Friday, 2014-12-12

*** bradjones has quit IRC00:18
*** exploreshaifali has quit IRC00:21
openstackgerritVictoria Martínez de la Cruz proposed openstack/zaqar: Adds websockets driver to Zaqar server  https://review.openstack.org/14015501:07
*** amalagon has quit IRC01:16
*** boris-42 has quit IRC02:03
*** X019 has quit IRC02:26
*** amalagon has joined #openstack-zaqar02:30
openstackgerritVictoria Martínez de la Cruz proposed openstack/zaqar: Adds websockets driver to Zaqar server  https://review.openstack.org/14015502:58
vkmcflwang1, around?03:39
flwang1vkmc: yep, i'm reviewing your patch now :)03:54
flwang1's up?03:54
vkmcohh03:54
vkmcI'm trying to come for a decent solution for a change03:54
vkmc>.<03:54
flwang1anything i can help?03:54
vkmcmaybe you can give me your opinion03:54
vkmcwe are working on separating the API layer from the transport03:55
vkmcthat is... the transport receives the message and sends it to the API... and the API validates and send the message to the storage03:55
flwang1so another layer above current API layer?03:55
vkmcwell.. not really... its the same layer but it has more responsabilities03:56
flwang1what's the API layer mean 'wsgi/websocket'?03:56
vkmcthis new architecture will be used for the persistent transports only... we are not changing wsgi03:56
flwang1vkmc: i think i need a clear definition about the 'new' transport layer and the api layer03:57
vkmcsure03:57
vkmcboth changes are related03:57
vkmcthis new API, besides validating the schemas as we were doing, will be in charge processing the requests03:58
flwang1which will be like an abstract layer above wsgi and websocket,  is it?03:59
vkmcnope, in the middle03:59
vkmce.g. websocket -> api -> mongodb03:59
flwang1and wsgi->api->mongodb?04:00
vkmcwsgi won't be changed for now04:00
vkmcits too complicated04:00
vkmcso this will be for persistent transports only04:00
flwang1so that means zaqar client need to handle the difference?04:00
vkmcright now the wsgi transport is in charge of validating and processing the messages received... that is something we discussed it was better to remove from the transport side04:01
vkmcand put it in the api side04:01
vkmcthe transport will be in charge of transport only04:01
flwang1so what's the benefit to have an new, even thin, layer between transport and storage?04:01
flwang1ah, ok04:02
flwang1but seems flaper87 is also trying to move the validation(or part of them) to storage layer, right?04:02
vkmcthe benefit... less code I guess04:02
vkmcwe won't have to perform controls for every transport we might want to add04:02
vkmcyeah, flaper87 is working on moving the data controls to the storage layer04:03
flwang1so any overlap/conflict with yours idea?04:03
vkmcthis is the description of the blueprint of the separated api thing04:04
vkmcDefine an API spec that supports extensions, versioning and that is also cross-transport. This will allow Zaqar to define its API in a single module and allow transport for translating the API to something supported by the protocol they implement.04:04
vkmcits fancier than my 'less code' response haha04:04
vkmcno, flaper's change is not overlapping my change because I'm making controls on the request received04:05
flwang1so from the api consumer view, they may be a little bit confused04:06
flwang1like me :)04:06
vkmchaha I totally understand that because it took some time to me to process the whole idea04:06
flwang1unless we can make sure we will cover the magic in client side04:07
flwang1and provide the consistent 'api' (if i still can use this term) for our consumer/developers04:07
vkmcthe client has a similar structure as the one we are trying to achieve in the server04:07
flwang1vkmc: i see.04:08
flwang1my concern is I don't want to see any big change for our api04:08
flwang1given we want to graduate, and a stable api is really important04:08
vkmcyeah... I feel the same way04:08
vkmcalso... the idea of having a way of handling websockets and another way to handle wsgi04:09
flwang1for websocket, IMHO, that's the way we are looking for improve zaqar's performance04:10
vkmcyeah, but in the case of websocket04:10
vkmcwe can do something similar to what we have for wsgi04:10
flwang1you mean wsgi->^api^->storage?04:11
vkmcand I think its a good adition that doesn't affect what we have already04:11
vkmcright now its wsgi->storage04:11
flwang1yep, i see04:11
vkmcwe can do websockets->storage04:11
vkmcits not mandatory for websockets to do this change04:11
flwang1i would like to see the new arch won't impact what we have currently04:11
vkmcit won't I think04:12
vkmcbecause we are not changing wsgi04:12
flwang1okay04:12
vkmcbut it may look odd :)04:12
flwang1right04:12
flwang1that's what my concern either ;)04:12
vkmchaha04:13
vkmcI mean04:13
flwang1the api/arch of zaqar will be weird04:13
vkmcI have been thinking all night different architectures04:13
vkmcand honestly I haven't come to a decent solution04:13
vkmcI don't want to do a handler with a ton of if clauses04:13
flwang1vkmc: i have to go, my son got fervor04:14
flwang1vkmc: i will ping you later, sorry for that04:14
vkmcflwang1, please, don't be sorry... poor lil hacker04:14
vkmcflwang1, ttyl04:14
*** flwang1 has quit IRC04:18
*** amalagon has quit IRC04:44
*** amitgandhinz has quit IRC06:05
*** sgotliv has joined #openstack-zaqar06:08
openstackgerritVictoria Martínez de la Cruz proposed openstack/zaqar: API handler and API v1.1 endpoints  https://review.openstack.org/14128006:12
*** sgotliv has quit IRC06:13
*** reed has quit IRC06:28
*** vkmc has quit IRC06:31
*** achanda has joined #openstack-zaqar06:31
*** sgotliv has joined #openstack-zaqar06:57
*** achanda has quit IRC07:04
*** sgotliv has quit IRC07:04
*** achanda has joined #openstack-zaqar07:05
*** amalagon has joined #openstack-zaqar07:05
*** achanda_ has joined #openstack-zaqar07:07
*** achanda has quit IRC07:09
*** amalagon has quit IRC07:10
*** achanda_ has quit IRC07:17
flaper87flwang: hey there07:24
*** achanda has joined #openstack-zaqar07:46
flaper87achanda: hey hey :)07:47
achandaflapper87: Hi there07:47
*** amitgandhinz has joined #openstack-zaqar07:49
*** achanda has quit IRC08:11
*** achanda has joined #openstack-zaqar08:12
*** achanda has quit IRC08:16
*** achanda has joined #openstack-zaqar08:18
*** sgotliv has joined #openstack-zaqar10:08
*** achanda has quit IRC10:28
*** sgotliv has quit IRC10:40
*** malini has joined #openstack-zaqar11:34
*** jasondotstar has joined #openstack-zaqar11:43
*** boris-42 has joined #openstack-zaqar11:55
*** sgotliv has joined #openstack-zaqar12:18
*** achanda has joined #openstack-zaqar12:39
*** jasondotstar has quit IRC12:48
openstackgerritZhi Yan Liu proposed openstack/zaqar: Integrate OSprofiler with Zaqar  https://review.openstack.org/14135612:58
*** vkmc has joined #openstack-zaqar13:09
*** vkmc has quit IRC13:09
*** vkmc has joined #openstack-zaqar13:09
vkmco/13:10
flaper87vkmc: hey hey13:10
vkmcflaper87, hey dud13:10
vkmcflaper87, https://review.openstack.org/#/c/141280/13:13
vkmc^ :D13:13
* flaper87 clicks13:15
vkmcneed your feedback to continue13:15
kragnizmorning, zaqhaar13:23
vkmczakhaaaar13:24
flaper87vkmc: commented13:24
flaper87vkmc: it's looking great13:24
flaper87vkmc: any reason you made those methods private?13:25
vkmcflaper87, thanks for reviewing :)13:25
vkmcwell, given that those are for internal use13:26
vkmcbut there is no need not that you mention13:27
vkmcI'll fix that13:28
vkmcI have one corcern, as always13:29
*** amitgandhinz has quit IRC13:29
vkmcwell, tbh two13:29
vkmc1. yesterday we were talking with flwang that it might look weird to have one architecture for wsgi and another for websockets13:30
vkmcand without a logic explanation13:31
vkmcwe should go all the way or nothing13:31
flaper87as in, use the API layer for ws and not wsgi ?13:31
vkmcyea13:31
flaper87FWIW, we talked about this at the summit and when we discussed the spec13:32
flaper87We tried using this work for wsgi when the idea first came out13:32
vkmcdid we discuss this spec? I wasn't there13:32
vkmcprobably is one of the times I got lost13:32
flaper87and we didn't move forward because it'd have ended in code duplication13:32
flaper87(when we discussed the persistent transport one)13:32
flaper87the reason is that, in order to be RESTFul compliant, we need to raise proper exceptions and whatnot13:33
flaper87this means me need to register endpoints, catch exceptions and re-raise them13:33
vkmcyes I'm aware about that, we discussed it13:33
flaper87it's different for wire protocols because we just translate the exception and send it back13:33
flaper87We can treat the protocol pretty much like rpc13:34
flaper87In addition to that, the wsgi transport is our *reference* transport13:34
vkmcindeed13:34
flaper87what's coded there, must be supported by all other transports13:34
flaper87we don't want to mess with it13:34
flaper87if it makes it more clear, we can just rename the `api` package into `protocol/wire` ?13:35
vkmcwell, I'm raising this topic because its something that I noticed flwang didn't know about and had some doubts about it13:35
flaper87dunno, something that does not give the impression it's the API13:35
flaper87sure, re-visiting arguments is good13:35
vkmcand it seems that this requires a clear up in the next meeting13:35
flaper87the next meeting is at 15 UTC13:35
flaper87I think he won't be around13:35
flaper87:(13:35
flaper87flwang: you, boy, read this ^13:36
vkmcok :/13:36
vkmclet's pull all this comments in the blueprint board13:36
vkmcI'll do so13:36
flaper87vkmc: could you also update the cross-api spec ?13:36
flaper87as in add these comments to the `Proposed Change` section13:37
flaper87I think it's worth it13:37
vkmcthat is what I meant13:37
flaper87ah sorry13:37
flaper87the blueprint term made me think of LP13:37
vkmcno problem :D13:37
flaper87my bad13:37
flaper87:D13:37
* vkmc is a little worried that she is thinking like flaper8713:37
*** jasondotstar has joined #openstack-zaqar13:37
vkmcok, will update both13:38
vkmcyour comments are welcome as well13:38
vkmchehe13:38
vkmcaaaaaaaaaaaand, second concern13:39
vkmcI noticed you or Cindy coded a validation method in the api13:39
vkmcwhich is JSON only13:39
vkmcda real thing is the validation in transport13:39
flaper87LP ruined that word for me13:39
vkmcthis validation https://github.com/openstack/zaqar/blob/master/zaqar/transport/validation.py13:40
vkmcI know we chatted that the API should be in charge of doing the validation13:40
vkmcbut I think we should keep that to the client13:40
vkmcsorry13:40
vkmctransport13:40
vkmcotherwise we are overcharging the api13:41
vkmcand there are some attributes that are part of the transport13:41
flaper87so, that validation thing in the API validates the request form: It's based on jsonschema13:41
vkmcyeah13:41
flaper87the remaining validations should go through transport.validation13:41
vkmcthat seems right13:41
flaper87we can't move the request validation to the client because one may use raw websockets13:42
flaper87I mean, one could send random requests to the server without using any client13:42
vkmco_o13:42
vkmcso... we validate in both layers13:43
flaper87yup13:43
vkmcok13:43
flaper87I mean, we can allow clients to skip client validation13:44
flaper87but the server-side validation must exist13:44
vkmcwhat do you mean by client here?13:44
vkmcboth validations are server side13:44
flaper87I don't know13:44
flaper87ah wait13:44
flaper87I thought by layers you meant client library and server13:45
flaper87you actually meant request schema and request data13:45
flaper87the answer is yes anyway13:45
flaper87:D13:45
vkmchttps://github.com/openstack/zaqar/blob/master/zaqar/common/api.py#L5213:45
vkmcthat is the validation I'm confused about13:46
vkmcbut what I just understood is13:46
vkmctransport validates incoming data, creates the request13:46
vkmcapi validates that the request is well formed?13:46
vkmcseems unnecesary because we are building the request13:46
vkmcwe should be smart enough to build the request13:46
vkmcin a sane way13:47
flaper87wait13:47
vkmcI have to rush for a bit :(13:47
flaper87This evaluates the request form: https://github.com/openstack/zaqar/blob/master/zaqar/common/api.py#L5213:47
flaper87this evaluates the request data, server limits, etc: https://github.com/openstack/zaqar/blob/master/zaqar/transport/validation.py13:48
flaper87those are different validations13:48
flaper87one validates required fields are in the request (former) the other validates the values (the later)13:48
flaper87We don't need the former for the wsgi transport13:48
flaper87which is why they are separated13:48
vkmccool13:53
vkmcbbl13:53
vkmcthanks for clear up things for meeee >.>13:54
*** malini has quit IRC13:55
*** jchai has joined #openstack-zaqar13:57
openstackgerritFlavio Percoco proposed openstack/zaqar: Wrap abstract method with base methods  https://review.openstack.org/13907314:06
openstackgerritFlavio Percoco proposed openstack/zaqar: Add capabilities property to the DataDriver  https://review.openstack.org/13563714:06
*** sriram has joined #openstack-zaqar14:07
*** achanda has quit IRC14:11
*** achanda has joined #openstack-zaqar14:12
*** dynarro has joined #openstack-zaqar14:23
*** jasondotstar has quit IRC14:23
*** jchai has quit IRC14:24
*** jchai has joined #openstack-zaqar14:32
*** ametts has joined #openstack-zaqar14:46
*** mpanetta has joined #openstack-zaqar14:49
*** malini has joined #openstack-zaqar14:59
*** jchai has quit IRC15:00
openstackgerritZhi Yan Liu proposed openstack/zaqar: Integrate OSprofiler with Zaqar  https://review.openstack.org/14135615:03
*** jchai has joined #openstack-zaqar15:06
*** X019 has joined #openstack-zaqar15:08
*** amitgandhinz has joined #openstack-zaqar15:09
*** malini1 has joined #openstack-zaqar15:13
*** malini has quit IRC15:15
openstackgerritZhi Yan Liu proposed openstack/zaqar-specs: Add OSProfiler to Zaqar  https://review.openstack.org/13561215:34
zhiyanvkmc: hi, around?15:34
*** malini1 has left #openstack-zaqar15:35
zhiyanvkmc, kgriffs|afk flaper87 : when/if you ok, could you pls review osprofiler change and its spec (I'd like it can be flag in k1), any input are welcome, thanks.15:41
vkmczhiyan, heeeeeeey15:42
vkmczhiyan, sure thing15:42
vkmczhiyan, thanks for working on that15:42
zhiyanvkmc: btw, could you educate me in the sepc change? i don't know which path should new proposed rst file save..15:43
zhiyanvkmc: glade to do this stuff in zaqar, useful for diagnose perf issue for dev and operator.15:45
vkmczhiyan, totally :)15:45
vkmczhiyan, so... the specs are in zaqar-specs repository15:45
vkmczhiyan, https://github.com/openstack/zaqar-specs15:45
vkmczhiyan, you can clone that repo, grab my change and amend it with your comments15:46
zhiyanvkmc: oh, yes, i mean this http://logs.openstack.org/12/135612/5/check/gate-zaqar-specs-docs/cacf6f5/console.html#_2014-12-12_15_40_54_10115:46
vkmcohhh  I see15:47
zhiyanseems "zaqar-specs/specs/kilo" path is wrong , at least fro the doc test15:48
openstackgerritZhi Yan Liu proposed openstack/zaqar-specs: Add OSProfiler to Zaqar  https://review.openstack.org/13561216:00
zhiyanvkmc: fixed ^^16:00
vkmczhiyan, thanks, I got distracted, sorry16:01
*** cpallares has joined #openstack-zaqar16:11
*** achanda has quit IRC16:11
*** achanda has joined #openstack-zaqar16:11
*** achanda has quit IRC16:16
*** achanda has joined #openstack-zaqar16:18
*** jasondotstar has joined #openstack-zaqar16:19
*** reed has joined #openstack-zaqar16:19
*** X019 has quit IRC16:21
*** dynarro has quit IRC16:21
*** boris-42 has quit IRC16:32
*** achanda has quit IRC16:50
*** achanda has joined #openstack-zaqar16:51
*** achanda has quit IRC16:56
jasondotstarhi all16:57
jasondotstarnew here. just saying hi.16:58
*** amalagon has joined #openstack-zaqar17:00
* kragniz huggles jasondotstar 17:02
*** achanda has joined #openstack-zaqar17:03
*** miqui_ has quit IRC17:04
*** miqui_ has joined #openstack-zaqar17:04
* jasondotstar feels warm and fuzzy17:05
jasondotstarhonestly, I'm looking for a project to get involved in. make my first contribs to.17:05
jasondotstarsaw the PTL update on Zaqar17:06
jasondotstarinteresting project you guys have here.17:06
kragnizjasondotstar: you should totally work on zaqar17:09
kragnizI personally don't, but they have the best irc channel around17:09
jasondotstaris that rt? nice!17:09
vkmchey jasondotstar!17:25
vkmcwelcome :)17:25
jasondotstarthx17:25
jasondotstarfeeling welcome already17:25
vkmcthat sounds great17:25
vkmcI read you are interested in contributing to Zaqar17:25
vkmcso please let me know if there is something I can do to make that easier for you to get involved17:26
jasondotstarsure.17:26
jasondotstarI'm starting to poke around the low-hanging fruit in the bug triage17:26
vkmcthat's cool17:27
jasondotstaris there a roadmap for the project stashed somewhere?17:28
vkmclet me see.. I don't know how updated is that17:29
vkmcyeah, its outdated17:30
vkmcbut well, I can tell you right now that our main work for K-1 will be to implement a persistent transport (websockets), notifications, capabilities17:32
vkmcand possibly... getting rid of the queue concept17:32
vkmcwe are not longer a queing service, but a messaging service17:32
vkmca more detailed information about those changes can be checked here https://github.com/openstack/zaqar-specs/tree/master/specs/kilo/approved17:33
*** openstackgerrit has quit IRC17:34
*** openstackgerrit has joined #openstack-zaqar17:34
vkmcwe also have some pending tasks that might be interesting to tackle on kilo17:35
vkmcas the integration with the OpenStack Dashboard (Horizon)17:35
jasondotstarcool.17:37
*** openstackgerrit has quit IRC17:49
*** openstackgerrit has joined #openstack-zaqar17:49
*** amitgandhinz has quit IRC18:22
*** JAHoagie has joined #openstack-zaqar18:27
*** sgotliv has quit IRC18:57
*** boris-42 has joined #openstack-zaqar19:03
*** amitgandhinz has joined #openstack-zaqar19:16
* vkmc lurks19:43
*** ametts has quit IRC20:27
*** achanda has quit IRC20:37
*** ametts has joined #openstack-zaqar20:41
*** jchai is now known as jchai_afk20:45
*** jasondotstar has quit IRC20:46
*** jchai_afk is now known as jchai20:52
*** amalagon is now known as alagon20:56
*** X019 has joined #openstack-zaqar20:57
*** jchai has quit IRC21:13
*** cpallares has quit IRC21:14
*** boris-42 has quit IRC21:33
*** sriram has quit IRC21:40
*** boris-42 has joined #openstack-zaqar21:55
*** JAHoagie has quit IRC22:00
*** mpanetta has quit IRC22:13
openstackgerritOpenStack Proposal Bot proposed openstack/zaqar: Updated from global requirements  https://review.openstack.org/14096322:22
*** flwang1 has joined #openstack-zaqar22:31
openstackgerritMerged openstack/zaqar: Add capabilities property to the DataDriver  https://review.openstack.org/13563722:59
openstackgerritMerged openstack/zaqar: Split Control and Data planes of Storage layer  https://review.openstack.org/13491022:59
*** amitgandhinz has quit IRC23:00
*** flwang1 has quit IRC23:21
openstackgerritMerged openstack/zaqar: Wrap abstract method with base methods  https://review.openstack.org/13907323:30
*** ametts has quit IRC23:30
*** flwang1 has joined #openstack-zaqar23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!