Tuesday, 2016-04-19

*** elynn has joined #senlin00:09
*** elynn has quit IRC00:34
*** Qiming has quit IRC00:47
*** flwang has joined #senlin01:31
*** Qiming has joined #senlin01:45
*** yanyanhu has joined #senlin01:45
*** elynn has joined #senlin01:52
*** wei1 has joined #senlin01:53
*** wei1 has quit IRC01:57
flwangQiming: ping01:59
yanyanhuflwang, feilong?02:05
Qiminghello sir02:09
elynnmorning senlin :)02:09
yanyanhuhi, Qiming, have talked with rally team. I think the first patch for basic support of senlin will be merged after their 4.0 release is done02:10
yanyanhuand they also hope us to add those plugins into rally repo02:11
Qimingthey are not synced with openstack release?02:11
Qimingwhat does it mean by a 4.0 release02:12
yanyanhuno I guess02:12
yanyanhuoh, sorry, 0.4.002:12
yanyanhulooks like their release was still not done till yesterday02:13
Qimingoh, I see02:13
yanyanhuso senlin support will not be included in that release02:13
Qimingthere is no deliverables registered in openstack/releases project02:13
Qimingnot a big deal02:14
yanyanhuso, they use different release mode from us?02:14
yanyanhuso I think this patch will be used for test of gate job https://review.openstack.org/30717002:14
yanyanhuand finally those plugins will be delivered to rally I guess02:14
yanyanhujust leave jobs inside senlin repo02:15
Qimingseems they are still doing independent releases02:15
yanyanhuoh, I see02:15
yanyanhuI recalled they joined big tent a little bit earlier than us?02:16
yanyanhuok. I will keep working on this and talk with them if needed02:17
Qimingbtw, patch 307170 makes little to no sense to me02:17
Qimingit is doing exactly what we were trying to avoid02:17
yanyanhuoh, you mean the duplicated code?02:17
yanyanhuyes. But looks like there is no other way to do so...02:18
yanyanhusince tempest scenario in rally is not for this purpose02:18
Qimingwhy we cannot share the code?02:18
yanyanhuand those plugins will finally go to rally side, so we don't need to maintain them in senlin02:19
yanyanhuyou mean with tempest test cases defined in senlin?02:19
Qimingfor example, https://review.openstack.org/#/c/307170/2/rally-jobs/plugins/senlin_plugin.py, the _create_cluster() function is doing nothing new02:19
yanyanhuthat's because rally doesn't support benchmarking using tempest scenario. Only verification support is provided02:19
Qimingif we are adding a 'health_threshold' property to a cluster02:20
yanyanhuso if we want to benchmark something like cluster_create/cluster_delete individually, we need to define them as individual actions02:20
Qimingthere are too many places to sync now02:20
yanyanhuthat's true...02:20
Qimingwhen the code wasn't updated, it becomes outdated, not accurate, or even wrong02:21
yanyanhuyes, but I guess if we want to benchmarking senlin in that fine granularity using rally, we may have to do so02:21
yanyanhuactually we don't implement too much cases using rally I think02:22
yanyanhumuch less then cases we defined in tempest plugin02:22
Qimingline 71-9002:22
Qimingfor example02:22
*** zzxwill__ has quit IRC02:22
*** zzxwill__ has joined #senlin02:23
Qimingfunction _create_cluster was defined as just a generic function02:23
Qimingit has too many parameters and most of them have default values02:23
yanyanhuyou mean we should remove those default values?02:23
Qimingit looks like a new fork of the senlinclient library02:24
Qimingif we are benchmarking things, we'd better have a good plan, what are the cases we care02:24
yanyanhuyes, since user can only customize the parameter that defined in the action function when performing benchmarking02:24
Qimingwhat are the cases that are representative02:24
Qimingand ... this plugin is using its own client, instead of invoking the APIs directly02:26
Qimingis that the expectation, or I'm missing something02:26
yanyanhuyes, that is also following rally's osclient02:26
Qimingso the benchmarking is about an end-to-end scenario, right?02:26
yanyanhuQiming, yes02:27
yanyanhuI think the time counting is with the granularity of that function02:27
Qimingit is not about the service time between the moment API receiving a request to the moment the action is done02:27
yanyanhuso includes all interaction though client02:28
yanyanhuyes, it's not for senlin's action granularity02:28
Qimingit is counting REST call time, SDK processing time, keystone authentication delay, middleware processing time, all those stuff02:28
Qimingso it has nothing to do with tempest then02:29
yanyanhuI guess if we want to collect time data in that level, we need some checkpoint inside senlin?02:29
Qimingem, don't think so02:29
Qimingyou can get an idea by checking the logs already02:30
Qimingall log entries have timestamps02:30
yanyanhuyes, but that's not the way we should use I guess02:30
Qimingthen we should use tempest for that purpose02:31
yanyanhubenchmark should be done with treat test object as blackbox I guess?02:31
Qimingleave rally outside of this test02:31
QimingI have no idea what rally is trying to do02:31
Qimingif you are testing the performance by invoking senlinclient02:31
Qimingyou are doing an end-to-end performance test02:32
Qimingit is about your cloud deployment as a whole02:32
yanyanhuthe ability we want to leverage from rally is statistics and result collecting and visualization02:32
Qimingbut if you have hard-coded the invocation by bringing in senlinclient02:33
yanyanhuif we want to test every detail of our own service, I guess we need more work to do by ourselves02:33
yanyanhuinside senlin to collecting those time data02:33
Qimingthe only thing it can do is for end-to-end benchmarking02:33
yanyanhuI see02:33
Qimingthis line:     @atomic.action_timer("senlin.list_clusters")02:33
yanyanhuso it's not for benchmarking senlin's service performance02:34
Qimingis it just a label or something?02:34
Qimingthat is my concern02:34
yanyanhuoh, this label will be shown in test results02:34
QimingI'm not against rally, but we have to be sure what we are doing02:34
flwangQiming: i have question about your api-ref02:34
flwangi saw you copied some code from nova02:34
Qimingit could be:     @atomic.action_timer("senlin.make_clusters_dance_blah_blah")02:34
Qimingyes, flwang02:35
Qimingthose code, I suppose will eventually go back to api-site02:35
flwangbut nova does that change is because swagger doesn't support micro version02:35
Qimingit is there only to make the tox job happy, and to generate the same look-n-feel of api doc locally02:36
flwangdoes senlin has the same issue?02:36
Qimingflwang, we just added micro-versioning support02:36
Qimingwe believe it is very very important to keep API versioned02:36
Qimingso we will, have the same problem02:36
QimingI hate the fact that the doc team is "swaggering" this way and that02:37
QimingI had a WIP for swagger locally, not committed02:37
Qimingthen I learned from Anne, that won't be the plan any more02:38
Qimingthey are more leaning towards a (short-sighted) hack02:38
flwangdo you mean swagger won't the plan?02:38
Qimingthis one? http://lists.openstack.org/pipermail/openstack-dev/2016-March/090659.html02:39
flwangi don't think so.02:39
Qiminghile we're not completely giving02:39
Qimingup on Swagger, we're also recognizing the engineering effort to maintain02:39
Qimingand sustain those efforts isn't going to magically appear.02:39
flwangbased on my understanding, swagger still could be the way we will go for02:40
QimingI don't think docs team has technical arsenal to drive this02:40
QimingI don't like the way Sean is doing either02:40
flwangQiming: i will forward a mail between anne and me02:41
flwangpersonally, i would like each team just work out a swagger json and api-site will extract them and build it02:41
flwanganne said it should work, and i'm going to do that02:41
*** elynn_ has joined #senlin02:41
QimingIdeally, the whole community should make a decision02:42
QimingI was creating some json using the fairy-slipper thing I guess?02:42
flwangQiming: btw, are you guys still interested in using zaqar in senlin?02:42
Qimingflwang, definitely02:43
Qimingbut ... I tried ping you several times02:43
flwangis there a bp/spec to track it?02:43
flwangQiming: sorry, pls blame timezone02:43
Qimingto report that zaqar functional tests break02:43
flwangi owe you a beer :)02:43
flwangfunctional test?02:43
Qimingthere are many api return code doesn't match02:43
Qimingtox -e functional02:43
QimingCharles has been looking into zaqar recently02:44
Qimingalthough the whole team is keen to know what we can achieve during Newton02:44
*** elynn has quit IRC02:45
flwangQiming: i think we don't support tox -e functional02:45
flwangwe use tox -e integration02:45
QimingWe are not tracking this using a BP yet, it is documented in a TODO.rst file so we won't forget02:46
Qimingokay, you are using 'integeration' to run functional tests then02:47
flwangbased on the name 'Zaqar queue based receiver.'  it could be very easy i think02:48
flwangyou just need create a queue and post message to the queue, done02:48
Qimingif it is a receiver02:48
Qimingit means senlin will create a "listener" or "subscriber"02:49
flwangif you can let me know what's a 'receiver' in senlin, i can give you some ideas :)02:49
Qimingit is an abstraction, with different flavors02:49
Qimingby default, when you do senlin reiver-create, you will get a webhook, to which you can HTTP POST a signal02:50
Qimingwe want to extend that to receiver-create -t message, so the caller can get a queue name/address/id or something02:50
flwangah, aodh just done a similar thing02:50
Qimingwhen another alert system is posting things into that queue, senlin will get notified02:51
flwangso senlin is the 'subscriber'?02:51
flwangdoes it need an auth?02:52
flwangtoken i mean02:52
Qimingin our context, we want to enable some (or any) senlin operations to be triggered by external alerts02:52
Qimingfor webhook, no02:53
Qimingfor zaqar, that is a good question02:53
flwangif so, it should be easy02:53
flwanglet me repeat the user case02:53
flwang1. user create a queue in zaqar named A02:54
flwang2. create a subscription in zaqar on the queue A and the 'subscriber' is an url of senlin, so that senlin can be notified02:54
flwang3. post messages the queue A02:54
flwang4. senlin is notified02:54
flwangis it correct?02:54
Qimingin step 1, we don't want to bother a user to do queue creation02:55
flwangQiming: no worries, zaqar support lazq queue02:55
flwangyou don't have to create the queue explicitly02:56
Qimingstep 1 will be invoked by senlin internally02:56
flwangso we even don't need the step102:56
Qimingthe queue name 'A' is something the event alarm system need to know02:56
flwangok, it could be a pre-defined patten i think02:57
Qimingthen when senlin-receiver is successful, it means senlin is saying: okay, I have a queue for you to post your request, come on02:58
Qimingjust like it used to say: okay, I have a webhook for your to invoke, let me know if you have a request02:59
flwangthen, the question is when zaqar notify senlin, the subscriber is a link/endpoint of senlin, right?03:00
flwangdoes the endpoint need auth?03:00
Qimingwhat are the formats you support for a subscriber?03:01
QimingCan I just look into the queue directly?03:01
Qimingwhen are user wants to post a message, I was expecting he/she was authenticated by zaqar03:02
flwangwhen you create a subscriptoin, the post body is 'subscriber', 'ttl' and 'options'03:04
flwangsubscriber could be a http/https, email03:04
flwangyes, you can look into the queue for sure03:04
Qiming... so for senlin, it is just another webhook03:04
Qimingnot triggered by the monitoring system directly03:05
flwangTBH, i'm a little bit confused03:05
flwangso what's the 'subscriber' look like?03:05
flwangit's another 3rd party URL or a senlin's endpoint?03:06
QimingI'm not sure I understand what a 'subscriber' means03:07
Qimingokay, let's get back to the webhook use case03:07
flwang'subscriber' is a URL or email address03:07
flwangso when there is a message posted to the queue, zaqar will forward the message to post it to the URL or send it to the email address03:08
Qimingif what I should (have to) provide you is a senlin endpoint03:08
Qimingfor zaqar to invoke03:08
Qimingthat is no difference for us, as a webhook03:08
Qiminga webhook is something like this: http://{host:port}/v1/webhooks/{webhook_id}/trigger?V=103:09
Qimingthat would be the so-called subscriber to a zaqar queue03:10
Qiminga monitoring service/system can invoke that URL directly03:10
Qimingthus eliminating the need of zaqar in between03:10
flwangso user(3rd party) send notification to the webhook?03:11
flwangand the webhook call senlin?03:12
Qimingthe webhook is a senlin endpoint03:12
Qimingwhen you are doing a HTTP POST to that URI, an action will be triggered on a specific cluster03:13
flwangdoes the webhook need auth to access?03:13
Qimingunless I tell you where it is03:13
Qimingthere is no listing or showing API related to webhook03:14
flwangi see03:14
Qimingwebhook_id is a UUID03:14
Qimingnote that in "http://{host:port}/v1/webhooks/{webhook_id}/trigger?V=1" there is a "V=1"03:14
Qimingthat means a version 1 protocol03:15
Qimingwhen security concern rises, we can move to V=2, for example, while requiring more parameters on the URL telling me who you are03:15
flwangok, so based on my understanding, even zaqar is a good thing for this case, it's not senlin's business, right?03:16
Qimingit was not what we have imagined03:16
flwangso what you were looking for?03:17
Qimingwhen creating a receiver, we hope we can create something other than webhooks03:17
Qimingbecause webhooks are exposing the controller's IP address03:17
Qimingthat is dangerous03:17
flwangok, if that's the key03:18
Qiminginstead, we are looking to something like 'senlin receiver-create -c my_cluster -a CLUSTER_SCALE_OUT -f message'03:18
flwangthen we could meet somewhere03:18
Qimingthen 'senlin receiver-show'03:18
flwanglet me repeat and pls fix it03:18
flwang1. senlin receiver-create -c my_cluster -a CLUSTER_SCALE_OUT -f message   will return a queue name to user(3rd party)03:19
Qimingyou will get something like '{'receiver': {'id': xxx, 'channel': {'queue': {....}}}'03:19
flwangand meanwhile, senlin will create the subscription with his webhook03:19
flwanguser(3rd party) can't access the subscriptions, so they can't know the webhook url(ip)03:20
Qimingwhen creating that receiver, senlin has invoked zaqar to create a queue, and senlin will listen on that queue03:20
flwang2. user(3rd party) send messages to the queue03:20
flwangsenlin is notified03:20
flwangsenlin don't have to listen03:21
flwangsenlin will be notified03:21
Qiming... why we have to mix webhooks and message queues?03:21
Qimingthey are different kinds of 'receivers' to senlin03:21
flwangok, sorry03:21
Qimingwhen a user decides that a webhook is appropriate, we don't need man-in-the-middle03:21
flwangi seeeeeeeeeeeeeeeeee03:22
Qimingif they believe a messaging based notification is more appropriate, they will abandon webhook03:22
flwangso for the zaqar case, you don't want to(have to) be notified03:22
flwangyou just want to listen/poll the queue, right?03:22
Qimingit is a pretty harsh binary ... this assumption may be totally invalid, ;)03:22
flwangi see03:23
Qimingjust like another piece of code where we are listening to oslo.messaging03:23
Qimingwe need to know the topic and some other parameters before doing so03:23
Qiming(we were trying to listen to oslo.messaging so that we know a VM we created is down ...)03:24
flwangQiming: ok, if that's what you guys want, super easy i think03:28
flwangi was confused just because i always think you want to be notified instead of listening/polling03:29
Qimingvery interested in seeing a strawman running, then we can figure out how to solve the authentication problem, if any03:29
flwangand i'm sure you guys will be benefited from queue03:29
flwangmay i know what's the message looks like?03:30
flwanglike an alarm?03:30
Qiminga json body03:30
Qimingsomething like that03:30
flwangi mean the content03:31
flwangok, then queue is good for you guys03:31
flwangimagine a scenario03:31
flwangyou got 2 alarms and they are different, what does senlin to do?03:31
Qimingit could be as simple as this: {"parameters": {"count": 3, "strict": false}}03:32
Qimingthis is a parameter to a CLUSTER_SCALE_IN operation03:32
Qimingall optional03:32
flwangi mean an alarm received at 13:01  say cpu usage is 95%, and alarm got at 13:05 say cpu usage is 60%03:32
Qimingyes, users decide what parameters they will pass in the above body03:33
flwangi mean senlin could be benefited from a queue system03:33
flwangso that it can make some decisions more smart03:33
Qimingwe are trying to make senlin stupid03:33
flwanginstead of processing each message even they're confict03:33
Qimingleaving the decisions to users03:33
flwangok, then skip my words :)03:34
Qimingusers will say when 13:01  say cpu usage is 95%, they will do CLUSTER_SCALE_OUT by 50%, they instruct the monitoring system to do so03:34
Qimingsenlin will check the request parameters and do what it was asked03:35
Qiming5 minutes later, another alarm comes in03:35
flwangi'm just thinking aloud, i'm a newbie for senlin03:35
Qimingsenlin will say ... no, you just said that 95%, isn't you?03:35
QimingI'm not interested, you can try tell me something new at 13:1103:36
Qimingyour cluster is in a 'cool_down' period03:36
flwangQiming: that's what i'm talking about i think03:36
flwangsenlin could do it more smart03:36
Qimingright, it is a tricky area03:37
flwanginstead of doing anything03:37
flwangat least, it could be a config or something like that03:37
Qimingwe want to build the mechanisms layer, make it robust and flexible03:37
Qimingthen we can work on the fancy things, ;)03:37
flwangfair enough03:38
Qimingand ... we are already making some of those higher level decisions abstracted and exposed to users as Policies03:38
Qimingalways hesitate when coming up with something smart, :P03:39
Qimingit is sometimes poison, :)03:39
Qimingflwang, speaking of the API tihng03:45
QimingI'm wondering if we should use swagger or something alike03:45
Qimingteam has proposed some ideas about API schema validation03:46
Qimingchecking parameter's data type, value range, making sure required parameters are there and unknown parameters are rejected03:46
QimingI was thinking about this last night03:47
Qiminggoing the swagger way would benefit both the API documentation and schema validation03:47
*** elynn_ has quit IRC04:19
flwangQiming: yep, and we're using json schema04:23
*** elynn_ has joined #senlin04:45
*** elynn_ has quit IRC04:49
*** elynn_ has joined #senlin04:50
yanyanhuhi, Qiming, can I directly revert a patch in gerrit UI?05:44
yanyanhuwill remove this change https://review.openstack.org/#/c/303923/05:44
openstackgerritYanyan Hu proposed openstack/senlin: Revert "Add Rally plugin for Senlin tempest"  https://review.openstack.org/30756505:46
*** elynn_ has quit IRC06:26
*** yanyanhu has quit IRC06:26
*** Qiming has quit IRC06:29
*** Qiming has joined #senlin08:00
xuhaiweiQiming, available now?08:02
xuhaiweishall we do it on phone or irc?08:09
Qiminglet's try phone call now08:10
Qimingwe are in xinhui's office08:10
Qimingrehearsaling the deep dive presentation08:10
xuhaiweiagain, the password, sorry about it08:11
QimingParticipant Code:  2157537108:12
Qimingsorry I dropped dialing in again08:32
xuhaiweican you enter again?08:32
*** Qiming has quit IRC09:04
*** Qiming has joined #senlin10:25
*** Qiming has quit IRC10:37
*** Qiming has joined #senlin12:11
*** Qiming_ has joined #senlin12:14
*** Qiming has quit IRC12:16
*** elynn has joined #senlin12:18
*** xuhaiwei has quit IRC12:19
*** elynn has quit IRC12:28
Qiming_hi, cschulz, it is cancelled, sorry for the late notice13:27
cschulzFigured that since noticed you were working on presentations13:30
Qiming_sorry for that, :)13:31
cschulzHope it is going well13:31
*** openstackstatus has joined #senlin13:38
*** ChanServ sets mode: +v openstackstatus13:38
-openstackstatus- NOTICE: We have recovered one of our cloud providers, but there is a huge backlog of jobs to process. Please have patience until your jobs are processed13:41
*** zzxwill__ has quit IRC14:32
*** zzxwill__ has joined #senlin14:33
*** Qiming_ has quit IRC15:38
*** zzxwill__ has quit IRC17:22
*** dnievas has joined #senlin18:58
dnievashi guys18:59
*** flwang has left #senlin21:32
*** zzxwill__ has joined #senlin22:05
*** zzxwill__ has quit IRC22:35
*** Qiming has joined #senlin23:10
*** Qiming has quit IRC23:25
*** dnievas has quit IRC23:27
*** xuhaiwei has joined #senlin23:36

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!