Wednesday, 2017-03-29

*** jkilpatr has quit IRC06:53
*** joseppc has joined #openstack-cyborg08:52
*** joseppc has joined #openstack-cyborg08:52
*** jkilpatr has joined #openstack-cyborg11:02
*** jkilpatr has quit IRC11:07
*** jkilpatr has joined #openstack-cyborg11:07
*** mikeH has joined #openstack-cyborg11:43
*** NokMikeR has joined #openstack-cyborg12:43
*** crushil has quit IRC13:32
*** crushil has joined #openstack-cyborg13:49
*** skelso has joined #openstack-cyborg13:58
*** sekelso has joined #openstack-cyborg14:00
*** sekelso has quit IRC14:02
*** sekelso has joined #openstack-cyborg14:03
*** skelso has quit IRC14:04
*** zhipeng has joined #openstack-cyborg14:07
*** sekelso has quit IRC14:10
*** sekelso has joined #openstack-cyborg14:10
*** sekelso has quit IRC14:33
*** sekelso has joined #openstack-cyborg14:34
*** sekelso is now known as skelso14:35
*** zhipeng has quit IRC14:41
*** mpaolino has joined #openstack-cyborg14:47
jkilpatranyone alive in here?14:53
jkilpatroh man I should check on my commits more often.14:54
*** zhipeng has joined #openstack-cyborg14:58
zhipeng#startmeeting openstack-cyborg15:00
openstackMeeting started Wed Mar 29 15:00:18 2017 UTC and is due to finish in 60 minutes.  The chair is zhipeng. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
openstackThe meeting name has been set to 'openstack_cyborg'15:00
zhipeng#topic Roll Call15:00
zhipengwaiting for more folks if we have15:02
zhipengokey let's proceed15:06
zhipeng#topic BP review15:07
zhipengi must appologize i was buried with kubecon work this week so haven't got the time for the reviews15:07
zhipengi see _gryf posted a lot of good comments15:08
zhipengas well as rushil and others15:08
zhipengif we have any outstanding issues, please feel free to shout out15:08
jkilpatrdo we want to consolidate all DB interaction to the api end point (presumably on the controllers) or have the agent on the computes update the database themselves.15:11
jkilpatressentially that's a question of where to handle communication and parallelism.15:11
zhipengif we choose the latter one, does it mean the controller side will be sometimes out of sync with the agents ?15:13
jkilpatrit would make it a concern, it would be possible to prevent if we tried.15:14
jkilpatron the other hand if the agent's don't store anything themselves then we can lose accelerator state if anything disrupts the agent.15:14
zhipenghow could we prevent the out-of-sync problem ? using heartbeat ?15:17
crushilI would prefer the latter option> And I believe heartbeat can be an option15:17
jkilpatrjust make sure both sides refresh info from the DB when it might have changed. So proper cache invalidation.15:18
jkilpatrand probably more DB load15:18
crushilBut the agent should definitely keep its database updated15:18
zhipengbut in real deployment that always hard to manage, I always hear complaints from our product team about the heartbeat15:19
zhipengcache invalidation is prong to go wrong15:19
zhipengis there a way for us to simplify so that we could avoid the common shortcomings15:19
jkilpatrso agent <-> rabbit <-> api end point <-> db15:20
jkilpatrthat way we don't need invalidation because the end point is the only one that interacts with the DB15:20
crushilIsn't that overkill though? Going through hoops to update db?15:22
crushilCan't we have  agent maintain a local copy of the db and agent keeps updating it?15:23
zhipengbut if say we have 3 compute node that all have FPGA based iNIC on it15:23
jkilpatrwhy does the agent need info about all accelerators? it only needs to manage one machine (there are many agents on many compute nodes, but each one only needs to care about it's own scope) I guess they could have a mini db each but what good is that.15:24
zhipengif the DB is updated only via local copy15:24
zhipengi'm terminating my thoughts here~~15:24
crushilWell, I feel that the agent needs to have a more efficient way of updating the db15:25
jkilpatrmaybe we just have the agent update the db for things that are longer term? Like a new accelerator?15:26
zhipenghow about the config changes15:26
zhipengthat would be part of the life cycle mgmt usually15:27
zhipengor we have two seperate DBs (sorta), the main one resides with the control node15:28
jkilpatrI guess agents need to read from the db for config updates, we don't want to obligate the operator to ssh into every node to change a config value.15:28
zhipenglike jkilpatr it only get updated on major events15:28
zhipengand agent locally manages a cache15:28
zhipengfor the more constant changes ?15:28
crushilThat seems fine, although how would you define major events?15:29
zhipengthe info in the locan cache should be of no concern of either other compute nodes or the control node15:29
zhipenglike when accelerator attached, created, deleted15:30
zhipengand for small changes15:30
zhipengagent could advertise to the api-endpoint15:30
zhipengas a bulk of its local cache info, if it deemed important by the control node15:31
zhipengso what I'm think is that user could explicitly define events that they care15:31
zhipengin the cyborg.conf for example15:31
zhipengand those events will be aggregated and reported by the agents to the api-end-point15:32
* _gryf waves late15:32
crushilSo, you're suggesting maintaining a central db and a local cached db. The central db would be updated on all events and cached db will be updated only on select events?15:32
zhipengalongside the usual big event15:32
ttk2[m]Jkilpatr here had to go mobile.15:32
zhipengcentral/main DB will get updated on all the major events (attach/detach/delete/create) and smaller events that user specified15:33
zhipengthe local copy is just cache for all the local updates, refreshed constantly15:33
zhipengthat is my thinking15:33
_gryfzhipeng, +1 for the cache instead of full blown db15:34
ttk2[m]Sounds like a workable compromise.15:34
crushilthat sounds ok to me15:34
zhipenggreat :) then the DB design would remain the same, and there will be an additional design on the local cache for the agent15:35
zhipengis this ttk2[m] or crushil's ?15:35
ttk2[m]I'll do the agent cache.15:36
zhipengfantastic :)15:37
zhipeng#action Justin to update with the agent cache design15:37
ttk2[m]Should I put that in the same blueprint?15:37
zhipengi think you could do that15:37
zhipengit should not be a big deal15:38
zhipengokey any other topics ?15:40
zhipengany questions on Roman's BP re the interaction between Nova and Cyborg15:40
crushilnothing from me15:43
zhipengwell let's keep review the spec anyways :P15:44
zhipeng#topic AoB15:44
zhipengany other buisness ?15:44
ttk2[m]Do we want to set a blueprint finish goal?15:44
zhipengttk[m] what do you mean ? for a deadline date ?15:44
_gryfttk2[m], like a deadline or smth?15:44
ttk2[m]Yes. Just something to keep things moving.15:45
_gryfin other projects that depends on how big is the core reviewers pipe15:45
zhipengi would love to set a deadline lol15:46
_gryfin nova there are deadlines, so they can be sure, that core reviewers spend quality time on desired and accepted features15:46
zhipengwhat we called in China is the lazy cancer kicked in all the time :P15:46
_gryfotoh, I've seen projects, (smaller ones), that they obey only openstack release schedule15:47
zhipengI think setting a deadline/milestone would be a good idea15:47
_gryfthat fine.15:47
zhipengso that everyone on the same page and pace15:47
zhipengothers ?15:48
ttk2[m]As long as it's not too tight.15:48
crushilSo, for current specs, should we say EOR should be the deadline?15:48
zhipeng(if we have a milestone then maybe we need a sprint someday ...)15:48
zhipengcrushil for specs that would be too relax15:48
zhipengwe need to get code in for Pike15:48
crushilI meant implementation of the specs15:48
zhipengyes that'd agre15:49
zhipengI would agree15:49
ttk2[m]Let's just put a deadline on specs for now.15:49
zhipengApr 15th ?15:49
ttk2[m]That's what 3 meetings?15:49
zhipengsounds about right15:50
zhipengtoo relax or too tight ? lol15:50
crushilI think for most of the specs, the owners know what needs to be added15:50
crushilSo, it is slightly on the more relaxed side15:50
zhipengit is the reviews that takes time15:51
* _gryf seen blueprints which lasts several months, before being accepted ;]15:51
zhipeng_gryf we are not that famous project so we could move faster XD15:51
_gryfwhich obviously means slipping to the next release :>15:51
crushilLet's do April 15th then15:51
zhipeng#vote Apr 15th as the deadline for specs15:52
zhipengokey wrong cmd ...15:52
zhipengbut anyway we should all agree on this15:52
zhipeng#info Apr 15th for the first milestone on spec freeze15:52
zhipeng#agreed Apr 15th for the first milestone on spec freeze15:53
zhipenggreat discussions folks15:53
zhipengany other buisness ?15:53
zhipengh[m]My Chromebook just died..15:54
crushilzhipengh[m], Get a real computer. :P15:54
zhipengh[m]lol give me money15:55
*** joseppc has quit IRC15:55
*** zhipeng has quit IRC15:58
zhipengh[m]Okey if no other biz, let me try to end meeting using this handle... Not sure it will work15:58
*** NokMikeR has quit IRC16:01
*** mpaolino has quit IRC16:21
*** mikeH has quit IRC17:16
*** mikeH has joined #openstack-cyborg17:40
*** zhipeng has joined #openstack-cyborg17:50
zhipengthe longest meeting ever...17:51
openstackMeeting ended Wed Mar 29 17:51:05 2017 UTC.  Information about MeetBot at . (v 0.1.4)17:51
openstackMinutes (text):
*** zhipeng has quit IRC17:55
*** mikeH has quit IRC18:08
*** mikeH has joined #openstack-cyborg18:10
*** crushil has quit IRC18:16
*** crushil has joined #openstack-cyborg18:17
*** openstackstatus has joined #openstack-cyborg18:43
*** ChanServ sets mode: +v openstackstatus18:43
*** crushil has quit IRC19:09
*** crushil has joined #openstack-cyborg19:26
*** crushil has quit IRC19:58
*** joseppc has joined #openstack-cyborg20:09
*** joseppc has joined #openstack-cyborg20:09
*** mikeH has quit IRC20:34
*** mikeH has joined #openstack-cyborg21:13
*** skelso has quit IRC21:26
*** skelso has joined #openstack-cyborg21:26
*** skelso has quit IRC22:10
*** mikeH has quit IRC22:24

Generated by 2.14.0 by Marius Gedminas - find it at!