Tuesday, 2017-03-28

*** thorst has joined #openstack-watcher00:12
*** thorst has quit IRC00:13
*** thorst has joined #openstack-watcher00:43
*** hidekazu has joined #openstack-watcher00:47
*** thorst has quit IRC01:02
*** thorst has joined #openstack-watcher01:11
*** thorst has quit IRC01:11
*** sanfern has quit IRC01:51
*** zhurong has joined #openstack-watcher01:53
*** karthikpr has joined #openstack-watcher02:04
*** thorst has joined #openstack-watcher02:12
*** thorst has quit IRC02:17
*** karthikpr has quit IRC03:00
*** karthikpr has joined #openstack-watcher03:04
*** karthikpr has quit IRC03:08
*** thorst has joined #openstack-watcher03:13
*** Tamayo has quit IRC03:32
*** thorst has quit IRC03:33
*** sanfern has joined #openstack-watcher03:40
*** zhurong has quit IRC03:57
openstackgerritlicanwei proposed openstack/watcher master: exception when running 'watcher audit start XXX'  https://review.openstack.org/45055503:58
*** karthikpr has joined #openstack-watcher04:05
*** hidekazu has quit IRC04:09
*** karthikpr has quit IRC04:10
*** hidekazu has joined #openstack-watcher04:13
*** thorst has joined #openstack-watcher04:30
*** thorst has quit IRC04:34
*** adisky_ has joined #openstack-watcher04:42
openstackgerritHidekazu Nakamura proposed openstack/watcher-specs master: Add spec for Cinder model integration  https://review.openstack.org/45017304:43
*** zhurong has joined #openstack-watcher04:55
*** karthikpr has joined #openstack-watcher04:57
*** karthikpr has quit IRC06:24
*** thorst has joined #openstack-watcher06:31
*** thorst has quit IRC06:35
*** vincentfrancoise has joined #openstack-watcher07:22
openstackgerritJeremy Zhang proposed openstack/watcher master: Use HostAddressOpt for opts that accept IP and hostnames  https://review.openstack.org/45059607:22
*** danpawlik has joined #openstack-watcher07:22
*** alexchadin has joined #openstack-watcher07:29
alexchadinvincentfrancoise: hi07:29
vincentfrancoisealexchadin: morning07:29
alexchadinvincentfrancoise: how are you?07:29
vincentfrancoisealexchadin: great and you?07:30
alexchadinvincentfrancoise: I'm ok :)07:30
alexchadinvincentfrancoise: I'd like to discuss background-jobs-ha with you07:30
vincentfrancoisealexchadin: ok (I have 15mn before my daily scrum)07:30
alexchadinvincentfrancoise: https://review.openstack.org/#/c/449659/1/watcher/db/sqlalchemy/job_store.py line 7207:31
vincentfrancoisealexchadin: what about the pickling?07:31
alexchadinvincentfrancoise: job_state contains 'func' key that is only reference to the method of class07:32
alexchadinvincentfrancoise: so when we add our jobs to db, apscheduler saves only link, not object07:32
alexchadinvincentfrancoise: it means that when we get job from db, we get the following error: TypeError: unbound method launch_audits_periodically() must be called with ContinuousAuditHandler instance as first argument (got nothing instead)07:33
alexchadinvincentfrancoise: fair enough. Cause ContinuousAuditHandler object is not initialised07:33
alexchadinvincentfrancoise: I propose to make ContinuousAuditHandler like singleton07:34
vincentfrancoisealexchadin: hum let me fully understand the problem07:34
vincentfrancoisealexchadin: I kind of get what you mean but I need to read the code to make more sense out of what you told me just now :p07:35
alexchadinvincentfrancoise: we save only link like watcher.decision_engine.audit.continuous:ContinuousAuditHandler.launch_audits_periodically07:35
alexchadinvincentfrancoise: ok ok07:35
vincentfrancoisealexchadin: ok so why is it not an object when it gets serialized?07:41
vincentfrancoisealexchadin: the way I understand is that what seems to be serialized is the actual class instead of the instance07:41
alexchadinapscheduler don't save instance or class, only a link to them07:42
alexchadinvincentfrancoise: here is launch of function after getting it from db: https://github.com/agronholm/apscheduler/blob/master/apscheduler/executors/base.py#L12507:43
vincentfrancoisealexchadin: oh ok so can't we re-initialize them when we unpickle them?07:43
alexchadinvincentfrancoise: as you can see, it tries to launch function without init07:43
alexchadinvincentfrancoise: I thought about it, but IMHO it isn't right way07:44
alexchadinvincentfrancoise: We will get a lot of ContinuousAuditHandler instances07:44
alexchadinvincentfrancoise: IMHO, ContinuousAuditHandler should be init once07:44
alexchadinvincentfrancoise: I need to go so let's meet later07:50
*** alexchadin has quit IRC07:51
openstackgerritSanthosh Fernandes proposed openstack/watcher master: Add gnocchi support in workload_balance strategy  https://review.openstack.org/45061508:05
*** acabot_ has joined #openstack-watcher08:06
*** acabot has quit IRC08:11
*** acabot_ has quit IRC08:11
*** acabot has joined #openstack-watcher08:11
*** acabot has quit IRC08:13
*** pksingh has joined #openstack-watcher08:20
*** acabot has joined #openstack-watcher08:30
*** acabot has left #openstack-watcher08:30
*** acabot has joined #openstack-watcher08:30
*** thorst has joined #openstack-watcher08:33
*** alexchadin has joined #openstack-watcher08:36
alexchadinvincentfrancoise: ping08:36
vincentfrancoisealexchadin: pong08:36
vincentfrancoisealexchadin: sorry I left for my meeting and forgot to tell you08:37
alexchadinvincentfrancoise: no worries08:37
vincentfrancoisealexchadin: ok so if I resume from where we left08:40
vincentfrancoisealexchadin: I agree that creating many instances isn't a good option08:41
alexchadinvincentfrancoise: DE instantiate ContinuousAuditHandler only once for now08:42
vincentfrancoisealexchadin: since they would all instantiate a self.scheduler08:42
alexchadinvincentfrancoise: +108:42
vincentfrancoisealexchadin: so IMHO, we either make the ContinuousAuditHandler a singleton08:43
vincentfrancoisealexchadin: or we make the BackgroundScheduler a singleton08:43
vincentfrancoisealexchadin: then the other problem I saw08:45
vincentfrancoisealexchadin: the "from apscheduler.schedulers import background" import in continuous.py should not be done like this08:45
alexchadinvincentfrancoise: I vote for the first option since rewriting BackgroundScheduler may call some collisions08:46
vincentfrancoisealexchadin: all background schedulers in Watcher should instead inherit from https://github.com/openstack/watcher/blob/master/watcher/common/scheduling.py#L2608:46
vincentfrancoisealexchadin: not rewriting08:46
vincentfrancoisealexchadin: IMHO, they should all rely on DecisionEngineSchedulingService https://github.com/openstack/watcher/blob/master/watcher/decision_engine/scheduling.py#L3508:48
vincentfrancoisealexchadin: and then DecisionEngineSchedulingService should actually be a singleton08:48
alexchadinvincentfrancoise: you suggest to inherit DecisionEngineSchedulingService?08:49
openstackgerritHidekazu Nakamura proposed openstack/watcher master: [WIP] Cinder model integration  https://review.openstack.org/45064208:52
vincentfrancoisealexchadin: this https://github.com/openstack/watcher/blob/master/watcher/decision_engine/audit/continuous.py#L4608:52
*** thorst has quit IRC08:52
vincentfrancoisealexchadin: should be a DecisionEngineSchedulingService instance08:52
alexchadinvincentfrancoise: ok08:52
vincentfrancoisealexchadin: and we should modify DecisionEngineSchedulingService to make it a singleton08:52
vincentfrancoisealexchadin: because right now it seems we already have 2 background scheduler running at the same time08:53
alexchadinvincentfrancoise: ah, yeah I see that BackgroundSchedulerService inherit background.BackgroundScheduler08:53
vincentfrancoisealexchadin: yeah that's why08:53
alexchadinvincentfrancoise: good point, let's see what we can do with it:)08:53
vincentfrancoisealexchadin: problem is that it will probably break many tests08:54
vincentfrancoisealexchadin: because adding singletons usually creates test dependencies08:54
alexchadinvincentfrancoise: but how we can run job then? it will still tries to run launch_audits_periodically without instantiating ContinuousAuditHandler08:55
vincentfrancoisealexchadin: so maybe you'll have to add a cleanup that would destroy the background scheduler singleton instance everytime to avoid that08:55
vincentfrancoisealexchadin: ah yeah so once this is done, I suggest we make launch_audits_periodically a classmethod08:57
vincentfrancoisealexchadin: the self.scheduler can then be replaced by an instantiation that would do nothing if the singleton is already created08:57
alexchadinvincentfrancoise: launch_audits_periodically adds self.execute_audit as method to execute08:57
vincentfrancoisealexchadin: ah I missed that one...08:58
alexchadinvincentfrancoise: there are dependencies after dependencies...08:58
vincentfrancoisealexchadin: ok so let me think about another solution then09:00
alexchadinI'll be back soon09:00
vincentfrancoisealexchadin: ok09:01
*** alexchadin has quit IRC09:16
*** necuser__ has quit IRC09:20
*** gabor_antal_ is now known as gabor_antal09:23
*** karthikpr has joined #openstack-watcher09:26
*** karthikpr has quit IRC09:31
*** acabot has quit IRC09:32
*** acabot has joined #openstack-watcher09:34
*** necuser__ has joined #openstack-watcher09:34
*** thorst has joined #openstack-watcher09:47
aspiersjwcroppe: we don't package it yet but I would really like to10:03
openstackgerritSanthosh Fernandes proposed openstack/watcher master: Add gnocchi support in VM-Workload-Consolidation strategy  https://review.openstack.org/45069310:05
openstackgerritZhong Luyao proposed openstack/watcher-specs master: Noisy Neighbor Strategy renamed cache occupancy strategy to noisy neighbor  https://review.openstack.org/39816210:06
*** vincentfrancoise has quit IRC10:16
*** thorst has quit IRC10:19
*** zhurong has quit IRC10:20
*** zhurong has joined #openstack-watcher10:28
*** sanfern has quit IRC10:28
*** acabot has quit IRC10:28
*** acabot has joined #openstack-watcher10:29
*** pksingh has quit IRC10:33
*** acabot has quit IRC10:41
*** alexchadin has joined #openstack-watcher10:43
*** alexchadin has quit IRC10:43
*** alexchadin has joined #openstack-watcher10:44
*** alexchadin has quit IRC10:48
aspiershi all, is there a concise summary anywhere of the difference between Watcher and Neat? If not, it would be really helpful to create one10:51
*** alexchadin has joined #openstack-watcher10:52
aspiersit seems that Neat is only focusing on VM consolidation, right? but seems to have done some great work in that area10:58
aspiersit seems a shame that they implemented it as a separate project though10:59
aspiersI wonder why that happened10:59
openstackgerritZhong Luyao proposed openstack/watcher-specs master: Noisy Neighbor Strategy renamed cache occupancy strategy to noisy neighbor  https://review.openstack.org/39816211:02
openstackgerritlicanwei proposed openstack/watcher master: Add action description  https://review.openstack.org/45072011:11
*** thorst has joined #openstack-watcher11:20
aspiersalso I get the impression they take a decentralized approach11:23
*** zhurong has quit IRC11:25
*** thorst has quit IRC11:25
*** vincentfrancoise has joined #openstack-watcher11:27
*** alexchadin has quit IRC11:30
*** openstackgerrit has quit IRC11:33
vincentfrancoiseaspiers: hi11:39
aspiershi vincentfrancoise11:39
vincentfrancoiseaspiers: just read about the comparison with openstack neat11:40
vincentfrancoiseaspiers: basically they were focusing on the consolidation whereas watcher has taken a more generic path to address any goal11:41
vincentfrancoiseaspiers: also I didn't test it by their only algorithm is bin packing, which is also the case for basic_consolidation in Watcher11:42
vincentfrancoiseaspiers: and at the time, the solution wasn't really well maintained and it seemed the project was dying out11:44
vincentfrancoiseaspiers: so it made more sense for us to start from a clean slate (at least that's what what I was told when I also asked the question)11:45
*** thorst has joined #openstack-watcher11:45
aspiersoh, so Neat came first? interesting11:45
vincentfrancoiseaspiers: as for explaining the difference, since they were not within the openstack big tent or even partially integrated within the community, we didn't really bother11:46
vincentfrancoiseaspiers: if you look closely into the readme, you can see this is a PhD project and I assume the guy just dropped the project at the end of it11:48
aspiersvincentfrancoise: seems that there has been very little activity since 2013 https://github.com/beloglazov/openstack-neat/graphs/contributors11:50
aspiersthe guy now works for Google, so I assume he's not working on OpenStack any more11:50
aspiersthat's a shame11:50
aspiersnot much activity here either https://groups.google.com/forum/#!forum/openstack-neat11:51
*** openstackgerrit has joined #openstack-watcher11:58
openstackgerritaditi sharma proposed openstack/watcher master: WIP: run watcher-api with mod-wsgi  https://review.openstack.org/45074011:58
*** karthikpr has joined #openstack-watcher13:27
*** jwcroppe has quit IRC13:28
*** karthik__ has joined #openstack-watcher13:29
*** karthikpr has quit IRC13:31
*** karthik__ has quit IRC13:34
*** jwcroppe has joined #openstack-watcher13:42
*** sanfern has joined #openstack-watcher13:43
*** karthikpr has joined #openstack-watcher13:47
*** karthikpr has quit IRC13:52
*** karthikpr has joined #openstack-watcher14:05
*** karthikpr has quit IRC14:09
*** Tamayo has joined #openstack-watcher14:10
*** gabor_antal has quit IRC15:05
*** karthikpr has joined #openstack-watcher15:10
*** karthikpr has quit IRC15:15
*** vincentfrancois1 has joined #openstack-watcher15:18
*** sanfern has quit IRC15:19
*** karthikpr has joined #openstack-watcher15:27
*** sanfern has joined #openstack-watcher15:31
*** karthikpr has quit IRC15:32
*** alexchadin has joined #openstack-watcher15:45
alexchadinvincentfrancoise: ping15:45
alexchadinaspiers: hi15:47
alexchadinaspiers: I agree with you about Neat achievements, Anton did a great work15:47
aspiersyes it looks impressive15:48
aspiersI met this guy https://cs.uwaterloo.ca/~mkarsten/ earlier this year - he is interested in this field and was asking about Neat, Watcher etc.15:49
alexchadinaspiers: I've studied Anton's work, some of his suggests was used to build Nova-LoadBalancer, which then has been integrated into Watcher15:50
alexchadinaspiers: Martin is welcome here!15:51
aspiersalexchadin: I suggested him to join :)15:51
alexchadinaspiers: great! We are appreciate fresh looks and minds :)15:52
aspiersI thought so ;)15:52
alexchadinaspiers: back to Neat, unfortunately this project was made in different code-style unlike OpenStack15:53
alexchadinaspiers: each great ideas should be wrapped in appropriate form15:54
*** vincentfrancoise has quit IRC15:56
*** alexchadin has quit IRC16:00
*** alexchadin has joined #openstack-watcher16:19
*** gabor_antal has joined #openstack-watcher16:43
*** gabor_antal_ has joined #openstack-watcher16:55
*** gabor_antal has quit IRC16:55
*** alexchadin has quit IRC16:59
*** gabor_antal_ is now known as gabor_antal17:14
*** wootehfoot has joined #openstack-watcher19:04
*** hidekazu has quit IRC20:26
*** wootehfoot has quit IRC20:30
*** wootehfoot has joined #openstack-watcher20:37
*** ianychoi has quit IRC21:07
*** ianychoi has joined #openstack-watcher21:08
*** thorst has quit IRC21:51
*** wootehfoot has quit IRC22:11
*** jwcroppe has quit IRC22:49
*** jwcroppe has joined #openstack-watcher22:49
*** jwcroppe has quit IRC22:50
*** jwcroppe has joined #openstack-watcher22:51
*** thorst has joined #openstack-watcher22:51
*** thorst has quit IRC22:55
*** jwcroppe has quit IRC22:56
*** thorst has joined #openstack-watcher23:08
*** jwcroppe has joined #openstack-watcher23:22
*** thorst has quit IRC23:25
*** thorst has joined #openstack-watcher23:25
*** thorst has quit IRC23:29
*** thorst has joined #openstack-watcher23:56
*** hidekazu has joined #openstack-watcher23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!