| *** vladikr__ has quit IRC | 00:00 | |
| arvindn05 | i think either of the 2 options we boiled down to work for me....i put this in open dicussion for tommorow's meeting...hopefully we get concensus then :) | 00:01 |
|---|---|---|
| *** gjayavelu has quit IRC | 00:04 | |
| *** gjayavelu has joined #openstack-nova | 00:09 | |
| *** Nel1x has joined #openstack-nova | 00:09 | |
| *** arvindn05 is now known as arvindn05_away | 00:13 | |
| *** lbragstad has quit IRC | 00:17 | |
| *** mriedem_inlaws has quit IRC | 00:23 | |
| openstackgerrit | Jake Yip proposed openstack/nova master: Add --before to nova-manage db archive_deleted_rows https://review.openstack.org/556751 | 00:25 |
| *** yamamoto has joined #openstack-nova | 00:36 | |
| *** BrinZhang has quit IRC | 00:38 | |
| *** BrinZhang has joined #openstack-nova | 00:39 | |
| *** mriedem has joined #openstack-nova | 00:41 | |
| *** yamamoto has quit IRC | 00:41 | |
| *** fragatin_ has quit IRC | 00:42 | |
| *** hoangcx has joined #openstack-nova | 00:43 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add nova-manage placement heal_allocations CLI https://review.openstack.org/565886 | 00:46 |
| *** r-daneel has joined #openstack-nova | 00:46 | |
| *** r-daneel_ has joined #openstack-nova | 00:49 | |
| *** r-daneel has quit IRC | 00:50 | |
| *** r-daneel_ is now known as r-daneel | 00:50 | |
| *** jichen has joined #openstack-nova | 00:52 | |
| *** phuongnh has joined #openstack-nova | 01:04 | |
| *** threestrands has joined #openstack-nova | 01:05 | |
| *** r-daneel has quit IRC | 01:06 | |
| *** gjayavelu has quit IRC | 01:06 | |
| *** yamahata has quit IRC | 01:08 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: WIP new migration threads control https://review.openstack.org/563505 | 01:13 |
| *** zhaochao has joined #openstack-nova | 01:16 | |
| *** Guest10014 is now known as jlvillal | 01:19 | |
| *** fragatina has joined #openstack-nova | 01:22 | |
| openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova-specs master: Complex (Anti)-Affinity Policies https://review.openstack.org/546925 | 01:23 |
| *** Tom-Tom has quit IRC | 01:28 | |
| *** Tom-Tom has joined #openstack-nova | 01:28 | |
| *** namnh has joined #openstack-nova | 01:31 | |
| *** namnh has quit IRC | 01:31 | |
| *** Tom-Tom has quit IRC | 01:33 | |
| *** Nil_ has quit IRC | 01:34 | |
| *** yamamoto has joined #openstack-nova | 01:38 | |
| *** dpawlik has joined #openstack-nova | 01:44 | |
| *** yamamoto has quit IRC | 01:44 | |
| *** lei-zh has joined #openstack-nova | 01:47 | |
| *** dpawlik has quit IRC | 01:48 | |
| *** Tom-Tom has joined #openstack-nova | 01:49 | |
| *** yamamoto has joined #openstack-nova | 01:50 | |
| *** r-daneel has joined #openstack-nova | 01:53 | |
| *** bkopilov has quit IRC | 01:57 | |
| *** bkopilov_ has quit IRC | 01:57 | |
| *** gcb has joined #openstack-nova | 02:00 | |
| *** hongbin has joined #openstack-nova | 02:12 | |
| *** annp has joined #openstack-nova | 02:12 | |
| *** tiendc has joined #openstack-nova | 02:16 | |
| *** psachin has joined #openstack-nova | 02:18 | |
| *** lei-zh has quit IRC | 02:19 | |
| *** lei-zh has joined #openstack-nova | 02:19 | |
| *** gjayavelu has joined #openstack-nova | 02:21 | |
| *** lbragstad has joined #openstack-nova | 02:24 | |
| *** dougshelley66 has quit IRC | 02:30 | |
| *** moshele has joined #openstack-nova | 02:35 | |
| *** moshele has quit IRC | 02:42 | |
| *** tbachman has quit IRC | 02:46 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix the request context in ServiceFixture https://review.openstack.org/558160 | 02:52 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova-manage placement heal_allocations CLI https://review.openstack.org/565886 | 02:52 |
| *** mriedem has quit IRC | 02:52 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add multi-cell negative test for cold migration with target host https://review.openstack.org/524027 | 02:54 |
| *** spsurya has joined #openstack-nova | 02:54 | |
| *** moshele has joined #openstack-nova | 02:55 | |
| *** tbachman has joined #openstack-nova | 02:57 | |
| *** moshele has quit IRC | 03:02 | |
| *** zhangbailin_ has joined #openstack-nova | 03:04 | |
| *** BrinZhang has quit IRC | 03:07 | |
| *** dpawlik has joined #openstack-nova | 03:10 | |
| *** bkopilov has joined #openstack-nova | 03:13 | |
| openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: Spawn and destroy function of z/VM driver https://review.openstack.org/527658 | 03:14 |
| openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add snapshot function https://review.openstack.org/534240 | 03:14 |
| openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add power actions https://review.openstack.org/543340 | 03:14 |
| openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add get console output https://review.openstack.org/543344 | 03:14 |
| *** dpawlik has quit IRC | 03:14 | |
| *** bkopilov_ has joined #openstack-nova | 03:16 | |
| *** bkopilov__ has joined #openstack-nova | 03:16 | |
| *** nicolasbock has quit IRC | 03:21 | |
| *** liverpooler has quit IRC | 03:31 | |
| *** hamzy has joined #openstack-nova | 03:34 | |
| *** udesale has joined #openstack-nova | 03:36 | |
| *** Gorian has quit IRC | 03:36 | |
| *** anquan_wu has joined #openstack-nova | 03:37 | |
| *** namnh has joined #openstack-nova | 03:38 | |
| *** liverpooler has joined #openstack-nova | 03:40 | |
| *** Gorian has joined #openstack-nova | 03:41 | |
| *** hongbin has quit IRC | 03:42 | |
| *** fragatina has quit IRC | 03:43 | |
| *** lpetrut has joined #openstack-nova | 03:44 | |
| *** jogo has quit IRC | 03:45 | |
| *** janki has joined #openstack-nova | 03:47 | |
| *** jogo has joined #openstack-nova | 03:50 | |
| *** jchhatbar has joined #openstack-nova | 04:00 | |
| *** germs has quit IRC | 04:02 | |
| *** liverpooler has quit IRC | 04:03 | |
| *** janki has quit IRC | 04:03 | |
| *** gjayavelu has quit IRC | 04:05 | |
| *** fragatina has joined #openstack-nova | 04:12 | |
| *** fragatin_ has joined #openstack-nova | 04:13 | |
| *** fragatina has quit IRC | 04:17 | |
| *** lpetrut has quit IRC | 04:18 | |
| *** hoonetorg has quit IRC | 04:21 | |
| *** lpetrut has joined #openstack-nova | 04:21 | |
| *** jpena|off has quit IRC | 04:29 | |
| *** weshay has quit IRC | 04:29 | |
| *** markmc has quit IRC | 04:30 | |
| *** rnoriega_ has quit IRC | 04:31 | |
| *** jpena|off has joined #openstack-nova | 04:31 | |
| *** ltomasbo has quit IRC | 04:31 | |
| *** markmc has joined #openstack-nova | 04:32 | |
| *** rnoriega has joined #openstack-nova | 04:33 | |
| *** abhishekk has joined #openstack-nova | 04:34 | |
| *** weshay has joined #openstack-nova | 04:35 | |
| *** Tom-Tom has quit IRC | 04:41 | |
| *** Nel1x has quit IRC | 04:41 | |
| *** gyee has quit IRC | 04:41 | |
| *** Nel1x has joined #openstack-nova | 04:41 | |
| *** lei-zh has quit IRC | 04:42 | |
| *** lpetrut has quit IRC | 04:43 | |
| *** Tom-Tom has joined #openstack-nova | 04:43 | |
| *** links has joined #openstack-nova | 04:46 | |
| *** Tom-Tom has quit IRC | 04:47 | |
| *** anquan_wu has quit IRC | 04:48 | |
| *** jichen has quit IRC | 04:48 | |
| *** Nel1x has quit IRC | 04:54 | |
| openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add power actions https://review.openstack.org/543340 | 04:55 |
| openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add get console output https://review.openstack.org/543344 | 04:55 |
| *** sree_ has joined #openstack-nova | 04:56 | |
| *** sree_ is now known as Guest41888 | 04:57 | |
| *** Guest41888 has quit IRC | 04:57 | |
| *** Zames has joined #openstack-nova | 05:06 | |
| *** yamahata has joined #openstack-nova | 05:06 | |
| *** markvoelker has quit IRC | 05:08 | |
| *** Zames has quit IRC | 05:08 | |
| *** anquan_wu has joined #openstack-nova | 05:10 | |
| *** dpawlik has joined #openstack-nova | 05:11 | |
| *** dpawlik has quit IRC | 05:15 | |
| *** moshele has joined #openstack-nova | 05:16 | |
| *** Eran_Kuris has joined #openstack-nova | 05:24 | |
| *** armaan has joined #openstack-nova | 05:28 | |
| *** Tom-Tom has joined #openstack-nova | 05:29 | |
| *** Tom-Tom has quit IRC | 05:33 | |
| *** sree_ has joined #openstack-nova | 05:34 | |
| *** sree_ is now known as Guest22258 | 05:34 | |
| *** wolverineav has joined #openstack-nova | 05:34 | |
| *** evin has quit IRC | 05:39 | |
| *** sridharg has joined #openstack-nova | 05:47 | |
| *** wolverineav has quit IRC | 05:52 | |
| openstackgerrit | Vishakha Agarwal proposed openstack/nova master: Re-using the code of os brick cinder https://review.openstack.org/565703 | 05:55 |
| *** hoonetorg has joined #openstack-nova | 05:55 | |
| openstackgerrit | Vishakha Agarwal proposed openstack/nova master: Re-using the code of os brick cinder https://review.openstack.org/565703 | 06:08 |
| *** markvoelker has joined #openstack-nova | 06:09 | |
| *** evin has joined #openstack-nova | 06:12 | |
| *** andreas_s has joined #openstack-nova | 06:16 | |
| *** Tom-Tom has joined #openstack-nova | 06:16 | |
| *** mingyu has joined #openstack-nova | 06:24 | |
| *** lpetrut has joined #openstack-nova | 06:29 | |
| *** anquan_wu_ has joined #openstack-nova | 06:32 | |
| *** anquan_wu has quit IRC | 06:36 | |
| *** anquan_wu_ has quit IRC | 06:36 | |
| *** kholkina has joined #openstack-nova | 06:38 | |
| *** lei-zh has joined #openstack-nova | 06:39 | |
| *** damien_r has joined #openstack-nova | 06:40 | |
| *** markvoelker has quit IRC | 06:44 | |
| *** hoonetorg has quit IRC | 06:44 | |
| *** bhagyashris has joined #openstack-nova | 06:46 | |
| *** zhuli has quit IRC | 06:50 | |
| *** hoonetorg has joined #openstack-nova | 06:50 | |
| *** pcaruana has joined #openstack-nova | 06:53 | |
| *** do3meli has joined #openstack-nova | 06:55 | |
| *** rcernin has quit IRC | 06:58 | |
| *** threestrands has quit IRC | 06:58 | |
| *** ccamacho has joined #openstack-nova | 06:58 | |
| *** aloga has joined #openstack-nova | 07:03 | |
| *** dpawlik has joined #openstack-nova | 07:12 | |
| *** moshele has quit IRC | 07:14 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: WIP new migration threads control https://review.openstack.org/563505 | 07:16 |
| *** dpawlik has quit IRC | 07:16 | |
| *** tesseract has joined #openstack-nova | 07:18 | |
| *** Tom-Tom has quit IRC | 07:27 | |
| *** sahid has joined #openstack-nova | 07:29 | |
| *** ragiman_ has joined #openstack-nova | 07:33 | |
| *** jpena|off is now known as jpena | 07:35 | |
| *** tssurya has joined #openstack-nova | 07:35 | |
| *** markvoelker has joined #openstack-nova | 07:41 | |
| *** alexchadin has joined #openstack-nova | 07:43 | |
| sahid | stephenfin: if you can have a look to the serie https://review.openstack.org/#/c/561912/ | 07:43 |
| *** Tom-Tom has joined #openstack-nova | 07:45 | |
| *** amoralej|off is now known as amoralej | 07:50 | |
| *** anquan_wu has joined #openstack-nova | 07:58 | |
| bauzas | morning stackers | 08:00 |
| *** anquan_wu_ has joined #openstack-nova | 08:00 | |
| gibi | good morning nova | 08:02 |
| *** mgoddard has joined #openstack-nova | 08:02 | |
| *** anquan_wu has quit IRC | 08:04 | |
| *** moshele has joined #openstack-nova | 08:08 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Consider nested RPs in get_all_with_shared https://review.openstack.org/556450 | 08:08 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Support shared and nested allocation candidates https://review.openstack.org/556514 | 08:08 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Support nested alloc cands without shared RPs https://review.openstack.org/561842 | 08:08 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Return all resources in provider_summaries https://review.openstack.org/558045 | 08:08 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Return all nested providers in tree https://review.openstack.org/559480 | 08:08 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Return resource providers without inventories https://review.openstack.org/559554 | 08:08 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Support nested allocation candidates in placement https://review.openstack.org/565487 | 08:08 |
| *** markvoelker has quit IRC | 08:13 | |
| *** yamahata has quit IRC | 08:14 | |
| *** wolverineav has joined #openstack-nova | 08:15 | |
| *** tuanla____ has joined #openstack-nova | 08:17 | |
| *** lucas-afk is now known as lucasagomes | 08:17 | |
| *** mdnadeem has joined #openstack-nova | 08:21 | |
| gibi | bauzas: could you look at a small spec template update that already has +2 from mriedem? https://review.openstack.org/#/c/562265/ | 08:23 |
| bauzas | sure | 08:24 |
| gibi | bauzas: thanks | 08:24 |
| bauzas | done | 08:25 |
| bauzas | with a comment but +Wd | 08:25 |
| *** wolverineav has quit IRC | 08:27 | |
| gibi | bauzas: I can quickfix your nit then you can fastapprove it | 08:27 |
| *** alexchad_ has joined #openstack-nova | 08:27 | |
| *** wolverineav has joined #openstack-nova | 08:28 | |
| bauzas | gibi: nah, no worries | 08:28 |
| bauzas | I understand what's an API entity | 08:28 |
| bauzas | no need for a followup | 08:28 |
| bauzas | and I'm not a API SIG expert ;) | 08:28 |
| bauzas | but I use a vocabulary | 08:28 |
| bauzas | REST verbs : GET, PUT, etc. | 08:29 |
| gibi | bauzas: OK then let's merge it as is :) | 08:29 |
| bauzas | REST resource : /servers, etc. | 08:29 |
| gibi | bauzas: I think your vocabulary is the correct one | 08:29 |
| bauzas | actions are not RESTful | 08:29 |
| *** mdbooth has joined #openstack-nova | 08:30 | |
| bauzas | but let's call it a subcollection | 08:30 |
| kashyap | Those who can +2 backports, can you put this through please: https://review.openstack.org/#/c/565668/ | 08:30 |
| kashyap | (It already has a +2.) | 08:30 |
| *** anquan_wu__ has joined #openstack-nova | 08:30 | |
| *** alexchadin has quit IRC | 08:31 | |
| bauzas | kashyap: just did | 08:31 |
| *** mdbooth_ has joined #openstack-nova | 08:31 | |
| kashyap | bauzas: Fantastic. I've got two more, mind looking there too? | 08:31 |
| *** liuzz_ has joined #openstack-nova | 08:31 | |
| bauzas | kashyap: see matt's comment | 08:31 |
| * kashyap looks | 08:31 | |
| bauzas | kashyap: https://review.openstack.org/#/c/565670/1 | 08:32 |
| * kashyap clicks | 08:32 | |
| openstackgerrit | Merged openstack/nova-specs master: Suggest when to propose notification changes https://review.openstack.org/562265 | 08:32 |
| kashyap | bauzas: Actually, that's what I did | 08:32 |
| kashyap | But let me double-check | 08:32 |
| *** wolverineav has quit IRC | 08:32 | |
| gibi | bauzas: subcollection sound weird. as PUT and GET is called verb, the 'action' term for our unRESTful actions makes more sense to me | 08:33 |
| *** liuzz has quit IRC | 08:33 | |
| *** anquan_wu_ has quit IRC | 08:34 | |
| *** mdbooth has quit IRC | 08:34 | |
| *** derekh has joined #openstack-nova | 08:36 | |
| kashyap | bauzas++ | 08:36 |
| bauzas | gibi: fair enoguh | 08:38 |
| bauzas | gibi: let's call it an API action | 08:38 |
| gibi | bauzas: action on an API resource :) | 08:38 |
| bauzas | gibi: but not a REST action, which is an oxymore | 08:38 |
| bauzas | anyway, we're furiously ratholing | 08:39 |
| gibi | a good morning rathole to get us up to speed | 08:39 |
| *** links has quit IRC | 08:43 | |
| *** links has joined #openstack-nova | 08:44 | |
| *** Guest22258 has quit IRC | 08:45 | |
| *** hemna_ has quit IRC | 08:46 | |
| *** anquan_wu_ has joined #openstack-nova | 08:47 | |
| *** namnh has quit IRC | 08:48 | |
| *** trungnv has quit IRC | 08:48 | |
| *** tiendc has quit IRC | 08:48 | |
| *** tiendc has joined #openstack-nova | 08:48 | |
| *** namnh has joined #openstack-nova | 08:48 | |
| *** trungnv has joined #openstack-nova | 08:48 | |
| *** anquan_wu__ has quit IRC | 08:51 | |
| *** slaweq has quit IRC | 08:56 | |
| *** slaweq has joined #openstack-nova | 08:57 | |
| *** markvoelker has joined #openstack-nova | 09:10 | |
| *** dpawlik has joined #openstack-nova | 09:13 | |
| *** yingjun has joined #openstack-nova | 09:16 | |
| *** dpawlik has quit IRC | 09:17 | |
| openstackgerrit | Merged openstack/nova-specs master: Document Rocky review priorities https://review.openstack.org/557796 | 09:23 |
| openstackgerrit | Vishakha Agarwal proposed openstack/nova master: Re-using the code of os brick cinder https://review.openstack.org/565703 | 09:29 |
| *** khappone has joined #openstack-nova | 09:32 | |
| *** khappone_ has quit IRC | 09:36 | |
| openstackgerrit | jichenjc proposed openstack/nova master: [doc] enhance admin/configuration/api.rst https://review.openstack.org/566008 | 09:38 |
| *** blkart has quit IRC | 09:40 | |
| *** blkart has joined #openstack-nova | 09:41 | |
| *** yassine has joined #openstack-nova | 09:42 | |
| *** yassine has quit IRC | 09:42 | |
| *** yassine has joined #openstack-nova | 09:43 | |
| *** yassine is now known as Guest12236 | 09:43 | |
| *** markvoelker has quit IRC | 09:44 | |
| *** yingjun has quit IRC | 09:44 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add tests for alloc_cands with member_of https://review.openstack.org/561399 | 09:46 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Fix member_of with sharing providers https://review.openstack.org/561400 | 09:46 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Expand member_of functional test cases https://review.openstack.org/566011 | 09:46 |
| openstackgerrit | jichenjc proposed openstack/nova master: [doc]remove nova-cert leftover in doc https://review.openstack.org/566013 | 09:48 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Don't use single backticks in man pages https://review.openstack.org/540887 | 09:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Start using openstackdoctheme's extlink extension https://review.openstack.org/540888 | 09:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Fix file permissions https://review.openstack.org/566014 | 09:49 |
| *** mdbooth_ has quit IRC | 09:50 | |
| openstackgerrit | Merged openstack/nova stable/queens: libvirt: Make `cpu_model_extra_flags` case-insensitive for real https://review.openstack.org/565668 | 09:51 |
| *** lei-zh has quit IRC | 09:52 | |
| *** Eran_Kuris has quit IRC | 09:54 | |
| *** mdbooth has joined #openstack-nova | 10:01 | |
| *** rdopiera has joined #openstack-nova | 10:06 | |
| *** rdopiera has left #openstack-nova | 10:06 | |
| *** alexchad_ has quit IRC | 10:07 | |
| *** namnh has quit IRC | 10:11 | |
| *** ragiman_ has quit IRC | 10:13 | |
| *** bkopilov has quit IRC | 10:13 | |
| *** bkopilov_ has quit IRC | 10:14 | |
| *** bkopilov__ has quit IRC | 10:14 | |
| *** sahid has quit IRC | 10:17 | |
| *** anquan_wu_ has quit IRC | 10:18 | |
| *** yamamoto has quit IRC | 10:21 | |
| *** hoangcx has quit IRC | 10:21 | |
| *** jmccarthy has joined #openstack-nova | 10:25 | |
| *** nicolasbock has joined #openstack-nova | 10:31 | |
| openstackgerrit | Jianghua Wang proposed openstack/nova master: XenAPI: deprecate the config for image handler class path https://review.openstack.org/497201 | 10:31 |
| openstackgerrit | Jianghua Wang proposed openstack/nova master: XenAPI: define a new image handler to use vdi streaming https://review.openstack.org/486475 | 10:31 |
| *** abhishekk has quit IRC | 10:31 | |
| openstackgerrit | Jianghua Wang proposed openstack/nova master: XenAPI: document change for supporting image handler https://review.openstack.org/563077 | 10:31 |
| *** edmondsw has joined #openstack-nova | 10:31 | |
| *** mdbooth has quit IRC | 10:35 | |
| *** edmondsw has quit IRC | 10:36 | |
| *** phuongnh has quit IRC | 10:36 | |
| *** yamamoto has joined #openstack-nova | 10:37 | |
| *** Tom-Tom has quit IRC | 10:37 | |
| *** annp has quit IRC | 10:40 | |
| *** markvoelker has joined #openstack-nova | 10:41 | |
| *** alexchadin has joined #openstack-nova | 10:44 | |
| *** mdbooth has joined #openstack-nova | 10:48 | |
| *** alexchadin has quit IRC | 10:49 | |
| *** dtantsur|afk is now known as dtantsur | 10:50 | |
| *** Eran_Kuris has joined #openstack-nova | 10:51 | |
| *** fragatin_ has quit IRC | 10:52 | |
| *** dougshelley66 has joined #openstack-nova | 10:53 | |
| *** fragatina has joined #openstack-nova | 10:53 | |
| *** jmccarthy has left #openstack-nova | 10:54 | |
| kashyap | bauzas: and stephenfin Can you put this through: https://review.openstack.org/#/c/565707/ | 10:54 |
| kashyap | Quite mechanical change | 10:54 |
| *** alexchadin has joined #openstack-nova | 10:54 | |
| kashyap | Or gibi even :-) | 10:54 |
| stephenfin | kashyap: done | 10:55 |
| kashyap | Splendid | 10:55 |
| *** jpena is now known as jpena|lunch | 10:58 | |
| openstackgerrit | Merged openstack/nova master: Cleanup patch for the cell-disable series https://review.openstack.org/559299 | 10:58 |
| *** tbachman has quit IRC | 10:58 | |
| *** slaweq has quit IRC | 10:58 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova stable/pike: libvirt: Make `cpu_model_extra_flags` case-insensitive for real https://review.openstack.org/565670 | 10:59 |
| gibi | kashyap: done :) | 10:59 |
| kashyap | Thanks! | 11:05 |
| kashyap | gibi: I can't really figure why this is the only one that is failing: https://review.openstack.org/#/c/564012/ | 11:07 |
| kashyap | (Where you did the 'recheck') | 11:07 |
| moshele | stephenfin: hi can you review this pike backport https://review.openstack.org/#/c/563501/? | 11:07 |
| gibi | kashyap: based on the fail tempest test case names it feels unrelated to me | 11:08 |
| gibi | kashyap: if the recheck fails with similar result then we have to dig | 11:08 |
| kashyap | Indeed. Anyway, let's see if the new 'recheck' helps | 11:08 |
| * kashyap is reasonably pessimistic that it will fail | 11:08 | |
| kashyap | Anyway, I have other irons in the fire...So onto them | 11:09 |
| *** lucasagomes is now known as lucas-hungry | 11:10 | |
| *** slaweq has joined #openstack-nova | 11:10 | |
| *** mingyu has quit IRC | 11:11 | |
| *** kotra03_2 has joined #openstack-nova | 11:12 | |
| *** markvoelker has quit IRC | 11:14 | |
| *** slaweq has quit IRC | 11:15 | |
| *** dougshelley66 has quit IRC | 11:21 | |
| openstackgerrit | Merged openstack/nova master: Include only required fields in ironic node cache https://review.openstack.org/539508 | 11:21 |
| *** opetrenko has quit IRC | 11:26 | |
| *** tuanla____ has quit IRC | 11:26 | |
| *** opetrenko has joined #openstack-nova | 11:27 | |
| *** pchavva has joined #openstack-nova | 11:35 | |
| *** bkopilov has joined #openstack-nova | 11:39 | |
| *** bkopilov_ has joined #openstack-nova | 11:40 | |
| *** bkopilov__ has joined #openstack-nova | 11:41 | |
| jaypipes | stephenfin: answered your query on the query on https://review.openstack.org/#/c/561315 :) | 11:46 |
| *** yamamoto has quit IRC | 11:48 | |
| *** eharney has quit IRC | 11:49 | |
| *** salv-orlando has joined #openstack-nova | 11:53 | |
| *** amoralej is now known as amoralej|off | 11:53 | |
| *** amoralej|off is now known as amoralej|lunch | 11:54 | |
| *** yamamoto has joined #openstack-nova | 11:55 | |
| *** yamamoto_ has joined #openstack-nova | 11:56 | |
| *** yamamoto_ has quit IRC | 11:57 | |
| *** mingyu has joined #openstack-nova | 11:58 | |
| *** salv-orlando has quit IRC | 11:58 | |
| *** yamamoto_ has joined #openstack-nova | 11:59 | |
| *** yamamoto has quit IRC | 12:00 | |
| *** armaan has quit IRC | 12:00 | |
| *** armaan has joined #openstack-nova | 12:01 | |
| *** wolverineav has joined #openstack-nova | 12:01 | |
| *** yamamoto_ has quit IRC | 12:02 | |
| *** gcb has quit IRC | 12:09 | |
| *** jchorin has joined #openstack-nova | 12:10 | |
| *** jchorin has left #openstack-nova | 12:11 | |
| *** jpena|lunch is now known as jpena | 12:11 | |
| *** markvoelker has joined #openstack-nova | 12:11 | |
| *** tbachman has joined #openstack-nova | 12:13 | |
| *** yamamoto has joined #openstack-nova | 12:13 | |
| *** tiendc has quit IRC | 12:14 | |
| *** edmondsw has joined #openstack-nova | 12:16 | |
| *** lucas-hungry is now known as lucasagomes | 12:16 | |
| *** markvoelker has quit IRC | 12:20 | |
| *** markvoelker has joined #openstack-nova | 12:20 | |
| *** yamamoto has quit IRC | 12:20 | |
| *** yamamoto has joined #openstack-nova | 12:21 | |
| *** abhishekk has joined #openstack-nova | 12:23 | |
| *** yamamoto has quit IRC | 12:26 | |
| *** moshele has quit IRC | 12:27 | |
| openstackgerrit | Jim Rollenhagen proposed openstack/nova master: Ironic: report 0 for vcpus/memory_mb/disk_gb resources https://review.openstack.org/565841 | 12:28 |
| jroll | dtantsur: fyi, rebased ^ on vdrok's stuff with a depends-on for the scheduler changes instead | 12:28 |
| *** moshele has joined #openstack-nova | 12:29 | |
| dtantsur | thanks! | 12:31 |
| *** liverpooler has joined #openstack-nova | 12:31 | |
| stephenfin | jaypipes: Makes sense. Mind addressing these two comments and I can ack https://review.openstack.org/#/c/561315/15/nova/api/openstack/placement/util.py@533 https://review.openstack.org/#/c/561315/15/nova/api/openstack/placement/handlers/resource_provider.py@200 | 12:32 |
| *** liverpoo1er has joined #openstack-nova | 12:32 | |
| stephenfin | bauzas, gibi: Easy doc win here, if you fancy taking a look https://review.openstack.org/#/c/540888/ | 12:33 |
| gibi | stephenfin: looking | 12:33 |
| *** liverpooler has quit IRC | 12:36 | |
| *** zhangbailin_ has quit IRC | 12:36 | |
| gibi | stephenfin: you missed one link :) https://review.openstack.org/#/c/540888/3/api-guide/source/server_concepts.rst@895 | 12:47 |
| gibi | stephenfin: fix it quickly and I'm +2 | 12:47 |
| *** lyan has joined #openstack-nova | 12:47 | |
| *** lyan is now known as Guest49729 | 12:47 | |
| *** armaan has quit IRC | 12:54 | |
| *** mvk has quit IRC | 12:55 | |
| *** nicolasbock has quit IRC | 12:57 | |
| *** liuzz has joined #openstack-nova | 12:58 | |
| *** psachin has quit IRC | 12:59 | |
| *** liuzz_ has quit IRC | 13:00 | |
| *** READ10 has joined #openstack-nova | 13:02 | |
| *** jchhatbar has quit IRC | 13:02 | |
| openstackgerrit | Jay Pipes proposed openstack/nova master: support multiple member_of qparams https://review.openstack.org/561315 | 13:02 |
| jaypipes | stephenfin: righty-o. all done ^ | 13:02 |
| *** mchlumsky has joined #openstack-nova | 13:07 | |
| * gibi is wondering: "do I need to run the nova meeting today?" | 13:09 | |
| *** lionx has joined #openstack-nova | 13:09 | |
| *** slaweq has joined #openstack-nova | 13:10 | |
| *** yamamoto has joined #openstack-nova | 13:11 | |
| *** lionx has quit IRC | 13:11 | |
| openstackgerrit | Merged openstack/nova master: trivial: Fix file permissions https://review.openstack.org/566014 | 13:11 |
| *** dpawlik has joined #openstack-nova | 13:12 | |
| *** nicolasbock has joined #openstack-nova | 13:14 | |
| *** slaweq has quit IRC | 13:16 | |
| *** dpawlik has quit IRC | 13:17 | |
| *** yamamoto has quit IRC | 13:17 | |
| *** yamamoto has joined #openstack-nova | 13:17 | |
| *** slaweq has joined #openstack-nova | 13:19 | |
| *** mriedem has joined #openstack-nova | 13:20 | |
| *** yamamoto has quit IRC | 13:22 | |
| *** Eran_Kuris has quit IRC | 13:24 | |
| *** sahid has joined #openstack-nova | 13:25 | |
| *** amoralej|lunch is now known as amoralej | 13:26 | |
| *** mvk has joined #openstack-nova | 13:28 | |
| *** moshele has quit IRC | 13:31 | |
| *** eharney has joined #openstack-nova | 13:32 | |
| *** tetsuro has joined #openstack-nova | 13:33 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Start using openstackdoctheme's extlink extension https://review.openstack.org/540888 | 13:35 |
| stephenfin | gibi: There were actually a few more. Think I've addressed everything that uses a versioned URL now | 13:35 |
| gibi | stephenfin: thanks, looking | 13:36 |
| sahid | stephenfin: can you look at this serie https://review.openstack.org/#/c/561912/9, jaypipes already +2ed most of the patches | 13:37 |
| stephenfin | sahid: Can do | 13:37 |
| gibi | stephenfin: is the new link at https://review.openstack.org/#/c/540888/3..4/doc/source/admin/manage-the-cloud.rst@25 more general than the original was? | 13:37 |
| stephenfin | gibi: Possibly. I couldn't find an equivalent for the former | 13:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: trivial: Fix file permissions https://review.openstack.org/566064 | 13:38 |
| gibi | stephenfin: ahh so the original was a dead link already | 13:39 |
| gibi | stephenfin: then it is OK for me | 13:39 |
| stephenfin | Yup. A couple of them were | 13:39 |
| gibi | stephenfin: then this fix is better than I thought :) | 13:39 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove [scheduler]/host_manager config option https://review.openstack.org/565806 | 13:39 |
| *** Eran_Kuris has joined #openstack-nova | 13:39 | |
| *** vladikr_ is now known as vladikr | 13:39 | |
| *** awaugama has joined #openstack-nova | 13:48 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova-manage placement heal_allocations CLI https://review.openstack.org/565886 | 13:52 |
| gibi | stephenfin: why do we need this ref? https://review.openstack.org/#/c/540888/3..4/nova/api/openstack/compute/rest_api_version_history.rst@761 | 13:52 |
| *** fragatina has quit IRC | 13:52 | |
| *** fragatina has joined #openstack-nova | 13:53 | |
| stephenfin | gibi: for this https://review.openstack.org/#/c/540888/3..4/doc/source/admin/manage-volumes.rst@37 | 13:53 |
| mriedem | johnthetubaguy: want to re-approve this? https://review.openstack.org/#/c/524027/ had to rebase | 13:54 |
| gibi | stephenfin: cool, thanks | 13:54 |
| johnthetubaguy | mriedem: sorry missed that one, done | 13:54 |
| melwitt | nova meeting in 5 minutes | 13:55 |
| mriedem | johnthetubaguy: thanks | 13:56 |
| mriedem | johnthetubaguy: it's also your mandatory weekly bugging for https://review.openstack.org/#/c/552078 | 13:57 |
| *** takashin has joined #openstack-nova | 13:59 | |
| johnthetubaguy | mriedem: so I managed to loose track of what we are doing with specs re freeze, although this one should get approved eiher way | 13:59 |
| mriedem | johnthetubaguy: spec freeze is r-2 | 13:59 |
| mriedem | june 7 i think | 13:59 |
| *** rajinir has joined #openstack-nova | 14:00 | |
| johnthetubaguy | mriedem: ah, I had in my head r-1 for some reason | 14:00 |
| mriedem | it usually is the 1st milestone, | 14:01 |
| *** kotra03_krk has joined #openstack-nova | 14:01 | |
| mriedem | but because of runways we decided to move out the spec freeze | 14:01 |
| johnthetubaguy | yeah, makes good sense | 14:01 |
| johnthetubaguy | I am just out of the loop (again), too many stupid hospital trips, but thankfully that is all done now! | 14:01 |
| Shilpa | stephenfin: hi, did you gone through http://paste.openstack.org/show/720231/, is it okay to discuss now? | 14:01 |
| *** kotra03_2 has quit IRC | 14:02 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Convert websocketproxy to use db for token validation https://review.openstack.org/333990 | 14:02 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Update layout docs for running console proxies https://review.openstack.org/557489 | 14:02 |
| mriedem | johnthetubaguy: hopefully everything is ok, didn't know that was happening | 14:02 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate the nova-consoleauth service https://review.openstack.org/565367 | 14:02 |
| mriedem | stephenfin: rebased and fixed up https://review.openstack.org/#/c/565367/ - should be happy now | 14:03 |
| * kashyap wonders if mriedem has a special dashboard that you check to keep on top of "right stuff". | 14:03 | |
| johnthetubaguy | mriedem: no worries, not really mentioned it, wasn't too bit a deal, just took time I didn't have! | 14:03 |
| *** evin has quit IRC | 14:03 | |
| kashyap | Because when I just wanted to ping on something, Matt was already on it & addressed what needs addressing. | 14:04 |
| kashyap | (Not the first time it happened, though...) | 14:04 |
| mriedem | johnthetubaguy: babies are just so gd needy! | 14:04 |
| mriedem | kashyap: the right stuff? https://www.youtube.com/watch?v=tbIEwIwYz-c | 14:05 |
| * kashyap gingerly clicks on it | 14:05 | |
| efried | without clicking, /me assumes nkotb video. | 14:05 |
| kashyap | Haha | 14:05 |
| johnthetubaguy | mriedem: its true | 14:05 |
| *** Guest12236 has quit IRC | 14:06 | |
| *** esberglu has joined #openstack-nova | 14:06 | |
| kashyap | mriedem: Didn't know of this vintage song; thanks for the education. | 14:06 |
| *** hemna_ has joined #openstack-nova | 14:06 | |
| kashyap | efried: You know your guess was right. | 14:08 |
| * kashyap wonders will mriedem be doing that dance sequence at Vancouver (or Berlin - likely I'll be there) | 14:09 | |
| kashyap | As it's closer to me | 14:09 |
| *** andreas_s has quit IRC | 14:10 | |
| *** mlavalle has joined #openstack-nova | 14:11 | |
| mriedem | tssurya: question in https://bugs.launchpad.net/nova/+bug/1767303 for you | 14:11 |
| openstack | Launchpad bug 1767303 in OpenStack Compute (nova) "Scheduler connects to all cells DBs to gather compute nodes info" [Undecided,New] - Assigned to Surya Seetharaman (tssurya) | 14:11 |
| mriedem | tssurya: maybe you just want to propose a backport of https://review.openstack.org/#/c/539617/ for queens? | 14:12 |
| *** READ10 has quit IRC | 14:13 | |
| bauzas | stephenfin: do we really need RsT for a docstring ? https://review.openstack.org/#/c/564214/2/nova/conf/devices.py | 14:14 |
| *** andreas_s has joined #openstack-nova | 14:14 | |
| stephenfin | bauzas: It's not a docstring. That's help text that's parsed by oslo_config.sphinext to generate this | 14:15 |
| stephenfin | https://docs.openstack.org/nova/queens/configuration/config.html | 14:15 |
| bauzas | shit | 14:15 |
| bauzas | because yes, it's a docstring | 14:15 |
| bauzas | it's just we use the docstring elsewhere | 14:15 |
| stephenfin | I think our definition of docstring differs :) | 14:15 |
| bauzas | IMHO, help text is for nova.conf | 14:16 |
| bauzas | but meh | 14:16 |
| stephenfin | Um, that is nova.conf? | 14:16 |
| bauzas | a *python* docstring | 14:16 |
| bauzas | 3 brackets | 14:16 |
| bauzas | that's what I call "docstring" | 14:16 |
| edleafe | bauzas: that's just a string | 14:16 |
| stephenfin | That's a mult-line string | 14:16 |
| stephenfin | Yeah | 14:16 |
| edleafe | A docstring is the comment that follows the method definition | 14:17 |
| stephenfin | https://www.python.org/dev/peps/pep-0257/#id15 | 14:17 |
| bauzas | well, if you should name something because of its usage, I disagree | 14:18 |
| bauzas | but meh | 14:18 |
| bauzas | :p | 14:18 |
| edleafe | bauzas: for example: | 14:19 |
| edleafe | def some_method(): | 14:19 |
| edleafe | "this is a docstring" | 14:19 |
| edleafe | return True | 14:19 |
| edleafe | Doesn't need triple quotes | 14:19 |
| edleafe | unless it's multi-line | 14:19 |
| bauzas | I should write my own language | 14:19 |
| *** yamamoto has joined #openstack-nova | 14:20 | |
| stephenfin | Yeah, theoretically you could even avoid using triple quotes then by escaping the new lines but no one would be so evil | 14:20 |
| stephenfin | bauzas: http://logs.openstack.org/14/564214/2/check/build-openstack-sphinx-docs/a23348b/html/configuration/config.html#devices.enabled_vgpu_types | 14:20 |
| bauzas | grmblblb :p | 14:21 |
| stephenfin | Eventually I'll have you all writing perfect rST | 14:21 |
| *** evin has joined #openstack-nova | 14:21 | |
| stephenfin | At which point I can happily retire to Spain or something | 14:21 |
| mriedem | *perfect* rST? | 14:21 |
| stephenfin | (TM) | 14:22 |
| *** Eran_Kuris has quit IRC | 14:22 | |
| bauzas | that's glorious | 14:22 |
| *** esberglu has quit IRC | 14:23 | |
| *** hongbin has joined #openstack-nova | 14:24 | |
| mriedem | stephenfin: speaking of https://review.openstack.org/#/c/565526/2..3/nova/conf/compute.py@a894 | 14:25 |
| mriedem | you should appreciate that | 14:25 |
| stephenfin | I do :) oslo_config.sphinxext should soon start reporting those warnings too (though it won't raise an error, to avoid breaking all the stuff) | 14:26 |
| *** mikal has quit IRC | 14:27 | |
| *** esberglu has joined #openstack-nova | 14:34 | |
| *** esberglu_ has joined #openstack-nova | 14:36 | |
| *** r-daneel has quit IRC | 14:36 | |
| *** Eran_Kuris has joined #openstack-nova | 14:38 | |
| *** germs has joined #openstack-nova | 14:38 | |
| *** germs has quit IRC | 14:38 | |
| *** germs has joined #openstack-nova | 14:38 | |
| *** kaisers has quit IRC | 14:38 | |
| *** germs has quit IRC | 14:39 | |
| *** kaisers has joined #openstack-nova | 14:39 | |
| *** germs has joined #openstack-nova | 14:39 | |
| *** esberglu has quit IRC | 14:40 | |
| *** esberglu_ is now known as esberglu | 14:40 | |
| *** moshele has joined #openstack-nova | 14:40 | |
| *** mdbooth has quit IRC | 14:41 | |
| *** mvk has quit IRC | 14:42 | |
| *** moshele has quit IRC | 14:42 | |
| *** mdbooth has joined #openstack-nova | 14:43 | |
| *** takashin has left #openstack-nova | 14:45 | |
| *** tetsuro has quit IRC | 14:45 | |
| *** arvindn05_away is now known as arvindn05 | 14:45 | |
| *** yamamoto has quit IRC | 14:46 | |
| *** _d34dh0r53_ is now known as d34dh0r53 | 14:46 | |
| *** alexchadin has quit IRC | 14:49 | |
| *** kotra03_krk has quit IRC | 14:50 | |
| *** mikal has joined #openstack-nova | 14:50 | |
| mriedem | efried: https://bugs.launchpad.net/nova/+bug/1767139 | 14:52 |
| openstack | Launchpad bug 1767139 in OpenStack Compute (nova) "TypeError in _get_inventory_and_update_provider_generation" [Undecided,New] | 14:52 |
| mriedem | remember https://review.openstack.org/#/c/562739/ ? | 14:52 |
| mriedem | apparently people are hitting a variant of that in pike starting in 16.1.1 which is when the pike version of that backport was released | 14:52 |
| mriedem | looking at the code, i can't figure out what might be causing that | 14:52 |
| mriedem | they are saying in the bug report that it happens if they start the compute before the control plane is up, but i'm not sure what difference that would make - but placement would have to be running | 14:53 |
| efried | mriedem: Sorry, is it the backport of the original or the backport of the revert that's breaking? | 14:53 |
| mriedem | backport of the original | 14:54 |
| *** afazekas_ has quit IRC | 14:54 | |
| mriedem | there is no backport of the revert | 14:54 |
| mriedem | so this https://review.openstack.org/#/c/524618/ | 14:54 |
| efried | ^ needs to be reverted is what you're saying? | 14:55 |
| mriedem | in the bug they said this happens if compute starts before controller, so i have to assume that includes placement isn't up yet, so we're hitting a flow where the RT is posting inventory for the first time and the compute node RP can't be created | 14:55 |
| mriedem | i would like to avoid a revert if possible | 14:56 |
| mriedem | i'm trying to inspect the change and flow to see if there is something we're missing | 14:56 |
| efried | mriedem: Is it weird that https://review.openstack.org/#/c/562739/ is a revert in ocata, but pike was skipped? | 14:57 |
| mriedem | we only saw the ci failures in ocata | 14:57 |
| mriedem | i haven't seen those in pike | 14:58 |
| efried | okay, I just thought that wasn't allowed. | 14:58 |
| *** Nil_ has joined #openstack-nova | 14:58 | |
| *** afazekas has joined #openstack-nova | 14:58 | |
| mriedem | i bet i know what it is | 14:58 |
| mriedem | bazinga! | 14:58 |
| efried | I guess it wasn't a cherry-pick of a revert; it was a revert of a cherry-pick. | 14:58 |
| mriedem | set_inventory_for_provider -> _ensure_resource_provider -> _create_resource_provider -> safe_connect returns None because it can't talk to placement yet | 14:59 |
| mriedem | https://review.openstack.org/#/c/524618/2/nova/scheduler/client/report.py@516 | 14:59 |
| mriedem | so we put None in the cache | 14:59 |
| *** r-daneel has joined #openstack-nova | 14:59 | |
| efried | That sounds totally right. | 14:59 |
| mriedem | kafuckingboom | 14:59 |
| melwitt | mriedem: do you have any idea if this bug is still a thing? it's ancient https://bugs.launchpad.net/nova/+bug/1366168 | 14:59 |
| * efried looks up "safe" | 14:59 | |
| openstack | Launchpad bug 1366168 in OpenStack Compute (nova) "delete private flavor will cause access problem with existed instance" [Medium,Confirmed] | 15:00 |
| *** abhishekk has quit IRC | 15:00 | |
| *** AlexeyAbashkin has joined #openstack-nova | 15:00 | |
| *** yamahata has joined #openstack-nova | 15:00 | |
| mriedem | melwitt: would have to look later, we need a fix for this placement thing in stable | 15:00 |
| melwitt | k, mah bad | 15:00 |
| mriedem | mp | 15:00 |
| mriedem | *np | 15:00 |
| mriedem | I'll wip up a thing | 15:02 |
| mriedem | why we hit this in ocata ci and not pike, i'm not sure | 15:02 |
| mriedem | maybe devstack changed the order of install stuff in pike for the fleetify | 15:02 |
| *** AlexeyAbashkin has quit IRC | 15:03 | |
| *** links has quit IRC | 15:06 | |
| *** kholkina has quit IRC | 15:09 | |
| openstackgerrit | Merged openstack/nova stable/pike: libvirt: Make `cpu_model_extra_flags` case-insensitive for real https://review.openstack.org/565670 | 15:11 |
| *** ejat has quit IRC | 15:12 | |
| *** dpawlik has joined #openstack-nova | 15:13 | |
| *** ejat has joined #openstack-nova | 15:15 | |
| melwitt | frickler: hi, could you please lift the -2 on this devstack change https://review.openstack.org/484973? the change that depends on it has been approved https://review.openstack.org/333990 | 15:16 |
| kashyap | mriedem: Thanks for the review here, I wrote that warning in a hurry (hence Workflow -1): https://review.openstack.org/#/c/565242/1/nova/virt/libvirt/driver.py | 15:16 |
| *** slaweq has quit IRC | 15:17 | |
| melwitt | frickler: and then the follow up patch (that depends-on the nova change) can be approved to re-enable the novnc tests https://review.openstack.org/527812 | 15:17 |
| kashyap | I want get the wording right. I will update that, and address the unit tests. | 15:17 |
| *** dpawlik has quit IRC | 15:17 | |
| *** ejat has quit IRC | 15:19 | |
| *** ejat has joined #openstack-nova | 15:19 | |
| *** janki has joined #openstack-nova | 15:24 | |
| *** ccamacho has quit IRC | 15:24 | |
| *** ccamacho1 has joined #openstack-nova | 15:24 | |
| efried | gibi: o notification deity supreme... | 15:26 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle @safe_connect returns None side effect in _ensure_resource_provider https://review.openstack.org/566096 | 15:26 |
| efried | gibi: Do you know of any examples you can point to where an op/admin wrote a script/daemon that listens for notifications and triggers some action? | 15:26 |
| gibi | efried: there should be a couple of services listening to notifications, let me dig up links | 15:27 |
| *** andreas_s has quit IRC | 15:27 | |
| *** andreas_s has joined #openstack-nova | 15:28 | |
| gibi | efried: searchlight: https://github.com/openstack/searchlight/blob/master/searchlight/elasticsearch/plugins/nova/notification_handler.py | 15:29 |
| mriedem | designate sink | 15:29 |
| mriedem | ceilometer | 15:29 |
| mriedem | mistral | 15:29 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: Configure multiple vGPU types https://review.openstack.org/564214 | 15:29 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: mdevs returning parent and vendor PCI info https://review.openstack.org/562304 | 15:29 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: Get inventories per vGPU type https://review.openstack.org/564215 | 15:29 |
| efried | gibi: Thanks. | 15:30 |
| gibi | efried: https://github.com/openstack/watcher/blob/master/watcher/decision_engine/model/notification/nova.py | 15:31 |
| gibi | efried: I mriedem listed the others in the meanwhile | 15:32 |
| efried | gibi, mriedem: Thanks y'all. | 15:32 |
| *** ccamacho1 has quit IRC | 15:35 | |
| *** exarlos has joined #openstack-nova | 15:36 | |
| *** andreas_s has quit IRC | 15:37 | |
| mriedem | TheJulia: left some comments in https://bugs.launchpad.net/nova/+bug/1766661 | 15:38 |
| openstack | Launchpad bug 1766661 in OpenStack Compute (nova) "'host' property is missing for Cinder volume connector when boot from volume" [Medium,Confirmed] | 15:38 |
| mriedem | tl;dr nova knows how to get the host connector from the db if needed | 15:38 |
| openstackgerrit | Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247 | 15:39 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306 | 15:39 |
| *** slaweq has joined #openstack-nova | 15:40 | |
| *** felipemonteiro__ has joined #openstack-nova | 15:41 | |
| *** andreas_s has joined #openstack-nova | 15:42 | |
| exarlos | Hi all. I had a user attempt a resize of a VM (horizon) and it has failed and reduced the VM to an error state | 15:42 |
| exarlos | ocata / ubuntu / lvm storage | 15:43 |
| exarlos | It mentions not being able to stat the /var/lib/nova/instances folder | 15:43 |
| mriedem | https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7412 | 15:44 |
| *** yamamoto has joined #openstack-nova | 15:44 | |
| mriedem | is it volume-backed or not? | 15:44 |
| exarlos | mriedem: local lvm disks as the backend | 15:45 |
| mriedem | then you'll eventually be failing because ^ | 15:45 |
| *** felipemonteiro_ has quit IRC | 15:45 | |
| mriedem | i think cfriesen might have a patch floating around for that | 15:45 |
| mriedem | https://review.openstack.org/#/c/337334/ | 15:46 |
| exarlos | mriedem: It has worked many times in the past. :-( | 15:46 |
| melwitt | so, you're saying regression from newton -> ocata? | 15:46 |
| mriedem | have you patched nova to make that happen? | 15:46 |
| *** gyee has joined #openstack-nova | 15:47 | |
| exarlos | I hadn't patched. There is a allow_resize_to_same_host option in nova.conf that permitted this to happen. | 15:48 |
| mriedem | given https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7412 i don't see how that's possible | 15:50 |
| exarlos | Let me check local code. | 15:50 |
| *** pcaruana has quit IRC | 15:50 | |
| exarlos | Honestly though :-) | 15:50 |
| *** liuzz has quit IRC | 15:52 | |
| *** liuzz has joined #openstack-nova | 15:53 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove support for monitoring Intel CMT `perf` events https://review.openstack.org/565242 | 15:53 |
| bauzas | jaypipes: efried: naichuans (I know it's late for you): I have a thought about providing traits for vGPU types | 15:53 |
| efried | bauzas: Listening. Though I confess I'm behind on those reviews. | 15:53 |
| bauzas | jaypipes: efried: naichuans: the problem is that vGPU types differ in between virt drivers, and even can be depending on the GPU host driver (eg. nvidia vs. intel) | 15:53 |
| bauzas | efried: no worries, it's unrelate | 15:54 |
| *** damien_r has quit IRC | 15:54 | |
| efried | bauzas: The thing doing the dev discovery == the thing setting up the resource providers == the thing doing the provisioning == the virt driver. | 15:55 |
| bauzas | jaypipes: efried: naichuans: so, say, I'd like to provide a flavor asking for *this* type, should we provide specific traits like https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#supported-gpus-grid-vgpu ? | 15:55 |
| efried | That's how it boils down for me. | 15:55 |
| bauzas | efried: ok, so you're not concerned by the virt driver setting for example a trait like "nvidia-11" while Xen would set a different one ? | 15:55 |
| *** yamamoto has quit IRC | 15:55 | |
| *** sahid has quit IRC | 15:55 | |
| efried | not at all. | 15:55 |
| bauzas | if so, that's super cool, it prevents me to write a mapping table | 15:56 |
| efried | well | 15:56 |
| exarlos | 15.0.7 is the version of nova components i'm running. | 15:56 |
| efried | bauzas: If we can standardize some traits that all virt drivers could reasonably use when talking about the same device, that's the best thing. | 15:56 |
| *** hamzy has quit IRC | 15:56 | |
| *** andreas_s has quit IRC | 15:56 | |
| bauzas | efried: see for example https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#vgpu-types-tesla-m60 | 15:56 |
| exarlos | Looks like the code is different, so probably I am a bit behind. | 15:56 |
| efried | bauzas: IOW, the value the virt driver discovers may be different from one virt driver to the next, but as long as it can map it to the standard trait, we're good. | 15:56 |
| bauzas | efried: the GPU type named "M60-0B" is named "nvidia-11" by the nvidia gpu driver | 15:57 |
| bauzas | efried: so the mediated devices etc. use that "nvidia-11" name | 15:57 |
| bauzas | efried: but if you use xen, it's totally named differently | 15:57 |
| efried | bauzas: But is it still a 1-1 mapping? | 15:57 |
| bauzas | efried: https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#how-to-discover-a-gpu-type | 15:58 |
| efried | bauzas: eek. | 15:58 |
| bauzas | efried: in libvirt, yes, a 1:1 | 15:58 |
| efried | Let's talk through the use case real quick. | 15:59 |
| exarlos | mriedem: So, whilst I look at a few things, how best do I pull this back from an error state, and set it so it's bootable again? :-/ | 15:59 |
| bauzas | efried: blame the gpu driver. :p | 15:59 |
| efried | bauzas: From the user pov, is it likely that he's going to put together a flavor that asks for a vgpu and then not care whether he gets a libvirt or xen instance to satisfy it? | 15:59 |
| bauzas | efried: that's a flavor, right? so I don't expect at all end users requesting that | 16:00 |
| efried | sorry, I never remember the right term. | 16:00 |
| bauzas | efried: I rather see operators setting conf file accordingly with flavors | 16:00 |
| bauzas | they could name the flavor GRID M60-0Q I'm fine | 16:00 |
| efried | bauzas: So what I'm trying to figure out is whether there's such a thing as a flavor that would include a VGPU, that's possible to use to deploy instances on different hypervisors? | 16:01 |
| bauzas | but the extra spec would be "requested=nvidia-11" | 16:01 |
| bauzas | efried: mixing environments is highly discouraged | 16:01 |
| efried | Well then I think it doesn't matter whether the traits are named the same from one hypervisor to the next. | 16:01 |
| bauzas | you can do that but then split your cloud | 16:01 |
| *** andreas_s has joined #openstack-nova | 16:01 | |
| bauzas | efried: that said, there is a flaw | 16:01 |
| exarlos | mriedem: and also, should I expect each "server" to have a respective folder in /var/lib/nova/instances/<uuid> ? | 16:01 |
| bauzas | efried: the mdev GPU type name depends on the nvidia driver | 16:02 |
| bauzas | efried: since it depends on a vendor driver, we have no guarantee that 'nvidia-11' will still correspond to a specific GPU type in the future | 16:02 |
| efried | bauzas: Right, so that's what I meant by 1-1 | 16:03 |
| bauzas | efried: if nvidia bumps a new driver that passes types that are not backwards compatible, then operators need to set again their flavors | 16:03 |
| bauzas | that's not a public API, and I hate that | 16:03 |
| bauzas | the kernel just leaks out what the VFIO device tells | 16:04 |
| efried | bauzas: So wait, it's possible that 'nvidia-11' today corresponds to M60-0B and tomorrow will correspond to M99-0X ?? | 16:04 |
| bauzas | efried: possibly yeah | 16:04 |
| bauzas | crazy, isn't it ? | 16:04 |
| efried | that's bullshit, yo. | 16:04 |
| bauzas | that's a vendor API | 16:04 |
| *** liuzz has quit IRC | 16:04 | |
| bauzas | vendors can do crazy things | 16:04 |
| bauzas | including breaking changes | 16:04 |
| mriedem | exarlos: without knowing where this actually failed and how it failed, it's hard to say | 16:05 |
| mriedem | if the guest is still running on the source host, which you could verify via virsh (assuming libvirt), then you could probably just reset the state of the instance | 16:05 |
| efried | bauzas: Well, we can only do so much. I think this mapping is something the virt driver will need to be able to keep track of. And if the stoopid vendor driver makes a breaking change, the virt driver will just have to scramble to accomodate. | 16:05 |
| bauzas | efried: I think I reached the same conclusion | 16:05 |
| mriedem | https://developer.openstack.org/api-ref/compute/#reset-server-state-os-resetstate-action | 16:05 |
| bauzas | efried: for some reason, xen is better designed than libvirt | 16:06 |
| efried | bauzas: But I think as much as we possibly can, we should create standard traits and ask the virt drivers to map to them. | 16:06 |
| bauzas | efried: it already makes the translation I guess | 16:06 |
| efried | bauzas: Which one is doing translation, though? | 16:06 |
| bauzas | efried: because if you look the Xen GPU types, those look very identical to the nvidia doc | 16:06 |
| bauzas | efried: the xen API | 16:06 |
| efried | bauzas: Right, so it's libvirt that's translating. | 16:06 |
| bauzas | efried: no | 16:06 |
| efried | xen is just using 'em as is | 16:06 |
| efried | no? | 16:07 |
| bauzas | efried: libvirt is straight setting what the kernel module tells | 16:07 |
| bauzas | https://www.kernel.org/doc/Documentation/vfio-mediated-device.txt | 16:07 |
| efried | oh, so xen is mapping those back to the model names in the literature | 16:07 |
| exarlos | mriedem: Let me go have a read of that. | 16:08 |
| exarlos | It is on the same host still. | 16:08 |
| bauzas | efried: that's what I guess, yes | 16:08 |
| efried | bauzas: so the point is that we need to choose one or the other, and then the docs for a given hypervisor have to tell me how to discover and determine the appropriate corresponding trait name. | 16:08 |
| bauzas | efried: ok, I take the challenge | 16:08 |
| bauzas | efried: I'm in favor of an unified trait name | 16:09 |
| efried | bauzas: Which needs to happen regardless, because at the very least 'nvidia-11' will need to be translated to 'trait:HW_VGPU_TYPE_NVIDIA_11=required' | 16:09 |
| bauzas | yup | 16:09 |
| efried | bauzas: What would be neat is if each hypervisor provided a script to discover the devs and list them along with their trait names. | 16:10 |
| efried | that might be asking too much, though :) | 16:10 |
| bauzas | efried: what I wonder if whether the mapping should be hardcoded in the virt driver code, or if we should leave operators setting that | 16:10 |
| bauzas | efried: heh, I have the same wish | 16:10 |
| bauzas | efried: xen does that | 16:10 |
| *** andreas_s has quit IRC | 16:10 | |
| bauzas | efried: but see my docs on libvirt usage | 16:10 |
| bauzas | that's crazy | 16:11 |
| bauzas | you have to lookup sysfs | 16:11 |
| efried | bauzas: The operator mapping thing - that's going back to the yaml config we started brainstorming in Denver. | 16:11 |
| bauzas | we don't have any libvirt API that shows you that | 16:11 |
| bauzas | efried: I know | 16:11 |
| efried | bauzas: BUT that mapping is going to be more for device aliasing, white/blacklisting, etc. I don't think we should be using it to define traits, unless maybe they're CUSTOM_* | 16:12 |
| bauzas | efried: just a thought | 16:12 |
| bauzas | efried: about custom traits | 16:12 |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290 | 16:12 |
| bauzas | efried: can you set those with nested RPs ? | 16:12 |
| *** yamahata has quit IRC | 16:12 | |
| bauzas | efried: because then, I wouldn't care a bit about that | 16:12 |
| bauzas | efried: I could just ask operators to provide custom traits | 16:13 |
| efried | bauzas: You can set traits on any RP. We originally talked about traits "inheriting" or "propagating" in various directions, but we're not doing that. | 16:13 |
| efried | bauzas: "on any RP" including child, sharing, etc. | 16:13 |
| bauzas | efried: I'm a bit rusty on traits | 16:13 |
| bauzas | efried: but I guess operators have to play with placement client to set the trait ? | 16:13 |
| bauzas | if so, that's perfect | 16:13 |
| * jaypipes reads back... | 16:13 | |
| bauzas | jaypipes: for your sanity, don't read the vendor crazypants I wrote | 16:14 |
| efried | bauzas: Well, what we talked about in Dublin is that each of various entities (nova-compute, neutron, operator) will have a set of traits it owns/controls for a given provider. It's allowed to set/unset those traits, but the other entities are not. | 16:15 |
| efried | bauzas: So in this example, I would expect nova-compute (specifically the virt driver) to "own" the trait indicating the GPU type. The operator doesn't get to set that. That wouldn't make sense. | 16:15 |
| bauzas | efried: keep in mind we will have one inventory per vGPU type | 16:16 |
| efried | bauzas: Same entity owns the inventory. | 16:16 |
| bauzas | efried: what if I want to set custom traits ? | 16:16 |
| *** felipemonteiro__ has quit IRC | 16:17 | |
| *** felipemonteiro_ has joined #openstack-nova | 16:17 | |
| bauzas | efried: say the virt driver now provides a tree of a root RP and two children RP, each one being a vGPU type | 16:17 |
| bauzas | efried: as an operator, I'd like to pick only one type based on a custom trait | 16:17 |
| efried | bauzas: From this perspective, we don't really make a distinction between standard and custom traits. But I would say the op is allowed to invent and assign a random custom trait as she sees fit. | 16:17 |
| bauzas | efried: exactly what we wrote in the spec, actually | 16:17 |
| bauzas | https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html | 16:18 |
| efried | bauzas: Legit. Basically the op just has to be careful not to stomp on a trait owned by compute. | 16:18 |
| efried | bauzas: beyond that, sky's the limit. | 16:18 |
| efried | bauzas: which is exactly why we made this a requirement in Dublin. | 16:18 |
| efried | (dansmith and I almost came to blows over it) | 16:18 |
| bauzas | yeah I remember | 16:18 |
| *** zhaochao has quit IRC | 16:18 | |
| efried | I still believe it's going to bite us, but time will tell. | 16:19 |
| bauzas | efried: ok, so what's the way to set custom traits as of now ? | 16:19 |
| bauzas | thru placement API ? (and the client) | 16:19 |
| efried | bauzas: Sure, that works. | 16:19 |
| *** moshele has joined #openstack-nova | 16:20 | |
| bauzas | efried: ok, what I need is testing then | 16:20 |
| bauzas | efried: my patch series is close to provide inventories using update_provider_tree() | 16:20 |
| *** andreas_s has joined #openstack-nova | 16:20 | |
| efried | bauzas: There's no change to the placement API for any of that; the only thing we did was update the u-p-t spec/docs to make it clear that a dance is required to make sure you're not stepping on each other. | 16:20 |
| efried | bauzas: Oh, are you aware of https://review.openstack.org/#/c/560444/ ? | 16:21 |
| bauzas | efried: ok, will look | 16:21 |
| *** andrewbogott has quit IRC | 16:21 | |
| *** andrewbogott has joined #openstack-nova | 16:21 | |
| bauzas | efried: no, I wasn't | 16:21 |
| *** lpetrut_ has joined #openstack-nova | 16:21 | |
| bauzas | efried: my top patch in the series returns a dict of dicts | 16:21 |
| bauzas | efried: I'll use that dict of dicts to populate the tree | 16:22 |
| bauzas | efried: but I was considering to implement by a change the new public interface for libvirt | 16:22 |
| openstackgerrit | Merged openstack/nova master: Get anchors for sharing providers https://review.openstack.org/565279 | 16:22 |
| openstackgerrit | Merged openstack/nova master: Make association_refresh configurable https://review.openstack.org/565526 | 16:22 |
| efried | bauzas: Have you been following the xen driver work for vgpus? | 16:23 |
| bauzas | efried: barely but yes | 16:23 |
| efried | bauzas: I think they're the furthest along with using upt in their driver. | 16:23 |
| bauzas | efried: well, I'm pretty close too | 16:23 |
| efried | bauzas: Of course, they're based on code that hasn't merged and/or won't work until the nrp-in-alloc-cands series lands. | 16:23 |
| bauzas | efried: last change in my series is https://review.openstack.org/#/c/564215/ | 16:23 |
| bauzas | efried: I know, and I was about to propose your series for a runway slot | 16:24 |
| bauzas | I'm a bit out of review as of those days due to my patch series, but I can switch to reviewing if I'm blocked due to that series :p | 16:24 |
| *** lpetrut has quit IRC | 16:25 | |
| bauzas | anyway, I need to drop for today | 16:25 |
| efried | bauzas: Granular is ready as well. (Except both series are going to bounce on merge conflict soon) | 16:25 |
| bauzas | granular is already a runway candidate | 16:25 |
| jaypipes | bauzas: I'm a strong -2 for having standard traits (or really, any traits) representing GPU vendor types | 16:25 |
| efried | I guess I could put both of those series into the runway queue. | 16:25 |
| efried | jaypipes: eh?? | 16:25 |
| bauzas | jaypipes: I'm cool with that | 16:25 |
| exarlos | mriedem: pastebin.com/SPpHu9Tu is the current error state of the VM. | 16:25 |
| bauzas | jaypipes: I guess your concern is the os-traits explosion | 16:26 |
| melwitt | looks like the gate is timing out a lot in response to me saying it had been good in the nova meeting | 16:26 |
| efried | jaypipes: Oh, you're looking for GPU *features* as opposed to types. | 16:26 |
| bauzas | melwitt: yup, I noticed that in the meeting too | 16:26 |
| melwitt | :( sorry everyone | 16:26 |
| *** lpetrut_ has quit IRC | 16:26 | |
| exarlos | mriedem: I would think it's recoverable, but i'm still learning. | 16:26 |
| bauzas | jaypipes: efried: anyway, I need to drop | 16:26 |
| efried | bauzas: à bientôt | 16:27 |
| bauzas | the first step is to make nested RPs a thing | 16:27 |
| bauzas | the second step will be to test my patches using custom traits | 16:27 |
| mriedem | melwitt: excellent catch on https://review.openstack.org/#/c/541420/ - i reproduced in one of the unit tests | 16:27 |
| bauzas | so I'm super not concerned by having standard traits as of now, in particular for vendor purposes | 16:28 |
| bauzas | jaypipes: hope that clarifies ^ | 16:28 |
| *** tesseract has quit IRC | 16:28 | |
| * bauzas calls it a day | 16:28 | |
| jaypipes | bauzas: you don't need custom traits for any of this. | 16:28 |
| *** hongbin has quit IRC | 16:28 | |
| jaypipes | bauzas: you don't need GPU types to be traits either. | 16:29 |
| bauzas | what I want is a pony | 16:29 |
| *** hongbin has joined #openstack-nova | 16:29 | |
| mriedem | exarlos: so you're hitting https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7450 | 16:29 |
| jaypipes | bauzas: we had long conversations with Jianghua on the original VGPU spec about this. | 16:29 |
| *** slaweq has quit IRC | 16:29 | |
| bauzas | jaypipes: what do you suggest ? | 16:29 |
| mriedem | exarlos: which comes after https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7413 | 16:29 |
| melwitt | mriedem: woot (though concerned the unit test was covering it up?) | 16:29 |
| mriedem | exarlos: are you sure this isn't a volume-backed instance? and by volume-backed, i mean cinder volume, not local lvm disk | 16:29 |
| *** slaweq has joined #openstack-nova | 16:29 | |
| *** slaweq has quit IRC | 16:30 | |
| jaypipes | bauzas: use the traits for GPUs that are already in os-traits for describing the *capabilities* of the GPU. | 16:30 |
| mriedem | melwitt: well, the unit test was missing a check for the bdm turducken | 16:30 |
| melwitt | a-ha | 16:30 |
| jaypipes | bauzas: "nvidia-11" isn't a capability. | 16:30 |
| mriedem | melwitt: the DriverVolumeBlockDevice wraps BlockDeviceMapping and proxies the fields to save on the internal BDM object | 16:30 |
| jaypipes | bauzas: support for, say, OpenCL 2.0 is a capability. | 16:30 |
| bauzas | got it, good idea | 16:30 |
| jaypipes | bauzas: support for 4 max display heads is a capability (weirdly, but correct) | 16:30 |
| jaypipes | bauzas: support for a particular resolution size is a capability. | 16:30 |
| openstackgerrit | Merged openstack/os-vif stable/queens: Fix VF-rep lookup routine to use parent PF number https://review.openstack.org/554917 | 16:30 |
| melwitt | mriedem: super turducken | 16:31 |
| exarlos | mriedem: Ohhh. So, yes, I use cinder! Sorry. | 16:31 |
| exarlos | I thought you mean ceph or some other storage backend. | 16:31 |
| *** do3meli has left #openstack-nova | 16:32 | |
| exarlos | mriedem: Again, i'll mention that i'm running package version 15.0.7 - so could well be some way behind on the code. | 16:34 |
| *** andreas_s has quit IRC | 16:35 | |
| exarlos | (I don't see locally the lines you're suggesting on github) | 16:35 |
| mriedem | exarlos: ah ok then that is why you get past this check https://github.com/openstack/nova/blob/15.0.7/nova/virt/libvirt/driver.py#L7294 | 16:37 |
| mriedem | and you fail here https://github.com/openstack/nova/blob/15.0.7/nova/virt/libvirt/driver.py#L7331 | 16:37 |
| mriedem | exarlos: and your "instances_path" config option value must be /var/lib/nova/instances right? | 16:38 |
| mriedem | you're hitting a ProcessExecutionError on the source node during the resize, and the instance is getting put into ERROR state here https://github.com/openstack/nova/blob/stable/ocata/nova/compute/manager.py#L6792 | 16:40 |
| mriedem | the guest is otherwise fine on the source node, and you could reset it's state to ACTIVE | 16:40 |
| mriedem | you'll have to debug why you're getting "Stderr: u"mv: cannot stat \'/var/lib/nova/instance'" | 16:41 |
| mriedem | this is just trying to rename the directory mv /var/lib/nova/instances/97346b0a-71a5-4a66-a966-a41565cf3c80 /var/lib/nova/instances/97346b0a-71a5-4a66-a966-a41565cf3c80_resize | 16:41 |
| exarlos | mriedem: yeah, that folder does not exist. Hence my question as to whether all VMs should have a folder in this respective /vat/lib/nova/instances folder. | 16:44 |
| exarlos | They only contain the console.log file. | 16:44 |
| exarlos | And only half of the folders are present, if I should expect all "servers" to have a folder here :-| | 16:44 |
| exarlos | instances_path is default: $state_path/instances | 16:45 |
| mriedem | and state_path is /var/lib/nova | 16:46 |
| exarlos | mriedem: correct. | 16:47 |
| mriedem | yes all instances should have that directory, spawn() eventually calls this https://github.com/openstack/nova/blob/15.0.7/nova/virt/libvirt/driver.py#L3104 | 16:48 |
| *** andreas_s has joined #openstack-nova | 16:49 | |
| mriedem | hard rebooting the instance should recreate that path | 16:49 |
| *** yamahata has joined #openstack-nova | 16:49 | |
| *** burt has quit IRC | 16:50 | |
| *** burt has joined #openstack-nova | 16:51 | |
| *** wwriverrat has joined #openstack-nova | 16:51 | |
| *** mdbooth has quit IRC | 16:53 | |
| exarlos | mriedem: ah, okay. Concerns me a little bit that this folder isn't already there :-| | 16:54 |
| mriedem | yeah, i don't know why it wouldn't be | 16:54 |
| mriedem | although, | 16:55 |
| mriedem | https://github.com/openstack/nova/blob/15.0.7/nova/virt/libvirt/driver.py#L7229 might have done something naughty | 16:55 |
| *** andreas_s has quit IRC | 16:55 | |
| mriedem | well nvm, probably not because if utils.execute('mv', inst_base, inst_base_resize) failed then inst_base_resize shouldn't exist | 16:56 |
| exarlos | it doesn't exist. I check my other compute nodes, and they do seem to have the folders present to match their VMs, but only show a console.log | 16:57 |
| *** tesseract has joined #openstack-nova | 16:57 | |
| exarlos | okay, i'll try and reset the state and fire her up. | 16:57 |
| mriedem | they only have a console.log because the actual image data for the guest is in the root cinder volume | 16:58 |
| mriedem | you can also just reboot the instance in ERROR state | 16:58 |
| mriedem | and verify that it recreates the instance dir | 16:59 |
| *** derekh has quit IRC | 16:59 | |
| mriedem | it will likely have to be a HARD reboot though | 16:59 |
| mriedem | https://developer.openstack.org/api-ref/compute/#reboot-server-reboot-action | 16:59 |
| *** itlinux has joined #openstack-nova | 17:01 | |
| *** jpena is now known as jpena|off | 17:01 | |
| *** liuzz has joined #openstack-nova | 17:02 | |
| *** dtantsur is now known as dtantsur|afk | 17:02 | |
| exarlos | mriedem: It's starting up :-) | 17:03 |
| *** hamzy has joined #openstack-nova | 17:03 | |
| *** mgoddard has quit IRC | 17:03 | |
| exarlos | And that folder exists now in /var/lib/nova/instances/ for the server. | 17:04 |
| exarlos | I guess that using local storage for openstack is really being left behind? | 17:05 |
| exarlos | Is ceph the go-to storage option of choice? | 17:07 |
| *** lucasagomes is now known as lucas-afk | 17:08 | |
| larsks | Will nova permit someone to set up PCI passthrough for a device that device is already bound to an inactive (e.g. powered off) instance? | 17:08 |
| exarlos | (for those of us who can't afford/have space at home for a real storage solution) :-) | 17:08 |
| mriedem | exarlos: ceph is the most widely used cinder backend, i don't know how much people also use it for local ephemeral storage, but i assume if you're doing it with persistent block devices then you might as well also use it for local ephemeral | 17:10 |
| *** slaweq has joined #openstack-nova | 17:11 | |
| mriedem | larsks: we don't support hot plugging pci devices | 17:11 |
| mriedem | cyborg is the long-term solution for that i believe | 17:11 |
| mriedem | exarlos: i'm not sure why that local disk was gone, or if it has anything to do with your storage option of choice | 17:11 |
| exarlos | mriedem: Thankyou for your time today. I suspect you're rather busy. | 17:12 |
| mriedem | unless you had a hard drive failure on that host or something | 17:12 |
| larsks | mriedem: I wasn't asking about hot plugging...or was I? I didn't think so. I just meant you have instance A configure w/ pci passthrough, you shut down A and then try to create B with the same configuration...would nova allow that? | 17:12 |
| mriedem | larsks: i hope not | 17:12 |
| larsks | Fair enough. | 17:12 |
| mriedem | exarlos: np, thanks and good luck | 17:12 |
| *** tssurya has quit IRC | 17:14 | |
| *** dpawlik has joined #openstack-nova | 17:14 | |
| *** damien_r has joined #openstack-nova | 17:15 | |
| *** slaweq has quit IRC | 17:16 | |
| *** dpawlik has quit IRC | 17:18 | |
| moshele | mriedem: hi, what are the chances to approve this pike backport https://review.openstack.org/#/c/563501/ ? the packport wan't trivial at all | 17:20 |
| *** damien_r has quit IRC | 17:23 | |
| *** janki has quit IRC | 17:24 | |
| mriedem | moshele: that enables the nova side of a neutron feature added in pike correct? | 17:24 |
| mriedem | and the change on master was originally reverted https://review.openstack.org/#/c/518833/ | 17:25 |
| mriedem | master/queens | 17:25 |
| *** tesseract has quit IRC | 17:25 | |
| moshele | meridem: the first version was revert because it broken heat, but the second one was merged https://review.openstack.org/#/c/519066/ | 17:27 |
| *** damien_r has joined #openstack-nova | 17:27 | |
| *** mvk has joined #openstack-nova | 17:28 | |
| mriedem | sure, but i'm still not sure if we should backport something to nova pike to enable a feature in neutron pike | 17:28 |
| mriedem | afk for lunch | 17:28 |
| *** gjayavelu has joined #openstack-nova | 17:29 | |
| moshele | meridem: this is the feature enablement in pike https://review.openstack.org/#/c/398265/. the other patch was a bug fix (https://review.openstack.org/#/c/519066/ ) which was introduce by something I change in neutron | 17:30 |
| moshele | merdem: this is the neutron change https://review.openstack.org/#/c/499203/ | 17:31 |
| moshele | mriedem: this is the neutron change https://review.openstack.org/#/c/499203/ | 17:31 |
| moshele | mriedem: ok I will try to catch you after lunchת bon appetit! | 17:32 |
| *** damien_r has quit IRC | 17:34 | |
| *** spsurya has quit IRC | 17:42 | |
| *** mgoddard has joined #openstack-nova | 17:44 | |
| exarlos | Thanks again, ciao for now. | 17:44 |
| *** exarlos has quit IRC | 17:44 | |
| *** udesale has quit IRC | 17:50 | |
| *** mdnadeem has quit IRC | 17:51 | |
| Tahvok | Hey guys! | 17:56 |
| *** sridharg has quit IRC | 17:57 | |
| Tahvok | I'm having some boot issues with kvm (using ocata): After creating an instance, I configured some volumes on it, and after shutdown, I cannot bring it up, it's getting stuck in seabios with 'no bootable device' message. If I remove all other volumes I've created, it manages to boot, but fails later as my configuration needs the volumes available | 17:58 |
| Tahvok | So I've checked the bootindex configuration, and it looks like it's completely fine, bootindex is set only on the bootable volume. | 17:58 |
| Tahvok | I should also mention that I'm using ceph | 17:58 |
| mriedem | Tahvok: do you have this https://review.openstack.org/#/c/561613/ ? | 18:00 |
| mriedem | just released in 15.1.1 | 18:00 |
| Tahvok | Nova,version=15.1.1 | 18:00 |
| Tahvok | Let me check the file just to be sure | 18:01 |
| Tahvok | So, I've just checked this, and not I do not have it | 18:03 |
| melwitt | if you do a 'virsh dumpxml' on the instance, you might see the boot_index set incorrectly on your bootable volume (it should be 0). and if it's not, you need the bug fix from 5.1.1 | 18:06 |
| melwitt | this was the bug https://bugs.launchpad.net/nova/+bug/1729584 | 18:06 |
| openstack | Launchpad bug 1729584 in OpenStack Compute (nova) queens "boot from volume + configdrive with virtio-scsi broken (regression)" [High,Fix committed] - Assigned to Artom Lifshitz (notartom) | 18:06 |
| frickler | mriedem: melwitt: removed the -2, will do a closer review tomorrow | 18:08 |
| melwitt | cool, thanks frickler | 18:08 |
| Tahvok | melwitt: yes, it's set as 1 and not 0 | 18:09 |
| Tahvok | I'm restarting nova-compute now, to check the patch | 18:09 |
| Tahvok | And it's working! | 18:10 |
| melwitt | sweet | 18:10 |
| Tahvok | I wonder why it says that I have nova 15.1.1 when I definitely do not have this patch | 18:10 |
| melwitt | hm, yeah, it should be in there | 18:11 |
| melwitt | 5.1.1 was released yesterday, very recent | 18:11 |
| Tahvok | So I definitely do not have this | 18:12 |
| Tahvok | As we rerun ansible a week ago | 18:12 |
| melwitt | do you build packages yourself from stable/ocata? or where did you get the packages? | 18:13 |
| Tahvok | I'm using openstack-ansible project | 18:13 |
| melwitt | okay. I'm not familiar with how it works but I'm guessing maybe it builds packages off stable as the "next version" which would have been 5.1.1 | 18:14 |
| melwitt | that is, if you were to run it today it would give you 5.1.2, just a guess | 18:14 |
| Tahvok | Yes, it's uses hashes from git | 18:15 |
| Tahvok | Ok, so that might be why I see 5.1.1 | 18:15 |
| eandersson | Random question but why does the scheduler need to log every time it does an instance sync? | 18:15 |
| mriedem | eandersson: need to? | 18:15 |
| Tahvok | I will request to bump the hashes so I'll get this fix 'officially' | 18:16 |
| mriedem | because if thinks you care | 18:16 |
| eandersson | :D | 18:16 |
| mriedem | *it | 18:16 |
| eandersson | It's just with 800+ nodes it gets a bit spammy :D | 18:16 |
| melwitt | Tahvok: ah, k. yeah. so when we released 5.1.1, that means it was tagged as 15.1.1 and pushed to pypi etc as that version | 18:16 |
| mriedem | i had a patch for oslo.log at one point to be able to drop stuff like that in the scheduler to AUDIT log | 18:16 |
| mriedem | below DEBUG | 18:16 |
| mriedem | eandersson: is it debug today? | 18:16 |
| eandersson | INFO | 18:16 |
| melwitt | Tahvok: this was the release patch. I'm not sure if/how openstack-ansible consumes that https://review.openstack.org/564044 | 18:16 |
| eandersson | but we are still on mitaka (so might have changed) | 18:17 |
| mriedem | eandersson: which specific message? | 18:17 |
| *** moshele has quit IRC | 18:17 | |
| mriedem | Successfully synced instances from host ? | 18:17 |
| mriedem | yeah those should probably be debug | 18:18 |
| eandersson | Yea | 18:18 |
| mriedem | the "received <operation> from unknown host" is probably ok at INFO | 18:18 |
| mriedem | but all the other "OH HI I DID SOMETHING I"M SUPPOSED TO DO!!!" is debug | 18:18 |
| mriedem | *should be debug | 18:18 |
| eandersson | I can throw up a quick commit on gerrit | 18:18 |
| mriedem | ack | 18:18 |
| mriedem | just don't get any on me | 18:18 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Create volume attachment during boot from volume in compute https://review.openstack.org/541420 | 18:20 |
| mriedem | melwitt: this time with 10x more bdmturducken ^ call in the next 20 minutes and we'll double your bdms | 18:20 |
| melwitt | and the peasants rejoiced | 18:20 |
| Tahvok | melwitt: mriedem thanks a lot for helping with the issue! And didn't know what was going on... | 18:21 |
| Tahvok | We've had it for a few months now... | 18:21 |
| mriedem | i can't say that's it, but it sounds right | 18:21 |
| mriedem | yeah it was a regression in ocata i believe | 18:21 |
| melwitt | np, I'd been chasing that bug for a long time to get the fix out | 18:21 |
| *** tssurya has joined #openstack-nova | 18:35 | |
| *** felipemonteiro__ has joined #openstack-nova | 18:39 | |
| *** moshele has joined #openstack-nova | 18:39 | |
| *** amoralej is now known as amoralej|off | 18:42 | |
| *** felipemonteiro_ has quit IRC | 18:42 | |
| mriedem | so uh, i don't think this is ever going to be true https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3196 | 18:49 |
| mriedem | is it possible to reboot a rescued instance? | 18:50 |
| mriedem | the api doesn't allow it | 18:50 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/compute/vm_states.py#L69-L72 | 18:50 |
| mriedem | is there some other place in nova where we internally reboot an instance when it's rescued? | 18:51 |
| melwitt | I know not | 18:52 |
| *** fragatina has quit IRC | 18:53 | |
| mriedem | git blame history here i go | 18:53 |
| *** mgoddard has quit IRC | 18:53 | |
| *** fragatina has joined #openstack-nova | 18:53 | |
| *** moshele has quit IRC | 18:53 | |
| mriedem | https://github.com/openstack/nova/commit/39ffe806cdfe97721fee5e48c8591c4c3b52766b | 18:54 |
| openstackgerrit | Merged openstack/nova master: support multiple member_of qparams https://review.openstack.org/561315 | 18:54 |
| mriedem | https://github.com/openstack/nova/blob/39ffe806cdfe97721fee5e48c8591c4c3b52766b/nova/compute/api.py#L1702 | 18:54 |
| *** imacdonn has quit IRC | 18:54 | |
| mriedem | wtf, you couldn't reboot a rescue vm | 18:54 |
| *** imacdonn has joined #openstack-nova | 18:54 | |
| *** eharney has quit IRC | 18:54 | |
| melwitt | what? that patch is about the vm_state, just making sure it stays "rescued" after the reboot. right? | 18:55 |
| mriedem | https://review.openstack.org/#/q/topic:bug/1170237+(status:open+OR+status:merged) | 18:56 |
| mriedem | yeah, but if you look at the reboot api code in that same hash, we don't allow rebooting a rescued vm | 18:56 |
| melwitt | oh :\ | 18:56 |
| mriedem | i mean, we don't allow rebooting an instance with vm_state=RESCUE | 18:56 |
| mriedem | https://review.openstack.org/#/q/topic:bug/1170237+(status:open+OR+status:merged) | 18:56 |
| mriedem | looks like some rax/xen thing | 18:56 |
| mriedem | that likely never made the api change upstream | 18:56 |
| mriedem | gd rax | 18:56 |
| melwitt | guh, weird. so we have patches that say "now you can reboot a rescued instance" but you can't because the API kicks you out | 18:57 |
| mriedem | the rax api code probably lets you | 18:57 |
| melwitt | right | 18:57 |
| mriedem | where was that one guy that still works at rax? | 18:57 |
| mriedem | the guy always in korea.. | 18:58 |
| melwitt | tbh, I don't know whether we're suppose to be able to reboot rescued instances. I'm not that familiar with the "rescue" function | 18:58 |
| mriedem | neither am i, | 18:58 |
| mriedem | but i can read code :) | 18:58 |
| mriedem | that's how i found this | 18:58 |
| melwitt | well, yeah. I mean whether or not it makes sense. I would guess from the lack of user complaints that it's not usual to try to reboot one | 18:58 |
| mriedem | mikal: johnthetubaguy: when you're around, maybe you can sort this out - did rax have a proprietary change to allow rebooting rescued instances? https://review.openstack.org/#/q/topic:bug/1170237+(status:open+OR+status:merged) | 18:59 |
| mriedem | because the upstream api doesn't allow that | 18:59 |
| mriedem | see https://github.com/openstack/nova/blob/master/nova/compute/vm_states.py#L69-L72 | 18:59 |
| *** openstackgerrit has quit IRC | 19:05 | |
| *** moshele has joined #openstack-nova | 19:06 | |
| *** damien_r has joined #openstack-nova | 19:07 | |
| *** openstackgerrit has joined #openstack-nova | 19:09 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix being able to hard reboot a pausing instance https://review.openstack.org/566143 | 19:09 |
| *** damien_r has quit IRC | 19:13 | |
| *** dpawlik has joined #openstack-nova | 19:15 | |
| *** damien_r has joined #openstack-nova | 19:15 | |
| *** mgoddard has joined #openstack-nova | 19:16 | |
| *** sapd_ has joined #openstack-nova | 19:19 | |
| *** dpawlik has quit IRC | 19:19 | |
| *** felipemonteiro_ has joined #openstack-nova | 19:21 | |
| *** felipemonteiro__ has quit IRC | 19:21 | |
| *** slaweq has joined #openstack-nova | 19:21 | |
| *** slaweq has quit IRC | 19:22 | |
| *** slaweq has joined #openstack-nova | 19:22 | |
| *** sapd has quit IRC | 19:22 | |
| *** mgoddard has quit IRC | 19:23 | |
| openstackgerrit | Oliver Walsh proposed openstack/nova master: Fix handling of connect issues in _ensure_resource_provider https://review.openstack.org/566148 | 19:26 |
| mriedem | owalsh: i already have a patch up for that | 19:28 |
| mriedem | beat you by a few hours | 19:28 |
| mriedem | https://review.openstack.org/#/c/566096/ | 19:28 |
| owalsh | mriedem: snap | 19:29 |
| owalsh | mriedem: worse on pike, it cache None | 19:32 |
| mriedem | yeah it was a backport regression | 19:32 |
| melwitt | oh, so _that's_ where the None RP comes from. geesh | 19:33 |
| melwitt | I remember we reverted a thing while we were trying to get backports done for the stable releases | 19:34 |
| mriedem | yeah that was ocata | 19:34 |
| mriedem | i suspect the fleetify devstack stuff in pike+ was hiding it for us | 19:35 |
| mriedem | somehow | 19:35 |
| mriedem | although even in ocata devstack i thought we started the compute last | 19:35 |
| *** zzzeek_ has quit IRC | 19:35 | |
| *** zzzeek has joined #openstack-nova | 19:37 | |
| *** mgoddard has joined #openstack-nova | 19:38 | |
| *** fragatina has quit IRC | 19:41 | |
| *** awaugama has quit IRC | 19:43 | |
| mriedem | dansmith: what do you think a safe batch size is for this heal allocations CLI? at first i default to CONF.api.max_limit but that's 1000 which seems way too big https://review.openstack.org/#/c/565886/5/nova/cmd/manage.py@1776 - map_instances defaults to 50 | 19:44 |
| mriedem | so was thinking about using 50 | 19:44 |
| dansmith | yeah 1000 is too much | 19:44 |
| dansmith | 50 is probably good | 19:44 |
| mriedem | online_data_migrations also does 50 | 19:44 |
| dansmith | yup | 19:44 |
| melwitt | did we do an audit of other uses of safe_connect where None can be returned? | 19:44 |
| mriedem | melwitt: jaypipes has been working on untangling that | 19:45 |
| melwitt | k, cool | 19:45 |
| *** eharney has joined #openstack-nova | 19:49 | |
| *** mgoddard has quit IRC | 19:52 | |
| *** damien_r has quit IRC | 19:53 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix wrong arguments for 'detach_volume' https://review.openstack.org/566152 | 19:59 |
| *** gjayavelu has quit IRC | 20:02 | |
| *** liverpoo1er has quit IRC | 20:04 | |
| *** damien_r has joined #openstack-nova | 20:04 | |
| *** mingyu has quit IRC | 20:06 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova-manage placement heal_allocations CLI https://review.openstack.org/565886 | 20:06 |
| *** moshele has quit IRC | 20:08 | |
| *** tbachman has quit IRC | 20:09 | |
| mriedem | owalsh: if you want to do the backport to queens and pike for https://review.openstack.org/#/c/566096/ that would speed things along so i can +2 the backports | 20:11 |
| mriedem | owalsh: beware: the provider tree stuff in rocky will likely mean merge conflicts for the backports | 20:11 |
| arvindn05 | dansmith: looks like there was agreement on the rebuild instance with traits thread. Can you send out an update on ML on the final approach? | 20:16 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.openstack.org/566153 | 20:17 |
| *** mchlumsky has quit IRC | 20:17 | |
| dansmith | arvindn05: I started it a bit ago but got distracted.. if there is agreement you're not blocked right? | 20:17 |
| *** wolverineav has quit IRC | 20:19 | |
| *** wolverineav has joined #openstack-nova | 20:20 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Create volume attachment during boot from volume in compute https://review.openstack.org/541420 | 20:23 |
| arvindn05 | dansmith: i am blocked at this point because my approved patch https://review.openstack.org/#/c/560596/ is also holding based on the decision on the rebuild issue | 20:24 |
| *** mingyu has joined #openstack-nova | 20:24 | |
| *** wolverineav has quit IRC | 20:24 | |
| arvindn05 | my next patches would be update the spec with decision on rebuild and propose the code patch for the same | 20:25 |
| dansmith | arvindn05: you mean you're blocked because your patches don't do what mriedem wants yeah? | 20:25 |
| dansmith | arvindn05: if you'd do what we said in the meeting this morning, then he'd remove his -W and everything would move along, AFAICT | 20:25 |
| arvindn05 | dansmith: yup. but i was unfortunately not in the meeting and not entirely sure what approach was decided | 20:26 |
| dansmith | if only there was a log... | 20:27 |
| arvindn05 | dansmith: is it fair to summarize it as we want to go check the allocations route | 20:27 |
| arvindn05 | (8:25:17 AM) efried: arvindn05: That was the impression I got. But yeah, let's see what dansmith has to say. | 20:27 |
| dansmith | arvindn05: yes | 20:27 |
| arvindn05 | dansmith: great. Thanks for confirming....i will start with the spec and the code patch. glad the deadlock was resolved :) | 20:28 |
| efried | ++ | 20:28 |
| *** moshele has joined #openstack-nova | 20:28 | |
| *** damien_r has quit IRC | 20:29 | |
| mriedem | arvindn05: fyi http://eavesdrop.openstack.org/meetings/nova/2018/nova.2018-05-03-14.00.log.html#l-161 | 20:30 |
| *** weshay is now known as weshay|rover | 20:30 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix being able to hard reboot a pausing instance https://review.openstack.org/566143 | 20:31 |
| owalsh | mriedem: sure | 20:31 |
| mriedem | thanks, hopefully it's not too bad, the patch is pretty isolated | 20:32 |
| arvindn05 | mriedem: thanks got that from gibi as well :) last statement was <dansmith> I shall commentificate upon the threadage and reviewage so wanted to confirm :) | 20:32 |
| *** moshele has quit IRC | 20:33 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup placement policy generator docs https://review.openstack.org/565225 | 20:34 |
| arvindn05 | dansmith: one other clarification, we want to validate the allocations vs traits always correct? Even when we are rebuilding with the same image and no traits have changed | 20:41 |
| *** evin has quit IRC | 20:42 | |
| *** arvindn05 has quit IRC | 20:46 | |
| *** arvindn05 has joined #openstack-nova | 20:48 | |
| melwitt | I would think so -- while the image hasn't changed, the underlying resource provider traits could have | 20:48 |
| mriedem | arvindn05: that gets tricky depending on where you do the validation, | 20:50 |
| mriedem | if the image doesn't change, i'm not sure if you get to the point in conductor where we'd be calling the scheduler | 20:50 |
| mriedem | i realze you're not calling the scheduler to do the validation, but i assumed it would be in the same block | 20:50 |
| arvindn05 | mriedem: i am thinking it would be in the conductor | 20:51 |
| arvindn05 | conductor already has the placement client and all its dependencies need to make the validation | 20:51 |
| arvindn05 | conductor.manager.ComputeTaskManager#rebuild_instance somewhere in here is where the validation logic would lie | 20:52 |
| *** andreas_s has joined #openstack-nova | 20:53 | |
| *** hamzy has quit IRC | 20:53 | |
| mriedem | arvindn05: you'd have an else block here https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/conductor/manager.py#L944 when recreate is False | 20:55 |
| arvindn05 | in case of rebuild we would add logic here https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L897 | 20:55 |
| arvindn05 | mriedem: sorry...looking at your code pointer now | 20:56 |
| mriedem | melwitt: so i'm looking at https://review.openstack.org/#/c/540258/ and something is bugging me, | 20:56 |
| mriedem | with an affinity group policy, the group members would all have to be in the same cell because they have to be on the same host | 20:57 |
| mriedem | also soft-affinity might screw with that... | 20:57 |
| *** andreas_s has quit IRC | 20:57 | |
| mriedem | for an anti-affinity group, we could have instances in different cells i'd thinkg | 20:57 |
| mriedem | *think | 20:57 |
| melwitt | yeah, I think so | 20:58 |
| melwitt | are you suggesting that if policy == affinity then limit to same cell? | 20:58 |
| mriedem | so if i'm doing a move operation on an instance in an anti-affinity group, we have a targeted context for the cell that instance being moved is in, | 20:58 |
| arvindn05 | mriedem: from the comment in line 916 that whole block executes only in case we are doing a rebuild with a new image.... | 20:58 |
| mriedem | and we'll only get group hosts in that cell to make sure the instance doesn't go to a host that another member of the same group in the same cell is in, since we can't move across cells | 20:59 |
| melwitt | right | 20:59 |
| mriedem | which is correct | 20:59 |
| arvindn05 | if we want to run the validation always...we would have to do outside the block, correct? | 20:59 |
| mriedem | arvindn05: well, rebuild + new image or evacuate | 21:00 |
| mriedem | evacuate is a rebuild on another new host | 21:00 |
| mriedem | but same image | 21:00 |
| mriedem | in that case, the scheduler will go through GET /allocation_candidates and do a claim, | 21:00 |
| mriedem | so we don't need to do any image validation for traits in conductor for evacuate b/c that would be redundant | 21:00 |
| arvindn05 | yup...i got that for evacuate...how about the case of rebuild without the image being changed | 21:01 |
| mriedem | melwitt: ok and if we're not targeted (not a move, just instance create), then we need to get group hosts from all cells because we don't know in which cell the instance is going to land in | 21:01 |
| melwitt | mriedem: right | 21:02 |
| mriedem | and this happens before we ever run the affinity/anti-affinity filters right? | 21:02 |
| mriedem | yeah called from schedule_and_build_instances | 21:03 |
| melwitt | yes, this is the setup actually pretty early on in compute/api, I think | 21:03 |
| melwitt | okay maybe not (sorry I forgot) | 21:03 |
| mriedem | ok yeah so if i'm creating an instance an in anti-affinity group then the other members of the group could be in any cell and i need to iterate all cells to find which hosts those other members are in, | 21:04 |
| melwitt | setup_instance_group is called in some places and that's what does it (if group.hosts is not yet populated) | 21:04 |
| melwitt | it will do that the first time it's called in a code path | 21:04 |
| mriedem | and if i'm creating an instance in an affinity group, if that group already has one member in it, we should know exactly which cell the new instance and new member of that group should get scheduled to | 21:04 |
| mriedem | so in that case, we wouldn't really need to scatter/gather all cells, | 21:05 |
| mriedem | i don't think... | 21:05 |
| melwitt | hm, well, I don't know actually. affinity just means once the cell is selected, affine with an instance in the group if there is one. I think that's how it would behave | 21:05 |
| melwitt | or does it actually choose the cell based on where the members are? I would have doubted that | 21:06 |
| melwitt | you could have group members in different cells even with the affinity policy, I think | 21:06 |
| melwitt | like, they affine with each other on a per cell basis | 21:07 |
| melwitt | and if you're the first group member to land in a cell, you can build anywhere. it's just after that, new members have to affine if they land in that cell | 21:07 |
| *** fragatina has joined #openstack-nova | 21:09 | |
| openstackgerrit | Oliver Walsh proposed openstack/nova stable/queens: Handle @safe_connect returns None side effect in _ensure_resource_provider https://review.openstack.org/566161 | 21:09 |
| mriedem | i left some comments, | 21:10 |
| *** fragatina has quit IRC | 21:11 | |
| mriedem | but the strict affinity policy should filter such that members of the same group are on the same host | 21:11 |
| mriedem | which implies same cell | 21:11 |
| mriedem | if you built 2 instances in concurrent requests to the same strict affinity group, we don't guarantee that those end up on the same host via the scheduler b/c it doesn't have that information; that's why we have the late affinity check in the compute, | 21:11 |
| mriedem | HOWEVER, | 21:11 |
| *** tssurya has quit IRC | 21:11 | |
| mriedem | with multiple cells, if instance A goes on host 1 in cell1 and instance B goes on host 2 in cell2, i'm not sure the late affinity check will catch that | 21:12 |
| melwitt | yeah, that's a good question. I don't think this has been brought up before. I hadn't been thinking to target an instance to a cell where its affinity member lives | 21:12 |
| *** wolverineav has joined #openstack-nova | 21:12 | |
| mriedem | although it should.....the late check will ask the instance group for hosts that the other members are in, and if they are in another cell, it will fail | 21:12 |
| mriedem | https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/compute/manager.py#L1287 | 21:13 |
| *** dtruong_ has quit IRC | 21:13 | |
| mriedem | yeah so i think we'll fail the late affinity check, and reschedule, however, we'll never find a host to fit that policy b/c it's going to be in the wrong cell at that point | 21:13 |
| mriedem | anti-affinity is much easier, different host in same cell, or different cell, doesn't matter | 21:14 |
| melwitt | yeah ... I think my patch makes it so that check would do a "all cells" query and maybe it shouldn't ... getting confused | 21:14 |
| mriedem | well, | 21:14 |
| mriedem | i think there are two cases here: | 21:14 |
| mriedem | for affintiy | 21:14 |
| mriedem | 1. a group member is already in a host (so a cell) and your lookup finds it, we know our new instance has to go in that cell to affine to the existing member | 21:15 |
| mriedem | worst case there is we're needlessly iterating multiple cells when we already know which one we want | 21:15 |
| mriedem | so, perf optimization for later maybe, | 21:15 |
| *** edmondsw has quit IRC | 21:15 | |
| *** dpawlik has joined #openstack-nova | 21:16 | |
| mriedem | 2. we're creating multiple instances concurrently where there are no existing members in the affinity group - iterating the cells is fine but we can still have the scheduler put the members in the wrong cells | 21:16 |
| *** edmondsw has joined #openstack-nova | 21:16 | |
| mriedem | i.e. it's the same affinity race for concurrent creates that we've always had, | 21:16 |
| mriedem | but the late check in the compute can't fix that one | 21:16 |
| mriedem | because we don't reschedule across cells, only within the same cell we're already in | 21:16 |
| melwitt | I see, yeah | 21:16 |
| *** mingyu has quit IRC | 21:17 | |
| melwitt | I think we have to check all cells though right? we have to pull instances to find out what host they're on | 21:17 |
| mriedem | so in case 2 you get a novalidhost / maxretriesexceeded and have to delete and recreate the instance, which at that point should work | 21:17 |
| *** fragatina has joined #openstack-nova | 21:18 | |
| openstackgerrit | Oliver Walsh proposed openstack/nova stable/pike: Handle @safe_connect returns None side effect in _ensure_resource_provider https://review.openstack.org/566164 | 21:18 |
| melwitt | that is, I don't think there's a way to avoid having to look in all cells unless you mean do it iteratively instead of a scatter-gather and stop early when found | 21:18 |
| mriedem | melwitt: yeah i think the latter, | 21:18 |
| openstackgerrit | karim proposed openstack/nova master: Pushing image traits to ironic node https://review.openstack.org/565620 | 21:18 |
| melwitt | gotcha | 21:18 |
| mriedem | like, if policy == 'affinity' lookup the instance mappings for the group members, and they should all be in the same cell, so just get hosts from that one | 21:19 |
| mriedem | or just get one instance mapping for one member in the affinity group and pick it's host, because the new instance *has* to go on that same host | 21:19 |
| mriedem | anyway, like i said, perf optimization for later if we care | 21:19 |
| mriedem | i just wanted to make sure i was thinking through this correctly | 21:20 |
| *** dpawlik has quit IRC | 21:21 | |
| *** edmondsw has quit IRC | 21:21 | |
| melwitt | oh, I see what you mean now, get the group members and choose a cell based on that | 21:21 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: rework how we pass candidate request information https://review.openstack.org/566166 | 21:21 |
| *** arvindn05 has left #openstack-nova | 21:22 | |
| melwitt | yeah, I think the only potential wrinkle there is, that would mean affinity always means one cell forever | 21:22 |
| melwitt | and I'm not 100% sure that's how it's supposed to work considering cells are transparent, that is, if you imagine you have some application that will run faster if its instances are affined to a host, does that mean you want your application to only be able to run in one cell? or do you want it to run in multiple cells where each cell will have its instances running on the same host | 21:23 |
| dansmith | if you want the latter you need two groups regardless of cells right? | 21:24 |
| melwitt | and have multiple instances of your application running from multiple cells? maybe that's not a thing that makes sense but just for the sake of thinking about it | 21:24 |
| dansmith | if you asked for host affinity, then "same cell" is kinda implied no? | 21:24 |
| melwitt | maybe, I'm not that much of an affinity expert | 21:24 |
| dansmith | affinity means "same host" currently | 21:24 |
| melwitt | if it is, then that's cool. I just didn't know | 21:24 |
| dansmith | anti-affinity means "any other host" | 21:24 |
| dansmith | or rather "not the same host" | 21:25 |
| melwitt | right, okay. that makes it clear then. I don't know why I was thinking about multiple per cell affinity | 21:25 |
| *** esberglu has quit IRC | 21:26 | |
| mriedem | looking at https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/compute/manager.py#L1289 again, | 21:32 |
| *** arvindn05 has joined #openstack-nova | 21:32 | |
| mriedem | i think it's possible that you could have 2 affinity group members in different cells and we wouldn't fail | 21:32 |
| mriedem | because if the group has instA and instB in cell1 and cell2 respectively, group_hosts = group.get_hosts(exclude=[instance.uuid]) for instA won't return the host for instB because it's in another cell, and we're not targeted for that other cell | 21:33 |
| mriedem | so group_hosts would be empty and we'll say all is clear | 21:33 |
| * mriedem has a hard time thinking with jesus christ super star blaring from laura's phone in the other room | 21:34 | |
| melwitt | yeah ... you're talking about the current state of affinity with multiple cells? | 21:34 |
| mriedem | yeah regardless of this change | 21:34 |
| melwitt | yeah, what I found is that since we're not targeted, we just get an empty hosts list every time | 21:34 |
| mriedem | hmm, is the group targeted when we get down to compute? https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/compute/manager.py#L1289 | 21:35 |
| melwitt | no because the group is pulled early before we target anything. I hope I wrote comments in the patch about that, I'm having trouble remembering it even now | 21:36 |
| melwitt | oh, that in compute manager, that will be local to a cell | 21:37 |
| dansmith | that'd be a bug, of course, | 21:37 |
| *** gjayavelu has joined #openstack-nova | 21:37 | |
| dansmith | if two instances in an affinity group were in different cells | 21:37 |
| dansmith | which maybe could have happened if we did something dumb in scheduler | 21:37 |
| dansmith | (maybe related to melwitt's patch) | 21:37 |
| dansmith | but that would be, like, bad, not a legit arrangement | 21:38 |
| melwitt | yeah. well, it definitely happens today because the query for hosts will always be empty | 21:38 |
| mriedem | dansmith: yeah it's totally a bug but i think it's something that could happen today with multiple cells and concurrent requests to create instances in the same affinity group | 21:38 |
| dansmith | ack, okay | 21:38 |
| melwitt | it doesn't have to be concurrent because no matter what, when you go to boot an instance and say "give me the hosts that are in this group" it will hand you an empty list because it's an untargeted DB access by nova-api | 21:39 |
| *** esberglu has joined #openstack-nova | 21:39 | |
| dansmith | probably worth backporting that fix then and calling it out in a reno that people should audit their groups if they are running multiple cells | 21:39 |
| dansmith | melwitt: yeah | 21:39 |
| openstackgerrit | Julia Kreger proposed openstack/nova master: ironic: add instance_uuid before any other spawn activity https://review.openstack.org/563722 | 21:40 |
| mriedem | so i still can't tell, | 21:40 |
| mriedem | during an instance create, | 21:41 |
| mriedem | when we get to the late affinity check in compute https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/compute/manager.py#L1288 | 21:41 |
| mriedem | is that group going to have a cell-targeted context in it? | 21:41 |
| dansmith | no, | 21:41 |
| dansmith | because it's local to the cell | 21:41 |
| dansmith | if it's compute manager | 21:41 |
| dansmith | it can't go anywhere other than the local cell | 21:41 |
| mriedem | so it just reads the [database]/connection | 21:41 |
| mriedem | ah right | 21:41 |
| dansmith | right | 21:41 |
| melwitt | the thing I'm concerned about is whether my patch will make that go to all cells, which would be wrong? | 21:42 |
| dansmith | does compute manager use setup_instance_group? | 21:42 |
| dansmith | I didn't think so | 21:42 |
| melwitt | oh, no it doesn't. okay, my bad | 21:43 |
| mriedem | setup_instance_group is only called from conductor i think, prior to sending the request spec to the scheduler to pick a host | 21:43 |
| dansmith | you know that there's still a race though, right? | 21:44 |
| dansmith | that's what the late-binding check is for, | 21:44 |
| dansmith | and we can't do that from the cell anyway | 21:44 |
| dansmith | so melwitt's patch will close the race by not just getting an empty list every time, | 21:44 |
| dansmith | but if we're concurrently booting several instances, you can still sprinkle them among the cells | 21:45 |
| dansmith | although, that makes me wonder... | 21:45 |
| mriedem | right i know for the concurrent boot scenario, we're already in trouble | 21:45 |
| dansmith | instead of doing the check late like we do now, | 21:45 |
| dansmith | what if we create the instance mapping (which means we've destined it to a cell) and then do the affinity check right there to make sure that the group hasn't spread across cells, | 21:46 |
| dansmith | then once we've done that, we can let the late check happen in computemanager like normal, which will only need to consider the local cell anyway | 21:46 |
| mriedem | there is still a race there though isn't there? | 21:47 |
| mriedem | it's the same race as for getting on different computes | 21:47 |
| dansmith | no, because if we know everything is in the same cell, then the late check as it is today works fine yeah? | 21:47 |
| dansmith | oh, although we can't look up the server group members there | 21:48 |
| dansmith | that's why | 21:48 |
| mriedem | my brain hurts | 21:48 |
| mriedem | so to summarize, i think with the concurrent affinity group create race, before multi-cell, you could race past the scheduler and late affinity check in the compute has to fix the race, | 21:49 |
| mriedem | with multiple cells, you could race past the scheduler and the late affinity check in compute might not fail because it can't "see" that the group has members in another cell | 21:49 |
| mriedem | to which it's not affined | 21:49 |
| mriedem | and you end up with an affinity group with members in multiple cells | 21:49 |
| dansmith | no, | 21:49 |
| dansmith | the late check is just for anti-affinity | 21:50 |
| dansmith | right? | 21:50 |
| mriedem | no | 21:50 |
| * mriedem links again https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/compute/manager.py#L1288 | 21:50 | |
| dansmith | oh, I guess it does check both | 21:50 |
| dansmith | the comment only says anti-affinity | 21:50 |
| dansmith | mriedem: so yes, your second comment is right, except that: | 21:51 |
| mriedem | so before multi-cell, you'd reschedule until you find the right host or fail, | 21:51 |
| dansmith | 1. for anti-affinity it doesn't matter since anything it doesn't find is clearly not on the same host and, | 21:51 |
| dansmith | 2. the problem is we can't upcall to get the members, not just that we can't see them in our db | 21:51 |
| mriedem | after multi-cell, we might not even see the problem | 21:51 |
| dansmith | right but with multi-cell #2 gets you anyway | 21:51 |
| *** liverpooler has joined #openstack-nova | 21:51 | |
| mriedem | agree with 1 yes, anti-affinity is totes fine | 21:51 |
| dansmith | because of the upcall | 21:51 |
| *** Guest49729 has quit IRC | 21:52 | |
| mriedem | if the cell isn't configured to hit the api db, then yes we can't upcall to iterate the cells to find hosts for all members in the group | 21:52 |
| *** yamamoto has joined #openstack-nova | 21:53 | |
| mriedem | this https://github.com/openstack/nova/blob/0ef3c685b9d2e0049f38fcf1a268870e69a5b9cf/nova/objects/instance_group.py#L473 | 21:53 |
| mriedem | yeah? | 21:53 |
| *** moshele has joined #openstack-nova | 21:53 | |
| dansmith | well, | 21:53 |
| *** brault has quit IRC | 21:53 | |
| dansmith | we won't iterate cells anyway, but yeah | 21:53 |
| *** itlinux_ has joined #openstack-nova | 21:53 | |
| *** wolverineav has quit IRC | 21:55 | |
| *** itlinux has quit IRC | 21:56 | |
| *** moshele has quit IRC | 21:57 | |
| *** yamamoto has quit IRC | 21:57 | |
| *** itlinux_ has quit IRC | 21:58 | |
| mriedem | so the only way to really fix this is for the scheduler to be aware of the location of the group members at the time of picking a host, and that has to be global, so placement, | 21:58 |
| mriedem | and we likely need to model affinity (distance) for the group members using....provider aggregates? i know this is a rathole, | 21:58 |
| mriedem | i keep thinking of cells as a provider aggregate | 21:59 |
| dansmith | well, this is why I've been punting the problem because placement affinity is better for lots of reasons | 21:59 |
| mriedem | melwitt: anyway, i think the functional change in your patch is ok, my main issue / concern is with the test | 22:00 |
| melwitt | mriedem: yeah, in the middle of replying now | 22:00 |
| mriedem | melwitt: a functional test with real services would be ideal | 22:00 |
| melwitt | well, I think we need the unit test to cover all the weird paths, unless I'm just not thinking creatively enough | 22:00 |
| mriedem | the majority of the work in a functional test for stuff like this is in the setup, | 22:01 |
| melwitt | I'm gonna try to take a step back and see if I can come up with a real world scenario(s) that will take all of the paths that we can verify | 22:01 |
| mriedem | after that it's just running scenarios and asserting stuff | 22:01 |
| mriedem | https://review.openstack.org/#/c/565886/6/nova/tests/functional/test_nova_manage.py@357 is an example with 2 working cells | 22:02 |
| melwitt | this is weird because there are two steps: 1) look up the hosts for members of the requested group 2) look up the hosts for members of the group the instance is a member of | 22:03 |
| melwitt | and I was trying to test that both of those queries do the multi-cell thing. but yeah, probably could set this up for a functional test. I'll try it | 22:04 |
| mriedem | with complicated changes like this, i find it's easier to write the functional test to setup the environment like the user would run a use case | 22:05 |
| mriedem | using the actual APIs | 22:05 |
| mriedem | to create the groups and add members to them and such | 22:05 |
| *** eharney has quit IRC | 22:06 | |
| *** mingyu has joined #openstack-nova | 22:06 | |
| mriedem | otherwise it's too easy to fake things out in the db that aren't accurate | 22:06 |
| melwitt | for whatever reason, I did not expect it would be easy to do a functional test. I agree it would be a lot better to reason about too | 22:07 |
| mriedem | mostly just copy/paste the setup | 22:07 |
| mriedem | pretty easy | 22:07 |
| *** rcernin has joined #openstack-nova | 22:07 | |
| mriedem | the one thing with multi-cell functional and having different hosts in different cells, you'll need https://review.openstack.org/#/c/558160/ | 22:08 |
| mriedem | otherwise the computes all get created in the default cell1 | 22:08 |
| mriedem | but that's approved now so shouldn't be a problem - you just need to specify the cell you want the compute in when you create it | 22:08 |
| melwitt | I know, I tried to solve that problem with my CellDatabases patch months ago but people weren't okay with it because I did the ServiceWrapper thing | 22:09 |
| mriedem | i've been told i need to go workout because i've become somewhat of a troll, so ttyl | 22:09 |
| *** mriedem is now known as mrjazzercise | 22:09 | |
| *** gjayavelu has quit IRC | 22:20 | |
| openstackgerrit | Jay Pipes proposed openstack/nova master: process groups individually and merge candidates https://review.openstack.org/566180 | 22:23 |
| *** slaweq has quit IRC | 22:23 | |
| *** moshele has joined #openstack-nova | 22:29 | |
| *** slaweq has joined #openstack-nova | 22:35 | |
| *** pchavva has quit IRC | 22:36 | |
| *** vladikr has quit IRC | 22:36 | |
| *** moshele has quit IRC | 22:38 | |
| *** felipemonteiro__ has joined #openstack-nova | 22:43 | |
| *** felipemonteiro__ has quit IRC | 22:44 | |
| *** liverpooler has quit IRC | 22:44 | |
| *** felipemonteiro__ has joined #openstack-nova | 22:44 | |
| *** felipemonteiro_ has quit IRC | 22:47 | |
| openstackgerrit | Merged openstack/nova master: Handle @safe_connect returns None side effect in _ensure_resource_provider https://review.openstack.org/566096 | 22:52 |
| *** moshele has joined #openstack-nova | 22:53 | |
| *** damien_r has joined #openstack-nova | 22:55 | |
| *** gjayavelu has joined #openstack-nova | 22:57 | |
| openstackgerrit | Merged openstack/nova master: Fix the request context in ServiceFixture https://review.openstack.org/558160 | 22:58 |
| *** threestrands has joined #openstack-nova | 22:59 | |
| *** damien_r has quit IRC | 22:59 | |
| *** felipemonteiro__ has quit IRC | 23:04 | |
| *** hongbin has quit IRC | 23:06 | |
| openstackgerrit | Arvind Nadendla proposed openstack/nova-specs master: Handle rebuild of instance with new image https://review.openstack.org/560718 | 23:07 |
| *** gouthamr has quit IRC | 23:08 | |
| *** moshele has quit IRC | 23:08 | |
| *** rajinir has quit IRC | 23:09 | |
| *** gouthamr has joined #openstack-nova | 23:10 | |
| *** slaweq has quit IRC | 23:13 | |
| *** dpawlik has joined #openstack-nova | 23:14 | |
| *** dpawlik has quit IRC | 23:18 | |
| *** gouthamr has quit IRC | 23:18 | |
| *** masber has joined #openstack-nova | 23:23 | |
| idlemind | http://paste.openstack.org/show/720326/ | 23:33 |
| idlemind | I need those database connections to be updated to .9 not .11 ... what's the best way to do that? | 23:34 |
| *** mlavalle has quit IRC | 23:34 | |
| idlemind | can i just drop the cells (delete) and they'll get recreated? | 23:34 |
| idlemind | or should i update them | 23:34 |
| openstackgerrit | Oliver Walsh proposed openstack/nova stable/pike: Handle @safe_connect returns None side effect in _ensure_resource_provider https://review.openstack.org/566164 | 23:46 |
| openstackgerrit | Arvind Nadendla proposed openstack/nova-specs master: Handle rebuild of instance with new image https://review.openstack.org/560718 | 23:48 |
| *** gouthamr has joined #openstack-nova | 23:51 | |
| *** yamamoto has joined #openstack-nova | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!