| *** derekh has quit IRC | 00:08 | |
| *** derekh has joined #openstack-nova | 00:08 | |
| *** gyee has quit IRC | 00:10 | |
| *** mlavalle has quit IRC | 00:29 | |
| *** avolkov has quit IRC | 00:34 | |
| *** artom has quit IRC | 00:38 | |
| *** ricolin has joined #openstack-nova | 00:43 | |
| brinzhang | amotoki: What do you mean https://review.opendev.org/#/c/673485/3/novaclient/v2/shell.py@2609 ? | 00:45 |
|---|---|---|
| amotoki | brinzhang: sorry, I looked some wrong line when scrolling the review... I will add a comment. | 00:46 |
| brinzhang | as mriedem said, this patch do the ``nova volume-attach`` CLI, not server create | 00:46 |
| brinzhang | amotoki: np | 00:46 |
| brinzhang | amotoki: And I want to keep the option delete-on-termination in novaclient with '--delete-on-termination', but I am not sure use this in OSC is fine? | 00:48 |
| brinzhang | amotoki: As sain in PS3, how about add enable/disable befor the 'delete-on-termination', what do you think? | 00:49 |
| amotoki | brinzhang: sounds good for --delete-on-termination. | 00:49 |
| brinzhang | amotoki: I will keep --delete-on-termination in novaclient. | 00:50 |
| amotoki | brinzhang: if we need an option to specify 'delete_on_termination=False', I see several options: (a) --delete-on-termination / --no-delete-on-termination (b) --enable-delete-on-termination/--disable-delete-on-termination (c) --delete-on-termination / --keep-on-termination | 00:51 |
| *** markvoelker has joined #openstack-nova | 00:51 | |
| amotoki | brinzhang: thought? | 00:52 |
| brinzhang | amotoki: I tend to (b) --enable-delete-on-termination/--disable-delete-on-termination | 00:53 |
| brinzhang | amotoki: I will rebase the novaclient patch, could please review the OSC patch, and you can leave that comment in it? Or after I rebased it. https://review.opendev.org/#/c/673725/6 ^^ | 00:54 |
| *** markvoelker has quit IRC | 00:56 | |
| *** brinzhang_ has joined #openstack-nova | 00:57 | |
| amotoki | brinzhang: sure. | 00:57 |
| amotoki | brinzhang: btw, is there any compute API ref corresponding to this? | 00:58 |
| *** brinzhang has quit IRC | 01:00 | |
| brinzhang_ | amotoki: is this what dou you want?https://docs.openstack.org/api-ref/compute/#attach-a-volume-to-an-instance | 01:01 |
| amotoki | brinzhang_: but I cannot find information on delete_on_termination. | 01:02 |
| brinzhang_ | amotoki: this field in this compute API is new, after https://review.opendev.org/#/c/673133/ is merged, you can look at it. | 01:04 |
| amotoki | brinzhang_: thanks. It would help me when reviewing CLI patch. | 01:04 |
| brinzhang_ | amotoki: but in create server API in request body(block_device_mapping_v2.delete_on_termination (Optional)) has this field, you can reference https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server | 01:05 |
| amotoki | brinzhang_: mainly I would like to check the default value of the option. | 01:06 |
| openstackgerrit | Akira KAMIO proposed openstack/nova master: VMware: disk_io_limits settings are not reflected when resize https://review.opendev.org/680296 | 01:08 |
| gmann | amotoki: this one - https://review.opendev.org/#/c/673133/16/nova/api/openstack/compute/volumes.py L342 | 01:08 |
| amotoki | gmann: thanks. | 01:10 |
| *** slaweq has joined #openstack-nova | 01:11 | |
| *** slaweq has quit IRC | 01:16 | |
| *** spsurya has joined #openstack-nova | 01:27 | |
| *** BjoernT has joined #openstack-nova | 01:36 | |
| openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.79: Add delete_on_termination to volume-attach API https://review.opendev.org/673485 | 01:37 |
| openstackgerrit | Merged openstack/nova master: Handle VirtDriverNotReady in _cleanup_running_deleted_instances https://review.opendev.org/679473 | 01:39 |
| openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.79: Add delete_on_termination to volume-attach API https://review.opendev.org/673485 | 01:41 |
| *** hongbin has joined #openstack-nova | 02:13 | |
| *** derekh has quit IRC | 02:29 | |
| *** derekh has joined #openstack-nova | 02:29 | |
| *** dave-mccowan has quit IRC | 02:38 | |
| *** gbarros has joined #openstack-nova | 02:39 | |
| *** tinwood has quit IRC | 02:49 | |
| *** tinwood has joined #openstack-nova | 02:50 | |
| *** larainema has joined #openstack-nova | 03:01 | |
| openstackgerrit | Akihiro Motoki proposed openstack/nova master: PDF documentation build https://review.opendev.org/676730 | 03:06 |
| *** slaweq has joined #openstack-nova | 03:11 | |
| *** slaweq has quit IRC | 03:16 | |
| *** slaweq has joined #openstack-nova | 03:32 | |
| *** mvkr has joined #openstack-nova | 03:34 | |
| *** gbarros has quit IRC | 03:45 | |
| *** BjoernT has quit IRC | 03:57 | |
| *** nicolasbock has quit IRC | 03:58 | |
| *** yedongcan has joined #openstack-nova | 04:02 | |
| *** derekh has quit IRC | 04:02 | |
| *** derekh has joined #openstack-nova | 04:03 | |
| *** igordc has joined #openstack-nova | 04:10 | |
| *** etp has joined #openstack-nova | 04:17 | |
| *** igordc has quit IRC | 04:23 | |
| *** jaosorior has quit IRC | 04:34 | |
| *** jaosorior has joined #openstack-nova | 04:34 | |
| *** hongbin has quit IRC | 04:35 | |
| *** HagunKim has joined #openstack-nova | 04:39 | |
| *** Luzi has joined #openstack-nova | 04:52 | |
| *** shilpasd has joined #openstack-nova | 04:53 | |
| *** shilpasd has quit IRC | 04:57 | |
| *** ileixe has left #openstack-nova | 05:05 | |
| *** mkrai has joined #openstack-nova | 05:08 | |
| *** markvoelker has joined #openstack-nova | 05:15 | |
| *** markvoelker has quit IRC | 05:20 | |
| *** pcaruana has joined #openstack-nova | 05:25 | |
| *** ratailor has joined #openstack-nova | 05:43 | |
| *** etp has quit IRC | 05:47 | |
| *** lpetrut has joined #openstack-nova | 05:49 | |
| *** jaosorior has quit IRC | 05:49 | |
| *** jaosorior has joined #openstack-nova | 05:51 | |
| *** udesale has joined #openstack-nova | 05:54 | |
| *** yedongcan has quit IRC | 05:54 | |
| *** etp has joined #openstack-nova | 05:55 | |
| *** maciejjozefczyk has joined #openstack-nova | 05:55 | |
| *** itlinux has joined #openstack-nova | 06:00 | |
| *** markvoelker has joined #openstack-nova | 06:06 | |
| openstackgerrit | Michael Davies proposed openstack/os-traits master: Add support for ppc64le platforms https://review.opendev.org/680580 | 06:10 |
| *** markvoelker has quit IRC | 06:10 | |
| *** brinzhang_ has quit IRC | 06:14 | |
| *** brinzhang_ has joined #openstack-nova | 06:15 | |
| *** brinzhang_ has quit IRC | 06:16 | |
| *** lpetrut has quit IRC | 06:16 | |
| *** etp has quit IRC | 06:16 | |
| *** lpetrut has joined #openstack-nova | 06:16 | |
| *** etp has joined #openstack-nova | 06:16 | |
| openstackgerrit | ya.wang proposed openstack/nova master: vCPU model selection https://review.opendev.org/670298 | 06:31 |
| openstackgerrit | ya.wang proposed openstack/nova master: Add compatibility checks for CPU mode and CPU models and extra flags https://review.opendev.org/670299 | 06:31 |
| openstackgerrit | ya.wang proposed openstack/nova master: Support reporting multi CPU model traits https://review.opendev.org/670300 | 06:31 |
| *** markvoelker has joined #openstack-nova | 06:36 | |
| *** jawad_axd has joined #openstack-nova | 06:38 | |
| *** ildikov has joined #openstack-nova | 06:40 | |
| *** markvoelker has quit IRC | 06:41 | |
| *** trident has quit IRC | 06:53 | |
| *** trident has joined #openstack-nova | 07:04 | |
| *** markvoelker has joined #openstack-nova | 07:06 | |
| *** brault has joined #openstack-nova | 07:07 | |
| *** itlinux has quit IRC | 07:09 | |
| *** threestrands has quit IRC | 07:10 | |
| *** rcernin|brb has quit IRC | 07:10 | |
| *** tesseract has joined #openstack-nova | 07:11 | |
| *** luksky has joined #openstack-nova | 07:11 | |
| *** markvoelker has quit IRC | 07:11 | |
| *** ccamacho has joined #openstack-nova | 07:13 | |
| gibi | bauzas: hi! I'm about the start changing the bottom of the bw series https://review.opendev.org/#/c/656422 Yesterday we agreed with mriedem that the bottom can be fixed in a follow up, so if you are about to +A that patch then I will go with the followup. If not then I go and fix the patch itself instead | 07:14 |
| bauzas | gibi: yup, I was reviewing and I saw matt's comments | 07:14 |
| bauzas | gibi: as you want | 07:14 |
| bauzas | gibi: I'm not done with reviewing your bottom change but I'm absolutely not opposed to a followup | 07:15 |
| gibi | bauzas: then please continue reviewing it and I do a follow up | 07:15 |
| bauzas | cool | 07:15 |
| gibi | bauzas: and thank you :) | 07:15 |
| bauzas | (on a train with limited connection but whatever :) ) | 07:15 |
| * bauzas 's getting audited for his first visa application and is afraid :) | 07:16 | |
| *** markvoelker has joined #openstack-nova | 07:16 | |
| * gibi is about to file that visa application today | 07:16 | |
| gibi | fun times | 07:16 |
| cervigni | os summit? | 07:16 |
| gibi | yepp | 07:17 |
| *** brinzhang has joined #openstack-nova | 07:17 | |
| cervigni | oh god... I will procrastinate that until 2 weeks before | 07:17 |
| bauzas | yeah | 07:17 |
| bauzas | I need to provide my fingerprints | 07:17 |
| cervigni | I don't even know if taking my laptop or not | 07:17 |
| bauzas | hence me traveling to the closest consulate place | 07:17 |
| cervigni | really? | 07:17 |
| bauzas | yeah, even my passport is biometrical | 07:18 |
| bauzas | and already having my 10 fingerprints | 07:18 |
| cervigni | is this country dependant or not? | 07:18 |
| cervigni | I don't think I have a Chinese consulate here | 07:18 |
| bauzas | but you basically need to go to a visa app center that'll retain your passport by the time it'll take them to check your application | 07:18 |
| cervigni | WHAT? | 07:19 |
| cervigni | hahaha | 07:19 |
| cervigni | RETAIN your passport? | 07:19 |
| bauzas | for Lyon, France, the average delay is 8 working days | 07:19 |
| bauzas | correct | 07:19 |
| cervigni | I don't want anyone to "retain" my passport | 07:19 |
| cervigni | I cannot be legally here without my passport | 07:19 |
| gibi | that sounds pretty bad. I thought in Hungary I only need to send a photocopy of my passport | 07:19 |
| bauzas | cervigni: https://visaforchina.org | 07:19 |
| cervigni | "org" | 07:20 |
| cervigni | haha | 07:20 |
| bauzas | gibi: you don't need to be present yourself ? | 07:20 |
| cervigni | wow nice website | 07:20 |
| cervigni | is this an official website? | 07:20 |
| gibi | bauzas: it seems the travell agencyt can make it work without mee | 07:20 |
| *** markvoelker has quit IRC | 07:20 | |
| bauzas | gibi: you can pay a TA | 07:20 |
| gibi | my employer pays for it (i hope) | 07:20 |
| bauzas | gibi: but for us, it will still mean that we have to physically move to an appointment to get our fingerprints taken | 07:21 |
| bauzas | gibi: me too | 07:21 |
| cervigni | are we on the F category? | 07:21 |
| cervigni | or M ? | 07:21 |
| gibi | this information is still missing for me, I trust the TA | 07:21 |
| bauzas | cervigni: some asked for F but were said it should be a M | 07:21 |
| bauzas | gibi: just check the website above | 07:21 |
| cervigni | yeah, I don't think we are though commerce and trade | 07:22 |
| bauzas | gibi: if they don't tell you to physically attend an appointment, then cool | 07:22 |
| bauzas | cervigni: I should be able to tell you in 3 hours -ish | 07:22 |
| bauzas | I filed a M | 07:22 |
| cervigni | ok, could you write it in a PM? | 07:22 |
| cervigni | here gmt+8 | 07:22 |
| bauzas | sure | 07:23 |
| cervigni | many thanks | 07:23 |
| gibi | bauzas: there is not even a service center in hungary based on that page. But we have a consulate | 07:23 |
| gibi | anyhow good luck bauzas | 07:23 |
| cervigni | on that website from Australia it seems there is no fingerprint requirements | 07:24 |
| cervigni | though I am italian so it is very confusing | 07:24 |
| bauzas | gibi: good luck for you too then | 07:25 |
| bauzas | gibi: that's probably why it's a good idea for you to follow a TA :) | 07:25 |
| gibi | bauzas: yepp. I saw it working. One of my collague went to China last year | 07:25 |
| gibi | successfully :) | 07:26 |
| bauzas | cervigni: I guess you have to look at the closest service center in Australia | 07:26 |
| bauzas | they should be able to give you instructions | 07:26 |
| cervigni | Yeah I check and thanks god there is one in Perth | 07:26 |
| bauzas | even if you're not Aussie | 07:26 |
| cervigni | Will do that... or I will just forget to come to the os summit and wait for the next one | 07:27 |
| cervigni | I don't think I will give my fingerprints to the chinese governament for any reasons | 07:27 |
| bauzas | oh dear god, I paid for a 1st class train which *has* power plugs but which are dysfunctional | 07:27 |
| cervigni | I already give them allto the Australian one and I was pissed of for a month | 07:27 |
| cervigni | bauzas: I guess it is an Italian train | 07:28 |
| bauzas | Alstom, take it as you wish | 07:28 |
| cervigni | I have never found an outlet working in an Italian train | 07:28 |
| cervigni | going home, good luck with your visas | 07:29 |
| cervigni | let me know how it goes | 07:29 |
| *** macz has joined #openstack-nova | 07:31 | |
| *** macz has quit IRC | 07:36 | |
| bauzas | gibi: +Wd with concerns | 07:42 |
| bauzas | gibi: my understanding is that you'll fail the resize when you wanna migrate/resize from host A to B when B isn't upgraded, right? | 07:43 |
| *** ralonsoh has joined #openstack-nova | 07:45 | |
| * bauzas drops since the train arrives at destination | 07:47 | |
| bauzas | ttyl | 07:47 |
| *** avolkov has joined #openstack-nova | 07:54 | |
| *** ivve has joined #openstack-nova | 08:01 | |
| openstackgerrit | Yongli He proposed openstack/python-novaclient master: Microversion 2.78 - show server topology https://review.opendev.org/670790 | 08:01 |
| *** macz has joined #openstack-nova | 08:04 | |
| *** pcaruana has quit IRC | 08:08 | |
| kashyap | Does anyone here how to remove an instance that is stuck in 'pre-migrating' (it's the migration status) state? | 08:12 |
| kashyap | I mean, not _remove_ the instance, but more unjamming it from that state. (Besides some database muckery) | 08:12 |
| *** ociuhandu has joined #openstack-nova | 08:13 | |
| *** pcaruana has joined #openstack-nova | 08:13 | |
| *** ociuhandu has quit IRC | 08:18 | |
| *** cdent has joined #openstack-nova | 08:19 | |
| yaawang | alex_xu: stephenfin Hi, I have updated patches of vCPU models selection, can you review them if you have time :) https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/cpu-model-selection | 08:24 |
| *** macz has quit IRC | 08:27 | |
| *** tkajinam has quit IRC | 08:32 | |
| *** trident has quit IRC | 08:40 | |
| *** gbarros has joined #openstack-nova | 08:40 | |
| *** cdent has quit IRC | 08:44 | |
| gibi | bauzas: thanks. replied. | 08:46 |
| * gibi was pulled in to a downstream discussion in the mean time | 08:47 | |
| *** trident has joined #openstack-nova | 08:51 | |
| *** zigo has quit IRC | 08:57 | |
| *** zigo has joined #openstack-nova | 09:02 | |
| *** slaweq has quit IRC | 09:06 | |
| aspiers | kashyap: some trivial changes for you to review: https://review.opendev.org/#/c/680527/ | 09:07 |
| aspiers | https://review.opendev.org/#/c/680526/ | 09:07 |
| kashyap | aspiers: Stuck in something urgent; but already have them open. | 09:08 |
| openstackgerrit | Guo Jingyu proposed openstack/nova master: Define and catch InstanceAgentNotRunning for set-password failure https://review.opendev.org/661466 | 09:08 |
| *** markvoelker has joined #openstack-nova | 09:10 | |
| kashyap | aspiers: FWIW, ACK to the tests move; nice catch (by "SEV work") | 09:16 |
| aspiers | kashyap: thanks! | 09:17 |
| *** markvoelker has quit IRC | 09:20 | |
| *** cdent has joined #openstack-nova | 09:25 | |
| *** spatel has joined #openstack-nova | 09:31 | |
| openstackgerrit | Merged openstack/nova master: Add delete_on_termination to volume-attach API https://review.opendev.org/673133 | 09:31 |
| *** shilpasd has joined #openstack-nova | 09:33 | |
| *** spatel has quit IRC | 09:35 | |
| *** yedongcan has joined #openstack-nova | 09:35 | |
| *** ivve has quit IRC | 09:36 | |
| *** slaweq has joined #openstack-nova | 09:36 | |
| *** dtantsur|afk is now known as dtantsur | 09:37 | |
| *** ivve has joined #openstack-nova | 09:39 | |
| dtantsur | morning folks, we seem to have broken ironic with the recent nova patches | 09:44 |
| cdent | dtantsur: sounds fun | 09:45 |
| dtantsur | as always! | 09:46 |
| cdent | when did it stop working? | 09:50 |
| dtantsur | cdent: it's because of https://review.opendev.org/#/c/656027/, a fix is already posted | 09:51 |
| cdent | ah | 09:51 |
| dtantsur | efried: and I also wonder why our regular job did not detect ^^ | 09:51 |
| dtantsur | maybe list_instances is not normally hit or is not critical? | 09:51 |
| cdent | i would guess that most of the time individual instances are retrieved | 09:55 |
| cdent | but that's totally a guess | 09:55 |
| openstackgerrit | hulina proposed openstack/nova master: Nova raise exceptions when extending volume fails https://review.opendev.org/680648 | 10:00 |
| *** artom has joined #openstack-nova | 10:07 | |
| artom | stephenfin, sean-k-mooney, can I get your thoughts on a thing? | 10:09 |
| artom | You know this bit: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4525-L4528 | 10:10 |
| artom | Normally on a host not intended for NUMA instances, https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4549 will return None because vcpu_pin_set will not be set | 10:10 |
| stephenfin | looking | 10:11 |
| stephenfin | artom: Yeah, correct | 10:12 |
| artom | If, however, an instance without a NUMA topology lands on a host with vcpu_pin_set not None, guest.cpuset being not None ends up triggering https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L2587-L2590 and we end up pinning guest CPUs that way | 10:12 |
| artom | (via https://libvirt.org/formatdomain.html#elementsCPUAllocation, not our usual vcpupin XML bits) | 10:13 |
| * bauzas was released from the visa application center | 10:14 | |
| artom | Do we care about *that* form of CPU pinning (the one via <vcpu cpuset=blah>) for live migration? | 10:14 |
| stephenfin | artom: um, I think that's correct, yeah | 10:14 |
| artom | Theoretically we should be recalculating it | 10:14 |
| bauzas | now I'm a bit afraid of my application be rejected because of some coffee traces on my passport :/ | 10:14 |
| stephenfin | yup | 10:14 |
| stephenfin | we should | 10:14 |
| artom | But... in practice, does it actually happen? | 10:14 |
| stephenfin | ohh | 10:14 |
| stephenfin | hmm, I'm not actually sure | 10:15 |
| *** ricolin has quit IRC | 10:15 | |
| artom | We tell everyone to split NUMA instances into host aggregates | 10:15 |
| bauzas | stephenfin: I also think we should correctly document the weird case you mention in https://review.opendev.org/#/c/680107/3 | 10:15 |
| stephenfin | I had this written off as being fixed by your series | 10:15 |
| artom | So a non-NUMA instances landing on a host with vcpu_pin_set is... unlikely? | 10:15 |
| artom | stephenfin, hah, I've just noticed that it's a thing | 10:15 |
| bauzas | stephenfin: because the behaviour will change for those not RTFM'ing | 10:15 |
| stephenfin | bauzas: Nope, it won't change | 10:15 |
| *** ociuhandu has joined #openstack-nova | 10:15 | |
| stephenfin | *Break | 10:15 |
| stephenfin | bauzas: see here | 10:15 |
| stephenfin | bauzas: https://review.opendev.org/#/c/680107/3/nova/compute/manager.py@861 | 10:16 |
| stephenfin | I'm only warning | 10:16 |
| stephenfin | Not raising | 10:16 |
| stephenfin | bauzas: For those other two conditionals, not that I'm only checking the _new_ options (cpu_dedicated_set, cpu_shared_set) | 10:17 |
| bauzas | see my comment | 10:17 |
| stephenfin | *note | 10:17 |
| stephenfin | Yup. The conditional is here https://review.opendev.org/#/c/680107/3/nova/compute/manager.py@827 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: db: Add resources column in instance_extra table https://review.opendev.org/678447 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: object: Introduce Resource and ResourceList objs https://review.opendev.org/678448 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Add resources dict into _Provider https://review.opendev.org/678449 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Retrieve the allocations early https://review.opendev.org/678450 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Claim resources in resource tracker https://review.opendev.org/678452 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver discovering PMEM namespaces https://review.opendev.org/678453 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree https://review.opendev.org/678454 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup https://review.opendev.org/678455 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec https://review.opendev.org/678456 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory https://review.opendev.org/678470 | 10:17 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: doc: attaching virtual persistent memory to guests https://review.opendev.org/680300 | 10:17 |
| stephenfin | and I early return here https://review.opendev.org/#/c/680107/3/nova/compute/manager.py@839 | 10:17 |
| stephenfin | so lines 841 etc. will only be triggered if the instance has a NUMA topology and that topology has pinning enabled | 10:18 |
| bauzas | oh shit, that's what happen when you review in a crowded Starbucks place | 10:18 |
| bauzas | you're totally right | 10:18 |
| stephenfin | bauzas: but at least you've mediocre coffee on hand :P | 10:18 |
| stephenfin | If it's not clear, I can rework to use an if-else | 10:18 |
| * stephenfin tends to favour early returns since they need less indentation, but maybe they're less clear | 10:19 | |
| *** jawad_axd has quit IRC | 10:19 | |
| *** ociuhandu has quit IRC | 10:19 | |
| openstackgerrit | Merged openstack/nova master: update allocation in binding profile during migrate https://review.opendev.org/656422 | 10:20 |
| bauzas | stephenfin: nah nah, it's cool | 10:21 |
| bauzas | stephenfin: we could nitpick on getting an extra comment, but meh | 10:21 |
| bauzas | and yeah Starbucks is mediocre, but that's the only place where you can guaranttee both decent wifi coverage and working outlets | 10:22 |
| stephenfin | In the code or the commit message? I can do the latter in a follow-up | 10:22 |
| stephenfin | Yeah, agreed. Costa just isn't the same (and it's UK only) | 10:23 |
| bauzas | I wish I could afford a Regus usage :) | 10:23 |
| bauzas | stephenfin: well, I was thinking of just a comment line in the code itself but that's gross | 10:23 |
| bauzas | nevermind it | 10:23 |
| bauzas | it's just me who can't read coee | 10:23 |
| bauzas | code* | 10:24 |
| openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.79: Add delete_on_termination to volume-attach API https://review.opendev.org/673485 | 10:24 |
| stephenfin | 'sall good :D | 10:24 |
| *** shilpasd has quit IRC | 10:25 | |
| *** ociuhandu has joined #openstack-nova | 10:29 | |
| *** ociuhandu has quit IRC | 10:30 | |
| *** luksky has quit IRC | 10:36 | |
| stephenfin | artom: how do you decide to recalculate the '<cpuset>' values on a destination host in your series? | 10:36 |
| artom | stephenfin, I currently don't, at all | 10:37 |
| sean-k-mooney | you just always recaluate it right | 10:37 |
| sean-k-mooney | as part of the claim | 10:37 |
| *** maciejjozefczyk has quit IRC | 10:37 | |
| artom | sean-k-mooney, no, currently it's not handled at all | 10:37 |
| sean-k-mooney | which cpuset | 10:38 |
| artom | But in theory it should happen when https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4525-L4528 | 10:38 |
| artom | sean-k-mooney, the one in https://libvirt.org/formatdomain.html#elementsCPUAllocation | 10:38 |
| sean-k-mooney | oh that should be the union of hte cores used for the vcpus | 10:38 |
| sean-k-mooney | and should be updated | 10:38 |
| artom | It only matters when an instance without a NUMA topology lands on a host with vcpu_pin | 10:38 |
| stephenfin | alex_xu: You around today? I'm planning to work through the VPMEM series again, but is there any chance you could look at https://review.opendev.org/#/c/671800 ? | 10:38 |
| sean-k-mooney | artom: yes | 10:38 |
| artom | sean-k-mooney, right, so is that... a real thing? | 10:39 |
| artom | We tell people to split them by aggregates | 10:39 |
| sean-k-mooney | yes | 10:39 |
| stephenfin | artom: wait for it | 10:39 |
| artom | They don't? | 10:39 |
| sean-k-mooney | you shoudl set vcpu_pin_set on all host ideally | 10:39 |
| stephenfin | (sean-k-mooney is about to tell you that vcpu_pin_set has nothing to do with pinning and should always be set) | 10:39 |
| sean-k-mooney | and it will be need for the new stuff stephenfin is working on | 10:39 |
| artom | Haha | 10:39 |
| sean-k-mooney | stephenfin: :) | 10:39 |
| openstackgerrit | Gorka Eguileor proposed openstack/nova master: Use os-brick locking for volume attach and detach https://review.opendev.org/614190 | 10:40 |
| stephenfin | but yeah, vcpu_pin_set != pinning (or NUMA, for that matter) | 10:40 |
| stephenfin | we use it for this stuff | 10:40 |
| sean-k-mooney | its the set of core that the hypvervior may use for vms | 10:40 |
| stephenfin | and also to decide how much VCPU inventory to report to placement | 10:40 |
| sean-k-mooney | nothing more or less | 10:40 |
| *** tbachman has quit IRC | 10:40 | |
| sean-k-mooney | yep its used for the inventory in the RT which is used for plamcent | 10:41 |
| stephenfin | I don't agree with it being set by default, but yeah, it being set even when you don't have NUMA or pinning is a thing | 10:41 |
| artom | OK, so it needs to be handled | 10:41 |
| stephenfin | afraid so | 10:41 |
| sean-k-mooney | stephenfin: you realise that for you vcpu work | 10:41 |
| sean-k-mooney | we shoudl also be handelign shared and dedicated sets | 10:41 |
| artom | sean-k-mooney, I'll most likely as you to test that bit when it's implemented | 10:42 |
| stephenfin | sean-k-mooney: what do you mean? | 10:42 |
| artom | Or if we can set it in the gate and show that working... | 10:42 |
| sean-k-mooney | stephenfin: well for non pinned instance the cpuset will need to be updated based on the dest cpu_shared_set | 10:42 |
| sean-k-mooney | and for pinned instnce it need to be updated based on the dedicated set | 10:43 |
| sean-k-mooney | although the second half will be done by artoms exisitng code | 10:43 |
| stephenfin | sean-k-mooney: Yeah, correct. That code has been updated here https://review.opendev.org/#/c/671800/28/nova/virt/libvirt/driver.py | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: object: Introduce Resource and ResourceList objs https://review.opendev.org/678448 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Add resources dict into _Provider https://review.opendev.org/678449 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Retrieve the allocations early https://review.opendev.org/678450 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Claim resources in resource tracker https://review.opendev.org/678452 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver discovering PMEM namespaces https://review.opendev.org/678453 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree https://review.opendev.org/678454 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup https://review.opendev.org/678455 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec https://review.opendev.org/678456 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory https://review.opendev.org/678470 | 10:44 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: doc: attaching virtual persistent memory to guests https://review.opendev.org/680300 | 10:44 |
| sean-k-mooney | by the way people know that cpuset in the vcpu element is funtionally the same as generatin vcpuin elemnt for each core pinning the core to the set of host core set in vcpu pin set | 10:46 |
| artom | stephenfin, sean-k-mooney, actually, while we're on the topic, I thought you could get CPU pinning with https://libvirt.org/formatdomain.html#elementsCPUTuning without NUMA pinning with https://libvirt.org/formatdomain.html#elementsNUMATuning | 10:46 |
| artom | For example by using only the cpu_policy=dedicated extra spec | 10:46 |
| sean-k-mooney | artom: yes you can | 10:46 |
| sean-k-mooney | no | 10:46 |
| sean-k-mooney | if you use cpu_policy=dedicated | 10:47 |
| *** maciejjozefczyk has joined #openstack-nova | 10:47 | |
| sean-k-mooney | you will have a numa toplogy | 10:47 |
| sean-k-mooney | if you set vcpu_pin_set on the host | 10:47 |
| artom | But looking at the code of _get_guets_numa_config, IIUC if it sets vcpupin, it'll also set the memnode stuff | 10:47 |
| sean-k-mooney | and set nothing on the flavor | 10:47 |
| sean-k-mooney | you get <vcpu placement='static' cpuset="1-4,^3,6" current="1">2</vcpu> | 10:47 |
| artom | sean-k-mooney, right, that's the case with <vcpu cpuset=blah> that I'm not handling | 10:47 |
| sean-k-mooney | or whatever the vcpu_pin_set is but no numa affinity | 10:47 |
| artom | I was talking about <cputune> without <numatune> | 10:48 |
| artom | Is that possible? Don't think so | 10:48 |
| sean-k-mooney | oh libvirt allows that but we never generate it | 10:48 |
| artom | Aha, ok | 10:48 |
| artom | Confirms what I was seeing in the code, thanks | 10:48 |
| sean-k-mooney | we only generate the cputune element if you have pinning nebaled which give you a numa toplogy | 10:49 |
| sean-k-mooney | we do generate numatune without cpu tune i think | 10:49 |
| sean-k-mooney | e.g. if you just enable hugepages but no pinning | 10:49 |
| sean-k-mooney | or hw:numa_nodes=X | 10:49 |
| artom | sean-k-mooney, you sure? It would happen only if https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4609 doesn't iterate at all | 10:50 |
| sean-k-mooney | we definitly numa affine hugepages | 10:50 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree https://review.opendev.org/678454 | 10:50 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup https://review.opendev.org/678455 | 10:50 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec https://review.opendev.org/678456 | 10:50 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | 10:50 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory https://review.opendev.org/678470 | 10:50 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: doc: attaching virtual persistent memory to guests https://review.opendev.org/680300 | 10:50 |
| sean-k-mooney | im just not sure if we use vcpupin to pin the cores to the numa node or cpuset | 10:51 |
| artom | sean-k-mooney, right, I'm saying in those cases we also affine guest CPUs with cputune | 10:51 |
| sean-k-mooney | ah ok | 10:51 |
| sean-k-mooney | well as i said cpuset is just a shortcurt for vcpupin | 10:51 |
| sean-k-mooney | let me give an example | 10:51 |
| *** ivve has quit IRC | 10:52 | |
| sean-k-mooney | those are the same | 10:52 |
| sean-k-mooney | <vcpu placement='static' cpuset="1-4" current="1">2</vcpu> | 10:52 |
| sean-k-mooney | <cputune> | 10:52 |
| sean-k-mooney | <vcpupin vcpu="0" cpuset="1-4"/> | 10:52 |
| sean-k-mooney | <vcpupin vcpu="1" cpuset="1,2,3,4"/> | 10:52 |
| sean-k-mooney | <cputune> | 10:52 |
| sean-k-mooney | libvirt internally generate the vcpupin element form the cpu set | 10:52 |
| sean-k-mooney | and generge the same qemu commandline | 10:53 |
| artom | Wait a second. | 10:53 |
| *** shilpasd has joined #openstack-nova | 10:53 | |
| artom | My *entire* code triggers if the instance has a NUMA topology | 10:53 |
| sean-k-mooney | yes | 10:53 |
| artom | So the insance with no NUMA topology on a host with vcpu_pin_set case will have to...? | 10:53 |
| sean-k-mooney | you will need to add support for it | 10:54 |
| sean-k-mooney | well you or stepen | 10:54 |
| *** mkrai has quit IRC | 10:54 | |
| sean-k-mooney | technically this is a long standing issue | 10:54 |
| *** mkrai has joined #openstack-nova | 10:54 | |
| sean-k-mooney | we always should have been updating the xml | 10:54 |
| sean-k-mooney | it will be fixed via a hard reboot however | 10:54 |
| sean-k-mooney | libvirt will reject the migration if the cores do not exist on the dest | 10:55 |
| artom | Right, true | 10:55 |
| *** ociuhandu has joined #openstack-nova | 10:55 | |
| artom | I'm surprised more people haven't been hitting that | 10:55 |
| *** ivve has joined #openstack-nova | 10:55 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Follow-up: Add delete_on_termination to volume-attach API https://review.opendev.org/680655 | 10:55 |
| sean-k-mooney | well the havent be casue we tell peole ot use the same cpu_set on all host in teh same aggreagte/az | 10:56 |
| stephenfin | artom: You wouldn't see an issue if you had the same hardware and the same vcpu_pin_set on all hosts | 10:56 |
| stephenfin | yeah | 10:56 |
| aspiers | sean-k-mooney: I've implemented your suggestions from yesterday in https://review.opendev.org/#/c/680065/ and https://review.opendev.org/#/c/644565/ | 10:56 |
| sean-k-mooney | and as i said since its fixed on a hard reboot sicne we dont claim anything and just regenreate it form the config | 10:56 |
| sean-k-mooney | aspiers: did it fix the import loop? | 10:57 |
| aspiers | sean-k-mooney: yes | 10:57 |
| sean-k-mooney | :) | 10:57 |
| stephenfin | sean-k-mooney: Has https://blueprints.launchpad.net/nova/+spec/image-metadata-prefiltering been punted to U? | 10:57 |
| aspiers | sean-k-mooney: it worked very nicely as you should see from the comments I just posted | 10:57 |
| stephenfin | If so, can I kick it out of the runway? | 10:57 |
| sean-k-mooney | stephenfin: so i was suggesting since your code and aspiers keeps breaking it | 10:57 |
| sean-k-mooney | and no one is revieing it | 10:57 |
| stephenfin | I was, in fairness | 10:58 |
| sean-k-mooney | that we could kick it out to get your stuff landed instead | 10:58 |
| aspiers | sean-k-mooney: I'm hoping SEV is now done | 10:58 |
| aspiers | all feedback should be now addressed, even stephenfin's request for extra API checks | 10:58 |
| sean-k-mooney | stephenfin: well yes but you wer also reviing the sev stuff which keeps breaking it | 10:58 |
| stephenfin | True | 10:59 |
| sean-k-mooney | i had hoped that it would merge before either the cpu or sev stuff starte merging to avoid that | 10:59 |
| sean-k-mooney | so im ok with punting ot U | 10:59 |
| sean-k-mooney | we did it internally | 10:59 |
| *** mkrai has quit IRC | 10:59 | |
| sean-k-mooney | or i can rebase and we can merge it quickly | 10:59 |
| stephenfin | I was going to suggest rebasing on top of cpu-resources and I'd review | 11:00 |
| stephenfin | since it's small | 11:00 |
| stephenfin | but just on master could be fine too | 11:00 |
| sean-k-mooney | erric alredy -2'd it since i said he could | 11:00 |
| stephenfin | the merge conflicts should be slight | 11:00 |
| sean-k-mooney | so ill kick it out of the run way | 11:00 |
| sean-k-mooney | stephenfin: they are | 11:00 |
| aspiers | stephenfin: the list of SEV patches got slightly bigger but don't be fooled, it's just 2 trivial extra refactorings plus the API checks you requested | 11:01 |
| *** spatel has joined #openstack-nova | 11:01 | |
| * aspiers just realised he forgot to add the extra reviews to the runway etherpad :-( | 11:01 | |
| * aspiers updates the etherpad | 11:01 | |
| sean-k-mooney | stephenfin: focus on the sev series and your own | 11:02 |
| aspiers | sean-k-mooney: hopefully there will be no more major changes to SEV to cause further merge conflicts | 11:04 |
| sean-k-mooney | aspiers: sev conflict with the cpu series i think | 11:05 |
| sean-k-mooney | but we should finish the sev seires as soon as we can i think | 11:05 |
| sean-k-mooney | then push to land the cpu series | 11:05 |
| aspiers | +1 for that ;-) | 11:05 |
| *** luksky has joined #openstack-nova | 11:05 | |
| *** spatel has quit IRC | 11:05 | |
| aspiers | It's unfortunate that there was a Gerrit issue causing the current Zuul backlog | 11:06 |
| aspiers | But hopefully it should catch up over the weekend | 11:06 |
| bauzas | aspiers: gtk, then we should avoid asking for respins in case of nits | 11:14 |
| aspiers | bauzas: yes, currently some jobs are in the check queue for >20 hours | 11:14 |
| gibi | aspiers: I think those jobs stuck, as other, newly proposed patches went through | 11:16 |
| aspiers | gibi: hrmm :-( I'll ask on #openstack-infra | 11:17 |
| aspiers | gibi: Which ones do you see going through which are newer? I don't see them | 11:19 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add user_id and project_id colume to Migration https://review.opendev.org/673990 | 11:19 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add operator user_id/project_id to the migrations https://review.opendev.org/679413 | 11:19 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Filter migrations by user_id/project_id https://review.opendev.org/674243 | 11:19 |
| gibi | aspiers: bauzas approved this couple of hours ago https://review.opendev.org/#/c/656422/ | 11:19 |
| *** brinzhang has quit IRC | 11:19 | |
| gibi | aspiers: and it went through the gate since | 11:20 |
| * bauzas finds it fun to review things around some corner in some crowdy starbucks | 11:20 | |
| bauzas | but the flakey network makes me crazy | 11:20 |
| bauzas | dudes, move out my lawn ! | 11:21 |
| bauzas | (and don't take all bandwith for crazy netflix thingies) | 11:21 |
| aspiers | gibi: but the most recent patch set was uploaded yesterday afternoon | 11:21 |
| aspiers | gibi: so I don't think it jumped the queue | 11:21 |
| gibi | aspiers: true, so it can be that the check queue is owercrowded but the gate queue isny | 11:22 |
| bauzas | aspiers: gibi: I haven't paid attention to the status recently | 11:22 |
| gibi | isnt | 11:22 |
| aspiers | gibi: yes exactly that is the case | 11:22 |
| bauzas | aspiers: you know the difference between the gate and check pipelines, right? | 11:22 |
| aspiers | I don't know why they are different resource pools | 11:22 |
| aspiers | bauzas: yes of course | 11:22 |
| gibi | could be that patches sitting in the check queue consuming resouce but now moving forward? | 11:22 |
| * bauzas looks at the status page then | 11:23 | |
| bauzas | oh, freaky | 11:23 |
| bauzas | http://zuul.openstack.org/status | 11:23 |
| aspiers | bauzas: yes this is what we are talking about :) | 11:23 |
| bauzas | hard to see with a flakey connection :) | 11:24 |
| aspiers | The gate looks fine, but not check | 11:25 |
| sean-k-mooney | gibi: infra were seeing quota issue causing nodepool failures | 11:25 |
| sean-k-mooney | as a result the queue built up a bit | 11:25 |
| gibi | sean-k-mooney: thanks, so we need to be patient | 11:25 |
| sean-k-mooney | i think they were fixed yesterday evening | 11:25 |
| sean-k-mooney | yes | 11:25 |
| sean-k-mooney | it will just take a while | 11:25 |
| aspiers | yup | 11:26 |
| sean-k-mooney | although if it does not wew might need a zuul restart to clear the queue | 11:26 |
| *** cdent has quit IRC | 11:32 | |
| *** nicolasbock has joined #openstack-nova | 11:38 | |
| *** cdent has joined #openstack-nova | 11:43 | |
| *** johnsom has quit IRC | 11:53 | |
| *** johnsom has joined #openstack-nova | 11:55 | |
| *** bnemec has quit IRC | 11:55 | |
| *** bnemec has joined #openstack-nova | 11:56 | |
| *** etp has quit IRC | 11:57 | |
| openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Nova object changes for forbidden aggregates request filter https://review.opendev.org/671072 | 12:02 |
| openstackgerrit | Shilpa Devharakar proposed openstack/nova master: DB API changes to get non-matching aggregates from metadata https://review.opendev.org/671074 | 12:02 |
| openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Add a new request filter to isolate aggregates https://review.opendev.org/671075 | 12:02 |
| openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Docs for isolated aggregates request filter https://review.opendev.org/667952 | 12:02 |
| *** rcernin|brb has joined #openstack-nova | 12:03 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add min service level check for migrate with bandwidth https://review.opendev.org/680394 | 12:04 |
| openstackgerrit | Dongcan Ye proposed openstack/nova master: Get pci_devices from _list_devices https://review.opendev.org/680674 | 12:06 |
| *** tbachman has joined #openstack-nova | 12:08 | |
| *** ratailor has quit IRC | 12:09 | |
| dtantsur | efried, dustinc, update: the grenade failure in our CI probably is not caused by the instance_id problem.. | 12:09 |
| *** maciejjozefczyk_ has joined #openstack-nova | 12:12 | |
| *** ociuhandu has quit IRC | 12:14 | |
| *** ociuhandu has joined #openstack-nova | 12:15 | |
| *** tbachman has quit IRC | 12:15 | |
| *** maciejjozefczyk has quit IRC | 12:16 | |
| *** jawad_axd has joined #openstack-nova | 12:17 | |
| *** larainema has quit IRC | 12:17 | |
| *** tbachman has joined #openstack-nova | 12:17 | |
| gibi | bauzas: if you still have energy then this is the next patch in the bw series. I fixed up mriedem's comments https://review.opendev.org/#/c/680394 | 12:19 |
| *** ociuhandu has quit IRC | 12:19 | |
| *** ociuhandu has joined #openstack-nova | 12:21 | |
| *** spsurya has quit IRC | 12:26 | |
| *** mdbooth has joined #openstack-nova | 12:29 | |
| *** rcernin|brb has quit IRC | 12:30 | |
| *** dave-mccowan has joined #openstack-nova | 12:30 | |
| *** dave-mccowan has quit IRC | 12:31 | |
| *** yedongcan has quit IRC | 12:32 | |
| *** dave-mccowan has joined #openstack-nova | 12:33 | |
| *** dave-mccowan has quit IRC | 12:38 | |
| *** markvoelker has joined #openstack-nova | 12:45 | |
| *** mriedem has joined #openstack-nova | 12:46 | |
| bauzas | gibi: heh, sure, I'm now waiting for my train | 12:49 |
| gibi | bauzas: meanwhile I found that I left some debug print statement in that patch :/ so I will need to go back and remove them. | 12:49 |
| bauzas | gibi: oh, which one ? did I miss it too ? | 12:50 |
| bauzas | if so, Starbucks, I hate you | 12:50 |
| gibi | bauzas: in https://review.opendev.org/680394 | 12:50 |
| gibi | bauzas: not in the patch you approved | 12:50 |
| * bauzas whispers | 12:50 | |
| mriedem | yonglihe: will you have time to address these comments in https://review.opendev.org/#/c/670790/ today or should someone else do those? | 12:58 |
| *** BjoernT has joined #openstack-nova | 13:05 | |
| mriedem | artom: let me know if you want to talk about any of my comments before you push changes | 13:12 |
| mdbooth | mriedem dansmith: Looks like I've got another customer report affected by https://review.opendev.org/#/c/677736/ | 13:14 |
| mdbooth | Any chance we could merge that? | 13:14 |
| artom | mriedem, ack - you saw I noticed a thing we all missed until now? The -1 on the massive patch | 13:15 |
| *** nweinber_ has joined #openstack-nova | 13:15 | |
| *** jawad_axd has quit IRC | 13:15 | |
| artom | Not that hard a fix, but kind of a bitch to only spot it now | 13:15 |
| mriedem | i haven't looked back on responses yet | 13:15 |
| artom | mriedem, https://review.opendev.org/#/c/635229/58/nova/virt/libvirt/driver.py@7345 for when you're ready to look | 13:16 |
| *** jawad_axd has joined #openstack-nova | 13:16 | |
| mriedem | yonglihe: also, as i said on that novaclient change, i'd like to see what it actually looks like in the output | 13:16 |
| mriedem | if it's just a bunch of json blobs dumped to the console that's not very useful | 13:16 |
| mriedem | artom: is that because i was asking about guest_cpu_tune and guest_numa_tune being exclusive? | 13:18 |
| openstackgerrit | Dmitry Tantsur proposed openstack/nova master: Fix incorrect invocation of openstacksdk's baremetal.nodes() https://review.opendev.org/680684 | 13:18 |
| dtantsur | efried, dustinc, mriedem, another issue ^^^ | 13:18 |
| dtantsur | this one probably affects ironic CI as well | 13:18 |
| artom | mriedem, yeah, that's what lead me down that path | 13:18 |
| mriedem | \o/ | 13:18 |
| *** eharney has joined #openstack-nova | 13:18 | |
| artom | mriedem, indeed :) | 13:18 |
| mriedem | mdbooth: i'll follow dansmith's lead on that one | 13:18 |
| *** jawad_ax_ has joined #openstack-nova | 13:19 | |
| *** nweinber_ has quit IRC | 13:19 | |
| mriedem | dustinc: commented | 13:20 |
| *** nweinber has joined #openstack-nova | 13:20 | |
| *** jawad_a__ has joined #openstack-nova | 13:21 | |
| *** jawad_axd has quit IRC | 13:21 | |
| yonglihe | mriedem: do you mean put json to release notes? It is definitely json blob. | 13:23 |
| yonglihe | https://review.opendev.org/#/c/621476/62/doc/api_samples/os-server-topology/v2.78/servers-topology-resp.json | 13:23 |
| *** jawad_ax_ has quit IRC | 13:23 | |
| dtantsur | mriedem: I can try to reproduce my thoughts process in a bug, although it was a bit of guessing initially :) do you still use launchpad? | 13:23 |
| yonglihe | mriedem: BTW, I'm afraid i don't have time to do it today, sorry. | 13:25 |
| *** jawad_a__ has quit IRC | 13:25 | |
| mdbooth | mriedem: Ack, thanks. | 13:26 |
| mriedem | yonglihe: no i meant i'd like to see what the CLI output looks like | 13:26 |
| mriedem | dtantsur: yeah | 13:26 |
| dtantsur | so old school :) | 13:26 |
| mriedem | best school | 13:26 |
| mriedem | dtantsur: btw how is that we're not seeing issues in ironic CI? | 13:27 |
| mriedem | or is it just because the job is non-voting in nova? | 13:27 |
| dtantsur | mriedem: it has only one node at all. so picking a random node works. | 13:27 |
| dtantsur | our multinode grenade uses several nodes, so it got hard broken on this. | 13:27 |
| dtantsur | I'm writing a bug with explanation. | 13:27 |
| mriedem | ok well just a link to a failed instance of one of those jobs with logs would be good in a bug report | 13:27 |
| mriedem | thanks | 13:27 |
| yonglihe | mriedem: then i need to running it after this weekend, sorry. | 13:28 |
| *** macz has joined #openstack-nova | 13:29 | |
| *** macz has quit IRC | 13:29 | |
| *** macz has joined #openstack-nova | 13:29 | |
| mriedem | ok, maybe sean-k-mooney or stephenfin or artom have a system with servers that have numa topology and they can pull this down and see what it looks like | 13:30 |
| dtantsur | mriedem: https://bugs.launchpad.net/nova/+bug/1843050 | 13:30 |
| openstack | Launchpad bug 1843050 in OpenStack Compute (nova) "Regression in the ironic driver: wait_for_active checks a random node" [Undecided,New] | 13:30 |
| yonglihe | mriedem: that's so nice. | 13:31 |
| stephenfin | mriedem: This is the patch to expose NUMA topology API change in novaclient? | 13:31 |
| openstackgerrit | Dmitry Tantsur proposed openstack/nova master: Fix incorrect invocation of openstacksdk's baremetal.nodes() https://review.opendev.org/680684 | 13:31 |
| mriedem | stephenfin: yeah | 13:31 |
| artom | mriedem, they could probably pile on to sean-k-mooney's new nfv job | 13:31 |
| mriedem | artom: it's novaclient | 13:31 |
| mriedem | i just want someone to show me a paste | 13:32 |
| stephenfin | Sure, I can do that, though to be clear it doesn't need a NUMA-based system | 13:32 |
| stephenfin | well, a two node+ system | 13:32 |
| artom | mriedem, ah, my machine's under my desk at the office, and not responding to ping. Since daughter started school I haven't had a chance to get over there. | 13:32 |
| mriedem | stephenfin: i could probably also just hack it up with the fake data fixtures in novaclient using the api-ref sample from the docs and trigger a failure in shell tests to see the output | 13:34 |
| mriedem | my goal is to get that change merged so we can also get the 2.79 novaclient change merged, which is much simpler, but is held up on the 2.78 change | 13:34 |
| mriedem | i can ffffart around with that today | 13:34 |
| stephenfin | Whatever suits. I'll take it as an opportunity to redeploy my DevStack node so I'll have something but it'll be a few hours? | 13:35 |
| *** nweinber has quit IRC | 13:35 | |
| *** nweinber has joined #openstack-nova | 13:35 | |
| mriedem | stephenfin: if you want, but it's friday and you've probably got other plates spinning | 13:35 |
| mriedem | dtantsur: do we have an ironic change that depends-on the nova fix to see the multinode grenade job pass with the fix? knowing that even if we do, it could be several hours before it even runs | 13:36 |
| stephenfin | many plates. much spinning :) | 13:36 |
| dtantsur | mriedem: https://review.opendev.org/#/c/680643/ (depends on both fixes) | 13:36 |
| mriedem | oh i didn't know there were 2 | 13:37 |
| mriedem | ok will get those merged today | 13:37 |
| dtantsur | thx! | 13:37 |
| *** jawad_axd has joined #openstack-nova | 13:38 | |
| mriedem | dtantsur: is this the job we care about? ironic-grenade-dsvm-multinode-multitenant | 13:41 |
| dtantsur | lemme double-check | 13:41 |
| mriedem | that's the only multinode grenade job i see in zuul on that change | 13:42 |
| dtantsur | mriedem: oh, it seems to be a normal (non-grenade) multinode job: ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode | 13:42 |
| *** jawad_axd has quit IRC | 13:42 | |
| mriedem | ack | 13:43 |
| dtantsur | although I did see something similar on one or two grenade runs, but they don't seem consistently broken | 13:43 |
| mriedem | maybe randomly picking the correct node | 13:43 |
| dtantsur | yep | 13:43 |
| dansmith | mriedem: mdbooth decided to talk about some details about that patch downstream, so let me fill you in | 13:43 |
| dansmith | mriedem: there are (suddenly?) two customer cases he thinks are caused by this issue, yet we apparently have no confirmation from either see a problem solved by applying this | 13:44 |
| dansmith | mriedem: one clearly isn't fixed by it, and we don't have logs to show why | 13:44 |
| alex_xu | stephenfin: thanks, actually I already go through that patch, I didn't found problem, I'm just waiting for huaqiang to do some manual test, then I will go through all the left patches and +2 | 13:45 |
| dansmith | mriedem: so while the rationale for making the change seems fairly legit to me, I don't know if (a) that one place was never wrapped for some reason or (b) whether changing that is going to cause other thread interaction or timing-related problems | 13:45 |
| dansmith | mriedem: so I'm not sure what to say... we could just merge it and watch carefully and hope that any negative impact is not in two years when people actually roll to it, as it seems likely to be just an oversight | 13:46 |
| yonglihe | mriedem: sorry this is a barrier, i did not bring back my laptop today, and stephenfin, thanks. | 13:46 |
| dansmith | but it definitely makes me nervous not having any "yes this fixes my problem" confirmation for code that has been around for ten years | 13:47 |
| *** macz has quit IRC | 13:47 | |
| mriedem | dansmith: an oversight that we don't know if it has caused problems until now, though that code hasn't changed in a long time | 13:47 |
| stephenfin | alex_xu: Huzzah. Good to hear (y) | 13:47 |
| mriedem | dansmith: any chance of some weird eventlet patches in these customer environments? | 13:47 |
| mriedem | so it would be rhosp specific? | 13:47 |
| *** ociuhandu has quit IRC | 13:47 | |
| mriedem | or python-libvirt patches, something external to nova? | 13:48 |
| dansmith | mriedem: I doubt it, but if they did and it was related, then they'd have been fixed by this (if that's what you mean) | 13:48 |
| mriedem | also, if we land this on master, will backports be proposed upstream? | 13:48 |
| dansmith | mriedem: they're probably on newer *libvirt*, but I dunno how often python-libvirt changes | 13:48 |
| mriedem | i think you have to rebuild python-libvirt from whatever version of libvirt binary you're using | 13:49 |
| mriedem | otherwise things get weird | 13:49 |
| dansmith | mriedem: I dunno, ask mdbooth. I would expect he would want to | 13:49 |
| mriedem | i remember we had issues in the past with running libvirt x in the gate jobs but python-libvirt was built against y and reporting y to nova, so we'd try to use features that weren't actually there | 13:49 |
| dansmith | mriedem: right but that's just because the bindings are auto-generated I think | 13:49 |
| mriedem | ok, so i'm less comfortable with backporting a thing that we're not sure fixes the issue | 13:50 |
| mriedem | it's one thing to land on master and let it simmer until people get to it, | 13:50 |
| mriedem | than to backport it and force it on people with otherwise stable environments | 13:50 |
| dansmith | although we are right before FF, so not as much runway | 13:50 |
| mriedem | rh probably just cares that it's merged on master and can backport internally is less of an issue yeah? | 13:51 |
| mriedem | merge conflict risk on stable branches and such i know | 13:51 |
| dansmith | maybe we could get efried to comment on the risk, and maybe cdent on the actual eventlet bits? | 13:51 |
| dansmith | I'm not sure what context cdent has on eventlet, but it seems like sometimes he has a lot | 13:51 |
| cdent | hello what? | 13:51 |
| cdent | which thing? | 13:51 |
| mriedem | efried isn't a stable branch maintainer though, or you just mean risk for FF? | 13:51 |
| dansmith | mriedem: I mean for merging to master right now or in general yeah | 13:52 |
| mriedem | cdent: https://review.opendev.org/#/c/677736/ | 13:52 |
| * cdent reads | 13:52 | |
| mriedem | cdent: tl;dr old ass code, | 13:52 |
| mriedem | suddenly 2 rhosp customers are seeing weird issues which mdbooth thinks are related to libvirt + python + eventlet, | 13:52 |
| dansmith | (I wish I was working on things that *have* to merge by FF right now) | 13:52 |
| mriedem | but there isn't a clear indication that patch fixes those problems | 13:52 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | 13:52 |
| aspiers | stephenfin: Your doc feedback should be addressed now ^^^ | 13:53 |
| dansmith | mriedem: I see no indication that it fixes those problems.. I think the only indication is that it "seems wrong" and "these tests pass" | 13:53 |
| mriedem | sure, so ruling it out i guess | 13:53 |
| *** ratailor has joined #openstack-nova | 13:53 | |
| mriedem | i also need to investigate these fixes that are blocking the ironic gate atm | 13:53 |
| mriedem | since i'm guessing ironic would like to merge code before FF | 13:54 |
| aspiers | stephenfin: https://review.opendev.org/#/c/666616/47..48/nova/conf/libvirt.py | 13:54 |
| donnyd | sean-k-mooney: Is the NUMA job still throwing errors? | 13:54 |
| *** mlavalle has joined #openstack-nova | 13:54 | |
| aspiers | stephenfin: BTW I've checked that the rst all renders OK | 13:54 |
| sean-k-mooney | donnyd: i have not re run it | 13:54 |
| sean-k-mooney | with the queue thats in the gate i was going to wiat until the weekend | 13:55 |
| sean-k-mooney | donnyd: i can kick it off now if you like and see | 13:55 |
| donnyd | sure | 13:55 |
| * cdent reads the code | 13:55 | |
| *** udesale has quit IRC | 13:56 | |
| *** udesale has joined #openstack-nova | 13:57 | |
| sean-k-mooney | donnyd: there is a build stitting in the experimental pipeline now | 13:58 |
| sean-k-mooney | donnyd: ill let you know how it goes | 13:58 |
| donnyd | kk | 13:58 |
| donnyd | where can i check its status? | 13:58 |
| sean-k-mooney | http://zuul.openstack.org/status | 13:58 |
| sean-k-mooney | filter by 679656 | 13:58 |
| sean-k-mooney | currently all jobs are queued | 13:59 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Reject live migration and suspend on SEV guests https://review.opendev.org/680158 | 13:59 |
| *** Luzi has quit IRC | 13:59 | |
| luyao | dansmith, efried: I updated the patch for 'resource' object, many thanks for your detailed comments. It helps me a lot. https://review.opendev.org/#/c/678448/17 | 14:00 |
| cdent | mriedem, dansmith : I'd be inclined to think that the non-proxy original code was wrong and the fix is good and we've tended to get lucky. However, without any clear proof that presenting problem(s) is fixed by the change...hard to say. I would think merge to master and watch wouldn't be a bad thing. | 14:00 |
| sean-k-mooney | i proably should have kicked off one of the later patces to run less tests | 14:00 |
| sean-k-mooney | but i did want that to report back in anycase | 14:00 |
| aspiers | mriedem: thanks for feedback; addressed - https://review.opendev.org/#/c/680158/4..5/nova/compute/api.py | 14:00 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | 14:01 |
| alex_xu | anyone can tell me...can we boot instance without allocation | 14:01 |
| sean-k-mooney | am we should be able to start them. but create new instances no | 14:02 |
| sean-k-mooney | e.g. you deleted the allocation in placmenet for an instance nova should still be able to start/stop the vm | 14:02 |
| dansmith | cdent: specifically one customer clearly *still* has the problem after applying this patch | 14:03 |
| cdent | ah | 14:03 |
| cdent | well | 14:03 |
| dansmith | cdent: but I agree that it *seems* like the change we'd want to make, I'd just like confirmation that it does something positive you know :) | 14:03 |
| alex_xu | sean-k-mooney: probably we can before https://github.com/openstack/nova/blob/master/nova/tests/functional/test_nova_manage.py#L405 | 14:04 |
| cdent | is there concrete evidence it is stuck in eventlet, or just that it is stuck? | 14:04 |
| *** lbragstad has quit IRC | 14:04 | |
| dansmith | cdent: there's some guru meditation that makes it look like it, but we don't get to control when that happens and thus it's pretty likely to always be blocking on some C call on a busy system I think | 14:05 |
| artom | mriedem, FWIW, I've done an inital read-through of all your comments, and they all make sense, so I should have answers/fixes later today | 14:05 |
| alex_xu | sean-k-mooney: that functional test ensure we can boot instance without allocation, but the goal is testing the heal_allocations. | 14:06 |
| *** ociuhandu has joined #openstack-nova | 14:06 | |
| cdent | dansmith: terrific! Does it happen often? Therapuetic process kills to the rescue? | 14:06 |
| *** lbragstad has joined #openstack-nova | 14:07 | |
| dansmith | yeah, just nuke the compute service I think | 14:07 |
| dansmith | and sometimes they can't reproduce it, like when we got on a call to watch :) | 14:07 |
| dansmith | what's strange to me is that this code has been in place for ages and suddenly there are two customers hitting it simultaneously, but not fixed by this patch | 14:07 |
| *** markvoelker has quit IRC | 14:07 | |
| dansmith | just makes me highly skeptical :) | 14:07 |
| cdent | yeah. eventlet, libc, libvirt changes all possible contributors | 14:08 |
| melwitt | does anyone know anything about the nova.task_log table? we have customers whose nova.task_log table grows indefinitely and I found there's no methods for ever deleting task_log entries. does anyone know what people usually do about this? manually truncate the task_log table? what are these records for? is arbitrary truncation safe? | 14:10 |
| mriedem | dtantsur: dustinc: efried: what do you think about these comments? https://review.opendev.org/#/c/680684/2 | 14:10 |
| mriedem | the fix is fine, but wondering about the 'this shouldn't happen' case | 14:10 |
| dtantsur | will check in a few minutes | 14:13 |
| dtantsur | yeah, I was also unsure whether to log or to raise | 14:13 |
| mriedem | alex_xu: with the caching scheduler yes it was possible to create an instance with no allocations in placement, | 14:15 |
| mriedem | because the caching scheduler didn't use placement or GET /allocatoin_candidates | 14:15 |
| mriedem | the caching scheduler was removed in stein, | 14:15 |
| mriedem | but you can still load up out of tree scheduler drivers that may not use placement | 14:15 |
| mriedem | alex_xu: coincidentally, see https://review.opendev.org/#/c/680521/ | 14:15 |
| cdent | mriedem: that mention of logging reminds me of something I wanted to ask about: In python 2, when using oslo_log and LOG.error it includes whatever the most recent exception was, even if it is out of scope. In pytyhon3 sys.exc_info() "expires" sooner, more predictably. This means that error messages in the scheduler report client often include bogus tails in the message | 14:16 |
| cdent | do we care? or can we just ignore python2 soon enough? | 14:16 |
| mriedem | cdent: i think i know what you're talking about, and melwitt/dhellmann had made some fixes to oslo.log to squash that weirdness, but maybe that was only for warning level logs, | 14:18 |
| mriedem | e.g. you'd see a stacktrace for a ComputeNodeNotFound_Remote warning even though you weren't using exc_info=True on LOG.warning, | 14:18 |
| cdent | it's warning and worse where oslo_log does the inspection exc_info | 14:18 |
| mriedem | something like that | 14:18 |
| melwitt | oh yeah, good memory | 14:18 |
| * melwitt gets a link | 14:19 | |
| melwitt | https://review.opendev.org/528036 | 14:19 |
| mriedem | melwitt: i'm pretty sure task log is related to https://docs.openstack.org/api-ref/compute/#server-usage-audit-log-os-instance-usage-audit-log which is related to ceilometer which no one uses | 14:19 |
| mriedem | i'd like to burn out all of that code | 14:19 |
| melwitt | thanks. I was surprised in all my time working on nova, I've never heard of the task_log table before now | 14:20 |
| melwitt | so I'd support burning the code that no one uses if indeed no one uses it | 14:21 |
| efried | dtantsur: I looked briefly at the ironic CI job to see why it didn't hit that bug; I think it's just because those methods are used in the course of specific operations that tempest doesn't cover. We should add that, for sure. I'm not completely sure how to get started with that though. | 14:21 |
| dtantsur | yeah | 14:21 |
| dtantsur | the 2nd bug is hit when you have >1 nodes | 14:21 |
| mriedem | rackspace probably had internal scripts or something to purge task log | 14:21 |
| mriedem | melwitt: well clearly your customer is using it | 14:22 |
| cdent | thanks melwitt, I think that corresponds to something I was looking at when poking at the problem, but doesn't address the issue with the exception name being included in the log even though it was "elsewhere". I'm inclined to ignore it if nobody is complaining, since it gets better with python3. I only noticed becasue somebody asked why there was a "NotImplemted" at the end of rp generation conflict message | 14:22 |
| melwitt | mriedem: oh, you think table grow means they are using the audit APIs? I had assumed task_log entries get added regardless (like on any instance action) | 14:22 |
| openstackgerrit | Dmitry Tantsur proposed openstack/nova master: Fix incorrect invocation of openstacksdk's baremetal.nodes() https://review.opendev.org/680684 | 14:22 |
| dtantsur | mriedem: better ^^? | 14:22 |
| *** ociuhandu has quit IRC | 14:23 | |
| alex_xu | mriedem: I'm talking to luyao, we should stop the instance booting if there is no allocation, since vgpu and vpmem should depend on allocation | 14:23 |
| melwitt | I need to check | 14:23 |
| mriedem | melwitt: it's a periodic task in the compute service which you have to enable, and it's disabled by default | 14:23 |
| mriedem | so if they have task log table entries they are configured to run that task | 14:23 |
| melwitt | oh, dang | 14:23 |
| mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.instance_usage_audit | 14:23 |
| mriedem | and they are probably using ceilometer to hit that compute api | 14:23 |
| mriedem | if they aren't, then they should disable the task and purge the db | 14:24 |
| melwitt | thanks. I shouldn't have assumed | 14:24 |
| mriedem | please see topic new contributor :P | 14:24 |
| melwitt | :( | 14:24 |
| mriedem | alex_xu: yeah that's probably fine, there is no real contract on support for out of tree scheduler drivers | 14:25 |
| *** macz has joined #openstack-nova | 14:25 | |
| mriedem | alex_xu: see https://opendev.org/openstack/nova/src/branch/master/nova/scheduler/driver.py#L35 | 14:25 |
| *** brault has quit IRC | 14:27 | |
| efried | dansmith: risk on https://review.opendev.org/#/c/677736/ I really have no idea I'm afraid. If cdent thinks it's okay to merge-and-watch, that's good enough for me. Remember FF is FF so we have a little bit of time before the release to flush out issues; so I don't think current proximity to FF is a reason to block it. | 14:27 |
| dansmith | efried: ack | 14:28 |
| *** cdent has quit IRC | 14:28 | |
| mriedem | dansmith: before artom moves forward with this rpc/object change in the numa lm series https://review.opendev.org/#/c/634827/57//COMMIT_MSG@12 might want to see if you are on board | 14:33 |
| mriedem | i'm basically asking that we're consistent in the flags saying both source and dest are ready for action | 14:33 |
| alex_xu | mriedem: i see now, thanks | 14:33 |
| dansmith | mriedem: ack looking | 14:34 |
| artom | mriedem, actually yeah, thanks. I think it should be fine, but dansmith's lack of veto would boost confidence | 14:34 |
| dansmith | artom: mriedem you're thinking set that field on the dest using the same can-do-numa thing in rpcapi? because without that, the only way to know is to pass a new param and have it *not* show up at the other end | 14:35 |
| dansmith | we resolved that first, and then artom added the can-do check to rpcapi for the source, who has nobody to call | 14:35 |
| dansmith | so if you're saying make the source and dest both call that, and set their corresponding field, yeah I guess that's okay | 14:35 |
| artom | dansmith, not quite - I'm saying the dest doesn't need to actually send RPC 5.3, so check_can_live_migrate_source doesn't need that numa_live_migraiton flag, the dest can just shove it in migrate_data | 14:37 |
| aspiers | Just asked in #openstack-qa but figured worth asking here too: Is it considered good/bad practice for tempest tests to check the message string of nova exceptions returned by assertRaises? | 14:38 |
| dansmith | artom: but the dest needs to support the 5.3 inbound call, but you're saying actually report the conductor side? meaning the did-i-get-the-migration-from-conductor flag? | 14:38 |
| *** jaosorior has quit IRC | 14:38 | |
| aspiers | I'm thinking that if we wanted to reword an exception's message string, this could break tempest, and then you'd need a circular Depends-On across repos to fix it | 14:38 |
| artom | dansmith, right, the dest needs to receive 5.3 - specifically migration and limits form the conductor | 14:39 |
| artom | So if gets those, it can set the flag inside migrate_data | 14:39 |
| dansmith | artom: and the new drop call | 14:39 |
| dansmith | from source | 14:39 |
| artom | dansmith, yeah | 14:39 |
| artom | No, not from source | 14:39 |
| artom | The source still needs to send 5.3 | 14:39 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix incorrect invocation of openstacksdk's baremetal.nodes() https://review.opendev.org/680684 | 14:39 |
| mriedem | dtantsur: +2 now ^ | 14:39 |
| mriedem | efried: ^ | 14:39 |
| dansmith | artom: the source calls the new drop-claim against the dest right? | 14:39 |
| artom | dansmith, yeah | 14:40 |
| artom | So if the dest sets the flag in migrate_data, these things could happen on the source: | 14:40 |
| dansmith | artom: that's what I mean by inbound 5.3 | 14:40 |
| *** mtreinish has joined #openstack-nova | 14:40 | |
| dansmith | anyway, yeah I think that's okay | 14:40 |
| mriedem | aspiers: error messages in the api aren't a contract | 14:41 |
| mriedem | only error codes like placement uses, but nova doesn't | 14:41 |
| mriedem | dansmith: yeah basically remove the numa_live_migration flag that goes from dest to source and instead put that as an attribute in migrate_data instead | 14:42 |
| efried | dustinc: I'm merging https://review.opendev.org/#/c/680684/ but please have a look at it when you get in. | 14:42 |
| mriedem | for consistency | 14:42 |
| artom | dansmith, right, dest needs to receive 5.3 | 14:42 |
| artom | But nothing needs to actually send it | 14:42 |
| aspiers | mriedem: OK thanks, so I'll just check for HTTPBadRequest and leave it there | 14:42 |
| efried | mriedem, dtantsur: +A | 14:42 |
| artom | So if it sets the flag in migrate_data, we know it's the new code that can receive 5.3 | 14:42 |
| artom | And then on the source: | 14:42 |
| artom | Either it's old code and that flag gets dropped by the conductor | 14:42 |
| dansmith | mriedem: right I was just confirming how we calculate that because right now the flag making it through the dest's rpcapi to the source is the sentinel | 14:42 |
| artom | Either it's new code but pinned, in which case the can_send check will fail | 14:43 |
| artom | Or it's new code unpinned, and we proceed with the NUMA LM | 14:43 |
| mriedem | dansmith: yup | 14:44 |
| mriedem | i didn't want to start a pattern of putting random feature flags on those compute rpc api methods, like sev=True, vpmems=True, suchandsuch=True | 14:44 |
| *** sapd1_x has joined #openstack-nova | 14:45 | |
| dansmith | mriedem: yeah, but we needed it to figure out if the dest could *send* 5.3, but that's actually not a thing we need to know | 14:45 |
| dansmith | and now we have that can-do method to tell us without actually trying to make the call | 14:46 |
| artom | dansmith, that's the thing, I don't think dest actually needs to send 5.3 | 14:46 |
| artom | The changes are all in objects | 14:46 |
| artom | From its POV | 14:46 |
| dansmith | artom: no it doesn't, I'm agreeing | 14:48 |
| dansmith | artom: I think we just kinda munged the "does it support" with "is it configured to allow sending" ... 5.3 when we first discussed it | 14:48 |
| artom | dansmith, yep | 14:48 |
| dansmith | if we were adding new calls on both sides, they'd both need to be configured for 5.3, but in this case, we can cheat | 14:48 |
| artom | dansmith, ack, looks like we're all on the same page, I'll continue down that route and with the rest of mriedem's feedback after the bug call | 14:50 |
| dansmith | cool | 14:51 |
| *** markvoelker has joined #openstack-nova | 14:51 | |
| mriedem | stephenfin: alex_xu: yonglihe: on that topology cli change, this is wrong, correct? https://review.opendev.org/#/c/670790/7/doc/source/cli/nova.rst@3663 | 14:52 |
| mriedem | compute:server:topology:host:index is admin-only by default and used to expose host fields | 14:53 |
| *** jawad_axd has joined #openstack-nova | 14:53 | |
| *** jawad_axd has quit IRC | 14:53 | |
| mriedem | https://review.opendev.org/#/c/621476/62/nova/api/openstack/compute/server_topology.py@58 | 14:54 |
| mriedem | yeah looks like it | 14:54 |
| *** ociuhandu has joined #openstack-nova | 14:54 | |
| *** markvoelker has quit IRC | 14:55 | |
| donnyd | sean-k-mooney: looks like we are still getting a node failure | 14:56 |
| stephenfin | looking | 14:58 |
| *** ociuhandu has quit IRC | 14:59 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add min service level check for migrate with bandwidth https://review.opendev.org/680394 | 14:59 |
| stephenfin | mriedem: Yup, good catch | 15:01 |
| mriedem | efried: i'm +2 on the other ironic fix https://review.opendev.org/#/c/680542/ but we can wait until the test patch passes https://review.opendev.org/#/c/680643/ - given the state of the gate that might be sometime this weekend though, so maybe you want to +2 and one of us can approve once we have results? | 15:01 |
| sean-k-mooney | donnyd: it might be because its requesting two vms getting one and timeing out getting the second? | 15:01 |
| mriedem | actually that ironic patch already got test nodes, it must be nice to not be nova | 15:02 |
| efried | mriedem: I was thinking along the same lines. But iiuc dtantsur followed up later and said the grenade fix wasn't actually related to this bug. We (still) don't have CI coverage that will hit it. I just proposed https://review.opendev.org/680718 in tripleo which will hopefully hit the job that originally uncovered the problem. | 15:02 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: migrate: Add bw min service level check of source compute https://review.opendev.org/680395 | 15:02 |
| efried | s/grenade fix/grenade failure/ | 15:03 |
| *** Sundar has joined #openstack-nova | 15:03 | |
| *** lpetrut has quit IRC | 15:03 | |
| mriedem | alright then | 15:03 |
| mriedem | moving on | 15:03 |
| sean-k-mooney | donnyd: ill check it at the weekend agian when the ci does not have as high a load | 15:04 |
| efried | mriedem: but given how simple and obviously-right the fix is, and that dtantsur says it is legit, I think we might want to merge it anyway. | 15:04 |
| dtantsur | efried: I'd make your tripleo patch depending on both fixes. I'm not sure which one of them is most breaking. | 15:04 |
| mriedem | efried: i'm good with that | 15:04 |
| mriedem | i definitely trust dtantsur here and these are simple fixes | 15:04 |
| efried | dtantsur: the symptoms would be different | 15:04 |
| *** bnemec is now known as beekneemech | 15:04 | |
| *** luksky has quit IRC | 15:05 | |
| dtantsur | okie | 15:06 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: resize: Add bw min service level check of source compute https://review.opendev.org/680396 | 15:06 |
| *** ociuhandu has joined #openstack-nova | 15:07 | |
| efried | dtantsur, mriedem: +A | 15:08 |
| dtantsur | now let's see if you'll regret trusting me :D | 15:08 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth https://review.opendev.org/655109 | 15:09 |
| efried | It's not clear that the tripleo patch will work as is; I don't understand the job chain that would get us there, or whether nova needs to be added to required_projects *somewhere* (but where??). So it may take a few iterations to figure that out. | 15:09 |
| mriedem | dtantsur: please don't break my achy breaky heart | 15:10 |
| dtantsur | okay | 15:11 |
| mriedem | i'm assuming you get early 90s billy ray cyrus references of course | 15:11 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Func test for migrate server with ports having resource request https://review.opendev.org/655113 | 15:11 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Make _rever_allocation nested allocation aware https://review.opendev.org/676138 | 15:14 |
| *** ratailor has quit IRC | 15:17 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support reverting migration / resize with bandwidth https://review.opendev.org/676140 | 15:18 |
| gibi | mriedem, bauzas: the next 5 patches in the bw series is up-to-date. first patch: https://review.opendev.org/#/c/680394/ last up-to-date patch: https://review.opendev.org/#/c/655113 | 15:18 |
| mriedem | gibi: yup, on my list for today | 15:19 |
| gibi | mriedem: thanks. I still have to work on comments in th second half of the patch series and also need to do a follow up for the already merged https://review.opendev.org/#/c/656422 | 15:20 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Func test for migrate re-schedule with bandwidth https://review.opendev.org/676972 | 15:21 |
| *** cdent has joined #openstack-nova | 15:22 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support migrating SRIOV port with bandwidth https://review.opendev.org/676980 | 15:24 |
| mriedem | stephenfin: is it not possible to deprecate an entire group of config options? | 15:25 |
| mriedem | you have to do them all individually? | 15:25 |
| stephenfin | you can deprecate opts but not groups, afaik | 15:25 |
| stephenfin | so no | 15:25 |
| mriedem | blast | 15:25 |
| stephenfin | soz | 15:25 |
| mriedem | is that hipster millenial for sorry? | 15:26 |
| * mriedem writes it down | 15:26 | |
| mriedem | i had to look up what a "stan" was last night | 15:26 |
| stephenfin | let's say...yes | 15:26 |
| cdent | you'll be cool, eventually, mriedem | 15:26 |
| * stephenfin Googles "stan" | 15:26 | |
| cdent | each day, get better and better | 15:26 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow migrating server with port resource request https://review.opendev.org/671497 | 15:27 |
| mriedem | https://media1.tenor.com/images/fc73ef705412eeaeb838763055918587/tenor.gif?itemid=5653229 | 15:27 |
| mriedem | oooo https://www.youtube.com/watch?v=BGrfhsxxmdE | 15:28 |
| jangutter | The only "stan" I know is https://mc-stan.org | 15:28 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Do not query allocations twice in finish_revert_resize https://review.opendev.org/678827 | 15:30 |
| *** avolkov has quit IRC | 15:33 | |
| *** maciejjozefczyk has joined #openstack-nova | 15:34 | |
| *** bbobrov has quit IRC | 15:36 | |
| *** trident has quit IRC | 15:37 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow resizing server with port resource request https://review.opendev.org/679019 | 15:37 |
| *** johnthetubaguy has quit IRC | 15:37 | |
| *** maciejjozefczyk_ has quit IRC | 15:37 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate the XenAPIDriver https://review.opendev.org/680732 | 15:41 |
| mriedem | efried: dansmith: gibi: bauzas: alex_xu: melwitt: stephenfin: fyi ^ | 15:41 |
| dansmith | hoo boy | 15:41 |
| *** bbobrov has joined #openstack-nova | 15:41 | |
| mriedem | i'll post to the ML | 15:42 |
| stephenfin | Ima +2 that so hard. | 15:42 |
| stephenfin | after the ML post, of course | 15:42 |
| * aspiers grabs some popcorn | 15:42 | |
| *** dtantsur is now known as dtantsur|afk | 15:44 | |
| *** trident has joined #openstack-nova | 15:46 | |
| mriedem | http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009180.html | 15:46 |
| mriedem | stephenfin: since you smelled it, maybe you should deal up an etherpad for rc todo stuff like the prelude reno | 15:47 |
| mriedem | i'm auto delegating efried delegating that to me | 15:47 |
| stephenfin | mriedem: Sure https://etherpad.openstack.org/p/nova-train-prelude | 15:48 |
| efried | thanks | 15:51 |
| efried | is a prelude related to cycle highlights? | 15:51 |
| mriedem | sort of, | 15:51 |
| efried | cause I could use a delegate for that as well | 15:51 |
| mriedem | the prelude generally feeds into highlights | 15:51 |
| mriedem | the highlights have to be wordsmithed a bit for marketing speak | 15:52 |
| efried | it's cruel that first draft of highlights is due the day of FF. | 15:52 |
| *** tesseract has quit IRC | 15:52 | |
| mriedem | "the compute team leveraged the synergy of the cloud to...deprecate the xenapi driver" | 15:52 |
| mriedem | efried: yup | 15:52 |
| mriedem | i've complained about that before | 15:52 |
| mriedem | smcginnis is good at suggestions on the highlights, | 15:53 |
| mriedem | he's nearly executive level management at this point | 15:53 |
| stephenfin | efried: I'm hoping to put together one of those internally to figure out what we need to test/document for Train/OSP16. I'll share it but it won't be 'til Wednesday week, which is probably too late? | 15:53 |
| mriedem | he only speaks that langauge | 15:53 |
| mriedem | stephenfin: FF is thursday | 15:53 |
| efried | stephenfin: unless you accept that this task has been delegated to you. | 15:53 |
| efried | then Thursday is the "deadline" for the "first draft", so Wednesday is a whole day early! | 15:54 |
| efried | iow, it's all yours, and I don't need to vet/edit/revise it. | 15:54 |
| smcginnis | efried: I'm not happy about the deadline either, but the OSF marketing folks said they needed that info earlier. :/ | 15:55 |
| *** gyee has joined #openstack-nova | 15:55 | |
| mriedem | need to copy/paste into a chart asap | 15:55 |
| stephenfin | efried, mriedem: okay, I'll do it Monday | 15:55 |
| * stephenfin writes that down to make sure he doesn't forget | 15:56 | |
| *** shilpasd has quit IRC | 15:56 | |
| mriedem | maybe we should put an easter egg in the highlights... | 15:56 |
| mriedem | see who's paying attention | 15:56 |
| efried | I'm not blaming anyone, and I'm sure there's a "good" reason. It's just like, we're already all one-armed paper hangers already. | 15:56 |
| efried | mriedem: love that idea. | 15:56 |
| mriedem | "nova is deprecated, all hail serverless" | 15:56 |
| *** ivve has quit IRC | 15:56 | |
| stephenfin | There's one non-doc patch left in the remove-consoleauth series that I thought had already merged. If anyone has bandwidth to push it through (melwitt is +2), that'd be dandy https://review.opendev.org/#/c/652970/ | 16:00 |
| stephenfin | I'm just going to abandon the other two | 16:00 |
| dustinc | mriedem, efried, dtantsur|afk: all caught up, thanks for your help | 16:02 |
| efried | stephenfin: left a question there | 16:04 |
| efried | it looks like a behavior change to me | 16:04 |
| mriedem | efried: +2 on that deps update for the sighup fix https://review.opendev.org/#/c/679974/ - mnaser / dansmith fyi on that | 16:05 |
| efried | thanks mriedem | 16:05 |
| mriedem | definitely rc blocker | 16:05 |
| dansmith | efried: I was thinking you had tested something related to sighup and decided that it was *not* properly restarting us after the fix | 16:06 |
| dansmith | efried: did I misunderstand that or did something else change? | 16:06 |
| efried | dansmith: that was at the point where we only had the oslo.service fix. I dug further and fixed the oslo.privsep issue. With both fixes together, shit works. | 16:07 |
| dansmith | gotcha okay cool | 16:07 |
| efried | links from that patch --^ | 16:07 |
| dansmith | efried: yeah okay I didn't get that from the reno when I first read it, | 16:07 |
| dansmith | but I didn't study both bugs and just wanted to make sure | 16:08 |
| *** sapd1_x has quit IRC | 16:08 | |
| efried | ++ | 16:08 |
| melwitt | efried, stephenfinI think you're right :/ I didn't notice that. commenting | 16:08 |
| openstackgerrit | sean mooney proposed openstack/nova master: [DNM] testing with new lable. https://review.opendev.org/680738 | 16:08 |
| openstackgerrit | sean mooney proposed openstack/nova master: [DNM] testing with old lable https://review.opendev.org/680739 | 16:08 |
| efried | melwitt: looking again, I think the one I commented on is okay, but the one a couple lines down should be access_url, not access_url_base. | 16:08 |
| sean-k-mooney | ignore those by the way ^ | 16:08 |
| sean-k-mooney | im trying to debug stuff. | 16:09 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Find instance in another cell during floating IP re-association https://review.opendev.org/656594 | 16:09 |
| melwitt | efried: I think they should both be access_url. the second one works because it's just checking the scheme, which would be the same on both access_url_base and access_url | 16:09 |
| efried | melwitt: access_url is a @property tho, not a field. | 16:09 |
| melwitt | yeah, but I think you need to get the access_url first and if None, raise | 16:10 |
| melwitt | that check is supposed to be checking whether there is an access_url available and the property can return None | 16:11 |
| stephenfin | efried: Yeah, what melwitt is saying. I've replied on the patch | 16:11 |
| melwitt | but I dunno, maybe it doesn't really matter because all that method is doing is validating the scheme | 16:11 |
| *** mriedem is now known as mriedem_lunch | 16:11 | |
| melwitt | it's never going to look at anything beyond the "https" or whatever the scheme is | 16:11 |
| efried | ah, that makes sense | 16:12 |
| efried | arguably a bit unrelated, but sokay I guess. | 16:12 |
| melwitt | yeah, I'd agree that using the actual access_url here would be unrelated to what this method is doing | 16:13 |
| efried | melwitt: If you're still good, I'll +A | 16:14 |
| melwitt | (realized that as I typed the next two sentences) | 16:14 |
| melwitt | efried: yeah I think it's fine after thinking through | 16:15 |
| * melwitt wonders if I went through this thought loop before and completely forgot | 16:16 | |
| efried | ack, thanks for bearing with me melwitt & stephenfin | 16:16 |
| melwitt | probably did | 16:16 |
| stephenfin | thanks for the review (y) | 16:16 |
| efried | sean-k-mooney: can you please ack this https://review.opendev.org/#/c/680094/ ? | 16:22 |
| sean-k-mooney | oh the os-vif request | 16:23 |
| sean-k-mooney | yes i wanned to double check one thing | 16:24 |
| sean-k-mooney | but i think its fine | 16:24 |
| efried | sean-k-mooney: ...per http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009182.html | 16:24 |
| efried | thank you | 16:24 |
| sean-k-mooney | we did 2 train release already for m1 and m2 so this tecnically is not the first os-vif release of train | 16:24 |
| sean-k-mooney | but i think 1.17.0 still make sense for other reasons | 16:24 |
| efried | sean-k-mooney: right, that was the main thing I was concerned about. It looked like there were two bugfixes since the last release, but one of them seemed like it might be feature-y. | 16:25 |
| efried | I'm sure it's not the end of the world if we go feature bump when we should have gone bugfix bump... is it smcginnis? | 16:25 |
| sean-k-mooney | one of them was a securiy bug which has been backported | 16:25 |
| sean-k-mooney | honestly version number are cheap so i would rather bump more then we neeeded too then not so 1.17 looks good to me | 16:26 |
| sean-k-mooney | efried: no if we bump major it would be an issue but bumping feature should not be | 16:27 |
| sean-k-mooney | stien is on the 1.15.x seriese anyway | 16:27 |
| efried | I guess the only downside could be confusion of having .16 and .17 both in train. But meh. | 16:28 |
| smcginnis | efried: So that one wasn't a new feature? I didn't go much further than the commit message. | 16:28 |
| efried | smcginnis: the commit message sounded feature-y for sure, but there was a bug (not a bp) associated. This is what I wanted sean-k-mooney to confirm for us. | 16:29 |
| efried | I could look at the code, but... | 16:29 |
| efried | jangutter: you have an opinion here? | 16:29 |
| efried | specifically: is this https://review.opendev.org/#/c/658786/ a "feature" or a "bugfix", for purposes of release numbering? | 16:30 |
| jangutter | efried: it leans more to "feature" than bugfix, especially since the original idea was "feature". | 16:31 |
| efried | jangutter: Okay, good enough for me, thanks. | 16:31 |
| efried | sean-k-mooney, smcginnis: ^ cool? | 16:31 |
| smcginnis | Works for me. | 16:31 |
| jangutter | efried: os-vif uses bugs to track features (like neutron) | 16:32 |
| efried | smcginnis: acked, thanks for your patience. | 16:32 |
| efried | jangutter: okay, gtk, then I'm going to have this question every release :P | 16:32 |
| jangutter | efried, sean-k-mooney: is there a way to distinguish between [RFE] bugs and your run of the mill annoyances? | 16:33 |
| smcginnis | Thanks for following up on that. | 16:33 |
| efried | jangutter: With careful use of tagging, I'm sure there is. You may have even had that in place in this case - I admit I didn't look. | 16:34 |
| smcginnis | It's been pretty common in my experience that new "features" also fix bugs, so I've never really used BPs vs bugs as an indicator. | 16:34 |
| efried | But that would also require an understanding of whatever tag scheme by whoever's doing the looking | 16:35 |
| efried | and yeah, more broadly, what smcginnis says. | 16:35 |
| efried | it's on a team by team basis, so really we need team members to make this call one way or another (either by proposing the release patches themselves and noting this reasoning in the commit message, or by being pestered like this when releases are proposed "automatically" or otherwise externally to the team) | 16:36 |
| sean-k-mooney | jangutter: we use RFE bugs in os-vif to track feature | 16:37 |
| sean-k-mooney | jangutter: because we dont want to have to use specs or blueprints | 16:37 |
| sean-k-mooney | so RFE bugs cant be backported as they are features | 16:37 |
| sean-k-mooney | other then that no | 16:37 |
| *** pcaruana has quit IRC | 16:37 | |
| efried | aspiers: I'm looking at https://review.opendev.org/#/c/680065/8/nova/tests/unit/scheduler/test_utils.py -- I was digging for this answer myself, but am being pulled in multiple directions -- can you confirm that there are other tests covering the behavior when hw_machine_type is absent (which is to allow it through) ? | 16:40 |
| efried | I think tests above those are doing it, yah? | 16:40 |
| aspiers | efried: checking | 16:40 |
| efried | iow I wanted to say "hey, instead of doing this to all three, just do it to one or two, so it proves the other thing" -- but don't want to block on that if other tests are already covering that. | 16:41 |
| aspiers | efried: testing hw_machine_type being absent was missing until very recently, I just added it like 2 days ago | 16:41 |
| aspiers | let me find where | 16:41 |
| aspiers | it's mentioned in a commit message somewhere | 16:41 |
| aspiers | oh yeah it's this review | 16:41 |
| aspiers | let me find the lines | 16:42 |
| *** derekh has quit IRC | 16:42 | |
| aspiers | efried: https://review.opendev.org/#/c/680065/8/nova/tests/unit/virt/test_hardware.py@3837 | 16:42 |
| efried | that's asserting failure due to no uefi... | 16:43 |
| aspiers | doh | 16:43 |
| aspiers | brain fried | 16:43 |
| efried | where's a case that succeeds with no mach type | 16:43 |
| aspiers | ah OK | 16:43 |
| aspiers | checking | 16:43 |
| efried | perhaps L3926 could have a None in it too? | 16:43 |
| aspiers | ah that has to be tested in the driver | 16:44 |
| aspiers | since with no machine type in the image, it relies on pulling it from utils | 16:44 |
| aspiers | which reads nova.conf on compute node | 16:44 |
| aspiers | I think it's maybe in a later patch | 16:44 |
| aspiers | checking ... | 16:44 |
| aspiers | efried: https://review.opendev.org/#/c/644565/52/nova/tests/unit/virt/libvirt/test_driver.py@2700 | 16:46 |
| aspiers | although maybe there is a test missing for when CONF.libvirt.hw_machine_type is set to x86_64=q35 | 16:47 |
| efried | aspiers: that one is *also* a failure scenario | 16:47 |
| aspiers | yeah that's what I'm saying | 16:48 |
| *** vesper11 has quit IRC | 16:48 | |
| aspiers | CONF.libvirt.hw_machine_type set to x86_64=q35 would be the non-failure scenario | 16:48 |
| aspiers | I can add that | 16:48 |
| *** vesper11 has joined #openstack-nova | 16:48 | |
| aspiers | efried: kudos for eagle eyes as usual :) | 16:48 |
| efried | where are you going to add it? | 16:49 |
| efried | aspiers: is it adequate to undo one of those three lines of delta here https://review.opendev.org/#/c/680065/8/nova/tests/unit/scheduler/test_utils.py ? | 16:50 |
| efried | and then maybe also add the case next to the one in the fourth patch? | 16:50 |
| *** pcaruana has joined #openstack-nova | 16:51 | |
| efried | aspiers: there's one more comment in there, in case you agree and want to make that change at the same time. | 16:51 |
| * efried moves on to other things. | 16:51 | |
| *** efried is now known as efried_afk | 16:51 | |
| aspiers | efried_afk: ok thanks! | 16:51 |
| *** gyee has quit IRC | 16:56 | |
| *** gyee has joined #openstack-nova | 16:56 | |
| *** cdent has quit IRC | 16:58 | |
| *** ociuhandu has quit IRC | 17:03 | |
| *** gbarros has quit IRC | 17:08 | |
| *** markvoelker has joined #openstack-nova | 17:25 | |
| *** udesale has quit IRC | 17:27 | |
| *** mriedem_lunch is now known as mriedem | 17:38 | |
| mriedem | dansmith: so linkedin told me it was a big day for you and i was looking at the suggested comments and i thought this one was a good fit https://photos.app.goo.gl/WrC1wjQYu7Btfz2g8 | 17:45 |
| dansmith | heh | 17:48 |
| dansmith | yeah, I wish linkedin wouldn't ask people to do that | 17:48 |
| dansmith | and/or I wish people would ignore all that crap like I do :) | 17:48 |
| sean-k-mooney | i think mose people do | 17:49 |
| aspiers | haha | 17:50 |
| sean-k-mooney | dansmith: the real question is does it feel like 6 or 12 | 17:51 |
| dansmith | sean-k-mooney: see, you're doing it, although indirectly :) | 17:51 |
| *** gbarros has joined #openstack-nova | 17:53 | |
| *** jangutter has quit IRC | 17:54 | |
| *** jawad_axd has joined #openstack-nova | 17:55 | |
| mriedem | i just laugh at the AI/ML generated auto responses | 17:55 |
| *** mvkr has quit IRC | 17:55 | |
| mriedem | i hate them on my phone but am too lazy to figure out how to disable | 17:55 |
| mriedem | because no my every response to my wife's text messages shouldn't be, "I know, right?" | 17:55 |
| melwitt | I know that you can disable profile updates from notifying the world but I don't know if that encompasses work anniversaries too | 17:56 |
| mriedem | heh, messaging > settings > smart reply (disable) | 17:56 |
| mriedem | that was easy | 17:56 |
| *** markvoelker has quit IRC | 17:57 | |
| mriedem | "I know, right?" | 17:57 |
| *** markvoelker has joined #openstack-nova | 18:03 | |
| sean-k-mooney | ok i havent got anything useful done in an hour some im goign to get dinner. i might be back online later. if not have a nice weekend o/ | 18:05 |
| dansmith | sean-k-mooney: it's friday. turn. off. the. laptop. | 18:05 |
| *** markvoelker has quit IRC | 18:12 | |
| *** tbachman has quit IRC | 18:13 | |
| mriedem | but is bauzas still up working on a friday night? | 18:13 |
| mriedem | b/c i'm +2 on https://review.opendev.org/#/c/680394/4 and the 2 above it | 18:13 |
| *** luksky has joined #openstack-nova | 18:13 | |
| *** markvoelker has joined #openstack-nova | 18:15 | |
| mriedem | gibi: when you're back and collecting comments, i approved but left a comment about defining constants for https://review.opendev.org/#/c/655109/24 | 18:16 |
| *** tbachman has joined #openstack-nova | 18:20 | |
| *** factor has joined #openstack-nova | 18:24 | |
| *** markvoelker has quit IRC | 18:26 | |
| *** ralonsoh has quit IRC | 18:27 | |
| mriedem | if anyone is adventurous, there are +2s waiting for a +W on 4 of the bw provider move series https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports+status:open+label:Code-Review=2 | 18:27 |
| mriedem | they are all about min compute service version checking and functional tests | 18:28 |
| mriedem | so pretty easy | 18:28 |
| *** markvoelker has joined #openstack-nova | 18:28 | |
| mriedem | the bottom one is only big b/c of a lot of test coverage | 18:28 |
| *** maciejjozefczyk has quit IRC | 18:36 | |
| *** markvoelker has quit IRC | 18:38 | |
| mriedem | dansmith: so i've got an rpc compat thing i'd like to remove but need to get out of my head, | 18:38 |
| mriedem | before this change, https://review.opendev.org/#/c/582417/ conductor would always send a legacy request spec dict to compute's prep_resize method during cold migrate/resize and compute would never send it back to conductor on reschedule, | 18:39 |
| mriedem | with that change, prep_resize on the comput takes a RequestSpec object if you're not pinned | 18:39 |
| mriedem | and compute passes back whatever it gets | 18:39 |
| mriedem | there were no changes to the conductor compute task api rpc interface b/c it always took a request spec and passed it through (for the api calling conductor initially) | 18:40 |
| mriedem | what i'd really like to do is remove this compat code when compute doesn't send a request spec back on reschedule https://review.opendev.org/#/c/582417/7/nova/conductor/manager.py | 18:40 |
| mriedem | which should be possible if we aren't supporting older-than-stein computes in train or ussuri, | 18:41 |
| dansmith | violates the rpcapi though | 18:41 |
| mriedem | where i think i f'ed up was that the compat code in conductor isn't handling request spec being a dict | 18:41 |
| mriedem | no i don't think it violates the rpc api because there was no change to the conductor rpc api to remove the request spec that compute passes back | 18:41 |
| mriedem | it's just a pass through | 18:41 |
| mriedem | the only interface that changed was the compute rpc api | 18:41 |
| *** jawad_axd has quit IRC | 18:42 | |
| mriedem | so the only time compute won't send a request spec back up is if it's pre-stein | 18:42 |
| dansmith | the object could be None there which means old version right? | 18:42 |
| mriedem | well, if you're pinned to less than conductor 1.13 yeah... | 18:42 |
| dansmith | right | 18:42 |
| mriedem | which was mitaka https://review.opendev.org/#/c/277800/ but yeah | 18:43 |
| dansmith | so you can't remove the ability to take an N.0 call unless you bump majors | 18:43 |
| dansmith | I mean, | 18:43 |
| mriedem | right so conductor 2.0 to make request spec required and RequestSpec object | 18:43 |
| dansmith | I realize other shit will break if you go that old, but either we stick to these rules or we don't | 18:43 |
| mriedem | yeah | 18:43 |
| mriedem | i just came across that todo while reviewing gibi's series and though i'm glad we have that, and was going to remove the compat, but then realized i can't | 18:44 |
| mriedem | so i'll be amending that todo comment i guess since i don't feel like taking on a conductor 2.0 bump | 18:44 |
| dansmith | aye | 18:45 |
| openstackgerrit | Merged openstack/nova master: objects: Remove ConsoleAuthToken.to_dict https://review.opendev.org/652970 | 18:51 |
| *** igordc has joined #openstack-nova | 18:53 | |
| *** efried_afk is now known as efried | 18:54 | |
| *** gbarros has quit IRC | 19:03 | |
| *** markvoelker has joined #openstack-nova | 19:11 | |
| *** markvoelker has quit IRC | 19:15 | |
| *** gbarros has joined #openstack-nova | 19:24 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle legacy request spec dict in ComputeTaskManager._cold_migrate https://review.opendev.org/680762 | 19:25 |
| mriedem | ok i think that handles both issues (bad comment and dict compat) | 19:25 |
| openstackgerrit | Merged openstack/nova master: Bump min for oslo.service & .privsep to fix SIGHUP https://review.opendev.org/679974 | 19:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup request spec handling in _cold_migrate https://review.opendev.org/680763 | 19:40 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Isolate request spec handling from _cold_migrate https://review.opendev.org/680763 | 19:40 |
| *** markvoelker has joined #openstack-nova | 19:41 | |
| efried | dansmith: I haven't looked at all yet, but it's almost 3 and I'm still not dug out, and wanted to catch you today if possible -- did a) the vpmem Resource ovo stuff and b) the forbidden aggs stuff shake out satisfactorily? | 19:41 |
| dansmith | I haven't gotten to them yet | 19:41 |
| *** markvoelker has quit IRC | 19:45 | |
| dansmith | efried: do you know why the isolated aggs set was rebased this morning? | 19:50 |
| dansmith | doesnt' seem like anything changed and it means we still don't have a ci run on it yet | 19:50 |
| efried | dansmith: I don't. Maybe so the owner can retain commiter-hood? | 19:50 |
| * efried looks... | 19:51 | |
| dansmith | that's not how that works | 19:51 |
| aspiers | efried: on further thought, I think https://review.opendev.org/#/c/680065/ is fine as it stands - see the comments I just posted | 19:51 |
| efried | dansmith: oh, it looks like the bottom patch needed an ovo hash bump -- oops, that's my bad. | 19:51 |
| dansmith | oh okay I thought I checked all the files but maybe I missed that one | 19:52 |
| dansmith | (for changes I mean) | 19:52 |
| efried | https://review.opendev.org/#/c/671072/14..15/nova/tests/unit/objects/test_objects.py | 19:52 |
| dansmith | yeah I know | 19:53 |
| efried | aspiers: beaut, thanks for following up. +A | 19:59 |
| *** nweinber has quit IRC | 20:00 | |
| aspiers | efried: yw. Don't hesitate to ask if I can (attempt to) help explain other oddities in the remaining two patches | 20:04 |
| *** BjoernT has quit IRC | 20:06 | |
| *** ricolin has joined #openstack-nova | 20:08 | |
| *** markvoelker has joined #openstack-nova | 20:19 | |
| * efried ==> chauffeur | 20:19 | |
| *** efried is now known as efried_afk | 20:19 | |
| mriedem | artom: dansmith: since stephen and sean aren't around, i'll ask you this https://review.opendev.org/#/c/621476/62/nova/api/openstack/compute/server_topology.py@66 | 20:22 |
| mriedem | since i don't know numa | 20:22 |
| dansmith | same pagesize on all cells, is that what you mean? | 20:22 |
| mriedem | yeah | 20:22 |
| *** tbachman has quit IRC | 20:23 | |
| dansmith | they will all have a pagesize and they'll all be identical | 20:23 |
| dansmith | I asked that earlier in artom's series | 20:23 |
| mriedem | if they will all have a page size why is that conditional? | 20:23 |
| dansmith | oh, not sure they will all always have a pagesize, but if any do, they'll be the same | 20:24 |
| dansmith | now, that's a libvirt/x86 thing.. I can imagine power allowing multiples because some team in DB2 asked for it in 1996 | 20:24 |
| mriedem | ok, that works for me | 20:24 |
| dansmith | all that probably depends on the hypervisor and arch | 20:24 |
| mriedem | this is an intel (tm) api so don't worry about it | 20:25 |
| dansmith | so I'm sure this api will be broken for other ones that aren't considered here | 20:25 |
| dansmith | WONderful | 20:25 |
| mriedem | powerstackers was burned out of governance b/c eric wouldn't lead them anymore | 20:25 |
| mriedem | and the z folks have moved onto openshift on mainframes | 20:25 |
| dansmith | shocking | 20:26 |
| mriedem | has to be weird when the german customers ask for sles + openshift on their z mainframe | 20:26 |
| *** markvoelker has quit IRC | 20:29 | |
| *** tbachman has joined #openstack-nova | 20:30 | |
| *** gbarros has quit IRC | 20:44 | |
| *** markvoelker has joined #openstack-nova | 20:50 | |
| mriedem | so this is the 2.78 server numa topology command output right now in the proposed change: http://paste.openstack.org/show/772110/ | 20:52 |
| mriedem | that....sucks | 20:52 |
| *** markvoelker has quit IRC | 20:54 | |
| sean-k-mooney | libvirt/qemu support having muliple page size even in the same numa node but we dont support that in openstack and likely never will | 20:58 |
| mriedem | sean-k-mooney: ok i'll stop writing up that spec for ussuri | 20:58 |
| mriedem | i thought you'd be on board | 20:58 |
| mriedem | but i guess you don't like that kind of excitement | 20:58 |
| sean-k-mooney | if you want to debug it when it does not work feel free | 20:59 |
| mriedem | heh | 20:59 |
| mriedem | obviously i'm joking | 20:59 |
| mriedem | i've been trying to come up with some kind of hardware schmorgasbord name generator thing to get your goat | 20:59 |
| *** gbarros has joined #openstack-nova | 21:00 | |
| sean-k-mooney | just pick 3 random letters and its proably some future NFV hardware thing | 21:00 |
| mriedem | FUK | 21:00 |
| sean-k-mooney | :) | 21:01 |
| mriedem | failover uber kernel | 21:01 |
| mriedem | it's a new DR thing | 21:01 |
| sean-k-mooney | i thought it was a brexit reference | 21:01 |
| sean-k-mooney | but ok :) | 21:01 |
| mriedem | you've been boris'ed | 21:01 |
| *** tbachman has quit IRC | 21:01 | |
| mriedem | btw https://www.youtube.com/watch?v=dXyO_MC9g3k if you haven't seen that, it's pretty good | 21:01 |
| sean-k-mooney | i have not, i look forword to it :) | 21:02 |
| mriedem | ok, time for some cli design, | 21:04 |
| mriedem | who's in? | 21:04 |
| mriedem | http://paste.openstack.org/show/772111/ | 21:04 |
| *** ociuhandu has joined #openstack-nova | 21:04 | |
| mriedem | that's better than before, but the list of nodes in json string is not great, | 21:05 |
| sean-k-mooney | is that the output for the toplogy api? | 21:05 |
| mriedem | but if you dumped each node dict as a separate property, what would you name the property? because that host_node field is only in the response for admins, | 21:05 |
| mriedem | yeah, in the cli | 21:05 |
| mriedem | proposed cli | 21:05 |
| mriedem | so if you're an admin, you could name each one as node{host_node} so like node0 and node1 | 21:05 |
| sean-k-mooney | can we pretty print the json blob | 21:06 |
| mriedem | but if you're not admin, calling them node0 or node1 could make the user think they are actually on those physical host numa nodes | 21:06 |
| mriedem | hmm, maybe | 21:06 |
| *** ricolin has quit IRC | 21:14 | |
| sean-k-mooney | its a litte more readable. http://paste.openstack.org/show/772112/ i didnt put it in the ascis table | 21:14 |
| *** ociuhandu has quit IRC | 21:15 | |
| sean-k-mooney | but you could also do somithing like | 21:15 |
| mriedem | that's what i already have http://paste.openstack.org/show/772111/ | 21:15 |
| sean-k-mooney | |node.[0].cpu_pinning | {"0": 0,"1": 5} | 21:16 |
| mriedem | anyway, i've got this better than it was so i'll push it up and someone else can hack on it | 21:16 |
| mriedem | it's not even my patch | 21:16 |
| sean-k-mooney | the latest windows update sucks ... | 21:17 |
| mriedem | 10...? | 21:17 |
| sean-k-mooney | ya | 21:18 |
| mriedem | i'm rocking 7 | 21:18 |
| sean-k-mooney | it install an update last night and now its haning if i boot it in a vm | 21:18 |
| sean-k-mooney | if i reboot the host indo the same windwos install its fine | 21:18 |
| *** rcernin|brb has joined #openstack-nova | 21:19 | |
| sean-k-mooney | i did like 7. i think i still have my 7 ultimate 64bit dvd around somewhere. | 21:21 |
| mnaser | efried_afk: thanks for driving the SIGHUP patches stuff, i ran out of bandwidth | 21:21 |
| *** efried_afk is now known as efried | 21:22 | |
| efried | mnaser: yahyoubetcha. Glad to have that fixed finally. | 21:22 |
| *** prometheanfire has joined #openstack-nova | 21:22 | |
| prometheanfire | os-brick update is breaking nova :( https://review.opendev.org/680339 | 21:22 |
| efried | prometheanfire: does cinder know? | 21:22 |
| efried | (I didn't click yet) | 21:23 |
| mnaser | AttributeError: 'module' object has no attribute 'NVMeConnector' | 21:23 |
| mnaser | https://github.com/openstack/os-brick/commit/0f057aee6e0351bdef68fadf1fc56ac315027c98 | 21:23 |
| prometheanfire | no, test fails were in nova so pinged you first | 21:23 |
| mnaser | im guessing those are the fails | 21:24 |
| mnaser | the module was renamed | 21:24 |
| openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Microversion 2.78 - show server topology https://review.opendev.org/670790 | 21:24 |
| sean-k-mooney | are the nova unit test mocking stuff in os-bricks | 21:25 |
| prometheanfire | try import except import? | 21:25 |
| mnaser | btw | 21:25 |
| mnaser | there is some context here https://review.opendev.org/#/c/643421/ | 21:25 |
| sean-k-mooney | ok so we are importing the conector object in the test | 21:27 |
| mnaser | `initiator.NVME` that was left around for backwards compat | 21:27 |
| mnaser | but the connector did not keep a backwards compat | 21:28 |
| mnaser | i mean as a work around you could fall back to asserting the initiator used instead of the connector used | 21:29 |
| *** macz has quit IRC | 21:29 | |
| sean-k-mooney | so the real driver code use the factory method to consurct it | 21:29 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/volume/nvme.py#L34 | 21:29 |
| mnaser | sean-k-mooney: it uses `initiator.NVME` which is still around | 21:30 |
| sean-k-mooney | i think the real fix is to not assert thing abot the data type of an external lib | 21:30 |
| *** macz has joined #openstack-nova | 21:30 | |
| mnaser | https://github.com/openstack/os-brick/blob/0f057aee6e0351bdef68fadf1fc56ac315027c98/os_brick/initiator/connector.py#L117-L122 | 21:30 |
| mnaser | cause thats still around | 21:30 |
| sean-k-mooney | mnaser: yes but i guess they considerd as part of the public interface | 21:30 |
| sean-k-mooney | and the conector object are not | 21:31 |
| mnaser | maybe the test should mock `connector.InitiatorConnector.factory` and make sure it calls it with initiator.NVME ? | 21:31 |
| mnaser | which is enough to know we are doing the right thing(tm) | 21:31 |
| sean-k-mooney | maybe but its stil mocking stuff we dont own | 21:32 |
| sean-k-mooney | but that seams safer | 21:32 |
| sean-k-mooney | sice we are mocking somethign we are calling | 21:32 |
| mnaser | well at least it makes sure that we're creating a connector with the right type of initiator | 21:32 |
| mnaser | which i think is the biggest thing that test is supported to do.. | 21:32 |
| sean-k-mooney | right but that is not novas job to test | 21:32 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: fix server topology "host_numa_node" field param name https://review.opendev.org/680775 | 21:32 |
| sean-k-mooney | that is os-bricks job to test | 21:32 |
| sean-k-mooney | we shoudl jsut assert we called ti with the correct input | 21:33 |
| *** rcernin|brb has quit IRC | 21:33 | |
| mnaser | exactly thats what i meant about mocking `connector.InitiatorConnector.factory` | 21:33 |
| mnaser | aka make sure we're calling it with initiator.NVME as an argument (instead of $some_other_weird_thing) | 21:33 |
| sean-k-mooney | ya which seams more robust | 21:33 |
| sean-k-mooney | yep | 21:33 |
| mnaser | its friday and im waiting for my food | 21:33 |
| efried | dansmith: responded in https://review.opendev.org/671075 | 21:34 |
| mnaser | let me push up a patch quick.. | 21:34 |
| dansmith | efried: ack, but I'm EOD | 21:35 |
| efried | nod | 21:35 |
| dansmith | replied for action | 21:36 |
| * dansmith & | 21:36 | |
| efried | thanks | 21:38 |
| *** KeithMnemonic1 has quit IRC | 21:40 | |
| mriedem | it's in-law o'clock | 21:41 |
| mriedem | o/ | 21:41 |
| *** mriedem has quit IRC | 21:41 | |
| *** macz has quit IRC | 21:43 | |
| *** gbarros has quit IRC | 21:45 | |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Apply SEV-specific guest config when SEV is required https://review.opendev.org/644565 | 21:48 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Reject live migration and suspend on SEV guests https://review.opendev.org/680158 | 21:49 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Decouple NVMe tests from os-brick https://review.opendev.org/680779 | 21:52 |
| mnaser | sean-k-mooney: ^ unit tests and flake8 passes here locally | 21:52 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | 21:53 |
| mnaser | efried: fyi the requirements breakage should be solved by that ^ | 21:53 |
| mnaser | oh rap | 21:53 |
| mnaser | i have an extra file there | 21:53 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Decouple NVMe tests from os-brick https://review.opendev.org/680779 | 21:53 |
| efried | mnaser: ugh, do we really need root helper? | 21:54 |
| mnaser | efried: i mean i can skip the mock and import utils ? | 21:55 |
| mnaser | but i mean its being passed down there | 21:55 |
| efried | sorry, I just remember recently seeing where we were trying to get rid of that thing, so adding another reference to it feels like increasing tech debt. | 21:55 |
| mnaser | https://docs.openstack.org/os-brick/stein/reference/os_brick/initiator/connector.html | 21:55 |
| mnaser | well i only touched the tests, it was already referencing it | 21:56 |
| mnaser | and it looks like the library relies on needing it | 21:56 |
| efried | yeah, it just makes me sad. | 21:56 |
| efried | mnaser, prometheanfire: Ima update the req patch to dep-on this guy to make sure it clears up, mkay? | 21:56 |
| *** macz has joined #openstack-nova | 21:56 | |
| mnaser | efried: i mean i can get rid of the mock and have it call the real function but still gonna have a reference at the end of teh day | 21:57 |
| efried | don't worry about it. | 21:57 |
| mnaser | cools | 21:57 |
| * mnaser goes back to food | 21:57 | |
| aspiers | efried: woot, SEV tempest test is working, exercising many code paths, and even found a bug which is now fixed :) | 22:07 |
| aspiers | now I just have to wait a week for Zuul to catch up :( | 22:07 |
| efried | nice | 22:12 |
| efried | aspiers: did you do the tempest work? | 22:12 |
| aspiers | yes | 22:12 |
| aspiers | I'd never touched tempest before yesterday | 22:12 |
| aspiers | It's pretty straight-forward though, and nicely documented | 22:12 |
| efried | feel like writing one to cover https://review.opendev.org/#/c/680542/ ? | 22:12 |
| aspiers | Not right now ;-) It's 11am and yet another week where I'm about 20 hours over time | 22:13 |
| efried | it was worth a try | 22:13 |
| aspiers | But hey, if all the SEV stuff lands early next week then I'd have spare time on my hands ... hint hint ;-) | 22:13 |
| aspiers | In fixing the designer.py found by tempest, the code got a lot simpler - look: https://review.opendev.org/#/c/644565/53/nova/virt/libvirt/designer.py@202 | 22:14 |
| aspiers | *missing "bug" in above sentence | 22:15 |
| *** markvoelker has joined #openstack-nova | 22:15 | |
| efried | ack | 22:15 |
| aspiers | Other than reinstating W+1 on the rebased https://review.opendev.org/#/c/680158/, there's only that patch and the final master switch one left needing reviews | 22:16 |
| aspiers | so it feels like we're close, CI permitting | 22:17 |
| *** markvoelker has quit IRC | 22:20 | |
| prometheanfire | efried: sgtm | 22:25 |
| *** luksky has quit IRC | 22:51 | |
| openstackgerrit | Merged openstack/nova master: doc: cleanup references to conductor doc https://review.opendev.org/680535 | 22:54 |
| *** ociuhandu has joined #openstack-nova | 23:15 | |
| *** ociuhandu has quit IRC | 23:20 | |
| openstackgerrit | Merged openstack/nova master: Remove old comments about caching scheduler compat https://review.opendev.org/680521 | 23:28 |
| *** mdbooth has quit IRC | 23:42 | |
| *** mdbooth has joined #openstack-nova | 23:44 | |
| *** gyee has quit IRC | 23:45 | |
| *** mtreinish has quit IRC | 23:49 | |
| *** efried has quit IRC | 23:51 | |
| *** gyee has joined #openstack-nova | 23:55 | |
| *** gyee has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
