Thursday, 2019-05-23

*** gyee has quit IRC00:04
*** ttsiouts has joined #openstack-nova00:10
*** luksky has quit IRC00:14
*** ttsiouts has quit IRC00:16
*** Sundar has quit IRC00:22
openstackgerritBrin Zhang proposed openstack/python-novaclient master: Set the lower limit of api_version for volume_type  https://review.opendev.org/66086100:25
*** brinzhang has joined #openstack-nova00:29
*** frank_wang has quit IRC00:39
*** itlinux has joined #openstack-nova00:44
*** ttsiouts has joined #openstack-nova00:45
*** frankwang has joined #openstack-nova00:47
*** itlinux has quit IRC00:52
*** frankwang has quit IRC00:52
*** frankwang has joined #openstack-nova00:53
*** slaweq has joined #openstack-nova00:57
*** slaweq has quit IRC01:01
openstackgerritJake Yip proposed openstack/nova master: refactor nova-manage archive_deleted_rows  https://review.opendev.org/64377901:07
openstackgerritJake Yip proposed openstack/nova master: Add --before to nova-manage db archive_deleted_rows  https://review.opendev.org/55675101:08
*** itlinux has joined #openstack-nova01:10
*** itlinux has quit IRC01:10
*** ttsiouts has quit IRC01:18
*** bbowen has quit IRC01:19
*** guozijn has joined #openstack-nova01:22
*** guozijn has quit IRC01:27
*** guozijn has joined #openstack-nova01:27
*** itlinux has joined #openstack-nova01:30
*** whoami-rajat has joined #openstack-nova01:31
*** guozijn_ has joined #openstack-nova01:34
*** ricolin has joined #openstack-nova01:36
*** guozijn has quit IRC01:37
*** guozijn_ is now known as guozijn01:37
*** tbachman has quit IRC01:40
*** bbowen has joined #openstack-nova01:50
openstackgerritMerged openstack/nova stable/stein: Restore connection_info after live migration rollback  https://review.opendev.org/66037002:06
*** slaweq has joined #openstack-nova02:11
*** slaweq has quit IRC02:15
*** ykarel|away has joined #openstack-nova02:20
*** edleafe has quit IRC02:26
*** irclogbot_0 has quit IRC02:26
*** irclogbot_3 has joined #openstack-nova02:29
*** ttsiouts has joined #openstack-nova02:30
*** itlinux has quit IRC02:39
openstackgerritMerged openstack/nova stable/rocky: xenapi/agent: Change openssl error handling  https://review.opendev.org/65630702:48
*** nicolasbock has quit IRC02:51
*** _erlon_ has quit IRC02:52
*** itlinux has joined #openstack-nova02:53
*** ttsiouts has quit IRC03:04
*** itlinux has quit IRC03:08
openstackgerritMerged openstack/python-novaclient master: Allow passing negative values for the locked search_opt in cs.servers.list  https://review.opendev.org/65978303:13
*** itlinux has joined #openstack-nova03:35
*** itlinux has quit IRC03:39
*** itlinux has joined #openstack-nova03:42
*** itlinux has quit IRC03:43
*** samueldmq has quit IRC03:50
*** yikun_ has quit IRC03:50
*** frankwang has quit IRC03:59
*** itlinux has joined #openstack-nova04:00
*** itlinux has quit IRC04:05
*** ttsiouts has joined #openstack-nova04:09
*** itlinux has joined #openstack-nova04:10
*** slaweq has joined #openstack-nova04:11
*** slaweq has quit IRC04:16
*** ykarel|away has quit IRC04:17
openstackgerritMerged openstack/nova stable/stein: libvirt: Do not reraise DiskNotFound exceptions during resize  https://review.opendev.org/66036104:28
*** udesale has joined #openstack-nova04:28
*** ykarel|away has joined #openstack-nova04:31
*** luksky has joined #openstack-nova04:35
*** luksky has quit IRC04:41
*** luksky has joined #openstack-nova04:42
*** ttsiouts has quit IRC04:43
*** itlinux has quit IRC04:43
*** luksky has quit IRC04:56
*** lpetrut has joined #openstack-nova05:11
*** ttsiouts has joined #openstack-nova05:12
*** ratailor has joined #openstack-nova05:16
*** ttsiouts has quit IRC05:16
*** ykarel|away is now known as ykarel05:20
*** ccamacho has quit IRC05:29
*** itlinux has joined #openstack-nova05:30
*** janki has joined #openstack-nova05:31
*** zbr has joined #openstack-nova05:34
*** imacdonn has quit IRC05:41
*** imacdonn has joined #openstack-nova05:42
*** frankwang has joined #openstack-nova05:43
openstackgerritArthur Dayne proposed openstack/nova stable/rocky: Add the missing ' in routes.py  https://review.opendev.org/66088505:49
*** ttsiouts has joined #openstack-nova05:49
*** slaweq has joined #openstack-nova05:59
*** Luzi has joined #openstack-nova06:02
*** slaweq has quit IRC06:03
*** slaweq has joined #openstack-nova06:11
*** cfriesen has quit IRC06:14
*** dpawlik has joined #openstack-nova06:16
*** jchhatbar has joined #openstack-nova06:19
*** janki has quit IRC06:20
*** itlinux has quit IRC06:20
*** ttsiouts has quit IRC06:23
*** ileixe has joined #openstack-nova06:38
ileixeHi forks.06:38
ileixeI got an simple test using cpu pinning lately, and the result is somewhat unexpected. So if you are experince in the same environment, please let me know what's wrong.06:40
ileixeMy test is that VMs with cpu pinning host can interfere affect to each other.06:41
*** tesseract has joined #openstack-nova06:41
ileixeI assume that VM with CPU pinning can achieve their own workload though, but the fact is If nosiy VM (which is also CPU pinned) is there they affect to each other.06:42
*** tbachman has joined #openstack-nova06:49
*** rcernin has quit IRC07:00
*** markvoelker has joined #openstack-nova07:03
*** tbachman has quit IRC07:03
*** pcaruana has joined #openstack-nova07:07
*** markvoelker has quit IRC07:08
openstackgerritMerged openstack/nova stable/stein: Log when port resource is leaked during port delete  https://review.opendev.org/65758107:09
*** helenafm has joined #openstack-nova07:12
*** rpittau|afk is now known as rpittau07:20
*** awalende has joined #openstack-nova07:20
*** ircuser-1 has quit IRC07:26
*** jchhatbar has quit IRC07:27
*** ircuser-1 has joined #openstack-nova07:28
*** ttsiouts has joined #openstack-nova07:29
openstackgerritMerged openstack/nova stable/stein: Include all network devices in nova diagnostics  https://review.opendev.org/65712507:30
openstackgerritMerged openstack/nova master: Update description of valid whitelist for non-admin user  https://review.opendev.org/64240307:30
openstackgerritHamdy Khader proposed openstack/os-vif master: OVS DPDK port representors support  https://review.opendev.org/65878607:34
*** ccamacho has joined #openstack-nova07:35
*** ircuser-1 has quit IRC07:36
*** ykarel is now known as ykarel|afk07:37
openstackgerritAlexandre arents proposed openstack/nova master: Fix live-migration when glance image deleted  https://review.opendev.org/65905407:37
*** ircuser-1 has joined #openstack-nova07:38
*** ivve has joined #openstack-nova07:42
openstackgerritMerged openstack/nova master: Skip existing VMs when hosts apply force_config_drive  https://review.opendev.org/65970307:43
*** janki has joined #openstack-nova07:44
openstackgerritLee Yarwood proposed openstack/nova stable/stein: Skip existing VMs when hosts apply force_config_drive  https://review.opendev.org/66091407:44
openstackgerritLee Yarwood proposed openstack/nova stable/rocky: Skip existing VMs when hosts apply force_config_drive  https://review.opendev.org/66091507:45
openstackgerritLee Yarwood proposed openstack/nova stable/queens: Skip existing VMs when hosts apply force_config_drive  https://review.opendev.org/66091707:45
*** tssurya has joined #openstack-nova07:46
*** ykarel|afk is now known as ykarel08:03
*** trident has quit IRC08:04
*** ralonsoh has joined #openstack-nova08:04
alex_xuileixe: what kind of nosiy you pointed to?08:05
*** tssurya has quit IRC08:05
ileixeCPU load08:05
ileixealex_xu: I used stress and ssl encryption08:05
ileixetool called 'stress'08:05
*** trident has joined #openstack-nova08:05
alex_xuileixe: the vcpu and pcpu is 1:1 pinning, but in the same cpu socket, they still share cpu cache08:08
ileixeI made them to be separated by using cpu_thread_policy08:08
ileixeto 'require'08:08
ileixeso I think no VMs share physical core for the test environment.08:09
ileixealex_xu: Have you ever expereince cpu pinning? Is it assure that VMs could not interfere each other?08:11
ileixe(if there are only cpu loads)08:12
lyarwood# git review08:13
lyarwoodCould not connect to gerrit.08:13
lyarwoodhmm anyone seeing this or have I just borked something locally?08:13
*** tkajinam has quit IRC08:14
lyarwood`RSA host key for review.openstack.org has changed and you have requested strict checking.` - nope it's remote08:15
*** ykarel is now known as ykarel|lunch08:16
*** tobias-urdin has joined #openstack-nova08:19
*** tssurya has joined #openstack-nova08:20
alex_xuileixe: i only can suggestion you to check whether is system process running on the those pinned pcpu. and setting the emulator thread policy ensure the emulator thread not running on those pcpu.08:21
*** janki has quit IRC08:23
ileixealex_xu: Hm.. I double checked libvirt configuration and PID/stats at proc interface, and it was expected. Thanks anyway.08:23
alex_xuileixe: np, good luck08:25
*** frankwang has quit IRC08:30
*** frankwang has joined #openstack-nova08:31
*** ttsiouts has quit IRC08:32
*** panda|rover has quit IRC08:40
*** panda has joined #openstack-nova08:42
*** derekh has joined #openstack-nova08:46
*** jangutter has joined #openstack-nova08:54
*** ttsiouts has joined #openstack-nova09:03
*** markvoelker has joined #openstack-nova09:04
*** panda is now known as panda|rover09:11
openstackgerritStephen Finucane proposed openstack/nova master: Remove 'ComputeManager._reschedule'  https://review.opendev.org/66077409:15
openstackgerritStephen Finucane proposed openstack/nova master: Remove cells v1 parameter from 'ComputeTaskAPI.resize_instance'  https://review.opendev.org/65131109:15
openstackgerritStephen Finucane proposed openstack/nova master: Stop passing 'kwargs' to 'rebuild_instance'  https://review.opendev.org/65131209:15
openstackgerritStephen Finucane proposed openstack/nova master: Stop passing 'delete_type' to 'terminate_instance'  https://review.opendev.org/65131309:15
openstackgerritStephen Finucane proposed openstack/nova master: filters: Stop handling cells v1  https://review.opendev.org/65131409:15
openstackgerritStephen Finucane proposed openstack/nova master: Remove nova.compute.*API() shims  https://review.opendev.org/66052709:15
openstackgerritStephen Finucane proposed openstack/nova master: Ensure controllers all call super  https://review.opendev.org/66095009:15
*** ricolin has quit IRC09:15
*** luksky has joined #openstack-nova09:29
*** ykarel|lunch is now known as ykarel09:34
*** ttsiouts has quit IRC09:37
*** markvoelker has quit IRC09:38
kashyapstephenfin: Hi09:49
kashyapstephenfin: I want to submit this doc to one of the upstream Nova admin guides -- which guide do you suggest: https://kashyapc.fedorapeople.org/Nova-Mitigation-for-Intel-MDS-Flaws.txt09:50
stephenfinkashyap: probably the reference guide (doc/source/reference)09:52
*** derekh has quit IRC09:54
*** derekh has joined #openstack-nova09:54
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove global state from the FakeDriver  https://review.opendev.org/65670909:58
openstackgerritBalazs Gibizer proposed openstack/nova master: Enhance service restart in functional env  https://review.opendev.org/51255209:59
openstackgerritBalazs Gibizer proposed openstack/nova master: Reproduce bug 1724172 in the functional test env  https://review.opendev.org/51255310:02
openstackbug 1724172 in OpenStack Compute (nova) "Allocation of an evacuated instance is not cleaned on the source host if instance is not defined on the hypervisor" [Medium,In progress] https://launchpad.net/bugs/1724172 - Assigned to Balazs Gibizer (balazs-gibizer)10:02
openstackgerritBalazs Gibizer proposed openstack/nova master: cleanup evacuated instances not on hypervisor  https://review.opendev.org/51262310:02
openstackgerritArnaud Morin proposed openstack/nova master: Force refresh instance network info on deletion  https://review.opendev.org/66076110:02
*** cdent has joined #openstack-nova10:03
*** sridharg has joined #openstack-nova10:14
*** frankwang has quit IRC10:15
*** guozijn has quit IRC10:23
*** guozijn has joined #openstack-nova10:26
*** guozijn has quit IRC10:32
*** markvoelker has joined #openstack-nova10:35
*** sapd1_x has joined #openstack-nova10:37
*** ttsiouts has joined #openstack-nova10:40
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove global state from the FakeDriver  https://review.opendev.org/65670910:44
openstackgerritBalazs Gibizer proposed openstack/nova master: Enhance service restart in functional env  https://review.opendev.org/51255210:44
openstackgerritBalazs Gibizer proposed openstack/nova master: Reproduce bug 1724172 in the functional test env  https://review.opendev.org/51255310:44
openstackbug 1724172 in OpenStack Compute (nova) "Allocation of an evacuated instance is not cleaned on the source host if instance is not defined on the hypervisor" [Medium,In progress] https://launchpad.net/bugs/1724172 - Assigned to Balazs Gibizer (balazs-gibizer)10:44
openstackgerritBalazs Gibizer proposed openstack/nova master: cleanup evacuated instances not on hypervisor  https://review.opendev.org/51262310:44
*** bbowen has quit IRC10:50
*** ttsiouts has quit IRC10:59
*** ttsiouts has joined #openstack-nova11:00
openstackgerritBalazs Gibizer proposed openstack/nova master: Do not start nova-network in the notification func test  https://review.opendev.org/66097511:06
*** markvoelker has quit IRC11:08
*** davidsha has joined #openstack-nova11:09
*** nicolasbock has joined #openstack-nova11:11
*** janki has joined #openstack-nova11:11
*** udesale has quit IRC11:13
*** udesale has joined #openstack-nova11:13
*** ttsiouts has quit IRC11:14
*** hrw has joined #openstack-nova11:22
hrwmorning11:22
*** sridharg has quit IRC11:25
*** dave-mccowan has joined #openstack-nova11:31
*** eharney has quit IRC11:34
*** jaosorior has joined #openstack-nova11:38
openstackgerritLee Yarwood proposed openstack/nova master: Use os-brick locking for volume attach and detach  https://review.opendev.org/61419011:38
openstackgerritHamdy Khader proposed openstack/os-vif master: OVS DPDK port representors support  https://review.opendev.org/65878611:43
*** jaosorior has quit IRC11:49
*** _erlon_ has joined #openstack-nova11:54
*** bbowen has joined #openstack-nova11:57
*** hrw has left #openstack-nova11:59
*** panda|rover is now known as panda|rover|eat11:59
*** markvoelker has joined #openstack-nova11:59
openstackgerritArnaud Morin proposed openstack/nova master: Use force_refresh=True on network info update  https://review.opendev.org/66098211:59
*** Alon_KS has joined #openstack-nova12:00
*** bbowen_ has joined #openstack-nova12:03
*** hemna has quit IRC12:04
*** janki has quit IRC12:04
*** janki has joined #openstack-nova12:05
*** bbowen has quit IRC12:05
*** dpawlik has quit IRC12:06
*** dpawlik has joined #openstack-nova12:10
*** dpawlik has quit IRC12:12
*** dpawlik has joined #openstack-nova12:13
*** edleafe has joined #openstack-nova12:16
*** Sundar has joined #openstack-nova12:17
*** Sundar has quit IRC12:17
*** eharney has joined #openstack-nova12:18
*** ttsiouts has joined #openstack-nova12:28
*** luksky has quit IRC12:33
*** brinzhang has quit IRC12:36
*** brinzhang has joined #openstack-nova12:36
*** ratailor has quit IRC12:47
*** ykarel is now known as ykarel|afk12:48
openstackgerritAdam Spiers proposed openstack/nova master: Move patch_exists() to nova.test.TestCase for reuse  https://review.opendev.org/66050012:50
*** Luzi has quit IRC12:54
*** ttsiouts has quit IRC12:58
*** panda|rover|eat is now known as panda|rover13:00
*** mriedem has joined #openstack-nova13:03
*** brinzhang has quit IRC13:05
*** dave-mccowan has quit IRC13:12
*** ykarel|afk is now known as ykarel13:12
*** dave-mccowan has joined #openstack-nova13:13
kashyapstephenfin: Thx13:22
*** luksky has joined #openstack-nova13:25
efriedkashyap: Did you see my patch answering the "how do we support renamed arch-specific cpu flag traits" question?13:27
kashyapefried: Afraid, not yet.  I'm at a one-day conference in the Netherlands, so my connectivity is a bit spotty today.13:28
kashyapefried: Got the URL handy?13:28
efriedkashyap: https://review.opendev.org/#/c/660515/13:28
*** burt has joined #openstack-nova13:29
efriedkashyap: it's all red in the CI because it depends on your os-traits patch, but the dep chain doesn't work easily13:29
efriedkashyap: But once your patch is in an os-traits release, we put that in requirements.txt/lower-constraints.txt in this patch and it should go green.13:29
efriedanyway, point is it's not super hard to make it work, and we're probably going to have to do it eventually anyway, so might as well.13:30
efriedwhat this means to you is: no reason to hold off "moving" those traits in your patch.13:30
*** janki has quit IRC13:31
openstackgerritAdam Spiers proposed openstack/nova master: Move patch_exists() to nova.test.TestCase for reuse  https://review.opendev.org/66050013:31
*** tbachman has joined #openstack-nova13:32
aspiersefried: ^^^ this feels like the cleanest approach to me13:32
kashyapefried: Thank you ... "For backward compatibility reasons, we have to continue exposing those versions of those traits" :-)13:33
aspiersefried: I'm working on something similar for open()13:33
aspiersbut for now I need to go vote and get lunch. biab13:33
kashyapefried: So my patch, shall I go back to making it a new PS of the older PS to preserve history?  (Sorry for not doing that to begin with)13:33
kashyapOr shall I just mention it in this new one, saying: "please read here for history"?13:33
efriedI would like someone libvirt-y, like stephenfin and aspiers, to have a look at that solution and make sure it's not insane.13:34
efriedkashyap: I would like it all kept in the original patch please.13:34
* aspiers wonders if he merits being called libvirt-y13:34
efriedaspiers: Well, you were the last one to touch the CPU_TRAITS_MAPPING anyway :)13:34
aspierstrue13:34
aspiersreminder to self: voting and lunching means not being here :-/13:35
kashyapefried: Will keep it in the original (then you need to update your PS, too)13:35
efriedkashyap: Thanks. Yeah, at this point I've proven the concept; I'll just wait until your patch is merged and released and then update mine properely.13:35
efriedproperly13:35
efried(sean-k-mooney moment)13:35
kashyapefried: Yeah, noted.  Tomm I'm off the afternoon.  Can get to it only next week.  Since there's no urgency here, hope that's OK13:36
*** mfeder has joined #openstack-nova13:36
*** samueldmq has joined #openstack-nova13:36
* efried caffeinates más13:36
efriedkashyap: Sure, that's fine, no hurry from my end13:36
kashyapefried: Also you might want to have a gander at this (which I'll submit as a doc to Nova): https://kashyapc.fedorapeople.org/Nova-Mitigation-for-Intel-MDS-Flaws.txt13:36
openstackgerritDan Smith proposed openstack/nova master: Make nova-next archive using --before  https://review.opendev.org/66100213:37
efriedkashyap: lgtm13:39
efriedI'll have a few comments when it's up in gerrit13:39
efriedbut overall clear and concise13:39
kashyapefried: Thx; sure.  Posting a variant of it now...before I forget.13:40
kashyapAnything that improves clarity.13:40
mfederHi all,13:42
mfederIs there any option how to execute command as another user (linux way : "sudo -u user `command`") ?13:42
mfederI would like to access regular-user's key pair using admin user.13:42
mfederE.g.13:42
mfeder# as regular user :13:42
mfeder`openstack keypair create kp1`13:42
mfeder# as admin user:13:42
mfeder`openstack keypair show kp1 `13:42
mfederThanks for any hint !13:42
*** roukoswarf has quit IRC13:43
mfeder There seems to be an option in the nova API to specify a user ID for keypair requests: https://developer.openstack.org/api-ref/compute/?expanded=list-keypairs-detail,create-or-import-keypair-detail#keypairs-keypairs    But I don't know how to do this via `openstack` command13:44
*** Sundar has joined #openstack-nova13:45
*** ricolin has joined #openstack-nova13:49
*** tbachman has quit IRC13:49
*** awalende_ has joined #openstack-nova13:52
fricklermfeder: there's a patch proposed for that, so currently it isn't possible yet, see https://review.opendev.org/576507 . you may want to ask dtroyer or mordred in #openstack-sdks for help with getting this merged and released13:55
mfederfrickler: Thank you !13:56
*** awalende has quit IRC13:57
*** awalende_ has quit IRC13:57
*** ttsiouts has joined #openstack-nova14:02
*** mmethot has joined #openstack-nova14:05
*** mfeder has left #openstack-nova14:05
openstackgerritMerged openstack/nova stable/stein: Skip _exclude_nested_providers() if not nested  https://review.opendev.org/65920614:09
*** pcaruana has quit IRC14:10
*** sridharg has joined #openstack-nova14:12
*** mlavalle has joined #openstack-nova14:12
*** tbachman has joined #openstack-nova14:16
*** cmart has joined #openstack-nova14:21
*** dpawlik has quit IRC14:27
*** pcaruana has joined #openstack-nova14:29
*** yankcrime has quit IRC14:30
*** hongbin has joined #openstack-nova14:33
*** ttsiouts has quit IRC14:36
mordredfrickler: wow. that's 10 months old14:36
*** READ10 has joined #openstack-nova14:36
fricklermordred: yeah, progress in osc land is ... not fast14:37
efriedaspiers: +2, some tweaks if you feel so inclined.14:37
openstackgerritMerged openstack/python-novaclient master: Set the lower limit of api_version for volume_type  https://review.opendev.org/66086114:38
mriedemget'ya simple perf optimizations hey! https://review.opendev.org/#/c/651647/14:43
*** dave-mccowan has quit IRC14:52
*** dave-mccowan has joined #openstack-nova14:52
openstackgerritBrin Zhang proposed openstack/python-novaclient stable/stein: Set the lower limit of api_version for volume_type  https://review.opendev.org/66102414:54
*** sridharg has quit IRC14:56
openstackgerritAdam Spiers proposed openstack/nova master: Move patch_exists() to nova.test.TestCase for reuse  https://review.opendev.org/66050014:57
aspiersefried: ^^^14:57
aspiersdoh14:58
aspiersdocstring error14:58
openstackgerritAdam Spiers proposed openstack/nova master: Move patch_exists() to nova.test.TestCase for reuse  https://review.opendev.org/66050014:58
efriedyeah, you could even remove the word 'static' entirely to convey the meaning.14:59
efriedas the call doesn't change regardless14:59
aspierstrue14:59
aspierswell, it says "static method" now14:59
aspiersthat should be good enough14:59
efriedyup14:59
efriedstephenfin: easy and fun if you're so inclined --^15:00
aspiersI hadn't previously realised that @contextlib.contextmanager gives you something which can be both a decorator and a context manager. Pretty nice15:00
*** cfriesen has joined #openstack-nova15:02
efriedI didn't know that either. TIL15:05
*** ivve has quit IRC15:06
*** lpetrut has quit IRC15:08
*** luksky has quit IRC15:12
aspiersthis is comically wrong https://www.stackalytics.com/?metric=commits&release=train&project_type=all&module=nova15:13
aspiersefried: if there were only 16 commits so far this release, I guess you'd be feeling a lot more relaxed :)15:13
aspiersI've already reported the issue, BTW15:14
openstackgerritMatt Riedemann proposed openstack/nova master: Avoid unnecessary joins in InstanceGroup.get_hosts  https://review.opendev.org/66103215:20
mriedemanother easy optimization ^15:20
*** hongbin has quit IRC15:20
*** hongbin has joined #openstack-nova15:21
*** ccamacho has quit IRC15:22
efriedaspiers: Yup, saw the thread, thanks for reporting.15:26
*** itlinux has joined #openstack-nova15:26
mriedemmlavalle: sean-k-mooney: btw i fixed the revert cross-cell cold migration port binding thing from the other day - activating the source host binding before deleting the dest host binding did the trick as expected15:26
mlavallemriedem: good to know :-)15:29
*** priteau has joined #openstack-nova15:31
*** helenafm has quit IRC15:31
*** tssurya has quit IRC15:35
sean-k-mooneymriedem: cool good to know. i need to add that review serise back to my review queue15:35
sean-k-mooneymriedem:are you are only using the mulitple port binding  flow for cross cell by the way or will it now be ussed for all resizes/cold migrations too15:36
mriedemthe former15:36
sean-k-mooneyok15:36
sean-k-mooneyi guess we can expand it after in U to non cross cell migration too if it makes sense15:37
*** macza has joined #openstack-nova15:38
*** ykarel is now known as ykarel|away15:39
*** wwriverrat has quit IRC15:40
*** boxiang has joined #openstack-nova15:43
*** ttsiouts has joined #openstack-nova15:43
*** hamzy has quit IRC15:44
*** ttsiouts has quit IRC15:45
*** ttsiouts has joined #openstack-nova15:46
openstackgerritMerged openstack/nova-specs master: Train Cycle Themes  https://review.opendev.org/65717115:46
*** ykarel|away has quit IRC15:46
*** whoami-rajat has quit IRC15:50
*** rpittau is now known as rpittau|afk15:51
openstackgerritMatt Riedemann proposed openstack/nova master: libvirt: check job status for VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED event  https://review.opendev.org/59452715:58
*** itlinux has quit IRC16:00
*** boxiang has quit IRC16:01
*** davidsha has quit IRC16:05
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Enable cross-cell resize in the nova-multi-cell job  https://review.opendev.org/65665616:06
*** ykarel|away has joined #openstack-nova16:08
*** pcaruana has quit IRC16:09
*** cdent has quit IRC16:15
*** gyee has joined #openstack-nova16:16
*** ttsiouts has quit IRC16:16
*** ivve has joined #openstack-nova16:23
*** dpawlik has joined #openstack-nova16:25
stephenfinefried: I have a "remove-nova-consoleauth specless blueprint" topic on the agenda for the team meeting today but I won't be around (it's 10pm my time). If you've any questions, stick my nick in there and I'll revisit tomorrow16:34
stephenfin(*remove-consoleauth, fwiw)16:36
*** mgoddard has quit IRC16:38
*** mgoddard has joined #openstack-nova16:39
efriedstephenfin: roger that.16:43
efriedmain question was, have we gotten the all-clear from... whoever we thought might care?16:43
mriedemconsoleauth != console16:44
mriedemthe latter is xen specific16:44
mriedemthe former is not16:44
efriedo16:44
stephenfinwhat mriedem said16:45
*** zbr has quit IRC16:45
stephenfinthe only reason to keep consoleauth around was cells v1 so that should deem it no longer necessary16:46
stephenfinnova-console and nova-xvpvncproxy and on my chopping block too though but later, after nove-network16:46
stephenfinremove allll the code16:46
* stephenfin -> 🏡16:46
mriedemthere is also a migration involved in the consoleauth stuff, need to determine if that is done before we remove it, using a blocker migration or something perhaps,16:47
mriedemi know we have an upgrade check but not sure if it fails or warns16:47
*** ttsiouts has joined #openstack-nova16:48
melwittit warns. and I don't think there's a migration about it, just an upgrade status check to advise whether to enable the [workarounds] config option or not16:49
*** whoami-rajat has joined #openstack-nova16:51
mriedemok so workarounds.enable_consoleauth is already deprecated, that's good at least16:52
*** ttsiouts has quit IRC16:53
mriedemwould we change the upgrade check from warning to failure?16:53
mriedemit only warns if there are non-deleted, enabled nova-consoleauth service records in any cell and there are computes < rocky16:54
*** dtantsur is now known as dtantsur|afk16:54
*** sapd1_x has quit IRC16:54
mriedemthe latter shouldn't be the case by the time someone upgrades to train16:55
*** ricolin has quit IRC16:55
*** tjgresha has quit IRC16:55
*** tjgresha has joined #openstack-nova16:55
*** derekh has quit IRC17:00
*** pcaruana has joined #openstack-nova17:00
*** ykarel|away has quit IRC17:02
*** udesale has quit IRC17:03
melwittI guess if it would fail for the combo of both service records and computes < rocky, that seems like it would be ok. I wouldn't be sure about failing it for the service records alone, because I'm not sure operators definitely deleted the services after they stopped running nova-consoleauth17:07
aspiersefried: I've got the patch_open() helper ready now - should I submit that separately to switching stuff over to it?17:13
efriedaspiers: I don't see why17:13
aspiersso, altogether then?17:14
efriedIf you've identified instances where it can be used, yeah, fo sho.17:14
aspiersthere's a whole bunch of other stuff which could be converted to use patch_exists() too17:14
efriedokay17:14
efriedup to you, I'm fine either way.17:14
aspiershah, just found some other reimplementations17:15
aspiershttps://opendev.org/openstack/nova/src/branch/master/nova/tests/unit/virt/disk/mount/test_nbd.py17:17
*** dpawlik has quit IRC17:17
aspiersbut those conditions are more specific17:17
aspiershmm17:17
efriedone of them is. the other could be swapped out as it stands.17:19
efriedYou could juice up patch_exists so the path arg can be a str or a regex :)17:20
efriedthat would satisfy the other one.17:21
efriedoh, no it wouldn't.17:22
efriedaspiers: You could make it a str or a callable...17:22
efriedstarts to smell a lot like side_effect at that point17:22
*** ykarel has joined #openstack-nova17:24
*** ttsiouts has joined #openstack-nova17:26
*** dpawlik has joined #openstack-nova17:26
*** hemna has joined #openstack-nova17:26
aspiersefried: yeah, I was thinking of supporting callables, but ... scope creep, I dunno17:27
*** hamzy has joined #openstack-nova17:28
efriedaspiers: Feel free to push it to the wishlist for later. Add a `TODO(aspiers): figure out how to use patch_exists here` and move on.17:31
openstackgerritMerged openstack/nova stable/stein: Handle placement error during re-schedule  https://review.opendev.org/65760217:35
openstackgerritMerged openstack/nova stable/stein: Only call _fill_provider_mapping if claim succeeds  https://review.opendev.org/65760317:35
aspiersefried: good idea17:36
*** hamzy has quit IRC17:40
*** hamzy has joined #openstack-nova17:40
openstackgerritMerged openstack/nova stable/stein: Stop logging traceback when skipping quiesce  https://review.opendev.org/66032517:42
*** hamzy_ has joined #openstack-nova17:46
*** itlinux has joined #openstack-nova17:46
*** hamzy has quit IRC17:49
*** ralonsoh has quit IRC17:51
*** itlinux has quit IRC17:56
*** ttsiouts has quit IRC17:59
*** lpetrut has joined #openstack-nova18:03
*** lpetrut has quit IRC18:07
*** itlinux has joined #openstack-nova18:09
*** itlinux has quit IRC18:11
*** READ10 has quit IRC18:13
*** klindgren has quit IRC18:13
*** klindgren has joined #openstack-nova18:14
*** luksky has joined #openstack-nova18:20
*** hamzy__ has joined #openstack-nova18:26
*** hamzy_ has quit IRC18:26
gansolyarwood: Hi! Could you please take a look at this patch? https://review.opendev.org/#/c/659338 Thanks in advance!18:32
*** ykarel has quit IRC18:32
*** hamzy__ has quit IRC18:34
*** mriedem has quit IRC18:37
*** hamzy has joined #openstack-nova18:43
*** mriedem has joined #openstack-nova18:43
openstackgerritmelanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement  https://review.opendev.org/63807318:43
openstackgerritmelanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next  https://review.opendev.org/65314618:43
openstackgerritmelanie witt proposed openstack/nova master: Use instance mappings to count server group members  https://review.opendev.org/63832418:43
openstackgerritmelanie witt proposed openstack/nova master: Add documentation for counting quota usage from placement  https://review.opendev.org/65384518:43
*** priteau has quit IRC18:44
*** hamzy has quit IRC18:47
*** hamzy has joined #openstack-nova18:47
*** tasker has joined #openstack-nova19:02
taskerI'm running Pike and trying to turn the logging level down to WARNING ( instead of the default INFO ), and "verbose = False" no longer works. The documentation at pike/admin/manage-logs.html and pike/configuration/config.html provide non-matching information.19:04
taskerone ( admin ) says to use "log-config" while the other ( configuration ) says "log-config" is deprecated and that all other log items will be ignored if the new "log_config_append" is used.19:04
taskerwhat's the easiest way to turn Nova's output down to WARNING?19:04
*** hamzy has quit IRC19:05
*** tesseract has quit IRC19:05
*** ttsiouts has joined #openstack-nova19:09
*** aram1s has joined #openstack-nova19:10
*** hemna has quit IRC19:10
*** hamzy has joined #openstack-nova19:26
*** bbowen_ has quit IRC19:27
sean-k-mooneyefried: do you know what the policy on deprecating parts of config vaules is19:34
sean-k-mooneyefried: i want to deprecate and remove one of the ways of whitelisting device in the pci_white_list this cycle19:34
sean-k-mooneyefried: specificly the devicename parmater19:34
efriedthat's interesting.19:35
sean-k-mooneywould it need a blueprint and or spec? or jsut a bug ?19:35
efriedFor something like that, you would just have to note the deprecation in the help text of the opt and in a reno; and add a warning log to the code that parses it.19:35
efriedoh, you mean *paperwork*19:35
efrieddepends, is there a bug? :)19:36
sean-k-mooneyya so we woudl need to keep it around for train and log a warning if its used19:36
sean-k-mooneynot for this but there are bugs caused by that parmaater19:36
efriedI would think so, yes.19:36
sean-k-mooneythe issue is if your kernel is configured not to bind back the PFs/VFs to the nic driver e.g. they stay boudn to vfio19:37
sean-k-mooneythen if you restart the compute agent it cant dicover the VFs or PFs again19:37
sean-k-mooneybecause the PF/VF does not have a netdev and therfore you cant find it by name19:38
efriedAnd this is the best way to "fix" the bug? Deprecating/removing the ability to specify a device by name?19:38
sean-k-mooneybut also the name can somethime change after it rebound19:38
sean-k-mooneyefried: ya basically19:38
efriedare there not scenarios where identifying a device by name *doesn't* break?19:38
efried(s/not// for same meaning with fewer negatives)19:39
*** tssurya has joined #openstack-nova19:39
sean-k-mooneyit only works if the device is reliable bound back to the nic driver when nolonger used by the guest and the name does not change whne it rebound to the host19:39
sean-k-mooneyso it can work but its really fragile19:39
sean-k-mooneyusing vendor id and product id or useing the pci address are both more robost as they dont depned on teh driver or the semanitc of the nic vendor19:40
efriedsean-k-mooney: I guess what I would do in this case is write the patch and start a thread on the ML with your intent, soliciting opinions on whether this is okay to kill. Link the patch and ML thread from each other. IMO that's sufficient paperwork. And if someone thinks otherwise, you can always write a bug/bp later.19:40
efriedthough it sounds like what you're talking about actually *is* a bug, and this is just one possible remedy.19:41
sean-k-mooneyefried: ok sound good.19:41
efriedIn my ignorance, another remedy could be "blow up if devname is used for devices it has the potential to break on"19:41
sean-k-mooneyit depend on how you configre your driver at the kernel levle19:42
sean-k-mooneyand to some extent the nic vendor19:42
efriedno way to detect it, other than going all the way through until you hit the bug??19:42
*** ttsiouts has quit IRC19:42
sean-k-mooneyi belive intel nics prefer the networkign driver over vfio-pci but i think it depend on the model to some extend and your udev rules19:42
sean-k-mooneyefried: the main issue comes down to will udev/systemd chose to bind the driver to i40e or leave it bound to vfio-pci19:43
sean-k-mooneyso its not really a nova issue19:44
sean-k-mooneyits more a distro/packaging/install/confituion thing on the host19:44
sean-k-mooneywhich is what make it hard for nova to dectet and or warn about19:45
efriedtasker: Does19:45
efried[DEFAULT]19:45
efrieddefault_log_levels = nova=WARNING19:45
efriednot work?19:45
efriedsean-k-mooney: ack, and shrug. I hope we can soon deprecate *all* of [pci]passthrough_whitelist and [pci]alias.19:46
sean-k-mooneyefried: maybe in U19:46
sean-k-mooneyof course then we woudl have a cyborg whitelist and device profile19:46
sean-k-mooneybut then its there problem :)19:46
efriedheh19:46
sean-k-mooneyif we get the cyborg integration done fully this cycle that might be a nice thing to port over early next cycle19:47
* sean-k-mooney that is partly a comment to my self but i also dont want to sign up to it yet :)19:48
efriedsean-k-mooney: cyborg/fpga + providers.yaml we might have all we need.19:48
sean-k-mooneywell i was thinig more generic pci passhtough rather then smart nic19:48
sean-k-mooneye.g. replaceing all the uses of the pci tracker in nova with a very basic generic pci driver in cyborg19:49
sean-k-mooneybut making that work with hardare offload ovs and all the other legacy things we have would be a lot of work19:50
sean-k-mooneymanyly on the nova side to make it transparent19:50
sean-k-mooneyefried: anyway thanks ill send an email to the list19:51
efriedcoo19:51
taskerefried: did not. I still see a bunch of "HEAD / => generated 0 bytes"20:02
efriedtasker: Coming from nova?20:02
taskeryes. specifically the "nova-api-os-compute" container ( if that distinction matters. )20:03
efriedtasker: Can you paste me the entire log line please?20:04
efriedI'm thinking that might be coming from the wsgi layer20:04
taskerhttp://dpaste.com/2C5P2PY20:04
taskerI concur. to complicate things, I don't know how to turn that thing down, either.20:05
efriedin which case you may wish to try adding ,eventlet.wsgi.server=WARNING20:05
efriedI think the left-hand-side of that '=' is a substring match. So if you say foo.bar=WARNING then log lines coming from foo.bar and foo.bar.baz and foo.bar.etc will all get filtered at a WARNING level.20:06
efriedhmph, no indication in that line where it's coming from. Is there a log line template that will add that? Looking...20:08
efriedtasker: Looks like you might be able to add %(pathname)s to your logging_default_format_string to get a clue where that line is being generated from; then use the prefix trick above to suppress it.20:09
taskerefried: did not work. "%(pathname)s" was applied to another set of messages, but not the one I'm trying to filter out.20:17
efriedinteresting20:18
taskerwait .. scratch that. it wasn't applied at all.20:18
*** hemna has joined #openstack-nova20:18
efriedassume you're restarting services each time20:19
taskeryup.20:19
taskerhttp://dpaste.com/1ZSB78D -- what the "wsgi" logs look like from "nova-api-wsgi.log" when all testing options are removed. this is the original behaviour.20:19
efriedguess you could try adding the pathname to logging_debug_format_suffix20:20
taskerwhen I set anything to the config file, the lines in nova-api-wsgi.log stop, but not to nova-api-os-compute.log20:20
taskerthe latter file contains lines of the first paste I sent you.20:20
efriedhmph. I'm afraid I'm out of ideas.20:21
efriedCourse, I'm not what you'd call an "operator".20:22
taskerhehe. can we bring back "verbose = False"?20:22
taskerefried: I appreciate you taking the time to help me out. You've shed some light on some things I did not know and given me a lot of good information to play around with.20:23
efriedyou're welcome, sorry I couldn't be more help.20:23
taskerI did match the "pid" in the log file to the PIDs of the uWSGI helpers that spawn whenever the nova service starts.20:30
taskerthat lends weight to it being the WSGI module(s).20:30
taskerare the modules using the same file independent of Nova?20:31
*** whoami-rajat has quit IRC20:31
*** aram1s has quit IRC20:31
efriedtasker: You're out of my wheelhouse at this point. Throwing it around earlier like I knew what I was talking about, but I can barely spell WSGI.20:32
taskerit's alright. those were mostly musings. I tend to "stream of conciousness" debug and use any open chat window to get ideas out.20:33
efriedheh, wfm.20:33
efriedtasker: I've got a guy who might know a bit more. He's on his way into the channel, if you're going to hang around for a bit.20:34
*** donnyd has joined #openstack-nova20:34
taskeryeah. thanks!20:34
*** dpawlik has quit IRC20:34
efriedo/ donnyd20:34
efrieddonnyd: meet tasker. tasker: likewise20:35
donnydHi20:35
efrieddonnyd has actual experience deploying openstack20:35
taskerdonnyd: helo!20:35
donnydI'm just getting caught up on what the issue is20:35
taskertrying to turn down all logging in Nova to WARNING because I'm getting these messages that indicate the API endpoints are hit at ( what I guess is INFO ) level. efried was helping and about all we can do is determine that they're coming from WSGI.20:36
donnydDo you have an example of the log entry you don't want to see?20:37
*** itlinux has joined #openstack-nova20:37
taskerhttp://dpaste.com/2C5P2PY20:37
taskerthis is Pike, and prior to it, I could set "verbose = False" in nova.conf and call it a day.20:39
*** bbowen has joined #openstack-nova20:39
donnydNext question is what deployment method did you use?20:39
taskerOSA20:39
*** slaweq has quit IRC20:41
donnydThat surely looks like a wsgi log to me. (I see the same log entry type in my ister server).20:42
taskerknow if there's a comparable "verbose = False" flag for uWSGI?20:43
taskertheir docs don't hint at such.20:43
*** pcaruana has quit IRC20:47
openstackgerritMatt Riedemann proposed openstack/nova master: Link versioned notification talk into docs  https://review.opendev.org/66111520:50
taskerappending "--disable-logging" to the uwsgi ExecStart string in the service file killed the messages, but that's too heavy-handed.20:53
donnydStill searching20:55
*** ttsiouts has joined #openstack-nova20:56
taskerand having "--disable-logging" and having Nova at "debug_log_levels = nova=INFO20:56
tasker" I can still get Nova's notices of API hits.20:56
openstackgerritMerged openstack/nova master: refactor nova-manage archive_deleted_rows  https://review.opendev.org/64377920:57
taskerI'm stepping out for some "fresh air". back in a few.20:58
efriedNova meeting in 2 minutes in #openstack-meeting20:58
*** nicolasbock has quit IRC20:58
*** nicolasbock has joined #openstack-nova20:59
efriedpawing through uwsgi docs, I'm not actually finding any hints that they *have* different log levels.21:00
efriedI think it's just on or off.21:00
efriedin which case your heavy-handed solution might be the only solution, beyond `grep` :P21:01
donnydThats what it looks like to me @efried21:01
efriedtasker: Is there anything in nova-api-os-compute.log that you actually care about?21:02
openstackgerritmelanie witt proposed openstack/nova stable/rocky: [stable-only] Improve the reno for consoleauth upgrade check  https://review.opendev.org/66111821:03
*** Sundar has quit IRC21:06
taskerefried: errors and warnings.  . )   what are you thinking?21:07
efriedtasker: I was just wondering whether you could filter the whole thing out :)21:09
openstackgerritMerged openstack/nova master: Use InstanceList.get_count_by_hosts when deleting a compute service  https://review.opendev.org/65164721:10
donnydhttps://github.com/unbit/uwsgi-docs/blob/master/ManagementFlag.rst21:10
*** itlinux has quit IRC21:10
donnyd0loggingenable/disable logging21:10
donnydLooks like its mostly an off or on kinda thing21:10
*** itlinux has joined #openstack-nova21:11
taskerefried: donnyd: thank you both very much for your assistance. with your help, I think I have a solution that will work. I'll be leaving it overnight and trying to trigger various warnings and events to verify it does what I need.21:15
efriedgood luck tasker21:15
*** Sundar has joined #openstack-nova21:18
*** eharney has quit IRC21:20
*** ttsiouts has quit IRC21:26
sean-k-mooneyon the nova console auth topic kolla-ansibel still defualt to deploying it on master21:34
efriedwhee21:35
openstackgerritmelanie witt proposed openstack/nova stable/stein: [stable-only] Improve the reno for consoleauth upgrade check  https://review.opendev.org/66112321:37
sean-k-mooneyosa looks like id does not but i only quickly looked21:38
openstackgerritmelanie witt proposed openstack/nova stable/rocky: [stable-only] Improve the reno for consoleauth upgrade check  https://review.opendev.org/66111821:39
sean-k-mooneyand tripleo remvoed it 14 days ago https://github.com/openstack/tripleo-heat-templates/blob/c33f8cb1174146e3cbdca6241914a78a623ea505/releasenotes/notes/nova_remove_nova-consoleauth-227cbeb44e9ab5ef.yaml21:39
melwittnice detective work :)21:41
*** ttsiouts has joined #openstack-nova21:42
sean-k-mooneymelwitt: what was the console auth service used for with cells v1 again21:42
melwittall console token auth. for cells v1 the database backend stuff was _not_ implemented21:43
sean-k-mooneyah right so its jsust issueing the tokens to connect to the websockets?21:44
mriedemosa is done https://github.com/openstack/openstack-ansible-os_nova/commit/1df0bd51ee5031039d3852cefbb63162e29b22a521:44
melwittsean-k-mooney: console auth service? yeah it stores the token auths locally and responds yea/nay when a request to connect to console proxy comes in21:45
melwittand the change we did for non-cells v1 was store the token auths in the cell databases and have console proxies (websockets) run per cell so they can query the database to validate token auths21:46
sean-k-mooney melwitt ok so looking at the patches to remove it in osa and triplo if we just dont deploy it will it work with nova today?21:46
sean-k-mooneyi dont see any config cahnge in those patches so it seam like nova just does the right thing now?21:47
melwittsean-k-mooney: yeah, the default behavior is to use the database as the console token auth storage. so running it is not necessary. unless you're running cells v121:47
sean-k-mooneywhich kolla never supported21:47
sean-k-mooneyso i can write a quick patch to delete the container form kolla and remove it form the kolla-ansibel playbook if that is all thats needed21:48
sean-k-mooneywell the upgade playbook would also need to stop any running containers21:48
sean-k-mooneybut that seams trivial too21:49
melwittI don't know if this is relevant at this point but during a live, rolling upgrade from queens to rocky, you would need to keep nova-consoleauth running until everything is rocky21:50
melwittif you were not OK with disrupting existing console sessions21:50
sean-k-mooneylooks like https://bugs.launchpad.net/kolla-ansible/+bug/1804474 is open but i am gessing its not been worked on21:50
openstackLaunchpad bug 1804474 in kolla-ansible "nova-consoleauth is deprecated" [Undecided,In progress] - Assigned to caoyuan (cao-yuan)21:50
sean-k-mooneymelwitt: well if i just remove it in the train branch that should not be an issue right21:51
melwittyeah should not be21:51
sean-k-mooneye.g. going form stien to train it wont be used anyway21:51
sean-k-mooneyand it  woudl be safe to backprot technically to rock->stien too21:51
sean-k-mooney... that said im not ment to be starting new things until i finish some of my curent in flight tasks.21:53
* sean-k-mooney shakes fist at kanban board :)21:53
melwitthah21:53
*** tasker has quit IRC21:57
*** ivve has quit IRC21:59
*** Sundar has quit IRC22:12
*** mriedem has quit IRC22:13
openstackgerritMerged openstack/nova master: Do not start nova-network in the notification func test  https://review.opendev.org/66097522:30
openstackgerritMatt Riedemann proposed openstack/nova master: Link versioned notification talk into docs  https://review.opendev.org/66111522:34
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Enable cross-cell resize in the nova-multi-cell job  https://review.opendev.org/65665622:36
*** mlavalle has quit IRC22:47
*** tkajinam has joined #openstack-nova22:57
*** panda|rover has quit IRC22:59
*** panda has joined #openstack-nova22:59
*** macza has quit IRC23:03
*** itlinux has quit IRC23:04
*** sapd1_x has joined #openstack-nova23:06
*** rcernin has joined #openstack-nova23:18
*** _erlon_ has quit IRC23:24
*** itlinux has joined #openstack-nova23:27
*** tssurya has quit IRC23:38
*** ttsiouts has quit IRC23:42
*** trident has quit IRC23:51
*** hongbin has quit IRC23:52
*** trident has joined #openstack-nova23:53
openstackgerritMerged openstack/python-novaclient stable/stein: Set the lower limit of api_version for volume_type  https://review.opendev.org/66102423:57
*** ttsiouts has joined #openstack-nova23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!