Thursday, 2020-06-04

*** hamalq has quit IRC00:01
*** _mlavalle_1 has joined #openstack-nova00:19
*** tetsuro has joined #openstack-nova00:20
*** mlavalle has quit IRC00:23
*** brinzhang has joined #openstack-nova00:29
*** tbachman has quit IRC00:39
*** tbachman has joined #openstack-nova00:40
*** spatel has quit IRC00:43
*** Liang__ has joined #openstack-nova01:08
*** Liang__ is now known as LiangFang01:09
*** brinzhang_ has joined #openstack-nova01:10
*** brinzhang has quit IRC01:13
*** tbachman has quit IRC01:16
*** tbachman has joined #openstack-nova01:18
*** xiaolin has joined #openstack-nova01:25
*** gyee has quit IRC01:26
*** rcernin has quit IRC01:27
*** rcernin has joined #openstack-nova01:30
*** brinzhang_ has quit IRC01:32
*** brinzhang_ has joined #openstack-nova01:32
*** brinzhang0 has joined #openstack-nova01:44
*** brinzhang_ has quit IRC01:47
*** _mlavalle_1 has quit IRC01:51
*** xiaolin has quit IRC01:53
*** boxiang has joined #openstack-nova02:33
*** boxiang_ has quit IRC02:35
*** rcernin has quit IRC02:37
*** brinzhang_ has joined #openstack-nova02:50
*** brinzhang0 has quit IRC02:53
*** brinzhang0 has joined #openstack-nova03:02
*** brinzhang_ has quit IRC03:05
*** sapd1 has joined #openstack-nova03:11
*** xiaolin has joined #openstack-nova03:25
*** rcernin has joined #openstack-nova03:37
*** ksk_ has quit IRC03:48
*** ociuhandu has joined #openstack-nova03:52
*** ociuhandu has quit IRC03:57
*** huaqiang has joined #openstack-nova04:04
*** LiangFang has quit IRC04:04
*** Liang__ has joined #openstack-nova04:05
*** brinzhang_ has joined #openstack-nova04:18
*** huaqiang has quit IRC04:22
*** brinzhang0 has quit IRC04:22
*** huaqiang has joined #openstack-nova04:22
*** sapd1 has quit IRC04:22
*** Liang__ has quit IRC04:23
*** Liang__ has joined #openstack-nova04:23
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-nova04:33
*** sapd1 has joined #openstack-nova04:37
*** lbragstad has quit IRC04:59
*** ratailor has joined #openstack-nova05:06
*** Liang__ has quit IRC05:15
*** Liang__ has joined #openstack-nova05:16
*** udesale has joined #openstack-nova05:36
*** links has joined #openstack-nova05:50
*** vishalmanchanda has joined #openstack-nova05:51
*** josephillips has quit IRC06:11
*** belmoreira has joined #openstack-nova06:12
*** dklyle has quit IRC06:13
*** tetsuro has quit IRC06:19
*** tetsuro has joined #openstack-nova06:19
*** tetsuro has quit IRC06:36
*** redrobot has quit IRC06:39
*** ratailor has quit IRC06:43
*** ratailor has joined #openstack-nova06:44
*** nightmare_unreal has joined #openstack-nova06:47
*** boxiang_ has joined #openstack-nova06:54
*** boxiang has quit IRC06:58
*** xek has joined #openstack-nova06:58
*** jaosorior has quit IRC07:00
*** jaosorior has joined #openstack-nova07:06
*** Liang__ has quit IRC07:12
*** ttsiouts has joined #openstack-nova07:12
*** ccamacho has joined #openstack-nova07:14
*** rcernin has quit IRC07:15
*** tesseract has joined #openstack-nova07:16
*** maciejjozefczyk_ has joined #openstack-nova07:17
*** rcernin has joined #openstack-nova07:20
*** jsuchome has joined #openstack-nova07:20
*** maciejjozefczyk has quit IRC07:20
*** rcernin has quit IRC07:21
*** rcernin has joined #openstack-nova07:21
*** xek has quit IRC07:26
*** xek_ has joined #openstack-nova07:26
*** slaweq has joined #openstack-nova07:28
*** sapd1 has quit IRC07:29
*** xinranwang has quit IRC07:30
*** ralonsoh has joined #openstack-nova07:32
*** ociuhandu has joined #openstack-nova07:40
*** rcernin has quit IRC07:41
*** xek__ has joined #openstack-nova07:45
*** tosky has joined #openstack-nova07:46
*** ttsiouts has quit IRC07:46
*** ttsiouts has joined #openstack-nova07:47
*** jawad_axd has joined #openstack-nova07:48
*** xek_ has quit IRC07:48
*** rpittau|afk is now known as rpittau07:50
*** jmlowe has quit IRC07:54
*** jmlowe has joined #openstack-nova07:55
*** ttsiouts has quit IRC08:00
*** avolkov has joined #openstack-nova08:03
*** martinkennelly has joined #openstack-nova08:05
*** ttsiouts has joined #openstack-nova08:11
*** ociuhandu_ has joined #openstack-nova08:12
*** dtantsur|afk is now known as dtantsur08:13
*** ociuhandu has quit IRC08:13
*** ociuhandu_ has quit IRC08:19
*** xiaolin has quit IRC08:20
*** links has quit IRC08:26
*** ociuhandu has joined #openstack-nova08:28
*** boxiang has joined #openstack-nova08:30
*** boxiang_ has quit IRC08:33
*** ociuhandu has quit IRC08:38
*** ociuhandu has joined #openstack-nova08:39
bauzasgood morning Nova08:41
gibibauzas: good morning08:47
*** links has joined #openstack-nova08:48
*** tkajinam has quit IRC08:48
*** ociuhandu has quit IRC08:49
*** ociuhandu has joined #openstack-nova08:50
*** ociuhandu has quit IRC08:51
*** ociuhandu has joined #openstack-nova08:51
* bauzas wraps his head to understand the routed networks story08:52
*** ociuhandu has quit IRC08:58
*** priteau has joined #openstack-nova08:58
*** martinkennelly has quit IRC09:00
*** xiaolin has joined #openstack-nova09:03
xiaolinhello guys, I'm trying to run openstack on a host with MIPS architecture, but got an error "Instance failed to spawn: libvirtError: internal error: unexpected host-model CPU for mips64el architecture" while creating an instance.09:04
xiaolinAnyone can guide me how I can contribute a fix to upstream?  need I report a bug first?09:04
*** links has quit IRC09:09
*** links has joined #openstack-nova09:13
gibixiaolin: does this similar to what belmoreira requested here? https://bugs.launchpad.net/nova/+bug/186372809:18
openstackLaunchpad bug 1863728 in OpenStack Compute (nova) "Nova can't create instances for a different arch" [Wishlist,Invalid]09:18
gibixiaolin: we will discuss ^^ today during the PTG https://etherpad.opendev.org/p/nova-victoria-ptg09:19
*** Liang__ has joined #openstack-nova09:25
xiaolingibi: The problem I encountered is not similar to  what belmoreira requested09:27
xiaolingibi: the host and instance are both MIPS architecture09:28
alex_xugibi: sean-k-mooney does neutron already report physical network interface resource provider? accroding to the spec https://specs.openstack.org/openstack/neutron-specs/specs/rocky/minimum-bandwidth-allocation-placement-api.html09:28
* alex_xu sorry for inject the discussion09:29
gibixiaolin: I see. Then I don't have a direct answer. the libvirt error suggest that something is missing from the libvirt side. Are you able to start a VM purely with libvirt / virsh on your host?09:30
belmoreiragibi xiaolin unfortunately I won't be able to join09:30
gibibelmoreira: thanks for the headsup, we will discuss it anyhow09:31
gibialex_xu: neutron reports SRIOV PF and OVS bridge as RP in placement with physnet and vnic type trait09:31
alex_xugibi: got it09:31
gibialex_xu: but only if bandwidth is defined in the agent conf for these devices09:32
alex_xugibi: just read the cyborg sriov doc, begin to think about..cyborg will report any RP for PF...09:32
alex_xus/report any/report another/09:32
xiaolingibi: Yes, I can start a VM purely with libvirt / virsh on my host09:32
gibialex_xu: I guess we have to decide if neutron or cyborg owns the PF09:33
alex_xugibi: ok, if they are totally two feature, should be ok. Since those cyborg PF's RP should be managed by cyborg...not sure how cyborg and neutron merge them...09:33
alex_xugibi: yea, that is something to decide09:33
gibixiaolin: please look at the host-model documentation https://docs.openstack.org/nova/latest/admin/configuration/hypervisor-kvm.html#specify-the-cpu-model-of-kvm-guests you might need to configure the nova a bit differently for using your hardware.09:36
gibixiaolin: also I hope kashyap could help from libvirt perspective09:37
*** udesale has quit IRC09:39
xiaolingibi: Thank you. I will read the documentation09:41
*** udesale has joined #openstack-nova09:49
*** songwenping_ has joined #openstack-nova09:53
*** ikla has quit IRC09:56
*** ircuser-1 has quit IRC09:56
*** songwenping__ has quit IRC09:57
*** dpawlik has quit IRC09:57
*** kevinz has quit IRC09:57
*** dpawlik3 has joined #openstack-nova09:57
*** rpittau is now known as rpittau|bbl10:02
*** dpawlik3 has quit IRC10:05
*** dpawlik3 has joined #openstack-nova10:05
*** derekh has joined #openstack-nova10:08
* kashyap blinks and reads the scroll10:11
kashyapxiaolin: Oh, MIPS ... you are the first I've come across here :)10:12
kashyapgibi: xiaolin: So the error "unexpected host-model" simply means, MIPS in libvirt doesn't have the notion of 'host-model'10:12
kashyapSo, let me check what's possible there...10:13
kashyapxiaolin: What you could try is: launch your `qemu-system-mips -cpu help` (or whatever the MIPS QEMU binary is called)10:15
kashyapxiaolin: On the Compute host, and configure that as an explicit CPU model in Nova:10:15
gibikashyap: thanks for the help :)10:16
kashyap    [libvirt]10:16
kashyap    cpu_mode = custom10:16
kashyap    cpu_model = $My_MIPS_CPU_FIXME10:16
kashyapgibi: Happy to be useful :)10:17
* kashyap goes back to kitchen to finish up the meal10:17
* gibi realizes that it is lunchtime10:19
*** hemna has quit IRC10:26
*** Liang__ has quit IRC10:31
sean-k-mooneyalex_xu: they way the cyborg driver work there is no filtering so yes it would report any PF that the driver could manage10:32
*** hemna has joined #openstack-nova10:33
sean-k-mooneygibi: today nova owns the pf not neutron10:33
sean-k-mooneyneutron only owns the bandwidth10:33
sean-k-mooneywhen we start reporting pci device to placement form nova we will be the ones creating the inventories for the VFs10:34
*** breizhkoala has joined #openstack-nova10:43
gibisean-k-mooney: today neutron creates the PF RPs and reports bandiwdth inventory on it, when we start reporting VF inventory from nova we need to agree with neutron who creates the PF RP or we have to create two RPs for a PF one from neutron perspective and one from nova perspective and then connect them somehow10:43
sean-k-mooneygibi: nova need to do it because it needs to be consitient with non nic pci devices10:44
*** ociuhandu has joined #openstack-nova10:44
sean-k-mooneyunless we are going to specal case when the device has a physnet tag on the pci whitelist10:44
sean-k-mooneywhich we coudl do but then we need two different code paths10:44
sean-k-mooneygibi: ideally it would end up being one RP10:45
sean-k-mooneyunless we extend placemets data model with a way to link rps10:45
sean-k-mooneygibi: quick question do you want a blueprint or a spec to track the machine type recodeing work?10:54
sean-k-mooneyas part of that i would like to start reporing avialble machine types as traits to placemnt and the schdlueling on that10:54
sean-k-mooneye.g. if we do a move opperation on an exsting instance i want to ensure we land it on a host that supports that machine type10:55
sean-k-mooneyand if we are creating a new instance and you request a machine type in the image i want to ensure we land on a host that supports that too10:56
sean-k-mooneyboth of which would be done by a prefilter10:56
sean-k-mooneyjust addign the required tratit based on the instance system metadata or image metdata10:57
gibisean-k-mooney: sorry I was lunching.11:00
sean-k-mooneyno worries11:00
*** ociuhandu has quit IRC11:00
*** songwenping__ has joined #openstack-nova11:00
gibisean-k-mooney: agree that it would be nice to have on RP for one physical entity (like a PF or other PCI device)11:01
gibisean-k-mooney: I just wanted to note that this might requires changes both from nova and neutron11:01
gibibtw placement has two ways today to link RPs, one is the parent-child relationship and the other is aggregates11:01
sean-k-mooneyyep altought i was thinking more of something like a symlink11:02
gibisean-k-mooney: I guess you don't need db schema rpc or REST API changes for the machine type recording work so I think a bp is enough11:03
sean-k-mooneyaggreate wont work because we can do member of in this case11:03
gibisean-k-mooney: yeah aggregates would be a bit ugly I agree11:03
*** songwenping_ has quit IRC11:03
sean-k-mooneyand parent-child likely wont work because we neither nova or neutron knows the uuid/name that will be used for the RP11:03
sean-k-mooneythat said we can work this out when we need too11:04
*** ttsiouts has quit IRC11:05
sean-k-mooneygibi: are you ok with also doing the traits reporting and prefileter in the same specless blueprint.11:05
sean-k-mooneyill write it up and i guess we can decide then11:06
*** mgariepy has quit IRC11:06
sean-k-mooneyi assume a specless blueprint is also fine for the doc change to add the common profiles(nfv,realtime,modren) that we talked about too11:07
sean-k-mooneyim going to try and get those up before we resume today11:07
*** udesale_ has joined #openstack-nova11:08
gibisean-k-mooney: the prefilter work is a bit of a grey area as I guess that needs a new config flag to enable since old computes will not report the traits11:11
*** udesale has quit IRC11:11
gibiso you might only want to enable the prefilter after all your computes are on V11:11
sean-k-mooneyyes although we could auto enable based on the min compute service version too.11:12
gibisean-k-mooney: specless bp for the hw:profile doc is totally OK11:12
gibisean-k-mooney: OK, note this upgrade wrinkel in the specless bp then I'm OK11:12
sean-k-mooneysure will do. if i end up having to write a small spec its not the end of the world either but we will see what other think once its up11:13
gibisean-k-mooney: we will bring this bp up in the next nova meeting for approval so the rest of the team can complain if they need more words about it11:13
sean-k-mooneysure works for me11:13
gibiawesome11:13
gibithanks11:13
*** martinkennelly has joined #openstack-nova11:18
*** brinzhang_ has quit IRC11:20
*** ravsingh has joined #openstack-nova11:21
*** xek__ has quit IRC11:25
*** donnyd_ has quit IRC11:33
*** donnyd_ has joined #openstack-nova11:34
*** donnyd_ has quit IRC11:34
*** rpittau|bbl is now known as rpittau11:35
*** donnyd_ has joined #openstack-nova11:36
*** TobbeCN has joined #openstack-nova11:37
*** songwenping_ has joined #openstack-nova11:38
*** songwenping__ has quit IRC11:41
*** donnyd_ has joined #openstack-nova11:42
*** donnyd_ is now known as donnyd11:44
openstackgerritBalazs Gibizer proposed openstack/nova-specs master: Make Feature Liaison optional  https://review.opendev.org/73063811:48
*** martinkennelly has quit IRC11:49
*** martinkennelly has joined #openstack-nova11:52
*** raildo has joined #openstack-nova11:55
*** ociuhandu has joined #openstack-nova12:00
*** ociuhandu has quit IRC12:02
*** ociuhandu has joined #openstack-nova12:03
*** TobbeCN has quit IRC12:07
*** mgariepy has joined #openstack-nova12:08
*** links has quit IRC12:10
*** links has joined #openstack-nova12:25
*** arxcruz has quit IRC12:28
*** arxcruz has joined #openstack-nova12:29
*** xek__ has joined #openstack-nova12:29
*** links has quit IRC12:29
gibinova will continue the PTG session in 30 minutes in the Juno Zoom room.12:30
*** tbachman_ has joined #openstack-nova12:32
sean-k-mooneycrap already i guess i better get coffee and water ready so12:32
*** dpawlik6 has joined #openstack-nova12:33
*** damien_r has quit IRC12:33
*** damien_r has joined #openstack-nova12:33
*** csatari_ has joined #openstack-nova12:33
gibiI've just did the same, water, coffee, and replaced the air in the room :)12:34
*** hemna_ has joined #openstack-nova12:34
*** ralonsoh_ has joined #openstack-nova12:34
*** sapd1_x has joined #openstack-nova12:34
gmanngibi: added one more topic (in the last) about removing the old and deprecated user level policy restriction. it was reminded by current ML thread.12:34
*** tobiash_ has joined #openstack-nova12:35
gmannit should be ok to remove as we already doing warning for 4 years but good to check in PTG if anyone still using that way.12:35
*** benj_- has joined #openstack-nova12:36
sean-k-mooneygibi: i bought a https://www.amazon.co.uk/Samson-SAQ2U-Microphone-Recording-Pack/dp/B001R747SG dnymic mic and stand so i can turn on my ceiling fan without out it being picked up without having to use a headset12:36
*** owalsh_ has joined #openstack-nova12:36
*** zzzeek has quit IRC12:36
*** tobberydberg has quit IRC12:36
*** pots has quit IRC12:36
*** csatari has quit IRC12:36
*** arxcruz has quit IRC12:36
*** hemna has quit IRC12:36
*** dpawlik3 has quit IRC12:36
*** owalsh has quit IRC12:36
*** tobiash has quit IRC12:36
*** gryf has quit IRC12:36
*** ioni has quit IRC12:36
*** udesale_ has quit IRC12:36
*** tbachman has quit IRC12:36
*** mvkr has quit IRC12:36
*** kukacz_ has quit IRC12:36
*** zigo has quit IRC12:36
*** csatari_ is now known as csatari12:36
*** tbachman_ is now known as tbachman12:36
*** benj_ has quit IRC12:36
*** tonyb has quit IRC12:36
*** averi has quit IRC12:36
*** derekh has quit IRC12:36
*** ralonsoh has quit IRC12:36
*** elod has quit IRC12:36
*** rmk has quit IRC12:36
*** sapd1_y has quit IRC12:36
*** benj_- is now known as benj_12:36
*** elod_ has joined #openstack-nova12:37
gibigmann: Is it OK to for you if I move that topic up to the first slot today where we anyhow talk about removals and deprecation?12:37
*** pots has joined #openstack-nova12:37
*** arxcruz has joined #openstack-nova12:37
gibisean-k-mooney: that mic look serious :)12:37
*** mvkr has joined #openstack-nova12:37
gmanngibi: yeah, that is totally fine.12:37
gibigmann: cool12:38
*** zzzeek has joined #openstack-nova12:38
*** tobberydberg has joined #openstack-nova12:38
gmannthanks12:38
*** lbragstad has joined #openstack-nova12:40
sean-k-mooneygibi: well i work form home permently and have calls regualarly but ya its a live stage mic that alot of twitch/game stream also use. for like 60 euro its good value too12:40
*** ioni has joined #openstack-nova12:41
openstackgerritMerged openstack/nova-specs master: Make Feature Liaison optional  https://review.opendev.org/73063812:42
*** boxiang_ has joined #openstack-nova12:44
*** zigo_ has joined #openstack-nova12:46
gibisean-k-mooney: yeah, sounds like a good idea for a home office. I only do full home office due to COVID12:46
*** nweinber has joined #openstack-nova12:46
*** boxiang has quit IRC12:48
gmannjohnthetubaguy: you want to do quick fix on release name in https://review.opendev.org/#/c/732173/112:48
johnthetubaguygmann: oops, looking12:50
*** dosaboy has quit IRC12:50
openstackgerritJohn Garbutt proposed openstack/nova-specs master: Re-propose Unified Limits in Nova  https://review.opendev.org/73217312:51
*** dosaboy has joined #openstack-nova12:51
alex_xusean-k-mooney: gibi so for neutron sriov agent, I guess it has no way to discover the VF? since it won't read from libvirt12:53
*** ociuhandu has quit IRC12:56
*** spatel has joined #openstack-nova12:57
sean-k-mooneyalex_xu: it does but not in a useful way12:59
sean-k-mooneyyou have to list the netdev names of the PF in its config12:59
sean-k-mooneyand it uses iptool i think13:00
gibialex_xu: nova sends pci_slot information to neutron in the binding profile but yeah as sean-k-mooney said the pci whitelist is on nova side today to specify what VF and PF resource is usable13:00
sean-k-mooneygibi: we do also send the pci addres in the pci_slot key13:00
gibiso nova has the VF resource view13:00
*** spatel has quit IRC13:00
sean-k-mooneybut that is used only when we are plugging or unplugging the port13:01
*** tkajinam has joined #openstack-nova13:02
*** yoctozepto has joined #openstack-nova13:02
*** alex_xu_ has joined #openstack-nova13:04
*** Guest10631 has joined #openstack-nova13:04
*** Guest10631 is now known as redrobot13:05
*** mlavalle has joined #openstack-nova13:06
*** slaweq has quit IRC13:07
artomgdi, y'all are talking and I can't hear you, can I?13:07
*** slaweq_ has joined #openstack-nova13:07
*** martinkennelly has quit IRC13:09
*** gryf has joined #openstack-nova13:10
alex_xu_gibi: sean-k-mooney I'm just thinking of that the neutron sriov agent won't be able to manage vf even in the future. (not today, today nova manage it)13:11
openstackgerritMerged openstack/nova-specs master: Re-propose Unified Limits in Nova  https://review.opendev.org/73217313:12
*** derekh has joined #openstack-nova13:14
sean-k-mooneyalex_xu_: right i dont think it will or should manage the vfs13:16
*** elod_ is now known as elod13:17
*** ociuhandu has joined #openstack-nova13:17
*** xek_ has joined #openstack-nova13:20
*** xek__ has quit IRC13:23
*** rpittau is now known as rpittau|brb13:24
artomDammit, you need a Zoom account to change your audio output settings >_<13:25
*** slaweq_ is now known as slaweq13:28
sean-k-mooneyartom: you should not13:29
sean-k-mooneyi can change mine and i dont have one13:29
artomsean-k-mooney, in the client?13:29
sean-k-mooneyin both the client and the browser13:29
artomsean-k-mooney, 'cuz I didn't find any way to do it. Even their official doc states you have to be logged in13:29
dansmithadvanced settings need an app I think13:29
artomsean-k-mooney, https://support.zoom.us/hc/en-us/articles/201362283-Testing-computer-or-device-audio13:29
dansmithincluding advanced network13:29
sean-k-mooneyartom: what are you trying to change13:30
artomsean-k-mooney, audio output to go to my BT headphones13:30
sean-k-mooneyif you click the arror on mic13:30
artomIt was outputting to stereo out, where nothing is currently plugged in13:30
sean-k-mooneythen you can change the output13:30
artomsean-k-mooney, oh christ, yes, it's there, you're right13:30
*** Liang__ has joined #openstack-nova13:32
*** iokiwi3 has joined #openstack-nova13:37
*** iokiwi has quit IRC13:37
stephenfinartom: your gain is waaaay too high13:38
sean-k-mooneyartom: the zoom app auto adjust your volume13:39
*** iokiwi3 is now known as iokiwi13:39
sean-k-mooneyor can so maybe look at that it in the extra options via the same arror menue13:39
*** iokiwi3 has joined #openstack-nova13:40
artomstephenfin, but my gainz are way too low :( Stupid gyms closing13:41
stephenfinartom: you know where the door is13:41
*** ociuhandu has quit IRC13:42
*** breizhkoala has quit IRC13:42
*** munimeha1 has joined #openstack-nova13:43
*** KeithMnemonic has joined #openstack-nova13:46
sean-k-mooneyartom: you are now a bit quite but that is an imporvement13:50
gibigmann: it is policy stuff in the nova room13:54
bauzasgibi: patches from dansmith about the compute 5.0 API bump https://review.opendev.org/#/q/topic:queens-rpc-bump+(status:open+OR+status:merged)13:54
dansmithbauzas: yeah13:55
gmanngibi: 1 min joinin13:55
bauzasgibi: the proxy dansmith told https://review.opendev.org/#/c/541005/6/nova/compute/manager.py@775013:55
gibigmann: we will discuss the policy stuff after the break at 5 minute past the hour13:55
gibigmann: so no rush13:55
bauzasanyway, less a deal to write the 6.0 proxy than the 5.013:55
gibibauzas: thanks!13:55
gmanngibi: ok13:55
bauzaserr, the 5.x proxy I mean13:56
*** ratailor has quit IRC13:58
*** ralonsoh_ is now known as ralonsoh14:00
*** rpittau|brb is now known as rpittau14:01
*** dklyle has joined #openstack-nova14:21
*** ociuhandu has joined #openstack-nova14:21
*** tobiash_ is now known as tobiash14:23
openstackgerritHervé Beraud proposed openstack/nova master: Always reset CONF when starting the wsgi app  https://review.opendev.org/73362714:27
*** martinkennelly has joined #openstack-nova14:28
*** munimeha1 has quit IRC14:28
*** boxiang has joined #openstack-nova14:31
*** boxiang_ has quit IRC14:32
*** ociuhandu has quit IRC14:41
*** gyee has joined #openstack-nova14:46
stephenfinkashyap: you're up next14:47
kashyapstephenfin: Just a moment, wrapping up another thing, sorry14:49
kashyapstephenfin: What's the topic?  /me goes to check the Etherpad...14:51
stephenfinq3514:51
*** songwenping_ has quit IRC14:51
*** songwenping_ has joined #openstack-nova14:52
*** jawad_axd has quit IRC14:52
kashyapLine-435?  We already talked a bit; see line-445 on the AGREED bit14:52
kashyapstephenfin: ^14:52
kashyapBut sure, if there's anything else to be touched on that, can certainly speak up...14:53
*** jawad_axd has joined #openstack-nova14:54
kashyapgibi: stephenfin: Has the topic flew by?  Which line this whole ARM topic on?14:56
stephenfinthe previous one (line 427)14:56
kashyapAh, we're on line-427 currently14:56
gibikashyap: 427 yes14:56
kashyapstephenfin: Yep, just see it, "virtualized architectures"14:56
dansmithsean-k-mooney: so you could just float one DNM patch to some thing to add that queue to get one run of nova on arm yeah?14:58
dansmithI mean, I assume we've never seen that work?14:58
sean-k-mooneyyep15:00
*** mriedem has joined #openstack-nova15:00
dansmithyep which? can float a job or yep, we've seen it work?15:00
sean-k-mooneyyou can just do a DNM patch15:01
sean-k-mooneyand yes i was looking at patches that do that previously15:01
sean-k-mooneyi think there are still a few issue they are working through15:01
dansmithI didn't know we could do that so easily.. now I'm curious if it works as is15:01
sean-k-mooneyyou just need to refecne the correct pipeline ill see if i can find the patch i was looking at before15:02
*** Luzi has joined #openstack-nova15:05
*** Luzi has quit IRC15:05
*** grandchild has joined #openstack-nova15:10
sean-k-mooneyits 8 on 3.x kernels15:14
johnthetubaguysean-k-mooney: ah...15:14
sean-k-mooneybecuase that si the most that vhost kernel module supports15:14
johnthetubaguysean-k-mooney: guest limit, or host limit, or both?15:14
johnthetubaguyah15:14
sean-k-mooneyin 4.X its 25615:14
johnthetubaguyah, interesting, thanks15:15
sean-k-mooneywe now have a config option so you can tell nova what it is if its different15:15
kashyapgibi: I need to drop to run a quick errand, afraid15:15
gibikashyap: do you have other topics on the etherpad?15:15
kashyapgibi: Not that I know of15:16
gibiI don't see any15:16
gibiOK. thanks for joining15:16
kashyapI'm curious to hear stephenfin's topic of "documentation is awful" (line-474) - please take some notes on it on the Etherpad :)15:16
* kashyap bbiab15:16
gibikashyap: ack15:17
sean-k-mooneyhttps://docs.openstack.org/nova/latest/configuration/config.html#filter_scheduler.max_io_ops_per_host15:30
*** mgariepy has quit IRC15:34
*** xek_ has quit IRC15:46
*** lpetrut has joined #openstack-nova15:48
*** alex_xu_ has quit IRC15:51
melwittsean-k-mooney: just curious, how can you know there's not enough swap configured before any instances are booted so you know the flavor?15:56
CeeMachi sean-k-mooney and melwitt ; regarding or chat yesterday around shelving, we've done some testing today and it would appear that even in shelved_offloaded state an instance is accumulating usage and the resources still appear allocated in the project quota15:56
CeeMacim back to being perplexed15:57
melwittCeeMac: oh... this is sounding familiar :(15:57
CeeMacwell, that doesnt sound good :/15:57
CeeMaci'm running rocky btw15:58
melwittlong ago, when the shelve API was added I am recalling that we don't release quota while instance is shelved. but I didn't realize that would cause it to count as the simple tenant usage. but it makes sense that it would15:58
melwittthe reasoning behind that decision was, not to let the user ever be in a position where they cannot unshelve for lack of quota15:59
*** factor has quit IRC15:59
*** jsuchome has quit IRC15:59
CeeMacyeah, i was actually having that conversation with my colleagues around implications to consider when shelving16:00
CeeMaci guess, at least in the way I'm ingesting the data, it depends on how horizon is building its usage information for Admin | Overview | Usage summary16:01
*** lpetrut has quit IRC16:03
CeeMaclet me see if I can pull some data from the openstack cli16:03
*** tkajinam has quit IRC16:05
*** Liang__ has quit IRC16:12
*** ebbex has joined #openstack-nova16:14
*** mgariepy has joined #openstack-nova16:14
CeeMacapparently I cant /fail16:16
melwittCeeMac: trying to get the same data that horizon shows you?16:17
CeeMacyeah for the instances specifically16:17
CeeMaci see the project usage in general16:17
CeeMacalthough it wont show any details for the admin project16:17
CeeMacwhich is odd (maybe)16:18
*** hamalq has joined #openstack-nova16:18
CeeMacopenstack usage show/list I've tried16:18
melwittok I was about to ask, that should be the right command16:18
*** hamalq has quit IRC16:18
CeeMacI could have sworn i'd seen something about instance usage when I was reading up yesterday, but I can't find it now if I did16:18
CeeMacyeah, list only has some projects16:19
CeeMaci'll maybe try with a different rc file, one sec16:19
*** hamalq has joined #openstack-nova16:19
CeeMacyeah, thats not helped.  I'll retry the test in demo project as I can see that16:22
CeeMacalthough openstack usage show only gives the totals, not per instance16:23
CeeMacok I'm stumped16:29
melwittI need to look a deeper at it because obviously horizon is getting it somehow, to be able to tell you how to do it with the cli16:29
CeeMacno worries16:31
*** rpittau is now known as rpittau|afk16:31
CeeMaci think maybe the actual api is more capable than what is implemented in the cli just now, from what I remember seeing reading through api-ref last night16:32
CeeMacdirect call api stuff is a bit beyond me at the moment though16:32
*** dtantsur is now known as dtantsur|afk16:33
gmannstephenfin: this one i was talking, i added index page for tempest plugins having some link on tempest do "Stable Branch * Policy" - https://docs.openstack.org/tempest/latest/plugins/index.html16:35
*** ravsingh has quit IRC16:49
*** priteau has quit IRC16:49
dansmithstephenfin: your picture is dark so I can't tell if you were receiving my good-natured jabbing properly17:02
stephenfinHeh, all good. See Zoom chat17:03
dansmithah heh yeah17:03
*** nightmare_unreal has quit IRC17:06
*** derekh has quit IRC17:07
*** tesseract has quit IRC17:16
sean-k-mooneymelwitt: sorry didnt see that till now. total_ram * allocation ratio - reserved should <= total_ram + swap17:23
sean-k-mooneyso that would be the init_host check17:24
sean-k-mooneythe per vm check is if i add up the MEMORY_MB for all vms on this host is it larger then total ram17:24
sean-k-mooneyif so warn17:24
sean-k-mooneythe less agressive warning would be to warn on total_ram + swap17:25
sean-k-mooneyinstead of > then total ram but once your are over total ram you are in OOM killer terrorty17:25
melwittsean-k-mooney: ah I see, thanks17:26
sean-k-mooneymelwitt: do you rememebr the nova bug for that transport thing i borught up at the end17:27
*** lbragstad has quit IRC17:27
sean-k-mooneymelwitt: i remember debuging it with you at the time17:27
melwittyeah vaguely17:27
melwittit's the mandatory flag for oslo messaging17:28
sean-k-mooneyyes17:28
melwittI don't remember if we opened a bug about that specifically or what bug we referred to when talking about it17:28
sean-k-mooneyso it looks like there is an example of using it here17:28
sean-k-mooneyhttps://github.com/Gsantomaggio/rabbitmq-utils/blob/master/openstack/mandatory_test/mandatory_client_fail.py#L40-L5417:28
sean-k-mooneywell i guess https://github.com/Gsantomaggio/rabbitmq-utils/blob/master/openstack/mandatory_test/mandatory_test.py#L35-L4517:29
sean-k-mooneyis a better example17:29
*** lbragstad has joined #openstack-nova17:30
sean-k-mooneybut what im not clear on is shoudl we just be setting it on every call or just when we do a topic send17:30
sean-k-mooneythe examples seam to be for topic sends17:31
melwittyeah, prob have the ask oslo peeps17:32
melwittcause I dunno either17:32
sean-k-mooneyi guess i could propose a patch and try adding it to things and see what breaks17:33
sean-k-mooneyhttps://bugs.launchpad.net/oslo.messaging/+bug/184974117:33
openstackLaunchpad bug 1849741 in oslo.messaging "TransportOptions are not documented" [Medium,Triaged]17:33
sean-k-mooneythat does not help however17:33
openstackgerritStephen Finucane proposed openstack/nova master: Add reproducer for bug #1881455  https://review.opendev.org/73366717:34
openstackbug 1881455 in OpenStack Compute (nova) "migrate server reporting list index out of bound" [Undecided,New] https://launchpad.net/bugs/188145517:34
openstackgerritStephen Finucane proposed openstack/nova master: Use 'Exception.__traceback__' for versioned notifications  https://review.opendev.org/73366817:34
*** lbragstad has quit IRC17:35
*** lbragstad has joined #openstack-nova17:37
sean-k-mooneystephenfin: by the way i don tthink you have to use Exception.__traceback__ directly anymore17:39
sean-k-mooneyinfact that is an anti patteren i think17:39
melwittsean-k-mooney: could it have been this? https://bugs.launchpad.net/nova/+bug/185499217:40
openstackLaunchpad bug 1854992 in OpenStack Compute (nova) "Frequent instances stuck in BUILD with no apparent failure" [Undecided,Incomplete]17:40
sean-k-mooneymelwitt: yep that was the nova bug17:40
sean-k-mooneywell i think it was the second one17:41
melwittok, can you write a note on it so we can have more hope to find it next time xD17:41
melwittthe only reason I found that was because I thought I remembered erik reporting it17:41
*** ociuhandu has joined #openstack-nova17:42
sean-k-mooneyyep erik is from blizzard right17:42
melwittyeah17:42
sean-k-mooneyoptions = oslo_messaging.TransportOptions(at_least_once=True)17:42
sean-k-mooneyclient = oslo_messaging.RPCClient(transport,17:42
sean-k-mooney                                  target,17:42
sean-k-mooney                                  transport_options=options)17:42
sean-k-mooneyso i think that is all we have to do17:42
melwittI wonder if there's a way we could test that it does something17:43
sean-k-mooneywhere we do this https://github.com/openstack/nova/blob/master/nova/rpc.py#L208-L21217:43
sean-k-mooneyand well the other RPC clients17:43
sean-k-mooneythat is what https://github.com/Gsantomaggio/rabbitmq-utils/tree/master/openstack/mandatory_test is trying to do17:44
sean-k-mooneywhen it fails to deliver a message to the queue17:45
sean-k-mooneywe will get a oslo_messaging.exceptions.MessageUndeliverable17:45
sean-k-mooneyexception17:45
melwittyeah I mean, how do you create that env where it would have raised MessageUndeliverable?17:45
melwittjust stop rabbitmq process or?17:45
melwittjust curious how to repro and see it fix the problem17:46
sean-k-mooneyno you need to send a message to a queue that does not exist17:46
melwittok, I don't know how to do that I guess17:47
sean-k-mooneywe could create an exchange without any queue and jsut send a message to a random queue name17:47
melwittor is that the 'foo' in the example17:47
sean-k-mooneyya so we can sed to foo17:48
sean-k-mooneybut not create foo17:48
melwittok. shows how much I know about rabbitmq :P17:48
sean-k-mooneywell i think that is how this works17:48
melwittcool. I'm just thinking about a one-off local test just so we know we set the option in the right place and get the improved behavior. since the unit tests that go with the patch will only be checking "did you pass mandatory at this call site"17:49
sean-k-mooneyi wondering if we could do an opertunistic functional test or something17:50
*** ociuhandu has quit IRC17:50
sean-k-mooneye.g. if rabbitmq is avaiable and we have a nova.conf with creads connect and try and to do cause the excption17:51
sean-k-mooneyi would assume/hope oslo messaging does that17:51
melwittyeah I dunno about that17:52
melwittI didn't think we have rabbitmq available in those17:52
sean-k-mooneyi can hope17:52
sean-k-mooneywe dont in nova17:52
sean-k-mooneywe use the inmemory oslo messaging driver17:53
sean-k-mooneyfor our functional tests17:53
melwittyeah but that's the 'fake' driver. yeah I mean maybe it models that scenario? I was doubting17:54
sean-k-mooneyhttps://github.com/openstack/oslo.messaging/blob/e44c9883066d9b2d081a594b97aac3d598d491c9/oslo_messaging/tests/functional/test_functional.py#L154-L18617:56
*** jawad_axd has quit IRC17:57
sean-k-mooneyso based on the fact that skips if its not rabbit://17:57
sean-k-mooneyi would guess that it only works if its aviable17:57
melwittok. besides that, that makes it look like we need to do more than just set the flag right? we have to do something to handle MessageUndeliverable17:59
sean-k-mooneyyes17:59
sean-k-mooneywhich woudl depend on the rpc call17:59
sean-k-mooneyfor spwan i guess put the vm to error state?17:59
sean-k-mooneynot really sure about what we woudl do for anything else18:00
melwittah so this is more complicated than I thought18:01
melwittI thought all we'd have to do is set the flag and then oslo.messaging would reconnect us and retry or something like that18:01
sean-k-mooneythe issue is that the queue that the compute agent created nolonger exsits but the compute agent does not know that. so unless we had the compute agent call itself on that queue in a heart beat we would not know we had to recreate the queue18:01
sean-k-mooneymelwitt: im not sure if we can create the queue or if it has to be the compute agent18:02
sean-k-mooneythe compute agent normally creates the queue when it connect to rabbitmq18:02
sean-k-mooneywhich is why restarting the compuate agent fixes the problem18:02
sean-k-mooneyso at a minium we can log that the agent is unreachable. we could even mark the agent as down but that leaves the question of who will make it up again once its restarted18:03
melwittyeah. well, then I'd wonder if we could put try-except in the nova/rpc layer that will create the queue if we get MessageUndeliverable? that way whoever gets the "no queue" state will recreate it?18:03
sean-k-mooneyif we had a periodic task in teh compute agent that sent to its own queue then that would work18:04
sean-k-mooneyit might work if the conductor or api created it too18:04
sean-k-mooneyi just dont know if the compute agent will start litening to the queue when its created by someone else18:05
melwittyeah I guess I'm thinking it wouldn't matter who creates the queue. like if nova-conductor gets MessageUndeliverable trying to talk to compute, assume the queue is gone and recreate it, and resend18:05
melwittoh, right18:05
melwitthm18:05
sean-k-mooneythat is why i was suggesting havign the compute agent send to its own queue18:06
sean-k-mooneyjsut a simple ping/heatbeat18:06
melwittperiodic does sound like it would work but that doesn't help someone trying to boot an instance and it fails due to this18:06
melwittbut of course better than what we have today18:07
sean-k-mooneywe could retry after a short interval?18:07
sean-k-mooneyhoping the agent fixes its self?18:07
*** ralonsoh has quit IRC18:07
sean-k-mooneymaybe if we have time tommorow and ben or some of the oslo folks are aroudn we could ask them18:08
sean-k-mooneyour i guess i could try it locally18:08
melwittyeah... I don't love that idea but yeah, would take some thought on some options18:08
sean-k-mooneyi could use the rabbitmq gui to delete the queue manually18:08
sean-k-mooneyi dont think the manamge ui is isntall by devstack by default but i dont think its that hard to enable18:09
sean-k-mooneyif i could repoduce it manually then we could test teh recreate behavior18:10
sean-k-mooneye.g. if i make the condutor recreate it and i still can boot a vm i know it need to be the agent18:10
melwittright18:10
melwittyeah doing that would demystify a lot. but yeah gonna be a pain/not ideal if nova-compute would need to re-init to pick up the new queue18:11
sean-k-mooneyim going to go figure out what im doing for dinner and then i might give it a try although i kind of want to look into it someother time18:14
sean-k-mooneycan you bug me about this if you remember tommorow or next week18:14
bnemecIf you have messaging stuff to talk about I would suggest pinging kgiusti to make sure he's available.18:14
melwittsure. I'll try to write something on the launchpad bug so I don't forget about this a sixth time or whatever we're at now18:14
melwittin addition18:15
sean-k-mooneybnemec: well we are just trying to figure out how to use the mandataory flag that ye enabled via the transport options correctly18:15
sean-k-mooneybut my rabbit mq knoladge is really not good enough to have an intuition about this18:16
melwittuse it + handle the exception that will raise as a result of setting it18:16
melwitt*set it18:16
sean-k-mooneymelwitt: hehe yep 6th might be a bit much but at least thrid or forth :)18:17
melwitt:)18:17
sean-k-mooneywe finally merge the patch t silance the amqp heart beat error by the way18:19
melwitt#success18:19
openstackstatusmelwitt: Added success to Success page (https://wiki.openstack.org/wiki/Successes)18:19
melwittlol oops18:20
sean-k-mooneyhehe18:20
sean-k-mooneyhttps://review.opendev.org/#/c/724188/518:20
sean-k-mooneyok time for food o/18:21
melwitto/18:21
melwittI forgot about the hashtag success being a thing 😬18:22
*** lbragstad has quit IRC18:33
*** lbragstad has joined #openstack-nova18:37
dansmithmelwitt: from a quick scan, nobody has ever made that mistake in the history of the hashtag18:56
dansmithexcept for the second most recent :P18:57
dansmith(three days go)18:57
*** martinkennelly has quit IRC19:06
*** vishalmanchanda has quit IRC19:11
*** belmoreira has quit IRC19:12
melwittdansmith: I noticed that. go me19:30
melwitt#special19:30
*** boxiang has quit IRC19:34
*** boxiang has joined #openstack-nova19:34
*** songwenping__ has joined #openstack-nova19:50
*** songwenping_ has quit IRC19:53
*** amodi has quit IRC20:02
*** amodi has joined #openstack-nova20:03
*** maciejjozefczyk_ has quit IRC20:09
*** maciejjozefczyk has joined #openstack-nova20:13
*** damien_r has quit IRC20:22
*** NostawRm has quit IRC20:28
*** NostawRm has joined #openstack-nova20:28
*** NostawRm has quit IRC20:39
*** nweinber has quit IRC20:44
*** xek_ has joined #openstack-nova20:44
*** gyee has quit IRC20:55
*** ccamacho has quit IRC20:56
*** gyee has joined #openstack-nova20:57
*** mriedem has left #openstack-nova20:58
openstackgerritSylvain Bauza proposed openstack/nova-specs master: Routed network scheduling spec  https://review.opendev.org/73370321:23
*** xiaolin has quit IRC21:25
*** _mlavalle_1 has joined #openstack-nova21:30
*** xek_ has quit IRC21:30
*** mlavalle has quit IRC21:33
*** bbowen has quit IRC21:39
*** amodi has quit IRC21:52
*** rcernin has joined #openstack-nova22:06
*** _mlavalle_1 has quit IRC22:10
*** rcernin has quit IRC22:11
*** mlavalle has joined #openstack-nova22:11
*** raildo has quit IRC22:18
*** rajinir has quit IRC22:35
*** CeeMac has quit IRC22:35
*** samueldmq has quit IRC22:35
*** avolkov has quit IRC22:35
*** csatari has quit IRC22:35
*** tonyb[m] has quit IRC22:35
*** jungleboyj has quit IRC22:35
*** nicolasbock has quit IRC22:36
*** pas-ha has quit IRC22:36
*** knikolla has quit IRC22:36
*** luyao has quit IRC22:36
*** masayukig has quit IRC22:36
*** rpittau|afk has quit IRC22:36
*** mnaser has quit IRC22:36
*** donnyd has quit IRC22:36
*** _erlon_ has quit IRC22:36
*** NobodyCam has quit IRC22:36
*** larainema has quit IRC22:36
*** gregwork has quit IRC22:36
*** bcm has quit IRC22:37
*** mnasiadka has quit IRC22:37
*** cz3 has quit IRC22:37
*** cz3 has joined #openstack-nova22:37
*** portdirect has quit IRC22:37
*** lseki has quit IRC22:37
*** TheJulia has quit IRC22:37
*** pas-ha has joined #openstack-nova22:37
*** NobodyCam has joined #openstack-nova22:37
*** jrosser has quit IRC22:37
*** fyx has quit IRC22:37
*** johnsom has quit IRC22:37
*** arne_wiebalck has quit IRC22:37
*** gmann has quit IRC22:37
*** vdrok has quit IRC22:37
*** CeeMac has joined #openstack-nova22:37
*** rm_work has quit IRC22:38
*** rajinir has joined #openstack-nova22:38
*** gregwork has joined #openstack-nova22:38
*** larainema has joined #openstack-nova22:38
*** masayukig has joined #openstack-nova22:38
*** gmann has joined #openstack-nova22:38
*** knikolla has joined #openstack-nova22:38
*** tonyb[m] has joined #openstack-nova22:39
*** mnasiadka has joined #openstack-nova22:39
*** csatari has joined #openstack-nova22:39
*** _erlon_ has joined #openstack-nova22:39
*** avolkov has joined #openstack-nova22:40
*** arne_wiebalck has joined #openstack-nova22:40
*** mnaser has joined #openstack-nova22:41
*** jrosser has joined #openstack-nova22:41
*** bcm has joined #openstack-nova22:41
*** donnyd has joined #openstack-nova22:41
*** TheJulia has joined #openstack-nova22:41
*** lseki has joined #openstack-nova22:42
*** portdirect has joined #openstack-nova22:42
*** nicolasbock has joined #openstack-nova22:42
*** luyao has joined #openstack-nova22:43
*** johnsom has joined #openstack-nova22:43
*** rcernin has joined #openstack-nova22:48
*** avolkov has quit IRC22:50
*** rm_work has joined #openstack-nova22:51
*** jungleboyj has joined #openstack-nova22:53
*** vdrok has joined #openstack-nova22:54
*** tkajinam has joined #openstack-nova22:56
*** samueldmq has joined #openstack-nova23:02
*** rpittau|afk has joined #openstack-nova23:04
*** fyx has joined #openstack-nova23:12
*** huaqiang has quit IRC23:14
*** hamalq has quit IRC23:22
*** slaweq has quit IRC23:25
*** slaweq has joined #openstack-nova23:29
*** slaweq has quit IRC23:34
*** tosky has quit IRC23:35
*** grandchild has quit IRC23:39
*** gyee has quit IRC23:46
*** rcernin has quit IRC23:49
*** ociuhandu has joined #openstack-nova23:53
*** mlavalle has quit IRC23:57
*** ociuhandu has quit IRC23:58

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!