Thursday, 2022-06-23

*** ysandeep|out is now known as ysandeep04:53
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible stable/yoga: Update .gitreview for stable/yoga  https://review.opendev.org/c/openstack/openstack-ansible/+/84724206:33
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible stable/yoga: Update TOX_CONSTRAINTS_FILE for stable/yoga  https://review.opendev.org/c/openstack/openstack-ansible/+/84724306:33
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible master: Update master for stable/yoga  https://review.opendev.org/c/openstack/openstack-ansible/+/84724406:33
*** ysandeep is now known as ysandeep|afk07:30
opendevreviewMerged openstack/openstack-ansible stable/yoga: Update .gitreview for stable/yoga  https://review.opendev.org/c/openstack/openstack-ansible/+/84724208:02
opendevreviewMerged openstack/openstack-ansible stable/yoga: Update TOX_CONSTRAINTS_FILE for stable/yoga  https://review.opendev.org/c/openstack/openstack-ansible/+/84724308:02
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Continue Zed development  https://review.opendev.org/c/openstack/openstack-ansible/+/84727208:12
jrosser_should we drop rocky-8 and centos-8-stream straight away?08:19
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Create backends when running against manila_share  https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/84727608:26
noonedeadpunkYeah, I think so. I just wanted to see with my eyes that they fail first :D08:41
jrosser_hmm i wonder if we are pulling in openstacksdk 0.99 https://zuul.opendev.org/t/openstack/build/41651e569b0a4170994f1bb8ed68fbfa/log/job-output.txt#953209:08
jrosser_yeah https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L45609:11
jrosser_thats going to end badly i think09:11
*** ysandeep|afk is now known as ysandeep09:20
jrosser_noonedeadpunk: i wonder if we now need to add some stuff to globally edit the u-c file we download09:29
jrosser_because changing the openstacksdk version constraint everywhere is going to be a huge mess otherwise09:29
noonedeadpunkI think it all depends on collections version we are to use tbh09:31
noonedeadpunkAs basically we need 0.99 as well as master sdk...09:31
jrosser_and also anywhere that sdk is used inside services, which might have different version requirements again09:32
jrosser_perhaps the first thing to do is be more in control of the openstacksdk version that gets used in the ansible-runtime venv09:33
jrosser_then we can be a bit independant about that compared to what we deploy with the services09:33
noonedeadpunkmaybe it's time to try https://review.opendev.org/c/openstack/openstack-ansible/+/836382 again?09:35
noonedeadpunkbtw let me bump collections anyway...09:35
noonedeadpunkbecause tbh, for system scopes in Zed we one way or another should have 0.99....09:36
noonedeadpunkmaybe not not though09:36
jrosser_omg there are so many moving parts here :(09:36
jrosser_for tooling we have here, using the published collections and sdk 0.99 just doesnt work09:37
jrosser_we have to pin to < 0.9909:37
noonedeadpunkI'd say - there's sooooo much mess in clients/collections/sdk now 09:38
noonedeadpunkand eventually on master collections some things were already fixed....09:41
noonedeadpunkor well, a lot of09:41
noonedeadpunkbut yes. overriding it is another way around09:44
noonedeadpunkWe will fail at least here I guess https://review.opendev.org/c/openstack/ansible-collections-openstack/+/84530409:44
noonedeadpunk*here - on image upload in tempest role09:45
noonedeadpunkoh my, for god's sake https://pypi.org/project/ansible-config-template/09:47
noonedeadpunkhow we should deprecate that////09:47
jrosser_i know09:51
jrosser_did we do that my mistake or has it come from the tripleo side?09:52
jrosser_*by mistake09:52
noonedeadpunkI think we did it intentionally before galaxy times09:54
noonedeadpunkas that was the neatest way to install plugin I guess09:54
jrosser_i do remember a time when there was lots of discussion if to publish ansible things to pypi09:55
noonedeadpunkbut then galaxy came in I believe09:56
noonedeadpunkok. then let's update readme, publish on galaxy and drop from pypi as a next step...10:00
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-config_template master: Update documentation about installation methods  https://review.opendev.org/c/openstack/ansible-config_template/+/84730210:16
noonedeadpunkRegarding config_template. My suggestion is to merge deperecation notice of pypi and publish to galaxy. Release 2.0.0 version with that. Afterwards drop publishing to pypi and setup.py and do another release with that10:23
noonedeadpunkregarding master branch - maybe we should land several bugfixes like https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/847276 https://review.opendev.org/c/openstack/openstack-ansible/+/846799 and https://review.opendev.org/c/openstack/openstack-ansible/+/846580 and backport them?10:25
noonedeadpunkthere's also bug with etcd/zun which I will work on today10:25
noonedeadpunkregarding sdk/collections let's see how bad master is regarding collections. Maybe it's close to working state today and jsut needs small push10:26
noonedeadpunkSo for config_template basically we need 2 things now I guess? https://review.opendev.org/c/openstack/ansible-config_template/+/847302 and https://review.opendev.org/c/openstack/ansible-config_template/+/84639110:27
noonedeadpunkdamiandabrowski[m]: having some reviews would be helpful10:28
damiandabrowski[m]okok11:04
*** dviroel|afk is now known as dviroel11:20
noonedeadpunkjrosser_: yes, we need to install sdk<0.99 in ansible venv...11:21
noonedeadpunkhttps://hackmd.io/7NtovjRkRn-tKraBXfz9jw?view that's todo list11:21
jrosser_we need a way to override parts of u-c for the ansible venv then11:22
jrosser_i think we should have a list of things which should be excluded from u-c11:25
jrosser_and then another list of things that we add in11:25
jrosser_for the ansible venv11:25
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Continue Zed development  https://review.opendev.org/c/openstack/openstack-ansible/+/84727211:31
jrosser_the messaging around sdk and collection versions is not totally clear11:42
opendevreviewMerged openstack/openstack-ansible-os_manila stable/yoga: Updated from OpenStack Ansible Tests  https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/84677812:16
opendevreviewMerged openstack/openstack-ansible-os_masakari stable/yoga: Updated from OpenStack Ansible Tests  https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/84677912:30
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Continue Zed development  https://review.opendev.org/c/openstack/openstack-ansible/+/84727212:41
spateljrosser_ noonedeadpunk didn't we fix this issue in wallaby ? - TASK [python_venv_build : Fail if wheels are not built for multiple hosts] 12:41
spatelThis is related rally 12:41
noonedeadpunkspatel: nope https://review.opendev.org/c/openstack/openstack-ansible/+/84568512:42
spatelI meant disable rally 12:42
spateloh!! so its in WIP 12:42
noonedeadpunkseems that rabbitmq install is broken on bionic for Wallaby12:43
noonedeadpunkso CI's not passing and didn't have time to follow that up12:43
spatelgood to know, its kind of annoying to see that error every time build new deployment :)12:43
spatelI will see if i can fix it in my free time.. 12:45
jrosser_noonedeadpunk: didnt we have a patch to switch wallaby to cloudsmith, maybe i think of something else12:53
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Continue Zed development  https://review.opendev.org/c/openstack/openstack-ansible/+/84727212:54
noonedeadpunkI made a backport but it failed close to same way12:54
jrosser_aha https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/84612212:54
noonedeadpunkok it failed way worse :)12:55
*** ysandeep is now known as ysandeep|afk13:39
*** ysandeep|afk is now known as ysandeep13:47
*** ThiagoCMC is now known as UltraForgot14:15
UltraForgotHi folks! I'm trying to deploy OSA AIO, but the `scripts/bootstrap-ansible.sh` is failing to clone `python_venv_build`, saying "fatal: reference is not a tree". Any idea? I'm trying to deploy OSA 23.0.014:17
UltraForgotI also tried 22.0.0, same error.14:17
noonedeadpunkUltraForgot: if I was you, I'd try latest tag in each series14:19
noonedeadpunkSo 22.4.4 or 23.3.114:19
noonedeadpunkI believe this is actually depth issue (ie, how deep git clones repo). By default we set it to 20. But since 22.0.0 for some repos there could be more then 20 patches landed14:20
noonedeadpunkNow we hopefully fixed that with replacing depth with shallow_since14:21
noonedeadpunkbut anyway I'd say it's worth using latest version in series 14:21
noonedeadpunkWhole 22.*.* is still Victoria, whole 23.*.* is still Wallaby14:22
UltraForgotHmm... I see, makes sense, thanks! I'll try.14:24
UltraForgotI was planning to experiment with upgrades from 22.0.0, some minor versions pump first, then 23.0.0, and etc... For a PoC14:25
noonedeadpunkUltraForgot: oh, well. Then try setting `role_clone_default_depth: 50`  https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/get-ansible-role-requirements.yml#L17614:26
noonedeadpunklikely, you can set that also in user_variables.yml but not 100% sure14:27
noonedeadpunkbut if you want to jump between releases it's usually not required to do minor upgrade first. So you for example you can jump from 22.0.0 to 23.3.1 directly14:28
jrosser_22.0.0 will be the very first release of that branch that we made14:29
noonedeadpunkThere was 1 issue though with rabbitmq version, that if you upgrade from bionic to focal, you must upgrade to some minor release first. 14:29
jrosser_i can bet almost 100% there will be something broken14:29
noonedeadpunkoh, yes, it will :D14:29
jrosser_don't we generally make 22.1.0 be the first stable release we have increased confidence in14:30
UltraForgotCool, 23.2.0 worked lol14:36
*** ysandeep is now known as ysandeep|out15:21
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Don not try to run etcd in serial mode  https://review.opendev.org/c/openstack/openstack-ansible/+/84736215:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not try to run etcd in serial mode  https://review.opendev.org/c/openstack/openstack-ansible/+/84736215:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Move etcd installation to playbook  https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/84736315:34
UltraForgotSomething else is happening now... lol 16:15
UltraForgottested multiple versions 16:15
UltraForgotThe TASK systemd_mount Download EPEL gpg keys is failing, but I'm on Ubuntu 20.04.16:15
jrosser_having some context around that would help16:21
jrosser_which playbook and pasting some helpful output to paste.opendev.org16:21
*** frenzy_friday is now known as frenzyfriday|PTO16:21
UltraForgotI'm trying to run `scripts/bootstrap-aio.sh` and that TASK fails.16:22
UltraForgotOSA version 23.3.016:23
UltraForgotIt says: "The conditional check ''s3fs' in systemd_mount_types' failed.16:24
spatelQuick question do you guys prefer hw_disk_bus=scsi property for glance image?16:28
spatelor hw_disk_bus=virtio ?16:28
UltraForgotVirtIO SCSI FTW16:28
spatel?16:29
jrosser_noonedeadpunk: from a focal job: `2021-12-14T09:41:52,047 Looking in indexes: https://mirror-int.dfw.rax.opendev.org/pypi/simple, https://mirror-int.dfw.rax.opendev.org/wheel/centos-8-x86_64`16:31
jrosser_UltraForgot: thats really not enough information to be able to help unfortunately16:35
UltraForgotok, well, I'm following the bootstrap AIO and it's stopping on that task16:43
UltraForgotThis one: https://docs.openstack.org/openstack-ansible/wallaby/user/aio/quickstart.html16:44
UltraForgotI'll try again (it's a test in a VM, no big deal)...   =)16:44
jrosser_we have some requirements trouble with horizon https://github.com/fusionbox/django-pyscss/blob/master/setup.py#L1416:57
jrosser_vs. https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L7516:58
jrosser_which results in this https://paste.opendev.org/show/bMIVHl4BBfo6TUo05bqm/16:59
UltraForgotOk, tried again bootstrap-aio.sh from scratch and it passed that part! Now it failed in TASK [systemd_networkd : Create systemd-networkd network network(s)], error: no filter named 'ipaddr' - Any idea?17:18
UltraForgotSeems fixed in 22.4.2, gonna try again!17:21
jrosser_UltraForgot: this is the price to pay for using older versions17:24
jrosser_in this case the early 22.x releases were ok at the time that they were made17:24
jrosser_but a ansibke unfortunately made breaking changes particularly to the ipaddr module so that’s only fixed in newer tags17:25
noonedeadpunkhuh, metal jobs passed https://review.opendev.org/c/openstack/openstack-ansible/+/847272 with master collections....17:28
UltraForgotjrosser_, okay, thanks!17:29
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Return facts gathering to pre_tasks for masakari  https://review.opendev.org/c/openstack/openstack-ansible/+/84726017:31
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Fix facts gathering for zun  https://review.opendev.org/c/openstack/openstack-ansible/+/84726117:33
damiandabrowski[m]noonedeadpunk: please let me know if I missed reviews for some urgent changes17:35
jrosser_noonedeadpunk: yes I saw the metal all passed - does that mean we still don’t run horizon for metal jobs…..17:53
jrosser_horizon fails certainly for the focal lxc one17:53
UltraForgotOops... Just realized I'm running on M1 Parallels, it's arch64... Nobody building an amr64-based cloud around, right? lol17:55
UltraForgotNo 22.4.2/ubuntu-20.04-aarch64 ...  :-D17:56
jrosser_I did work with OSA for aarch64 a while ago but could never get a CI job to complete inside 3 hours18:13
jrosser_that’s probably better now that there should be aarch64 python wheels already built in the CI environment18:14
UltraForgotAny tips to point it to a repository that contains the aarch64 container images?18:17
noonedeadpunkoh, well, that's yet another thing that likely not needed on newer releases :D18:17
UltraForgotOh no... lol18:17
UltraForgotWhich one was the first?18:17
noonedeadpunkbut eventually, for lxc images (if you mean them) arch should be respected from what I see18:18
UltraForgotYep, the lxc images18:18
UltraForgotI didn't work for 22.4.218:18
noonedeadpunkat least it's stated there https://opendev.org/openstack/openstack-ansible-lxc_hosts/src/branch/stable/victoria/vars/ubuntu-20.04.yml#L1818:18
noonedeadpunkbut we switched to building images with debootstrap starting Wallaby only18:19
UltraForgotCool! It looks like it should work in Victoria - http://cdimage.ubuntu.com/ubuntu-base/releases/20.04/release have arm64 base image18:21
jrosser_why not use newer code :)18:21
noonedeadpunkI think issue there is that ubuntu names it arm64 and ansible detects as aarch6418:21
UltraForgotTrue18:21
jrosser_I had this working and committed a bunch of stuff for arm18:21
noonedeadpunkBut you can override that easily in usera_variables18:21
jrosser_so the problems will be relatively small18:22
UltraForgotI'm building a PoC to prove how awesome is to upgrade OpenStack with OSA18:22
UltraForgotSo, start with old and upgrade to new18:22
UltraForgot;-)18:22
noonedeadpunkJust put that content under variable `lxc_cache_map` https://opendev.org/openstack/openstack-ansible-lxc_hosts/src/branch/stable/victoria/vars/ubuntu-20.04.yml#L21-L3218:22
noonedeadpunkexcept adjust arch value to arm64 18:22
UltraForgotNice!18:22
jrosser_here is my effort for arm https://review.opendev.org/c/openstack/openstack-ansible/+/61830518:25
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Update doc index page  https://review.opendev.org/c/openstack/openstack-ansible/+/84739718:27
jrosser_https://review.opendev.org/q/topic:arm64+owner:jrosser18:27
UltraForgotCool, I'll give it a try.18:49
*** dviroel is now known as dviroel|out21:14
opendevreviewMerged openstack/ansible-config_template master: Update documentation about installation methods  https://review.opendev.org/c/openstack/ansible-config_template/+/84730222:15
opendevreviewMerged openstack/openstack-ansible-os_zun stable/yoga: Updated from OpenStack Ansible Tests  https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/84678023:56

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!