Tuesday, 2019-07-23

*** iyamahat has quit IRC00:14
*** Liang__ has joined #openstack-meeting-400:50
*** hongbin has joined #openstack-meeting-400:55
*** igordc has quit IRC00:55
*** TxGirlGeek has joined #openstack-meeting-400:58
*** TxGirlGeek has quit IRC01:32
*** yamamoto has quit IRC01:34
*** TxGirlGeek has joined #openstack-meeting-401:50
*** TxGirlGeek has quit IRC01:54
*** yamamoto has joined #openstack-meeting-401:58
*** yamamoto has quit IRC02:17
*** yamamoto has joined #openstack-meeting-402:21
*** gcheresh_ has joined #openstack-meeting-402:43
*** michael-beaver has quit IRC02:54
*** gcheresh_ has quit IRC03:20
*** psachin has joined #openstack-meeting-403:28
*** iyamahat has joined #openstack-meeting-403:44
*** hongbin has quit IRC03:45
*** gkadam has joined #openstack-meeting-403:49
*** gkadam has quit IRC03:50
*** hongbin has joined #openstack-meeting-403:50
*** hongbin has quit IRC03:50
*** gcheresh_ has joined #openstack-meeting-404:22
*** gcheresh_ has quit IRC04:27
*** pcaruana has joined #openstack-meeting-404:43
*** links has joined #openstack-meeting-405:04
*** Luzi has joined #openstack-meeting-405:05
*** jraju__ has joined #openstack-meeting-405:08
*** links has quit IRC05:09
*** jraju__ has quit IRC05:10
*** TxGirlGeek has joined #openstack-meeting-405:16
*** notmyname has quit IRC05:41
*** notmyname has joined #openstack-meeting-405:42
*** TxGirlGeek has quit IRC05:43
*** igordc has joined #openstack-meeting-406:08
*** igordc has quit IRC06:15
*** diablo_rojo has joined #openstack-meeting-406:27
*** diablo_rojo has quit IRC06:32
*** e0ne has joined #openstack-meeting-406:40
*** e0ne has quit IRC06:41
*** gcheresh_ has joined #openstack-meeting-406:49
*** yamamoto has quit IRC07:05
*** yamamoto has joined #openstack-meeting-407:06
*** slaweq has joined #openstack-meeting-407:07
*** irclogbot_0 has quit IRC07:20
*** irclogbot_2 has joined #openstack-meeting-407:21
*** dansmith has quit IRC07:23
*** dansmith has joined #openstack-meeting-407:24
*** k_mouza has joined #openstack-meeting-408:19
*** yamamoto has quit IRC08:20
*** mfuller has joined #openstack-meeting-408:22
*** ralonsoh has joined #openstack-meeting-408:26
*** mfuller has quit IRC08:43
*** e0ne has joined #openstack-meeting-408:46
*** yamamoto has joined #openstack-meeting-408:50
*** yamamoto has quit IRC09:02
*** Liang__ has quit IRC09:19
*** psachin has quit IRC09:20
*** k_mouza has quit IRC09:22
*** psachin has joined #openstack-meeting-409:35
*** k_mouza has joined #openstack-meeting-409:43
*** iyamahat has quit IRC10:04
*** lpetrut has joined #openstack-meeting-410:21
*** yamamoto has joined #openstack-meeting-410:27
*** yamamoto has quit IRC10:28
*** yamamoto has joined #openstack-meeting-410:28
*** shachar has joined #openstack-meeting-410:29
*** snapiri has quit IRC10:32
*** yamamoto has quit IRC10:48
*** yamamoto has joined #openstack-meeting-410:57
*** yamamoto has quit IRC11:02
*** yamamoto has joined #openstack-meeting-411:05
*** yamamoto has quit IRC12:03
*** yamamoto has joined #openstack-meeting-412:08
*** bh526r has joined #openstack-meeting-412:56
*** ktibi has joined #openstack-meeting-413:30
*** cliffparsons has quit IRC13:33
*** lemko has joined #openstack-meeting-413:38
*** Luzi has quit IRC13:51
*** yamamoto has quit IRC13:53
*** yamamoto has joined #openstack-meeting-413:54
*** yamamoto has quit IRC13:58
*** michael-beaver has joined #openstack-meeting-414:07
*** belmoreira has joined #openstack-meeting-414:14
*** yamamoto has joined #openstack-meeting-414:37
*** dosaboy has joined #openstack-meeting-414:41
*** yamamoto has quit IRC14:43
*** jsuchome has joined #openstack-meeting-414:45
*** dwalt has joined #openstack-meeting-414:46
*** cliffparsons has joined #openstack-meeting-414:51
*** gagehugo has joined #openstack-meeting-414:52
*** TxGirlGeek has joined #openstack-meeting-414:52
* srwilkers pokes portdirect15:00
portdirect#startmeeting openstack-helm15:01
openstackMeeting started Tue Jul 23 15:01:13 2019 UTC and is due to finish in 60 minutes.  The chair is portdirect. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
*** openstack changes topic to " (Meeting topic: openstack-helm)"15:01
openstackThe meeting name has been set to 'openstack_helm'15:01
portdirectlets give it a few mins15:01
lamto/15:01
dwalto/15:01
gagehugoo/15:01
dwaltI made it15:01
howello/15:01
roman_go/15:01
cliffparsonso/15:01
portdirectagenda is here: https://etherpad.openstack.org/p/openstack-helm-meeting-2019-07-2315:01
srwilkerso/15:01
srwilkershappy tuesday e'rybody15:01
cheng1o/15:01
zhipeng[m]hello15:02
zhipeng[m]have a good day, everyone15:02
rihabb2o/15:03
portdirectok - lets go15:06
portdirect#topic Apologies15:06
*** openstack changes topic to "Apologies (Meeting topic: openstack-helm)"15:06
zhipeng[m]lets go:)15:06
portdirectI need to hold my hands up here, and apologise for the admin backlock thats built up15:06
portdirectin particular im behind on the meeting time re-arangment15:06
portdirectand the release versioning spec15:07
portdirectim sorry - things have been hectic15:07
portdirectand im trying to get back on track15:07
portdirectthough may take another week or two15:07
srwilkersportdirect: you're only human15:07
portdirect#topic Gating for more than just the ocata release.15:09
*** openstack changes topic to "Gating for more than just the ocata release. (Meeting topic: openstack-helm)"15:09
*** gcheresh_ has quit IRC15:09
portdirectso we have had checks in place for all openstack realeases from ocata to stein for some time now15:09
portdirectand also opensuse support for rocky15:09
portdirectits probably time we considered them for promotion to gates15:10
portdirector at least a subset of them15:10
portdirectthoughts?15:10
srwilkersi think that makes sense15:11
lamt++, but do we need to cover all the releases from ocata -> stein? or just a subset of those releases15:11
srwilkersi'm checking through now and don't see any particular subset of jobs that seems to fail often enough to cause concern, at least from my perspective15:12
evrardjpsrwilkers: the simple fact of enabling things in gates will make things fail, Murphy's law still apply15:13
srwilkerswell, i take that back -- it seems the compute jobs fail more frequently than anything else, but that's not any different than what i've noticed in the past both with single node and multinode jobs15:13
srwilkersevrardjp: obviously15:13
portdirectlamt: im wondering if just a subset makes sense?15:13
evrardjpI would ofc vote (pun intended) to make SUSE voting on rocky :D15:14
portdirectpossibly gate on the default images, and the current 'supported' openstack releases?15:14
lamtI don't recall any major failure - most of the issues I remember is with supporting various distribution like fedora and centos - sometimes the package repo change and things break15:14
evrardjpportdirect: or we can just slowly advance, and report on the progress in this meeting?15:14
evrardjp"this week we enabled x as voting, and things seems to be still fine"15:15
jsuchomewhat is slowly advancing?15:15
portdirectim more worried about srwilkers concern15:15
evrardjp"next week we're gonna add x"15:15
portdirectin that if we have too many gates - then murphy will hit us hard15:15
evrardjpthat's happening in every project15:15
portdirectits pretty common atm to see one check fail each time15:15
portdirectand the distrobution of that is fairly even :(15:16
evrardjpis the test flaky ?15:16
portdirecti think mostly infra15:16
evrardjpso maybe we should start on improving reliability, by listing the causes of the rechecks?15:16
srwilkersit's not just the tests -- at times, its the initial chart deployments that fail15:16
portdirectrunning openstack in k8s, in a 8gb ram, 4vcpu vm with horrible i/o is fun15:16
srwilkers^15:16
evrardjpsrwilkers: on that note, reliability is for everyone, gates or not15:17
srwilkersevrardjp: I dont think i stated otherwise15:17
portdirectthe horrible i/o is the reall killer15:17
evrardjpsrwilkers: I know, I am just pointing out the fact it's important to be reliable at all times :D15:17
srwilkersevrardjp: as am i.  however, we're limited by the infrastructure we're provided, as portdirect pointed out15:17
srwilkersunfortunately, thats outside of our control15:17
portdirecthow about 'walking back down'15:18
portdirectie what evrardjp suggested, but starting with stein15:18
evrardjpportdirect: what do you mean?15:18
portdirectand then adding rocky15:18
*** trident has quit IRC15:18
*** belmoreira has quit IRC15:18
portdirectand so on progessivly, untill we feel we've reached the nadir15:18
evrardjpI think the concern is not about the way up or down, it's the fact we can trust gate reliability15:18
portdirectyup15:18
evrardjp(and the fact that we have ppl monitoring those jobs)15:19
srwilkersthat's a valid point and i'm glad you brought it up15:20
*** trident has joined #openstack-meeting-415:20
portdirectok - sounds like we dont think we are ready to add new gates atm?15:22
portdirectevrardjp / srwilkers ?15:22
evrardjpI know I won't check periodics for things outside suse/rocky. When I will do a PR, I will check if things work, and try to fix things if not working, whether SUSE or not. But I can't promise more.15:22
evrardjpI prefer not promising and over perform than the other way around. :p15:22
srwilkersif a change is made, it should ideally be vetted against all jobs we've decided to promote to gates or periodics15:22
srwilkersquite frankly, i find it frustrating that there's not much concern outside of myself for making sure the periodic jobs run successfully15:23
evrardjpcould you clarify srwilkers?15:23
evrardjpsrwilkers: I understand, and feel your pain.15:23
evrardjpI am sorry there aren't more eyes on this.15:23
evrardjpyou know you can ping ppl if necessary too, right? :p15:24
evrardjpportdirect: I let the decision to ppl monitoring said jobs, and to your authority15:24
evrardjpI leave the decision to*15:25
evrardjpor whatever it is in english15:25
portdirectok - lets revisit this once we have some stats on where we see failures currently15:26
portdirectand untill then leave the voting jobs as they are.15:26
portdirect#topic Backup/restore Enhancements for MariaDB and Postgresql Databases15:27
*** openstack changes topic to "Backup/restore Enhancements for MariaDB and Postgresql Databases (Meeting topic: openstack-helm)"15:27
evrardjpportdirect: sounds good15:27
portdirectcliff is this you?15:27
cliffparsonsportdirect: yes15:28
cliffparsonsHello everyone, good morning!15:28
cliffparsonsI wanted to make everyone aware of the pending backup/restore enhancements for MariaDB and Postgresql.15:28
cliffparsonsThe patchsets that introduce these enhancements: https://review.opendev.org/#/c/655231/ and https://review.opendev.org/#/c/661853/15:28
cliffparsonsThe Postgresql and MariaDB releases already have automatic backup and restore capability from a local PVC or host path volume.15:29
cliffparsonsThe two patchsets above are enhancing it so that the databases can be backed up to a remote platform via Swift API;15:29
cliffparsonsin our case we have chosen Ceph RGW, which will be running in a different cluster.  But to test it in the OSH gate environment, we15:29
cliffparsonsuse the ceph rgw in the same cluster. The keystone dependency is required to create a keystone user to use the Swift API.15:29
cliffparsonsSo I was wondering if anyone has seen these patchsets, and if so, are there any concerns?15:30
srwilkerstaking a peek real quick15:30
portdirecti'll need to look properly at: https://review.opendev.org/#/c/655231/56/tools/deployment/backup-restore/backup-restore-tests.sh15:32
srwilkershttps://review.opendev.org/#/c/661853/62/helm-toolkit/templates/manifests/_job-ks-user.yaml.tpl seems like a sane change, but maybe should have been a separate change along with defaults for these into the jobs that consume that template15:32
*** ktibi has quit IRC15:32
cliffparsonssrwilkers: I can split out the job definition if that makes more sense15:33
cliffparsonsjust wanted to show that it's tested15:33
portdirectcliffparsons: that would be great - it would probably also make sense to add this to all the jobs?15:33
srwilkersportdirect: i think it does -- at the moment, we've got some jobs that support these overrides, and others that dont15:34
cliffparsonsportdirect: assuming you're referring to enabling the backups and testing even on the voting gates?15:34
srwilkerswould be nice to make that standard15:34
portdirectcliffparsons: i was refering to the params added here: https://review.opendev.org/#/c/661853/62/helm-toolkit/templates/manifests/_job-ks-user.yaml.tpl15:34
portdirectwould be a great addition to all jobs15:35
cliffparsonsoh15:35
srwilkersalso, wondering if it makes sense to just add testing this functionality to the openstack-support job we already have15:35
portdirectalso i wonder if we should make the default to retry 'indefinately'15:35
srwilkersinstead of creating an entirely new job for testing backup functionality15:35
portdirecti think that makes sense15:35
srwilkersportdirect: that default would make me happy honestly15:35
cliffparsonssrwilkers: originally started doing that but was a little unsure if we wanted to do that15:36
lamt++15:36
cliffparsonsportdirect: thanks for the suggestion, I'll work on that15:36
srwilkersfor a start, it would make debugging chart job failures in our zuul jobs easier15:36
cliffparsonssrwilkers: agree15:36
cliffparsonsAny other concerns or comments?15:37
portdirectnone atm - thanks cliffparsons for taking this on15:37
srwilkerscliffparsons: nope15:37
cliffparsonsportdirect: Thanks for the floor15:37
portdirect#topic OVS-DPDK CI needs KVM support in nodepool VM and DPDK support of the virtual NIC in VM15:38
*** openstack changes topic to "OVS-DPDK CI needs KVM support in nodepool VM and DPDK support of the virtual NIC in VM (Meeting topic: openstack-helm)"15:38
cheng1this is mine15:38
portdirectcheng1: floor is yours15:38
cheng1I am working on ovs-dpdk CI job15:38
cheng1As I can see we have two kinds of VMs in openstack CI15:39
cheng1allocated from nodepool15:39
cheng1#1. Has two interfaces, one with a public ip and the other with a private ip. They don't have pci address. KVM is not supported in the VM15:39
*** e0ne has quit IRC15:39
cheng1#2. With only one interface which has pci address. KVM is supported in the VM15:39
srwilkersah15:40
cheng1For ovs-dpdk test, we need kvm support and the NIC should have pci address15:40
*** david-lyle is now known as dklyle15:40
cheng1Not sure if anyone here is similar with openstack CI VMs15:41
cheng1anyway to customize the VM15:41
cheng1?15:41
portdirectok so we need to run this in #2?15:41
portdirector do neither of the profiles provide what we need?15:42
cheng1#2 is good except it has only 1 NIC15:42
cheng1which has a public ip15:43
portdirectok - lets see what we can do15:43
cheng1thanks portdirect15:43
portdirectfollowing this meeting do you have time to join #openstack-infra, and we can ask them there?15:43
cheng1let me check the meeting time first. It's already 11:44 PM15:44
portdirectok - that makes sense!15:44
portdirectI'll ping them while we discuss the next topic and see if we can circle back here15:44
portdirectok to move on for now?15:44
cheng1sure, portdirect thanks15:44
portdirect#topic How to make metadata update/create/delete available on Horizon?15:44
*** openstack changes topic to "How to make metadata update/create/delete available on Horizon? (Meeting topic: openstack-helm)"15:44
zhipeng[m]y15:45
zhipeng[m]Anyone can help on this question?15:46
portdirectnot off the top of my head15:46
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/JWUOrYLfNTMegzKgndZRZXzC >15:46
portdirectcan you provide the config that you'd be looking to apply at the pod end15:46
portdirectand we can then work out how to get that into inputs for the chart15:47
portdirector make any required changes15:47
zhipeng[m]ok15:48
lamtI can help with that if you can provide the configuration you have in mind15:48
zhipeng[m]overides for horizon?15:49
cheng1Do you have the same issue with other interface more than horizon? zhipeng[m]15:49
cheng1like openstack cmdline15:50
zhipeng[m]not sure about it15:50
srwilkerszhipeng[m]: i think the ask here from portdirect and lamt is to provide them the required configuration for enabling those operations on metadata through horizon, so that they can work to get that functionality into the horizon chart if it's missing15:51
srwilkersas i'm not entirely sure what configurations required to make that happen either15:51
zhipeng[m]@lamt,  which config you want to check?15:52
portdirectthough here we need to bottom out cheng1's question as well15:52
*** trident has quit IRC15:52
cheng1Then you can't make sure the problem is on horizon or nova itself15:52
portdirectif its not just horizon, then is a policy.json issue somewhere else15:52
portdirectand right now - our policy.jsons in the charts are default ocata era ones15:53
lamt++, want to make sure the problem is just with horizon and not with nova15:53
portdirectso that may well be it for starlinx15:53
zhipeng[m]OK, I will check further, thanks!15:53
lamtdoes starlingX use ocata as well?15:53
zhipeng[m]no15:53
cheng1stein I think15:54
zhipeng[m]we switch to stein now15:54
lamtzhipeng[m]: if you have questions, feel free to ping folks in the #openstack-helm channel15:54
lamtah15:54
zhipeng[m]thanks lamt!   will ping you later:)15:54
portdirect#topic In cinder pod, the default type is missing. Is it a helm chart issue?15:55
*** openstack changes topic to "In cinder pod, the default type is missing. Is it a helm chart issue? (Meeting topic: openstack-helm)"15:55
portdirectzhipeng[m] think this is you?15:55
*** trident has joined #openstack-meeting-415:55
portdirectok - lets discuss this further in the channel15:56
portdirect#topic DPDK checks revisited15:57
*** openstack changes topic to "DPDK checks revisited (Meeting topic: openstack-helm)"15:57
zhipeng[m]y15:58
portdirectok - I reached out to -infra and both fungi and clarkb were very helpful15:58
portdirectunfortaulty we cannot select which provider we use in nodepool15:58
fungihelpful as we can be anyway15:58
portdirectand if we could, there is no gtee that the prfile of the vms would be the same15:58
portdirecthttps://docs.openstack.org/infra/manual/testing.html15:58
portdirect^ a great ref here15:59
cheng1If that, we can test very limited cases for ovs-dpdk feature15:59
portdirectcheng1: that would be a great start15:59
fungithe short answer is that the environment we provide is targeted at testing software in isolation, it's not really geared toward testing features of the environment itself15:59
fungiso jobs should avoid depending on test nodes having a specific number of network interfaces, hypervisor, processor flags, et cetera16:00
portdirectfungi: thats fair - and we love what we can get out of -infra atm :D16:00
*** e0ne has joined #openstack-meeting-416:01
portdirectok - we out of time16:03
portdirectthanks for coming peeps16:03
portdirectsee you in #openstack-helm16:03
portdirect#endmeeting16:03
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:03
openstackMeeting ended Tue Jul 23 16:03:21 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:03
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_helm/2019/openstack_helm.2019-07-23-15.01.html16:03
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_helm/2019/openstack_helm.2019-07-23-15.01.txt16:03
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_helm/2019/openstack_helm.2019-07-23-15.01.log.html16:03
zhipeng[m]see you , thanks!16:03
*** cheng1 has quit IRC16:04
*** lemko has quit IRC16:08
*** cheng1 has joined #openstack-meeting-416:08
*** bh526r has quit IRC16:15
*** lpetrut has quit IRC16:19
*** iyamahat has joined #openstack-meeting-416:19
*** gagehugo has left #openstack-meeting-416:19
*** yamamoto has joined #openstack-meeting-416:43
*** yamamoto has quit IRC16:48
*** yamamoto has joined #openstack-meeting-416:49
*** k_mouza has quit IRC16:49
*** bh526r has joined #openstack-meeting-416:54
*** roman_g has quit IRC17:03
*** roman_g has joined #openstack-meeting-417:05
*** k_mouza has joined #openstack-meeting-417:06
*** trident has quit IRC17:07
*** trident has joined #openstack-meeting-417:10
*** k_mouza has quit IRC17:10
*** yamamoto has quit IRC17:11
*** yamamoto has joined #openstack-meeting-417:13
*** yamamoto has quit IRC17:18
*** e0ne has quit IRC17:33
*** ralonsoh has quit IRC17:36
*** iyamahat has quit IRC17:43
*** cliffparsons has left #openstack-meeting-418:02
*** psachin has quit IRC18:19
*** igordc has joined #openstack-meeting-418:27
*** jamesgu has quit IRC18:32
*** e0ne has joined #openstack-meeting-418:42
*** igordc has quit IRC19:00
*** dwalt has left #openstack-meeting-419:05
*** iyamahat has joined #openstack-meeting-419:43
*** jsuchome has quit IRC19:46
*** obi12341 has quit IRC20:02
*** e0ne has quit IRC20:06
*** slaweq has quit IRC20:08
*** roman_g has quit IRC20:12
*** iyamahat has quit IRC20:18
*** pcaruana has quit IRC20:50
*** slaweq has joined #openstack-meeting-420:56
*** slaweq has quit IRC21:01
*** iyamahat has joined #openstack-meeting-421:20
*** lpetrut has joined #openstack-meeting-421:22
*** e0ne has joined #openstack-meeting-421:22
*** lpetrut has quit IRC21:22
*** lpetrut has joined #openstack-meeting-421:23
*** iyamahat has quit IRC21:26
*** lpetrut has quit IRC21:30
*** irclogbot_2 has quit IRC21:32
*** altlogbot_3 has quit IRC21:33
*** altlogbot_2 has joined #openstack-meeting-421:34
*** irclogbot_3 has joined #openstack-meeting-421:34
*** yamamoto has joined #openstack-meeting-421:50
*** e0ne has quit IRC21:50
*** yamamoto has quit IRC21:55
*** mfuller has joined #openstack-meeting-421:56
*** irclogbot_3 has quit IRC21:59
*** altlogbot_2 has quit IRC22:01
*** altlogbot_3 has joined #openstack-meeting-422:22
*** TxGirlGeek has quit IRC22:25
*** altlogbot_3 has quit IRC22:27
*** mfuller has quit IRC22:34
*** diablo_rojo has joined #openstack-meeting-422:41
*** bh526r has quit IRC22:45
*** altlogbot_1 has joined #openstack-meeting-423:14
*** altlogbot_1 has quit IRC23:19
*** altlogbot_3 has joined #openstack-meeting-423:28
*** irclogbot_2 has joined #openstack-meeting-423:32
*** igordc has joined #openstack-meeting-423:35
*** diablo_rojo has quit IRC23:54
*** mfuller has joined #openstack-meeting-423:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!