Wednesday, 2018-12-12

*** munimeha1 has quit IRC00:06
*** sthussey has quit IRC01:19
*** jamesgu_ has quit IRC01:25
*** aaronsheffield has quit IRC01:27
*** cfriesen has quit IRC01:59
*** cfriesen has joined #openstack-helm02:44
*** cfriesen has quit IRC06:31
*** nguyenhai_ has quit IRC06:47
*** nguyenhai has joined #openstack-helm07:07
*** openstackgerrit has quit IRC07:29
*** openstackgerrit has joined #openstack-helm10:04
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add openSUSE Leap 15 openvswitch image  https://review.openstack.org/62114510:04
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: Add openSUSE Leap 15 openvswitch image  https://review.openstack.org/62114510:05
*** gkadam has joined #openstack-helm10:05
*** gkadam has quit IRC10:05
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-helm master: Imported Translations from Zanata  https://review.openstack.org/62285810:26
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: Add periodics  https://review.openstack.org/62465410:50
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120611:36
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120611:38
evrardjpsrwilkers: https://review.openstack.org/#/c/624654/11:43
evrardjpif those periodic jobs fail, there are changes in upstream images/content we need to care.11:44
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120612:11
*** dimitris_ has joined #openstack-helm12:14
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120612:30
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120612:51
evrardjpportdirect: is the user created?13:05
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: Ceph: Allow multiple test pods to be present in clusters  https://review.openstack.org/62469313:30
srwilkerso/13:59
*** aojea has joined #openstack-helm14:01
openstackgerritMike Pham proposed openstack/openstack-helm master: Enable Egress policy enforcement  https://review.openstack.org/61453814:05
openstackgerritMike Pham proposed openstack/openstack-helm master: WIP: Implement East-West EGRESS policies for the Cloud Core services  https://review.openstack.org/62331514:20
*** irclogbot_1 has quit IRC14:32
openstackgerritRahul Khiyani proposed openstack/openstack-helm master: Keystone podSecurityContext  https://review.openstack.org/61401114:34
*** irclogbot_1 has joined #openstack-helm14:41
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: Publish images on DockerHub and Quay.io  https://review.openstack.org/62471314:42
evrardjpportdirect: I am now waiting for a chat with you for credentials.14:42
evrardjpsrwilkers: https://review.openstack.org/#/c/624654/114:43
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: DNM: Test elastic query in post run job  https://review.openstack.org/62443514:52
*** irclogbot_1 has quit IRC14:55
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Fluentbit: Add Decode_Field_As config to docker parser  https://review.openstack.org/62472114:55
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120614:58
evrardjpsrwilkers: you had questions about periodic, right, or did I dream about it?14:59
srwilkersevrardjp: i had some.  it's clear how to define them in the projects zuul layout, but was difficult to find more information beyond just that (how to define frequency, where the results are published to, etc)15:00
evrardjpSo15:02
evrardjpYou can't change frequency15:02
evrardjpit's daily15:02
evrardjpjust like that :)15:02
evrardjpfor publication of results:15:03
evrardjphttp://zuul.openstack.org/builds?pipeline=periodic15:03
evrardjpso,for OSH: http://zuul.openstack.org/builds?pipeline=periodic&project=openstack%2Fopenstack-helm15:04
srwilkersoh nice15:07
srwilkersthanks :)15:07
srwilkerslearned something new today15:08
*** irclogbot_1 has joined #openstack-helm15:08
openstackgerritMike Pham proposed openstack/openstack-helm master: WIP: Implement East-West EGRESS policies for the Cloud Core services  https://review.openstack.org/62331515:09
*** irclogbot_1 has joined #openstack-helm15:19
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Update ironic multinode periodic check nodeset  https://review.openstack.org/62473415:22
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: WIP: Update ironic multinode periodic check nodeset  https://review.openstack.org/62473415:23
evrardjpsrwilkers: for more periodics: https://review.openstack.org/#/c/624654/115:26
evrardjpTrying to get my votes!15:27
evrardjpportdirect: please have a look at https://review.openstack.org/#/c/624713/15:27
evrardjpI even gave you, in the comments, what you should do so that we have our secrets done.15:27
evrardjpportdirect: please use outfile to avoid copy paste issues15:28
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120615:38
*** aaronsheffield has joined #openstack-helm16:02
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: Ceph: Allow multiple test pods for ceph-client to be present in clusters  https://review.openstack.org/62474816:03
*** jamesgu_ has joined #openstack-helm16:06
evrardjphow is everyone here?16:19
evrardjpAwaken up?16:19
evrardjpdo you need more coffee?16:19
openstackgerritSophie Huang proposed openstack/openstack-helm master: Adding livenessProbe to cinder-api container  https://review.openstack.org/62421116:22
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: DNM: Test elastic query in post run job  https://review.openstack.org/62443516:24
evrardjpportdirect: Should we add the OSH-images process into 1.0 storyboard?16:25
evrardjpThere are a few tasks about moving charts, but there is no mention of images16:25
openstackgerritMike Pham proposed openstack/openstack-helm master: WIP: Implement East-West EGRESS policies for the Cloud Core services  https://review.openstack.org/62331516:25
*** cfriesen has joined #openstack-helm16:42
*** cfriesen has joined #openstack-helm16:42
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120616:43
openstackgerritGage Hugo proposed openstack/openstack-helm master: WIP - Add credential delete hook to keystone chart  https://review.openstack.org/62413116:58
cfriesenhas anyone tested pod recovery with a single-replica mariadb?  when we reboot the kubernetes node that mariadb is running on, it doesn't get rescheduled to the other controller node.  This is with the latest mariadb chart updates from "master".16:58
cfriesenportdirect: you submitted the recent mariadb changes...any thoughts on this?  ^17:03
cfriesenportdirect: when the original node comes back up, mariadb ends up in CrashBackOff with the following logs: http://paste.openstack.org/show/737153/17:04
cfriesenfor what it's worth, /var/lib/mysql/grastate.dat from the crashed container appears to be empty.17:19
openstackgerritchinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph: Journal partition automation  https://review.openstack.org/61433117:20
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120617:28
evrardjpcfriesen: I have not17:34
evrardjpbut at least I answered you.17:34
cfriesen:)17:35
evrardjpHas anyone talked with the SAPCC folks?17:35
evrardjpI can try to help you cfriesen if you want17:35
cfriesenI haven't talked with anyone from SAPCC.  happy to take any help.  I was just about to start reading through the handling of grastate.dat in the chart.17:36
*** ianychoi has joined #openstack-helm17:38
mattmceuenhey cfriesen, just saw this - sorry, I haven't tested that scenario.  give portdirect a bit to get back to you, he's one of a handful of folks shouldering a lot in the office while many others (like me) are on holiday :)17:46
cfriesenno problem, I know how it goes.17:46
*** jamesgu__ has joined #openstack-helm18:00
*** jamesgu_ has quit IRC18:04
openstackgerritMike Pham proposed openstack/openstack-helm-infra master: Add Egress Helm-toolkit function  https://review.openstack.org/61250618:12
*** aojea has quit IRC18:23
*** sgrasley_ has joined #openstack-helm18:43
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Elasticsearch: Update helm test  https://review.openstack.org/62478318:44
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120618:46
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: DNM: Test elastic query in post run job  https://review.openstack.org/62443518:47
*** sgrasley has quit IRC18:47
*** sgrasley_ has quit IRC18:48
openstackgerritDmitrii Kabanov proposed openstack/openstack-helm master: Add startingDeadlineSeconds field to cronJobs  https://review.openstack.org/60985919:08
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120619:14
openstackgerritMike Pham proposed openstack/openstack-helm-infra master: Add Egress Helm-toolkit function  https://review.openstack.org/61250619:14
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120619:18
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add SUSE Leap15 for newton images  https://review.openstack.org/62162319:18
openstackgerritMerged openstack/openstack-helm-infra master: Ceph: Allow multiple test pods to be present in clusters  https://review.openstack.org/62469319:18
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add SUSE Leap15 for newton images  https://review.openstack.org/62162319:19
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add SUSE Leap15 for newton images  https://review.openstack.org/62162319:21
*** jamesgu__ has quit IRC19:21
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: Add openSUSE Leap 15 images  https://review.openstack.org/62162319:21
*** ln0 has joined #openstack-helm19:36
openstackgerritchinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph: Journal partition automation  https://review.openstack.org/61433119:37
ln0Hello. Anybody with issues regarding Keystone ? I was following the development guide, went fine until this:19:38
ln0https://pastebin.com/G2aJuGWQ19:39
portdirectln0: can you get the logs from the keystone API init container19:40
ln0kubectl logs keystone-api-b78b499b8-6cncd -n openstack19:41
ln0Error from server (BadRequest): container "keystone-api" in pod "keystone-api-b78b499b8-6cncd" is waiting to start: PodInitializing19:41
openstackgerritAndreas Jaeger proposed openstack/openstack-helm master: Update docs building  https://review.openstack.org/62479319:44
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120619:44
portdirectAdd '-c init' to the end of that command19:44
ln0Uh, thanks.19:45
ln0Entrypoint WARNING: 2018/12/12 19:45:07 entrypoint.go:71: Resolving dependency Job keystone-db-sync in namespace openstack failed: Job Job keystone-db-sync in namespace openstack is not completed yet19:45
openstackgerritAndreas Jaeger proposed openstack/openstack-helm master: Update docs building  https://review.openstack.org/62479319:46
portdirectln0: can you get logs from that jobs pod?19:47
ln0portdirect: I can describe it, but not sure if kubectl logs can help ?19:50
openstackgerritSophie Huang proposed openstack/openstack-helm master: Adding livenessProbe to cinder-api container  https://review.openstack.org/62421119:55
ln0Looks like similar to: https://bugs.launchpad.net/openstack-helm/+bug/178091519:57
openstackLaunchpad bug 1780915 in openstack-helm "Fail to connect to keystone after deploy following the guide(NFS provider)" [Undecided,Confirmed] - Assigned to Naeil ZOUEIDI (naeilzoueidi)19:57
portdirectln0: i think these are unrelated20:10
portdirectthe pods from the job would really help, but a describe would too20:11
ln0Here it is: https://pastebin.com/QbQDnGhy20:12
portdirecthmm - the lack of events there is odd20:13
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120620:22
openstackgerritMike Pham proposed openstack/openstack-helm-infra master: Add Egress Helm-toolkit function  https://review.openstack.org/61250620:24
*** ln0 has quit IRC20:26
*** ln0 has joined #openstack-helm20:27
cfriesenportdirect: when you get a minute, could you look at my issue from earlier?20:27
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Elasticsearch: Define success criteria for adding snapshot repo  https://review.openstack.org/62480820:45
*** ln0 has quit IRC20:55
openstackgerritJean-Philippe Evrard proposed openstack/openstack-helm-images master: [WIP] Add LOCI to OSH images  https://review.openstack.org/62120621:19
portdirectcfriesen: yeah - this is a side effect of using a statefulset21:53
portdirectk8s will not move the pods when a node goes down21:53
portdirectso if you're running a single replica then yes you'll have to manually intervene if that node goes down21:54
cfriesenportdirect: okay, that explains why it didn't move when the node went down.  what about why it's in CrashBackOff when the node comes back up?  I can see the contents of the "osh-openstack-mariadb-mariadb-state" configmap, but for some reason the container seems to be having problems with it.21:58
cfriesenincidentally, what manual intervention would you expect this to need?21:59
*** jamesgu__ has joined #openstack-helm22:01
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: DNM: Test elastic query in post run job  https://review.openstack.org/62443522:03
evrardjpportdirect: don't forget to act on https://review.openstack.org/#/c/624713/22:04
openstackgerritMerged openstack/openstack-helm-infra master: Ingress: Remove server headers from response  https://review.openstack.org/62442922:13
*** mnasiadka_ has joined #openstack-helm22:23
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Fluentbit: Add Decode_Field_As config to docker parser  https://review.openstack.org/62472122:24
*** mnasiadka has quit IRC22:29
*** mnasiadka_ is now known as mnasiadka22:29
openstackgerritMerged openstack/openstack-helm-infra master: Elasticsearch: Update helm test  https://review.openstack.org/62478322:30
cfriesenportdirect: it appears that my problem is due to grastate.dat being present but empty.  This passes the if os.path.exists() test, but then chokes in get_grastate_val()22:32
cfriesenportdirect: I modified the code to delete  grastate.dat if it was zero-size, then hit this: http://paste.openstack.org/show/737172/22:44
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: DNM: Test elastic query in post run job  https://review.openstack.org/62443522:50
*** sgrasley has joined #openstack-helm22:55
*** sgrasley_ has joined #openstack-helm22:58
*** sgrasley_ has quit IRC23:02
*** sgrasley has quit IRC23:02
openstackgerritMerged openstack/openstack-helm-infra master: Elasticsearch: Define success criteria for adding snapshot repo  https://review.openstack.org/62480823:23

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!