Monday, 2019-09-30

*** kranthikirang has joined #openstack-helm00:33
*** kranthikirang has quit IRC00:38
*** Liang__ has joined #openstack-helm01:08
*** Liang__ has left #openstack-helm01:17
openstackgerritTin Lam proposed openstack/openstack-helm master: Implements egress network policy  https://review.opendev.org/68563401:37
openstackgerritTin Lam proposed openstack/openstack-helm master: Implements egress network policy  https://review.opendev.org/68563401:40
*** kranthikirang has joined #openstack-helm02:21
*** kranthikirang has quit IRC02:26
zhipeng[m]Hi all, after upgrade openstack-helm, I always encounter below issue for horizon02:38
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/vZrXJTqJzjsbQGydUrZxBOib >02:38
zhipeng[m]Is there anyone experiencing the same issue?02:39
lamtzhipeng[m]: I have not seen a readiness probe failure with horizon. Are you seeing this using what's in master without any customization?02:42
zhipeng[m]Before upgrading ,it really works with our costomization02:43
lamtwhile working on network policies, I had seen some probe failures, but that's not enabled02:43
zhipeng[m]now,  horizon container is running but always not ready02:44
portdirectIs it actually serving requests on port 80?02:44
zhipeng[m]yes02:44
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/aGCeseZbPxzAoVYQulspmZVn >02:45
zhipeng[m]this part added in latest version, is there any impact02:45
portdirectOk, so to confirm, you curl the pods ip at port 80, and get a <400 http response back?02:45
*** markvoelker has joined #openstack-helm02:46
portdirectIf so, then this looks to be a k8s  or network policy issue.02:46
zhipeng[m]you mean?02:47
zhipeng[m]controller-0:~$ curl 172.16.192.109:8002:47
zhipeng[m]curl: (7) Failed to connect to 172.16.192.109: Invalid argument02:47
zhipeng[m]controller-0:/var/log/containers$ curl 172.16.192.109:8002:48
zhipeng[m]curl: (7) Failed connect to 172.16.192.109:80; Connection refused02:48
zhipeng[m] *02:48
portdirectOk, the invalid argument is odd, you running some dual stackery?02:49
zhipeng[m]see my update above02:50
portdirectBut the latter, well that's not looking great :(02:50
lamtyou don't have any network policy right?02:50
zhipeng[m]not have this one02:50
portdirectCan you paste the logs from the horizon pod?02:51
zhipeng[m] network_policy: false02:51
zhipeng[m]we set it to false before02:51
lamtYeah, curious if horizon is actually running and listening for request02:52
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/lRTgPREXApeCWlimsZzhnoQJ >02:53
lamtcan you do kubectl -n openstack logs <horizon pod>?02:54
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/HLlWREJTSgoqdAnthGLxPkLP >02:54
portdirectWhat cni are you running with?02:56
portdirectLooks like calico, multus/chi genie and somthing else?02:56
zhipeng[m]calico02:57
portdirectWhat's providing this annotation02:58
portdirectk8s.v1.cni.cncf.io/networks-status02:58
portdirectThat looks like multus?02:59
zhipeng[m]not sure03:00
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/pjMZmoVhUAKPMMOsADxbUtQA >03:03
zhipeng[m] * multus is a top level cni, it will call calico03:04
zhipeng[m] * multus is a top level cni, it will call calico03:04
portdirectI suspect somthing is up with your network config03:12
portdirectCan you attempt connection to apache from within the same network ns as your pod03:13
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/gIGBvmgAIPrWAtRdRsNYbElo >03:23
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/GnRsUVKkTaHxnmWcnHiBmhJJ >03:23
portdirectTry 127.0.0.103:24
zhipeng[m]apache not started?03:24
portdirectWhy would it be 8080?03:24
portdirectFrom your logs it looks like it has03:25
zhipeng[m]typo03:25
zhipeng[m]controller-0:/var/log/containers$ kubectl -n openstack exec -it  horizon-7f6499b6df-b2468 bash03:25
zhipeng[m][root@horizon-7f6499b6df-b2468 /]# curl 127.0.0.1:8003:25
zhipeng[m]curl: (7) Failed connect to 127.0.0.1:80; Connection refused03:25
portdirectOk, so yeah looks like Apache ain't listening03:25
portdirectI'd check your image at this point03:26
zhipeng[m]The image works with old openstack-helm version03:27
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/VupfQBDqzRsmeorDnsCRisOI >03:27
portdirectOk, can you give the commit you were using, and the one your using now?03:27
zhipeng[m]latest verson add <IfVersion < 2.4>03:27
zhipeng[m]ok03:28
portdirectLooks like that may be an issue for your Apache version?03:28
portdirectIts tested on ubuntu and opensuse03:28
zhipeng[m]apache2.4.703:28
portdirectIf starlingx would add image overrides, we'd be happy to test them too ;)03:29
zhipeng[m]great!!!03:29
zhipeng[m]openstack-helm-6c71637222f47d85681038994f02feac92f75bd2.tar.gz03:30
zhipeng[m]openstack-helm-82c72367c85ca94270f702661c7b984899c1ae38.tar.gz03:30
zhipeng[m]6c71 works   82c7 not work03:30
zhipeng[m]I will paste our overrides03:31
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/pCzPYCUgvRJBfTdwBgEfrNOB >03:34
portdirectJust over-ride the Apache conf to the way it used to be for now03:36
portdirectIf that works for you, then let's get this fixed03:36
portdirectAnd also please let's get starlingx images in the osh gates03:36
zhipeng[m]Apache conf?03:37
zhipeng[m]actually, I override the  values.conf.horizon.apache   with the one we used before03:38
zhipeng[m]however, it still doesn't work.03:40
openstackgerritTin Lam proposed openstack/openstack-helm master: Implements egress network policy  https://review.opendev.org/68563403:44
openstackgerritMerged openstack/openstack-helm-infra master: Prometheus: Relabeling the node-exporter label  https://review.opendev.org/68511903:45
* zhipeng[m] sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/oroflVLwzZQmmKAiOPwNKbhd >03:54
zhipeng[m]portdirect:  Is it possible for below setting04:06
zhipeng[m] allowPrivilegeEscalation: false04:06
*** kranthikirang has joined #openstack-helm04:09
*** kranthikirang has quit IRC04:13
zhipeng[m]@I might find the issue04:34
zhipeng[m] site_dir: /etc/apache2/sites-enable04:35
zhipeng[m]changed to sites-available04:35
zhipeng[m]Apache when starting actually reads the configuration from any .conf files (or symlinks to .conf files) in the sites-enabled folder.04:39
*** markvoelker has quit IRC05:02
*** jamesgu has joined #openstack-helm05:16
*** rezroo has joined #openstack-helm05:17
*** dpawlik has joined #openstack-helm05:28
*** rezroo has quit IRC05:34
zhipeng[m]@port05:38
zhipeng[m] * @freenode_portdirect:matrix.org  I confirmed it caused by this issue05:39
zhipeng[m]btw, does our latest  openstack-helm support  openstack train?05:40
zhipeng[m]we are upgrading openstack to train05:40
zhipeng[m]https://github.com/openstack/openstack-helm/commit/8b8e568d21228c16cfa065e46c709a43f32f81c205:47
zhipeng[m]this patch change    conf->software->apache2-> site_dir:05:47
zhipeng[m] /etc/apache2/sites-available05:47
zhipeng[m]From my side, it should be changed to /etc/apache2/sites-enabled05:47
zhipeng[m]- name: horizon-etc05:47
zhipeng[m]              mountPath: /etc/apache2/sites-enabled/000-default.conf05:47
zhipeng[m]Otherwise,  Apache2 could not read 000-default.conf from above folder05:48
*** kranthikirang has joined #openstack-helm05:58
*** markvoelker has joined #openstack-helm05:58
*** dpawlik has quit IRC06:01
*** markvoelker has quit IRC06:02
*** kranthikirang has quit IRC06:03
*** nick_kar has joined #openstack-helm06:03
*** dpawlik has joined #openstack-helm06:29
*** vsaienk0 has joined #openstack-helm06:36
*** pgaxatte has joined #openstack-helm06:38
*** jamesgu has quit IRC06:47
*** pgaxatte has quit IRC06:58
*** pgaxatte has joined #openstack-helm07:00
*** jsuchome has joined #openstack-helm07:08
*** Darcidride has joined #openstack-helm07:34
*** bobmel has joined #openstack-helm07:39
*** dpawlik has quit IRC07:45
*** kranthikirang has joined #openstack-helm07:46
*** kranthikirang has quit IRC07:50
*** witek has joined #openstack-helm07:52
*** markvoelker has joined #openstack-helm08:00
*** dpawlik has joined #openstack-helm08:04
*** markvoelker has quit IRC08:04
*** jsuchome has quit IRC08:31
*** dpawlik has quit IRC08:39
*** georgk5 has quit IRC08:41
*** georgk has joined #openstack-helm08:41
openstackgerritVasyl Saienko proposed openstack/openstack-helm-infra master: [mysql-exporter] Use flags depending on version  https://review.opendev.org/68396208:42
*** dpawlik has joined #openstack-helm08:59
*** vsaienk0 has quit IRC09:00
*** vsaienk0 has joined #openstack-helm09:01
*** dpawlik has quit IRC09:04
*** cheng1 has quit IRC09:04
openstackgerritVasyl Saienko proposed openstack/openstack-helm master: [Keystone] Fix indentation in values.yaml  https://review.opendev.org/68566309:05
*** dpawlik has joined #openstack-helm09:13
*** cheng1 has joined #openstack-helm09:14
*** dpawlik has quit IRC09:29
*** kranthikirang has joined #openstack-helm09:34
*** kranthikirang has quit IRC09:39
*** dpawlik has joined #openstack-helm09:48
*** vsaienk0 has quit IRC10:12
*** obravo has joined #openstack-helm10:19
*** jsuchome has joined #openstack-helm10:20
*** vsaienk0 has joined #openstack-helm10:20
*** vsaienk0 has quit IRC10:47
*** vsaienk0 has joined #openstack-helm10:57
*** markvoelker has joined #openstack-helm11:01
*** markvoelker has quit IRC11:06
openstackgerritVasyl Saienko proposed openstack/openstack-helm-infra master: [mysql-exporter] Use flags depending on version  https://review.opendev.org/68396211:11
pma78please review https://review.opendev.org/683111 (Support override of ironic agent image)11:14
*** witek has quit IRC11:20
*** kranthikirang has joined #openstack-helm11:22
*** witek has joined #openstack-helm11:23
*** kranthikirang has quit IRC11:26
*** ianychoi has quit IRC11:39
*** ianychoi has joined #openstack-helm11:40
openstackgerritzhipeng liu proposed openstack/openstack-helm master: Add placement chart  https://review.opendev.org/66222912:02
*** nicolasbock has joined #openstack-helm12:03
*** vsaienk0 has quit IRC12:42
*** vsaienk0 has joined #openstack-helm12:45
*** aaronsheffield has joined #openstack-helm12:52
*** bobmel has quit IRC12:59
*** bobmel has joined #openstack-helm12:59
*** markvoelker has joined #openstack-helm13:02
*** markvoelker has quit IRC13:06
*** kranthikirang has joined #openstack-helm13:10
*** kranthikirang has quit IRC13:15
*** sthussey has joined #openstack-helm13:30
*** dpawlik has quit IRC13:36
*** roman_g has joined #openstack-helm13:41
openstackgerritHari proposed openstack/openstack-helm-addons master: Include retry logic for checking status  https://review.opendev.org/68518813:43
*** kranthikirang has joined #openstack-helm13:46
openstackgerritHari proposed openstack/openstack-helm-addons master: Include retry logic for checking status  https://review.opendev.org/68518813:47
*** munimeha1 has joined #openstack-helm13:49
*** obravo has quit IRC14:00
*** bobmel has quit IRC14:15
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Heat: Add ingress network policy overrides  https://review.opendev.org/68572314:25
*** pgaxatte has quit IRC14:32
*** rezroo has joined #openstack-helm14:35
openstackgerritHari proposed openstack/openstack-helm-addons master: Remove ssh configurations  https://review.opendev.org/68539514:48
*** kranthikirang has quit IRC14:48
*** jamesgu has joined #openstack-helm15:01
*** markvoelker has joined #openstack-helm15:02
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Heat: Add ingress network policy overrides  https://review.opendev.org/68572315:06
*** roman_g has quit IRC15:07
*** witek has quit IRC15:17
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Heat: Add ingress network policy overrides  https://review.opendev.org/68572315:20
*** bobmel has joined #openstack-helm15:21
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Heat: Add ingress network policy overrides  https://review.opendev.org/68572315:37
openstackgerritHari proposed openstack/openstack-helm-addons master: Remove ssh configurations  https://review.opendev.org/68539515:48
openstackgerritTin Lam proposed openstack/openstack-helm master: Add horizon ingress override  https://review.opendev.org/68546715:56
*** roman_g has joined #openstack-helm15:59
openstackgerritCliff Parsons proposed openstack/openstack-helm master: Add Cinder ingress network policy overrides  https://review.opendev.org/68575116:02
*** itlinux has joined #openstack-helm16:06
*** AlexNoskov has joined #openstack-helm16:07
openstackgerritSangeet Gupta proposed openstack/openstack-helm-infra master: Memcached: Add ingress network policy overrides  https://review.opendev.org/68575516:29
*** kranthikirang has joined #openstack-helm16:33
*** obravo has joined #openstack-helm16:33
*** coboluxx has joined #openstack-helm16:35
*** obravo has quit IRC16:37
*** kranthikirang has quit IRC16:38
openstackgerritSophie Huang proposed openstack/openstack-helm master: Add keystone ingress netpol  https://review.opendev.org/68193916:38
openstackgerritSophie Huang proposed openstack/openstack-helm master: Add keystone ingress netpol  https://review.opendev.org/68193916:39
openstackgerritSophie Huang proposed openstack/openstack-helm master: Add keystone ingress netpol  https://review.opendev.org/68193916:46
*** cfriesen has joined #openstack-helm17:01
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Tools: Update test-networkpolicy.sh  https://review.opendev.org/68576117:03
*** vsaienk0 has quit IRC17:04
openstackgerritCliff Parsons proposed openstack/openstack-helm master: [WIP] Add Nova ingress network policy overrides  https://review.opendev.org/68576217:05
*** Darcidride has quit IRC17:07
*** igordc has joined #openstack-helm17:08
*** markvoelker has quit IRC17:17
*** markvoelker has joined #openstack-helm17:20
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Heat: Add ingress network policy overrides  https://review.opendev.org/68572317:53
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Tools: Update test-networkpolicy.sh  https://review.opendev.org/68576117:54
*** itlinux has quit IRC17:59
*** gmann is now known as gmann_afk18:19
*** kranthikirang has joined #openstack-helm18:21
*** kranthikirang has quit IRC18:25
*** bobmel has quit IRC18:31
*** bobmel has joined #openstack-helm18:32
*** bobmel has quit IRC18:35
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577518:43
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577519:02
openstackgerritSangeet Gupta proposed openstack/openstack-helm master: Heat: Add ingress network policy overrides  https://review.opendev.org/68572319:08
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577519:13
*** jsuchome has quit IRC19:27
openstackgerritSophie Huang proposed openstack/openstack-helm master: Add keystone ingress netpol  https://review.opendev.org/68193919:30
openstackgerritSophie Huang proposed openstack/openstack-helm master: Add keystone ingress netpol  https://review.opendev.org/68193919:35
*** kranthikirang has joined #openstack-helm19:41
*** markvoelker has quit IRC20:04
*** obravo has joined #openstack-helm20:07
openstackgerritNishant Kumar proposed openstack/openstack-helm-infra master: [PodSecurityPolicy] Move to 'policy/v1beta1' apiVersion  https://review.opendev.org/68544220:08
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577520:09
*** obravo has quit IRC20:13
openstackgerritChi Lo proposed openstack/openstack-helm-addons master: Health probe for Ranger-agent pods  https://review.opendev.org/68550120:16
*** obravo has joined #openstack-helm20:25
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577520:39
*** markvoelker has joined #openstack-helm20:52
*** irclogbot_2 has quit IRC20:54
*** irclogbot_1 has joined #openstack-helm20:55
*** irclogbot_1 has quit IRC21:01
*** irclogbot_3 has joined #openstack-helm21:01
*** irclogbot_3 has quit IRC21:07
*** irclogbot_0 has joined #openstack-helm21:07
*** irclogbot_0 has quit IRC21:13
*** irclogbot_0 has joined #openstack-helm21:14
*** irclogbot_0 has quit IRC21:19
*** irclogbot_3 has joined #openstack-helm21:20
*** irclogbot_3 has quit IRC21:25
*** irclogbot_3 has joined #openstack-helm21:25
*** irclogbot_3 has quit IRC21:33
*** irclogbot_2 has joined #openstack-helm21:35
*** irclogbot_2 has quit IRC21:41
*** irclogbot_2 has joined #openstack-helm21:41
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577521:42
openstackgerritSteve Wilkerson proposed openstack/openstack-helm-infra master: Update Kubernetes version to 1.16.0  https://review.opendev.org/68577521:44
*** irclogbot_2 has quit IRC21:46
*** irclogbot_2 has joined #openstack-helm21:47
openstackgerritMerged openstack/openstack-helm-infra master: Add extensible volume mounts to node-exporter  https://review.opendev.org/68517221:49
*** irclogbot_2 has quit IRC21:53
*** irclogbot_3 has joined #openstack-helm21:53
*** irclogbot_3 has quit IRC21:59
*** irclogbot_2 has joined #openstack-helm21:59
*** irclogbot_2 has quit IRC22:05
*** irclogbot_3 has joined #openstack-helm22:05
*** kranthikirang has quit IRC22:06
*** kranthikirang has joined #openstack-helm22:07
openstackgerritCliff Parsons proposed openstack/openstack-helm master: Add Cinder ingress network policy overrides  https://review.opendev.org/68575122:09
openstackgerritTin Lam proposed openstack/openstack-helm master: Add horizon ingress override  https://review.opendev.org/68546722:10
*** kranthikirang has quit IRC22:12
*** irclogbot_3 has quit IRC22:14
*** irclogbot_2 has joined #openstack-helm22:15
*** kranthikirang has joined #openstack-helm22:17
*** munimeha1 has quit IRC22:28
*** obravo has quit IRC22:35
*** obravo has joined #openstack-helm22:36
*** obravo has quit IRC22:38
*** kranthikirang has quit IRC22:45
*** kranthikirang has joined #openstack-helm22:46
*** kranthikirang has quit IRC22:50
*** AlexNoskov has quit IRC22:58
*** aaronsheffield has quit IRC23:12
openstackgerritAndrii Ostapenko proposed openstack/openstack-helm-infra master: Make all prints python3 compatible  https://review.opendev.org/68586123:30
openstackgerritAndrii Ostapenko proposed openstack/openstack-helm-infra master: Make all prints python3 compatible  https://review.opendev.org/68586123:33
openstackgerritTin Lam proposed openstack/openstack-helm-infra master: Add default netpol to LMA charts  https://review.opendev.org/68586223:38
openstackgerritTin Lam proposed openstack/openstack-helm-infra master: Add default netpol to LMA charts  https://review.opendev.org/68586223:40
*** kranthikirang has joined #openstack-helm23:45
*** kranthikirang has quit IRC23:50
*** sthussey has quit IRC23:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!