| *** jaypipes has quit IRC | 00:09 | |
| openstackgerrit | Koffi Nogbe proposed openstack/openstack-helm-infra master: Add enhancement to mariadb backup https://review.opendev.org/661853 | 00:15 |
|---|---|---|
| *** sreejithp_ has quit IRC | 00:16 | |
| *** Liang__ has joined #openstack-helm | 00:16 | |
| *** Liang__ has left #openstack-helm | 00:21 | |
| *** michael-beaver has quit IRC | 00:25 | |
| *** itlinux has joined #openstack-helm | 00:55 | |
| *** sreejithp has joined #openstack-helm | 01:13 | |
| *** happyhemant has quit IRC | 01:23 | |
| *** itlinux has quit IRC | 01:37 | |
| *** sreejithp has quit IRC | 01:39 | |
| *** bobmel has joined #openstack-helm | 01:43 | |
| *** itlinux has joined #openstack-helm | 01:45 | |
| *** bobmel has quit IRC | 01:47 | |
| *** itlinux has quit IRC | 01:56 | |
| *** sreejithp has joined #openstack-helm | 02:22 | |
| *** itlinux has joined #openstack-helm | 02:25 | |
| *** Talion has quit IRC | 02:43 | |
| openstackgerrit | Tin Lam proposed openstack/openstack-helm-infra master: Replace git.openstack.org URLs with opendev.org URLs https://review.opendev.org/655048 | 02:44 |
| *** cfriesen has quit IRC | 02:47 | |
| *** sreejithp_ has joined #openstack-helm | 02:50 | |
| *** sreejithp has quit IRC | 02:53 | |
| *** bobmel has joined #openstack-helm | 03:44 | |
| *** altlogbot_3 has quit IRC | 03:44 | |
| *** altlogbot_0 has joined #openstack-helm | 03:45 | |
| *** bobmel has quit IRC | 03:48 | |
| *** altlogbot_0 has quit IRC | 04:38 | |
| *** altlogbot_1 has joined #openstack-helm | 04:39 | |
| *** altlogbot_1 has quit IRC | 04:40 | |
| *** altlogbot_3 has joined #openstack-helm | 04:41 | |
| *** sreejithp_ has quit IRC | 04:49 | |
| *** akki58 has joined #openstack-helm | 05:10 | |
| akki58 | hello team, i am running the following command in order to deploy the ingress controller | 05:10 |
| akki58 | ./tools/deployment/multinode/020-ingress.sh | 05:10 |
| akki58 | it makes two pods ingress-error-pages-899888c7-pnxhv and ingress-error-pages-899888c7-w5d5b which remains in pending state | 05:11 |
| akki58 | in kubectl describe it is showing Warning FailedScheduling 60s (x3 over 2m12s) default-scheduler 0/3 nodes are available: 3 node(s) didn't match node selector | 05:11 |
| akki58 | please guide me how to resolve this issue | 05:11 |
| *** akki58 has quit IRC | 05:43 | |
| *** cfriesen has joined #openstack-helm | 05:54 | |
| *** dpawlik has joined #openstack-helm | 06:03 | |
| *** aojea has joined #openstack-helm | 06:17 | |
| *** jamesgu_ has quit IRC | 06:29 | |
| *** jsuchome has joined #openstack-helm | 06:36 | |
| *** bobmel has joined #openstack-helm | 06:40 | |
| *** roman_g has joined #openstack-helm | 06:49 | |
| *** cfriesen has quit IRC | 07:02 | |
| *** witek has joined #openstack-helm | 07:27 | |
| *** itlinux has quit IRC | 07:42 | |
| *** Emine has joined #openstack-helm | 07:43 | |
| *** bobmel has quit IRC | 08:10 | |
| openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm master: Change set variable in openstack helm gate job https://review.opendev.org/660695 | 08:32 |
| mbuil | In other to deploy multinode, I am trying to understand how to handle config which is different in each node (e.g. the name of the interfaces). I found this patch https://review.opendev.org/#/c/639087/ which apparently provides support for different configs per host, however, I don't see how it can do different configs per host. Can anyone help please? thx | 09:34 |
| mbuil | s/other/order | 09:34 |
| openstackgerrit | saichirumarri proposed openstack/openstack-helm master: [DNM] Updating weight value from 10 to 100 https://review.opendev.org/661730 | 09:51 |
| *** HagunKim has quit IRC | 10:14 | |
| openstackgerrit | saichirumarri proposed openstack/openstack-helm-infra master: [DNM] updating weight value from 10 to 100 https://review.opendev.org/661935 | 10:28 |
| itxaka | akki58 your nodes probably dont have the proper labels | 10:53 |
| itxaka | afaics the ingress-error-pages needs openstack-control-plane=enabled in the node labels | 10:54 |
| itxaka | mbuil seems that any special host-only config needs to be dealt with in the init scripts for the service | 10:55 |
| itxaka | of course, it seems that you need to know in advance the config values per host, unless those are dynamic, in which case I suppose you should just use the init scripts to do so | 10:56 |
| *** happyhemant has joined #openstack-helm | 10:56 | |
| *** jaypipes has joined #openstack-helm | 10:57 | |
| itxaka | mbuil, what are you exactly trying to do? Maybe with a specific example someone can provide guidance on that | 10:57 |
| happyhemant | Hello everyone, I am tunning openstack monitoring services and I found out that openstack placement api service is in critical state does anybody has any idea how to resolve this issue | 11:54 |
| happyhemant | in promethus-openstack-exporter logs i found this `Service placement check failed (returned '200' but expected '[401]')` | 11:55 |
| happyhemant | help will be much appreciated thanks | 11:55 |
| mbuil | itxaka: I am trying to set the variable local_ip in the openvswitch_agent config file. That ip is fetch from the interface set as "tunnel_interface" ==> https://github.com/openstack/openstack-helm/blob/aaea081765c74b0366254d90ef0a423b19bd85da/neutron/templates/bin/_neutron-openvswitch-agent-init.sh.tpl#L65 | 11:56 |
| mbuil | itxaka: but what if the interface I want to use is different in each node? How that would be solved? Is there a way to add a conditional in Helm that checks the name of the host where it is deploying? | 11:57 |
| openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm-infra master: mariadb: fix hostpath values https://review.opendev.org/661955 | 12:06 |
| openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm master: [DNM] mariadb single cluster fix https://review.opendev.org/661957 | 12:09 |
| itxaka | mbuil, helm only creates the expected yaml files and pushes them to k8s, it has no knowledge of whats in the hosts | 12:11 |
| itxaka | so no idea there, sorry :( | 12:11 |
| *** bobmel has joined #openstack-helm | 12:27 | |
| *** howell has joined #openstack-helm | 12:31 | |
| *** bobmel has quit IRC | 12:35 | |
| *** cheng1__ has joined #openstack-helm | 12:35 | |
| openstackgerrit | Jiri Suchomel proposed openstack/openstack-helm master: Set the execution flag for 085-horizon.sh script https://review.opendev.org/661965 | 12:39 |
| itxaka | johnhaan, I think I fixed the mariadb hostpath | 12:47 |
| itxaka | you should add Depends-on: https://review.opendev.org/661955 to the commit message so it can build of the fixed patchset :) | 12:48 |
| *** gmann has quit IRC | 12:53 | |
| johnhaan | @itxaka: Thank you. I've got it~! :) | 12:54 |
| *** cheng1__ has quit IRC | 12:58 | |
| *** cheng1__ has joined #openstack-helm | 12:58 | |
| openstackgerrit | Merged openstack/openstack-helm master: Imported Translations from Zanata https://review.opendev.org/661439 | 12:59 |
| rbarreto | Hi all, I'm trying deploy openstack with ceph but it's getting error when deploy the ceph mon keyring generator | 13:08 |
| rbarreto | Uploaded file: https://uploads.kiwiirc.com/files/e3286412ea0e3dba7fa703f49a4a0ea9/pasted.txt | 13:08 |
| rbarreto | command ran: ./tools/deployment/developer/ceph/040-ceph.sh | 13:09 |
| rbarreto | the pod ceph-mon-keyring got in CrashLoop | 13:10 |
| rbarreto | the pod ceph-mon-keyring stuck in CrashLoop | 13:11 |
| *** aaronsheffield has joined #openstack-helm | 13:19 | |
| *** bline has joined #openstack-helm | 13:20 | |
| *** kranthikirang has joined #openstack-helm | 13:20 | |
| *** kranthikirang has quit IRC | 13:27 | |
| *** michael-beaver has joined #openstack-helm | 13:28 | |
| srwilkers | o/ | 13:33 |
| rbarreto | I'm not able to find out any similar issue on kubernetes about that: error: SchemaError(io.k8s.api.authorization.v1beta1.SubjectAccessReviewStatus): invalid object doesn't have additional properties | 13:40 |
| itxaka | rbarreto, can you do a describe pod for the ceph-mon-keyring pod? | 13:41 |
| itxaka | afaics any chrashloopbackoff are only shown in there and ont he events | 13:41 |
| *** sreejithp has joined #openstack-helm | 13:43 | |
| itxaka | srwilkers, portdirect, I did an oopsie with the maridb hostpath, here is the fix (if zuul want to collaborate) https://review.opendev.org/#/c/661955/ | 13:43 |
| itxaka | tested by https://review.opendev.org/#/c/661957/ | 13:44 |
| srwilkers | oh good catch | 13:44 |
| openstackgerrit | Merged openstack/openstack-helm master: [Fix] Add Prometheus mysql exporter label to network policy https://review.opendev.org/651931 | 13:45 |
| srwilkers | and thanks for hitting this one itxaka: https://review.opendev.org/#/c/660486/4 -- i appreciate it | 13:45 |
| itxaka | np, trying to move forward some of the clean patches ready to merge. It helps that they are easy enough to review/understand :D | 13:48 |
| openstackgerrit | caoyuan proposed openstack/openstack-helm-infra master: Replace git.openstack.org URLs with opendev.org URLs https://review.opendev.org/655048 | 13:48 |
| happyhemant | Hi again I alos tried to see the logs of dependent pods but couldnt able to get anything. i cam only see it for prometheius-openstack-exporter pod :( | 13:50 |
| openstackgerrit | caoyuan proposed openstack/openstack-helm-infra master: Replace git.openstack.org URLs with opendev.org URLs https://review.opendev.org/655048 | 13:52 |
| itxaka | happyhemant, that checks seems weird (expecting 401 but returning 200), is the placement service broken or is the check broken? | 13:55 |
| openstackgerrit | Koffi Nogbe proposed openstack/openstack-helm-infra master: Add enhancement to mariadb backup https://review.opendev.org/661853 | 13:56 |
| happyhemant | https://www.irccloud.com/pastebin/RQ6HifBI/itxaka%2C%20I%20think%20no%20all%20my%20placement%20pods%20works%20fine%20 | 13:58 |
| happyhemant | itxaka, I think no all my placement pods works fine | 13:59 |
| itxaka | I see that the openstack-prometheus-thingie-exporter does indeed expects a 401 from the placement path: https://github.com/att-comdev/prometheus-openstack-exporter/blob/26b499b32fe4d97c1287ed2e8f990f35a89f6c09/exporter/check_os_api.py#L53 | 13:59 |
| happyhemant | yeah thats wjhy i surprised its strange error | 14:00 |
| itxaka | that was from a year ago, so maybe the check has changed? I dont have my env working currently so I cannot test it sorry, maybe tonigth if it finishes deploying :PO | 14:00 |
| happyhemant | well thanks alot will try first myself and will let you know | 14:01 |
| happyhemant | thanks alot for help :) | 14:01 |
| *** jamesgu_ has joined #openstack-helm | 14:02 | |
| rbarreto | itxaka follow: | 14:07 |
| rbarreto | Uploaded file: https://uploads.kiwiirc.com/files/0254c4d88bd8f62d40fdde658b66af22/pasted.txt | 14:07 |
| *** gmann has joined #openstack-helm | 14:10 | |
| *** itlinux has joined #openstack-helm | 14:11 | |
| openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Add helm tests for Ceph Provisioners chart https://review.opendev.org/636735 | 14:24 |
| openstackgerrit | John Haan proposed openstack/openstack-helm master: Change set variable in openstack helm gate job https://review.opendev.org/660695 | 14:25 |
| itxaka | rbarreto, ummm I see, I see, according to that and the post you logged before it seems that its missing the secret yaml that is created on the fly | 14:25 |
| itxaka | this one: https://github.com/openstack/openstack-helm-infra/blob/master/ceph-mon/templates/bin/keys/_bootstrap-keyring-manager.sh.tpl#L42 | 14:26 |
| itxaka | in your previous logs I see the "cat" but not the text that it should cat... | 14:26 |
| itxaka | can you check your configmap ceph-mon-bin for the values of keys-bootstrap-keyring-manager.sh ? | 14:29 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Elasticsearch: Update client and data pod deps https://review.opendev.org/661997 | 14:37 |
| rbarreto | itxaka follow | 14:38 |
| rbarreto | Uploaded file: https://uploads.kiwiirc.com/files/70c0624938f9ad578d5a9414205a22a4/pasted.txt | 14:39 |
| srwilkers | this moves us to use the osh built image for fluentd: https://review.opendev.org/#/c/661819/ | 14:39 |
| srwilkers | that gives us access to the systemd plugin | 14:39 |
| *** itlinux has quit IRC | 14:41 | |
| rbarreto | I don't see any odd in this script as well, but when it is running it is getting: | 14:41 |
| rbarreto | error: SchemaError(io.k8s.api.authorization.v1beta1.SubjectAccessReviewStatus): invalid object doesn't have additional properties | 14:41 |
| itxaka | I still think that the script may be wrong because its telling you that the object is invalid | 14:46 |
| itxaka | so it may be that its not cat-ing it correctly somehow | 14:46 |
| rbarreto | I see. I'll try cat the script before the contianer terminate | 14:49 |
| *** rezroo has joined #openstack-helm | 14:50 | |
| rbarreto | I can't | 14:55 |
| itxaka | rbarreto, ah I just tested locally and no, it seems to generate it correctly | 14:56 |
| itxaka | so Im kind of lost there | 14:56 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Elasticsearch: Fix typos in cluster wait script https://review.opendev.org/662012 | 14:59 |
| itxaka | rbarreto, how about your configmap ceph-templates mon.keyring ? | 15:02 |
| rbarreto | mon.keyring: | [mon.] key = {{ key }} caps mon = "allow *" | 15:03 |
| rbarreto | Uploaded file: https://uploads.kiwiirc.com/files/d54ab559365e5eb2d56430b627d40a4e/pasted.txt | 15:04 |
| *** cfriesen has joined #openstack-helm | 15:08 | |
| itxaka | ¬_¬ | 15:09 |
| itxaka | that all looks good | 15:09 |
| itxaka | Im not sure I can help you further there rbarreto, sorry about that | 15:09 |
| itxaka | you may want to bring this up during office hours | 15:10 |
| *** rezroo has quit IRC | 15:12 | |
| itxaka | thats 20:00 UTC today | 15:12 |
| *** cheng1__ has quit IRC | 15:14 | |
| openstackgerrit | Krishna Venkata proposed openstack/openstack-helm-infra master: Switch from default values being populated for upgrade strategy for ceph components https://review.opendev.org/658492 | 15:21 |
| openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Ceph-osd: Fix security context for pod/container https://review.opendev.org/661796 | 15:28 |
| openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Ceph-mon: Fix security context for pod/container https://review.opendev.org/661831 | 15:31 |
| openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Ceph-client: Fix security context for pod/container https://review.opendev.org/661805 | 15:32 |
| openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Ingress: Fix read-only-fs https://review.opendev.org/659719 | 15:33 |
| *** altlogbot_3 has quit IRC | 15:35 | |
| *** altlogbot_2 has joined #openstack-helm | 15:35 | |
| *** irclogbot_2 has quit IRC | 15:35 | |
| *** irclogbot_2 has joined #openstack-helm | 15:37 | |
| openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Add helm tests for Ceph Provisioners chart https://review.opendev.org/636735 | 15:43 |
| *** itlinux has joined #openstack-helm | 15:54 | |
| *** arunkant_ has joined #openstack-helm | 16:02 | |
| *** arunkant_ has quit IRC | 16:03 | |
| *** arunkant has joined #openstack-helm | 16:03 | |
| *** arunkant_ has joined #openstack-helm | 16:03 | |
| *** arunkant_ has quit IRC | 16:03 | |
| *** happyhemant has quit IRC | 16:24 | |
| *** Emine has quit IRC | 16:31 | |
| *** aojea has quit IRC | 16:35 | |
| *** witek has quit IRC | 16:40 | |
| openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Add helm tests for Ceph Provisioners chart https://review.opendev.org/636735 | 16:44 |
| *** howell has quit IRC | 17:01 | |
| rbarreto | ok itxaka no worry, thank you for your time | 17:05 |
| openstackgerrit | Evgeniy L proposed openstack/openstack-helm-images master: Add periodic building https://review.opendev.org/655426 | 17:10 |
| rbarreto | itxaka I added --validate=false on the kubectl apply and the secret was created successfully | 17:44 |
| rbarreto | Uploaded file: https://uploads.kiwiirc.com/files/93c06576344157d8ea863728c2c850ed/pasted.txt | 17:44 |
| *** bobmel has joined #openstack-helm | 18:03 | |
| *** bobmel has quit IRC | 18:05 | |
| openstackgerrit | Koffi Nogbe proposed openstack/openstack-helm-infra master: Add enhancement to mariadb backup https://review.opendev.org/661853 | 18:13 |
| *** howell has joined #openstack-helm | 18:15 | |
| *** howell has quit IRC | 19:57 | |
| itxaka | are open hours now started portdirect, srwilkers ? | 20:10 |
| itxaka | shame on me but I have something to bring up :D | 20:10 |
| portdirect | itxaka: no problem :D | 20:10 |
| portdirect | what they are here for | 20:10 |
| portdirect | sup? | 20:10 |
| itxaka | I want to bring attention to 3 patches that are really needed on our side, and we have a need for those like asap unfortunately | 20:11 |
| portdirect | sure - you got a link to em? | 20:11 |
| itxaka | https://review.opendev.org/#/c/640115/ - Add support for multiple VIPS | 20:11 |
| itxaka | ufortunately this one hasnt seen a review from anyone but suse people | 20:12 |
| itxaka | so maybe there is not a real need or people dont like it | 20:12 |
| itxaka | and both rbd app name value overrides: | 20:12 |
| itxaka | https://review.opendev.org/661722 | 20:12 |
| itxaka | and https://review.opendev.org/661723 | 20:12 |
| itxaka | would really appreciate some feedback on those, especially important so we can estimate if they will be ready for our deadline or not | 20:13 |
| itxaka | *any* kind of feedback is appreciated in there | 20:13 |
| itxaka | and with that said, I need to leave to put my daughter to bed, will be back later in case any questions are needed for those patches, thanks y'all! | 20:14 |
| portdirect | the latter two - easy - lgtm | 20:17 |
| portdirect | looking at the 1st again now | 20:17 |
| portdirect | im really not sure about it - in that how do you actually split the requests? | 20:19 |
| portdirect | looks like its just creating two routable ips on the host? | 20:19 |
| itxaka | thats a good question and I think there is someone in the channel that can answer that | 20:25 |
| itxaka | let me se... | 20:25 |
| itxaka | sgrasley_, arunkant, jamesgu, jamesgu_ ^^^^ | 20:26 |
| itxaka | ummm, maybe they are out, not sure what timezone they are in, oregon maybe? I dont remember :D | 20:28 |
| *** jsuchome has quit IRC | 20:48 | |
| jamesgu_ | portdirect: sgrasley is not in... I'll try my best to answer the question on his behalf :-) | 21:02 |
| jamesgu_ | yes, the idea is to have multiple vips, one for public end point of openstack services, one for admin/internal/airshup ucp end points. The two vips generally speaking, are from different networks for security. | 21:03 |
| *** itlinux has quit IRC | 21:05 | |
| openstackgerrit | Merged openstack/openstack-helm master: glance: allow configuring the rbd app name https://review.opendev.org/661722 | 21:08 |
| openstackgerrit | Merged openstack/openstack-helm master: cinder: allow configuring the rbd app name https://review.opendev.org/661723 | 21:12 |
| *** itlinux has joined #openstack-helm | 21:13 | |
| portdirect | jamesgu_: what security do you get, thats the bit thats confusing me? | 21:14 |
| portdirect | if you used somthing like metal-lb and had multiple lb's declared with diff endpoints on each id totally get this | 21:14 |
| portdirect | but as it sounds this doesnt get anything over just a static route? | 21:14 |
| jamesgu_ | our k8s distro doesn't support metal-lb yet. So the idea is that we make a subset of endpoints only available thru a specific vip. | 21:18 |
| jamesgu_ | But I think I see a hole in this PS that doesn't fully achieve that goal. | 21:19 |
| jamesgu_ | anyway, in the external DNS, one can define routes to the specific virtual hosts of those service endpoints to the vip depending on which network it is on. | 21:22 |
| jamesgu_ | I am not sure I did a very good job explaining it :-( | 21:23 |
| portdirect | but just sending the host header explicitly in a request would leave the door wide open? | 21:23 |
| jamesgu_ | yes, I think it is missing the logic (if feasible) that which service endpoint goes to which vip. | 21:25 |
| portdirect | ++ | 21:25 |
| openstackgerrit | Koffi Nogbe proposed openstack/openstack-helm-infra master: Add enhancement to mariadb backup https://review.opendev.org/661853 | 21:26 |
| jamesgu_ | do you think it makes sense to add that kind of logic in the ingress controller chart (I don't know it's the right place or not) or should leave to an external load balancer? | 21:27 |
| portdirect | I think this is really where somthing like metal-lb or an external lb would help | 21:28 |
| portdirect | its pushing the ingress controller pretty hard | 21:28 |
| *** sreejithp has quit IRC | 21:28 | |
| portdirect | i think you'd want to extend the controller logic to allow you to annotate rules with the iface/ip you wanted it to bind a partiaular vhost/route to | 21:28 |
| portdirect | at which point, frankly you may be better off running two controllers - each bound to an interface - and then use the ingress class to controll how requests are routed | 21:29 |
| *** rezroo has joined #openstack-helm | 21:33 | |
| jamesgu_ | yes, two controllers sounds more reasonable approach. Do you have any pointers where I can start to look into how we control the ingress rules for different type of end points in the osh charts? | 21:35 |
| jamesgu_ | where I cn -> where we can... I ry to stay away from network topics as far as I can | 21:36 |
| *** itxaka has quit IRC | 21:39 | |
| *** jaypipes has quit IRC | 21:41 | |
| *** rezroo has quit IRC | 21:42 | |
| openstackgerrit | Georg Kunz proposed openstack/openstack-helm master: Extending Neutron chart with support for OVS DPDK https://review.opendev.org/643284 | 22:24 |
| *** ianychoi has quit IRC | 22:32 | |
| *** ianychoi has joined #openstack-helm | 22:33 | |
| *** aaronsheffield has quit IRC | 22:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!