| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Cinder: Enable backup https://review.openstack.org/530286 | 00:21 |
|---|---|---|
| *** yangyapeng has quit IRC | 00:22 | |
| *** yangyapeng has joined #openstack-helm | 00:23 | |
| *** yangyapeng has quit IRC | 00:27 | |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Glance: Update RBD pool management to better support luminious https://review.openstack.org/530414 | 00:31 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Glance: Update RBD pool management to better support luminious https://review.openstack.org/530414 | 00:34 |
| anticw | mwynne: sometimes | 00:40 |
| mwynne | anticw: Do you know if the ubuntu VM needs LVM volumes for this to work? | 00:43 |
| anticw | it does not | 00:44 |
| mwynne | Ok | 00:44 |
| mwynne | even for ceph/cinder? | 00:44 |
| mwynne | I'm getting this during the ceph steps of hte AIO "fs.go:418] Stat fs failed. Error: no such file or directory" | 00:44 |
| mwynne | "kubelet[28833]: E1229 00:46:34.147454 28833 fs.go:418] Stat fs failed. Error: no such file or directory" Over and over again :S | 00:47 |
| mwynne | And this: "kubelet[28833]: W1229 00:47:52.395619 28833 conversion.go:110] Could not get instant cpu stats: different number of cpus" | 00:48 |
| mwynne | anticw: portdirect This make sense to either of you guys: http://paste.openstack.org/show/632833/ | 00:49 |
| mwynne | That's what I get while mysql is failing to do anything | 00:49 |
| mwynne | And this: http://paste.openstack.org/show/632834/ | 00:49 |
| osh-chatbot4 | <portdirect> so in my laptop it can take upto 4 mins for the volume to mount | 00:50 |
| osh-chatbot4 | <portdirect> that kubelet error looks like cadvisor doing cadvisor things... | 00:51 |
| osh-chatbot4 | <portdirect> https://github.com/kubernetes/kubernetes/issues/35062 | 00:51 |
| mwynne | Ok, so that's a known issue. | 00:51 |
| mwynne | http://paste.openstack.org/show/632835/ | 00:52 |
| mwynne | fsck errors | 00:52 |
| osh-chatbot4 | <portdirect> look at the last line in that log.... | 00:53 |
| mwynne | Saw that | 00:53 |
| osh-chatbot4 | <portdirect> is the pod alive now? | 00:53 |
| mwynne | Looks like it finishes the format shortly after that | 00:54 |
| mwynne | but no | 00:54 |
| mwynne | the pod never comes up | 00:54 |
| osh-chatbot4 | <portdirect> is this the same setup as you were trying yesterday? | 00:55 |
| mwynne | yeah :( | 00:55 |
| osh-chatbot4 | <portdirect> you have access to anything a bit faster? | 00:55 |
| osh-chatbot4 | <portdirect> It really looks like resources are holding you back :( | 00:56 |
| mwynne | This is running on a box with 8 physical CPUS and 64 gigs of ram | 00:56 |
| mwynne | I've allocated 16GB and 8 CPUs to this VM | 00:56 |
| osh-chatbot4 | <portdirect> but the IOPs. | 00:56 |
| openstackgerrit | Merged openstack/openstack-helm master: Docs: remove potentially confusing references to versions and RBAC https://review.openstack.org/530412 | 00:56 |
| mwynne | all the disk writes? | 00:56 |
| osh-chatbot4 | <portdirect> you are running ceph, in a vm ontop of raid :S | 00:56 |
| osh-chatbot4 | <portdirect> I'm surprised its not working, but am really out of ideas as to what else could be up | 00:57 |
| mwynne | portdirect: No RAID on my compute nodes | 00:57 |
| mwynne | I was mistaken yesterday | 00:58 |
| osh-chatbot4 | <portdirect> anything else from the kubelet logs after the attempring to format? | 00:58 |
| mwynne | I mean, I can shut down the other VMs on that compute node if I have to :S | 00:58 |
| mwynne | but I'd like to avoid that | 00:58 |
| mwynne | portdirect: http://paste.openstack.org/show/632845/ | 01:00 |
| mwynne | http://paste.openstack.org/show/632846/ | 01:00 |
| portdirect | for context could you dump `kubectl get --all-namespaces pods` | 01:01 |
| mwynne | http://paste.openstack.org/show/632848/ | 01:02 |
| osh-chatbot4 | <portdirect> you have 1 restart on the mariadb pod | 01:02 |
| osh-chatbot4 | <portdirect> that means it did come up | 01:03 |
| osh-chatbot4 | <portdirect> but did not get ready in time | 01:03 |
| osh-chatbot4 | <portdirect> can I lend you a vm to try this on? | 01:03 |
| mwynne | Right. | 01:04 |
| mwynne | So that means it wasn't able to complete everything, so it rebooted? | 01:05 |
| mwynne | http://paste.openstack.org/show/632851/ | 01:05 |
| mwynne | I'm assuming that's due to the reboot? | 01:05 |
| portdirect | yup | 01:05 |
| mwynne | ok | 01:05 |
| portdirect | looks like it crashed during init... | 01:05 |
| mwynne | Makes sense.. | 01:05 |
| portdirect | (or was killed) | 01:05 |
| mwynne | Can I increase timeouts all over the place? | 01:05 |
| mwynne | I guess I can kill VMs on my compute nodes as well | 01:05 |
| portdirect | I'd go the latter route | 01:06 |
| mwynne | but with the specs I have I wouldn't have expected to have this issue :S | 01:06 |
| portdirect | as increasing timouts wont solve the underlying issue | 01:06 |
| mwynne | But I'll give that a go anyway. Don't seem to have another option. | 01:06 |
| mwynne | Thanks for all your help :) | 01:06 |
| portdirect | what processors are they? 8 cpus and 64gb ram? | 01:06 |
| mwynne | New Ryzens | 01:07 |
| mwynne | (Ryzens with the known hardware issue... we're waiting for replacements) | 01:07 |
| portdirect | and to clarify, you mean 8 cores? or 8 physical cpus? | 01:07 |
| mwynne | sorry, 8 physical CORES | 01:07 |
| mwynne | 1 CPU | 01:08 |
| portdirect | yeah - I'd kill the other vms.. | 01:08 |
| mwynne | Alright. Will do. Thanks again for the help! | 01:08 |
| mwynne | How does this work on your laptop in a VM? | 01:08 |
| *** ljjjustin has joined #openstack-helm | 01:08 | |
| portdirect | np - if that doesnt work I can lend you a vm for a day that will 100% work. | 01:08 |
| mwynne | What kind of laptop do you have?! haha | 01:09 |
| portdirect | nothing special | 01:09 |
| portdirect | hp g3, 32gb ram, ssd | 01:09 |
| mwynne | huh, ok. | 01:09 |
| mwynne | Well I'll give it a try anyway. | 01:09 |
| portdirect | though i run direct of the metal... | 01:10 |
| portdirect | *on | 01:10 |
| mwynne | oh, no VM? | 01:10 |
| portdirect | I run both | 01:10 |
| portdirect | osh on the metal | 01:10 |
| portdirect | then a cluster in vms | 01:10 |
| portdirect | but having a really fast ssd is key there. | 01:10 |
| mwynne | Yeah, an SSD would be nice. | 01:11 |
| mwynne | I might have to pick some up | 01:11 |
| *** yangyapeng has joined #openstack-helm | 01:13 | |
| mwynne | portdirect: Alternatively... Can I just do the AIO without ceph? | 01:17 |
| portdirect | yes - though its not currently documented | 01:17 |
| portdirect | you will need to deploy the nfs chart | 01:18 |
| portdirect | and then set glance to use the "pvc" backend | 01:18 |
| portdirect | and turn off ceph for libvirt and nova, though I cannot rememebr the flags for that offhand | 01:19 |
| openstackgerrit | Merged openstack/openstack-helm-infra master: Move fluentbit and fluentd configs to values.yaml https://review.openstack.org/527489 | 01:27 |
| mwynne | openstackgerrit: hmm. That all sounds like unfamiliar territory for me unfortunately | 01:32 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Gnocchi: many chart fixes https://review.openstack.org/530421 | 01:43 |
| openstackgerrit | Merged openstack/openstack-helm-infra master: Elasticsearch: Add curator snapshot action and PVC for fs repo https://review.openstack.org/527539 | 01:44 |
| openstackgerrit | Merged openstack/openstack-helm-infra master: Rename elasticsearch configmaps and provide config via toYaml https://review.openstack.org/530391 | 01:44 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Ceilometer: RBAC for helm test https://review.openstack.org/530422 | 01:53 |
| *** larainema has quit IRC | 01:55 | |
| *** ljjjustin has quit IRC | 01:58 | |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: WIP: RBAC: add rules to all remaing helm test pods https://review.openstack.org/530423 | 02:00 |
| yangyapeng | openstack-helm-core | 02:07 |
| yangyapeng | hello | 02:07 |
| mattmceuen | o/ yangyapeng | 02:07 |
| yangyapeng | mattmceuen: hello, I want to install OpenStack use openstack-helm, Have any good Suggestions？ | 02:08 |
| mattmceuen | that's awesome - sure do :) | 02:09 |
| mattmceuen | For a test environment -- I'd suggest starting here https://docs.openstack.org/openstack-helm/latest/install/developer/all-in-one.html | 02:09 |
| mattmceuen | For a production environment -- by the time you get through the developer guide, we should have an updated multinode guide available | 02:10 |
| yangyapeng | mattmceuen: thank you mattmceuen, i have a try. ^^ | 02:10 |
| mattmceuen | Cool yangyapeng. Let us know how it goes, or if you have any questions! | 02:10 |
| yangyapeng | mattmceuen: ok, i will install it on centos 7 , try. thank you | 02:11 |
| *** yamamoto has joined #openstack-helm | 02:15 | |
| *** larainema has joined #openstack-helm | 02:21 | |
| yangyapeng | mattmceuen: i meet a question In the process of installation K8S，it have no automatic installation docker. if i install it(docker), any docker version is available? | 02:33 |
| *** yamamoto has quit IRC | 02:34 | |
| yangyapeng | mattmceuen: TASK [deploy-docker : check if docker deploy is needed], ansible check faild, but i have a docker in /usr/bin/docker | 02:35 |
| *** mdih has joined #openstack-helm | 02:36 | |
| mattmceuen | We're using 1.13.1 right now yangyapeng, as it's the default docker.io package in ubuntu | 02:41 |
| mattmceuen | what version do you have? I'll take a look at the check | 02:42 |
| *** mdih has quit IRC | 02:42 | |
| mattmceuen | for centos -- please upgrade to the latest version of the docker-latest package | 02:43 |
| yangyapeng | mattmceuen: ok. thank for your replay. i have a try | 02:44 |
| *** yamamoto has joined #openstack-helm | 03:35 | |
| *** yamamoto has quit IRC | 03:41 | |
| yangyapeng | mattmceuen: Hello, i install K8s and docker and helm， and it is ok. | 03:48 |
| srwilkers | o/ | 03:49 |
| yangyapeng | mattmceuen: srwilkers but i have some other question | 03:49 |
| yangyapeng | http://paste.openstack.org/show/633018/ | 03:49 |
| yangyapeng | mattmceuen: srwilkers because i install k8s and helm Manual， if i should do more ? | 03:51 |
| *** gagehugo has quit IRC | 03:52 | |
| *** gagehugo has joined #openstack-helm | 03:53 | |
| mattmceuen | yangyapeng: are you running the script as root or as a normal user? | 03:55 |
| openstackgerrit | Tin Lam proposed openstack/openstack-helm master: RBAC: Fix issue with inclusion of empty jobs or daemonsets https://review.openstack.org/530431 | 04:05 |
| *** gagehugo has quit IRC | 04:08 | |
| yangyapeng | mattmceuen: root | 04:41 |
| mattmceuen | yangyapeng: can you try running as a normal user and see if that resolves it? | 04:57 |
| *** yamamoto has joined #openstack-helm | 05:05 | |
| *** yamamoto has quit IRC | 05:05 | |
| openstackgerrit | Tin Lam proposed openstack/openstack-helm master: Update docs building infra https://review.openstack.org/530435 | 05:57 |
| *** yamamoto has joined #openstack-helm | 06:05 | |
| openstackgerrit | Tin Lam proposed openstack/openstack-helm master: Update docs building infra https://review.openstack.org/530435 | 06:07 |
| *** yamamoto has quit IRC | 06:12 | |
| *** haint has joined #openstack-helm | 06:55 | |
| *** ljjjustin has joined #openstack-helm | 07:23 | |
| *** nguyentrihai has quit IRC | 07:48 | |
| *** yamamoto has joined #openstack-helm | 07:55 | |
| *** yamamoto has quit IRC | 08:08 | |
| *** yamamoto has joined #openstack-helm | 08:08 | |
| *** yamamoto has quit IRC | 08:13 | |
| *** gagehugo has joined #openstack-helm | 08:59 | |
| *** yamamoto has joined #openstack-helm | 09:11 | |
| *** yamamoto has quit IRC | 09:17 | |
| *** yamamoto has joined #openstack-helm | 09:40 | |
| *** ljjjustin has quit IRC | 11:04 | |
| *** yamamoto has quit IRC | 11:14 | |
| *** yamamoto has joined #openstack-helm | 12:00 | |
| *** yamamoto has quit IRC | 12:10 | |
| *** yamamoto has joined #openstack-helm | 12:19 | |
| *** yamamoto has quit IRC | 12:33 | |
| *** yangyapeng has quit IRC | 12:40 | |
| *** yangyapeng has joined #openstack-helm | 12:41 | |
| *** yangyapeng has quit IRC | 12:48 | |
| *** yamamoto has joined #openstack-helm | 13:05 | |
| *** jistr has quit IRC | 13:14 | |
| *** yangyapeng has joined #openstack-helm | 13:15 | |
| *** jistr has joined #openstack-helm | 13:16 | |
| *** yamamoto has quit IRC | 13:23 | |
| openstackgerrit | Matt McEuen proposed openstack/openstack-helm master: Add some clarifications to AIO dev guide https://review.openstack.org/530468 | 13:33 |
| *** mdih has joined #openstack-helm | 14:03 | |
| *** yamamoto has joined #openstack-helm | 14:24 | |
| *** mdih has quit IRC | 14:27 | |
| *** yamamoto has quit IRC | 14:30 | |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm master: Add out-of-date warning to openstack-helm gates README https://review.openstack.org/530400 | 14:33 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Add elasticsearch exporter https://review.openstack.org/523634 | 14:34 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Add elasticsearch exporter https://review.openstack.org/523634 | 14:40 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Add elasticsearch exporter https://review.openstack.org/523634 | 14:41 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Add elasticsearch exporter https://review.openstack.org/523634 | 14:45 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: Add elasticsearch exporter https://review.openstack.org/523634 | 14:47 |
| openstackgerrit | Merged openstack/openstack-helm master: Update docs building infra https://review.openstack.org/530435 | 14:55 |
| osh-chatbot4 | <kelvin633> openstack-helm uses `ubuntu-source` images. Can we expect `ubuntu-binary` images to work as well? | 15:17 |
| openstackgerrit | Merged openstack/openstack-helm master: RBAC: Fix issue with inclusion of empty jobs or daemonsets https://review.openstack.org/530431 | 15:34 |
| openstackgerrit | Merged openstack/openstack-helm-infra master: RBAC: Fix issue with inclusion of empty jobs or daemonsets https://review.openstack.org/530337 | 15:56 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Ceph: Dont create volume pool by default https://review.openstack.org/530478 | 16:16 |
| mattmceuen | kelvin633: I think they should, although I haven't tried them myself. We're working to be image-agnostic. If you try it out, can you let us know how it goes? | 16:18 |
| osh-chatbot4 | <kelvin633> i rebuilt neutron to use binary and they dont work. perhaps it’s the mixing of binary and source, but i’m rebuilding them as source and see if that’s the case | 16:19 |
| *** mdih has joined #openstack-helm | 16:49 | |
| *** mdih has quit IRC | 16:53 | |
| openstackgerrit | Merged openstack/openstack-helm master: Cinder: Enable backup https://review.openstack.org/530286 | 17:01 |
| openstackgerrit | Merged openstack/openstack-helm-infra master: Gate: simplify helm release log gathering https://review.openstack.org/530398 | 18:28 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: WIP: Use OpenStack-Helm-Infra log collection playbook in zuul https://review.openstack.org/530369 | 19:07 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: RBAC: add rules to all remaing helm test pods https://review.openstack.org/530423 | 19:10 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Ceph: Add mgr caps to the admin keyring https://review.openstack.org/529636 | 19:18 |
| openstackgerrit | dave kormann proposed openstack/openstack-helm master: Ceph OSD disk targeting https://review.openstack.org/527505 | 19:21 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: WIP: Docs: Multinode gated guide https://review.openstack.org/530505 | 19:23 |
| *** larainema has quit IRC | 19:45 | |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: WIP: Docs: Multinode gated guide https://review.openstack.org/530505 | 20:30 |
| openstackgerrit | Merged openstack/openstack-helm master: Glance: Update RBD pool management to better support luminious https://review.openstack.org/530414 | 20:31 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: WIP: Docs: Multinode gated guide https://review.openstack.org/530505 | 20:31 |
| *** mdih has joined #openstack-helm | 20:52 | |
| *** mdih has quit IRC | 21:17 | |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm-infra master: WIP - run kibana behind apache https://review.openstack.org/530510 | 21:45 |
| openstackgerrit | Matt McEuen proposed openstack/openstack-helm master: [WIP] Add Swift Chart https://review.openstack.org/508780 | 22:20 |
| v1k0d3n | hey everyone...i hope everyone has a fun and safe new years! | 22:34 |
| *** sebastian-w has quit IRC | 22:44 | |
| mattmceuen | Hey thanks v1k0d3n -- ditto to you, Happy New Year!! | 23:00 |
| openstackgerrit | Merged openstack/openstack-helm master: RBAC: add rules to all remaing helm test pods https://review.openstack.org/530423 | 23:08 |
| openstackgerrit | Matt McEuen proposed openstack/openstack-helm master: [WIP] Add Swift Chart https://review.openstack.org/508780 | 23:08 |
| openstackgerrit | Merged openstack/openstack-helm master: Add out-of-date warning to openstack-helm gates README https://review.openstack.org/530400 | 23:16 |
| srwilkers | Hey v1k0d3n - you too | 23:51 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!