Wednesday, 2018-04-04

« Tuesday, 2018-04-03 Index Thursday, 2018-04-05 »
openstackgerritTin Lam proposed openstack/openstack-helm master: Implement nginx sidecar  https://review.openstack.org/55538600:01
*** sawblade6 has joined #openstack-helm00:02
*** sawblade_ has quit IRC00:05
*** unicell has quit IRC00:06
*** sawblade_ has joined #openstack-helm00:20
*** sawblade6 has quit IRC00:22
*** yamamoto has joined #openstack-helm00:28
*** yamamoto has quit IRC00:34
*** sawblade6 has joined #openstack-helm00:35
*** sawblade_ has quit IRC00:37
*** sawblade_ has joined #openstack-helm01:04
*** sawblade6 has quit IRC01:06
*** sawblade6 has joined #openstack-helm01:09
*** sawblade_ has quit IRC01:10
*** sawblade_ has joined #openstack-helm01:12
*** sawblade6 has quit IRC01:13
*** sawblade6 has joined #openstack-helm01:16
*** sawblade_ has quit IRC01:17
openstackgerritPete Birley proposed openstack/openstack-helm master: Ceph: allow un-interupted operation during partial mon outage  https://review.openstack.org/55866101:26
*** yamamoto has joined #openstack-helm01:30
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implement Ceph-Mon chart  https://review.openstack.org/55832601:30
*** yamamoto has quit IRC01:35
*** yangyapeng has joined #openstack-helm01:48
*** yangyape_ has joined #openstack-helm01:50
*** yangyapeng has quit IRC01:50
*** unicell has joined #openstack-helm02:09
*** unicell has quit IRC02:10
*** unicell has joined #openstack-helm02:11
*** unicell has quit IRC02:12
openstackgerritChris Wedgwood proposed openstack/openstack-helm master: [DNM] experiment: use alternate (smaller) entrypoint container  https://review.openstack.org/55867002:19
*** felipemonteiro has joined #openstack-helm02:22
*** yamamoto has joined #openstack-helm02:32
*** sawblade_ has joined #openstack-helm02:34
*** sawblade6 has quit IRC02:34
*** yamamoto has quit IRC02:37
*** sawblade6 has joined #openstack-helm02:40
*** mdih has joined #openstack-helm02:40
*** sawblade_ has quit IRC02:42
*** yangyape_ has quit IRC02:50
*** yangyapeng has joined #openstack-helm02:50
*** sawblade_ has joined #openstack-helm02:54
*** felipemonteiro has quit IRC02:54
*** yangyapeng has quit IRC02:55
*** sawblade6 has quit IRC02:56
*** gagehugo has quit IRC02:59
*** gagehugo has joined #openstack-helm03:03
*** sawblade6 has joined #openstack-helm03:16
*** sawblade_ has quit IRC03:18
*** yangyapeng has joined #openstack-helm03:19
*** sawblade_ has joined #openstack-helm03:20
*** sawblade6 has quit IRC03:22
*** yangyapeng has quit IRC03:24
*** yamamoto has joined #openstack-helm03:33
*** yangyapeng has joined #openstack-helm03:35
*** yamamoto has quit IRC03:38
*** yangyapeng has quit IRC03:40
*** sawblade6 has joined #openstack-helm03:52
*** sawblade_ has quit IRC03:55
*** yangyapeng has joined #openstack-helm04:01
*** yangyapeng has quit IRC04:06
*** sawblade_ has joined #openstack-helm04:28
*** yangyapeng has joined #openstack-helm04:30
*** sawblade6 has quit IRC04:31
*** sawblade6 has joined #openstack-helm04:33
*** yamamoto has joined #openstack-helm04:35
*** sawblade_ has quit IRC04:35
*** yamamoto has quit IRC04:40
*** sawblade_ has joined #openstack-helm04:40
*** sawblade6 has quit IRC04:42
openstackgerritMerged openstack/openstack-helm master: RabbitMQ: only request 256Mi for PVC by default  https://review.openstack.org/55825804:48
*** yamamoto has joined #openstack-helm04:51
*** sawblade6 has joined #openstack-helm04:59
*** sawblade_ has quit IRC05:01
*** yangyapeng has quit IRC05:18
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implement Ceph-Mon chart  https://review.openstack.org/55832605:29
*** sawblade_ has joined #openstack-helm05:42
*** yangyapeng has joined #openstack-helm05:43
*** sawblade6 has quit IRC05:45
*** yangyapeng has quit IRC05:48
openstackgerritRenis Makadia proposed openstack/openstack-helm master: fixes image for ceph mon-check  https://review.openstack.org/55869505:50
*** yangyapeng has joined #openstack-helm06:02
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP - splitting ceph-client chart  https://review.openstack.org/55870006:06
*** yangyapeng has quit IRC06:07
*** yangyapeng has joined #openstack-helm06:23
*** sawblade6 has joined #openstack-helm06:24
*** sawblade_ has quit IRC06:27
*** yangyapeng has quit IRC06:28
*** sawblade_ has joined #openstack-helm07:06
*** sawblade6 has quit IRC07:09
openstackgerritChris Wedgwood proposed openstack/openstack-helm master: [DNM] experiment: use alternate (smaller) entrypoint container  https://review.openstack.org/55867007:09
*** sawblade6 has joined #openstack-helm07:15
*** sawblade_ has quit IRC07:18
openstackgerritChris Wedgwood proposed openstack/openstack-helm master: Neutron: Update daemonset labels and mixins  https://review.openstack.org/55546107:32
*** yangyapeng has joined #openstack-helm07:34
*** gkadam has joined #openstack-helm07:49
*** sawblade_ has joined #openstack-helm07:52
*** sawblade6 has quit IRC07:54
*** sawblade6 has joined #openstack-helm08:04
*** sawblade_ has quit IRC08:06
*** goldenfri has quit IRC08:27
*** korzen has quit IRC08:31
*** yangyapeng has quit IRC08:50
*** yangyapeng has joined #openstack-helm09:09
*** yangyapeng has quit IRC09:14
*** sawblade_ has joined #openstack-helm09:17
*** sawblade6 has quit IRC09:20
*** sawblade6 has joined #openstack-helm09:23
*** sawblade_ has quit IRC09:25
*** yangyapeng has joined #openstack-helm09:26
*** yangyapeng has quit IRC09:31
openstackgerritHyunsun Moon proposed openstack/openstack-helm master: openvswitch: allow setting ovsdb connection using vsctl  https://review.openstack.org/55876409:31
openstackgerritHyunsun Moon proposed openstack/openstack-helm master: openvswitch: use pidfile option  https://review.openstack.org/55877209:49
*** yangyapeng has joined #openstack-helm09:53
*** yangyapeng has quit IRC09:58
*** yamamoto_ has joined #openstack-helm10:08
*** anticw_ has joined #openstack-helm10:11
*** yamamoto has quit IRC10:15
*** anticw has quit IRC10:15
*** sawblade_ has joined #openstack-helm10:26
*** sawblade6 has quit IRC10:28
*** sawblade6 has joined #openstack-helm10:43
*** sawblade_ has quit IRC10:45
*** pbourke has quit IRC10:47
*** pbourke has joined #openstack-helm10:47
*** sawblade6 has quit IRC10:49
*** sawblade6 has joined #openstack-helm10:51
*** sawblade_ has joined #openstack-helm11:07
*** sawblade6 has quit IRC11:09
*** sawblade6 has joined #openstack-helm11:17
*** sawblade_ has quit IRC11:21
*** mdih has quit IRC11:25
*** sawblade_ has joined #openstack-helm11:52
*** sawblade6 has quit IRC11:54
*** sawblade_ has quit IRC11:57
*** sawblade6 has joined #openstack-helm11:58
*** yangyapeng has joined #openstack-helm12:02
*** sawblade_ has joined #openstack-helm12:02
*** sawblade6 has quit IRC12:04
*** yangyapeng has quit IRC12:06
*** yangyapeng has joined #openstack-helm12:14
*** sawblade6 has joined #openstack-helm12:17
*** sawblade_ has quit IRC12:19
*** sawblade6 has quit IRC12:23
*** yangyapeng has quit IRC12:25
*** sawblade6 has joined #openstack-helm12:26
*** mdih has joined #openstack-helm12:28
*** sawblade_ has joined #openstack-helm12:35
*** sawblade6 has quit IRC12:37
*** adrianreza has quit IRC12:37
*** yangyapeng has joined #openstack-helm12:42
*** yangyapeng has quit IRC12:47
*** sawblade6 has joined #openstack-helm12:51
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863712:52
*** sawblade_ has quit IRC12:53
*** sawblade_ has joined #openstack-helm12:56
*** sawblade6 has quit IRC12:57
*** sawblade_ has quit IRC13:02
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863713:10
*** sawblade6 has joined #openstack-helm13:10
*** yangyapeng has joined #openstack-helm13:30
*** yangyapeng has quit IRC13:31
*** goutham has joined #openstack-helm13:33
*** mdih has quit IRC13:37
*** julim has quit IRC13:39
*** sawblade_ has joined #openstack-helm13:40
*** sawblade6 has quit IRC13:43
*** goutham has quit IRC13:49
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863714:01
*** sawblade6 has joined #openstack-helm14:05
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863714:06
*** sawblade_ has quit IRC14:08
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863714:13
*** julim has joined #openstack-helm14:26
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863714:28
*** sawblade_ has joined #openstack-helm14:35
*** sawblade6 has quit IRC14:35
*** mdih has joined #openstack-helm14:35
*** mdih has quit IRC14:37
*** eeiden has joined #openstack-helm14:43
openstackgerritScott Hussey proposed openstack/openstack-helm master: [WIP] Postgres Helm Test support  https://review.openstack.org/55863714:43
*** mdih has joined #openstack-helm14:46
*** felipemonteiro has joined #openstack-helm14:52
openstackgerritScott Hussey proposed openstack/openstack-helm master: Postgres Helm Test support  https://review.openstack.org/55863715:05
openstackgerritTin Lam proposed openstack/openstack-helm master: DNM: Adding TLS support for sidecar  https://review.openstack.org/55824815:09
openstackgerritSean Eagan proposed openstack/openstack-helm master: Add image build for kubernetes-entrypoint  https://review.openstack.org/55862215:11
*** sawblade6 has joined #openstack-helm15:17
*** sawblade_ has quit IRC15:19
*** felipemonteiro_ has joined #openstack-helm15:22
*** MarkBaker has quit IRC15:23
*** MarkBaker has joined #openstack-helm15:24
*** felipemonteiro has quit IRC15:26
*** felipemonteiro__ has joined #openstack-helm15:33
*** felipemonteiro_ has quit IRC15:33
*** pabelanger has joined #openstack-helm15:34
openstackgerritPete Birley proposed openstack/openstack-helm master: Ceph: Fix config helper image and bump version  https://review.openstack.org/55885115:35
*** felipemonteiro has joined #openstack-helm15:36
pabelangermorning, please http://lists.openstack.org/pipermail/openstack-dev/2018-April/129060.html we've proposed the removal of fedora-26 nodes from nodepool and the helm project is still using them. This is going to result in NODE_FAILURE for the project, blocking CI. I wanted to give a heads up and hope somebody is able to land the patch linked in the ML. Thanks!15:36
*** mdih has quit IRC15:36
openstackgerritRakesh Patnaik proposed openstack/openstack-helm-infra master: Alert rules in prometheus to support nagios based monitoring via alert metric queries  https://review.openstack.org/55570015:39
*** goldenfri has joined #openstack-helm15:45
openstackgerritPete Birley proposed openstack/openstack-helm master: Ceph: Update armada manifests to remove jobs on upgrade correctly  https://review.openstack.org/55885415:46
*** ianychoi__ is now known as ianychoi15:50
*** MarkBaker has quit IRC15:53
openstackgerritPete Birley proposed openstack/openstack-helm master: Ceph: Update armada manifests to remove jobs on upgrade correctly  https://review.openstack.org/55885416:02
openstackgerritPete Birley proposed openstack/openstack-helm master: DNM: Armada: test update/upgrade  https://review.openstack.org/55886016:05
openstackgerritScott Hussey proposed openstack/openstack-helm master: Postgres Helm Test support  https://review.openstack.org/55863716:09
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Gate: Add heat template to bring up a KubeADM cluster with OSH-Infra  https://review.openstack.org/55068416:15
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870016:17
*** sawblade_ has joined #openstack-helm16:31
openstackgerritPaul Belanger proposed openstack/openstack-helm master: Move openstack-helm-multinode-(centos|fedora) to experimental pipeline  https://review.openstack.org/55887016:32
*** sawblade6 has quit IRC16:33
openstackgerritPaul Belanger proposed openstack/openstack-helm master: Move openstack-helm-multinode-(centos|fedora) to experimental pipeline  https://review.openstack.org/55887016:37
pabelangerportdirect: v1k0d3n: I've created a ML post about ^ too, please take a moment to review.  This will help reduce the number of nodes each helm patch uses from nodepool16:39
*** sawblade6 has joined #openstack-helm16:39
*** sawblade_ has quit IRC16:41
*** MarkBaker has joined #openstack-helm16:43
v1k0d3npabelanger looking over now. thank you for the PS and suggestion.16:49
anticw_pabelanger: i wondered if there might be better logic for when to run jobs17:00
anticw_pabelanger: or even set a priority on them, so that non-voting 'good to know' jobs only run when there is nothing more pressing17:01
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870017:01
pabelangeranticw_: zuul has some ability to do conditional job runs, eg; files / irrelevant-files. But ideally, non-voting jobs should only be non-voting for a minimal about of time. Otherwise, we find they tend to bitrot.  In the case here, since they are 5 nodes for centos / fedora, it does get expensive to keep them non-voting17:03
pabelangermoving them the experimental still give users the ability to run time, and help get them back to voting17:04
anticw_i think that's fair, we can look at running those experimental jobs internally when/if we want17:04
pabelangeryah, and nothing saying they can be moved back to check / gate when ready17:05
pabelangercan't*17:05
*** felipemonteiro_ has joined #openstack-helm17:05
v1k0d3nlamt, portdirect and mattmceuen: you guys around?17:05
v1k0d3nportdirect17:06
mattmceuenHey yep17:06
v1k0d3nsounds all fair to me, but lamt portdirect and anticw_  have done most of the work. changes look/sound gtm pabelanger17:06
pabelangerthanks!17:07
v1k0d3nthanks for the help/guidance pabelanger.17:08
mattmceuenpabelanger catching up and will get back to you, afk at the moment. Thanks for the heads up!17:08
*** felipemonteiro__ has quit IRC17:08
clarkbanticw_: pabelanger right I think the bigger issue isn't that you want to run them only when its useful but that they don't seem to function at all17:13
clarkbanticw_: pabelanger you need to fix the don't seem to function at all before it makes sense to optimize beyond that17:13
*** felipemonteiro_ has quit IRC17:13
*** felipemonteiro_ has joined #openstack-helm17:14
clarkbthe other day portdirect offered to dig into the failures but I haven't heard back on that yet. Basically the nova-db-sync container was eating up a ton of time failing then eventually succeeding but the time it consumed was enough the job timed out by the time everything behind it started moving again17:14
clarkband I couldn't find logs for the failing nova-db-sync17:14
*** felipemonteiro_ has quit IRC17:18
*** eeiden has quit IRC17:19
*** felipemonteiro_ has joined #openstack-helm17:20
*** MarkBaker has quit IRC17:25
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870017:28
*** sawblade_ has joined #openstack-helm17:43
*** sawblade6 has quit IRC17:46
*** sawblade6 has joined #openstack-helm17:56
anticw_clarkb: nova-db-sync does a *lot*17:58
*** sawblade_ has quit IRC17:58
anticw_it typically runs 2-3 tims for me here, that is it does at most 100 schema updates at a time i think then terminates17:58
clarkbanticw_: sure but it doesn't take 16 minutes normally17:58
anticw_so to get all 350+ it seens it has to restart a few times17:58
*** yamamoto_ has quit IRC17:58
anticw_clarkb: no, but it's a *lot* of sync IO17:59
clarkbanticw_: ya but in the failing case the container went ot lunch for 16 minutes or so17:59
clarkband then failed17:59
anticw_we should probably just make db IO async and avoid the innodb (whatever maria calls it now) double write in the gates17:59
clarkbwhich prevented a bunch of others from starting behind it17:59
clarkbthe one that succeeded took ~2 minutes17:59
*** yamamoto has joined #openstack-helm17:59
clarkbbut by then the job had timed out the test17:59
anticw_if here are spinning disks and it's sync IO w/ something to take the edge off, you end up feeling the rotation/seek limitations of disks17:59
anticw_clarkb: diff vm on a diff node with a subtle different raid config?18:00
clarkbanticw_: no clue, we consume what are basically public cloud resources and their specific configurations are not somethign we know a ton about. We know some clouds boot off of ceph and others use local storage but thats about it18:00
anticw_clarkb: later i will see if i can get mariadb in 'wreckless abandon mode' (my words) and put up a PS to see if that helps18:00
anticw_clarkb: some builders have *really* slow IO18:01
clarkbanticw_: it would also be good to collect the logs from failed containers18:01
*** felipemonteiro__ has joined #openstack-helm18:01
anticw_clarkb: yeah, we should def do that as well18:01
clarkbat least I don't seem to see where they are collected if they are collected18:01
anticw_it should be there18:01
anticw_do you recall the PS# ?18:01
*** felipemonteiro_ has quit IRC18:02
clarkbanticw_: http://logs.openstack.org/14/558214/2/check/openstack-helm-multinode-fedora/090b77f/primary/pod-logs/openstack/nova-db-sync-d7lwm/nova-db-sync.txt was the one I was looking at18:02
clarkbit started 18 minutes after http://logs.openstack.org/14/558214/2/check/openstack-helm-multinode-fedora/090b77f/primary/pod-logs/openstack/nova-db-sync-d7lwm/init.txt said it was done initing. But then portdirect found a different instance ran an failed /me looks for that log18:03
clarkbhttp://logs.openstack.org/14/558214/2/check/openstack-helm-multinode-fedora/090b77f/primary/objects/namespaced/openstack/pods/nova-db-sync-d7lwm.txt18:03
clarkbits that missing ~16 minutes or so that we want to debug I think18:03
openstackgerritSean Eagan proposed openstack/openstack-helm master: Add image build for kubernetes-entrypoint  https://review.openstack.org/55862218:04
anticw_oh, that looks like the db hung18:06
*** yamamoto_ has joined #openstack-helm18:06
clarkbanticw_: and basically by the time it succeeded 18 minutes later the test had decided thigns were timed out and failed the job18:07
anticw_is it usually nova-db-sync ?18:09
*** yamamoto has quit IRC18:10
*** felipemonteiro has quit IRC18:10
portdirectanticw_: clarkb sorry snowed under atm18:10
anticw_portdirect: it's fine, this can sit on the back-burner for a bit18:10
portdirectbut it would be great if we added a `--previous` to the log collection playbooks18:11
portdirectwhich would get us the logs from pods that had failed, and then been respawened18:11
clarkbanticw_: http://logs.openstack.org/14/558214/2/check/openstack-helm-multinode-fedora/090b77f/primary/objects/namespaced/openstack/pods/nova-db-sync-d7lwm.txt if you look there it appears to be. One was terminated for reason error after ~16 minutes18:11
portdirectsry not to be able to help atm18:11
clarkbanticw_: then the subsequent attempt succeeds much more quikcly18:11
srwilkersportdirect: if nobody adds it before I get back to a keyboard, I can throw something together to get em18:16
*** spiette has quit IRC18:16
*** spiette has joined #openstack-helm18:19
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870018:21
*** felipemonteiro has joined #openstack-helm18:38
*** felipemonteiro__ has quit IRC18:38
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870018:41
openstackgerritChris Wedgwood proposed openstack/openstack-helm master: [DNN] MariaDB: use unsafe defaults to test performance  https://review.openstack.org/55889518:42
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870018:51
*** gkadam has quit IRC19:00
*** sawblade_ has joined #openstack-helm19:08
*** sawblade6 has quit IRC19:10
*** felipemonteiro_ has joined #openstack-helm19:18
*** felipemonteiro has quit IRC19:21
*** sawblade6 has joined #openstack-helm19:24
*** pabelanger has left #openstack-helm19:25
*** sawblade_ has quit IRC19:26
*** sawblade_ has joined #openstack-helm19:27
*** sawblade6 has quit IRC19:29
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870019:41
*** gkadam has joined #openstack-helm19:42
*** felipemonteiro__ has joined #openstack-helm19:48
*** felipemonteiro__ has quit IRC19:49
*** felipemonteiro__ has joined #openstack-helm19:49
*** sawblade6 has joined #openstack-helm19:51
*** felipemonteiro_ has quit IRC19:52
*** sawblade_ has quit IRC19:53
*** felipemonteiro_ has joined #openstack-helm19:54
*** felipemonteiro__ has quit IRC19:57
*** sawblade6 has quit IRC19:59
openstackgerritStacey Fletcher proposed openstack/openstack-helm-addons master: Jenkins chart  https://review.openstack.org/54478020:03
*** sawblade6 has joined #openstack-helm20:05
*** yangyapeng has joined #openstack-helm20:05
openstackgerritStacey Fletcher proposed openstack/openstack-helm-addons master: Jenkins chart  https://review.openstack.org/54478020:05
*** sawblade_ has joined #openstack-helm20:09
*** yangyapeng has quit IRC20:09
*** sawblade6 has quit IRC20:11
openstackgerritMerged openstack/openstack-helm master: Move openstack-helm-multinode-(centos|fedora) to experimental pipeline  https://review.openstack.org/55887020:12
anticw_clarkb: i'm not even sure if nova-db-sync is working properly, it ends with ERROR in a way i hadn't noticed until today20:17
*** sawblade6 has joined #openstack-helm20:17
*** sawblade_ has quit IRC20:19
openstackgerritTin Lam proposed openstack/openstack-helm master: [WIP] Adding TLS support for sidecar  https://review.openstack.org/55824820:26
clarkbanticw_: ya that is why I was hoping we could get logs for it to see how it is failing20:27
anticw_clarkb: so i did a change that should make it faster ... which it maybe does, hard to say20:28
anticw_but in testing that i see db migration stop (prematurely) in a way i never noticed20:28
clarkbcan we get logs for these types of failures too?20:28
clarkbthat way this and other issues become debuggable20:28
anticw_http://logs.openstack.org/70/558870/2/check/openstack-helm-dev-deploy-nfs-ovs/f60c86e/primary/pod-logs/openstack/nova-db-sync-hs25q/nova-db-sync.txt20:29
anticw_see the end of that20:29
anticw_googling i see a couple (perhaps 3) fixes over time for that error, as best as i can tell those are in the image we are using20:29
anticw_i don't know if portdirect has some ideas ... could the sr-iov stuff have affected this?20:30
*** yamamoto_ has quit IRC20:31
*** unicell has joined #openstack-helm20:31
clarkbanticw_: ya we noticed those errors to but they occur in the much faster and successful sync aiui20:33
clarkbbut we don't have logs to see why the earlier sync is so slow20:33
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870020:34
anticw_'ktail -t --all-namespace f > some/file.log' would potentially be useful if a bit painful to deal with20:35
clarkbit could be that the earlier failed sync leads to these errors we do see20:37
clarkbso getting that information will be very valuable20:37
clarkb(also we can fix these errors we do see any it may not fix the underlying issue)20:37
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870020:38
openstackgerritRandeep Jalli proposed openstack/openstack-helm master: WIP: Implement Ceph-Mon chart  https://review.openstack.org/55832620:40
*** sawblade_ has joined #openstack-helm20:41
*** sawblade6 has quit IRC20:41
*** julim has quit IRC20:42
*** sawblade6 has joined #openstack-helm20:49
*** sawblade_ has quit IRC20:51
openstackgerritPete Birley proposed openstack/openstack-helm master: Ceph: Fix config helper image and bump version  https://review.openstack.org/55885120:52
openstackgerritChris Wedgwood proposed openstack/openstack-helm master: [DNN] MariaDB: use unsafe defaults to test performance  https://review.openstack.org/55889521:26
*** yamamoto has joined #openstack-helm21:28
*** yamamoto has quit IRC21:33
openstackgerritRenis Makadia proposed openstack/openstack-helm master: WIP: Implements ceph-client chart  https://review.openstack.org/55870021:34
*** gkadam has quit IRC21:36
*** julim has joined #openstack-helm21:38
*** yangyapeng has joined #openstack-helm21:53
*** yangyapeng has quit IRC21:58
*** sawblade_ has joined #openstack-helm22:03
*** sawblade6 has quit IRC22:05
*** felipemonteiro_ has quit IRC22:21
*** sawblade6 has joined #openstack-helm22:30
*** yamamoto has joined #openstack-helm22:30
*** sawblade_ has quit IRC22:33
*** yamamoto has quit IRC22:36
*** sawblade_ has joined #openstack-helm22:53
*** sawblade6 has quit IRC22:55
*** felipemonteiro has joined #openstack-helm23:00
*** sawblade6 has joined #openstack-helm23:07
*** sawblade_ has quit IRC23:10
*** sawblade_ has joined #openstack-helm23:16
*** sawblade6 has quit IRC23:18
openstackgerritRandeep Jalli proposed openstack/openstack-helm master: WIP: Implement Ceph-Mon chart  https://review.openstack.org/55832623:21
*** yamamoto has joined #openstack-helm23:32
*** sawblade6 has joined #openstack-helm23:37
*** yamamoto has quit IRC23:38
*** sawblade_ has quit IRC23:39
*** sawblade6 has quit IRC23:49
*** sawblade6 has joined #openstack-helm23:51
« Tuesday, 2018-04-03 Index Thursday, 2018-04-05 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!