Thursday, 2020-12-03

*** noonedeadpunk has quit IRC00:08
*** hamalq_ has joined #opendev00:15
*** tosky has quit IRC00:18
*** hamalq has quit IRC00:19
*** hamalq_ has quit IRC00:21
*** mlavalle has quit IRC00:22
*** hamalq has joined #opendev00:25
*** cgoncalves has quit IRC00:28
openstackgerritIan Wienand proposed opendev/gerritlib master: integration test: run manage-projects with debug  https://review.opendev.org/c/opendev/gerritlib/+/76522300:35
openstackgerritIan Wienand proposed opendev/system-config master: WIP: initalize gerrit in testing  https://review.opendev.org/c/opendev/system-config/+/76522400:55
*** DSpider has quit IRC00:57
*** cgoncalves has joined #opendev01:06
fungii'll check in a bit to see if we're set for a restart01:23
*** brinzhang0 has joined #opendev02:01
*** brinzhang_ has quit IRC02:04
*** hamalq has quit IRC02:32
ianwfungi: do you want me to pull the latest image?02:38
openstackgerritIan Wienand proposed opendev/system-config master: WIP: initalize gerrit in testing  https://review.opendev.org/c/opendev/system-config/+/76522402:54
*** brinzhang0 has quit IRC03:20
*** brinzhang has joined #opendev03:29
fungiyeah, sorry, got sidetracked but checking stuff now03:35
fungiopendevorg/gerrit                                         3.2                                    b12bec20e800        7 hours ago         796MB03:38
fungithat's after pulling03:38
fungi763656 merged at 21:10z so that seems reasonable timing03:40
fungidocker-compose.yaml has the db bindmount in it too03:40
fungiother than the tripleo gate queue just getting longer and longer, things seem to have generally calmed down03:43
fungitaking things down and up quickly now03:46
fungi#status log restarted the gerrit service on review.o.o for the openjdk 11 upgrade from https://review.opendev.org/76365603:46
openstackstatusfungi: finished logging03:46
ianwfungi: i wonder, the latest digest is 221051f8d64e?03:47
ianwfor 763656 according to https://registry.hub.docker.com/layers/opendevorg/gerrit/3.2/images/sha256-221051f8d64e3128c155b983ffb8aa584a61125336c0033c2529e1aa3642f7bd?context=explore03:47
fungimmm03:47
ianwanyway it has java 11 now, but i just wonder what i'm reading wrong03:50
fungiwhy would pull have a different hash?03:50
fungiyeah03:50
fungithat's output from `docker image ls` so maybe it uses a different hash algorithm?03:51
fungibut yeah03:52
fungiJVM: Oracle Corporation OpenJDK 64-Bit Server VM 11.0.9.1+103:52
fungii don't expect we would have built additional gerrit 3.2 images after that change landed03:52
fungithere would only be the one with java 1103:52
ianw        "RepoDigests": [03:53
ianw            "opendevorg/gerrit@sha256:221051f8d64e3128c155b983ffb8aa584a61125336c0033c2529e1aa3642f7bd"03:53
ianw        ],03:53
fungianyway, things seem to be working normally, but i'm around for a while still of something turns out weird03:53
ianwthat lines up03:53
fungicould docker image ls be giving sha-1?03:54
ianwyeah i think it's "docker image ls --digests" to get what lines up with the dockerhub entry03:56
ianw"the image id is a hash of the local image JSON configuration"03:56
ianwjust ... because03:56
fungithat's... fun03:57
ianwanyway, right image :)03:57
ianwi just locally took a screenshot using a selenium docker container of the locally running gerrit.  i think this might work.03:58
fungineat!04:00
fungiyeah, that sounds promising04:00
*** brinzhang_ has joined #opendev04:04
fungifor webapps we can't create a simple draft build of, that seems like a great alternative for getting an idea of what certain webui alterations will look like04:04
*** brinzhang has quit IRC04:07
*** ykarel|away has joined #opendev04:34
*** rishabhhpe has joined #opendev04:45
*** rishabhhpe has quit IRC04:46
*** rishabhhpe has joined #opendev04:49
*** ysandeep|away is now known as ysandeep04:53
*** ykarel|away is now known as ykarel05:23
openstackgerritIan Wienand proposed opendev/system-config master: WIP: initalize gerrit in testing  https://review.opendev.org/c/opendev/system-config/+/76522405:23
*** ianw is now known as ianw_pto05:24
*** marios has joined #opendev05:59
*** chengsheng1 is now known as chengsheng06:07
*** d34dh0r53 has quit IRC07:09
*** d34dh0r53 has joined #opendev07:10
*** sboyron has joined #opendev07:20
*** rpittau|afk is now known as rpittau07:27
*** eolivare has joined #opendev07:29
*** slaweq has joined #opendev07:41
*** lpetrut has joined #opendev07:46
*** fressi has joined #opendev08:01
*** rishabhhpe has quit IRC08:31
*** tosky has joined #opendev08:34
*** fressi_ has joined #opendev08:42
*** rishabhhpe has joined #opendev08:43
*** fressi has quit IRC08:44
*** fressi_ has quit IRC08:47
*** andrewbonney has joined #opendev08:49
*** fressi has joined #opendev08:52
*** hashar has joined #opendev08:54
*** noonedeadpunk has joined #opendev09:03
*** ykarel_ has joined #opendev09:08
*** ykarel has quit IRC09:11
*** ykarel_ has quit IRC09:12
*** ykarel_ has joined #opendev09:18
*** sshnaidm|afk is now known as sshnaidm|ruck09:22
*** rishabhhpe has quit IRC09:34
*** ykarel_ is now known as ykarel09:46
*** rishabhhpe has joined #opendev09:56
*** rishabhhpe has quit IRC09:59
*** DSpider has joined #opendev10:26
*** dtantsur|afk is now known as dtantsur10:26
openstackgerritMerged opendev/elastic-recheck master: Add query for bug 1903979  https://review.opendev.org/c/opendev/elastic-recheck/+/76250212:03
openstackbug 1903979 in OpenStack Compute (nova) "nova-live-migration job fails during evacuate negative test" [High,In progress] https://launchpad.net/bugs/1903979 - Assigned to Lee Yarwood (lyarwood)12:03
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517712:29
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517712:52
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517713:00
*** ysandeep is now known as ysandeep|afk13:02
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517713:13
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517713:17
*** brinzhang_ has quit IRC13:18
*** mugsie has quit IRC13:32
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517713:44
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517713:49
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517713:59
*** ysandeep|afk is now known as ysandeep14:04
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517714:14
*** sshnaidm|ruck is now known as sshnaidm|afk14:17
fungihttp://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=26&rra_id=all15:00
fungiopenjdk 11 doesn't seem to have resulted in a huge performance boost15:00
fungithough it does seem to have consumed memory more aggrssively15:00
fungihttp://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=27&rra_id=all15:01
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517715:10
*** fressi has quit IRC15:14
*** hashar is now known as hasharAway15:22
mnasero/ are we safe to land projects that will run manage-projects (i.e. gerrit acl changes etc?)15:26
*** sshnaidm|afk is now known as sshnaidm|ruck15:27
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu  https://review.opendev.org/c/zuul/zuul-jobs/+/76517715:29
*** lpetrut has quit IRC15:30
*** redrobot has joined #opendev15:35
*** ysandeep is now known as ysandeep|away15:36
openstackgerritMerged openstack/project-config master: End project gating for retiring Searchlight project  https://review.opendev.org/c/openstack/project-config/+/76451915:36
openstackgerritMerged openstack/project-config master: End project gating for retiring Qinling project  https://review.opendev.org/c/openstack/project-config/+/76452015:41
fungimnaser: yes, absolutely15:42
fungiworst case we may encounter bugs in the manage-projects script which delays them taking effect, but things should be fine15:43
fungiwe had to merge a change to fix a behavior change in new gerrit around group creation, but i think it's merged now and i shouldn't impact those config changes anyway15:44
clarkbfungi: mnaser well that fix isn't in our images yet15:53
clarkband I'm not sure it would work at all. It might be nice get the current broken state happy before doing more?15:53
mnaserah- in this case it was an acl change..15:54
mnaseri think i can remove my +w still15:54
clarkbya the current issue we've discovered is an acl update15:54
clarkbwe think we have a fix but it isn't in the gerrit docker image yet I think15:54
mnaserok i've removed my +W from https://review.opendev.org/c/openstack/project-config/+/764534 and https://review.opendev.org/c/openstack/project-config/+/76453215:54
fungiit was specifically an acl adding a group thought right?15:56
*** mlavalle has joined #opendev15:57
clarkbfungi: no, it is an acl adding vote categories. The bug is we don't properly detect the group already exists so try to create it and that is an error15:57
clarkbthe fix updates our gerrit ssh query to use the correct flag to query for group existence15:57
clarkbI think new groups will also fail because we won't get the uuid necessary to put in the groups file15:57
clarkbbasically any acl update is broken unless it only uses built in groups15:57
clarkbthat was the deficiency in our testing, we only used the built in groups in testing15:58
fungithough they'll continue to get retried on subsequent runs until the fix is in, right?15:58
clarkbno the acls won't be updated at all15:58
clarkbthough amybe in this case if there are no groups involved it would work15:58
clarkb(I think there are groups involved in retirements now though)15:58
fungionce the gerritlib fix lands, do we need to tag a release or are we continuously deploying the master branch state into our images?16:08
fungii can check, just figure you've already done so16:09
clarkbfungi: I think we need a gerritlib tag, then a jeepyb/system-config gerrit image update to update the image16:09
fungiahh, okay16:10
*** hasharAway is now known as hashar16:11
*** ykarel is now known as ykarel|away16:19
fungilooks like the gerritlib fix hit a mirror server timeout in limestone16:34
fungiurllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='mirror.regionone.limestone.opendev.org', port=443): Read timed out.16:34
fungii can reach it, so likely internal networking issues16:35
clarkbfungi: note that I think that fix is for tests only. We could theoretically tag the change from yesterday but that feels incomplete16:35
fungiyep16:35
*** marios is now known as marios|out16:37
fungiclarkb: so looking at the gerritlib changes which have merged since 0.9.0, i think the tcp keepalive support addition and configurable connection retries justify a minor version bump to 0.10.0 for the next tag16:45
fungionce the two approved changes merge16:45
clarkbya that seems reasonable, what is the second approved change?16:46
clarkboh the one for debug logging in testing ++16:46
fungiianw added some m-p testing improvements on the heels of yours16:46
fungiyeah, that16:46
fungii figured they should both go in togther16:46
clarkbyup agreed as that makes the tests a lot easier to understand16:47
fungii'll be on hand to push the 0.10.0 tag once those are in, and then when it's up on pypi i'll push a trivial change to trigger another manage-projects container image build i guess?16:48
*** ykarel|away has quit IRC16:48
clarkbfungi: ya maybe bump up the gerritlib version in jeepyb requirements?16:49
clarkbI think that should do it and reflect the improvements that have been made16:49
*** rpittau is now known as rpittau|afk16:49
fungithat's a good point16:53
fungii'll push that first after the release is done16:53
*** marios|out has quit IRC16:55
*** eolivare has quit IRC16:56
openstackgerritMerged opendev/gerritlib master: Test manage-projects repos harder  https://review.opendev.org/c/opendev/gerritlib/+/76521717:12
fungiso there's one of them17:21
fungithe other is about to enter the gate now17:22
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: Add unit test for modify_registries_conf  https://review.opendev.org/c/zuul/zuul-jobs/+/76535217:29
*** tosky has quit IRC17:29
*** Tengu has quit IRC17:32
*** dtantsur is now known as dtantsur|afk17:39
*** Tengu has joined #opendev17:39
openstackgerritMerged opendev/gerritlib master: integration test: run manage-projects with debug  https://review.opendev.org/c/opendev/gerritlib/+/76522317:43
fungiokay, prepping to tag it now17:45
fungitag pushed17:51
clarkbthanks!17:52
fungi#status log released gerritlib 0.10.0 to get a gerrit upgrade related group management fix into our manage-projects image17:53
openstackstatusfungi: finished logging17:53
fungias soon as it appears on pypi i'll push the version bump in jeepyb17:56
clarkbfungi: I've just double checked the jeepyb gerrit image build job file matchers and it is set to .* so should be fine to update the requirements file17:58
openstackgerritPaul Belanger proposed zuul/zuul-jobs master: Add unit test for modify_registries_conf  https://review.opendev.org/c/zuul/zuul-jobs/+/76535218:03
openstackgerritJeremy Stanley proposed opendev/jeepyb master: Bump gerritlib requirement to 0.10.0  https://review.opendev.org/c/opendev/jeepyb/+/76535718:04
fungiclarkb: thanks for confirming! ^18:04
zbrfungi: re jeepyb i wonder what made https://review.opendev.org/c/opendev/jeepyb/+/714729 missing a merge18:13
fungizbr: i was wondering that myself. it's possible there were gerrit/jgit issues with it18:14
zbri guess you can do the -W/+W trick, probably faster than a recheck comment.18:15
fungizuul tries to guess whether gerrit will be able to merge a change, but it doesn't always guess accurately since the only real way to assess that would be to test each merge action with an actual jgit implementation18:15
zbrwell.. i doubt there were too many changes on this repo happening at the same time :D18:16
zbrmy guess, probably slipped during a restart or so.18:16
fungiyeah but if a bunch of other changes merged between the commit that change was parented on and when it was approved, jgit can choke18:16
fungialso possible gerrit was offline at the exact moment the submit action was called, i guess18:17
fungithough we didn't log any gerrit restart that day: https://wiki.openstack.org/wiki/Infrastructure_Status18:18
zbrhttps://review.opendev.org/c/opendev/jeepyb/+/758595 also seams safe18:21
*** sshnaidm|ruck is now known as sshnaidm|afk18:22
zbri wonder why ancient (3yy+ old) changes are not just abandoned, it could make it cleaner.18:24
zbrdoes anyone benefit from having PR-zombies? on the other hand abandoning them could motivate someone to revive it.18:25
clarkbwe'ev given you the ability to abandon your own changes if you feel that is appropriate18:27
clarkbwe think thati s better than mass abandoning and losing things as a result18:27
clarkbthis conversation has come up over and over and over again. Basically if teams want to do that they can, we give tehm that ability but don't do it wholesale as its a bad approach from a hosting perspective18:27
*** andrewbonney has quit IRC18:29
zbrmost of the time the original author left, so only cores can abandon. and for the record, I never recommended bulk abandon.18:29
clarkb"i wonder why ancient (3yy+ old) changes are not just abandoned, it could make it cleaner." implies bulk abandons based on timestamps18:30
zbri was referring to a nice one, with a comment: "this seams outdated and we will abandon to cleanup the review queue but if you want to revive...."18:30
fungiwe've had a loose policy that if a core reviewer notices a change is no longer relevant, they can politely abandon it18:31
zbrjust abandoned by a human, while I love automation, some stuff are better left to humans18:31
fungii usually try to say something like "this was fixed in change NNNN instead" or "this doesn't fit with the project plans, sorry" or whatever18:32
zbri know, i used the button quite often, i do not remember any complaints, afaik restore can be done even by original author.18:32
zbrprobably people would use it more if it would be called "archive"18:33
zbrbecause in the end, that's is what it does.18:33
fungithe underlying problem is that we lack sufficient time to devote to reviewing every change which gets proposed. abandoning a change in a reasonable manner is part of review, but when some of those changes don't get reviewed they don't get abandoned. i don't thnik abandonnig changes i haven't reviewed is a wise choice though18:33
fungiany more than closing bugs nobody's had time to triage is a good idea18:36
*** hashar has quit IRC18:40
zbrwhile i totally agree that lack of time is a major issue, my personal approach is to (at least attempt) to review incoming contributions on projects i take care of, and not allow them to pile-up. Why? because ignoring means deterring someone from ever contributing back, which  to some extend loosing someone that could potential help you with18:40
zbrmaintenance.18:40
clarkbright but we cannot physically accomplish that because there is simply too much18:43
clarkbif I started doing that today I wouldn't get to see my family or have sleep or get any other work done18:43
clarkbbecause the inputs are greater than the human processing power18:44
clarkbinstead we do our best to prioritize and apply the time we do have to the most important things of the moment18:44
* zbr going out, is already 7pm here, see you tomorrow.18:45
openstackgerritMerged opendev/jeepyb master: Just use flake8 instead of hacking  https://review.opendev.org/c/opendev/jeepyb/+/71472918:57
*** hamalq has joined #opendev18:58
*** hashar has joined #opendev18:58
*** deepthot has joined #opendev18:59
fungidockerhub says our gerrit and gerrit-base images updated 2 minutes ago19:00
fungii went ahead and pulled since our automation hadn't yet19:03
fungiopendevorg/gerrit                                         3.2                                    712c96672bbb        9 minutes ago       796MB19:03
fungilooks like our new manage-projects wrapper doesn't log to /var/log/manage_projects.log19:05
fungilooks like it logs to /var/log/ansible/manage-projects.yaml.log on bridge.o.o now19:09
fungithe continued error is with openstack/tripleo-ci so i'll test running manage-projects for it first by itself and see what happens now19:09
fungithat seems to have worked19:10
fungigoing to do a full m-p run of everything now to make sure there are no other lingering errors this was hiding19:10
fungiapparently there's nothing else to do19:12
fungimnaser: if you want to reapply your approval on one of those retirement changes we'll make sure it gets applied correctly19:12
clarkbfungi: cool so we're happy with the state of things now?19:16
clarkband thank you for picking that up after I had to step away yesterday19:16
fungiclarkb: absolutely19:20
fungiyour fix seems to be working a treat19:20
*** DSpider has quit IRC19:46
*** sshnaidm|afk has quit IRC20:30
*** tosky has joined #opendev20:55
*** deepthot has quit IRC21:00
openstackgerritMerged openstack/project-config master: [check-release-approval] Handle PTL-less projects  https://review.opendev.org/c/openstack/project-config/+/76477521:04
openstackgerritMerged openstack/project-config master: Notify placement changes on nova channel  https://review.opendev.org/c/openstack/project-config/+/76446921:04
openstackgerritMerged openstack/project-config master: Update retiring Qinling repo ACL to openstack/retired.config  https://review.opendev.org/c/openstack/project-config/+/76453221:04
openstackgerritMerged openstack/project-config master: Update retiring Searchlight repo ACL to openstack/retired.config  https://review.opendev.org/c/openstack/project-config/+/76453421:04
*** openstackgerrit has quit IRC21:08
*** hashar has quit IRC21:16
*** deepthot has joined #opendev22:16
*** openstackgerrit has joined #opendev22:41
openstackgerritJeremy Stanley proposed openstack/project-config master: Announce openstack/ossa in #openstack-security  https://review.opendev.org/c/openstack/project-config/+/76538922:41
*** tkajinam has quit IRC23:03
*** tkajinam has joined #opendev23:04
openstackgerritJeremy Stanley proposed opendev/system-config master: Add an OpenDev theme into our Gerrit image  https://review.opendev.org/c/opendev/system-config/+/76542223:53
fungipaladox: ^23:53
fungithanks again!23:54
paladoxshould change wm-styles to opendev-styles23:54
fungioh, can do23:54
paladoxbut otherwise looks good to me :)23:54
openstackgerritJeremy Stanley proposed opendev/system-config master: Add an OpenDev theme into our Gerrit image  https://review.opendev.org/c/opendev/system-config/+/76542223:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!