Monday, 2023-02-13

*** priteau_ is now known as priteau12:25
opendevreviewMerged openstack/swift master: proxy-server exception logging shows replication_ip/port  https://review.opendev.org/c/openstack/swift/+/86086621:34
mcapeHi! The rocky - yoga update story, part 4. 
Object layer went without any issues. Hurray! 
However,  update for container servers (just servers first, replicators/sharders were left on rocky)  got quite a bunch of 500 errors. 22:42
mcapeThey fall into two categories, and look like https://pastebin.com/23iCcPmP, and https://pastebin.com/dDES2KYM.
I rolled back quickly, to try to digest what is going on.

It feels like that this errors are coming from same deleted shards i am getting a lot of container-sharder warnings about, 22:42
mcapehowever it is unclear now how to better proceed.22:42
mcapeHere's an example of such a shard:  https://pastebin.com/7X0AKYPG22:42
mcapeI thought it will be probably better to apply patches:  "Identify deleted shards as shards" https://review.opendev.org/c/openstack/swift/+/738113 and "Do not delete root_path on ContainerBroker.delete_db" https://review.opendev.org/c/openstack/swift/+/773832, as timburke pointed before, and look how it will work out.  
However "Identify deleted shards as shards" is not applying cleanly on my files from rocky. 22:42
mcapeSo I guess i should bite the bullet and just do upgrade for all container-level stuff and hope that new code will sort everything out without crashing the clusters. 22:43
mcapeAny input is much appreciated. 22:43
mcapeHave there been changes in the structure of the container databases in the new versions that will not allow me to go back in case of problems?22:43

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!