Lack of cache eviction results in infinitely-sized memory caches

What happened?

Several memory caches do not have an eviction policy and can grow to contain millions of entries per active process. This is highly inefficient and may even be a cause of the slow memory leak that affects some instances. In particular, these caches have an infinite eviction time:

• User objects (this cache is duplicated 4 additional times with varying keys)
• Public keys (this cache is duplicated with an alternate key)
• Client app metadata

As a solution, I propose that we define a reasonable cache eviction timeline for all permanent caches. We should also allow some caches to specify a maximum number of entries. If the size is exceeded, the oldest entry is evicted. This latter change would be most effective for the public key and client app caches. If we're willing to invest additional development time, then we can gain a much greater improvement by using Least Recently Used eviction logic. This would allow us to extend eviction times and reduce cache size while still improving cache hit rate.

---

Additional discussion: Misskey issue #14310.

What did you expect to happen?

I expected caches to have a reasonable eviction method to prevent infinite memory usage.

Version

v2024.6.0-dev (change was introduced in March 2021)

Instance

N/A - affects all instances

What type of issue is this?

backend

How do you deploy Sharkey on your server? (Server-side issues only)

N/A - affects all deployment methods

What operating system are you using? (Server-side issues only)

N/A - affects all operating systems

Relevant log output

N/A - this code does not produce logs

Contribution Guidelines By submitting this issue, you agree to follow our Contribution Guidelines

• I agree to follow this project's Contribution Guidelines
• I have searched the issue tracker for similar issues, and this is not a duplicate.

added bugbackend from-upstream labels

possibly useful https://iceshrimp.dev/iceshrimp/iceshrimp-pre-lfs-migration/commit/355b1e0063ab7dea54ce1388f0eb3ad01137180b (thanks @kopper)

although that cache looks like a mix of our memorycache and rediscache, so we may not be able to just copy those changes

I'm working on a fix for this in https://activitypub.software/fEmber/Sharkey/-/commits/hazelnoot/fix-memory-queues

That branch is broken somehow, so work is restarted in https://activitypub.software/fEmber/Sharkey/-/commits/hazelnoot%2F600-remove-infinite-caches

assigned to @fEmber

added in progress label

This could be upstreamed as it is important to eliminate possible memory leak (https://github.com/misskey-dev/misskey/issues/10984)

mentioned in merge request !580 (closed)

changed the description

mentioned in merge request !587 (merged)

mentioned in commit 4e7df7a5

closed with merge request !587 (merged)