Skip to content

Draft: Draft Add option to use Elasticsearch

James Moy requested to merge chikorita157/Sharkey:feature/elasticsearch into develop

What does this PR do? (Please give us a brief description of what this PR does.) This is a port of the Elasticsearch feature from the Misskey io fork of Misskey, ported to Sharkey. Sakurajima Social is running this patch without issues.

The motivation to add Elasticsearch support is that Meilisearch despite marketing itself as being lightweight, uses a lot of disk space when it comes to the index (almost using over 60 GB or more just to make the notes indexable while our Postgres database is smaller than that. It's also uses more CPU.

From the reasons that motivated Misskey IO to add ElasticSearch to their fork was:

  • Poor search accuracy (for Japanese) compared to Elastic
  • Too high of disk usage
  • High CPU when updating indexes (I noticed that CPU usage is a lot higher for Meilisearch than Elastic))

While Elastic will probably use more RAM since it's based on Java, but from experience, the search comes back quickly and it doesn't use nearly as much space for indexes, at least from experience with Mastodon.

However, this implementation is missing two things. It has no scope settings, so it will index all notes. Also, you can't search notes with filetypes. If someone has more experience with TypeScript, maybe someone can help me implement this and make ElasticSearch on par with Meilisearch in terms of what it can search for.

Contribution Guidelines By submitting this merge request, you agree to follow our Contribution Guidelines

  • I agree to follow this project's Contribution Guidelines
  • I have made sure to test this pull request

(While the patch is running on Sakurajima Social, the second is not checked as this is a draft, it's missing features)

Merge request reports