Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • TransFem-org/Documentation
  • minneyar/Documentation
  • polygon/Documentation
  • puzzler995/Documentation
  • phoenix_fairy/Documentation
  • Dalek/Documentation
  • piuvas/Documentation
  • Sneexy/Documentation
  • maciejla/Documentation
  • ChloeCat/Documentation
  • nune/Documentation
  • gdude2002/Documentation
  • AverageDood/Documentation
  • Daniel/Documentation
14 results
Show changes
Commits on Source (29)
Hide whitespace changes
Inline Side-by-side
Showing
with 990 additions and 454 deletions
content/en/docs/comparison/misskey.md
View file @ 5343b59c
......@@ -68,6 +68,7 @@ toc: true
* buttons to show/hide all notes with CWs in a conversation
* you can choose to always show long notes in their entirety
* you can choose to always expand notes that have a CW
* you can set a default CW for your notes
* the "parent" note of a reply can be collapsed in timelines
* one-button "like" (plus custom reactions on a separate button)
* animated MFM can be enabled/disabled on each note
......@@ -77,6 +78,8 @@ toc: true
* pop-up user profiles show if follow requests to the user require
approval, have an "open remote profile" option, and show custom
fields (e.g. the user's website address)
* you can accept or reject follow requests from the user profile
* you can "follow back" from a following notification
* user profile pages always show the whole user description
* MFM cheatsheet when composing notes
* MFM won't create a search box just because a sentence ends in
......@@ -166,6 +169,9 @@ toc: true
* admins can set a URL for donations
* moderators can delete all of a user's files
* moderators can delete all files from a remote instance
* moderators can force a CW for a user
* moderators can remove the "quote" feature for specific users (local
or remote) and remote instances
* the PWA icon matches the instance icon
* deleted custom emoji are automatically removed from Drive
* container images are much smaller (we don't ship unused
......@@ -214,3 +220,12 @@ toc: true
without having to fork Sharkey
* if your donation link goes to Open Collective, your supporters will
be shown in the "about" page
* you can generate VAPID keys for push notifications right from the
control panel
* you can set the contents of `robots.txt`
* you can configure the IP address that the server listens on (instead
of "all of them")
* supports building and running on BSD
* you can log all ActivityPub objects to the database, which can help
debugging federation problems
* you can use PostgreSQL `tsvector` search engine
content/en/docs/customisation/search.md 0 → 100644
View file @ 5343b59c
---
title: "Search Engine Options"
weight: 3200
toc: true
---
Sharkey can use a few different methods to implement its note
search. Regardless of which one you use, you always need to enable
note search via the roles system, by enabling "Usage of note search"
either in the role template, or in a role applied to some or all local
users.
## Simple `LIKE`
This is the default. It's very basic and slow, it can only find
exact substring matches (for example, searching for "my cat" will
match "yummy catch" but not "my white cat").
You can make this method less slow by creating a trigram index:
```sql
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE INDEX CONCURRENTLY IF NOT EXISTS "CUSTOM_IDX_note_text_trgm"
ON "note"
USING gin ("text" gin_trgm_ops);
ANALYZE "note";
```
Your Sharkey configuration file (usually `.config/default.yml`) should
already contain the following:
```yaml
fulltextSearch:
provider: sqlLike
```
## Meilisearch
Meilisearch is a very powerful search engine, but its installation is
a bit fiddly. We have [a whole guide for setting it up](meilisearch/).
## PGroonga
This requires Sharkey 2025.2.2 or later.
[PGroonga](https://pgroonga.github.io/) is a PostgreSQL extension that
uses the [Groonga](https://groonga.org/) full-text search engine. It
works pretty well across a variety of languages.
This engine supports some advanced search syntax, read [the
documentation of Groonga's query
syntax](https://groonga.org/docs/reference/grn_expr/query_syntax.html),
including things like `text:@tasty cw:@food` to search for a note
whose CW contains "food" and whose main text contains "tasty".
You have to install PGroonga on the machine that's running your
PostgreSQL, or use one of their [Docker
images](https://hub.docker.com/r/groonga/pgroonga/tags) (be very
careful, if you're switching from a plain PostgreSQL image, to select
one with the same PostgreSQL version!).
Once PGroonga is installed, you can run:
```sql
CREATE EXTENSION IF NOT EXISTS pgroonga;
CREATE INDEX CONCURRENTLY IF NOT EXISTS "CUSTOM_IDX_note_text_pg"
ON "note"
USING pgroonga ("text","cw","name")
WITH (query_allow_column=true);
ANALYZE "note";
```
Then edit your Sharkey configuration file (usually
`.config/default.yml`) so it contains the following, and restart
Sharkey.
```yaml
fulltextSearch:
provider: sqlPgroonga
```
## TSVector
This requires Sharkey 2025.2.2 or later.
[TSVector](https://www.postgresql.org/docs/current/textsearch.html) is
a decent full-text search engine built into PostgreSQL. It mainly
works with languages that use an alphabet, and needs to know what
language each note is written in (but Sharkey doesn't currently know
about note languages).
This engine supports some advanced search syntax, read [the
documentation of
`websearch_to_tsquery`](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-PARSING-QUERIES).
You need to create a dedicated column and an index:
```sql
ALTER TABLE "note"
ADD COLUMN tsvector_embedding tsvector
GENERATED ALWAYS AS (
to_tsvector('english',
COALESCE("text", '') || ' ' ||
COALESCE("cw", '') || ' ' ||
COALESCE("name", '')
)
) STORED;
CREATE INDEX CUSTOM_IDX_note_text_tsv
ON "note"
USING gin (tsvector_embedding);
ANALYZE "note";
```
Notice that we're telling it to consider the notes written in
English. You can use a different dictionary for better results if your
main instance language is not English. To get a list, run `SELECT
cfgname FROM pg_ts_config;`, then replace `'english'` in the call to
`to_tsvector` with the desired dictionary name.
Then edit your Sharkey configuration file (usually
`.config/default.yml`) so it contains the following, and restart
Sharkey.
```yaml
fulltextSearch:
provider: sqlTsvector
```
content/en/docs/customisation/meilisearch.md → content/en/docs/customisation/search/meilisearch.md
View file @ 5343b59c
---
title: "Meilisearch"
weight: 3200
weight: 3210
toc: true
---
......@@ -51,16 +51,19 @@ List keys:
curl -s 'http://localhost:7700/keys' -H "Authorization: Bearer $MASTER_KEY" | jq .
```
## Update sharkey default.yml
## Update Sharkey default.yml
Update sharkey's default.yml to contain the below and restart.
Update Sharkey's default.yml to contain the below and restart.
```yaml
meilisearch:
host: localhost # Host for sharkey to connect to. Docker configurations replace with 'meilisearch' instead.
port: 7700
apiKey: '<Key from previus step>'
ssl: false
index: '<Index to use>' # `shonk_social` in this example. Omit the `---notes`
scope: global # use local to only index local notes and not remote ones.
fulltextSearch:
provider: meilisearch
meilisearch:
host: localhost # Host for sharkey to connect to. Docker configurations replace with 'meilisearch' instead.
port: 7700
apiKey: '<Key from previus step>'
ssl: false
index: '<Index to use>' # `shonk_social` in this example. Omit the `---notes`
scope: global # use local to only index local notes and not remote ones.
```
content/en/docs/install/faqs.md
View file @ 5343b59c
......@@ -37,6 +37,9 @@ either expand the "role template" (if you want to give every user
access to search) or create a new role (if you want to give access to
only some users), then change the "**Usage of note search**" setting.
Also look at [the various search engines that you can set
up](../../customisation/search/).
## How do I give my users more Drive space?
You use the "roles" system. Log in as administrator, go to the
......@@ -48,7 +51,13 @@ capacity**" setting.
## How do I enable push notifications for the web interface?
First of all, you need to generate a pair of so-called "VAPID" keys.
Log in as administrator, go to the "control panel", select the
"general" section (under "settings"), scroll to the "ServiceWorker"
bit, enable the "Enable Push-Notifications for your Browser" toggle,
press the "Generate keys" button, then the "save" button.
For versions of Sharkey before 2025.2.2, you need to generate the
"VAPID" keys by hand.
One way to do that is, from your Sharkey directory (git clone, or
inside the Docker image):
......@@ -60,10 +69,8 @@ inside the Docker image):
Alternatively, you can use [an online
generator](https://www.stephane-quantin.com/en/tools/generators/vapid-keys).
Once you have that public and private keys, log in as administrator,
go to the "control panel", select the "general" section (under
"settings"), scroll to the "ServiceWorker" bit, enter both keys, and
enable the "Enable Push-Notifications for your Browser" toggle.
Once you have that public and private keys, you can copy them into the
two input boxes in "ServiceWorker" form, and press "save".
## Image upload doesn't work!
......
content/en/docs/install/fresh.md
View file @ 5343b59c
......@@ -12,6 +12,8 @@ Before you start, you really need:
* a web server running on that machine, with valid TLS certificates
for the domain name
Please note that the first registered user account of your instance is subject to some restrictions and cannot be migrated, for example. Therefore, the account should not be used as a normal user account.
## With Docker
Prerequisites:
......@@ -57,8 +59,9 @@ the various ways you can set configuration values.
Edit `docker-compose.yml`, there are multiple comments there as
well. If you want to set up note search with meilisearch, uncomment
all of meilisearch options, otherwise proceed to do the following
changes in the `services:` / `web:` section:
all of meilisearch options (but also check out [all the search engine
options](../../customisation/search/)), otherwise proceed to do the
following changes in the `services:` / `web:` section:
* uncomment the line that starts with `image:`
* comment out the line that starts with `build: .`
......@@ -113,6 +116,9 @@ Prerequisites:
* FFmpeg
* all the various packages to compile and build C code, and Python (on
Debian-style systems, that's `build-essential` & `python`)
* some system libraries and their development headers: pango, cairo,
pixman (on Debian-style systems, that's `libpango1.0-dev
libcairo2-dev libpixman-1-dev`)
Create a `sharkey` user:
......@@ -158,7 +164,8 @@ instance will be protected between when you start it and when you
create your first user.
Refer to [the Configuration Options page](../configuration/) for all
the various ways you can set configuration values.
the various ways you can set configuration values, and check out [all
the search engine options](../../customisation/search/).
Build Sharkey:
......@@ -371,6 +378,27 @@ reverse_proxy localhost:3000
}
```
### Apache
Something like this works for Apache 2.4.47 or later:
```apacheconf
<VirtualHost *>
DocumentRoot /var/empty
ServerName sharkey.example
SSLEngine On
AllowEncodedSlashes NoDecode
ProxyPass "/" "http://127.0.0.1:3000/" upgrade=websocket
# or, if your Sharkey listens on a UNIX socket
# ProxyPass "/" "unix:/opt/sharkey/sharkey.sock|http://127.0.0.1/" upgrade=websocket
ProxyPassReverse / https://sharkey.example/
</VirtualHost>
```
## Update Sharkey
### With Docker
......@@ -378,7 +406,7 @@ reverse_proxy localhost:3000
Fetch the latest image, restart:
```bash
docker compose down && docker compose build --pull && docker compose up -d
docker compose pull && docker compose up -d
```
**NOTE**: if you are upgrading from 2024.3.1 or earlier, and are *not*
......@@ -425,6 +453,11 @@ needs Node.js 22.11, and you will almost certainly need to run `pnpm
run clean-all` before `pnpm install`, otherwise you'll end up with
some dependencies (e.g. `re2`) linked to the wrong libraries.
**NOTE**: if you upgrading from 2024.11.2 or earlier, Sharkey now
needs some system libraries and their development headers: pango,
cairo, pixman (on Debian-style systems, that's `libpango1.0-dev
libcairo2-dev libpixman-1-dev`)
### Caching issues
It's possible that the web server (or the CDN, if you've configured
......
content/en/docs/install/migrate.md
View file @ 5343b59c
......@@ -4,439 +4,12 @@ weight: 1200
toc: true
---
## From Misskey
* [from Misskey](misskey/)
* from Firefish:
Let's say you have a working Misskey, running as user `misskey` from
`/home/misskey/misskey`.
we include guides for [Firefish v1.0.5-RC](firefish-1.0.5/) and
[Firefish v20241205](firefish-20241205/); other versions and
instances may differ quite a lot from the ones we used, especially
if you have custom patches applied.
Migrating to Sharkey is the same as updating to a newer Misskey
version:
```shell
sudo -u misskey -i
cd misskey
git remote rename origin misskey
git remote add origin https://activitypub.software/TransFem-org/Sharkey.git
git remote update -p
git checkout -b stable --track origin/stable
git pull --recurse-submodules
pnpm install --frozen-lockfile
pnpm run build
pnpm run migrate
```
Then you can restart your service.
If you see weirdness like service not starting, or missing labels in
the web UI, you should first make sure the build worked (check all the
error messages!), then try building again from scratch:
```shell
pnpm run clean-all
pnpm install --frozen-lockfile
pnpm run build
pnpm run migrate
```
Also, clear your browser's cache and local storage (this will log your
browser out of Misskey/Sharkey).
### Using Docker
If you are using the `docker-compose.yml` file from Misskey which builds locally follow these steps:
```shell
git remote rename origin misskey
git remote add origin https://activitypub.software/TransFem-org/Sharkey.git
git remote update -p
git checkout -b stable --track origin/stable
git pull --recurse-submodules
pnpm install --frozen-lockfile
sudo docker compose build
```
## From Firefish
This guide was only tested on a Firefish 1.0.5-RC instance: other
versions and instances may differ quite a lot from the one we used,
especially if you have custom patches applied.
If any of the steps fails, especially SQL queries, seek help from us
[Via Discord](https://discord.gg/8hF6pMVWja). Do *not* try fixing things
in the DB yourself unless you *really* know what you are doing.
Before you begin, please take a backup of your database. While you're
there, make sure you know how to restore from that backup! Using the
`plain` format for `pg_dump` is probably the simplest way.
It's a good idea to have a separate backup of your list of silenced
instances and of your `user` table.
### Using Docker
Stop / shut down the entire stack: PostgreSQL, Redis / KeyDB /
DragonflyDB, Sonic / ElasticSearch / MeiliSearch, Firefish
itself. Stop all of it.
Edit you docker compose and replace the Firefish image with
`registry.activitypub.software/transfem-org/sharkey:latest`
If you use Sonic or ElasticSearch replace that section of the Docker
Compose with the following, as Sharkey currently only supports
meilisearch:
```yaml
meilisearch:
restart: always
image: getmeili/meilisearch:v1.3.4
environment:
- MEILI_NO_ANALYTICS=true
- MEILI_ENV=production
env_file:
- .config/meilisearch.env
networks:
- calcnet # <-- Use whatever network name is used in the docker compose here
volumes:
- ./meili_data:/meili_data # <--- make sure to replace the volume with one that fits your existing docker compose
```
If you use DragonflyDB replace it with Redis or KeyDB, as Sharkey
currently does not support DragonflyDB. To do this replace the section
in the docker compose with the following:
```yaml
redis:
restart: always
image: redis:7-alpine
networks:
- calcnet # <-- Use whatever network name is used in the docker compose here
volumes:
- ./redis:/data # <-- Make sure to replace the volume with the one used in your firefish docker compose
healthcheck:
test: "redis-cli ping"
interval: 5s
retries: 20
```
Backup your Firefish config and replace it with the [default
Sharkey
one](https://activitypub.software/TransFem-org/Sharkey/-/raw/stable/.config/docker_example.yml)
Edit the config inline with your instance settings. Make sure to use
the same `db` & `redis` settings as in your Firefish config.
Now is the time to *backup* your database and Redis volumes!
Firefish's docker-compose uses PostgreSQL version 12. We strongly
recommend upgrading to 15 or 16. You can do this by making a new db
volume, starting the newer PostgreSQL on it, and importing the data
from the backup you just made. Refer to [the PostgreSQL
documentation](https://www.postgresql.org/docs/current/backup-dump.html).
Make sure to update the *mount paths* for volumes of the Sharkey
container from `/firefish/` to `/sharkey/` (so your existing volumes
will show up at the new path, inside the container).
Now start *only* the database with `docker compose up -d db` (instead
of `db`, you may need to use whatever name is set for the service in
your docker compose config), and start a `psql` shell with `docker
exec -it db psql -U firefish -d firefish` (replace `db` as before, and
`-U firefish -d firefish` with the database user and database name,
respectively, if they're different from `firefish`).
You should now be connected to your Firefish database. We need to
massage it into shape so that Sharkey database migrations will
work. The following series of SQL queries / commands should do it, but
please read the comments and pay attention to the results after each
query!
```sql
-- start a transaction, so we won't leave the db in a halfway state if
-- things go wrong
BEGIN;
-- we need to add back some columns that Firefish removed, but that
-- Sharkey migrations expect
ALTER TABLE "user_profile" ADD "integrations" JSONB NOT NULL DEFAULT '{}';
ALTER TABLE "meta" ADD "twitterConsumerSecret" VARCHAR(128);
ALTER TABLE "meta" ADD "twitterConsumerKey" VARCHAR(128);
ALTER TABLE "meta" ADD "enableTwitterIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "enableGithubIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "githubClientId" VARCHAR(128);
ALTER TABLE "meta" ADD "githubClientSecret" VARCHAR(128);
ALTER TABLE "meta" ADD "enableDiscordIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "discordClientId" VARCHAR(128);
ALTER TABLE "meta" ADD "discordClientSecret" VARCHAR(128);
-- also an extra table, for the same reasons
CREATE TABLE antenna_note();
-- Misskey used to have a Reversi game, Firefish dropped the tables,
-- now Misskey uses them again
CREATE TABLE "reversi_game" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "startedAt" TIMESTAMP WITH TIME ZONE, "user1Id" character varying(32) NOT NULL, "user2Id" character varying(32) NOT NULL, "user1Accepted" boolean NOT NULL DEFAULT false, "user2Accepted" boolean NOT NULL DEFAULT false, "black" integer, "isStarted" boolean NOT NULL DEFAULT false, "isEnded" boolean NOT NULL DEFAULT false, "winnerId" character varying(32), "surrendered" character varying(32), "logs" jsonb NOT NULL DEFAULT '[]', "map" character varying(64) array NOT NULL, "bw" character varying(32) NOT NULL, "isLlotheo" boolean NOT NULL DEFAULT false, "canPutEverywhere" boolean NOT NULL DEFAULT false, "loopedBoard" boolean NOT NULL DEFAULT false, "form1" jsonb DEFAULT null, "form2" jsonb DEFAULT null, "crc32" character varying(32), CONSTRAINT "PK_76b30eeba71b1193ad7c5311c3f" PRIMARY KEY ("id"));
CREATE INDEX "IDX_b46ec40746efceac604142be1c" ON "reversi_game" ("createdAt");
CREATE TABLE "reversi_matching" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "parentId" character varying(32) NOT NULL, "childId" character varying(32) NOT NULL, CONSTRAINT "PK_880bd0afbab232f21c8b9d146cf" PRIMARY KEY ("id"));
CREATE INDEX "IDX_b604d92d6c7aec38627f6eaf16" ON "reversi_matching" ("createdAt");
CREATE INDEX "IDX_3b25402709dd9882048c2bbade" ON "reversi_matching" ("parentId");
CREATE INDEX "IDX_e247b23a3c9b45f89ec1299d06" ON "reversi_matching" ("childId");
-- move aside some FireFish columns; Sharkey migrations will
-- re-create them; we don't `DROP` them because we want to keep the data
ALTER TABLE "user" RENAME COLUMN "movedToUri" TO "ff_movedToUri";
ALTER TABLE "user" RENAME COLUMN "alsoKnownAs" TO "ff_alsoKnownAs";
ALTER TABLE "user" RENAME COLUMN "isIndexable" TO "ff_isIndexable";
ALTER TABLE "user" RENAME COLUMN "speakAsCat" TO "ff_speakAsCat";
ALTER TABLE "user_profile" RENAME COLUMN "preventAiLearning" TO "ff_preventAiLearning";
ALTER TABLE "meta" RENAME COLUMN "silencedHosts" TO "ff_silencedHosts";
-- this column was added by both Firefish and Misskey, but with
-- different names, let's fix it
ALTER TABLE "meta" RENAME COLUMN "ToSUrl" TO "termsOfServiceUrl";
-- update antenna types, this is only needed on some instances but
-- recommend to run anyway
--
-- this *removes* any antennas of types not supported by Sharkey!
CREATE TYPE public.new_antenna_src_enum AS ENUM ('home', 'all', 'list');
ALTER TABLE antenna ADD COLUMN new_src public.new_antenna_src_enum;
DELETE FROM antenna WHERE src NOT IN ('home', 'all', 'list');
ALTER TABLE antenna DROP COLUMN src;
ALTER TABLE antenna RENAME COLUMN new_src TO src;
DROP TYPE public.antenna_src_enum;
ALTER TYPE new_antenna_src_enum RENAME TO antenna_src_enum;
-- optional but recommended: delete all empty moderation log entries
DELETE FROM moderation_log WHERE info = '{}';
-- only needed on some instances, run this if
-- `\dT+ user_profile_mutingnotificationtypes_enum`
-- does not show `note` in the "elements" section
ALTER TYPE "public"."user_profile_mutingnotificationtypes_enum" ADD VALUE 'note';
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Matrix](https://matrix.to/#/#sharkey-support:shourai.de) or
[Discord](https://discord.gg/8hF6pMVWja)!
Start Sharkey, and let it run all its migrations. Once that's done,
and Starkey says it's listening, stop Sharkey but keep the database
running.
Open another `psql` shell like before (`docker exec -it db psql -U
firefish -d firefish`, replacing things as before). We need another
small pass of massaging.
```sql
BEGIN;
-- all existing users are approved, because Firefish doesn't have a
-- concept of approvals
UPDATE "user" SET approved = true;
-- now we put back the data we moved aside
UPDATE "user" SET "movedToUri" = "ff_movedToUri" WHERE "ff_movedToUri" IS NOT NULL;
UPDATE "user" SET "alsoKnownAs" = "ff_alsoKnownAs" WHERE "ff_alsoKnownAs" IS NOT NULL;
UPDATE "user" SET "noindex" = NOT (COALESCE("ff_isIndexable", true));
UPDATE "user" SET "speakAsCat" = COALESCE("ff_speakAsCat", false);
UPDATE "user_profile" SET "preventAiLearning" = COALESCE("ff_preventAiLearning", true);
UPDATE "meta" SET "silencedHosts" = COALESCE("ff_silencedHosts",'{}');
ALTER TABLE "user" DROP COLUMN "ff_movedToUri";
ALTER TABLE "user" DROP COLUMN "ff_alsoKnownAs";
ALTER TABLE "user" DROP COLUMN "ff_isIndexable";
ALTER TABLE "user" DROP COLUMN "ff_speakAsCat";
ALTER TABLE "user_profile" DROP COLUMN "ff_preventAiLearning";
ALTER TABLE "meta" DROP COLUMN "ff_silencedHosts";
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start everything up again, you should see no errors in the logs.
Log in as an administrator, and go to the control panel.
If you use an object store such as S3, double-check your settings
(it's possible, for example, that the URL now looks like
`https://https://yourdomain.com`, fix it). If you want your users to
be able to search notes, you must enable it via the "roles" system.
Congratulations, you're now running Sharkey!
## From Iceshrimp
This guide was only tested on an Iceshrimp-JS instance, on version v2023.12.11. Iceshrimp.NET is not supported,
and as far as known, the migration from Iceshrimp-JS to Iceshrimp.NET is a one way street.
Furthermore, this was tested on a **vanilla** Iceshrimp-JS instance. Other Iceshrimp instances
that may run with custom patches may require a different treatment.
If any of the steps fails, especially SQL queries, seek help from us
[Via Discord](https://discord.gg/8hF6pMVWja). Do *not* try fixing things
in the DB yourself unless you *really* know what you are doing.
Before you begin, please take a backup of your database. While you're
there, make sure you know how to restore from that backup! Using the
`plain` format for `pg_dump` is probably the simplest way.
It's a good idea to have a separate backup of your list of silenced
instances and of your `user` table.
### Using Docker
Stop / shut down the entire stack: PostgreSQL, Redis / Valkey and
Iceshrimp itself. Stop all of it.
Now is the time to *backup* your database and Redis volumes!
You will need to revert Iceshrimp migrations back to the last common one with
Firefish, which should leave you on Firefish 1.0.2 schema.
First, connect to the database container. Start *only* the database with `docker compose up -d db`
(instead of `db`, you may need to use whatever name is set for the service in
your docker compose config), and start a `psql` shell with `docker
exec -it db psql -U iceshrimp -d iceshrimp` (replace `db` as before, and
`-U iceshrimp -d iceshrimp` with the database user and database name,
respectively, if they're different from `iceshrimp`).
You should now be connected to your Iceshrimp database. We need first to delete from the
tables 3 of the Iceshrimp migrations that are not able to be reverted, regarding OAuth and
instance's max character limit. Don't worry, the OAuth is going to be dropped enterily by
one of the migration reverts, and the max character limit does not conflict with Sharkey, and
in fact, could be overwritten safely by a Sharkey migration.
Make sure you have a backup safely stored, and then delete the mentioned
migrations with the following queries:
```sql
DELETE FROM "migrations" WHERE "name" = 'IncreaseHostCharLimit1701118152149';
DELETE FROM "migrations" WHERE "name" = 'IncreaseOauthTokenRedirecturisLength1697286869039';
DELETE FROM "migrations" WHERE "name" = 'IncreaseOAuthRedirecturisLength1697246035867';
```
We are now ready to start reverting Iceshrimp migrations.
Edit the `docker-compose.yml` file and add `entrypoint: /bin/sh` to the web section of the file.
Save the file and start the container with `docker-compose up -d web`, and then run
`docker exec -it iceshrimp_web sh`.
Replace "iceshrimp_web" with the name of your container if you've changed it.
Now, navigate to the Iceshrimp directory, and inside of it, nagivate to `packages/backend`.
The command to run in order to revert the latest applies migration is
```bash
yarn run typeorm migration:revert -d built/ormconfig.js
```
You'll have to run it several times, until you see that `IceshrimpRepo1689965609061` has been reverted.
You're now back on the Firefish 1.0.2 database scheme.
Exit the shell and run `docker compose down web` to stop the Iceshrimp container.
Now, edit you docker compose and replace the Iceshrimp image with
`registry.activitypub.software/transfem-org/sharkey:latest`
Backup your Iceshrimp config and replace it with the [default
Sharkey one](https://activitypub.software/TransFem-org/Sharkey/-/raw/stable/.config/docker_example.yml)
Edit the config inline with your instance settings. Make sure to use
the same `db` & `redis` settings as in your Iceshrimp config.
Make sure to update the *mount paths* for volumes of the Sharkey
container from `/iceshrimp/` to `/sharkey/` (so your existing volumes
will show up at the new path, inside the container).
Now we need to massage the database into shape so that Sharkey database migrations will
work. The following series of SQL queries / commands should do it, but
please read the comments and pay attention to the results after each
query!
```sql
-- start a transaction, so we won't leave the db in a halfway state if
-- things go wrong
BEGIN;
-- move aside some existing columns; Sharkey migrations will
-- re-create them; we don't `DROP` them because we want to keep the data
ALTER TABLE "user" RENAME COLUMN "movedToUri" TO "ish_movedToUri";
ALTER TABLE "user" RENAME COLUMN "alsoKnownAs" TO "ish_alsoKnownAs";
ALTER TABLE "user" RENAME COLUMN "speakAsCat" TO "ish_speakAsCat";
ALTER TABLE "user_profile" RENAME COLUMN "preventAiLearning" TO "ish_preventAiLearning";
ALTER TABLE "meta" RENAME COLUMN "silencedHosts" TO "ish_silencedHosts";
-- this column was added by both Iceshrimp, and Misskey, but with
-- different names, let's fix it
ALTER TABLE "meta" RENAME COLUMN "ToSUrl" TO "termsOfServiceUrl";
-- update antenna types, this is only needed on some instances but
-- recommend to run anyway
--
-- this *removes* any antennas of types not supported by Sharkey!
CREATE TYPE public.new_antenna_src_enum AS ENUM ('home', 'all', 'list');
ALTER TABLE antenna ADD COLUMN new_src public.new_antenna_src_enum;
DELETE FROM antenna WHERE src NOT IN ('home', 'all', 'list');
ALTER TABLE antenna DROP COLUMN src;
ALTER TABLE antenna RENAME COLUMN new_src TO src;
DROP TYPE public.antenna_src_enum;
ALTER TYPE new_antenna_src_enum RENAME TO antenna_src_enum;
-- optional but recommended: delete all empty moderation log entries
DELETE FROM moderation_log WHERE info = '{}';
-- only needed on some instances, run this if
-- `\dT+ user_profile_mutingnotificationtypes_enum`
-- does not show `note` in the "elements" section
ALTER TYPE "public"."user_profile_mutingnotificationtypes_enum" ADD VALUE 'note';
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Matrix](https://matrix.to/#/#sharkey-support:shourai.de) or
[Discord](https://discord.gg/8hF6pMVWja)!
Start Sharkey, and let it run all its migrations. Once that's done,
and Starkey says it's listening, stop Sharkey but keep the database
running.
Open another `psql` shell like before (`docker exec -it db psql -U
firefish -d firefish`, replacing things as before). We need another
small pass of massaging.
```sql
BEGIN;
-- all existing users are approved, because Firefish doesn't have a
-- concept of approvals
UPDATE "user" SET approved = true;
-- now we put back the data we moved aside
UPDATE "user" SET "movedToUri" = "ish_movedToUri" WHERE "ish_movedToUri" IS NOT NULL;
UPDATE "user" SET "alsoKnownAs" = "ish_alsoKnownAs" WHERE "ish_alsoKnownAs" IS NOT NULL;
UPDATE "user" SET "speakAsCat" = COALESCE("ish_speakAsCat", false);
UPDATE "user_profile" SET "preventAiLearning" = COALESCE("ish_preventAiLearning", true);
UPDATE "meta" SET "silencedHosts" = COALESCE("ish_silencedHosts",'{}');
ALTER TABLE "user" DROP COLUMN "ish_movedToUri";
ALTER TABLE "user" DROP COLUMN "ish_alsoKnownAs";
ALTER TABLE "user" DROP COLUMN "ish_speakAsCat";
ALTER TABLE "user_profile" DROP COLUMN "ish_preventAiLearning";
ALTER TABLE "meta" DROP COLUMN "ish_silencedHosts";
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start everything up again, you should see no errors in the logs.
Log in as an administrator, and go to the control panel.
If you use an object store such as S3, double-check your settings
(it's possible, for example, that the URL now looks like
`https://https://yourdomain.com`, fix it). If you want your users to
be able to search notes, you must enable it via the "roles" system.
Congratulations, you're now running Sharkey!
* [from Iceshrimp](iceshrimp/)
content/en/docs/install/migrate/firefish-1.0.5.md 0 → 100644
View file @ 5343b59c
---
title: "Migrating from Firefish v1.0.5-RC"
weight: 1220
toc: false
---
If any of the steps fails, especially SQL queries, seek help from us
[Via Discord](https://discord.gg/8hF6pMVWja). Do *not* try fixing things
in the DB yourself unless you *really* know what you are doing.
Before you begin, please take a backup of your database. While you're
there, make sure you know how to restore from that backup! Using the
`plain` format for `pg_dump` is probably the simplest way.
It's a good idea to have a separate backup of your list of silenced
instances and of your `user` table.
### Using Docker (v1.0.5-RC)
This guide was tested on Firefish version 1.0.5-RC. If you're using
version 20241205 or later, [please see the other
quide](../firefish-20241205/).
Stop / shut down the entire stack: PostgreSQL, Redis / KeyDB /
DragonflyDB, Sonic / ElasticSearch / MeiliSearch, Firefish
itself. Stop all of it.
Edit you docker compose and replace the Firefish image with
`registry.activitypub.software/transfem-org/sharkey:latest`
If you use Sonic or ElasticSearch replace that section of the Docker
Compose with the following, as Sharkey currently only supports
meilisearch:
```yaml
meilisearch:
restart: always
image: getmeili/meilisearch:v1.3.4
environment:
- MEILI_NO_ANALYTICS=true
- MEILI_ENV=production
env_file:
- .config/meilisearch.env
networks:
- calcnet # <-- Use whatever network name is used in the docker compose here
volumes:
- ./meili_data:/meili_data # <--- make sure to replace the volume with one that fits your existing docker compose
```
If you use DragonflyDB replace it with Redis or KeyDB or Valkey, as
Sharkey currently does not support DragonflyDB. To do this replace the
section in the docker compose with the following:
```yaml
redis:
restart: always
image: redis:7-alpine
networks:
- calcnet # <-- Use whatever network name is used in the docker compose here
volumes:
- ./redis:/data # <-- Make sure to replace the volume with the one used in your firefish docker compose
healthcheck:
test: "redis-cli ping"
interval: 5s
retries: 20
```
You may need to migrate the data from DragonflyDB to the new software,
the [`librdb` command line tool](https://github.com/redis/librdb) may
help with that.
Backup your Firefish config and replace it with the [default
Sharkey
one](https://activitypub.software/TransFem-org/Sharkey/-/raw/stable/.config/docker_example.yml)
Edit the config inline with your instance settings. Make sure to use
the same `db` & `redis` settings as in your Firefish config.
Now is the time to *backup* your database and Redis volumes!
Firefish's docker-compose uses PostgreSQL version 12. We strongly
recommend upgrading to 15 or 16. You can do this by making a new db
volume, starting the newer PostgreSQL on it, and importing the data
from the backup you just made. Refer to [the PostgreSQL
documentation](https://www.postgresql.org/docs/current/backup-dump.html).
Make sure to update the *mount paths* for volumes of the Sharkey
container from `/firefish/` to `/sharkey/` (so your existing volumes
will show up at the new path, inside the container).
Now start *only* the database with `docker compose up -d db` (instead
of `db`, you may need to use whatever name is set for the service in
your docker compose config), and start a `psql` shell with `docker
exec -it db psql -U firefish -d firefish` (replace `db` as before, and
`-U firefish -d firefish` with the database user and database name,
respectively, if they're different from `firefish`).
You should now be connected to your Firefish database. We need to
massage it into shape so that Sharkey database migrations will
work. The following series of SQL queries / commands should do it, but
please read the comments and pay attention to the results after each
query!
```sql
-- start a transaction, so we won't leave the db in a halfway state if
-- things go wrong
BEGIN;
-- we need to add back some columns that Firefish removed, but that
-- Sharkey migrations expect
ALTER TABLE "user_profile" ADD "integrations" JSONB NOT NULL DEFAULT '{}';
ALTER TABLE "meta" ADD "twitterConsumerSecret" VARCHAR(128);
ALTER TABLE "meta" ADD "twitterConsumerKey" VARCHAR(128);
ALTER TABLE "meta" ADD "enableTwitterIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "enableGithubIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "githubClientId" VARCHAR(128);
ALTER TABLE "meta" ADD "githubClientSecret" VARCHAR(128);
ALTER TABLE "meta" ADD "enableDiscordIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "discordClientId" VARCHAR(128);
ALTER TABLE "meta" ADD "discordClientSecret" VARCHAR(128);
-- also an extra table, for the same reasons
CREATE TABLE antenna_note();
-- Misskey used to have a Reversi game, Firefish dropped the tables,
-- now Misskey uses them again
CREATE TABLE "reversi_game" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "startedAt" TIMESTAMP WITH TIME ZONE, "user1Id" character varying(32) NOT NULL, "user2Id" character varying(32) NOT NULL, "user1Accepted" boolean NOT NULL DEFAULT false, "user2Accepted" boolean NOT NULL DEFAULT false, "black" integer, "isStarted" boolean NOT NULL DEFAULT false, "isEnded" boolean NOT NULL DEFAULT false, "winnerId" character varying(32), "surrendered" character varying(32), "logs" jsonb NOT NULL DEFAULT '[]', "map" character varying(64) array NOT NULL, "bw" character varying(32) NOT NULL, "isLlotheo" boolean NOT NULL DEFAULT false, "canPutEverywhere" boolean NOT NULL DEFAULT false, "loopedBoard" boolean NOT NULL DEFAULT false, "form1" jsonb DEFAULT null, "form2" jsonb DEFAULT null, "crc32" character varying(32), CONSTRAINT "PK_76b30eeba71b1193ad7c5311c3f" PRIMARY KEY ("id"));
CREATE INDEX "IDX_b46ec40746efceac604142be1c" ON "reversi_game" ("createdAt");
CREATE TABLE "reversi_matching" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "parentId" character varying(32) NOT NULL, "childId" character varying(32) NOT NULL, CONSTRAINT "PK_880bd0afbab232f21c8b9d146cf" PRIMARY KEY ("id"));
CREATE INDEX "IDX_b604d92d6c7aec38627f6eaf16" ON "reversi_matching" ("createdAt");
CREATE INDEX "IDX_3b25402709dd9882048c2bbade" ON "reversi_matching" ("parentId");
CREATE INDEX "IDX_e247b23a3c9b45f89ec1299d06" ON "reversi_matching" ("childId");
-- move aside some FireFish columns; Sharkey migrations will
-- re-create them; we don't `DROP` them because we want to keep the data
ALTER TABLE "user" RENAME COLUMN "movedToUri" TO "ff_movedToUri";
ALTER TABLE "user" RENAME COLUMN "alsoKnownAs" TO "ff_alsoKnownAs";
ALTER TABLE "user" RENAME COLUMN "isIndexable" TO "ff_isIndexable";
ALTER TABLE "user" RENAME COLUMN "speakAsCat" TO "ff_speakAsCat";
ALTER TABLE "user_profile" RENAME COLUMN "preventAiLearning" TO "ff_preventAiLearning";
ALTER TABLE "meta" RENAME COLUMN "silencedHosts" TO "ff_silencedHosts";
-- this column was added by both Firefish and Misskey, but with
-- different names, let's fix it
ALTER TABLE "meta" RENAME COLUMN "ToSUrl" TO "termsOfServiceUrl";
-- update antenna types, this is only needed on some instances but
-- recommend to run anyway
--
-- this *removes* any antennas of types not supported by Sharkey!
CREATE TYPE public.new_antenna_src_enum AS ENUM ('home', 'all', 'list');
ALTER TABLE antenna ADD COLUMN new_src public.new_antenna_src_enum;
DELETE FROM antenna WHERE src NOT IN ('home', 'all', 'list');
ALTER TABLE antenna DROP COLUMN src;
ALTER TABLE antenna RENAME COLUMN new_src TO src;
DROP TYPE public.antenna_src_enum;
ALTER TYPE new_antenna_src_enum RENAME TO antenna_src_enum;
-- optional but recommended: delete all empty moderation log entries
DELETE FROM moderation_log WHERE info = '{}';
-- only needed on some instances, run this if
-- `\dT+ user_profile_mutingnotificationtypes_enum`
-- does not show `note` in the "elements" section
ALTER TYPE "public"."user_profile_mutingnotificationtypes_enum" ADD VALUE 'note';
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start Sharkey, and let it run all its migrations. Once that's done,
and Sharkey says it's listening, stop Sharkey but keep the database
running.
Open another `psql` shell like before (`docker exec -it db psql -U
firefish -d firefish`, replacing things as before). We need another
small pass of massaging.
```sql
BEGIN;
-- all existing users are approved, because Firefish doesn't have a
-- concept of approvals
UPDATE "user" SET approved = true;
-- now we put back the data we moved aside
UPDATE "user" SET "movedToUri" = "ff_movedToUri" WHERE "ff_movedToUri" IS NOT NULL;
UPDATE "user" SET "alsoKnownAs" = "ff_alsoKnownAs" WHERE "ff_alsoKnownAs" IS NOT NULL;
UPDATE "user" SET "noindex" = NOT (COALESCE("ff_isIndexable", true));
UPDATE "user" SET "speakAsCat" = COALESCE("ff_speakAsCat", false);
UPDATE "user_profile" SET "preventAiLearning" = COALESCE("ff_preventAiLearning", true);
UPDATE "meta" SET "silencedHosts" = COALESCE("ff_silencedHosts",'{}');
ALTER TABLE "user" DROP COLUMN "ff_movedToUri";
ALTER TABLE "user" DROP COLUMN "ff_alsoKnownAs";
ALTER TABLE "user" DROP COLUMN "ff_isIndexable";
ALTER TABLE "user" DROP COLUMN "ff_speakAsCat";
ALTER TABLE "user_profile" DROP COLUMN "ff_preventAiLearning";
ALTER TABLE "meta" DROP COLUMN "ff_silencedHosts";
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start everything up again, you should see no errors in the logs.
Log in as an administrator, and go to the control panel.
If you use an object store such as S3, double-check your settings
(it's possible, for example, that the URL now looks like
`https://https://yourdomain.com`, fix it). If you want your users to
be able to search notes, you must enable it via the "roles" system.
Congratulations, you're now running Sharkey!
content/en/docs/install/migrate/firefish-20241205.md 0 → 100644
View file @ 5343b59c
---
title: "Migrating from Firefish v20241205"
weight: 1230
toc: false
---
If any of the steps fails, especially SQL queries, seek help from us
[Via Discord](https://discord.gg/8hF6pMVWja). Do *not* try fixing things
in the DB yourself unless you *really* know what you are doing.
Before you begin, please take a backup of your database. While you're
there, make sure you know how to restore from that backup! Using the
`plain` format for `pg_dump` is probably the simplest way.
It's a good idea to have a separate backup of your list of silenced
instances and of your `user` table.
### Using Docker (v20241205)
This guide was tested on Firefish version 20241205. If you're using
version 1.0.5-RC, [please see the other guide](../firefish-1.0.5/).
{{< callout context="caution" icon="alert-triangle" >}}
Antennas from this version of Firefish are not compatible with Sharkey,
and you will need to delete them as part of this guide.
Please take note of any important antennas and recreate them when you're finished with the guide.
{{< /callout >}}
Stop / shut down the entire stack: PostgreSQL, Redis / KeyDB /
DragonflyDB, Sonic / ElasticSearch / MeiliSearch, Firefish
itself. Stop all of it.
Edit your docker compose and replace the Firefish image with
`registry.activitypub.software/transfem-org/sharkey:latest`
If you use Sonic or ElasticSearch replace that section of the Docker
Compose with the following, as Sharkey currently only supports
meilisearch:
```yaml
meilisearch:
restart: always
image: getmeili/meilisearch:v1.3.4
environment:
- MEILI_NO_ANALYTICS=true
- MEILI_ENV=production
env_file:
- .config/meilisearch.env
networks:
- calcnet # <-- Use whatever network name is used in the docker compose here
volumes:
- ./meili_data:/meili_data # <--- make sure to replace the volume with one that fits your existing docker compose
```
If you use DragonflyDB replace it with Redis or KeyDB or Valkey, as
Sharkey currently does not support DragonflyDB. To do this replace the
section in the docker compose with the following:
```yaml
redis:
restart: always
image: redis:7-alpine
networks:
- calcnet # <-- Use whatever network name is used in the docker compose here
volumes:
- ./redis:/data # <-- Make sure to replace the volume with the one used in your firefish docker compose
healthcheck:
test: "redis-cli ping"
interval: 5s
retries: 20
```
You may need to migrate the data from DragonflyDB to the new software,
the [`librdb` command line tool](https://github.com/redis/librdb) may
help with that.
Backup your Firefish config and replace it with the [default
Sharkey
one](https://activitypub.software/TransFem-org/Sharkey/-/raw/stable/.config/docker_example.yml)
Edit the config inline with your instance settings. Make sure to use
the same `db` & `redis` settings as in your Firefish config.
Now is the time to *backup* your database and Redis volumes!
Firefish's docker-compose uses PostgreSQL version 12. We strongly
recommend upgrading to 15 or 16. You can do this by making a new db
volume, starting the newer PostgreSQL on it, and importing the data
from the backup you just made. Refer to [the PostgreSQL
documentation](https://www.postgresql.org/docs/current/backup-dump.html).
Make sure to update the *mount paths* for volumes of the Sharkey
container from `/firefish/` to `/sharkey/` (so your existing volumes
will show up at the new path, inside the container).
Now start *only* the database with `docker compose up -d db` (instead
of `db`, you may need to use whatever name is set for the service in
your docker compose config), and start a `psql` shell with `docker
exec -it db psql -U firefish -d firefish` (replace `db` as before, and
`-U firefish -d firefish` with the database user and database name,
respectively, if they're different from `firefish`).
You should now be connected to your Firefish database. We need to
massage it into shape so that Sharkey database migrations will
work. The following series of SQL queries / commands should do it, but
please read the comments and pay attention to the results after each
query!
```sql
-- start a transaction, so we won't leave the db in a halfway state if
-- things go wrong
BEGIN;
-- undo Firefish removing "_enum" from enum names
ALTER TYPE "antenna_src" RENAME TO "antenna_src_enum";
ALTER TYPE "drive_file_usage_hint" RENAME TO "drive_file_usage_hint_enum";
ALTER TYPE "muted_note_reason" RENAME TO "muted_note_reason_enum";
ALTER TYPE "note_visibility" RENAME TO "note_visibility_enum";
ALTER TYPE "notification_type" RENAME TO "notification_type_enum";
ALTER TYPE "page_visibility" RENAME TO "page_visibility_enum";
ALTER TYPE "poll_note_visibility" RENAME TO "poll_notevisibility_enum";
ALTER TYPE "relay_status" RENAME TO "relay_status_enum";
ALTER TYPE "user_emoji_mod_perm" RENAME TO "user_emojimodperm_enum";
ALTER TYPE "user_profile_ffvisibility" RENAME TO "user_profile_ffvisibility_enum";
ALTER TYPE "user_profile_muting_notification_types" RENAME TO "user_profile_mutingnotificationtypes_enum";
-- fix meta column names that Firefish changed
ALTER TABLE "meta" RENAME COLUMN "tosUrl" TO "ToSUrl";
ALTER TABLE "meta" RENAME COLUMN "objectStorageUseSsl" TO "objectStorageUseSSL";
ALTER TABLE "meta" RENAME COLUMN "customMotd" TO "customMOTD";
-- we need to add back some indexes that Firefish removed, but that Sharkey
-- expects
CREATE INDEX "IDX_01f4581f114e0ebd2bbb876f0b" ON "note_reaction" ("createdAt");
CREATE INDEX "IDX_0610ebcfcfb4a18441a9bcdab2" ON "poll" ("userId");
CREATE INDEX "IDX_25dfc71b0369b003a4cd434d0b" ON "note" ("attachedFileTypes");
CREATE INDEX "IDX_2710a55f826ee236ea1a62698f" ON "hashtag" ("mentionedUsersCount");
CREATE INDEX "IDX_4c02d38a976c3ae132228c6fce" ON "hashtag" ("mentionedRemoteUsersCount");
CREATE INDEX "IDX_51c063b6a133a9cb87145450f5" ON "note" ("fileIds");
CREATE INDEX "IDX_54ebcb6d27222913b908d56fd8" ON "note" ("mentions");
CREATE INDEX "IDX_7fa20a12319c7f6dc3aed98c0a" ON "poll" ("userHost");
CREATE INDEX "IDX_88937d94d7443d9a99a76fa5c0" ON "note" ("tags");
CREATE INDEX "IDX_b11a5e627c41d4dc3170f1d370" ON "notification" ("createdAt");
CREATE INDEX "IDX_c8dfad3b72196dd1d6b5db168a" ON "drive_file" ("createdAt");
CREATE INDEX "IDX_d57f9030cd3af7f63ffb1c267c" ON "hashtag" ("attachedUsersCount");
CREATE INDEX "IDX_e5848eac4940934e23dbc17581" ON "drive_file" ("uri");
CREATE INDEX "IDX_fa99d777623947a5b05f394cae" ON "user" ("tags");
-- we also need to add back some columns that Sharkey expects
ALTER TABLE "user_profile" ADD "integrations" JSONB NOT NULL DEFAULT '{}';
ALTER TABLE "meta" ADD "twitterConsumerSecret" VARCHAR(128);
ALTER TABLE "meta" ADD "twitterConsumerKey" VARCHAR(128);
ALTER TABLE "meta" ADD "enableTwitterIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "enableGithubIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "githubClientId" VARCHAR(128);
ALTER TABLE "meta" ADD "githubClientSecret" VARCHAR(128);
ALTER TABLE "meta" ADD "enableDiscordIntegration" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "meta" ADD "discordClientId" VARCHAR(128);
ALTER TABLE "meta" ADD "discordClientSecret" VARCHAR(128);
-- also an extra table, for the same reasons
CREATE TABLE antenna_note();
-- Misskey used to have a Reversi game, Firefish dropped the tables,
-- now Misskey uses them again
CREATE TABLE "reversi_game" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "startedAt" TIMESTAMP WITH TIME ZONE, "user1Id" character varying(32) NOT NULL, "user2Id" character varying(32) NOT NULL, "user1Accepted" boolean NOT NULL DEFAULT false, "user2Accepted" boolean NOT NULL DEFAULT false, "black" integer, "isStarted" boolean NOT NULL DEFAULT false, "isEnded" boolean NOT NULL DEFAULT false, "winnerId" character varying(32), "surrendered" character varying(32), "logs" jsonb NOT NULL DEFAULT '[]', "map" character varying(64) array NOT NULL, "bw" character varying(32) NOT NULL, "isLlotheo" boolean NOT NULL DEFAULT false, "canPutEverywhere" boolean NOT NULL DEFAULT false, "loopedBoard" boolean NOT NULL DEFAULT false, "form1" jsonb DEFAULT null, "form2" jsonb DEFAULT null, "crc32" character varying(32), CONSTRAINT "PK_76b30eeba71b1193ad7c5311c3f" PRIMARY KEY ("id"));
CREATE INDEX "IDX_b46ec40746efceac604142be1c" ON "reversi_game" ("createdAt");
CREATE TABLE "reversi_matching" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "parentId" character varying(32) NOT NULL, "childId" character varying(32) NOT NULL, CONSTRAINT "PK_880bd0afbab232f21c8b9d146cf" PRIMARY KEY ("id"));
CREATE INDEX "IDX_b604d92d6c7aec38627f6eaf16" ON "reversi_matching" ("createdAt");
CREATE INDEX "IDX_3b25402709dd9882048c2bbade" ON "reversi_matching" ("parentId");
CREATE INDEX "IDX_e247b23a3c9b45f89ec1299d06" ON "reversi_matching" ("childId");
-- move aside some FireFish columns; Sharkey migrations will
-- re-create them; we don't `DROP` them because we want to keep the data
ALTER TABLE "user" RENAME COLUMN "movedToUri" TO "ff_movedToUri";
ALTER TABLE "user" RENAME COLUMN "alsoKnownAs" TO "ff_alsoKnownAs";
ALTER TABLE "user" RENAME COLUMN "isIndexable" TO "ff_isIndexable";
ALTER TABLE "user" RENAME COLUMN "speakAsCat" TO "ff_speakAsCat";
ALTER TABLE "user_profile" RENAME COLUMN "preventAiLearning" TO "ff_preventAiLearning";
ALTER TABLE "meta" RENAME COLUMN "silencedHosts" TO "ff_silencedHosts";
-- this column was added by both Firefish and Misskey, but with
-- different names, let's fix it
ALTER TABLE "meta" RENAME COLUMN "ToSUrl" TO "termsOfServiceUrl";
-- update antenna types, this is only needed on some instances but
-- recommend to run anyway
--
-- this *removes* any antennas of types not supported by Sharkey!
CREATE TYPE public.new_antenna_src_enum AS ENUM ('home', 'all', 'list');
ALTER TABLE antenna ADD COLUMN new_src public.new_antenna_src_enum;
DELETE FROM antenna WHERE src NOT IN ('home', 'all', 'list');
ALTER TABLE antenna DROP COLUMN src;
ALTER TABLE antenna RENAME COLUMN new_src TO src;
DROP TYPE public.antenna_src_enum;
ALTER TYPE new_antenna_src_enum RENAME TO antenna_src_enum;
-- optional but recommended: delete all empty moderation log entries
DELETE FROM moderation_log WHERE info = '{}';
-- only needed on some instances, run this if
-- `\dT+ user_profile_mutingnotificationtypes_enum`
-- does not show `note` in the "elements" section
ALTER TYPE "public"."user_profile_mutingnotificationtypes_enum" ADD VALUE 'note';
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start Sharkey, and let it run all its migrations. Once that's done,
and Sharkey says it's listening, stop Sharkey but keep the database
running.
Open another `psql` shell like before (`docker exec -it db psql -U
firefish -d firefish`, replacing things as before). We need another
small pass of massaging.
```sql
BEGIN;
-- all existing users are approved, because Firefish doesn't have a
-- concept of approvals
UPDATE "user" SET approved = true;
-- now we put back the data we moved aside
UPDATE "user" SET "movedToUri" = "ff_movedToUri" WHERE "ff_movedToUri" IS NOT NULL;
UPDATE "user" SET "alsoKnownAs" = "ff_alsoKnownAs" WHERE "ff_alsoKnownAs" IS NOT NULL;
UPDATE "user" SET "noindex" = NOT (COALESCE("ff_isIndexable", true));
UPDATE "user" SET "speakAsCat" = COALESCE("ff_speakAsCat", false);
UPDATE "user_profile" SET "preventAiLearning" = COALESCE("ff_preventAiLearning", true);
UPDATE "meta" SET "silencedHosts" = COALESCE("ff_silencedHosts",'{}');
ALTER TABLE "user" DROP COLUMN "ff_movedToUri";
ALTER TABLE "user" DROP COLUMN "ff_alsoKnownAs";
ALTER TABLE "user" DROP COLUMN "ff_isIndexable";
ALTER TABLE "user" DROP COLUMN "ff_speakAsCat";
ALTER TABLE "user_profile" DROP COLUMN "ff_preventAiLearning";
ALTER TABLE "meta" DROP COLUMN "ff_silencedHosts";
-- next, add NSFW detection settings that Firefish removed
-- Sharkey doesn't use these right now, but it expects them to be here
CREATE TYPE "public"."meta_sensitivemediadetectionsensitivity_enum" AS ENUM('medium', 'low', 'high', 'veryLow', 'veryHigh');
CREATE TYPE "public"."meta_sensitivemediadetection_enum" AS ENUM('none', 'all', 'local', 'remote');
ALTER TABLE "user_profile" ADD "autoSensitive" BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE "meta" ADD "enableSensitiveMediaDetectionForVideos" BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE "meta" ADD "setSensitiveFlagAutomatically" BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE "meta" ADD "sensitiveMediaDetectionSensitivity" "public"."meta_sensitivemediadetectionsensitivity_enum" NOT NULL DEFAULT 'medium';
ALTER TABLE "meta" ADD "sensitiveMediaDetection" "public"."meta_sensitivemediadetection_enum" NOT NULL DEFAULT 'none';
ALTER TABLE "drive_file" ADD "maybePorn" BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE "drive_file" ADD "maybeSensitive" BOOLEAN NOT NULL DEFAULT FALSE;
COMMENT ON COLUMN "drive_file"."maybeSensitive" IS 'Whether the DriveFile is NSFW. (predict)';
-- recreate user profile columns that Firefish removed
ALTER TABLE "user_profile" ADD "room" JSONB NOT NULL DEFAULT '{}';
ALTER TABLE "user_profile" ADD "clientData" JSONB NOT NULL DEFAULT '{}';
COMMENT ON COLUMN "user_profile"."room" IS 'The room data of the User.';
COMMENT ON COLUMN "user_profile"."clientData" IS 'The client-specific data of the User.';
-- update incorrect user profile column types
ALTER TABLE user_profile ALTER "mutedWords" DROP DEFAULT;
ALTER TABLE user_profile ALTER "mutedWords" TYPE JSONB USING array_to_json("mutedWords")::JSONB;
ALTER TABLE user_profile ALTER "mutedWords" SET DEFAULT '[]'::JSONB;
ALTER TABLE user_profile ALTER "mutedInstances" DROP DEFAULT;
ALTER TABLE user_profile ALTER "mutedInstances" TYPE JSONB USING array_to_json("mutedInstances")::JSONB;
ALTER TABLE user_profile ALTER "mutedInstances" SET DEFAULT '[]'::JSONB;
-- clear the antenna table
-- antennas from this version of Firefish aren't compatible with Sharkey
TRUNCATE TABLE "antenna";
-- update incorrect antenna column types
ALTER TABLE antenna ALTER "keywords" DROP DEFAULT;
ALTER TABLE antenna ALTER "keywords" TYPE JSONB USING array_to_json("keywords")::jsonb;
ALTER TABLE antenna ALTER "keywords" SET DEFAULT '[]'::jsonb;
-- bring back chart columns removed by Firefish
CREATE TABLE public.__chart__ap_request (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "___deliverFailed" INTEGER DEFAULT 0 NOT NULL, "___deliverSucceeded" INTEGER DEFAULT 0 NOT NULL, "___inboxReceived" INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE public.__chart__drive (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "___local_incCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_incSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_decCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_decSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_incCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_incSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_decCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_decSize" INTEGER DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart__federation (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "unique_temp___deliveredInstances" CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, "___deliveredInstances" SMALLINT DEFAULT '0'::SMALLINT NOT NULL, "unique_temp___inboxInstances" CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, "___inboxInstances" SMALLINT DEFAULT '0'::SMALLINT NOT NULL, unique_temp___stalled CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, ___stalled SMALLINT DEFAULT '0'::SMALLINT NOT NULL, ___sub SMALLINT DEFAULT '0'::SMALLINT NOT NULL, ___pub SMALLINT DEFAULT '0'::SMALLINT NOT NULL, ___pubsub SMALLINT DEFAULT '0'::SMALLINT NOT NULL, "___subActive" SMALLINT DEFAULT '0'::SMALLINT NOT NULL, "___pubActive" SMALLINT DEFAULT '0'::SMALLINT NOT NULL);
CREATE TABLE public.__chart__hashtag (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___local_users INTEGER DEFAULT 0 NOT NULL, ___remote_users INTEGER DEFAULT 0 NOT NULL, unique_temp___local_users CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, unique_temp___remote_users CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL);
CREATE TABLE public.__chart__instance (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___requests_failed SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___requests_succeeded SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___requests_received SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___notes_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_inc INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_dec INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_diffs_normal INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_diffs_reply INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_diffs_renote INTEGER DEFAULT '0'::BIGINT NOT NULL, ___users_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___users_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___users_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___following_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___following_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___following_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___followers_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___followers_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___followers_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___drive_totalFiles" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_incFiles" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_incUsage" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_decFiles" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_decUsage" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___notes_diffs_withFile" INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE public.__chart__network (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "___incomingRequests" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___outgoingRequests" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___totalTime" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___incomingBytes" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___outgoingBytes" INTEGER DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart__notes (id SERIAL NOT NULL, DATE INTEGER NOT NULL, ___local_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_inc INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_dec INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_diffs_normal INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_diffs_reply INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_diffs_renote INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_inc INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_dec INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_diffs_normal INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_diffs_reply INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_diffs_renote INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_diffs_withFile" INTEGER DEFAULT 0 NOT NULL, "___remote_diffs_withFile" INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE public.__chart__per_user_drive (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, "___totalCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___totalSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___incCount" SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___incSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___decCount" SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___decSize" INTEGER DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart__per_user_following (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___local_followings_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_followings_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_followings_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_followers_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_followers_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_followers_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followings_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_followings_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followings_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followers_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_followers_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followers_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart__per_user_notes (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___DEC SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___diffs_normal SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___diffs_reply SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___diffs_renote SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___diffs_withFile" SMALLINT DEFAULT '0'::SMALLINT NOT NULL);
CREATE TABLE public.__chart__per_user_reaction (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___local_count SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_count SMALLINT DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart__test (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128), ___foo_total BIGINT NOT NULL, ___foo_inc BIGINT NOT NULL, ___foo_dec BIGINT NOT NULL);
CREATE TABLE public.__chart__test_grouped (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128), ___foo_total BIGINT NOT NULL, ___foo_inc BIGINT NOT NULL, ___foo_dec BIGINT NOT NULL);
CREATE TABLE public.__chart__test_unique (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128), ___foo CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL);
CREATE TABLE public.__chart__users (id SERIAL NOT NULL, DATE INTEGER NOT NULL, ___local_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart_day__ap_request (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "___deliverFailed" INTEGER DEFAULT 0 NOT NULL, "___deliverSucceeded" INTEGER DEFAULT 0 NOT NULL, "___inboxReceived" INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE public.__chart_day__drive (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "___local_incCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_incSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_decCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_decSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_incCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_incSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_decCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___remote_decSize" INTEGER DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart_day__federation (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "unique_temp___deliveredInstances" CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, "___deliveredInstances" SMALLINT DEFAULT '0'::SMALLINT NOT NULL, "unique_temp___inboxInstances" CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, "___inboxInstances" SMALLINT DEFAULT '0'::SMALLINT NOT NULL, unique_temp___stalled CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, ___stalled SMALLINT DEFAULT '0'::SMALLINT NOT NULL, ___sub SMALLINT DEFAULT '0'::SMALLINT NOT NULL, ___pub SMALLINT DEFAULT '0'::SMALLINT NOT NULL, ___pubsub SMALLINT DEFAULT '0'::SMALLINT NOT NULL, "___subActive" SMALLINT DEFAULT '0'::SMALLINT NOT NULL, "___pubActive" SMALLINT DEFAULT '0'::SMALLINT NOT NULL);
CREATE TABLE public.__chart_day__hashtag (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___local_users INTEGER DEFAULT 0 NOT NULL, ___remote_users INTEGER DEFAULT 0 NOT NULL, unique_temp___local_users CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL, unique_temp___remote_users CHARACTER VARYING[] DEFAULT '{}'::CHARACTER VARYING[] NOT NULL);
CREATE TABLE public.__chart_day__instance (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___requests_failed SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___requests_succeeded SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___requests_received SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___notes_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_inc INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_dec INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_diffs_normal INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_diffs_reply INTEGER DEFAULT '0'::BIGINT NOT NULL, ___notes_diffs_renote INTEGER DEFAULT '0'::BIGINT NOT NULL, ___users_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___users_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___users_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___following_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___following_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___following_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___followers_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___followers_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___followers_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___drive_totalFiles" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_incFiles" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_incUsage" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_decFiles" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___drive_decUsage" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___notes_diffs_withFile" INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE public.__chart_day__network (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "___incomingRequests" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___outgoingRequests" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___totalTime" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___incomingBytes" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___outgoingBytes" INTEGER DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart_day__notes (id SERIAL NOT NULL, DATE INTEGER NOT NULL, ___local_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_inc INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_dec INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_diffs_normal INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_diffs_reply INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_diffs_renote INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_inc INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_dec INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_diffs_normal INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_diffs_reply INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_diffs_renote INTEGER DEFAULT '0'::BIGINT NOT NULL, "___local_diffs_withFile" INTEGER DEFAULT 0 NOT NULL, "___remote_diffs_withFile" INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE public.__chart_day__per_user_drive (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, "___totalCount" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___totalSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___incCount" SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___incSize" INTEGER DEFAULT '0'::BIGINT NOT NULL, "___decCount" SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___decSize" INTEGER DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart_day__per_user_following (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___local_followings_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_followings_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_followings_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_followers_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_followers_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_followers_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followings_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_followings_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followings_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followers_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_followers_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_followers_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart_day__per_user_notes (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___DEC SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___diffs_normal SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___diffs_reply SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___diffs_renote SMALLINT DEFAULT '0'::BIGINT NOT NULL, "___diffs_withFile" SMALLINT DEFAULT '0'::SMALLINT NOT NULL);
CREATE TABLE public.__chart_day__per_user_reaction (id SERIAL NOT NULL, DATE INTEGER NOT NULL, "GROUP" CHARACTER VARYING(128) NOT NULL, ___local_count SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_count SMALLINT DEFAULT '0'::BIGINT NOT NULL);
CREATE TABLE public.__chart_day__users (id SERIAL NOT NULL, DATE INTEGER NOT NULL, ___local_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___local_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___local_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_total INTEGER DEFAULT '0'::BIGINT NOT NULL, ___remote_inc SMALLINT DEFAULT '0'::BIGINT NOT NULL, ___remote_dec SMALLINT DEFAULT '0'::BIGINT NOT NULL);
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start everything up again, you should see no errors in the logs.
Log in as an administrator, and go to the control panel.
If you use an object store such as S3, double-check your settings
(it's possible, for example, that the URL now looks like
`https://https://yourdomain.com`, fix it). If you want your users to
be able to search notes, you must enable it via the "roles" system.
Congratulations, you're now running Sharkey!
content/en/docs/install/migrate/iceshrimp.md 0 → 100644
View file @ 5343b59c
---
title: "Migrating from Iceshrimp"
weight: 1240
toc: false
---
This guide was only tested on an Iceshrimp-JS instance, on version v2023.12.11. Iceshrimp.NET is not supported,
and as far as known, the migration from Iceshrimp-JS to Iceshrimp.NET is a one way street.
Furthermore, this was tested on a **vanilla** Iceshrimp-JS instance. Other Iceshrimp instances
that may run with custom patches may require a different treatment.
If any of the steps fails, especially SQL queries, seek help from us
[Via Discord](https://discord.gg/8hF6pMVWja). Do *not* try fixing things
in the DB yourself unless you *really* know what you are doing.
Before you begin, please take a backup of your database. While you're
there, make sure you know how to restore from that backup! Using the
`plain` format for `pg_dump` is probably the simplest way.
It's a good idea to have a separate backup of your list of silenced
instances and of your `user` table.
### Using Docker
Stop / shut down the entire stack: PostgreSQL, Redis / Valkey and
Iceshrimp itself. Stop all of it.
Now is the time to *backup* your database and Redis volumes!
You will need to revert Iceshrimp migrations back to the last common one with
Firefish, which should leave you on Firefish 1.0.2 schema.
First, connect to the database container. Start *only* the database with `docker compose up -d db`
(instead of `db`, you may need to use whatever name is set for the service in
your docker compose config), and start a `psql` shell with `docker
exec -it db psql -U iceshrimp -d iceshrimp` (replace `db` as before, and
`-U iceshrimp -d iceshrimp` with the database user and database name,
respectively, if they're different from `iceshrimp`).
You should now be connected to your Iceshrimp database. We need first to delete from the
tables 3 of the Iceshrimp migrations that are not able to be reverted, regarding OAuth and
instance's max character limit. Don't worry, the OAuth is going to be dropped enterily by
one of the migration reverts, and the max character limit does not conflict with Sharkey, and
in fact, could be overwritten safely by a Sharkey migration.
Make sure you have a backup safely stored, and then delete the mentioned
migrations with the following queries:
```sql
DELETE FROM "migrations" WHERE "name" = 'IncreaseHostCharLimit1701118152149';
DELETE FROM "migrations" WHERE "name" = 'IncreaseOauthTokenRedirecturisLength1697286869039';
DELETE FROM "migrations" WHERE "name" = 'IncreaseOAuthRedirecturisLength1697246035867';
```
We are now ready to start reverting Iceshrimp migrations.
Edit the `docker-compose.yml` file and add `entrypoint: /bin/sh` to the web section of the file.
Save the file and start the container with `docker-compose up -d web`, and then run
`docker exec -it iceshrimp_web sh`.
Replace "iceshrimp_web" with the name of your container if you've changed it.
Now, navigate to the Iceshrimp directory, and inside of it, nagivate to `packages/backend`.
The command to run in order to revert the latest applies migration is
```bash
yarn run typeorm migration:revert -d built/ormconfig.js
```
You'll have to run it several times, until you see that `IceshrimpRepo1689965609061` has been reverted.
You're now back on the Firefish 1.0.2 database scheme.
Exit the shell and run `docker compose down web` to stop the Iceshrimp container.
Now, edit you docker compose and replace the Iceshrimp image with
`registry.activitypub.software/transfem-org/sharkey:latest`
Backup your Iceshrimp config and replace it with the [default
Sharkey one](https://activitypub.software/TransFem-org/Sharkey/-/raw/stable/.config/docker_example.yml)
Edit the config inline with your instance settings. Make sure to use
the same `db` & `redis` settings as in your Iceshrimp config.
Make sure to update the *mount paths* for volumes of the Sharkey
container from `/iceshrimp/` to `/sharkey/` (so your existing volumes
will show up at the new path, inside the container).
Now we need to massage the database into shape so that Sharkey database migrations will
work. The following series of SQL queries / commands should do it, but
please read the comments and pay attention to the results after each
query!
```sql
-- start a transaction, so we won't leave the db in a halfway state if
-- things go wrong
BEGIN;
-- move aside some existing columns; Sharkey migrations will
-- re-create them; we don't `DROP` them because we want to keep the data
ALTER TABLE "user" RENAME COLUMN "movedToUri" TO "ish_movedToUri";
ALTER TABLE "user" RENAME COLUMN "alsoKnownAs" TO "ish_alsoKnownAs";
ALTER TABLE "user" RENAME COLUMN "speakAsCat" TO "ish_speakAsCat";
ALTER TABLE "user_profile" RENAME COLUMN "preventAiLearning" TO "ish_preventAiLearning";
ALTER TABLE "meta" RENAME COLUMN "silencedHosts" TO "ish_silencedHosts";
-- this column was added by both Iceshrimp, and Misskey, but with
-- different names, let's fix it
ALTER TABLE "meta" RENAME COLUMN "ToSUrl" TO "termsOfServiceUrl";
-- update antenna types, this is only needed on some instances but
-- recommend to run anyway
--
-- this *removes* any antennas of types not supported by Sharkey!
CREATE TYPE public.new_antenna_src_enum AS ENUM ('home', 'all', 'list');
ALTER TABLE antenna ADD COLUMN new_src public.new_antenna_src_enum;
DELETE FROM antenna WHERE src NOT IN ('home', 'all', 'list');
ALTER TABLE antenna DROP COLUMN src;
ALTER TABLE antenna RENAME COLUMN new_src TO src;
DROP TYPE public.antenna_src_enum;
ALTER TYPE new_antenna_src_enum RENAME TO antenna_src_enum;
-- optional but recommended: delete all empty moderation log entries
DELETE FROM moderation_log WHERE info = '{}';
-- only needed on some instances, run this if
-- `\dT+ user_profile_mutingnotificationtypes_enum`
-- does not show `note` in the "elements" section
ALTER TYPE "public"."user_profile_mutingnotificationtypes_enum" ADD VALUE 'note';
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start Sharkey, and let it run all its migrations. Once that's done,
and Starkey says it's listening, stop Sharkey but keep the database
running.
Open another `psql` shell like before (`docker exec -it db psql -U
firefish -d firefish`, replacing things as before). We need another
small pass of massaging.
```sql
BEGIN;
-- all existing users are approved, because Firefish doesn't have a
-- concept of approvals
UPDATE "user" SET approved = true;
-- now we put back the data we moved aside
UPDATE "user" SET "movedToUri" = "ish_movedToUri" WHERE "ish_movedToUri" IS NOT NULL;
UPDATE "user" SET "alsoKnownAs" = "ish_alsoKnownAs" WHERE "ish_alsoKnownAs" IS NOT NULL;
UPDATE "user" SET "speakAsCat" = COALESCE("ish_speakAsCat", false);
UPDATE "user_profile" SET "preventAiLearning" = COALESCE("ish_preventAiLearning", true);
UPDATE "meta" SET "silencedHosts" = COALESCE("ish_silencedHosts",'{}');
ALTER TABLE "user" DROP COLUMN "ish_movedToUri";
ALTER TABLE "user" DROP COLUMN "ish_alsoKnownAs";
ALTER TABLE "user" DROP COLUMN "ish_speakAsCat";
ALTER TABLE "user_profile" DROP COLUMN "ish_preventAiLearning";
ALTER TABLE "meta" DROP COLUMN "ish_silencedHosts";
```
If everything worked and you saw no errors, you can run `COMMIT;` in
that same `psql` shell, to commit all the changes, then close that
shell. Again, if anything went wrong, come talk to us on
[Discord](https://discord.gg/8hF6pMVWja)!
Start everything up again, you should see no errors in the logs.
Log in as an administrator, and go to the control panel.
If you use an object store such as S3, double-check your settings
(it's possible, for example, that the URL now looks like
`https://https://yourdomain.com`, fix it). If you want your users to
be able to search notes, you must enable it via the "roles" system.
Congratulations, you're now running Sharkey!
content/en/docs/install/migrate/misskey.md 0 → 100644
View file @ 5343b59c
---
title: "Migrating from Misskey"
weight: 1210
toc: false
---
Let's say you have a working Misskey, running as user `misskey` from
`/home/misskey/misskey`.
Migrating to Sharkey is the same as updating to a newer Misskey
version:
```shell
sudo -u misskey -i
cd misskey
git remote rename origin misskey
git remote add origin https://activitypub.software/TransFem-org/Sharkey.git
git remote update -p
git checkout -b stable --track origin/stable
git pull --recurse-submodules
pnpm install --frozen-lockfile
pnpm run build
pnpm run migrate
```
Then you can restart your service.
If you see weirdness like service not starting, or missing labels in
the web UI, you should first make sure the build worked (check all the
error messages!), then try building again from scratch:
```shell
pnpm run clean-all
pnpm install --frozen-lockfile
pnpm run build
pnpm run migrate
```
Also, clear your browser's cache and local storage (this will log your
browser out of Misskey/Sharkey).
**NOTE**: Sharkey 2025.2.2 and later needs some system libraries and
their development headers: pango, cairo, pixman (on Debian-style
systems, that's `libpango1.0-dev libcairo2-dev libpixman-1-dev`)
### Using Docker
If you are using the `docker-compose.yml` file from Misskey which builds locally follow these steps:
```shell
git remote rename origin misskey
git remote add origin https://activitypub.software/TransFem-org/Sharkey.git
git remote update -p
git checkout -b stable --track origin/stable
git pull --recurse-submodules
pnpm install --frozen-lockfile
sudo docker compose build
```
tools/get-new-changes
View file @ 5343b59c
......@@ -22,7 +22,7 @@
# script printed at the end, commit all the changes
declare -A last_commit_checked=(
["Sharkey"]=d14f0b140bbc3174a5e187af36e25e5198125fd0
["Sharkey"]=93ffd4611c6fb3a9b3deb3180b3f88b5d855323d
["sfm-js"]=eb43eb0298c04b3133e240cc235efce96213666f
)
......