mostlyobvious

Migrating to Your Own Fediverse Instance

17 min read

See Twitter Migration for more information on Twitter data export etc, other networks, etc.

Why run your own instance

Choosing an ActivityPub instance is similar to choosing ISP or Email Provider, except for since most servers are run by small independent operators as informal coops, it’s maybe a bit more akin to old BBS’s.

Currently (2022-11), there aren’t the best discovery tools (official Mastodon list, Fediverse Observer) for which instance to choose - I am assuming that most people are picking either semi-randomly based on what their client recommends when onboarding, or via what some people they like/follow are choosing (the Twitter export tool Debirdify even makes a nice pie chart of instance distribution of followers/following).

I had an old Mastodon.social account but when I signed back up on Mastodon (more broadly, the Fediverse), I picked Fosstodon as my instance since the idea of joining a community focused around FOSS appealed to me, and running your own Mastodon instance seemed like a huge pain (and most managed instance providers like Spacebear and Mastohost were overbooked already).

As I dug in more though, I decided it probably was worth it to run my own. The good thing is that migrating instances is relatively easy, and Mastodon includes an export and redirection service, even allowing for migration of followers, so friction is lowered.

Some of my reasons for running your own instance:

  • Sustainability: With the Twitter exodus, most instances are experiencing huge growth, which on the one hand is great, but on the other hand, is not so great if the instance is dependent on a few dedicated volunteers not getting burnt out and on donations to pay for infra costs especially as costs skyrocket. Long-term sustainability is possible, and I’ve seen some tools and best practices pointing towards how to make running instances/communities sustainable, but, it’s a consideration…
  • Moderation, filtering, and federation: this probably needs a lot more unpacking, but at the simplest level, your instance admins run your server and they are responsible for user/content moderation and also which other servers they talk to. I’m not going to make any grand statements on Postel’s Law, credible neutrality, or anything like that there, just that personally, I’d like to be in control of who I can communicate with and how I behave, and that’s much easier when self-hosting.
  • Owning your own data: Yes, most instances allow you to export your data, but I’ve been on a self-hosted (r/selfhosted) kick for a while. This maybe ties in a bit with “because you can” as a reason, but also from a network health perspective, decentralizing is key. Another concept from the cryptoeconomic world is the Nakamoto coefficient, and I think it’s a useful lens when thinking about what it means when decentralizing networks.
  • Improving your experience: There is a Mastodon Client API, but (often by design), Mastodon is missing a lot of QoL features. Almost all alternative Fediverse software gives you useful features that you’ll never get from Mastodon.
  • Hacking and improving: if you’re a developer, it makes a lot of sense to run your own instance and work to improve things. As the ActivityPub protocol is a standard, APIs are open, and almost all servers and clients are open source, you won’t get the rugged pulled out from under you like with commercial APIs, so it’s actually worth the investment (or at least, worth poking around)

Some other people’s opinions:

Why I Chose Akkoma

Why not run Mastodon?

I wasn’t super excited about the idea of maintaining a large/bloated stack for Mastodon even with the help of docker-compose, but I was aware from previous conversations that there were lighterweight, mostly compatible alternatives available.

(Note: there are a few other Mastodon forks, notably Hometown and glitch-soc that make some tweaks/add some functionality, but they don’t really differ in resource consumption or setup complexity)

A good summary of a number of reasons not to run Mastodon are brought up in this 2021 piece by Jake Bauer and I’d recommend giving it a read if you’re interested:

The biggest consideration for a single user instance is probably resource consumption.

  • Switching from Mastodon to Plemora lowered memory usage from 2.5GiB to 700MiB for this user in 2020
  • More recently (Nov 24 2022) a user migrated from Mastodon to Plemora and went from using >2.5GiB of memory down to 400MiB
  • Large instances aren’t so relevant to me, but in August 2020, migrating a 10K+ user instance from Mastodon to Pleroma cut server costs by over half.
  • Akkoma claims to be another 30-50% faster than Plemora when handling common operations on their site.

What other options are there?

One thing to be aware of, is that Mastodon, while being far and away the highest profile and most popular ActivityPub service, is only a subset of a newtork more broadly termed the Fediverse.

Considerations

The most popular alternatives to Mastodon at the moment are Plemora-likes and Misskey-likes. There are some neat projects like microblog.pub and honk however they may be a bit barebones for those looking for a richer experience. GoToSocial is one last project worth a mention. It’s still under very active development, but it is ultra-lightweight while notably, supporting the Mastodon client API.

This last bit was probably the most important thing for me. If you’re using Mastodon now and want to continue using your favorite clients, then you will need to run something with Mastodon client API support. While I was pretty interested in running something like Misskey (Calckey in particular seems cool), client compatibility was on my mind. Pleroma was designed to be largely Mastodon compatible and as a result has a huge list of compatible clients.

One other thing that put Misskey-likes out of the running is that account move support is still in development, whereas for Plemora (well, Akkoma for me), it basically “just works.”

There are 3 active Plemora-like projects, the original project Plemora (2019; repo) and two forks, Akkoma (2022; repo) and Rebased (nee Soapbox BE) (2021; rep ).

Plemora is obviously the most established project - it has the longest history, the most servers running it, and I would have gone with that, but one of the things I wanted was webfinger expansion, and while is now supported in the development branch (and in the docs) it’s actually not supported yet. Sadly, while development is pretty active, the release schedule is glacial. I’m not an Elixir guy and was hoping to at least be able to get something working out of the gate w/o having to futz w/ branches or patches, so I took a look at the forks…

First, there’s not a lot out there comparing the two. I found two Reddit threads (almost the same post, but the comments are different and good for color).

Based on the discussion, it seems like Rebased would be more polished/a better option, the main dev is quite active, and also runs a commercial Rebased hosting service.

Akkoma only forked from Plemora half a year ago with this announcement: Akkoma: A vision to refocus Pleroma. Since then it’s also been quite active and it looks like it’s settled onto a monthly release cadence. (Here’s a just published 6 month retrospective.)

My original plan was to try them both out, with the expectation that one might not work well anyway, and the main reason I decided to check Akkoma first is that it has proper setup Docker documentation, whereas Rebased does not and that overall Akkoma seemed closer to Plemora (the docs are almost the same).

Setting up Akkoma took a little longer than expected (DNS propagation, Nginx/Caddy hijinx, search setup hiccups), so with everything working, I decided I’d just be settled. Since then, I’ve also discovered why Pleroma had a big schism. This may or may not be relevant to your platform choice, but I figure I’d post it here, since it’s not obvious to anyone who wasn’t a Plemora dev before this year, apparently.

  • The Akkoma announcement is pretty circumspect about it:
    • “I shan’t elaborate too much on that schism here, but the catalyst was one developer who both aligned with the “free speech” group and refused to treat other developers with any sort of respect (whilst being a generally unpleasant person to boot)”
  • This similarly circumspect thread doesn’t really illuminate much except that perhaps the (erstwhile?) founder of Plemora hasn’t done a great job managing things…
  • Oh, and then found a reference to the “one developer” via this post recounting some Fediverse history/ruminations and it started coming together.
    • “Around the same time, the Soapbox project was launched, and they decided to use Pleroma as their backend. This led to a lot of friction inside the project, because the Soapbox author had a tendency to share his ideological positions inside the project space as part of his anti-trans activism. I wound up leaving Pleroma toward the middle of 2020 because of the scalability issues in the database with Pleroma 2.0 and the lack of any effort to maintain a welcoming space for everyone.”
  • Well, that’s a bummer. Here’s a summary from another ex-Maintainer
  • To be fair, here’s Alex Gleason’s side of things where he claims mistreatment from others for his (unpopular) TERF views

Anyway, all of that is a bit of a mess and bummer but beyond code smell, I think keyman questions and vibe checks are worth asking for all of these smaller projects.

While I’m planning on sticking with Akkoma for now, I’m also keeping an eye on GoToSocial and some other lightweight alternatives, although migration would likely be… extremely unpleasant.

Setting up an Akkoma Instance

Server

The first thing you’ll want to do is to get an Linux server. For now, I’m using a t3a.small and we’ll see where it goes from there.

If you’re using Ubuntu LTS, first get Docker and Docker Compose up and running:

Make sure you have a firewall and only allow :80 and :443 open. If you want to enable HTTP/3 you will need to make sure you specify 443:443/udp in Docker.

Basic Setup

The Akkoma Installing in Docker docs are pretty good and for the most part work.

Since I’m using Docker Compose V2, I needed to modify docker-resources/build.sh and docker-resources/manage.sh to use docker compose and not docker-compose.

One issue I ran into is that the POSTGRES_PASSWORD credentials didn’t match up. I ended up passing in the default akkoma password in the docker-compose.yml and config/prod.secret.exs and that seemed to make things work. Just make sure your Postgres ports aren’t exposed to the public.

If you’re using this as a single user instance, you’ll want to set registrations_open as false in your config/prod.secret.exs as well.

I used the included commands to install plemora-fe and admin-fe.

NOTE: on weaker instances, it takes a a minute or so for Akkoma to run as it needs to compile on startup. I’d recommend running with docker compose up (not in daemon mode) while you’re setting up so you can easily see what it’s up to.

For those looking for an alternative setup, I spotted this repo with a “simpler” Docker setup (does a few things more automatically, but less documented and may be harder to customize). I haven’t used it, just putting it out there for those interested: https://github.com/chrisbanes/docker-akkoma

For those looking for another step-by-step writeup on setting Akkoma up, you can check out this Akkoma install overview.

Webfinger

After some thought on using a new domain, I decided that I’d just run it on my 25yo domain that I’ve been using forever, because well, why not… (spam can’t get worse, can it?) Since I’m running the instance on a subdomain at https://fediverse.randomfoo.net/ but would like my ID to be from the main domain, I used this Akkoma doc on How to use a different domain name for Akkoma and the users it serves]

First, I added this to my .config/prod.secret.exs:

config :pleroma, Pleroma.Web.WebFinger, 
   domain: "randomfoo.net"

Basically you’ll want to make sure that your webfinger query returns the proper acct when queried for:

curl 'https://fediverse.randomfoo.net/.well-known/webfinger?resource=desired-user@example.com'

Next, it’s just a matter of adding the redirect from the main domain to the subdomain. The docs include the Nginx stanza. If you’re using Caddy on your main site, you’d use the redir directive.

Transferring Your Account

Once everything is set up, the next thing you’ll want to do is move your old account over. You can basically transfer followers (and import exported following) over, but not old posts (you can export and probably could figure out a way to import old posts, favorites, etc but I haven’t looked into that yet).

This is a bi-directional transfer so requires you to do some stuff on your new Akkoma instance. In the plemora-fe UI lets you should add your old account address as an account alias. That’s located in Settings > Security > Account aliases.

Then you follow the Mastodon docs (use the Settings > Account > Account migration) to set a redirect and move your followers over.

There’s a cooldown period for moving, so make sure you have everything all setup properly first before the migration.

Search isn’t built into Akkoma by default, but can be added. Akkoma has support for Meilisearch which is easier to setup than Elasticsearch (and it turns out, much lighter weight).

First I created a data.ms folder and then I added the following to my docker-compose.yml:

  # https://github.com/ezpaarse-project/meilisearch/blob/master/docker-compose.yml
  meilisearch:
    image: getmeili/meilisearch
    restart: unless-stopped
    environment:
      - http_proxy
      - https_proxy
      - MEILI_MASTER_KEY=${MEILI_MASTER_KEY:-masterKey}
      - MEILI_NO_ANALYTICS=${MEILI_NO_ANALYTICS:-true}
      - MEILI_ENV=${MEILI_ENV:-development}
      - MEILI_LOG_LEVEL
      - MEILI_DB_PATH=${MEILI_DB_PATH:-/data.ms}
    ports:
      - ${MEILI_PORT:-7700}:7700
    volumes:
      - ./data.ms:/data.ms

Then I added the following to the config/prod.secret.exs:

config :pleroma, Pleroma.Search.Meilisearch,
  url: "http://meilisearch:7700/",
  private_key: "masterKey",
  initial_indexing_chunk_size: 100_000

After restarting the containers, what I did was to docker exec -it [akko-container] /bin/sh and run:

mix pleroma.search.meilisearch index
mix pleroma.search.meilisearch stats

To do the initial run and make sure it was working.

Privacy

Apparently by default, there are a couple leaky APIs that probably should be disabled. Akkoma only has one of them. I have it blocked off in my Caddyfile

  @blocked {
    path /api/v1/instance/peers
  }
  respond @blocked 403

Resource Consumption

Here’s what docker stats looks like after about 5 days of running (about 100 followers and following 100, 135K statuses in the DB):

NAME                   CPU %     MEM USAGE / LIMIT     MEM %     NET I/O
akkoma-meilisearch-1   0.03%     3.422MiB / 1.898GiB   0.18%     8.95kB / 0B
akkoma-proxy-1         0.87%     29.47MiB / 1.898GiB   1.52%     5.02GB / 2.36GB
akkoma-akkoma-1        31.29%    506.3MiB / 1.898GiB   26.05%    15.2GB / 15.1GB
akkoma-db-1            17.47%    363.8MiB / 1.898GiB   18.72%    13.2GB / 15.8GB

Upgrading

Using: https://docs.akkoma.dev/stable/installation/docker_en/#updating-docker-installs

Testing upgrade w/ 2022-12 release: https://meta.akkoma.dev/t/akkoma-stable-2022-12-automatic-static/225

git pull
./docker-resources/build.sh
./docker-resources/manage.sh mix deps.get
./docker-resources/manage.sh mix compile
./docker-resources/manage.sh mix ecto.migrate
# Use space for Docker Compose 2.0
docker compose restart akkoma db

While you’re doing restart, you can follow the logs to make sure things are working:

docker compose logs akkoma --tail 100 -f

2023-09-23

  • Ran out of space on drive
  • Cleared out root
cd /srv/fediverse/akkoma
docker compose restart akkoma db

Works, now lets try to upgrade

First we need to checkout our scripts…

Updating 9d7c877de..ebfb617b2
error: Your local changes to the following files would be overwritten by merge:
	docker-resources/build.sh
	docker-resources/manage.sh
Please commit your changes or stash them before you merge.
Aborting
root@snowball7 /s/f/akkoma (stable) [1]# df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            13G  2.0M   13G   1% /run
/dev/md2        2.0T  1.9T   11G 100% /
tmpfs            63G   96K   63G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/md1        989M  255M  683M  28% /boot
/dev/md3        1.5T  1.2T  240G  84% /home
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/9afddd686e450852523434bd349e748905b8351d31ff015746fd4da6fb5b72cc/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/3c525976ddb643521ae43512341c2c8d42e4e4f660e5e774b7a3c1474b61b012/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/a7cba50e1ab81c196381067eea0c1d2396fe36f07f122018912aebcb81287a5f/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/fdb382f17ee721fff897bf9175af7021351b0cf950ba7f39c0ecdbc1daadd19b/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/4e4c758cffdd8f3f0f12f0cb534815bc8c53fc7465aec8d9159fd11a78b0a4b5/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/62420d854d961f42a89599510ac4fdeb6d2af95491da51cc6e70c55ec29b6c75/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/31ccf5d895797919878e20872475d6939ce7dcbf1f6c8ddc0cf3904f52ea4c18/merged
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/4638e2587fbe341670dfed1606fd0a43d9c9d3f525736258264ae1b8637c6d70/merged
tmpfs            13G     0   13G   0% /run/user/1000
overlay         2.0T  1.9T   11G 100% /var/lib/docker/overlay2/579f31bb34887315f3bad50dafcc28a293df860f3527756480f4e03f6dfc1644/merged
root@snowball7 /s/f/akkoma (stable)# git diff docker-resources/build.sh
diff --git a/docker-resources/build.sh b/docker-resources/build.sh
index daa653da6..ce4f30f8d 100755
--- a/docker-resources/build.sh
+++ b/docker-resources/build.sh
@@ -1,4 +1,4 @@
 #!/bin/sh
 
-docker-compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) akkoma
-docker-compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) db
+docker compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) akkoma
+docker compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) db
root@snowball7 /s/f/akkoma (stable)# git diff docker-resources/manage.sh 
diff --git a/docker-resources/manage.sh b/docker-resources/manage.sh
index 944f5e2e2..acb6618c3 100755
--- a/docker-resources/manage.sh
+++ b/docker-resources/manage.sh
@@ -1,3 +1,3 @@
 #!/bin/sh
 
-docker-compose run --rm akkoma $@
+docker compose run --rm akkoma $@
root@snowball7 /s/f/akkoma (stable)# git checkout docker-resources/
Updated 2 paths from the index
git pull

We set this in our ~/.config/fish/config.fish for the future:

alias docker-compose="docker compose"

Run this as regular user otherwise $GID will be no bueno

cd /srv/fediverse/akkoma
git pull
./docker-resources/build.sh
./docker-resources/manage.sh mix deps.get
./docker-resources/manage.sh mix compile
./docker-resources/manage.sh mix ecto.migrate
# Use space for Docker Compose 2.0
docker compose restart akkoma db

2023-03-20

2023-03 Update worked the same https://meta.akkoma.dev/t/akkoma-stable-2023-03-recently-drafted-purge-requests/394

And logs looked OK, but no front-end. Weird? Here’s the solution:

# Make sure we have frontends installed:
./docker-resources/manage.sh mix pleroma.frontend install pleroma-fe --ref stable
./docker-resources/manage.sh mix pleroma.frontend install admin-fe --ref stable

# Add this to config/prod.secret.exs
config :pleroma, :frontends,
  primary: %{
    "name" => "pleroma-fe",
    "ref" => "stable"
  },
  admin: %{
    "name" => "admin-fe",
    "ref" => "stable"
  },
  mastodon: %{
    "name" => "mastodon-fe",
    "ref" => "akkoma"
  }

2023-05-25

Update didn’t work b/c 8GB EC2 root partition ran out of space. Was planning on migrating anyway, so did a DB backup: https://docs.akkoma.dev/stable/administration/backup/ with pgdump and then transferred all of akkoma folder to new machine.

This was relatively straightforward, the big change here is switching to using an external caddy docker, which requires adding the akkoma_default network to that docker-compose.yml and doing a docker compose up -d (rebuild) for that to access the Akkoma network properly.

Clients

As mentioned, since Akkoma is largely compatible with the Mastodon Client API, most Mastodon clients should work. Personally, I’ve been using Whalebird and Sengi on the desktop and Fedilab and Husky on Android.

The one caveat is that currently, filtering of replies to users you don’t mutually follow is done on the client-side by plemora-fe, not in the timeline stream so the client views are a bit messy.

Also, Akkoma has deprecated some v1 APIs including the search that Sengi uses to pull up reply context. I jank-solved this by adding a rewrite to my Caddyfile…

  uri replace /v1/search /v2/search

Other

There’s on other comprehensive guides I’ve found that covers similar ground:

There are also some other resources that cover different aspects of setting up your own instance:

Server Migration

https://icyphox.sh/blog/mastodon-to-pleroma/ https://git.pleroma.social/pleroma/pleroma/-/issues/162 https://gitlab.com/soapbox-pub/migratord

Backlinks