Mastodon has supported using a dedicated namespace for keys in Redis for a long time. The idea was that several applications (or instances of Mastodon) could share a single Redis database this way.

We soft-deprecated this option in Mastodon 4.3, mainly because newer versions of sidekiq no longer support it.

Keeping this option means we cannot update sidekiq which has since released two new major versions. Even worse, due to sidekiq's version requirements for some transitive dependencies, this holds back updates of other dependencies as well.

This reduces Mastodon's security, keeps us from adding some "free" features gained by updating dependencies and holds back future feature development in Mastodon.

While we know this is very unfortunate for server admins that use this option, given the reasons above we do not think we have a choice other than removing support for Redis namespaces from Mastodon.

Sadly, a fully automated migration as part of a Mastodon update is not possible. This is mainly due to different scenarios needing different approaches.

After having discussed this with some server administrators, we have identified two main scenarios:

1. Dedicated Redis instances that use namespaces, even though they do not strictly need to. Sometimes this in an artifact of a formerly shared installation, or just because it was there as an option. In this case it is sufficient to just strip the namespace from all the keys in Redis.
2. A shared Redis instance used by more than one Mastodon installation. In this case you will have to use a dedicated Redis instance per Mastodon installation, which means moving keys to another Redis instance and then renaming them.

This document is meant to help server administrators to perform a migration in one of the two scenarios mentioned above. It should also give you an idea what to do if your setup does not match one of those perfectly. But feel free to reach out if you need help and/or think this document is missing something.

If you use a REDIS_NAMESPACE you cannot migrate away from that without some downtime. The exact downtime depends on the number of keys you have in Redis and where you migrate to. We tested a very simple case with 500,000 tiny key/value pairs which took a couple of seconds, but this may vary a lot on real-world production data, so you should probably test this on your infrastructure with a dump of the Redis database.

The process is always roughly the same:

1. Stop all Mastodon services (web, sidekiq, streaming).
2. Migrate Redis keys to not use namespaces.
3. Update Mastodon config (no more namespaces, possibly different Redis location)
4. Restart Mastodon services

The difference lies in where to migrate Redis keys to.

Redis namespaces were introduced to share a single Redis database with other applications. In the most simple case, you do not (or no longer) need this. This would apply if

• You only ever used this Redis database for a single Mastodon instance to begin with, or
• You used to share the Redis database with other software or other instances of Mastodon, but no longer do so, or
• You share the Redis database with other applications but can migrate those away to use something else before starting the Mastodon migration.

In this case you only need to rename the keys in Redis. An example script to do so is rename.rb in this repository.

You can copy it into your Mastodon code directory (e.g. /home/mastodon/live), make sure your Mastodon services are stopped and run it like this:

RAILS_ENV=production bin/rails runner rename.rb

After this, edit your .env.production and depending on your current setup either:

1. Rename REDIS_NAMESPACE to ES_PREFIX if (and only if) you are using ElasticSearch and did not have ES_PREFIX configured. (In this case REDIS_NAMESPACE was used as the default for prefixes in ElasticSearch.) OR
2. Remove REDIS_NAMESPACE altogether if not using ElasticSearch or if you already have a ES_PREFIX variable set.

Save your configuration and restart all Mastodon services.

Redis makes it relatively easy to start different instances even on the same server. Just use a different port for each instance and you should be good. The memory overhead for running another instance of Redis is very small.

So if you currently share a single Redis instance between different installations of Mastodon, spinning up a separate, dedicated instance for each Mastodon installation is a good solution.

In this case you need to move keys from one Redis instance to another, then rename them. An example script to do so is move.rb in this repository.

You can copy it into your Mastodon code directory (e.g. /home/mastodon/live), make sure your Mastodon services are stopped and run it like this:

NEW_REDIS_URL=<Redis URL> RAILS_ENV=production bin/rails runner move.rb

Make sure to set NEW_REDIS_URL to an Redis connection URL pointing to your new, dedicated Redis instance. Also make sure your old Redis instance can actually reach this URL.

After this, update your env.production file to point to the new, dedicated Redis instance and depending on your current setup, either:

1. Rename REDIS_NAMESPACE to ES_PREFIX if (and only if) you are using ElasticSearch and did not have ES_PREFIX configured. (In this case REDIS_NAMESPACE was used as the default for prefixes in ElasticSearch.) OR
2. Remove REDIS_NAMESPACE altogether if not using ElasticSearch or if you already have a ES_PREFIX variable set.

Save your configuration and restart all Mastodon services.

Redis databases are a built-in way to isolate different key namespaces from each other. It is one of the options mentioned in the Sidekiq article linked to above.

While this looks good as a way to share a single Redis instance between different Mastodon installations on the surface, there are a couple of problems with it.

The most obvious one is that while databases offer real isolation of key namespaces, channels used for Publish/Subscribe still share a single, global namespace. Mastodon uses Publish/Subscribe for its streaming server, so databases will not work here.

We thought about providing different workarounds for this, but then were made aware of this comment here by a Redis developer saying that the Redis core team does not recommend this setup.

With that in mind we decided to not build a half-baked solution based on a mechanism that is not recommended by upstream developers.

Databases may still become an alternative in the future, though: The Valkey team is currently contemplating offering full isolation between databases, including pub/sub. See this issue here for details.