Category: Ubuntu

In part 1 of this tutorial, I dived into my reasons for setting up a Matrix Synapse homeserver, and how to set up the basics of the server and it’s required software. In part 2, I’m going to register myself an admin account, log into the server using the online chat client, to verify it’s all working as it should, and migrate the database engine from SQLite to Postgres.

A quick reminder, all server commands are run as the root user.

Register an admin user account

Installing the Matrix Synapse server software also installs a few executables on the server, that can be used for specific tasks. register_new_matrix_user is one such executable, and it allows you to register a new user from the command line of the server.

register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml http://localhost:8008

The process will ask you for a user name, password, and whether to make this user an admin or not.

Once you’ve created the user, you can log in to the homeserver via the Element web client hosted at https://app.element.io/#/login, or download Element for your operating system, and Sign in from the client.

Either way, to sign in to your server, click on the Change link, next to “Sign in to your Matrix account on matrix.org”.

Enter the server domain as the homeserver URL and click Next.

Enter your user name and password and click Sign in.

Switch Database Engine

While Matrix Synapse shops with SQLite by default, the official documentation suggests this only for testing purposes or for servers with light workloads. Otherwise, it’s recommended to switch to Postgres, as it provides significant performance improvements due to the superior threading and caching model, and smarter query optimiser, and allows the database to be run on separate hardware.

Install Postgres

First step is to install Postgres on the server.

apt install postgresql postgresql-contrib

We then need to create a user for synapse to use to access the database. We do this by switching to the postgres system user, and running the createuser command

su - postgres
createuser --pwprompt synapse_user

The createuser command will ask you for a password, and create the synapse_user with that password.

Now we can create the database, by logging into the Postgres database server, while operating as the postgres system user.

psql

Once logged in, we can create the database, and assign it to the synapse_user

CREATE DATABASE synapse
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER synapse_user;

Then we need to allow the synapse_user to connect to the database. The Postgres docs for Synapse talk about possibly needing to enable password authentication, but I found that by default this was already endabled, so all I had to do was add the relevant file to the pg_hba.conf file. I wasn’t sure how to find the pg_hba.conf file on my system, but this Stack Overflow thread explained what commands I could use when logged into the Postgres database server.

show hba_file;

My pb_hba.conf file was located at /etc/postgresql/12/main/pg_hba.conf.

Towards the bottom of the file, I added the Synapse local connection section to allow the synapse user access to Postgres.

# Database administrative login by Unix domain socket
local   all             postgres                                peer

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     peer
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# Synapse local connection:
host    synapse         synapse_user    ::1/128                 md5
# IPv6 local connections:
host    all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     peer
host    replication     all             127.0.0.1/32            md5
host    replication     all             ::1/128                 md5

Because line order matters in pg_hba.conf, the Synapse Postgres docs make a point of the fact that the Synapse local connection needs to be just above the IPv6 local connections.

Migrating the SQLite data to the Postgres database

In order for the data to be migrated from the SQLite database to the PostgreSQL database, we need to use the synapse_port_db executable, which requires that the homeserver.yaml file includes the correct server_name. So edit the homeserver.yaml, and set the server_name to your domain name from part 1.

nano /etc/matrix-synapse/homeserver.yaml

server_name: "domain.com"

The next step is to make a copy of the homeserver.yaml file, in preparation for the Postgres set up

cp /etc/matrix-synapse/homeserver.yaml /etc/matrix-synapse/homeserver-postgres.yaml

Then, edit the new /etc/matrix-synapse/homeserver-postgres.yaml file, so that the database settings point to the newly created Postgres database.

#database:
#  name: sqlite3
#  args:
#    database: /var/lib/matrix-synapse/homeserver.db

database:
  name: psycopg2
  args:
    user: synapse_user
    password: dbpassword
    database: synapse
    host: localhost
    cp_min: 5
    cp_max: 10

Make sure that the newly created /etc/matrix-synapse/homeserver-postgres.yaml file is owned by the correct system user.

chown matrix-synapse:nogroup /etc/matrix-synapse/homeserver-postgres.yaml

Next step is to copy the SQLite database, so that we can import from the copy, and give that copy the correct permissions.

cp /var/lib/matrix-synapse/homeserver.db /var/lib/matrix-synapse/homeserver.db.snapshot
chown matrix-synapse:nogroup /var/lib/matrix-synapse/homeserver.db.snapshot

We should then stop the matrix-synapse server, before we run the import.

systemctl stop matrix-synapse

Now we can use the synapse_port_db command to import the data from SQLite to Postgres, using the SQLite snapshot, and the Postgres enabled homeserver-postgres.yaml.

synapse_port_db --curses --sqlite-database homeserver.db.snapshot --postgres-config /etc/matrix-synapse/homeserver-postgres.yaml

Once the import successfully completes, we can backup the current SQLite enabled configuration.

mv /etc/matrix-synapse/homeserver.yaml /etc/matrix-synapse/homeserver-old-sqlite.yaml

Finally, we set the Postgres enabled yaml as the default.

mv /etc/matrix-synapse/homeserver-postgres.yaml /etc/matrix-synapse/homeserver.yaml

At this stage, it’s a good idea to make sure that the new /etc/matrix-synapse/homeserver.yaml file has the right permissions set, and if not, set them correctly.

chown matrix-synapse:nogroup /etc/matrix-synapse/homeserver.yaml

And that’s it, log into your Matrix server via the web client or desktop client, and if you’ve completed the import correctly, everything should be working as before.

That concludes the main set up requirements to have a working Matrix homeserver.

I do however plan to have two follow-up posts. In the first, I’ll dive into some of the quirks of signing into and verifying additional clients, once your initial sign-in has been successful.

For the second, I’ll be adding some testers to the mix, and I’ll document and any configuration changes I’ve needed to make, to get things usable as a community communication platform. This one might take a bit longer, as it relies on other folks testing the platform, giving me feedback, and me tweaking the server until it’s a workable solution.

For about the past six months or so, I’ve been interested in the open, decentralized communications standard called Matrix. In May of this year, it was announced on TechCrunch that Automattic had invested $4.6 million into the company behind the Matrix standard. The company in question, New Vector (now rebranded as Element) also develop an open-source chat client, which is a rival to Slack, that runs on the Matrix standard. This is the part that made me sit up and take notice. Matt Mullenweg is the CEO of Automattic and the co-founder and BDFL for WordPress. His decision to invest in this open-source communications standard, which offers an alternative to Slack, could very well mean that in the near future, both the internal communication at Automattic and that of the WordPress project, could end up running on Matrix.

For some time now, myself and the other admins of the WP South Africa Slack community, have been considering switching away from Slack. The main reason for this is that Slack doesn’t offer an option for open source communities to get any premium plan features, so unless we’re prepared to pay $8 USD per person for our over 1000 members we’re stuck with the free version. It also means we don’t “own” our open source community’s communication channels. If Slack ever dropped the free plan, which could happen, we’d be stuck up the proverbial creek.

I decided it was time I took a look at what it would take to get a Matrix homeserver set up, what the pros and cons would be, and if it would be possible to migrate all our users over to our own homeserver.

Requirements

In order to set up a Matrix homeserver, I would need a web server instance to host it. Fortunately, I have a Digital Ocean account, so spinning up a new basic VPS droplet with Ubuntu 20.04, 1GB of RAM, 1 vCPU and 25GB of storage at $5 a month was a click of a few buttons. The other requirement was to have a public domain pointing to the IP address of the server. I asked Hugh, who manages the wpsouthafrica.org domain, to set up an A record in the DNS to point the subdomain matrix.wpsouthafrica.org to the new server IP address. You can also use a regular top-level domain (eg domain.com) but if you already have a domain (for example for your community website), using a subdomain for the Matrix server means not needing to purchase a new one.

Initial set up

A note on commands, I’m running all the commands for this server as the root user. If you have access to your web server via another user with sudo privileges, I suggest switching to the root user, it will just make everything easier.

sudo su

The first step with a new server is to make sure the server software is up to date.

apt update && apt upgrade

Then, I needed to ensure that any package dependencies for the matrix-synapse server software are installed.

apt install lsb-release wget apt-transport-https

While there are official matrix-synapse packages in the Ubuntu repositories, the matrix-synapse docs recommend using their own packages. To do that I first had to enable those packages, by adding the GPG key and the matrix-synapse repository for a Debian/Ubuntu-based system.

wget -qO /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/matrix-org.list

Once the repository is set up, I can get the latest package updates, which will now include the Matrix ones, and install the matrix-synapse homeserver software.

apt update
apt install matrix-synapse-py3

During the install, I enter the chosen domain (in my case our subdomain) as the server name. I can also choose not to send Anonymous Data Statistics, but that’s entirely up to you.

Once it’s all installed, I start the matrix-synapse server, and enable it to auto-start on system boot.

systemctl start matrix-synapse
systemctl enable matrix-synapse

Configure Matrix Synapse

Firstly, I needed to generate a random string which is used as the Matrix Synapse registration secret. This I did using the following command.

cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1

The server outputs the random key, which I copied and saved somewhere safe for later.

rgzzglba4KiJo5qIMcAUu3eEWsMtV8wu

No, this isn’t the key for our Matrix homeserver! 🙂

The next step is to edit the homeserver.yaml configuration file, which is in the /etc/matrix-synapse directory. For this tutorial, I’m using nano, but you can use whatever CLI text editor you prefer.

nano /etc/matrix-synapse/homeserver.yaml

I searched for and changed the registration_shared_secret entry, and used the randomly generated key created earlier. It’s important to note that the key should be enclosed by double-quotes.

registration_shared_secret: "YC9h8djIiCaCinWxkE2zt9cgvXYGX25P"

I then saved and closed the homeserver.yaml file.

Then, I restarted the matrix-synapse service, which will apply the new configuration.

systemctl restart matrix-synapse

Generate SSL

My next step was setting up the webserver software Nginx as a reverse proxy for the Matrix Synapse server. Before I could do that, I needed to generate an SSL certificate, to make sure the traffic to and from the server is secure. This can be accomplished using a LetsEncrypt certificate, for which I need to install certbot.

apt install certbot

Once certbot is installed, I generated the SSL certificate, using my email address, and the subdomain we have pointing to the IP address of the webserver.

certbot certonly --rsa-key-size 2048 --standalone --agree-tos --no-eff-email --email user@domain.com -d domain.com

Once this is completed, the SSL certificate and chain were saved at /etc/letsencrypt/live/domain.com/fullchain.pem and the SSL key file was been saved at /etc/letsencrypt/live/domain.com/privkey.pem. This information will become useful when I set up Nginx in the next step.

Setup Nginx as a Reverse Proxy

Setting up a reverse proxy allows Matrix clients to connect to your server securely through the default HTTPS port (443) without needing to run Synapse with root privileges. So first I install Nginx.

apt install nginx

Once installed, I create a virtual host file, to manage the incoming connections.

nano /etc/nginx/sites-available/matrix

The virtual host file configuration redirects all traffic from port 80 (HTTP) to port 443 (HTTPS), configures the SSL port with the certificates created earlier, redirects any requests to the /_matrix endpoint on the domain to the Matrix Synpse server, and configures port 8448, which is used by the Matrix Federation APIs to allow Matrix homeservers to communicate with each other.

server {
    listen 80;
    server_name domain.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name domain.com;

    ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem;

    location /_matrix {
        proxy_pass http://localhost:8008;
        proxy_set_header X-Forwarded-For $remote_addr;
        # Nginx by default only allows file uploads up to 1M in size
        # Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
        client_max_body_size 10M;
    }
}

# This is used for Matrix Federation
# which is using default TCP port '8448'
server {
    listen 8448 ssl;
    server_name domain.com;

    ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem;

    location / {
        proxy_pass http://localhost:8008;
        proxy_set_header X-Forwarded-For $remote_addr;
    }
}

Once the file is saved and closed, I can enable the Nginx virtual host, and test to make sure there were no issues.

ln -s /etc/nginx/sites-available/matrix /etc/nginx/sites-enabled/
nginx -t

Finally, I restart Nginx, and enable it to start at system boot.

systemctl restart nginx
systemctl enable nginx

UFW Firewall

With the basics set up, it would now be a good idea to add some firewall rules, to ensure the rest of the server ports are locked down. I add the ssh, http, https, and the TCP port ‘8448’ to the UFW firewall using the command below.

for svc in ssh http https 8448
do
ufw allow $svc
done

After the rules are added, I enable the firewall and check the firewall rules, using these two commands.

ufw enable
ufw status numbered

At this point, I usually log into the server via SSH in a new terminal just to be sure I’ve not locked myself out of SSH access.

Test the Matrix Synapse homeserver

If everything is set up correctly, at this point you should be able to browse to the below URL, enter the server domain, and check if a successful call can be made to your homeserver.

https://federationtester.matrix.org/

If you see Success message, congratulations! You’ve successfully set up your Matrix Synapse server. However, there’s still more to be done, which we will dive into in part 2 of this tutorial.