webmail-client/docs/nas-install.md

# Running Webmail on a Synology or QNAP NAS

This is the operator runbook: how to install it, how to keep your mail safe from
anything Docker does to itself, and how to update without holding your breath.
It assumes the NAS is doing other work too, so the stack is deliberately modest:
**two published ports, no host networking, memory caps on every container** —
mail on one, the admin panel on the other (§5.1).

Everything here works on any Intel/AMD Docker host — the NAS parts are just the
specifics for DSM, QTS and QuTS hero. Synology users want §2.5; QNAP users
§2.6.

---

## 1. What gets installed

| Container | What it does | Published? |
|---|---|---|
| `web` | nginx: serves the UI, proxies `/api` (incl. the WebSocket) | **yes — the only one** |
| `api` | FastAPI app | no, internal network only |
| `worker-sync` | keeps IMAP connections open, syncs mail | no |
| `worker-jobs` | background jobs (send, backfill, purge, plugin jobs) | no |
| `postgres` | the database | no |
| `redis` | job queues + sync leases | no |
| `backup` | nightly verified database dump | no |
| `meilisearch` | optional better search (see §7) | no |
| `migrate`, `init` | run once at start, then exit | — |

Plugins (calendar, contacts, labels, filters) are baked into the backend image
and their database schema is applied automatically on every start.

**Requirements:** an Intel/AMD NAS with Docker (Synology *Container Manager*,
QNAP *Container Station*), ~2 GB RAM free, and 10 GB+ of disk for a typical
personal mailbox. Nothing is needed from the internet except your mail servers.

---

## 2. Install

### 2.1 Pick where the data lives

This is the single most important decision. Use a **shared folder**, never a
Docker named volume:

| NAS | Use |
|---|---|
| Synology | `/volume1/docker/webmail/data` |
| QNAP (QTS / QuTS hero) | `/share/Container/webmail/data` |

On **QuTS hero** `/share/<shared-folder>` is the stable path — use it rather
than the underlying `/share/<POOL>_DATA/...` form, which carries the pool name
and changes if you ever move the folder. ZFS is a genuine advantage here:
snapshot the `Container` shared folder and you get cheap, instant, restorable
points-in-time for the whole install, on top of the nightly dump in §3.

Why: named volumes live inside Docker's internal storage, where Hyper Backup,
Snapshot Replication and HBS3 cannot see them, and where "delete the project" in
the NAS UI can take your mail with it. A shared folder is backed up by the tools
you already run, survives every container operation, and can be copied to
another machine wholesale.

> **Never put this on an NFS or SMB mount.** Postgres on a network filesystem
> corrupts. `./deploy/webmail.sh doctor` checks this and refuses to be quiet
> about it.

### 2.2 Get the code onto the NAS

Enable SSH on the NAS (DSM: *Control Panel → Terminal & SNMP → Enable SSH*;
QTS: *Control Panel → Telnet/SSH*), then:

```bash
ssh admin@your-nas
sudo -i
mkdir -p /volume1/docker/webmail && cd /volume1/docker/webmail
git clone <your-gitea-url>/webmail.git src && cd src
```

No git on the NAS? Copy the folder over with File Station or `rsync` instead.

### 2.3 Configure

```bash
./deploy/webmail.sh init
```

This creates `.env` with a freshly generated `MASTER_KEY` and
database password. Then open it and set:

```ini
DATA_ROOT=/volume1/docker/webmail/data   # or /share/Container/webmail/data
APP_DOMAIN=mail.home.lan                 # your handles become you@this
TZ=Europe/Warsaw
WEB_PORT=8088                            # mail; avoid 5000/5001 (DSM), 8080 (QTS)
ADMIN_PORT=8089                          # admin; keep this one LAN-only (§5.1)
WEB_BIND=0.0.0.0                         # or one NAS IP, e.g. 192.168.1.185
ADMIN_BIND=0.0.0.0
```

> ### Copy `MASTER_KEY` into your password manager. Now.
> It encrypts every stored mail-account password. It exists **only** in this
> file. A perfect database backup without this key gives you your mail but not
> your credentials — you would have to re-enter every account password by hand.
> `.env` is `chmod 600` and git-ignored; treat it as a backup
> artefact in its own right.

### 2.4 Start

```bash
./deploy/webmail.sh doctor    # sanity-checks config, disk, filesystem, ports
./deploy/webmail.sh up        # builds the images on first run, then starts
```

First run builds both images. The frontend build wants ~2 GB of RAM: if your NAS
is small or you'd rather not tie it up, build on your PC instead — see §4
"Building images elsewhere".

Open `http://your-nas:8088/`, create your account, and add a mail account
through the wizard.

### 2.5 Synology: installing entirely from the DSM GUI (no SSH)

The whole install can be done without ever enabling SSH. The trick is **Task
Scheduler**: it runs any shell command as root from the DSM web UI, which covers
the three things the Container Manager UI cannot do on its own.

**1. Get the code onto the NAS.** In Gitea, open the repository and use
*Download ZIP* (the `<>` Code button). In DSM, open *File Station*, create
`docker/webmail/`, upload the zip into it, then right-click → *Extract here*.
Rename the extracted folder to `src`, so you end up with
`/volume1/docker/webmail/src/docker-compose.yml`.

**2. Configure it — one Task Scheduler job.** *Control Panel → Task Scheduler →
Create → Scheduled Task → User-defined script*. Set **User: root**, leave it
unscheduled, and paste this as the script (adjust the four values):

```bash
cd /volume1/docker/webmail/src
chmod +x deploy/*.sh docker/*.sh
bash deploy/webmail.sh init \
  --data-root /volume1/docker/webmail/data \
  --domain mail.home.lan \
  --tz Europe/Warsaw \
  --port 8088
```

Save, select the task, and click *Run*. The `chmod` line matters: a zip download
loses the executable bit. This generates `MASTER_KEY`, a database password, and
the `.env` that Container Manager will read — with no hidden files to edit.

**3. Back up the master key — a second one-off task.** Same recipe, script:

```bash
cd /volume1/docker/webmail/src
bash deploy/webmail.sh show-key > /volume1/docker/webmail/MASTER_KEY.txt
```

Run it, open `MASTER_KEY.txt` in File Station, copy the key into your password
manager, then **delete the file**.

> Doing this from an SSH shell instead? `init` created that file as root with
> mode 600, so run it as root or you get "Permission denied":
> `sudo bash deploy/webmail.sh show-key`. The same goes for every other command
> here — the data directories are root-owned, so start with `sudo -i`. Skipping this is the one mistake with no
recovery: without that key, no backup can decrypt your saved mail-account
passwords.

**4. Create the project.** *Container Manager → Project → Create*:

| Field | Value |
|---|---|
| Project name | `webmail-prod` |
| Path | `/volume1/docker/webmail/src` |
| Source | it will find `docker-compose.yml` — choose to use the existing file |

Pick the **repo root**, not `docker/`. Both contain a `docker-compose.yml`, and
the one in `docker/` is the throwaway *dev* service stack — starting it gives
you containers but no webmail server. Use the project name `webmail-prod`
exactly: the name typed here overrides the one in the file, and matching them
means the GUI and `deploy/webmail.sh` drive the same stack.

Click through, and it builds. **First build takes 5-15 minutes and wants ~2 GB
of RAM** — it compiles the frontend on the NAS. This is the one place the GUI
route is meaningfully worse than SSH, where you can build on your PC and copy
the images over (§4 "Building images elsewhere").

**5. Open `http://your-nas:8088/`** and register your account. The first account
created on a fresh install becomes the **administrator**, and registration closes
behind it — see §5.1.

**What you will see in Container Manager:** `init` and `migrate` sitting at
**Exited (0)**. That is correct — they are one-shot jobs that did their work and
stopped, and everything else waited on them.

#### Running the maintenance commands without a shell

Task Scheduler is not just for setup — it runs any of these, and can schedule
them. Each is a user-defined script, user **root**, starting with
`cd /volume1/docker/webmail/src`:

| Task | Script line | When |
|---|---|---|
| Update safely | `bash deploy/webmail.sh update` | after pulling new code |
| Roll back | `bash deploy/webmail.sh rollback` | if an update misbehaves |
| Backup now | `bash deploy/webmail.sh backup` | before anything risky |
| Restore | `echo RESTORE \| bash deploy/webmail.sh restore <file>` | disaster recovery |
| Health check | `bash deploy/webmail.sh doctor` | when something looks wrong |
| Rebuild search | `bash deploy/webmail.sh reindex --purge` | after a restore |

Updating *through the Container Manager UI instead* (project → Build) does work
and does apply migrations — but it takes no backup first, has no health gate,
and will not roll back if the new version fails to start. The `update` task
above does all three. Use it.

### 2.6 QNAP: Container Station and QuTS hero

The stack itself is identical — same compose file, same `deploy/webmail.sh`.
Four things differ from the Synology route.

**1. Docker is off the PATH.** Container Station keeps `docker` and
`docker-compose` inside its QPKG directory, so a fresh SSH session finds
neither. `deploy/webmail.sh` locates them itself (it globs
`/share/*/.qpkg/container-station/bin`, which covers both QTS's
`CACHEDEV1_DATA` and QuTS hero's pool name). For your own commands, put it on
the PATH for the session:

```bash
export PATH="$(ls -d /share/*/.qpkg/container-station/bin | head -1):$PATH"
docker ps
```

**2. Drive it from the shell, not the Container Station wizard.** Two reasons,
one of which holds whatever your Container Station version offers. The weaker
one: depending on version, its *Applications* create-flow may want YAML pasted
into a text box rather than a folder path, and pasted YAML has no `.env` beside
it — which is where `MASTER_KEY` and the database password live. The one that
always holds: its update path takes no backup, has no health gate and cannot
roll back, exactly like Synology's project *Build* button. Run the install the
normal way:

```bash
ssh admin@your-qnap
sudo -i
mkdir -p /share/Container/webmail && cd /share/Container/webmail
git clone <your-gitea-url>/webmail.git src && cd src
bash deploy/webmail.sh init --data-root /share/Container/webmail/data \
  --domain mail.home.lan --tz Europe/Warsaw --port 8088
bash deploy/webmail.sh up
```

You still get the GUI. Containers started this way use the same Docker daemon,
so Container Station lists them, shows logs and stats, and can stop or restart
them — you simply use `deploy/webmail.sh update` instead of its update button.

**3. Pick ports around QTS.** The web UI owns **8080** (and 443/80 if you have
HTTPS on), so the defaults `WEB_PORT=8088` / `ADMIN_PORT=8089` are already
clear of it. `deploy/webmail.sh doctor` warns if either is taken.

**4. There is no DSM firewall problem.** §8 — the chain that silently drops
container traffic — is Synology-specific. QuTS hero has no such rule by default;
container-to-container and container-to-internet both just work. If you install
**QuFirewall**, re-read §8 first: it describes the same class of failure and,
more usefully, how to prove where a packet dies.

#### Pulling images instead of building on the NAS

Building on the NAS costs 5-15 minutes and ~2 GB of RAM per update. Pulling
pre-built images makes an update a download and a restart. The catch is that
your Gitea speaks plain **HTTP**, and Docker refuses a non-HTTPS registry
unless told otherwise.

On the QNAP, once:

```bash
# find the daemon's config file rather than guessing its path
ps -ef | grep -o '\--config-file[= ][^ ]*' | head -1
```

Add the registry to that `daemon.json`:

```json
{ "insecure-registries": ["192.168.1.185:8418"] }
```

Restart Container Station afterwards so the daemon reloads, then:

```bash
docker login 192.168.1.185:8418
```

From then on, on your PC:

```bash
REGISTRY_PREFIX=192.168.1.185:8418/<you>/webmail ./deploy/build-push.sh v1.0.0
```

and on the NAS, with `REGISTRY_PREFIX` and `IMAGE_TAG` set in `.env`:

```bash
bash deploy/webmail.sh pull v1.0.0      # first install
bash deploy/webmail.sh update v1.0.0    # every time after
```

`pull` exists because the other two don't cover a first install from a
registry: `up` only builds what is missing locally, and `update` refuses to run
without an existing database to back up.

> If you would rather not loosen the daemon, `./deploy/build-push.sh v1.0.0
> --save` writes a tar you copy over and `docker load` — no registry, no
> insecure flag, but a manual copy each time.

### 2.7 Starting automatically after a reboot

Every service is `restart: unless-stopped`, so Docker brings the stack back on
boot by itself. On Synology, make sure Container Manager itself is set to start
on boot (it is by default); on QNAP, the same goes for Container Station.

---

## 3. The three things that matter

Containers are disposable. These are not:

| # | What | Where | If you lose it |
|---|---|---|---|
| 1 | **Database** | `$DATA_ROOT/postgres` + nightly dumps in `$DATA_ROOT/backups` | Everything: accounts, threads, flags, calendar, contacts |
| 2 | **Blob store** | `$DATA_ROOT/blobs` | Mail bodies and attachments (re-downloadable from the servers, slowly) |
| 3 | **`MASTER_KEY`** | `.env` + your password manager | Every saved mail-account password, permanently |

Everything else is derived and needs no backup: Redis holds queues and leases,
Meilisearch's index rebuilds from Postgres (§7), and images rebuild from source.

### Backups

Two layers, and you want both:

**Nightly dumps (automatic).** The `backup` container writes a compressed
`pg_dump` to `$DATA_ROOT/backups` at `BACKUP_AT` (default 03:30) and keeps
`BACKUP_KEEP_DAYS` (14) of them. Every dump is verified with `pg_restore --list`
before it counts, so you never find out at restore time that it was truncated.

**Off-box copies (you set this up, once).** Point your NAS backup tool at
`$DATA_ROOT`:

- Synology: *Hyper Backup* → local/remote/cloud destination → include the
  `docker/webmail` shared folder. Add *Snapshot Replication* on a Btrfs volume
  for fast local rollback.
- QNAP: *HBS 3* → the `Container/webmail` folder. Add snapshots on QuTS hero.

A snapshot of a *running* Postgres directory is crash-consistent — it usually
replays fine, but it is not a substitute for the dumps, which are verified and
restorable into a different Postgres version. Keep both.

Also back up `.env` somewhere that is not the NAS.

### Restore drill — do this once, before you need it

```bash
./deploy/webmail.sh status                          # see available dumps
./deploy/webmail.sh restore webmail-20260914-033000.dump
```

It stops the app containers, restores in a single transaction (a failure leaves
the old database untouched), and starts everything back up.

Full disaster recovery onto a new box:

1. Install Docker, copy `$DATA_ROOT` back from your backup.
2. Copy the repo and `.env` — **with the original `MASTER_KEY`**.
3. `./deploy/webmail.sh up`

That's it. If the blob store was lost but the database survived, mail bodies
refill from the mail servers as accounts re-sync.

---

## 4. Updating

```bash
git pull
./deploy/webmail.sh update
```

`update` does, in this order, stopping the moment anything fails:

1. **Takes a verified database backup.** No backup, no update.
2. Builds (or pulls) the new images. A failure here changes nothing.
3. Runs migrations as a one-shot container — **before** any service restarts, so
   a bad migration never leaves new code running against an old schema.
4. Rolls workers first, then API, then web. Sync leases transfer to the
   replacement worker within ~90 s and message upserts are idempotent, so no
   mail is lost or duplicated by a roll.
5. Health-gates for 120 s. **If the new version doesn't come up healthy it rolls
   itself back automatically** and leaves you on the previous version.

To go back manually:

```bash
./deploy/webmail.sh rollback          # previous version
./deploy/webmail.sh rollback v1.2.0   # a specific one
```

Rolling *code* back does not require rolling the *schema* back, as long as
migrations stay expand-then-contract (see `docs/migrations.md` discipline in
`deployment-setup-instructions.md` §9). For the rare breaking migration, restore
the dump that `update` took in step 1.

### Refreshing the source tree (installs without git)

`update` replaces the **images**. The files around them — `docker-compose.yml`,
`deploy/webmail.sh`, `docker/rspamd/local.d` (read by the antispam engine) — come
from the source tree on the NAS, and change only when that tree does. On an
install that was unpacked from a tarball rather than cloned, refresh it now and
then, keeping `.env` and the data directory:

```bash
cd /share/Container/webmail          # the folder that holds src/
curl -fsSL -o main.tar.gz http://192.168.1.185:8418/lukas/webmail-client/archive/main.tar.gz
tar -xzf main.tar.gz                 # unpacks to ./webmail-client
cp -p src/.env webmail-client/.env
[ -f src/docker/.env.prod ] && cp -p src/docker/.env.prod webmail-client/docker/.env.prod
./src/deploy/webmail.sh down         # stop with the OLD files
mv src src-old-$(date +%Y%m%d)       # kept as a backup
mv webmail-client src
chmod +x src/deploy/*.sh src/docker/*.sh
cd src && ./deploy/webmail.sh up && ./deploy/webmail.sh doctor
```

`down`/`up` rather than `update`: every container is recreated from the new
compose file, including the ones `update` leaves alone, and bind mounts point at
the new folder instead of the renamed one. Data lives in `DATA_ROOT`, outside
`src/`, and `.env` keeps `IMAGE_TAG`, so the same version comes back up.

### Building images elsewhere

Building on the NAS is simple but slow and RAM-hungry. Two alternatives:

```bash
# On your PC — push to your Gitea registry (the namespace is your Gitea user):
REGISTRY_PREFIX=192.168.1.185:8418/<you>/webmail ./deploy/build-push.sh v1.0.0
# then on the NAS, set REGISTRY_PREFIX + IMAGE_TAG in .env and:
./deploy/webmail.sh update v1.0.0

# Or with no registry at all — build to a file and carry it over:
./deploy/build-push.sh v1.0.0 --save
# copy webmail-v1.0.0.tar to the NAS, then:
docker load -i webmail-v1.0.0.tar && ./deploy/webmail.sh update v1.0.0
```

---

## 5. Reaching it from outside the LAN

### 5.1 Read this first

**Registration is open only until the first account exists.** On a fresh install
the signup page is available to anyone who can reach the port; whoever uses it
first becomes the administrator, and signup closes automatically behind them. So
the window is narrow, but it is real: create your account before exposing the
port, or someone else owns the instance.

After that, new accounts need an administrator to reopen registration — which
brings us to the two ports.

#### The two ports

One container, two apps, two nginx listeners:

| Port | Serves | Expose to |
|---|---|---|
| `WEB_PORT` (8088) | The mail client. The admin panel and `/api/admin/*` both return **404** here. | Wherever you like |
| `ADMIN_PORT` (8089) | The admin panel — its own app, its own sign-in. | **LAN only** |

They are two separate applications built from one source tree: `index.html` is
the mail client, `admin.html` is the panel, and nginx serves one per listener.
The panel is a few KB of its own code and never loads the mail client. Opening
`http://your-nas:8089/` lands directly on the administrator sign-in.

Signing in there is the ordinary account login — same session cookie as mail.
An account without administrator rights gets in and is told it is not an
administrator; it sees no settings.

Understand what this boundary is: nginx refusing a path on one listener. That is
enough to keep the admin API off an internet-facing port, and it fails closed if
the app ever forgets a permission check. It is **not** process isolation — both
listeners proxy to the same API container, so anything that reaches `ADMIN_PORT`
reaches every admin route, and the API's own administrator check is still what
guards each one. Restricting `ADMIN_PORT` is therefore not optional.

#### Restricting ADMIN_PORT to the LAN

**On QNAP** there is usually no firewall in the way at all, which cuts both
ways: nothing filters `ADMIN_PORT` for you either. Two things keep it off the
internet, and you want both:

1. **Do not forward 8089 on your router.** This is what actually matters. A
   published port is only reachable from outside if something routes to it, and
   a home router forwards nothing by default.
2. **Bind it to the LAN interface.** Set `ADMIN_BIND` in `.env` to the NAS's LAN
   address, e.g. `ADMIN_BIND=192.168.1.50`. The port then isn't listening on any
   other interface — useful if the box ever gains a second one (a VPN tunnel, a
   second NIC, a Docker macvlan).

For an actual source-IP rule on QNAP you need the optional **QuFirewall** app;
its allow/deny-by-subnet model works like the DSM recipe below.

**On Synology:** *Control Panel → Security → Firewall → Edit Rules → Create*

| Field | Value |
|---|---|
| Ports | Custom → TCP → `8089` |
| Source IP | Specific IP → Subnet → `192.168.1.0` / `255.255.255.0` |
| Action | Allow |

Then a second rule for the same port with Source IP **All** and Action **Deny**,
placed directly below it. Order is everything: DSM evaluates top-down, so the
allow must come first. Verify from off-LAN — or just trust the deny and check
that `curl http://nas:8089/healthz` still answers from a LAN machine.

If you would rather not publish it at all, comment the `ADMIN_PORT` line out of
the `web` service's `ports:` in `docker-compose.yml` and reach admin over VPN,
or temporarily `docker compose port` it when you need it.

Reopening registration, once you are on `ADMIN_PORT`:

```bash
# the panel does this for you; this is the same call by hand, from a
# signed-in browser session:
curl -X PUT http://your-nas:8089/api/admin/settings \
  -H 'Content-Type: application/json' -H "X-CSRF: $CSRF" \
  --cookie "session=$SESSION; csrf=$CSRF" \
  -d '{"registration_open": true}'
```

`GET /api/admin/settings` reads the current state, and `{"registration_open":
false}` closes it again. The same endpoint carries `default_quota_bytes`, the
quota handed to accounts created from then on — changing it never resizes a
mailbox that already exists.

#### And the mail port

Even with admin split off, do not put `WEB_PORT` straight on the public
internet unguarded. Pick one:

- **VPN** (Tailscale, WireGuard, or the NAS's own VPN Server) — simplest and
  safest. Both ports stay LAN-only.
- **NAS reverse proxy with HTTPS**, restricted by firewall rules to the networks
  you trust. Proxy `WEB_PORT` only — never create a proxy entry for
  `ADMIN_PORT`, or you have undone the split.

### 5.2 HTTPS through the NAS reverse proxy

DSM owns ports 80/443, which is why the stack publishes 8088. To put DSM's
certificate in front of it:

*Control Panel → Login Portal → Advanced → Reverse Proxy → Create*

| Field | Value |
|---|---|
| Source protocol / hostname / port | HTTPS / `mail.yourdomain` / 443 |
| Destination protocol / hostname / port | HTTP / `localhost` / `8088` |

Then open **Custom Header → Create → WebSocket** and save. Without that,
`/api/ws` is blocked and the UI stops updating live.

QNAP: *Network & Virtual Switch → Reverse Proxy* — same idea, and likewise
enable WebSocket support.

Session cookies get the `Secure` and `__Host-` treatment automatically as soon
as the app sees an HTTPS request: nginx forwards `X-Forwarded-Proto`, and the
API is started with `--proxy-headers`. Nothing to configure.

**Plain HTTP and HSTS.** If DSM also has a rule for **HTTP** / port 80 to the
same hostname, keep it: the web container answers any request DSM forwarded
over plain HTTP with a `301` to `https://` (it recognises one by DSM's
`X-Forwarded-Proto: http`), and sends `Strict-Transport-Security` on HTTPS
responses. Leave DSM's own *Enable HSTS* box unticked — the container already
sends the header. Opening `http://NAS:8088` on the LAN is not redirected
(no reverse proxy, no such header).

**Security headers** (CSP, framing, `nosniff`, referrer, HSTS) are set by the
web container on every response — `docker/nginx-security.conf`. Don't add
Custom Headers for them in DSM: two differing CSPs are both enforced.

**Breached-password check.** New passwords (sign-up, reset) are checked
against Have I Been Pwned's range API — only the first 5 hex digits of the
password's SHA-1 leave the server. The API container needs outbound HTTPS to
`api.pwnedpasswords.com`; when it cannot reach it the check is skipped, never
fatal. `PASSWORD_BREACH_CHECK=false` turns it off.

### 5.3 Installing as an app, and notifications about new mail

Over HTTPS the web app is installable (Chrome/Edge: the install icon in the
address bar; Android: *Add to Home screen*; iPhone/iPad: *Share → Add to Home
Screen*). Each user turns on **Settings → Accounts → Notifications** per
device; new mail reaching an Inbox then shows a notification even with the app
closed. Desktop Opera shows them too but has no "install app"; Opera on Android
can install. On iPhone/iPad notifications work only from the Home Screen app
(iOS 16.4+).

- The server signs notifications with a key pair it makes on first use, stored
  in the database (private half encrypted with `MASTER_KEY`). Nothing to set.
- Set **Public URL** in the admin panel (e.g. `https://ultimo.email`): push
  services see it as the contact for our notifications. Without it they see
  `mailto:postmaster@APP_DOMAIN`.
- The API container must reach the push services over HTTPS (port 443):
  `fcm.googleapis.com`, `*.google.com`, `*.push.services.mozilla.com`,
  `*.push.apple.com`, `*.notify.windows.com`.
- The notification shows sender, subject and the first line; it travels
  end-to-end encrypted through the browser vendor's push service.

**Later: an Android app (TWA).** The installable web app is what a Trusted
Web Activity wraps (e.g. with Bubblewrap from `/manifest.webmanifest`). It
then also needs `https://ultimo.email/.well-known/assetlinks.json` naming the
Android package and its signing-certificate fingerprint — served by nginx
like any other static file.

### 5.4 Calendars and contacts in other apps (CalDAV / CardDAV)

Ultimo serves each user's calendars and address books to standard apps, which
sync two-way with the web app: iPhone/iPad and macOS Calendar & Contacts,
Thunderbird, Android through DAVx⁵, GNOME/Evolution.

- **Server:** `https://ultimo.email/dav/` — most apps also find it from just
  `ultimo.email` (`/.well-known/caldav`, `/.well-known/carddav`).
- **User name:** the full Ultimo address. **Password:** an **app password**
  (Settings → Security → App passwords) — the account password is refused, so
  2FA cannot be bypassed and a lost device is one revocation away. Settings shows
  the exact server and user name to type.
- nginx passes `/dav/` and the two well-known paths to the API (in
  `docker/nginx-app.conf`). A reverse proxy in front (QNAP, DSM) must pass them
  too, with the `Authorization` header — the default.
- HTTPS is required by iOS/macOS for these accounts.
- Repeated wrong passwords are throttled per address and per IP (15 minutes).

---

## 6. Resource usage

Steady state for one or two mail accounts, `SEARCH_BACKEND=pg`:

| | Typical | Cap (`.env`) |
|---|---|---|
| postgres | 150-300 MB | `PG_MEM_LIMIT=768m` |
| api | 150-250 MB | `API_MEM_LIMIT=512m` |
| worker-sync | 120-250 MB | `WORKER_SYNC_MEM_LIMIT=512m` |
| worker-jobs | 150-350 MB | `WORKER_JOBS_MEM_LIMIT=640m` |
| redis / web / backup | < 100 MB total | — |
| rspamd + its redis | 150-300 MB | `RSPAMD_MEM_LIMIT=256m`, `RSPAMD_REDIS_MEM_LIMIT=160m` |
| **total** | **~0.9-1.5 GB** | ~2.6 GB of limits |
| meilisearch (optional) | 300-800 MB | `MEILI_MEM_LIMIT=1g` |

On a 2 GB NAS: stay on `SEARCH_BACKEND=pg` and drop `PG_SHARED_BUFFERS` to
`128MB`. Tight but workable. On 4 GB+ you have room for Meilisearch.

### The anti-spam engine

`rspamd` and `redis-rspamd` start with everything else — spam filtering is part
of the stack, not an add-on. Two things follow from that:

- **Nothing waits for them.** No service declares `depends_on: rspamd`, so an
  engine that is down, slow or restarting cannot delay mail. Classification is
  best-effort; delivery is not. The health gate ignores it for the same reason.
- **`webmail.sh init` generates `RSPAMD_CONTROLLER_PASSWORD`.** It is internal —
  nobody types it — but the engine will not start without it. Upgrading an
  install that predates the engine means re-running `init`, which only fills in
  values that are empty and never touches `MASTER_KEY`.

Its Bayes database lives in `$DATA_ROOT/rspamd-redis`. Like the search index,
that is derived data: it rides along in backups but never needs restoring, and
retraining rebuilds it.

### The spell checker (LanguageTool)

`languagetool` checks spelling and grammar in the compose window, in some 30
languages. Like the anti-spam engine nothing waits for it: when it is down the
compose window falls back to the browser's own spell checker. It is never
published to the host — the app's API is its only client.

It needs about **1.2 GB of RAM** (`LANGUAGETOOL_HEAP=1g` plus the JVM) and is
capped at one CPU core (`LANGUAGETOOL_CPUS`), so checking can never slow sync,
search or the API.

**Optional n-gram data** teaches it to catch real words used wrongly — "the
affect was huge", "Das Buch liegt ihm Schrank" — beyond its hand-written rules
(English has ~780 such word pairs, German ~70). Install it once:

```bash
./deploy/webmail.sh ngrams            # en de fr es — or name languages: ngrams en de
```

That downloads about **13 GB** (English 8.4 GB, German/French/Spanish ~1.6 GB
each) into `$DATA_ROOT/languagetool-ngrams`, unpacks it to about **26 GB**, and
restarts the engine to load it. An interrupted download resumes when you run
the command again. There is no n-gram data for Polish; Polish text is checked
exactly as without it.

The data is read from disk on demand and cached by the kernel, so the memory
limit (`LANGUAGETOOL_MEM_LIMIT=28g`) is room for that cache, not memory in use.
At start the container reads the files once in the background, at low
priority, so even the first checks after a restart are served from RAM. On
mechanical disks that warm-up takes several minutes; an NVMe cache in front of
the volume makes it — and any later cache misses — fast.

Two NAS-specific realities worth knowing:

- **Disk hibernation won't happen.** Active accounts hold IMAP IDLE connections
  open and write as mail arrives. That is the feature working, not a bug.
- **Keep NTP on.** Two-factor codes fail on a drifting clock.

---

## 7. Search: Postgres or Meilisearch

`SEARCH_BACKEND=pg` (default) uses Postgres full-text search. No extra
container, no extra RAM, and it is perfectly good for a personal mailbox.

`SEARCH_BACKEND=meili` adds typo tolerance, better ranking and stays fast on
large mailboxes, at the cost of a container and 300-800 MB.

To switch:

```bash
# in .env:  SEARCH_BACKEND=meili  and set MEILI_MASTER_KEY
./deploy/webmail.sh up          # starts Meilisearch automatically
./deploy/webmail.sh reindex     # build the index from existing mail
```

The index is **derived data** — every field it holds also lives in Postgres — so
it never needs backing up. If the Meilisearch volume is lost, or you switch
backends, or search ever looks stale, just rebuild it:

```bash
./deploy/webmail.sh reindex             # upsert everything (safe to re-run)
./deploy/webmail.sh reindex --purge     # drop stale documents first
```

---

## 8. Synology only: the DSM firewall drops container traffic

**On QNAP, skip this section** unless you have installed *QuFirewall* — QTS and
QuTS hero add no such rule of their own. It is still worth skimming §8.2, which
is mostly a method for proving *where* a packet dies; that part is portable.

There are two separate failures here with the same silent-timeout signature.

### 8.1 Containers cannot reach each other

**Symptom:** everything reports healthy, but `migrate` fails with an asyncpg
`TimeoutError`, or services intermittently cannot reach `postgres:5432`. Testing
by hand is maddening — the same command succeeds, then fails, then succeeds:

```
postgres:5432 - no response
postgres:5432 - accepting connections
postgres:5432 - no response
```

**Cause:** with Synology's firewall enabled, DSM puts its own chain ahead of
Docker's in `FORWARD`:

```
-A FORWARD -j FORWARD_FIREWALL      <- DSM's, first, ends in -j DROP
-A FORWARD -j DEFAULT_FORWARD       <- Docker's ACCEPT rules, never reached
```

and `net.bridge.bridge-nf-call-iptables=1` (the DSM default) forces traffic
between two containers *on the same bridge* through that chain. Your database
port is not in DSM's allow list — and should not be — so new connections are
dropped. The `RELATED,ESTABLISHED` accept near the top is what makes it look
intermittent rather than broken: flows that got established keep working.

Adding an iptables rule does **not** hold. DSM reloads its firewall
periodically, which flushes and rebuilds the chains; you can watch the packet
counters reset to zero and any rule you inserted disappear with them.

**Fix — take bridge traffic out of iptables entirely:**

```bash
sysctl -w net.bridge.bridge-nf-call-iptables=0
```

Two containers on one bridge are a layer-2 conversation; that flag is what
forces it through a layer-3 firewall in the first place. With it off the packets
cross the bridge directly, where DSM's reloading cannot reach them.

Verify — expect 10 out of 10:

```bash
for i in $(seq 10); do
  docker run --rm --network webmail-prod postgres:16 pg_isready -h postgres -p 5432 -t 5
done
```

**Make it permanent.** *Control Panel → Task Scheduler → Create → Triggered
Task → User-defined script*, event **Boot-up**, user **root**:

```bash
sleep 60
sysctl -w net.bridge.bridge-nf-call-iptables=0
```

Re-check it after a DSM upgrade with `sysctl net.bridge.bridge-nf-call-iptables`.

#### Is that safe?

Yes, for this deployment, and the scope is narrower than it first sounds.

The flag only affects packets **bridged within a single network** — that is,
containers on the same Docker network talking to each other. That they can do so
is Docker's normal behaviour on any ordinary Linux host; DSM's default is the
unusual one.

What does **not** change:

- **Isolation between different Docker networks.** That traffic is *routed*, not
  bridged, so it still traverses `FORWARD` and Docker's
  `DOCKER-ISOLATION-STAGE-1/2` rules still apply.
- **Traffic from your LAN or the internet into a published port.** Still routed,
  still filtered by DSM's firewall. The published ports here are `WEB_PORT` and
  `ADMIN_PORT` — and that filtering is exactly what keeps `ADMIN_PORT` on the
  LAN, so this is a place you *want* DSM's firewall doing its job (§5.1).
- **Traffic from containers out to the network.** Still routed and filtered.
- **DSM's own protection.** The `INPUT` chain, which guards the NAS itself, is
  untouched.

The one thing to be aware of: it is a host-wide setting, so it applies to every
bridge on the box. If you run virtual machines on a bridged network *and* rely
on DSM's firewall to filter traffic between VMs on that same bridge, that
filtering stops too. Filtering of VM traffic to anywhere else is unaffected.

### 8.2 Containers cannot reach the internet

The `bridge-nf` fix above does **not** cover this one, and the symptom is easy
to misread.

**Symptom:** adding a mail account fails with `connection timed out`, while the
setup wizard happily auto-discovers the provider's server names. That looks like
"the app found the server, so the network is fine" — it is not. Autodiscovery
answers from a local presets file for the common providers, and never touches
the network at all.

**Cause:** `FORWARD_FIREWALL` is an allow-list of destination ports ending in
`DROP`. Inspect it and the shape is obvious:

```
23  RETURN  udp  --  udp dpt:53                 <- DNS allowed
24  RETURN  tcp  --  multiport dports 80,443    <- HTTPS allowed
    ...  no 993, no 465, no 587 ...
    DROP    all                                 <- IMAP/SMTP dies here
```

So DNS resolves and HTTPS works, while IMAP's SYN goes unanswered. A *timeout*
rather than a refusal is the tell: a closed port answers with RST, a firewall
says nothing.

Note that `bridge-nf-call-iptables=0` cannot help, because this traffic is
**routed** (bridge → eth0), not bridged, so it crosses `FORWARD` regardless.

**Prove it** — the host reaches the server, a container does not:

```bash
# A: from the NAS itself
timeout 5 bash -c 'exec 3<>/dev/tcp/imap.gmail.com/993' && echo "HOST OK" || echo "HOST FAIL"

# B: from inside the stack's network
docker run --rm --network webmail-prod postgres:16 \
  bash -c 'timeout 5 bash -c "exec 3<>/dev/tcp/imap.gmail.com/993" && echo "CTR OK" || echo "CTR FAIL"'
```

A ok + B fail confirms it. To see precisely which rule kills it, zero the
counters, re-run B, and look for the one that moved:

```bash
iptables -Z FORWARD_FIREWALL
iptables -vnL FORWARD_FIREWALL --line-numbers
```

**Fix — allow the Docker subnets outbound, in DSM's own GUI.** Not with
`iptables`: a hand-inserted rule is flushed the next time DSM rebuilds its
chains (§8.1), whereas a GUI rule is part of what gets rebuilt.

*Control Panel → Security → Firewall → Edit Rules → Create*

| Field | Value |
|---|---|
| Ports | All |
| Protocol | All |
| Source IP | Specific IP → Subnet → `172.16.0.0` / `255.240.0.0` |
| Action | Allow |

Place it above the deny-all. Use the whole `/12` rather than the stack's current
subnet: Docker assigns those dynamically, so a rule naming `172.21.0.0/16` stops
matching the day the network is recreated on a different subnet — a trap that is
easy to fall into and hard to spot afterwards.

## 9. Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| `up` hangs on `postgres` | `DATA_ROOT` on NFS/SMB, or wrong permissions | `./deploy/webmail.sh doctor` |
| UI loads, no live updates | WebSocket not enabled in the NAS reverse proxy | §5.2 Custom Header → WebSocket |
| Login works, logs out on reload | HTTPS terminated upstream but `X-Forwarded-Proto` stripped | Use the NAS reverse proxy as documented; don't strip headers |
| "port is already allocated" | 8088/8089 taken by another app | change `WEB_PORT` / `ADMIN_PORT` |
| Admin panel 404s | you are on `WEB_PORT`; the panel is only on `ADMIN_PORT` | open `http://nas:8089/` (§5.1) |
| Panel says "Not an administrator" | signed in with a non-admin account | sign out, sign in as the first-created account (§5.1) |
| `migrate` fails with asyncpg TimeoutError | Synology: DSM firewall dropping container traffic (§8.1) | `sysctl -w net.bridge.bridge-nf-call-iptables=0` |
| Adding a mail account times out, but autodiscovery worked | Synology: DSM firewall dropping container *egress* (§8.2) | allow `172.16.0.0/12` outbound in the DSM firewall GUI |
| `no compose found` on QNAP over SSH | Container Station's binaries are off-PATH | nothing to do — the script globs for them (§2.6) |
| Mail not syncing | worker stopped, or credentials rejected | `./deploy/webmail.sh logs worker-sync` |
| Search finds nothing after a restore | index not rebuilt | `./deploy/webmail.sh reindex --purge` |
| App won't start after editing `.env` | malformed `MASTER_KEY` | `./deploy/webmail.sh doctor` |

Logs: `./deploy/webmail.sh logs [service] -f`. All logs are capped at 10 MB ×3
per container, so they can't fill the volume.

---

## 10. Command reference

Run these as root (`sudo -i` first): the env file is mode 600 and the data
directories are root-owned.

```
./deploy/webmail.sh init [flags]     create data dirs + generate secrets
./deploy/webmail.sh show-key         print MASTER_KEY so you can back it up
./deploy/webmail.sh doctor           pre-flight checks
./deploy/webmail.sh up [--meili]     start everything
./deploy/webmail.sh down             stop (data untouched)
./deploy/webmail.sh status           what's running, versions, disk, backups
./deploy/webmail.sh logs [svc] [-f]  logs
./deploy/webmail.sh update [tag]     backup -> migrate -> roll -> health-gate
./deploy/webmail.sh rollback [tag]   back to the previous version
./deploy/webmail.sh backup           verified dump, right now
./deploy/webmail.sh restore <dump>   restore a dump (asks for confirmation)
./deploy/webmail.sh reindex [--purge] rebuild the search index
./deploy/webmail.sh ngrams [langs]   spell-check n-gram data (default: en de fr es)
./deploy/webmail.sh psql             database shell
./deploy/webmail.sh shell [svc]      shell in a container
```