Files
webmail-client/docker-compose.yml
T
lukasandClaude Opus 5 059fa0ade3 Antispam: the app could never authenticate to the learning engine
Rspamd answers on two ports and only one of them asks for a password:
the scan worker takes /checkv2 anonymously, the controller demands
`Password:` for /stat and /learnspam. `webmail.sh init` generates
RSPAMD_CONTROLLER_PASSWORD and docker-compose hands it to the rspamd
container — but the shared x-backend-env block that feeds api and
worker-* never carried it. The plugin reads it from the environment,
found "", and sent no header.

So classification worked and learning never did. Every learn returned
401 and the plugin swallowed it, as it is built to: an engine problem
must not lose mail. Bootstrap logged "engine unavailable", passive
learning from Inbox->Spam moves taught nothing, and the single visible
trace was `"bayes": null` on GET /api/plugins/antispam/stats, on an
install whose verdict counts were ticking up normally.

Verified against a live engine: with no password /stat and /learnspam
both 401 and rspamd.learn() returns False; with it, stats returns and
learn returns True.

  - pass RSPAMD_CONTROLLER_PASSWORD to the app containers, in both the
    production and dev compose files
  - doctor now makes the real call from inside the api container, with
    the password that container actually holds, and says plainly that
    nothing will be learned if it fails
  - a test reads both compose files, because this is a missing line that
    review already let through once

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-21 22:17:29 +02:00

430 lines
17 KiB
YAML

# Production stack for a single box (Synology / QNAP / any Docker host).
#
# Data lives in bind mounts under ${DATA_ROOT} — NOT in named volumes — so it
# survives `docker compose down`, image updates, project deletion in the NAS
# UI, and a full reinstall, and so the NAS backup tools (Hyper Backup,
# Snapshot Replication, HBS3) can actually see it.
#
# Nothing is published except ${WEB_PORT}. Postgres, Redis, Meilisearch and the
# API are reachable only on the internal `webmail-prod` network.
#
# This file lives at the repo ROOT on purpose. Synology Container Manager and
# QNAP Container Station build a project from a folder and read
# `docker-compose.yml` from it, and DSM's validator rejects newer compose
# features (`include:` among them) — so this is one plain, flat file with no
# indirection. Keep it that way.
#
# Note `docker/docker-compose.yml` is something else entirely: the throwaway
# DEV service stack (Postgres/Redis/Meilisearch/GreenMail on dev ports). Do
# not point a NAS project at the docker/ folder.
#
# Start: ./deploy/webmail.sh up
# Manual: docker compose up -d (reads .env from this directory)
#
# The odd-looking DATA_ROOT default is deliberate: if no .env is found, the
# daemon's "Bind mount failed: '<path>' does not exist" error then names the
# fix instead of pointing at a directory nobody meant to create.
#
# Likewise, no `cpus:` limits: that maps to NanoCPUs, which DSM's Docker
# daemon rejects ("NanoCPUs can not be set"). mem_limit uses the much older
# Memory field and works fine, so memory caps are the throttle here.
#
# Interpolation here must stay on the oldest, dumbest forms: ${VAR} and
# ${VAR:-default}. DSM's bundled Compose rejects `${VAR:?message}` with
# "invalid interpolation format", and rejects `include:` outright. Required
# values are enforced by deploy/webmail.sh and `webmail.sh doctor` instead.
#
# Project name is deliberately NOT "webmail": docker/docker-compose.yml uses
# that, and two compose files sharing one project name fight over each other's
# containers — running `make up` would stop production.
name: webmail-prod
x-logging: &logging
# A shared NAS must never have its volume filled by container logs.
driver: json-file
options: { max-size: "10m", max-file: "3" }
x-backend-image: &backend-image
image: ${REGISTRY_PREFIX:-webmail}/backend:${IMAGE_TAG:-local}
build:
context: .
dockerfile: docker/Dockerfile.backend
args:
APP_VERSION: ${IMAGE_TAG:-local}
GIT_SHA: ${GIT_SHA:-unknown}
x-backend-env: &backend-env
APP_DOMAIN: "${APP_DOMAIN:-APP_DOMAIN-unset}"
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-webmail}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-webmail}
REDIS_URL: redis://redis:6379/0
MASTER_KEY: "${MASTER_KEY}"
BLOB_ROOT: /data/blobs
SEARCH_BACKEND: ${SEARCH_BACKEND:-pg}
MEILI_URL: ${MEILI_URL:-http://meilisearch:7700}
MEILI_KEY: ${MEILI_MASTER_KEY:-}
# The antispam engine splits across two ports: the scan worker (:11333) takes
# anonymous /checkv2, the controller (:11334) demands this password for /stat
# and /learnspam. Without it here, classification works and LEARNING SILENTLY
# DOES NOT — the controller answers 401, the plugin fails open, and the only
# visible symptom is "bayes": null on the stats endpoint.
RSPAMD_CONTROLLER_PASSWORD: ${RSPAMD_CONTROLLER_PASSWORD}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
UNDO_SECONDS: ${UNDO_SECONDS:-10}
SESSION_TTL_DAYS: ${SESSION_TTL_DAYS:-30}
MAX_ATTACHMENT_MB: ${MAX_ATTACHMENT_MB:-25}
TRASH_RETENTION_DAYS: ${TRASH_RETENTION_DAYS:-30}
SENDER_AVATARS_ENABLED: ${SENDER_AVATARS_ENABLED:-true}
GOOGLE_OAUTH_ENABLED: ${GOOGLE_OAUTH_ENABLED:-false}
OAUTH_GOOGLE_CLIENT_ID: ${OAUTH_GOOGLE_CLIENT_ID:-}
OAUTH_GOOGLE_CLIENT_SECRET: ${OAUTH_GOOGLE_CLIENT_SECRET:-}
OAUTH_MS_CLIENT_ID: ${OAUTH_MS_CLIENT_ID:-}
OAUTH_MS_CLIENT_SECRET: ${OAUTH_MS_CLIENT_SECRET:-}
# Worker sizing — see docker/.env.prod.example for what these cost in RAM.
INTERACTIVE_CONCURRENCY: ${INTERACTIVE_CONCURRENCY:-8}
BULK_CONCURRENCY: ${BULK_CONCURRENCY:-1}
PARSE_CONCURRENCY: ${PARSE_CONCURRENCY:-2}
STREAM_PARSE_MB: ${STREAM_PARSE_MB:-8}
PER_HOST_CONN_CAP: ${PER_HOST_CONN_CAP:-10}
MAX_ACCOUNTS_PER_WORKER: ${MAX_ACCOUNTS_PER_WORKER:-400}
HOT_BUDGET_PER_WORKER: ${HOT_BUDGET_PER_WORKER:-50}
APP_VERSION: ${IMAGE_TAG:-local}
TZ: ${TZ:-UTC}
services:
# One-shot: make the bind-mounted host folders writable by the unprivileged
# container users before anything tries to use them. Runs as root by design.
init:
image: alpine:3.20
user: root
command:
- sh
- -c
- |
set -e
mkdir -p /mnt/postgres /mnt/blobs /mnt/backups /mnt/redis /mnt/meili /mnt/state
# Only recurse when ownership is actually wrong: `chown -R` over a large
# blob store on every start would be minutes of pointless disk churn.
[ "$$(stat -c %u /mnt/blobs)" = "10001" ] || chown -R 10001:10001 /mnt/blobs
[ "$$(stat -c %u /mnt/backups)" = "999" ] || chown -R 999:999 /mnt/backups
[ "$$(stat -c %u /mnt/redis)" = "999" ] || chown -R 999:999 /mnt/redis
chmod 700 /mnt/backups
echo "[init] data directories ready"
volumes:
# ONE mount, of DATA_ROOT itself. Synology's Docker daemon refuses to
# create a missing bind-mount source ("Bind mount failed: ... doesn't
# exists") instead of creating it the way Docker does elsewhere, so every
# subdirectory mounted below must exist before its container is created.
# Mounting the parent here means only DATA_ROOT has to pre-exist, and this
# service creates the rest — which is why everything now depends on it.
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}:/mnt
restart: "no"
logging: *logging
postgres:
image: postgres:16
environment:
POSTGRES_USER: ${POSTGRES_USER:-webmail}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB:-webmail}
# Keeps the data directory one level down, so the bind mount can hold lost+found
# and NAS metadata files without initdb refusing to start.
PGDATA: /var/lib/postgresql/data/pgdata
TZ: ${TZ:-UTC}
command:
- postgres
- -c
- shared_buffers=${PG_SHARED_BUFFERS:-256MB}
- -c
- max_connections=${PG_MAX_CONNECTIONS:-100}
- -c
- work_mem=${PG_WORK_MEM:-8MB}
- -c
- maintenance_work_mem=${PG_MAINTENANCE_WORK_MEM:-128MB}
- -c
- effective_cache_size=${PG_EFFECTIVE_CACHE_SIZE:-768MB}
# Full page writes stay ON: NAS filesystems (btrfs/ext4) are not 8k-atomic.
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/postgres:/var/lib/postgresql/data
depends_on:
init: { condition: service_completed_successfully }
healthcheck:
# -h 127.0.0.1 forces a TCP check. Without it pg_isready uses the Unix
# socket, which the temporary server that the postgres image runs during
# first-time initdb DOES answer (it listens on the socket only) — so the
# container went healthy before TCP was up, and dependents connecting to
# postgres:5432 hung until they timed out.
test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -p 5432 -U ${POSTGRES_USER:-webmail} -d ${POSTGRES_DB:-webmail}"]
interval: 10s
timeout: 5s
retries: 12
start_period: 30s
restart: unless-stopped
mem_limit: ${PG_MEM_LIMIT:-768m}
stop_grace_period: 60s
logging: *logging
redis:
image: redis:7-alpine
# Leases and the arq queues live here. AOF everysec means a NAS power cut
# costs at most a second of queue state instead of the whole queue.
command:
- redis-server
- --appendonly
- "yes"
- --appendfsync
- everysec
- --maxmemory
- ${REDIS_MAXMEMORY:-96mb}
- --maxmemory-policy
- noeviction
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/redis:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 3s
retries: 10
depends_on:
init: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${REDIS_MEM_LIMIT:-160m}
logging: *logging
# Opt-in: `./deploy/webmail.sh up --meili` (or COMPOSE_PROFILES=meili).
# Costs ~700MB-1GB RSS; with SEARCH_BACKEND=pg the stack runs without it.
meilisearch:
image: getmeili/meilisearch:v1.8
profiles: ["meili"]
environment:
MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-}
MEILI_ENV: production
MEILI_NO_ANALYTICS: "true"
TZ: ${TZ:-UTC}
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/meili:/meili_data
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:7700/health | grep -q available"]
interval: 10s
timeout: 5s
retries: 12
start_period: 20s
depends_on:
init: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${MEILI_MEM_LIMIT:-1g}
logging: *logging
# --- anti-spam engine (SPEC Appendix C) ------------------------------------
# Not optional: it starts with everything else. Note what does NOT depend on
# it — api, workers and web have no `depends_on: rspamd`. That is deliberate
# and load-bearing: classification is best-effort, delivery is not, so an
# engine that is down, slow or mid-restart must never hold up mail (§C.4).
rspamd:
image: rspamd/rspamd:3.9
# The controller password is an internal secret nobody types; humans reach
# Rspamd through the webmail admin panel, which authenticates with the
# ordinary administrator account. local.d is read-only (the shipped privacy
# defaults), so the generated password is written to override.d at start.
# Starts as root only to write that file — /etc/rspamd is root-owned and the
# image's default user (_rspamd, uid 11333) cannot write it. rspamd then
# drops to _rspamd itself via -u/-g, which is how it normally runs: a root
# master supervising unprivileged workers.
user: root
entrypoint:
- /bin/sh
- -c
- |
mkdir -p /etc/rspamd/override.d
printf 'password = "%s";\nenable_password = "%s";\n' \
"$$RSPAMD_CONTROLLER_PASSWORD" "$$RSPAMD_CONTROLLER_PASSWORD" \
> /etc/rspamd/override.d/worker-controller.inc
exec /usr/bin/rspamd -f -u _rspamd -g _rspamd
environment:
RSPAMD_CONTROLLER_PASSWORD: ${RSPAMD_CONTROLLER_PASSWORD}
TZ: ${TZ:-UTC}
volumes:
- ./docker/rspamd/local.d:/etc/rspamd/local.d:ro
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/rspamd:/var/lib/rspamd
healthcheck:
# rspamc, not wget/curl: the rspamd image ships neither, so the obvious
# HTTP probe fails 127 forever and the container sits "unhealthy" while
# the engine is answering perfectly.
test: ["CMD-SHELL", "rspamc stat > /dev/null 2>&1"]
interval: 30s
timeout: 5s
retries: 5
start_period: 20s
depends_on:
redis-rspamd: { condition: service_healthy }
init: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${RSPAMD_MEM_LIMIT:-256m}
logging: *logging
# Bayes tokens and fuzzy hashes. Separate from the app's redis on purpose
# (§C.2): this data is derived and retrainable, so it trades durability for
# room to grow — RDB snapshots every 5 min, and an eviction policy, where the
# app's redis uses AOF + noeviction because losing a sync lease loses work.
redis-rspamd:
image: redis:7-alpine
command:
- redis-server
- --save
- "300"
- "1"
- --maxmemory
- ${RSPAMD_REDIS_MAXMEMORY:-128mb}
- --maxmemory-policy
- allkeys-lru
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/rspamd-redis:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 3s
retries: 10
depends_on:
init: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${RSPAMD_REDIS_MEM_LIMIT:-160m}
logging: *logging
# One-shot schema upgrade: core Alembic head + every plugin's own branch.
# Everything else waits for this to exit 0, so a failed migration stops the
# roll instead of starting code against an old schema.
migrate:
<<: *backend-image
command: ["migrate"]
environment:
<<: *backend-env
ENABLED_PLUGINS: ${ENABLED_PLUGINS:-calendar contacts labels filters}
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/blobs:/data/blobs
depends_on:
postgres: { condition: service_healthy }
init: { condition: service_completed_successfully }
restart: "no"
logging: *logging
api:
<<: *backend-image
command: ["api"]
environment: *backend-env
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/blobs:/data/blobs
healthcheck:
test: ["CMD-SHELL", "python -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/api/healthz', timeout=5).status==200 else 1)\""]
interval: 30s
timeout: 10s
retries: 5
start_period: 30s
depends_on:
postgres: { condition: service_healthy }
redis: { condition: service_healthy }
migrate: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${API_MEM_LIMIT:-512m}
logging: *logging
worker-sync:
<<: *backend-image
command: ["worker-sync"]
environment: *backend-env
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/blobs:/data/blobs
# Persistent IMAP IDLE connections; the default 1024 is fine for a personal
# install but this costs nothing and matches the sync-engine v2 sizing.
ulimits:
nofile: { soft: 8192, hard: 8192 }
depends_on:
postgres: { condition: service_healthy }
redis: { condition: service_healthy }
migrate: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${WORKER_SYNC_MEM_LIMIT:-512m}
# Leases are released on SIGTERM (workers/run.py targets <10 s); give it room.
stop_grace_period: 30s
logging: *logging
worker-jobs:
<<: *backend-image
command: ["worker-jobs"]
environment: *backend-env
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/blobs:/data/blobs
ulimits:
nofile: { soft: 8192, hard: 8192 }
depends_on:
postgres: { condition: service_healthy }
redis: { condition: service_healthy }
migrate: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${WORKER_JOBS_MEM_LIMIT:-640m}
stop_grace_period: 30s
logging: *logging
web:
image: ${REGISTRY_PREFIX:-webmail}/web:${IMAGE_TAG:-local}
build:
context: .
dockerfile: docker/Dockerfile.frontend
args:
APP_VERSION: ${IMAGE_TAG:-local}
GIT_SHA: ${GIT_SHA:-unknown}
environment:
TZ: ${TZ:-UTC}
ports:
# Bind to a specific NAS interface by setting WEB_BIND / ADMIN_BIND
# (e.g. 192.168.1.185) to keep a port off other networks.
#
# WEB_PORT -> :80, the mail app; /api/admin/* is refused on this one.
# ADMIN_PORT -> :81, the same app with the admin API reachable.
#
# Restrict ADMIN_PORT to the LAN in the NAS firewall — publishing it is
# not what protects it (docs/nas-install.md §5.1). Comment the second
# line out entirely if you would rather reach admin over VPN only.
- "${WEB_BIND:-0.0.0.0}:${WEB_PORT:-8088}:80"
- "${ADMIN_BIND:-0.0.0.0}:${ADMIN_PORT:-8089}:81"
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/healthz | grep -q ok"]
interval: 30s
timeout: 5s
retries: 5
depends_on:
api: { condition: service_started }
restart: unless-stopped
mem_limit: ${WEB_MEM_LIMIT:-96m}
logging: *logging
# Nightly logical backup of Postgres into ${DATA_ROOT}/backups, pruned to
# BACKUP_KEEP_DAYS. Point Hyper Backup / HBS3 at ${DATA_ROOT} and this plus the
# blob store is the whole recoverable state (the other half is MASTER_KEY —
# see docs/nas-install.md §"The three things that matter").
backup:
image: postgres:16
user: "999:999"
entrypoint: ["/bin/bash", "/usr/local/bin/backup.sh"]
environment:
PGHOST: postgres
PGUSER: ${POSTGRES_USER:-webmail}
PGPASSWORD: ${POSTGRES_PASSWORD}
PGDATABASE: ${POSTGRES_DB:-webmail}
BACKUP_DIR: /backups
BACKUP_AT: ${BACKUP_AT:-03:30}
BACKUP_KEEP_DAYS: ${BACKUP_KEEP_DAYS:-14}
TZ: ${TZ:-UTC}
volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/backups:/backups
- ./docker/backup.sh:/usr/local/bin/backup.sh:ro
depends_on:
postgres: { condition: service_healthy }
init: { condition: service_completed_successfully }
restart: unless-stopped
mem_limit: ${BACKUP_MEM_LIMIT:-256m}
logging: *logging
networks:
default:
name: webmail-prod