Deploy: production Docker stack for a single host (Synology/QNAP NAS)
Everything needed to run the app on an Intel NAS that is also doing other jobs: one published port, no host networking, memory/CPU caps and log rotation on every container, and all state in bind mounts under DATA_ROOT so mail survives image updates, project deletion in the NAS UI, and full reinstalls — and so Hyper Backup/HBS3 can actually see it. - docker/Dockerfile.backend: multi-stage, non-root (uid 10001), tini for the <10s graceful lease release. Installs the four plugins as entry-point distributions. One image, several roles via docker/entrypoint.sh (api | worker-sync | worker-jobs | migrate | reindex | plugins). - docker/Dockerfile.frontend + nginx.conf: SPA + /api reverse proxy with the WebSocket upgrade for /api/ws, and X-Forwarded-Proto preserved so the API still emits __Host-/Secure cookies when TLS is terminated by the NAS reverse proxy (identity/cookies.py keys off request scheme). - docker/docker-compose.prod.yml: project name webmail-prod, NOT webmail — docker-compose.yml (dev services) owns that name, and two compose files sharing one project name fight over each other's containers. Meilisearch is an opt-in profile; SEARCH_BACKEND=pg is the default so a small NAS pays nothing for search. - migrate runs as a one-shot (core Alembic head + every plugin branch) that everything else waits on, so a bad migration can never leave new code running against an old schema. - docker/backup.sh: nightly pg_dump verified with `pg_restore --list` before it counts as a backup, with retention and a baseline on first start. - deploy/webmail.sh: init/doctor/up/down/status/logs/update/rollback/ backup/restore/reindex/psql/shell. `update` is backup -> images -> migrate -> roll workers, api, web -> health-gate, with automatic rollback to the previous version if the gate fails. `doctor` refuses to be quiet when DATA_ROOT is on NFS/SMB (silent Postgres corruption). - deploy/build-push.sh: build images off-NAS (the frontend build wants ~2GB RAM) and push to Gitea, or --save a tar to carry over. - scripts/reindex_search.py: rebuild the search index from Postgres alone. Every SearchDoc field is already stored, so the index is derived data that never needs backing up — this closes the only real objection to running Meilisearch on a box whose volumes may be wiped. - pyproject: package-data for accounts/presets.yaml and the shared plugin Alembic env. Without it a non-editable install (i.e. any image) silently ships a package that fails at runtime. - GET /api/version, used by the deploy health gate. Verified end to end on a real stack: fresh install, all services healthy, core + 4 plugin migration branches applied, login/SPA/plugin bundles through nginx, WS upgrade returning 101, __Host- cookies under a simulated TLS proxy, backup -> restore round-trip preserving user and plugin schema, failed-update auto-rollback (both a bad migration and a boot-crashing API), and Meilisearch profile serving a live query. ruff + mypy --strict clean, 363/363 backend tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nux3LCu4zP9nih4BRhPyHw
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,30 @@
|
|||||||
|
# Build context hygiene: the backend image builds from the repo root (it needs
|
||||||
|
# backend/ *and* plugins/), so everything heavy or secret is excluded here.
|
||||||
|
.git
|
||||||
|
.github
|
||||||
|
.gitea
|
||||||
|
.claude
|
||||||
|
**/node_modules
|
||||||
|
**/dist
|
||||||
|
**/__pycache__
|
||||||
|
**/*.py[cod]
|
||||||
|
**/.mypy_cache
|
||||||
|
**/.ruff_cache
|
||||||
|
**/.pytest_cache
|
||||||
|
**/*.egg-info
|
||||||
|
# Runtime data: mail content and local blobs must never enter an image layer.
|
||||||
|
var/
|
||||||
|
backend/var/
|
||||||
|
**/var/blobs
|
||||||
|
# Secrets
|
||||||
|
.env.dev
|
||||||
|
.env.*.local
|
||||||
|
gitea-server-credentials.md
|
||||||
|
docker/.env.prod
|
||||||
|
deploy/*.env
|
||||||
|
# Docs / screenshots / scratch
|
||||||
|
screenshots/
|
||||||
|
docs/
|
||||||
|
*.md
|
||||||
|
!README.md
|
||||||
|
claude-session.txt
|
||||||
@@ -1,5 +1,7 @@
|
|||||||
# Secrets / local env
|
# Secrets / local env
|
||||||
.env.dev
|
.env.dev
|
||||||
|
docker/.env.prod
|
||||||
|
docker/.env
|
||||||
.env.*.local
|
.env.*.local
|
||||||
gitea-server-credentials.md
|
gitea-server-credentials.md
|
||||||
|
|
||||||
|
|||||||
@@ -21,3 +21,22 @@ test:
|
|||||||
cd backend && pytest -q && cd ../frontend && npx vitest run
|
cd backend && pytest -q && cd ../frontend && npx vitest run
|
||||||
mail-test: ## start throwaway IMAP/SMTP server
|
mail-test: ## start throwaway IMAP/SMTP server
|
||||||
docker compose -f docker/docker-compose.yml up -d greenmail
|
docker compose -f docker/docker-compose.yml up -d greenmail
|
||||||
|
|
||||||
|
# --- production (NAS / single-host Docker deploy; see docs/nas-install.md) ---
|
||||||
|
prod-init: ## generate docker/.env.prod + data dirs
|
||||||
|
./deploy/webmail.sh init
|
||||||
|
prod-up: ## start the production stack
|
||||||
|
./deploy/webmail.sh up
|
||||||
|
prod-down: ## stop the production stack (data kept)
|
||||||
|
./deploy/webmail.sh down
|
||||||
|
prod-status: ## what is running, versions, disk, backups
|
||||||
|
./deploy/webmail.sh status
|
||||||
|
prod-update: ## backup -> migrate -> roll -> health-gate (auto-rollback on failure)
|
||||||
|
./deploy/webmail.sh update
|
||||||
|
prod-backup: ## take a verified database dump now
|
||||||
|
./deploy/webmail.sh backup
|
||||||
|
prod-doctor: ## pre-flight checks on a deployment
|
||||||
|
./deploy/webmail.sh doctor
|
||||||
|
prod-images: ## build production images locally (no push)
|
||||||
|
docker build -f docker/Dockerfile.backend -t webmail/backend:local .
|
||||||
|
docker build -f docker/Dockerfile.frontend -t webmail/web:local .
|
||||||
|
|||||||
@@ -32,6 +32,9 @@ class Settings(BaseSettings):
|
|||||||
session_ttl_days: int = Field(default=30, alias="SESSION_TTL_DAYS")
|
session_ttl_days: int = Field(default=30, alias="SESSION_TTL_DAYS")
|
||||||
max_attachment_mb: int = Field(default=25, alias="MAX_ATTACHMENT_MB")
|
max_attachment_mb: int = Field(default=25, alias="MAX_ATTACHMENT_MB")
|
||||||
log_level: str = Field(default="INFO", alias="LOG_LEVEL")
|
log_level: str = Field(default="INFO", alias="LOG_LEVEL")
|
||||||
|
# Release identifier baked into the container image at build time; surfaced
|
||||||
|
# by GET /api/version so a deploy can verify what is actually running.
|
||||||
|
app_version: str = Field(default="dev", alias="APP_VERSION")
|
||||||
|
|
||||||
# --- Sync engine v2 (workers layer) --------------------------------------
|
# --- Sync engine v2 (workers layer) --------------------------------------
|
||||||
# Worker role: "sync" (lease-owned account runners) | "jobs" (arq consumer)
|
# Worker role: "sync" (lease-owned account runners) | "jobs" (arq consumer)
|
||||||
|
|||||||
@@ -45,6 +45,11 @@ def create_app() -> FastAPI:
|
|||||||
await get_redis().ping()
|
await get_redis().ping()
|
||||||
return {"status": "ok"}
|
return {"status": "ok"}
|
||||||
|
|
||||||
|
@app.get("/api/version")
|
||||||
|
async def version() -> dict[str, str]:
|
||||||
|
"""Report the running release, for deploy/rollback verification."""
|
||||||
|
return {"version": get_settings().app_version}
|
||||||
|
|
||||||
from app.api import register_routers
|
from app.api import register_routers
|
||||||
|
|
||||||
register_routers(app)
|
register_routers(app)
|
||||||
|
|||||||
@@ -52,6 +52,13 @@ dev = [
|
|||||||
where = ["."]
|
where = ["."]
|
||||||
include = ["app*", "workers*"]
|
include = ["app*", "workers*"]
|
||||||
|
|
||||||
|
# Non-Python files that a *non-editable* install (the Docker image) must carry:
|
||||||
|
# the account presets config (SPEC §6.2) and the shared Alembic env used by the
|
||||||
|
# per-plugin migration runner (app/plugins/migrate.py). Without these, a
|
||||||
|
# `pip install ./backend` produces a package that fails at runtime.
|
||||||
|
[tool.setuptools.package-data]
|
||||||
|
app = ["accounts/presets.yaml", "plugins/alembic/*"]
|
||||||
|
|
||||||
[tool.ruff]
|
[tool.ruff]
|
||||||
line-length = 100
|
line-length = 100
|
||||||
target-version = "py311"
|
target-version = "py311"
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
"""Rebuild the search index for every message, from Postgres alone.
|
||||||
|
|
||||||
|
Both backends index the same :class:`SearchDoc` (SPEC §5.2), and every field of
|
||||||
|
it is already stored in the database — subject/addresses/flags on ``messages``,
|
||||||
|
the plain-text body on ``body_parts``, filenames on ``attachments``. So the
|
||||||
|
index is fully derivable and never needs its own backup: if the Meilisearch
|
||||||
|
volume is lost, or you switch SEARCH_BACKEND, run this.
|
||||||
|
|
||||||
|
python scripts/reindex_search.py # everything
|
||||||
|
python scripts/reindex_search.py --user <uuid> # one user
|
||||||
|
python scripts/reindex_search.py --purge # drop stale docs first
|
||||||
|
|
||||||
|
In Docker: ./deploy/webmail.sh reindex [--purge]
|
||||||
|
|
||||||
|
``--purge`` clears each user's existing documents before rebuilding, which is
|
||||||
|
what you want after deleting mail outside the app or when the index has drifted;
|
||||||
|
without it the rebuild is a pure upsert (safe to re-run, safe to interrupt).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import asyncio
|
||||||
|
import uuid
|
||||||
|
from collections import defaultdict
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from app.db import get_sessionmaker
|
||||||
|
from app.models import Attachment, BodyPart, Mailbox, Message
|
||||||
|
from app.search.base import SearchBackend, SearchDoc
|
||||||
|
from app.search.factory import get_search_backend
|
||||||
|
|
||||||
|
# Rows pulled per round-trip, and how many documents are pushed to the backend
|
||||||
|
# concurrently. Deliberately modest: this runs on a NAS alongside live sync.
|
||||||
|
BATCH = 500
|
||||||
|
CONCURRENCY = 8
|
||||||
|
|
||||||
|
|
||||||
|
def _epoch(value: datetime) -> int:
|
||||||
|
return int(value.timestamp())
|
||||||
|
|
||||||
|
|
||||||
|
async def _index_batch(backend: SearchBackend, docs: list[SearchDoc]) -> None:
|
||||||
|
sem = asyncio.Semaphore(CONCURRENCY)
|
||||||
|
|
||||||
|
async def one(doc: SearchDoc) -> None:
|
||||||
|
async with sem:
|
||||||
|
await backend.index(doc)
|
||||||
|
|
||||||
|
await asyncio.gather(*(one(d) for d in docs))
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Rebuild the search index from Postgres")
|
||||||
|
parser.add_argument("--user", help="only this user_id (UUID)")
|
||||||
|
parser.add_argument(
|
||||||
|
"--purge",
|
||||||
|
action="store_true",
|
||||||
|
help="delete each user's existing documents before rebuilding",
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
only_user = uuid.UUID(args.user) if args.user else None
|
||||||
|
backend = get_search_backend()
|
||||||
|
sm = get_sessionmaker()
|
||||||
|
|
||||||
|
if args.purge:
|
||||||
|
async with sm() as session:
|
||||||
|
users = (
|
||||||
|
[only_user]
|
||||||
|
if only_user
|
||||||
|
else list(await session.scalars(select(Message.user_id).distinct()))
|
||||||
|
)
|
||||||
|
for user_id in users:
|
||||||
|
await backend.delete_user(user_id)
|
||||||
|
print(f"purged existing documents for {len(users)} user(s)")
|
||||||
|
|
||||||
|
total = 0
|
||||||
|
after: uuid.UUID | None = None
|
||||||
|
while True:
|
||||||
|
async with sm() as session:
|
||||||
|
stmt = (
|
||||||
|
select(Message, Mailbox.special_use)
|
||||||
|
.join(Mailbox, Mailbox.id == Message.mailbox_id)
|
||||||
|
.order_by(Message.id)
|
||||||
|
.limit(BATCH)
|
||||||
|
)
|
||||||
|
if after is not None:
|
||||||
|
stmt = stmt.where(Message.id > after)
|
||||||
|
if only_user is not None:
|
||||||
|
stmt = stmt.where(Message.user_id == only_user)
|
||||||
|
rows = list(await session.execute(stmt))
|
||||||
|
if not rows:
|
||||||
|
break
|
||||||
|
|
||||||
|
ids = [row[0].id for row in rows]
|
||||||
|
after = ids[-1]
|
||||||
|
|
||||||
|
# Two batched lookups instead of 2N per-message queries.
|
||||||
|
bodies: dict[uuid.UUID, str] = {}
|
||||||
|
for message_id, content in await session.execute(
|
||||||
|
select(BodyPart.message_id, BodyPart.content).where(
|
||||||
|
BodyPart.message_id.in_(ids), BodyPart.kind == "text"
|
||||||
|
)
|
||||||
|
):
|
||||||
|
bodies.setdefault(message_id, content)
|
||||||
|
|
||||||
|
names: dict[uuid.UUID, list[str]] = defaultdict(list)
|
||||||
|
for message_id, filename in await session.execute(
|
||||||
|
select(Attachment.message_id, Attachment.filename).where(
|
||||||
|
Attachment.message_id.in_(ids),
|
||||||
|
Attachment.content_id.is_(None), # real attachments, not inline images
|
||||||
|
)
|
||||||
|
):
|
||||||
|
names[message_id].append(filename)
|
||||||
|
|
||||||
|
docs = [
|
||||||
|
SearchDoc(
|
||||||
|
message_id=msg.id,
|
||||||
|
thread_id=msg.thread_id or msg.id,
|
||||||
|
user_id=msg.user_id,
|
||||||
|
account_id=msg.account_id,
|
||||||
|
mailbox_special_use=special_use,
|
||||||
|
from_=msg.from_addr,
|
||||||
|
to=msg.to_addr,
|
||||||
|
cc=msg.cc_addr,
|
||||||
|
subject=msg.subject,
|
||||||
|
body_text=bodies.get(msg.id, ""),
|
||||||
|
attachment_names=names.get(msg.id, []),
|
||||||
|
has_attachments=msg.has_attachments,
|
||||||
|
is_unread="\\Seen" not in msg.flags,
|
||||||
|
is_starred="\\Flagged" in msg.flags,
|
||||||
|
date=_epoch(msg.internal_date),
|
||||||
|
size_bytes=msg.size_bytes,
|
||||||
|
)
|
||||||
|
for msg, special_use in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
await _index_batch(backend, docs)
|
||||||
|
total += len(docs)
|
||||||
|
print(f"indexed {total} messages", flush=True)
|
||||||
|
|
||||||
|
print(f"done: {total} messages reindexed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
Executable
+77
@@ -0,0 +1,77 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Build the two production images on THIS machine and either push them to a
|
||||||
|
# registry or export them as a file you can copy to the NAS.
|
||||||
|
#
|
||||||
|
# Why you'd use this instead of building on the NAS: the frontend build (Vite +
|
||||||
|
# tsc) wants ~2 GB of RAM and a lot of CPU. On a 2-4 GB NAS that is slow at best
|
||||||
|
# and an OOM at worst, and it competes with everything else the box is doing.
|
||||||
|
#
|
||||||
|
# ./deploy/build-push.sh v1.0.0 # build + push to $REGISTRY_PREFIX
|
||||||
|
# REGISTRY_PREFIX=192.168.1.185:8418/claude/webmail \
|
||||||
|
# ./deploy/build-push.sh v1.0.0
|
||||||
|
# ./deploy/build-push.sh v1.0.0 --save # build + write a .tar to load on the NAS
|
||||||
|
#
|
||||||
|
# On the NAS afterwards:
|
||||||
|
# push mode: set REGISTRY_PREFIX and IMAGE_TAG in docker/.env.prod, then
|
||||||
|
# ./deploy/webmail.sh update v1.0.0
|
||||||
|
# save mode: docker load -i webmail-v1.0.0.tar && ./deploy/webmail.sh update v1.0.0
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
TAG="${1:-}"
|
||||||
|
MODE="${2:-push}"
|
||||||
|
REGISTRY_PREFIX="${REGISTRY_PREFIX:-webmail}"
|
||||||
|
PLATFORM="${PLATFORM:-linux/amd64}" # Synology and QNAP Intel boxes
|
||||||
|
|
||||||
|
[ -n "$TAG" ] || { echo "usage: $0 <version-tag> [--save]" >&2; exit 1; }
|
||||||
|
|
||||||
|
GIT_SHA="$(git -C "$ROOT_DIR" rev-parse --short HEAD 2>/dev/null || echo unknown)"
|
||||||
|
if ! git -C "$ROOT_DIR" diff --quiet 2>/dev/null; then
|
||||||
|
echo "warning: working tree has uncommitted changes; tagging ${TAG} anyway" >&2
|
||||||
|
fi
|
||||||
|
|
||||||
|
build() {
|
||||||
|
local name="$1" dockerfile="$2"
|
||||||
|
echo "== building ${REGISTRY_PREFIX}/${name}:${TAG}"
|
||||||
|
docker build \
|
||||||
|
--platform "$PLATFORM" \
|
||||||
|
-f "${ROOT_DIR}/docker/${dockerfile}" \
|
||||||
|
--build-arg "APP_VERSION=${TAG}" \
|
||||||
|
--build-arg "GIT_SHA=${GIT_SHA}" \
|
||||||
|
-t "${REGISTRY_PREFIX}/${name}:${TAG}" \
|
||||||
|
-t "${REGISTRY_PREFIX}/${name}:latest" \
|
||||||
|
"$ROOT_DIR"
|
||||||
|
}
|
||||||
|
|
||||||
|
build backend Dockerfile.backend
|
||||||
|
build web Dockerfile.frontend
|
||||||
|
|
||||||
|
if [ "$MODE" = "--save" ]; then
|
||||||
|
out="${ROOT_DIR}/webmail-${TAG}.tar"
|
||||||
|
echo "== saving both images to ${out}"
|
||||||
|
docker save -o "$out" \
|
||||||
|
"${REGISTRY_PREFIX}/backend:${TAG}" \
|
||||||
|
"${REGISTRY_PREFIX}/web:${TAG}"
|
||||||
|
echo "done. Copy it to the NAS and run: docker load -i $(basename "$out")"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$REGISTRY_PREFIX" = "webmail" ]; then
|
||||||
|
cat >&2 <<'MSG'
|
||||||
|
REGISTRY_PREFIX is still the local default ("webmail"), so there is nothing to
|
||||||
|
push to. Either set it to your registry path, e.g.
|
||||||
|
|
||||||
|
REGISTRY_PREFIX=192.168.1.185:8418/claude/webmail ./deploy/build-push.sh v1.0.0
|
||||||
|
|
||||||
|
(run `docker login 192.168.1.185:8418` once first), or re-run with --save to
|
||||||
|
produce a tar file instead.
|
||||||
|
MSG
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "== pushing"
|
||||||
|
docker push "${REGISTRY_PREFIX}/backend:${TAG}"
|
||||||
|
docker push "${REGISTRY_PREFIX}/web:${TAG}"
|
||||||
|
docker push "${REGISTRY_PREFIX}/backend:latest"
|
||||||
|
docker push "${REGISTRY_PREFIX}/web:latest"
|
||||||
|
echo "pushed ${TAG}. On the NAS: ./deploy/webmail.sh update ${TAG}"
|
||||||
Executable
+378
@@ -0,0 +1,378 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Webmail operator control script. One command per thing you might need to do.
|
||||||
|
#
|
||||||
|
# ./deploy/webmail.sh init create data dirs + a filled-in .env.prod
|
||||||
|
# ./deploy/webmail.sh up [--meili] start (builds images if missing)
|
||||||
|
# ./deploy/webmail.sh down stop (data is untouched)
|
||||||
|
# ./deploy/webmail.sh status what is running, and which version
|
||||||
|
# ./deploy/webmail.sh logs [svc] [-f] tail logs
|
||||||
|
# ./deploy/webmail.sh update [tag] backup -> new images -> migrate -> roll -> health-gate
|
||||||
|
# ./deploy/webmail.sh rollback [tag] go back to the previous version
|
||||||
|
# ./deploy/webmail.sh backup take a verified database dump now
|
||||||
|
# ./deploy/webmail.sh restore <file> DESTRUCTIVE: restore a dump
|
||||||
|
# ./deploy/webmail.sh reindex [args] rebuild the search index from Postgres
|
||||||
|
# ./deploy/webmail.sh psql open a database shell
|
||||||
|
# ./deploy/webmail.sh shell [svc] shell inside a container (default: api)
|
||||||
|
# ./deploy/webmail.sh doctor check the install before it bites you
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(dirname "$SCRIPT_DIR")"
|
||||||
|
COMPOSE_FILE="${ROOT_DIR}/docker/docker-compose.prod.yml"
|
||||||
|
ENV_FILE="${ROOT_DIR}/docker/.env.prod"
|
||||||
|
ENV_EXAMPLE="${ROOT_DIR}/docker/.env.prod.example"
|
||||||
|
|
||||||
|
# Services that must come back healthy after a roll.
|
||||||
|
HEALTH_SERVICES=(postgres redis api web)
|
||||||
|
HEALTH_TIMEOUT=120
|
||||||
|
|
||||||
|
red() { printf '\033[31m%s\033[0m\n' "$*"; }
|
||||||
|
grn() { printf '\033[32m%s\033[0m\n' "$*"; }
|
||||||
|
ylw() { printf '\033[33m%s\033[0m\n' "$*"; }
|
||||||
|
die() { red "error: $*" >&2; exit 1; }
|
||||||
|
|
||||||
|
compose() {
|
||||||
|
docker compose --env-file "$ENV_FILE" -f "$COMPOSE_FILE" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
require_env() {
|
||||||
|
[ -f "$ENV_FILE" ] || die "$ENV_FILE not found. Run: ./deploy/webmail.sh init"
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
set -a; . "$ENV_FILE"; set +a
|
||||||
|
[ -n "${DATA_ROOT:-}" ] || die "DATA_ROOT is not set in $ENV_FILE"
|
||||||
|
[ -n "${MASTER_KEY:-}" ] || die "MASTER_KEY is not set in $ENV_FILE"
|
||||||
|
[ -n "${POSTGRES_PASSWORD:-}" ] || die "POSTGRES_PASSWORD is not set in $ENV_FILE"
|
||||||
|
}
|
||||||
|
|
||||||
|
state_dir() { echo "${DATA_ROOT}/state"; }
|
||||||
|
|
||||||
|
# busybox sed (what you may get on DSM/QTS) has no -i suffix argument, so every
|
||||||
|
# in-place edit goes through a temp file.
|
||||||
|
set_env_value() {
|
||||||
|
local key="$1" value="$2"
|
||||||
|
sed "s|^${key}=.*|${key}=${value}|" "$ENV_FILE" > "${ENV_FILE}.tmp" \
|
||||||
|
&& mv "${ENV_FILE}.tmp" "$ENV_FILE" && chmod 600 "$ENV_FILE"
|
||||||
|
}
|
||||||
|
|
||||||
|
record_version() {
|
||||||
|
local new="$1" cur
|
||||||
|
mkdir -p "$(state_dir)" 2>/dev/null || true
|
||||||
|
if ! { : > "$(state_dir)/.probe"; } 2>/dev/null; then
|
||||||
|
ylw "cannot write ${DATA_ROOT}/state — version history not recorded (run as the owner of DATA_ROOT)"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
rm -f "$(state_dir)/.probe"
|
||||||
|
cur="$(cat "$(state_dir)/version" 2>/dev/null || echo "")"
|
||||||
|
[ -n "$cur" ] && [ "$cur" != "$new" ] && echo "$cur" > "$(state_dir)/previous"
|
||||||
|
echo "$new" > "$(state_dir)/version"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- commands ----------------------------------------------------------------
|
||||||
|
|
||||||
|
cmd_init() {
|
||||||
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
|
cp "$ENV_EXAMPLE" "$ENV_FILE"
|
||||||
|
chmod 600 "$ENV_FILE"
|
||||||
|
local master pgpass
|
||||||
|
master="$(openssl rand -base64 32)"
|
||||||
|
pgpass="$(openssl rand -base64 24 | tr -d '/+=' | cut -c1-24)"
|
||||||
|
# Portable in-place edit (BSD and GNU sed disagree about -i).
|
||||||
|
sed "s|^MASTER_KEY=.*|MASTER_KEY=${master}|; s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=${pgpass}|" \
|
||||||
|
"$ENV_FILE" > "${ENV_FILE}.tmp" && mv "${ENV_FILE}.tmp" "$ENV_FILE"
|
||||||
|
chmod 600 "$ENV_FILE"
|
||||||
|
grn "created $ENV_FILE with freshly generated secrets"
|
||||||
|
ylw ""
|
||||||
|
ylw " >> BACK UP MASTER_KEY NOW. Copy it into your password manager. <<"
|
||||||
|
ylw " Without it, saved mail-account passwords cannot be decrypted even"
|
||||||
|
ylw " from a perfect database backup."
|
||||||
|
ylw ""
|
||||||
|
echo "Then edit $ENV_FILE and set at least: DATA_ROOT, APP_DOMAIN, TZ, WEB_PORT."
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
ylw "$ENV_FILE already exists; leaving it alone."
|
||||||
|
require_env
|
||||||
|
mkdir -p "${DATA_ROOT}"/{postgres,blobs,redis,backups,meili,state} 2>/dev/null \
|
||||||
|
|| ylw "could not create every directory under ${DATA_ROOT} (the init container will)"
|
||||||
|
grn "data directories ready under ${DATA_ROOT}"
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_doctor() {
|
||||||
|
require_env
|
||||||
|
local ok=0
|
||||||
|
|
||||||
|
echo "config:"
|
||||||
|
echo " DATA_ROOT ${DATA_ROOT}"
|
||||||
|
echo " APP_DOMAIN ${APP_DOMAIN:-<unset>}"
|
||||||
|
echo " WEB_PORT ${WEB_PORT:-8088}"
|
||||||
|
echo " SEARCH_BACKEND ${SEARCH_BACKEND:-pg}"
|
||||||
|
echo " IMAGE_TAG ${IMAGE_TAG:-local}"
|
||||||
|
echo
|
||||||
|
|
||||||
|
if [ "$(printf '%s' "${MASTER_KEY}" | base64 -d 2>/dev/null | wc -c)" -ne 32 ]; then
|
||||||
|
red " MASTER_KEY is not 32 bytes of base64 — the app will refuse to start"; ok=1
|
||||||
|
else
|
||||||
|
grn " MASTER_KEY looks well-formed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$(stat -c %a "$ENV_FILE" 2>/dev/null || echo 600)" != "600" ]; then
|
||||||
|
ylw " $ENV_FILE is not chmod 600 (it holds your master key)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Missing directories are not a failure before the first start: `up` and the
|
||||||
|
# `init` container create them. Only say so, so a pre-install doctor is green.
|
||||||
|
local missing=0
|
||||||
|
for d in postgres blobs redis backups state; do
|
||||||
|
[ -d "${DATA_ROOT}/${d}" ] || missing=1
|
||||||
|
done
|
||||||
|
if [ "$missing" -eq 1 ]; then
|
||||||
|
ylw " data directories not created yet — 'up' will create them"
|
||||||
|
else
|
||||||
|
grn " data directories present"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Probe the nearest existing ancestor: the NFS/SMB check has to work BEFORE
|
||||||
|
# the first install, which is the only time it can still save you.
|
||||||
|
local fstype probe="${DATA_ROOT}"
|
||||||
|
while [ ! -d "$probe" ] && [ "$probe" != "/" ] && [ -n "$probe" ]; do
|
||||||
|
probe="$(dirname "$probe")"
|
||||||
|
done
|
||||||
|
# `|| true`: with `set -e -o pipefail`, df failing on a not-yet-created
|
||||||
|
# DATA_ROOT would abort doctor on exactly the run where you need it most.
|
||||||
|
fstype="$(df -PT "$probe" 2>/dev/null | awk 'NR==2{print $2}' || true)"
|
||||||
|
[ -n "$fstype" ] || fstype="$(stat -f -c %T "$probe" 2>/dev/null || echo "")"
|
||||||
|
case "$fstype" in
|
||||||
|
nfs*|cifs|smbfs|fuse*)
|
||||||
|
red " ${DATA_ROOT} is on a network filesystem — Postgres WILL corrupt there."
|
||||||
|
red " Move DATA_ROOT to a local volume (btrfs/ext4)."; ok=1 ;;
|
||||||
|
"") ylw " could not determine the filesystem of ${DATA_ROOT}" ;;
|
||||||
|
*) grn " ${DATA_ROOT} is on a local filesystem" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
local avail
|
||||||
|
avail="$(df -Pm "$probe" 2>/dev/null | awk 'NR==2{print $4}' || true)"
|
||||||
|
[ -n "$avail" ] && {
|
||||||
|
if [ "$avail" -lt 5120 ]; then ylw " only ${avail} MB free on ${DATA_ROOT}"
|
||||||
|
else grn " ${avail} MB free on ${DATA_ROOT}"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
if command -v ss >/dev/null 2>&1 && ss -ltn 2>/dev/null | grep -q ":${WEB_PORT:-8088} "; then
|
||||||
|
ylw " port ${WEB_PORT:-8088} already has a listener — change WEB_PORT if that is not us"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# chmod 700 and owned by uid 999, so count them as root in a container.
|
||||||
|
local recent
|
||||||
|
recent="$(docker run --rm -v "${DATA_ROOT}/backups:/b:ro" alpine:3.20 \
|
||||||
|
sh -c 'find /b -maxdepth 1 -name "*.dump" -mtime -2 2>/dev/null | wc -l' 2>/dev/null || echo 0)"
|
||||||
|
if [ "${recent:-0}" -gt 0 ]; then grn " ${recent} database backup(s) from the last 48 h"
|
||||||
|
else ylw " no database backup in the last 48 h"; fi
|
||||||
|
|
||||||
|
[ "$ok" -eq 0 ] && grn "doctor: no blocking problems" || red "doctor: problems found (above)"
|
||||||
|
return "$ok"
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_up() {
|
||||||
|
require_env
|
||||||
|
local profile=() build=()
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--meili) profile=(--profile meili) ;;
|
||||||
|
--build) build=(--build) ;;
|
||||||
|
*) die "unknown option: $arg" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
[ "${SEARCH_BACKEND:-pg}" = "meili" ] && [ ${#profile[@]} -eq 0 ] && {
|
||||||
|
ylw "SEARCH_BACKEND=meili — starting the Meilisearch container too"
|
||||||
|
profile=(--profile meili)
|
||||||
|
}
|
||||||
|
# Best-effort only: Docker creates any missing bind-mount path itself, and
|
||||||
|
# the `init` service fixes ownership. This line must not be fatal, because
|
||||||
|
# DATA_ROOT is typically root-owned while you may be running as yourself.
|
||||||
|
mkdir -p "${DATA_ROOT}"/{postgres,blobs,redis,backups,meili,state} 2>/dev/null || true
|
||||||
|
compose "${profile[@]}" up -d "${build[@]}" --wait
|
||||||
|
record_version "${IMAGE_TAG:-local}"
|
||||||
|
grn "up. open http://<this-host>:${WEB_PORT:-8088}/"
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_down() { require_env; compose --profile meili down; grn "stopped (data kept in ${DATA_ROOT})"; }
|
||||||
|
cmd_restart() { require_env; compose restart "${@:-}"; }
|
||||||
|
|
||||||
|
cmd_status() {
|
||||||
|
require_env
|
||||||
|
compose --profile meili ps
|
||||||
|
echo
|
||||||
|
echo "recorded version : $(cat "$(state_dir)/version" 2>/dev/null || echo unknown)"
|
||||||
|
echo "previous version : $(cat "$(state_dir)/previous" 2>/dev/null || echo none)"
|
||||||
|
local running
|
||||||
|
running="$(compose exec -T api python -c \
|
||||||
|
"import urllib.request,json;print(json.load(urllib.request.urlopen('http://127.0.0.1:8000/api/version'))['version'])" \
|
||||||
|
2>/dev/null || echo unreachable)"
|
||||||
|
echo "running version : ${running}"
|
||||||
|
echo
|
||||||
|
# The data directories are owned by the container users (postgres 999,
|
||||||
|
# backend 10001) and backups are chmod 700, so read them as root inside a
|
||||||
|
# throwaway container rather than reporting misleading zeroes.
|
||||||
|
docker run --rm -v "${DATA_ROOT}:/d:ro" alpine:3.20 sh -c '
|
||||||
|
echo "data usage:"
|
||||||
|
du -sh /d/postgres /d/blobs /d/backups 2>/dev/null | sed "s|/d/| |"
|
||||||
|
echo
|
||||||
|
echo "latest backups:"
|
||||||
|
ls -lh /d/backups/*.dump 2>/dev/null | tail -3 | sed "s|/d/backups/| |" || echo " none yet"
|
||||||
|
'
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_logs() {
|
||||||
|
require_env
|
||||||
|
compose --profile meili logs --tail 200 "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
health_gate() {
|
||||||
|
local deadline=$(( $(date +%s) + HEALTH_TIMEOUT )) svc cid state
|
||||||
|
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||||
|
local all_ok=1
|
||||||
|
for svc in "${HEALTH_SERVICES[@]}"; do
|
||||||
|
cid="$(compose ps -q "$svc" 2>/dev/null || true)"
|
||||||
|
[ -n "$cid" ] || { all_ok=0; break; }
|
||||||
|
state="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' "$cid" 2>/dev/null || echo missing)"
|
||||||
|
case "$state" in
|
||||||
|
healthy|running) ;;
|
||||||
|
*) all_ok=0; break ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
[ "$all_ok" -eq 1 ] && return 0
|
||||||
|
sleep 5
|
||||||
|
done
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_backup() {
|
||||||
|
require_env
|
||||||
|
compose run --rm backup once
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_update() {
|
||||||
|
require_env
|
||||||
|
local target="${1:-${IMAGE_TAG:-local}}"
|
||||||
|
local previous="${IMAGE_TAG:-local}"
|
||||||
|
|
||||||
|
echo "== 1/5 backing up the database before anything changes"
|
||||||
|
cmd_backup || die "backup failed — refusing to update. Fix the backup first."
|
||||||
|
|
||||||
|
echo "== 2/5 obtaining images for tag '${target}'"
|
||||||
|
if [ "${REGISTRY_PREFIX:-webmail}" = "webmail" ] || [ "$target" = "local" ]; then
|
||||||
|
IMAGE_TAG="$target" compose build
|
||||||
|
else
|
||||||
|
IMAGE_TAG="$target" compose pull || die "pull failed — nothing has changed, you are still on ${previous}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "== 3/5 applying migrations"
|
||||||
|
IMAGE_TAG="$target" compose run --rm migrate \
|
||||||
|
|| die "migration failed — services untouched, still on ${previous}. Restore from ${DATA_ROOT}/backups if the schema is damaged."
|
||||||
|
|
||||||
|
echo "== 4/5 rolling services"
|
||||||
|
# Workers first (leases transfer to the replacement within ~90 s; the lease
|
||||||
|
# protocol and idempotent (mailbox_id, uid) upserts make this lossless),
|
||||||
|
# then the API, then the web front end.
|
||||||
|
IMAGE_TAG="$target" compose up -d worker-sync worker-jobs
|
||||||
|
IMAGE_TAG="$target" compose up -d api web
|
||||||
|
|
||||||
|
echo "== 5/5 health gate (${HEALTH_TIMEOUT}s)"
|
||||||
|
if health_gate; then
|
||||||
|
record_version "$target"
|
||||||
|
set_env_value IMAGE_TAG "$target"
|
||||||
|
grn "updated to ${target}"
|
||||||
|
else
|
||||||
|
red "health gate failed — rolling back to ${previous}"
|
||||||
|
IMAGE_TAG="$previous" compose up -d --wait || true
|
||||||
|
die "update to ${target} failed; production is back on ${previous}"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_rollback() {
|
||||||
|
require_env
|
||||||
|
local target="${1:-$(cat "$(state_dir)/previous" 2>/dev/null || echo "")}"
|
||||||
|
[ -n "$target" ] || die "no previous version recorded; pass a tag: ./deploy/webmail.sh rollback v1.0.0"
|
||||||
|
ylw "rolling back to ${target}"
|
||||||
|
IMAGE_TAG="$target" compose up -d --wait
|
||||||
|
if health_gate; then
|
||||||
|
record_version "$target"
|
||||||
|
set_env_value IMAGE_TAG "$target"
|
||||||
|
grn "rolled back to ${target}"
|
||||||
|
else
|
||||||
|
die "rollback to ${target} did not come up healthy — check: ./deploy/webmail.sh logs"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_restore() {
|
||||||
|
require_env
|
||||||
|
local arg="${1:-}" dir base
|
||||||
|
[ -n "$arg" ] || die "usage: ./deploy/webmail.sh restore <file.dump> (run \`status\` to list them)"
|
||||||
|
|
||||||
|
# Backups are chmod 700 owned by uid 999, so a plain `[ -f ]` from an
|
||||||
|
# unprivileged shell says "missing" for a file that is right there. Resolve
|
||||||
|
# and verify inside a container instead.
|
||||||
|
if [ -f "$arg" ]; then
|
||||||
|
dir="$(cd "$(dirname "$arg")" && pwd)"; base="$(basename "$arg")"
|
||||||
|
else
|
||||||
|
dir="${DATA_ROOT}/backups"; base="$(basename "$arg")"
|
||||||
|
fi
|
||||||
|
if ! docker run --rm -v "${dir}:/b:ro" alpine:3.20 test -f "/b/${base}" 2>/dev/null; then
|
||||||
|
red "no such dump: ${base}"
|
||||||
|
echo "available in ${DATA_ROOT}/backups:"
|
||||||
|
docker run --rm -v "${DATA_ROOT}/backups:/b:ro" alpine:3.20 \
|
||||||
|
sh -c 'ls -1 /b/*.dump 2>/dev/null | sed "s|/b/| |"' || true
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
local dump="${dir}/${base}"
|
||||||
|
|
||||||
|
red "THIS REPLACES THE CURRENT DATABASE with ${base}."
|
||||||
|
red "Mail synced since that dump will be gone (it re-syncs from the servers)."
|
||||||
|
printf 'Type RESTORE to continue: '
|
||||||
|
read -r confirm
|
||||||
|
[ "$confirm" = "RESTORE" ] || die "aborted"
|
||||||
|
|
||||||
|
echo "stopping application containers (postgres stays up)"
|
||||||
|
compose stop api worker-sync worker-jobs web backup
|
||||||
|
|
||||||
|
echo "restoring $(basename "$dump")"
|
||||||
|
# --clean --if-exists drops and recreates objects; a single transaction so a
|
||||||
|
# failure leaves the old database intact.
|
||||||
|
docker run --rm -i --network webmail-prod \
|
||||||
|
-e PGPASSWORD="${POSTGRES_PASSWORD}" \
|
||||||
|
-v "$(dirname "$dump")":/restore:ro \
|
||||||
|
postgres:16 \
|
||||||
|
pg_restore --clean --if-exists --no-owner --single-transaction \
|
||||||
|
-h postgres -U "${POSTGRES_USER:-webmail}" -d "${POSTGRES_DB:-webmail}" \
|
||||||
|
"/restore/$(basename "$dump")" \
|
||||||
|
|| die "restore failed; the database was left as it was"
|
||||||
|
|
||||||
|
echo "starting application containers"
|
||||||
|
compose up -d --wait
|
||||||
|
grn "restored. If the blob store was lost too, mail bodies will refill on next sync."
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd_reindex() { require_env; compose run --rm api reindex "$@"; }
|
||||||
|
cmd_psql() { require_env; compose exec postgres psql -U "${POSTGRES_USER:-webmail}" -d "${POSTGRES_DB:-webmail}"; }
|
||||||
|
cmd_shell() { require_env; compose exec "${1:-api}" /bin/sh; }
|
||||||
|
|
||||||
|
usage() { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's|^# \?||'; }
|
||||||
|
|
||||||
|
cmd="${1:-}"; shift || true
|
||||||
|
case "$cmd" in
|
||||||
|
init) cmd_init "$@" ;;
|
||||||
|
doctor) cmd_doctor "$@" ;;
|
||||||
|
up) cmd_up "$@" ;;
|
||||||
|
down) cmd_down "$@" ;;
|
||||||
|
restart) cmd_restart "$@" ;;
|
||||||
|
status) cmd_status "$@" ;;
|
||||||
|
logs) cmd_logs "$@" ;;
|
||||||
|
update) cmd_update "$@" ;;
|
||||||
|
rollback) cmd_rollback "$@" ;;
|
||||||
|
backup) cmd_backup "$@" ;;
|
||||||
|
restore) cmd_restore "$@" ;;
|
||||||
|
reindex) cmd_reindex "$@" ;;
|
||||||
|
psql) cmd_psql "$@" ;;
|
||||||
|
shell) cmd_shell "$@" ;;
|
||||||
|
""|-h|--help|help) usage ;;
|
||||||
|
*) die "unknown command: $cmd (try --help)" ;;
|
||||||
|
esac
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# ============================================================================
|
||||||
|
# Webmail production configuration.
|
||||||
|
#
|
||||||
|
# Copy to docker/.env.prod, fill in the three REQUIRED values, keep it safe:
|
||||||
|
# cp docker/.env.prod.example docker/.env.prod && chmod 600 docker/.env.prod
|
||||||
|
#
|
||||||
|
# THIS FILE IS A BACKUP ARTEFACT. MASTER_KEY is not stored anywhere else: lose
|
||||||
|
# it and every saved mail-account password becomes permanently undecryptable,
|
||||||
|
# even from a perfect database backup. Keep a copy in a password manager.
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
# --- REQUIRED ---------------------------------------------------------------
|
||||||
|
|
||||||
|
# Where all persistent data lives on the host. Use a real shared folder your NAS
|
||||||
|
# backup tool can see — not a Docker named volume.
|
||||||
|
# Synology: /volume1/docker/webmail/data
|
||||||
|
# QNAP: /share/Container/webmail/data
|
||||||
|
DATA_ROOT=/volume1/docker/webmail/data
|
||||||
|
|
||||||
|
# 32 random bytes, base64. Generate ONCE, then never change it:
|
||||||
|
# openssl rand -base64 32
|
||||||
|
MASTER_KEY=
|
||||||
|
|
||||||
|
# Database password. Generate once:
|
||||||
|
# openssl rand -base64 24 | tr -d '/+=' | cut -c1-24
|
||||||
|
POSTGRES_PASSWORD=
|
||||||
|
|
||||||
|
# --- Identity ---------------------------------------------------------------
|
||||||
|
# The domain your user handles are minted under (user@APP_DOMAIN) and the issuer
|
||||||
|
# shown in 2FA authenticator apps. It does NOT have to be publicly resolvable.
|
||||||
|
APP_DOMAIN=mail.home.lan
|
||||||
|
|
||||||
|
# --- Network ----------------------------------------------------------------
|
||||||
|
# The single published port. Avoid 5000/5001 (Synology DSM) and 8080 (QNAP QTS).
|
||||||
|
WEB_PORT=8088
|
||||||
|
# Bind to one NAS interface instead of every network, e.g. WEB_BIND=192.168.1.185
|
||||||
|
WEB_BIND=0.0.0.0
|
||||||
|
|
||||||
|
TZ=Europe/Warsaw
|
||||||
|
|
||||||
|
# --- Search -----------------------------------------------------------------
|
||||||
|
# pg = Postgres full-text. No extra container, no extra RAM. Default.
|
||||||
|
# meili = Meilisearch: typo tolerance, better ranking, faster on large mailboxes.
|
||||||
|
# Costs a container and ~0.7-1 GB RAM. Start it with:
|
||||||
|
# ./deploy/webmail.sh up --meili
|
||||||
|
# and after switching an existing install, rebuild the index:
|
||||||
|
# ./deploy/webmail.sh reindex
|
||||||
|
SEARCH_BACKEND=pg
|
||||||
|
# Required if SEARCH_BACKEND=meili — generate with: openssl rand -base64 32
|
||||||
|
MEILI_MASTER_KEY=
|
||||||
|
|
||||||
|
# --- Images -----------------------------------------------------------------
|
||||||
|
# `local` = built on this host from the repo. To deploy prebuilt images from
|
||||||
|
# your Gitea registry instead, set both of these (see deploy/build-push.sh):
|
||||||
|
# REGISTRY_PREFIX=192.168.1.185:8418/claude/webmail
|
||||||
|
# IMAGE_TAG=v1.0.0
|
||||||
|
REGISTRY_PREFIX=webmail
|
||||||
|
IMAGE_TAG=local
|
||||||
|
|
||||||
|
# --- Application ------------------------------------------------------------
|
||||||
|
LOG_LEVEL=INFO
|
||||||
|
UNDO_SECONDS=10
|
||||||
|
SESSION_TTL_DAYS=30
|
||||||
|
MAX_ATTACHMENT_MB=25
|
||||||
|
# Auto-purge Trash/Spam older than N days (0 disables).
|
||||||
|
TRASH_RETENTION_DAYS=30
|
||||||
|
# Server-side sender avatars (Gravatar + favicon), fetched by the server so
|
||||||
|
# senders never see your IP. Set false to make zero outbound web requests.
|
||||||
|
SENDER_AVATARS_ENABLED=true
|
||||||
|
|
||||||
|
GOOGLE_OAUTH_ENABLED=false
|
||||||
|
OAUTH_GOOGLE_CLIENT_ID=
|
||||||
|
OAUTH_GOOGLE_CLIENT_SECRET=
|
||||||
|
OAUTH_MS_CLIENT_ID=
|
||||||
|
OAUTH_MS_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Plugins whose schema is migrated on every start. Remove one to stop shipping
|
||||||
|
# its tables; `hello` is the SDK demo and is intentionally not listed.
|
||||||
|
ENABLED_PLUGINS=calendar contacts labels filters
|
||||||
|
|
||||||
|
# --- Backups ----------------------------------------------------------------
|
||||||
|
BACKUP_AT=03:30
|
||||||
|
BACKUP_KEEP_DAYS=14
|
||||||
|
|
||||||
|
# --- Sizing (defaults suit a personal install on a shared NAS) --------------
|
||||||
|
# Peak worker RAM is roughly PARSE_CONCURRENCY x STREAM_PARSE_MB plus overhead.
|
||||||
|
INTERACTIVE_CONCURRENCY=8
|
||||||
|
BULK_CONCURRENCY=1
|
||||||
|
PARSE_CONCURRENCY=2
|
||||||
|
STREAM_PARSE_MB=8
|
||||||
|
PER_HOST_CONN_CAP=10
|
||||||
|
HOT_BUDGET_PER_WORKER=50
|
||||||
|
|
||||||
|
# Container caps. Total without Meilisearch: ~2.2 GB of *limits*; real steady
|
||||||
|
# usage for one or two mail accounts is around 700 MB-1 GB.
|
||||||
|
PG_MEM_LIMIT=768m
|
||||||
|
PG_SHARED_BUFFERS=256MB
|
||||||
|
PG_MAX_CONNECTIONS=100
|
||||||
|
PG_WORK_MEM=8MB
|
||||||
|
PG_MAINTENANCE_WORK_MEM=128MB
|
||||||
|
PG_EFFECTIVE_CACHE_SIZE=768MB
|
||||||
|
REDIS_MEM_LIMIT=160m
|
||||||
|
REDIS_MAXMEMORY=96mb
|
||||||
|
API_MEM_LIMIT=512m
|
||||||
|
API_CPUS=1.0
|
||||||
|
WORKER_SYNC_MEM_LIMIT=512m
|
||||||
|
WORKER_JOBS_MEM_LIMIT=640m
|
||||||
|
WORKER_CPUS=1.5
|
||||||
|
WEB_MEM_LIMIT=96m
|
||||||
|
BACKUP_MEM_LIMIT=256m
|
||||||
|
MEILI_MEM_LIMIT=1g
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
# Production backend image: API, both worker roles, and the migration/admin CLI.
|
||||||
|
#
|
||||||
|
# One image, several commands (see docker/entrypoint.sh) — matches SPEC §6.3's
|
||||||
|
# WORKER_ROLE split. Build context is the REPO ROOT, because the image needs
|
||||||
|
# backend/ *and* plugins/ (the five plugins are separate distributions that the
|
||||||
|
# loader discovers through the `webmail.plugins` entry-point group, SPEC §11).
|
||||||
|
#
|
||||||
|
# docker build -f docker/Dockerfile.backend -t webmail/backend .
|
||||||
|
|
||||||
|
# ---------- builder: resolve wheels into a self-contained venv ----------------
|
||||||
|
FROM python:3.11-slim AS builder
|
||||||
|
|
||||||
|
ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||||
|
PIP_NO_CACHE_DIR=1 \
|
||||||
|
PYTHONDONTWRITEBYTECODE=1
|
||||||
|
|
||||||
|
# Only needed if a pinned dep has no manylinux wheel for this platform; the
|
||||||
|
# whole stage is discarded, so it costs nothing in the runtime image.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends build-essential \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN python -m venv /opt/venv
|
||||||
|
ENV PATH="/opt/venv/bin:$PATH"
|
||||||
|
|
||||||
|
# Backend first: its dependency set is the slowest layer and changes least often.
|
||||||
|
COPY backend/pyproject.toml /src/backend/pyproject.toml
|
||||||
|
COPY backend/app /src/backend/app
|
||||||
|
COPY backend/workers /src/backend/workers
|
||||||
|
RUN pip install /src/backend
|
||||||
|
|
||||||
|
# Plugins are ordinary distributions installed on top (each owns its own deps).
|
||||||
|
COPY plugins /src/plugins
|
||||||
|
RUN pip install \
|
||||||
|
/src/plugins/webmail_plugin_calendar \
|
||||||
|
/src/plugins/webmail_plugin_contacts \
|
||||||
|
/src/plugins/webmail_plugin_labels \
|
||||||
|
/src/plugins/webmail_plugin_filters
|
||||||
|
|
||||||
|
# ---------- runtime -----------------------------------------------------------
|
||||||
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
ARG APP_VERSION=dev
|
||||||
|
ARG GIT_SHA=unknown
|
||||||
|
LABEL org.opencontainers.image.title="webmail-backend" \
|
||||||
|
org.opencontainers.image.version="${APP_VERSION}" \
|
||||||
|
org.opencontainers.image.revision="${GIT_SHA}"
|
||||||
|
|
||||||
|
ENV PYTHONUNBUFFERED=1 \
|
||||||
|
PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PATH="/opt/venv/bin:$PATH" \
|
||||||
|
APP_VERSION="${APP_VERSION}" \
|
||||||
|
GIT_SHA="${GIT_SHA}"
|
||||||
|
|
||||||
|
# tini reaps zombies and forwards SIGTERM, which the workers need for the <10 s
|
||||||
|
# graceful lease release in workers/run.py.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends tini \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* \
|
||||||
|
&& useradd --system --uid 10001 --create-home --home-dir /home/webmail webmail
|
||||||
|
|
||||||
|
COPY --from=builder /opt/venv /opt/venv
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
# Alembic's core tree lives outside the installed package, so it is copied in.
|
||||||
|
COPY backend/alembic.ini ./alembic.ini
|
||||||
|
COPY backend/migrations ./migrations
|
||||||
|
COPY backend/scripts ./scripts
|
||||||
|
COPY docker/entrypoint.sh /usr/local/bin/entrypoint
|
||||||
|
RUN chmod +x /usr/local/bin/entrypoint
|
||||||
|
|
||||||
|
# Blob store mount point (BLOB_ROOT). Ownership is set by the compose `init`
|
||||||
|
# service, which runs before anything else touches the bind mount.
|
||||||
|
RUN mkdir -p /data/blobs && chown webmail:webmail /data/blobs
|
||||||
|
VOLUME ["/data/blobs"]
|
||||||
|
|
||||||
|
USER webmail
|
||||||
|
ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/entrypoint"]
|
||||||
|
CMD ["api"]
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
# Production web image: the built SPA served by nginx, which also reverse-proxies
|
||||||
|
# /api (including the /api/ws WebSocket) to the API container. Build context is
|
||||||
|
# the REPO ROOT so the build stage can see frontend/.
|
||||||
|
#
|
||||||
|
# docker build -f docker/Dockerfile.frontend -t webmail/web .
|
||||||
|
|
||||||
|
# ---------- builder -----------------------------------------------------------
|
||||||
|
FROM node:20-slim AS builder
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
COPY frontend/package.json frontend/package-lock.json* ./
|
||||||
|
# `npm ci` when a lockfile is present (reproducible), `npm install` otherwise.
|
||||||
|
RUN if [ -f package-lock.json ]; then npm ci; else npm install; fi
|
||||||
|
|
||||||
|
COPY frontend/ ./
|
||||||
|
# Type-check + bundle. `npm run build` runs `tsc --noEmit && vite build`, so a
|
||||||
|
# type error fails the image build rather than shipping a broken bundle.
|
||||||
|
RUN npm run build
|
||||||
|
|
||||||
|
# ---------- runtime -----------------------------------------------------------
|
||||||
|
FROM nginx:1.27-alpine
|
||||||
|
|
||||||
|
ARG APP_VERSION=dev
|
||||||
|
ARG GIT_SHA=unknown
|
||||||
|
LABEL org.opencontainers.image.title="webmail-web" \
|
||||||
|
org.opencontainers.image.version="${APP_VERSION}" \
|
||||||
|
org.opencontainers.image.revision="${GIT_SHA}"
|
||||||
|
|
||||||
|
COPY docker/nginx.conf /etc/nginx/conf.d/default.conf
|
||||||
|
COPY --from=builder /src/dist /usr/share/nginx/html
|
||||||
|
|
||||||
|
# Surface the running version to `deploy/webmail.sh status` without needing the API.
|
||||||
|
RUN printf '%s\n' "${APP_VERSION}" > /usr/share/nginx/html/version.txt
|
||||||
|
|
||||||
|
EXPOSE 80
|
||||||
Executable
+88
@@ -0,0 +1,88 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Nightly logical backup of the webmail database.
|
||||||
|
#
|
||||||
|
# Runs as a long-lived sidecar: sleeps until BACKUP_AT, dumps, verifies, prunes,
|
||||||
|
# repeats. `docker compose run --rm backup once` takes a single backup now (this
|
||||||
|
# is what `deploy/webmail.sh backup` does, e.g. immediately before an update).
|
||||||
|
#
|
||||||
|
# Why logical dumps and not just a NAS snapshot: a btrfs/Hyper Backup snapshot of
|
||||||
|
# a *running* Postgres data directory is crash-consistent, which usually replays
|
||||||
|
# fine but is not guaranteed and cannot be restored into a different Postgres
|
||||||
|
# version. A pg_dump is version-portable and verifiable. Keep both.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
BACKUP_DIR="${BACKUP_DIR:-/backups}"
|
||||||
|
BACKUP_AT="${BACKUP_AT:-03:30}"
|
||||||
|
BACKUP_KEEP_DAYS="${BACKUP_KEEP_DAYS:-14}"
|
||||||
|
DB="${PGDATABASE:-webmail}"
|
||||||
|
|
||||||
|
log() { echo "[backup $(date -Is)] $*"; }
|
||||||
|
|
||||||
|
take_backup() {
|
||||||
|
local stamp out globals
|
||||||
|
stamp="$(date +%Y%m%d-%H%M%S)"
|
||||||
|
out="${BACKUP_DIR}/${DB}-${stamp}.dump"
|
||||||
|
globals="${BACKUP_DIR}/globals-${stamp}.sql"
|
||||||
|
|
||||||
|
log "dumping ${DB} -> $(basename "$out")"
|
||||||
|
# -Fc: compressed custom format, restorable selectively with pg_restore.
|
||||||
|
if ! pg_dump --format=custom --compress=6 --file="${out}.partial" "$DB"; then
|
||||||
|
log "ERROR: pg_dump failed; leaving no partial file"
|
||||||
|
rm -f "${out}.partial"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Verify the archive is readable before it counts as a backup.
|
||||||
|
if ! pg_restore --list "${out}.partial" >/dev/null 2>&1; then
|
||||||
|
log "ERROR: dump failed verification (pg_restore --list); discarding"
|
||||||
|
rm -f "${out}.partial"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mv "${out}.partial" "$out"
|
||||||
|
chmod 600 "$out"
|
||||||
|
|
||||||
|
# Roles/passwords live outside the database; tiny, and makes a restore onto a
|
||||||
|
# bare Postgres one step shorter.
|
||||||
|
pg_dumpall --globals-only --file="$globals" 2>/dev/null || true
|
||||||
|
[ -f "$globals" ] && chmod 600 "$globals"
|
||||||
|
|
||||||
|
ln -sfn "$(basename "$out")" "${BACKUP_DIR}/latest.dump"
|
||||||
|
log "ok: $(du -h "$out" | cut -f1) $(basename "$out")"
|
||||||
|
|
||||||
|
log "pruning dumps older than ${BACKUP_KEEP_DAYS} days"
|
||||||
|
find "$BACKUP_DIR" -maxdepth 1 -name "${DB}-*.dump" -mtime "+${BACKUP_KEEP_DAYS}" -print -delete || true
|
||||||
|
find "$BACKUP_DIR" -maxdepth 1 -name "globals-*.sql" -mtime "+${BACKUP_KEEP_DAYS}" -print -delete || true
|
||||||
|
}
|
||||||
|
|
||||||
|
seconds_until() {
|
||||||
|
local target now next
|
||||||
|
target="$1"
|
||||||
|
now=$(date +%s)
|
||||||
|
next=$(date -d "today ${target}" +%s 2>/dev/null || echo 0)
|
||||||
|
if [ "$next" -le "$now" ]; then
|
||||||
|
next=$(date -d "tomorrow ${target}" +%s)
|
||||||
|
fi
|
||||||
|
echo $(( next - now ))
|
||||||
|
}
|
||||||
|
|
||||||
|
if [ "${1:-}" = "once" ]; then
|
||||||
|
take_backup
|
||||||
|
exit $?
|
||||||
|
fi
|
||||||
|
|
||||||
|
log "backup sidecar started; schedule ${BACKUP_AT} daily, retention ${BACKUP_KEEP_DAYS} days"
|
||||||
|
|
||||||
|
# Baseline on first start: if there is no dump from the last 24 h, take one now
|
||||||
|
# rather than leaving a fresh install unprotected until the first scheduled run.
|
||||||
|
if ! find "$BACKUP_DIR" -maxdepth 1 -name "${DB}-*.dump" -mtime -1 | grep -q .; then
|
||||||
|
log "no recent dump found; taking a baseline backup"
|
||||||
|
take_backup || log "baseline backup failed; will retry on schedule"
|
||||||
|
fi
|
||||||
|
|
||||||
|
while true; do
|
||||||
|
wait_s="$(seconds_until "$BACKUP_AT")"
|
||||||
|
log "next backup in ${wait_s}s (at ${BACKUP_AT})"
|
||||||
|
sleep "$wait_s"
|
||||||
|
take_backup || log "scheduled backup failed; continuing"
|
||||||
|
done
|
||||||
@@ -0,0 +1,307 @@
|
|||||||
|
# Production stack for a single box (Synology / QNAP / any Docker host).
|
||||||
|
#
|
||||||
|
# Data lives in bind mounts under ${DATA_ROOT} — NOT in named volumes — so it
|
||||||
|
# survives `docker compose down`, image updates, project deletion in the NAS
|
||||||
|
# UI, and a full reinstall, and so the NAS backup tools (Hyper Backup,
|
||||||
|
# Snapshot Replication, HBS3) can actually see it.
|
||||||
|
#
|
||||||
|
# Nothing is published except ${WEB_PORT}. Postgres, Redis, Meilisearch and the
|
||||||
|
# API are reachable only on the internal `webmail` network.
|
||||||
|
#
|
||||||
|
# Start: ./deploy/webmail.sh up
|
||||||
|
# Manual: docker compose --env-file docker/.env.prod -f docker/docker-compose.prod.yml up -d
|
||||||
|
|
||||||
|
# Deliberately NOT "webmail": docker/docker-compose.yml (the dev service stack)
|
||||||
|
# uses that project name, and two compose files sharing one project name fight
|
||||||
|
# over each other's containers — running `make up` would stop production.
|
||||||
|
name: webmail-prod
|
||||||
|
|
||||||
|
x-logging: &logging
|
||||||
|
# A shared NAS must never have its volume filled by container logs.
|
||||||
|
driver: json-file
|
||||||
|
options: { max-size: "10m", max-file: "3" }
|
||||||
|
|
||||||
|
x-backend-image: &backend-image
|
||||||
|
image: ${REGISTRY_PREFIX:-webmail}/backend:${IMAGE_TAG:-local}
|
||||||
|
build:
|
||||||
|
context: ..
|
||||||
|
dockerfile: docker/Dockerfile.backend
|
||||||
|
args:
|
||||||
|
APP_VERSION: ${IMAGE_TAG:-local}
|
||||||
|
GIT_SHA: ${GIT_SHA:-unknown}
|
||||||
|
|
||||||
|
x-backend-env: &backend-env
|
||||||
|
APP_DOMAIN: ${APP_DOMAIN}
|
||||||
|
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-webmail}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-webmail}
|
||||||
|
REDIS_URL: redis://redis:6379/0
|
||||||
|
MASTER_KEY: ${MASTER_KEY}
|
||||||
|
BLOB_ROOT: /data/blobs
|
||||||
|
SEARCH_BACKEND: ${SEARCH_BACKEND:-pg}
|
||||||
|
MEILI_URL: ${MEILI_URL:-http://meilisearch:7700}
|
||||||
|
MEILI_KEY: ${MEILI_MASTER_KEY:-}
|
||||||
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
|
UNDO_SECONDS: ${UNDO_SECONDS:-10}
|
||||||
|
SESSION_TTL_DAYS: ${SESSION_TTL_DAYS:-30}
|
||||||
|
MAX_ATTACHMENT_MB: ${MAX_ATTACHMENT_MB:-25}
|
||||||
|
TRASH_RETENTION_DAYS: ${TRASH_RETENTION_DAYS:-30}
|
||||||
|
SENDER_AVATARS_ENABLED: ${SENDER_AVATARS_ENABLED:-true}
|
||||||
|
GOOGLE_OAUTH_ENABLED: ${GOOGLE_OAUTH_ENABLED:-false}
|
||||||
|
OAUTH_GOOGLE_CLIENT_ID: ${OAUTH_GOOGLE_CLIENT_ID:-}
|
||||||
|
OAUTH_GOOGLE_CLIENT_SECRET: ${OAUTH_GOOGLE_CLIENT_SECRET:-}
|
||||||
|
OAUTH_MS_CLIENT_ID: ${OAUTH_MS_CLIENT_ID:-}
|
||||||
|
OAUTH_MS_CLIENT_SECRET: ${OAUTH_MS_CLIENT_SECRET:-}
|
||||||
|
# Worker sizing — see docker/.env.prod.example for what these cost in RAM.
|
||||||
|
INTERACTIVE_CONCURRENCY: ${INTERACTIVE_CONCURRENCY:-8}
|
||||||
|
BULK_CONCURRENCY: ${BULK_CONCURRENCY:-1}
|
||||||
|
PARSE_CONCURRENCY: ${PARSE_CONCURRENCY:-2}
|
||||||
|
STREAM_PARSE_MB: ${STREAM_PARSE_MB:-8}
|
||||||
|
PER_HOST_CONN_CAP: ${PER_HOST_CONN_CAP:-10}
|
||||||
|
MAX_ACCOUNTS_PER_WORKER: ${MAX_ACCOUNTS_PER_WORKER:-400}
|
||||||
|
HOT_BUDGET_PER_WORKER: ${HOT_BUDGET_PER_WORKER:-50}
|
||||||
|
APP_VERSION: ${IMAGE_TAG:-local}
|
||||||
|
TZ: ${TZ:-UTC}
|
||||||
|
|
||||||
|
services:
|
||||||
|
# One-shot: make the bind-mounted host folders writable by the unprivileged
|
||||||
|
# container users before anything tries to use them. Runs as root by design.
|
||||||
|
init:
|
||||||
|
image: alpine:3.20
|
||||||
|
user: root
|
||||||
|
command:
|
||||||
|
- sh
|
||||||
|
- -c
|
||||||
|
- |
|
||||||
|
set -e
|
||||||
|
mkdir -p /mnt/blobs /mnt/backups /mnt/redis /mnt/meili
|
||||||
|
chown -R 10001:10001 /mnt/blobs # backend runs as uid 10001
|
||||||
|
chown -R 999:999 /mnt/backups # pg_dump runs as postgres (999)
|
||||||
|
chown -R 999:999 /mnt/redis # redis runs as uid 999
|
||||||
|
chmod 700 /mnt/backups
|
||||||
|
echo "[init] data directories ready"
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/blobs:/mnt/blobs
|
||||||
|
- ${DATA_ROOT}/backups:/mnt/backups
|
||||||
|
- ${DATA_ROOT}/redis:/mnt/redis
|
||||||
|
- ${DATA_ROOT}/meili:/mnt/meili
|
||||||
|
restart: "no"
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
postgres:
|
||||||
|
image: postgres:16
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: ${POSTGRES_USER:-webmail}
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||||
|
POSTGRES_DB: ${POSTGRES_DB:-webmail}
|
||||||
|
# Keeps the data directory one level down, so the bind mount can hold lost+found
|
||||||
|
# and NAS metadata files without initdb refusing to start.
|
||||||
|
PGDATA: /var/lib/postgresql/data/pgdata
|
||||||
|
TZ: ${TZ:-UTC}
|
||||||
|
command:
|
||||||
|
- postgres
|
||||||
|
- -c
|
||||||
|
- shared_buffers=${PG_SHARED_BUFFERS:-256MB}
|
||||||
|
- -c
|
||||||
|
- max_connections=${PG_MAX_CONNECTIONS:-100}
|
||||||
|
- -c
|
||||||
|
- work_mem=${PG_WORK_MEM:-8MB}
|
||||||
|
- -c
|
||||||
|
- maintenance_work_mem=${PG_MAINTENANCE_WORK_MEM:-128MB}
|
||||||
|
- -c
|
||||||
|
- effective_cache_size=${PG_EFFECTIVE_CACHE_SIZE:-768MB}
|
||||||
|
# Full page writes stay ON: NAS filesystems (btrfs/ext4) are not 8k-atomic.
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/postgres:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-webmail} -d ${POSTGRES_DB:-webmail}"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
start_period: 30s
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${PG_MEM_LIMIT:-768m}
|
||||||
|
stop_grace_period: 60s
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
# Leases and the arq queues live here. AOF everysec means a NAS power cut
|
||||||
|
# costs at most a second of queue state instead of the whole queue.
|
||||||
|
command:
|
||||||
|
- redis-server
|
||||||
|
- --appendonly
|
||||||
|
- "yes"
|
||||||
|
- --appendfsync
|
||||||
|
- everysec
|
||||||
|
- --maxmemory
|
||||||
|
- ${REDIS_MAXMEMORY:-96mb}
|
||||||
|
- --maxmemory-policy
|
||||||
|
- noeviction
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/redis:/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 10
|
||||||
|
depends_on:
|
||||||
|
init: { condition: service_completed_successfully }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${REDIS_MEM_LIMIT:-160m}
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
# Opt-in: `./deploy/webmail.sh up --meili` (or COMPOSE_PROFILES=meili).
|
||||||
|
# Costs ~700MB-1GB RSS; with SEARCH_BACKEND=pg the stack runs without it.
|
||||||
|
meilisearch:
|
||||||
|
image: getmeili/meilisearch:v1.8
|
||||||
|
profiles: ["meili"]
|
||||||
|
environment:
|
||||||
|
MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-}
|
||||||
|
MEILI_ENV: production
|
||||||
|
MEILI_NO_ANALYTICS: "true"
|
||||||
|
TZ: ${TZ:-UTC}
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/meili:/meili_data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:7700/health | grep -q available"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
start_period: 20s
|
||||||
|
depends_on:
|
||||||
|
init: { condition: service_completed_successfully }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${MEILI_MEM_LIMIT:-1g}
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
# One-shot schema upgrade: core Alembic head + every plugin's own branch.
|
||||||
|
# Everything else waits for this to exit 0, so a failed migration stops the
|
||||||
|
# roll instead of starting code against an old schema.
|
||||||
|
migrate:
|
||||||
|
<<: *backend-image
|
||||||
|
command: ["migrate"]
|
||||||
|
environment:
|
||||||
|
<<: *backend-env
|
||||||
|
ENABLED_PLUGINS: ${ENABLED_PLUGINS:-calendar contacts labels filters}
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/blobs:/data/blobs
|
||||||
|
depends_on:
|
||||||
|
postgres: { condition: service_healthy }
|
||||||
|
init: { condition: service_completed_successfully }
|
||||||
|
restart: "no"
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
api:
|
||||||
|
<<: *backend-image
|
||||||
|
command: ["api"]
|
||||||
|
environment: *backend-env
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/blobs:/data/blobs
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "python -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/api/healthz', timeout=5).status==200 else 1)\""]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 5
|
||||||
|
start_period: 30s
|
||||||
|
depends_on:
|
||||||
|
postgres: { condition: service_healthy }
|
||||||
|
redis: { condition: service_healthy }
|
||||||
|
migrate: { condition: service_completed_successfully }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${API_MEM_LIMIT:-512m}
|
||||||
|
cpus: ${API_CPUS:-1.0}
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
worker-sync:
|
||||||
|
<<: *backend-image
|
||||||
|
command: ["worker-sync"]
|
||||||
|
environment: *backend-env
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/blobs:/data/blobs
|
||||||
|
# Persistent IMAP IDLE connections; the default 1024 is fine for a personal
|
||||||
|
# install but this costs nothing and matches the sync-engine v2 sizing.
|
||||||
|
ulimits:
|
||||||
|
nofile: { soft: 8192, hard: 8192 }
|
||||||
|
depends_on:
|
||||||
|
postgres: { condition: service_healthy }
|
||||||
|
redis: { condition: service_healthy }
|
||||||
|
migrate: { condition: service_completed_successfully }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${WORKER_SYNC_MEM_LIMIT:-512m}
|
||||||
|
cpus: ${WORKER_CPUS:-1.5}
|
||||||
|
# Leases are released on SIGTERM (workers/run.py targets <10 s); give it room.
|
||||||
|
stop_grace_period: 30s
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
worker-jobs:
|
||||||
|
<<: *backend-image
|
||||||
|
command: ["worker-jobs"]
|
||||||
|
environment: *backend-env
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/blobs:/data/blobs
|
||||||
|
ulimits:
|
||||||
|
nofile: { soft: 8192, hard: 8192 }
|
||||||
|
depends_on:
|
||||||
|
postgres: { condition: service_healthy }
|
||||||
|
redis: { condition: service_healthy }
|
||||||
|
migrate: { condition: service_completed_successfully }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${WORKER_JOBS_MEM_LIMIT:-640m}
|
||||||
|
cpus: ${WORKER_CPUS:-1.5}
|
||||||
|
stop_grace_period: 30s
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
web:
|
||||||
|
image: ${REGISTRY_PREFIX:-webmail}/web:${IMAGE_TAG:-local}
|
||||||
|
build:
|
||||||
|
context: ..
|
||||||
|
dockerfile: docker/Dockerfile.frontend
|
||||||
|
args:
|
||||||
|
APP_VERSION: ${IMAGE_TAG:-local}
|
||||||
|
GIT_SHA: ${GIT_SHA:-unknown}
|
||||||
|
environment:
|
||||||
|
TZ: ${TZ:-UTC}
|
||||||
|
ports:
|
||||||
|
# The ONLY published port. Bind to a specific NAS interface by setting
|
||||||
|
# WEB_BIND (e.g. 192.168.1.185) to keep it off other networks.
|
||||||
|
- "${WEB_BIND:-0.0.0.0}:${WEB_PORT:-8088}:80"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/healthz | grep -q ok"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
depends_on:
|
||||||
|
api: { condition: service_started }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${WEB_MEM_LIMIT:-96m}
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
# Nightly logical backup of Postgres into ${DATA_ROOT}/backups, pruned to
|
||||||
|
# BACKUP_KEEP_DAYS. Point Hyper Backup / HBS3 at ${DATA_ROOT} and this plus the
|
||||||
|
# blob store is the whole recoverable state (the other half is MASTER_KEY —
|
||||||
|
# see docs/nas-install.md §"The three things that matter").
|
||||||
|
backup:
|
||||||
|
image: postgres:16
|
||||||
|
user: "999:999"
|
||||||
|
entrypoint: ["/bin/bash", "/usr/local/bin/backup.sh"]
|
||||||
|
environment:
|
||||||
|
PGHOST: postgres
|
||||||
|
PGUSER: ${POSTGRES_USER:-webmail}
|
||||||
|
PGPASSWORD: ${POSTGRES_PASSWORD}
|
||||||
|
PGDATABASE: ${POSTGRES_DB:-webmail}
|
||||||
|
BACKUP_DIR: /backups
|
||||||
|
BACKUP_AT: ${BACKUP_AT:-03:30}
|
||||||
|
BACKUP_KEEP_DAYS: ${BACKUP_KEEP_DAYS:-14}
|
||||||
|
TZ: ${TZ:-UTC}
|
||||||
|
volumes:
|
||||||
|
- ${DATA_ROOT}/backups:/backups
|
||||||
|
- ./backup.sh:/usr/local/bin/backup.sh:ro
|
||||||
|
depends_on:
|
||||||
|
postgres: { condition: service_healthy }
|
||||||
|
init: { condition: service_completed_successfully }
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: ${BACKUP_MEM_LIMIT:-256m}
|
||||||
|
logging: *logging
|
||||||
|
|
||||||
|
networks:
|
||||||
|
default:
|
||||||
|
name: webmail-prod
|
||||||
Executable
+62
@@ -0,0 +1,62 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# One image, several roles. Called as: entrypoint <command> [args...]
|
||||||
|
#
|
||||||
|
# api uvicorn, behind the `web` nginx container
|
||||||
|
# worker-sync lease-owned account sync supervisor (SPEC §6.3)
|
||||||
|
# worker-jobs arq consumer (interactive + bulk queues)
|
||||||
|
# worker-all both in one process (small installs; halves RAM use)
|
||||||
|
# migrate core Alembic upgrade + per-plugin install; exits 0 when done
|
||||||
|
# reindex rebuild the search index from Postgres
|
||||||
|
# plugins `python -m app.plugins.cli ...` passthrough
|
||||||
|
# shell /bin/sh (debugging)
|
||||||
|
#
|
||||||
|
# Any other argument is exec'd verbatim, so `docker compose run api python -c ...`
|
||||||
|
# still works.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
: "${API_HOST:=0.0.0.0}"
|
||||||
|
: "${API_PORT:=8000}"
|
||||||
|
: "${API_WORKERS:=1}"
|
||||||
|
# Plugins whose migrations are applied on every `migrate` run. Space-separated so
|
||||||
|
# an operator can add a third-party plugin without editing the image.
|
||||||
|
: "${ENABLED_PLUGINS:=calendar contacts labels filters}"
|
||||||
|
|
||||||
|
cmd="${1:-api}"
|
||||||
|
[ "$#" -gt 0 ] && shift || true
|
||||||
|
|
||||||
|
run_migrations() {
|
||||||
|
echo "[entrypoint] core schema: alembic upgrade head"
|
||||||
|
alembic upgrade head
|
||||||
|
for plugin in $ENABLED_PLUGINS; do
|
||||||
|
echo "[entrypoint] plugin schema: $plugin"
|
||||||
|
python -m app.plugins.cli install "$plugin"
|
||||||
|
done
|
||||||
|
echo "[entrypoint] migrations complete"
|
||||||
|
}
|
||||||
|
|
||||||
|
case "$cmd" in
|
||||||
|
api)
|
||||||
|
# --proxy-headers + --forwarded-allow-ips are what let the app see the
|
||||||
|
# real scheme through nginx/DSM. Without them every request looks like
|
||||||
|
# plain http and the session cookie loses its __Host- prefix and Secure
|
||||||
|
# flag (app/identity/cookies.py:22). The allow-list is the internal
|
||||||
|
# docker network only; this port is never published directly.
|
||||||
|
exec uvicorn app.main:create_app \
|
||||||
|
--factory \
|
||||||
|
--host "$API_HOST" \
|
||||||
|
--port "$API_PORT" \
|
||||||
|
--workers "$API_WORKERS" \
|
||||||
|
--proxy-headers \
|
||||||
|
--forwarded-allow-ips "${FORWARDED_ALLOW_IPS:-*}" \
|
||||||
|
--no-server-header \
|
||||||
|
"$@"
|
||||||
|
;;
|
||||||
|
worker-sync) WORKER_ROLE=sync exec python -m workers.run "$@" ;;
|
||||||
|
worker-jobs) WORKER_ROLE=jobs exec python -m workers.run "$@" ;;
|
||||||
|
worker-all) WORKER_ROLE=all exec python -m workers.run "$@" ;;
|
||||||
|
migrate) run_migrations ;;
|
||||||
|
reindex) exec python scripts/reindex_search.py "$@" ;;
|
||||||
|
plugins) exec python -m app.plugins.cli "$@" ;;
|
||||||
|
shell) exec /bin/sh "$@" ;;
|
||||||
|
*) exec "$cmd" "$@" ;;
|
||||||
|
esac
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
# Webmail front door: static SPA + reverse proxy to the API container.
|
||||||
|
#
|
||||||
|
# This file is dropped into /etc/nginx/conf.d/, i.e. it is already inside the
|
||||||
|
# `http` block — hence the bare `map` below.
|
||||||
|
|
||||||
|
# Preserve the original scheme when something else terminates TLS in front of us
|
||||||
|
# (Synology's Login Portal reverse proxy, QNAP's, Caddy, Cloudflare Tunnel...).
|
||||||
|
# The API only emits Secure/__Host- cookies when it believes the request was
|
||||||
|
# https, so losing this header silently downgrades session cookies.
|
||||||
|
map $http_x_forwarded_proto $fwd_proto {
|
||||||
|
default $scheme;
|
||||||
|
https https;
|
||||||
|
http http;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Connection header for the WebSocket upgrade dance on /api/ws.
|
||||||
|
map $http_upgrade $connection_upgrade {
|
||||||
|
default upgrade;
|
||||||
|
'' close;
|
||||||
|
}
|
||||||
|
|
||||||
|
upstream webmail_api {
|
||||||
|
server api:8000;
|
||||||
|
keepalive 16;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name _;
|
||||||
|
|
||||||
|
root /usr/share/nginx/html;
|
||||||
|
index index.html;
|
||||||
|
|
||||||
|
# Attachment uploads. Keep this comfortably above MAX_ATTACHMENT_MB (default
|
||||||
|
# 25) to leave room for multipart/base64 overhead.
|
||||||
|
client_max_body_size 64m;
|
||||||
|
client_body_timeout 300s;
|
||||||
|
|
||||||
|
# The NAS is doing other jobs too — don't buffer big uploads to disk twice.
|
||||||
|
proxy_request_buffering off;
|
||||||
|
|
||||||
|
gzip on;
|
||||||
|
gzip_vary on;
|
||||||
|
gzip_min_length 1024;
|
||||||
|
gzip_types text/plain text/css application/javascript application/json
|
||||||
|
image/svg+xml application/manifest+json;
|
||||||
|
|
||||||
|
# Clickjacking / sniffing basics. The app's own CSP for mail HTML is applied
|
||||||
|
# server-side in app/indexer/sanitize.py; this is only the shell document.
|
||||||
|
add_header X-Content-Type-Options nosniff always;
|
||||||
|
add_header X-Frame-Options SAMEORIGIN always;
|
||||||
|
add_header Referrer-Policy no-referrer always;
|
||||||
|
|
||||||
|
# ---- API ----------------------------------------------------------------
|
||||||
|
location /api/ {
|
||||||
|
proxy_pass http://webmail_api;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $fwd_proto;
|
||||||
|
proxy_set_header X-Forwarded-Host $host;
|
||||||
|
|
||||||
|
# WebSocket (/api/ws) and any long-poll: no upgrade means no live mail.
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection $connection_upgrade;
|
||||||
|
|
||||||
|
# Live updates hold the connection open; don't cut them at 60 s.
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
proxy_buffering off;
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- static -------------------------------------------------------------
|
||||||
|
# Vite emits content-hashed filenames into /assets, so they are immutable.
|
||||||
|
location /assets/ {
|
||||||
|
expires 1y;
|
||||||
|
add_header Cache-Control "public, immutable";
|
||||||
|
try_files $uri =404;
|
||||||
|
}
|
||||||
|
|
||||||
|
location /fonts/ {
|
||||||
|
expires 30d;
|
||||||
|
add_header Cache-Control "public";
|
||||||
|
try_files $uri =404;
|
||||||
|
}
|
||||||
|
|
||||||
|
location = /healthz {
|
||||||
|
access_log off;
|
||||||
|
return 200 "ok\n";
|
||||||
|
add_header Content-Type text/plain;
|
||||||
|
}
|
||||||
|
|
||||||
|
# SPA fallback: every non-file route renders index.html (react-router).
|
||||||
|
location / {
|
||||||
|
try_files $uri $uri/ /index.html;
|
||||||
|
}
|
||||||
|
|
||||||
|
# index.html itself must never be cached, or a deploy won't be picked up.
|
||||||
|
location = /index.html {
|
||||||
|
add_header Cache-Control "no-store, must-revalidate";
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,354 @@
|
|||||||
|
# Running Webmail on a Synology or QNAP NAS
|
||||||
|
|
||||||
|
This is the operator runbook: how to install it, how to keep your mail safe from
|
||||||
|
anything Docker does to itself, and how to update without holding your breath.
|
||||||
|
It assumes the NAS is doing other work too, so the stack is deliberately modest:
|
||||||
|
**one published port, no host networking, memory caps on every container.**
|
||||||
|
|
||||||
|
Everything here works on any Intel/AMD Docker host — the NAS parts are just the
|
||||||
|
specifics for DSM and QTS.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What gets installed
|
||||||
|
|
||||||
|
| Container | What it does | Published? |
|
||||||
|
|---|---|---|
|
||||||
|
| `web` | nginx: serves the UI, proxies `/api` (incl. the WebSocket) | **yes — the only one** |
|
||||||
|
| `api` | FastAPI app | no, internal network only |
|
||||||
|
| `worker-sync` | keeps IMAP connections open, syncs mail | no |
|
||||||
|
| `worker-jobs` | background jobs (send, backfill, purge, plugin jobs) | no |
|
||||||
|
| `postgres` | the database | no |
|
||||||
|
| `redis` | job queues + sync leases | no |
|
||||||
|
| `backup` | nightly verified database dump | no |
|
||||||
|
| `meilisearch` | optional better search (see §8) | no |
|
||||||
|
| `migrate`, `init` | run once at start, then exit | — |
|
||||||
|
|
||||||
|
Plugins (calendar, contacts, labels, filters) are baked into the backend image
|
||||||
|
and their database schema is applied automatically on every start.
|
||||||
|
|
||||||
|
**Requirements:** an Intel/AMD NAS with Docker (Synology *Container Manager*,
|
||||||
|
QNAP *Container Station*), ~2 GB RAM free, and 10 GB+ of disk for a typical
|
||||||
|
personal mailbox. Nothing is needed from the internet except your mail servers.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Install
|
||||||
|
|
||||||
|
### 2.1 Pick where the data lives
|
||||||
|
|
||||||
|
This is the single most important decision. Use a **shared folder**, never a
|
||||||
|
Docker named volume:
|
||||||
|
|
||||||
|
| NAS | Use |
|
||||||
|
|---|---|
|
||||||
|
| Synology | `/volume1/docker/webmail/data` |
|
||||||
|
| QNAP | `/share/Container/webmail/data` |
|
||||||
|
|
||||||
|
Why: named volumes live inside Docker's internal storage, where Hyper Backup,
|
||||||
|
Snapshot Replication and HBS3 cannot see them, and where "delete the project" in
|
||||||
|
the NAS UI can take your mail with it. A shared folder is backed up by the tools
|
||||||
|
you already run, survives every container operation, and can be copied to
|
||||||
|
another machine wholesale.
|
||||||
|
|
||||||
|
> **Never put this on an NFS or SMB mount.** Postgres on a network filesystem
|
||||||
|
> corrupts. `./deploy/webmail.sh doctor` checks this and refuses to be quiet
|
||||||
|
> about it.
|
||||||
|
|
||||||
|
### 2.2 Get the code onto the NAS
|
||||||
|
|
||||||
|
Enable SSH on the NAS (DSM: *Control Panel → Terminal & SNMP → Enable SSH*;
|
||||||
|
QTS: *Control Panel → Telnet/SSH*), then:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh admin@your-nas
|
||||||
|
sudo -i
|
||||||
|
mkdir -p /volume1/docker/webmail && cd /volume1/docker/webmail
|
||||||
|
git clone <your-gitea-url>/webmail.git src && cd src
|
||||||
|
```
|
||||||
|
|
||||||
|
No git on the NAS? Copy the folder over with File Station or `rsync` instead.
|
||||||
|
|
||||||
|
### 2.3 Configure
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/webmail.sh init
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates `docker/.env.prod` with a freshly generated `MASTER_KEY` and
|
||||||
|
database password. Then open it and set:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
DATA_ROOT=/volume1/docker/webmail/data # or /share/Container/webmail/data
|
||||||
|
APP_DOMAIN=mail.home.lan # your handles become you@this
|
||||||
|
TZ=Europe/Warsaw
|
||||||
|
WEB_PORT=8088 # avoid 5000/5001 (DSM), 8080 (QTS)
|
||||||
|
WEB_BIND=0.0.0.0 # or one NAS IP, e.g. 192.168.1.185
|
||||||
|
```
|
||||||
|
|
||||||
|
> ### Copy `MASTER_KEY` into your password manager. Now.
|
||||||
|
> It encrypts every stored mail-account password. It exists **only** in this
|
||||||
|
> file. A perfect database backup without this key gives you your mail but not
|
||||||
|
> your credentials — you would have to re-enter every account password by hand.
|
||||||
|
> `docker/.env.prod` is `chmod 600` and git-ignored; treat it as a backup
|
||||||
|
> artefact in its own right.
|
||||||
|
|
||||||
|
### 2.4 Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/webmail.sh doctor # sanity-checks config, disk, filesystem, ports
|
||||||
|
./deploy/webmail.sh up # builds the images on first run, then starts
|
||||||
|
```
|
||||||
|
|
||||||
|
First run builds both images. The frontend build wants ~2 GB of RAM: if your NAS
|
||||||
|
is small or you'd rather not tie it up, build on your PC instead — see §6.3.
|
||||||
|
|
||||||
|
Open `http://your-nas:8088/`, create your account, and add a mail account
|
||||||
|
through the wizard.
|
||||||
|
|
||||||
|
### 2.5 Synology Container Manager (UI instead of SSH)
|
||||||
|
|
||||||
|
If you prefer the GUI: *Container Manager → Project → Create*, path
|
||||||
|
`/volume1/docker/webmail/src/docker`, and select `docker-compose.prod.yml`.
|
||||||
|
Container Manager reads a file named exactly `.env` from that folder, so run
|
||||||
|
`cp docker/.env.prod docker/.env` first (and re-copy it whenever you change
|
||||||
|
settings). The SSH route is still the better one — `webmail.sh` gives you the
|
||||||
|
backup, update, health-gate and rollback logic that the UI has no equivalent for.
|
||||||
|
|
||||||
|
### 2.6 Starting automatically after a reboot
|
||||||
|
|
||||||
|
Every service is `restart: unless-stopped`, so Docker brings the stack back on
|
||||||
|
boot by itself. On Synology, make sure Container Manager itself is set to start
|
||||||
|
on boot (it is by default).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The three things that matter
|
||||||
|
|
||||||
|
Containers are disposable. These are not:
|
||||||
|
|
||||||
|
| # | What | Where | If you lose it |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | **Database** | `$DATA_ROOT/postgres` + nightly dumps in `$DATA_ROOT/backups` | Everything: accounts, threads, flags, calendar, contacts |
|
||||||
|
| 2 | **Blob store** | `$DATA_ROOT/blobs` | Mail bodies and attachments (re-downloadable from the servers, slowly) |
|
||||||
|
| 3 | **`MASTER_KEY`** | `docker/.env.prod` + your password manager | Every saved mail-account password, permanently |
|
||||||
|
|
||||||
|
Everything else is derived and needs no backup: Redis holds queues and leases,
|
||||||
|
Meilisearch's index rebuilds from Postgres (§8), and images rebuild from source.
|
||||||
|
|
||||||
|
### Backups
|
||||||
|
|
||||||
|
Two layers, and you want both:
|
||||||
|
|
||||||
|
**Nightly dumps (automatic).** The `backup` container writes a compressed
|
||||||
|
`pg_dump` to `$DATA_ROOT/backups` at `BACKUP_AT` (default 03:30) and keeps
|
||||||
|
`BACKUP_KEEP_DAYS` (14) of them. Every dump is verified with `pg_restore --list`
|
||||||
|
before it counts, so you never find out at restore time that it was truncated.
|
||||||
|
|
||||||
|
**Off-box copies (you set this up, once).** Point your NAS backup tool at
|
||||||
|
`$DATA_ROOT`:
|
||||||
|
|
||||||
|
- Synology: *Hyper Backup* → local/remote/cloud destination → include the
|
||||||
|
`docker/webmail` shared folder. Add *Snapshot Replication* on a Btrfs volume
|
||||||
|
for fast local rollback.
|
||||||
|
- QNAP: *HBS 3* → the `Container/webmail` folder. Add snapshots on QuTS hero.
|
||||||
|
|
||||||
|
A snapshot of a *running* Postgres directory is crash-consistent — it usually
|
||||||
|
replays fine, but it is not a substitute for the dumps, which are verified and
|
||||||
|
restorable into a different Postgres version. Keep both.
|
||||||
|
|
||||||
|
Also back up `docker/.env.prod` somewhere that is not the NAS.
|
||||||
|
|
||||||
|
### Restore drill — do this once, before you need it
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/webmail.sh status # see available dumps
|
||||||
|
./deploy/webmail.sh restore webmail-20260914-033000.dump
|
||||||
|
```
|
||||||
|
|
||||||
|
It stops the app containers, restores in a single transaction (a failure leaves
|
||||||
|
the old database untouched), and starts everything back up.
|
||||||
|
|
||||||
|
Full disaster recovery onto a new box:
|
||||||
|
|
||||||
|
1. Install Docker, copy `$DATA_ROOT` back from your backup.
|
||||||
|
2. Copy the repo and `docker/.env.prod` — **with the original `MASTER_KEY`**.
|
||||||
|
3. `./deploy/webmail.sh up`
|
||||||
|
|
||||||
|
That's it. If the blob store was lost but the database survived, mail bodies
|
||||||
|
refill from the mail servers as accounts re-sync.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Updating
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git pull
|
||||||
|
./deploy/webmail.sh update
|
||||||
|
```
|
||||||
|
|
||||||
|
`update` does, in this order, stopping the moment anything fails:
|
||||||
|
|
||||||
|
1. **Takes a verified database backup.** No backup, no update.
|
||||||
|
2. Builds (or pulls) the new images. A failure here changes nothing.
|
||||||
|
3. Runs migrations as a one-shot container — **before** any service restarts, so
|
||||||
|
a bad migration never leaves new code running against an old schema.
|
||||||
|
4. Rolls workers first, then API, then web. Sync leases transfer to the
|
||||||
|
replacement worker within ~90 s and message upserts are idempotent, so no
|
||||||
|
mail is lost or duplicated by a roll.
|
||||||
|
5. Health-gates for 120 s. **If the new version doesn't come up healthy it rolls
|
||||||
|
itself back automatically** and leaves you on the previous version.
|
||||||
|
|
||||||
|
To go back manually:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/webmail.sh rollback # previous version
|
||||||
|
./deploy/webmail.sh rollback v1.2.0 # a specific one
|
||||||
|
```
|
||||||
|
|
||||||
|
Rolling *code* back does not require rolling the *schema* back, as long as
|
||||||
|
migrations stay expand-then-contract (see `docs/migrations.md` discipline in
|
||||||
|
`deployment-setup-instructions.md` §9). For the rare breaking migration, restore
|
||||||
|
the dump that `update` took in step 1.
|
||||||
|
|
||||||
|
### Building images elsewhere
|
||||||
|
|
||||||
|
Building on the NAS is simple but slow and RAM-hungry. Two alternatives:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On your PC — push to your Gitea registry:
|
||||||
|
REGISTRY_PREFIX=192.168.1.185:8418/claude/webmail ./deploy/build-push.sh v1.0.0
|
||||||
|
# then on the NAS, set REGISTRY_PREFIX + IMAGE_TAG in docker/.env.prod and:
|
||||||
|
./deploy/webmail.sh update v1.0.0
|
||||||
|
|
||||||
|
# Or with no registry at all — build to a file and carry it over:
|
||||||
|
./deploy/build-push.sh v1.0.0 --save
|
||||||
|
# copy webmail-v1.0.0.tar to the NAS, then:
|
||||||
|
docker load -i webmail-v1.0.0.tar && ./deploy/webmail.sh update v1.0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Reaching it from outside the LAN
|
||||||
|
|
||||||
|
### 5.1 Read this first
|
||||||
|
|
||||||
|
**Anyone who can open the page can create an account on it.** Registration is
|
||||||
|
open by design (it's a personal server), so do not put this straight on the
|
||||||
|
public internet. Pick one:
|
||||||
|
|
||||||
|
- **VPN** (Tailscale, WireGuard, or the NAS's own VPN Server) — simplest and
|
||||||
|
safest. The port stays LAN-only.
|
||||||
|
- **NAS reverse proxy with HTTPS**, restricted by firewall rules to the networks
|
||||||
|
you trust.
|
||||||
|
|
||||||
|
### 5.2 HTTPS through the NAS reverse proxy
|
||||||
|
|
||||||
|
DSM owns ports 80/443, which is why the stack publishes 8088. To put DSM's
|
||||||
|
certificate in front of it:
|
||||||
|
|
||||||
|
*Control Panel → Login Portal → Advanced → Reverse Proxy → Create*
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|---|---|
|
||||||
|
| Source protocol / hostname / port | HTTPS / `mail.yourdomain` / 443 |
|
||||||
|
| Destination protocol / hostname / port | HTTP / `localhost` / `8088` |
|
||||||
|
|
||||||
|
Then open **Custom Header → Create → WebSocket** and save. Without that,
|
||||||
|
`/api/ws` is blocked and the UI stops updating live.
|
||||||
|
|
||||||
|
QNAP: *Network & Virtual Switch → Reverse Proxy* — same idea, and likewise
|
||||||
|
enable WebSocket support.
|
||||||
|
|
||||||
|
Session cookies get the `Secure` and `__Host-` treatment automatically as soon
|
||||||
|
as the app sees an HTTPS request: nginx forwards `X-Forwarded-Proto`, and the
|
||||||
|
API is started with `--proxy-headers`. Nothing to configure.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Resource usage
|
||||||
|
|
||||||
|
Steady state for one or two mail accounts, `SEARCH_BACKEND=pg`:
|
||||||
|
|
||||||
|
| | Typical | Cap (`docker/.env.prod`) |
|
||||||
|
|---|---|---|
|
||||||
|
| postgres | 150-300 MB | `PG_MEM_LIMIT=768m` |
|
||||||
|
| api | 150-250 MB | `API_MEM_LIMIT=512m` |
|
||||||
|
| worker-sync | 120-250 MB | `WORKER_SYNC_MEM_LIMIT=512m` |
|
||||||
|
| worker-jobs | 150-350 MB | `WORKER_JOBS_MEM_LIMIT=640m` |
|
||||||
|
| redis / web / backup | < 100 MB total | — |
|
||||||
|
| **total** | **~0.7-1.2 GB** | ~2.2 GB of limits |
|
||||||
|
| meilisearch (optional) | 300-800 MB | `MEILI_MEM_LIMIT=1g` |
|
||||||
|
|
||||||
|
On a 2 GB NAS: stay on `SEARCH_BACKEND=pg` and drop `PG_SHARED_BUFFERS` to
|
||||||
|
`128MB`. Tight but workable. On 4 GB+ you have room for Meilisearch.
|
||||||
|
|
||||||
|
Two NAS-specific realities worth knowing:
|
||||||
|
|
||||||
|
- **Disk hibernation won't happen.** Active accounts hold IMAP IDLE connections
|
||||||
|
open and write as mail arrives. That is the feature working, not a bug.
|
||||||
|
- **Keep NTP on.** Two-factor codes fail on a drifting clock.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Search: Postgres or Meilisearch
|
||||||
|
|
||||||
|
`SEARCH_BACKEND=pg` (default) uses Postgres full-text search. No extra
|
||||||
|
container, no extra RAM, and it is perfectly good for a personal mailbox.
|
||||||
|
|
||||||
|
`SEARCH_BACKEND=meili` adds typo tolerance, better ranking and stays fast on
|
||||||
|
large mailboxes, at the cost of a container and 300-800 MB.
|
||||||
|
|
||||||
|
To switch:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in docker/.env.prod: SEARCH_BACKEND=meili and set MEILI_MASTER_KEY
|
||||||
|
./deploy/webmail.sh up # starts Meilisearch automatically
|
||||||
|
./deploy/webmail.sh reindex # build the index from existing mail
|
||||||
|
```
|
||||||
|
|
||||||
|
The index is **derived data** — every field it holds also lives in Postgres — so
|
||||||
|
it never needs backing up. If the Meilisearch volume is lost, or you switch
|
||||||
|
backends, or search ever looks stale, just rebuild it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/webmail.sh reindex # upsert everything (safe to re-run)
|
||||||
|
./deploy/webmail.sh reindex --purge # drop stale documents first
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Troubleshooting
|
||||||
|
|
||||||
|
| Symptom | Cause | Fix |
|
||||||
|
|---|---|---|
|
||||||
|
| `up` hangs on `postgres` | `DATA_ROOT` on NFS/SMB, or wrong permissions | `./deploy/webmail.sh doctor` |
|
||||||
|
| UI loads, no live updates | WebSocket not enabled in the NAS reverse proxy | §5.2 Custom Header → WebSocket |
|
||||||
|
| Login works, logs out on reload | HTTPS terminated upstream but `X-Forwarded-Proto` stripped | Use the NAS reverse proxy as documented; don't strip headers |
|
||||||
|
| "port is already allocated" | 8088 taken by another app | change `WEB_PORT` |
|
||||||
|
| Mail not syncing | worker stopped, or credentials rejected | `./deploy/webmail.sh logs worker-sync` |
|
||||||
|
| Search finds nothing after a restore | index not rebuilt | `./deploy/webmail.sh reindex --purge` |
|
||||||
|
| App won't start after editing `.env.prod` | malformed `MASTER_KEY` | `./deploy/webmail.sh doctor` |
|
||||||
|
|
||||||
|
Logs: `./deploy/webmail.sh logs [service] -f`. All logs are capped at 10 MB ×3
|
||||||
|
per container, so they can't fill the volume.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Command reference
|
||||||
|
|
||||||
|
```
|
||||||
|
./deploy/webmail.sh init create data dirs + generate secrets
|
||||||
|
./deploy/webmail.sh doctor pre-flight checks
|
||||||
|
./deploy/webmail.sh up [--meili] start everything
|
||||||
|
./deploy/webmail.sh down stop (data untouched)
|
||||||
|
./deploy/webmail.sh status what's running, versions, disk, backups
|
||||||
|
./deploy/webmail.sh logs [svc] [-f] logs
|
||||||
|
./deploy/webmail.sh update [tag] backup -> migrate -> roll -> health-gate
|
||||||
|
./deploy/webmail.sh rollback [tag] back to the previous version
|
||||||
|
./deploy/webmail.sh backup verified dump, right now
|
||||||
|
./deploy/webmail.sh restore <dump> restore a dump (asks for confirmation)
|
||||||
|
./deploy/webmail.sh reindex [--purge] rebuild the search index
|
||||||
|
./deploy/webmail.sh psql database shell
|
||||||
|
./deploy/webmail.sh shell [svc] shell in a container
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user