From 8f96a1ecae35766a829d464097213d2e228bb613 Mon Sep 17 00:00:00 2001 From: Lukas Remis Date: Mon, 14 Sep 2026 20:31:03 +0200 Subject: [PATCH] Deploy: fix the initdb race that made migrate time out On a real Synology first run, `migrate` exited 1 with a bare asyncpg TimeoutError while the postgres log showed a perfectly healthy server. The healthcheck was `pg_isready` with no -h, so it checked the UNIX SOCKET. During first-time initialisation the postgres image runs a temporary server that listens on the socket only (listen_addresses=''), so pg_isready answered and the container was marked healthy while nothing was listening on TCP yet. Dependents then connected to postgres:5432, hung, and timed out. It only bites on the very first start of a fresh data directory, which is exactly the run an operator is doing when they have least idea what is wrong. - healthcheck uses `pg_isready -h 127.0.0.1 -p 5432`, so health means TCP is actually accepting. - `migrate` additionally waits for the port itself (DB_WAIT_SECONDS, default 180) before running alembic. It is a one-shot with no restart policy, so a few seconds of slow NAS disk should not fail the whole bring-up; the API and workers already self-heal through their restart policy. Verified with a first-time bring-up against a fresh data directory: migrate logs "database accepting connections", applies core + all four plugin branches, and the stack comes up serving. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Nux3LCu4zP9nih4BRhPyHw --- docker-compose.yml | 7 ++++++- docker/entrypoint.sh | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/docker-compose.yml b/docker-compose.yml index f78e79a..e367058 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -136,7 +136,12 @@ services: volumes: - ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/postgres:/var/lib/postgresql/data healthcheck: - test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-webmail} -d ${POSTGRES_DB:-webmail}"] + # -h 127.0.0.1 forces a TCP check. Without it pg_isready uses the Unix + # socket, which the temporary server that the postgres image runs during + # first-time initdb DOES answer (it listens on the socket only) — so the + # container went healthy before TCP was up, and dependents connecting to + # postgres:5432 hung until they timed out. + test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -p 5432 -U ${POSTGRES_USER:-webmail} -d ${POSTGRES_DB:-webmail}"] interval: 10s timeout: 5s retries: 12 diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh index 1809be1..f233433 100755 --- a/docker/entrypoint.sh +++ b/docker/entrypoint.sh @@ -24,7 +24,35 @@ set -eu cmd="${1:-api}" [ "$#" -gt 0 ] && shift || true +# Belt and braces on top of the compose healthcheck: a NAS with slow disks can +# take a while over initdb, and `migrate` is a one-shot with no restart policy — +# if it starts a moment early it fails the whole bring-up. +wait_for_db() { + python - <<'PYEOF' +import os, socket, sys, time +from urllib.parse import urlparse + +url = os.environ.get("DATABASE_URL", "") +parsed = urlparse(url.replace("+asyncpg", "")) +host = parsed.hostname or "postgres" +port = parsed.port or 5432 +deadline = time.time() + float(os.environ.get("DB_WAIT_SECONDS", "180")) + +while time.time() < deadline: + try: + with socket.create_connection((host, port), timeout=3): + print(f"[entrypoint] database accepting connections at {host}:{port}") + sys.exit(0) + except OSError: + time.sleep(2) + +print(f"[entrypoint] ERROR: {host}:{port} never accepted a connection", file=sys.stderr) +sys.exit(1) +PYEOF +} + run_migrations() { + wait_for_db echo "[entrypoint] core schema: alembic upgrade head" alembic upgrade head for plugin in $ENABLED_PLUGINS; do