Deploy: fix the initdb race that made migrate time out

On a real Synology first run, `migrate` exited 1 with a bare asyncpg
TimeoutError while the postgres log showed a perfectly healthy server.

The healthcheck was `pg_isready` with no -h, so it checked the UNIX SOCKET.
During first-time initialisation the postgres image runs a temporary server
that listens on the socket only (listen_addresses=''), so pg_isready answered
and the container was marked healthy while nothing was listening on TCP yet.
Dependents then connected to postgres:5432, hung, and timed out. It only bites
on the very first start of a fresh data directory, which is exactly the run an
operator is doing when they have least idea what is wrong.

- healthcheck uses `pg_isready -h 127.0.0.1 -p 5432`, so health means TCP is
  actually accepting.
- `migrate` additionally waits for the port itself (DB_WAIT_SECONDS, default
  180) before running alembic. It is a one-shot with no restart policy, so a
  few seconds of slow NAS disk should not fail the whole bring-up; the API and
  workers already self-heal through their restart policy.

Verified with a first-time bring-up against a fresh data directory: migrate
logs "database accepting connections", applies core + all four plugin
branches, and the stack comes up serving.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nux3LCu4zP9nih4BRhPyHw
This commit is contained in:
2026-09-14 20:31:03 +02:00
co-authored by Claude Opus 5
parent d74e1f97b2
commit 8f96a1ecae
2 changed files with 34 additions and 1 deletions
+6 -1
View File
@@ -136,7 +136,12 @@ services:
volumes: volumes:
- ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/postgres:/var/lib/postgresql/data - ${DATA_ROOT:-/NO-.env-FOUND-RUN-deploy_webmail.sh-init-FIRST}/postgres:/var/lib/postgresql/data
healthcheck: healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-webmail} -d ${POSTGRES_DB:-webmail}"] # -h 127.0.0.1 forces a TCP check. Without it pg_isready uses the Unix
# socket, which the temporary server that the postgres image runs during
# first-time initdb DOES answer (it listens on the socket only) — so the
# container went healthy before TCP was up, and dependents connecting to
# postgres:5432 hung until they timed out.
test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -p 5432 -U ${POSTGRES_USER:-webmail} -d ${POSTGRES_DB:-webmail}"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 12 retries: 12
+28
View File
@@ -24,7 +24,35 @@ set -eu
cmd="${1:-api}" cmd="${1:-api}"
[ "$#" -gt 0 ] && shift || true [ "$#" -gt 0 ] && shift || true
# Belt and braces on top of the compose healthcheck: a NAS with slow disks can
# take a while over initdb, and `migrate` is a one-shot with no restart policy —
# if it starts a moment early it fails the whole bring-up.
wait_for_db() {
python - <<'PYEOF'
import os, socket, sys, time
from urllib.parse import urlparse
url = os.environ.get("DATABASE_URL", "")
parsed = urlparse(url.replace("+asyncpg", ""))
host = parsed.hostname or "postgres"
port = parsed.port or 5432
deadline = time.time() + float(os.environ.get("DB_WAIT_SECONDS", "180"))
while time.time() < deadline:
try:
with socket.create_connection((host, port), timeout=3):
print(f"[entrypoint] database accepting connections at {host}:{port}")
sys.exit(0)
except OSError:
time.sleep(2)
print(f"[entrypoint] ERROR: {host}:{port} never accepted a connection", file=sys.stderr)
sys.exit(1)
PYEOF
}
run_migrations() { run_migrations() {
wait_for_db
echo "[entrypoint] core schema: alembic upgrade head" echo "[entrypoint] core schema: alembic upgrade head"
alembic upgrade head alembic upgrade head
for plugin in $ENABLED_PLUGINS; do for plugin in $ENABLED_PLUGINS; do