sync-v2 step 7: observability + graceful shutdown (§7)

- Heartbeat JSON now carries {owned, sockets_open, tiers:{hot,warm,cold}, ts} per
  §2 so peers and operators can see each worker's load and socket footprint.
- run.py runs the sync coordinator directly (not via a placeholder arq worker) so
  SIGINT/SIGTERM can stop the claimer, cancel runners, and release every lease
  explicitly within a 10s drain budget — peers adopt the accounts immediately
  instead of waiting out the lease TTL. Removed the sync:control-queue hack and
  the supervisor indirection (start/stop_supervisor, sync_on_startup/shutdown).
- Acceptance: graceful-shutdown test proves a peer adopts released accounts within
  seconds under a 30s TTL (i.e. via release, not expiry).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-19 07:18:19 +02:00
co-authored by Claude Opus 4.8
parent 9dbc5ed998
commit 6d83f7cde8
5 changed files with 80 additions and 93 deletions
+2 -17
View File
@@ -1,8 +1,9 @@
"""Worker job functions and the sync-worker lifecycle (refactor §1, §6).
"""Worker job callables and queue function-sets (refactor §1, §6).
The process is launched via ``workers.run`` (``WORKER_ROLE`` decides which arq
workers run). This module only defines the job callables and the two queue
function-sets; the runner wires them onto the ``interactive`` / ``bulk`` queues.
The sync role runs the coordinator directly (see ``workers.run``), not via arq.
"""
from __future__ import annotations
@@ -46,19 +47,3 @@ INTERACTIVE_FUNCTIONS = [
# Bulk work (§6): long timeout, low concurrency, isolated from interactive jobs.
BULK_FUNCTIONS = [sync_bootstrap_account]
async def sync_on_startup(ctx: dict[str, Any]) -> None:
"""Sync-worker startup: launch the account-sync supervisor."""
from app.sync.supervisor import start_supervisor
await start_supervisor(ctx)
logger.info("sync worker: supervisor started")
async def sync_on_shutdown(ctx: dict[str, Any]) -> None:
"""Sync-worker shutdown: stop the supervisor (process cleanup is in the runner)."""
from app.sync.supervisor import stop_supervisor
await stop_supervisor(ctx)
logger.info("sync worker: supervisor stopped")