lukasandClaude Opus 4.8
6d83f7cde8
- Heartbeat JSON now carries {owned, sockets_open, tiers:{hot,warm,cold}, ts} per
§2 so peers and operators can see each worker's load and socket footprint.
- run.py runs the sync coordinator directly (not via a placeholder arq worker) so
SIGINT/SIGTERM can stop the claimer, cancel runners, and release every lease
explicitly within a 10s drain budget — peers adopt the accounts immediately
instead of waiting out the lease TTL. Removed the sync:control-queue hack and
the supervisor indirection (start/stop_supervisor, sync_on_startup/shutdown).
- Acceptance: graceful-shutdown test proves a peer adopts released accounts within
seconds under a 30s TTL (i.e. via release, not expiry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
50 lines
1.6 KiB
Python
50 lines
1.6 KiB
Python
"""Worker job callables and queue function-sets (refactor §1, §6).
|
|
|
|
The process is launched via ``workers.run`` (``WORKER_ROLE`` decides which arq
|
|
workers run). This module only defines the job callables and the two queue
|
|
function-sets; the runner wires them onto the ``interactive`` / ``bulk`` queues.
|
|
The sync role runs the coordinator directly (see ``workers.run``), not via arq.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import uuid
|
|
from typing import Any
|
|
|
|
from app.outbox.draft_sync import push_draft_to_imap_job, remove_imap_draft_job
|
|
from app.outbox.jobs import send_job
|
|
from app.sync.actions import (
|
|
delete_thread_job,
|
|
move_job,
|
|
move_thread_to_folder_job,
|
|
push_flags_job,
|
|
)
|
|
from app.sync.engine import bootstrap_and_backfill, refresh_account_job
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
async def sync_bootstrap_account(ctx: dict[str, Any], account_id: str) -> int:
|
|
"""Bootstrap + backfill an account (enqueued on account creation)."""
|
|
logger.info("sync.bootstrap_account start account=%s", account_id)
|
|
count = await bootstrap_and_backfill(uuid.UUID(account_id))
|
|
logger.info("sync.bootstrap_account done account=%s messages=%d", account_id, count)
|
|
return count
|
|
|
|
|
|
# Fast, interactive user actions (§6): short timeout, high concurrency.
|
|
INTERACTIVE_FUNCTIONS = [
|
|
push_flags_job,
|
|
move_job,
|
|
move_thread_to_folder_job,
|
|
delete_thread_job,
|
|
send_job,
|
|
push_draft_to_imap_job,
|
|
remove_imap_draft_job,
|
|
refresh_account_job,
|
|
]
|
|
|
|
# Bulk work (§6): long timeout, low concurrency, isolated from interactive jobs.
|
|
BULK_FUNCTIONS = [sync_bootstrap_account]
|