sync-v2 step 5: activity tiers (hot/warm/cold)

Each owned account is tiered by the coordinator; the runner shapes its work to
match — correctness identical, only latency/socket cost differ.

- hot: persistent INBOX IDLE + secondary poll every POLL_WARM.
- warm: no persistent socket; transient INBOX+special poll every POLL_WARM.
- cold: same transient poll every POLL_COLD.
- engine: poll_all_loop (transient full poll); secondary_poll_loop interval is
  now a parameter; POLL_INTERVAL removed in favor of POLL_WARM/POLL_COLD.
- coordinator: tier eval every 120s from session heat (sess:active:{user}) +
  recent mail (max internal_date within TIER_HOT_ACTIVITY_H), with the
  HOT_BUDGET_PER_WORKER guard shedding least-recently-active session-less hot
  accounts; user.active event promotes a user's accounts to hot at once (≤5s).
- API: login + WS connect/ping publish user.active and refresh sess:active
  (TTL TIER_SESSION_GRACE_MIN). New sync:tier:{id} key, default warm on claim.
- config: POLL_WARM/POLL_COLD/TIER_*/HOT_BUDGET_PER_WORKER.
- tests: 5 tier tests (session/recent-mail promotion, cold, user.active,
  budget demotion) alongside the lease/lifecycle acceptance tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-19 07:03:54 +02:00
co-authored by Claude Opus 4.8
parent 245672d3f2
commit 19335116da
9 changed files with 365 additions and 47 deletions
+26 -6
View File
@@ -255,7 +255,7 @@ async def live_loop(
from app.db import session_scope
stop = stop or asyncio.Event()
poll_interval = get_settings().poll_interval
idle_fallback = get_settings().poll_warm
async with session_scope() as session:
account = await _load_account(session, account_id)
@@ -267,7 +267,7 @@ async def live_loop(
try:
await provider.idle(mailbox_path)
except ProviderNotSupported:
await asyncio.sleep(poll_interval)
await asyncio.sleep(idle_fallback)
if stop.is_set():
break
async with session_scope() as session:
@@ -341,19 +341,39 @@ async def refresh_account_job(ctx: dict[str, Any], account_id: str) -> None:
await poll_folders_once(uuid.UUID(account_id), include_inbox=True)
async def secondary_poll_loop(account_id: uuid.UUID, stop: asyncio.Event | None = None) -> None:
async def secondary_poll_loop(
account_id: uuid.UUID, stop: asyncio.Event | None = None, interval: float | None = None
) -> None:
"""Periodically CONDSTORE-poll non-INBOX folders (Sent/Drafts/Archive/Trash/Junk).
INBOX has its own IDLE loop (real-time). This keeps the other folders in step
with server-side changes (e.g. Gmail-filed sent mail) and changes made by other
clients. CONDSTORE/QRESYNC makes each pass a cheap MODSEQ delta.
clients. CONDSTORE/QRESYNC makes each pass a cheap MODSEQ delta. Used by hot-tier
runners alongside the INBOX IDLE loop.
"""
stop = stop or asyncio.Event()
interval = get_settings().poll_interval
period = interval if interval is not None else get_settings().poll_warm
while not stop.is_set():
# Wait first: the initial state was already covered by the backfill.
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(stop.wait(), timeout=period)
if stop.is_set():
break
await _poll_secondary_once(account_id)
async def poll_all_loop(
account_id: uuid.UUID, interval: float, stop: asyncio.Event | None = None
) -> None:
"""Transient CONDSTORE poll of INBOX + special folders every ``interval`` seconds.
The warm/cold tiers use this instead of a persistent IDLE connection: no socket
is held between passes. Correctness matches the hot tier — only latency differs.
"""
stop = stop or asyncio.Event()
while not stop.is_set():
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(stop.wait(), timeout=interval)
if stop.is_set():
break
await _poll_secondary_once(account_id)
await poll_folders_once(account_id, include_inbox=True)