sync-v2 step 1: worker roles + interactive/bulk queue split

- config: WORKER_ROLE (sync|jobs|all), INTERACTIVE_CONCURRENCY, BULK_CONCURRENCY.
- queue: named queues (interactive/bulk) + enqueue_job() router; sync_bootstrap
  goes to bulk, all other jobs to interactive.
- workers.run: single entrypoint; role=jobs runs both arq consumers (per-queue
  timeout/concurrency), role=sync runs the supervisor, all=both (dev default).
- docker-compose: worker-sync + worker-jobs services (shared image, nofile
  ulimit); backend/Dockerfile; Makefile worker/worker-sync/worker-jobs targets.
- tests repointed to app.queue.get_arq_pool; supervisor behavior unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-18 19:53:31 +02:00
co-authored by Claude Opus 4.8
parent 9cb946f75d
commit e0a45c5a46
18 changed files with 262 additions and 102 deletions
+1 -3
View File
@@ -127,9 +127,7 @@ async def test_draft_messages_are_not_threaded(ctx: Ctx) -> None:
m.set_content("body")
drafts_mb = await ctx.session.scalar(
select(Mailbox).where(
Mailbox.account_id == ctx.account_id, Mailbox.special_use == "drafts"
)
select(Mailbox).where(Mailbox.account_id == ctx.account_id, Mailbox.special_use == "drafts")
)
assert drafts_mb is not None
remote = RemoteMessage(
+2 -2
View File
@@ -16,7 +16,7 @@ from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
from app.api import compose as compose_api
from app import queue as app_queue
from app.config import get_settings
from app.db import session_scope
from app.identity.passwords import hash_password
@@ -181,7 +181,7 @@ async def ctx(monkeypatch) -> AsyncIterator[Ctx]: # type: ignore[no-untyped-def
async def fake_pool() -> FakePool:
return FakePool()
monkeypatch.setattr(compose_api, "get_arq_pool", fake_pool)
monkeypatch.setattr(app_queue, "get_arq_pool", fake_pool)
token = await create_session(user.id, "pytest")
c.client = httpx.AsyncClient(