lukas/webmail-client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sync-v2 step 1: worker roles + interactive/bulk queue split

Browse Source 
- config: WORKER_ROLE (sync|jobs|all), INTERACTIVE_CONCURRENCY, BULK_CONCURRENCY.
- queue: named queues (interactive/bulk) + enqueue_job() router; sync_bootstrap
  goes to bulk, all other jobs to interactive.
- workers.run: single entrypoint; role=jobs runs both arq consumers (per-queue
  timeout/concurrency), role=sync runs the supervisor, all=both (dev default).
- docker-compose: worker-sync + worker-jobs services (shared image, nofile
  ulimit); backend/Dockerfile; Makefile worker/worker-sync/worker-jobs targets.
- tests repointed to app.queue.get_arq_pool; supervisor behavior unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
lukas
2026-07-18 19:53:31 +02:00
co-authored by Claude Opus 4.8
parent 9cb946f75d
commit e0a45c5a46
18 changed files with 262 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/config.py
+7
View File
@@ -34,6 +34,13 @@ class Settings(BaseSettings):
max_attachment_mb: int = Field(default=25, alias="MAX_ATTACHMENT_MB")
log_level: str = Field(default="INFO", alias="LOG_LEVEL")
# --- Sync engine v2 (workers layer) --------------------------------------
# Worker role: "sync" (lease-owned account runners) | "jobs" (arq consumer)
# | "all" (both, single-process dev default).
worker_role: str = Field(default="all", alias="WORKER_ROLE")
interactive_concurrency: int = Field(default=20, alias="INTERACTIVE_CONCURRENCY")
bulk_concurrency: int = Field(default=2, alias="BULK_CONCURRENCY")
oauth_google_client_id: str = Field(default="", alias="OAUTH_GOOGLE_CLIENT_ID")
oauth_google_client_secret: str = Field(default="", alias="OAUTH_GOOGLE_CLIENT_SECRET")
oauth_ms_client_id: str = Field(default="", alias="OAUTH_MS_CLIENT_ID")