sync-v2 step 1: worker roles + interactive/bulk queue split
- config: WORKER_ROLE (sync|jobs|all), INTERACTIVE_CONCURRENCY, BULK_CONCURRENCY. - queue: named queues (interactive/bulk) + enqueue_job() router; sync_bootstrap goes to bulk, all other jobs to interactive. - workers.run: single entrypoint; role=jobs runs both arq consumers (per-queue timeout/concurrency), role=sync runs the supervisor, all=both (dev default). - docker-compose: worker-sync + worker-jobs services (shared image, nofile ulimit); backend/Dockerfile; Makefile worker/worker-sync/worker-jobs targets. - tests repointed to app.queue.get_arq_pool; supervisor behavior unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,16 @@
|
||||
# Shared image for the worker roles (refactor §1). Built from backend/.
|
||||
FROM python:3.11-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install deps first for layer caching.
|
||||
COPY pyproject.toml ./
|
||||
COPY app ./app
|
||||
COPY workers ./workers
|
||||
COPY migrations ./migrations
|
||||
COPY alembic.ini ./alembic.ini
|
||||
RUN pip install --no-cache-dir .
|
||||
|
||||
# Default to the combined role; docker-compose overrides WORKER_ROLE per service.
|
||||
ENV WORKER_ROLE=all
|
||||
CMD ["python", "-m", "workers.run"]
|
||||
Reference in New Issue
Block a user