dev-environment-setup.md

# Development Environment — Agent Setup Instructions

Companion to `SPEC.md` (webmail-client-spec). This file tells the implementing agent exactly how to build and verify the development environment **before writing any application code**. Approach: **conda for the toolchain** (Python 3.11, Node 20), **Docker for stateful services** (PostgreSQL, Redis, Meilisearch, test mail servers) — identical images in dev and CI.

Rules:
- Do not install anything into the host system Python/apt beyond verifying prerequisites. All language tooling lives in the conda env; all services live in containers.
- Do not substitute versions or images. If something is unavailable, stop and record it in `docs/open-questions.md`.
- Every step below has a verification command. A step is not done until its verification passes.

---

## 1. Prerequisites (verify, don't install blindly)

```bash
docker --version          # need >= 24.x
docker compose version    # need v2 plugin
conda --version           # need conda >= 23.x from Miniforge
```

- If conda is missing, install **Miniforge** (NOT Anaconda — conda-forge channel only, avoids `defaults`-channel licensing):
  `curl -fsSL https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh -o /tmp/miniforge.sh && bash /tmp/miniforge.sh -b && ~/miniforge3/bin/conda init bash`
- If Docker is missing, halt and report — Docker installation is a host-admin decision, not the agent's.
- Verify the host user can run Docker without sudo: `docker run --rm hello-world`.

## 2. Conda environment (toolchain only — no services here)

Create `environment.yml` at repo root:

```yaml
name: webmail
channels:
  - conda-forge
dependencies:
  - python=3.11
  - nodejs=20
  - pip
```

```bash
conda env create -f environment.yml
conda activate webmail
python --version   # must print 3.11.x
node --version     # must print v20.x
```

Install backend tooling and deps (after `backend/pyproject.toml` exists per SPEC §3):

```bash
pip install -e "./backend[dev]"    # dev extra: ruff, mypy, pytest, pytest-asyncio, respx
cd frontend && npm ci && cd ..
```

**Auto-config on activate** — create `$CONDA_PREFIX/etc/conda/activate.d/webmail.sh`:

```bash
#!/usr/bin/env bash
set -a
[ -f "$(git rev-parse --show-toplevel 2>/dev/null)/.env.dev" ] && . "$(git rev-parse --show-toplevel)/.env.dev"
set +a
```

## 3. Service containers

Create `docker/docker-compose.yml`. Host ports are deliberately **non-default** (5433/6380) so they never collide with anything already running on the host — do not change them.

```yaml
name: webmail
services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: webmail
      POSTGRES_PASSWORD: webmail_dev
      POSTGRES_DB: webmail_dev
    ports: ["127.0.0.1:5433:5432"]
    volumes: [pgdata:/var/lib/postgresql/data]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U webmail -d webmail_dev"]
      interval: 5s
      timeout: 3s
      retries: 10

  redis:
    image: redis:7
    ports: ["127.0.0.1:6380:6379"]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 10

  meilisearch:
    image: getmeili/meilisearch:v1.8
    environment:
      MEILI_MASTER_KEY: dev_master_key_change_me
      MEILI_ENV: development
    ports: ["127.0.0.1:7700:7700"]
    volumes: [meilidata:/meili_data]
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:7700/health | grep -q available"]
      interval: 5s
      timeout: 3s
      retries: 10

  # Test mail servers (integration tests + manual dev against a throwaway mailbox)
  greenmail:
    image: greenmail/standalone:2.0.1
    environment:
      GREENMAIL_OPTS: >-
        -Dgreenmail.setup.test.all -Dgreenmail.hostname=0.0.0.0
        -Dgreenmail.users=dev:devpass@dev.local -Dgreenmail.verbose
    ports:
      - "127.0.0.1:3025:3025"   # SMTP
      - "127.0.0.1:3143:3143"   # IMAP
      - "127.0.0.1:3993:3993"   # IMAPS

volumes:
  pgdata:
  meilidata:
```

```bash
docker compose -f docker/docker-compose.yml up -d postgres redis meilisearch
docker compose -f docker/docker-compose.yml ps   # all must be "healthy"
```

Greenmail is started on demand by integration tests and for manual testing (`--profile` it later if noise bothers; keep the definition here).

## 4. Environment file

Create `.env.dev` at repo root (add to `.gitignore`; commit a `.env.example` copy with the same content — these are dev-only credentials, still never reuse them anywhere):

```bash
APP_DOMAIN=dev.webmail.local
DATABASE_URL=postgresql+asyncpg://webmail:webmail_dev@127.0.0.1:5433/webmail_dev
REDIS_URL=redis://127.0.0.1:6380/0
SEARCH_BACKEND=pg                    # switch to meili in M2 work
MEILI_URL=http://127.0.0.1:7700
MEILI_KEY=dev_master_key_change_me
MASTER_KEY=$(python -c "import os,base64;print(base64.b64encode(os.urandom(32)).decode())")
BLOB_ROOT=./var/blobs
UNDO_SECONDS=10
POLL_INTERVAL=120
SESSION_TTL_DAYS=30
MAX_ATTACHMENT_MB=25
LOG_LEVEL=DEBUG
```

Generate `MASTER_KEY` once and paste the literal value — do not leave the `$(...)` in the file. `mkdir -p var/blobs`.

## 5. Initialize and run

```bash
conda activate webmail                       # loads .env.dev via activate hook
cd backend && alembic upgrade head && cd ..  # migrations against 5433

# Terminal 1 — API
uvicorn app.main:create_app --factory --reload --port 8000 --app-dir backend

# Terminal 2 — worker (sync supervisor + arq job consumers in one process)
cd backend && WORKER_ROLE=all python -m workers.run

# Terminal 3 — frontend
cd frontend && npm run dev                   # Vite on 5173, proxies /api -> 8000
```

Vite dev proxy (in `vite.config.ts`): `/api` → `http://127.0.0.1:8000` with `ws: true` for `/api/ws`.

## 6. Verification checklist (run in order; all must pass)

```bash
docker compose -f docker/docker-compose.yml ps          # postgres, redis, meilisearch healthy
psql postgresql://webmail:webmail_dev@127.0.0.1:5433/webmail_dev -c 'select 1'
redis-cli -p 6380 ping                                   # PONG (via docker exec if no local cli)
curl -s http://127.0.0.1:7700/health                     # {"status":"available"}
curl -s http://127.0.0.1:8000/api/healthz                # implement: checks DB + Redis, returns 200
cd backend && ruff check . && mypy --strict app && pytest -q
cd frontend && npx tsc --noEmit && npm run lint && npx vitest run
```

Integration smoke (once M0 sync exists): start greenmail, configure account `dev@dev.local` / `devpass` against `127.0.0.1:3143` (plain, dev-only), send a message into Greenmail via SMTP 3025, assert it appears via the API within 10 s.

## 7. Day-to-day commands (put these in a root `Makefile`)

```make
up:        ## start services
	docker compose -f docker/docker-compose.yml up -d postgres redis meilisearch
down:      ## stop services (data kept)
	docker compose -f docker/docker-compose.yml down
reset-db:  ## DESTRUCTIVE: wipe db + search + blobs, re-migrate
	docker compose -f docker/docker-compose.yml down -v
	rm -rf var/blobs && mkdir -p var/blobs
	docker compose -f docker/docker-compose.yml up -d --wait postgres redis meilisearch
	cd backend && alembic upgrade head
api:
	uvicorn app.main:create_app --factory --reload --port 8000 --app-dir backend
worker:      ## run both sync + jobs in one process (dev)
	cd backend && WORKER_ROLE=all python -m workers.run
worker-sync: ## run only the account-sync supervisor
	cd backend && WORKER_ROLE=sync python -m workers.run
worker-jobs: ## run only the arq job consumers (interactive + bulk)
	cd backend && WORKER_ROLE=jobs python -m workers.run
web:
	cd frontend && npm run dev
test:
	cd backend && pytest -q && cd ../frontend && npx vitest run
mail-test:  ## start throwaway IMAP/SMTP server
	docker compose -f docker/docker-compose.yml up -d greenmail
```

## 8. CI parity

`docker/docker-compose.ci.yml` extends the same file: adds Dovecot (`dovecot/dovecot:2.3`) for provider-compat tests and runs Playwright against a production build. CI must use the same image tags as this file — when bumping a service version, bump it in exactly one place (this compose file) and update SPEC §3 in the same commit.

## 9. Troubleshooting

- **Port already in use** — something on the host occupies 5433/6380/7700/8000/5173. Do not change this file's ports without recording it in `docs/open-questions.md`; prefer stopping the conflicting service.
- **`alembic upgrade` connection refused** — services not healthy yet; `docker compose ... up -d --wait`.
- **mypy/ruff not found** — you're outside the conda env; `conda activate webmail`.
- **Frontend can't reach WS** — check the Vite proxy `ws: true` and that the session cookie is set (login first).
- **Blob permission errors** — `BLOB_ROOT` must be inside the repo (`./var/blobs`), never a system path.

## 10. Testing

Use: test@imaginary-pixels.com account for testing. Login: test@imaginary-pixels.com, password: webmail!@A
and: test@tank-mouse.com. Login: test@tank-mouse.com, password: webmail!@A
SMTP: serwer2295308.home.pl; Port: 587  Port SSL: 465
IMAP: serwer2295308.home.pl; Port: 143  Port SSL: 993