- Invitations: each user can invite two friends by email (sign-up works
through the link even while registration is closed); admin-editable
template, subject and sender.
- Invitation sender defaults to the system-mail From address, which the
SMTP account is allowed to send as (was postmaster@APP_DOMAIN, refused
with 553 when the two differ).
- Sign-in, sign-up and forgot-password fields show the fixed
@domain after the typed name; the registration status reports it.
- System mail gets a Date header; the invitation's plain-text part keeps
its paragraphs and line breaks.
- SMTP EHLO uses SMTP_HELO_NAME / APP_DOMAIN, not the container name.
- DNS recommendations for the mail domains.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Security (ultimo-security-report.md):
- Security headers on every response, the app shell included: they lived at
server level and nginx dropped them in any location with its own header.
CSP, framing, nosniff, referrer, permissions (docker/nginx-security.conf).
- Plain HTTP forwarded by the reverse proxy is redirected to HTTPS; HSTS on
HTTPS responses. server_tokens off.
- Cross-site state-changing /api requests refused (login CSRF), WebSocket too.
- Validation errors in the usual error shape, without the request model.
- New-password policy: max 128, common/own-address/repeated refused,
breached-password check (HIBP k-anonymity, fails open).
- Message HTML: links keep only http(s)/mailto (sanitizer v6); the HTML
endpoint carries its own no-script CSP.
- Outbound fetches pinned to the checked IP (DNS rebinding); external
IMAP/SMTP servers re-checked before each connection; CGNAT and mapped
addresses counted as non-public.
Fixes from testing:
- IDLE loop catches up before idling: mail that came while the worker was
away no longer waits up to 10 minutes.
- Restore from Spam/Trash/Archive moves only the removed messages, not your
reply in Sent.
- Narrow desktop list stacks sender/subject/snippet (subject was squeezed
to nothing at the default width), with hover actions.
- Phone: wide emails scale to fit the screen.
- Singular forms ("1 message"); no zip button for a single attachment;
app passwords no longer claim "never used" for mail apps.
- ruff format across the backend.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosted mailboxes (SPEC App. D), behind HOSTED_MAILBOXES (off: unchanged):
- Stalwart mail server, provisioned by `python -m app.hosted.provision`;
directory read from a Postgres view, app passwords only for mail apps
- native account per user, the worker signs in as master for the user
- raw messages of hosted mailboxes only cached, fetched back on demand
- migrations 0025 (accounts.kind, directory view), 0026 (IMAP ids bigint)
Mail:
- dark message mode switch in the reading pane (Thunderbird's algorithm)
- moves and folder creation quote the folder name ("Deleted Items"):
swipe-to-delete / Trash no longer bounce back
- thread unread counters: lock before recount, sorted lock order,
UIDVALIDITY re-import recounts, nightly repair_threads
- sidebar accounts sorted alphabetically, more space above headings
Settings: Security headings renamed; antispam accounts are explicit
(none ticked = none filtered), input widths, Activity hidden.
Plugin bundles served with Cache-Control: no-cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A self-hosted LanguageTool (dev: docker/docker-compose.yml, started by
`make up`) checks what the user writes in the compose window, through
POST /api/spellcheck — the browser never talks to the engine directly.
- Marks are CSS highlights over the text, never written into the message: red
wavy for spelling, blue for grammar/style. Clicking one shows the message,
up to five suggestions (applied as typed, so Ctrl+Z undoes them) and Ignore.
The popup, like the link popup, opens above its anchor when there is no
room below.
- Checked after a pause in typing; the signature and quoted original are
skipped; marks on words just edited drop at once; a check overtaken by a
newer one, by turning checking off, or by closing the window is discarded.
- With "auto", text is checked in chunks of lines, each in its own detected
language — LanguageTool settles on one language per request, so a mixed
Polish/English message had every English word flagged as a Polish typo.
- Settings > Appearance: on/off and language (Auto-detect or any of the
engine's languages); the compose window's More menu overrides it for one
message. The browser's own spellcheck steps aside while ours runs, and comes
back when it is off or the server has no engine (503 from the endpoint).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything needed to run the app on an Intel NAS that is also doing other
jobs: one published port, no host networking, memory/CPU caps and log
rotation on every container, and all state in bind mounts under DATA_ROOT
so mail survives image updates, project deletion in the NAS UI, and full
reinstalls — and so Hyper Backup/HBS3 can actually see it.
- docker/Dockerfile.backend: multi-stage, non-root (uid 10001), tini for
the <10s graceful lease release. Installs the four plugins as entry-point
distributions. One image, several roles via docker/entrypoint.sh
(api | worker-sync | worker-jobs | migrate | reindex | plugins).
- docker/Dockerfile.frontend + nginx.conf: SPA + /api reverse proxy with
the WebSocket upgrade for /api/ws, and X-Forwarded-Proto preserved so
the API still emits __Host-/Secure cookies when TLS is terminated by
the NAS reverse proxy (identity/cookies.py keys off request scheme).
- docker/docker-compose.prod.yml: project name webmail-prod, NOT webmail —
docker-compose.yml (dev services) owns that name, and two compose files
sharing one project name fight over each other's containers.
Meilisearch is an opt-in profile; SEARCH_BACKEND=pg is the default so a
small NAS pays nothing for search.
- migrate runs as a one-shot (core Alembic head + every plugin branch)
that everything else waits on, so a bad migration can never leave new
code running against an old schema.
- docker/backup.sh: nightly pg_dump verified with `pg_restore --list`
before it counts as a backup, with retention and a baseline on first
start.
- deploy/webmail.sh: init/doctor/up/down/status/logs/update/rollback/
backup/restore/reindex/psql/shell. `update` is backup -> images ->
migrate -> roll workers, api, web -> health-gate, with automatic
rollback to the previous version if the gate fails. `doctor` refuses to
be quiet when DATA_ROOT is on NFS/SMB (silent Postgres corruption).
- deploy/build-push.sh: build images off-NAS (the frontend build wants
~2GB RAM) and push to Gitea, or --save a tar to carry over.
- scripts/reindex_search.py: rebuild the search index from Postgres alone.
Every SearchDoc field is already stored, so the index is derived data
that never needs backing up — this closes the only real objection to
running Meilisearch on a box whose volumes may be wiped.
- pyproject: package-data for accounts/presets.yaml and the shared plugin
Alembic env. Without it a non-editable install (i.e. any image) silently
ships a package that fails at runtime.
- GET /api/version, used by the deploy health gate.
Verified end to end on a real stack: fresh install, all services healthy,
core + 4 plugin migration branches applied, login/SPA/plugin bundles
through nginx, WS upgrade returning 101, __Host- cookies under a
simulated TLS proxy, backup -> restore round-trip preserving user and
plugin schema, failed-update auto-rollback (both a bad migration and a
boot-crashing API), and Meilisearch profile serving a live query.
ruff + mypy --strict clean, 363/363 backend tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nux3LCu4zP9nih4BRhPyHw
Sender avatars (privacy-safe, server-fetched, cached):
- New GET /api/avatar resolves Gravatar (d=404) then the sender domain's favicon
under the SSRF guard, Redis-cached (hit+miss); gated by SENDER_AVATARS_ENABLED.
GuardedResponse gains raw bytes. Shown in the mail list, reading pane, People
panel, and Contacts (overlay that falls back to initials on a miss).
PDF preview:
- Password-protected PDFs now prompt in-place and retry (pdf.js PasswordException),
with a clear error for other load failures instead of a blank overlay. The
password is remembered per attachment until the thread is left; clicking the
field no longer dismisses the overlay.
Contacts:
- Autosave reply recipients: message.sent now carries to/cc (name+email); the
contacts plugin subscribes and adds any non-contact recipients to the Autosaved
group. Contact avatars show the gravatar/favicon via /api/avatar.
Mail views / retention:
- Clicking an account shows all its mail except Trash/Spam (was inbox-only).
- Daily cron autopurge_trash_spam permanently deletes Trash/Spam older than
TRASH_RETENTION_DAYS (default 30; 0 disables) — server EXPUNGE + local rows.
UI polish:
- Ultimo signet logo in the Mail/Calendar/Contacts sidebar header and collapsed
rail; consistent 11px top margin and vertically-aligned header row across states
(fixed-size flex icon buttons); SVG favicon on the browser tab.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
Implements SPEC v1.5 §6.2.1–6.2.3 so Gmail (and Yahoo/iCloud/Outlook-personal)
onboard via a provider app password on the existing plain-password sync path —
no OAuth, no new sync code. Gmail was oauth2-only with OAuth unimplemented, so it
was previously un-addable.
Backend:
- presets.yaml: each preset declares an ordered `auth_strategy`; Gmail becomes
`auth: password` + `[app_password, oauth2]`. App-password providers ship a
data-driven `help:` block (deep_link, app_name_hint, i18n step keys, docs_url).
- discovery/types: CandidateConfig carries `auth_strategy` + `HelpBlock`; presets
load them and autoconfig candidates derive a default from the detected auth.
- schemas/api: CandidateOut exposes auth_strategy + help; DiscoverResponse adds
`google_oauth_enabled`.
- config: GOOGLE_OAUTH_ENABLED flag (default false) gates the OAuth fallback.
Frontend:
- ProviderCredentialPanel (wizard step 1.5): deep-link popup (user gesture),
rendered steps, single paste field that strips whitespace, instant verify
(auto POST /accounts/test at 16 chars → Connected ✓ / mapped error), and the
auth_failed recovery branch with a gated "Sign in with Google instead" button.
- AccountWizard routes app-password providers to the panel, then a details step.
- i18n keys (en) for the panel + provider steps.
Honors the §6.2.2 prohibitions: no page automation, no real-password capture,
no scraping — deep-link + guide + paste + test only. Adds the v1.5 spec.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
- concurrency.loop_semaphore: per-event-loop semaphores (safe across pytest loops
and process restarts via a WeakKeyDictionary keyed by the running loop).
- provider: holds a per-account (≤4) and per-host (PER_HOST_CONN_CAP) slot for the
whole lifetime of an IMAP socket, released on close; SMTP send takes a per-host
slot for its duration. A worker can no longer open hundreds of sockets to one
host/account.
- indexer: parsing goes through a PARSE_CONCURRENCY semaphore, and a message over
STREAM_PARSE_MB is parsed from its blob file (parse_message_file) instead of an
in-memory tree, so peak RAM ~= PARSE_CONCURRENCY x STREAM_PARSE_MB regardless of
mailbox sizes. Backfill parallelism stays bounded by the bulk queue only.
- config: PER_HOST_CONN_CAP, PARSE_CONCURRENCY, STREAM_PARSE_MB.
- tests: per-account cap blocks the 5th connection / distinct accounts independent;
file-vs-bytes parse equivalence + large-message streaming routing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each owned account is tiered by the coordinator; the runner shapes its work to
match — correctness identical, only latency/socket cost differ.
- hot: persistent INBOX IDLE + secondary poll every POLL_WARM.
- warm: no persistent socket; transient INBOX+special poll every POLL_WARM.
- cold: same transient poll every POLL_COLD.
- engine: poll_all_loop (transient full poll); secondary_poll_loop interval is
now a parameter; POLL_INTERVAL removed in favor of POLL_WARM/POLL_COLD.
- coordinator: tier eval every 120s from session heat (sess:active:{user}) +
recent mail (max internal_date within TIER_HOT_ACTIVITY_H), with the
HOT_BUDGET_PER_WORKER guard shedding least-recently-active session-less hot
accounts; user.active event promotes a user's accounts to hot at once (≤5s).
- API: login + WS connect/ping publish user.active and refresh sess:active
(TTL TIER_SESSION_GRACE_MIN). New sync:tier:{id} key, default warm on claim.
- config: POLL_WARM/POLL_COLD/TIER_*/HOT_BUDGET_PER_WORKER.
- tests: 5 tier tests (session/recent-mail promotion, cold, user.active,
budget demotion) alongside the lease/lifecycle acceptance tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ownership of an account now moves through a Redis lease so any number of sync
workers can run against the same account set with no double-IDLE.
- leases.py: sync:live registry + sync:lease:{id} (SET NX EX) with owner-checked
Lua renew/release; mark_live / mark_not_live helpers.
- coordinator.py: SyncCoordinator claims from sync:live up to MAX_ACCOUNTS_PER_
WORKER, renews every LEASE_RENEW (fences the runner on lease loss), reconciles
sync:live against Postgres, heartbeats a worker key, and rebalances when it
holds well above a fair share.
- supervisor start/stop now drive a coordinator; engine._set_state and
delete_account maintain sync:live membership (+ lease drop = fencing teardown).
- config: MAX_ACCOUNTS_PER_WORKER, LEASE_TTL, LEASE_RENEW.
- tests: acceptance 1-3 (single ownership, crash failover, create/delete
lifecycle) driving the real coordinator + Redis with faked IMAP loops.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lukasandClaude Opus 5.5