Security (ultimo-security-report.md):
- Security headers on every response, the app shell included: they lived at
server level and nginx dropped them in any location with its own header.
CSP, framing, nosniff, referrer, permissions (docker/nginx-security.conf).
- Plain HTTP forwarded by the reverse proxy is redirected to HTTPS; HSTS on
HTTPS responses. server_tokens off.
- Cross-site state-changing /api requests refused (login CSRF), WebSocket too.
- Validation errors in the usual error shape, without the request model.
- New-password policy: max 128, common/own-address/repeated refused,
breached-password check (HIBP k-anonymity, fails open).
- Message HTML: links keep only http(s)/mailto (sanitizer v6); the HTML
endpoint carries its own no-script CSP.
- Outbound fetches pinned to the checked IP (DNS rebinding); external
IMAP/SMTP servers re-checked before each connection; CGNAT and mapped
addresses counted as non-public.
Fixes from testing:
- IDLE loop catches up before idling: mail that came while the worker was
away no longer waits up to 10 minutes.
- Restore from Spam/Trash/Archive moves only the removed messages, not your
reply in Sent.
- Narrow desktop list stacks sender/subject/snippet (subject was squeezed
to nothing at the default width), with hover actions.
- Phone: wide emails scale to fit the screen.
- Singular forms ("1 message"); no zip button for a single attachment;
app passwords no longer claim "never used" for mail apps.
- ruff format across the backend.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosted mailboxes (SPEC App. D), behind HOSTED_MAILBOXES (off: unchanged):
- Stalwart mail server, provisioned by `python -m app.hosted.provision`;
directory read from a Postgres view, app passwords only for mail apps
- native account per user, the worker signs in as master for the user
- raw messages of hosted mailboxes only cached, fetched back on demand
- migrations 0025 (accounts.kind, directory view), 0026 (IMAP ids bigint)
Mail:
- dark message mode switch in the reading pane (Thunderbird's algorithm)
- moves and folder creation quote the folder name ("Deleted Items"):
swipe-to-delete / Trash no longer bounce back
- thread unread counters: lock before recount, sorted lock order,
UIDVALIDITY re-import recounts, nightly repair_threads
- sidebar accounts sorted alphabetically, more space above headings
Settings: Security headings renamed; antispam accounts are explicit
(none ticked = none filtered), input widths, Activity hidden.
Plugin bundles served with Cache-Control: no-cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sending: a send the server could not take (unreachable, timeout, 4xx) is no
longer failed at once. It waits in Scheduled ("retrying") and is tried again
after 1, 2, 5, 10, 30 min, then hourly, for 24 hours — on the server, so it
works with the browser closed. A permanent refusal (5xx, wrong password) is
"failed" at once, with the server's reason. GET /outbox lists both;
POST /outbox/{id}/retry (Retry now) and /cancel (Edit, Delete) act on them,
and they are left out of Drafts. Mail delivered but not filed in Sent now
counts as sent, and a conditional claim keeps two jobs from sending one
message twice. The app says when a send failed and when it finally went.
Actions: arq retries a job only on Retry, never on a plain exception, so the
"retry x5, then revert" for flags and moves never happened. Flag, move,
archive, restore, delete and empty-folder jobs now raise Retry while the
server is unreachable (~24 hours of growing gaps), skip a change a later one
replaced, stop at the first connection failure instead of 20 s per message,
and delete/empty keep the rows until the server has the messages gone.
Account health: 3 failed connections in a row mark an account unreachable
(amber in the sidebar and Settings, cleared on the next success); 3 refused
sign-ins stop its syncing as auth_failed until the password is entered
again, so the provider does not lock the account. Sync loops back off 30 s
doubling to 10 min and log one line per expected failure; a refused
connection fails at once with its reason (the connect task is kept), and a
silently dead IDLE connection is noticed within ~11 min instead of ~29.
Also: the empty reading pane is clipped to its width when dragged very
narrow; tests no longer leave live accounts and outbox rows in the database.
Migration 0021: outbox attempts / next_attempt_at / retry_until, accounts
sync_error / sync_error_since.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Mark as Spam" moved the message but left it on screen. Two causes, both fixed
generally rather than for this one action.
`move_job` published message.changed without a thread_id, so the WS handler
invalidated the message list but never the thread the user was reading — the
panel had no reason to refetch. It now names the thread.
And a thread whose last visible message has gone should not linger. MailLayout
closes the panel when the open thread comes back empty, whoever emptied it —
core action or plugin.
`move_to_special` now also publishes the move immediately rather than waiting
for the IMAP job, since a plugin acting on the user's behalf has no client-side
handle to update optimistically the way the core endpoint's caller does.
That exposed an ordering bug: the optimistic event reaches the passive-learning
hook, which would re-teach the lesson /learn had just taught. /learn now records
`learned` before moving, so the hook's dedup stands down — and records it with
an upsert, because provider-filed spam has no verdict row of ours to update.
One real mistake on the way: the event first reported the destination folder as
the source, which would have made every move look like a no-op to anything
reading the pair.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
Explicit teaching shipped with C1 (the Spam / Not spam buttons). This adds the
two halves that need no buttons at all.
Passive learning (§C.5). Moving mail into Spam says "this is spam" more honestly
than any button, because the user is doing it for their own reasons rather than
to train a filter; moving it back out says the opposite. Only those two
transitions teach — Spam to Trash is someone tidying up, Inbox to Archive says
nothing about spam. Teaching the same message the same thing twice is skipped,
since users move things back and forth, but reversing a lesson is allowed. If
the engine is down nothing is recorded, so a later move can teach it again.
This needed the transition, not just the destination: `message.changed` carried
only where a message landed, so `move_job` now also publishes
previous_mailbox_id and both special_use values.
Provider bootstrap (§C.5). An account going live enqueues a job that teaches
Bayes from mail the user already owns — up to 500 messages from the provider's
Junk folder as spam, and up to 500 Inbox messages older than 30 days as ham.
This is the constructive use of the history that classification deliberately
refuses to act on: the same backfill, read instead of judged. It sleeps between
messages so live classification never queues behind training, stops rather than
grinding if the engine disappears, and marks the account done in KV so it runs
once.
That needed `PluginMessages.sample`: the SDK could fetch a message by id but not
enumerate a folder. Ids only — bodies are pulled one at a time, so a 500-message
sample never materialises 500 bodies.
Verified against the running engine: teaching moved its `learned` counter 0 -> 2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
Sender avatars (privacy-safe, server-fetched, cached):
- New GET /api/avatar resolves Gravatar (d=404) then the sender domain's favicon
under the SSRF guard, Redis-cached (hit+miss); gated by SENDER_AVATARS_ENABLED.
GuardedResponse gains raw bytes. Shown in the mail list, reading pane, People
panel, and Contacts (overlay that falls back to initials on a miss).
PDF preview:
- Password-protected PDFs now prompt in-place and retry (pdf.js PasswordException),
with a clear error for other load failures instead of a blank overlay. The
password is remembered per attachment until the thread is left; clicking the
field no longer dismisses the overlay.
Contacts:
- Autosave reply recipients: message.sent now carries to/cc (name+email); the
contacts plugin subscribes and adds any non-contact recipients to the Autosaved
group. Contact avatars show the gravatar/favicon via /api/avatar.
Mail views / retention:
- Clicking an account shows all its mail except Trash/Spam (was inbox-only).
- Daily cron autopurge_trash_spam permanently deletes Trash/Spam older than
TRASH_RETENTION_DAYS (default 30; 0 disables) — server EXPUNGE + local rows.
UI polish:
- Ultimo signet logo in the Mail/Calendar/Contacts sidebar header and collapsed
rail; consistent 11px top margin and vertically-aligned header row across states
(fixed-size flex icon buttons); SVG favicon on the browser tab.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
Backend:
- Restore returns each message to its original folder (e.g. Sent/Archive) instead
of always the Inbox: new messages.restore_mailbox_id (migration 0010) +
restore_thread_job.
- Drafts: expunge reconciliation for the Drafts poll (provider.list_uids /
_reconcile_drafts) clears phantom "remote" drafts; send + discard drop the
mirrored DB row; adopting a remote draft carries its real attachments into the
compose flow.
- Thread detail/list are folder-aware: individually trashed/spam messages are
hidden outside their folder, Trash/Spam show only their own messages, and the
mail-list message count matches the view.
Frontend:
- Thread reading panel: fix newest-first card overlap/spacing (expand-all and
single expand), hide Reply/Forward on trashed messages, close the Actions menu
on message-body (iframe) clicks, scroll to the inline composer, and prompt
Save draft / Discard on the X close.
- Composer: remount when switching drafts, dirty-tracking to stop duplicate IMAP
mirrors, and close the From/More dropdowns on outside click.
- Sidebar: collapse/expand toggle (icons-only ~60px rail) and a log-out icon on
Sign out.
- Mail-standard printing in a new window with reliable page margins (lib/print.ts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
lukasandClaude Opus 5.5