Commit Graph
13 Commits
Author SHA1 Message Date
lukasandClaude Opus 5.5 0ddc0cb001 Admin user management, password reset by email, and the Ultimo Email name
Branding: the page titles and app name are now "Ultimo Email".

Admin panel (now two columns with a divider):
- Users: search by login or by a mail account's address or name (results
  only, nothing listed until something is typed). A user opens a window to
  change their quota, block/unblock them, or delete them; block and delete ask
  for confirmation, and an admin cannot do either to their own account.
- Blocking (users.blocked_at, migration 0019) refuses sign-in with a clear
  message and revokes every session; a new user.signed_out event sends the
  user's open tabs to the sign-in screen at once. Any "not signed in" answer
  now does the same, not only /auth/me. Deleting removes mail accounts (sync
  stops), plugin rows (cleared inside the user's RLS context, since plugin
  tables have no FK to users), blobs, the search index, sessions and the user.
- Outgoing system mail (migration 0020): SMTP server, port, security, login,
  password (vault-encrypted, never sent back), From address and the public
  address reset links point to, plus a "Send test" that reports the server's
  error.

Password reset:
- "Forgot password?" on the sign-in page asks for the user name and always
  answers "If this account exists you will receive an email to your recovery
  email address"; the mail goes out after the response, so neither the reply
  nor its timing reveals an account, and it is capped at 3 per hour.
- The emailed link (token in the URL fragment, single use, 30 minutes) opens a
  page to set a new password. With 2-step verification on, the code is checked
  before anything changes; five wrong codes burn the link.
- Settings > Profile: the recovery email can be added, changed or removed,
  confirmed with the current password, which must be typed — autofill and
  paste are refused (TypedPasswordInput).

Also adds frontend/icons/ultimo-logo-bimi.svg, an SVG Tiny PS version of the
logo for a BIMI record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 20:16:27 +02:00
lukas 428e81efa5 Revert "Mail list: show who wrote, not everyone involved"
This reverts commit 550b8cf503.
2026-09-18 18:31:16 +02:00
lukasandClaude Opus 5 550b8cf503 Mail list: show who wrote, not everyone involved
The first column listed a thread's `participants`, which the threader builds from
from + to + cc — so every ordinary email read "Sender, You", and the +N badge
beside the avatar counted the user themselves.

`participants` cannot simply be narrowed: the subject-fallback join matches an
incoming message's recipients against it (threader._by_subject), so dropping
recipients would change how mail is threaded. Hence a separate `threads.senders`,
holding only from addresses, newest writer first — which is also the right thing
for a conversation where several people replied.

Migration 0019 backfills it, so existing threads do not wait to be touched again.
The list falls back to `participants` if a row somehow has no senders, rather
than rendering an empty column.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-18 18:27:42 +02:00
lukasandClaude Opus 5 e58a2f0ef4 Sync: a flag set on the server left the thread counters stale
Reported from the running app: a message showed as unread in the list — bold,
blue dot — while the reading pane showed it as read, and only a manual "mark as
read" reconciled them.

The two views read different things. The pane reads the message's own flags; the
list reads `threads.unread_count`, denormalized from them. `apply_delta` updated
message flags with a bare UPDATE and never recomputed the thread, so a flag set
anywhere but in this app — read on a phone, filed by a server-side rule — moved
one and not the other. Auto-mark-on-read could not fix it either: it sees a
message that is already \Seen and correctly does nothing, leaving the counter
wrong forever.

Expunges had the same hole, which also left `message_count` overstated.

`apply_delta` now collects the affected threads and re-denormalizes them after
the deletes, so a thread emptied by an expunge is removed rather than recounted.

Migration 0018 repairs rows that already drifted — recomputed from the messages,
so it is a no-op on a healthy database. One thread was stale here; installs that
have been syncing longer will have more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-18 18:21:30 +02:00
lukasandClaude Opus 5 8fce86559e Admin panel on its own port, and a registration gate
Three strands that only make sense together, so they land together.

Registration gate (migration 0016). POST /auth/register was open to anyone
who could reach the port. Now the first account on an empty instance is
always allowed, becomes the administrator, and closes registration behind
itself; after that it takes an admin reopening it. The decision is made
under a row lock on app_settings so two simultaneous first signups cannot
both see an empty table and both claim administrator. Existing installs
keep their oldest account as admin and start closed.

Instance default quota (migration 0017). users.quota_bytes is snapshotted
at creation from app_settings.default_quota_bytes, so changing the default
affects later accounts only and never resizes a mailbox that exists.

The panel is a separate app, not a page in the mail client. admin.html is
its own Vite entry (5.8 KB next to the client's 1.7 MB; they share only the
vendor chunk), with its own administrator sign-in. One nginx now serves two
listeners: :80 is the mail client and 404s both /admin.html and
/api/admin/*, :81 is the panel. docker-compose publishes them as WEB_PORT
and ADMIN_PORT. The boundary is nginx refusing paths on one listener, not
process isolation — CurrentAdmin still guards every admin route — so
ADMIN_PORT must be restricted to the LAN, which the runbook now covers for
both DSM and QNAP.

Also QNAP: deploy/webmail.sh finds Container Station's docker under
/share/*/.qpkg/container-station/bin, which is off an SSH session's PATH,
before anything shells out to it. docs/nas-install.md gains a QuTS hero
runbook (§2.6), splits the DSM firewall section into container-to-container
(§8.1) and the egress case that makes account-add time out while
autodiscovery appears to work (§8.2), and scopes both to Synology.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-16 20:34:08 +02:00
lukasandClaude Opus 4.8 4c200ab94a Gmail label sync + no-dup backfill, live label/UI updates, account-delete cleanup, Sync-SPAM toggle, sidebar logo
Gmail sync (SPEC: Gmail sync):
- Provider fetches X-GM-LABELS when X-GM-EXT-1 is advertised; parses custom labels
  (system labels dropped, mUTF-7 decoded) onto RemoteMessage.
- Engine syncs only system folders + All Mail for Gmail, skips every label folder,
  and dedupes the All Mail pass by Message-ID — kills the INBOX/All-Mail duplication.
- Custom labels ride the message.new event; the labels plugin subscribes and maps
  them to chips as "<name> [Gmail]" (per-message + thread aggregate + known set),
  so labels fill in gradually with the sync window. Non-ASCII labels decode right.

Labels self-healing (message-driven reconcile):
- Per-message keys are the source of truth; t: aggregates and _labels are derived.
  Reconcile drops labels of deleted messages, regroups survivors by their current
  thread (fixing JWZ thread merges), rewrites changed t: keys, and rebuilds _labels
  from live messages — keeping user labels, dropping orphaned Gmail ones.
- New SDK reads: sdk.threads.existing / sdk.messages.threads_of. Sidebar refreshes
  live via a plugin.labels.changed WS event on new labels.

Account deletion: prune threads left empty after the cascade (they were user-scoped,
so only messages cascaded) + de-list the account from surviving cross-account threads.

Sync SPAM: opt-in per-account sync_spam (migration 0015, default off) — the engine
skips the Junk folder in backfill and poll unless enabled; wizard checkbox added.

Robustness / UX:
- WebSocket auto-reconnects with backoff and resyncs on (re)connect, and account
  add/delete invalidates threads/labels — no more hard refresh for live updates.
- Backfill banner + sidebar ring also require done < total, so a completed account
  can't leave a stuck "X of X".
- Scrub lone UTF-16 surrogates from parsed headers/bodies so bad messages no longer
  crash the backfill on Postgres text/JSONB.
- Ultimo signet logo in the Mail/Calendar/Contacts sidebar header (and at the top of
  the collapsed rail), via the shared plugin-ui.css + MailLayout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-05 18:04:49 +02:00
lukasandClaude Opus 4.8 4e077f1a37 M3 plugin platform: SDK + loader, RLS-isolated tables, migrations, jobs
Backend foundation for the plugin platform (SPEC §11), Phases 1-3.

Phase 1 — SDK + loader:
- app/plugins/types.py: frozen Plugin/UiManifest/Slot/Page/JobSpec contract.
- app/plugins/sdk.py: sdk(name) facade — the only app.* import allowed for plugins
  (scoped session, plugin.{name}.* publish, blob, outbox, settings, SSRF http_get, KV, enqueue).
- app/plugins/loader.py: entry-point discovery (webmail.plugins), hook forwarding
  (API + worker), router mounting at /api/plugins/{name}, ui-manifest aggregation.
- plugin_kv core table (migration 0013) + GET /api/plugins/ui-manifest.

Phase 2 — non-removable core plugins + DB-enforced privacy:
- builtin flag (calendar/contacts will be builtin=True; uninstall refuses them).
- RLS per-user isolation on plg_* tables: least-privilege webmail_plugin role
  (migration 0014), plugin_session(user_id) sets LOCAL ROLE + app.current_user_id,
  rls.enable_rls() (ENABLE+FORCE RLS + USING/WITH CHECK policy). Mail untouched.
- Per-plugin Alembic runner (shared env, dedicated version table) + install/uninstall
  --drop-data CLI; skeleton fixture proves install -> run -> uninstall --drop-data
  leaves zero residue (§12 M3 acceptance).

Phase 3 — plugin jobs (§11.2):
- JobSpec registered as arq function plg.{plugin}.{func} + cron (5-field parser);
  singleton=True guarded by a per-(plugin,user) Redis lock; wired onto the shared
  interactive worker.

Gates: ruff, mypy --strict (94 files + tests), full suite 326 passed. RLS isolation,
migration round-trip, and singleton lock covered by tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcj3TpydMTzTSmU7kunzf8
2026-07-26 21:42:35 +02:00
lukasandClaude Opus 4.8 49738740c0 M2: Meili search + saved searches, sync windows + backfill progress, threading & image fixes
Search:
- Meilisearch backend (app/search/meili.py) + factory wiring; pinned SDK 2.7
- Search-as-you-type: debounced query + Postgres prefix matching, built from
  the DB tokenizer so full email addresses match (email stays one lexeme)
- Saved searches API + sidebar virtual mailboxes; save affordance in the box

Account sync windows (SPEC §12 M2):
- Choose at setup: All (recent 30d first, then remainder), Last 3 months, New only
- Per-account sync_window + "Sync remaining mail" from settings (idempotent re-run)
- Provider windowed backfill (SINCE/BEFORE), message_count, newest_uid
- migration 0012: accounts.sync_window + backfill_total/backfill_done

Backfill progress UX:
- Floating progress card (auto-dissolves after 10s) + spinning sidebar ring that
  fills with the account colour; driven by sync.progress over WS

Thread list pagination:
- Server-side offset + accurate grand total, with filter/order in SQL; working
  prev/next across all synced mail

Reading panel:
- "Load images" one-off now actually renders images (?images=1) without trusting
  the sender; envelope carries images_allowed so a trusted sender shows no prompt

Threading:
- Subject fallback no longer merges messages that declare an explicit (different)
  parent; true siblings still group via a shared-parent reference match. Fixes
  unrelated auto-replies with the same subject landing in one thread

Backend 257 tests green, ruff + mypy --strict clean; frontend tsc/eslint/vitest clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-23 23:00:08 +02:00
lukasandClaude Opus 4.8 b4e07f11aa collapse cross-account dupes, Lato UI font, mail-list selection & density tuning
Backend:
- Collapse identical cross-account deliveries in a thread: new messages.delivery_id
  (migration 0011) preserves the real Message-ID even when message_id_hdr is
  synthesized (§7.1); get_thread groups by it and reports also_in_account_ids; the
  list count counts distinct deliveries. Backfill script for existing rows.

Frontend:
- Bundle Lato (SIL OFL) as the UI font; unread mail-list rows use Black weight.
- Mail-list selection: hover-avatar checkbox, tri-state "select all", bulk
  Archive/Pin/Delete (Restore/Delete in Archive/Trash/Spam) with a count in the
  reading pane; unread blue dot; "also delivered to" note in the thread.
- Spacing/size tuning: sender-column width, icon gaps, avatar size, bar heights,
  and a round of -1px font sizes across the sidebar, mail list, and reading pane.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-22 21:49:48 +02:00
lukasandClaude Opus 4.8 d650f16feb mail UX: restore-to-folder, draft/attachment fixes, printing, sidebar collapse
Backend:
- Restore returns each message to its original folder (e.g. Sent/Archive) instead
  of always the Inbox: new messages.restore_mailbox_id (migration 0010) +
  restore_thread_job.
- Drafts: expunge reconciliation for the Drafts poll (provider.list_uids /
  _reconcile_drafts) clears phantom "remote" drafts; send + discard drop the
  mirrored DB row; adopting a remote draft carries its real attachments into the
  compose flow.
- Thread detail/list are folder-aware: individually trashed/spam messages are
  hidden outside their folder, Trash/Spam show only their own messages, and the
  mail-list message count matches the view.

Frontend:
- Thread reading panel: fix newest-first card overlap/spacing (expand-all and
  single expand), hide Reply/Forward on trashed messages, close the Actions menu
  on message-body (iframe) clicks, scroll to the inline composer, and prompt
  Save draft / Discard on the X close.
- Composer: remount when switching drafts, dirty-tracking to stop duplicate IMAP
  mirrors, and close the From/More dropdowns on outside click.
- Sidebar: collapse/expand toggle (icons-only ~60px rail) and a log-out icon on
  Sign out.
- Mail-standard printing in a new window with reliable page margins (lib/print.ts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-20 23:12:25 +02:00
lukasandClaude Opus 4.8 e0a45c5a46 sync-v2 step 1: worker roles + interactive/bulk queue split
- config: WORKER_ROLE (sync|jobs|all), INTERACTIVE_CONCURRENCY, BULK_CONCURRENCY.
- queue: named queues (interactive/bulk) + enqueue_job() router; sync_bootstrap
  goes to bulk, all other jobs to interactive.
- workers.run: single entrypoint; role=jobs runs both arq consumers (per-queue
  timeout/concurrency), role=sync runs the supervisor, all=both (dev default).
- docker-compose: worker-sync + worker-jobs services (shared image, nofile
  ulimit); backend/Dockerfile; Makefile worker/worker-sync/worker-jobs targets.
- tests repointed to app.queue.get_arq_pool; supervisor behavior unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:53:31 +02:00
lukasandClaude Opus 4.8 3951df9755 Two-way draft sync, signatures, sent/bcc fixes, and mail-list UI
Backend
- Sent copies: persist locally on send + periodic multi-folder CONDSTORE poll
  (Sent/Archive/Trash/Junk) so they thread and show without an IDLE-only INBOX.
- Bcc: transmitted copy omits Bcc, Sent copy keeps it; store/display bcc_addr.
- Drafts: mirror local drafts to the IMAP Drafts folder (append/replace/expunge);
  inbound sync of foreign drafts (un-threaded) with adopt-to-edit + dedup by UID.
- Per-user storage quota (5 GiB default) and manual "sync all folders" action.
- Account editing (display/color/server/password) and per-account HTML signature.
- Account view scoped to that account's inbox; new migrations 0006–0009.

Frontend
- Mail-list pagination, folder filters (All/Unread/Pinned), sort toggle, refresh.
- Aligned + widened mail-list margins; reading-pane bars fixed height.
- Rich-text composer: color palette, highlight, image; attachments; expand/pop-out.
- Signatures: auto-insert for new mail, replace on From change, reply insert.
- Restore/Delete actions per folder; drafts view merges local + remote, date-sorted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 15:27:09 +02:00
lukasandClaude Opus 4.8 a5ec0d21cb Initial commit: webmail client (M0/M1)
Three-pane webmail client.
- Backend: FastAPI + async SQLAlchemy + arq worker (Postgres/Redis), IMAP/SMTP
  sync, threading, search, drafts/outbox with undo, attachments, 2FA.
- Frontend: React 18 + Vite + TypeScript + TanStack Query + Tailwind; three-pane
  UI with rich-text composer, light/dark themes, resizable panels.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 23:20:34 +02:00