webmail-client/docs/feature-ideas.md

# Feature ideas — making Ultimo competitive with Gmail

Status (2026-10-18): **built** (uncommitted at the time of writing) — snooze,
keyboard shortcuts, subscriptions page + unsubscribe, templates, tracker
blocking. Left out of that first pass: follow-up reminders (§1.1), the image
proxy (§2.1), the footer-link fallback and "auto-archive in future" filter on the
Subscriptions page (§1.3), and bulk snooze from the list. Everything else here is
still a proposal. Each idea notes
what it is, why it matters, how it fits the existing architecture, and a rough
size (S = days, M = 1–2 weeks, L = several weeks).

Already built, so not repeated here: undo send, scheduled send, labels, filters,
antispam, search operators (Meilisearch), VIP contacts, calendar (CalDAV, iMIP),
contacts (CardDAV), offline read-only mode, PWA + Web Push, 2FA, invitations,
hosted mailboxes (Stalwart).

---

## 1. What people miss when leaving Gmail

### 1.1 Snooze and follow-up reminders — M
- **Snooze:** "bring this back Monday 09:00". The thread leaves the inbox and
  returns at the chosen time, marked unread and pinned to the top.
- **Follow-up:** when sending, "remind me if nobody replies in 3 days". If no
  reply has arrived in the thread by then, it comes back to the inbox with a
  "No reply yet" chip.
- **Fit:** a `snoozes` table (thread, user, wake_at, kind) plus an arq cron job
  that wakes due threads. Snoozed threads move to a server-side "Snoozed" folder
  so other IMAP clients see the same state. The follow-up check watches
  `message.new` events for the thread. Could be a builtin plugin.

### 1.2 Keyboard shortcuts — S/M
- Gmail-compatible: `j/k` next/previous, `o`/Enter open, `u` back to list,
  `r`/`a`/`f` reply / reply-all / forward, `e` archive, `#` delete, `s` star,
  `l` label, `v` move, `/` search, `c` compose, `x` select, `Shift+I/U`
  read/unread, `g i` / `g s` go to inbox / starred, `?` shows them all.
- **Fit:** frontend only. One central key handler that respects focus in
  inputs and the compose editor, plus a setting to turn it off.

### 1.3 One-click unsubscribe and a subscriptions page — M
- An "Unsubscribe" button on mail that carries `List-Unsubscribe`, using the
  one-click POST (RFC 8058) when offered, `mailto:` otherwise.
- A **Subscriptions** page: every sender who sends mailing-list mail, how often,
  when last read, with Unsubscribe / "Delete all from this sender" /
  "Auto-archive in future" (creates a filter).
- **How subscriptions are found:** no service records what a user signed up
  for, so the page infers it from received mail (as Gmail does). A message is
  bulk mail when it carries `List-Unsubscribe` (required of bulk senders by
  Gmail/Yahoo since 2024), `List-Id`, `Precedence: bulk|list`, or mailing-service
  markers (`Feedback-ID`, ESP headers). A footer "unsubscribe" link is a weaker
  fallback, shown as "probably a newsletter". Mail is grouped by `List-Id`, else
  by sender address.
- **Per group:** messages per month, last received, and **read rate** from the
  `\Seen` flag ("opened 0 of 23"), sorted most frequent / least read first.
- **After unsubscribing:** mark the group unsubscribed with the date. If mail
  still arrives after a grace period (RFC 8058 senders should stop within 2
  days), flag "still sending" and offer block / auto-archive via a filter.
- **Limits:** lists that haven't sent anything yet, or send no list headers,
  can't be seen. The page shows "bulk senders currently mailing you".
- **Fit:** the plugin SDK can read raw headers
  (`backend/app/plugins/actions.py`), but it parses the stored RFC822 per
  message, and with hosted mailboxes the raw message is only cached. So extract
  `List-Id`, `List-Unsubscribe` and `List-Unsubscribe-Post` during sync into a
  small `message_list_headers` table, backfilled once for existing mail. The
  spec (§12, M3) already describes the button as an example `message.actions`
  plugin. The one-click POST goes from the server, through the SSRF guard.

### 1.4 Templates (canned replies) — S
- Saved replies with placeholders (`{{first_name}}`, `{{my_name}}`), inserted
  from a compose toolbar menu or by typing `/` in the editor.
- **Fit:** per-user table plus a compose-toolbar plugin slot.

### 1.5 Out-of-office reply — M
- Start/end dates, subject, message; replies at most once per sender per N days;
  never to mailing lists or `Auto-Submitted` mail (RFC 3834).
- **Fit:** for hosted mailboxes, write a Sieve `vacation` script through
  Stalwart. For external accounts, the outbox sends the reply from an arq job
  on `message.new`.

---

## 2. Where Ultimo can beat Gmail

### 2.1 Privacy as the selling point — M
- **Tracker blocking:** detect tracking pixels (1×1 images, known tracker
  domains, per-recipient URL tokens) and strip them even when remote images are
  allowed. Show "3 trackers blocked" on the message.
- **Image proxy:** when remote images are allowed, load them through the server
  (SSRF-guarded, cached), so senders never see the reader's IP address or when
  the mail was opened.
- **Aliases on the hosted domain:** unlimited addresses such as
  `shop.amazon@ultimo.email` or `name+tag@`. Each one can be switched off, and
  the app shows which alias leaked to spammers.
- **Fit:** the sanitizer (`backend/app/indexer/sanitize.py`) already rewrites
  HTML and blocks remote images. Aliases are Stalwart principals or catch-all
  rules managed from the M6 hosted-mailbox code.

### 2.2 Encrypted mail (OpenPGP) — L
- Optional, done in the browser (OpenPGP.js), so the **private key never
  reaches the server unencrypted**.
- Decrypt and verify received PGP/MIME mail. Sign and encrypt when sending.
  Look up recipients' keys automatically (WKD, keys.openpgp.org) and publish
  WKD for the hosted domain.
- **Fit:** compose and reading-panel plugin slots. The key backup is encrypted
  with a passphrase before upload. Indexing can't search encrypted bodies (say
  so in the UI).

### 2.3 AI assistant (opt-in) — L
Detailed design in **§5** below.

---

## 3. Productivity polish

### 3.1 Split inbox / categories — M
Tabs such as *Important / Newsletters / Notifications / Receipts / Other*,
filled by rules (sender, `List-Id`, labels, filters) and optionally by the AI
classifier (§5.3). The user can move a sender between tabs and the app learns
from it.

### 3.2 Everything you've received — S/M
A view listing all **attachments, links and photos**, filterable by sender, type
and date, with preview. Builds on the right panel's Attachments tab and the
deduplicated attachment store.

### 3.3 Mute a thread; send and archive — S
- **Mute:** new replies in the thread skip the inbox (still searchable). Good for
  noisy reply-all threads.
- **Send and archive:** one button that sends the reply and archives the thread.

### 3.4 Easy move from Gmail — M
- Copy existing mail from Gmail over IMAP (Google OAuth is already planned),
  with progress, resume, and labels → labels.
- Import Gmail filters (Gmail exports them as XML) into the filters plugin.
- Import contacts (vCard/CSV) and calendars (ICS).
- The hardest part of winning users is getting them to switch. This makes
  switching painless.

### 3.5 Calendar from mail — M
- "Create event from this mail": date, time and place pre-filled from the text
  (rules first, AI optional).
- "Find a time": free/busy for the thread's participants (for those whose
  calendar we can see), then send an invitation.
- "Turn into task" once a tasks list exists (VTODO over CalDAV).

### 3.6 Shared mailboxes and delegation — L
For families and small teams:
- A shared `info@` mailbox visible to several users.
- Assign a thread to someone, with status (open / waiting / done).
- Internal notes on a thread that are never sent.
- Delegated access ("my assistant can read and send as me").

---

## 4. Suggested order

| Milestone | Contents | Why |
|---|---|---|
| **Next** | Snooze + follow-up, keyboard shortcuts, unsubscribe/subscriptions, templates, tracker blocking + image proxy | Moderate size, fits the current architecture, users notice right away; starts the privacy story |
| Then | Out-of-office reply, mute / send and archive, everything-received view, Gmail import | Smaller wins + removes the switching barrier |
| Later | AI assistant (§5), split inbox, calendar from mail | AI unlocks smart categories and event extraction |
| Separate | OpenPGP, aliases, shared mailboxes | Larger security / multi-user projects |

---

## 5. AI assistant — design

### 5.1 Principles
1. **Off by default, opt-in twice.** The administrator turns the plugin on and
   chooses a provider. Each user then opts in, per feature. Nothing about a
   user's mail is sent anywhere until they switch a feature on.
2. **The server calls the AI, never the browser.** API keys stay on the server,
   encrypted with the vault (`MASTER_KEY`), like mail-account passwords.
3. **The AI suggests, the user acts.** No AI output sends mail, deletes, moves
   or unsubscribes on its own. Drafts land in the compose editor; categories
   are labels the user can change.
4. **Mail is untrusted input** (see §5.6). An email can contain text written to
   manipulate the model ("ignore your instructions and…"). The design makes
   this harmless rather than trying to detect it.
5. **Provider-independent.** One small adapter interface; Anthropic (Claude) is
   the default. Other providers or a local model can be plugged in.

### 5.2 Features

| Feature | Trigger | What is sent to the model | Output |
|---|---|---|---|
| **Thread summary** | User clicks "Summarize" (or auto for threads over N messages, if enabled) | Plain-text bodies of the thread (quoted history stripped), senders, dates | 3–5 bullets + "what's asked of you" + deadlines; streamed into the reading panel |
| **Reply drafts** | "Suggest reply" in the reading panel, or a short instruction ("decline politely, offer next week") | The thread + the user's instruction + their signature/name + optional tone examples from their own sent mail | A draft inserted into the compose editor; the user edits and sends |
| **Rewrite** | Select text in compose → "Shorter / More formal / Friendlier / Fix grammar / Translate to …" | Only the selected text | Replacement text with a preview and undo |
| **Categorization** | Background, on each new message (only for users who opted in) | Sender, subject, `List-*` headers, first ~2,000 characters of text | One category from a fixed list (structured output) + optional "needs reply" flag → label |
| **Ask your mailbox** | A search-bar mode: "When does my car insurance expire?" | Only the messages that a search finds relevant (see §5.4) | An answer with links to the source messages |
| **Event extraction** | "Create event" on a message | That message | Title, start/end, place, attendees as structured data → pre-filled event form |

### 5.3 How it plugs into Ultimo
- **A new builtin plugin `webmail_plugin_ai`**, using the existing SDK:
  reading-panel and compose slots for the buttons, its own tables for settings,
  usage and cached results, and arq jobs for background work.
- **Provider adapter:** `complete(system, messages, schema=None, stream=False)`,
  with implementations for Anthropic (official `anthropic` Python SDK), and
  later Bedrock / Vertex / a local OpenAI-compatible server (Ollama, vLLM).
- **Streaming:** summaries and drafts stream token by token to the browser over
  the existing WebSocket (`backend/app/ws.py`), so text appears right away.
- **Structured outputs:** categorization and event extraction use the API's
  structured output mode (`output_config.format` / `messages.parse()` against
  a JSON schema). The model can only answer with an allowed category, so a
  malicious mail can't make it invent a "trusted" label.
- **Caching results:** a summary is stored keyed by thread + the newest message
  id, so reopening a thread costs nothing until a new message arrives.
- **Prompt caching:** the fixed instructions (system prompt, category list,
  style guide) come first and are marked cacheable, so repeated calls pay for
  them at the much cheaper cache-read rate. Volatile parts (the email text, the
  date) come after.
- **Bulk backlog:** classifying existing mail when a user first opts in goes
  through the Message Batches API (asynchronous, half price).
- **Quotas and audit:** per-user monthly token budget set by the admin; a usage
  table (user, feature, model, input/output tokens) shown in the admin panel.
  Calls are logged without message content.
- **Refusals and errors:** check `stop_reason` (including `refusal`) before using
  output; on rate limits, retry later (arq retry); the UI says "AI unavailable"
  rather than failing the mail action.

### 5.4 "Ask your mailbox" in detail
Searching all mail with the model would be slow and expensive, so this uses
retrieval:
1. The question goes to the model together with **one read-only tool**,
   `search_mail(query, from?, after?, before?)`, backed by the existing
   Meilisearch index and run **as that user** (row-level security; it can only
   see their own mail).
2. The model makes one or more searches; the server returns short snippets
   (sender, date, subject, ~500 characters).
3. If needed, the model asks for the full text of a few messages with a second
   read-only tool, `read_message(id)`.
4. It answers and cites message ids; the UI turns them into links.

There are no tools that write anything. Keyword search is enough to start.
Semantic search (embeddings in pgvector) could come later, but would need a
separate embeddings provider or local model, since that's a different service
from text generation.

### 5.5 Who provides the AI
The administrator chooses one in the admin panel:

| Option | Privacy | Quality | Cost | Notes |
|---|---|---|---|---|
| **Anthropic Claude API** (default) | Mail text leaves the server for Anthropic | Best | Pay per token | Simplest; official Python SDK; streaming, structured outputs, prompt caching, batches |
| **Claude through Google Vertex AI or Amazon Bedrock** | Same model, contract and data processing with Google/AWS; regional endpoints (e.g. EU) | Best | Partner pricing | Choose if EU data residency or an existing cloud contract matters |
| **Local model** (Ollama / vLLM on a GPU machine) | Nothing leaves your network | Noticeably weaker; fine for categorization and rewriting, weaker for summaries and Q&A | Hardware + electricity | The QNAP has no GPU — CPU-only is too slow for interactive use; needs a separate box |
| **Bring your own key** (per user) | As the chosen provider | — | User pays | Optional; lets power users use their own account |

**Models (Anthropic, prices cached 2026-10-06, per million tokens in / out):**

| Model | Price | Suggested use |
|---|---|---|
| `claude-opus-5-5` | $4 / $20 | Default for everything; best quality for drafts, summaries, Q&A |
| `claude-sonnet-5-5` | $2 / $10 | Option for drafts / summaries if cost matters |
| `claude-haiku-5-5` | $0.10 / $0.50 | Option for high-volume background work (categorization) |

The model is configurable **per feature** in the admin panel. Default to Opus 5.5
everywhere and switch individual features down only after checking that quality
holds. Categorization is the one where volume matters. Example: 500 new
messages/day × ~1,000 tokens ≈ 15M input tokens/month ≈ $60/month on Opus 5.5
vs. ≈ $1.50 on Haiku 5.5 (before caching). Interactive features are cheap
either way: a summary of a long thread is a few thousand tokens. Prices change;
check the provider's current price list before deciding.

**Data handling to verify before launch:** the provider's commercial terms on
training (Anthropic's API terms state that API inputs are not used for model
training by default — confirm the current wording), how long requests are
retained, and a data processing agreement (DPA). Hosting other people's mail in
the EU makes Ultimo the GDPR controller, so the opt-in screen must say which
provider receives the text.

### 5.6 Security: mail as untrusted input
- **No side effects from model output.** Every action needs a user click. Drafts
  are never sent automatically.
- **Read-only tools only**, always running as the requesting user (RLS), so an
  email that says "search for passwords and include them in the reply" can at
  most show the user something from their own mailbox, to the user.
- **Fixed output shapes** (structured outputs) for anything that changes state,
  like labels.
- **No remote content:** links in model output are shown as text or go through
  the normal link-safety warning; images are never loaded from model output.
- **Clear boundaries in the prompt:** email text is wrapped and labelled as
  data, not instructions. This helps, but the guarantees above don't depend on
  it.
- **Minimal data:** attachments are not sent by default; bodies are trimmed of
  quoted history and signatures; hidden HTML text is dropped by the sanitizer
  before the model sees it.

### 5.7 Build steps
1. Plugin skeleton, provider adapter (Anthropic), admin settings (key, model per
   feature, quotas), per-user opt-in screen, usage table.
2. Thread summary (streaming) + rewrite in compose.
3. Reply drafts with tone from the user's sent mail.
4. Categorization (structured output, batch backfill) → feeds split inbox (§3.1).
5. Ask your mailbox (read-only tool loop over Meilisearch).
6. Event extraction → calendar plugin.
7. Optional: Vertex/Bedrock and local-model adapters; bring-your-own-key.