webmail-client/docs/feature-ideas.md

Feature ideas — making Ultimo competitive with Gmail

Status (2026-10-18): built (uncommitted at the time of writing) — snooze, keyboard shortcuts, subscriptions page + unsubscribe, templates, tracker blocking. Left out of that first pass: follow-up reminders (§1.1), the image proxy (§2.1), the footer-link fallback and "auto-archive in future" filter on the Subscriptions page (§1.3), and bulk snooze from the list. Everything else here is still a proposal. Each idea notes what it is, why it matters, how it fits the existing architecture, and a rough size (S = days, M = 1–2 weeks, L = several weeks).

Already built, so not repeated here: undo send, scheduled send, labels, filters, antispam, search operators (Meilisearch), VIP contacts, calendar (CalDAV, iMIP), contacts (CardDAV), offline read-only mode, PWA + Web Push, 2FA, invitations, hosted mailboxes (Stalwart).

---

1. What people miss when leaving Gmail

1.1 Snooze and follow-up reminders — M

• Snooze: "bring this back Monday 09:00". The thread leaves the inbox and returns at the chosen time, marked unread and pinned to the top.
• Follow-up: when sending, "remind me if nobody replies in 3 days". If no reply has arrived in the thread by then, it comes back to the inbox with a "No reply yet" chip.
• Fit: a snoozes table (thread, user, wake_at, kind) plus an arq cron job that wakes due threads. Snoozed threads move to a server-side "Snoozed" folder so other IMAP clients see the same state. The follow-up check watches message.new events for the thread. Could be a builtin plugin.

1.2 Keyboard shortcuts — S/M

• Gmail-compatible: j/k next/previous, o/Enter open, u back to list, r/a/f reply / reply-all / forward, e archive, # delete, s star, l label, v move, / search, c compose, x select, Shift+I/U read/unread, g i / g s go to inbox / starred, ? shows them all.
• Fit: frontend only. One central key handler that respects focus in inputs and the compose editor, plus a setting to turn it off.

1.3 One-click unsubscribe and a subscriptions page — M

• An "Unsubscribe" button on mail that carries List-Unsubscribe, using the one-click POST (RFC 8058) when offered, mailto: otherwise.
• A Subscriptions page: every sender who sends mailing-list mail, how often, when last read, with Unsubscribe / "Delete all from this sender" / "Auto-archive in future" (creates a filter).
• How subscriptions are found: no service records what a user signed up for, so the page infers it from received mail (as Gmail does). A message is bulk mail when it carries List-Unsubscribe (required of bulk senders by Gmail/Yahoo since 2024), List-Id, Precedence: bulk|list, or mailing-service markers (Feedback-ID, ESP headers). A footer "unsubscribe" link is a weaker fallback, shown as "probably a newsletter". Mail is grouped by List-Id, else by sender address.
• Per group: messages per month, last received, and read rate from the \Seen flag ("opened 0 of 23"), sorted most frequent / least read first.
• After unsubscribing: mark the group unsubscribed with the date. If mail still arrives after a grace period (RFC 8058 senders should stop within 2 days), flag "still sending" and offer block / auto-archive via a filter.
• Limits: lists that haven't sent anything yet, or send no list headers, can't be seen. The page shows "bulk senders currently mailing you".
• Fit: the plugin SDK can read raw headers (backend/app/plugins/actions.py), but it parses the stored RFC822 per message, and with hosted mailboxes the raw message is only cached. So extract List-Id, List-Unsubscribe and List-Unsubscribe-Post during sync into a small message_list_headers table, backfilled once for existing mail. The spec (§12, M3) already describes the button as an example message.actions plugin. The one-click POST goes from the server, through the SSRF guard.

1.4 Templates (canned replies) — S

• Saved replies with placeholders ({{first_name}}, {{my_name}}), inserted from a compose toolbar menu or by typing / in the editor.
• Fit: per-user table plus a compose-toolbar plugin slot.

1.5 Out-of-office reply — M

• Start/end dates, subject, message; replies at most once per sender per N days; never to mailing lists or Auto-Submitted mail (RFC 3834).
• Fit: for hosted mailboxes, write a Sieve vacation script through Stalwart. For external accounts, the outbox sends the reply from an arq job on message.new.

---

2. Where Ultimo can beat Gmail

2.1 Privacy as the selling point — M

• Tracker blocking: detect tracking pixels (1×1 images, known tracker domains, per-recipient URL tokens) and strip them even when remote images are allowed. Show "3 trackers blocked" on the message.
• Image proxy: when remote images are allowed, load them through the server (SSRF-guarded, cached), so senders never see the reader's IP address or when the mail was opened.
• Aliases on the hosted domain: unlimited addresses such as shop.amazon@ultimo.email or name+tag@. Each one can be switched off, and the app shows which alias leaked to spammers.
• Fit: the sanitizer (backend/app/indexer/sanitize.py) already rewrites HTML and blocks remote images. Aliases are Stalwart principals or catch-all rules managed from the M6 hosted-mailbox code.

2.2 Encrypted mail (OpenPGP) — L

• Optional, done in the browser (OpenPGP.js), so the private key never reaches the server unencrypted.
• Decrypt and verify received PGP/MIME mail. Sign and encrypt when sending. Look up recipients' keys automatically (WKD, keys.openpgp.org) and publish WKD for the hosted domain.
• Fit: compose and reading-panel plugin slots. The key backup is encrypted with a passphrase before upload. Indexing can't search encrypted bodies (say so in the UI).

2.3 AI assistant (opt-in) — L

Detailed design in §5 below.

---

3. Productivity polish

3.1 Split inbox / categories — M

Tabs such as Important / Newsletters / Notifications / Receipts / Other, filled by rules (sender, List-Id, labels, filters) and optionally by the AI classifier (§5.3). The user can move a sender between tabs and the app learns from it.

3.2 Everything you've received — S/M

A view listing all attachments, links and photos, filterable by sender, type and date, with preview. Builds on the right panel's Attachments tab and the deduplicated attachment store.

3.3 Mute a thread; send and archive — S

• Mute: new replies in the thread skip the inbox (still searchable). Good for noisy reply-all threads.
• Send and archive: one button that sends the reply and archives the thread.

3.4 Easy move from Gmail — M

• Copy existing mail from Gmail over IMAP (Google OAuth is already planned), with progress, resume, and labels → labels.
• Import Gmail filters (Gmail exports them as XML) into the filters plugin.
• Import contacts (vCard/CSV) and calendars (ICS).
• The hardest part of winning users is getting them to switch. This makes switching painless.

3.5 Calendar from mail — M

• "Create event from this mail": date, time and place pre-filled from the text (rules first, AI optional).
• "Find a time": free/busy for the thread's participants (for those whose calendar we can see), then send an invitation.
• "Turn into task" once a tasks list exists (VTODO over CalDAV).

3.6 Shared mailboxes and delegation — L

For families and small teams:

• A shared info@ mailbox visible to several users.
• Assign a thread to someone, with status (open / waiting / done).
• Internal notes on a thread that are never sent.
• Delegated access ("my assistant can read and send as me").

---

4. Suggested order

Milestone	Contents	Why
Next	Snooze + follow-up, keyboard shortcuts, unsubscribe/subscriptions, templates, tracker blocking + image proxy	Moderate size, fits the current architecture, users notice right away; starts the privacy story
Then	Out-of-office reply, mute / send and archive, everything-received view, Gmail import	Smaller wins + removes the switching barrier
Later	AI assistant (§5), split inbox, calendar from mail	AI unlocks smart categories and event extraction
Separate	OpenPGP, aliases, shared mailboxes	Larger security / multi-user projects

---

5. AI assistant — design

5.1 Principles

1. Off by default, opt-in twice. The administrator turns the plugin on and chooses a provider. Each user then opts in, per feature. Nothing about a user's mail is sent anywhere until they switch a feature on.
2. The server calls the AI, never the browser. API keys stay on the server, encrypted with the vault (MASTER_KEY), like mail-account passwords.
3. The AI suggests, the user acts. No AI output sends mail, deletes, moves or unsubscribes on its own. Drafts land in the compose editor; categories are labels the user can change.
4. Mail is untrusted input (see §5.6). An email can contain text written to manipulate the model ("ignore your instructions and…"). The design makes this harmless rather than trying to detect it.
5. Provider-independent. One small adapter interface; Anthropic (Claude) is the default. Other providers or a local model can be plugged in.

5.2 Features

Feature	Trigger	What is sent to the model	Output
Thread summary	User clicks "Summarize" (or auto for threads over N messages, if enabled)	Plain-text bodies of the thread (quoted history stripped), senders, dates	3–5 bullets + "what's asked of you" + deadlines; streamed into the reading panel
Reply drafts	"Suggest reply" in the reading panel, or a short instruction ("decline politely, offer next week")	The thread + the user's instruction + their signature/name + optional tone examples from their own sent mail	A draft inserted into the compose editor; the user edits and sends
Rewrite	Select text in compose → "Shorter / More formal / Friendlier / Fix grammar / Translate to …"	Only the selected text	Replacement text with a preview and undo
Categorization	Background, on each new message (only for users who opted in)	Sender, subject, List-* headers, first ~2,000 characters of text	One category from a fixed list (structured output) + optional "needs reply" flag → label
Ask your mailbox	A search-bar mode: "When does my car insurance expire?"	Only the messages that a search finds relevant (see §5.4)	An answer with links to the source messages
Event extraction	"Create event" on a message	That message	Title, start/end, place, attendees as structured data → pre-filled event form

5.3 How it plugs into Ultimo

• A new builtin plugin webmail_plugin_ai, using the existing SDK: reading-panel and compose slots for the buttons, its own tables for settings, usage and cached results, and arq jobs for background work.
• Provider adapter: complete(system, messages, schema=None, stream=False), with implementations for Anthropic (official anthropic Python SDK), and later Bedrock / Vertex / a local OpenAI-compatible server (Ollama, vLLM).
• Streaming: summaries and drafts stream token by token to the browser over the existing WebSocket (backend/app/ws.py), so text appears right away.
• Structured outputs: categorization and event extraction use the API's structured output mode (output_config.format / messages.parse() against a JSON schema). The model can only answer with an allowed category, so a malicious mail can't make it invent a "trusted" label.
• Caching results: a summary is stored keyed by thread + the newest message id, so reopening a thread costs nothing until a new message arrives.
• Prompt caching: the fixed instructions (system prompt, category list, style guide) come first and are marked cacheable, so repeated calls pay for them at the much cheaper cache-read rate. Volatile parts (the email text, the date) come after.
• Bulk backlog: classifying existing mail when a user first opts in goes through the Message Batches API (asynchronous, half price).
• Quotas and audit: per-user monthly token budget set by the admin; a usage table (user, feature, model, input/output tokens) shown in the admin panel. Calls are logged without message content.
• Refusals and errors: check stop_reason (including refusal) before using output; on rate limits, retry later (arq retry); the UI says "AI unavailable" rather than failing the mail action.

5.4 "Ask your mailbox" in detail

Searching all mail with the model would be slow and expensive, so this uses retrieval:

1. The question goes to the model together with one read-only tool, search_mail(query, from?, after?, before?), backed by the existing Meilisearch index and run as that user (row-level security; it can only see their own mail).
2. The model makes one or more searches; the server returns short snippets (sender, date, subject, ~500 characters).
3. If needed, the model asks for the full text of a few messages with a second read-only tool, read_message(id).
4. It answers and cites message ids; the UI turns them into links.

There are no tools that write anything. Keyword search is enough to start. Semantic search (embeddings in pgvector) could come later, but would need a separate embeddings provider or local model, since that's a different service from text generation.

5.5 Who provides the AI

The administrator chooses one in the admin panel:

Option	Privacy	Quality	Cost	Notes
Anthropic Claude API (default)	Mail text leaves the server for Anthropic	Best	Pay per token	Simplest; official Python SDK; streaming, structured outputs, prompt caching, batches
Claude through Google Vertex AI or Amazon Bedrock	Same model, contract and data processing with Google/AWS; regional endpoints (e.g. EU)	Best	Partner pricing	Choose if EU data residency or an existing cloud contract matters
Local model (Ollama / vLLM on a GPU machine)	Nothing leaves your network	Noticeably weaker; fine for categorization and rewriting, weaker for summaries and Q&A	Hardware + electricity	The QNAP has no GPU — CPU-only is too slow for interactive use; needs a separate box
Bring your own key (per user)	As the chosen provider	—	User pays	Optional; lets power users use their own account

Models (Anthropic, prices cached 2026-10-06, per million tokens in / out):

Model	Price	Suggested use
claude-opus-5-5	$4 / $20	Default for everything; best quality for drafts, summaries, Q&A
claude-sonnet-5-5	$2 / $10	Option for drafts / summaries if cost matters
claude-haiku-5-5	$0.10 / $0.50	Option for high-volume background work (categorization)

The model is configurable per feature in the admin panel. Default to Opus 5.5 everywhere and switch individual features down only after checking that quality holds. Categorization is the one where volume matters. Example: 500 new messages/day × ~1,000 tokens ≈ 15M input tokens/month ≈ $60/month on Opus 5.5 vs. ≈ $1.50 on Haiku 5.5 (before caching). Interactive features are cheap either way: a summary of a long thread is a few thousand tokens. Prices change; check the provider's current price list before deciding.

Data handling to verify before launch: the provider's commercial terms on training (Anthropic's API terms state that API inputs are not used for model training by default — confirm the current wording), how long requests are retained, and a data processing agreement (DPA). Hosting other people's mail in the EU makes Ultimo the GDPR controller, so the opt-in screen must say which provider receives the text.

5.6 Security: mail as untrusted input

• No side effects from model output. Every action needs a user click. Drafts are never sent automatically.
• Read-only tools only, always running as the requesting user (RLS), so an email that says "search for passwords and include them in the reply" can at most show the user something from their own mailbox, to the user.
• Fixed output shapes (structured outputs) for anything that changes state, like labels.
• No remote content: links in model output are shown as text or go through the normal link-safety warning; images are never loaded from model output.
• Clear boundaries in the prompt: email text is wrapped and labelled as data, not instructions. This helps, but the guarantees above don't depend on it.
• Minimal data: attachments are not sent by default; bodies are trimmed of quoted history and signatures; hidden HTML text is dropped by the sanitizer before the model sees it.

5.7 Build steps

1. Plugin skeleton, provider adapter (Anthropic), admin settings (key, model per feature, quotas), per-user opt-in screen, usage table.
2. Thread summary (streaming) + rewrite in compose.
3. Reply drafts with tone from the user's sent mail.
4. Categorization (structured output, batch backfill) → feeds split inbox (§3.1).
5. Ask your mailbox (read-only tool loop over Meilisearch).
6. Event extraction → calendar plugin.
7. Optional: Vertex/Bedrock and local-model adapters; bring-your-own-key.