lukas/webmail-client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Productivity features, UI translations, local email translation

Browse Source 
- Snooze (with a Snoozed folder), keyboard shortcuts with a Settings
  section, mail templates (Compose > More > Templates), a Subscriptions
  folder with bulk unsubscribe/delete, and tracker blocking with a
  Privacy toggle (on by default)
- Own dialogs instead of browser popups; emoji picker in compose; compose
  windows survive a visit to Settings
- Sentence completion learned from the user's sent mail (no AI)
- Interface in English and Polish (one file per language, English as the
  fallback), picked from the browser on first run, then kept
- Email translation on the server via LibreTranslate: Translate button
  and "Translate to...", messages in several languages translated part
  by part, names/addresses/emoji/file names/phone numbers kept, an
  animated line while translating
- Antispam "Mark as spam" loads with the message, not when the menu opens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
lukas
2026-10-18 20:00:31 +02:00
co-authored by Claude Opus 5.5
parent 463f96ad57
commit 8a1b9979e9
83 changed files with 8677 additions and 293 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/feature-ideas.md
+298
View File
@@ -0,0 +1,298 @@
# Feature ideas — making Ultimo competitive with Gmail
Status (2026-10-18): **built** (uncommitted at the time of writing) — snooze,
keyboard shortcuts, subscriptions page + unsubscribe, templates, tracker
blocking. Left out of that first pass: follow-up reminders (§1.1), the image
proxy (§2.1), the footer-link fallback and "auto-archive in future" filter on the
Subscriptions page (§1.3), and bulk snooze from the list. Everything else here is
still a proposal. Each idea notes
what it is, why it matters, how it fits the existing architecture, and a rough
size (S = days, M = 1–2 weeks, L = several weeks).
Already built, so not repeated here: undo send, scheduled send, labels, filters,
antispam, search operators (Meilisearch), VIP contacts, calendar (CalDAV, iMIP),
contacts (CardDAV), offline read-only mode, PWA + Web Push, 2FA, invitations,
hosted mailboxes (Stalwart).
---
## 1. What people miss when leaving Gmail
### 1.1 Snooze and follow-up reminders — M
- **Snooze:** "bring this back Monday 09:00". The thread leaves the inbox and
returns at the chosen time, marked unread and pinned to the top.
- **Follow-up:** when sending, "remind me if nobody replies in 3 days". If no
reply has arrived in the thread by then, it comes back to the inbox with a
"No reply yet" chip.
- **Fit:** a `snoozes` table (thread, user, wake_at, kind) plus an arq cron job
that wakes due threads. Snoozed threads move to a server-side "Snoozed" folder
so other IMAP clients see the same state. The follow-up check watches
`message.new` events for the thread. Could be a builtin plugin.
### 1.2 Keyboard shortcuts — S/M
- Gmail-compatible: `j/k` next/previous, `o`/Enter open, `u` back to list,
`r`/`a`/`f` reply / reply-all / forward, `e` archive, `#` delete, `s` star,
`l` label, `v` move, `/` search, `c` compose, `x` select, `Shift+I/U`
read/unread, `g i` / `g s` go to inbox / starred, `?` shows them all.
- **Fit:** frontend only. One central key handler that respects focus in
inputs and the compose editor, plus a setting to turn it off.
### 1.3 One-click unsubscribe and a subscriptions page — M
- An "Unsubscribe" button on mail that carries `List-Unsubscribe`, using the
one-click POST (RFC 8058) when offered, `mailto:` otherwise.
- A **Subscriptions** page: every sender who sends mailing-list mail, how often,
when last read, with Unsubscribe / "Delete all from this sender" /
"Auto-archive in future" (creates a filter).
- **How subscriptions are found:** no service records what a user signed up
for, so the page infers it from received mail (as Gmail does). A message is
bulk mail when it carries `List-Unsubscribe` (required of bulk senders by
Gmail/Yahoo since 2024), `List-Id`, `Precedence: bulk|list`, or mailing-service
markers (`Feedback-ID`, ESP headers). A footer "unsubscribe" link is a weaker
fallback, shown as "probably a newsletter". Mail is grouped by `List-Id`, else
by sender address.
- **Per group:** messages per month, last received, and **read rate** from the
`\Seen` flag ("opened 0 of 23"), sorted most frequent / least read first.
- **After unsubscribing:** mark the group unsubscribed with the date. If mail
still arrives after a grace period (RFC 8058 senders should stop within 2
days), flag "still sending" and offer block / auto-archive via a filter.
- **Limits:** lists that haven't sent anything yet, or send no list headers,
can't be seen. The page shows "bulk senders currently mailing you".
- **Fit:** the plugin SDK can read raw headers
(`backend/app/plugins/actions.py`), but it parses the stored RFC822 per
message, and with hosted mailboxes the raw message is only cached. So extract
`List-Id`, `List-Unsubscribe` and `List-Unsubscribe-Post` during sync into a
small `message_list_headers` table, backfilled once for existing mail. The
spec (§12, M3) already describes the button as an example `message.actions`
plugin. The one-click POST goes from the server, through the SSRF guard.
### 1.4 Templates (canned replies) — S
- Saved replies with placeholders (`{{first_name}}`, `{{my_name}}`), inserted
from a compose toolbar menu or by typing `/` in the editor.
- **Fit:** per-user table plus a compose-toolbar plugin slot.
### 1.5 Out-of-office reply — M
- Start/end dates, subject, message; replies at most once per sender per N days;
never to mailing lists or `Auto-Submitted` mail (RFC 3834).
- **Fit:** for hosted mailboxes, write a Sieve `vacation` script through
Stalwart. For external accounts, the outbox sends the reply from an arq job
on `message.new`.
---
## 2. Where Ultimo can beat Gmail
### 2.1 Privacy as the selling point — M
- **Tracker blocking:** detect tracking pixels (1×1 images, known tracker
domains, per-recipient URL tokens) and strip them even when remote images are
allowed. Show "3 trackers blocked" on the message.
- **Image proxy:** when remote images are allowed, load them through the server
(SSRF-guarded, cached), so senders never see the reader's IP address or when
the mail was opened.
- **Aliases on the hosted domain:** unlimited addresses such as
`shop.amazon@ultimo.email` or `name+tag@`. Each one can be switched off, and
the app shows which alias leaked to spammers.
- **Fit:** the sanitizer (`backend/app/indexer/sanitize.py`) already rewrites
HTML and blocks remote images. Aliases are Stalwart principals or catch-all
rules managed from the M6 hosted-mailbox code.
### 2.2 Encrypted mail (OpenPGP) — L
- Optional, done in the browser (OpenPGP.js), so the **private key never
reaches the server unencrypted**.
- Decrypt and verify received PGP/MIME mail. Sign and encrypt when sending.
Look up recipients' keys automatically (WKD, keys.openpgp.org) and publish
WKD for the hosted domain.
- **Fit:** compose and reading-panel plugin slots. The key backup is encrypted
with a passphrase before upload. Indexing can't search encrypted bodies (say
so in the UI).
### 2.3 AI assistant (opt-in) — L
Detailed design in **§5** below.
---
## 3. Productivity polish
### 3.1 Split inbox / categories — M
Tabs such as *Important / Newsletters / Notifications / Receipts / Other*,
filled by rules (sender, `List-Id`, labels, filters) and optionally by the AI
classifier (§5.3). The user can move a sender between tabs and the app learns
from it.
### 3.2 Everything you've received — S/M
A view listing all **attachments, links and photos**, filterable by sender, type
and date, with preview. Builds on the right panel's Attachments tab and the
deduplicated attachment store.
### 3.3 Mute a thread; send and archive — S
- **Mute:** new replies in the thread skip the inbox (still searchable). Good for
noisy reply-all threads.
- **Send and archive:** one button that sends the reply and archives the thread.
### 3.4 Easy move from Gmail — M
- Copy existing mail from Gmail over IMAP (Google OAuth is already planned),
with progress, resume, and labels → labels.
- Import Gmail filters (Gmail exports them as XML) into the filters plugin.
- Import contacts (vCard/CSV) and calendars (ICS).
- The hardest part of winning users is getting them to switch. This makes
switching painless.
### 3.5 Calendar from mail — M
- "Create event from this mail": date, time and place pre-filled from the text
(rules first, AI optional).
- "Find a time": free/busy for the thread's participants (for those whose
calendar we can see), then send an invitation.
- "Turn into task" once a tasks list exists (VTODO over CalDAV).
### 3.6 Shared mailboxes and delegation — L
For families and small teams:
- A shared `info@` mailbox visible to several users.
- Assign a thread to someone, with status (open / waiting / done).
- Internal notes on a thread that are never sent.
- Delegated access ("my assistant can read and send as me").
---
## 4. Suggested order
| Milestone | Contents | Why |
|---|---|---|
| **Next** | Snooze + follow-up, keyboard shortcuts, unsubscribe/subscriptions, templates, tracker blocking + image proxy | Moderate size, fits the current architecture, users notice right away; starts the privacy story |
| Then | Out-of-office reply, mute / send and archive, everything-received view, Gmail import | Smaller wins + removes the switching barrier |
| Later | AI assistant (§5), split inbox, calendar from mail | AI unlocks smart categories and event extraction |
| Separate | OpenPGP, aliases, shared mailboxes | Larger security / multi-user projects |
---
## 5. AI assistant — design
### 5.1 Principles
1. **Off by default, opt-in twice.** The administrator turns the plugin on and
chooses a provider. Each user then opts in, per feature. Nothing about a
user's mail is sent anywhere until they switch a feature on.
2. **The server calls the AI, never the browser.** API keys stay on the server,
encrypted with the vault (`MASTER_KEY`), like mail-account passwords.
3. **The AI suggests, the user acts.** No AI output sends mail, deletes, moves
or unsubscribes on its own. Drafts land in the compose editor; categories
are labels the user can change.
4. **Mail is untrusted input** (see §5.6). An email can contain text written to
manipulate the model ("ignore your instructions and…"). The design makes
this harmless rather than trying to detect it.
5. **Provider-independent.** One small adapter interface; Anthropic (Claude) is
the default. Other providers or a local model can be plugged in.
### 5.2 Features
| Feature | Trigger | What is sent to the model | Output |
|---|---|---|---|
| **Thread summary** | User clicks "Summarize" (or auto for threads over N messages, if enabled) | Plain-text bodies of the thread (quoted history stripped), senders, dates | 3–5 bullets + "what's asked of you" + deadlines; streamed into the reading panel |
| **Reply drafts** | "Suggest reply" in the reading panel, or a short instruction ("decline politely, offer next week") | The thread + the user's instruction + their signature/name + optional tone examples from their own sent mail | A draft inserted into the compose editor; the user edits and sends |
| **Rewrite** | Select text in compose → "Shorter / More formal / Friendlier / Fix grammar / Translate to …" | Only the selected text | Replacement text with a preview and undo |
| **Categorization** | Background, on each new message (only for users who opted in) | Sender, subject, `List-*` headers, first ~2,000 characters of text | One category from a fixed list (structured output) + optional "needs reply" flag → label |
| **Ask your mailbox** | A search-bar mode: "When does my car insurance expire?" | Only the messages that a search finds relevant (see §5.4) | An answer with links to the source messages |
| **Event extraction** | "Create event" on a message | That message | Title, start/end, place, attendees as structured data → pre-filled event form |
### 5.3 How it plugs into Ultimo
- **A new builtin plugin `webmail_plugin_ai`**, using the existing SDK:
reading-panel and compose slots for the buttons, its own tables for settings,
usage and cached results, and arq jobs for background work.
- **Provider adapter:** `complete(system, messages, schema=None, stream=False)`,
with implementations for Anthropic (official `anthropic` Python SDK), and
later Bedrock / Vertex / a local OpenAI-compatible server (Ollama, vLLM).
- **Streaming:** summaries and drafts stream token by token to the browser over
the existing WebSocket (`backend/app/ws.py`), so text appears right away.
- **Structured outputs:** categorization and event extraction use the API's
structured output mode (`output_config.format` / `messages.parse()` against
a JSON schema). The model can only answer with an allowed category, so a
malicious mail can't make it invent a "trusted" label.
- **Caching results:** a summary is stored keyed by thread + the newest message
id, so reopening a thread costs nothing until a new message arrives.
- **Prompt caching:** the fixed instructions (system prompt, category list,
style guide) come first and are marked cacheable, so repeated calls pay for
them at the much cheaper cache-read rate. Volatile parts (the email text, the
date) come after.
- **Bulk backlog:** classifying existing mail when a user first opts in goes
through the Message Batches API (asynchronous, half price).
- **Quotas and audit:** per-user monthly token budget set by the admin; a usage
table (user, feature, model, input/output tokens) shown in the admin panel.
Calls are logged without message content.
- **Refusals and errors:** check `stop_reason` (including `refusal`) before using
output; on rate limits, retry later (arq retry); the UI says "AI unavailable"
rather than failing the mail action.
### 5.4 "Ask your mailbox" in detail
Searching all mail with the model would be slow and expensive, so this uses
retrieval:
1. The question goes to the model together with **one read-only tool**,
`search_mail(query, from?, after?, before?)`, backed by the existing
Meilisearch index and run **as that user** (row-level security; it can only
see their own mail).
2. The model makes one or more searches; the server returns short snippets
(sender, date, subject, ~500 characters).
3. If needed, the model asks for the full text of a few messages with a second
read-only tool, `read_message(id)`.
4. It answers and cites message ids; the UI turns them into links.
There are no tools that write anything. Keyword search is enough to start.
Semantic search (embeddings in pgvector) could come later, but would need a
separate embeddings provider or local model, since that's a different service
from text generation.
### 5.5 Who provides the AI
The administrator chooses one in the admin panel:
| Option | Privacy | Quality | Cost | Notes |
|---|---|---|---|---|
| **Anthropic Claude API** (default) | Mail text leaves the server for Anthropic | Best | Pay per token | Simplest; official Python SDK; streaming, structured outputs, prompt caching, batches |
| **Claude through Google Vertex AI or Amazon Bedrock** | Same model, contract and data processing with Google/AWS; regional endpoints (e.g. EU) | Best | Partner pricing | Choose if EU data residency or an existing cloud contract matters |
| **Local model** (Ollama / vLLM on a GPU machine) | Nothing leaves your network | Noticeably weaker; fine for categorization and rewriting, weaker for summaries and Q&A | Hardware + electricity | The QNAP has no GPU — CPU-only is too slow for interactive use; needs a separate box |
| **Bring your own key** (per user) | As the chosen provider | — | User pays | Optional; lets power users use their own account |
**Models (Anthropic, prices cached 2026-10-06, per million tokens in / out):**
| Model | Price | Suggested use |
|---|---|---|
| `claude-opus-5-5` | $4 / $20 | Default for everything; best quality for drafts, summaries, Q&A |
| `claude-sonnet-5-5` | $2 / $10 | Option for drafts / summaries if cost matters |
| `claude-haiku-5-5` | $0.10 / $0.50 | Option for high-volume background work (categorization) |
The model is configurable **per feature** in the admin panel. Default to Opus 5.5
everywhere and switch individual features down only after checking that quality
holds. Categorization is the one where volume matters. Example: 500 new
messages/day × ~1,000 tokens ≈ 15M input tokens/month ≈ $60/month on Opus 5.5
vs. ≈ $1.50 on Haiku 5.5 (before caching). Interactive features are cheap
either way: a summary of a long thread is a few thousand tokens. Prices change;
check the provider's current price list before deciding.
**Data handling to verify before launch:** the provider's commercial terms on
training (Anthropic's API terms state that API inputs are not used for model
training by default — confirm the current wording), how long requests are
retained, and a data processing agreement (DPA). Hosting other people's mail in
the EU makes Ultimo the GDPR controller, so the opt-in screen must say which
provider receives the text.
### 5.6 Security: mail as untrusted input
- **No side effects from model output.** Every action needs a user click. Drafts
are never sent automatically.
- **Read-only tools only**, always running as the requesting user (RLS), so an
email that says "search for passwords and include them in the reply" can at
most show the user something from their own mailbox, to the user.
- **Fixed output shapes** (structured outputs) for anything that changes state,
like labels.
- **No remote content:** links in model output are shown as text or go through
the normal link-safety warning; images are never loaded from model output.
- **Clear boundaries in the prompt:** email text is wrapped and labelled as
data, not instructions. This helps, but the guarantees above don't depend on
it.
- **Minimal data:** attachments are not sent by default; bodies are trimmed of
quoted history and signatures; hidden HTML text is dropped by the sanitizer
before the model sees it.
### 5.7 Build steps
1. Plugin skeleton, provider adapter (Anthropic), admin settings (key, model per
feature, quotas), per-user opt-in screen, usage table.
2. Thread summary (streaming) + rewrite in compose.
3. Reply drafts with tone from the user's sent mail.
4. Categorization (structured output, batch backfill) → feeds split inbox (§3.1).
5. Ask your mailbox (read-only tool loop over Meilisearch).
6. Event extraction → calendar plugin.
7. Optional: Vertex/Bedrock and local-model adapters; bring-your-own-key.