Commit Graph
62 Commits
Author SHA1 Message Date
lukasandClaude Opus 5.5 463f96ad57 Invite a friend; handle field shows the domain; system mail fixes
- Invitations: each user can invite two friends by email (sign-up works
  through the link even while registration is closed); admin-editable
  template, subject and sender.
- Invitation sender defaults to the system-mail From address, which the
  SMTP account is allowed to send as (was postmaster@APP_DOMAIN, refused
  with 553 when the two differ).
- Sign-in, sign-up and forgot-password fields show the fixed
  @domain after the typed name; the registration status reports it.
- System mail gets a Date header; the invitation's plain-text part keeps
  its paragraphs and line breaks.
- SMTP EHLO uses SMTP_HELO_NAME / APP_DOMAIN, not the container name.
- DNS recommendations for the mail domains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:34:31 +02:00
lukasandClaude Opus 5.5 addb2232a1 Security hardening from the assessment; testing fixes
Security (ultimo-security-report.md):
- Security headers on every response, the app shell included: they lived at
  server level and nginx dropped them in any location with its own header.
  CSP, framing, nosniff, referrer, permissions (docker/nginx-security.conf).
- Plain HTTP forwarded by the reverse proxy is redirected to HTTPS; HSTS on
  HTTPS responses. server_tokens off.
- Cross-site state-changing /api requests refused (login CSRF), WebSocket too.
- Validation errors in the usual error shape, without the request model.
- New-password policy: max 128, common/own-address/repeated refused,
  breached-password check (HIBP k-anonymity, fails open).
- Message HTML: links keep only http(s)/mailto (sanitizer v6); the HTML
  endpoint carries its own no-script CSP.
- Outbound fetches pinned to the checked IP (DNS rebinding); external
  IMAP/SMTP servers re-checked before each connection; CGNAT and mapped
  addresses counted as non-public.

Fixes from testing:
- IDLE loop catches up before idling: mail that came while the worker was
  away no longer waits up to 10 minutes.
- Restore from Spam/Trash/Archive moves only the removed messages, not your
  reply in Sent.
- Narrow desktop list stacks sender/subject/snippet (subject was squeezed
  to nothing at the default width), with hover actions.
- Phone: wide emails scale to fit the screen.
- Singular forms ("1 message"); no zip button for a single attachment;
  app passwords no longer claim "never used" for mail apps.
- ruff format across the backend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:08:06 +02:00
lukasandClaude Opus 5.5 b88855931e M6 hosted mailboxes (off by default); dark message mode; fixes
Hosted mailboxes (SPEC App. D), behind HOSTED_MAILBOXES (off: unchanged):
- Stalwart mail server, provisioned by `python -m app.hosted.provision`;
  directory read from a Postgres view, app passwords only for mail apps
- native account per user, the worker signs in as master for the user
- raw messages of hosted mailboxes only cached, fetched back on demand
- migrations 0025 (accounts.kind, directory view), 0026 (IMAP ids bigint)

Mail:
- dark message mode switch in the reading pane (Thunderbird's algorithm)
- moves and folder creation quote the folder name ("Deleted Items"):
  swipe-to-delete / Trash no longer bounce back
- thread unread counters: lock before recount, sorted lock order,
  UIDVALIDITY re-import recounts, nightly repair_threads
- sidebar accounts sorted alphabetically, more space above headings

Settings: Security headings renamed; antispam accounts are explicit
(none ticked = none filtered), input widths, Activity hidden.
Plugin bundles served with Cache-Control: no-cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:25:10 +02:00
lukasandClaude Opus 5.5 0e3793a4d6 App passwords; CalDAV / CardDAV server for phones, Thunderbird, macOS
App passwords (SPEC App. D §D.5): one generated password per app, shown once,
revocable on its own, revoked on a password reset; never accepted by the web
sign-in. Settings → Security → App passwords (migration 0024).

CalDAV / CardDAV server (SPEC Appendix E), built in: /dav/ with
/.well-known/caldav|carddav, HTTP Basic with app passwords only (cached checks,
throttled failures). The calendar and contacts plugins serve their own tables as
collections (Plugin(dav=...)):
- calendars as iCalendar: series with overrides, time zones, all-day, attendees
  and RSVP, alarms <-> reminders; MKCALENDAR / PROPPATCH / DELETE of calendars;
  subscriptions read-only; writes to upstream-synced calendars flow on upstream;
- address books as vCard (3.0 served, 3.0/4.0 read), groups as CATEGORIES;
- what a client stores is served back verbatim until the web app edits it;
- sync tokens / CTags from change logs filled by triggers (calendar0008,
  contacts0002), so every write path counts; trimmed nightly after 90 days.
REPORTs: calendar-query (time range), calendar-/addressbook-multiget,
addressbook-query, sync-collection. nginx and the Vite proxy route /dav/.
Settings shows the server and user name to enter; install guide §5.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:49:04 +02:00
lukasandClaude Opus 5.5 fa552392c8 Calendar reminders as push notifications
A job every minute finds the event reminders that fell due — repeating events
and moved occurrences expanded as the calendar view does — and pushes each
once to every device with notifications on (sdk.push, new: plugins can notify
through the new-mail push delivery). A reminder missed while the worker was down
is still sent within 10 minutes; cancelled events and declined invitations stay
quiet. Title, then "In 15 minutes, 14:00 · Room 2" in the user's zone and clock
format; it stays until dismissed and shows even with the app open; clicking it
opens that day (/p/calendar?date=…, new deep link).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:49:04 +02:00
lukasandClaude Opus 5.5 5bf8fc81f6 Search: all results with a real count; bulk actions on results stay done
Search showed at most the first 50 matching messages ("1–50 of 50"). The
first page now carries the number of matching conversations, further pages
load as the list is paged through (a conversation spanning two appears once;
oldest-first loads them all, up to 40 pages), and the Unread/Pinned tabs filter
on the server, so their count covers every result.

Bulk Delete / Archive / Restore over selected results acts at once: the rows
leave the list (greyed instead when archived from a search, which still finds
them), the search results are refreshed, and the reading pane returns to
"Select a conversation". Each move landing on the server refreshes them again,
and a hidden row only comes back when no move has landed for 30 seconds — a
long batch no longer brings deleted results back.

Settings → Accounts → Notifications: shorter description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:15:06 +02:00
lukasandClaude Opus 5.5 3957dd540a Attachments stored once; nightly clean-up of unused files
Attachments are no longer stored apart from their message (they were kept
twice: base64 inside the raw message and decoded on their own). The indexer
records the MIME part (attachments.part_index, migration 0023); downloads,
zip, inline images, thumbnails and forwarding take it out of the message;
videos stream from a short-lived copy (kept a day, 1 GiB per user). A
background pass on worker start removes the copies stored before, each only
after checking it can be taken out again (spec App. D §D.6 item 1, criterion
corrected to <= 60%).

A nightly job (03:47, after the Trash/Spam purge) removes files nothing
refers to any more — deleted or purged messages, previews of attachments
gone, uploads no draft has used for a week, expired streaming copies. Files
younger than an hour are left alone; a sent message waiting for its Sent copy
is held until its job is done. Account removal keeps files a draft still uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:20:21 +02:00
lukasandClaude Opus 5.5 0514053252 Messages size at once; quoted inline images travel inside the reply
The message frame is set up as soon as its document is parsed instead of at
its load event, which waits for every image (remote ones included): it no
longer sits at its first height until the slowest image arrives.

A reply quoted an original's inline (cid:) images as this server's own URLs,
and they were sent like that — broken for the recipient, and mangled copies
("http:///api/…") made browsers look up a host named "api". Outgoing mail
(and the IMAP Drafts copy) now carries them as inline parts with new
Content-IDs; one that can no longer be had is left out. Shown messages load
such URLs from this server (sanitizer v5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:20:21 +02:00
lukasandClaude Opus 5.5 fda4cc6f16 Installable app, push notifications about new mail, Install app entry
The webmail is now a PWA: web app manifest, icons (any + maskable, a status-bar
badge), theme-coloured title bar following the chosen theme, and an app-icon
badge with the unread Inbox count. The service worker is always installed;
offline access is on only when it is registered as /sw.js?offline=1, so
turning it on or off is an update that applies at once.

Notifications (Settings → Accounts → Notifications, per device) use Web Push.
Encryption (RFC 8291) and VAPID signing (RFC 8292) are implemented in
app/push/webpush.py — pywebpush needs a newer cryptography than we pin — and
checked against the RFC's worked example. The server makes its key pair on
first use (private half encrypted with MASTER_KEY). New mail found by the live
sync in an Inbox enqueues a job 5 s later that tells the user's devices about
messages still unread there, under 2 hours old and not from the user; more
than three become one "N new messages". A subscription belongs to the session
it was made under: signed out or revoked, nothing more is sent there. Endpoints
are limited to the known push services (no SSRF). Nothing shows while the app
is in front (except on Apple, which requires it). Tested with real pushes
through Google's service in Chromium and Opera.

The mobile drawer gets "Install app": Chrome's own install dialog where the
browser offers one, else short steps for iPhone/iPad, Opera and Firefox, with
"Don't show this again". docs/nas-install.md §5.3 covers installing,
notifications, Public URL and a later Android app (TWA).

Migration 0022: push_subscriptions, app_settings.vapid_*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:52:26 +02:00
lukasandClaude Opus 5.5 1982e8964b Calendar: cancel events for guests, apply organizers' cancellations, invitation settings
Deleting an event the user organizes, with other guests, asks "Would you like
to inform the participants that this event is canceled?" (Yes, inform them and
delete / No, just delete / Cancel; for a recurring one after which occurrences).
Yes sends each guest an iTIP CANCEL from the organizer's mailbox — of one
occurrence via RECURRENCE-ID — or, for "this and following", the shortened
series. Invitations now carry their RRULE, so recurring events repeat for
guests too.

An organizer's cancellation is applied as the email arrives (message.new hook)
or, for older mail, when the card is opened — always, whatever the settings:
only from that event's organizer and not older than the calendar's copy; one
occurrence becomes an EXDATE. The card says "Event was removed from calendar".

Settings → Calendars → Invitations now work, defaulting to "Do not
automatically update my calendar": "Only update events that I have already
added" applies the organizer's changes (keeping the user's answer), "Add new
events…" also adds new invitations, unanswered — from contacts or anyone — to
the chosen calendar, and "mark as read and file" archives such messages. With
no automatic updates, a changed invitation says so on its card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:50:17 +02:00
lukasandClaude Opus 5.5 2337375bb7 Event invitations in Mail, video previews, People tab, matching avatars
Invitations received by email (iMIP): above the message body, a card with
when, what, where (or "Join video call"), the organizer, what else is in the
calendar then (a conflict, else the previous event that day), and Yes / No /
Maybe. Yes and Maybe put the event in the calendar (or update it), No takes
it out — and is still shown as the answer given; each sends the organizer
the iTIP REPLY, then a "Reply sent" toast, and an open calendar refreshes.
A cancellation offers "Remove from calendar". It is the calendar plugin's
message.banner slot (ui/invite.js, imip.py, /invitations endpoints); the SDK
gains the user's own accounts and settings. An Outlook-style invitation sent
only as text/calendar is now kept as invite.ics too.

Calendar: clicking an empty day in the side panel's agenda opens a new event
on that date; a video-call link can be removed (✕ beside Copy); the page
guests see after answering, and our invitations' footer, say Ultimo Calendar
(with the logo).

Attachments: videos (mp4, mov, webm, …) play in the full-screen preview,
paused until played; attachments are now served from disk with HTTP Range,
so a video starts at once and can be seeked.

People tab: everyone in the thread, the user included, A–Z, under the name
any message gives them. Initials circles share one colour rule in the list,
the reading pane and the People tab. The contact view drops its duplicate
"Message" button. A general corner toast (webmail:toast) joins Undo's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 21:50:29 +02:00
lukasandClaude Opus 5.5 fba3c947bf Attachment previews, forwarded attachments, row dividers
Attachments in the reading pane are tiles with a preview: images and a PDF's
first page, rendered on the server (Pillow, pypdfium2) as WebP of at most
480 px, cached beside the blob store and in the browser. Tiles are all one
size — at most 230 px wide, narrower so three still fit in a row — and frame
each preview by its shape: close to the tile's it fills it, very tall / wide /
square ones show whole over a blurred copy, small ones are not enlarged, PDFs
show the top of the page. The name bar (checkbox, type, name, size, download)
slides in on hover and adapts to the tile's width; files without a preview
get a large type icon. Previews of opened conversations are kept offline.

Forward now carries the original's attachments: new compose attachments
sharing the stored blobs (POST /messages/{id}/forward-attachments).

Mailbox display / Settings → Appearance: "Show a dividing line between
rows" (off by default, per device), with a new switch. Settings drop the
"On a phone…" note, and "Language" reads "Spellchecking language".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:15:17 +02:00
lukasandClaude Opus 5.5 bfe011cd2c Fix: IMAP over SSL connected without TLS; never lose a sent message's copy
v1.3.26's IMAP4_SSL subclass put its create_client ahead of aioimaplib's,
which is where the TLS context is made — so port 993 got plain TCP, the
server waited for a handshake, and every SSL account timed out ("Can't
reach the mail server"). The context is now made in our create_client too,
and a test checks an SSL client really gets one (it fails without the fix).

With IMAP down, a send could still be delivered by SMTP while filing its copy
in Sent failed — the send then counted as done and the copy was lost. The
copy is now kept and filed by file_sent_copy_job, retried (arq Retry) for
about 24 hours, then shown in Sent as usual.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 22:24:23 +02:00
lukasandClaude Opus 5.5 609e7c58f9 Unreachable mail servers: send retries in Scheduled, real action retries, account health
Sending: a send the server could not take (unreachable, timeout, 4xx) is no
longer failed at once. It waits in Scheduled ("retrying") and is tried again
after 1, 2, 5, 10, 30 min, then hourly, for 24 hours — on the server, so it
works with the browser closed. A permanent refusal (5xx, wrong password) is
"failed" at once, with the server's reason. GET /outbox lists both;
POST /outbox/{id}/retry (Retry now) and /cancel (Edit, Delete) act on them,
and they are left out of Drafts. Mail delivered but not filed in Sent now
counts as sent, and a conditional claim keeps two jobs from sending one
message twice. The app says when a send failed and when it finally went.

Actions: arq retries a job only on Retry, never on a plain exception, so the
"retry x5, then revert" for flags and moves never happened. Flag, move,
archive, restore, delete and empty-folder jobs now raise Retry while the
server is unreachable (~24 hours of growing gaps), skip a change a later one
replaced, stop at the first connection failure instead of 20 s per message,
and delete/empty keep the rows until the server has the messages gone.

Account health: 3 failed connections in a row mark an account unreachable
(amber in the sidebar and Settings, cleared on the next success); 3 refused
sign-ins stop its syncing as auth_failed until the password is entered
again, so the provider does not lock the account. Sync loops back off 30 s
doubling to 10 min and log one line per expected failure; a refused
connection fails at once with its reason (the connect task is kept), and a
silently dead IDLE connection is noticed within ~11 min instead of ~29.

Also: the empty reading pane is clipped to its width when dragged very
narrow; tests no longer leave live accounts and outbox rows in the database.

Migration 0021: outbox attempts / next_attempt_at / retry_until, accounts
sync_error / sync_error_since.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 22:08:42 +02:00
lukasandClaude Opus 5.5 e1bda37188 Offline compose: write mail offline, Scheduled folder, undo countdown fix
Compose, Reply and Forward work offline. Drafts are kept on the device
(IndexedDB, lib/scheduled.ts), with files attached offline; Send becomes
"Schedule" and puts the message in a new Scheduled folder. When the server
is reachable again, scheduled messages are sent through the normal outbox
(Undo offered for a single one) and offline drafts become server drafts —
files uploaded first, each step recorded so nothing is created or sent
twice, under a Web Lock so two tabs never send the same message. Drafts
written offline show in Drafts as "On this device". Sign-out sends what
is scheduled, then wipes the device's copy.

Also: offline failures from keyboard shortcuts show "Not available
offline"; no recipient suggestions are asked for offline; a reply no
longer quotes an error page when the original is not saved.

The undo countdown counted to the server's deadline on the browser's
clock, so a few seconds of skew showed (18 s instead of 10). /send now
also returns undo_seconds, and the composer counts down from that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 21:11:41 +02:00
lukasandClaude Opus 5.5 1f2e539eb2 Compose: spelling and grammar check with LanguageTool
A self-hosted LanguageTool (dev: docker/docker-compose.yml, started by
`make up`) checks what the user writes in the compose window, through
POST /api/spellcheck — the browser never talks to the engine directly.

- Marks are CSS highlights over the text, never written into the message: red
  wavy for spelling, blue for grammar/style. Clicking one shows the message,
  up to five suggestions (applied as typed, so Ctrl+Z undoes them) and Ignore.
  The popup, like the link popup, opens above its anchor when there is no
  room below.
- Checked after a pause in typing; the signature and quoted original are
  skipped; marks on words just edited drop at once; a check overtaken by a
  newer one, by turning checking off, or by closing the window is discarded.
- With "auto", text is checked in chunks of lines, each in its own detected
  language — LanguageTool settles on one language per request, so a mixed
  Polish/English message had every English word flagged as a Polish typo.
- Settings > Appearance: on/off and language (Auto-detect or any of the
  engine's languages); the compose window's More menu overrides it for one
  message. The browser's own spellcheck steps aside while ours runs, and comes
  back when it is off or the server has no engine (503 from the endpoint).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 23:51:31 +02:00
lukasandClaude Opus 5.5 165442d897 Logo in system mail; "don't ask for the 2FA code on this device"
System mail: the password-reset email (and the admin panel's test email) now
opens with the Ultimo logo, centered. It is a PNG rendered from
icons/ultimo-logo.svg and embedded in the message (multipart/related, cid:),
so it shows where remote images are blocked and in clients that do not
render SVG. The PNG is listed in package-data so the Docker image carries it.

Trusted devices: a 2FA sign-in offers "Don't ask for the code on this device
in future". Ticked with a correct code, the browser gets its own HttpOnly
trusted-device cookie (token in Redis, 90 days, renewed on use); later
sign-ins to the same account from that browser need only the password. Other
accounts and browsers are unaffected. The trust is revoked on a password
reset, when 2FA is turned off, and when an admin blocks or deletes the user;
signing out keeps it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 21:02:44 +02:00
lukasandClaude Opus 5.5 0ddc0cb001 Admin user management, password reset by email, and the Ultimo Email name
Branding: the page titles and app name are now "Ultimo Email".

Admin panel (now two columns with a divider):
- Users: search by login or by a mail account's address or name (results
  only, nothing listed until something is typed). A user opens a window to
  change their quota, block/unblock them, or delete them; block and delete ask
  for confirmation, and an admin cannot do either to their own account.
- Blocking (users.blocked_at, migration 0019) refuses sign-in with a clear
  message and revokes every session; a new user.signed_out event sends the
  user's open tabs to the sign-in screen at once. Any "not signed in" answer
  now does the same, not only /auth/me. Deleting removes mail accounts (sync
  stops), plugin rows (cleared inside the user's RLS context, since plugin
  tables have no FK to users), blobs, the search index, sessions and the user.
- Outgoing system mail (migration 0020): SMTP server, port, security, login,
  password (vault-encrypted, never sent back), From address and the public
  address reset links point to, plus a "Send test" that reports the server's
  error.

Password reset:
- "Forgot password?" on the sign-in page asks for the user name and always
  answers "If this account exists you will receive an email to your recovery
  email address"; the mail goes out after the response, so neither the reply
  nor its timing reveals an account, and it is capped at 3 per hour.
- The emailed link (token in the URL fragment, single use, 30 minutes) opens a
  page to set a new password. With 2-step verification on, the code is checked
  before anything changes; five wrong codes burn the link.
- Settings > Profile: the recovery email can be added, changed or removed,
  confirmed with the current password, which must be typed — autofill and
  paste are refused (TypedPasswordInput).

Also adds frontend/icons/ultimo-logo-bimi.svg, an SVG Tiny PS version of the
logo for a BIMI record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 20:16:27 +02:00
lukasandClaude Opus 5.5 8dc60dbf5c Avatars: a sender without one logged a console error; phone list rows spaced 20px
A missing avatar was a 404, which the browser reports as an error for every
such sender on every page. It is now an empty 204: the <img> still fails to
decode it and the initials show through, but nothing is logged, and the miss
carries the same one-day browser cache as a hit so it stops being re-fetched.

On a phone the mail list's rows sit 20px apart instead of 7px.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 21:35:26 +02:00
lukasandClaude Opus 5.5 caed8543ea Avatars: a cached avatar or miss failed on every later request
The shared Redis client decodes replies as UTF-8 text, but the avatar cache
holds raw image bytes and a b"none" miss marker: an image could not be read
back at all, and a miss came back as str and crashed on .partition(b"\n").
Only a sender's first request worked; after that, a 500. The cache now goes
through a binary-safe client, get_redis_bytes().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 08:09:09 +02:00
lukasandClaude Opus 5.5 02011f23f6 Sanitizer: a newsletter's <title> showed as a line above the body
<title> is not an allowed tag, so nh3 unwrapped it and kept its text —
"mailer", "Email template" or the subject rendered before the content.
It is now dropped with its content, both in the sanitized HTML and in the
text derived from HTML-only mail (snippet and search). SANITIZER_VERSION
goes to 4 so stored bodies re-sanitize from the raw blob on next render.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 20:35:42 +02:00
lukasandClaude Opus 5.5 08d428de8a Mail list: name everyone who wrote in the thread, and call the user "me"
The sender column showed participants[0], and participants are rebuilt
newest message first — so replying to a thread put the user in that slot
and the list named them instead of the person who wrote in.

Thread summaries (list and search) now carry `senders`: the distinct From
addresses of the thread's visible messages, oldest first, with the same
Trash/Spam visibility as the message count. The list joins them and shows
the user's own account addresses as "me" ("me, Alice" for a thread the
user started), and the avatar prefers the other side of the conversation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 20:17:37 +02:00
lukas 428e81efa5 Revert "Mail list: show who wrote, not everyone involved"
This reverts commit 550b8cf503.
2026-09-18 18:31:16 +02:00
lukasandClaude Opus 5 550b8cf503 Mail list: show who wrote, not everyone involved
The first column listed a thread's `participants`, which the threader builds from
from + to + cc — so every ordinary email read "Sender, You", and the +N badge
beside the avatar counted the user themselves.

`participants` cannot simply be narrowed: the subject-fallback join matches an
incoming message's recipients against it (threader._by_subject), so dropping
recipients would change how mail is threaded. Hence a separate `threads.senders`,
holding only from addresses, newest writer first — which is also the right thing
for a conversation where several people replied.

Migration 0019 backfills it, so existing threads do not wait to be touched again.
The list falls back to `participants` if a row somehow has no senders, rather
than rendering an empty column.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-18 18:27:42 +02:00
lukasandClaude Opus 5 e58a2f0ef4 Sync: a flag set on the server left the thread counters stale
Reported from the running app: a message showed as unread in the list — bold,
blue dot — while the reading pane showed it as read, and only a manual "mark as
read" reconciled them.

The two views read different things. The pane reads the message's own flags; the
list reads `threads.unread_count`, denormalized from them. `apply_delta` updated
message flags with a bare UPDATE and never recomputed the thread, so a flag set
anywhere but in this app — read on a phone, filed by a server-side rule — moved
one and not the other. Auto-mark-on-read could not fix it either: it sees a
message that is already \Seen and correctly does nothing, leaving the counter
wrong forever.

Expunges had the same hole, which also left `message_count` overstated.

`apply_delta` now collects the affected threads and re-denormalizes them after
the deletes, so a thread emptied by an expunge is removed rather than recounted.

Migration 0018 repairs rows that already drifted — recomputed from the messages,
so it is a no-op on a healthy database. One thread was stale here; installs that
have been syncing longer will have more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-18 18:21:30 +02:00
lukasandClaude Opus 5 8eb699a05f Antispam: bootstrap every account, not just the ones we happen to see go live
Warming Bayes hung on a single event — `account.state_changed` reaching "live".
That covers an account added from now on and nothing else. An account that went
live before this plugin existed never gets one; neither does one whose Junk
folder was empty at the time and filled up later, or one whose owner enabled
spam-folder sync afterwards. A worker restarting at the wrong moment loses it
too. The result was a feature that quietly required someone to run it by hand
per account, which is not a feature.

An hourly sweep now walks every live account and offers each a bootstrap. That
works because the per-account job was already cheap to repeat: it returns at once
for an account already done, and declines — without marking it done — when the
corpus is still too small to learn from. So the sweep simply keeps asking until
there is something worth learning, then does it once.

`s.accounts.live()` is the new SDK surface it needs: cross-user by design,
because a sweep runs for the instance rather than a request. Ids and state only,
so reading anything still goes through the user-scoped accessors.

One failing account cannot stop the sweep — the mailbox that raises is logged and
the rest still get their turn.

docs/plugins.md now says this in general terms next to `JobSpec`: a state-change
hook is one-shot, so pair it with a sweep when the work must eventually happen
for everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 21:04:38 +02:00
lukasandClaude Opus 5 8e3252679a Clear the reading panel when its last message leaves
"Mark as Spam" moved the message but left it on screen. Two causes, both fixed
generally rather than for this one action.

`move_job` published message.changed without a thread_id, so the WS handler
invalidated the message list but never the thread the user was reading — the
panel had no reason to refetch. It now names the thread.

And a thread whose last visible message has gone should not linger. MailLayout
closes the panel when the open thread comes back empty, whoever emptied it —
core action or plugin.

`move_to_special` now also publishes the move immediately rather than waiting
for the IMAP job, since a plugin acting on the user's behalf has no client-side
handle to update optimistically the way the core endpoint's caller does.

That exposed an ordering bug: the optimistic event reaches the passive-learning
hook, which would re-teach the lesson /learn had just taught. /learn now records
`learned` before moving, so the hook's dedup stands down — and records it with
an upsert, because provider-filed spam has no verdict row of ours to update.

One real mistake on the way: the event first reported the destination folder as
the source, which would have made every move look like a no-op to anything
reading the pair.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 19:47:42 +02:00
lukasandClaude Opus 5 c76b186b10 Antispam: "Mark as Spam" belongs in the Actions menu
A button pinned to every message header is a persistent control; marking spam is
a verb you reach for occasionally, like Reply or Delete. It now lives in the
Actions menu, between Labels and Delete.

That needed a slot inside the dropdown, which the catalog had no name for:
`message.actions.menu`. The host wraps it so any click closes the menu, leaving
a bundle to do nothing but act.

The UI kit gains `.wm-menu-item`, @applying the exact utilities ThreadView's own
menu items use — the same trick the sidebar shell already relies on, so a plugin
verb is indistinguishable from a core one rather than approximately similar.

Labels follow the folder as before: "Mark as Spam" in the inbox or archive,
"Mark as not spam" in the spam folder, nothing in Sent/Drafts/Trash. A message
already taught shows that, disabled, instead of offering the lesson again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 19:36:29 +02:00
lukasandClaude Opus 5 149c11581e Antispam C2: learn from what the user does
Explicit teaching shipped with C1 (the Spam / Not spam buttons). This adds the
two halves that need no buttons at all.

Passive learning (§C.5). Moving mail into Spam says "this is spam" more honestly
than any button, because the user is doing it for their own reasons rather than
to train a filter; moving it back out says the opposite. Only those two
transitions teach — Spam to Trash is someone tidying up, Inbox to Archive says
nothing about spam. Teaching the same message the same thing twice is skipped,
since users move things back and forth, but reversing a lesson is allowed. If
the engine is down nothing is recorded, so a later move can teach it again.

This needed the transition, not just the destination: `message.changed` carried
only where a message landed, so `move_job` now also publishes
previous_mailbox_id and both special_use values.

Provider bootstrap (§C.5). An account going live enqueues a job that teaches
Bayes from mail the user already owns — up to 500 messages from the provider's
Junk folder as spam, and up to 500 Inbox messages older than 30 days as ham.
This is the constructive use of the history that classification deliberately
refuses to act on: the same backfill, read instead of judged. It sleeps between
messages so live classification never queues behind training, stops rather than
grinding if the engine disappears, and marks the account done in KV so it runs
once.

That needed `PluginMessages.sample`: the SDK could fetch a message by id but not
enumerate a folder. Ids only — bodies are pulled one at a time, so a 500-message
sample never materialises 500 bodies.

Verified against the running engine: teaching moved its `learned` counter 0 -> 2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 08:36:51 +02:00
lukasandClaude Opus 5 6414504f6e Settings: a section per plugin, and call the folder Spam
Plugin settings pages were stacked at the bottom of Settings -> Accounts, which
put antispam's thresholds and lists under a heading about mail accounts. Each
`settings.page` now gets its own entry in the Settings nav: "Antispam filtering"
and "Mail filters".

Slot gains an optional `title` for this — the host needs a label when it gives a
slot an area of its own, and falls back to the plugin name. PluginSlot gains an
optional `plugin` filter so a host can render one plugin's bundle rather than
every binding for a slot name.

Wording: the app calls that folder Spam (i18n `nav.spam`), so the plugin's copy
now does too — "filed into Spam", "Move to Spam", "marked as spam". The IMAP
special-use stays `junk`, which is what the protocol calls it; only what the
user reads changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 08:19:11 +02:00
lukasandClaude Opus 5 dee1983b84 Antispam: never classify during backfill
message.new fires for imported history as well as new arrivals, and EventBus
awaits hooks inline. As written, adding an account would have scored every
message in its backfill — filing years-old inbox mail into Junk on the server,
and adding ~70 ms of serialized HTTP per message to the backfill loop (twelve
minutes on a 10k-message account). It is also when the filter is least able to
judge, because Bayes has learned nothing yet.

The envelope now carries account_state, and the hook returns unless it is
"live". The backfill window belongs to the C2 learning bootstrap (§C.5), which
reads that history as training data instead of acting on it.

Tests pin the gating: backfilling accounts are not even read from the blob
store, non-inbox folders are ignored, a live inbox arrival still classifies, and
an unreachable engine still delivers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 07:59:15 +02:00
lukasandClaude Opus 5 428ec58c71 Plugin SDK: raw message bytes, per-message move, mailbox in the envelope
Three additive surfaces the antispam plugin (M7, Appendix C) needs and the
frozen §11 API did not have. None of them changes a core mail path.

`actions.move_to_special` moves ONE message into an account's folder by
special-use, wrapping the same optimistic-update + move_job path the core
endpoint uses. The existing `trash` moves a whole thread, which is right for
Delete and wrong for a spam verdict: one spam reply must not drag a legitimate
conversation into Junk. It returns False rather than raising when the account
has no such folder — an account whose server lacks a Junk folder must still
receive its mail.

`messages.raw` returns the stored RFC822 bytes. Spam scoring needs headers and
body; the SDK could only hand out one header at a time.

`envelope` now also carries account_id and mailbox_special_use. Appendix C must
classify only inbox arrivals, and neither the message.new payload nor the
envelope previously said where a message landed.

docs/open-questions.md gains OQ-6 (why these are additive rather than a spec
violation of "zero core diffs") and OQ-7 (why the engine is reached with plain
httpx instead of the SSRF-guarded sdk.http).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-16 22:14:24 +02:00
lukasandClaude Opus 5 8fce86559e Admin panel on its own port, and a registration gate
Three strands that only make sense together, so they land together.

Registration gate (migration 0016). POST /auth/register was open to anyone
who could reach the port. Now the first account on an empty instance is
always allowed, becomes the administrator, and closes registration behind
itself; after that it takes an admin reopening it. The decision is made
under a row lock on app_settings so two simultaneous first signups cannot
both see an empty table and both claim administrator. Existing installs
keep their oldest account as admin and start closed.

Instance default quota (migration 0017). users.quota_bytes is snapshotted
at creation from app_settings.default_quota_bytes, so changing the default
affects later accounts only and never resizes a mailbox that exists.

The panel is a separate app, not a page in the mail client. admin.html is
its own Vite entry (5.8 KB next to the client's 1.7 MB; they share only the
vendor chunk), with its own administrator sign-in. One nginx now serves two
listeners: :80 is the mail client and 404s both /admin.html and
/api/admin/*, :81 is the panel. docker-compose publishes them as WEB_PORT
and ADMIN_PORT. The boundary is nginx refusing paths on one listener, not
process isolation — CurrentAdmin still guards every admin route — so
ADMIN_PORT must be restricted to the LAN, which the runbook now covers for
both DSM and QNAP.

Also QNAP: deploy/webmail.sh finds Container Station's docker under
/share/*/.qpkg/container-station/bin, which is off an SSH session's PATH,
before anything shells out to it. docs/nas-install.md gains a QuTS hero
runbook (§2.6), splits the DSM firewall section into container-to-container
(§8.1) and the egress case that makes account-add time out while
autodiscovery appears to work (§8.2), and scopes both to Synology.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-16 20:34:08 +02:00
lukasandClaude Opus 5 5553b5febe Deploy: production Docker stack for a single host (Synology/QNAP NAS)
Everything needed to run the app on an Intel NAS that is also doing other
jobs: one published port, no host networking, memory/CPU caps and log
rotation on every container, and all state in bind mounts under DATA_ROOT
so mail survives image updates, project deletion in the NAS UI, and full
reinstalls — and so Hyper Backup/HBS3 can actually see it.

- docker/Dockerfile.backend: multi-stage, non-root (uid 10001), tini for
  the <10s graceful lease release. Installs the four plugins as entry-point
  distributions. One image, several roles via docker/entrypoint.sh
  (api | worker-sync | worker-jobs | migrate | reindex | plugins).
- docker/Dockerfile.frontend + nginx.conf: SPA + /api reverse proxy with
  the WebSocket upgrade for /api/ws, and X-Forwarded-Proto preserved so
  the API still emits __Host-/Secure cookies when TLS is terminated by
  the NAS reverse proxy (identity/cookies.py keys off request scheme).
- docker/docker-compose.prod.yml: project name webmail-prod, NOT webmail —
  docker-compose.yml (dev services) owns that name, and two compose files
  sharing one project name fight over each other's containers.
  Meilisearch is an opt-in profile; SEARCH_BACKEND=pg is the default so a
  small NAS pays nothing for search.
- migrate runs as a one-shot (core Alembic head + every plugin branch)
  that everything else waits on, so a bad migration can never leave new
  code running against an old schema.
- docker/backup.sh: nightly pg_dump verified with `pg_restore --list`
  before it counts as a backup, with retention and a baseline on first
  start.
- deploy/webmail.sh: init/doctor/up/down/status/logs/update/rollback/
  backup/restore/reindex/psql/shell. `update` is backup -> images ->
  migrate -> roll workers, api, web -> health-gate, with automatic
  rollback to the previous version if the gate fails. `doctor` refuses to
  be quiet when DATA_ROOT is on NFS/SMB (silent Postgres corruption).
- deploy/build-push.sh: build images off-NAS (the frontend build wants
  ~2GB RAM) and push to Gitea, or --save a tar to carry over.
- scripts/reindex_search.py: rebuild the search index from Postgres alone.
  Every SearchDoc field is already stored, so the index is derived data
  that never needs backing up — this closes the only real objection to
  running Meilisearch on a box whose volumes may be wiped.
- pyproject: package-data for accounts/presets.yaml and the shared plugin
  Alembic env. Without it a non-editable install (i.e. any image) silently
  ships a package that fails at runtime.
- GET /api/version, used by the deploy health gate.

Verified end to end on a real stack: fresh install, all services healthy,
core + 4 plugin migration branches applied, login/SPA/plugin bundles
through nginx, WS upgrade returning 101, __Host- cookies under a
simulated TLS proxy, backup -> restore round-trip preserving user and
plugin schema, failed-update auto-rollback (both a bad migration and a
boot-crashing API), and Meilisearch profile serving a live query.
ruff + mypy --strict clean, 363/363 backend tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nux3LCu4zP9nih4BRhPyHw
2026-09-14 18:53:50 +02:00
lukasandClaude Opus 4.8 12a67825cc Sender avatars, PDF password preview, contacts autosave, account-view + Trash/Spam auto-purge, UI polish
Sender avatars (privacy-safe, server-fetched, cached):
- New GET /api/avatar resolves Gravatar (d=404) then the sender domain's favicon
  under the SSRF guard, Redis-cached (hit+miss); gated by SENDER_AVATARS_ENABLED.
  GuardedResponse gains raw bytes. Shown in the mail list, reading pane, People
  panel, and Contacts (overlay that falls back to initials on a miss).

PDF preview:
- Password-protected PDFs now prompt in-place and retry (pdf.js PasswordException),
  with a clear error for other load failures instead of a blank overlay. The
  password is remembered per attachment until the thread is left; clicking the
  field no longer dismisses the overlay.

Contacts:
- Autosave reply recipients: message.sent now carries to/cc (name+email); the
  contacts plugin subscribes and adds any non-contact recipients to the Autosaved
  group. Contact avatars show the gravatar/favicon via /api/avatar.

Mail views / retention:
- Clicking an account shows all its mail except Trash/Spam (was inbox-only).
- Daily cron autopurge_trash_spam permanently deletes Trash/Spam older than
  TRASH_RETENTION_DAYS (default 30; 0 disables) — server EXPUNGE + local rows.

UI polish:
- Ultimo signet logo in the Mail/Calendar/Contacts sidebar header and collapsed
  rail; consistent 11px top margin and vertically-aligned header row across states
  (fixed-size flex icon buttons); SVG favicon on the browser tab.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-06 19:23:27 +02:00
lukasandClaude Opus 4.8 4c200ab94a Gmail label sync + no-dup backfill, live label/UI updates, account-delete cleanup, Sync-SPAM toggle, sidebar logo
Gmail sync (SPEC: Gmail sync):
- Provider fetches X-GM-LABELS when X-GM-EXT-1 is advertised; parses custom labels
  (system labels dropped, mUTF-7 decoded) onto RemoteMessage.
- Engine syncs only system folders + All Mail for Gmail, skips every label folder,
  and dedupes the All Mail pass by Message-ID — kills the INBOX/All-Mail duplication.
- Custom labels ride the message.new event; the labels plugin subscribes and maps
  them to chips as "<name> [Gmail]" (per-message + thread aggregate + known set),
  so labels fill in gradually with the sync window. Non-ASCII labels decode right.

Labels self-healing (message-driven reconcile):
- Per-message keys are the source of truth; t: aggregates and _labels are derived.
  Reconcile drops labels of deleted messages, regroups survivors by their current
  thread (fixing JWZ thread merges), rewrites changed t: keys, and rebuilds _labels
  from live messages — keeping user labels, dropping orphaned Gmail ones.
- New SDK reads: sdk.threads.existing / sdk.messages.threads_of. Sidebar refreshes
  live via a plugin.labels.changed WS event on new labels.

Account deletion: prune threads left empty after the cascade (they were user-scoped,
so only messages cascaded) + de-list the account from surviving cross-account threads.

Sync SPAM: opt-in per-account sync_spam (migration 0015, default off) — the engine
skips the Junk folder in backfill and poll unless enabled; wizard checkbox added.

Robustness / UX:
- WebSocket auto-reconnects with backoff and resyncs on (re)connect, and account
  add/delete invalidates threads/labels — no more hard refresh for live updates.
- Backfill banner + sidebar ring also require done < total, so a completed account
  can't leave a stuck "X of X".
- Scrub lone UTF-16 surrogates from parsed headers/bodies so bad messages no longer
  crash the backfill on Postgres text/JSONB.
- Ultimo signet logo in the Mail/Calendar/Contacts sidebar header (and at the top of
  the collapsed rail), via the shared plugin-ui.css + MailLayout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-05 18:04:49 +02:00
lukasandClaude Opus 4.8 e5a87b37b1 Gmail (app-password) onboarding: auth_strategy routing + credential-helper panel
Implements SPEC v1.5 §6.2.1–6.2.3 so Gmail (and Yahoo/iCloud/Outlook-personal)
onboard via a provider app password on the existing plain-password sync path —
no OAuth, no new sync code. Gmail was oauth2-only with OAuth unimplemented, so it
was previously un-addable.

Backend:
- presets.yaml: each preset declares an ordered `auth_strategy`; Gmail becomes
  `auth: password` + `[app_password, oauth2]`. App-password providers ship a
  data-driven `help:` block (deep_link, app_name_hint, i18n step keys, docs_url).
- discovery/types: CandidateConfig carries `auth_strategy` + `HelpBlock`; presets
  load them and autoconfig candidates derive a default from the detected auth.
- schemas/api: CandidateOut exposes auth_strategy + help; DiscoverResponse adds
  `google_oauth_enabled`.
- config: GOOGLE_OAUTH_ENABLED flag (default false) gates the OAuth fallback.

Frontend:
- ProviderCredentialPanel (wizard step 1.5): deep-link popup (user gesture),
  rendered steps, single paste field that strips whitespace, instant verify
  (auto POST /accounts/test at 16 chars → Connected ✓ / mapped error), and the
  auth_failed recovery branch with a gated "Sign in with Google instead" button.
- AccountWizard routes app-password providers to the panel, then a details step.
- i18n keys (en) for the panel + provider steps.

Honors the §6.2.2 prohibitions: no page automation, no real-password capture,
no scraping — deep-link + guide + paste + test only. Adds the v1.5 spec.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-04 19:59:52 +02:00
lukasandClaude Opus 4.8 0a478a6b26 Labels in Mail UI (chips + Actions + sidebar), plain-text Lato, calendar-panel polish
Labels:
- Moved labelling out of the "+label" message slot into Mail: label chips on the
  thread subject bar (removable ✕) and before list subjects, and a Labels submenu
  in the per-message Actions menu (multi-select checkboxes + "Create new…").
  Per-message storage kept; the reading pane writes a thread-union aggregate so the
  list (no message ids) can show chips.
- Sidebar LABELS section rewritten: aligned monocolor tag rows, collapsible
  (collapsed by default), inline "+" create popover, hover "−" delete with a confirm
  popover, and click-to-filter the mail list by label.
- Backend: batch message/thread label reads, thread aggregate, create + delete
  (delete unhooks from all mail), and GET /labels/{name}/threads. Core GET /threads
  gained an `ids` param to power the label view. Removed the plugin-pages list from
  the Mail sidebar.

Plain-text mail now renders in Lato (self-hosted at /fonts, @font-face'd into the
sandboxed message iframe) instead of the monospace <pre> default.

Calendar right-panel (embed mode): restored the "…" view-options menu, pinned the
toolbar + mini-calendar (only the agenda scrolls), plain "All calendars" button
(inline resets), and the event popover now closes on any outside click.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-03 20:53:35 +02:00
lukasandClaude Opus 4.8 421b44a8ab Mail mark-all-read + iframe fixes; Contacts detail/selection/groups + resizable panels
Mail:
- New "More" (⋯) menu in the list sorting bar with "Mark all as read": POST
  /threads/mark-read marks every unread thread in the current view (unified /
  folder:<special> / account:<uuid>) as \Seen, pushes + publishes updates.
- Message iframe: no longer shows a spurious vertical scrollbar (display:block,
  getBoundingClientRect+ceil sizing, overflow-y:hidden on the doc) and has
  rounded corners.
- Deep links from Contacts: /mail?compose=1&to= opens a maximized prefilled
  composer; /mail?q= prefills the search box.

Contacts:
- Detail pane shows all populated fields (work subtitle, phones, address, urls,
  nickname, birthday). Company name is used as the display name when there's no
  personal name (existing rows too). Editable "Add notes".
- Selection: checkbox reveals on avatar hover only; Groups/More bulk menus in
  select mode; "N contacts selected" panel; filled yellow VIP star in list +
  detail; pagination bottom bar; Lato pinned on form controls.
- Groups: "+" on the GROUPS header and "Create new" in the group menus (bulk
  add/remove via POST /groups/{id}/add|remove, ownership-guarded).
- Rounded Mail-style list rows; removed the ⋯ button from the search bar;
  Compose→Mail, Find-all-conversations→Mail search; Message button removed;
  delete confirmation; resizable sidebar + list/detail split.

Calendar: the right panel (agenda / search) is now resizable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-02 21:34:22 +02:00
lukasandClaude Opus 4.8 57a1f1562f M4 Calendar: reminders form, invitations, short publish links, settings UX
Calendar settings (Settings > Calendars):
- Reminders are now {method, minutes}: Add-reminder opens a boxed form with a
  method dropdown (Show an alert / Email me) + time dropdown, bell icons.
  Backend normalizes the legacy list[int] transparently.
- New Invitations section (auto-add emailed events): policy radios, "Add new
  events from" (My contacts / Anyone), "Add to calendar", mark-read+file toggle.
  Persisted to user settings (inbound iMIP processing still deferred).
- Time zone options show GMT codes, e.g. "Central European Summer Time (GMT+2)".
- Default calendar now defaults to "Personal" instead of a generic placeholder.
- Colour picker is a circular-swatch-only dropdown (sized so swatches don't overlap).
- Delete calendar / remove subscription now confirm via a modal.
- Create form: dropped the breadcrumb; heading stays "Create calendar" even after
  a Publish toggle auto-creates the calendar.

Publish links: replaced the 244-char vault-sealed token with a 16-char random
slug (secrets.token_urlsafe). New sdk.public_session() (privileged, non-RLS) lets
the public /public/{token} feed resolve the slug directly with explicit filters.

Settings shell: "<- Mail" is now a context-aware "<- Back" via ?from=; Calendar
opens Settings with from=/p/calendar so Back returns to the calendar.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-02 08:48:33 +02:00
lukasandClaude Opus 4.8 e2bef3c41d M4 Calendar: publish public .ics links, subscriptions, multi-calendar settings
Calendar (builtin plugin):
- Publish read-only public .ics feed per calendar via vault-sealed token
  (full-detail or free/busy-only); unauthenticated GET /public/{token},
  404 on bad/absent token. Migration calendar0004 adds publish_token,
  publish_busy, subscription_url + partial unique index.
- Subscribe to external iCal (.ics) URLs (SSRF-guarded) as read-only
  calendars; CSRF-guarded POST /subscriptions.
- Settings > Calendars: create/edit/delete, publish toggles with live
  shareable link + copy, Add-account stubs (Google/iCloud/CalDAV, deferred),
  Subscriptions section; default calendar + default reminders; auto-detected
  time zone; color-dropdown contrast fix; 3x-larger edit arrow.
- Left panel: shared Mail-parity sidebar template (.wm-side-*), calendar
  search panel, font/spacing tuning, per-section "+" create shortcut.

Frontend: CalendarOut publish fields, api publish/subscribe, settings keys.
Backend: settings default_calendar + reminder lists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-08-01 20:22:53 +02:00
lukasandClaude Opus 4.8 8c634f7967 M4: Calendar plugin (builtin) + mail HTML/attachment fixes
Calendar plugin (webmail_plugin_calendar, non-removable core plugin):
- Local calendars/events CRUD with RLS isolation; recurrence expanded at
  query time (DST-correct), EXDATE + detached-override merge.
- Calendar app bundle: month/week/day/3-day views, mini-month, event form
  (color, show-as, timezone combobox, repeat, reminders, guests w/ recipient
  autocomplete, Jitsi video call), event popover with RSVP badges, resizable/
  collapsible left panel + theme toggle, overlap layout.
- iMIP invites: sdk.mail sends REQUEST from the organizer's mailbox (HTML +
  text/calendar + invite.ics); public /rsvp endpoint (vault-sealed token)
  records accepted/declined/tentative per attendee.
- "This and following" edit + scoped delete (this / this-and-following), with
  a delete confirmation dialog for recurring events.

Core:
- Settings: time_format (12/24h, default 24h) + jitsi_server; new Integrations
  section; date order follows time format (en-GB vs en-US) across Mail.
- Plugin SDK: mail (send from a user's account), vault (envelope encrypt),
  multi-method SSRF-guarded http; AppSwitcher product switcher; full-bleed
  PluginPage.

Mail HTML rendering:
- Sanitizer allows <style> blocks + class (CSS sanitized: drop script-like,
  strip @import/url() when remote images blocked); SANITIZER_VERSION 3 so
  already-indexed mail re-sanitizes on view.
- Attachments: a Content-Disposition: attachment part is a real attachment
  even when it also carries a Content-ID (fixes hidden invite.ics).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-28 16:33:34 +02:00
lukasandClaude Opus 4.8 d46ad13b7b M3 plugin platform: frontend host, reference plugins, UI kit + docs
Completes M3 (SPEC §11 + §12 acceptance) on top of the backend platform (4e077f1).

Frontend host (§11.3):
- src/plugins/: manifest loader, runtime ESM bundle loader (framework-agnostic
  mount contract), PluginSlot, PluginPage (app.page at /p/*), plugin context
  (scoped api/events/i18n), capabilities hook. Wired slots into ThreadView
  (message.actions/message.banner), MailLayout sidebar (sidebar.section + page nav)
  and Settings→Accounts (settings.page); ws.ts fans out core + plugin.* events.
- plugin-ui.css: theme-aware .wm-* classes so bundles don't re-derive inline styles.

SDK additions:
- CurrentUserId + require_csrf (plugin-router auth), messages.envelope/header,
  actions.set_flags/trash, sdk.enqueue.

Reference plugins (plugins/, installed via webmail.plugins entry points):
- hello: skeleton page-plugin (app.page + plg_hello_note w/ RLS + cron job) —
  install→run→uninstall --drop-data leaves zero residue.
- labels: message.actions + sidebar.section slots, KV only, zero core diffs.
- filters: message.new hook -> match (subject/from/to) -> Mark read / Pin / Delete
  (delete = move thread to Trash via the core path).

docs/plugins.md: full authoring guide + a complete "Unsubscribe button" worked
example (the §12 "writable from docs alone" bar).

Also: conftest resets the arq pool between tests (loop-bound global, like engine/redis).

Gates: backend ruff + mypy --strict (107) + full suite 337; frontend tsc + eslint
+ vitest (11) + vite build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcj3TpydMTzTSmU7kunzf8
2026-07-27 08:30:24 +02:00
lukasandClaude Opus 4.8 4e077f1a37 M3 plugin platform: SDK + loader, RLS-isolated tables, migrations, jobs
Backend foundation for the plugin platform (SPEC §11), Phases 1-3.

Phase 1 — SDK + loader:
- app/plugins/types.py: frozen Plugin/UiManifest/Slot/Page/JobSpec contract.
- app/plugins/sdk.py: sdk(name) facade — the only app.* import allowed for plugins
  (scoped session, plugin.{name}.* publish, blob, outbox, settings, SSRF http_get, KV, enqueue).
- app/plugins/loader.py: entry-point discovery (webmail.plugins), hook forwarding
  (API + worker), router mounting at /api/plugins/{name}, ui-manifest aggregation.
- plugin_kv core table (migration 0013) + GET /api/plugins/ui-manifest.

Phase 2 — non-removable core plugins + DB-enforced privacy:
- builtin flag (calendar/contacts will be builtin=True; uninstall refuses them).
- RLS per-user isolation on plg_* tables: least-privilege webmail_plugin role
  (migration 0014), plugin_session(user_id) sets LOCAL ROLE + app.current_user_id,
  rls.enable_rls() (ENABLE+FORCE RLS + USING/WITH CHECK policy). Mail untouched.
- Per-plugin Alembic runner (shared env, dedicated version table) + install/uninstall
  --drop-data CLI; skeleton fixture proves install -> run -> uninstall --drop-data
  leaves zero residue (§12 M3 acceptance).

Phase 3 — plugin jobs (§11.2):
- JobSpec registered as arq function plg.{plugin}.{func} + cron (5-field parser);
  singleton=True guarded by a per-(plugin,user) Redis lock; wired onto the shared
  interactive worker.

Gates: ruff, mypy --strict (94 files + tests), full suite 326 passed. RLS isolation,
migration round-trip, and singleton lock covered by tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcj3TpydMTzTSmU7kunzf8
2026-07-26 21:42:35 +02:00
lukasandClaude Opus 4.8 4d5a30cffb sync: QRESYNC VANISHED expunge detection (SPEC §6.3 M2 fast resync)
Server-side deletions now propagate to the DB. Previously poll() detected new
messages and CONDSTORE flag deltas but never expunges (the old TODO), so mail
deleted in another client lingered as phantoms outside the small Drafts reconcile.

- provider: enable QRESYNC when advertised (_maybe_enable_qresync + qresync_enabled
  property). poll() harvests VANISHED (expunged UIDs) and changed FLAGS in one pass
  via a raw `SELECT <mbox> (QRESYNC (uidvalidity modseq))` — issued as a low-level
  SELECT because aioimaplib only surfaces untagged VANISHED lines on a sync command,
  not on an async UID FETCH. CONDSTORE-only servers (e.g. Gmail) keep the CHANGEDSINCE
  flag delta and fall back to the engine reconcile.
- parsing: parse_vanished() for `VANISHED [(EARLIER)] <uid-set>` with range expansion.
- engine: apply_delta now drops expunged rows from the search index, un-links any
  local draft mirroring a vanished UID, and publishes message.deleted for the UI.
- tests: parse_vanished unit cases + apply_delta expunge (removal, draft un-link,
  modseq advance).

Verified live: the worker issues `SELECT "INBOX" (QRESYNC (uidvalidity modseq))`
against QRESYNC-capable accounts. ruff + mypy --strict clean; 296 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcj3TpydMTzTSmU7kunzf8
2026-07-26 18:28:09 +02:00
lukasandClaude Opus 4.8 dd0b089049 attachments preview overlay, docx/xlsx/odf rendering, composer drag-drop & undo-send UX
Frontend:
- Full-screen attachment preview overlay (AttachmentPreview) for image/pdf/txt
  plus office docs: .docx via docx-preview (rendered live into the sandboxed
  iframe so header/footer VML images survive), .xlsx via xlsx-preview, and
  .odt/.ods/.odp/.odg via a JSZip+DOMParser ODF walker (officeConvert).
- Type-aware attachment icons (green spreadsheet, blue document, red PDF,
  image/archive); new document/image/spreadsheet/print SVGs.
- Composer: drag-drop attachments (body drop inserts images inline, else
  attaches); inserted images are selectable/resizable/deletable (RichTextEditor).
- Undo-send: "Sending…" banner with live "N seconds to Undo" countdown.
- Search result list: pin/archive/delete update correctly; grey-out only for
  search hits; scoped "Search in Trash/Spam".

Backend:
- Search: prefix (search-as-you-type) matching, email-address search, and
  address-first result ordering; Trash/Junk excluded with scoped folder search.
- Attachments: RFC 6266 Content-Disposition for non-ASCII filenames; one-time
  inline image loading.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcj3TpydMTzTSmU7kunzf8
2026-07-26 16:15:49 +02:00
lukasandClaude Opus 4.8 c023cfa15e search: email address search + address-first ranking; inline images; folder i18n; link security
Search (PG backend):
- '@' terms match address substrings (from/to/cc), so a partial email like
  "dyrektor@wfdif" resolves and narrows predictably as you type
- index each address split into parts, so a bare domain/local word ("wfdif")
  finds the address (indexed as one lexeme otherwise); dotted terms fall back to
  an address substring
- rank sender/recipient name+address matches above body/subject-only ones
  (addr_hit tier, then recency); cursor extended to keep keyset pagination correct
- scripts/reindex_address_parts.py backfills address tokens into existing rows

Inline images:
- extract attachments via a full MIME walk instead of shallow iter_attachments(),
  which missed inline images nested in mixed>related>alternative (cid: rendered
  broken); has_attachments counts only real (non-cid) attachments
- scripts/backfill_inline_attachments.py stores previously-missed inline images

Folder-name i18n (SPEC §12 M2):
- accent/punctuation-insensitive, multi-delimiter (/ \ .) special-use inference
  across ~13 languages; curated names (no substring false positives)

Link security (reading panel):
- links never navigate the message iframe: a parent-side click interceptor opens
  http/https in a new tab (noopener) and mailto in the composer (recipient
  pre-filled); all other schemes blocked

Backend 292 tests green, ruff + mypy --strict clean; frontend tsc/eslint/vitest clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-26 09:28:48 +02:00
lukasandClaude Opus 4.8 49738740c0 M2: Meili search + saved searches, sync windows + backfill progress, threading & image fixes
Search:
- Meilisearch backend (app/search/meili.py) + factory wiring; pinned SDK 2.7
- Search-as-you-type: debounced query + Postgres prefix matching, built from
  the DB tokenizer so full email addresses match (email stays one lexeme)
- Saved searches API + sidebar virtual mailboxes; save affordance in the box

Account sync windows (SPEC §12 M2):
- Choose at setup: All (recent 30d first, then remainder), Last 3 months, New only
- Per-account sync_window + "Sync remaining mail" from settings (idempotent re-run)
- Provider windowed backfill (SINCE/BEFORE), message_count, newest_uid
- migration 0012: accounts.sync_window + backfill_total/backfill_done

Backfill progress UX:
- Floating progress card (auto-dissolves after 10s) + spinning sidebar ring that
  fills with the account colour; driven by sync.progress over WS

Thread list pagination:
- Server-side offset + accurate grand total, with filter/order in SQL; working
  prev/next across all synced mail

Reading panel:
- "Load images" one-off now actually renders images (?images=1) without trusting
  the sender; envelope carries images_allowed so a trusted sender shows no prompt

Threading:
- Subject fallback no longer merges messages that declare an explicit (different)
  parent; true siblings still group via a shared-parent reference match. Fixes
  unrelated auto-replies with the same subject landing in one thread

Backend 257 tests green, ruff + mypy --strict clean; frontend tsc/eslint/vitest clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-23 23:00:08 +02:00
lukasandClaude Opus 4.8 b4e07f11aa collapse cross-account dupes, Lato UI font, mail-list selection & density tuning
Backend:
- Collapse identical cross-account deliveries in a thread: new messages.delivery_id
  (migration 0011) preserves the real Message-ID even when message_id_hdr is
  synthesized (§7.1); get_thread groups by it and reports also_in_account_ids; the
  list count counts distinct deliveries. Backfill script for existing rows.

Frontend:
- Bundle Lato (SIL OFL) as the UI font; unread mail-list rows use Black weight.
- Mail-list selection: hover-avatar checkbox, tri-state "select all", bulk
  Archive/Pin/Delete (Restore/Delete in Archive/Trash/Spam) with a count in the
  reading pane; unread blue dot; "also delivered to" note in the thread.
- Spacing/size tuning: sender-column width, icon gaps, avatar size, bar heights,
  and a round of -1px font sizes across the sidebar, mail list, and reading pane.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-22 21:49:48 +02:00
lukasandClaude Opus 4.8 fdb6ee002e mail list redesign, folder actions, empty-folder, unread badges & empty-state art
Frontend:
- Redesign the mail list rows (avatar + sender + subject/snippet + right rail),
  even-spaced attachment/pin/archive/trash icons, hover-reveal actions, pink
  always-on pin, monocolor paperclip, row gaps, wider sender-subject gap.
- Sidebar folder badges: Inbox unread + Drafts counts (live via WS + mutations).
- Trash/Spam/Archive: hover Restore icon (replaces Archive), permanent delete in
  Trash; "Empty trash" / "Empty spam" buttons on the sort bar.
- Reading pane empty state: email illustration above an enlarged
  "Select a conversation".
- Search field clear (X) button; mark-read after a 4s dwell instead of instantly.

Backend:
- GET /api/mailboxes/counts (inbox unread + drafts) for the sidebar badges.
- POST /api/folders/{special}/empty + empty_folder_job to permanently clear a
  Trash/Spam/Archive folder (server EXPUNGE + DB rows + thread cleanup).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
2026-07-22 07:40:29 +02:00