mail UX: restore-to-folder, draft/attachment fixes, printing, sidebar collapse

Backend:
- Restore returns each message to its original folder (e.g. Sent/Archive) instead
  of always the Inbox: new messages.restore_mailbox_id (migration 0010) +
  restore_thread_job.
- Drafts: expunge reconciliation for the Drafts poll (provider.list_uids /
  _reconcile_drafts) clears phantom "remote" drafts; send + discard drop the
  mirrored DB row; adopting a remote draft carries its real attachments into the
  compose flow.
- Thread detail/list are folder-aware: individually trashed/spam messages are
  hidden outside their folder, Trash/Spam show only their own messages, and the
  mail-list message count matches the view.

Frontend:
- Thread reading panel: fix newest-first card overlap/spacing (expand-all and
  single expand), hide Reply/Forward on trashed messages, close the Actions menu
  on message-body (iframe) clicks, scroll to the inline composer, and prompt
  Save draft / Discard on the X close.
- Composer: remount when switching drafts, dirty-tracking to stop duplicate IMAP
  mirrors, and close the From/More dropdowns on outside click.
- Sidebar: collapse/expand toggle (icons-only ~60px rail) and a log-out icon on
  Sign out.
- Mail-standard printing in a new window with reliable page margins (lib/print.ts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
This commit is contained in:
2026-07-20 23:12:25 +02:00
co-authored by Claude Opus 4.8
parent 1a83e0fb9b
commit d650f16feb
33 changed files with 1927 additions and 105 deletions
+7 -2
View File
@@ -111,8 +111,13 @@ async def trash_thread(thread_id: uuid.UUID, user: CurrentUser, db: DbSession) -
@router.post("/threads/{thread_id}/restore") @router.post("/threads/{thread_id}/restore")
async def restore_thread(thread_id: uuid.UUID, user: CurrentUser, db: DbSession) -> dict[str, str]: async def restore_thread(thread_id: uuid.UUID, user: CurrentUser, db: DbSession) -> dict[str, str]:
"""Move a thread back to the Inbox (from Archive or Trash).""" """Restore a thread (from Archive or Trash) to each message's original folder."""
await _move_thread(user.id, thread_id, db, "inbox") thread = await db.scalar(
select(Thread).where(Thread.id == thread_id, Thread.user_id == user.id)
)
if thread is None:
raise_api_error(404, "not_found", "thread not found")
await enqueue_job("restore_thread_job", str(thread_id))
return {"status": "queued"} return {"status": "queued"}
+37 -3
View File
@@ -16,7 +16,7 @@ from app.config import get_settings
from app.db import get_session from app.db import get_session
from app.identity.deps import CurrentUser, require_csrf from app.identity.deps import CurrentUser, require_csrf
from app.indexer import blobstore from app.indexer import blobstore
from app.models import BodyPart, ComposeAttachment, Draft, Mailbox, Message from app.models import Attachment, BodyPart, ComposeAttachment, Draft, Mailbox, Message
from app.outbox.schemas import ( from app.outbox.schemas import (
DraftIn, DraftIn,
DraftOut, DraftOut,
@@ -24,7 +24,13 @@ from app.outbox.schemas import (
SendRequest, SendRequest,
SendResponse, SendResponse,
) )
from app.outbox.service import DraftError, cancel_outbox, create_outbox, upsert_draft from app.outbox.service import (
DraftError,
cancel_outbox,
create_outbox,
delete_mirrored_draft_row,
upsert_draft,
)
from app.queue import enqueue_job from app.queue import enqueue_job
router = APIRouter(tags=["compose"], dependencies=[Depends(require_csrf)]) router = APIRouter(tags=["compose"], dependencies=[Depends(require_csrf)])
@@ -183,6 +189,30 @@ async def adopt_remote_draft(message_id: uuid.UUID, user: CurrentUser, db: DbSes
parts = list(await db.scalars(select(BodyPart).where(BodyPart.message_id == msg.id))) parts = list(await db.scalars(select(BodyPart).where(BodyPart.message_id == msg.id)))
html = next((p.content for p in parts if p.kind == "html_sanitized"), "") html = next((p.content for p in parts if p.kind == "html_sanitized"), "")
text = next((p.content for p in parts if p.kind == "text"), "") text = next((p.content for p in parts if p.kind == "text"), "")
# Carry the draft's real (non-inline) attachments over as compose attachments,
# so they show in the editor and are re-attached on save/send. The underlying
# blobs are shared by sha256 — we only add new ComposeAttachment references.
att_rows = list(
await db.scalars(
select(Attachment)
.where(Attachment.message_id == msg.id, Attachment.content_id.is_(None))
.order_by(Attachment.filename)
)
)
attachment_ids: list[str] = []
for a in att_rows:
ca = ComposeAttachment(
user_id=user.id,
filename=a.filename,
mime_type=a.mime_type,
size_bytes=a.size_bytes,
blob_sha256=a.blob_sha256,
)
db.add(ca)
await db.flush()
attachment_ids.append(str(ca.id))
payload = { payload = {
"to": [_fmt_addr(a) for a in msg.to_addr], "to": [_fmt_addr(a) for a in msg.to_addr],
"cc": [_fmt_addr(a) for a in msg.cc_addr], "cc": [_fmt_addr(a) for a in msg.cc_addr],
@@ -190,7 +220,7 @@ async def adopt_remote_draft(message_id: uuid.UUID, user: CurrentUser, db: DbSes
"subject": msg.subject, "subject": msg.subject,
"html": html, "html": html,
"text": text, "text": text,
"attachment_ids": [], "attachment_ids": attachment_ids,
} }
draft = Draft( draft = Draft(
user_id=user.id, user_id=user.id,
@@ -244,6 +274,10 @@ async def delete_draft(draft_id: uuid.UUID, user: CurrentUser, db: DbSession) ->
raise_api_error(404, "not_found", "draft not found") raise_api_error(404, "not_found", "draft not found")
account_id, imap_uid = draft.account_id, draft.imap_uid account_id, imap_uid = draft.account_id, draft.imap_uid
await db.delete(draft) await db.delete(draft)
# Drop the mirrored Drafts-folder message row now so the draft doesn't
# resurface as an "other client" remote draft after the local row is gone.
if imap_uid:
await delete_mirrored_draft_row(db, account_id=account_id, imap_uid=imap_uid)
await db.commit() await db.commit()
# Remove the mirrored copy from the server Drafts folder too. # Remove the mirrored copy from the server Drafts folder too.
if imap_uid: if imap_uid:
+50 -6
View File
@@ -6,7 +6,7 @@ import uuid
from typing import Annotated from typing import Annotated
from fastapi import APIRouter, Depends, Query from fastapi import APIRouter, Depends, Query
from sqlalchemy import literal, select, tuple_ from sqlalchemy import func, literal, or_, select, tuple_
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.api.cursors import decode_cursor, encode_cursor from app.api.cursors import decode_cursor, encode_cursor
@@ -31,14 +31,16 @@ def _addresses(raw: list[dict[str, str]]) -> list[Address]:
return [Address(name=a.get("name", ""), email=a.get("email", "")) for a in raw] return [Address(name=a.get("name", ""), email=a.get("email", "")) for a in raw]
def _summary(thread: Thread) -> ThreadSummary: def _summary(thread: Thread, message_count: int | None = None) -> ThreadSummary:
return ThreadSummary( return ThreadSummary(
id=thread.id, id=thread.id,
subject=thread.subject_norm or "(no subject)", subject=thread.subject_norm or "(no subject)",
snippet=thread.snippet, snippet=thread.snippet,
participants=_addresses(thread.participants), participants=_addresses(thread.participants),
account_ids=list(thread.account_ids), account_ids=list(thread.account_ids),
message_count=thread.message_count, # Count only the messages this view actually shows (deleted/spam excluded
# elsewhere); falls back to the denormalized total.
message_count=message_count if message_count is not None else thread.message_count,
unread_count=thread.unread_count, unread_count=thread.unread_count,
has_attachments=thread.has_attachments, has_attachments=thread.has_attachments,
pinned=thread.pinned, pinned=thread.pinned,
@@ -123,7 +125,32 @@ async def list_threads(
next_cursor = ( next_cursor = (
encode_cursor(threads[-1].last_message_at, threads[-1].id) if has_more and threads else None encode_cursor(threads[-1].last_message_at, threads[-1].id) if has_more and threads else None
) )
return ThreadListOut(threads=[_summary(t) for t in threads], cursor=next_cursor)
# Per-thread visible message count matching what get_thread shows for this view:
# Trash/Spam count only their own messages; every other view excludes them. This
# keeps the list count in step with a single deleted message.
counts: dict[uuid.UUID, int] = {}
if threads:
count_stmt = (
select(Message.thread_id, func.count())
.join(Mailbox, Mailbox.id == Message.mailbox_id)
.where(Message.user_id == user.id, Message.thread_id.in_([t.id for t in threads]))
.group_by(Message.thread_id)
)
current_special = view.split(":", 1)[1] if view.startswith("folder:") else None
if current_special in _REMOVAL_FOLDERS:
count_stmt = count_stmt.where(Mailbox.special_use == current_special)
else:
count_stmt = count_stmt.where(
or_(Mailbox.special_use.is_(None), Mailbox.special_use.notin_(_REMOVAL_FOLDERS))
)
counts = {
r[0]: r[1] for r in (await db.execute(count_stmt)).all() if r[0] is not None
}
return ThreadListOut(
threads=[_summary(t, counts.get(t.id)) for t in threads], cursor=next_cursor
)
async def _resolve_thread(db: AsyncSession, user: User, thread_id: uuid.UUID) -> Thread: async def _resolve_thread(db: AsyncSession, user: User, thread_id: uuid.UUID) -> Thread:
@@ -135,23 +162,40 @@ async def _resolve_thread(db: AsyncSession, user: User, thread_id: uuid.UUID) ->
return thread return thread
# Trash/Spam are message-scoped folders: an individually deleted message lives
# there on its own. Viewing a thread from Trash/Spam shows ONLY the messages filed
# there; every other view shows the conversation with trashed/spam messages hidden.
_REMOVAL_FOLDERS = frozenset({"trash", "junk"})
@router.get("/{thread_id}", response_model=ThreadDetail) @router.get("/{thread_id}", response_model=ThreadDetail)
async def get_thread( async def get_thread(
thread_id: uuid.UUID, thread_id: uuid.UUID,
user: CurrentUser, user: CurrentUser,
db: DbSession, db: DbSession,
sort: str = Query("asc"), sort: str = Query("asc"),
view: str = Query("unified"),
) -> ThreadDetail: ) -> ThreadDetail:
thread = await _resolve_thread(db, user, thread_id) thread = await _resolve_thread(db, user, thread_id)
current_special = view.split(":", 1)[1] if view.startswith("folder:") else None
# Messages are always returned ASC (oldest first); the client reverses for # Messages are always returned ASC (oldest first); the client reverses for
# display per the requested order (invariant I-5 / §9.4). # display per the requested order (invariant I-5 / §9.4).
messages = list( stmt = (
await db.scalars(
select(Message) select(Message)
.join(Mailbox, Mailbox.id == Message.mailbox_id)
.where(Message.thread_id == thread_id, Message.user_id == user.id) .where(Message.thread_id == thread_id, Message.user_id == user.id)
.order_by(Message.internal_date.asc(), Message.id.asc()) .order_by(Message.internal_date.asc(), Message.id.asc())
) )
if current_special in _REMOVAL_FOLDERS:
# Show only the messages actually in this Trash/Spam folder.
stmt = stmt.where(Mailbox.special_use == current_special)
else:
# Conversation view: hide messages that were individually trashed or spammed
# (keep custom folders, special_use IS NULL, and every non-removal folder).
stmt = stmt.where(
or_(Mailbox.special_use.is_(None), Mailbox.special_use.notin_(_REMOVAL_FOLDERS))
) )
messages = list(await db.scalars(stmt))
order = "desc" if sort == "desc" else "asc" order = "desc" if sort == "desc" else "asc"
return ThreadDetail( return ThreadDetail(
id=thread.id, id=thread.id,
+5
View File
@@ -147,6 +147,11 @@ class Message(Base):
mailbox_id: Mapped[uuid.UUID] = mapped_column( mailbox_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True), ForeignKey("mailboxes.id", ondelete="CASCADE"), nullable=False UUID(as_uuid=True), ForeignKey("mailboxes.id", ondelete="CASCADE"), nullable=False
) )
# The folder this message was in before being moved to Archive/Trash/Junk, so
# Restore can return it there (e.g. Sent) instead of the Inbox. NULL otherwise.
restore_mailbox_id: Mapped[uuid.UUID | None] = mapped_column(
UUID(as_uuid=True), ForeignKey("mailboxes.id", ondelete="SET NULL")
)
uid: Mapped[int] = mapped_column(Integer, nullable=False) uid: Mapped[int] = mapped_column(Integer, nullable=False)
thread_id: Mapped[uuid.UUID | None] = mapped_column( thread_id: Mapped[uuid.UUID | None] = mapped_column(
UUID(as_uuid=True), ForeignKey("threads.id") UUID(as_uuid=True), ForeignKey("threads.id")
+7
View File
@@ -24,6 +24,7 @@ from app.indexer.indexer import persist_message
from app.models import Account, Draft, KnownRecipient, Mailbox, Message, Outbox from app.models import Account, Draft, KnownRecipient, Mailbox, Message, Outbox
from app.outbox.attachments import load_attachments from app.outbox.attachments import load_attachments
from app.outbox.mime_build import build_mime from app.outbox.mime_build import build_mime
from app.outbox.service import delete_mirrored_draft_row
from app.providers.base import RemoteMessage from app.providers.base import RemoteMessage
from app.search.factory import get_search_backend from app.search.factory import get_search_backend
from app.sync.provider_factory import build_provider from app.sync.provider_factory import build_provider
@@ -178,6 +179,12 @@ async def send_job(ctx: dict[str, Any], outbox_id: str) -> None:
draft_row = await session.get(Draft, draft.id) draft_row = await session.get(Draft, draft.id)
if draft_row is not None: if draft_row is not None:
await session.delete(draft_row) await session.delete(draft_row)
# Also drop the mirrored Drafts-folder copy's DB row (we expunged it on the
# server above) so it doesn't resurface as a "remote" draft.
if draft_imap_uid:
await delete_mirrored_draft_row(
session, account_id=account_obj.id, imap_uid=draft_imap_uid
)
await _harvest_recipients(session, user_id, envelope_to) await _harvest_recipients(session, user_id, envelope_to)
bus = get_event_bus() bus = get_event_bus()
+24 -1
View File
@@ -9,7 +9,7 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.config import get_settings from app.config import get_settings
from app.models import Account, Draft, Outbox from app.models import Account, Draft, Mailbox, Message, Outbox
from app.outbox.schemas import DraftIn from app.outbox.schemas import DraftIn
@@ -17,6 +17,29 @@ class DraftError(Exception):
"""Draft could not be created/updated (e.g. account not owned).""" """Draft could not be created/updated (e.g. account not owned)."""
async def delete_mirrored_draft_row(
session: AsyncSession, *, account_id: uuid.UUID, imap_uid: int
) -> None:
"""Delete the DB row for a draft that was mirrored into the IMAP Drafts folder.
The server copy is expunged separately (send_job / remove_imap_draft_job).
Without also dropping this row the draft resurfaces in ``list_remote_drafts``
as an "other client" draft once its local ``drafts`` row is gone — until the
next Drafts poll would catch the expunge. Used on both send and discard.
"""
mirror = await session.scalar(
select(Message)
.join(Mailbox, Mailbox.id == Message.mailbox_id)
.where(
Message.account_id == account_id,
Message.uid == imap_uid,
Mailbox.special_use == "drafts",
)
)
if mirror is not None:
await session.delete(mirror)
async def upsert_draft( async def upsert_draft(
session: AsyncSession, *, user_id: uuid.UUID, body: DraftIn, draft_id: uuid.UUID | None session: AsyncSession, *, user_id: uuid.UUID, body: DraftIn, draft_id: uuid.UUID | None
) -> Draft: ) -> Draft:
+4
View File
@@ -83,6 +83,10 @@ class MailProvider(Protocol):
otherwise UID SEARCH + FETCH FLAGS diff.""" otherwise UID SEARCH + FETCH FLAGS diff."""
... ...
async def list_uids(self, mailbox: str) -> set[int]:
"""Full set of UIDs currently present in the mailbox (expunge reconciliation)."""
...
async def idle(self, mailbox: str) -> None: async def idle(self, mailbox: str) -> None:
"""Blocks until the server signals change or 25 min elapse (re-issue IDLE). May raise """Blocks until the server signals change or 25 min elapse (re-issue IDLE). May raise
ProviderNotSupported => sync loop falls back to poll() every POLL_WARM (default 180 s). ProviderNotSupported => sync loop falls back to poll() every POLL_WARM (default 180 s).
@@ -198,6 +198,18 @@ class ImapSmtpProvider(MailProvider):
if messages: if messages:
yield messages yield messages
async def list_uids(self, mailbox: str) -> set[int]:
"""Full set of UIDs currently present in the mailbox (expunge reconciliation).
Cheap — a single ``UID SEARCH ALL``. The CONDSTORE poll only reports new and
changed messages, not vanished ones (that needs QRESYNC/VANISHED); callers use
this to detect server-side expunges for small folders like Drafts.
"""
await self._select(mailbox)
imap = await self._connect()
resp = await imap.uid_search("ALL")
return set(parse_search_uids(resp.lines))
async def _fetch_messages(self, imap: aioimaplib.IMAP4, uids: list[int]) -> list[RemoteMessage]: async def _fetch_messages(self, imap: aioimaplib.IMAP4, uids: list[int]) -> list[RemoteMessage]:
if not uids: if not uids:
return [] return []
+75
View File
@@ -32,6 +32,10 @@ _MAX_TRIES = 5
# Folder names to create when a target special-use folder is missing on the server. # Folder names to create when a target special-use folder is missing on the server.
_DEFAULT_FOLDER_NAME = {"archive": "Archive", "trash": "Trash", "junk": "Spam"} _DEFAULT_FOLDER_NAME = {"archive": "Archive", "trash": "Trash", "junk": "Spam"}
# Moves into these folders record where the message came from (restore_mailbox_id)
# so the Restore action can return it to its original folder.
_REMOVAL_FOLDERS = frozenset({"archive", "trash", "junk"})
def sanitize_flags(flags: Iterable[str]) -> frozenset[str]: def sanitize_flags(flags: Iterable[str]) -> frozenset[str]:
return frozenset(f for f in flags if f in ALLOWED_FLAGS) return frozenset(f for f in flags if f in ALLOWED_FLAGS)
@@ -214,6 +218,9 @@ async def move_thread_to_folder_job(ctx: dict[str, Any], thread_id: str, special
) )
message = await session.get(Message, mid) message = await session.get(Message, mid)
if message is not None and resolved is not None: if message is not None and resolved is not None:
# Remember where it came from so Restore returns it there, not the Inbox.
if special_use in _REMOVAL_FOLDERS and message.mailbox_id != resolved:
message.restore_mailbox_id = message.mailbox_id
message.mailbox_id = resolved message.mailbox_id = resolved
message.uid = new_uid message.uid = new_uid
@@ -222,6 +229,74 @@ async def move_thread_to_folder_job(ctx: dict[str, Any], thread_id: str, special
) )
async def restore_thread_job(ctx: dict[str, Any], thread_id: str) -> None:
"""Move every message in a thread back to its original folder (Restore action).
Each message returns to ``restore_mailbox_id`` (the folder it was in before it
was archived/trashed) — e.g. a deleted Sent message goes back to Sent. Messages
with no recorded origin fall back to the account's Inbox.
"""
tid = uuid.UUID(thread_id)
async with session_scope() as session:
messages = list(await session.scalars(select(Message).where(Message.thread_id == tid)))
if not messages:
return
user_id = messages[0].user_id
account_ids = list({m.account_id for m in messages})
accounts = {
a.id: a
for a in await session.scalars(select(Account).where(Account.id.in_(account_ids)))
}
mailboxes = list(
await session.scalars(select(Mailbox).where(Mailbox.account_id.in_(account_ids)))
)
path_by_id = {mb.id: mb.path for mb in mailboxes}
inbox_by_account = {
mb.account_id: mb.id for mb in mailboxes if mb.special_use == "inbox"
}
# (message_id, account_id, source_path, target_id) — resolve the target now.
plan: list[tuple[uuid.UUID, uuid.UUID, str, uuid.UUID, int]] = []
for m in messages:
target_id = m.restore_mailbox_id
if target_id is None or target_id not in path_by_id:
target_id = inbox_by_account.get(m.account_id)
if target_id is None or target_id == m.mailbox_id:
continue # nowhere to restore to, or already there
plan.append(
(m.id, m.account_id, path_by_id.get(m.mailbox_id, "INBOX"), target_id, m.uid)
)
# (message_id, target_id, new_uid) applied after the network moves.
moves: list[tuple[uuid.UUID, uuid.UUID, int]] = []
for account_id in account_ids:
provider = build_provider(accounts[account_id])
try:
for mid, m_account, src_path, target_id, uid in plan:
if m_account != account_id:
continue
target_path = path_by_id[target_id]
try:
new_uid = await provider.move(uid, src_path, target_path)
except Exception:
logger.exception("restore move failed for message %s -> %s", mid, target_path)
continue
moves.append((mid, target_id, new_uid))
finally:
await provider.close()
async with session_scope() as session:
for mid, target_id, new_uid in moves:
message = await session.get(Message, mid)
if message is not None:
message.mailbox_id = target_id
message.uid = new_uid
message.restore_mailbox_id = None
await get_event_bus().publish(
"thread.updated", {"user_id": str(user_id), "thread_id": str(tid)}
)
async def delete_thread_job(ctx: dict[str, Any], thread_id: str) -> None: async def delete_thread_job(ctx: dict[str, Any], thread_id: str) -> None:
"""Permanently delete every message in a thread (server EXPUNGE + DB rows). """Permanently delete every message in a thread (server EXPUNGE + DB rows).
+44 -1
View File
@@ -15,7 +15,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.config import get_settings from app.config import get_settings
from app.events import get_event_bus from app.events import get_event_bus
from app.indexer.indexer import PersistResult, persist_message from app.indexer.indexer import PersistResult, persist_message
from app.models import Account, Mailbox, Message from app.models import Account, Draft, Mailbox, Message
from app.providers.base import MailboxDelta, ProviderNotSupported from app.providers.base import MailboxDelta, ProviderNotSupported
from app.providers.imap_smtp import ImapSmtpProvider from app.providers.imap_smtp import ImapSmtpProvider
from app.search.factory import get_search_backend from app.search.factory import get_search_backend
@@ -248,6 +248,49 @@ async def apply_delta(
}, },
) )
# Expunge reconciliation for Drafts. The mirror mechanism (and other clients)
# expunge server copies, but a CONDSTORE poll doesn't report vanished UIDs, so
# expunged drafts would linger in the DB and resurface as phantom "remote"
# drafts. Drafts folders are small, so a full UID reconcile per poll is cheap.
if mailbox.special_use == "drafts":
await _reconcile_drafts(provider, account, mailbox)
async def _reconcile_drafts(
provider: ImapSmtpProvider, account: Account, mailbox: Mailbox
) -> None:
"""Drop DB rows for drafts that no longer exist on the server (were expunged)."""
from app.db import session_scope
try:
server_uids = await provider.list_uids(mailbox.path)
except Exception:
logger.exception("drafts reconcile: list_uids failed for %s", mailbox.path)
return
async with session_scope() as session:
db_uids = set(
await session.scalars(select(Message.uid).where(Message.mailbox_id == mailbox.id))
)
gone = db_uids - server_uids
if not gone:
return
await session.execute(
delete(Message).where(Message.mailbox_id == mailbox.id, Message.uid.in_(gone))
)
# A local draft whose mirrored copy vanished becomes local-only again.
await session.execute(
update(Draft)
.where(Draft.account_id == account.id, Draft.imap_uid.in_(gone))
.values(imap_uid=None)
)
logger.info("drafts reconcile: removed %d expunged row(s) in %s", len(gone), mailbox.path)
await get_event_bus().publish(
"message.deleted",
{"user_id": str(account.user_id), "account_id": str(account.id), "mailbox": mailbox.path},
)
async def live_loop( async def live_loop(
account_id: uuid.UUID, mailbox_path: str = "INBOX", stop: asyncio.Event | None = None account_id: uuid.UUID, mailbox_path: str = "INBOX", stop: asyncio.Event | None = None
@@ -0,0 +1,32 @@
"""add messages.restore_mailbox_id
Records the folder a message was in before it was moved to Archive/Trash/Junk, so
the Restore action can return it to where it came from (e.g. Sent) instead of
always dumping it in the Inbox.
Revision ID: 0010
Revises: 0009
Create Date: 2026-07-20
"""
from __future__ import annotations
from collections.abc import Sequence
from alembic import op
revision: str = "0010"
down_revision: str | None = "0009"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.execute(
"ALTER TABLE messages ADD COLUMN restore_mailbox_id uuid "
"REFERENCES mailboxes(id) ON DELETE SET NULL"
)
def downgrade() -> None:
op.execute("ALTER TABLE messages DROP COLUMN IF EXISTS restore_mailbox_id")
+44
View File
@@ -254,6 +254,50 @@ async def test_thread_detail_messages_ascending(api: ApiCtx) -> None:
assert dates == sorted(dates) # ascending assert dates == sorted(dates) # ascending
async def test_thread_detail_hides_trashed_message_unless_viewing_trash(api: ApiCtx) -> None:
assert api.session is not None
# Give the account a Trash folder and delete a single message of the 2-message
# kickoff thread (move it to Trash) — as the per-message Delete action does.
trash = Mailbox(
account_id=api.account_id, user_id=api.user_id, path="Trash", special_use="trash"
)
api.session.add(trash)
await api.session.flush()
resp = await api.client.get("/api/threads")
kickoff = next(t for t in resp.json()["threads"] if t["message_count"] == 2)
detail = (await api.client.get(f"/api/threads/{kickoff['id']}")).json()
victim = uuid.UUID(detail["messages"][-1]["id"])
msg = await api.session.get(Message, victim)
assert msg is not None
msg.mailbox_id = trash.id
await api.session.commit()
# Viewing the thread from the Inbox: the trashed message is gone.
inbox_view = await api.client.get(
f"/api/threads/{kickoff['id']}", params={"view": "folder:inbox"}
)
ids = [m["id"] for m in inbox_view.json()["messages"]]
assert str(victim) not in ids
assert len(ids) == 1
# Viewing from the Trash folder: only the trashed message is shown (Trash is
# message-scoped), not the rest of the conversation.
trash_view = await api.client.get(
f"/api/threads/{kickoff['id']}", params={"view": "folder:trash"}
)
ids_trash = [m["id"] for m in trash_view.json()["messages"]]
assert ids_trash == [str(victim)]
# The Inbox LIST count drops the trashed message (was 2, now 1); the Trash list
# counts only the one trashed message.
inbox_list = await api.client.get("/api/threads", params={"view": "folder:inbox"})
row = next(t for t in inbox_list.json()["threads"] if t["id"] == kickoff["id"])
assert row["message_count"] == 1
trash_list = await api.client.get("/api/threads", params={"view": "folder:trash"})
trow = next(t for t in trash_list.json()["threads"] if t["id"] == kickoff["id"])
assert trow["message_count"] == 1
async def test_message_html_is_sanitized(api: ApiCtx) -> None: async def test_message_html_is_sanitized(api: ApiCtx) -> None:
# the lunch message has HTML with a script # the lunch message has HTML with a script
resp = await api.client.get("/api/threads") resp = await api.client.get("/api/threads")
+8 -2
View File
@@ -225,14 +225,20 @@ async def test_archive_unknown_thread_404(ctx: Ctx) -> None:
assert ctx.enqueued == [] assert ctx.enqueued == []
async def test_restore_thread_enqueues_move_to_inbox(ctx: Ctx) -> None: async def test_restore_thread_enqueues_restore_job(ctx: Ctx) -> None:
resp = await ctx.client.post(f"/api/threads/{ctx.thread_id}/restore", headers=_hdr()) resp = await ctx.client.post(f"/api/threads/{ctx.thread_id}/restore", headers=_hdr())
assert resp.status_code == 200 assert resp.status_code == 200
assert ctx.enqueued == [ assert ctx.enqueued == [
("move_thread_to_folder_job", str(ctx.thread_id), "inbox"), ("restore_thread_job", str(ctx.thread_id)),
] ]
async def test_restore_unknown_thread_404(ctx: Ctx) -> None:
resp = await ctx.client.post(f"/api/threads/{uuid.uuid4()}/restore", headers=_hdr())
assert resp.status_code == 404
assert ctx.enqueued == []
async def test_delete_thread_enqueues_permanent_delete(ctx: Ctx) -> None: async def test_delete_thread_enqueues_permanent_delete(ctx: Ctx) -> None:
resp = await ctx.client.post(f"/api/threads/{ctx.thread_id}/delete", headers=_hdr()) resp = await ctx.client.post(f"/api/threads/{ctx.thread_id}/delete", headers=_hdr())
assert resp.status_code == 200 assert resp.status_code == 200
+67
View File
@@ -274,6 +274,73 @@ async def test_remote_drafts_listed_and_adopted(ctx: Ctx) -> None:
assert again.json()["id"] == draft["id"] assert again.json()["id"] == draft["id"]
async def test_adopt_remote_draft_carries_attachments(ctx: Ctx) -> None:
from datetime import UTC, datetime
from app.indexer import blobstore
from app.models import Attachment
drafts_mb = Mailbox(
account_id=ctx.account_id, user_id=ctx.user_id, path="DRAFTS", special_use="drafts"
)
ctx.session.add(drafts_mb)
await ctx.session.flush()
msg = Message(
user_id=ctx.user_id,
account_id=ctx.account_id,
mailbox_id=drafts_mb.id,
uid=95,
subject="Home draft with file",
to_addr=[{"name": "", "email": "bob@example.com"}],
cc_addr=[],
bcc_addr=[],
internal_date=datetime.now(UTC),
blob_sha256="y" * 64,
snippet="",
has_attachments=True,
)
ctx.session.add(msg)
await ctx.session.flush()
# A real (downloadable) attachment plus an inline cid image, which must NOT be carried.
sha = blobstore.store(ctx.user_id, b"file contents")
ctx.session.add(
Attachment(
message_id=msg.id,
user_id=ctx.user_id,
filename="report.pdf",
mime_type="application/pdf",
size_bytes=13,
content_id=None,
blob_sha256=sha,
)
)
ctx.session.add(
Attachment(
message_id=msg.id,
user_id=ctx.user_id,
filename="logo.png",
mime_type="image/png",
size_bytes=3,
content_id="logo@x",
blob_sha256=blobstore.store(ctx.user_id, b"img"),
)
)
await ctx.session.commit()
adopted = await ctx.client.post(f"/api/drafts/adopt/{msg.id}", headers=_hdr())
assert adopted.status_code == 200, adopted.text
ids = adopted.json()["payload"]["attachment_ids"]
assert len(ids) == 1 # only the real attachment, not the inline image
# The compose-attachment metadata resolves — this is what the editor renders and
# what load_attachments re-attaches on send.
meta = await ctx.client.get("/api/attachments", params={"ids": ",".join(ids)})
assert meta.status_code == 200
body = meta.json()
assert [a["filename"] for a in body] == ["report.pdf"]
assert body[0]["size_bytes"] == 13
async def test_publish_draft_enqueues_imap_mirror(ctx: Ctx) -> None: async def test_publish_draft_enqueues_imap_mirror(ctx: Ctx) -> None:
did = (await ctx.client.post("/api/drafts", json=_draft_body(ctx), headers=_hdr())).json()["id"] did = (await ctx.client.post("/api/drafts", json=_draft_body(ctx), headers=_hdr())).json()["id"]
ctx.enqueued.clear() ctx.enqueued.clear()
+71 -1
View File
@@ -13,7 +13,7 @@ from sqlalchemy.pool import NullPool
from app.config import get_settings from app.config import get_settings
from app.db import session_scope from app.db import session_scope
from app.identity.passwords import hash_password from app.identity.passwords import hash_password
from app.models import Account, Mailbox, Message, User from app.models import Account, Draft, Mailbox, Message, User
from app.providers.base import MailboxDelta, RemoteMessage from app.providers.base import MailboxDelta, RemoteMessage
from app.sync import engine as sync_engine from app.sync import engine as sync_engine
@@ -97,3 +97,73 @@ async def test_secondary_poll_syncs_special_folders_not_inbox(monkeypatch) -> No
await cleanup.execute(delete(User).where(User.id == user.id)) await cleanup.execute(delete(User).where(User.id == user.id))
await session.close() await session.close()
await engine.dispose() await engine.dispose()
async def test_drafts_poll_reconciles_expunged_rows(monkeypatch) -> None: # type: ignore[no-untyped-def]
"""A Drafts poll drops DB rows whose server copy was expunged (phantoms) and
un-links a local draft whose mirrored copy vanished."""
engine = create_async_engine(get_settings().database_url, poolclass=NullPool)
sm = async_sessionmaker(engine, expire_on_commit=False)
session = sm()
user = User(handle=f"rec{uuid.uuid4().hex[:8]}@test.local", password_hash=hash_password("pw"))
session.add(user)
await session.flush()
account = Account(
user_id=user.id,
email="me@example.com",
imap_config={},
smtp_config={},
secret_ct=b"",
discovery_src="manual",
)
session.add(account)
await session.flush()
drafts = Mailbox(account_id=account.id, user_id=user.id, path="DRAFTS", special_use="drafts")
session.add(drafts)
await session.flush()
# uid 2 survives on the server; uids 1 and 3 were expunged (phantom DB rows).
for uid in (1, 2, 3):
session.add(
Message(
user_id=user.id,
account_id=account.id,
mailbox_id=drafts.id,
uid=uid,
internal_date=datetime.now(UTC),
blob_sha256="",
)
)
# A local draft still linking the vanished uid 3 must be un-linked (imap_uid -> NULL).
linked = Draft(user_id=user.id, account_id=account.id, kind="new", payload={}, imap_uid=3)
kept = Draft(user_id=user.id, account_id=account.id, kind="new", payload={}, imap_uid=2)
session.add_all([linked, kept])
await session.commit()
class FakeProvider:
async def poll(self, mailbox: str, *, last_seen_uid: int, highestmodseq: int | None): # type: ignore[no-untyped-def]
return MailboxDelta([], [], [], None, None)
async def list_uids(self, mailbox: str) -> set[int]:
return {2} # only uid 2 remains on the server
async def close(self) -> None:
return None
monkeypatch.setattr(sync_engine, "build_provider", lambda _a: FakeProvider())
try:
await sync_engine._poll_secondary_once(account.id)
session.expunge_all()
remaining = set(
await session.scalars(select(Message.uid).where(Message.mailbox_id == drafts.id))
)
assert remaining == {2} # phantoms 1 and 3 removed
assert (await session.get(Draft, linked.id)).imap_uid is None # un-linked
assert (await session.get(Draft, kept.id)).imap_uid == 2 # untouched
finally:
async with session_scope() as cleanup:
await cleanup.execute(delete(User).where(User.id == user.id))
await session.close()
await engine.dispose()
+2
View File
@@ -19,6 +19,7 @@ from app.sync.actions import (
move_job, move_job,
move_thread_to_folder_job, move_thread_to_folder_job,
push_flags_job, push_flags_job,
restore_thread_job,
) )
from app.sync.engine import bootstrap_and_backfill, refresh_account_job from app.sync.engine import bootstrap_and_backfill, refresh_account_job
@@ -38,6 +39,7 @@ INTERACTIVE_FUNCTIONS = [
push_flags_job, push_flags_job,
move_job, move_job,
move_thread_to_folder_job, move_thread_to_folder_job,
restore_thread_job,
delete_thread_job, delete_thread_job,
send_job, send_job,
push_draft_to_imap_job, push_draft_to_imap_job,
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="64" height="64" viewBox="0 0 24 24" style="color: rgb(74, 85, 101); opacity: 1; transform: rotate(0deg);"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="m4 4l16 16m-3.5-3.244C15.147 17.485 13.618 18 12 18c-3.53 0-6.634-2.452-8.413-4.221c-.47-.467-.705-.7-.854-1.159c-.107-.327-.107-.913 0-1.24c.15-.459.385-.693.855-1.16c.897-.892 2.13-1.956 3.584-2.793M19.5 14.634c.333-.293.638-.582.912-.854l.003-.003c.468-.466.703-.7.852-1.156c.107-.327.107-.914 0-1.241c-.15-.458-.384-.692-.854-1.159C18.633 8.452 15.531 6 12 6q-.507 0-1 .064m2.323 7.436a2 2 0 0 1-2.762-2.889"></path></svg>

After

Width:  |  Height:  |  Size: 759 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="64" height="64" viewBox="0 0 24 24" style="color: rgb(74, 85, 101); opacity: 1; transform: rotate(0deg);"><g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"><path d="M3.587 13.779c1.78 1.769 4.883 4.22 8.413 4.22s6.634-2.451 8.413-4.22c.47-.467.705-.7.854-1.159c.107-.327.107-.913 0-1.24c-.15-.458-.385-.692-.854-1.159C18.633 8.452 15.531 6 12 6c-3.53 0-6.634 2.452-8.413 4.221c-.47.467-.705.7-.854 1.159c-.107.327-.107.913 0 1.24c.15.458.384.692.854 1.159"></path><path d="M10 12a2 2 0 1 0 4 0a2 2 0 0 0-4 0"></path></g></svg>

After

Width:  |  Height:  |  Size: 697 B

+8
View File
@@ -8,12 +8,16 @@ import compose from "../../icons/compose.svg?raw";
import drafts from "../../icons/drafts-28-regular.svg?raw"; import drafts from "../../icons/drafts-28-regular.svg?raw";
import expandAll from "../../icons/expand-all.svg?raw"; import expandAll from "../../icons/expand-all.svg?raw";
import forward from "../../icons/forward.svg?raw"; import forward from "../../icons/forward.svg?raw";
import hide from "../../icons/hide.svg?raw";
import inbox from "../../icons/inbox.svg?raw"; import inbox from "../../icons/inbox.svg?raw";
import logOut from "../../icons/log-out.svg?raw";
import pin from "../../icons/pin.svg?raw"; import pin from "../../icons/pin.svg?raw";
import print from "../../icons/print.svg?raw"; import print from "../../icons/print.svg?raw";
import replyAll from "../../icons/reply-all-line.svg?raw"; import replyAll from "../../icons/reply-all-line.svg?raw";
import reply from "../../icons/reply.svg?raw"; import reply from "../../icons/reply.svg?raw";
import sent from "../../icons/sent-02.svg?raw"; import sent from "../../icons/sent-02.svg?raw";
import settings from "../../icons/settings.svg?raw";
import show from "../../icons/show.svg?raw";
import spam from "../../icons/spam.svg?raw"; import spam from "../../icons/spam.svg?raw";
import trash from "../../icons/trash.svg?raw"; import trash from "../../icons/trash.svg?raw";
@@ -24,12 +28,16 @@ const RAW: Record<string, string> = {
drafts, drafts,
"expand-all": expandAll, "expand-all": expandAll,
forward, forward,
hide,
inbox, inbox,
"log-out": logOut,
pin, pin,
print, print,
reply, reply,
"reply-all": replyAll, "reply-all": replyAll,
sent, sent,
settings,
show,
spam, spam,
trash, trash,
}; };
+117 -12
View File
@@ -137,16 +137,37 @@ export function Composer({
const [showToolbar, setShowToolbar] = useState(true); const [showToolbar, setShowToolbar] = useState(true);
const [fromOpen, setFromOpen] = useState(false); const [fromOpen, setFromOpen] = useState(false);
const [moreOpen, setMoreOpen] = useState(false); const [moreOpen, setMoreOpen] = useState(false);
const moreRef = useRef<HTMLDivElement>(null);
const fromRef = useRef<HTMLDivElement>(null);
const [attachments, setAttachments] = useState<AttachmentOut[]>([]); const [attachments, setAttachments] = useState<AttachmentOut[]>([]);
const [attachError, setAttachError] = useState(""); const [attachError, setAttachError] = useState("");
const [status, setStatus] = useState<"" | "saving" | "saved" | "sending" | "error">(""); const [status, setStatus] = useState<"" | "saving" | "saved" | "sending" | "error">("");
const [undo, setUndo] = useState<{ outboxId: string } | null>(null); const [undo, setUndo] = useState<{ outboxId: string } | null>(null);
const [showClosePrompt, setShowClosePrompt] = useState(false);
const draftId = useRef<string | null>(init.draftId ?? null); const draftId = useRef<string | null>(init.draftId ?? null);
const sentRef = useRef(false); // send initiated -> don't mirror to Drafts on close const sentRef = useRef(false); // send initiated -> don't mirror to Drafts on close
// Whether the user actually edited anything. Merely opening a draft to view it
// must not re-save/re-mirror it (that spawned duplicate IMAP copies).
const dirtyRef = useRef(false);
const markDirty = () => {
dirtyRef.current = true;
};
const undoTimer = useRef<number | null>(null); const undoTimer = useRef<number | null>(null);
const fileInputRef = useRef<HTMLInputElement>(null); const fileInputRef = useRef<HTMLInputElement>(null);
const editorRef = useRef<RichTextEditorHandle>(null); const editorRef = useRef<RichTextEditorHandle>(null);
// Close the From / More dropdowns on any click outside them.
useEffect(() => {
if (!moreOpen && !fromOpen) return;
const onDown = (e: MouseEvent) => {
const target = e.target as Node;
if (moreOpen && moreRef.current && !moreRef.current.contains(target)) setMoreOpen(false);
if (fromOpen && fromRef.current && !fromRef.current.contains(target)) setFromOpen(false);
};
document.addEventListener("mousedown", onDown);
return () => document.removeEventListener("mousedown", onDown);
}, [moreOpen, fromOpen]);
const account = accounts.find((a) => a.id === accountId); const account = accounts.find((a) => a.id === accountId);
const fromName = account?.display_name || ""; const fromName = account?.display_name || "";
const fromEmail = account?.email ?? ""; const fromEmail = account?.email ?? "";
@@ -204,6 +225,7 @@ export function Composer({
}, []); }, []);
const addFiles = async (files: FileList) => { const addFiles = async (files: FileList) => {
markDirty();
setAttachError(""); setAttachError("");
for (const file of Array.from(files)) { for (const file of Array.from(files)) {
if (file.size > MAX_ATTACHMENT_BYTES) { if (file.size > MAX_ATTACHMENT_BYTES) {
@@ -219,8 +241,10 @@ export function Composer({
} }
}; };
const removeAttachment = (id: string) => const removeAttachment = (id: string) => {
markDirty();
setAttachments((prev) => prev.filter((a) => a.id !== id)); setAttachments((prev) => prev.filter((a) => a.id !== id));
};
// The editor's seed content. New messages get the account signature (2 blank // The editor's seed content. New messages get the account signature (2 blank
// lines above it); replies keep the quoted original in a marked block so the // lines above it); replies keep the quoted original in a marked block so the
@@ -263,6 +287,7 @@ export function Composer({
useEffect(() => { useEffect(() => {
if (undo) return; if (undo) return;
const handle = window.setTimeout(async () => { const handle = window.setTimeout(async () => {
if (!dirtyRef.current) return; // nothing the user changed -> don't touch the draft
if (!to && !subject && !bodyHtml) return; if (!to && !subject && !bodyHtml) return;
setStatus("saving"); setStatus("saving");
try { try {
@@ -284,8 +309,9 @@ export function Composer({
() => () => { () => () => {
if (undoTimer.current) window.clearTimeout(undoTimer.current); if (undoTimer.current) window.clearTimeout(undoTimer.current);
// On close, mirror the draft to the IMAP Drafts folder so it can be // On close, mirror the draft to the IMAP Drafts folder so it can be
// continued on any mail client (skip if it was sent or discarded). // continued on any mail client — but only if the user actually edited it
if (draftId.current && !sentRef.current) { // (skip if it was sent, discarded, or merely opened to view).
if (draftId.current && !sentRef.current && dirtyRef.current) {
void api.publishDraft(draftId.current).catch(() => {}); void api.publishDraft(draftId.current).catch(() => {});
} }
}, },
@@ -333,6 +359,26 @@ export function Composer({
onClose(); onClose();
}; };
// X button: if there's a draft worth keeping, ask before closing; otherwise
// (nothing typed, no draft) just close.
const requestClose = () => {
if (dirtyRef.current || draftId.current) setShowClosePrompt(true);
else onClose();
};
const saveDraftAndClose = async () => {
setShowClosePrompt(false);
try {
if (dirtyRef.current) {
if (draftId.current) await api.updateDraft(draftId.current, payload());
else draftId.current = (await api.createDraft(payload())).id;
}
} catch {
/* the close-time IMAP mirror is best-effort too */
}
onClose(); // the unmount effect mirrors the (dirty) draft to the IMAP Drafts folder
};
const popOut = async () => { const popOut = async () => {
// Persist first so the new window can load this exact draft. // Persist first so the new window can load this exact draft.
try { try {
@@ -367,12 +413,44 @@ export function Composer({
return ( return (
<div <div
className={`flex flex-col overflow-hidden border-app-border bg-app-elevated ${ className={`relative flex flex-col overflow-hidden border-app-border bg-app-elevated ${
fullWindow fullWindow
? "h-full" ? "h-full"
: `rounded-xl border shadow-2xl ${expanded ? "h-full" : ""}` : `rounded-xl border shadow-2xl ${expanded ? "h-full" : ""}`
}`} }`}
> >
{showClosePrompt && (
<div
className="absolute inset-0 z-40 flex items-center justify-center bg-black/40 p-4"
onClick={() => setShowClosePrompt(false)}
>
<div
className="w-72 rounded-lg border border-app-border bg-app-elevated p-4 shadow-2xl"
onClick={(e) => e.stopPropagation()}
>
<p className="mb-4 text-sm text-app-fg">{t("compose.closePrompt")}</p>
<div className="flex justify-end gap-2">
<button
type="button"
onClick={() => {
setShowClosePrompt(false);
void discard();
}}
className="rounded-md px-3 py-1.5 text-sm text-red-500 hover:bg-app-hover"
>
{t("compose.discard")}
</button>
<button
type="button"
onClick={() => void saveDraftAndClose()}
className="rounded-md bg-app-accent px-3 py-1.5 text-sm font-medium text-app-accent-fg hover:opacity-90"
>
{t("compose.saveDraft")}
</button>
</div>
</div>
</div>
)}
{/* Header: Send · Attach · formatting toggle · More · window controls */} {/* Header: Send · Attach · formatting toggle · More · window controls */}
<div className="flex items-center gap-1.5 border-b border-app-border bg-app-surface px-3 py-2"> <div className="flex items-center gap-1.5 border-b border-app-border bg-app-surface px-3 py-2">
<button <button
@@ -410,7 +488,7 @@ export function Composer({
> >
<span className="text-sm font-semibold underline">A</span> <span className="text-sm font-semibold underline">A</span>
</HeaderButton> </HeaderButton>
<div className="relative"> <div className="relative" ref={moreRef}>
<HeaderButton onClick={() => setMoreOpen((v) => !v)} title={t("compose.more")}> <HeaderButton onClick={() => setMoreOpen((v) => !v)} title={t("compose.more")}>
{t("compose.more")} {t("compose.more")}
</HeaderButton> </HeaderButton>
@@ -469,7 +547,7 @@ export function Composer({
</svg> </svg>
</HeaderButton> </HeaderButton>
)} )}
<HeaderButton onClick={onClose} title={t("common.close")}> <HeaderButton onClick={requestClose} title={t("common.close")}>
<svg width="15" height="15" viewBox="0 0 16 16" fill="none" stroke="currentColor" strokeWidth="1.5" strokeLinecap="round"> <svg width="15" height="15" viewBox="0 0 16 16" fill="none" stroke="currentColor" strokeWidth="1.5" strokeLinecap="round">
<path d="M4 4l8 8M12 4l-8 8" /> <path d="M4 4l8 8M12 4l-8 8" />
</svg> </svg>
@@ -478,7 +556,10 @@ export function Composer({
</div> </div>
{/* From */} {/* From */}
<div className="relative flex items-center gap-2 border-b border-app-border px-3 py-2"> <div
className="relative flex items-center gap-2 border-b border-app-border px-3 py-2"
ref={fromRef}
>
<span className="w-10 shrink-0 text-xs font-semibold text-app-fg">{t("compose.from")}</span> <span className="w-10 shrink-0 text-xs font-semibold text-app-fg">{t("compose.from")}</span>
<button <button
type="button" type="button"
@@ -500,6 +581,7 @@ export function Composer({
key={a.id} key={a.id}
className="flex w-full items-center gap-2 px-3 py-1.5 text-left text-sm hover:bg-app-hover" className="flex w-full items-center gap-2 px-3 py-1.5 text-left text-sm hover:bg-app-hover"
onClick={() => { onClick={() => {
markDirty();
setAccountId(a.id); setAccountId(a.id);
setFromOpen(false); setFromOpen(false);
}} }}
@@ -517,7 +599,10 @@ export function Composer({
<RecipientChips <RecipientChips
label={t("compose.to")} label={t("compose.to")}
value={to} value={to}
onChange={setTo} onChange={(v) => {
markDirty();
setTo(v);
}}
autoFocus={init.kind === "new" || init.kind === "forward"} autoFocus={init.kind === "new" || init.kind === "forward"}
rightSlot={ rightSlot={
!showCc ? ( !showCc ? (
@@ -533,15 +618,32 @@ export function Composer({
/> />
{showCc && ( {showCc && (
<> <>
<RecipientChips label={t("compose.cc")} value={cc} onChange={setCc} /> <RecipientChips
<RecipientChips label={t("compose.bcc")} value={bcc} onChange={setBcc} /> label={t("compose.cc")}
value={cc}
onChange={(v) => {
markDirty();
setCc(v);
}}
/>
<RecipientChips
label={t("compose.bcc")}
value={bcc}
onChange={(v) => {
markDirty();
setBcc(v);
}}
/>
</> </>
)} )}
<input <input
className="border-b border-app-border bg-transparent px-3 py-2 text-sm text-app-fg outline-none placeholder:text-app-faint" className="border-b border-app-border bg-transparent px-3 py-2 text-sm text-app-fg outline-none placeholder:text-app-faint"
placeholder={t("compose.subject")} placeholder={t("compose.subject")}
value={subject} value={subject}
onChange={(e) => setSubject(e.target.value)} onChange={(e) => {
markDirty();
setSubject(e.target.value);
}}
/> />
{(attachments.length > 0 || attachError) && ( {(attachments.length > 0 || attachError) && (
@@ -575,7 +677,10 @@ export function Composer({
<RichTextEditor <RichTextEditor
ref={editorRef} ref={editorRef}
initialHtml={initialHtml} initialHtml={initialHtml}
onInput={setBodyHtml} onInput={(html) => {
markDirty();
setBodyHtml(html);
}}
showToolbar={showToolbar} showToolbar={showToolbar}
/> />
) : ( ) : (
+56 -18
View File
@@ -3,6 +3,7 @@ import { Fragment, useEffect, useMemo, useRef, useState } from "react";
import { useT } from "../../i18n"; import { useT } from "../../i18n";
import { api } from "../../lib/api"; import { api } from "../../lib/api";
import { printMessages } from "../../lib/print";
import type { AccountOut, MailboxOut, MessageEnvelope } from "../../lib/types"; import type { AccountOut, MailboxOut, MessageEnvelope } from "../../lib/types";
import { Composer, type ComposerInit } from "../compose/Composer"; import { Composer, type ComposerInit } from "../compose/Composer";
import { Icon } from "../Icon"; import { Icon } from "../Icon";
@@ -137,10 +138,12 @@ function ActionsMenu({
onReply, onReply,
onPrint, onPrint,
onDelete, onDelete,
replyDisabled = false,
}: { }: {
onReply: (kind: string) => void; onReply: (kind: string) => void;
onPrint: () => void; onPrint: () => void;
onDelete: () => void; onDelete: () => void;
replyDisabled?: boolean;
}) { }) {
const t = useT(); const t = useT();
const [open, setOpen] = useState(false); const [open, setOpen] = useState(false);
@@ -150,13 +153,25 @@ function ActionsMenu({
const onDoc = (e: MouseEvent) => { const onDoc = (e: MouseEvent) => {
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false); if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
}; };
// Clicking inside the sandboxed message-body iframe doesn't reach document
// mousedown, but it blurs the parent window — close the menu on that too.
const onBlur = () => setOpen(false);
document.addEventListener("mousedown", onDoc); document.addEventListener("mousedown", onDoc);
return () => document.removeEventListener("mousedown", onDoc); window.addEventListener("blur", onBlur);
return () => {
document.removeEventListener("mousedown", onDoc);
window.removeEventListener("blur", onBlur);
};
}, [open]); }, [open]);
const item = (label: string, onClick: () => void) => ( const item = (label: string, onClick: () => void, disabled = false) => (
<button <button
className="block w-full px-3 py-1.5 text-left text-xs text-app-fg hover:bg-app-hover" disabled={disabled}
className={`block w-full px-3 py-1.5 text-left text-xs ${
disabled
? "cursor-not-allowed text-app-faint"
: "text-app-fg hover:bg-app-hover"
}`}
onClick={() => { onClick={() => {
setOpen(false); setOpen(false);
onClick(); onClick();
@@ -177,10 +192,10 @@ function ActionsMenu({
</button> </button>
{open && ( {open && (
<div className="absolute right-0 z-20 mt-1 w-52 overflow-hidden rounded-md border border-app-border bg-app-elevated py-1 shadow-lg"> <div className="absolute right-0 z-20 mt-1 w-52 overflow-hidden rounded-md border border-app-border bg-app-elevated py-1 shadow-lg">
{item(t("mail.reply"), () => onReply("reply"))} {item(t("mail.reply"), () => onReply("reply"), replyDisabled)}
{item(t("mail.replyAll"), () => onReply("reply_all"))} {item(t("mail.replyAll"), () => onReply("reply_all"), replyDisabled)}
{item(t("mail.forward"), () => onReply("forward"))} {item(t("mail.forward"), () => onReply("forward"), replyDisabled)}
{item(t("mail.forwardAsAttachment"), () => onReply("forward"))} {item(t("mail.forwardAsAttachment"), () => onReply("forward"), replyDisabled)}
{item(t("mail.print"), onPrint)} {item(t("mail.print"), onPrint)}
<div className="my-1 border-t border-app-border" /> <div className="my-1 border-t border-app-border" />
{item(t("mail.delete"), onDelete)} {item(t("mail.delete"), onDelete)}
@@ -199,6 +214,7 @@ function MessageItem({
boxClassName, boxClassName,
zIndex, zIndex,
showReplyBar, showReplyBar,
canReply,
onToggle, onToggle,
onReply, onReply,
onToggleRead, onToggleRead,
@@ -214,6 +230,7 @@ function MessageItem({
boxClassName: string; boxClassName: string;
zIndex: number; zIndex: number;
showReplyBar: boolean; showReplyBar: boolean;
canReply: boolean;
onToggle: () => void; onToggle: () => void;
onReply: (kind: string) => void; onReply: (kind: string) => void;
onToggleRead: () => void; onToggleRead: () => void;
@@ -321,6 +338,7 @@ function MessageItem({
}`} }`}
/> />
</button> </button>
{canReply && (
<button <button
className="flex items-center gap-1 text-app-muted hover:text-app-fg" className="flex items-center gap-1 text-app-muted hover:text-app-fg"
onClick={() => onReply(multiRecipient ? "reply_all" : "reply")} onClick={() => onReply(multiRecipient ? "reply_all" : "reply")}
@@ -328,7 +346,13 @@ function MessageItem({
<Icon name={multiRecipient ? "reply-all" : "reply"} size={14} /> <Icon name={multiRecipient ? "reply-all" : "reply"} size={14} />
{multiRecipient ? t("mail.replyAll") : t("mail.reply")} {multiRecipient ? t("mail.replyAll") : t("mail.reply")}
</button> </button>
<ActionsMenu onReply={onReply} onPrint={onPrint} onDelete={onDelete} /> )}
<ActionsMenu
onReply={onReply}
onPrint={onPrint}
onDelete={onDelete}
replyDisabled={!canReply}
/>
</div> </div>
</div> </div>
{showDetails && ( {showDetails && (
@@ -356,7 +380,7 @@ function MessageItem({
<div className="px-4 pb-3"> <div className="px-4 pb-3">
<MessageBody message={message} /> <MessageBody message={message} />
</div> </div>
{showReplyBar && ( {showReplyBar && canReply && (
<div className="flex gap-2 border-t border-app-border px-4 py-3"> <div className="flex gap-2 border-t border-app-border px-4 py-3">
{replyBtn("reply", t("mail.reply"), "reply")} {replyBtn("reply", t("mail.reply"), "reply")}
{replyBtn("reply_all", t("mail.replyAll"), "reply-all")} {replyBtn("reply_all", t("mail.replyAll"), "reply-all")}
@@ -440,6 +464,7 @@ export function ThreadView({
}; };
const openReply = (msg: MessageEnvelope, kind: string) => { const openReply = (msg: MessageEnvelope, kind: string) => {
if (specialUse.get(msg.mailbox_id) === "trash") return; // no reply/forward from Trash
setExpanded((prev) => new Set(prev).add(msg.id)); setExpanded((prev) => new Set(prev).add(msg.id));
setComposing({ anchorId: msg.id, init: buildReplyInit(subject, msg, kind) }); setComposing({ anchorId: msg.id, init: buildReplyInit(subject, msg, kind) });
}; };
@@ -471,9 +496,15 @@ export function ThreadView({
allExpanded ? new Set(newest ? [newest.id] : []) : new Set(messages.map((m) => m.id)), allExpanded ? new Set(newest ? [newest.id] : []) : new Set(messages.map((m) => m.id)),
); );
const composerRef = useRef<HTMLDivElement | null>(null);
// Bring the inline composer into view when it opens (reply/reply-all/forward).
useEffect(() => {
if (composing) composerRef.current?.scrollIntoView({ behavior: "smooth", block: "center" });
}, [composing]);
const composerFor = (id: string) => const composerFor = (id: string) =>
composing?.anchorId === id ? ( composing?.anchorId === id ? (
<div className="px-4 py-3"> <div ref={composerRef} className="px-4 py-3">
<Composer init={composing.init} onClose={() => setComposing(null)} /> <Composer init={composing.init} onClose={() => setComposing(null)} />
</div> </div>
) : null; ) : null;
@@ -487,7 +518,7 @@ export function ThreadView({
className="rounded p-1.5 text-app-muted hover:bg-app-hover hover:text-app-fg" className="rounded p-1.5 text-app-muted hover:bg-app-hover hover:text-app-fg"
title={t("mail.print")} title={t("mail.print")}
aria-label={t("mail.print")} aria-label={t("mail.print")}
onClick={() => window.print()} onClick={() => void printMessages(subject, messages)}
> >
<Icon name="print" size={16} /> <Icon name="print" size={16} />
</button> </button>
@@ -507,17 +538,23 @@ export function ThreadView({
const isExpanded = expanded.has(message.id); const isExpanded = expanded.has(message.id);
const prev = i > 0 ? visual[i - 1] : undefined; const prev = i > 0 ? visual[i - 1] : undefined;
const prevCollapsed = prev ? !expanded.has(prev.id) : false; const prevCollapsed = prev ? !expanded.has(prev.id) : false;
const prevIsNewest = Boolean(prev && newest && prev.id === newest.id); // Cards overlap by a negative top margin (tuck up under the card above),
// Overlap onto a preceding collapsed card, and also onto the always-open // with the newer card kept on top by z-index. The card above is OLDER in
// newest message — so the newest stacks with its neighbour in both sort // oldest-first but NEWER in newest-first, so the rule differs by order:
// orders (bottom card in oldest-first, top card in newest-first). z-index // - oldest-first: an expanded card tucks up over the older card above it
// keeps the newer/"current" edge on top so text stays visible. // (its own top stays clear). Only overlap a collapsed card above.
const overlap = prevCollapsed || prevIsNewest; // - newest-first: the card above is newer, so tucking an expanded card up
// would hide its header under that newer card. Only collapsed cards tuck
// up; an expanded card keeps its gap, so its top stays clear of the newer
// message and the older card BELOW tucks under it instead.
const overlap = order === "desc" ? !isExpanded : prevCollapsed;
const margin = i === 0 ? "" : overlap ? "-mt-[20px]" : "mt-2"; const margin = i === 0 ? "" : overlap ? "-mt-[20px]" : "mt-2";
const z = order === "asc" ? i + 1 : visual.length - i; const z = order === "asc" ? i + 1 : visual.length - i;
const boxClassName = `${margin} rounded-lg border border-app-border bg-app-surface ${ const boxClassName = `${margin} rounded-lg border border-app-border bg-app-surface ${
isExpanded ? "" : "shadow-sm" isExpanded ? "" : "shadow-sm"
}`; }`;
// Reply/Reply-all/Forward make no sense on a trashed message.
const canReply = specialUse.get(message.mailbox_id) !== "trash";
return ( return (
<Fragment key={message.id}> <Fragment key={message.id}>
<MessageItem <MessageItem
@@ -529,10 +566,11 @@ export function ThreadView({
boxClassName={boxClassName} boxClassName={boxClassName}
zIndex={z} zIndex={z}
showReplyBar={isNewest && composing?.anchorId !== message.id} showReplyBar={isNewest && composing?.anchorId !== message.id}
canReply={canReply}
onToggle={() => toggle(message.id)} onToggle={() => toggle(message.id)}
onReply={(kind) => openReply(message, kind)} onReply={(kind) => openReply(message, kind)}
onToggleRead={() => toggleRead(message)} onToggleRead={() => toggleRead(message)}
onPrint={() => window.print()} onPrint={() => void printMessages(subject, [message])}
onDelete={() => deleteMessage(message)} onDelete={() => deleteMessage(message)}
innerRef={isNewest ? (el) => (newestRef.current = el) : undefined} innerRef={isNewest ? (el) => (newestRef.current = el) : undefined}
/> />
+4
View File
@@ -38,6 +38,8 @@ export const en = {
"nav.mail": "Mail", "nav.mail": "Mail",
"nav.settings": "Settings", "nav.settings": "Settings",
"nav.compose": "Compose", "nav.compose": "Compose",
"nav.collapseSidebar": "Collapse sidebar",
"nav.expandSidebar": "Expand sidebar",
"nav.allAccounts": "All accounts", "nav.allAccounts": "All accounts",
"nav.inbox": "Inbox", "nav.inbox": "Inbox",
"nav.archive": "Archive", "nav.archive": "Archive",
@@ -103,6 +105,8 @@ export const en = {
"compose.send": "Send", "compose.send": "Send",
"compose.sending": "Sending…", "compose.sending": "Sending…",
"compose.discard": "Discard", "compose.discard": "Discard",
"compose.saveDraft": "Save draft",
"compose.closePrompt": "Save this message as a draft?",
"compose.sent": "Message sent.", "compose.sent": "Message sent.",
"compose.undo": "Undo", "compose.undo": "Undo",
"compose.sendFailed": "Send failed.", "compose.sendFailed": "Send failed.",
+2 -2
View File
@@ -94,8 +94,8 @@ export const api = {
if (cursor) params.set("cursor", cursor); if (cursor) params.set("cursor", cursor);
return apiFetch<ThreadListOut>(`/threads?${params.toString()}`); return apiFetch<ThreadListOut>(`/threads?${params.toString()}`);
}, },
thread: (id: string, sort: "asc" | "desc" = "asc") => thread: (id: string, sort: "asc" | "desc" = "asc", view = "unified") =>
apiFetch<ThreadDetail>(`/threads/${id}?sort=${sort}`), apiFetch<ThreadDetail>(`/threads/${id}?sort=${sort}&view=${encodeURIComponent(view)}`),
search: (q: string, cursor?: string) => { search: (q: string, cursor?: string) => {
const params = new URLSearchParams({ q }); const params = new URLSearchParams({ q });
+200
View File
@@ -0,0 +1,200 @@
// Mail-style printing: assemble a clean, print-ready document (message headers +
// sanitized body, one section per message for a thread) in a new window and invoke
// the browser's print dialog. Reuses the same sanitized HTML the reader shows.
import { api } from "./api";
import type { Address, AttachmentOut, MessageEnvelope } from "./types";
function esc(s: string): string {
return s
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
function fmtAddr(a: Address): string {
if (!a.email) return a.name || "";
return a.name ? `${a.name} <${a.email}>` : a.email;
}
function fmtList(addrs: Address[]): string {
return addrs.map(fmtAddr).filter(Boolean).join(", ");
}
function fmtDate(iso: string): string {
const d = new Date(iso);
return d.toLocaleString(undefined, {
weekday: "short",
year: "numeric",
month: "short",
day: "numeric",
hour: "numeric",
minute: "2-digit",
});
}
function fmtBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`;
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
}
// Fetch a message's sanitized HTML (a full document) and return just its <body>.
async function bodyHtml(messageId: string): Promise<string> {
try {
const resp = await fetch(api.messageHtmlUrl(messageId), { credentials: "include" });
if (!resp.ok) throw new Error(String(resp.status));
const doc = new DOMParser().parseFromString(await resp.text(), "text/html");
return doc.body?.innerHTML ?? "";
} catch {
return `<p style="color:#999">[message content unavailable]</p>`;
}
}
async function attachmentsFor(msg: MessageEnvelope): Promise<AttachmentOut[]> {
if (!msg.has_attachments) return [];
try {
return await api.messageAttachments(msg.id);
} catch {
return [];
}
}
function metaRow(label: string, value: string): string {
return value
? `<tr><th>${esc(label)}</th><td>${esc(value)}</td></tr>`
: "";
}
function section(msg: MessageEnvelope, subject: string, body: string, atts: AttachmentOut[]): string {
const attachLine = atts.length
? `<tr><th>Attachments</th><td>${atts
.map((a) => `${esc(a.filename)} (${fmtBytes(a.size_bytes)})`)
.join(", ")}</td></tr>`
: "";
return `<section class="wmc-print-msg">
<div class="wmc-print-head">
<div class="wmc-print-subject">${esc(subject || "(no subject)")}</div>
<table class="wmc-print-meta">
${metaRow("From", fmtList(msg.from_addr))}
${metaRow("To", fmtList(msg.to_addr))}
${metaRow("Cc", fmtList(msg.cc_addr))}
${metaRow("Date", fmtDate(msg.date))}
${attachLine}
</table>
</div>
<div class="wmc-print-body">${body}</div>
</section>`;
}
const MARGIN = "16mm";
const PRINT_CSS = `
/* Let the sheet be the full page; the margin is content padding so it always
prints, regardless of the browser's "Margins" print-dialog setting. */
@page { margin: 0; }
* { box-sizing: border-box; }
html, body { margin: 0; padding: 0; }
body {
font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
color: #111; background: #e9e9ec; font-size: 12pt; line-height: 1.5;
-webkit-print-color-adjust: exact; print-color-adjust: exact;
}
/* On screen, present it like a padded sheet of paper. */
@media screen {
body {
max-width: 210mm; margin: 24px auto; padding: ${MARGIN};
background: #fff; box-shadow: 0 1px 8px rgba(0,0,0,.15);
}
}
/* In print the padding IS the page margin (survives a "None" margin setting). */
@media print {
html, body { background: #fff; }
body { padding: ${MARGIN}; }
}
.wmc-print-msg { padding-bottom: 18px; }
.wmc-print-msg + .wmc-print-msg { border-top: 2px solid #ddd; padding-top: 18px; }
.wmc-print-head { break-inside: avoid; margin-bottom: 12px; }
.wmc-print-subject { font-size: 16pt; font-weight: 600; margin: 0 0 8px; }
.wmc-print-meta { width: 100%; border-collapse: collapse; font-size: 10.5pt; }
.wmc-print-meta th {
text-align: left; vertical-align: top; color: #666; font-weight: 500;
width: 72px; padding: 1px 8px 1px 0; white-space: nowrap;
}
.wmc-print-meta td { vertical-align: top; padding: 1px 0; color: #111; }
.wmc-print-body { color: #1a1a1a; word-break: break-word; }
.wmc-print-body img { max-width: 100%; height: auto; }
.wmc-print-body table { max-width: 100%; border-collapse: collapse; }
.wmc-print-body blockquote {
margin: 0 0 0 12px; padding-left: 12px; border-left: 3px solid #e0e0e0; color: #555;
}
.wmc-print-body a { color: #2563eb; }
.wmc-print-body pre { white-space: pre-wrap; word-wrap: break-word; }
`;
// Wait for images to finish (so they're in the print), capped so a slow/broken
// image can never block printing indefinitely.
async function waitForImages(win: Window, timeoutMs = 3000): Promise<void> {
const pending = Array.from(win.document.images).filter((img) => !img.complete);
if (pending.length === 0) return;
await Promise.race([
Promise.all(
pending.map(
(img) =>
new Promise<void>((res) => {
img.addEventListener("load", () => res(), { once: true });
img.addEventListener("error", () => res(), { once: true });
}),
),
),
new Promise<void>((res) => setTimeout(res, timeoutMs)),
]);
}
/**
* Open a print-ready window for one or more messages and invoke the print dialog.
* Pass a single message to print just that email, or all thread messages (ascending)
* to print the conversation.
*/
export async function printMessages(subject: string, messages: MessageEnvelope[]): Promise<void> {
if (messages.length === 0) return;
// Open the window synchronously (inside the click) so it isn't blocked as a popup.
const win = window.open("", "_blank", "width=820,height=1000");
if (!win) {
window.alert("Please allow pop-ups for this site to print.");
return;
}
const title = esc(subject || "(no subject)");
win.document.write(
`<!doctype html><meta charset="utf-8"><title>${title}</title>` +
`<p style="font:12pt system-ui;color:#888;padding:16px">Preparing…</p>`,
);
win.document.close();
const [bodies, atts] = await Promise.all([
Promise.all(messages.map((m) => bodyHtml(m.id))),
Promise.all(messages.map((m) => attachmentsFor(m))),
]);
if (win.closed) return; // the user dismissed the window while it was preparing
const sections = messages
.map((m, i) => section(m, m.subject || subject, bodies[i], atts[i]))
.join("");
// Write the whole document in one pass so the print stylesheet parses cleanly.
win.document.open();
win.document.write(
`<!doctype html><html><head><meta charset="utf-8">` +
`<base href="${window.location.origin}/">` +
`<title>${title}</title><style>${PRINT_CSS}</style></head>` +
`<body>${sections}</body></html>`,
);
win.document.close();
await waitForImages(win);
if (win.closed) return;
win.focus();
win.onafterprint = () => win.close();
win.print();
}
+107 -29
View File
@@ -26,6 +26,9 @@ function quotaLabel(q: QuotaOut): string {
// Threads shown per page in the mail list. // Threads shown per page in the mail list.
const PAGE_SIZE = 50; const PAGE_SIZE = 50;
// Collapsed sidebar width: just wide enough for the icons.
const COLLAPSED_W = 60;
// Unified per-folder views. "Spam" maps to the IMAP special-use "junk". // Unified per-folder views. "Spam" maps to the IMAP special-use "junk".
const FOLDERS: { view: string; icon: string; label: MessageKey }[] = [ const FOLDERS: { view: string; icon: string; label: MessageKey }[] = [
{ view: "folder:inbox", icon: "inbox", label: "nav.inbox" }, { view: "folder:inbox", icon: "inbox", label: "nav.inbox" },
@@ -52,17 +55,23 @@ export function MailLayout() {
const [refreshing, setRefreshing] = useState(false); const [refreshing, setRefreshing] = useState(false);
const [composing, setComposing] = useState<ComposerInit | null>(null); const [composing, setComposing] = useState<ComposerInit | null>(null);
const [composerExpanded, setComposerExpanded] = useState(false); const [composerExpanded, setComposerExpanded] = useState(false);
// Bumped each time a composer is opened so the <Composer> remounts (fresh state)
// when switching between drafts instead of keeping the previous draft's editor.
const [composeKey, setComposeKey] = useState(0);
const [replySignal, setReplySignal] = useState<{ kind: string; seq: number }>(); const [replySignal, setReplySignal] = useState<{ kind: string; seq: number }>();
const searchRef = useRef<HTMLInputElement>(null); const searchRef = useRef<HTMLInputElement>(null);
// Resizable panel widths (persisted). Drag the dividers to adjust. // Resizable panel widths (persisted). Drag the dividers to adjust.
const [sidebarW, setSidebarW] = useState(() => Number(localStorage.getItem("panel.sidebar")) || 240); const [sidebarW, setSidebarW] = useState(() => Number(localStorage.getItem("panel.sidebar")) || 240);
const [listW, setListW] = useState(() => Number(localStorage.getItem("panel.list")) || 384); const [listW, setListW] = useState(() => Number(localStorage.getItem("panel.list")) || 384);
// Collapsed sidebar shows icons only (persisted).
const [collapsed, setCollapsed] = useState(() => localStorage.getItem("panel.collapsed") === "1");
const clampSidebar = (w: number) => Math.min(Math.max(w, 180), 420); const clampSidebar = (w: number) => Math.min(Math.max(w, 180), 420);
// The list can be dragged up to 75% of the window width. // The list can be dragged up to 75% of the window width.
const clampList = (w: number) => Math.min(Math.max(w, 260), window.innerWidth * 0.75); const clampList = (w: number) => Math.min(Math.max(w, 260), window.innerWidth * 0.75);
useEffect(() => localStorage.setItem("panel.sidebar", String(sidebarW)), [sidebarW]); useEffect(() => localStorage.setItem("panel.sidebar", String(sidebarW)), [sidebarW]);
useEffect(() => localStorage.setItem("panel.list", String(listW)), [listW]); useEffect(() => localStorage.setItem("panel.list", String(listW)), [listW]);
useEffect(() => localStorage.setItem("panel.collapsed", collapsed ? "1" : "0"), [collapsed]);
useEffect(() => localStorage.setItem("mail.listOrder", listOrder), [listOrder]); useEffect(() => localStorage.setItem("mail.listOrder", listOrder), [listOrder]);
const isDrafts = view === "folder:drafts"; const isDrafts = view === "folder:drafts";
@@ -117,8 +126,8 @@ export function MailLayout() {
threadsRef.current = threads; threadsRef.current = threads;
const detailQuery = useQuery({ const detailQuery = useQuery({
queryKey: ["thread", threadId, threadOrder], queryKey: ["thread", threadId, threadOrder, view],
queryFn: () => api.thread(threadId!, threadOrder), queryFn: () => api.thread(threadId!, threadOrder, view),
enabled: Boolean(threadId), enabled: Boolean(threadId),
}); });
@@ -132,6 +141,33 @@ export function MailLayout() {
// eslint-disable-next-line react-hooks/exhaustive-deps // eslint-disable-next-line react-hooks/exhaustive-deps
}, [detailQuery.data?.id]); }, [detailQuery.data?.id]);
// Open the composer with fresh state, remounting it if one is already open
// (e.g. clicking a different draft replaces the current editor).
const openComposer = (init: ComposerInit, expand = false) => {
setComposing(init);
setComposerExpanded(expand);
setComposeKey((k) => k + 1);
};
const closeComposer = () => {
setComposing(null);
setComposerExpanded(false);
queryClient.invalidateQueries({ queryKey: ["drafts"] });
queryClient.invalidateQueries({ queryKey: ["drafts-remote"] });
};
// Leaving a folder closes any open composer/draft editor: the reading pane
// should reflect the folder you navigated to, not a leftover draft editor.
const firstView = useRef(true);
useEffect(() => {
if (firstView.current) {
firstView.current = false;
return;
}
closeComposer();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [view]);
const afterMove = () => { const afterMove = () => {
queryClient.invalidateQueries({ queryKey: ["threads"] }); queryClient.invalidateQueries({ queryKey: ["threads"] });
queryClient.invalidateQueries({ queryKey: ["quota"] }); queryClient.invalidateQueries({ queryKey: ["quota"] });
@@ -201,7 +237,7 @@ export function MailLayout() {
useKeyboardShortcuts({ useKeyboardShortcuts({
onSearch: () => searchRef.current?.focus(), onSearch: () => searchRef.current?.focus(),
onCompose: () => accountsQuery.data?.[0] && setComposing({ accountId: accountsQuery.data[0].id, kind: "new" }), onCompose: () => accountsQuery.data?.[0] && openComposer({ accountId: accountsQuery.data[0].id, kind: "new" }),
onReply: () => setReplySignal({ kind: "reply", seq: Date.now() }), onReply: () => setReplySignal({ kind: "reply", seq: Date.now() }),
onReplyAll: () => setReplySignal({ kind: "reply_all", seq: Date.now() }), onReplyAll: () => setReplySignal({ kind: "reply_all", seq: Date.now() }),
onForward: () => setReplySignal({ kind: "forward", seq: Date.now() }), onForward: () => setReplySignal({ kind: "forward", seq: Date.now() }),
@@ -221,22 +257,39 @@ export function MailLayout() {
<div className="flex h-screen bg-app-bg text-app-fg"> <div className="flex h-screen bg-app-bg text-app-fg">
{/* Sidebar */} {/* Sidebar */}
<aside <aside
style={{ width: sidebarW }} style={{ width: collapsed ? COLLAPSED_W : sidebarW }}
className="flex shrink-0 flex-col border-r border-app-border bg-app-bg" className="flex shrink-0 flex-col border-r border-app-border bg-app-bg"
> >
<div className="flex items-center justify-between px-3 pb-1 pt-3"> <div
className={`flex pb-1 pt-3 ${
collapsed ? "flex-col items-center gap-1 px-2" : "items-center justify-between px-3"
}`}
>
{!collapsed && (
<span className="text-xs font-semibold uppercase tracking-wide text-app-faint"> <span className="text-xs font-semibold uppercase tracking-wide text-app-faint">
{t("common.appName")} {t("common.appName")}
</span> </span>
)}
<div className={`flex items-center gap-1 ${collapsed ? "flex-col" : ""}`}>
<button
onClick={() => setCollapsed((v) => !v)}
title={collapsed ? t("nav.expandSidebar") : t("nav.collapseSidebar")}
aria-label={collapsed ? t("nav.expandSidebar") : t("nav.collapseSidebar")}
className="rounded p-1.5 text-app-muted hover:bg-app-hover hover:text-app-fg"
>
<Icon name={collapsed ? "show" : "hide"} size={16} />
</button>
<ThemeToggle /> <ThemeToggle />
</div> </div>
<div className="px-3 pb-2"> </div>
<div className={`pb-2 ${collapsed ? "px-2" : "px-3"}`}>
<button <button
className="flex w-full items-center justify-center gap-2 rounded-lg bg-app-accent py-2 text-sm font-medium text-app-accent-fg shadow-sm hover:opacity-90" className="flex w-full items-center justify-center gap-2 rounded-lg bg-app-accent py-2 text-sm font-medium text-app-accent-fg shadow-sm hover:opacity-90"
onClick={() => accounts[0] && setComposing({ accountId: accounts[0].id, kind: "new" })} title={collapsed ? t("nav.compose") : undefined}
onClick={() => accounts[0] && openComposer({ accountId: accounts[0].id, kind: "new" })}
> >
<Icon name="compose" size={16} /> <Icon name="compose" size={16} />
{t("nav.compose")} {!collapsed && t("nav.compose")}
</button> </button>
</div> </div>
<nav className="flex-1 space-y-0.5 overflow-y-auto px-2 py-1 text-sm"> <nav className="flex-1 space-y-0.5 overflow-y-auto px-2 py-1 text-sm">
@@ -244,15 +297,20 @@ export function MailLayout() {
<SidebarItem <SidebarItem
key={f.view} key={f.view}
active={view === f.view} active={view === f.view}
collapsed={collapsed}
title={collapsed ? t(f.label) : undefined}
onClick={() => { onClick={() => {
setView(f.view); setView(f.view);
navigate("/mail"); navigate("/mail");
}} }}
> >
<Icon name={f.icon} size={18} /> <Icon name={f.icon} size={18} />
{t(f.label)} {!collapsed && t(f.label)}
</SidebarItem> </SidebarItem>
))} ))}
{/* Per-account views are hidden entirely when collapsed. */}
{!collapsed && (
<>
<div className="px-3 pb-1 pt-3 text-[11px] font-semibold uppercase tracking-wide text-app-faint"> <div className="px-3 pb-1 pt-3 text-[11px] font-semibold uppercase tracking-wide text-app-faint">
{t("settings.accounts")} {t("settings.accounts")}
</div> </div>
@@ -266,22 +324,35 @@ export function MailLayout() {
<span className="truncate">{a.email}</span> <span className="truncate">{a.email}</span>
</SidebarItem> </SidebarItem>
))} ))}
</>
)}
</nav> </nav>
<div className="border-t border-app-border p-2 text-sm"> <div className="border-t border-app-border p-2 text-sm">
<Link className="block rounded-lg px-3 py-2 text-app-muted hover:bg-app-hover hover:text-app-fg" to="/settings"> <Link
⚙ {t("nav.settings")} className={`flex items-center gap-2.5 rounded-lg py-2 text-app-muted hover:bg-app-hover hover:text-app-fg ${
collapsed ? "justify-center px-0" : "px-3"
}`}
title={collapsed ? t("nav.settings") : undefined}
to="/settings"
>
<Icon name="settings" size={16} />
{!collapsed && t("nav.settings")}
</Link> </Link>
<button <button
className="block w-full rounded-lg px-3 py-2 text-left text-app-muted hover:bg-app-hover hover:text-app-fg" className={`flex w-full items-center gap-2.5 rounded-lg py-2 text-left text-app-muted hover:bg-app-hover hover:text-app-fg ${
collapsed ? "justify-center px-0" : "px-3"
}`}
title={collapsed ? t("common.signOut") : undefined}
onClick={async () => { onClick={async () => {
await api.logout(); await api.logout();
queryClient.clear(); queryClient.clear();
navigate("/login"); navigate("/login");
}} }}
> >
{t("common.signOut")} <Icon name="log-out" size={16} />
{!collapsed && t("common.signOut")}
</button> </button>
{quotaQuery.data && ( {!collapsed && quotaQuery.data && (
<div <div
className="mt-1 border-t border-app-border px-3 pb-1 pt-2 text-xs text-app-faint" className="mt-1 border-t border-app-border px-3 pb-1 pt-2 text-xs text-app-faint"
title={t("nav.storageQuota")} title={t("nav.storageQuota")}
@@ -292,7 +363,7 @@ export function MailLayout() {
</div> </div>
</aside> </aside>
<Resizer onDelta={(dx) => setSidebarW((w) => clampSidebar(w + dx))} /> {!collapsed && <Resizer onDelta={(dx) => setSidebarW((w) => clampSidebar(w + dx))} />}
{/* Message list */} {/* Message list */}
<div <div
@@ -334,17 +405,13 @@ export function MailLayout() {
<DraftsList <DraftsList
drafts={draftsQuery.data ?? []} drafts={draftsQuery.data ?? []}
remoteDrafts={remoteDraftsQuery.data ?? []} remoteDrafts={remoteDraftsQuery.data ?? []}
onOpen={(draft) => { onOpen={(draft) => openComposer(draftToInit(draft), true)}
setComposing(draftToInit(draft));
setComposerExpanded(true);
}}
onOpenRemote={async (remote) => { onOpenRemote={async (remote) => {
// Adopt the server draft into an editable local draft, then open it. // Adopt the server draft into an editable local draft, then open it.
const draft = await api.adoptDraft(remote.message_id); const draft = await api.adoptDraft(remote.message_id);
queryClient.invalidateQueries({ queryKey: ["drafts"] }); queryClient.invalidateQueries({ queryKey: ["drafts"] });
queryClient.invalidateQueries({ queryKey: ["drafts-remote"] }); queryClient.invalidateQueries({ queryKey: ["drafts-remote"] });
setComposing(draftToInit(draft)); openComposer(draftToInit(draft), true);
setComposerExpanded(true);
}} }}
/> />
) : ( ) : (
@@ -466,15 +533,11 @@ export function MailLayout() {
} }
> >
<Composer <Composer
key={composeKey}
init={composing} init={composing}
expanded={composerExpanded} expanded={composerExpanded}
onToggleExpand={() => setComposerExpanded((v) => !v)} onToggleExpand={() => setComposerExpanded((v) => !v)}
onClose={() => { onClose={closeComposer}
setComposing(null);
setComposerExpanded(false);
queryClient.invalidateQueries({ queryKey: ["drafts"] });
queryClient.invalidateQueries({ queryKey: ["drafts-remote"] });
}}
/> />
</div> </div>
)} )}
@@ -570,11 +633,26 @@ function SortIcon() {
); );
} }
function SidebarItem({ active, onClick, children }: { active: boolean; onClick: () => void; children: React.ReactNode }) { function SidebarItem({
active,
onClick,
collapsed = false,
title,
children,
}: {
active: boolean;
onClick: () => void;
collapsed?: boolean;
title?: string;
children: React.ReactNode;
}) {
return ( return (
<button <button
onClick={onClick} onClick={onClick}
className={`flex w-full items-center gap-2.5 rounded-lg px-3 py-1.5 text-left ${ title={title}
className={`flex w-full items-center gap-2.5 rounded-lg py-1.5 text-left ${
collapsed ? "justify-center px-0" : "px-3"
} ${
active ? "bg-app-active font-medium text-app-fg" : "text-app-muted hover:bg-app-hover hover:text-app-fg" active ? "bg-app-active font-medium text-app-fg" : "text-app-muted hover:bg-app-hover hover:text-app-fg"
}`} }`}
> >
Binary file not shown.

Before

Width:  |  Height:  |  Size: 209 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 169 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 82 KiB

+811
View File
@@ -0,0 +1,811 @@
# Webmail Client — Implementation Specification (v1.4, agent-ready)
This document is a complete, self-contained build specification. It is written to be handed to an AI coding agent. Read it fully before writing code. The companion document `webmail-client-concept.md` gives background; where they differ, **this spec wins**.
---
## 0. Instructions to the implementing agent
1. **Build milestone by milestone** (§12), in order. Do not start a milestone before the previous one's acceptance criteria pass.
2. **Do not add features beyond this spec.** Anything not specified here is out of scope; leave `TODO(spec)` comments instead of inventing behavior.
3. **Definition of done** for every task: code + type hints + unit tests + passing `ruff check`, `ruff format --check`, `mypy --strict` (backend) / `tsc --noEmit`, `eslint` (frontend) + updated docs in `docs/` if an interface changed.
4. **Stability over novelty:** use exactly the pinned dependencies in §3. Do not swap libraries or add dependencies without a `# DEPENDENCY:` comment explaining why, and never for something the stdlib does adequately.
5. **All interfaces in §5 are frozen contracts.** Implementations may evolve; signatures may not (without bumping the version comment at the top of the interface file).
6. **Security rules in §10 are hard requirements**, not suggestions.
7. Every module publishing or consuming events must use the event names in §8 verbatim.
8. Commit in small, working increments; every commit must leave the test suite green.
## 1. Architecture invariants (never violate)
- **I-1 Sync-first:** the web UI and REST API never open IMAP connections. Only sync workers talk to mail servers. The API reads exclusively from Postgres/Meilisearch.
- **I-2 Independent identity:** login uses app accounts (`localpart@our-web-client.example`) with Argon2id hashes. Mailbox credentials are payload data, encrypted at rest, decrypted only inside sync workers.
- **I-3 One index, many views:** unified inbox, per-account views, virtual mailboxes, and search results are all parameterized queries over the same index. No duplicated storage per view.
- **I-4 Threading is ours:** JWZ in our backend (§7). Never use the IMAP `THREAD` extension.
- **I-5 Direction-agnostic thread UI:** thread ordering (oldest-first/newest-first) is a data-ordering concern. One layout, one composer-anchor slot (§9.4).
- **I-6 Plugins extend, core stays closed:** new features beyond this spec must be expressible through the plugin surfaces (§11).
- **I-7 Multi-user isolation:** every query (SQL and search) carries a mandatory `user_id` filter. No exceptions, including admin tooling.
- **I-8 Mail HTML is hostile input:** sanitize with nh3 server-side at index time AND render in a sandboxed iframe. Both, always.
## 2. Repository layout (monorepo)
```
webmail/
├── backend/
│ ├── pyproject.toml
│ ├── app/
│ │ ├── main.py # FastAPI app factory
│ │ ├── config.py # pydantic-settings; all config via env
│ │ ├── db.py # SQLAlchemy async engine/session
│ │ ├── events.py # EventBus (Redis pub/sub + arq enqueue)
│ │ ├── identity/ # §6.1 auth, sessions, handles, TOTP
│ │ ├── accounts/ # §6.2 wizard, autodiscovery, credential vault
│ │ ├── providers/
│ │ │ ├── base.py # MailProvider interface (§5.1)
│ │ │ └── imap_smtp/ # first implementation
│ │ ├── sync/ # §6.3 backfill, IDLE loops, reconciliation
│ │ ├── indexer/ # §6.4 MIME parsing → DB rows + blobs
│ │ ├── threading_/ # §7 JWZ (trailing _ avoids stdlib clash)
│ │ ├── search/
│ │ │ ├── base.py # SearchBackend interface (§5.2)
│ │ │ ├── queryparser.py # operator grammar (§8.2)
│ │ │ ├── pg.py # Postgres FTS backend (M0–M1)
│ │ │ └── meili.py # Meilisearch backend (M2)
│ │ ├── outbox/ # §6.5 drafts, MIME assembly, send queue, undo
│ │ ├── api/ # routers, one file per §9.1 group
│ │ ├── ws.py # WebSocket hub (§9.2)
│ │ └── plugins/ # §11 loader + plugin packages
│ ├── workers/worker.py # arq worker entrypoint
│ ├── migrations/ # alembic
│ └── tests/
├── frontend/
│ ├── package.json
│ └── src/
│ ├── api/ # generated client from OpenAPI + WS layer
│ ├── routes/ # §9.3
│ ├── components/
│ │ ├── thread/ThreadView.tsx # §9.4 direction-agnostic
│ │ ├── thread/Composer.tsx
│ │ ├── list/MessageList.tsx # virtualized
│ │ └── slots/ # plugin UI slots (§11)
│ └── lib/
├── docker/
│ ├── docker-compose.yml # app, worker, postgres:16, redis:7, meilisearch:v1.8, dovecot (test)
│ └── docker-compose.ci.yml # + greenmail for integration tests
└── docs/
```
## 3. Pinned stack
**Backend (Python 3.11):** fastapi ~0.115, uvicorn ~0.30, sqlalchemy ~2.0 (async, asyncpg), alembic ~1.13, pydantic ~2.8, pydantic-settings, arq ~0.26, redis ~5.0, aioimaplib ~1.1, imapclient ~3.0 (utility/fallback), aiosmtplib ~3.0, argon2-cffi ~23.1, pyotp ~2.9, dnspython ~2.6, httpx ~0.27, nh3 ~0.2, cryptography ~43.0, mail-parser ~4.0 (helpers only; primary parsing via stdlib `email.message.EmailMessage` with `policy.default`).
**Frontend:** TypeScript ~5.5, React ~18.3, Vite ~5, TanStack Query ~5, TanStack Virtual ~3, Radix UI, Tailwind ~3.4, zod (WS payload validation). API client generated from OpenAPI (`openapi-typescript`).
**Infra:** PostgreSQL 16 (with `citext` extension), Redis 7, Meilisearch v1.8 (M2+), MinIO or plain filesystem for the blob store (config-switchable; filesystem default).
## 4. Database schema (Postgres 16, authoritative DDL)
All PKs are `uuid DEFAULT gen_random_uuid()`. All tables carrying user data include `user_id` with an index. Timestamps are `timestamptz`.
```sql
CREATE EXTENSION IF NOT EXISTS citext;
CREATE TABLE users (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
handle citext UNIQUE NOT NULL, -- full form: localpart@<APP_DOMAIN>
password_hash text NOT NULL, -- argon2id
totp_secret bytea, -- encrypted; NULL = 2FA off
recovery_email citext,
settings jsonb NOT NULL DEFAULT '{}', -- incl. {"thread_order":"asc"|"desc"}
created_at timestamptz NOT NULL DEFAULT now()
);
-- Handle rules (enforce in code, mail-grade from day one):
-- localpart: 3–30 chars, [a-z0-9._-], no leading/trailing/double dot,
-- case-insensitive, reserved list (postmaster, abuse, admin, root, noreply, ...).
CREATE TABLE accounts (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
email citext NOT NULL,
display_name text NOT NULL DEFAULT '',
color text NOT NULL DEFAULT '#888888',
imap_config jsonb NOT NULL, -- {host,port,tls:"ssl"|"starttls",auth:"password"|"oauth2"}
smtp_config jsonb NOT NULL, -- same shape
secret_ct bytea NOT NULL, -- envelope-encrypted password or OAuth token bundle (§10.2)
discovery_src text NOT NULL, -- preset|autoconfig|ispdb|srv|probe|manual
state text NOT NULL DEFAULT 'backfilling', -- backfilling|live|error|disabled
state_detail text NOT NULL DEFAULT '',
last_sync_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (user_id, email)
);
CREATE TABLE mailboxes (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
account_id uuid NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
path text NOT NULL, -- IMAP mailbox name, UTF-7 decoded
special_use text, -- inbox|sent|drafts|trash|junk|archive|NULL
uidvalidity bigint NOT NULL DEFAULT 0,
highestmodseq bigint, -- NULL if no CONDSTORE
last_seen_uid bigint NOT NULL DEFAULT 0,
UNIQUE (account_id, path)
);
CREATE TABLE threads (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
subject_norm text NOT NULL, -- normalized subject (§7.3)
snippet text NOT NULL DEFAULT '',
participants jsonb NOT NULL DEFAULT '[]', -- [{name,email}], deduped, ordered by recency
account_ids uuid[] NOT NULL DEFAULT '{}',
message_count int NOT NULL DEFAULT 0,
unread_count int NOT NULL DEFAULT 0,
has_attachments boolean NOT NULL DEFAULT false,
last_message_at timestamptz NOT NULL
);
CREATE INDEX ON threads (user_id, last_message_at DESC);
CREATE TABLE messages (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
account_id uuid NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
mailbox_id uuid NOT NULL REFERENCES mailboxes(id) ON DELETE CASCADE,
uid bigint NOT NULL, -- IMAP UID within mailbox
thread_id uuid REFERENCES threads(id),
message_id_hdr text, -- Message-ID, angle brackets stripped
in_reply_to text,
references_hdr text[] NOT NULL DEFAULT '{}',
subject text NOT NULL DEFAULT '',
from_addr jsonb NOT NULL DEFAULT '[]',
to_addr jsonb NOT NULL DEFAULT '[]',
cc_addr jsonb NOT NULL DEFAULT '[]',
date_hdr timestamptz,
internal_date timestamptz NOT NULL,
flags text[] NOT NULL DEFAULT '{}', -- \Seen,\Flagged,\Answered,\Draft,\Deleted
size_bytes int NOT NULL DEFAULT 0,
has_attachments boolean NOT NULL DEFAULT false,
snippet text NOT NULL DEFAULT '', -- first ~160 chars of text body
blob_sha256 text NOT NULL, -- raw RFC822 in blob store (§6.4)
body_indexed tsvector, -- Postgres FTS (M0–M1 SearchBackend)
UNIQUE (mailbox_id, uid)
);
CREATE INDEX ON messages (user_id, thread_id);
CREATE INDEX ON messages (account_id, message_id_hdr);
CREATE INDEX ON messages USING gin (body_indexed);
CREATE TABLE body_parts (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
kind text NOT NULL, -- text|html_sanitized
content text NOT NULL
);
CREATE TABLE attachments (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
user_id uuid NOT NULL,
filename text NOT NULL,
mime_type text NOT NULL,
size_bytes int NOT NULL,
content_id text, -- for inline cid: images
blob_sha256 text NOT NULL
);
CREATE TABLE drafts (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
account_id uuid NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
thread_id uuid REFERENCES threads(id), -- NULL = new mail
reply_to_message_id uuid REFERENCES messages(id),
kind text NOT NULL DEFAULT 'reply', -- reply|reply_all|forward|new
payload jsonb NOT NULL, -- {to,cc,bcc,subject,html,text,attachment_ids}
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE outbox (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
draft_id uuid NOT NULL,
state text NOT NULL DEFAULT 'undo_window', -- undo_window|sending|sent|failed|cancelled
send_after timestamptz NOT NULL, -- now() + undo window
error text
);
CREATE TABLE saved_searches (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name text NOT NULL,
query text NOT NULL, -- raw operator-syntax string (§8.2)
pinned boolean NOT NULL DEFAULT true,
UNIQUE (user_id, name)
);
-- Core recipient autocomplete (Gmail-style, derived from mail history).
-- The Contacts plugin (Appendix B) ENRICHES suggestions; this table works without it.
CREATE TABLE known_recipients (
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
email citext NOT NULL,
name text NOT NULL DEFAULT '', -- last seen display name
times_used int NOT NULL DEFAULT 0, -- weighted: sent-to counts 3, received-from counts 1
last_used_at timestamptz NOT NULL,
PRIMARY KEY (user_id, email)
);
CREATE INDEX ON known_recipients (user_id, email text_pattern_ops);
```
**Blob store:** raw RFC822 messages and attachment payloads are stored content-addressed at `${BLOB_ROOT}/{user_id}/{sha256[:2]}/{sha256}` (filesystem) or the equivalent object key (MinIO). Postgres holds metadata only. Deleting a user removes their blob subtree.
**Sessions** live in Redis (`sess:{token}` → JSON `{user_id, created, ua}`, TTL 30 days sliding), not Postgres.
## 5. Frozen core interfaces (Python)
### 5.1 `providers/base.py` — MailProvider
```python
# INTERFACE VERSION: 1
from dataclasses import dataclass
from typing import AsyncIterator, Protocol
from datetime import datetime
@dataclass(frozen=True)
class RemoteMailbox:
path: str
special_use: str | None # "inbox"|"sent"|"drafts"|"trash"|"junk"|"archive"|None
uidvalidity: int
@dataclass(frozen=True)
class RemoteMessage:
uid: int
raw: bytes # full RFC822
flags: frozenset[str]
internal_date: datetime
@dataclass(frozen=True)
class FlagChange:
uid: int
flags: frozenset[str]
@dataclass(frozen=True)
class MailboxDelta:
new: list[RemoteMessage]
flag_changes: list[FlagChange]
expunged_uids: list[int]
new_uidvalidity: int | None # non-None => UIDVALIDITY changed, caller must full-resync
new_highestmodseq: int | None
class MailProvider(Protocol):
async def verify(self) -> None: ...
# Raises ProviderAuthError / ProviderConnectError with a human-readable .detail
async def list_mailboxes(self) -> list[RemoteMailbox]: ...
def backfill(self, mailbox: str, *, newest_first: bool = True,
batch_size: int = 200) -> AsyncIterator[list[RemoteMessage]]: ...
# Yields batches until the mailbox is exhausted. Headers+body in one pass.
async def poll(self, mailbox: str, *, last_seen_uid: int,
highestmodseq: int | None) -> MailboxDelta: ...
# One reconciliation pass. Uses QRESYNC/CONDSTORE when highestmodseq is not None,
# otherwise UID SEARCH + FETCH FLAGS diff.
async def idle(self, mailbox: str) -> None: ...
# Blocks until the server signals change or 25 min elapse (re-issue IDLE). May raise
# ProviderNotSupported => sync loop falls back to poll() every POLL_INTERVAL (default 120 s).
async def push_flags(self, mailbox: str, uid: int,
add: frozenset[str], remove: frozenset[str]) -> None: ...
async def move(self, uid: int, src: str, dst: str) -> int: ... # returns new UID
async def append(self, mailbox: str, raw: bytes,
flags: frozenset[str]) -> int: ... # Sent/Drafts upload
async def send(self, raw: bytes, envelope_from: str,
envelope_to: list[str]) -> None: ... # SMTP submission
```
`ImapSmtpProvider` is the only M0–M2 implementation. A future `NativeProvider` (hosted mailboxes) or `JmapProvider` implements the same protocol — nothing above this interface may import IMAP libraries.
### 5.2 `search/base.py` — SearchBackend
```python
# INTERFACE VERSION: 1
@dataclass(frozen=True)
class SearchQuery:
text: str | None # free-text remainder after operator extraction
filters: dict[str, object] # parsed operators, keys per §8.2
user_id: UUID # ALWAYS set (invariant I-7)
@dataclass(frozen=True)
class SearchHit:
message_id: UUID
thread_id: UUID
score: float
highlights: dict[str, str] # field -> HTML-escaped snippet with <mark>
class SearchBackend(Protocol):
async def index(self, doc: SearchDoc) -> None: ...
async def delete(self, message_id: UUID) -> None: ...
async def delete_user(self, user_id: UUID) -> None: ...
async def query(self, q: SearchQuery, *, limit: int, cursor: str | None
) -> tuple[list[SearchHit], str | None]: ...
```
`SearchDoc` fields: `message_id, thread_id, user_id, account_id, mailbox_special_use, from_ (name+email), to, cc, subject, body_text, attachment_names[], has_attachments, is_unread, is_starred, date (unix), size_bytes`. Both backends (pg.py, meili.py) index the same doc. Meilisearch: one index `messages`, filterable = every non-body field incl. `user_id`, searchable = `subject, body_text, from_, to, cc, attachment_names`, sortable = `date`; ranking rules default + `date:desc` tiebreak.
### 5.3 `events.py` — EventBus
```python
# INTERFACE VERSION: 1
class EventBus(Protocol):
async def publish(self, event: str, payload: dict) -> None:
# Fan-out to Redis pub/sub channel "evt:{event}" AND enqueue arq job
# "on_{event}" for durable consumers (indexer, plugins).
def subscribe(self, event: str, handler: Handler) -> None: ...
```
Payloads always include `user_id`. Event names are listed in §8.3 and are the only ones allowed.
## 6. Backend module specifications
### 6.1 Identity (`app/identity/`)
- Registration: validate localpart per §4 rules against reserved list in `identity/reserved.py`; store full handle `localpart@{APP_DOMAIN}` (env `APP_DOMAIN`).
- Login: constant-time verify (argon2id, `time_cost=3, memory_cost=64MiB, parallelism=4`); if `totp_secret` set, require valid TOTP (±1 window). Rate limit: 10 attempts / 15 min per handle AND per IP (Redis counters); lockout returns 429 with `Retry-After`.
- Session cookie: `__Host-session`, httpOnly, Secure, SameSite=Lax. CSRF: double-submit token header `X-CSRF` for all mutating requests.
- TOTP setup: `POST /auth/totp/setup` returns provisioning URI + QR payload; enable requires one valid code; secret stored envelope-encrypted.
- Password reset: only if `recovery_email` verified; single-use token, 30 min TTL, Redis-backed.
### 6.2 Accounts & autodiscovery (`app/accounts/`)
`discover(email) -> list[CandidateConfig]` runs the chain, **returning on first success** but recording every step's outcome in the response (`steps: [{method, status, detail}]`) for wizard UX:
1. **Presets** — `accounts/presets.yaml` (config, not code): match by domain; ships with gmail.com, googlemail.com, outlook.com/hotmail/live, office365, yahoo, icloud/me.com, gmx, wp.pl, onet.pl, o2.pl, interia.pl. Each entry: imap/smtp host+port+tls, auth kind, oauth provider id if applicable.
2. **Autoconfig** — GET `https://autoconfig.{domain}/mail/config-v1.1.xml?emailaddress={email}` then `https://{domain}/.well-known/autoconfig/mail/config-v1.1.xml`; parse Thunderbird XML (incomingServer type=imap, outgoingServer type=smtp; `%EMAILADDRESS%`/`%EMAILLOCALPART%` substitution).
3. **ISPDB** — GET `https://autoconfig.thunderbird.net/v1.1/{domain}`; same XML format.
4. **DNS SRV (RFC 6186)** — `_imaps._tcp.{domain}` (SSL 993), `_imap._tcp` (STARTTLS 143), `_submissions._tcp` (465), `_submission._tcp` (587); lowest priority wins.
5. **Probe** — try `{imap,mail}.{domain}` × ports 993(ssl)/143(starttls) and `{smtp,mail}.{domain}` × 465(ssl)/587(starttls); TCP connect + TLS handshake + IMAP/SMTP greeting, 5 s timeout each, all concurrently.
All HTTP/DNS lookups: 5 s timeout, no redirects off-domain, HTTPS only. `test(config, credentials)`: full IMAP LOGIN + `LIST`, and SMTP EHLO+AUTH (no message sent); errors mapped to precise user-facing strings (`auth_failed`, `tls_failed`, `connect_timeout`, `unknown_host`). On save: encrypt credentials (§10.2), create account row (`state=backfilling`), enqueue `sync.bootstrap_account`.
### 6.3 Sync engine v2 (`app/sync/`) — sharded, lease-owned, tiered
(Authoritative detail: `sync-engine-v2-refactor.md`; this section summarizes the binding rules.)
- **Two worker roles** (`WORKER_ROLE`): `jobs` = stateless arq consumer, N instances; `sync` = lease claimer + per-account runners, 1..N instances. Compose services `worker-jobs` / `worker-sync`, `nofile` ulimit 65536.
- **Lease ownership:** exactly one sync worker owns each live account. Redis SET `sync:live` (API-maintained, DB-reconciled every 300 s); lease `sync:lease:{account_id}` claimed `SET NX EX LEASE_TTL`, renewed every `LEASE_RENEW` via owner-checked Lua; **fencing** — on renewal failure, cancel that account's runner before any further provider/DB op. Worker heartbeats in `sync:workers:{id}`; rebalance releases excess above `target*1.2+5`. Crash failover = lease expiry + reclaim (≤ ~90 s), no coordinator.
- **Event-driven lifecycle:** `account.state_changed` adds/removes ids from `sync:live` at runtime — accounts go live ≤ 30 s after creation and stop ≤ 30 s after deletion with no restarts; runners re-verify `state='live'` each cycle (zombie-proof). New core event **`user.active`** (API publishes on login/WS connect) drives tier promotion.
- **Bootstrap (unchanged from v1, now a `bulk` job):** `list_mailboxes()` → upsert `mailboxes` (SPECIAL-USE mapping with name heuristics — Sent/Wysłane, Trash/Kosz, etc. in `sync/folder_names.py`); backfill newest-first per mailbox (INBOX first, then sent, then rest); `sync.progress` published per batch; on completion set `state='live'` → the API adds the id to `sync:live` and a sync worker claims it.
- **AccountRunner per tier:** *hot* = persistent INBOX IDLE + transient CONDSTORE poll of special folders every `POLL_WARM`; *warm* = transient poll of INBOX+specials every `POLL_WARM`; *cold* = every `POLL_COLD`. Promotion to hot on `user.active` (≤ 5 s) or new mail within `TIER_HOT_ACTIVITY_H`; demotion by inactivity; `HOT_BUDGET_PER_WORKER` degrades least-active non-session accounts. Delta application, UIDVALIDITY wipe-and-rebackfill, expunge soft-handling, and write-then-enqueue outbound flags are unchanged from v1.
- **Connection budgets:** ≤ 2 sync-side connections per account (4 total incl. jobs, shared semaphore); process-local per-destination-host semaphore cap `PER_HOST_CONN_CAP`.
- **Queues:** `interactive` (send/flags/move/draft jobs, timeout 120 s, concurrency 20) and `bulk` (backfills, timeout 3600 s, concurrency `BULK_CONCURRENCY`) — bulk work can never delay a send.
- **Memory bounds:** parse semaphore `PARSE_CONCURRENCY`; raw messages > `STREAM_PARSE_MB` stream to blob and parse from file — peak RAM ≈ `PARSE_CONCURRENCY × STREAM_PARSE_MB` per process regardless of mailbox size.
- **Shutdown:** SIGTERM → stop claimer → cancel runners → release leases → drain in < 10 s.
### 6.4 Indexer (`app/indexer/`)
Input: `RemoteMessage`. Steps: store raw blob (sha256, dedupe) → parse with `email` stdlib (`policy.default`) → extract: headers (decode RFC2047), `references_hdr` = parsed `References` + `In-Reply-To` message-ids in order; best text body and best HTML body (multipart/alternative preference: html for display, text for indexing; if only HTML, derive text via nh3 strip); sanitize HTML with nh3 (allowlist: structural tags, inline styles filtered, `cid:` images rewritten to `/api/messages/{id}/cid/{content_id}`, remote `src` rewritten to blocked-placeholder attributes for FR-23) → write `messages`, `body_parts` (kinds `text` and `html_sanitized`), `attachments` → hand to Threader (§7) → `bus.publish("message.new", ...)` (search indexing + thread denormalization are consumers). For non-junk received messages, upsert the `From` address into `known_recipients` (weight 1).
### 6.5 Outbox (`app/outbox/`)
- Drafts: autosave endpoint upserts `drafts` (client debounces 2 s); a background job mirrors drafts to the account's Drafts folder via `append()` at most every 60 s per draft (replace previous UID).
- Send: `POST /send {draft_id}` → build MIME (html + auto-generated text alternative via html-to-text; `In-Reply-To`/`References` from `reply_to_message_id`; attachments from blob store) → create `outbox` row with `send_after = now()+UNDO_SECONDS` (env, default 10) → schedule arq job at `send_after`. Undo endpoint flips state to `cancelled` if still in window. The job: SMTP `send()`, then `append()` to Sent (skip for Gmail preset — Gmail self-appends), mark `sent`, publish `message.sent`, and upsert every To/Cc/Bcc address into `known_recipients` (weight 3). Failure → `failed` + WS notification, draft preserved.
## 7. Threading specification (JWZ, `app/threading_/`)
### 7.1 Normalization
- `message_id_hdr`, `in_reply_to`, each `References` entry: strip angle brackets and whitespace, lowercase the domain part only. Discard obviously invalid ids (no `@`).
- Missing/duplicate `Message-ID`: synthesize `sha256(raw)[:32]@synthetic.local` and mark it non-authoritative (never used to join other messages to it).
### 7.2 Incremental algorithm (per new message)
1. Candidate set = all normalized ids in `references_hdr` + `in_reply_to` + own `message_id_hdr`.
2. `SELECT DISTINCT thread_id FROM messages WHERE user_id=$1 AND message_id_hdr = ANY($2) AND thread_id IS NOT NULL`.
3. 0 threads → subject fallback (§7.3); still none → create new thread.
4. 1 thread → attach.
5. ≥2 threads → **merge**: keep the thread with the oldest `last_message_at` as canonical, repoint messages of the others, delete emptied thread rows, publish `thread.updated` for all involved.
6. Recompute denormalized thread fields (count, unread, participants, snippet = newest message's snippet, has_attachments, account_ids, last_message_at) in one UPDATE.
Threads are cross-account by design: the candidate lookup is scoped by `user_id`, not `account_id` (invariant I-3). Sent-folder messages join threads exactly like received ones (FR-15).
### 7.3 Subject fallback
Only when reference joining fails: normalize subject (strip repeated `Re:/Fw:/Fwd:/Odp:/PD:` prefixes case-insensitively, collapse whitespace); join to a thread with the same `subject_norm` that has activity within 7 days AND shares ≥1 participant email; never join two messages that both lack references AND have empty subjects.
### 7.4 Test vectors (must pass in `tests/threading/`)
Linear chain; chain arriving out of order; reply missing `In-Reply-To` but carrying `References`; thread merge on late-arriving parent; broken client (subject-only) joins; distinct-topic same-subject ("hi") does NOT join; cross-account merge (message in account A references message in account B); duplicate Message-ID; synthetic-ID isolation.
## 8. Search specification
### 8.1 Pipeline
Consumer of `message.new` / `message.changed` / `message.deleted` → builds `SearchDoc` → active `SearchBackend` (env `SEARCH_BACKEND=pg|meili`). Backfill indexes inline with sync (per batch), never as a separate pass the user waits on (FR-30).
### 8.2 Query grammar (`search/queryparser.py`, engine-independent)
Tokenization: whitespace-split, respecting `"quoted phrases"`. A token matching `key:value` (or `-key:value`) with a known key becomes a filter; everything else joins free text. `-` prefix negates a term or filter. `OR` (uppercase) groups adjacent terms/filters into a disjunction; default combinator is AND.
| Operator | Filter semantics |
|---|---|
| `from:` `to:` `cc:` | substring match on name or email of that field |
| `subject:` | match constrained to subject |
| `account:` | account by `display_name` or email substring |
| `in:` | `inbox\|sent\|drafts\|trash\|junk\|archive` (special_use) or folder path |
| `is:` | `unread\|read\|starred\|answered` |
| `has:attachment` | boolean |
| `filename:` | attachment name substring |
| `before:` `after:` | `YYYY-MM-DD` |
| `newer_than:` `older_than:` | `Nd\|Nw\|Nm\|Ny` |
| `larger:` `smaller:` | `NK\|NM` bytes |
Parser output = `SearchQuery` (§5.2). Unknown `key:` tokens are treated as free text (forward compatibility). Unit tests must cover every operator, negation, quoting, `OR`, and garbage input (parser never raises).
### 8.3 Event catalog (complete)
`message.new`, `message.changed`, `message.deleted`, `message.sent`, `flags.changed`, `thread.updated`, `sync.progress`, `account.state_changed`, `draft.saved`, `outbox.state_changed`, `user.active` (published by the API on login and WS connect; consumed by sync workers for tier promotion — not forwarded to browsers). Payloads: `{user_id, ...ids, minimal-diff fields}` — documented in `docs/events.md` (agent: generate it from code).
Plugins may additionally publish events namespaced `plugin.{name}.*` (e.g. `plugin.calendar.event_updated`); core code never subscribes to plugin events, but the WS hub forwards them to the owning user like any other event.
## 9. API, realtime, frontend
### 9.1 REST API (prefix `/api`, session-cookie auth, JSON; errors: `{error:{code,message}}`)
**Auth:** `POST /auth/register` · `POST /auth/login` · `POST /auth/logout` · `GET /auth/me` · `GET /auth/sessions` · `DELETE /auth/sessions/{id}` · `POST /auth/totp/setup` · `POST /auth/totp/enable` · `POST /auth/password/reset-request` · `POST /auth/password/reset`
**Accounts:** `POST /accounts/discover {email}` → `{candidates:[{imap,smtp,source}], steps:[...]}` · `POST /accounts/test {config, credentials}` → `{ok}|{error:{code,detail}}` · `POST /accounts` · `GET /accounts` (incl. `state`, sync progress) · `PATCH /accounts/{id}` · `DELETE /accounts/{id}` (cascades rows, blobs, search docs)
**Mail:** `GET /threads?view=unified|account:{id}|mailbox:{id}|search&q=&cursor=&limit=50` → thread summaries (denormalized fields only; cursor = `last_message_at,id` keyset) · `GET /threads/{id}?sort=asc|desc` → ordered full messages (bodies of the N newest expanded per client hint `expand=`; older = envelope only) · `POST /threads/{id}/flags {add,remove}` (whole thread) · `POST /messages/{id}/flags` · `POST /messages/{id}/move {mailbox_id}` · `GET /messages/{id}/html` (sanitized, for iframe `srcdoc`) · `GET /messages/{id}/cid/{content_id}` · `GET /attachments/{id}` (Content-Disposition attachment; images/PDF also inline for preview) · `POST /messages/{id}/allow-remote-images` (per-sender allowlist in user settings)
**Compose:** `POST /drafts` · `PUT /drafts/{id}` · `DELETE /drafts/{id}` · `POST /send {draft_id}` → `{outbox_id, undo_until}` · `POST /send/{outbox_id}/undo` · `GET /recipients/suggest?q=` → top 10 from `known_recipients` (prefix match on email or name, ranked by `times_used` decayed by recency); the composer merges these with any plugin capability providers (§11.3)
**Search:** `GET /search?q=&cursor=` → `{threads:[...], hits_by_thread, highlights}` (grouped by thread, FR-28; `&group=message` for flat) · `GET/POST/DELETE /saved-searches`
**Settings:** `GET/PATCH /settings` (incl. `thread_order: "asc"|"desc"`, `tz` IANA timezone — default from browser at registration, undo window, per-sender image allowlist)
### 9.2 WebSocket `/api/ws`
Auth via session cookie. Server → client frames: `{event, payload}` for every §8.3 event belonging to the user (that's the whole contract — the UI refetches via TanStack Query invalidation keyed on ids in the payload). Client → server: `{type:"ping"}` only. Zod-validate all frames.
### 9.3 Frontend routes
`/login`, `/register`, `/mail` (unified inbox), `/mail/a/:accountId`, `/mail/a/:accountId/f/:mailboxId`, `/mail/t/:threadId` (thread pane; on desktop rendered beside the list), `/mail/search?q=`, `/settings/{profile|security|accounts|appearance}`; `/settings/accounts/new` = wizard.
**Account wizard flow:** step 1 email (+ password field revealed unless preset says oauth) → calls `discover` while typing pauses; step 2 shows resolved config with source badge ("found via ISPDB") and a Test button (auto-run); failure expands the manual form pre-filled with the best guess and the precise error; step 3 = created, live backfill progress bar (WS `sync.progress`) with "start reading now" (list fills as backfill proceeds).
### 9.4 ThreadView contract (invariant I-5)
```
ThreadView(props: { messages: Message[] /* ASC always */, order: 'asc'|'desc',
composerState, ... })
```
- Rendering order = `order==='asc' ? messages : [...messages].reverse()`. No other branching on `order` is allowed in layout code.
- **Composer anchor** = a slot component rendered adjacent to the newest message: after it when `asc` (bottom of thread), directly after it when `desc` (top of thread, above older messages). Same component, one placement rule: "composer is always attached to the newest message."
- Initial scroll: to the newest message (bottom for asc, top for desc). Collapse: all read messages except the newest are collapsed; "Expand all" toggles. Unread messages are never collapsed.
- Keyboard: `j`/`k` move in *visual* order; `o`/`Enter` expand; `r` reply, `a` reply-all, `f` forward (open composer at anchor); `e` archive; `/` focus search; `gi` go to unified inbox; `esc` close composer (draft persists). Toolbar has the per-thread order toggle; it updates local state only, and long-presses/menu offers "make default" (PATCH /settings).
- Message HTML renders in `<iframe sandbox="" srcdoc={...}>` sized via ResizeObserver postMessage shim; remote images show a "load images / always from this sender" bar when the sanitizer flagged any.
## 10. Security requirements (hard)
1. Argon2id params per §6.1; no other password hashing anywhere.
2. **Credential vault:** mailbox secrets encrypted with AES-256-GCM under per-account data keys, wrapped by a master key from env `MASTER_KEY` (32 B, base64) — `cryptography` library, key id embedded for future rotation. Decryption code path exists only in `workers/` and `accounts/test`. Secrets never appear in logs, API responses, or exceptions (custom repr).
3. CSRF double-submit on all mutating routes; `__Host-` cookie prefix; CSP: `default-src 'self'; frame-src 'self'; img-src 'self' data:` (remote mail images are proxied? **No** — v1 loads allowed remote images inside the sandboxed iframe only, which has its own relaxed CSP via srcdoc; document this in code).
4. nh3 sanitization at index time; re-sanitize on render if sanitizer version changed (store `sanitizer_v` on body_parts).
5. Rate limits: login (§6.1), `POST /accounts/discover` 10/min/user, `POST /send` 30/h/user (env-tunable).
6. All outbound autodiscovery HTTP: HTTPS only, 5 s timeout, response size cap 256 KB, no off-domain redirects (SSRF guard: resolve + reject private IP ranges).
7. Every SQL query and search query includes `user_id` (enforced by repository-layer helpers; raw session use in API code is a review error).
8. Uploads (attachments to drafts): 25 MB/file, 40 MB/message total, content-type sniffed server-side, stored to blob store, never executed/served with original content type unless in safe list (images/pdf/text) — others force `application/octet-stream` + attachment disposition.
## 11. Plugin API (freeze in M3 — including the heavy-plugin surfaces below)
A plugin = Python package `webmail_plugin_<name>` exposing:
```python
plugin = Plugin(
name, version,
hooks = {event: handler}, # subscribe to §8.3 events
router = APIRouter | None, # mounted at /api/plugins/{name}
ui_manifest = UiManifest | None, # slots + pages, see below
migrations = "migrations/" | None, # 11.1
jobs = [JobSpec, ...] | None, # 11.2
)
```
Loader discovers plugins via entry points group `webmail.plugins`, forwards declared events, and serves `GET /api/plugins/ui-manifest` for the frontend loader. Plugins may not import from `app.*` internals other than `app.plugins.sdk` (which exposes: DB session factory scoped to plugin tables, EventBus publish under `plugin.{name}.*`, blob store, outbox draft/send helpers, settings accessor, HTTP client with the §10.6 SSRF guard built in).
Every plugin gets the namespaced KV store (`plugin_kv(plugin, user_id, key)`) for light state. Heavy plugins additionally get:
### 11.1 Plugin-owned tables
A plugin may ship Alembic migrations in its own version branch (`alembic -n plugin_{name}`). All tables MUST be prefixed `plg_{name}_` and every user-data table MUST carry `user_id` (invariant I-7 applies to plugins). Core migrations never reference plugin tables and vice versa. Uninstalling a plugin offers `--drop-data` to run its downgrade chain.
### 11.2 Plugin background jobs
`JobSpec(func, schedule: cron_str | None, singleton: bool)` registers tasks on the shared arq worker: cron-scheduled (e.g. a sync loop trigger) or enqueued on demand from hooks/routes. `singleton=True` guarantees at most one running instance per (plugin, user) via a Redis lock. Plugin jobs run in the same worker process pool with the same retry semantics as core jobs and are visible in worker logs under `plg.{name}.*`.
### 11.3 Frontend surfaces
- **Slots** (lazy-loaded ESM injected into mail UI): `thread.toolbar`, `message.actions`, `message.banner` (a strip rendered above a message body — used for spam warnings, calendar invite cards, and similar per-message notices), `sidebar.section`, `settings.page`.
- **Pages** (`app.page`): a plugin may declare full views: `{route: "/calendar", nav: {icon, label, order}, bundle}`. The shell adds the nav entry, and the route (including sub-routes under it) lazy-loads the bundle into the **entire content area** — no mail list/thread components are rendered. Page bundles receive the SDK context (API client, WS event stream filtered to `plugin.{name}.*` + requested core events, design-system components) and are sandboxed to their route: importing mail-component internals is a review error. Removing the plugin removes the nav entry and route cleanly.
- **Capabilities**: the manifest may declare named capabilities mapping to plugin endpoints, e.g. `capabilities: {"recipients.suggest": "/api/plugins/contacts/suggest"}`. Core UI features query all providers of a capability they support and merge results client-side (defined capabilities in v1: `recipients.suggest` — used by the composer To/Cc/Bcc fields and by any other plugin needing person-picking, such as calendar attendees). This keeps core backend code plugin-unaware.
Acceptance test for the API: implement **filters** (rules on `message.new`: match → move/flag) and **labels** entirely as plugins using only slots + KV; then implement the **calendar** (Appendix A) using tables + jobs + a page — all with zero core diffs.
## 12. Milestones with acceptance criteria
**M0 — Architecture spike**
Login/register (no 2FA), one hardcoded-config IMAP account, backfill+IDLE for INBOX, JWZ, read-only unified list + thread view with working asc/desc toggle and correct composer-anchor placeholder, Postgres FTS free-text search.
✅ Accept: sync a real 10k-message Gmail account via app password; threads match Gmail's grouping on a 50-thread sample ≥ 90%; toggle order and the newest message + anchor swap ends correctly; search returns in < 500 ms.
**M1 — MVP**
Full identity (2FA, sessions UI, reset), account wizard with full discovery chain, multi-account + unified inbox with account chips, flags/move/archive both directions, inline reply + reply-all/forward + SMTP send + Sent append, recipient autocomplete from `known_recipients`, drafts autosave, sanitized HTML + blocked remote images, operator search (full §8.2 grammar) on PG FTS, WS live updates.
✅ Accept: add gmail.com, outlook.com, and one autoconfig-only domain through the wizard typing only address+password; reply from a thread lands correctly threaded in the recipient's client; all §8.2 operators pass integration tests; `docker-compose up` from clean checkout works.
**M2 — Hardening / v1.0**
OAuth2 (Google + Microsoft, XOAUTH2 for IMAP/SMTP, server-side refresh), QRESYNC fast resync, drafts mirrored to server, undo-send, attachment upload/preview, Meilisearch backend + instant search-as-you-type UI + saved searches as virtual mailboxes, session revocation, backfill progress UX, folder-name i18n heuristics.
✅ Accept: typo query (`invocie`) finds "invoice" mail < 200 ms on a 100k-message corpus; kill -9 the worker mid-backfill and restart resumes without duplicates (idempotency via `(mailbox_id, uid)` unique key); undo-send within window results in no SMTP traffic.
**M3 — Plugin platform**
Loader, full SDK per §11 **including** plugin migrations (11.1), plugin jobs (11.2), and `app.page` (11.3); filters + labels shipped as plugins; plugin docs.
✅ Accept: both plugins install/uninstall cleanly with zero core diffs; a third trivial plugin ("Unsubscribe header button" on `message.actions`) can be written from `docs/plugins.md` alone; a skeleton page-plugin (route + nav entry + one plugin table + one cron job) round-trips install → run → uninstall `--drop-data` leaving no residue.
**M4 — Calendar plugin (Appendix A)**
Phases A1–A2: local calendars + full page views + iMIP invite handling; then A3: CalDAV client sync + ICS subscriptions.
✅ Accept: criteria in Appendix A §A.7.
**M5 — Contacts plugin (Appendix B)**
Phases B1–B2: contact books, full page, groups, add-from-mail, `recipients.suggest` capability, vCard import/export; then B3: CardDAV client sync. May be built in parallel with M4 — both depend only on the M3 SDK.
✅ Accept: criteria in Appendix B §B.7.
**M6 (optional) — Hosted mailboxes**
Inbound MTA (Stalwart) delivering to a `NativeProvider`; handle becomes receiving address. Specified separately when reached.
**M7 — Anti-spam plugin (Appendix C)**
May be built any time after M3 (depends only on the plugin SDK; parallelizable with M4/M5). Phases C1–C2 client-side; C3 activates at the M6 edge.
✅ Accept: criteria in Appendix C §C.8.
## 13. Testing & CI
- **Unit:** pytest + pytest-asyncio; threading vectors (§7.4); query parser (§8.2); autodiscovery resolver with `respx`-mocked HTTP + `dnspython` stub; MIME corpus in `tests/fixtures/mime/` (agent: include ≥ 25 nasty real-world-style fixtures — encoded headers incl. RFC2047 Polish diacritics, nested multiparts, missing Message-ID, malformed dates, HTML-only, inline cid images, 8-bit subjects, winmail.dat placeholder).
- **Integration:** docker-compose.ci.yml with Greenmail (IMAP+SMTP) and Dovecot; full loop test: append message to Greenmail → appears in API < 10 s → reply via API → arrives in Greenmail SMTP sink with correct References.
- **Frontend:** vitest + React Testing Library; ThreadView ordering/anchor tests for both directions; Playwright smoke: login → add Greenmail account via wizard → read → inline reply → search.
- **CI gates:** ruff, mypy --strict, pytest, tsc, eslint, vitest, Playwright smoke; coverage floor 80% on `threading_`, `search/queryparser`, `accounts/autodiscovery`, `indexer`.
## 14. Configuration (env, complete list)
`APP_DOMAIN`, `DATABASE_URL`, `REDIS_URL`, `MASTER_KEY`, `BLOB_ROOT` or `S3_*`, `SEARCH_BACKEND=pg|meili`, `MEILI_URL`, `MEILI_KEY`, `UNDO_SECONDS=10`, `SESSION_TTL_DAYS=30`, `OAUTH_GOOGLE_CLIENT_ID/SECRET`, `OAUTH_MS_CLIENT_ID/SECRET`, `MAX_ATTACHMENT_MB=25`, `LOG_LEVEL`; sync engine v2 (§6.3): `WORKER_ROLE=jobs|sync`, `MAX_ACCOUNTS_PER_WORKER=400`, `LEASE_TTL=60`, `LEASE_RENEW=20`, `POLL_WARM=180`, `POLL_COLD=900`, `TIER_HOT_ACTIVITY_H=24`, `TIER_SESSION_GRACE_MIN=30`, `TIER_COLD_AFTER_D=3`, `HOT_BUDGET_PER_WORKER=250`, `PER_HOST_CONN_CAP=50`, `BULK_CONCURRENCY=2`, `PARSE_CONCURRENCY=4`, `STREAM_PARSE_MB=8`. All read once in `config.py` (pydantic-settings); no `os.environ` elsewhere.
---
## Appendix A — Calendar plugin (`webmail_plugin_calendar`)
The calendar is the reference "heavy plugin": it lives in the same app and repo, uses the §11 SDK only, and must build with **zero core diffs**. It ships as a page (`/calendar`) beside Mail — one product, one login.
### A.1 Scope by phase
- **A1 — Local calendar:** create/edit/delete events, multiple calendars per user, month/week/day/agenda views, recurrence, reminders (WS notification), timezone-correct display.
- **A2 — Invites over mail (iMIP):** detect `text/calendar` parts on `message.new`; render an invite card in the thread (`message.actions` + inline summary); Accept/Tentative/Decline sends an iTIP REPLY through the outbox; organizer flow sends REQUEST/CANCEL to attendees; updates (SEQUENCE bumps) modify the stored event.
- **A3 — External sources:** `CalendarProvider` interface with **CalDAV client** (first implementation; covers iCloud, Fastmail, Nextcloud) and read-only **ICS URL subscriptions** (holidays, team feeds); sync loop as a plugin job.
- **A4 (deferred, spec later):** Google/Microsoft API providers reusing M2 OAuth machinery; secret **ICS export feed** `GET /api/plugins/calendar/feed/{token}.ics`; CalDAV **server** endpoint for native phone sync — architecturally anticipated, not built now.
### A.2 Dependencies
`icalendar ~5.0` (parse/serialize RFC 5545), `python-dateutil` (RRULE expansion), `caldav ~1.3` (A3), `tzdata`. No other additions.
### A.3 Data model (plugin migrations, prefix `plg_calendar_`)
```sql
CREATE TABLE plg_calendar_calendars (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
name text NOT NULL,
color text NOT NULL DEFAULT '#3b82f6',
kind text NOT NULL DEFAULT 'local', -- local|caldav|ics_subscription
source_config jsonb NOT NULL DEFAULT '{}', -- caldav url/collection, ics url
secret_ct bytea, -- caldav credentials, §10.2 vault scheme
sync_token text, -- CalDAV sync-token / ICS etag
read_only boolean NOT NULL DEFAULT false,
UNIQUE (user_id, name)
);
CREATE TABLE plg_calendar_events (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
calendar_id uuid NOT NULL REFERENCES plg_calendar_calendars(id) ON DELETE CASCADE,
ical_uid text NOT NULL, -- UID from iCalendar; join key for iMIP updates
sequence int NOT NULL DEFAULT 0,
summary text NOT NULL DEFAULT '',
description text NOT NULL DEFAULT '',
location text NOT NULL DEFAULT '',
dtstart timestamptz NOT NULL,
dtend timestamptz,
all_day boolean NOT NULL DEFAULT false,
tz text NOT NULL DEFAULT 'UTC', -- original TZID; render in user tz, store UTC
rrule text, -- raw RRULE string, NULL = single
exdates timestamptz[] NOT NULL DEFAULT '{}',
recurrence_id timestamptz, -- non-NULL = detached exception instance
organizer jsonb, -- {name,email}
attendees jsonb NOT NULL DEFAULT '[]', -- [{name,email,partstat,role}]
status text NOT NULL DEFAULT 'confirmed', -- confirmed|tentative|cancelled
my_partstat text, -- needs-action|accepted|declined|tentative
reminders int[] NOT NULL DEFAULT '{}', -- minutes before start
origin_message_id uuid, -- link back to the invite mail thread
raw_ical text, -- round-trip fidelity for iMIP/CalDAV
UNIQUE (calendar_id, ical_uid, recurrence_id)
);
CREATE INDEX ON plg_calendar_events (user_id, dtstart);
```
**Recurrence rule:** store the RRULE string; expand instances at query time with `dateutil.rrule` over the requested view range only (no materialization table). Exceptions: `exdates` remove instances; a row with `recurrence_id` overrides one instance. Range queries must account for events starting before the window but recurring into it (expand from `dtstart`, cap iterations at 1000 per event).
### A.4 CalendarProvider interface (inside the plugin, mirrors §5.1)
```python
# PLUGIN INTERFACE VERSION: 1
class CalendarProvider(Protocol):
async def verify(self) -> None: ...
async def list_collections(self) -> list[RemoteCalendar]: ...
async def pull(self, collection: str, *, sync_token: str | None
) -> tuple[list[RemoteEvent], list[str], str]: ...
# returns (changed_events_as_ical, deleted_uids, new_sync_token)
async def push(self, collection: str, ical: str, *, uid: str,
etag: str | None) -> str: ... # returns new etag
async def delete(self, collection: str, uid: str, etag: str | None) -> None: ...
```
Implementations in A3: `CaldavProvider`, `IcsSubscriptionProvider` (pull-only, `push`/`delete` raise `ReadOnlyCalendar`). Sync loop = plugin cron job (`schedule="*/5 * * * *"`, `singleton=True` per user) doing pull → upsert by `(calendar_id, ical_uid, recurrence_id)` → local-dirty push with etag conflict handling (conflict ⇒ server wins, local copy saved as a duplicate marked `status='tentative'`, user notified).
### A.5 iMIP flow (A2)
- **Inbound:** hook on `message.new`; if a `text/calendar` part exists, parse METHOD:
`REQUEST` → upsert event (match `ical_uid`; apply only if `SEQUENCE` ≥ stored) into the user's default calendar with `my_partstat='needs-action'`, set `origin_message_id`; `CANCEL` → `status='cancelled'`; `REPLY` → update that attendee's `partstat` on the organizer's copy.
- **RSVP:** invite card buttons call `POST /api/plugins/calendar/events/{id}/rsvp {partstat}` → update row → build iTIP REPLY ics → send via SDK outbox helper to the organizer (subject `Accepted: {summary}` etc.) → publish `plugin.calendar.event_updated`.
- **Organizer:** creating an event with attendees sends METHOD:REQUEST to each; edits bump `SEQUENCE` and re-send; deletion sends CANCEL. All outgoing invite mail uses the account whose address matches the organizer identity (picker if ambiguous).
### A.6 UI (page `/calendar`, nav icon between Mail and Settings)
Routes: `/calendar` (month default), `/calendar/{month|week|day|agenda}/:date`, `/calendar/event/:id`, `/calendar/settings` (calendars CRUD, CalDAV/ICS wizard reusing the account-wizard UX pattern: URL+credentials → test → collection picker). Interactions: click-drag create, drag-move/resize (writes optimistically, reverts on API error), event popover with edit/RSVP, mini-month navigator, calendar visibility toggles (client state), `t` = today, arrows = prev/next period. Invite linkage both ways: invite card in thread links to `/calendar/event/:id`; event view links back to the mail thread when `origin_message_id` is set. Live updates via WS `plugin.calendar.*` events. Timezone: render in user setting `settings.tz` (add to §9.1 settings; default from browser at registration).
### A.7 Acceptance criteria
1. Plugin installs/uninstalls (`--drop-data`) with zero diffs in `app/` and `frontend/src/` outside `plugins/`/`slots/` loader code.
2. A weekly-recurring event with two EXDATEs and one moved instance renders correctly in month and week views across a DST boundary (Europe/Warsaw fixtures).
3. Round-trip with Google Calendar **via iMIP mail only**: receive a Google invite → Accept in our UI → organizer's Google Calendar shows attendee accepted; our organizer invite → Google attendee accepts → our event shows `partstat=accepted`.
4. CalDAV sync against a Nextcloud/Radicale test container: create/edit/delete propagate both directions in ≤ 5 min (one cron cycle); conflict case produces the A.4 tentative-duplicate, never silent loss.
5. ICS subscription to a public holiday feed renders read-only events; editing is blocked in UI and API (403 `read_only_calendar`).
6. Killing the worker mid-sync and restarting causes no duplicate events (upsert key discipline).
---
## Appendix B — Contacts plugin (`webmail_plugin_contacts`)
The second heavy plugin: same rules as the calendar — §11 SDK only, zero core diffs, own page. Division of labor with core: core owns *implicit* recipient history (`known_recipients`, §4/§9.1) so autocomplete works even without this plugin; the Contacts plugin owns *explicit*, user-curated contact records and enriches suggestions via the `recipients.suggest` capability (§11.3).
### B.1 Scope by phase
- **B1 — Local contacts:** address books, contact CRUD with structured fields, groups, photo, search, dedicated page `/contacts`; "Add sender to contacts" from the thread view; suggestion capability wired into the composer.
- **B2 — Portability:** vCard 3.0/4.0 import (file, multi-card) and export (single contact / book), duplicate detection + merge tool.
- **B3 — External sources:** `ContactProvider` interface with **CardDAV client** sync (iCloud, Fastmail, Nextcloud, Radicale); per-book sync job.
- **B4 (deferred, spec later):** Google People / Microsoft Graph providers (reusing M2 OAuth), CardDAV **server** endpoint for native phone sync, book sharing.
### B.2 Dependencies
`vobject ~0.9` (vCard parse/serialize). Photos stored via the SDK blob store (JPEG/PNG, re-encoded server-side, ≤ 512×512). Nothing else.
### B.3 Data model (plugin migrations, prefix `plg_contacts_`)
```sql
CREATE TABLE plg_contacts_books (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
name text NOT NULL,
kind text NOT NULL DEFAULT 'local', -- local|carddav
source_config jsonb NOT NULL DEFAULT '{}',-- carddav url/collection
secret_ct bytea, -- carddav credentials, §10.2 vault scheme
sync_token text,
read_only boolean NOT NULL DEFAULT false,
UNIQUE (user_id, name)
);
CREATE TABLE plg_contacts_contacts (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
book_id uuid NOT NULL REFERENCES plg_contacts_books(id) ON DELETE CASCADE,
vcard_uid text NOT NULL, -- UID property; join key for CardDAV
etag text, -- server etag (carddav books)
name_display text NOT NULL DEFAULT '', -- FN
name_parts jsonb NOT NULL DEFAULT '{}', -- {given,family,middle,prefix,suffix} (N)
emails jsonb NOT NULL DEFAULT '[]', -- [{value,type,pref}]
phones jsonb NOT NULL DEFAULT '[]',
addresses jsonb NOT NULL DEFAULT '[]', -- structured ADR
org text NOT NULL DEFAULT '',
title text NOT NULL DEFAULT '',
birthday date,
urls jsonb NOT NULL DEFAULT '[]',
notes text NOT NULL DEFAULT '',
photo_sha256 text,
raw_vcard text, -- round-trip fidelity: preserve unknown props
updated_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (book_id, vcard_uid)
);
CREATE INDEX ON plg_contacts_contacts (user_id, name_display);
CREATE INDEX ON plg_contacts_contacts USING gin (emails jsonb_path_ops);
CREATE TABLE plg_contacts_groups (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
name text NOT NULL,
UNIQUE (user_id, name)
);
CREATE TABLE plg_contacts_group_members (
group_id uuid NOT NULL REFERENCES plg_contacts_groups(id) ON DELETE CASCADE,
contact_id uuid NOT NULL REFERENCES plg_contacts_contacts(id) ON DELETE CASCADE,
PRIMARY KEY (group_id, contact_id)
);
```
vCard mapping discipline: parse known properties into columns, keep the full source in `raw_vcard`, and on export/push regenerate from columns **merged over** `raw_vcard` so properties we don't model (e.g. `X-` extensions) survive round-trips.
### B.4 ContactProvider interface (inside the plugin, mirrors A.4)
```python
# PLUGIN INTERFACE VERSION: 1
class ContactProvider(Protocol):
async def verify(self) -> None: ...
async def list_books(self) -> list[RemoteBook]: ...
async def pull(self, book: str, *, sync_token: str | None
) -> tuple[list[RemoteVcard], list[str], str]: ...
# (changed vcards, deleted uids, new sync_token)
async def push(self, book: str, vcard: str, *, uid: str,
etag: str | None) -> str: ...
async def delete(self, book: str, uid: str, etag: str | None) -> None: ...
```
Sync loop = plugin cron job (`*/5 * * * *`, singleton per user), upsert by `(book_id, vcard_uid)`, etag conflict ⇒ server wins + local copy kept as a duplicate flagged for the merge tool (never silent loss) — same policy as A.4.
### B.5 API & capability (`/api/plugins/contacts/`)
`GET /contacts?book=&group=&q=&cursor=` · `POST /contacts` · `GET/PUT/DELETE /contacts/{id}` · `POST /contacts/from-message {message_id}` (prefill from From header) · `GET/POST/DELETE /books`, `/groups`, group membership ops · `POST /import` (vCard file → report `{created, updated, skipped}`) · `GET /export?book=|contact=` (`.vcf`) · `GET /duplicates` (candidate pairs: same email, or same normalized name + fuzzy) · `POST /merge {keep_id, remove_id}` (field union, keep wins conflicts, group memberships merged) · **`GET /suggest?q=`** — the `recipients.suggest` capability endpoint: matches name/email prefix and substring, returns `[{name, email, source:"contact", contact_id, photo_url?}]`. Composer merge rule (client-side, defined here so behavior is deterministic): contact hits rank above `known_recipients` hits at equal prefix quality; dedupe by email, contact entry wins and carries the photo/display name.
### B.6 UI
Page `/contacts` (nav entry after Calendar): three-pane layout — books/groups sidebar, alphabetically grouped virtualized list with search-as-you-type, detail pane with edit-in-place form; import/export and merge review under `/contacts/tools`; CardDAV book wizard reusing the account-wizard UX pattern (URL + credentials → test → book picker). Mail integration via `message.actions` slot: "Add to Contacts" (hidden when the sender already matches a contact; then it becomes "View contact" linking to `/contacts/{id}`). Group compose: selecting a group in the composer expands to its members' preferred emails.
### B.7 Acceptance criteria
1. Zero core diffs; install/uninstall `--drop-data` round-trip clean.
2. Import a 500-card vCard file mixing v3.0/v4.0, quoted-printable encoding, and Polish diacritics in names (`Grzegorz Brzęczyszczykiewicz`) — all names, emails, and photos correct; re-export → re-import is lossless including `X-` properties (raw_vcard merge discipline).
3. Composer typing a 3-char prefix returns merged suggestions in < 100 ms p95 (contacts ranked above history, deduped by email, photos shown for contacts).
4. CardDAV sync against Radicale: create/edit/delete propagate both directions within one cron cycle; concurrent edit conflict produces a flagged duplicate, never silent loss; worker kill mid-sync ⇒ no duplicate contacts on restart.
5. "Add sender to contacts" from a thread creates a prefilled contact; the same sender's messages immediately render the contact's display name, and the action flips to "View contact".
6. Merge tool: merging two contacts unions emails/phones/groups, keeps the chosen primary's conflicting fields, and repoints nothing else in the system (calendar attendees and mail remain keyed by email, by design).
---
## Appendix C — Anti-spam plugin (`webmail_plugin_antispam`)
Third heavy plugin; §11 SDK only, zero core diffs. Architectural context: for **synced** accounts the upstream provider has usually already filtered spam, so this plugin is a *second-pass* filter (weak-provider accounts, unified experience, per-user trainable Bayes). It becomes the *first-pass* filter only for M6 hosted mailboxes, where the same engine also runs at the SMTP edge (§C.7).
### C.1 Scope by phase
- **C1 — Classification & actions:** score every newly synced inbox message; move high-scoring mail to Junk (through the existing move machinery, so the verdict propagates to the IMAP server); banner low-confidence suspects; per-user allow/deny lists; per-account enable toggle.
- **C2 — Learning loop:** explicit "Spam"/"Not spam" actions train per-user Bayes; passive learning from user moves in/out of Junk and (sampled, capped) from provider-classified Junk; stats page.
- **C3 (M6 era):** the same Rspamd configuration/expertise deployed at the mail edge (VPS), where full IP/SPF/DKIM/DMARC/greylisting checks are actually meaningful; hosted-mailbox delivery is filtered *before* it reaches the index.
### C.2 Engine decision
**Rspamd** as a service container (mature, fast, HTTP API) with a small **dedicated Redis** container for its Bayes/fuzzy storage (isolated from the app Redis on purpose — different persistence and flush semantics). No Python spam engine is written from scratch; the plugin is glue + policy. Compose additions: `rspamd` (`rspamd/rspamd:3.x`) + `redis-rspamd` (`redis:7`, `--save 300 1`). Rspamd's network checks (RBL/SURBL/fuzzy lookups leak message-derived hashes/URLs to third parties) are **off by default**, one settings toggle to enable (privacy default matches the product's self-hosted ethos).
### C.3 Data model (plugin migrations, prefix `plg_antispam_`)
```sql
CREATE TABLE plg_antispam_settings (
user_id uuid PRIMARY KEY,
enabled_accounts uuid[] NOT NULL DEFAULT '{}', -- empty = all accounts
move_threshold real NOT NULL DEFAULT 12.0, -- score >= : move to Junk
banner_threshold real NOT NULL DEFAULT 6.0, -- score >= : warn banner
network_checks boolean NOT NULL DEFAULT false,
learn_from_provider_junk boolean NOT NULL DEFAULT true
);
CREATE TABLE plg_antispam_rules (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL,
kind text NOT NULL, -- allow|deny
pattern citext NOT NULL, -- email or @domain
UNIQUE (user_id, kind, pattern)
);
CREATE TABLE plg_antispam_verdicts (
message_id uuid PRIMARY KEY,
user_id uuid NOT NULL,
score real NOT NULL,
symbols jsonb NOT NULL DEFAULT '[]', -- rspamd symbol list for explainability
action text NOT NULL, -- none|banner|moved|allow_listed|deny_listed
learned text -- spam|ham|NULL
);
```
### C.4 Classification flow
Hook on `message.new`; process only messages landing in `special_use='inbox'` mailboxes of enabled accounts (never Sent/Drafts/Junk/Trash; never re-classify on flag changes). Order: (1) deny list → move to Junk, action `deny_listed`; (2) allow list → action `allow_listed`, stop; (3) POST raw blob to Rspamd `/checkv2` with `Deliver-To`-style per-user context header (per-user Bayes statistics via Rspamd user settings); (4) apply thresholds → move via the existing move job (verdict thus syncs to the server's Junk folder) or record `banner`; (5) write verdict row, publish `plugin.antispam.classified`. Rspamd unavailability = fail-open (deliver normally, log, retry queue for learning only — spam filtering must never delay or lose mail).
### C.5 Learning
- Explicit: `message.actions` buttons "Spam" / "Not spam" → `POST /api/plugins/antispam/learn {message_id, verdict}` → Rspamd `/learnspam|/learnham` with the per-user context → move message accordingly → update `verdicts.learned`.
- Passive: hook on `flags.changed`/move events — user moves message out of Junk ⇒ learn ham; into Junk ⇒ learn spam (dedup via `verdicts.learned`).
- Provider bootstrap (per account, if `learn_from_provider_junk`): during/after backfill, sample up to 500 provider-Junk messages as spam corpus and 500 aged inbox messages as ham, rate-limited background job. Gives per-user Bayes a warm start.
### C.6 API & UI
`GET/PATCH /api/plugins/antispam/settings` · `GET/POST/DELETE /rules` · `POST /learn` · `GET /stats` (verdict counts, top symbols, Bayes confidence). UI: `settings.page` (thresholds with plain-language labels, per-account toggles, lists editor, privacy toggle, stats); `message.banner` for the suspect zone ("Looks like spam — [Move to Junk] [Not spam] [Always trust sender]"); `message.actions` Spam/Not-spam. Junk folder view is core (SPECIAL-USE) and needs no changes.
### C.7 M6 edge integration (assumption, specified fully with M6)
At the hosted-mail phase the VPS edge (Stalwart's built-in filter or Postfix+Rspamd) performs first-pass filtering with the checks that only make sense at SMTP time (connecting IP, SPF/DKIM/DMARC verification, greylisting, rate limits); accepted mail is tagged with its edge score in a header, and this plugin trusts that header for hosted accounts instead of re-scanning — one user-visible spam system, two enforcement points.
### C.8 Acceptance criteria
1. Zero core diffs; install/uninstall `--drop-data` clean; with the plugin absent, mail flow is byte-identical to v1.4 core.
2. Rspamd container stopped ⇒ new mail still indexes and renders normally within normal latency (fail-open proven by test).
3. Verdict propagation: a moved-to-Junk verdict appears in the provider's Junk folder via IMAP within one sync cycle; "Not spam" moves it back and the same message is not re-classified.
4. Learning: after 20 spam + 20 ham teachings on a fixture corpus, a held-out obvious-spam fixture scores above `move_threshold` for that user and a ham fixture below `banner_threshold` (per-user isolation: a second user's scores are unaffected).
5. Deny-listed sender is moved without engine involvement; allow-listed sender bypasses even a high engine score.
6. Privacy: with `network_checks=false` (default), a network-capture test during classification shows zero outbound connections from Rspamd.
---
*End of specification. Ambiguities: prefer the simplest behavior consistent with the invariants in §1, leave a `TODO(spec)` comment, and list open questions in `docs/open-questions.md` rather than guessing silently.*
+103
View File
@@ -0,0 +1,103 @@
How the worker functions (multi-user / multi-account)
=====================================================
The worker wears two hats (in one process, one event loop)
----------------------------------------------------------
It is a single arq process (`arq workers.worker.WorkerSettings`) running one
asyncio event loop. Inside it, two very different things run:
A) A job runner (the Redis queue side). The API enqueues discrete jobs; the
worker pulls and runs them, up to `max_jobs = 10` concurrently,
`job_timeout = 3600s`. The registered jobs:
sync_bootstrap_account (backfill a new account), push_flags_job, move_job,
move_thread_to_folder_job, delete_thread_job, send_job,
push_draft_to_imap_job, remove_imap_draft_job, refresh_account_job.
Each job builds a provider (which is where the mailbox credential is
decrypted with the MASTER_KEY - worker-only), does the IMAP/SMTP op, and
closes the connection. On failure they retry with backoff (~x5) and then
revert or mark failed.
B) A live-sync supervisor (long-running loops). Started once in on_startup and
is NOT part of the job queue.
The per-account model (the key point)
-------------------------------------
On startup, start_supervisor reads every account with state = "live" and, for
each account, spawns two persistent asyncio tasks:
1. INBOX IDLE loop (live_loop) - opens one IMAP connection and keeps it open,
blocking on IMAP IDLE (server push). It reacts within seconds when mail
arrives, and re-issues IDLE at least every 25 min. This is a *persistent*
connection held 24/7.
2. Secondary poll loop (secondary_poll_loop) - every POLL_INTERVAL (120 s) it
opens a fresh, transient connection, CONDSTORE-polls
Sent/Drafts/Archive/Trash/Junk (cheap MODSEQ deltas), and closes it.
So the steady-state cost scales linearly with the total number of email
accounts across all users:
- Persistent IMAP connections : ~1 per account (the INBOX IDLE) held
continuously
- asyncio tasks : 2 per account
- Transient connections : one burst per account every 120 s (secondary
poll) + per job
For 3 users x 3 accounts that is ~9 always-open IDLE connections and 18 tasks -
trivial. For 500 accounts it is ~500 sockets open 24/7 plus 1000 tasks, and you
will hit IMAP server per-user/global connection limits, file-descriptor limits,
and memory pressure long before anything else.
How updates reach the browsers
------------------------------
The worker never talks to browsers. When it persists a change it publishes to
Redis pub/sub (evt:{event}); the API's WebSocket hub relays each event to the
owning user's connected clients, which invalidate the relevant queries. So Redis
is both the job queue and the fan-out bus.
Honest scaling limitations
--------------------------
- Single process / single core. All those loops and jobs share one event loop.
It is fine because IMAP work is I/O-bound (mostly waiting), but a heavy
backfill or MIME parse briefly blocks everything else, and each raw message is
loaded fully into memory to parse + blob-store - so messages x concurrency is
your memory driver.
- The live-sync side does not horizontally scale. If you ran two workers for
throughput, both would start IDLE loops for all live accounts -> duplicate
connections and double-processing. The job side would shard fine across
workers (that is what arq does), but the supervisor assumes exactly one
instance.
- Account discovery is startup-only. New or re-configured accounts are not
picked up as live until the worker restarts (the code says so explicitly). A
new account does get its one-time backfill immediately (via the enqueued
sync_bootstrap_account job), but its ongoing IDLE loop will not start until a
restart. The same gap causes the "zombie loop" after account deletion.
- max_jobs = 10 is global, shared across all users - a burst of sends/moves
queues behind that cap.
What a real multi-tenant deployment would add
---------------------------------------------
- A pool of sync workers with account sharding: each account owned by exactly
one worker (e.g., a Redis lease/consistent hashing), so IDLE loops do not
duplicate and you can scale out.
- React to account.state_changed to start/stop loops at runtime (no restart on
add/edit/delete).
- Connection budgeting - e.g., keep IDLE only for recently-active mailboxes and
poll the long tail - plus per-server connection caps and backpressure.
- Move sync_bootstrap_account (a large backfill) onto a separate, higher-timeout
queue from the fast interactive jobs (send/flags/move).
Net: the current design is a clean, correct single-node model - great for a
handful of users with a few accounts each - but the per-account persistent IDLE
loop and the startup-only, single-supervisor assumptions are the things you
would re-architect before going multi-tenant at scale.