mail UX: restore-to-folder, draft/attachment fixes, printing, sidebar collapse
Backend: - Restore returns each message to its original folder (e.g. Sent/Archive) instead of always the Inbox: new messages.restore_mailbox_id (migration 0010) + restore_thread_job. - Drafts: expunge reconciliation for the Drafts poll (provider.list_uids / _reconcile_drafts) clears phantom "remote" drafts; send + discard drop the mirrored DB row; adopting a remote draft carries its real attachments into the compose flow. - Thread detail/list are folder-aware: individually trashed/spam messages are hidden outside their folder, Trash/Spam show only their own messages, and the mail-list message count matches the view. Frontend: - Thread reading panel: fix newest-first card overlap/spacing (expand-all and single expand), hide Reply/Forward on trashed messages, close the Actions menu on message-body (iframe) clicks, scroll to the inline composer, and prompt Save draft / Discard on the X close. - Composer: remount when switching drafts, dirty-tracking to stop duplicate IMAP mirrors, and close the From/More dropdowns on outside click. - Sidebar: collapse/expand toggle (icons-only ~60px rail) and a log-out icon on Sign out. - Mail-standard printing in a new window with reliable page margins (lib/print.ts). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
@@ -111,8 +111,13 @@ async def trash_thread(thread_id: uuid.UUID, user: CurrentUser, db: DbSession) -
|
||||
|
||||
@router.post("/threads/{thread_id}/restore")
|
||||
async def restore_thread(thread_id: uuid.UUID, user: CurrentUser, db: DbSession) -> dict[str, str]:
|
||||
"""Move a thread back to the Inbox (from Archive or Trash)."""
|
||||
await _move_thread(user.id, thread_id, db, "inbox")
|
||||
"""Restore a thread (from Archive or Trash) to each message's original folder."""
|
||||
thread = await db.scalar(
|
||||
select(Thread).where(Thread.id == thread_id, Thread.user_id == user.id)
|
||||
)
|
||||
if thread is None:
|
||||
raise_api_error(404, "not_found", "thread not found")
|
||||
await enqueue_job("restore_thread_job", str(thread_id))
|
||||
return {"status": "queued"}
|
||||
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ from app.config import get_settings
|
||||
from app.db import get_session
|
||||
from app.identity.deps import CurrentUser, require_csrf
|
||||
from app.indexer import blobstore
|
||||
from app.models import BodyPart, ComposeAttachment, Draft, Mailbox, Message
|
||||
from app.models import Attachment, BodyPart, ComposeAttachment, Draft, Mailbox, Message
|
||||
from app.outbox.schemas import (
|
||||
DraftIn,
|
||||
DraftOut,
|
||||
@@ -24,7 +24,13 @@ from app.outbox.schemas import (
|
||||
SendRequest,
|
||||
SendResponse,
|
||||
)
|
||||
from app.outbox.service import DraftError, cancel_outbox, create_outbox, upsert_draft
|
||||
from app.outbox.service import (
|
||||
DraftError,
|
||||
cancel_outbox,
|
||||
create_outbox,
|
||||
delete_mirrored_draft_row,
|
||||
upsert_draft,
|
||||
)
|
||||
from app.queue import enqueue_job
|
||||
|
||||
router = APIRouter(tags=["compose"], dependencies=[Depends(require_csrf)])
|
||||
@@ -183,6 +189,30 @@ async def adopt_remote_draft(message_id: uuid.UUID, user: CurrentUser, db: DbSes
|
||||
parts = list(await db.scalars(select(BodyPart).where(BodyPart.message_id == msg.id)))
|
||||
html = next((p.content for p in parts if p.kind == "html_sanitized"), "")
|
||||
text = next((p.content for p in parts if p.kind == "text"), "")
|
||||
|
||||
# Carry the draft's real (non-inline) attachments over as compose attachments,
|
||||
# so they show in the editor and are re-attached on save/send. The underlying
|
||||
# blobs are shared by sha256 — we only add new ComposeAttachment references.
|
||||
att_rows = list(
|
||||
await db.scalars(
|
||||
select(Attachment)
|
||||
.where(Attachment.message_id == msg.id, Attachment.content_id.is_(None))
|
||||
.order_by(Attachment.filename)
|
||||
)
|
||||
)
|
||||
attachment_ids: list[str] = []
|
||||
for a in att_rows:
|
||||
ca = ComposeAttachment(
|
||||
user_id=user.id,
|
||||
filename=a.filename,
|
||||
mime_type=a.mime_type,
|
||||
size_bytes=a.size_bytes,
|
||||
blob_sha256=a.blob_sha256,
|
||||
)
|
||||
db.add(ca)
|
||||
await db.flush()
|
||||
attachment_ids.append(str(ca.id))
|
||||
|
||||
payload = {
|
||||
"to": [_fmt_addr(a) for a in msg.to_addr],
|
||||
"cc": [_fmt_addr(a) for a in msg.cc_addr],
|
||||
@@ -190,7 +220,7 @@ async def adopt_remote_draft(message_id: uuid.UUID, user: CurrentUser, db: DbSes
|
||||
"subject": msg.subject,
|
||||
"html": html,
|
||||
"text": text,
|
||||
"attachment_ids": [],
|
||||
"attachment_ids": attachment_ids,
|
||||
}
|
||||
draft = Draft(
|
||||
user_id=user.id,
|
||||
@@ -244,6 +274,10 @@ async def delete_draft(draft_id: uuid.UUID, user: CurrentUser, db: DbSession) ->
|
||||
raise_api_error(404, "not_found", "draft not found")
|
||||
account_id, imap_uid = draft.account_id, draft.imap_uid
|
||||
await db.delete(draft)
|
||||
# Drop the mirrored Drafts-folder message row now so the draft doesn't
|
||||
# resurface as an "other client" remote draft after the local row is gone.
|
||||
if imap_uid:
|
||||
await delete_mirrored_draft_row(db, account_id=account_id, imap_uid=imap_uid)
|
||||
await db.commit()
|
||||
# Remove the mirrored copy from the server Drafts folder too.
|
||||
if imap_uid:
|
||||
|
||||
@@ -6,7 +6,7 @@ import uuid
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import literal, select, tuple_
|
||||
from sqlalchemy import func, literal, or_, select, tuple_
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.api.cursors import decode_cursor, encode_cursor
|
||||
@@ -31,14 +31,16 @@ def _addresses(raw: list[dict[str, str]]) -> list[Address]:
|
||||
return [Address(name=a.get("name", ""), email=a.get("email", "")) for a in raw]
|
||||
|
||||
|
||||
def _summary(thread: Thread) -> ThreadSummary:
|
||||
def _summary(thread: Thread, message_count: int | None = None) -> ThreadSummary:
|
||||
return ThreadSummary(
|
||||
id=thread.id,
|
||||
subject=thread.subject_norm or "(no subject)",
|
||||
snippet=thread.snippet,
|
||||
participants=_addresses(thread.participants),
|
||||
account_ids=list(thread.account_ids),
|
||||
message_count=thread.message_count,
|
||||
# Count only the messages this view actually shows (deleted/spam excluded
|
||||
# elsewhere); falls back to the denormalized total.
|
||||
message_count=message_count if message_count is not None else thread.message_count,
|
||||
unread_count=thread.unread_count,
|
||||
has_attachments=thread.has_attachments,
|
||||
pinned=thread.pinned,
|
||||
@@ -123,7 +125,32 @@ async def list_threads(
|
||||
next_cursor = (
|
||||
encode_cursor(threads[-1].last_message_at, threads[-1].id) if has_more and threads else None
|
||||
)
|
||||
return ThreadListOut(threads=[_summary(t) for t in threads], cursor=next_cursor)
|
||||
|
||||
# Per-thread visible message count matching what get_thread shows for this view:
|
||||
# Trash/Spam count only their own messages; every other view excludes them. This
|
||||
# keeps the list count in step with a single deleted message.
|
||||
counts: dict[uuid.UUID, int] = {}
|
||||
if threads:
|
||||
count_stmt = (
|
||||
select(Message.thread_id, func.count())
|
||||
.join(Mailbox, Mailbox.id == Message.mailbox_id)
|
||||
.where(Message.user_id == user.id, Message.thread_id.in_([t.id for t in threads]))
|
||||
.group_by(Message.thread_id)
|
||||
)
|
||||
current_special = view.split(":", 1)[1] if view.startswith("folder:") else None
|
||||
if current_special in _REMOVAL_FOLDERS:
|
||||
count_stmt = count_stmt.where(Mailbox.special_use == current_special)
|
||||
else:
|
||||
count_stmt = count_stmt.where(
|
||||
or_(Mailbox.special_use.is_(None), Mailbox.special_use.notin_(_REMOVAL_FOLDERS))
|
||||
)
|
||||
counts = {
|
||||
r[0]: r[1] for r in (await db.execute(count_stmt)).all() if r[0] is not None
|
||||
}
|
||||
|
||||
return ThreadListOut(
|
||||
threads=[_summary(t, counts.get(t.id)) for t in threads], cursor=next_cursor
|
||||
)
|
||||
|
||||
|
||||
async def _resolve_thread(db: AsyncSession, user: User, thread_id: uuid.UUID) -> Thread:
|
||||
@@ -135,23 +162,40 @@ async def _resolve_thread(db: AsyncSession, user: User, thread_id: uuid.UUID) ->
|
||||
return thread
|
||||
|
||||
|
||||
# Trash/Spam are message-scoped folders: an individually deleted message lives
|
||||
# there on its own. Viewing a thread from Trash/Spam shows ONLY the messages filed
|
||||
# there; every other view shows the conversation with trashed/spam messages hidden.
|
||||
_REMOVAL_FOLDERS = frozenset({"trash", "junk"})
|
||||
|
||||
|
||||
@router.get("/{thread_id}", response_model=ThreadDetail)
|
||||
async def get_thread(
|
||||
thread_id: uuid.UUID,
|
||||
user: CurrentUser,
|
||||
db: DbSession,
|
||||
sort: str = Query("asc"),
|
||||
view: str = Query("unified"),
|
||||
) -> ThreadDetail:
|
||||
thread = await _resolve_thread(db, user, thread_id)
|
||||
current_special = view.split(":", 1)[1] if view.startswith("folder:") else None
|
||||
# Messages are always returned ASC (oldest first); the client reverses for
|
||||
# display per the requested order (invariant I-5 / §9.4).
|
||||
messages = list(
|
||||
await db.scalars(
|
||||
select(Message)
|
||||
.where(Message.thread_id == thread_id, Message.user_id == user.id)
|
||||
.order_by(Message.internal_date.asc(), Message.id.asc())
|
||||
)
|
||||
stmt = (
|
||||
select(Message)
|
||||
.join(Mailbox, Mailbox.id == Message.mailbox_id)
|
||||
.where(Message.thread_id == thread_id, Message.user_id == user.id)
|
||||
.order_by(Message.internal_date.asc(), Message.id.asc())
|
||||
)
|
||||
if current_special in _REMOVAL_FOLDERS:
|
||||
# Show only the messages actually in this Trash/Spam folder.
|
||||
stmt = stmt.where(Mailbox.special_use == current_special)
|
||||
else:
|
||||
# Conversation view: hide messages that were individually trashed or spammed
|
||||
# (keep custom folders, special_use IS NULL, and every non-removal folder).
|
||||
stmt = stmt.where(
|
||||
or_(Mailbox.special_use.is_(None), Mailbox.special_use.notin_(_REMOVAL_FOLDERS))
|
||||
)
|
||||
messages = list(await db.scalars(stmt))
|
||||
order = "desc" if sort == "desc" else "asc"
|
||||
return ThreadDetail(
|
||||
id=thread.id,
|
||||
|
||||
@@ -147,6 +147,11 @@ class Message(Base):
|
||||
mailbox_id: Mapped[uuid.UUID] = mapped_column(
|
||||
UUID(as_uuid=True), ForeignKey("mailboxes.id", ondelete="CASCADE"), nullable=False
|
||||
)
|
||||
# The folder this message was in before being moved to Archive/Trash/Junk, so
|
||||
# Restore can return it there (e.g. Sent) instead of the Inbox. NULL otherwise.
|
||||
restore_mailbox_id: Mapped[uuid.UUID | None] = mapped_column(
|
||||
UUID(as_uuid=True), ForeignKey("mailboxes.id", ondelete="SET NULL")
|
||||
)
|
||||
uid: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
thread_id: Mapped[uuid.UUID | None] = mapped_column(
|
||||
UUID(as_uuid=True), ForeignKey("threads.id")
|
||||
|
||||
@@ -24,6 +24,7 @@ from app.indexer.indexer import persist_message
|
||||
from app.models import Account, Draft, KnownRecipient, Mailbox, Message, Outbox
|
||||
from app.outbox.attachments import load_attachments
|
||||
from app.outbox.mime_build import build_mime
|
||||
from app.outbox.service import delete_mirrored_draft_row
|
||||
from app.providers.base import RemoteMessage
|
||||
from app.search.factory import get_search_backend
|
||||
from app.sync.provider_factory import build_provider
|
||||
@@ -178,6 +179,12 @@ async def send_job(ctx: dict[str, Any], outbox_id: str) -> None:
|
||||
draft_row = await session.get(Draft, draft.id)
|
||||
if draft_row is not None:
|
||||
await session.delete(draft_row)
|
||||
# Also drop the mirrored Drafts-folder copy's DB row (we expunged it on the
|
||||
# server above) so it doesn't resurface as a "remote" draft.
|
||||
if draft_imap_uid:
|
||||
await delete_mirrored_draft_row(
|
||||
session, account_id=account_obj.id, imap_uid=draft_imap_uid
|
||||
)
|
||||
await _harvest_recipients(session, user_id, envelope_to)
|
||||
|
||||
bus = get_event_bus()
|
||||
|
||||
@@ -9,7 +9,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.config import get_settings
|
||||
from app.models import Account, Draft, Outbox
|
||||
from app.models import Account, Draft, Mailbox, Message, Outbox
|
||||
from app.outbox.schemas import DraftIn
|
||||
|
||||
|
||||
@@ -17,6 +17,29 @@ class DraftError(Exception):
|
||||
"""Draft could not be created/updated (e.g. account not owned)."""
|
||||
|
||||
|
||||
async def delete_mirrored_draft_row(
|
||||
session: AsyncSession, *, account_id: uuid.UUID, imap_uid: int
|
||||
) -> None:
|
||||
"""Delete the DB row for a draft that was mirrored into the IMAP Drafts folder.
|
||||
|
||||
The server copy is expunged separately (send_job / remove_imap_draft_job).
|
||||
Without also dropping this row the draft resurfaces in ``list_remote_drafts``
|
||||
as an "other client" draft once its local ``drafts`` row is gone — until the
|
||||
next Drafts poll would catch the expunge. Used on both send and discard.
|
||||
"""
|
||||
mirror = await session.scalar(
|
||||
select(Message)
|
||||
.join(Mailbox, Mailbox.id == Message.mailbox_id)
|
||||
.where(
|
||||
Message.account_id == account_id,
|
||||
Message.uid == imap_uid,
|
||||
Mailbox.special_use == "drafts",
|
||||
)
|
||||
)
|
||||
if mirror is not None:
|
||||
await session.delete(mirror)
|
||||
|
||||
|
||||
async def upsert_draft(
|
||||
session: AsyncSession, *, user_id: uuid.UUID, body: DraftIn, draft_id: uuid.UUID | None
|
||||
) -> Draft:
|
||||
|
||||
@@ -83,6 +83,10 @@ class MailProvider(Protocol):
|
||||
otherwise UID SEARCH + FETCH FLAGS diff."""
|
||||
...
|
||||
|
||||
async def list_uids(self, mailbox: str) -> set[int]:
|
||||
"""Full set of UIDs currently present in the mailbox (expunge reconciliation)."""
|
||||
...
|
||||
|
||||
async def idle(self, mailbox: str) -> None:
|
||||
"""Blocks until the server signals change or 25 min elapse (re-issue IDLE). May raise
|
||||
ProviderNotSupported => sync loop falls back to poll() every POLL_WARM (default 180 s).
|
||||
|
||||
@@ -198,6 +198,18 @@ class ImapSmtpProvider(MailProvider):
|
||||
if messages:
|
||||
yield messages
|
||||
|
||||
async def list_uids(self, mailbox: str) -> set[int]:
|
||||
"""Full set of UIDs currently present in the mailbox (expunge reconciliation).
|
||||
|
||||
Cheap — a single ``UID SEARCH ALL``. The CONDSTORE poll only reports new and
|
||||
changed messages, not vanished ones (that needs QRESYNC/VANISHED); callers use
|
||||
this to detect server-side expunges for small folders like Drafts.
|
||||
"""
|
||||
await self._select(mailbox)
|
||||
imap = await self._connect()
|
||||
resp = await imap.uid_search("ALL")
|
||||
return set(parse_search_uids(resp.lines))
|
||||
|
||||
async def _fetch_messages(self, imap: aioimaplib.IMAP4, uids: list[int]) -> list[RemoteMessage]:
|
||||
if not uids:
|
||||
return []
|
||||
|
||||
@@ -32,6 +32,10 @@ _MAX_TRIES = 5
|
||||
# Folder names to create when a target special-use folder is missing on the server.
|
||||
_DEFAULT_FOLDER_NAME = {"archive": "Archive", "trash": "Trash", "junk": "Spam"}
|
||||
|
||||
# Moves into these folders record where the message came from (restore_mailbox_id)
|
||||
# so the Restore action can return it to its original folder.
|
||||
_REMOVAL_FOLDERS = frozenset({"archive", "trash", "junk"})
|
||||
|
||||
|
||||
def sanitize_flags(flags: Iterable[str]) -> frozenset[str]:
|
||||
return frozenset(f for f in flags if f in ALLOWED_FLAGS)
|
||||
@@ -214,6 +218,9 @@ async def move_thread_to_folder_job(ctx: dict[str, Any], thread_id: str, special
|
||||
)
|
||||
message = await session.get(Message, mid)
|
||||
if message is not None and resolved is not None:
|
||||
# Remember where it came from so Restore returns it there, not the Inbox.
|
||||
if special_use in _REMOVAL_FOLDERS and message.mailbox_id != resolved:
|
||||
message.restore_mailbox_id = message.mailbox_id
|
||||
message.mailbox_id = resolved
|
||||
message.uid = new_uid
|
||||
|
||||
@@ -222,6 +229,74 @@ async def move_thread_to_folder_job(ctx: dict[str, Any], thread_id: str, special
|
||||
)
|
||||
|
||||
|
||||
async def restore_thread_job(ctx: dict[str, Any], thread_id: str) -> None:
|
||||
"""Move every message in a thread back to its original folder (Restore action).
|
||||
|
||||
Each message returns to ``restore_mailbox_id`` (the folder it was in before it
|
||||
was archived/trashed) — e.g. a deleted Sent message goes back to Sent. Messages
|
||||
with no recorded origin fall back to the account's Inbox.
|
||||
"""
|
||||
tid = uuid.UUID(thread_id)
|
||||
async with session_scope() as session:
|
||||
messages = list(await session.scalars(select(Message).where(Message.thread_id == tid)))
|
||||
if not messages:
|
||||
return
|
||||
user_id = messages[0].user_id
|
||||
account_ids = list({m.account_id for m in messages})
|
||||
accounts = {
|
||||
a.id: a
|
||||
for a in await session.scalars(select(Account).where(Account.id.in_(account_ids)))
|
||||
}
|
||||
mailboxes = list(
|
||||
await session.scalars(select(Mailbox).where(Mailbox.account_id.in_(account_ids)))
|
||||
)
|
||||
path_by_id = {mb.id: mb.path for mb in mailboxes}
|
||||
inbox_by_account = {
|
||||
mb.account_id: mb.id for mb in mailboxes if mb.special_use == "inbox"
|
||||
}
|
||||
# (message_id, account_id, source_path, target_id) — resolve the target now.
|
||||
plan: list[tuple[uuid.UUID, uuid.UUID, str, uuid.UUID, int]] = []
|
||||
for m in messages:
|
||||
target_id = m.restore_mailbox_id
|
||||
if target_id is None or target_id not in path_by_id:
|
||||
target_id = inbox_by_account.get(m.account_id)
|
||||
if target_id is None or target_id == m.mailbox_id:
|
||||
continue # nowhere to restore to, or already there
|
||||
plan.append(
|
||||
(m.id, m.account_id, path_by_id.get(m.mailbox_id, "INBOX"), target_id, m.uid)
|
||||
)
|
||||
|
||||
# (message_id, target_id, new_uid) applied after the network moves.
|
||||
moves: list[tuple[uuid.UUID, uuid.UUID, int]] = []
|
||||
for account_id in account_ids:
|
||||
provider = build_provider(accounts[account_id])
|
||||
try:
|
||||
for mid, m_account, src_path, target_id, uid in plan:
|
||||
if m_account != account_id:
|
||||
continue
|
||||
target_path = path_by_id[target_id]
|
||||
try:
|
||||
new_uid = await provider.move(uid, src_path, target_path)
|
||||
except Exception:
|
||||
logger.exception("restore move failed for message %s -> %s", mid, target_path)
|
||||
continue
|
||||
moves.append((mid, target_id, new_uid))
|
||||
finally:
|
||||
await provider.close()
|
||||
|
||||
async with session_scope() as session:
|
||||
for mid, target_id, new_uid in moves:
|
||||
message = await session.get(Message, mid)
|
||||
if message is not None:
|
||||
message.mailbox_id = target_id
|
||||
message.uid = new_uid
|
||||
message.restore_mailbox_id = None
|
||||
|
||||
await get_event_bus().publish(
|
||||
"thread.updated", {"user_id": str(user_id), "thread_id": str(tid)}
|
||||
)
|
||||
|
||||
|
||||
async def delete_thread_job(ctx: dict[str, Any], thread_id: str) -> None:
|
||||
"""Permanently delete every message in a thread (server EXPUNGE + DB rows).
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.config import get_settings
|
||||
from app.events import get_event_bus
|
||||
from app.indexer.indexer import PersistResult, persist_message
|
||||
from app.models import Account, Mailbox, Message
|
||||
from app.models import Account, Draft, Mailbox, Message
|
||||
from app.providers.base import MailboxDelta, ProviderNotSupported
|
||||
from app.providers.imap_smtp import ImapSmtpProvider
|
||||
from app.search.factory import get_search_backend
|
||||
@@ -248,6 +248,49 @@ async def apply_delta(
|
||||
},
|
||||
)
|
||||
|
||||
# Expunge reconciliation for Drafts. The mirror mechanism (and other clients)
|
||||
# expunge server copies, but a CONDSTORE poll doesn't report vanished UIDs, so
|
||||
# expunged drafts would linger in the DB and resurface as phantom "remote"
|
||||
# drafts. Drafts folders are small, so a full UID reconcile per poll is cheap.
|
||||
if mailbox.special_use == "drafts":
|
||||
await _reconcile_drafts(provider, account, mailbox)
|
||||
|
||||
|
||||
async def _reconcile_drafts(
|
||||
provider: ImapSmtpProvider, account: Account, mailbox: Mailbox
|
||||
) -> None:
|
||||
"""Drop DB rows for drafts that no longer exist on the server (were expunged)."""
|
||||
from app.db import session_scope
|
||||
|
||||
try:
|
||||
server_uids = await provider.list_uids(mailbox.path)
|
||||
except Exception:
|
||||
logger.exception("drafts reconcile: list_uids failed for %s", mailbox.path)
|
||||
return
|
||||
|
||||
async with session_scope() as session:
|
||||
db_uids = set(
|
||||
await session.scalars(select(Message.uid).where(Message.mailbox_id == mailbox.id))
|
||||
)
|
||||
gone = db_uids - server_uids
|
||||
if not gone:
|
||||
return
|
||||
await session.execute(
|
||||
delete(Message).where(Message.mailbox_id == mailbox.id, Message.uid.in_(gone))
|
||||
)
|
||||
# A local draft whose mirrored copy vanished becomes local-only again.
|
||||
await session.execute(
|
||||
update(Draft)
|
||||
.where(Draft.account_id == account.id, Draft.imap_uid.in_(gone))
|
||||
.values(imap_uid=None)
|
||||
)
|
||||
|
||||
logger.info("drafts reconcile: removed %d expunged row(s) in %s", len(gone), mailbox.path)
|
||||
await get_event_bus().publish(
|
||||
"message.deleted",
|
||||
{"user_id": str(account.user_id), "account_id": str(account.id), "mailbox": mailbox.path},
|
||||
)
|
||||
|
||||
|
||||
async def live_loop(
|
||||
account_id: uuid.UUID, mailbox_path: str = "INBOX", stop: asyncio.Event | None = None
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
"""add messages.restore_mailbox_id
|
||||
|
||||
Records the folder a message was in before it was moved to Archive/Trash/Junk, so
|
||||
the Restore action can return it to where it came from (e.g. Sent) instead of
|
||||
always dumping it in the Inbox.
|
||||
|
||||
Revision ID: 0010
|
||||
Revises: 0009
|
||||
Create Date: 2026-07-20
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0010"
|
||||
down_revision: str | None = "0009"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
"ALTER TABLE messages ADD COLUMN restore_mailbox_id uuid "
|
||||
"REFERENCES mailboxes(id) ON DELETE SET NULL"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute("ALTER TABLE messages DROP COLUMN IF EXISTS restore_mailbox_id")
|
||||
@@ -254,6 +254,50 @@ async def test_thread_detail_messages_ascending(api: ApiCtx) -> None:
|
||||
assert dates == sorted(dates) # ascending
|
||||
|
||||
|
||||
async def test_thread_detail_hides_trashed_message_unless_viewing_trash(api: ApiCtx) -> None:
|
||||
assert api.session is not None
|
||||
# Give the account a Trash folder and delete a single message of the 2-message
|
||||
# kickoff thread (move it to Trash) — as the per-message Delete action does.
|
||||
trash = Mailbox(
|
||||
account_id=api.account_id, user_id=api.user_id, path="Trash", special_use="trash"
|
||||
)
|
||||
api.session.add(trash)
|
||||
await api.session.flush()
|
||||
resp = await api.client.get("/api/threads")
|
||||
kickoff = next(t for t in resp.json()["threads"] if t["message_count"] == 2)
|
||||
detail = (await api.client.get(f"/api/threads/{kickoff['id']}")).json()
|
||||
victim = uuid.UUID(detail["messages"][-1]["id"])
|
||||
msg = await api.session.get(Message, victim)
|
||||
assert msg is not None
|
||||
msg.mailbox_id = trash.id
|
||||
await api.session.commit()
|
||||
|
||||
# Viewing the thread from the Inbox: the trashed message is gone.
|
||||
inbox_view = await api.client.get(
|
||||
f"/api/threads/{kickoff['id']}", params={"view": "folder:inbox"}
|
||||
)
|
||||
ids = [m["id"] for m in inbox_view.json()["messages"]]
|
||||
assert str(victim) not in ids
|
||||
assert len(ids) == 1
|
||||
|
||||
# Viewing from the Trash folder: only the trashed message is shown (Trash is
|
||||
# message-scoped), not the rest of the conversation.
|
||||
trash_view = await api.client.get(
|
||||
f"/api/threads/{kickoff['id']}", params={"view": "folder:trash"}
|
||||
)
|
||||
ids_trash = [m["id"] for m in trash_view.json()["messages"]]
|
||||
assert ids_trash == [str(victim)]
|
||||
|
||||
# The Inbox LIST count drops the trashed message (was 2, now 1); the Trash list
|
||||
# counts only the one trashed message.
|
||||
inbox_list = await api.client.get("/api/threads", params={"view": "folder:inbox"})
|
||||
row = next(t for t in inbox_list.json()["threads"] if t["id"] == kickoff["id"])
|
||||
assert row["message_count"] == 1
|
||||
trash_list = await api.client.get("/api/threads", params={"view": "folder:trash"})
|
||||
trow = next(t for t in trash_list.json()["threads"] if t["id"] == kickoff["id"])
|
||||
assert trow["message_count"] == 1
|
||||
|
||||
|
||||
async def test_message_html_is_sanitized(api: ApiCtx) -> None:
|
||||
# the lunch message has HTML with a script
|
||||
resp = await api.client.get("/api/threads")
|
||||
|
||||
@@ -225,14 +225,20 @@ async def test_archive_unknown_thread_404(ctx: Ctx) -> None:
|
||||
assert ctx.enqueued == []
|
||||
|
||||
|
||||
async def test_restore_thread_enqueues_move_to_inbox(ctx: Ctx) -> None:
|
||||
async def test_restore_thread_enqueues_restore_job(ctx: Ctx) -> None:
|
||||
resp = await ctx.client.post(f"/api/threads/{ctx.thread_id}/restore", headers=_hdr())
|
||||
assert resp.status_code == 200
|
||||
assert ctx.enqueued == [
|
||||
("move_thread_to_folder_job", str(ctx.thread_id), "inbox"),
|
||||
("restore_thread_job", str(ctx.thread_id)),
|
||||
]
|
||||
|
||||
|
||||
async def test_restore_unknown_thread_404(ctx: Ctx) -> None:
|
||||
resp = await ctx.client.post(f"/api/threads/{uuid.uuid4()}/restore", headers=_hdr())
|
||||
assert resp.status_code == 404
|
||||
assert ctx.enqueued == []
|
||||
|
||||
|
||||
async def test_delete_thread_enqueues_permanent_delete(ctx: Ctx) -> None:
|
||||
resp = await ctx.client.post(f"/api/threads/{ctx.thread_id}/delete", headers=_hdr())
|
||||
assert resp.status_code == 200
|
||||
|
||||
@@ -274,6 +274,73 @@ async def test_remote_drafts_listed_and_adopted(ctx: Ctx) -> None:
|
||||
assert again.json()["id"] == draft["id"]
|
||||
|
||||
|
||||
async def test_adopt_remote_draft_carries_attachments(ctx: Ctx) -> None:
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from app.indexer import blobstore
|
||||
from app.models import Attachment
|
||||
|
||||
drafts_mb = Mailbox(
|
||||
account_id=ctx.account_id, user_id=ctx.user_id, path="DRAFTS", special_use="drafts"
|
||||
)
|
||||
ctx.session.add(drafts_mb)
|
||||
await ctx.session.flush()
|
||||
msg = Message(
|
||||
user_id=ctx.user_id,
|
||||
account_id=ctx.account_id,
|
||||
mailbox_id=drafts_mb.id,
|
||||
uid=95,
|
||||
subject="Home draft with file",
|
||||
to_addr=[{"name": "", "email": "bob@example.com"}],
|
||||
cc_addr=[],
|
||||
bcc_addr=[],
|
||||
internal_date=datetime.now(UTC),
|
||||
blob_sha256="y" * 64,
|
||||
snippet="",
|
||||
has_attachments=True,
|
||||
)
|
||||
ctx.session.add(msg)
|
||||
await ctx.session.flush()
|
||||
# A real (downloadable) attachment plus an inline cid image, which must NOT be carried.
|
||||
sha = blobstore.store(ctx.user_id, b"file contents")
|
||||
ctx.session.add(
|
||||
Attachment(
|
||||
message_id=msg.id,
|
||||
user_id=ctx.user_id,
|
||||
filename="report.pdf",
|
||||
mime_type="application/pdf",
|
||||
size_bytes=13,
|
||||
content_id=None,
|
||||
blob_sha256=sha,
|
||||
)
|
||||
)
|
||||
ctx.session.add(
|
||||
Attachment(
|
||||
message_id=msg.id,
|
||||
user_id=ctx.user_id,
|
||||
filename="logo.png",
|
||||
mime_type="image/png",
|
||||
size_bytes=3,
|
||||
content_id="logo@x",
|
||||
blob_sha256=blobstore.store(ctx.user_id, b"img"),
|
||||
)
|
||||
)
|
||||
await ctx.session.commit()
|
||||
|
||||
adopted = await ctx.client.post(f"/api/drafts/adopt/{msg.id}", headers=_hdr())
|
||||
assert adopted.status_code == 200, adopted.text
|
||||
ids = adopted.json()["payload"]["attachment_ids"]
|
||||
assert len(ids) == 1 # only the real attachment, not the inline image
|
||||
|
||||
# The compose-attachment metadata resolves — this is what the editor renders and
|
||||
# what load_attachments re-attaches on send.
|
||||
meta = await ctx.client.get("/api/attachments", params={"ids": ",".join(ids)})
|
||||
assert meta.status_code == 200
|
||||
body = meta.json()
|
||||
assert [a["filename"] for a in body] == ["report.pdf"]
|
||||
assert body[0]["size_bytes"] == 13
|
||||
|
||||
|
||||
async def test_publish_draft_enqueues_imap_mirror(ctx: Ctx) -> None:
|
||||
did = (await ctx.client.post("/api/drafts", json=_draft_body(ctx), headers=_hdr())).json()["id"]
|
||||
ctx.enqueued.clear()
|
||||
|
||||
@@ -13,7 +13,7 @@ from sqlalchemy.pool import NullPool
|
||||
from app.config import get_settings
|
||||
from app.db import session_scope
|
||||
from app.identity.passwords import hash_password
|
||||
from app.models import Account, Mailbox, Message, User
|
||||
from app.models import Account, Draft, Mailbox, Message, User
|
||||
from app.providers.base import MailboxDelta, RemoteMessage
|
||||
from app.sync import engine as sync_engine
|
||||
|
||||
@@ -97,3 +97,73 @@ async def test_secondary_poll_syncs_special_folders_not_inbox(monkeypatch) -> No
|
||||
await cleanup.execute(delete(User).where(User.id == user.id))
|
||||
await session.close()
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
async def test_drafts_poll_reconciles_expunged_rows(monkeypatch) -> None: # type: ignore[no-untyped-def]
|
||||
"""A Drafts poll drops DB rows whose server copy was expunged (phantoms) and
|
||||
un-links a local draft whose mirrored copy vanished."""
|
||||
engine = create_async_engine(get_settings().database_url, poolclass=NullPool)
|
||||
sm = async_sessionmaker(engine, expire_on_commit=False)
|
||||
session = sm()
|
||||
user = User(handle=f"rec{uuid.uuid4().hex[:8]}@test.local", password_hash=hash_password("pw"))
|
||||
session.add(user)
|
||||
await session.flush()
|
||||
account = Account(
|
||||
user_id=user.id,
|
||||
email="me@example.com",
|
||||
imap_config={},
|
||||
smtp_config={},
|
||||
secret_ct=b"",
|
||||
discovery_src="manual",
|
||||
)
|
||||
session.add(account)
|
||||
await session.flush()
|
||||
drafts = Mailbox(account_id=account.id, user_id=user.id, path="DRAFTS", special_use="drafts")
|
||||
session.add(drafts)
|
||||
await session.flush()
|
||||
|
||||
# uid 2 survives on the server; uids 1 and 3 were expunged (phantom DB rows).
|
||||
for uid in (1, 2, 3):
|
||||
session.add(
|
||||
Message(
|
||||
user_id=user.id,
|
||||
account_id=account.id,
|
||||
mailbox_id=drafts.id,
|
||||
uid=uid,
|
||||
internal_date=datetime.now(UTC),
|
||||
blob_sha256="",
|
||||
)
|
||||
)
|
||||
# A local draft still linking the vanished uid 3 must be un-linked (imap_uid -> NULL).
|
||||
linked = Draft(user_id=user.id, account_id=account.id, kind="new", payload={}, imap_uid=3)
|
||||
kept = Draft(user_id=user.id, account_id=account.id, kind="new", payload={}, imap_uid=2)
|
||||
session.add_all([linked, kept])
|
||||
await session.commit()
|
||||
|
||||
class FakeProvider:
|
||||
async def poll(self, mailbox: str, *, last_seen_uid: int, highestmodseq: int | None): # type: ignore[no-untyped-def]
|
||||
return MailboxDelta([], [], [], None, None)
|
||||
|
||||
async def list_uids(self, mailbox: str) -> set[int]:
|
||||
return {2} # only uid 2 remains on the server
|
||||
|
||||
async def close(self) -> None:
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(sync_engine, "build_provider", lambda _a: FakeProvider())
|
||||
|
||||
try:
|
||||
await sync_engine._poll_secondary_once(account.id)
|
||||
|
||||
session.expunge_all()
|
||||
remaining = set(
|
||||
await session.scalars(select(Message.uid).where(Message.mailbox_id == drafts.id))
|
||||
)
|
||||
assert remaining == {2} # phantoms 1 and 3 removed
|
||||
assert (await session.get(Draft, linked.id)).imap_uid is None # un-linked
|
||||
assert (await session.get(Draft, kept.id)).imap_uid == 2 # untouched
|
||||
finally:
|
||||
async with session_scope() as cleanup:
|
||||
await cleanup.execute(delete(User).where(User.id == user.id))
|
||||
await session.close()
|
||||
await engine.dispose()
|
||||
|
||||
@@ -19,6 +19,7 @@ from app.sync.actions import (
|
||||
move_job,
|
||||
move_thread_to_folder_job,
|
||||
push_flags_job,
|
||||
restore_thread_job,
|
||||
)
|
||||
from app.sync.engine import bootstrap_and_backfill, refresh_account_job
|
||||
|
||||
@@ -38,6 +39,7 @@ INTERACTIVE_FUNCTIONS = [
|
||||
push_flags_job,
|
||||
move_job,
|
||||
move_thread_to_folder_job,
|
||||
restore_thread_job,
|
||||
delete_thread_job,
|
||||
send_job,
|
||||
push_draft_to_imap_job,
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="64" height="64" viewBox="0 0 24 24" style="color: rgb(74, 85, 101); opacity: 1; transform: rotate(0deg);"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="m4 4l16 16m-3.5-3.244C15.147 17.485 13.618 18 12 18c-3.53 0-6.634-2.452-8.413-4.221c-.47-.467-.705-.7-.854-1.159c-.107-.327-.107-.913 0-1.24c.15-.459.385-.693.855-1.16c.897-.892 2.13-1.956 3.584-2.793M19.5 14.634c.333-.293.638-.582.912-.854l.003-.003c.468-.466.703-.7.852-1.156c.107-.327.107-.914 0-1.241c-.15-.458-.384-.692-.854-1.159C18.633 8.452 15.531 6 12 6q-.507 0-1 .064m2.323 7.436a2 2 0 0 1-2.762-2.889"></path></svg>
|
||||
|
After Width: | Height: | Size: 759 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="64" height="64" viewBox="0 0 24 24" style="color: rgb(74, 85, 101); opacity: 1; transform: rotate(0deg);"><g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"><path d="M3.587 13.779c1.78 1.769 4.883 4.22 8.413 4.22s6.634-2.451 8.413-4.22c.47-.467.705-.7.854-1.159c.107-.327.107-.913 0-1.24c-.15-.458-.385-.692-.854-1.159C18.633 8.452 15.531 6 12 6c-3.53 0-6.634 2.452-8.413 4.221c-.47.467-.705.7-.854 1.159c-.107.327-.107.913 0 1.24c.15.458.384.692.854 1.159"></path><path d="M10 12a2 2 0 1 0 4 0a2 2 0 0 0-4 0"></path></g></svg>
|
||||
|
After Width: | Height: | Size: 697 B |
@@ -8,12 +8,16 @@ import compose from "../../icons/compose.svg?raw";
|
||||
import drafts from "../../icons/drafts-28-regular.svg?raw";
|
||||
import expandAll from "../../icons/expand-all.svg?raw";
|
||||
import forward from "../../icons/forward.svg?raw";
|
||||
import hide from "../../icons/hide.svg?raw";
|
||||
import inbox from "../../icons/inbox.svg?raw";
|
||||
import logOut from "../../icons/log-out.svg?raw";
|
||||
import pin from "../../icons/pin.svg?raw";
|
||||
import print from "../../icons/print.svg?raw";
|
||||
import replyAll from "../../icons/reply-all-line.svg?raw";
|
||||
import reply from "../../icons/reply.svg?raw";
|
||||
import sent from "../../icons/sent-02.svg?raw";
|
||||
import settings from "../../icons/settings.svg?raw";
|
||||
import show from "../../icons/show.svg?raw";
|
||||
import spam from "../../icons/spam.svg?raw";
|
||||
import trash from "../../icons/trash.svg?raw";
|
||||
|
||||
@@ -24,12 +28,16 @@ const RAW: Record<string, string> = {
|
||||
drafts,
|
||||
"expand-all": expandAll,
|
||||
forward,
|
||||
hide,
|
||||
inbox,
|
||||
"log-out": logOut,
|
||||
pin,
|
||||
print,
|
||||
reply,
|
||||
"reply-all": replyAll,
|
||||
sent,
|
||||
settings,
|
||||
show,
|
||||
spam,
|
||||
trash,
|
||||
};
|
||||
|
||||
@@ -137,16 +137,37 @@ export function Composer({
|
||||
const [showToolbar, setShowToolbar] = useState(true);
|
||||
const [fromOpen, setFromOpen] = useState(false);
|
||||
const [moreOpen, setMoreOpen] = useState(false);
|
||||
const moreRef = useRef<HTMLDivElement>(null);
|
||||
const fromRef = useRef<HTMLDivElement>(null);
|
||||
const [attachments, setAttachments] = useState<AttachmentOut[]>([]);
|
||||
const [attachError, setAttachError] = useState("");
|
||||
const [status, setStatus] = useState<"" | "saving" | "saved" | "sending" | "error">("");
|
||||
const [undo, setUndo] = useState<{ outboxId: string } | null>(null);
|
||||
const [showClosePrompt, setShowClosePrompt] = useState(false);
|
||||
const draftId = useRef<string | null>(init.draftId ?? null);
|
||||
const sentRef = useRef(false); // send initiated -> don't mirror to Drafts on close
|
||||
// Whether the user actually edited anything. Merely opening a draft to view it
|
||||
// must not re-save/re-mirror it (that spawned duplicate IMAP copies).
|
||||
const dirtyRef = useRef(false);
|
||||
const markDirty = () => {
|
||||
dirtyRef.current = true;
|
||||
};
|
||||
const undoTimer = useRef<number | null>(null);
|
||||
const fileInputRef = useRef<HTMLInputElement>(null);
|
||||
const editorRef = useRef<RichTextEditorHandle>(null);
|
||||
|
||||
// Close the From / More dropdowns on any click outside them.
|
||||
useEffect(() => {
|
||||
if (!moreOpen && !fromOpen) return;
|
||||
const onDown = (e: MouseEvent) => {
|
||||
const target = e.target as Node;
|
||||
if (moreOpen && moreRef.current && !moreRef.current.contains(target)) setMoreOpen(false);
|
||||
if (fromOpen && fromRef.current && !fromRef.current.contains(target)) setFromOpen(false);
|
||||
};
|
||||
document.addEventListener("mousedown", onDown);
|
||||
return () => document.removeEventListener("mousedown", onDown);
|
||||
}, [moreOpen, fromOpen]);
|
||||
|
||||
const account = accounts.find((a) => a.id === accountId);
|
||||
const fromName = account?.display_name || "";
|
||||
const fromEmail = account?.email ?? "";
|
||||
@@ -204,6 +225,7 @@ export function Composer({
|
||||
}, []);
|
||||
|
||||
const addFiles = async (files: FileList) => {
|
||||
markDirty();
|
||||
setAttachError("");
|
||||
for (const file of Array.from(files)) {
|
||||
if (file.size > MAX_ATTACHMENT_BYTES) {
|
||||
@@ -219,8 +241,10 @@ export function Composer({
|
||||
}
|
||||
};
|
||||
|
||||
const removeAttachment = (id: string) =>
|
||||
const removeAttachment = (id: string) => {
|
||||
markDirty();
|
||||
setAttachments((prev) => prev.filter((a) => a.id !== id));
|
||||
};
|
||||
|
||||
// The editor's seed content. New messages get the account signature (2 blank
|
||||
// lines above it); replies keep the quoted original in a marked block so the
|
||||
@@ -263,6 +287,7 @@ export function Composer({
|
||||
useEffect(() => {
|
||||
if (undo) return;
|
||||
const handle = window.setTimeout(async () => {
|
||||
if (!dirtyRef.current) return; // nothing the user changed -> don't touch the draft
|
||||
if (!to && !subject && !bodyHtml) return;
|
||||
setStatus("saving");
|
||||
try {
|
||||
@@ -284,8 +309,9 @@ export function Composer({
|
||||
() => () => {
|
||||
if (undoTimer.current) window.clearTimeout(undoTimer.current);
|
||||
// On close, mirror the draft to the IMAP Drafts folder so it can be
|
||||
// continued on any mail client (skip if it was sent or discarded).
|
||||
if (draftId.current && !sentRef.current) {
|
||||
// continued on any mail client — but only if the user actually edited it
|
||||
// (skip if it was sent, discarded, or merely opened to view).
|
||||
if (draftId.current && !sentRef.current && dirtyRef.current) {
|
||||
void api.publishDraft(draftId.current).catch(() => {});
|
||||
}
|
||||
},
|
||||
@@ -333,6 +359,26 @@ export function Composer({
|
||||
onClose();
|
||||
};
|
||||
|
||||
// X button: if there's a draft worth keeping, ask before closing; otherwise
|
||||
// (nothing typed, no draft) just close.
|
||||
const requestClose = () => {
|
||||
if (dirtyRef.current || draftId.current) setShowClosePrompt(true);
|
||||
else onClose();
|
||||
};
|
||||
|
||||
const saveDraftAndClose = async () => {
|
||||
setShowClosePrompt(false);
|
||||
try {
|
||||
if (dirtyRef.current) {
|
||||
if (draftId.current) await api.updateDraft(draftId.current, payload());
|
||||
else draftId.current = (await api.createDraft(payload())).id;
|
||||
}
|
||||
} catch {
|
||||
/* the close-time IMAP mirror is best-effort too */
|
||||
}
|
||||
onClose(); // the unmount effect mirrors the (dirty) draft to the IMAP Drafts folder
|
||||
};
|
||||
|
||||
const popOut = async () => {
|
||||
// Persist first so the new window can load this exact draft.
|
||||
try {
|
||||
@@ -367,12 +413,44 @@ export function Composer({
|
||||
|
||||
return (
|
||||
<div
|
||||
className={`flex flex-col overflow-hidden border-app-border bg-app-elevated ${
|
||||
className={`relative flex flex-col overflow-hidden border-app-border bg-app-elevated ${
|
||||
fullWindow
|
||||
? "h-full"
|
||||
: `rounded-xl border shadow-2xl ${expanded ? "h-full" : ""}`
|
||||
}`}
|
||||
>
|
||||
{showClosePrompt && (
|
||||
<div
|
||||
className="absolute inset-0 z-40 flex items-center justify-center bg-black/40 p-4"
|
||||
onClick={() => setShowClosePrompt(false)}
|
||||
>
|
||||
<div
|
||||
className="w-72 rounded-lg border border-app-border bg-app-elevated p-4 shadow-2xl"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<p className="mb-4 text-sm text-app-fg">{t("compose.closePrompt")}</p>
|
||||
<div className="flex justify-end gap-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => {
|
||||
setShowClosePrompt(false);
|
||||
void discard();
|
||||
}}
|
||||
className="rounded-md px-3 py-1.5 text-sm text-red-500 hover:bg-app-hover"
|
||||
>
|
||||
{t("compose.discard")}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void saveDraftAndClose()}
|
||||
className="rounded-md bg-app-accent px-3 py-1.5 text-sm font-medium text-app-accent-fg hover:opacity-90"
|
||||
>
|
||||
{t("compose.saveDraft")}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{/* Header: Send · Attach · formatting toggle · More · window controls */}
|
||||
<div className="flex items-center gap-1.5 border-b border-app-border bg-app-surface px-3 py-2">
|
||||
<button
|
||||
@@ -410,7 +488,7 @@ export function Composer({
|
||||
>
|
||||
<span className="text-sm font-semibold underline">A</span>
|
||||
</HeaderButton>
|
||||
<div className="relative">
|
||||
<div className="relative" ref={moreRef}>
|
||||
<HeaderButton onClick={() => setMoreOpen((v) => !v)} title={t("compose.more")}>
|
||||
{t("compose.more")}
|
||||
</HeaderButton>
|
||||
@@ -469,7 +547,7 @@ export function Composer({
|
||||
</svg>
|
||||
</HeaderButton>
|
||||
)}
|
||||
<HeaderButton onClick={onClose} title={t("common.close")}>
|
||||
<HeaderButton onClick={requestClose} title={t("common.close")}>
|
||||
<svg width="15" height="15" viewBox="0 0 16 16" fill="none" stroke="currentColor" strokeWidth="1.5" strokeLinecap="round">
|
||||
<path d="M4 4l8 8M12 4l-8 8" />
|
||||
</svg>
|
||||
@@ -478,7 +556,10 @@ export function Composer({
|
||||
</div>
|
||||
|
||||
{/* From */}
|
||||
<div className="relative flex items-center gap-2 border-b border-app-border px-3 py-2">
|
||||
<div
|
||||
className="relative flex items-center gap-2 border-b border-app-border px-3 py-2"
|
||||
ref={fromRef}
|
||||
>
|
||||
<span className="w-10 shrink-0 text-xs font-semibold text-app-fg">{t("compose.from")}</span>
|
||||
<button
|
||||
type="button"
|
||||
@@ -500,6 +581,7 @@ export function Composer({
|
||||
key={a.id}
|
||||
className="flex w-full items-center gap-2 px-3 py-1.5 text-left text-sm hover:bg-app-hover"
|
||||
onClick={() => {
|
||||
markDirty();
|
||||
setAccountId(a.id);
|
||||
setFromOpen(false);
|
||||
}}
|
||||
@@ -517,7 +599,10 @@ export function Composer({
|
||||
<RecipientChips
|
||||
label={t("compose.to")}
|
||||
value={to}
|
||||
onChange={setTo}
|
||||
onChange={(v) => {
|
||||
markDirty();
|
||||
setTo(v);
|
||||
}}
|
||||
autoFocus={init.kind === "new" || init.kind === "forward"}
|
||||
rightSlot={
|
||||
!showCc ? (
|
||||
@@ -533,15 +618,32 @@ export function Composer({
|
||||
/>
|
||||
{showCc && (
|
||||
<>
|
||||
<RecipientChips label={t("compose.cc")} value={cc} onChange={setCc} />
|
||||
<RecipientChips label={t("compose.bcc")} value={bcc} onChange={setBcc} />
|
||||
<RecipientChips
|
||||
label={t("compose.cc")}
|
||||
value={cc}
|
||||
onChange={(v) => {
|
||||
markDirty();
|
||||
setCc(v);
|
||||
}}
|
||||
/>
|
||||
<RecipientChips
|
||||
label={t("compose.bcc")}
|
||||
value={bcc}
|
||||
onChange={(v) => {
|
||||
markDirty();
|
||||
setBcc(v);
|
||||
}}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
<input
|
||||
className="border-b border-app-border bg-transparent px-3 py-2 text-sm text-app-fg outline-none placeholder:text-app-faint"
|
||||
placeholder={t("compose.subject")}
|
||||
value={subject}
|
||||
onChange={(e) => setSubject(e.target.value)}
|
||||
onChange={(e) => {
|
||||
markDirty();
|
||||
setSubject(e.target.value);
|
||||
}}
|
||||
/>
|
||||
|
||||
{(attachments.length > 0 || attachError) && (
|
||||
@@ -575,7 +677,10 @@ export function Composer({
|
||||
<RichTextEditor
|
||||
ref={editorRef}
|
||||
initialHtml={initialHtml}
|
||||
onInput={setBodyHtml}
|
||||
onInput={(html) => {
|
||||
markDirty();
|
||||
setBodyHtml(html);
|
||||
}}
|
||||
showToolbar={showToolbar}
|
||||
/>
|
||||
) : (
|
||||
|
||||
@@ -3,6 +3,7 @@ import { Fragment, useEffect, useMemo, useRef, useState } from "react";
|
||||
|
||||
import { useT } from "../../i18n";
|
||||
import { api } from "../../lib/api";
|
||||
import { printMessages } from "../../lib/print";
|
||||
import type { AccountOut, MailboxOut, MessageEnvelope } from "../../lib/types";
|
||||
import { Composer, type ComposerInit } from "../compose/Composer";
|
||||
import { Icon } from "../Icon";
|
||||
@@ -137,10 +138,12 @@ function ActionsMenu({
|
||||
onReply,
|
||||
onPrint,
|
||||
onDelete,
|
||||
replyDisabled = false,
|
||||
}: {
|
||||
onReply: (kind: string) => void;
|
||||
onPrint: () => void;
|
||||
onDelete: () => void;
|
||||
replyDisabled?: boolean;
|
||||
}) {
|
||||
const t = useT();
|
||||
const [open, setOpen] = useState(false);
|
||||
@@ -150,13 +153,25 @@ function ActionsMenu({
|
||||
const onDoc = (e: MouseEvent) => {
|
||||
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
|
||||
};
|
||||
// Clicking inside the sandboxed message-body iframe doesn't reach document
|
||||
// mousedown, but it blurs the parent window — close the menu on that too.
|
||||
const onBlur = () => setOpen(false);
|
||||
document.addEventListener("mousedown", onDoc);
|
||||
return () => document.removeEventListener("mousedown", onDoc);
|
||||
window.addEventListener("blur", onBlur);
|
||||
return () => {
|
||||
document.removeEventListener("mousedown", onDoc);
|
||||
window.removeEventListener("blur", onBlur);
|
||||
};
|
||||
}, [open]);
|
||||
|
||||
const item = (label: string, onClick: () => void) => (
|
||||
const item = (label: string, onClick: () => void, disabled = false) => (
|
||||
<button
|
||||
className="block w-full px-3 py-1.5 text-left text-xs text-app-fg hover:bg-app-hover"
|
||||
disabled={disabled}
|
||||
className={`block w-full px-3 py-1.5 text-left text-xs ${
|
||||
disabled
|
||||
? "cursor-not-allowed text-app-faint"
|
||||
: "text-app-fg hover:bg-app-hover"
|
||||
}`}
|
||||
onClick={() => {
|
||||
setOpen(false);
|
||||
onClick();
|
||||
@@ -177,10 +192,10 @@ function ActionsMenu({
|
||||
</button>
|
||||
{open && (
|
||||
<div className="absolute right-0 z-20 mt-1 w-52 overflow-hidden rounded-md border border-app-border bg-app-elevated py-1 shadow-lg">
|
||||
{item(t("mail.reply"), () => onReply("reply"))}
|
||||
{item(t("mail.replyAll"), () => onReply("reply_all"))}
|
||||
{item(t("mail.forward"), () => onReply("forward"))}
|
||||
{item(t("mail.forwardAsAttachment"), () => onReply("forward"))}
|
||||
{item(t("mail.reply"), () => onReply("reply"), replyDisabled)}
|
||||
{item(t("mail.replyAll"), () => onReply("reply_all"), replyDisabled)}
|
||||
{item(t("mail.forward"), () => onReply("forward"), replyDisabled)}
|
||||
{item(t("mail.forwardAsAttachment"), () => onReply("forward"), replyDisabled)}
|
||||
{item(t("mail.print"), onPrint)}
|
||||
<div className="my-1 border-t border-app-border" />
|
||||
{item(t("mail.delete"), onDelete)}
|
||||
@@ -199,6 +214,7 @@ function MessageItem({
|
||||
boxClassName,
|
||||
zIndex,
|
||||
showReplyBar,
|
||||
canReply,
|
||||
onToggle,
|
||||
onReply,
|
||||
onToggleRead,
|
||||
@@ -214,6 +230,7 @@ function MessageItem({
|
||||
boxClassName: string;
|
||||
zIndex: number;
|
||||
showReplyBar: boolean;
|
||||
canReply: boolean;
|
||||
onToggle: () => void;
|
||||
onReply: (kind: string) => void;
|
||||
onToggleRead: () => void;
|
||||
@@ -321,14 +338,21 @@ function MessageItem({
|
||||
}`}
|
||||
/>
|
||||
</button>
|
||||
<button
|
||||
className="flex items-center gap-1 text-app-muted hover:text-app-fg"
|
||||
onClick={() => onReply(multiRecipient ? "reply_all" : "reply")}
|
||||
>
|
||||
<Icon name={multiRecipient ? "reply-all" : "reply"} size={14} />
|
||||
{multiRecipient ? t("mail.replyAll") : t("mail.reply")}
|
||||
</button>
|
||||
<ActionsMenu onReply={onReply} onPrint={onPrint} onDelete={onDelete} />
|
||||
{canReply && (
|
||||
<button
|
||||
className="flex items-center gap-1 text-app-muted hover:text-app-fg"
|
||||
onClick={() => onReply(multiRecipient ? "reply_all" : "reply")}
|
||||
>
|
||||
<Icon name={multiRecipient ? "reply-all" : "reply"} size={14} />
|
||||
{multiRecipient ? t("mail.replyAll") : t("mail.reply")}
|
||||
</button>
|
||||
)}
|
||||
<ActionsMenu
|
||||
onReply={onReply}
|
||||
onPrint={onPrint}
|
||||
onDelete={onDelete}
|
||||
replyDisabled={!canReply}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{showDetails && (
|
||||
@@ -356,7 +380,7 @@ function MessageItem({
|
||||
<div className="px-4 pb-3">
|
||||
<MessageBody message={message} />
|
||||
</div>
|
||||
{showReplyBar && (
|
||||
{showReplyBar && canReply && (
|
||||
<div className="flex gap-2 border-t border-app-border px-4 py-3">
|
||||
{replyBtn("reply", t("mail.reply"), "reply")}
|
||||
{replyBtn("reply_all", t("mail.replyAll"), "reply-all")}
|
||||
@@ -440,6 +464,7 @@ export function ThreadView({
|
||||
};
|
||||
|
||||
const openReply = (msg: MessageEnvelope, kind: string) => {
|
||||
if (specialUse.get(msg.mailbox_id) === "trash") return; // no reply/forward from Trash
|
||||
setExpanded((prev) => new Set(prev).add(msg.id));
|
||||
setComposing({ anchorId: msg.id, init: buildReplyInit(subject, msg, kind) });
|
||||
};
|
||||
@@ -471,9 +496,15 @@ export function ThreadView({
|
||||
allExpanded ? new Set(newest ? [newest.id] : []) : new Set(messages.map((m) => m.id)),
|
||||
);
|
||||
|
||||
const composerRef = useRef<HTMLDivElement | null>(null);
|
||||
// Bring the inline composer into view when it opens (reply/reply-all/forward).
|
||||
useEffect(() => {
|
||||
if (composing) composerRef.current?.scrollIntoView({ behavior: "smooth", block: "center" });
|
||||
}, [composing]);
|
||||
|
||||
const composerFor = (id: string) =>
|
||||
composing?.anchorId === id ? (
|
||||
<div className="px-4 py-3">
|
||||
<div ref={composerRef} className="px-4 py-3">
|
||||
<Composer init={composing.init} onClose={() => setComposing(null)} />
|
||||
</div>
|
||||
) : null;
|
||||
@@ -487,7 +518,7 @@ export function ThreadView({
|
||||
className="rounded p-1.5 text-app-muted hover:bg-app-hover hover:text-app-fg"
|
||||
title={t("mail.print")}
|
||||
aria-label={t("mail.print")}
|
||||
onClick={() => window.print()}
|
||||
onClick={() => void printMessages(subject, messages)}
|
||||
>
|
||||
<Icon name="print" size={16} />
|
||||
</button>
|
||||
@@ -507,17 +538,23 @@ export function ThreadView({
|
||||
const isExpanded = expanded.has(message.id);
|
||||
const prev = i > 0 ? visual[i - 1] : undefined;
|
||||
const prevCollapsed = prev ? !expanded.has(prev.id) : false;
|
||||
const prevIsNewest = Boolean(prev && newest && prev.id === newest.id);
|
||||
// Overlap onto a preceding collapsed card, and also onto the always-open
|
||||
// newest message — so the newest stacks with its neighbour in both sort
|
||||
// orders (bottom card in oldest-first, top card in newest-first). z-index
|
||||
// keeps the newer/"current" edge on top so text stays visible.
|
||||
const overlap = prevCollapsed || prevIsNewest;
|
||||
// Cards overlap by a negative top margin (tuck up under the card above),
|
||||
// with the newer card kept on top by z-index. The card above is OLDER in
|
||||
// oldest-first but NEWER in newest-first, so the rule differs by order:
|
||||
// - oldest-first: an expanded card tucks up over the older card above it
|
||||
// (its own top stays clear). Only overlap a collapsed card above.
|
||||
// - newest-first: the card above is newer, so tucking an expanded card up
|
||||
// would hide its header under that newer card. Only collapsed cards tuck
|
||||
// up; an expanded card keeps its gap, so its top stays clear of the newer
|
||||
// message and the older card BELOW tucks under it instead.
|
||||
const overlap = order === "desc" ? !isExpanded : prevCollapsed;
|
||||
const margin = i === 0 ? "" : overlap ? "-mt-[20px]" : "mt-2";
|
||||
const z = order === "asc" ? i + 1 : visual.length - i;
|
||||
const boxClassName = `${margin} rounded-lg border border-app-border bg-app-surface ${
|
||||
isExpanded ? "" : "shadow-sm"
|
||||
}`;
|
||||
// Reply/Reply-all/Forward make no sense on a trashed message.
|
||||
const canReply = specialUse.get(message.mailbox_id) !== "trash";
|
||||
return (
|
||||
<Fragment key={message.id}>
|
||||
<MessageItem
|
||||
@@ -529,10 +566,11 @@ export function ThreadView({
|
||||
boxClassName={boxClassName}
|
||||
zIndex={z}
|
||||
showReplyBar={isNewest && composing?.anchorId !== message.id}
|
||||
canReply={canReply}
|
||||
onToggle={() => toggle(message.id)}
|
||||
onReply={(kind) => openReply(message, kind)}
|
||||
onToggleRead={() => toggleRead(message)}
|
||||
onPrint={() => window.print()}
|
||||
onPrint={() => void printMessages(subject, [message])}
|
||||
onDelete={() => deleteMessage(message)}
|
||||
innerRef={isNewest ? (el) => (newestRef.current = el) : undefined}
|
||||
/>
|
||||
|
||||
@@ -38,6 +38,8 @@ export const en = {
|
||||
"nav.mail": "Mail",
|
||||
"nav.settings": "Settings",
|
||||
"nav.compose": "Compose",
|
||||
"nav.collapseSidebar": "Collapse sidebar",
|
||||
"nav.expandSidebar": "Expand sidebar",
|
||||
"nav.allAccounts": "All accounts",
|
||||
"nav.inbox": "Inbox",
|
||||
"nav.archive": "Archive",
|
||||
@@ -103,6 +105,8 @@ export const en = {
|
||||
"compose.send": "Send",
|
||||
"compose.sending": "Sending…",
|
||||
"compose.discard": "Discard",
|
||||
"compose.saveDraft": "Save draft",
|
||||
"compose.closePrompt": "Save this message as a draft?",
|
||||
"compose.sent": "Message sent.",
|
||||
"compose.undo": "Undo",
|
||||
"compose.sendFailed": "Send failed.",
|
||||
|
||||
@@ -94,8 +94,8 @@ export const api = {
|
||||
if (cursor) params.set("cursor", cursor);
|
||||
return apiFetch<ThreadListOut>(`/threads?${params.toString()}`);
|
||||
},
|
||||
thread: (id: string, sort: "asc" | "desc" = "asc") =>
|
||||
apiFetch<ThreadDetail>(`/threads/${id}?sort=${sort}`),
|
||||
thread: (id: string, sort: "asc" | "desc" = "asc", view = "unified") =>
|
||||
apiFetch<ThreadDetail>(`/threads/${id}?sort=${sort}&view=${encodeURIComponent(view)}`),
|
||||
|
||||
search: (q: string, cursor?: string) => {
|
||||
const params = new URLSearchParams({ q });
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
// Mail-style printing: assemble a clean, print-ready document (message headers +
|
||||
// sanitized body, one section per message for a thread) in a new window and invoke
|
||||
// the browser's print dialog. Reuses the same sanitized HTML the reader shows.
|
||||
|
||||
import { api } from "./api";
|
||||
import type { Address, AttachmentOut, MessageEnvelope } from "./types";
|
||||
|
||||
function esc(s: string): string {
|
||||
return s
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, """);
|
||||
}
|
||||
|
||||
function fmtAddr(a: Address): string {
|
||||
if (!a.email) return a.name || "";
|
||||
return a.name ? `${a.name} <${a.email}>` : a.email;
|
||||
}
|
||||
|
||||
function fmtList(addrs: Address[]): string {
|
||||
return addrs.map(fmtAddr).filter(Boolean).join(", ");
|
||||
}
|
||||
|
||||
function fmtDate(iso: string): string {
|
||||
const d = new Date(iso);
|
||||
return d.toLocaleString(undefined, {
|
||||
weekday: "short",
|
||||
year: "numeric",
|
||||
month: "short",
|
||||
day: "numeric",
|
||||
hour: "numeric",
|
||||
minute: "2-digit",
|
||||
});
|
||||
}
|
||||
|
||||
function fmtBytes(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`;
|
||||
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`;
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
|
||||
}
|
||||
|
||||
// Fetch a message's sanitized HTML (a full document) and return just its <body>.
|
||||
async function bodyHtml(messageId: string): Promise<string> {
|
||||
try {
|
||||
const resp = await fetch(api.messageHtmlUrl(messageId), { credentials: "include" });
|
||||
if (!resp.ok) throw new Error(String(resp.status));
|
||||
const doc = new DOMParser().parseFromString(await resp.text(), "text/html");
|
||||
return doc.body?.innerHTML ?? "";
|
||||
} catch {
|
||||
return `<p style="color:#999">[message content unavailable]</p>`;
|
||||
}
|
||||
}
|
||||
|
||||
async function attachmentsFor(msg: MessageEnvelope): Promise<AttachmentOut[]> {
|
||||
if (!msg.has_attachments) return [];
|
||||
try {
|
||||
return await api.messageAttachments(msg.id);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function metaRow(label: string, value: string): string {
|
||||
return value
|
||||
? `<tr><th>${esc(label)}</th><td>${esc(value)}</td></tr>`
|
||||
: "";
|
||||
}
|
||||
|
||||
function section(msg: MessageEnvelope, subject: string, body: string, atts: AttachmentOut[]): string {
|
||||
const attachLine = atts.length
|
||||
? `<tr><th>Attachments</th><td>${atts
|
||||
.map((a) => `${esc(a.filename)} (${fmtBytes(a.size_bytes)})`)
|
||||
.join(", ")}</td></tr>`
|
||||
: "";
|
||||
return `<section class="wmc-print-msg">
|
||||
<div class="wmc-print-head">
|
||||
<div class="wmc-print-subject">${esc(subject || "(no subject)")}</div>
|
||||
<table class="wmc-print-meta">
|
||||
${metaRow("From", fmtList(msg.from_addr))}
|
||||
${metaRow("To", fmtList(msg.to_addr))}
|
||||
${metaRow("Cc", fmtList(msg.cc_addr))}
|
||||
${metaRow("Date", fmtDate(msg.date))}
|
||||
${attachLine}
|
||||
</table>
|
||||
</div>
|
||||
<div class="wmc-print-body">${body}</div>
|
||||
</section>`;
|
||||
}
|
||||
|
||||
const MARGIN = "16mm";
|
||||
|
||||
const PRINT_CSS = `
|
||||
/* Let the sheet be the full page; the margin is content padding so it always
|
||||
prints, regardless of the browser's "Margins" print-dialog setting. */
|
||||
@page { margin: 0; }
|
||||
* { box-sizing: border-box; }
|
||||
html, body { margin: 0; padding: 0; }
|
||||
body {
|
||||
font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
|
||||
color: #111; background: #e9e9ec; font-size: 12pt; line-height: 1.5;
|
||||
-webkit-print-color-adjust: exact; print-color-adjust: exact;
|
||||
}
|
||||
/* On screen, present it like a padded sheet of paper. */
|
||||
@media screen {
|
||||
body {
|
||||
max-width: 210mm; margin: 24px auto; padding: ${MARGIN};
|
||||
background: #fff; box-shadow: 0 1px 8px rgba(0,0,0,.15);
|
||||
}
|
||||
}
|
||||
/* In print the padding IS the page margin (survives a "None" margin setting). */
|
||||
@media print {
|
||||
html, body { background: #fff; }
|
||||
body { padding: ${MARGIN}; }
|
||||
}
|
||||
.wmc-print-msg { padding-bottom: 18px; }
|
||||
.wmc-print-msg + .wmc-print-msg { border-top: 2px solid #ddd; padding-top: 18px; }
|
||||
.wmc-print-head { break-inside: avoid; margin-bottom: 12px; }
|
||||
.wmc-print-subject { font-size: 16pt; font-weight: 600; margin: 0 0 8px; }
|
||||
.wmc-print-meta { width: 100%; border-collapse: collapse; font-size: 10.5pt; }
|
||||
.wmc-print-meta th {
|
||||
text-align: left; vertical-align: top; color: #666; font-weight: 500;
|
||||
width: 72px; padding: 1px 8px 1px 0; white-space: nowrap;
|
||||
}
|
||||
.wmc-print-meta td { vertical-align: top; padding: 1px 0; color: #111; }
|
||||
.wmc-print-body { color: #1a1a1a; word-break: break-word; }
|
||||
.wmc-print-body img { max-width: 100%; height: auto; }
|
||||
.wmc-print-body table { max-width: 100%; border-collapse: collapse; }
|
||||
.wmc-print-body blockquote {
|
||||
margin: 0 0 0 12px; padding-left: 12px; border-left: 3px solid #e0e0e0; color: #555;
|
||||
}
|
||||
.wmc-print-body a { color: #2563eb; }
|
||||
.wmc-print-body pre { white-space: pre-wrap; word-wrap: break-word; }
|
||||
`;
|
||||
|
||||
// Wait for images to finish (so they're in the print), capped so a slow/broken
|
||||
// image can never block printing indefinitely.
|
||||
async function waitForImages(win: Window, timeoutMs = 3000): Promise<void> {
|
||||
const pending = Array.from(win.document.images).filter((img) => !img.complete);
|
||||
if (pending.length === 0) return;
|
||||
await Promise.race([
|
||||
Promise.all(
|
||||
pending.map(
|
||||
(img) =>
|
||||
new Promise<void>((res) => {
|
||||
img.addEventListener("load", () => res(), { once: true });
|
||||
img.addEventListener("error", () => res(), { once: true });
|
||||
}),
|
||||
),
|
||||
),
|
||||
new Promise<void>((res) => setTimeout(res, timeoutMs)),
|
||||
]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Open a print-ready window for one or more messages and invoke the print dialog.
|
||||
* Pass a single message to print just that email, or all thread messages (ascending)
|
||||
* to print the conversation.
|
||||
*/
|
||||
export async function printMessages(subject: string, messages: MessageEnvelope[]): Promise<void> {
|
||||
if (messages.length === 0) return;
|
||||
|
||||
// Open the window synchronously (inside the click) so it isn't blocked as a popup.
|
||||
const win = window.open("", "_blank", "width=820,height=1000");
|
||||
if (!win) {
|
||||
window.alert("Please allow pop-ups for this site to print.");
|
||||
return;
|
||||
}
|
||||
const title = esc(subject || "(no subject)");
|
||||
win.document.write(
|
||||
`<!doctype html><meta charset="utf-8"><title>${title}</title>` +
|
||||
`<p style="font:12pt system-ui;color:#888;padding:16px">Preparing…</p>`,
|
||||
);
|
||||
win.document.close();
|
||||
|
||||
const [bodies, atts] = await Promise.all([
|
||||
Promise.all(messages.map((m) => bodyHtml(m.id))),
|
||||
Promise.all(messages.map((m) => attachmentsFor(m))),
|
||||
]);
|
||||
if (win.closed) return; // the user dismissed the window while it was preparing
|
||||
|
||||
const sections = messages
|
||||
.map((m, i) => section(m, m.subject || subject, bodies[i], atts[i]))
|
||||
.join("");
|
||||
// Write the whole document in one pass so the print stylesheet parses cleanly.
|
||||
win.document.open();
|
||||
win.document.write(
|
||||
`<!doctype html><html><head><meta charset="utf-8">` +
|
||||
`<base href="${window.location.origin}/">` +
|
||||
`<title>${title}</title><style>${PRINT_CSS}</style></head>` +
|
||||
`<body>${sections}</body></html>`,
|
||||
);
|
||||
win.document.close();
|
||||
|
||||
await waitForImages(win);
|
||||
if (win.closed) return;
|
||||
win.focus();
|
||||
win.onafterprint = () => win.close();
|
||||
win.print();
|
||||
}
|
||||
@@ -26,6 +26,9 @@ function quotaLabel(q: QuotaOut): string {
|
||||
// Threads shown per page in the mail list.
|
||||
const PAGE_SIZE = 50;
|
||||
|
||||
// Collapsed sidebar width: just wide enough for the icons.
|
||||
const COLLAPSED_W = 60;
|
||||
|
||||
// Unified per-folder views. "Spam" maps to the IMAP special-use "junk".
|
||||
const FOLDERS: { view: string; icon: string; label: MessageKey }[] = [
|
||||
{ view: "folder:inbox", icon: "inbox", label: "nav.inbox" },
|
||||
@@ -52,17 +55,23 @@ export function MailLayout() {
|
||||
const [refreshing, setRefreshing] = useState(false);
|
||||
const [composing, setComposing] = useState<ComposerInit | null>(null);
|
||||
const [composerExpanded, setComposerExpanded] = useState(false);
|
||||
// Bumped each time a composer is opened so the <Composer> remounts (fresh state)
|
||||
// when switching between drafts instead of keeping the previous draft's editor.
|
||||
const [composeKey, setComposeKey] = useState(0);
|
||||
const [replySignal, setReplySignal] = useState<{ kind: string; seq: number }>();
|
||||
const searchRef = useRef<HTMLInputElement>(null);
|
||||
|
||||
// Resizable panel widths (persisted). Drag the dividers to adjust.
|
||||
const [sidebarW, setSidebarW] = useState(() => Number(localStorage.getItem("panel.sidebar")) || 240);
|
||||
const [listW, setListW] = useState(() => Number(localStorage.getItem("panel.list")) || 384);
|
||||
// Collapsed sidebar shows icons only (persisted).
|
||||
const [collapsed, setCollapsed] = useState(() => localStorage.getItem("panel.collapsed") === "1");
|
||||
const clampSidebar = (w: number) => Math.min(Math.max(w, 180), 420);
|
||||
// The list can be dragged up to 75% of the window width.
|
||||
const clampList = (w: number) => Math.min(Math.max(w, 260), window.innerWidth * 0.75);
|
||||
useEffect(() => localStorage.setItem("panel.sidebar", String(sidebarW)), [sidebarW]);
|
||||
useEffect(() => localStorage.setItem("panel.list", String(listW)), [listW]);
|
||||
useEffect(() => localStorage.setItem("panel.collapsed", collapsed ? "1" : "0"), [collapsed]);
|
||||
useEffect(() => localStorage.setItem("mail.listOrder", listOrder), [listOrder]);
|
||||
|
||||
const isDrafts = view === "folder:drafts";
|
||||
@@ -117,8 +126,8 @@ export function MailLayout() {
|
||||
threadsRef.current = threads;
|
||||
|
||||
const detailQuery = useQuery({
|
||||
queryKey: ["thread", threadId, threadOrder],
|
||||
queryFn: () => api.thread(threadId!, threadOrder),
|
||||
queryKey: ["thread", threadId, threadOrder, view],
|
||||
queryFn: () => api.thread(threadId!, threadOrder, view),
|
||||
enabled: Boolean(threadId),
|
||||
});
|
||||
|
||||
@@ -132,6 +141,33 @@ export function MailLayout() {
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [detailQuery.data?.id]);
|
||||
|
||||
// Open the composer with fresh state, remounting it if one is already open
|
||||
// (e.g. clicking a different draft replaces the current editor).
|
||||
const openComposer = (init: ComposerInit, expand = false) => {
|
||||
setComposing(init);
|
||||
setComposerExpanded(expand);
|
||||
setComposeKey((k) => k + 1);
|
||||
};
|
||||
|
||||
const closeComposer = () => {
|
||||
setComposing(null);
|
||||
setComposerExpanded(false);
|
||||
queryClient.invalidateQueries({ queryKey: ["drafts"] });
|
||||
queryClient.invalidateQueries({ queryKey: ["drafts-remote"] });
|
||||
};
|
||||
|
||||
// Leaving a folder closes any open composer/draft editor: the reading pane
|
||||
// should reflect the folder you navigated to, not a leftover draft editor.
|
||||
const firstView = useRef(true);
|
||||
useEffect(() => {
|
||||
if (firstView.current) {
|
||||
firstView.current = false;
|
||||
return;
|
||||
}
|
||||
closeComposer();
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [view]);
|
||||
|
||||
const afterMove = () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["threads"] });
|
||||
queryClient.invalidateQueries({ queryKey: ["quota"] });
|
||||
@@ -201,7 +237,7 @@ export function MailLayout() {
|
||||
|
||||
useKeyboardShortcuts({
|
||||
onSearch: () => searchRef.current?.focus(),
|
||||
onCompose: () => accountsQuery.data?.[0] && setComposing({ accountId: accountsQuery.data[0].id, kind: "new" }),
|
||||
onCompose: () => accountsQuery.data?.[0] && openComposer({ accountId: accountsQuery.data[0].id, kind: "new" }),
|
||||
onReply: () => setReplySignal({ kind: "reply", seq: Date.now() }),
|
||||
onReplyAll: () => setReplySignal({ kind: "reply_all", seq: Date.now() }),
|
||||
onForward: () => setReplySignal({ kind: "forward", seq: Date.now() }),
|
||||
@@ -221,22 +257,39 @@ export function MailLayout() {
|
||||
<div className="flex h-screen bg-app-bg text-app-fg">
|
||||
{/* Sidebar */}
|
||||
<aside
|
||||
style={{ width: sidebarW }}
|
||||
style={{ width: collapsed ? COLLAPSED_W : sidebarW }}
|
||||
className="flex shrink-0 flex-col border-r border-app-border bg-app-bg"
|
||||
>
|
||||
<div className="flex items-center justify-between px-3 pb-1 pt-3">
|
||||
<span className="text-xs font-semibold uppercase tracking-wide text-app-faint">
|
||||
{t("common.appName")}
|
||||
</span>
|
||||
<ThemeToggle />
|
||||
<div
|
||||
className={`flex pb-1 pt-3 ${
|
||||
collapsed ? "flex-col items-center gap-1 px-2" : "items-center justify-between px-3"
|
||||
}`}
|
||||
>
|
||||
{!collapsed && (
|
||||
<span className="text-xs font-semibold uppercase tracking-wide text-app-faint">
|
||||
{t("common.appName")}
|
||||
</span>
|
||||
)}
|
||||
<div className={`flex items-center gap-1 ${collapsed ? "flex-col" : ""}`}>
|
||||
<button
|
||||
onClick={() => setCollapsed((v) => !v)}
|
||||
title={collapsed ? t("nav.expandSidebar") : t("nav.collapseSidebar")}
|
||||
aria-label={collapsed ? t("nav.expandSidebar") : t("nav.collapseSidebar")}
|
||||
className="rounded p-1.5 text-app-muted hover:bg-app-hover hover:text-app-fg"
|
||||
>
|
||||
<Icon name={collapsed ? "show" : "hide"} size={16} />
|
||||
</button>
|
||||
<ThemeToggle />
|
||||
</div>
|
||||
</div>
|
||||
<div className="px-3 pb-2">
|
||||
<div className={`pb-2 ${collapsed ? "px-2" : "px-3"}`}>
|
||||
<button
|
||||
className="flex w-full items-center justify-center gap-2 rounded-lg bg-app-accent py-2 text-sm font-medium text-app-accent-fg shadow-sm hover:opacity-90"
|
||||
onClick={() => accounts[0] && setComposing({ accountId: accounts[0].id, kind: "new" })}
|
||||
title={collapsed ? t("nav.compose") : undefined}
|
||||
onClick={() => accounts[0] && openComposer({ accountId: accounts[0].id, kind: "new" })}
|
||||
>
|
||||
<Icon name="compose" size={16} />
|
||||
{t("nav.compose")}
|
||||
{!collapsed && t("nav.compose")}
|
||||
</button>
|
||||
</div>
|
||||
<nav className="flex-1 space-y-0.5 overflow-y-auto px-2 py-1 text-sm">
|
||||
@@ -244,44 +297,62 @@ export function MailLayout() {
|
||||
<SidebarItem
|
||||
key={f.view}
|
||||
active={view === f.view}
|
||||
collapsed={collapsed}
|
||||
title={collapsed ? t(f.label) : undefined}
|
||||
onClick={() => {
|
||||
setView(f.view);
|
||||
navigate("/mail");
|
||||
}}
|
||||
>
|
||||
<Icon name={f.icon} size={18} />
|
||||
{t(f.label)}
|
||||
</SidebarItem>
|
||||
))}
|
||||
<div className="px-3 pb-1 pt-3 text-[11px] font-semibold uppercase tracking-wide text-app-faint">
|
||||
{t("settings.accounts")}
|
||||
</div>
|
||||
{accounts.map((a) => (
|
||||
<SidebarItem
|
||||
key={a.id}
|
||||
active={view === `account:${a.id}`}
|
||||
onClick={() => { setView(`account:${a.id}`); navigate("/mail"); }}
|
||||
>
|
||||
<span className="h-2.5 w-2.5 shrink-0 rounded-full" style={{ backgroundColor: a.color }} />
|
||||
<span className="truncate">{a.email}</span>
|
||||
{!collapsed && t(f.label)}
|
||||
</SidebarItem>
|
||||
))}
|
||||
{/* Per-account views are hidden entirely when collapsed. */}
|
||||
{!collapsed && (
|
||||
<>
|
||||
<div className="px-3 pb-1 pt-3 text-[11px] font-semibold uppercase tracking-wide text-app-faint">
|
||||
{t("settings.accounts")}
|
||||
</div>
|
||||
{accounts.map((a) => (
|
||||
<SidebarItem
|
||||
key={a.id}
|
||||
active={view === `account:${a.id}`}
|
||||
onClick={() => { setView(`account:${a.id}`); navigate("/mail"); }}
|
||||
>
|
||||
<span className="h-2.5 w-2.5 shrink-0 rounded-full" style={{ backgroundColor: a.color }} />
|
||||
<span className="truncate">{a.email}</span>
|
||||
</SidebarItem>
|
||||
))}
|
||||
</>
|
||||
)}
|
||||
</nav>
|
||||
<div className="border-t border-app-border p-2 text-sm">
|
||||
<Link className="block rounded-lg px-3 py-2 text-app-muted hover:bg-app-hover hover:text-app-fg" to="/settings">
|
||||
⚙ {t("nav.settings")}
|
||||
<Link
|
||||
className={`flex items-center gap-2.5 rounded-lg py-2 text-app-muted hover:bg-app-hover hover:text-app-fg ${
|
||||
collapsed ? "justify-center px-0" : "px-3"
|
||||
}`}
|
||||
title={collapsed ? t("nav.settings") : undefined}
|
||||
to="/settings"
|
||||
>
|
||||
<Icon name="settings" size={16} />
|
||||
{!collapsed && t("nav.settings")}
|
||||
</Link>
|
||||
<button
|
||||
className="block w-full rounded-lg px-3 py-2 text-left text-app-muted hover:bg-app-hover hover:text-app-fg"
|
||||
className={`flex w-full items-center gap-2.5 rounded-lg py-2 text-left text-app-muted hover:bg-app-hover hover:text-app-fg ${
|
||||
collapsed ? "justify-center px-0" : "px-3"
|
||||
}`}
|
||||
title={collapsed ? t("common.signOut") : undefined}
|
||||
onClick={async () => {
|
||||
await api.logout();
|
||||
queryClient.clear();
|
||||
navigate("/login");
|
||||
}}
|
||||
>
|
||||
{t("common.signOut")}
|
||||
<Icon name="log-out" size={16} />
|
||||
{!collapsed && t("common.signOut")}
|
||||
</button>
|
||||
{quotaQuery.data && (
|
||||
{!collapsed && quotaQuery.data && (
|
||||
<div
|
||||
className="mt-1 border-t border-app-border px-3 pb-1 pt-2 text-xs text-app-faint"
|
||||
title={t("nav.storageQuota")}
|
||||
@@ -292,7 +363,7 @@ export function MailLayout() {
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<Resizer onDelta={(dx) => setSidebarW((w) => clampSidebar(w + dx))} />
|
||||
{!collapsed && <Resizer onDelta={(dx) => setSidebarW((w) => clampSidebar(w + dx))} />}
|
||||
|
||||
{/* Message list */}
|
||||
<div
|
||||
@@ -334,17 +405,13 @@ export function MailLayout() {
|
||||
<DraftsList
|
||||
drafts={draftsQuery.data ?? []}
|
||||
remoteDrafts={remoteDraftsQuery.data ?? []}
|
||||
onOpen={(draft) => {
|
||||
setComposing(draftToInit(draft));
|
||||
setComposerExpanded(true);
|
||||
}}
|
||||
onOpen={(draft) => openComposer(draftToInit(draft), true)}
|
||||
onOpenRemote={async (remote) => {
|
||||
// Adopt the server draft into an editable local draft, then open it.
|
||||
const draft = await api.adoptDraft(remote.message_id);
|
||||
queryClient.invalidateQueries({ queryKey: ["drafts"] });
|
||||
queryClient.invalidateQueries({ queryKey: ["drafts-remote"] });
|
||||
setComposing(draftToInit(draft));
|
||||
setComposerExpanded(true);
|
||||
openComposer(draftToInit(draft), true);
|
||||
}}
|
||||
/>
|
||||
) : (
|
||||
@@ -466,15 +533,11 @@ export function MailLayout() {
|
||||
}
|
||||
>
|
||||
<Composer
|
||||
key={composeKey}
|
||||
init={composing}
|
||||
expanded={composerExpanded}
|
||||
onToggleExpand={() => setComposerExpanded((v) => !v)}
|
||||
onClose={() => {
|
||||
setComposing(null);
|
||||
setComposerExpanded(false);
|
||||
queryClient.invalidateQueries({ queryKey: ["drafts"] });
|
||||
queryClient.invalidateQueries({ queryKey: ["drafts-remote"] });
|
||||
}}
|
||||
onClose={closeComposer}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
@@ -570,11 +633,26 @@ function SortIcon() {
|
||||
);
|
||||
}
|
||||
|
||||
function SidebarItem({ active, onClick, children }: { active: boolean; onClick: () => void; children: React.ReactNode }) {
|
||||
function SidebarItem({
|
||||
active,
|
||||
onClick,
|
||||
collapsed = false,
|
||||
title,
|
||||
children,
|
||||
}: {
|
||||
active: boolean;
|
||||
onClick: () => void;
|
||||
collapsed?: boolean;
|
||||
title?: string;
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<button
|
||||
onClick={onClick}
|
||||
className={`flex w-full items-center gap-2.5 rounded-lg px-3 py-1.5 text-left ${
|
||||
title={title}
|
||||
className={`flex w-full items-center gap-2.5 rounded-lg py-1.5 text-left ${
|
||||
collapsed ? "justify-center px-0" : "px-3"
|
||||
} ${
|
||||
active ? "bg-app-active font-medium text-app-fg" : "text-app-muted hover:bg-app-hover hover:text-app-fg"
|
||||
}`}
|
||||
>
|
||||
|
||||
|
Before Width: | Height: | Size: 209 KiB |
|
Before Width: | Height: | Size: 169 KiB |
|
Before Width: | Height: | Size: 1.8 KiB |
|
After Width: | Height: | Size: 96 KiB |
|
After Width: | Height: | Size: 77 KiB |
|
After Width: | Height: | Size: 82 KiB |
@@ -0,0 +1,811 @@
|
||||
# Webmail Client — Implementation Specification (v1.4, agent-ready)
|
||||
|
||||
This document is a complete, self-contained build specification. It is written to be handed to an AI coding agent. Read it fully before writing code. The companion document `webmail-client-concept.md` gives background; where they differ, **this spec wins**.
|
||||
|
||||
---
|
||||
|
||||
## 0. Instructions to the implementing agent
|
||||
|
||||
1. **Build milestone by milestone** (§12), in order. Do not start a milestone before the previous one's acceptance criteria pass.
|
||||
2. **Do not add features beyond this spec.** Anything not specified here is out of scope; leave `TODO(spec)` comments instead of inventing behavior.
|
||||
3. **Definition of done** for every task: code + type hints + unit tests + passing `ruff check`, `ruff format --check`, `mypy --strict` (backend) / `tsc --noEmit`, `eslint` (frontend) + updated docs in `docs/` if an interface changed.
|
||||
4. **Stability over novelty:** use exactly the pinned dependencies in §3. Do not swap libraries or add dependencies without a `# DEPENDENCY:` comment explaining why, and never for something the stdlib does adequately.
|
||||
5. **All interfaces in §5 are frozen contracts.** Implementations may evolve; signatures may not (without bumping the version comment at the top of the interface file).
|
||||
6. **Security rules in §10 are hard requirements**, not suggestions.
|
||||
7. Every module publishing or consuming events must use the event names in §8 verbatim.
|
||||
8. Commit in small, working increments; every commit must leave the test suite green.
|
||||
|
||||
## 1. Architecture invariants (never violate)
|
||||
|
||||
- **I-1 Sync-first:** the web UI and REST API never open IMAP connections. Only sync workers talk to mail servers. The API reads exclusively from Postgres/Meilisearch.
|
||||
- **I-2 Independent identity:** login uses app accounts (`localpart@our-web-client.example`) with Argon2id hashes. Mailbox credentials are payload data, encrypted at rest, decrypted only inside sync workers.
|
||||
- **I-3 One index, many views:** unified inbox, per-account views, virtual mailboxes, and search results are all parameterized queries over the same index. No duplicated storage per view.
|
||||
- **I-4 Threading is ours:** JWZ in our backend (§7). Never use the IMAP `THREAD` extension.
|
||||
- **I-5 Direction-agnostic thread UI:** thread ordering (oldest-first/newest-first) is a data-ordering concern. One layout, one composer-anchor slot (§9.4).
|
||||
- **I-6 Plugins extend, core stays closed:** new features beyond this spec must be expressible through the plugin surfaces (§11).
|
||||
- **I-7 Multi-user isolation:** every query (SQL and search) carries a mandatory `user_id` filter. No exceptions, including admin tooling.
|
||||
- **I-8 Mail HTML is hostile input:** sanitize with nh3 server-side at index time AND render in a sandboxed iframe. Both, always.
|
||||
|
||||
## 2. Repository layout (monorepo)
|
||||
|
||||
```
|
||||
webmail/
|
||||
├── backend/
|
||||
│ ├── pyproject.toml
|
||||
│ ├── app/
|
||||
│ │ ├── main.py # FastAPI app factory
|
||||
│ │ ├── config.py # pydantic-settings; all config via env
|
||||
│ │ ├── db.py # SQLAlchemy async engine/session
|
||||
│ │ ├── events.py # EventBus (Redis pub/sub + arq enqueue)
|
||||
│ │ ├── identity/ # §6.1 auth, sessions, handles, TOTP
|
||||
│ │ ├── accounts/ # §6.2 wizard, autodiscovery, credential vault
|
||||
│ │ ├── providers/
|
||||
│ │ │ ├── base.py # MailProvider interface (§5.1)
|
||||
│ │ │ └── imap_smtp/ # first implementation
|
||||
│ │ ├── sync/ # §6.3 backfill, IDLE loops, reconciliation
|
||||
│ │ ├── indexer/ # §6.4 MIME parsing → DB rows + blobs
|
||||
│ │ ├── threading_/ # §7 JWZ (trailing _ avoids stdlib clash)
|
||||
│ │ ├── search/
|
||||
│ │ │ ├── base.py # SearchBackend interface (§5.2)
|
||||
│ │ │ ├── queryparser.py # operator grammar (§8.2)
|
||||
│ │ │ ├── pg.py # Postgres FTS backend (M0–M1)
|
||||
│ │ │ └── meili.py # Meilisearch backend (M2)
|
||||
│ │ ├── outbox/ # §6.5 drafts, MIME assembly, send queue, undo
|
||||
│ │ ├── api/ # routers, one file per §9.1 group
|
||||
│ │ ├── ws.py # WebSocket hub (§9.2)
|
||||
│ │ └── plugins/ # §11 loader + plugin packages
|
||||
│ ├── workers/worker.py # arq worker entrypoint
|
||||
│ ├── migrations/ # alembic
|
||||
│ └── tests/
|
||||
├── frontend/
|
||||
│ ├── package.json
|
||||
│ └── src/
|
||||
│ ├── api/ # generated client from OpenAPI + WS layer
|
||||
│ ├── routes/ # §9.3
|
||||
│ ├── components/
|
||||
│ │ ├── thread/ThreadView.tsx # §9.4 direction-agnostic
|
||||
│ │ ├── thread/Composer.tsx
|
||||
│ │ ├── list/MessageList.tsx # virtualized
|
||||
│ │ └── slots/ # plugin UI slots (§11)
|
||||
│ └── lib/
|
||||
├── docker/
|
||||
│ ├── docker-compose.yml # app, worker, postgres:16, redis:7, meilisearch:v1.8, dovecot (test)
|
||||
│ └── docker-compose.ci.yml # + greenmail for integration tests
|
||||
└── docs/
|
||||
```
|
||||
|
||||
## 3. Pinned stack
|
||||
|
||||
**Backend (Python 3.11):** fastapi ~0.115, uvicorn ~0.30, sqlalchemy ~2.0 (async, asyncpg), alembic ~1.13, pydantic ~2.8, pydantic-settings, arq ~0.26, redis ~5.0, aioimaplib ~1.1, imapclient ~3.0 (utility/fallback), aiosmtplib ~3.0, argon2-cffi ~23.1, pyotp ~2.9, dnspython ~2.6, httpx ~0.27, nh3 ~0.2, cryptography ~43.0, mail-parser ~4.0 (helpers only; primary parsing via stdlib `email.message.EmailMessage` with `policy.default`).
|
||||
|
||||
**Frontend:** TypeScript ~5.5, React ~18.3, Vite ~5, TanStack Query ~5, TanStack Virtual ~3, Radix UI, Tailwind ~3.4, zod (WS payload validation). API client generated from OpenAPI (`openapi-typescript`).
|
||||
|
||||
**Infra:** PostgreSQL 16 (with `citext` extension), Redis 7, Meilisearch v1.8 (M2+), MinIO or plain filesystem for the blob store (config-switchable; filesystem default).
|
||||
|
||||
## 4. Database schema (Postgres 16, authoritative DDL)
|
||||
|
||||
All PKs are `uuid DEFAULT gen_random_uuid()`. All tables carrying user data include `user_id` with an index. Timestamps are `timestamptz`.
|
||||
|
||||
```sql
|
||||
CREATE EXTENSION IF NOT EXISTS citext;
|
||||
|
||||
CREATE TABLE users (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
handle citext UNIQUE NOT NULL, -- full form: localpart@<APP_DOMAIN>
|
||||
password_hash text NOT NULL, -- argon2id
|
||||
totp_secret bytea, -- encrypted; NULL = 2FA off
|
||||
recovery_email citext,
|
||||
settings jsonb NOT NULL DEFAULT '{}', -- incl. {"thread_order":"asc"|"desc"}
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
-- Handle rules (enforce in code, mail-grade from day one):
|
||||
-- localpart: 3–30 chars, [a-z0-9._-], no leading/trailing/double dot,
|
||||
-- case-insensitive, reserved list (postmaster, abuse, admin, root, noreply, ...).
|
||||
|
||||
CREATE TABLE accounts (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
email citext NOT NULL,
|
||||
display_name text NOT NULL DEFAULT '',
|
||||
color text NOT NULL DEFAULT '#888888',
|
||||
imap_config jsonb NOT NULL, -- {host,port,tls:"ssl"|"starttls",auth:"password"|"oauth2"}
|
||||
smtp_config jsonb NOT NULL, -- same shape
|
||||
secret_ct bytea NOT NULL, -- envelope-encrypted password or OAuth token bundle (§10.2)
|
||||
discovery_src text NOT NULL, -- preset|autoconfig|ispdb|srv|probe|manual
|
||||
state text NOT NULL DEFAULT 'backfilling', -- backfilling|live|error|disabled
|
||||
state_detail text NOT NULL DEFAULT '',
|
||||
last_sync_at timestamptz,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
UNIQUE (user_id, email)
|
||||
);
|
||||
|
||||
CREATE TABLE mailboxes (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
account_id uuid NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
path text NOT NULL, -- IMAP mailbox name, UTF-7 decoded
|
||||
special_use text, -- inbox|sent|drafts|trash|junk|archive|NULL
|
||||
uidvalidity bigint NOT NULL DEFAULT 0,
|
||||
highestmodseq bigint, -- NULL if no CONDSTORE
|
||||
last_seen_uid bigint NOT NULL DEFAULT 0,
|
||||
UNIQUE (account_id, path)
|
||||
);
|
||||
|
||||
CREATE TABLE threads (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
subject_norm text NOT NULL, -- normalized subject (§7.3)
|
||||
snippet text NOT NULL DEFAULT '',
|
||||
participants jsonb NOT NULL DEFAULT '[]', -- [{name,email}], deduped, ordered by recency
|
||||
account_ids uuid[] NOT NULL DEFAULT '{}',
|
||||
message_count int NOT NULL DEFAULT 0,
|
||||
unread_count int NOT NULL DEFAULT 0,
|
||||
has_attachments boolean NOT NULL DEFAULT false,
|
||||
last_message_at timestamptz NOT NULL
|
||||
);
|
||||
CREATE INDEX ON threads (user_id, last_message_at DESC);
|
||||
|
||||
CREATE TABLE messages (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
account_id uuid NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||||
mailbox_id uuid NOT NULL REFERENCES mailboxes(id) ON DELETE CASCADE,
|
||||
uid bigint NOT NULL, -- IMAP UID within mailbox
|
||||
thread_id uuid REFERENCES threads(id),
|
||||
message_id_hdr text, -- Message-ID, angle brackets stripped
|
||||
in_reply_to text,
|
||||
references_hdr text[] NOT NULL DEFAULT '{}',
|
||||
subject text NOT NULL DEFAULT '',
|
||||
from_addr jsonb NOT NULL DEFAULT '[]',
|
||||
to_addr jsonb NOT NULL DEFAULT '[]',
|
||||
cc_addr jsonb NOT NULL DEFAULT '[]',
|
||||
date_hdr timestamptz,
|
||||
internal_date timestamptz NOT NULL,
|
||||
flags text[] NOT NULL DEFAULT '{}', -- \Seen,\Flagged,\Answered,\Draft,\Deleted
|
||||
size_bytes int NOT NULL DEFAULT 0,
|
||||
has_attachments boolean NOT NULL DEFAULT false,
|
||||
snippet text NOT NULL DEFAULT '', -- first ~160 chars of text body
|
||||
blob_sha256 text NOT NULL, -- raw RFC822 in blob store (§6.4)
|
||||
body_indexed tsvector, -- Postgres FTS (M0–M1 SearchBackend)
|
||||
UNIQUE (mailbox_id, uid)
|
||||
);
|
||||
CREATE INDEX ON messages (user_id, thread_id);
|
||||
CREATE INDEX ON messages (account_id, message_id_hdr);
|
||||
CREATE INDEX ON messages USING gin (body_indexed);
|
||||
|
||||
CREATE TABLE body_parts (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
|
||||
kind text NOT NULL, -- text|html_sanitized
|
||||
content text NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE attachments (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
|
||||
user_id uuid NOT NULL,
|
||||
filename text NOT NULL,
|
||||
mime_type text NOT NULL,
|
||||
size_bytes int NOT NULL,
|
||||
content_id text, -- for inline cid: images
|
||||
blob_sha256 text NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE drafts (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
account_id uuid NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||||
thread_id uuid REFERENCES threads(id), -- NULL = new mail
|
||||
reply_to_message_id uuid REFERENCES messages(id),
|
||||
kind text NOT NULL DEFAULT 'reply', -- reply|reply_all|forward|new
|
||||
payload jsonb NOT NULL, -- {to,cc,bcc,subject,html,text,attachment_ids}
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE outbox (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
draft_id uuid NOT NULL,
|
||||
state text NOT NULL DEFAULT 'undo_window', -- undo_window|sending|sent|failed|cancelled
|
||||
send_after timestamptz NOT NULL, -- now() + undo window
|
||||
error text
|
||||
);
|
||||
|
||||
CREATE TABLE saved_searches (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
name text NOT NULL,
|
||||
query text NOT NULL, -- raw operator-syntax string (§8.2)
|
||||
pinned boolean NOT NULL DEFAULT true,
|
||||
UNIQUE (user_id, name)
|
||||
);
|
||||
|
||||
-- Core recipient autocomplete (Gmail-style, derived from mail history).
|
||||
-- The Contacts plugin (Appendix B) ENRICHES suggestions; this table works without it.
|
||||
CREATE TABLE known_recipients (
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
email citext NOT NULL,
|
||||
name text NOT NULL DEFAULT '', -- last seen display name
|
||||
times_used int NOT NULL DEFAULT 0, -- weighted: sent-to counts 3, received-from counts 1
|
||||
last_used_at timestamptz NOT NULL,
|
||||
PRIMARY KEY (user_id, email)
|
||||
);
|
||||
CREATE INDEX ON known_recipients (user_id, email text_pattern_ops);
|
||||
```
|
||||
|
||||
**Blob store:** raw RFC822 messages and attachment payloads are stored content-addressed at `${BLOB_ROOT}/{user_id}/{sha256[:2]}/{sha256}` (filesystem) or the equivalent object key (MinIO). Postgres holds metadata only. Deleting a user removes their blob subtree.
|
||||
|
||||
**Sessions** live in Redis (`sess:{token}` → JSON `{user_id, created, ua}`, TTL 30 days sliding), not Postgres.
|
||||
|
||||
## 5. Frozen core interfaces (Python)
|
||||
|
||||
### 5.1 `providers/base.py` — MailProvider
|
||||
|
||||
```python
|
||||
# INTERFACE VERSION: 1
|
||||
from dataclasses import dataclass
|
||||
from typing import AsyncIterator, Protocol
|
||||
from datetime import datetime
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RemoteMailbox:
|
||||
path: str
|
||||
special_use: str | None # "inbox"|"sent"|"drafts"|"trash"|"junk"|"archive"|None
|
||||
uidvalidity: int
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RemoteMessage:
|
||||
uid: int
|
||||
raw: bytes # full RFC822
|
||||
flags: frozenset[str]
|
||||
internal_date: datetime
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FlagChange:
|
||||
uid: int
|
||||
flags: frozenset[str]
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MailboxDelta:
|
||||
new: list[RemoteMessage]
|
||||
flag_changes: list[FlagChange]
|
||||
expunged_uids: list[int]
|
||||
new_uidvalidity: int | None # non-None => UIDVALIDITY changed, caller must full-resync
|
||||
new_highestmodseq: int | None
|
||||
|
||||
class MailProvider(Protocol):
|
||||
async def verify(self) -> None: ...
|
||||
# Raises ProviderAuthError / ProviderConnectError with a human-readable .detail
|
||||
|
||||
async def list_mailboxes(self) -> list[RemoteMailbox]: ...
|
||||
|
||||
def backfill(self, mailbox: str, *, newest_first: bool = True,
|
||||
batch_size: int = 200) -> AsyncIterator[list[RemoteMessage]]: ...
|
||||
# Yields batches until the mailbox is exhausted. Headers+body in one pass.
|
||||
|
||||
async def poll(self, mailbox: str, *, last_seen_uid: int,
|
||||
highestmodseq: int | None) -> MailboxDelta: ...
|
||||
# One reconciliation pass. Uses QRESYNC/CONDSTORE when highestmodseq is not None,
|
||||
# otherwise UID SEARCH + FETCH FLAGS diff.
|
||||
|
||||
async def idle(self, mailbox: str) -> None: ...
|
||||
# Blocks until the server signals change or 25 min elapse (re-issue IDLE). May raise
|
||||
# ProviderNotSupported => sync loop falls back to poll() every POLL_INTERVAL (default 120 s).
|
||||
|
||||
async def push_flags(self, mailbox: str, uid: int,
|
||||
add: frozenset[str], remove: frozenset[str]) -> None: ...
|
||||
async def move(self, uid: int, src: str, dst: str) -> int: ... # returns new UID
|
||||
async def append(self, mailbox: str, raw: bytes,
|
||||
flags: frozenset[str]) -> int: ... # Sent/Drafts upload
|
||||
async def send(self, raw: bytes, envelope_from: str,
|
||||
envelope_to: list[str]) -> None: ... # SMTP submission
|
||||
```
|
||||
|
||||
`ImapSmtpProvider` is the only M0–M2 implementation. A future `NativeProvider` (hosted mailboxes) or `JmapProvider` implements the same protocol — nothing above this interface may import IMAP libraries.
|
||||
|
||||
### 5.2 `search/base.py` — SearchBackend
|
||||
|
||||
```python
|
||||
# INTERFACE VERSION: 1
|
||||
@dataclass(frozen=True)
|
||||
class SearchQuery:
|
||||
text: str | None # free-text remainder after operator extraction
|
||||
filters: dict[str, object] # parsed operators, keys per §8.2
|
||||
user_id: UUID # ALWAYS set (invariant I-7)
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SearchHit:
|
||||
message_id: UUID
|
||||
thread_id: UUID
|
||||
score: float
|
||||
highlights: dict[str, str] # field -> HTML-escaped snippet with <mark>
|
||||
|
||||
class SearchBackend(Protocol):
|
||||
async def index(self, doc: SearchDoc) -> None: ...
|
||||
async def delete(self, message_id: UUID) -> None: ...
|
||||
async def delete_user(self, user_id: UUID) -> None: ...
|
||||
async def query(self, q: SearchQuery, *, limit: int, cursor: str | None
|
||||
) -> tuple[list[SearchHit], str | None]: ...
|
||||
```
|
||||
|
||||
`SearchDoc` fields: `message_id, thread_id, user_id, account_id, mailbox_special_use, from_ (name+email), to, cc, subject, body_text, attachment_names[], has_attachments, is_unread, is_starred, date (unix), size_bytes`. Both backends (pg.py, meili.py) index the same doc. Meilisearch: one index `messages`, filterable = every non-body field incl. `user_id`, searchable = `subject, body_text, from_, to, cc, attachment_names`, sortable = `date`; ranking rules default + `date:desc` tiebreak.
|
||||
|
||||
### 5.3 `events.py` — EventBus
|
||||
|
||||
```python
|
||||
# INTERFACE VERSION: 1
|
||||
class EventBus(Protocol):
|
||||
async def publish(self, event: str, payload: dict) -> None:
|
||||
# Fan-out to Redis pub/sub channel "evt:{event}" AND enqueue arq job
|
||||
# "on_{event}" for durable consumers (indexer, plugins).
|
||||
def subscribe(self, event: str, handler: Handler) -> None: ...
|
||||
```
|
||||
|
||||
Payloads always include `user_id`. Event names are listed in §8.3 and are the only ones allowed.
|
||||
|
||||
## 6. Backend module specifications
|
||||
|
||||
### 6.1 Identity (`app/identity/`)
|
||||
- Registration: validate localpart per §4 rules against reserved list in `identity/reserved.py`; store full handle `localpart@{APP_DOMAIN}` (env `APP_DOMAIN`).
|
||||
- Login: constant-time verify (argon2id, `time_cost=3, memory_cost=64MiB, parallelism=4`); if `totp_secret` set, require valid TOTP (±1 window). Rate limit: 10 attempts / 15 min per handle AND per IP (Redis counters); lockout returns 429 with `Retry-After`.
|
||||
- Session cookie: `__Host-session`, httpOnly, Secure, SameSite=Lax. CSRF: double-submit token header `X-CSRF` for all mutating requests.
|
||||
- TOTP setup: `POST /auth/totp/setup` returns provisioning URI + QR payload; enable requires one valid code; secret stored envelope-encrypted.
|
||||
- Password reset: only if `recovery_email` verified; single-use token, 30 min TTL, Redis-backed.
|
||||
|
||||
### 6.2 Accounts & autodiscovery (`app/accounts/`)
|
||||
`discover(email) -> list[CandidateConfig]` runs the chain, **returning on first success** but recording every step's outcome in the response (`steps: [{method, status, detail}]`) for wizard UX:
|
||||
|
||||
1. **Presets** — `accounts/presets.yaml` (config, not code): match by domain; ships with gmail.com, googlemail.com, outlook.com/hotmail/live, office365, yahoo, icloud/me.com, gmx, wp.pl, onet.pl, o2.pl, interia.pl. Each entry: imap/smtp host+port+tls, auth kind, oauth provider id if applicable.
|
||||
2. **Autoconfig** — GET `https://autoconfig.{domain}/mail/config-v1.1.xml?emailaddress={email}` then `https://{domain}/.well-known/autoconfig/mail/config-v1.1.xml`; parse Thunderbird XML (incomingServer type=imap, outgoingServer type=smtp; `%EMAILADDRESS%`/`%EMAILLOCALPART%` substitution).
|
||||
3. **ISPDB** — GET `https://autoconfig.thunderbird.net/v1.1/{domain}`; same XML format.
|
||||
4. **DNS SRV (RFC 6186)** — `_imaps._tcp.{domain}` (SSL 993), `_imap._tcp` (STARTTLS 143), `_submissions._tcp` (465), `_submission._tcp` (587); lowest priority wins.
|
||||
5. **Probe** — try `{imap,mail}.{domain}` × ports 993(ssl)/143(starttls) and `{smtp,mail}.{domain}` × 465(ssl)/587(starttls); TCP connect + TLS handshake + IMAP/SMTP greeting, 5 s timeout each, all concurrently.
|
||||
|
||||
All HTTP/DNS lookups: 5 s timeout, no redirects off-domain, HTTPS only. `test(config, credentials)`: full IMAP LOGIN + `LIST`, and SMTP EHLO+AUTH (no message sent); errors mapped to precise user-facing strings (`auth_failed`, `tls_failed`, `connect_timeout`, `unknown_host`). On save: encrypt credentials (§10.2), create account row (`state=backfilling`), enqueue `sync.bootstrap_account`.
|
||||
|
||||
### 6.3 Sync engine v2 (`app/sync/`) — sharded, lease-owned, tiered
|
||||
(Authoritative detail: `sync-engine-v2-refactor.md`; this section summarizes the binding rules.)
|
||||
|
||||
- **Two worker roles** (`WORKER_ROLE`): `jobs` = stateless arq consumer, N instances; `sync` = lease claimer + per-account runners, 1..N instances. Compose services `worker-jobs` / `worker-sync`, `nofile` ulimit 65536.
|
||||
- **Lease ownership:** exactly one sync worker owns each live account. Redis SET `sync:live` (API-maintained, DB-reconciled every 300 s); lease `sync:lease:{account_id}` claimed `SET NX EX LEASE_TTL`, renewed every `LEASE_RENEW` via owner-checked Lua; **fencing** — on renewal failure, cancel that account's runner before any further provider/DB op. Worker heartbeats in `sync:workers:{id}`; rebalance releases excess above `target*1.2+5`. Crash failover = lease expiry + reclaim (≤ ~90 s), no coordinator.
|
||||
- **Event-driven lifecycle:** `account.state_changed` adds/removes ids from `sync:live` at runtime — accounts go live ≤ 30 s after creation and stop ≤ 30 s after deletion with no restarts; runners re-verify `state='live'` each cycle (zombie-proof). New core event **`user.active`** (API publishes on login/WS connect) drives tier promotion.
|
||||
- **Bootstrap (unchanged from v1, now a `bulk` job):** `list_mailboxes()` → upsert `mailboxes` (SPECIAL-USE mapping with name heuristics — Sent/Wysłane, Trash/Kosz, etc. in `sync/folder_names.py`); backfill newest-first per mailbox (INBOX first, then sent, then rest); `sync.progress` published per batch; on completion set `state='live'` → the API adds the id to `sync:live` and a sync worker claims it.
|
||||
- **AccountRunner per tier:** *hot* = persistent INBOX IDLE + transient CONDSTORE poll of special folders every `POLL_WARM`; *warm* = transient poll of INBOX+specials every `POLL_WARM`; *cold* = every `POLL_COLD`. Promotion to hot on `user.active` (≤ 5 s) or new mail within `TIER_HOT_ACTIVITY_H`; demotion by inactivity; `HOT_BUDGET_PER_WORKER` degrades least-active non-session accounts. Delta application, UIDVALIDITY wipe-and-rebackfill, expunge soft-handling, and write-then-enqueue outbound flags are unchanged from v1.
|
||||
- **Connection budgets:** ≤ 2 sync-side connections per account (4 total incl. jobs, shared semaphore); process-local per-destination-host semaphore cap `PER_HOST_CONN_CAP`.
|
||||
- **Queues:** `interactive` (send/flags/move/draft jobs, timeout 120 s, concurrency 20) and `bulk` (backfills, timeout 3600 s, concurrency `BULK_CONCURRENCY`) — bulk work can never delay a send.
|
||||
- **Memory bounds:** parse semaphore `PARSE_CONCURRENCY`; raw messages > `STREAM_PARSE_MB` stream to blob and parse from file — peak RAM ≈ `PARSE_CONCURRENCY × STREAM_PARSE_MB` per process regardless of mailbox size.
|
||||
- **Shutdown:** SIGTERM → stop claimer → cancel runners → release leases → drain in < 10 s.
|
||||
|
||||
### 6.4 Indexer (`app/indexer/`)
|
||||
Input: `RemoteMessage`. Steps: store raw blob (sha256, dedupe) → parse with `email` stdlib (`policy.default`) → extract: headers (decode RFC2047), `references_hdr` = parsed `References` + `In-Reply-To` message-ids in order; best text body and best HTML body (multipart/alternative preference: html for display, text for indexing; if only HTML, derive text via nh3 strip); sanitize HTML with nh3 (allowlist: structural tags, inline styles filtered, `cid:` images rewritten to `/api/messages/{id}/cid/{content_id}`, remote `src` rewritten to blocked-placeholder attributes for FR-23) → write `messages`, `body_parts` (kinds `text` and `html_sanitized`), `attachments` → hand to Threader (§7) → `bus.publish("message.new", ...)` (search indexing + thread denormalization are consumers). For non-junk received messages, upsert the `From` address into `known_recipients` (weight 1).
|
||||
|
||||
### 6.5 Outbox (`app/outbox/`)
|
||||
- Drafts: autosave endpoint upserts `drafts` (client debounces 2 s); a background job mirrors drafts to the account's Drafts folder via `append()` at most every 60 s per draft (replace previous UID).
|
||||
- Send: `POST /send {draft_id}` → build MIME (html + auto-generated text alternative via html-to-text; `In-Reply-To`/`References` from `reply_to_message_id`; attachments from blob store) → create `outbox` row with `send_after = now()+UNDO_SECONDS` (env, default 10) → schedule arq job at `send_after`. Undo endpoint flips state to `cancelled` if still in window. The job: SMTP `send()`, then `append()` to Sent (skip for Gmail preset — Gmail self-appends), mark `sent`, publish `message.sent`, and upsert every To/Cc/Bcc address into `known_recipients` (weight 3). Failure → `failed` + WS notification, draft preserved.
|
||||
|
||||
## 7. Threading specification (JWZ, `app/threading_/`)
|
||||
|
||||
### 7.1 Normalization
|
||||
- `message_id_hdr`, `in_reply_to`, each `References` entry: strip angle brackets and whitespace, lowercase the domain part only. Discard obviously invalid ids (no `@`).
|
||||
- Missing/duplicate `Message-ID`: synthesize `sha256(raw)[:32]@synthetic.local` and mark it non-authoritative (never used to join other messages to it).
|
||||
|
||||
### 7.2 Incremental algorithm (per new message)
|
||||
1. Candidate set = all normalized ids in `references_hdr` + `in_reply_to` + own `message_id_hdr`.
|
||||
2. `SELECT DISTINCT thread_id FROM messages WHERE user_id=$1 AND message_id_hdr = ANY($2) AND thread_id IS NOT NULL`.
|
||||
3. 0 threads → subject fallback (§7.3); still none → create new thread.
|
||||
4. 1 thread → attach.
|
||||
5. ≥2 threads → **merge**: keep the thread with the oldest `last_message_at` as canonical, repoint messages of the others, delete emptied thread rows, publish `thread.updated` for all involved.
|
||||
6. Recompute denormalized thread fields (count, unread, participants, snippet = newest message's snippet, has_attachments, account_ids, last_message_at) in one UPDATE.
|
||||
|
||||
Threads are cross-account by design: the candidate lookup is scoped by `user_id`, not `account_id` (invariant I-3). Sent-folder messages join threads exactly like received ones (FR-15).
|
||||
|
||||
### 7.3 Subject fallback
|
||||
Only when reference joining fails: normalize subject (strip repeated `Re:/Fw:/Fwd:/Odp:/PD:` prefixes case-insensitively, collapse whitespace); join to a thread with the same `subject_norm` that has activity within 7 days AND shares ≥1 participant email; never join two messages that both lack references AND have empty subjects.
|
||||
|
||||
### 7.4 Test vectors (must pass in `tests/threading/`)
|
||||
Linear chain; chain arriving out of order; reply missing `In-Reply-To` but carrying `References`; thread merge on late-arriving parent; broken client (subject-only) joins; distinct-topic same-subject ("hi") does NOT join; cross-account merge (message in account A references message in account B); duplicate Message-ID; synthetic-ID isolation.
|
||||
|
||||
## 8. Search specification
|
||||
|
||||
### 8.1 Pipeline
|
||||
Consumer of `message.new` / `message.changed` / `message.deleted` → builds `SearchDoc` → active `SearchBackend` (env `SEARCH_BACKEND=pg|meili`). Backfill indexes inline with sync (per batch), never as a separate pass the user waits on (FR-30).
|
||||
|
||||
### 8.2 Query grammar (`search/queryparser.py`, engine-independent)
|
||||
Tokenization: whitespace-split, respecting `"quoted phrases"`. A token matching `key:value` (or `-key:value`) with a known key becomes a filter; everything else joins free text. `-` prefix negates a term or filter. `OR` (uppercase) groups adjacent terms/filters into a disjunction; default combinator is AND.
|
||||
|
||||
| Operator | Filter semantics |
|
||||
|---|---|
|
||||
| `from:` `to:` `cc:` | substring match on name or email of that field |
|
||||
| `subject:` | match constrained to subject |
|
||||
| `account:` | account by `display_name` or email substring |
|
||||
| `in:` | `inbox\|sent\|drafts\|trash\|junk\|archive` (special_use) or folder path |
|
||||
| `is:` | `unread\|read\|starred\|answered` |
|
||||
| `has:attachment` | boolean |
|
||||
| `filename:` | attachment name substring |
|
||||
| `before:` `after:` | `YYYY-MM-DD` |
|
||||
| `newer_than:` `older_than:` | `Nd\|Nw\|Nm\|Ny` |
|
||||
| `larger:` `smaller:` | `NK\|NM` bytes |
|
||||
|
||||
Parser output = `SearchQuery` (§5.2). Unknown `key:` tokens are treated as free text (forward compatibility). Unit tests must cover every operator, negation, quoting, `OR`, and garbage input (parser never raises).
|
||||
|
||||
### 8.3 Event catalog (complete)
|
||||
`message.new`, `message.changed`, `message.deleted`, `message.sent`, `flags.changed`, `thread.updated`, `sync.progress`, `account.state_changed`, `draft.saved`, `outbox.state_changed`, `user.active` (published by the API on login and WS connect; consumed by sync workers for tier promotion — not forwarded to browsers). Payloads: `{user_id, ...ids, minimal-diff fields}` — documented in `docs/events.md` (agent: generate it from code).
|
||||
|
||||
Plugins may additionally publish events namespaced `plugin.{name}.*` (e.g. `plugin.calendar.event_updated`); core code never subscribes to plugin events, but the WS hub forwards them to the owning user like any other event.
|
||||
|
||||
## 9. API, realtime, frontend
|
||||
|
||||
### 9.1 REST API (prefix `/api`, session-cookie auth, JSON; errors: `{error:{code,message}}`)
|
||||
|
||||
**Auth:** `POST /auth/register` · `POST /auth/login` · `POST /auth/logout` · `GET /auth/me` · `GET /auth/sessions` · `DELETE /auth/sessions/{id}` · `POST /auth/totp/setup` · `POST /auth/totp/enable` · `POST /auth/password/reset-request` · `POST /auth/password/reset`
|
||||
|
||||
**Accounts:** `POST /accounts/discover {email}` → `{candidates:[{imap,smtp,source}], steps:[...]}` · `POST /accounts/test {config, credentials}` → `{ok}|{error:{code,detail}}` · `POST /accounts` · `GET /accounts` (incl. `state`, sync progress) · `PATCH /accounts/{id}` · `DELETE /accounts/{id}` (cascades rows, blobs, search docs)
|
||||
|
||||
**Mail:** `GET /threads?view=unified|account:{id}|mailbox:{id}|search&q=&cursor=&limit=50` → thread summaries (denormalized fields only; cursor = `last_message_at,id` keyset) · `GET /threads/{id}?sort=asc|desc` → ordered full messages (bodies of the N newest expanded per client hint `expand=`; older = envelope only) · `POST /threads/{id}/flags {add,remove}` (whole thread) · `POST /messages/{id}/flags` · `POST /messages/{id}/move {mailbox_id}` · `GET /messages/{id}/html` (sanitized, for iframe `srcdoc`) · `GET /messages/{id}/cid/{content_id}` · `GET /attachments/{id}` (Content-Disposition attachment; images/PDF also inline for preview) · `POST /messages/{id}/allow-remote-images` (per-sender allowlist in user settings)
|
||||
|
||||
**Compose:** `POST /drafts` · `PUT /drafts/{id}` · `DELETE /drafts/{id}` · `POST /send {draft_id}` → `{outbox_id, undo_until}` · `POST /send/{outbox_id}/undo` · `GET /recipients/suggest?q=` → top 10 from `known_recipients` (prefix match on email or name, ranked by `times_used` decayed by recency); the composer merges these with any plugin capability providers (§11.3)
|
||||
|
||||
**Search:** `GET /search?q=&cursor=` → `{threads:[...], hits_by_thread, highlights}` (grouped by thread, FR-28; `&group=message` for flat) · `GET/POST/DELETE /saved-searches`
|
||||
|
||||
**Settings:** `GET/PATCH /settings` (incl. `thread_order: "asc"|"desc"`, `tz` IANA timezone — default from browser at registration, undo window, per-sender image allowlist)
|
||||
|
||||
### 9.2 WebSocket `/api/ws`
|
||||
Auth via session cookie. Server → client frames: `{event, payload}` for every §8.3 event belonging to the user (that's the whole contract — the UI refetches via TanStack Query invalidation keyed on ids in the payload). Client → server: `{type:"ping"}` only. Zod-validate all frames.
|
||||
|
||||
### 9.3 Frontend routes
|
||||
`/login`, `/register`, `/mail` (unified inbox), `/mail/a/:accountId`, `/mail/a/:accountId/f/:mailboxId`, `/mail/t/:threadId` (thread pane; on desktop rendered beside the list), `/mail/search?q=`, `/settings/{profile|security|accounts|appearance}`; `/settings/accounts/new` = wizard.
|
||||
|
||||
**Account wizard flow:** step 1 email (+ password field revealed unless preset says oauth) → calls `discover` while typing pauses; step 2 shows resolved config with source badge ("found via ISPDB") and a Test button (auto-run); failure expands the manual form pre-filled with the best guess and the precise error; step 3 = created, live backfill progress bar (WS `sync.progress`) with "start reading now" (list fills as backfill proceeds).
|
||||
|
||||
### 9.4 ThreadView contract (invariant I-5)
|
||||
```
|
||||
ThreadView(props: { messages: Message[] /* ASC always */, order: 'asc'|'desc',
|
||||
composerState, ... })
|
||||
```
|
||||
- Rendering order = `order==='asc' ? messages : [...messages].reverse()`. No other branching on `order` is allowed in layout code.
|
||||
- **Composer anchor** = a slot component rendered adjacent to the newest message: after it when `asc` (bottom of thread), directly after it when `desc` (top of thread, above older messages). Same component, one placement rule: "composer is always attached to the newest message."
|
||||
- Initial scroll: to the newest message (bottom for asc, top for desc). Collapse: all read messages except the newest are collapsed; "Expand all" toggles. Unread messages are never collapsed.
|
||||
- Keyboard: `j`/`k` move in *visual* order; `o`/`Enter` expand; `r` reply, `a` reply-all, `f` forward (open composer at anchor); `e` archive; `/` focus search; `gi` go to unified inbox; `esc` close composer (draft persists). Toolbar has the per-thread order toggle; it updates local state only, and long-presses/menu offers "make default" (PATCH /settings).
|
||||
- Message HTML renders in `<iframe sandbox="" srcdoc={...}>` sized via ResizeObserver postMessage shim; remote images show a "load images / always from this sender" bar when the sanitizer flagged any.
|
||||
|
||||
## 10. Security requirements (hard)
|
||||
|
||||
1. Argon2id params per §6.1; no other password hashing anywhere.
|
||||
2. **Credential vault:** mailbox secrets encrypted with AES-256-GCM under per-account data keys, wrapped by a master key from env `MASTER_KEY` (32 B, base64) — `cryptography` library, key id embedded for future rotation. Decryption code path exists only in `workers/` and `accounts/test`. Secrets never appear in logs, API responses, or exceptions (custom repr).
|
||||
3. CSRF double-submit on all mutating routes; `__Host-` cookie prefix; CSP: `default-src 'self'; frame-src 'self'; img-src 'self' data:` (remote mail images are proxied? **No** — v1 loads allowed remote images inside the sandboxed iframe only, which has its own relaxed CSP via srcdoc; document this in code).
|
||||
4. nh3 sanitization at index time; re-sanitize on render if sanitizer version changed (store `sanitizer_v` on body_parts).
|
||||
5. Rate limits: login (§6.1), `POST /accounts/discover` 10/min/user, `POST /send` 30/h/user (env-tunable).
|
||||
6. All outbound autodiscovery HTTP: HTTPS only, 5 s timeout, response size cap 256 KB, no off-domain redirects (SSRF guard: resolve + reject private IP ranges).
|
||||
7. Every SQL query and search query includes `user_id` (enforced by repository-layer helpers; raw session use in API code is a review error).
|
||||
8. Uploads (attachments to drafts): 25 MB/file, 40 MB/message total, content-type sniffed server-side, stored to blob store, never executed/served with original content type unless in safe list (images/pdf/text) — others force `application/octet-stream` + attachment disposition.
|
||||
|
||||
## 11. Plugin API (freeze in M3 — including the heavy-plugin surfaces below)
|
||||
|
||||
A plugin = Python package `webmail_plugin_<name>` exposing:
|
||||
|
||||
```python
|
||||
plugin = Plugin(
|
||||
name, version,
|
||||
hooks = {event: handler}, # subscribe to §8.3 events
|
||||
router = APIRouter | None, # mounted at /api/plugins/{name}
|
||||
ui_manifest = UiManifest | None, # slots + pages, see below
|
||||
migrations = "migrations/" | None, # 11.1
|
||||
jobs = [JobSpec, ...] | None, # 11.2
|
||||
)
|
||||
```
|
||||
|
||||
Loader discovers plugins via entry points group `webmail.plugins`, forwards declared events, and serves `GET /api/plugins/ui-manifest` for the frontend loader. Plugins may not import from `app.*` internals other than `app.plugins.sdk` (which exposes: DB session factory scoped to plugin tables, EventBus publish under `plugin.{name}.*`, blob store, outbox draft/send helpers, settings accessor, HTTP client with the §10.6 SSRF guard built in).
|
||||
|
||||
Every plugin gets the namespaced KV store (`plugin_kv(plugin, user_id, key)`) for light state. Heavy plugins additionally get:
|
||||
|
||||
### 11.1 Plugin-owned tables
|
||||
A plugin may ship Alembic migrations in its own version branch (`alembic -n plugin_{name}`). All tables MUST be prefixed `plg_{name}_` and every user-data table MUST carry `user_id` (invariant I-7 applies to plugins). Core migrations never reference plugin tables and vice versa. Uninstalling a plugin offers `--drop-data` to run its downgrade chain.
|
||||
|
||||
### 11.2 Plugin background jobs
|
||||
`JobSpec(func, schedule: cron_str | None, singleton: bool)` registers tasks on the shared arq worker: cron-scheduled (e.g. a sync loop trigger) or enqueued on demand from hooks/routes. `singleton=True` guarantees at most one running instance per (plugin, user) via a Redis lock. Plugin jobs run in the same worker process pool with the same retry semantics as core jobs and are visible in worker logs under `plg.{name}.*`.
|
||||
|
||||
### 11.3 Frontend surfaces
|
||||
- **Slots** (lazy-loaded ESM injected into mail UI): `thread.toolbar`, `message.actions`, `message.banner` (a strip rendered above a message body — used for spam warnings, calendar invite cards, and similar per-message notices), `sidebar.section`, `settings.page`.
|
||||
- **Pages** (`app.page`): a plugin may declare full views: `{route: "/calendar", nav: {icon, label, order}, bundle}`. The shell adds the nav entry, and the route (including sub-routes under it) lazy-loads the bundle into the **entire content area** — no mail list/thread components are rendered. Page bundles receive the SDK context (API client, WS event stream filtered to `plugin.{name}.*` + requested core events, design-system components) and are sandboxed to their route: importing mail-component internals is a review error. Removing the plugin removes the nav entry and route cleanly.
|
||||
- **Capabilities**: the manifest may declare named capabilities mapping to plugin endpoints, e.g. `capabilities: {"recipients.suggest": "/api/plugins/contacts/suggest"}`. Core UI features query all providers of a capability they support and merge results client-side (defined capabilities in v1: `recipients.suggest` — used by the composer To/Cc/Bcc fields and by any other plugin needing person-picking, such as calendar attendees). This keeps core backend code plugin-unaware.
|
||||
|
||||
Acceptance test for the API: implement **filters** (rules on `message.new`: match → move/flag) and **labels** entirely as plugins using only slots + KV; then implement the **calendar** (Appendix A) using tables + jobs + a page — all with zero core diffs.
|
||||
|
||||
## 12. Milestones with acceptance criteria
|
||||
|
||||
**M0 — Architecture spike**
|
||||
Login/register (no 2FA), one hardcoded-config IMAP account, backfill+IDLE for INBOX, JWZ, read-only unified list + thread view with working asc/desc toggle and correct composer-anchor placeholder, Postgres FTS free-text search.
|
||||
✅ Accept: sync a real 10k-message Gmail account via app password; threads match Gmail's grouping on a 50-thread sample ≥ 90%; toggle order and the newest message + anchor swap ends correctly; search returns in < 500 ms.
|
||||
|
||||
**M1 — MVP**
|
||||
Full identity (2FA, sessions UI, reset), account wizard with full discovery chain, multi-account + unified inbox with account chips, flags/move/archive both directions, inline reply + reply-all/forward + SMTP send + Sent append, recipient autocomplete from `known_recipients`, drafts autosave, sanitized HTML + blocked remote images, operator search (full §8.2 grammar) on PG FTS, WS live updates.
|
||||
✅ Accept: add gmail.com, outlook.com, and one autoconfig-only domain through the wizard typing only address+password; reply from a thread lands correctly threaded in the recipient's client; all §8.2 operators pass integration tests; `docker-compose up` from clean checkout works.
|
||||
|
||||
**M2 — Hardening / v1.0**
|
||||
OAuth2 (Google + Microsoft, XOAUTH2 for IMAP/SMTP, server-side refresh), QRESYNC fast resync, drafts mirrored to server, undo-send, attachment upload/preview, Meilisearch backend + instant search-as-you-type UI + saved searches as virtual mailboxes, session revocation, backfill progress UX, folder-name i18n heuristics.
|
||||
✅ Accept: typo query (`invocie`) finds "invoice" mail < 200 ms on a 100k-message corpus; kill -9 the worker mid-backfill and restart resumes without duplicates (idempotency via `(mailbox_id, uid)` unique key); undo-send within window results in no SMTP traffic.
|
||||
|
||||
**M3 — Plugin platform**
|
||||
Loader, full SDK per §11 **including** plugin migrations (11.1), plugin jobs (11.2), and `app.page` (11.3); filters + labels shipped as plugins; plugin docs.
|
||||
✅ Accept: both plugins install/uninstall cleanly with zero core diffs; a third trivial plugin ("Unsubscribe header button" on `message.actions`) can be written from `docs/plugins.md` alone; a skeleton page-plugin (route + nav entry + one plugin table + one cron job) round-trips install → run → uninstall `--drop-data` leaving no residue.
|
||||
|
||||
**M4 — Calendar plugin (Appendix A)**
|
||||
Phases A1–A2: local calendars + full page views + iMIP invite handling; then A3: CalDAV client sync + ICS subscriptions.
|
||||
✅ Accept: criteria in Appendix A §A.7.
|
||||
|
||||
**M5 — Contacts plugin (Appendix B)**
|
||||
Phases B1–B2: contact books, full page, groups, add-from-mail, `recipients.suggest` capability, vCard import/export; then B3: CardDAV client sync. May be built in parallel with M4 — both depend only on the M3 SDK.
|
||||
✅ Accept: criteria in Appendix B §B.7.
|
||||
|
||||
**M6 (optional) — Hosted mailboxes**
|
||||
Inbound MTA (Stalwart) delivering to a `NativeProvider`; handle becomes receiving address. Specified separately when reached.
|
||||
|
||||
**M7 — Anti-spam plugin (Appendix C)**
|
||||
May be built any time after M3 (depends only on the plugin SDK; parallelizable with M4/M5). Phases C1–C2 client-side; C3 activates at the M6 edge.
|
||||
✅ Accept: criteria in Appendix C §C.8.
|
||||
|
||||
## 13. Testing & CI
|
||||
|
||||
- **Unit:** pytest + pytest-asyncio; threading vectors (§7.4); query parser (§8.2); autodiscovery resolver with `respx`-mocked HTTP + `dnspython` stub; MIME corpus in `tests/fixtures/mime/` (agent: include ≥ 25 nasty real-world-style fixtures — encoded headers incl. RFC2047 Polish diacritics, nested multiparts, missing Message-ID, malformed dates, HTML-only, inline cid images, 8-bit subjects, winmail.dat placeholder).
|
||||
- **Integration:** docker-compose.ci.yml with Greenmail (IMAP+SMTP) and Dovecot; full loop test: append message to Greenmail → appears in API < 10 s → reply via API → arrives in Greenmail SMTP sink with correct References.
|
||||
- **Frontend:** vitest + React Testing Library; ThreadView ordering/anchor tests for both directions; Playwright smoke: login → add Greenmail account via wizard → read → inline reply → search.
|
||||
- **CI gates:** ruff, mypy --strict, pytest, tsc, eslint, vitest, Playwright smoke; coverage floor 80% on `threading_`, `search/queryparser`, `accounts/autodiscovery`, `indexer`.
|
||||
|
||||
## 14. Configuration (env, complete list)
|
||||
|
||||
`APP_DOMAIN`, `DATABASE_URL`, `REDIS_URL`, `MASTER_KEY`, `BLOB_ROOT` or `S3_*`, `SEARCH_BACKEND=pg|meili`, `MEILI_URL`, `MEILI_KEY`, `UNDO_SECONDS=10`, `SESSION_TTL_DAYS=30`, `OAUTH_GOOGLE_CLIENT_ID/SECRET`, `OAUTH_MS_CLIENT_ID/SECRET`, `MAX_ATTACHMENT_MB=25`, `LOG_LEVEL`; sync engine v2 (§6.3): `WORKER_ROLE=jobs|sync`, `MAX_ACCOUNTS_PER_WORKER=400`, `LEASE_TTL=60`, `LEASE_RENEW=20`, `POLL_WARM=180`, `POLL_COLD=900`, `TIER_HOT_ACTIVITY_H=24`, `TIER_SESSION_GRACE_MIN=30`, `TIER_COLD_AFTER_D=3`, `HOT_BUDGET_PER_WORKER=250`, `PER_HOST_CONN_CAP=50`, `BULK_CONCURRENCY=2`, `PARSE_CONCURRENCY=4`, `STREAM_PARSE_MB=8`. All read once in `config.py` (pydantic-settings); no `os.environ` elsewhere.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A — Calendar plugin (`webmail_plugin_calendar`)
|
||||
|
||||
The calendar is the reference "heavy plugin": it lives in the same app and repo, uses the §11 SDK only, and must build with **zero core diffs**. It ships as a page (`/calendar`) beside Mail — one product, one login.
|
||||
|
||||
### A.1 Scope by phase
|
||||
- **A1 — Local calendar:** create/edit/delete events, multiple calendars per user, month/week/day/agenda views, recurrence, reminders (WS notification), timezone-correct display.
|
||||
- **A2 — Invites over mail (iMIP):** detect `text/calendar` parts on `message.new`; render an invite card in the thread (`message.actions` + inline summary); Accept/Tentative/Decline sends an iTIP REPLY through the outbox; organizer flow sends REQUEST/CANCEL to attendees; updates (SEQUENCE bumps) modify the stored event.
|
||||
- **A3 — External sources:** `CalendarProvider` interface with **CalDAV client** (first implementation; covers iCloud, Fastmail, Nextcloud) and read-only **ICS URL subscriptions** (holidays, team feeds); sync loop as a plugin job.
|
||||
- **A4 (deferred, spec later):** Google/Microsoft API providers reusing M2 OAuth machinery; secret **ICS export feed** `GET /api/plugins/calendar/feed/{token}.ics`; CalDAV **server** endpoint for native phone sync — architecturally anticipated, not built now.
|
||||
|
||||
### A.2 Dependencies
|
||||
`icalendar ~5.0` (parse/serialize RFC 5545), `python-dateutil` (RRULE expansion), `caldav ~1.3` (A3), `tzdata`. No other additions.
|
||||
|
||||
### A.3 Data model (plugin migrations, prefix `plg_calendar_`)
|
||||
```sql
|
||||
CREATE TABLE plg_calendar_calendars (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
name text NOT NULL,
|
||||
color text NOT NULL DEFAULT '#3b82f6',
|
||||
kind text NOT NULL DEFAULT 'local', -- local|caldav|ics_subscription
|
||||
source_config jsonb NOT NULL DEFAULT '{}', -- caldav url/collection, ics url
|
||||
secret_ct bytea, -- caldav credentials, §10.2 vault scheme
|
||||
sync_token text, -- CalDAV sync-token / ICS etag
|
||||
read_only boolean NOT NULL DEFAULT false,
|
||||
UNIQUE (user_id, name)
|
||||
);
|
||||
|
||||
CREATE TABLE plg_calendar_events (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
calendar_id uuid NOT NULL REFERENCES plg_calendar_calendars(id) ON DELETE CASCADE,
|
||||
ical_uid text NOT NULL, -- UID from iCalendar; join key for iMIP updates
|
||||
sequence int NOT NULL DEFAULT 0,
|
||||
summary text NOT NULL DEFAULT '',
|
||||
description text NOT NULL DEFAULT '',
|
||||
location text NOT NULL DEFAULT '',
|
||||
dtstart timestamptz NOT NULL,
|
||||
dtend timestamptz,
|
||||
all_day boolean NOT NULL DEFAULT false,
|
||||
tz text NOT NULL DEFAULT 'UTC', -- original TZID; render in user tz, store UTC
|
||||
rrule text, -- raw RRULE string, NULL = single
|
||||
exdates timestamptz[] NOT NULL DEFAULT '{}',
|
||||
recurrence_id timestamptz, -- non-NULL = detached exception instance
|
||||
organizer jsonb, -- {name,email}
|
||||
attendees jsonb NOT NULL DEFAULT '[]', -- [{name,email,partstat,role}]
|
||||
status text NOT NULL DEFAULT 'confirmed', -- confirmed|tentative|cancelled
|
||||
my_partstat text, -- needs-action|accepted|declined|tentative
|
||||
reminders int[] NOT NULL DEFAULT '{}', -- minutes before start
|
||||
origin_message_id uuid, -- link back to the invite mail thread
|
||||
raw_ical text, -- round-trip fidelity for iMIP/CalDAV
|
||||
UNIQUE (calendar_id, ical_uid, recurrence_id)
|
||||
);
|
||||
CREATE INDEX ON plg_calendar_events (user_id, dtstart);
|
||||
```
|
||||
|
||||
**Recurrence rule:** store the RRULE string; expand instances at query time with `dateutil.rrule` over the requested view range only (no materialization table). Exceptions: `exdates` remove instances; a row with `recurrence_id` overrides one instance. Range queries must account for events starting before the window but recurring into it (expand from `dtstart`, cap iterations at 1000 per event).
|
||||
|
||||
### A.4 CalendarProvider interface (inside the plugin, mirrors §5.1)
|
||||
```python
|
||||
# PLUGIN INTERFACE VERSION: 1
|
||||
class CalendarProvider(Protocol):
|
||||
async def verify(self) -> None: ...
|
||||
async def list_collections(self) -> list[RemoteCalendar]: ...
|
||||
async def pull(self, collection: str, *, sync_token: str | None
|
||||
) -> tuple[list[RemoteEvent], list[str], str]: ...
|
||||
# returns (changed_events_as_ical, deleted_uids, new_sync_token)
|
||||
async def push(self, collection: str, ical: str, *, uid: str,
|
||||
etag: str | None) -> str: ... # returns new etag
|
||||
async def delete(self, collection: str, uid: str, etag: str | None) -> None: ...
|
||||
```
|
||||
Implementations in A3: `CaldavProvider`, `IcsSubscriptionProvider` (pull-only, `push`/`delete` raise `ReadOnlyCalendar`). Sync loop = plugin cron job (`schedule="*/5 * * * *"`, `singleton=True` per user) doing pull → upsert by `(calendar_id, ical_uid, recurrence_id)` → local-dirty push with etag conflict handling (conflict ⇒ server wins, local copy saved as a duplicate marked `status='tentative'`, user notified).
|
||||
|
||||
### A.5 iMIP flow (A2)
|
||||
- **Inbound:** hook on `message.new`; if a `text/calendar` part exists, parse METHOD:
|
||||
`REQUEST` → upsert event (match `ical_uid`; apply only if `SEQUENCE` ≥ stored) into the user's default calendar with `my_partstat='needs-action'`, set `origin_message_id`; `CANCEL` → `status='cancelled'`; `REPLY` → update that attendee's `partstat` on the organizer's copy.
|
||||
- **RSVP:** invite card buttons call `POST /api/plugins/calendar/events/{id}/rsvp {partstat}` → update row → build iTIP REPLY ics → send via SDK outbox helper to the organizer (subject `Accepted: {summary}` etc.) → publish `plugin.calendar.event_updated`.
|
||||
- **Organizer:** creating an event with attendees sends METHOD:REQUEST to each; edits bump `SEQUENCE` and re-send; deletion sends CANCEL. All outgoing invite mail uses the account whose address matches the organizer identity (picker if ambiguous).
|
||||
|
||||
### A.6 UI (page `/calendar`, nav icon between Mail and Settings)
|
||||
Routes: `/calendar` (month default), `/calendar/{month|week|day|agenda}/:date`, `/calendar/event/:id`, `/calendar/settings` (calendars CRUD, CalDAV/ICS wizard reusing the account-wizard UX pattern: URL+credentials → test → collection picker). Interactions: click-drag create, drag-move/resize (writes optimistically, reverts on API error), event popover with edit/RSVP, mini-month navigator, calendar visibility toggles (client state), `t` = today, arrows = prev/next period. Invite linkage both ways: invite card in thread links to `/calendar/event/:id`; event view links back to the mail thread when `origin_message_id` is set. Live updates via WS `plugin.calendar.*` events. Timezone: render in user setting `settings.tz` (add to §9.1 settings; default from browser at registration).
|
||||
|
||||
### A.7 Acceptance criteria
|
||||
1. Plugin installs/uninstalls (`--drop-data`) with zero diffs in `app/` and `frontend/src/` outside `plugins/`/`slots/` loader code.
|
||||
2. A weekly-recurring event with two EXDATEs and one moved instance renders correctly in month and week views across a DST boundary (Europe/Warsaw fixtures).
|
||||
3. Round-trip with Google Calendar **via iMIP mail only**: receive a Google invite → Accept in our UI → organizer's Google Calendar shows attendee accepted; our organizer invite → Google attendee accepts → our event shows `partstat=accepted`.
|
||||
4. CalDAV sync against a Nextcloud/Radicale test container: create/edit/delete propagate both directions in ≤ 5 min (one cron cycle); conflict case produces the A.4 tentative-duplicate, never silent loss.
|
||||
5. ICS subscription to a public holiday feed renders read-only events; editing is blocked in UI and API (403 `read_only_calendar`).
|
||||
6. Killing the worker mid-sync and restarting causes no duplicate events (upsert key discipline).
|
||||
|
||||
---
|
||||
|
||||
## Appendix B — Contacts plugin (`webmail_plugin_contacts`)
|
||||
|
||||
The second heavy plugin: same rules as the calendar — §11 SDK only, zero core diffs, own page. Division of labor with core: core owns *implicit* recipient history (`known_recipients`, §4/§9.1) so autocomplete works even without this plugin; the Contacts plugin owns *explicit*, user-curated contact records and enriches suggestions via the `recipients.suggest` capability (§11.3).
|
||||
|
||||
### B.1 Scope by phase
|
||||
- **B1 — Local contacts:** address books, contact CRUD with structured fields, groups, photo, search, dedicated page `/contacts`; "Add sender to contacts" from the thread view; suggestion capability wired into the composer.
|
||||
- **B2 — Portability:** vCard 3.0/4.0 import (file, multi-card) and export (single contact / book), duplicate detection + merge tool.
|
||||
- **B3 — External sources:** `ContactProvider` interface with **CardDAV client** sync (iCloud, Fastmail, Nextcloud, Radicale); per-book sync job.
|
||||
- **B4 (deferred, spec later):** Google People / Microsoft Graph providers (reusing M2 OAuth), CardDAV **server** endpoint for native phone sync, book sharing.
|
||||
|
||||
### B.2 Dependencies
|
||||
`vobject ~0.9` (vCard parse/serialize). Photos stored via the SDK blob store (JPEG/PNG, re-encoded server-side, ≤ 512×512). Nothing else.
|
||||
|
||||
### B.3 Data model (plugin migrations, prefix `plg_contacts_`)
|
||||
```sql
|
||||
CREATE TABLE plg_contacts_books (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
name text NOT NULL,
|
||||
kind text NOT NULL DEFAULT 'local', -- local|carddav
|
||||
source_config jsonb NOT NULL DEFAULT '{}',-- carddav url/collection
|
||||
secret_ct bytea, -- carddav credentials, §10.2 vault scheme
|
||||
sync_token text,
|
||||
read_only boolean NOT NULL DEFAULT false,
|
||||
UNIQUE (user_id, name)
|
||||
);
|
||||
|
||||
CREATE TABLE plg_contacts_contacts (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
book_id uuid NOT NULL REFERENCES plg_contacts_books(id) ON DELETE CASCADE,
|
||||
vcard_uid text NOT NULL, -- UID property; join key for CardDAV
|
||||
etag text, -- server etag (carddav books)
|
||||
name_display text NOT NULL DEFAULT '', -- FN
|
||||
name_parts jsonb NOT NULL DEFAULT '{}', -- {given,family,middle,prefix,suffix} (N)
|
||||
emails jsonb NOT NULL DEFAULT '[]', -- [{value,type,pref}]
|
||||
phones jsonb NOT NULL DEFAULT '[]',
|
||||
addresses jsonb NOT NULL DEFAULT '[]', -- structured ADR
|
||||
org text NOT NULL DEFAULT '',
|
||||
title text NOT NULL DEFAULT '',
|
||||
birthday date,
|
||||
urls jsonb NOT NULL DEFAULT '[]',
|
||||
notes text NOT NULL DEFAULT '',
|
||||
photo_sha256 text,
|
||||
raw_vcard text, -- round-trip fidelity: preserve unknown props
|
||||
updated_at timestamptz NOT NULL DEFAULT now(),
|
||||
UNIQUE (book_id, vcard_uid)
|
||||
);
|
||||
CREATE INDEX ON plg_contacts_contacts (user_id, name_display);
|
||||
CREATE INDEX ON plg_contacts_contacts USING gin (emails jsonb_path_ops);
|
||||
|
||||
CREATE TABLE plg_contacts_groups (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
name text NOT NULL,
|
||||
UNIQUE (user_id, name)
|
||||
);
|
||||
CREATE TABLE plg_contacts_group_members (
|
||||
group_id uuid NOT NULL REFERENCES plg_contacts_groups(id) ON DELETE CASCADE,
|
||||
contact_id uuid NOT NULL REFERENCES plg_contacts_contacts(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (group_id, contact_id)
|
||||
);
|
||||
```
|
||||
|
||||
vCard mapping discipline: parse known properties into columns, keep the full source in `raw_vcard`, and on export/push regenerate from columns **merged over** `raw_vcard` so properties we don't model (e.g. `X-` extensions) survive round-trips.
|
||||
|
||||
### B.4 ContactProvider interface (inside the plugin, mirrors A.4)
|
||||
```python
|
||||
# PLUGIN INTERFACE VERSION: 1
|
||||
class ContactProvider(Protocol):
|
||||
async def verify(self) -> None: ...
|
||||
async def list_books(self) -> list[RemoteBook]: ...
|
||||
async def pull(self, book: str, *, sync_token: str | None
|
||||
) -> tuple[list[RemoteVcard], list[str], str]: ...
|
||||
# (changed vcards, deleted uids, new sync_token)
|
||||
async def push(self, book: str, vcard: str, *, uid: str,
|
||||
etag: str | None) -> str: ...
|
||||
async def delete(self, book: str, uid: str, etag: str | None) -> None: ...
|
||||
```
|
||||
Sync loop = plugin cron job (`*/5 * * * *`, singleton per user), upsert by `(book_id, vcard_uid)`, etag conflict ⇒ server wins + local copy kept as a duplicate flagged for the merge tool (never silent loss) — same policy as A.4.
|
||||
|
||||
### B.5 API & capability (`/api/plugins/contacts/`)
|
||||
`GET /contacts?book=&group=&q=&cursor=` · `POST /contacts` · `GET/PUT/DELETE /contacts/{id}` · `POST /contacts/from-message {message_id}` (prefill from From header) · `GET/POST/DELETE /books`, `/groups`, group membership ops · `POST /import` (vCard file → report `{created, updated, skipped}`) · `GET /export?book=|contact=` (`.vcf`) · `GET /duplicates` (candidate pairs: same email, or same normalized name + fuzzy) · `POST /merge {keep_id, remove_id}` (field union, keep wins conflicts, group memberships merged) · **`GET /suggest?q=`** — the `recipients.suggest` capability endpoint: matches name/email prefix and substring, returns `[{name, email, source:"contact", contact_id, photo_url?}]`. Composer merge rule (client-side, defined here so behavior is deterministic): contact hits rank above `known_recipients` hits at equal prefix quality; dedupe by email, contact entry wins and carries the photo/display name.
|
||||
|
||||
### B.6 UI
|
||||
Page `/contacts` (nav entry after Calendar): three-pane layout — books/groups sidebar, alphabetically grouped virtualized list with search-as-you-type, detail pane with edit-in-place form; import/export and merge review under `/contacts/tools`; CardDAV book wizard reusing the account-wizard UX pattern (URL + credentials → test → book picker). Mail integration via `message.actions` slot: "Add to Contacts" (hidden when the sender already matches a contact; then it becomes "View contact" linking to `/contacts/{id}`). Group compose: selecting a group in the composer expands to its members' preferred emails.
|
||||
|
||||
### B.7 Acceptance criteria
|
||||
1. Zero core diffs; install/uninstall `--drop-data` round-trip clean.
|
||||
2. Import a 500-card vCard file mixing v3.0/v4.0, quoted-printable encoding, and Polish diacritics in names (`Grzegorz Brzęczyszczykiewicz`) — all names, emails, and photos correct; re-export → re-import is lossless including `X-` properties (raw_vcard merge discipline).
|
||||
3. Composer typing a 3-char prefix returns merged suggestions in < 100 ms p95 (contacts ranked above history, deduped by email, photos shown for contacts).
|
||||
4. CardDAV sync against Radicale: create/edit/delete propagate both directions within one cron cycle; concurrent edit conflict produces a flagged duplicate, never silent loss; worker kill mid-sync ⇒ no duplicate contacts on restart.
|
||||
5. "Add sender to contacts" from a thread creates a prefilled contact; the same sender's messages immediately render the contact's display name, and the action flips to "View contact".
|
||||
6. Merge tool: merging two contacts unions emails/phones/groups, keeps the chosen primary's conflicting fields, and repoints nothing else in the system (calendar attendees and mail remain keyed by email, by design).
|
||||
|
||||
---
|
||||
|
||||
## Appendix C — Anti-spam plugin (`webmail_plugin_antispam`)
|
||||
|
||||
Third heavy plugin; §11 SDK only, zero core diffs. Architectural context: for **synced** accounts the upstream provider has usually already filtered spam, so this plugin is a *second-pass* filter (weak-provider accounts, unified experience, per-user trainable Bayes). It becomes the *first-pass* filter only for M6 hosted mailboxes, where the same engine also runs at the SMTP edge (§C.7).
|
||||
|
||||
### C.1 Scope by phase
|
||||
- **C1 — Classification & actions:** score every newly synced inbox message; move high-scoring mail to Junk (through the existing move machinery, so the verdict propagates to the IMAP server); banner low-confidence suspects; per-user allow/deny lists; per-account enable toggle.
|
||||
- **C2 — Learning loop:** explicit "Spam"/"Not spam" actions train per-user Bayes; passive learning from user moves in/out of Junk and (sampled, capped) from provider-classified Junk; stats page.
|
||||
- **C3 (M6 era):** the same Rspamd configuration/expertise deployed at the mail edge (VPS), where full IP/SPF/DKIM/DMARC/greylisting checks are actually meaningful; hosted-mailbox delivery is filtered *before* it reaches the index.
|
||||
|
||||
### C.2 Engine decision
|
||||
**Rspamd** as a service container (mature, fast, HTTP API) with a small **dedicated Redis** container for its Bayes/fuzzy storage (isolated from the app Redis on purpose — different persistence and flush semantics). No Python spam engine is written from scratch; the plugin is glue + policy. Compose additions: `rspamd` (`rspamd/rspamd:3.x`) + `redis-rspamd` (`redis:7`, `--save 300 1`). Rspamd's network checks (RBL/SURBL/fuzzy lookups leak message-derived hashes/URLs to third parties) are **off by default**, one settings toggle to enable (privacy default matches the product's self-hosted ethos).
|
||||
|
||||
### C.3 Data model (plugin migrations, prefix `plg_antispam_`)
|
||||
```sql
|
||||
CREATE TABLE plg_antispam_settings (
|
||||
user_id uuid PRIMARY KEY,
|
||||
enabled_accounts uuid[] NOT NULL DEFAULT '{}', -- empty = all accounts
|
||||
move_threshold real NOT NULL DEFAULT 12.0, -- score >= : move to Junk
|
||||
banner_threshold real NOT NULL DEFAULT 6.0, -- score >= : warn banner
|
||||
network_checks boolean NOT NULL DEFAULT false,
|
||||
learn_from_provider_junk boolean NOT NULL DEFAULT true
|
||||
);
|
||||
CREATE TABLE plg_antispam_rules (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL,
|
||||
kind text NOT NULL, -- allow|deny
|
||||
pattern citext NOT NULL, -- email or @domain
|
||||
UNIQUE (user_id, kind, pattern)
|
||||
);
|
||||
CREATE TABLE plg_antispam_verdicts (
|
||||
message_id uuid PRIMARY KEY,
|
||||
user_id uuid NOT NULL,
|
||||
score real NOT NULL,
|
||||
symbols jsonb NOT NULL DEFAULT '[]', -- rspamd symbol list for explainability
|
||||
action text NOT NULL, -- none|banner|moved|allow_listed|deny_listed
|
||||
learned text -- spam|ham|NULL
|
||||
);
|
||||
```
|
||||
|
||||
### C.4 Classification flow
|
||||
Hook on `message.new`; process only messages landing in `special_use='inbox'` mailboxes of enabled accounts (never Sent/Drafts/Junk/Trash; never re-classify on flag changes). Order: (1) deny list → move to Junk, action `deny_listed`; (2) allow list → action `allow_listed`, stop; (3) POST raw blob to Rspamd `/checkv2` with `Deliver-To`-style per-user context header (per-user Bayes statistics via Rspamd user settings); (4) apply thresholds → move via the existing move job (verdict thus syncs to the server's Junk folder) or record `banner`; (5) write verdict row, publish `plugin.antispam.classified`. Rspamd unavailability = fail-open (deliver normally, log, retry queue for learning only — spam filtering must never delay or lose mail).
|
||||
|
||||
### C.5 Learning
|
||||
- Explicit: `message.actions` buttons "Spam" / "Not spam" → `POST /api/plugins/antispam/learn {message_id, verdict}` → Rspamd `/learnspam|/learnham` with the per-user context → move message accordingly → update `verdicts.learned`.
|
||||
- Passive: hook on `flags.changed`/move events — user moves message out of Junk ⇒ learn ham; into Junk ⇒ learn spam (dedup via `verdicts.learned`).
|
||||
- Provider bootstrap (per account, if `learn_from_provider_junk`): during/after backfill, sample up to 500 provider-Junk messages as spam corpus and 500 aged inbox messages as ham, rate-limited background job. Gives per-user Bayes a warm start.
|
||||
|
||||
### C.6 API & UI
|
||||
`GET/PATCH /api/plugins/antispam/settings` · `GET/POST/DELETE /rules` · `POST /learn` · `GET /stats` (verdict counts, top symbols, Bayes confidence). UI: `settings.page` (thresholds with plain-language labels, per-account toggles, lists editor, privacy toggle, stats); `message.banner` for the suspect zone ("Looks like spam — [Move to Junk] [Not spam] [Always trust sender]"); `message.actions` Spam/Not-spam. Junk folder view is core (SPECIAL-USE) and needs no changes.
|
||||
|
||||
### C.7 M6 edge integration (assumption, specified fully with M6)
|
||||
At the hosted-mail phase the VPS edge (Stalwart's built-in filter or Postfix+Rspamd) performs first-pass filtering with the checks that only make sense at SMTP time (connecting IP, SPF/DKIM/DMARC verification, greylisting, rate limits); accepted mail is tagged with its edge score in a header, and this plugin trusts that header for hosted accounts instead of re-scanning — one user-visible spam system, two enforcement points.
|
||||
|
||||
### C.8 Acceptance criteria
|
||||
1. Zero core diffs; install/uninstall `--drop-data` clean; with the plugin absent, mail flow is byte-identical to v1.4 core.
|
||||
2. Rspamd container stopped ⇒ new mail still indexes and renders normally within normal latency (fail-open proven by test).
|
||||
3. Verdict propagation: a moved-to-Junk verdict appears in the provider's Junk folder via IMAP within one sync cycle; "Not spam" moves it back and the same message is not re-classified.
|
||||
4. Learning: after 20 spam + 20 ham teachings on a fixture corpus, a held-out obvious-spam fixture scores above `move_threshold` for that user and a ham fixture below `banner_threshold` (per-user isolation: a second user's scores are unaffected).
|
||||
5. Deny-listed sender is moved without engine involvement; allow-listed sender bypasses even a high engine score.
|
||||
6. Privacy: with `network_checks=false` (default), a network-capture test during classification shows zero outbound connections from Rspamd.
|
||||
|
||||
---
|
||||
*End of specification. Ambiguities: prefer the simplest behavior consistent with the invariants in §1, leave a `TODO(spec)` comment, and list open questions in `docs/open-questions.md` rather than guessing silently.*
|
||||
@@ -0,0 +1,103 @@
|
||||
How the worker functions (multi-user / multi-account)
|
||||
=====================================================
|
||||
|
||||
The worker wears two hats (in one process, one event loop)
|
||||
----------------------------------------------------------
|
||||
It is a single arq process (`arq workers.worker.WorkerSettings`) running one
|
||||
asyncio event loop. Inside it, two very different things run:
|
||||
|
||||
A) A job runner (the Redis queue side). The API enqueues discrete jobs; the
|
||||
worker pulls and runs them, up to `max_jobs = 10` concurrently,
|
||||
`job_timeout = 3600s`. The registered jobs:
|
||||
sync_bootstrap_account (backfill a new account), push_flags_job, move_job,
|
||||
move_thread_to_folder_job, delete_thread_job, send_job,
|
||||
push_draft_to_imap_job, remove_imap_draft_job, refresh_account_job.
|
||||
Each job builds a provider (which is where the mailbox credential is
|
||||
decrypted with the MASTER_KEY - worker-only), does the IMAP/SMTP op, and
|
||||
closes the connection. On failure they retry with backoff (~x5) and then
|
||||
revert or mark failed.
|
||||
|
||||
B) A live-sync supervisor (long-running loops). Started once in on_startup and
|
||||
is NOT part of the job queue.
|
||||
|
||||
|
||||
The per-account model (the key point)
|
||||
-------------------------------------
|
||||
On startup, start_supervisor reads every account with state = "live" and, for
|
||||
each account, spawns two persistent asyncio tasks:
|
||||
|
||||
1. INBOX IDLE loop (live_loop) - opens one IMAP connection and keeps it open,
|
||||
blocking on IMAP IDLE (server push). It reacts within seconds when mail
|
||||
arrives, and re-issues IDLE at least every 25 min. This is a *persistent*
|
||||
connection held 24/7.
|
||||
|
||||
2. Secondary poll loop (secondary_poll_loop) - every POLL_INTERVAL (120 s) it
|
||||
opens a fresh, transient connection, CONDSTORE-polls
|
||||
Sent/Drafts/Archive/Trash/Junk (cheap MODSEQ deltas), and closes it.
|
||||
|
||||
So the steady-state cost scales linearly with the total number of email
|
||||
accounts across all users:
|
||||
|
||||
- Persistent IMAP connections : ~1 per account (the INBOX IDLE) held
|
||||
continuously
|
||||
- asyncio tasks : 2 per account
|
||||
- Transient connections : one burst per account every 120 s (secondary
|
||||
poll) + per job
|
||||
|
||||
For 3 users x 3 accounts that is ~9 always-open IDLE connections and 18 tasks -
|
||||
trivial. For 500 accounts it is ~500 sockets open 24/7 plus 1000 tasks, and you
|
||||
will hit IMAP server per-user/global connection limits, file-descriptor limits,
|
||||
and memory pressure long before anything else.
|
||||
|
||||
|
||||
How updates reach the browsers
|
||||
------------------------------
|
||||
The worker never talks to browsers. When it persists a change it publishes to
|
||||
Redis pub/sub (evt:{event}); the API's WebSocket hub relays each event to the
|
||||
owning user's connected clients, which invalidate the relevant queries. So Redis
|
||||
is both the job queue and the fan-out bus.
|
||||
|
||||
|
||||
Honest scaling limitations
|
||||
--------------------------
|
||||
- Single process / single core. All those loops and jobs share one event loop.
|
||||
It is fine because IMAP work is I/O-bound (mostly waiting), but a heavy
|
||||
backfill or MIME parse briefly blocks everything else, and each raw message is
|
||||
loaded fully into memory to parse + blob-store - so messages x concurrency is
|
||||
your memory driver.
|
||||
|
||||
- The live-sync side does not horizontally scale. If you ran two workers for
|
||||
throughput, both would start IDLE loops for all live accounts -> duplicate
|
||||
connections and double-processing. The job side would shard fine across
|
||||
workers (that is what arq does), but the supervisor assumes exactly one
|
||||
instance.
|
||||
|
||||
- Account discovery is startup-only. New or re-configured accounts are not
|
||||
picked up as live until the worker restarts (the code says so explicitly). A
|
||||
new account does get its one-time backfill immediately (via the enqueued
|
||||
sync_bootstrap_account job), but its ongoing IDLE loop will not start until a
|
||||
restart. The same gap causes the "zombie loop" after account deletion.
|
||||
|
||||
- max_jobs = 10 is global, shared across all users - a burst of sends/moves
|
||||
queues behind that cap.
|
||||
|
||||
|
||||
What a real multi-tenant deployment would add
|
||||
---------------------------------------------
|
||||
- A pool of sync workers with account sharding: each account owned by exactly
|
||||
one worker (e.g., a Redis lease/consistent hashing), so IDLE loops do not
|
||||
duplicate and you can scale out.
|
||||
|
||||
- React to account.state_changed to start/stop loops at runtime (no restart on
|
||||
add/edit/delete).
|
||||
|
||||
- Connection budgeting - e.g., keep IDLE only for recently-active mailboxes and
|
||||
poll the long tail - plus per-server connection caps and backpressure.
|
||||
|
||||
- Move sync_bootstrap_account (a large backfill) onto a separate, higher-timeout
|
||||
queue from the fast interactive jobs (send/flags/move).
|
||||
|
||||
Net: the current design is a clean, correct single-node model - great for a
|
||||
handful of users with a few accounts each - but the per-account persistent IDLE
|
||||
loop and the startup-only, single-supervisor assumptions are the things you
|
||||
would re-architect before going multi-tenant at scale.
|
||||