App passwords; CalDAV / CardDAV server for phones, Thunderbird, macOS

App passwords (SPEC App. D §D.5): one generated password per app, shown once,
revocable on its own, revoked on a password reset; never accepted by the web
sign-in. Settings → Security → App passwords (migration 0024).

CalDAV / CardDAV server (SPEC Appendix E), built in: /dav/ with
/.well-known/caldav|carddav, HTTP Basic with app passwords only (cached checks,
throttled failures). The calendar and contacts plugins serve their own tables as
collections (Plugin(dav=...)):
- calendars as iCalendar: series with overrides, time zones, all-day, attendees
  and RSVP, alarms <-> reminders; MKCALENDAR / PROPPATCH / DELETE of calendars;
  subscriptions read-only; writes to upstream-synced calendars flow on upstream;
- address books as vCard (3.0 served, 3.0/4.0 read), groups as CATEGORIES;
- what a client stores is served back verbatim until the web app edits it;
- sync tokens / CTags from change logs filled by triggers (calendar0008,
  contacts0002), so every write path counts; trimmed nightly after 90 days.
REPORTs: calendar-query (time range), calendar-/addressbook-multiget,
addressbook-query, sync-collection. nginx and the Vite proxy route /dav/.
Settings shows the server and user name to enter; install guide §5.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 21:49:04 +02:00
co-authored by Claude Opus 5.5
parent fa552392c8
commit 0e3793a4d6
30 changed files with 3968 additions and 4 deletions
+4
View File
@@ -49,6 +49,10 @@ def register_routers(app: FastAPI) -> None:
): ):
app.include_router(router, prefix="/api") app.include_router(router, prefix="/api")
app.include_router(ws_router, prefix="/api") app.include_router(ws_router, prefix="/api")
# CalDAV / CardDAV (SPEC Appendix E): at the root, where clients look for it.
from app.dav.routes import router as dav_router
app.include_router(dav_router)
# Mount discovered plugin routers at /api/plugins/{name} (SPEC §11). Hooks are # Mount discovered plugin routers at /api/plugins/{name} (SPEC §11). Hooks are
# subscribed in the lifespan startup so they register once per process. # subscribed in the lifespan startup so they register once per process.
+1
View File
@@ -0,0 +1 @@
"""CalDAV / CardDAV server (SPEC Appendix E)."""
+93
View File
@@ -0,0 +1,93 @@
"""Signing in a DAV client: HTTP Basic with an app password (SPEC App. E.2).
Only app passwords: the account password (with or without 2FA) never works
here, so a lost phone is one revocation away and 2FA cannot be bypassed. A
sync makes many requests, each with the credentials; a successful check is
remembered for a few minutes under a hash of the password, and forgotten as
soon as the user's app passwords change (``app_passwords`` bumps a counter).
"""
from __future__ import annotations
import base64
import binascii
import datetime as dt
import hashlib
import uuid
from sqlalchemy import select, update
from app.db import session_scope
from app.identity import app_passwords
from app.identity.service import resolve_handle
from app.models import User, UserAppPassword
from app.redis_client import get_redis
CACHE_S = 300
FAIL_WINDOW_S = 900
MAX_FAILS_PER_ADDRESS = 10
MAX_FAILS_PER_IP = 30
_LAST_USED_EVERY = dt.timedelta(minutes=5)
class Throttled(Exception):
"""Too many failed sign-ins recently."""
def _credentials(header: str | None) -> tuple[str, str] | None:
if not header or not header.lower().startswith("basic "):
return None
try:
decoded = base64.b64decode(header[6:].strip(), validate=True).decode("utf-8")
except (binascii.Error, UnicodeDecodeError):
return None
login, sep, password = decoded.partition(":")
return (login.strip(), password) if sep and login.strip() else None
async def authenticate(header: str | None, ip: str) -> User | None:
"""The user these Basic credentials sign in, or None. Raises Throttled."""
creds = _credentials(header)
if creds is None:
return None
login, password = creds
redis = get_redis()
handle = resolve_handle(login)
fail_keys = (f"dav:fail:addr:{handle}", f"dav:fail:ip:{ip}")
fails = [int(await redis.get(k) or 0) for k in fail_keys]
if fails[0] >= MAX_FAILS_PER_ADDRESS or fails[1] >= MAX_FAILS_PER_IP:
raise Throttled
async with session_scope() as db:
user = await db.scalar(select(User).where(User.handle == handle))
if user is None or user.blocked_at is not None:
await _failed(fail_keys)
return None
gen = int(await redis.get(app_passwords.generation_key(user.id)) or 0)
digest = hashlib.sha256(f"{user.id}:{password}".encode()).hexdigest()
cache_key = f"dav:auth:{user.id}:{gen}:{digest}"
if await redis.get(cache_key):
return user
row = await app_passwords.verify(db, user.id, password)
if row is None:
await _failed(fail_keys)
return None
await redis.set(cache_key, str(row.id), ex=CACHE_S)
now = dt.datetime.now(dt.UTC)
if row.last_used_at is None or now - row.last_used_at > _LAST_USED_EVERY:
await db.execute(
update(UserAppPassword).where(UserAppPassword.id == row.id).values(last_used_at=now)
)
db.expunge(user)
return user
async def _failed(keys: tuple[str, ...]) -> None:
redis = get_redis()
for key in keys:
await redis.incr(key)
await redis.expire(key, FAIL_WINDOW_S)
def user_id_of(user: User) -> uuid.UUID:
return user.id
+118
View File
@@ -0,0 +1,118 @@
"""What a plugin implements to serve its collections over DAV (SPEC App. E.1).
The core (app/dav/routes.py) speaks the protocol; a provider only knows its
own data: collections (calendars, address books) and the resources in them
(one iCalendar object / one vCard each), named, with an ETag. Every method
gets the signed-in user's id; providers open the user's RLS session
(``sdk(name).session(user_id)``) themselves.
"""
from __future__ import annotations
import uuid
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal, Protocol
class DavError(Exception):
"""A request the provider refuses: ``status`` is the HTTP answer
(403 read-only, 404 missing, 409 conflict, 412 precondition failed, 415
unsupported data); ``condition`` an optional DAV precondition element
(e.g. ``{urn:ietf:params:xml:ns:caldav}valid-calendar-data``)."""
def __init__(self, status: int, message: str = "", condition: str | None = None) -> None:
super().__init__(message or str(status))
self.status = status
self.condition = condition
@dataclass
class Collection:
id: str
displayname: str
sync_token: int # the latest change number; also the CTag
description: str = ""
color: str | None = None # "#rrggbb"
read_only: bool = False
components: list[str] = field(default_factory=lambda: ["VEVENT"]) # calendars only
@dataclass
class Resource:
name: str # last path segment, e.g. "abc.ics"
etag: str # without quotes
data: str | None = None # the iCalendar / vCard text, when asked for
@dataclass
class Changes:
sync_token: int
changed: list[str] # resource names changed or added since the old token
deleted: list[str] # resource names gone since the old token
class DavProvider(Protocol):
kind: Literal["calendar", "addressbook"]
content_type: str # "text/calendar; charset=utf-8" / "text/vcard; charset=utf-8"
extension: str # ".ics" / ".vcf"
async def collections(self, user_id: uuid.UUID) -> list[Collection]: ...
async def resources(self, user_id: uuid.UUID, collection_id: str) -> list[Resource]: ...
async def get(
self, user_id: uuid.UUID, collection_id: str, names: list[str] | None
) -> list[Resource]: ...
async def put(
self,
user_id: uuid.UUID,
collection_id: str,
name: str,
data: str,
*,
if_match: str | None,
if_none_match: bool,
) -> tuple[str, bool]:
"""Store ``data`` as ``name``; returns (new ETag, created)."""
...
async def delete(
self, user_id: uuid.UUID, collection_id: str, name: str, *, if_match: str | None
) -> None: ...
async def changes(self, user_id: uuid.UUID, collection_id: str, since: int) -> Changes | None:
"""What changed after change number ``since``; None when that token is
too old to answer (the client then resyncs from scratch)."""
...
async def create_collection(
self, user_id: uuid.UUID, collection_id: str, displayname: str, color: str | None
) -> None:
"""Create a collection at the id the client chose (MKCALENDAR / MKCOL);
DavError(403) when that id cannot be used."""
...
async def update_collection(
self,
user_id: uuid.UUID,
collection_id: str,
*,
displayname: str | None = None,
color: str | None = None,
description: str | None = None,
) -> None: ...
async def delete_collection(self, user_id: uuid.UUID, collection_id: str) -> None: ...
async def query(
self,
user_id: uuid.UUID,
collection_id: str,
start: datetime | None,
end: datetime | None,
) -> list[str] | None:
"""Calendars: names of resources with an occurrence overlapping the
range; None when the provider cannot filter (the core then returns all)."""
...
+595
View File
@@ -0,0 +1,595 @@
"""The CalDAV / CardDAV endpoints (SPEC Appendix E).
/.well-known/caldav, /.well-known/carddav → 301 /dav/
/dav/ root: points at the principal
/dav/principals/{user}/ the signed-in user
/dav/calendars/{user}/[{calendar}/[{name}]] calendar home, calendars, events
/dav/addressbooks/{user}/[{book}/[{name}]] address-book home, books, contacts
The data comes from the plugins' providers (app/dav/provider.py): the calendar
plugin's for ``calendars``, the contacts plugin's for ``addressbooks``. Signed
in with HTTP Basic and an app password only (app/dav/auth.py); session cookies
are never looked at, so a web page cannot drive these endpoints.
"""
from __future__ import annotations
import logging
import uuid
import xml.etree.ElementTree as ET
from typing import Any
from urllib.parse import quote, unquote, urlsplit
from fastapi import APIRouter, Request, Response
from fastapi.responses import RedirectResponse
from sqlalchemy import select
from app.dav import auth
from app.dav.provider import Collection, DavError, DavProvider, Resource
from app.dav.xml import (
CR,
CS,
A,
BadRequest,
C,
D,
Multistatus,
PropValue,
el,
error_body,
href,
mkcol_props,
parse,
propfind_request,
proppatch_request,
q,
report_request,
)
from app.db import session_scope
from app.models import Account, User
logger = logging.getLogger(__name__)
router = APIRouter()
METHODS = [
"OPTIONS",
"PROPFIND",
"PROPPATCH",
"REPORT",
"GET",
"HEAD",
"PUT",
"DELETE",
"MKCALENDAR",
"MKCOL",
]
DAV_HEADER = "1, 3, calendar-access, addressbook, extended-mkcol"
XML_TYPE = "application/xml; charset=utf-8"
HOMES = {"calendars": "calendar", "addressbooks": "addressbook"}
SYNC_PREFIX = "http://ultimo.email/ns/sync/"
def _providers() -> dict[str, DavProvider]:
from app.plugins import loader
out: dict[str, DavProvider] = {}
for plugin in loader.get_plugins():
if plugin.dav is not None:
out[plugin.dav.kind] = plugin.dav
return out
# ---- entry points ------------------------------------------------------------------
@router.api_route("/.well-known/caldav", methods=["GET", "HEAD", "PROPFIND", "OPTIONS"])
@router.api_route("/.well-known/carddav", methods=["GET", "HEAD", "PROPFIND", "OPTIONS"])
async def well_known() -> Response:
return RedirectResponse("/dav/", status_code=301)
@router.api_route("/dav", methods=METHODS)
@router.api_route("/dav/{path:path}", methods=METHODS)
async def dav(request: Request, path: str = "") -> Response:
method = request.method.upper()
if method == "OPTIONS":
return Response(
status_code=200,
headers={"DAV": DAV_HEADER, "Allow": ", ".join(METHODS), "Content-Length": "0"},
)
ip = request.client.host if request.client else ""
try:
user = await auth.authenticate(request.headers.get("authorization"), ip)
except auth.Throttled:
return Response(status_code=429, headers={"Retry-After": "900"})
if user is None:
return Response(
status_code=401,
headers={"WWW-Authenticate": 'Basic realm="Ultimo", charset="UTF-8"'},
)
parts = [unquote(p) for p in path.split("/") if p]
ctx = _Ctx(request, user, parts)
try:
return await ctx.handle(method)
except BadRequest as err:
return Response(status_code=400, content=str(err))
except DavError as err:
body = error_body(err.condition) if err.condition else b""
return Response(
status_code=err.status,
content=body,
media_type=XML_TYPE if body else None,
)
class _Ctx:
def __init__(self, request: Request, user: User, parts: list[str]) -> None:
self.request = request
self.user = user
self.uid = str(user.id)
self.parts = parts
self.providers = _providers()
# -- paths -----------------------------------------------------------------------
def principal(self) -> str:
return f"/dav/principals/{self.uid}/"
def home(self, segment: str) -> str:
return f"/dav/{segment}/{self.uid}/"
def coll_href(self, segment: str, coll_id: str) -> str:
return f"{self.home(segment)}{quote(coll_id, safe='')}/"
def res_href(self, segment: str, coll_id: str, name: str) -> str:
return f"{self.coll_href(segment, coll_id)}{quote(name, safe='')}"
# -- dispatch --------------------------------------------------------------------
async def handle(self, method: str) -> Response:
p = self.parts
if not p:
return await self.root(method)
if p[0] == "principals":
if len(p) != 2 or p[1] != self.uid:
raise DavError(404)
return await self.principal_req(method)
if p[0] in HOMES and len(p) >= 2:
if p[1] != self.uid:
raise DavError(403)
provider = self.providers.get(HOMES[p[0]])
if provider is None:
raise DavError(404)
if len(p) == 2:
return await self.home_req(method, p[0], provider)
if len(p) == 3:
return await self.collection_req(method, p[0], provider, p[2])
if len(p) == 4:
return await self.resource_req(method, p[0], provider, p[2], p[3])
raise DavError(404)
async def body(self) -> ET.Element | None:
return parse(await self.request.body())
def depth(self) -> int:
value = (self.request.headers.get("depth") or "1").strip().lower()
return 0 if value == "0" else 1
@staticmethod
def multistatus(ms: Multistatus) -> Response:
return Response(status_code=207, content=ms.to_bytes(), media_type=XML_TYPE)
async def respond_props(
self, items: list[tuple[str, dict[str, PropValue]]], method_root: ET.Element | None
) -> Response:
req = propfind_request(method_root)
ms = Multistatus()
for path, props in items:
if req.names_only:
ms.response(path, dict.fromkeys(props))
elif req.names is None:
ms.response(path, props)
else:
found = {n: props[n] for n in req.names if n in props}
missing = [n for n in req.names if n not in props]
ms.response(path, found, missing)
return self.multistatus(ms)
# -- root and principal ----------------------------------------------------------
def common(self) -> dict[str, PropValue]:
return {
q(D, "current-user-principal"): [href(self.principal())],
q(D, "principal-URL"): [href(self.principal())],
q(D, "owner"): [href(self.principal())],
}
async def root(self, method: str) -> Response:
if method != "PROPFIND":
return Response(status_code=200 if method in ("GET", "HEAD") else 405)
props = {**self.common(), q(D, "resourcetype"): [el(q(D, "collection"))]}
return await self.respond_props([("/dav/", props)], await self.body())
async def addresses(self) -> list[str]:
async with session_scope() as db:
emails = list(
await db.scalars(select(Account.email).where(Account.user_id == self.user.id))
)
out = [self.user.handle, *emails]
return list(dict.fromkeys(e.lower() for e in out))
async def principal_props(self) -> dict[str, PropValue]:
props = {
**self.common(),
q(D, "resourcetype"): [el(q(D, "principal")), el(q(D, "collection"))],
q(D, "displayname"): self.user.handle,
q(C, "calendar-user-address-set"): [
href(f"mailto:{a}") for a in await self.addresses()
],
q(C, "calendar-user-type"): "INDIVIDUAL",
q(D, "principal-collection-set"): [href("/dav/principals/")],
q(D, "supported-report-set"): [],
}
if "calendar" in self.providers:
props[q(C, "calendar-home-set")] = [href(self.home("calendars"))]
if "addressbook" in self.providers:
props[q(CR, "addressbook-home-set")] = [href(self.home("addressbooks"))]
return props
async def principal_req(self, method: str) -> Response:
if method == "PROPFIND":
return await self.respond_props(
[(self.principal(), await self.principal_props())], await self.body()
)
if method == "REPORT": # principal-property-search, expand-property: nothing to add
return self.multistatus(Multistatus())
return Response(status_code=405)
# -- homes -------------------------------------------------------------------------
def privileges(self, read_only: bool) -> list[ET.Element]:
names = ["read", "read-current-user-privilege-set"]
if not read_only:
names += ["write", "write-content", "write-properties", "bind", "unbind"]
return [el(q(D, "privilege"), children=[el(q(D, n))]) for n in names]
def reports(self, kind: str) -> list[ET.Element]:
ns, reports = (
(C, ["calendar-query", "calendar-multiget"])
if kind == "calendar"
else (CR, ["addressbook-query", "addressbook-multiget"])
)
items = [(ns, r) for r in reports] + [(D, "sync-collection")]
return [
el(q(D, "supported-report"), children=[el(q(D, "report"), children=[el(q(n, r))])])
for n, r in items
]
async def home_req(self, method: str, segment: str, provider: DavProvider) -> Response:
if method == "PROPFIND":
home_props: dict[str, PropValue] = {
**self.common(),
q(D, "resourcetype"): [el(q(D, "collection"))],
q(D, "displayname"): "Calendars" if provider.kind == "calendar" else "Contacts",
q(D, "current-user-privilege-set"): self.privileges(False),
}
items = [(self.home(segment), home_props)]
if self.depth():
for coll in await provider.collections(self.user.id):
items.append(
(self.coll_href(segment, coll.id), self.collection_props(provider, coll))
)
return await self.respond_props(items, await self.body())
if method == "REPORT":
return self.multistatus(Multistatus())
if method in ("GET", "HEAD"):
return Response(status_code=200)
return Response(status_code=405)
# -- collections -------------------------------------------------------------------
def collection_props(self, provider: DavProvider, coll: Collection) -> dict[str, PropValue]:
token = f"{SYNC_PREFIX}{coll.sync_token}"
props: dict[str, PropValue] = {
**self.common(),
q(D, "displayname"): coll.displayname,
q(D, "current-user-privilege-set"): self.privileges(coll.read_only),
q(D, "supported-report-set"): self.reports(provider.kind),
q(D, "sync-token"): token,
q(CS, "getctag"): str(coll.sync_token),
}
if provider.kind == "calendar":
props[q(D, "resourcetype")] = [el(q(D, "collection")), el(q(C, "calendar"))]
props[q(C, "supported-calendar-component-set")] = [
el(q(C, "comp"), attrs={"name": comp}) for comp in coll.components
]
props[q(C, "supported-calendar-data")] = [
el(q(C, "calendar-data"), attrs={"content-type": "text/calendar", "version": "2.0"})
]
props[q(C, "calendar-description")] = coll.description
if coll.color:
props[q(A, "calendar-color")] = coll.color
else:
props[q(D, "resourcetype")] = [el(q(D, "collection")), el(q(CR, "addressbook"))]
props[q(CR, "supported-address-data")] = [
el(
q(CR, "address-data-type"),
attrs={"content-type": "text/vcard", "version": "3.0"},
),
el(
q(CR, "address-data-type"),
attrs={"content-type": "text/vcard", "version": "4.0"},
),
]
props[q(CR, "addressbook-description")] = coll.description
return props
async def find(self, provider: DavProvider, coll_id: str) -> Collection:
for coll in await provider.collections(self.user.id):
if coll.id == coll_id:
return coll
raise DavError(404)
def resource_props(
self, provider: DavProvider, res: Resource, with_data: bool
) -> dict[str, PropValue]:
props: dict[str, PropValue] = {
q(D, "getetag"): f'"{res.etag}"',
q(D, "getcontenttype"): provider.content_type,
q(D, "resourcetype"): [],
}
if with_data and res.data is not None:
data_prop = (
q(C, "calendar-data") if provider.kind == "calendar" else q(CR, "address-data")
)
props[data_prop] = res.data
return props
async def collection_req(
self, method: str, segment: str, provider: DavProvider, coll_id: str
) -> Response:
if method == "MKCALENDAR" or method == "MKCOL":
return await self.make_collection(segment, provider, coll_id)
coll = await self.find(provider, coll_id)
if method == "PROPFIND":
items = [(self.coll_href(segment, coll.id), self.collection_props(provider, coll))]
if self.depth():
for res in await provider.resources(self.user.id, coll.id):
items.append(
(
self.res_href(segment, coll.id, res.name),
self.resource_props(provider, res, False),
)
)
return await self.respond_props(items, await self.body())
if method == "REPORT":
return await self.report(segment, provider, coll)
if method == "PROPPATCH":
return await self.proppatch(segment, provider, coll)
if method == "DELETE":
if coll.read_only:
raise DavError(403)
await provider.delete_collection(self.user.id, coll.id)
return Response(status_code=204)
if method in ("GET", "HEAD"):
return Response(status_code=200)
return Response(status_code=405)
async def make_collection(self, segment: str, provider: DavProvider, coll_id: str) -> Response:
if any(c.id == coll_id for c in await provider.collections(self.user.id)):
raise DavError(405) # exists already
root = await self.body()
props = mkcol_props(root)
rtype = props.get(q(D, "resourcetype"))
if self.request.method.upper() == "MKCOL" and (
rtype is None
or rtype.find(q(CR, "addressbook")) is None
or provider.kind != "addressbook"
):
raise DavError(403)
if self.request.method.upper() == "MKCALENDAR" and provider.kind != "calendar":
raise DavError(403)
name_el = props.get(q(D, "displayname"))
color_el = props.get(q(A, "calendar-color"))
await provider.create_collection(
self.user.id,
coll_id,
(name_el.text or "").strip() if name_el is not None else "",
(color_el.text or "").strip() if color_el is not None else None,
)
return Response(status_code=201)
async def proppatch(self, segment: str, provider: DavProvider, coll: Collection) -> Response:
patch = proppatch_request(await self.body())
if coll.read_only:
raise DavError(403)
values: dict[str, Any] = {}
for name, value in patch.set.items():
text = (value.text or "").strip()
if name == q(D, "displayname"):
values["displayname"] = text
elif name == q(A, "calendar-color"):
values["color"] = text
elif name in (q(C, "calendar-description"), q(CR, "addressbook-description")):
values["description"] = text
if values:
await provider.update_collection(self.user.id, coll.id, **values)
# Properties not kept (calendar order, …) are accepted and ignored:
# clients treat a refusal as a failed sync.
ms = Multistatus()
ms.response(self.coll_href(segment, coll.id), dict.fromkeys([*patch.set, *patch.remove]))
return self.multistatus(ms)
async def report(self, segment: str, provider: DavProvider, coll: Collection) -> Response:
root = await self.body()
if root is None:
raise BadRequest("empty report")
rep = report_request(root)
wants_data = rep.props is None or any(
p in (q(C, "calendar-data"), q(CR, "address-data")) for p in rep.props
)
ms = Multistatus()
def add(res: Resource) -> None:
props = self.resource_props(provider, res, wants_data)
if rep.props is None:
ms.response(self.res_href(segment, coll.id, res.name), props)
else:
found = {n: props[n] for n in rep.props if n in props}
missing = [n for n in rep.props if n not in props]
ms.response(self.res_href(segment, coll.id, res.name), found, missing)
if rep.kind in ("calendar-multiget", "addressbook-multiget"):
names = [_last_segment(h) for h in rep.hrefs]
got = {r.name: r for r in await provider.get(self.user.id, coll.id, names)}
for h, name in zip(rep.hrefs, names, strict=True):
if name in got:
add(got[name])
else:
ms.response(h, {}, status=404)
return self.multistatus(ms)
if rep.kind in ("calendar-query", "addressbook-query"):
names_q = (
await provider.query(self.user.id, coll.id, rep.start, rep.end)
if rep.kind == "calendar-query"
else None
)
for res in (
await provider.get(self.user.id, coll.id, names_q)
if wants_data
else (
await provider.resources(self.user.id, coll.id)
if names_q is None
else [
r
for r in await provider.resources(self.user.id, coll.id)
if r.name in set(names_q)
]
)
):
add(res)
return self.multistatus(ms)
# sync-collection (RFC 6578)
since = _token_number(rep.sync_token)
if rep.sync_token and since is None:
raise DavError(403, condition=q(D, "valid-sync-token"))
if since is None:
changed = None
deleted: list[str] = []
token = coll.sync_token
else:
changes = await provider.changes(self.user.id, coll.id, since)
if changes is None:
raise DavError(403, condition=q(D, "valid-sync-token"))
changed, deleted, token = changes.changed, changes.deleted, changes.sync_token
if changed is None:
resources = (
await provider.get(self.user.id, coll.id, None)
if wants_data
else await provider.resources(self.user.id, coll.id)
)
elif changed:
resources = (
await provider.get(self.user.id, coll.id, changed)
if wants_data
else [
r
for r in await provider.resources(self.user.id, coll.id)
if r.name in set(changed)
]
)
else:
resources = []
present = {r.name for r in resources}
for res in resources:
add(res)
for name in deleted:
if name not in present:
ms.response(self.res_href(segment, coll.id, name), {}, status=404)
ms.sync_token(f"{SYNC_PREFIX}{token}")
return self.multistatus(ms)
# -- resources ---------------------------------------------------------------------
async def resource_req(
self, method: str, segment: str, provider: DavProvider, coll_id: str, name: str
) -> Response:
coll = await self.find(provider, coll_id)
if method in ("GET", "HEAD"):
got = await provider.get(self.user.id, coll.id, [name])
if not got:
raise DavError(404)
res = got[0]
content = (res.data or "").encode() if method == "GET" else b""
return Response(
status_code=200,
content=content,
media_type=provider.content_type,
headers={"ETag": f'"{res.etag}"'},
)
if method == "PROPFIND":
got = await provider.get(self.user.id, coll.id, [name])
if not got:
raise DavError(404)
return await self.respond_props(
[
(
self.res_href(segment, coll.id, name),
self.resource_props(provider, got[0], True),
)
],
await self.body(),
)
if method == "PUT":
if coll.read_only:
raise DavError(403, condition=q(D, "need-privileges"))
data = (await self.request.body()).decode("utf-8", "replace")
etag, created = await provider.put(
self.user.id,
coll.id,
name,
data,
if_match=_etag_header(self.request.headers.get("if-match")),
if_none_match=(self.request.headers.get("if-none-match") or "").strip() == "*",
)
return Response(status_code=201 if created else 204, headers={"ETag": f'"{etag}"'})
if method == "DELETE":
if coll.read_only:
raise DavError(403)
await provider.delete(
self.user.id,
coll.id,
name,
if_match=_etag_header(self.request.headers.get("if-match")),
)
return Response(status_code=204)
return Response(status_code=405)
def _last_segment(h: str) -> str:
path = urlsplit(h).path
return unquote(path.rstrip("/").rsplit("/", 1)[-1])
def _token_number(token: str) -> int | None:
if not token:
return None
tail = token.rsplit("/", 1)[-1]
return int(tail) if tail.isdigit() else None
def _etag_header(value: str | None) -> str | None:
if not value:
return None
value = value.strip()
if value == "*":
return "*"
if value.startswith("W/"):
value = value[2:]
return value.strip('"')
__all__ = ["router", "uuid"]
+236
View File
@@ -0,0 +1,236 @@
"""WebDAV XML: reading request bodies, writing multistatus answers (RFC 4918,
4791, 6352, 6578). Properties are ``{namespace}name`` strings, as ElementTree
names them."""
from __future__ import annotations
import xml.etree.ElementTree as ET
from dataclasses import dataclass, field
from datetime import UTC, datetime
from defusedxml import ElementTree as SafeET
D = "DAV:"
C = "urn:ietf:params:xml:ns:caldav"
CR = "urn:ietf:params:xml:ns:carddav"
CS = "http://calendarserver.org/ns/"
A = "http://apple.com/ns/ical/"
for _prefix, _ns in (("d", D), ("cal", C), ("card", CR), ("cs", CS), ("ical", A)):
ET.register_namespace(_prefix, _ns)
def q(ns: str, name: str) -> str:
return f"{{{ns}}}{name}"
class BadRequest(Exception):
pass
def parse(body: bytes) -> ET.Element | None:
if not body.strip():
return None
try:
root = SafeET.fromstring(body)
except Exception as err: # noqa: BLE001 - any malformed XML is the client's error
raise BadRequest(str(err)) from err
return root # type: ignore[no-any-return]
@dataclass
class PropRequest:
"""PROPFIND: which properties (None = all)."""
names: list[str] | None = None
names_only: bool = False
def propfind_request(root: ET.Element | None) -> PropRequest:
if root is None or root.find(q(D, "allprop")) is not None:
return PropRequest()
if root.find(q(D, "propname")) is not None:
return PropRequest(names_only=True)
prop = root.find(q(D, "prop"))
if prop is None:
return PropRequest()
return PropRequest(names=[child.tag for child in prop])
def requested_props(root: ET.Element) -> list[str] | None:
prop = root.find(q(D, "prop"))
if prop is None:
return None
return [child.tag for child in prop]
@dataclass
class Report:
# calendar-query | calendar-multiget | addressbook-query | addressbook-multiget
# | sync-collection
kind: str
props: list[str] | None
hrefs: list[str] = field(default_factory=list)
start: datetime | None = None
end: datetime | None = None
sync_token: str = ""
limit: int | None = None
def _ical_time(value: str | None) -> datetime | None:
if not value:
return None
try:
return datetime.strptime(value.strip(), "%Y%m%dT%H%M%SZ").replace(tzinfo=UTC)
except ValueError:
return None
def report_request(root: ET.Element) -> Report:
tag = root.tag
names = {
q(C, "calendar-query"): "calendar-query",
q(C, "calendar-multiget"): "calendar-multiget",
q(CR, "addressbook-query"): "addressbook-query",
q(CR, "addressbook-multiget"): "addressbook-multiget",
q(D, "sync-collection"): "sync-collection",
}
kind = names.get(tag)
if kind is None:
raise BadRequest(f"unsupported report {tag}")
rep = Report(kind=kind, props=requested_props(root))
rep.hrefs = [(h.text or "").strip() for h in root.findall(q(D, "href"))]
if kind == "calendar-query":
tr = root.find(f".//{q(C, 'time-range')}")
if tr is not None:
rep.start = _ical_time(tr.get("start"))
rep.end = _ical_time(tr.get("end"))
if kind == "sync-collection":
rep.sync_token = (root.findtext(q(D, "sync-token")) or "").strip()
nresults = root.find(f"{q(D, 'limit')}/{q(D, 'nresults')}")
if nresults is not None and (nresults.text or "").strip().isdigit():
rep.limit = int((nresults.text or "").strip())
return rep
@dataclass
class PropPatch:
set: dict[str, ET.Element] = field(default_factory=dict)
remove: list[str] = field(default_factory=list)
def proppatch_request(root: ET.Element | None) -> PropPatch:
out = PropPatch()
if root is None:
return out
for action in root:
prop = action.find(q(D, "prop"))
if prop is None:
continue
for child in prop:
if action.tag == q(D, "set"):
out.set[child.tag] = child
elif action.tag == q(D, "remove"):
out.remove.append(child.tag)
return out
def mkcol_props(root: ET.Element | None) -> dict[str, ET.Element]:
"""MKCALENDAR / extended MKCOL: the properties to set on the new collection."""
if root is None:
return {}
out: dict[str, ET.Element] = {}
for sett in root.findall(q(D, "set")):
prop = sett.find(q(D, "prop"))
if prop is not None:
for child in prop:
out[child.tag] = child
return out
# ---- answers --------------------------------------------------------------------------
# A property's value: text, a list of child elements, or an element builder.
PropValue = str | list[ET.Element] | None
class Multistatus:
def __init__(self) -> None:
self.root = ET.Element(q(D, "multistatus"))
def response(
self,
href: str,
found: dict[str, PropValue],
missing: list[str] | None = None,
status: int | None = None,
) -> None:
resp = ET.SubElement(self.root, q(D, "response"))
ET.SubElement(resp, q(D, "href")).text = href
if status is not None:
ET.SubElement(resp, q(D, "status")).text = _status_line(status)
return
if found:
ps = ET.SubElement(resp, q(D, "propstat"))
prop = ET.SubElement(ps, q(D, "prop"))
for name, value in found.items():
el = ET.SubElement(prop, name)
if isinstance(value, str):
el.text = value
elif isinstance(value, list):
el.extend(value)
ET.SubElement(ps, q(D, "status")).text = _status_line(200)
if missing:
ps = ET.SubElement(resp, q(D, "propstat"))
prop = ET.SubElement(ps, q(D, "prop"))
for name in missing:
ET.SubElement(prop, name)
ET.SubElement(ps, q(D, "status")).text = _status_line(404)
def sync_token(self, token: str) -> None:
ET.SubElement(self.root, q(D, "sync-token")).text = token
def to_bytes(self) -> bytes:
body: bytes = ET.tostring(self.root, encoding="utf-8")
return b'<?xml version="1.0" encoding="utf-8"?>\n' + body
def el(
tag: str,
text: str | None = None,
children: list[ET.Element] | None = None,
attrs: dict[str, str] | None = None,
) -> ET.Element:
e = ET.Element(tag, attrs or {})
if text is not None:
e.text = text
if children:
e.extend(children)
return e
def href(path: str) -> ET.Element:
return el(q(D, "href"), path)
def error_body(condition: str) -> bytes:
root = ET.Element(q(D, "error"))
ET.SubElement(root, condition)
return b'<?xml version="1.0" encoding="utf-8"?>\n' + bytes(ET.tostring(root, encoding="utf-8"))
_REASONS = {
200: "OK",
201: "Created",
204: "No Content",
207: "Multi-Status",
403: "Forbidden",
404: "Not Found",
409: "Conflict",
412: "Precondition Failed",
507: "Insufficient Storage",
}
def _status_line(code: int) -> str:
return f"HTTP/1.1 {code} {_REASONS.get(code, '')}".rstrip()
+104
View File
@@ -0,0 +1,104 @@
"""App passwords: one generated password per mail app (SPEC App. D §D.5).
A mail app (Thunderbird, a phone's Mail app) signs in to the hosted mailbox
over IMAP/SMTP, where no 2FA code can be asked for. It gets a password of its
own — 16 random letters, shown once — that can be revoked without touching the
main password or the other apps. The webmail itself never accepts one.
Stored as argon2id hashes (identity/passwords.py), the form the mail server's
SQL directory reads (§D.3). The hash is of the bare 16 letters: the mail
server compares exactly what an app sends, so the password is copied and
entered without the spaces it is shown with.
"""
from __future__ import annotations
import secrets
import string
import uuid
from sqlalchemy import delete, func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.identity.passwords import hash_password, verify_password
from app.models import UserAppPassword
LENGTH = 16 # 26^16 ≈ 2^75
MAX_PER_USER = 25
NAME_MAX = 60
_ALPHABET = string.ascii_lowercase
def generation_key(user_id: uuid.UUID) -> str:
"""Redis counter bumped whenever the user's app passwords change: DAV
sign-ins cached under the old value stop counting (app/dav/auth.py)."""
return f"apppw:gen:{user_id}"
async def _changed(user_id: uuid.UUID) -> None:
from app.redis_client import get_redis
await get_redis().incr(generation_key(user_id))
class TooMany(Exception):
"""The user already has MAX_PER_USER app passwords."""
def generate() -> str:
return "".join(secrets.choice(_ALPHABET) for _ in range(LENGTH))
async def list_for(db: AsyncSession, user_id: uuid.UUID) -> list[UserAppPassword]:
rows = await db.scalars(
select(UserAppPassword)
.where(UserAppPassword.user_id == user_id)
.order_by(UserAppPassword.created_at.desc())
)
return list(rows)
async def create(db: AsyncSession, user_id: uuid.UUID, name: str) -> tuple[UserAppPassword, str]:
"""A new app password named ``name``; returns the row and the password
itself, which is never available again."""
count = await db.scalar(
select(func.count()).select_from(UserAppPassword).where(UserAppPassword.user_id == user_id)
)
if (count or 0) >= MAX_PER_USER:
raise TooMany
password = generate()
row = UserAppPassword(
user_id=user_id, name=name.strip()[:NAME_MAX], password_hash=hash_password(password)
)
db.add(row)
await db.commit()
return row, password
async def revoke(db: AsyncSession, user_id: uuid.UUID, app_password_id: uuid.UUID) -> bool:
result = await db.execute(
delete(UserAppPassword).where(
UserAppPassword.id == app_password_id, UserAppPassword.user_id == user_id
)
)
await db.commit()
await _changed(user_id)
return bool(result.rowcount) # type: ignore[attr-defined]
async def revoke_all(db: AsyncSession, user_id: uuid.UUID) -> None:
"""Every app signs in again (the main password was reset)."""
await db.execute(delete(UserAppPassword).where(UserAppPassword.user_id == user_id))
await db.commit()
await _changed(user_id)
async def verify(db: AsyncSession, user_id: uuid.UUID, password: str) -> UserAppPassword | None:
"""The app password ``password`` matches, if any — for the mail server's
sign-in and for tests; never for the webmail's own sign-in."""
if len(password) != LENGTH:
return None
for row in await list_for(db, user_id):
if verify_password(row.password_hash, password):
return row
return None
+60 -1
View File
@@ -16,8 +16,8 @@ from app.api.errors import raise_api_error
from app.config import get_settings from app.config import get_settings
from app.db import get_session from app.db import get_session
from app.events import get_event_bus from app.events import get_event_bus
from app.identity import app_passwords, trusted_devices
from app.identity import totp as totp_mod from app.identity import totp as totp_mod
from app.identity import trusted_devices
from app.identity.cookies import ( from app.identity.cookies import (
clear_auth_cookies, clear_auth_cookies,
new_csrf_token, new_csrf_token,
@@ -34,6 +34,9 @@ from app.identity.reset import (
request_allowed, request_allowed,
) )
from app.identity.schemas import ( from app.identity.schemas import (
AppPasswordCreatedOut,
AppPasswordIn,
AppPasswordOut,
LoginRequest, LoginRequest,
MessageOut, MessageOut,
PasswordResetCheckIn, PasswordResetCheckIn,
@@ -312,12 +315,68 @@ async def password_reset(body: PasswordResetIn, db: DbSession) -> MessageOut:
user.password_hash = hash_password(body.password) user.password_hash = hash_password(body.password)
await db.commit() await db.commit()
await destroy_all_sessions(user.id) # force re-login everywhere after reset await destroy_all_sessions(user.id) # force re-login everywhere after reset
# Mail apps sign in again too: the reset may be the owner taking the account back.
await app_passwords.revoke_all(db, user.id)
# Whoever reset the password may be the owner taking the account back: no # Whoever reset the password may be the owner taking the account back: no
# device keeps skipping the code on the old trust. # device keeps skipping the code on the old trust.
await trusted_devices.revoke_all(user.id) await trusted_devices.revoke_all(user.id)
return MessageOut() return MessageOut()
# --- App passwords (SPEC App. D §D.5) -----------------------------------------
def _app_password_out(row: object) -> AppPasswordOut:
from app.models import UserAppPassword
assert isinstance(row, UserAppPassword)
return AppPasswordOut(
id=str(row.id),
name=row.name,
created_at=row.created_at.isoformat(),
last_used_at=row.last_used_at.isoformat() if row.last_used_at else None,
)
@router.get("/app-passwords", response_model=list[AppPasswordOut])
async def list_app_passwords(user: CurrentUser, db: DbSession) -> list[AppPasswordOut]:
return [_app_password_out(r) for r in await app_passwords.list_for(db, user.id)]
@router.post(
"/app-passwords", response_model=AppPasswordCreatedOut, dependencies=[Depends(require_csrf)]
)
async def create_app_password(
body: AppPasswordIn, user: CurrentUser, db: DbSession
) -> AppPasswordCreatedOut:
"""A new password for one mail app. Takes the account password: an app
password opens the mailbox without a 2FA code."""
if not verify_password(user.password_hash, body.password):
raise_api_error(403, "invalid_password", "incorrect password")
if not body.name.strip():
raise_api_error(400, "invalid_name", "give the app password a name")
try:
row, password = await app_passwords.create(db, user.id, body.name)
except app_passwords.TooMany:
raise_api_error(
409, "too_many", f"you can have up to {app_passwords.MAX_PER_USER} app passwords"
)
return AppPasswordCreatedOut(**_app_password_out(row).model_dump(), app_password=password)
@router.delete(
"/app-passwords/{app_password_id}",
response_model=MessageOut,
dependencies=[Depends(require_csrf)],
)
async def revoke_app_password(
app_password_id: uuid.UUID, user: CurrentUser, db: DbSession
) -> MessageOut:
if not await app_passwords.revoke(db, user.id, app_password_id):
raise_api_error(404, "not_found", "app password not found")
return MessageOut()
@router.put("/recovery-email", response_model=UserOut, dependencies=[Depends(require_csrf)]) @router.put("/recovery-email", response_model=UserOut, dependencies=[Depends(require_csrf)])
async def set_recovery_email(body: RecoveryEmailIn, user: CurrentUser, db: DbSession) -> UserOut: async def set_recovery_email(body: RecoveryEmailIn, user: CurrentUser, db: DbSession) -> UserOut:
"""Add, change or remove the recovery address. Takes the password: whoever """Add, change or remove the recovery address. Takes the password: whoever
+17
View File
@@ -63,6 +63,23 @@ class PasswordResetCheckOut(BaseModel):
totp_required: bool = False totp_required: bool = False
class AppPasswordIn(BaseModel):
name: str = Field(min_length=1, max_length=60) # "Thunderbird on laptop"
password: str = Field(min_length=1, max_length=1024) # the account password, to confirm
class AppPasswordOut(BaseModel):
id: str
name: str
created_at: str
last_used_at: str | None = None
class AppPasswordCreatedOut(AppPasswordOut):
# The generated password itself: in this answer only, never again.
app_password: str
class RecoveryEmailIn(BaseModel): class RecoveryEmailIn(BaseModel):
password: str = Field(min_length=1, max_length=1024) password: str = Field(min_length=1, max_length=1024)
# None (or blank) removes the recovery address. # None (or blank) removes the recovery address.
+16
View File
@@ -353,6 +353,22 @@ class SavedSearch(Base):
pinned: Mapped[bool] = mapped_column(nullable=False, server_default=text("true")) pinned: Mapped[bool] = mapped_column(nullable=False, server_default=text("true"))
class UserAppPassword(Base):
"""A password for one mail app (IMAP/SMTP), revocable on its own (§D.5)."""
__tablename__ = "user_app_passwords"
id: Mapped[uuid.UUID] = _uuid_pk()
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
)
name: Mapped[str] = mapped_column(Text, nullable=False) # "Thunderbird on laptop"
password_hash: Mapped[str] = mapped_column(Text, nullable=False) # argon2id
created_at: Mapped[dt.datetime] = _now()
# Set by the mail server's sign-in, once there is one.
last_used_at: Mapped[dt.datetime | None] = mapped_column(TIMESTAMP(timezone=True))
class PushSubscription(Base): class PushSubscription(Base):
"""A browser or installed app to tell about new mail (Web Push).""" """A browser or installed app to tell about new mail (Web Push)."""
+3
View File
@@ -112,6 +112,9 @@ class Plugin:
# Core plugins (calendar, contacts) ship builtin=True: always installed, never # Core plugins (calendar, contacts) ship builtin=True: always installed, never
# offered for removal in the UI, and the uninstall path refuses --drop-data. # offered for removal in the UI, and the uninstall path refuses --drop-data.
builtin: bool = False builtin: bool = False
# Serves its collections to CalDAV / CardDAV clients (app/dav/provider.py
# DavProvider; SPEC Appendix E).
dav: Any = None
def __post_init__(self) -> None: def __post_init__(self) -> None:
if not _NAME_RE.fullmatch(self.name): if not _NAME_RE.fullmatch(self.name):
@@ -0,0 +1,43 @@
"""app passwords: one password per mail app (SPEC App. D §D.5)
Mail apps (Thunderbird, a phone's Mail app) will sign in to the user's hosted
mailbox over IMAP/SMTP, where no 2FA code can be asked for. Each app gets its
own generated password instead, which can be revoked on its own; the main
password with 2FA stays for the webmail. Stored as argon2id hashes, the form
the mail server's SQL directory reads (§D.3).
Revision ID: 0024
Revises: 0023
Create Date: 2026-10-09
"""
from __future__ import annotations
from collections.abc import Sequence
from alembic import op
revision: str = "0024"
down_revision: str | None = "0023"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.execute(
"""
CREATE TABLE user_app_passwords (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name text NOT NULL,
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
last_used_at timestamptz
)
"""
)
op.execute("CREATE INDEX ix_user_app_passwords_user ON user_app_passwords (user_id)")
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS user_app_passwords")
@@ -0,0 +1,138 @@
"""App passwords (SPEC App. D §D.5): one generated password per mail app,
shown once, revocable on its own, never accepted by the webmail's sign-in."""
from __future__ import annotations
import re
import uuid
from collections.abc import AsyncIterator
from dataclasses import dataclass, field
import httpx
import pytest
import pytest_asyncio
from httpx import ASGITransport
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
from app.config import get_settings
from app.identity import app_passwords
from app.identity.passwords import hash_password
from app.identity.reset import issue_token
from app.identity.sessions import create_session
from app.main import create_app
from app.models import User, UserAppPassword
CSRF = "csrf-token"
PASSWORD = "hunter2pass"
@dataclass
class Ctx:
db: AsyncSession
anon: httpx.AsyncClient
created: list[uuid.UUID] = field(default_factory=list)
async def user(self) -> User:
user = User(
handle=f"ap{uuid.uuid4().hex[:8]}@test.local", password_hash=hash_password(PASSWORD)
)
self.db.add(user)
await self.db.commit()
self.created.append(user.id)
return user
async def signed_in(self, user: User) -> httpx.AsyncClient:
token = await create_session(user.id, "pytest")
return httpx.AsyncClient(
transport=ASGITransport(app=create_app()),
base_url="http://testserver",
cookies={"session": token, "csrf": CSRF},
headers={"X-CSRF": CSRF},
)
@pytest_asyncio.fixture
async def ctx() -> AsyncIterator[Ctx]:
engine = create_async_engine(get_settings().database_url, poolclass=NullPool)
db = async_sessionmaker(engine, expire_on_commit=False)()
anon = httpx.AsyncClient(transport=ASGITransport(app=create_app()), base_url="http://t")
c = Ctx(db=db, anon=anon)
try:
yield c
finally:
await anon.aclose()
await db.execute(delete(User).where(User.id.in_(c.created)))
await db.commit()
await db.close()
await engine.dispose()
async def _create(client: httpx.AsyncClient, name: str, password: str = PASSWORD) -> httpx.Response:
return await client.post("/api/auth/app-passwords", json={"name": name, "password": password})
async def test_created_once_listed_without_secret_and_revoked(ctx: Ctx) -> None:
user = await ctx.user()
client = await ctx.signed_in(user)
assert (await _create(client, "Thunderbird", "wrong")).status_code == 403
made = await _create(client, " Thunderbird on laptop ")
assert made.status_code == 200
body = made.json()
assert body["name"] == "Thunderbird on laptop"
assert re.fullmatch(r"[a-z]{16}", body["app_password"])
listed = (await client.get("/api/auth/app-passwords")).json()
assert [a["name"] for a in listed] == ["Thunderbird on laptop"]
assert "app_password" not in listed[0] and "password_hash" not in listed[0]
assert listed[0]["last_used_at"] is None
# What the mail server will check: this one matches, nothing else does.
row = await app_passwords.verify(ctx.db, user.id, body["app_password"])
assert row is not None and str(row.id) == body["id"]
assert await app_passwords.verify(ctx.db, user.id, PASSWORD) is None
assert await app_passwords.verify(ctx.db, user.id, "a" * 16) is None
stored = await ctx.db.scalar(
select(UserAppPassword.password_hash).where(UserAppPassword.id == row.id)
)
assert stored and stored.startswith("$argon2id$") and body["app_password"] not in stored
# The webmail's own sign-in never takes an app password.
login = await ctx.anon.post(
"/api/auth/login", json={"handle": user.handle, "password": body["app_password"]}
)
assert login.status_code == 401
# Someone else cannot revoke it; its owner can, once.
other = await ctx.signed_in(await ctx.user())
assert (await other.delete(f"/api/auth/app-passwords/{body['id']}")).status_code == 404
assert (await client.delete(f"/api/auth/app-passwords/{body['id']}")).status_code == 200
assert (await client.delete(f"/api/auth/app-passwords/{body['id']}")).status_code == 404
assert await app_passwords.verify(ctx.db, user.id, body["app_password"]) is None
await client.aclose()
await other.aclose()
async def test_at_most_a_few_per_user(ctx: Ctx, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(app_passwords, "MAX_PER_USER", 2)
client = await ctx.signed_in(await ctx.user())
assert (await _create(client, "one")).status_code == 200
assert (await _create(client, "two")).status_code == 200
third = await _create(client, "three")
assert third.status_code == 409 and third.json()["error"]["code"] == "too_many"
await client.aclose()
async def test_password_reset_revokes_them_all(ctx: Ctx) -> None:
user = await ctx.user()
client = await ctx.signed_in(user)
secret = (await _create(client, "Phone")).json()["app_password"]
token = await issue_token(user.id)
reset = await ctx.anon.post(
"/api/auth/password/reset", json={"token": token, "password": "brandNewPass9"}
)
assert reset.status_code == 200
assert await app_passwords.verify(ctx.db, user.id, secret) is None
await client.aclose()
+231
View File
@@ -0,0 +1,231 @@
"""CalDAV server (SPEC Appendix E): discovery, sign-in with app passwords, and
two-way changes between DAV clients and the web app on the same rows."""
from __future__ import annotations
import base64
import re
import uuid
from collections.abc import AsyncIterator
from dataclasses import dataclass
import httpx
import pytest_asyncio
from httpx import ASGITransport
from sqlalchemy import delete, text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
from app.config import get_settings
from app.db import session_scope
from app.identity import app_passwords
from app.identity.passwords import hash_password
from app.identity.sessions import create_session
from app.main import create_app
from app.models import User
CAL = "/api/plugins/calendar"
PROPFIND_ALL = b'<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:allprop/></d:propfind>'
EVENT = """BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//test//EN
BEGIN:VEVENT
UID:{uid}
DTSTAMP:20261009T120000Z
DTSTART;TZID=Europe/Warsaw:20311013T090000
DTEND;TZID=Europe/Warsaw:20311013T100000
SUMMARY:{summary}
X-CLIENT-ONLY:keep me
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:x
TRIGGER:-PT30M
END:VALARM
END:VEVENT
END:VCALENDAR
"""
@dataclass
class Ctx:
dav: httpx.AsyncClient # a CalDAV client (Basic, app password)
web: httpx.AsyncClient # the web app (session)
anon: httpx.AsyncClient
user: User
app_password: str
app_password_id: uuid.UUID
home: str
calendar_id: str
def _basic(login: str, password: str) -> dict[str, str]:
return {"Authorization": "Basic " + base64.b64encode(f"{login}:{password}".encode()).decode()}
@pytest_asyncio.fixture
async def ctx() -> AsyncIterator[Ctx]:
engine = create_async_engine(get_settings().database_url, poolclass=NullPool)
sm = async_sessionmaker(engine, expire_on_commit=False)
async with sm() as s:
user = User(
handle=f"dav{uuid.uuid4().hex[:8]}@test.local",
password_hash=hash_password("account-pw"),
)
s.add(user)
await s.commit()
async with session_scope() as db:
row, password = await app_passwords.create(db, user.id, "test client")
transport = ASGITransport(app=create_app())
token = await create_session(user.id, "pytest")
web = httpx.AsyncClient(
transport=transport,
base_url="http://testserver",
cookies={"session": token, "csrf": "c"},
headers={"X-CSRF": "c"},
)
calendar_id = (await web.get(f"{CAL}/calendars")).json()[0]["id"] # bootstraps "Personal"
dav = httpx.AsyncClient(
transport=transport, base_url="http://testserver", headers=_basic(user.handle, password)
)
anon = httpx.AsyncClient(transport=transport, base_url="http://testserver")
try:
yield Ctx(dav, web, anon, user, password, row.id, f"/dav/calendars/{user.id}/", calendar_id)
finally:
for c in (dav, web, anon):
await c.aclose()
async with sm() as s:
await s.execute(
text("DELETE FROM plg_calendar_calendars WHERE user_id = :u"), {"u": str(user.id)}
)
await s.execute(
text("DELETE FROM plg_calendar_dav_changes WHERE user_id = :u"), {"u": str(user.id)}
)
await s.execute(delete(User).where(User.id == user.id))
await s.commit()
await engine.dispose()
def _sync_report(token: str = "") -> bytes:
return (
'<?xml version="1.0"?><d:sync-collection xmlns:d="DAV:">'
f"<d:sync-token>{token}</d:sync-token><d:sync-level>1</d:sync-level>"
"<d:prop><d:getetag/></d:prop></d:sync-collection>"
).encode()
async def _sync(c: Ctx, token: str = "") -> tuple[str, str]:
r = await c.dav.request("REPORT", f"{c.home}{c.calendar_id}/", content=_sync_report(token))
assert r.status_code == 207, r.text
new = re.search(r"<d:sync-token>([^<]+)</d:sync-token>", r.text)
assert new
return new.group(1), r.text
async def test_discovery_and_sign_in(ctx: Ctx) -> None:
assert (await ctx.anon.get("/.well-known/caldav")).headers["location"] == "/dav/"
assert (await ctx.anon.request("PROPFIND", "/dav/")).status_code == 401
# The account password never works here — only app passwords.
wrong = await ctx.anon.request(
"PROPFIND", "/dav/", headers=_basic(ctx.user.handle, "account-pw")
)
assert wrong.status_code == 401
root = await ctx.dav.request("PROPFIND", "/dav/", headers={"Depth": "0"}, content=PROPFIND_ALL)
assert root.status_code == 207 and f"/dav/principals/{ctx.user.id}/" in root.text
principal = await ctx.dav.request(
"PROPFIND", f"/dav/principals/{ctx.user.id}/", headers={"Depth": "0"}
)
assert ctx.home in principal.text and "calendar-home-set" in principal.text
home = await ctx.dav.request("PROPFIND", ctx.home, headers={"Depth": "1"})
assert (
home.status_code == 207 and "Personal" in home.text and f"{ctx.calendar_id}/" in home.text
)
# Someone else's home: refused.
other = await ctx.dav.request(
"PROPFIND", f"/dav/calendars/{uuid.uuid4()}/", headers={"Depth": "1"}
)
assert other.status_code == 403
# Revoking the app password locks the client out at once (no stale cache).
async with session_scope() as db:
await app_passwords.revoke(db, ctx.user.id, ctx.app_password_id)
assert (await ctx.dav.request("PROPFIND", "/dav/")).status_code == 401
async def test_client_and_web_app_see_each_others_changes(ctx: Ctx) -> None:
url = f"{ctx.home}{ctx.calendar_id}/review.ics"
token0, _ = await _sync(ctx)
# A client adds an event: the web app shows it, alarm included.
put = await ctx.dav.put(
url,
content=EVENT.format(uid="dav-1@test", summary="Review"),
headers={"If-None-Match": "*"},
)
assert put.status_code == 201
etag = put.headers["etag"]
events = (
await ctx.web.get(
f"{CAL}/events", params={"start": "2031-10-13T00:00:00Z", "end": "2031-10-14T00:00:00Z"}
)
).json()["events"]
assert [(e["summary"], e["reminders"], e["tz"]) for e in events] == [
("Review", [30], "Europe/Warsaw")
]
event_id = events[0]["id"]
got = await ctx.dav.get(url)
assert got.headers["etag"] == etag and "X-CLIENT-ONLY:keep me" in got.text # verbatim
token1, body = await _sync(ctx, token0)
assert "review.ics" in body
# Creating it again, or editing an old version, is refused.
again = await ctx.dav.put(
url, content=EVENT.format(uid="dav-1@test", summary="x"), headers={"If-None-Match": "*"}
)
assert again.status_code == 412
stale = await ctx.dav.put(
url, content=EVENT.format(uid="dav-1@test", summary="x"), headers={"If-Match": '"nope"'}
)
assert stale.status_code == 412
# The web app renames it: the client gets the new version (regenerated).
assert (
await ctx.web.patch(f"{CAL}/events/{event_id}", json={"summary": "Review (web)"})
).status_code == 200
token2, body = await _sync(ctx, token1)
assert "review.ics" in body and token2 != token1
got = await ctx.dav.get(url)
assert got.headers["etag"] != etag and "SUMMARY:Review (web)" in got.text
assert "TRIGGER:-PT30M" in got.text and "TZID=Europe/Warsaw" in got.text
# The web app deletes it: the client hears it is gone.
assert (await ctx.web.delete(f"{CAL}/events/{event_id}")).status_code == 200
_, body = await _sync(ctx, token2)
assert "review.ics" in body and "404" in body
assert (await ctx.dav.get(url)).status_code == 404
async def test_client_delete_and_read_only_calendars(ctx: Ctx) -> None:
url = f"{ctx.home}{ctx.calendar_id}/gone.ics"
put = await ctx.dav.put(url, content=EVENT.format(uid="dav-2@test", summary="Temp"))
assert put.status_code == 201
assert (await ctx.dav.delete(url, headers={"If-Match": put.headers["etag"]})).status_code == 204
events = (
await ctx.web.get(
f"{CAL}/events", params={"start": "2031-10-13T00:00:00Z", "end": "2031-10-14T00:00:00Z"}
)
).json()["events"]
assert events == []
# A subscribed (read-only) calendar refuses writes.
async with session_scope() as db:
sub = await db.scalar(
text(
"INSERT INTO plg_calendar_calendars (user_id, name, kind, read_only) "
"VALUES (:u, 'Holidays', 'ics_subscription', true) RETURNING id"
),
{"u": str(ctx.user.id)},
)
ro = await ctx.dav.put(
f"{ctx.home}{sub}/x.ics", content=EVENT.format(uid="dav-3@test", summary="x")
)
assert ro.status_code == 403
+160
View File
@@ -0,0 +1,160 @@
"""CardDAV server (SPEC Appendix E.4): address books, vCards in and out, groups
as CATEGORIES, and changes flowing both ways with the web app."""
from __future__ import annotations
import base64
import re
import uuid
from collections.abc import AsyncIterator
from dataclasses import dataclass
import httpx
import pytest_asyncio
from httpx import ASGITransport
from sqlalchemy import delete, text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
from app.config import get_settings
from app.db import session_scope
from app.identity import app_passwords
from app.identity.passwords import hash_password
from app.identity.sessions import create_session
from app.main import create_app
from app.models import User
CON = "/api/plugins/contacts"
IPHONE_CARD = (
"BEGIN:VCARD\r\nVERSION:3.0\r\nPRODID:-//Apple Inc.//iPhone OS 18.0//EN\r\n"
"N:Kowalska;Anna;;;\r\nFN:Anna Kowalska\r\nORG:Imaginary Pixels;Compositing\r\n"
"item1.EMAIL;type=INTERNET;type=WORK;type=pref:anna@ip.example\r\n"
"TEL;type=CELL;type=VOICE:+48 600 100 200\r\nCATEGORIES:Clients\r\n"
"X-ABUID:ABC123\r\nUID:card-1\r\nEND:VCARD\r\n"
)
@dataclass
class Ctx:
dav: httpx.AsyncClient
web: httpx.AsyncClient
user: User
home: str
book_id: str
@pytest_asyncio.fixture
async def ctx() -> AsyncIterator[Ctx]:
engine = create_async_engine(get_settings().database_url, poolclass=NullPool)
sm = async_sessionmaker(engine, expire_on_commit=False)
async with sm() as s:
user = User(
handle=f"card{uuid.uuid4().hex[:8]}@test.local", password_hash=hash_password("pw")
)
s.add(user)
await s.commit()
async with session_scope() as db:
_, password = await app_passwords.create(db, user.id, "test client")
transport = ASGITransport(app=create_app())
token = await create_session(user.id, "pytest")
web = httpx.AsyncClient(
transport=transport,
base_url="http://testserver",
cookies={"session": token, "csrf": "c"},
headers={"X-CSRF": "c"},
)
auth = "Basic " + base64.b64encode(f"{user.handle}:{password}".encode()).decode()
dav = httpx.AsyncClient(
transport=transport, base_url="http://testserver", headers={"Authorization": auth}
)
home = f"/dav/addressbooks/{user.id}/"
listing = await dav.request("PROPFIND", home, headers={"Depth": "1"})
book_id = re.findall(rf"{home}([0-9a-f-]{{36}})/", listing.text)[0]
try:
yield Ctx(dav, web, user, home, book_id)
finally:
await dav.aclose()
await web.aclose()
async with sm() as s:
for table in ("plg_contacts_books", "plg_contacts_groups", "plg_contacts_dav_changes"):
await s.execute(
text(f"DELETE FROM {table} WHERE user_id = :u"), {"u": str(user.id)}
)
await s.execute(delete(User).where(User.id == user.id))
await s.commit()
await engine.dispose()
async def _sync(c: Ctx, token: str = "") -> tuple[str, str]:
body = (
'<?xml version="1.0"?><d:sync-collection xmlns:d="DAV:">'
f"<d:sync-token>{token}</d:sync-token><d:sync-level>1</d:sync-level>"
"<d:prop><d:getetag/></d:prop></d:sync-collection>"
)
r = await c.dav.request("REPORT", f"{c.home}{c.book_id}/", content=body)
assert r.status_code == 207, r.text
m = re.search(r"<d:sync-token>([^<]+)</d:sync-token>", r.text)
assert m
return m.group(1), r.text
async def test_discovery_finds_the_address_book(ctx: Ctx) -> None:
principal = await ctx.dav.request(
"PROPFIND", f"/dav/principals/{ctx.user.id}/", headers={"Depth": "0"}
)
assert "addressbook-home-set" in principal.text and ctx.home in principal.text
book = await ctx.dav.request("PROPFIND", f"{ctx.home}{ctx.book_id}/", headers={"Depth": "0"})
assert "Contacts" in book.text and "addressbook" in book.text
async def test_a_phone_contact_and_the_web_app(ctx: Ctx) -> None:
url = f"{ctx.home}{ctx.book_id}/ABC123.vcf"
token0, _ = await _sync(ctx)
put = await ctx.dav.put(url, content=IPHONE_CARD, headers={"If-None-Match": "*"})
assert put.status_code == 201
# The web app shows it, in the group its category named.
listed = (await ctx.web.get(f"{CON}/contacts", params={"filter": "all", "q": ""})).json()[
"contacts"
]
assert [c["display_name"] for c in listed] == ["Anna Kowalska"]
contact = (await ctx.web.get(f"{CON}/contacts/{listed[0]['id']}")).json()
assert contact["org"] == "Imaginary Pixels" and contact["department"] == "Compositing"
assert (
contact["emails"][0]["value"] == "anna@ip.example"
and contact["phones"][0]["type"] == "mobile"
)
assert [g["name"] for g in contact["groups"]] == ["Clients"]
got = await ctx.dav.get(url)
assert got.headers["etag"] == put.headers["etag"] and "X-ABUID:ABC123" in got.text # verbatim
token1, body = await _sync(ctx, token0)
assert "ABC123.vcf" in body
# The web app edits it: the phone gets a regenerated card.
edit = dict(contact, display_name="Anna K.", group_ids=[g["id"] for g in contact["groups"]])
assert (await ctx.web.put(f"{CON}/contacts/{contact['id']}", json=edit)).status_code == 200
token2, body = await _sync(ctx, token1)
assert "ABC123.vcf" in body
got = await ctx.dav.get(url)
assert "FN:Anna K." in got.text and "CATEGORIES:Clients" in got.text
# Deleting the group changes the contact's CATEGORIES.
gid = contact["groups"][0]["id"]
assert (await ctx.web.delete(f"{CON}/groups/{gid}")).status_code == 200
token3, body = await _sync(ctx, token2)
assert "ABC123.vcf" in body and "CATEGORIES" not in (await ctx.dav.get(url)).text
# Multiget, as phones fetch changed cards.
multiget = (
'<?xml version="1.0"?><card:addressbook-multiget xmlns:d="DAV:" '
'xmlns:card="urn:ietf:params:xml:ns:carddav"><d:prop><d:getetag/><card:address-data/>'
f"</d:prop><d:href>{url}</d:href><d:href>{ctx.home}{ctx.book_id}/missing.vcf</d:href>"
"</card:addressbook-multiget>"
)
r = await ctx.dav.request("REPORT", f"{ctx.home}{ctx.book_id}/", content=multiget)
assert r.status_code == 207 and "BEGIN:VCARD" in r.text and "404" in r.text
# Deleted in the web app: gone for the phone.
assert (await ctx.web.delete(f"{CON}/contacts/{contact['id']}")).status_code == 200
_, body = await _sync(ctx, token3)
assert "ABC123.vcf" in body and "404" in body
+20
View File
@@ -47,6 +47,26 @@ location /api/ {
proxy_buffering off; proxy_buffering off;
} }
# ---- CalDAV / CardDAV (SPEC Appendix E) -----------------------------------
# Phones, Thunderbird and macOS sync calendars and contacts here, signing in
# with an app password. The well-known paths are how clients find the server
# from just a host name.
location /dav/ {
proxy_pass http://webmail_api;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $fwd_proto;
proxy_set_header X-Forwarded-Host $host;
proxy_read_timeout 300s;
}
location ~ ^/\.well-known/(caldav|carddav)$ {
proxy_pass http://webmail_api;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $fwd_proto;
}
# ---- static ------------------------------------------------------------- # ---- static -------------------------------------------------------------
# Vite emits content-hashed filenames into /assets, so they are immutable. # Vite emits content-hashed filenames into /assets, so they are immutable.
location /assets/ { location /assets/ {
+18
View File
@@ -596,6 +596,24 @@ then also needs `https://ultimo.email/.well-known/assetlinks.json` naming the
Android package and its signing-certificate fingerprint — served by nginx Android package and its signing-certificate fingerprint — served by nginx
like any other static file. like any other static file.
### 5.4 Calendars and contacts in other apps (CalDAV / CardDAV)
Ultimo serves each user's calendars and address books to standard apps, which
sync two-way with the web app: iPhone/iPad and macOS Calendar & Contacts,
Thunderbird, Android through DAVx⁵, GNOME/Evolution.
- **Server:** `https://ultimo.email/dav/` — most apps also find it from just
`ultimo.email` (`/.well-known/caldav`, `/.well-known/carddav`).
- **User name:** the full Ultimo address. **Password:** an **app password**
(Settings → Security → App passwords) — the account password is refused, so
2FA cannot be bypassed and a lost device is one revocation away. Settings shows
the exact server and user name to type.
- nginx passes `/dav/` and the two well-known paths to the API (in
`docker/nginx-app.conf`). A reverse proxy in front (QNAP, DSM) must pass them
too, with the `Authorization` header — the default.
- HTTPS is required by iOS/macOS for these accounts.
- Repeated wrong passwords are throttled per address and per IP (15 minutes).
--- ---
## 6. Resource usage ## 6. Resource usage
+9
View File
@@ -2,6 +2,7 @@
// (SPEC §10.3) and includes cookies for the session (SPEC §6.1). // (SPEC §10.3) and includes cookies for the session (SPEC §6.1).
import type { import type {
AppPassword,
AccountDetail, AccountDetail,
AdminMail, AdminMail,
AdminSettings, AdminSettings,
@@ -242,6 +243,14 @@ export const api = {
body: JSON.stringify({ password }), body: JSON.stringify({ password }),
}), }),
sessions: () => apiFetch<SessionInfo[]>("/auth/sessions"), sessions: () => apiFetch<SessionInfo[]>("/auth/sessions"),
appPasswords: () => apiFetch<AppPassword[]>("/auth/app-passwords"),
createAppPassword: (name: string, password: string) =>
apiFetch<AppPassword & { app_password: string }>("/auth/app-passwords", {
method: "POST",
body: JSON.stringify({ name, password }),
}),
revokeAppPassword: (id: string) =>
apiFetch<{ message: string }>(`/auth/app-passwords/${id}`, { method: "DELETE" }),
revokeSession: (sid: string) => revokeSession: (sid: string) =>
apiFetch<{ status: string }>(`/auth/sessions/${sid}`, { method: "DELETE" }), apiFetch<{ status: string }>(`/auth/sessions/${sid}`, { method: "DELETE" }),
+8
View File
@@ -345,3 +345,11 @@ export interface RemoteDraftOut {
snippet: string; snippet: string;
updated_at: string; updated_at: string;
} }
/** A password for one mail app (Settings → Security → App passwords). */
export interface AppPassword {
id: string;
name: string;
created_at: string;
last_used_at: string | null;
}
+185
View File
@@ -509,6 +509,189 @@ function OfflineAccessSetting() {
); );
} }
/**
* Settings → Security: a password for each mail app (Thunderbird, a phone's
* Mail app) — they sign in over IMAP/SMTP, where no 2FA code can be asked for.
* The password is shown once, when made; each can be revoked on its own.
*/
function AppPasswordsSetting() {
const t = useT();
const handle = useMe().data?.handle ?? "";
const queryClient = useQueryClient();
const list = useQuery({ queryKey: ["app-passwords"], queryFn: api.appPasswords });
const [creating, setCreating] = useState(false);
const [name, setName] = useState("");
const [password, setPassword] = useState("");
const [error, setError] = useState("");
const [made, setMade] = useState<{ name: string; secret: string } | null>(null);
const [copied, setCopied] = useState(false);
const create = useMutation({
mutationFn: () => api.createAppPassword(name.trim(), password),
onSuccess: (out) => {
setMade({ name: out.name, secret: out.app_password });
setCreating(false);
setName("");
setPassword("");
setError("");
void queryClient.invalidateQueries({ queryKey: ["app-passwords"] });
},
onError: (e) => setError(e instanceof ApiError ? e.message : String(e)),
});
const revoke = useMutation({
mutationFn: (id: string) => api.revokeAppPassword(id),
onSuccess: () => void queryClient.invalidateQueries({ queryKey: ["app-passwords"] }),
});
const copy = async () => {
if (!made) return;
try {
await navigator.clipboard.writeText(made.secret);
setCopied(true);
window.setTimeout(() => setCopied(false), 2000);
} catch {
/* the text is on screen to copy by hand */
}
};
const date = (iso: string) => new Date(iso).toLocaleDateString(getDateLocale(), { day: "numeric", month: "short", year: "numeric" });
return (
<div className="space-y-2">
<h2 className="text-lg font-semibold">{t("appPasswords.title")}</h2>
<p className="text-sm text-app-muted">{t("appPasswords.help")}</p>
{/* What to type into a calendar / contacts app (CalDAV / CardDAV, SPEC App. E). */}
<div className="space-y-1.5 rounded-lg border border-app-border p-3 text-sm">
<p className="font-medium">{t("appPasswords.connectTitle")}</p>
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1">
<dt className="text-app-muted">{t("appPasswords.connectServer")}</dt>
<dd className="select-all break-all font-mono text-[13px]">{`${window.location.origin}/dav/`}</dd>
<dt className="text-app-muted">{t("appPasswords.connectUser")}</dt>
<dd className="select-all break-all font-mono text-[13px]">{handle}</dd>
<dt className="text-app-muted">{t("appPasswords.connectPassword")}</dt>
<dd className="text-app-muted">{t("appPasswords.connectPasswordValue")}</dd>
</dl>
<p className="text-xs text-app-faint">{t("appPasswords.connectHint")}</p>
</div>
{made && (
<div className="space-y-2 rounded-lg border border-emerald-500/40 bg-emerald-500/5 p-3">
<p className="text-sm">{t("appPasswords.madeFor", { name: made.name })}</p>
<div className="flex flex-wrap items-center gap-3">
{/* Shown in groups of four; copied (and entered) without spaces. */}
<code className="select-all rounded bg-app-surface px-3 py-2 font-mono text-lg tracking-wider">
{made.secret.match(/.{1,4}/g)?.map((group, i) => (
// Spacing only, no characters: copying by hand gives the bare letters.
<span key={i} className={i ? "ml-2.5" : ""}>
{group}
</span>
))}
</code>
<button
className="rounded border border-app-border px-3 py-1.5 text-sm hover:bg-app-hover"
onClick={() => void copy()}
>
{copied ? t("appPasswords.copied") : t("appPasswords.copy")}
</button>
</div>
<p className="text-xs text-app-muted">{t("appPasswords.onceOnly")}</p>
<button className="rounded bg-app-accent px-3 py-1 text-sm text-app-accent-fg" onClick={() => setMade(null)}>
{t("appPasswords.done")}
</button>
</div>
)}
{(list.data ?? []).length > 0 && (
<ul className="divide-y divide-app-border rounded-lg border border-app-border">
{(list.data ?? []).map((a) => (
<li key={a.id} className="flex items-center justify-between gap-3 px-3 py-2 text-sm">
<span className="min-w-0">
<span className="break-words">{a.name}</span>
<span className="block text-xs text-app-faint">
{t("appPasswords.created", { date: date(a.created_at) })} ·{" "}
{a.last_used_at
? t("appPasswords.lastUsed", { date: date(a.last_used_at) })
: t("appPasswords.neverUsed")}
</span>
</span>
<button
className="shrink-0 text-xs text-red-500 disabled:opacity-50"
disabled={revoke.isPending}
onClick={() => {
if (confirm(t("appPasswords.confirmRevoke", { name: a.name }))) revoke.mutate(a.id);
}}
>
{t("settings.revoke")}
</button>
</li>
))}
</ul>
)}
{creating ? (
<form
className="space-y-2 rounded-lg border border-app-border p-3"
onSubmit={(e) => {
e.preventDefault();
create.mutate();
}}
>
<label className="block text-sm">
<span className="text-app-muted">{t("appPasswords.name")}</span>
<input
className="mt-1 w-full rounded border border-app-border bg-app-surface px-2 py-1 text-sm"
placeholder={t("appPasswords.namePlaceholder")}
maxLength={60}
value={name}
onChange={(e) => setName(e.target.value)}
autoFocus
/>
</label>
<label className="block text-sm">
<span className="text-app-muted">{t("appPasswords.confirmPassword")}</span>
<input
className="mt-1 w-full rounded border border-app-border bg-app-surface px-2 py-1 text-sm"
type="password"
autoComplete="current-password"
value={password}
onChange={(e) => setPassword(e.target.value)}
/>
</label>
{error && <p className="text-sm text-red-500">{error}</p>}
<div className="flex gap-2">
<button
type="submit"
className="rounded bg-app-accent px-3 py-1 text-sm text-app-accent-fg disabled:opacity-50"
disabled={!name.trim() || !password || create.isPending}
>
{t("appPasswords.create")}
</button>
<button
type="button"
className="px-2 text-sm text-app-muted"
onClick={() => {
setCreating(false);
setError("");
}}
>
{t("common.cancel")}
</button>
</div>
</form>
) : (
<button
className="rounded border border-app-border px-3 py-1.5 text-sm text-app-fg hover:bg-app-hover"
onClick={() => {
setMade(null);
setCreating(true);
}}
>
{t("appPasswords.new")}
</button>
)}
</div>
);
}
function SecuritySection() { function SecuritySection() {
const t = useT(); const t = useT();
const queryClient = useQueryClient(); const queryClient = useQueryClient();
@@ -626,6 +809,8 @@ function SecuritySection() {
{error && <p className="text-sm text-red-500">{error}</p>} {error && <p className="text-sm text-red-500">{error}</p>}
</div> </div>
<AppPasswordsSetting />
<div className="space-y-2"> <div className="space-y-2">
<h2 className="text-lg font-semibold">{t("settings.sessions")}</h2> <h2 className="text-lg font-semibold">{t("settings.sessions")}</h2>
<ul className="divide-y divide-app-border rounded-lg border border-app-border"> <ul className="divide-y divide-app-border rounded-lg border border-app-border">
+4
View File
@@ -26,6 +26,10 @@ export default defineConfig({
changeOrigin: true, changeOrigin: true,
ws: true, ws: true,
}, },
// CalDAV / CardDAV (SPEC Appendix E), for trying clients against dev.
"/dav": { target: "http://127.0.0.1:8000", changeOrigin: true },
"/.well-known/caldav": { target: "http://127.0.0.1:8000", changeOrigin: true },
"/.well-known/carddav": { target: "http://127.0.0.1:8000", changeOrigin: true },
}, },
}, },
test: { test: {
@@ -12,7 +12,9 @@ from __future__ import annotations
from pathlib import Path from pathlib import Path
from app.plugins.sdk import JobSpec, Page, PageNav, Plugin, Slot, UiManifest from app.plugins.sdk import JobSpec, Page, PageNav, Plugin, Slot, UiManifest
from webmail_plugin_calendar.dav import calendar_dav, prune_changes
from webmail_plugin_calendar.hook import on_message_new from webmail_plugin_calendar.hook import on_message_new
from webmail_plugin_calendar.reminders import reminder_job
from webmail_plugin_calendar.router import router from webmail_plugin_calendar.router import router
from webmail_plugin_calendar.sync import caldav_sync from webmail_plugin_calendar.sync import caldav_sync
@@ -24,9 +26,17 @@ plugin = Plugin(
builtin=True, builtin=True,
# An organizer's cancellation, applied to the calendar as the email arrives. # An organizer's cancellation, applied to the calendar as the email arrives.
hooks={"message.new": on_message_new}, hooks={"message.new": on_message_new},
jobs=[
# Refresh connected CalDAV calendars every 15 minutes (also runs on connect). # Refresh connected CalDAV calendars every 15 minutes (also runs on connect).
jobs=[JobSpec(func=caldav_sync, schedule="*/15 * * * *")], JobSpec(func=caldav_sync, schedule="*/15 * * * *"),
# Event reminders, as notifications on the user's devices.
JobSpec(func=reminder_job, schedule="* * * * *", singleton=True),
# Trim the CalDAV change log nightly (sync tokens stay valid for 90 days).
JobSpec(func=prune_changes, schedule="20 4 * * *"),
],
router=router, router=router,
# Calendars served to CalDAV clients — phones, Thunderbird (SPEC App. E).
dav=calendar_dav,
ui_manifest=UiManifest( ui_manifest=UiManifest(
# An invitation received by email, above the message body (iMIP): renders # An invitation received by email, above the message body (iMIP): renders
# only for a message carrying an .ics, so ordinary mail shows nothing. # only for a message carrying an .ics, so ordinary mail shows nothing.
@@ -0,0 +1,464 @@
"""The calendar plugin's CalDAV provider (SPEC App. E.3): the user's calendars
as CalDAV collections, each event series one resource.
Reads and writes the same rows the web app does, in the user's RLS session, so
reminders, invitations and the upstream CalDAV write-back (the ``dav_dirty``
trigger; tombstones on synced calendars) behave as for any other change.
Change numbers come from ``plg_calendar_dav_changes``, filled by triggers.
"""
from __future__ import annotations
import hashlib
import json
import uuid
from collections import defaultdict
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import text
from app.dav.provider import Changes, Collection, DavError, Resource
from app.plugins.sdk import sdk
from webmail_plugin_calendar import ical_resource
from webmail_plugin_calendar.queries import expand_range
_SDK = sdk("calendar")
_C = "{urn:ietf:params:xml:ns:caldav}"
_EVENT_COLUMNS = (
"e.id, e.calendar_id, e.ical_uid, e.recurrence_id, e.sequence, e.summary, e.description, "
"e.location, e.dtstart, e.dtend, e.all_day, e.tz, e.rrule, e.exdates, e.status, "
"e.organizer, e.attendees, e.my_partstat, e.reminders, e.color, e.show_as, "
"e.conference_url, e.modified_at, e.srv_ical, e.srv_ical_ver, "
"plg_calendar_dav_name(e) AS dav_name"
)
class CalendarDav:
kind = "calendar"
content_type = "text/calendar; charset=utf-8"
extension = ".ics"
# -- helpers ---------------------------------------------------------------------
async def _addresses(self, user_id: uuid.UUID) -> set[str]:
return {email.lower() for _, email, _ in await _SDK.accounts.of_user(user_id)}
async def _calendar(self, s: Any, collection_id: str) -> dict[str, Any]:
try:
uuid.UUID(collection_id)
except ValueError:
raise DavError(404) from None
row = (
(
await s.execute(
text("SELECT id, kind, read_only FROM plg_calendar_calendars WHERE id = :id"),
{"id": collection_id},
)
)
.mappings()
.first()
)
if row is None:
raise DavError(404)
return dict(row)
async def _rows(
self, s: Any, collection_id: str, names: list[str] | None = None
) -> dict[str, list[dict[str, Any]]]:
"""The live rows of a calendar, by resource name."""
where = "e.calendar_id = :cal AND e.dav_deleted = false"
params: dict[str, Any] = {"cal": collection_id}
if names is not None:
if not names:
return {}
where += " AND plg_calendar_dav_name(e) = ANY(:names)"
params["names"] = names
rows = (
(
await s.execute(
text(f"SELECT {_EVENT_COLUMNS} FROM plg_calendar_events e WHERE {where}"),
params,
)
)
.mappings()
.all()
)
out: dict[str, list[dict[str, Any]]] = defaultdict(list)
for r in rows:
out[r["dav_name"]].append(dict(r))
return out
@staticmethod
def _serve(
rows: list[dict[str, Any]], addresses: set[str], with_data: bool
) -> tuple[str, str | None]:
"""(ETag, data) of a resource: the client's own text while the fields are
unchanged since it was stored, else generated from the fields."""
base = next((r for r in rows if r["recurrence_id"] is None), rows[0])
fp = ical_resource.fingerprint(rows)
raw = base.get("srv_ical")
if raw and base.get("srv_ical_ver") == fp:
return "r" + hashlib.sha1(raw.encode()).hexdigest(), raw # noqa: S324
return "g" + fp, ical_resource.build(rows, addresses) if with_data else None
# -- collections -----------------------------------------------------------------
async def collections(self, user_id: uuid.UUID) -> list[Collection]:
async with _SDK.session(user_id) as s:
rows = (
(
await s.execute(
text(
"SELECT c.id, c.name, c.color, c.kind, c.read_only, "
"COALESCE((SELECT max(id) FROM plg_calendar_dav_changes d "
" WHERE d.calendar_id = c.id), 0) AS token "
"FROM plg_calendar_calendars c ORDER BY c.name"
)
)
)
.mappings()
.all()
)
return [
Collection(
id=str(r["id"]),
displayname=r["name"],
sync_token=int(r["token"]),
color=r["color"],
read_only=bool(r["read_only"]) or r["kind"] not in ("local", "caldav"),
)
for r in rows
]
async def create_collection(
self, user_id: uuid.UUID, collection_id: str, displayname: str, color: str | None
) -> None:
try:
cid = uuid.UUID(collection_id)
except ValueError:
raise DavError(403) from None
name = (displayname or "Calendar").strip()[:100]
async with _SDK.session(user_id) as s:
taken = {
str(n).lower()
for n in (
await s.execute(text("SELECT name FROM plg_calendar_calendars"))
).scalars()
}
base, n = name, 2
while name.lower() in taken: # names are unique per user
name, n = f"{base} ({n})", n + 1
await s.execute(
text(
"INSERT INTO plg_calendar_calendars (id, user_id, name, color, kind) "
"VALUES (:id, :u, :name, :color, 'local')"
),
{
"id": str(cid),
"u": str(user_id),
"name": name,
"color": _color(color) or "#3b82f6",
},
)
async def update_collection(
self,
user_id: uuid.UUID,
collection_id: str,
*,
displayname: str | None = None,
color: str | None = None,
description: str | None = None,
) -> None:
sets, params = [], {"id": collection_id}
if displayname:
sets.append("name = :name")
params["name"] = displayname.strip()[:100]
if color and _color(color):
sets.append("color = :color")
params["color"] = _color(color) or ""
if not sets:
return
async with _SDK.session(user_id) as s:
await self._calendar(s, collection_id)
try:
await s.execute(
text(f"UPDATE plg_calendar_calendars SET {', '.join(sets)} WHERE id = :id"),
params,
)
except Exception as err: # noqa: BLE001 - a duplicate name
raise DavError(409) from err
async def delete_collection(self, user_id: uuid.UUID, collection_id: str) -> None:
async with _SDK.session(user_id) as s:
cal = await self._calendar(s, collection_id)
if cal["kind"] != "local":
raise DavError(403)
count = await s.scalar(
text("SELECT count(*) FROM plg_calendar_calendars WHERE kind = 'local'")
)
if (count or 0) <= 1:
raise DavError(403) # the last calendar stays
await s.execute(
text("DELETE FROM plg_calendar_calendars WHERE id = :id"), {"id": collection_id}
)
# -- resources -------------------------------------------------------------------
async def resources(self, user_id: uuid.UUID, collection_id: str) -> list[Resource]:
return await self._get(user_id, collection_id, None, with_data=False)
async def get(
self, user_id: uuid.UUID, collection_id: str, names: list[str] | None
) -> list[Resource]:
return await self._get(user_id, collection_id, names, with_data=True)
async def _get(
self, user_id: uuid.UUID, collection_id: str, names: list[str] | None, *, with_data: bool
) -> list[Resource]:
addresses = await self._addresses(user_id) if with_data else set()
async with _SDK.session(user_id) as s:
await self._calendar(s, collection_id)
by_name = await self._rows(s, collection_id, names)
out = []
for name, rows in by_name.items():
etag, data = self._serve(rows, addresses, with_data)
out.append(Resource(name=name, etag=etag, data=data))
return out
async def put(
self,
user_id: uuid.UUID,
collection_id: str,
name: str,
data: str,
*,
if_match: str | None,
if_none_match: bool,
) -> tuple[str, bool]:
addresses = await self._addresses(user_id)
settings = await _SDK.accounts.settings_of(user_id)
try:
new_rows = ical_resource.parse(data, addresses, str(settings.get("tz") or "UTC"))
except ical_resource.InvalidResource as err:
raise DavError(400, str(err), condition=f"{_C}valid-calendar-data") from err
uid = new_rows[0]["ical_uid"]
async with _SDK.session(user_id) as s:
cal = await self._calendar(s, collection_id)
if cal["read_only"] or cal["kind"] not in ("local", "caldav"):
raise DavError(403)
existing = (await self._rows(s, collection_id, [name])).get(name)
if existing:
etag, _ = self._serve(existing, addresses, False)
if if_none_match:
raise DavError(412)
if if_match and if_match != "*" and if_match != etag:
raise DavError(412)
if existing[0]["ical_uid"] != uid:
raise DavError(400, condition=f"{_C}no-uid-conflict")
else:
if if_match:
raise DavError(412)
clash = await s.scalar(
text(
"SELECT plg_calendar_dav_name(e) FROM plg_calendar_events e "
"WHERE calendar_id = :cal AND ical_uid = :uid "
"AND dav_deleted = false LIMIT 1"
),
{"cal": collection_id, "uid": uid},
)
if clash and clash != name:
raise DavError(409, condition=f"{_C}no-uid-conflict")
keep: list[str] = []
for r in new_rows:
params = {
"u": str(user_id),
"cal": collection_id,
"uid": uid,
"rid": r["recurrence_id"],
"seq": r["sequence"],
"sum": r["summary"],
"desc": r["description"],
"loc": r["location"],
"start": r["dtstart"],
"end": r["dtend"],
"allday": r["all_day"],
"tz": r["tz"],
"rrule": r["rrule"],
"ex": r["exdates"],
"status": r["status"],
"org": json.dumps(r["organizer"]) if r["organizer"] else None,
"att": json.dumps(r["attendees"]),
"mine": r["my_partstat"],
"rem": r["reminders"],
"color": r["color"],
"showas": r["show_as"],
"conf": r["conference_url"],
"name": name,
}
row_id = await s.scalar(
text(
"INSERT INTO plg_calendar_events (user_id, calendar_id, ical_uid, "
" recurrence_id, sequence, summary, description, location, dtstart, dtend, "
" all_day, tz, rrule, exdates, status, organizer, attendees, my_partstat, "
" reminders, color, show_as, conference_url, srv_name) "
"VALUES (:u, :cal, :uid, :rid, :seq, :sum, :desc, :loc, :start, :end, "
" :allday, :tz, :rrule, :ex, :status, CAST(:org AS jsonb), "
" CAST(:att AS jsonb), :mine, :rem, :color, :showas, :conf, :name) "
"ON CONFLICT (calendar_id, ical_uid, recurrence_id) DO UPDATE SET "
" sequence = EXCLUDED.sequence, summary = EXCLUDED.summary, "
" description = EXCLUDED.description, location = EXCLUDED.location, "
" dtstart = EXCLUDED.dtstart, dtend = EXCLUDED.dtend, "
" all_day = EXCLUDED.all_day, "
" tz = EXCLUDED.tz, rrule = EXCLUDED.rrule, exdates = EXCLUDED.exdates, "
" status = EXCLUDED.status, organizer = EXCLUDED.organizer, "
" attendees = EXCLUDED.attendees, my_partstat = EXCLUDED.my_partstat, "
" reminders = EXCLUDED.reminders, color = EXCLUDED.color, "
" show_as = EXCLUDED.show_as, conference_url = EXCLUDED.conference_url, "
" srv_name = EXCLUDED.srv_name, dav_deleted = false "
"RETURNING id"
),
params,
)
keep.append(str(row_id))
# Occurrences no longer in the client's version: gone.
await self._remove(s, cal, collection_id, uid, keep)
# Keep the client's text, with the fingerprint of the fields it gave.
stored = (await self._rows(s, collection_id, [name])).get(name, [])
fp = ical_resource.fingerprint(stored)
base = next((r for r in stored if r["recurrence_id"] is None), stored[0])
await s.execute(
text(
"UPDATE plg_calendar_events SET srv_ical = :raw, srv_ical_ver = :fp "
"WHERE id = :id"
),
{"raw": data, "fp": fp, "id": str(base["id"])},
)
return "r" + hashlib.sha1(data.encode()).hexdigest(), not existing # noqa: S324
async def _remove(
self, s: Any, cal: dict[str, Any], collection_id: str, uid: str, keep: list[str]
) -> None:
"""Delete a UID's rows (all, or those not in ``keep``) — tombstoned on a
calendar synced upstream, so the deletion reaches the server too."""
where = "calendar_id = :cal AND ical_uid = :uid AND dav_deleted = false"
params: dict[str, Any] = {"cal": collection_id, "uid": uid}
if keep:
where += " AND NOT (id = ANY(CAST(:keep AS uuid[])))"
params["keep"] = keep
if cal["kind"] == "caldav":
await s.execute(
text(f"UPDATE plg_calendar_events SET dav_deleted = true WHERE {where}"), params
)
else:
await s.execute(text(f"DELETE FROM plg_calendar_events WHERE {where}"), params)
async def delete(
self, user_id: uuid.UUID, collection_id: str, name: str, *, if_match: str | None
) -> None:
async with _SDK.session(user_id) as s:
cal = await self._calendar(s, collection_id)
if cal["read_only"] or cal["kind"] not in ("local", "caldav"):
raise DavError(403)
existing = (await self._rows(s, collection_id, [name])).get(name)
if not existing:
raise DavError(404)
if if_match and if_match != "*":
etag, _ = self._serve(existing, set(), False)
if if_match != etag:
raise DavError(412)
await self._remove(s, cal, collection_id, existing[0]["ical_uid"], [])
async def changes(self, user_id: uuid.UUID, collection_id: str, since: int) -> Changes | None:
async with _SDK.session(user_id) as s:
await self._calendar(s, collection_id)
rows = (
(
await s.execute(
text(
"SELECT id, name, deleted FROM plg_calendar_dav_changes "
"WHERE calendar_id = :cal AND id > :since ORDER BY id"
),
{"cal": collection_id, "since": since},
)
)
.mappings()
.all()
)
latest = await s.scalar(
text(
"SELECT COALESCE(max(id), 0) FROM plg_calendar_dav_changes "
"WHERE calendar_id = :cal"
),
{"cal": collection_id},
)
oldest = await s.scalar(
text("SELECT min(id) FROM plg_calendar_dav_changes WHERE calendar_id = :cal"),
{"cal": collection_id},
)
if oldest is not None and since < int(oldest) - 1 and since < int(latest or 0):
return None # older entries were trimmed (prune_changes): resync
if since > int(latest or 0):
return None # a token from the future (another server, a restore): resync
state: dict[str, bool] = {}
for r in rows:
if r["name"]:
state[r["name"]] = bool(r["deleted"])
return Changes(
sync_token=int(latest or 0),
changed=[n for n, gone in state.items() if not gone],
deleted=[n for n, gone in state.items() if gone],
)
async def query(
self, user_id: uuid.UUID, collection_id: str, start: datetime | None, end: datetime | None
) -> list[str] | None:
if start is None and end is None:
return None
lo = start or datetime(1900, 1, 1, tzinfo=UTC)
hi = end or datetime(2200, 1, 1, tzinfo=UTC)
async with _SDK.session(user_id) as s:
await self._calendar(s, collection_id)
by_name = await self._rows(s, collection_id, None)
names = []
for name, rows in by_name.items():
base: list[Any] = [
dict(r, exdates=r["exdates"] or []) for r in rows if r["recurrence_id"] is None
]
overrides: list[Any] = [r for r in rows if r["recurrence_id"] is not None]
for r in base + overrides:
r.setdefault("origin_message_id", None)
if expand_range(base, overrides, lo, hi):
names.append(name)
return names
def _color(value: str | None) -> str | None:
"""A "#rrggbb" colour from a client's ("#rrggbbaa" from Apple)."""
if not value:
return None
v = value.strip()
if v.startswith("#") and len(v) in (7, 9):
return v[:7].lower()
return None
calendar_dav = CalendarDav()
async def prune_changes(ctx: dict[str, Any]) -> int:
"""Nightly: drop change-log entries older than 90 days, keeping each
collection's newest (its sync token). A client holding an older token is
told to resync (``changes`` answers None)."""
async with _SDK.public_session() as pub:
result = await pub.execute(
text(
"DELETE FROM plg_calendar_dav_changes d "
"WHERE created_at < now() - interval '90 days' "
"AND id < (SELECT max(id) FROM plg_calendar_dav_changes k "
" WHERE k.calendar_id = d.calendar_id)"
)
)
return int(result.rowcount or 0) # type: ignore[attr-defined]
@@ -0,0 +1,310 @@
"""One event series as a CalDAV resource: iCalendar text <-> event rows
(SPEC App. E.3).
A resource holds every row of one UID: the series (no ``recurrence_id``) and
its moved or cancelled occurrences. :func:`build` writes the rows as a
VCALENDAR — time zones, all-day dates, recurrence, overrides, organizer and
attendees, alarms from the reminders, the event colour and video link (as X-
properties a client keeps); :func:`parse` reads what a client PUT back into row
fields. :func:`fingerprint` summarises the modelled fields, so a client's own
text can be served back verbatim for as long as they are unchanged.
"""
from __future__ import annotations
import hashlib
import json
from collections.abc import Mapping, Sequence
from datetime import UTC, date, datetime, timedelta
from typing import Any
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
Row = Mapping[str, Any]
# The columns a resource is made of (the fingerprint covers exactly these).
FIELDS = (
"ical_uid",
"recurrence_id",
"sequence",
"summary",
"description",
"location",
"dtstart",
"dtend",
"all_day",
"tz",
"rrule",
"exdates",
"status",
"organizer",
"attendees",
"my_partstat",
"reminders",
"color",
"show_as",
"conference_url",
)
_PARTSTATS = {"needs-action", "accepted", "declined", "tentative", "delegated"}
class InvalidResource(ValueError):
"""Not a calendar object we can store (no VEVENT, several UIDs, …)."""
def _zone(name: str | None) -> ZoneInfo:
try:
return ZoneInfo(name or "UTC")
except (ZoneInfoNotFoundError, ValueError):
return ZoneInfo("UTC")
def _norm(value: Any) -> Any:
if isinstance(value, datetime):
return value.astimezone(UTC).isoformat()
if isinstance(value, date):
return value.isoformat()
if isinstance(value, list | tuple):
return [_norm(v) for v in value]
if isinstance(value, dict):
return {k: _norm(v) for k, v in sorted(value.items())}
return value
def fingerprint(rows: Sequence[Row]) -> str:
"""A digest of the modelled fields of a resource's rows."""
items = sorted(
({f: _norm(r.get(f)) for f in FIELDS} for r in rows),
key=lambda d: str(d["recurrence_id"] or ""),
)
return hashlib.sha1(json.dumps(items, sort_keys=True, default=str).encode()).hexdigest()
# ---- rows -> iCalendar ----------------------------------------------------------------
def _when(value: datetime, all_day: bool, tz: str) -> date | datetime:
if all_day:
return value.astimezone(UTC).date()
zone = _zone(tz)
return value.astimezone(zone) if tz and tz != "UTC" else value.astimezone(UTC)
def _address(prop_name: str, person: Mapping[str, Any], partstat: str | None = None) -> Any:
from icalendar import vCalAddress
addr = vCalAddress(f"mailto:{person.get('email', '')}")
if person.get("name"):
addr.params["CN"] = person["name"]
if prop_name == "attendee":
addr.params["PARTSTAT"] = (partstat or person.get("partstat") or "needs-action").upper()
addr.params["ROLE"] = "REQ-PARTICIPANT"
return addr
def build(rows: Sequence[Row], my_addresses: set[str]) -> str:
"""The VCALENDAR for one resource's rows (series first, then overrides)."""
from icalendar import Alarm, Calendar, Event
from icalendar.prop import vRecur
cal = Calendar()
cal.add("prodid", "-//Ultimo//Calendar//EN")
cal.add("version", "2.0")
cal.add("calscale", "GREGORIAN")
ordered = sorted(
rows,
key=lambda r: (
r.get("recurrence_id") is not None,
r.get("recurrence_id") or datetime.min.replace(tzinfo=UTC),
),
)
for r in ordered:
ev = Event()
all_day = bool(r.get("all_day"))
tz = str(r.get("tz") or "UTC")
ev.add("uid", r["ical_uid"])
stamp = r.get("modified_at") or datetime(2000, 1, 1, tzinfo=UTC)
ev.add("dtstamp", stamp.astimezone(UTC))
ev.add("last-modified", stamp.astimezone(UTC))
ev.add("sequence", int(r.get("sequence") or 0))
ev.add("summary", r.get("summary") or "")
if r.get("description"):
ev.add("description", r["description"])
if r.get("location"):
ev.add("location", r["location"])
ev.add("dtstart", _when(r["dtstart"], all_day, tz))
if r.get("dtend") is not None:
ev.add("dtend", _when(r["dtend"], all_day, tz))
if r.get("recurrence_id") is not None:
ev.add("recurrence-id", _when(r["recurrence_id"], all_day, tz))
if r.get("rrule") and r.get("recurrence_id") is None:
ev.add("rrule", vRecur.from_ical(str(r["rrule"])))
for ex in r.get("exdates") or []:
ev.add("exdate", _when(ex, all_day, tz))
status = str(r.get("status") or "confirmed").upper()
if status in ("CONFIRMED", "TENTATIVE", "CANCELLED"):
ev.add("status", status)
ev.add("transp", "TRANSPARENT" if r.get("show_as") == "free" else "OPAQUE")
organizer = r.get("organizer")
if isinstance(organizer, dict) and organizer.get("email"):
ev.add("organizer", _address("organizer", organizer))
for att in r.get("attendees") or []:
if not isinstance(att, dict) or not att.get("email"):
continue
mine = str(att["email"]).lower() in my_addresses
ev.add("attendee", _address("attendee", att, r.get("my_partstat") if mine else None))
if r.get("conference_url"):
ev.add("x-ultimo-conference", r["conference_url"])
if r.get("color"):
ev.add("x-ultimo-color", r["color"])
for minutes in sorted(set(r.get("reminders") or [])):
alarm = Alarm()
alarm.add("action", "DISPLAY")
alarm.add("description", "Reminder")
alarm.add("trigger", timedelta(minutes=-int(minutes)))
ev.add_component(alarm)
cal.add_component(ev)
cal.add_missing_timezones()
return str(cal.to_ical().decode())
# ---- iCalendar -> rows ----------------------------------------------------------------
def _aware(value: date | datetime, default_tz: str) -> tuple[datetime, bool, str]:
"""(instant, all-day, tz name) of a DTSTART-like value."""
if not isinstance(value, datetime):
return datetime(value.year, value.month, value.day, tzinfo=UTC), True, "UTC"
if value.tzinfo is None: # floating: the user's own zone
zone = _zone(default_tz)
return value.replace(tzinfo=zone), False, default_tz or "UTC"
name = getattr(value.tzinfo, "key", None) or getattr(value.tzinfo, "zone", None) or ""
try:
ZoneInfo(str(name))
except (ZoneInfoNotFoundError, ValueError):
name = "UTC"
return value, False, str(name) or "UTC"
def _person(prop: Any) -> dict[str, str] | None:
if prop is None:
return None
email = str(prop)
if email.lower().startswith("mailto:"):
email = email[7:]
email = email.strip().lower()
if not email:
return None
name = str(prop.params.get("CN", "")).strip() if hasattr(prop, "params") else ""
return {"email": email, "name": "" if name.lower() == email else name}
def _list(value: Any) -> list[Any]:
if value is None:
return []
return value if isinstance(value, list) else [value]
def _reminders(comp: Any, start: datetime) -> list[int]:
out: set[int] = set()
for alarm in comp.walk("VALARM"):
trigger = alarm.get("trigger")
if trigger is None:
continue
value = trigger.dt
if isinstance(value, timedelta):
if str(trigger.params.get("RELATED", "START")).upper() != "START":
continue
minutes = int(-value.total_seconds() // 60)
elif isinstance(value, datetime):
minutes = int((start - value.astimezone(UTC)).total_seconds() // 60)
else:
continue
if 0 <= minutes <= 60 * 24 * 28:
out.add(minutes)
return sorted(out)
def parse(ics: str, my_addresses: set[str], default_tz: str) -> list[dict[str, Any]]:
"""The rows a client's VCALENDAR describes (one UID). Raises InvalidResource."""
from icalendar import Calendar
try:
cal = Calendar.from_ical(ics)
except Exception as err: # noqa: BLE001 - any parse failure: not valid calendar data
raise InvalidResource(str(err)) from err
events = list(cal.walk("VEVENT"))
if not events:
raise InvalidResource("no VEVENT")
uids = {str(e.get("uid") or "") for e in events}
if len(uids) != 1 or "" in uids:
raise InvalidResource("one UID per resource")
rows: list[dict[str, Any]] = []
for comp in events:
start_prop = comp.get("dtstart")
if start_prop is None:
raise InvalidResource("VEVENT without DTSTART")
dtstart, all_day, tz = _aware(start_prop.dt, default_tz)
end_prop = comp.get("dtend")
if end_prop is not None:
dtend = _aware(end_prop.dt, tz)[0]
elif comp.get("duration") is not None:
dtend = dtstart + comp.get("duration").dt
else:
dtend = dtstart + (timedelta(days=1) if all_day else timedelta(0))
rid_prop = comp.get("recurrence-id")
rrule = comp.get("rrule")
exdates: list[datetime] = []
for ex in _list(comp.get("exdate")):
for d in getattr(ex, "dts", []):
exdates.append(_aware(d.dt, tz)[0])
organizer = _person(comp.get("organizer"))
attendees = []
my_partstat = None
for prop in _list(comp.get("attendee")):
who = _person(prop)
if who is None:
continue
stat = str(prop.params.get("PARTSTAT", "NEEDS-ACTION")).lower()
who["partstat"] = stat if stat in _PARTSTATS else "needs-action"
if who["email"] in my_addresses:
my_partstat = who["partstat"]
attendees.append(who)
conference = ""
for key in ("x-ultimo-conference", "x-google-conference", "conference", "url"):
value = comp.get(key)
if value and str(value).startswith(("https://", "http://")):
conference = str(value)
break
color = str(comp.get("x-ultimo-color") or "") or None
status = str(comp.get("status") or "confirmed").lower()
rows.append(
{
"ical_uid": str(comp.get("uid")),
"recurrence_id": _aware(rid_prop.dt, tz)[0] if rid_prop is not None else None,
"sequence": int(comp.get("sequence") or 0),
"summary": str(comp.get("summary") or ""),
"description": str(comp.get("description") or ""),
"location": str(comp.get("location") or ""),
"dtstart": dtstart,
"dtend": dtend,
"all_day": all_day,
"tz": tz,
"rrule": rrule.to_ical().decode()
if rrule is not None and rid_prop is None
else None,
"exdates": exdates,
"status": status
if status in ("confirmed", "tentative", "cancelled")
else "confirmed",
"organizer": organizer,
"attendees": attendees,
"my_partstat": my_partstat,
"reminders": _reminders(comp, dtstart.astimezone(UTC)),
"color": color,
"show_as": "free"
if str(comp.get("transp") or "").upper() == "TRANSPARENT"
else "busy",
"conference_url": conference,
}
)
return rows
@@ -0,0 +1,154 @@
"""calendar: serve calendars to CalDAV clients (SPEC Appendix E.3)
* ``srv_name`` — the resource name a CalDAV client stored the event under (NULL:
derived from the UID, ``md5(uid).ics``);
* ``srv_ical`` / ``srv_ical_ver`` — the iCalendar text a client PUT, served back
verbatim while the event's fields still match (``srv_ical_ver`` is their
fingerprint when it was stored);
* ``modified_at`` — when the row last changed (LAST-MODIFIED, DTSTAMP);
* ``plg_calendar_dav_changes`` — every change to an event or a calendar, numbered:
a calendar's sync token / CTag is its latest number, and a client's token
says which changes it has seen. Filled by triggers, so every write path
counts — the web app, invitations by email, upstream CalDAV sync, DAV clients.
Revision ID: calendar0008
Revises: calendar0007
Create Date: 2026-10-09
"""
from __future__ import annotations
from collections.abc import Sequence
from alembic import op
from app.plugins.rls import enable_rls
revision: str = "calendar0008"
down_revision: str | None = "calendar0007"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.execute("ALTER TABLE plg_calendar_events ADD COLUMN srv_name text")
op.execute("ALTER TABLE plg_calendar_events ADD COLUMN srv_ical text")
op.execute("ALTER TABLE plg_calendar_events ADD COLUMN srv_ical_ver text")
op.execute(
"ALTER TABLE plg_calendar_events ADD COLUMN modified_at timestamptz NOT NULL DEFAULT now()"
)
op.execute(
"CREATE INDEX ix_plg_calendar_events_cal_uid ON plg_calendar_events (calendar_id, ical_uid)"
)
op.execute(
"""
CREATE TABLE plg_calendar_dav_changes (
id bigserial PRIMARY KEY,
user_id uuid NOT NULL,
calendar_id uuid NOT NULL,
name text, -- NULL: the calendar itself changed
deleted boolean NOT NULL DEFAULT false,
created_at timestamptz NOT NULL DEFAULT now()
)
"""
)
op.execute(
"CREATE INDEX ix_plg_calendar_dav_changes_cal ON plg_calendar_dav_changes (calendar_id, id)"
)
enable_rls("plg_calendar_dav_changes")
op.execute("GRANT USAGE ON SEQUENCE plg_calendar_dav_changes_id_seq TO webmail_plugin")
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_calendar_dav_name(e plg_calendar_events) RETURNS text AS $$
SELECT COALESCE(e.srv_name, md5(e.ical_uid) || '.ics')
$$ LANGUAGE sql IMMUTABLE;
"""
)
# Stamp modified_at on every change of a row.
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_calendar_touch() RETURNS trigger AS $$
BEGIN
NEW.modified_at := now();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_calendar_events_touch
BEFORE UPDATE ON plg_calendar_events
FOR EACH ROW EXECUTE FUNCTION plg_calendar_touch();
"""
)
# Log every event change under its resource name. A deleted (or tombstoned)
# row only deletes the resource when no live row of that UID is left — an
# occurrence removed from a series is a change to the series.
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_calendar_log_change() RETURNS trigger AS $$
BEGIN
-- The old resource, when this row left it (deleted, tombstoned for an
-- upstream CalDAV delete, or moved to another calendar). AFTER the
-- statement, so the query sees what is left of the UID.
IF TG_OP = 'DELETE'
OR (TG_OP = 'UPDATE' AND (NEW.calendar_id <> OLD.calendar_id
OR (NEW.dav_deleted AND NOT OLD.dav_deleted))) THEN
INSERT INTO plg_calendar_dav_changes (user_id, calendar_id, name, deleted)
VALUES (OLD.user_id, OLD.calendar_id, plg_calendar_dav_name(OLD),
NOT EXISTS (SELECT 1 FROM plg_calendar_events
WHERE calendar_id = OLD.calendar_id
AND ical_uid = OLD.ical_uid AND dav_deleted = false));
END IF;
-- The resource as it is now.
IF TG_OP <> 'DELETE' AND NOT NEW.dav_deleted THEN
INSERT INTO plg_calendar_dav_changes (user_id, calendar_id, name, deleted)
VALUES (NEW.user_id, NEW.calendar_id, plg_calendar_dav_name(NEW), false);
END IF;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_calendar_events_log
AFTER INSERT OR UPDATE OR DELETE ON plg_calendar_events
FOR EACH ROW EXECUTE FUNCTION plg_calendar_log_change();
"""
)
# A renamed / recoloured calendar changes its CTag too.
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_calendar_log_calendar() RETURNS trigger AS $$
BEGIN
INSERT INTO plg_calendar_dav_changes (user_id, calendar_id, name, deleted)
VALUES (NEW.user_id, NEW.id, NULL, false);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_calendar_calendars_log
AFTER INSERT OR UPDATE OF name, color ON plg_calendar_calendars
FOR EACH ROW EXECUTE FUNCTION plg_calendar_log_calendar();
"""
)
def downgrade() -> None:
op.execute("DROP TRIGGER IF EXISTS plg_calendar_calendars_log ON plg_calendar_calendars")
op.execute("DROP TRIGGER IF EXISTS plg_calendar_events_log ON plg_calendar_events")
op.execute("DROP TRIGGER IF EXISTS plg_calendar_events_touch ON plg_calendar_events")
op.execute("DROP FUNCTION IF EXISTS plg_calendar_log_calendar()")
op.execute("DROP FUNCTION IF EXISTS plg_calendar_log_change()")
op.execute("DROP FUNCTION IF EXISTS plg_calendar_touch()")
op.execute("DROP FUNCTION IF EXISTS plg_calendar_dav_name(plg_calendar_events)")
op.execute("DROP TABLE IF EXISTS plg_calendar_dav_changes")
op.execute("DROP INDEX IF EXISTS ix_plg_calendar_events_cal_uid")
for col in ("modified_at", "srv_ical_ver", "srv_ical", "srv_name"):
op.execute(f"ALTER TABLE plg_calendar_events DROP COLUMN IF EXISTS {col}")
@@ -13,7 +13,8 @@ from __future__ import annotations
from pathlib import Path from pathlib import Path
from app.plugins.sdk import Page, PageNav, Plugin, UiManifest from app.plugins.sdk import JobSpec, Page, PageNav, Plugin, UiManifest
from webmail_plugin_contacts.dav import contacts_dav, prune_changes
from webmail_plugin_contacts.hook import on_message_sent from webmail_plugin_contacts.hook import on_message_sent
from webmail_plugin_contacts.router import router from webmail_plugin_contacts.router import router
@@ -25,6 +26,10 @@ plugin = Plugin(
builtin=True, builtin=True,
hooks={"message.sent": on_message_sent}, hooks={"message.sent": on_message_sent},
router=router, router=router,
# Address books served to CardDAV clients — phones, Thunderbird (SPEC App. E).
dav=contacts_dav,
# Trim the CardDAV change log nightly (sync tokens stay valid for 90 days).
jobs=[JobSpec(func=prune_changes, schedule="25 4 * * *")],
ui_manifest=UiManifest( ui_manifest=UiManifest(
pages=[ pages=[
Page( Page(
@@ -0,0 +1,414 @@
"""The contacts plugin's CardDAV provider (SPEC App. E.4): address books as
CardDAV collections, each contact one vCard.
The same rows the web app reads and writes, in the user's RLS session. A
contact's groups are its CATEGORIES: a client's categories become (and are
kept in step with) the user's groups. Change numbers come from
``plg_contacts_dav_changes``, filled by triggers.
"""
from __future__ import annotations
import hashlib
import json
import uuid
from collections import defaultdict
from datetime import datetime
from typing import Any
from sqlalchemy import text
from app.dav.provider import Changes, Collection, DavError, Resource
from app.plugins.sdk import sdk
from webmail_plugin_contacts import vcard
_SDK = sdk("contacts")
_CR = "{urn:ietf:params:xml:ns:carddav}"
_COLUMNS = (
"c.id, c.book_id, c.vcard_uid, c.name_display, c.name_parts, c.nickname, c.emails, "
"c.default_email, c.phones, c.addresses, c.urls, c.org, c.department, c.title, "
"c.birthday, c.notes, c.starred, c.updated_at, c.srv_vcard, c.srv_vcard_ver, "
"plg_contacts_dav_name(c) AS dav_name"
)
class ContactsDav:
kind = "addressbook"
content_type = "text/vcard; charset=utf-8"
extension = ".vcf"
async def _book(self, s: Any, book_id: str) -> dict[str, Any]:
try:
uuid.UUID(book_id)
except ValueError:
raise DavError(404) from None
row = (
(
await s.execute(
text("SELECT id, kind, read_only FROM plg_contacts_books WHERE id = :id"),
{"id": book_id},
)
)
.mappings()
.first()
)
if row is None:
raise DavError(404)
return dict(row)
async def _rows(
self, s: Any, book_id: str, names: list[str] | None
) -> dict[str, dict[str, Any]]:
where = "c.book_id = :book"
params: dict[str, Any] = {"book": book_id}
if names is not None:
if not names:
return {}
where += " AND plg_contacts_dav_name(c) = ANY(:names)"
params["names"] = names
rows = (
(
await s.execute(
text(f"SELECT {_COLUMNS} FROM plg_contacts_contacts c WHERE {where}"), params
)
)
.mappings()
.all()
)
out = {r["dav_name"]: dict(r) for r in rows}
if out:
groups: dict[str, list[str]] = defaultdict(list)
for cid, name in (
await s.execute(
text(
"SELECT gm.contact_id, g.name FROM plg_contacts_group_members gm "
"JOIN plg_contacts_groups g ON g.id = gm.group_id "
"WHERE gm.contact_id = ANY(CAST(:ids AS uuid[])) ORDER BY g.name"
),
{"ids": [str(r["id"]) for r in out.values()]},
)
).all():
groups[str(cid)].append(name)
for r in out.values():
r["groups"] = groups.get(str(r["id"]), [])
return out
@staticmethod
def _serve(row: dict[str, Any], with_data: bool) -> tuple[str, str | None]:
fp = vcard.fingerprint(row, row.get("groups") or [])
raw = row.get("srv_vcard")
if raw and row.get("srv_vcard_ver") == fp:
return "r" + hashlib.sha1(raw.encode()).hexdigest(), raw # noqa: S324
return "g" + fp, vcard.build(row, row.get("groups") or []) if with_data else None
# -- collections -----------------------------------------------------------------
async def collections(self, user_id: uuid.UUID) -> list[Collection]:
from webmail_plugin_contacts.router import _ensure_book
async with _SDK.session(user_id) as s:
await _ensure_book(s, user_id) # a new user's first sync finds "Contacts"
rows = (
(
await s.execute(
text(
"SELECT b.id, b.name, b.kind, b.read_only, "
"COALESCE((SELECT max(id) FROM plg_contacts_dav_changes d "
" WHERE d.book_id = b.id), 0) AS token "
"FROM plg_contacts_books b ORDER BY b.name"
)
)
)
.mappings()
.all()
)
return [
Collection(
id=str(r["id"]),
displayname=r["name"],
sync_token=int(r["token"]),
read_only=bool(r["read_only"]) or r["kind"] != "local",
components=[],
)
for r in rows
]
async def create_collection(
self, user_id: uuid.UUID, collection_id: str, displayname: str, color: str | None
) -> None:
try:
bid = uuid.UUID(collection_id)
except ValueError:
raise DavError(403) from None
name = (displayname or "Address book").strip()[:100]
async with _SDK.session(user_id) as s:
taken = {
str(n).lower()
for n in (await s.execute(text("SELECT name FROM plg_contacts_books"))).scalars()
}
base, n = name, 2
while name.lower() in taken:
name, n = f"{base} ({n})", n + 1
await s.execute(
text("INSERT INTO plg_contacts_books (id, user_id, name) VALUES (:id, :u, :name)"),
{"id": str(bid), "u": str(user_id), "name": name},
)
async def update_collection(
self,
user_id: uuid.UUID,
collection_id: str,
*,
displayname: str | None = None,
color: str | None = None,
description: str | None = None,
) -> None:
if not displayname:
return
async with _SDK.session(user_id) as s:
await self._book(s, collection_id)
try:
await s.execute(
text("UPDATE plg_contacts_books SET name = :n WHERE id = :id"),
{"n": displayname.strip()[:100], "id": collection_id},
)
except Exception as err: # noqa: BLE001 - a duplicate name
raise DavError(409) from err
async def delete_collection(self, user_id: uuid.UUID, collection_id: str) -> None:
async with _SDK.session(user_id) as s:
book = await self._book(s, collection_id)
if book["kind"] != "local":
raise DavError(403)
count = await s.scalar(text("SELECT count(*) FROM plg_contacts_books"))
if (count or 0) <= 1:
raise DavError(403) # the last address book stays
await s.execute(
text("DELETE FROM plg_contacts_books WHERE id = :id"), {"id": collection_id}
)
# -- resources -------------------------------------------------------------------
async def resources(self, user_id: uuid.UUID, collection_id: str) -> list[Resource]:
return await self._get(user_id, collection_id, None, with_data=False)
async def get(
self, user_id: uuid.UUID, collection_id: str, names: list[str] | None
) -> list[Resource]:
return await self._get(user_id, collection_id, names, with_data=True)
async def _get(
self, user_id: uuid.UUID, collection_id: str, names: list[str] | None, *, with_data: bool
) -> list[Resource]:
async with _SDK.session(user_id) as s:
await self._book(s, collection_id)
rows = await self._rows(s, collection_id, names)
out = []
for name, row in rows.items():
etag, data = self._serve(row, with_data)
out.append(Resource(name=name, etag=etag, data=data))
return out
async def put(
self,
user_id: uuid.UUID,
collection_id: str,
name: str,
data: str,
*,
if_match: str | None,
if_none_match: bool,
) -> tuple[str, bool]:
try:
fields, groups = vcard.parse(data)
except vcard.InvalidVCard as err:
raise DavError(400, str(err), condition=f"{_CR}valid-address-data") from err
async with _SDK.session(user_id) as s:
book = await self._book(s, collection_id)
if book["read_only"] or book["kind"] != "local":
raise DavError(403)
existing = (await self._rows(s, collection_id, [name])).get(name)
if existing:
etag, _ = self._serve(existing, False)
if if_none_match or (if_match and if_match != "*" and if_match != etag):
raise DavError(412)
if existing["vcard_uid"] != fields["vcard_uid"]:
raise DavError(400, condition=f"{_CR}no-uid-conflict")
else:
if if_match:
raise DavError(412)
clash = await s.scalar(
text(
"SELECT plg_contacts_dav_name(c) FROM plg_contacts_contacts c "
"WHERE book_id = :b AND vcard_uid = :uid"
),
{"b": collection_id, "uid": fields["vcard_uid"]},
)
if clash and clash != name:
raise DavError(409, condition=f"{_CR}no-uid-conflict")
contact_id = await s.scalar(
text(
"INSERT INTO plg_contacts_contacts (user_id, book_id, vcard_uid, name_display, "
" name_parts, nickname, emails, default_email, phones, addresses, urls, org, "
" department, title, birthday, notes, starred, srv_name) "
"VALUES (:u, :b, :uid, :fn, CAST(:n AS jsonb), :nick, CAST(:emails AS jsonb), "
" :def, CAST(:phones AS jsonb), CAST(:adr AS jsonb), CAST(:urls AS jsonb), "
" :org, :dept, :title, :bday, :notes, :star, :name) "
"ON CONFLICT (book_id, vcard_uid) DO UPDATE SET "
" name_display = EXCLUDED.name_display, name_parts = EXCLUDED.name_parts, "
" nickname = EXCLUDED.nickname, emails = EXCLUDED.emails, "
" default_email = EXCLUDED.default_email, phones = EXCLUDED.phones, "
" addresses = EXCLUDED.addresses, urls = EXCLUDED.urls, org = EXCLUDED.org, "
" department = EXCLUDED.department, title = EXCLUDED.title, "
" birthday = EXCLUDED.birthday, notes = EXCLUDED.notes, "
" starred = EXCLUDED.starred, srv_name = EXCLUDED.srv_name "
"RETURNING id"
),
{
"u": str(user_id),
"b": collection_id,
"uid": fields["vcard_uid"],
"fn": fields["name_display"],
"n": json.dumps(fields["name_parts"]),
"nick": fields["nickname"],
"emails": json.dumps(fields["emails"]),
"def": fields["default_email"],
"phones": json.dumps(fields["phones"]),
"adr": json.dumps(fields["addresses"]),
"urls": json.dumps(fields["urls"]),
"org": fields["org"],
"dept": fields["department"],
"title": fields["title"],
"bday": fields["birthday"],
"notes": fields["notes"],
"star": fields["starred"],
"name": name,
},
)
await self._set_groups(s, user_id, str(contact_id), groups)
stored = (await self._rows(s, collection_id, [name]))[name]
fp = vcard.fingerprint(stored, stored.get("groups") or [])
await s.execute(
text(
"UPDATE plg_contacts_contacts SET srv_vcard = :raw, srv_vcard_ver = :fp "
"WHERE id = :id"
),
{"raw": data, "fp": fp, "id": str(contact_id)},
)
return "r" + hashlib.sha1(data.encode()).hexdigest(), not existing # noqa: S324
async def _set_groups(
self, s: Any, user_id: uuid.UUID, contact_id: str, names: list[str]
) -> None:
"""Make the contact's groups exactly ``names`` (creating missing ones)."""
ids: list[str] = []
for name in names:
gid = await s.scalar(
text(
"INSERT INTO plg_contacts_groups (user_id, name) VALUES (:u, :n) "
"ON CONFLICT (user_id, name) DO NOTHING RETURNING id"
),
{"u": str(user_id), "n": name[:100]},
)
if gid is None: # exists already (an update would mark all its members changed)
gid = await s.scalar(
text("SELECT id FROM plg_contacts_groups WHERE name = :n"), {"n": name[:100]}
)
ids.append(str(gid))
await s.execute(
text(
"DELETE FROM plg_contacts_group_members WHERE contact_id = :c "
"AND NOT (group_id = ANY(CAST(:ids AS uuid[])))"
),
{"c": contact_id, "ids": ids},
)
for gid in ids:
await s.execute(
text(
"INSERT INTO plg_contacts_group_members (user_id, group_id, contact_id) "
"VALUES (:u, :g, :c) ON CONFLICT DO NOTHING"
),
{"u": str(user_id), "g": gid, "c": contact_id},
)
async def delete(
self, user_id: uuid.UUID, collection_id: str, name: str, *, if_match: str | None
) -> None:
async with _SDK.session(user_id) as s:
book = await self._book(s, collection_id)
if book["read_only"] or book["kind"] != "local":
raise DavError(403)
existing = (await self._rows(s, collection_id, [name])).get(name)
if not existing:
raise DavError(404)
if if_match and if_match != "*" and if_match != self._serve(existing, False)[0]:
raise DavError(412)
await s.execute(
text("DELETE FROM plg_contacts_contacts WHERE id = :id"),
{"id": str(existing["id"])},
)
async def changes(self, user_id: uuid.UUID, collection_id: str, since: int) -> Changes | None:
async with _SDK.session(user_id) as s:
await self._book(s, collection_id)
rows = (
(
await s.execute(
text(
"SELECT id, name, deleted FROM plg_contacts_dav_changes "
"WHERE book_id = :b AND id > :since ORDER BY id"
),
{"b": collection_id, "since": since},
)
)
.mappings()
.all()
)
latest = await s.scalar(
text(
"SELECT COALESCE(max(id), 0) FROM plg_contacts_dav_changes WHERE book_id = :b"
),
{"b": collection_id},
)
oldest = await s.scalar(
text("SELECT min(id) FROM plg_contacts_dav_changes WHERE book_id = :b"),
{"b": collection_id},
)
if oldest is not None and since < int(oldest) - 1 and since < int(latest or 0):
return None # older entries were trimmed (prune_changes): resync
if since > int(latest or 0):
return None
state: dict[str, bool] = {}
for r in rows:
if r["name"]:
state[r["name"]] = bool(r["deleted"])
return Changes(
sync_token=int(latest or 0),
changed=[n for n, gone in state.items() if not gone],
deleted=[n for n, gone in state.items() if gone],
)
async def query(
self, user_id: uuid.UUID, collection_id: str, start: datetime | None, end: datetime | None
) -> list[str] | None:
return None # address books: the core returns them all
contacts_dav = ContactsDav()
async def prune_changes(ctx: dict[str, Any]) -> int:
"""Nightly: drop change-log entries older than 90 days, keeping each address
book's newest (its sync token). A client holding an older token is told to
resync (``changes`` answers None)."""
async with _SDK.public_session() as pub:
result = await pub.execute(
text(
"DELETE FROM plg_contacts_dav_changes d "
"WHERE created_at < now() - interval '90 days' "
"AND id < (SELECT max(id) FROM plg_contacts_dav_changes k "
" WHERE k.book_id = d.book_id)"
)
)
return int(result.rowcount or 0) # type: ignore[attr-defined]
@@ -0,0 +1,175 @@
"""contacts: serve address books to CardDAV clients (SPEC Appendix E.4)
* ``srv_name`` — the resource name a client stored the contact under (NULL:
``md5(uid).vcf``);
* ``srv_vcard`` / ``srv_vcard_ver`` — the vCard a client PUT, served back
verbatim while the contact's fields (and groups) still match;
* ``plg_contacts_dav_changes`` — numbered changes per address book (sync token /
CTag), filled by triggers: a contact changed, its groups changed (they are
its CATEGORIES), a group renamed, a book renamed.
Revision ID: contacts0002
Revises: contacts0001
Create Date: 2026-10-09
"""
from __future__ import annotations
from collections.abc import Sequence
from alembic import op
from app.plugins.rls import enable_rls
revision: str = "contacts0002"
down_revision: str | None = "contacts0001"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
for col in ("srv_name", "srv_vcard", "srv_vcard_ver"):
op.execute(f"ALTER TABLE plg_contacts_contacts ADD COLUMN {col} text")
op.execute(
"""
CREATE TABLE plg_contacts_dav_changes (
id bigserial PRIMARY KEY,
user_id uuid NOT NULL,
book_id uuid NOT NULL,
name text, -- NULL: the address book itself changed
deleted boolean NOT NULL DEFAULT false,
created_at timestamptz NOT NULL DEFAULT now()
)
"""
)
op.execute(
"CREATE INDEX ix_plg_contacts_dav_changes_book ON plg_contacts_dav_changes (book_id, id)"
)
enable_rls("plg_contacts_dav_changes")
op.execute("GRANT USAGE ON SEQUENCE plg_contacts_dav_changes_id_seq TO webmail_plugin")
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_contacts_dav_name(c plg_contacts_contacts) RETURNS text AS $$
SELECT COALESCE(c.srv_name, md5(c.vcard_uid) || '.vcf')
$$ LANGUAGE sql IMMUTABLE;
"""
)
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_contacts_touch() RETURNS trigger AS $$
BEGIN
NEW.updated_at := now();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_contacts_contacts_touch
BEFORE UPDATE ON plg_contacts_contacts
FOR EACH ROW EXECUTE FUNCTION plg_contacts_touch();
"""
)
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_contacts_log_change() RETURNS trigger AS $$
BEGIN
IF TG_OP = 'DELETE' OR (TG_OP = 'UPDATE' AND NEW.book_id <> OLD.book_id) THEN
INSERT INTO plg_contacts_dav_changes (user_id, book_id, name, deleted)
VALUES (OLD.user_id, OLD.book_id, plg_contacts_dav_name(OLD), true);
END IF;
IF TG_OP <> 'DELETE' THEN
INSERT INTO plg_contacts_dav_changes (user_id, book_id, name, deleted)
VALUES (NEW.user_id, NEW.book_id, plg_contacts_dav_name(NEW), false);
END IF;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_contacts_contacts_log
AFTER INSERT OR UPDATE OR DELETE ON plg_contacts_contacts
FOR EACH ROW EXECUTE FUNCTION plg_contacts_log_change();
"""
)
# Group membership is part of the vCard (CATEGORIES).
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_contacts_log_member() RETURNS trigger AS $$
DECLARE
m plg_contacts_group_members;
BEGIN
m := CASE WHEN TG_OP = 'DELETE' THEN OLD ELSE NEW END;
INSERT INTO plg_contacts_dav_changes (user_id, book_id, name, deleted)
SELECT c.user_id, c.book_id, plg_contacts_dav_name(c), false
FROM plg_contacts_contacts c WHERE c.id = m.contact_id;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_contacts_group_members_log
AFTER INSERT OR DELETE ON plg_contacts_group_members
FOR EACH ROW EXECUTE FUNCTION plg_contacts_log_member();
"""
)
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_contacts_log_group() RETURNS trigger AS $$
BEGIN
INSERT INTO plg_contacts_dav_changes (user_id, book_id, name, deleted)
SELECT c.user_id, c.book_id, plg_contacts_dav_name(c), false
FROM plg_contacts_group_members gm
JOIN plg_contacts_contacts c ON c.id = gm.contact_id
WHERE gm.group_id = NEW.id;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_contacts_groups_log
AFTER UPDATE OF name ON plg_contacts_groups
FOR EACH ROW EXECUTE FUNCTION plg_contacts_log_group();
"""
)
op.execute(
"""
CREATE OR REPLACE FUNCTION plg_contacts_log_book() RETURNS trigger AS $$
BEGIN
INSERT INTO plg_contacts_dav_changes (user_id, book_id, name, deleted)
VALUES (NEW.user_id, NEW.id, NULL, false);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
"""
)
op.execute(
"""
CREATE TRIGGER plg_contacts_books_log
AFTER INSERT OR UPDATE OF name ON plg_contacts_books
FOR EACH ROW EXECUTE FUNCTION plg_contacts_log_book();
"""
)
def downgrade() -> None:
op.execute("DROP TRIGGER IF EXISTS plg_contacts_books_log ON plg_contacts_books")
op.execute("DROP TRIGGER IF EXISTS plg_contacts_groups_log ON plg_contacts_groups")
op.execute(
"DROP TRIGGER IF EXISTS plg_contacts_group_members_log ON plg_contacts_group_members"
)
op.execute("DROP TRIGGER IF EXISTS plg_contacts_contacts_log ON plg_contacts_contacts")
op.execute("DROP TRIGGER IF EXISTS plg_contacts_contacts_touch ON plg_contacts_contacts")
for fn in ("log_book", "log_group", "log_member", "log_change", "touch"):
op.execute(f"DROP FUNCTION IF EXISTS plg_contacts_{fn}()")
op.execute("DROP FUNCTION IF EXISTS plg_contacts_dav_name(plg_contacts_contacts)")
op.execute("DROP TABLE IF EXISTS plg_contacts_dav_changes")
for col in ("srv_vcard_ver", "srv_vcard", "srv_name"):
op.execute(f"ALTER TABLE plg_contacts_contacts DROP COLUMN IF EXISTS {col}")
@@ -0,0 +1,338 @@
"""vCard (RFC 2426 v3.0, RFC 6350 v4.0) <-> contact fields (SPEC App. E.4).
:func:`parse` reads a vCard 3.0 or 4.0 a client sent into the columns of
``plg_contacts_contacts`` (plus its CATEGORIES, which are the user's groups);
:func:`build` writes a contact as vCard 3.0, which every client reads.
:func:`fingerprint` summarises the modelled fields, so a client's own text can
be served back verbatim for as long as they are unchanged.
"""
from __future__ import annotations
import hashlib
import json
import re
from collections.abc import Mapping, Sequence
from datetime import date, datetime
from typing import Any
FIELDS = (
"vcard_uid",
"name_display",
"name_parts",
"nickname",
"emails",
"default_email",
"phones",
"addresses",
"urls",
"org",
"department",
"title",
"birthday",
"notes",
"starred",
)
class InvalidVCard(ValueError):
pass
def fingerprint(row: Mapping[str, Any], groups: Sequence[str]) -> str:
data = {f: row.get(f) for f in FIELDS}
data["groups"] = sorted(groups, key=str.lower)
return hashlib.sha1(json.dumps(data, sort_keys=True, default=str).encode()).hexdigest()
# ---- text level -----------------------------------------------------------------------
def _unfold(text: str) -> list[str]:
lines: list[str] = []
for raw in text.replace("\r\n", "\n").replace("\r", "\n").split("\n"):
if raw[:1] in (" ", "\t") and lines:
lines[-1] += raw[1:]
elif raw.strip():
lines.append(raw)
return lines
def _split_unescaped(value: str, sep: str) -> list[str]:
parts, cur, esc = [], [], False
for ch in value:
if esc:
cur.append("\\" + ch)
esc = False
elif ch == "\\":
esc = True
elif ch == sep:
parts.append("".join(cur))
cur = []
else:
cur.append(ch)
parts.append("".join(cur))
return parts
def _unescape(value: str) -> str:
return re.sub(r"\\(.)", lambda m: "\n" if m.group(1) in "nN" else m.group(1), value)
def _escape(value: str) -> str:
return value.replace("\\", "\\\\").replace(";", "\\;").replace(",", "\\,").replace("\n", "\\n")
class _Line:
def __init__(self, raw: str) -> None:
head, sep, value = self._split(raw)
if not sep:
raise InvalidVCard(f"bad line: {raw[:40]}")
bits = _split_params(head)
name = bits[0]
self.name = name.rsplit(".", 1)[-1].upper() # "item1.EMAIL" -> EMAIL
self.params: dict[str, list[str]] = {}
for p in bits[1:]:
key, eq, val = p.partition("=")
if not eq: # vCard 2.1-style bare type ("WORK")
key, val = "TYPE", key
vals = [v.strip('"') for v in val.split(",")]
self.params.setdefault(key.upper(), []).extend(v for v in vals if v)
self.value = value
@staticmethod
def _split(raw: str) -> tuple[str, str, str]:
quoted = False
for i, ch in enumerate(raw):
if ch == '"':
quoted = not quoted
elif ch == ":" and not quoted:
return raw[:i], ":", raw[i + 1 :]
return raw, "", ""
def types(self) -> set[str]:
return {t.lower() for t in self.params.get("TYPE", [])}
def pref(self) -> bool:
return "pref" in self.types() or bool(self.params.get("PREF"))
def _split_params(head: str) -> list[str]:
out, cur, quoted = [], [], False
for ch in head:
if ch == '"':
quoted = not quoted
cur.append(ch)
elif ch == ";" and not quoted:
out.append("".join(cur))
cur = []
else:
cur.append(ch)
out.append("".join(cur))
return out
# ---- vCard -> fields ------------------------------------------------------------------
def _birthday(value: str) -> date | None:
v = value.strip()
for fmt in ("%Y-%m-%d", "%Y%m%d"):
try:
return datetime.strptime(v[:10] if "-" in v else v[:8], fmt).date()
except ValueError:
continue
return None # "--MMDD" (no year) and the like are not kept as a date
def parse(text: str) -> tuple[dict[str, Any], list[str]]:
"""(contact fields, group names) of one vCard. Raises InvalidVCard."""
lines = _unfold(text)
if not lines or lines[0].strip().upper() != "BEGIN:VCARD":
raise InvalidVCard("not a vCard")
if sum(1 for line in lines if line.strip().upper() == "BEGIN:VCARD") != 1:
raise InvalidVCard("one vCard per resource")
row: dict[str, Any] = {
"vcard_uid": "",
"name_display": "",
"name_parts": {"prefix": "", "given": "", "middle": "", "family": "", "suffix": ""},
"nickname": "",
"emails": [],
"default_email": "",
"phones": [],
"addresses": [],
"urls": [],
"org": "",
"department": "",
"title": "",
"birthday": None,
"notes": "",
"starred": False,
}
groups: list[str] = []
for raw in lines[1:]:
if raw.strip().upper() == "END:VCARD":
break
line = _Line(raw)
name, value, types = line.name, line.value, line.types()
if name == "UID":
v = _unescape(value).strip()
row["vcard_uid"] = v[9:] if v.lower().startswith("urn:uuid:") else v
elif name == "FN":
row["name_display"] = _unescape(value).strip()
elif name == "N":
parts = [_unescape(p).strip() for p in _split_unescaped(value, ";")] + [""] * 5
row["name_parts"] = {
"family": parts[0],
"given": parts[1],
"middle": parts[2],
"prefix": parts[3],
"suffix": parts[4],
}
elif name == "NICKNAME":
row["nickname"] = _unescape(_split_unescaped(value, ",")[0]).strip()
elif name == "EMAIL":
addr = _unescape(value).strip()
if addr.lower().startswith("mailto:"):
addr = addr[7:]
if addr:
kind = "work" if "work" in types else "personal" if "home" in types else "other"
row["emails"].append({"value": addr, "type": kind, "pref": line.pref()})
elif name == "TEL":
number = _unescape(value).strip()
if number.lower().startswith("tel:"):
number = number[4:]
if number:
kind = (
"mobile"
if "cell" in types
else "work"
if "work" in types
else "home"
if "home" in types
else "other"
)
row["phones"].append({"value": number, "type": kind})
elif name == "ADR":
p = [_unescape(x).strip() for x in _split_unescaped(value, ";")] + [""] * 7
street = ", ".join(x for x in (p[1], p[2]) if x) # extended + street
kind = "work" if "work" in types else "home" if "home" in types else "other"
if any((street, p[3], p[5], p[6])):
row["addresses"].append(
{
"type": kind,
"street": street,
"city": p[3],
"postal_code": p[5],
"country": p[6],
}
)
elif name == "URL":
url = _unescape(value).strip()
if url:
row["urls"].append({"value": url, "type": "website"})
elif name == "ORG":
p = [_unescape(x).strip() for x in _split_unescaped(value, ";")] + [""]
row["org"], row["department"] = p[0], p[1]
elif name == "TITLE":
row["title"] = _unescape(value).strip()
elif name == "BDAY":
row["birthday"] = _birthday(_unescape(value))
elif name == "NOTE":
row["notes"] = _unescape(value)
elif name == "CATEGORIES":
groups.extend(
g for g in (_unescape(x).strip() for x in _split_unescaped(value, ",")) if g
)
elif name == "X-ULTIMO-VIP":
row["starred"] = value.strip() in ("1", "TRUE", "true")
if not row["vcard_uid"]:
raise InvalidVCard("UID is required")
if not row["name_display"]:
n = row["name_parts"]
row["name_display"] = (
" ".join(
x for x in (n["prefix"], n["given"], n["middle"], n["family"], n["suffix"]) if x
)
or row["org"]
or (row["emails"][0]["value"] if row["emails"] else "")
)
pref = next((e["value"] for e in row["emails"] if e["pref"]), None)
row["default_email"] = pref or (row["emails"][0]["value"] if row["emails"] else "")
return row, list(dict.fromkeys(groups))
# ---- fields -> vCard ------------------------------------------------------------------
def _fold(line: str) -> str:
"""Fold at 75 octets (RFC 6350 §3.2), never inside a UTF-8 character."""
out: list[str] = []
cur = b""
for ch in line:
b = ch.encode()
if len(cur) + len(b) > (75 if not out else 74):
out.append(cur.decode())
cur = b""
cur += b
out.append(cur.decode())
return "\r\n ".join(out)
def build(row: Mapping[str, Any], groups: Sequence[str]) -> str:
"""A contact as vCard 3.0."""
lines = ["BEGIN:VCARD", "VERSION:3.0", "PRODID:-//Ultimo//Contacts//EN"]
lines.append(f"UID:{_escape(str(row['vcard_uid']))}")
lines.append(f"FN:{_escape(row.get('name_display') or '')}")
n = row.get("name_parts") or {}
lines.append(
"N:"
+ ";".join(
_escape(str(n.get(k, "") or ""))
for k in ("family", "given", "middle", "prefix", "suffix")
)
)
if row.get("nickname"):
lines.append(f"NICKNAME:{_escape(row['nickname'])}")
if row.get("org") or row.get("department"):
lines.append(f"ORG:{_escape(row.get('org') or '')};{_escape(row.get('department') or '')}")
if row.get("title"):
lines.append(f"TITLE:{_escape(row['title'])}")
default = (row.get("default_email") or "").lower()
for e in row.get("emails") or []:
kind = {"work": "WORK", "personal": "HOME"}.get(e.get("type", ""), "OTHER")
pref = (
",pref" if e.get("pref") or (default and e.get("value", "").lower() == default) else ""
)
lines.append(f"EMAIL;TYPE=INTERNET,{kind}{pref}:{_escape(e.get('value', ''))}")
for p in row.get("phones") or []:
kind = {"mobile": "CELL", "home": "HOME", "work": "WORK"}.get(p.get("type", ""), "VOICE")
lines.append(f"TEL;TYPE={kind}:{_escape(p.get('value', ''))}")
for a in row.get("addresses") or []:
kind = {"home": "HOME", "work": "WORK"}.get(a.get("type", ""), "OTHER")
parts = [
"",
"",
a.get("street", ""),
a.get("city", ""),
"",
a.get("postal_code", ""),
a.get("country", ""),
]
lines.append(f"ADR;TYPE={kind}:" + ";".join(_escape(str(x or "")) for x in parts))
for u in row.get("urls") or []:
lines.append(f"URL:{_escape(u.get('value', ''))}")
bday = row.get("birthday")
if bday:
lines.append(f"BDAY:{bday.isoformat() if hasattr(bday, 'isoformat') else bday}")
if row.get("notes"):
lines.append(f"NOTE:{_escape(row['notes'])}")
if groups:
lines.append("CATEGORIES:" + ",".join(_escape(g) for g in groups))
if row.get("starred"):
lines.append("X-ULTIMO-VIP:1")
updated = row.get("updated_at")
if isinstance(updated, datetime):
lines.append(f"REV:{updated.strftime('%Y%m%dT%H%M%SZ')}")
lines.append("END:VCARD")
return "\r\n".join(_fold(line) for line in lines) + "\r\n"
+32
View File
@@ -874,6 +874,11 @@ CREATE VIEW mail_principals AS
### D.5 External mail clients and 2FA ### D.5 External mail clients and 2FA
If phones/desktop clients may use the mailbox over IMAP/SMTP, a password alone would bypass TOTP. Rule: users with 2FA enabled must use **app passwords** for IMAP/SMTP — a `user_app_passwords` table (argon2id hashes) added to the directory view, plus a Settings screen to create/revoke them. Stalwart's login rate limits / auto-ban on. If external clients are not wanted, IMAP/SMTP submission is exposed only on the Docker network and this section shrinks to nothing. If phones/desktop clients may use the mailbox over IMAP/SMTP, a password alone would bypass TOTP. Rule: users with 2FA enabled must use **app passwords** for IMAP/SMTP — a `user_app_passwords` table (argon2id hashes) added to the directory view, plus a Settings screen to create/revoke them. Stalwart's login rate limits / auto-ban on. If external clients are not wanted, IMAP/SMTP submission is exposed only on the Docker network and this section shrinks to nothing.
**App passwords — built (v1.3.37, ahead of D5):** table `user_app_passwords` (migration 0024: `id, user_id, name, password_hash, created_at, last_used_at`), `app/identity/app_passwords.py`, routes `GET/POST /api/auth/app-passwords`, `DELETE /api/auth/app-passwords/{id}`, Settings → Security → App passwords.
- 16 random lowercase letters (~75 bits), shown once in groups of four, copied and entered **without spaces**: the hash is of the bare letters, because the mail server compares exactly what an app sends.
- Creating one takes the account password; at most 25 per user; each is revoked on its own; a password reset revokes them all (as Google does). The webmail's own sign-in never accepts one.
- Still for D2/D5: add the hashes to the directory's secrets query (a user's secrets = `users.password_hash` unless 2FA is on, plus every `user_app_passwords.password_hash`), and set `last_used_at` from the mail server's successful sign-ins.
### D.6 Storing mail once ### D.6 Storing mail once
Today the indexer keeps every message in full (raw `.eml` blob + sanitized body in `body_parts` + search index), so external accounts are already a second copy, and with Stalwart both copies would sit on the same disk. Ideas, best first: Today the indexer keeps every message in full (raw `.eml` blob + sanitized body in `body_parts` + search index), so external accounts are already a second copy, and with Stalwart both copies would sit on the same disk. Ideas, best first:
@@ -912,5 +917,32 @@ Today the indexer keeps every message in full (raw `.eml` blob + sanitized body
3. May external mail clients (IMAP/SMTP) use these mailboxes? 3. May external mail clients (IMAP/SMTP) use these mailboxes?
4. Ship D1 ahead of the rest? 4. Ship D1 ahead of the rest?
## Appendix E — CalDAV / CardDAV server (phones, Thunderbird, macOS)
Ultimo serves the calendars and address books it already holds to standard clients — iPhone/iPad and macOS Calendar & Contacts, Thunderbird, Android via DAVx5, GNOME/Evolution — so they sync two-way with the web app. No second store: clients read and write the calendar/contacts plugins' own tables. (Chosen over Stalwart's DAV, which would make Stalwart the store and the plugins its clients, §D.)
### E.1 Shape
- **Core `app/dav/`** — the protocol: routes, Basic sign-in, discovery, principal, XML (PROPFIND / PROPPATCH / REPORT multistatus), ETag preconditions, sync tokens (RFC 6578). No knowledge of events or contacts.
- **Providers in the plugins** — `Plugin(dav=...)`: the calendar plugin serves calendar collections (iCalendar, RFC 4791), the contacts plugin address books (vCard, RFC 6352). Interface: list collections; list resources (name + ETag); get; put (If-Match / If-None-Match); delete; changes since a sync token; create / update / delete a collection; calendar time-range query. Each runs in the user's RLS session (`sdk.session`).
- **URLs** — `/.well-known/caldav`, `/.well-known/carddav` → 301 `/dav/`; principal `/dav/principals/{user_id}/`; homes `/dav/calendars/{user_id}/`, `/dav/addressbooks/{user_id}/`; a collection `…/{collection_id}/`; a resource `…/{name}.ics|.vcf`. nginx proxies `/dav/` and the two well-known paths to the API.
### E.2 Sign-in
HTTP Basic over HTTPS: the handle (full address or local part) and an **app password** (§D.5) — never the account password, so 2FA cannot be bypassed and each device can be revoked. A successful check is cached briefly (keyed by a hash of the password, invalidated when app passwords change) so a sync's many requests don't each cost an argon2id verify; failures are rate-limited per address and IP. `last_used_at` is set (at most every few minutes).
### E.3 Calendars
- A **resource** is one iCalendar object per UID: the series plus its moved/cancelled occurrences (`RECURRENCE-ID`). Name: what the client PUT it as, else derived from the UID.
- **Fidelity:** what a client PUTs is kept verbatim and served back unchanged while the event's modelled fields are still what it said; once the web app (or iMIP, or an upstream sync) changes them, the object is regenerated from the fields — SUMMARY, DESCRIPTION, LOCATION, DTSTART/DTEND with TZID and VTIMEZONE, all-day dates, RRULE, EXDATE, overrides, STATUS, TRANSP, ORGANIZER, ATTENDEEs with PARTSTAT, VALARMs from reminders, COLOR, the video link.
- **Writes** go through the same columns the web app uses, so reminders, invitations and upstream CalDAV write-back (the `dav_dirty` trigger, tombstones on synced calendars) behave identically. Subscribed (ICS) and read-only calendars are served read-only.
- **Change tracking** by triggers on the plugin tables (any write path counts) into a per-calendar change log; a calendar's sync token / CTag is its latest change number; deletions are reported by name. Old log entries are pruned; a client holding a pruned token gets `valid-sync-token` and resyncs.
- REPORTs: `calendar-query` (time-range on VEVENT), `calendar-multiget`, `sync-collection`. MKCALENDAR and PROPPATCH (name, colour) create / restyle local calendars; DELETE of a collection deletes a local calendar.
### E.4 Contacts
Address books = `plg_contacts_books`; resources = contacts as vCard (3.0 served; 3.0 and 4.0 accepted): FN, N, NICKNAME, EMAIL (type, pref), TEL, ADR, URL, ORG/department, TITLE, BDAY, NOTE, CATEGORIES ↔ groups, the VIP star as an `X-` property. Same fidelity rule (`raw_vcard` kept until the web app edits), same change tracking. REPORTs: `addressbook-query` (basic), `addressbook-multiget`, `sync-collection`.
### E.5 Phases
- **E1** core + discovery + sign-in + calendars read/write (tested with a CalDAV client library);
- **E2** vCard read/write + address books;
- **E3** deployment (nginx, docs: how to add the account on each client), then checks with real devices.
--- ---
*End of specification. Ambiguities: prefer the simplest behavior consistent with the invariants in §1, leave a `TODO(spec)` comment, and list open questions in `docs/open-questions.md` rather than guessing silently.* *End of specification. Ambiguities: prefer the simplest behavior consistent with the invariants in §1, leave a `TODO(spec)` comment, and list open questions in `docs/open-questions.md` rather than guessing silently.*