v1.3.26's IMAP4_SSL subclass put its create_client ahead of aioimaplib's,
which is where the TLS context is made — so port 993 got plain TCP, the
server waited for a handshake, and every SSL account timed out ("Can't
reach the mail server"). The context is now made in our create_client too,
and a test checks an SSL client really gets one (it fails without the fix).
With IMAP down, a send could still be delivered by SMTP while filing its copy
in Sent failed — the send then counted as done and the copy was lost. The
copy is now kept and filed by file_sent_copy_job, retried (arq Retry) for
about 24 hours, then shown in Sent as usual.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sending: a send the server could not take (unreachable, timeout, 4xx) is no
longer failed at once. It waits in Scheduled ("retrying") and is tried again
after 1, 2, 5, 10, 30 min, then hourly, for 24 hours — on the server, so it
works with the browser closed. A permanent refusal (5xx, wrong password) is
"failed" at once, with the server's reason. GET /outbox lists both;
POST /outbox/{id}/retry (Retry now) and /cancel (Edit, Delete) act on them,
and they are left out of Drafts. Mail delivered but not filed in Sent now
counts as sent, and a conditional claim keeps two jobs from sending one
message twice. The app says when a send failed and when it finally went.
Actions: arq retries a job only on Retry, never on a plain exception, so the
"retry x5, then revert" for flags and moves never happened. Flag, move,
archive, restore, delete and empty-folder jobs now raise Retry while the
server is unreachable (~24 hours of growing gaps), skip a change a later one
replaced, stop at the first connection failure instead of 20 s per message,
and delete/empty keep the rows until the server has the messages gone.
Account health: 3 failed connections in a row mark an account unreachable
(amber in the sidebar and Settings, cleared on the next success); 3 refused
sign-ins stop its syncing as auth_failed until the password is entered
again, so the provider does not lock the account. Sync loops back off 30 s
doubling to 10 min and log one line per expected failure; a refused
connection fails at once with its reason (the connect task is kept), and a
silently dead IDLE connection is noticed within ~11 min instead of ~29.
Also: the empty reading pane is clipped to its width when dragged very
narrow; tests no longer leave live accounts and outbox rows in the database.
Migration 0021: outbox attempts / next_attempt_at / retry_until, accounts
sync_error / sync_error_since.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reported from the running app: a message showed as unread in the list — bold,
blue dot — while the reading pane showed it as read, and only a manual "mark as
read" reconciled them.
The two views read different things. The pane reads the message's own flags; the
list reads `threads.unread_count`, denormalized from them. `apply_delta` updated
message flags with a bare UPDATE and never recomputed the thread, so a flag set
anywhere but in this app — read on a phone, filed by a server-side rule — moved
one and not the other. Auto-mark-on-read could not fix it either: it sees a
message that is already \Seen and correctly does nothing, leaving the counter
wrong forever.
Expunges had the same hole, which also left `message_count` overstated.
`apply_delta` now collects the affected threads and re-denormalizes them after
the deletes, so a thread emptied by an expunge is removed rather than recounted.
Migration 0018 repairs rows that already drifted — recomputed from the messages,
so it is a no-op on a healthy database. One thread was stale here; installs that
have been syncing longer will have more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
Gmail sync (SPEC: Gmail sync):
- Provider fetches X-GM-LABELS when X-GM-EXT-1 is advertised; parses custom labels
(system labels dropped, mUTF-7 decoded) onto RemoteMessage.
- Engine syncs only system folders + All Mail for Gmail, skips every label folder,
and dedupes the All Mail pass by Message-ID — kills the INBOX/All-Mail duplication.
- Custom labels ride the message.new event; the labels plugin subscribes and maps
them to chips as "<name> [Gmail]" (per-message + thread aggregate + known set),
so labels fill in gradually with the sync window. Non-ASCII labels decode right.
Labels self-healing (message-driven reconcile):
- Per-message keys are the source of truth; t: aggregates and _labels are derived.
Reconcile drops labels of deleted messages, regroups survivors by their current
thread (fixing JWZ thread merges), rewrites changed t: keys, and rebuilds _labels
from live messages — keeping user labels, dropping orphaned Gmail ones.
- New SDK reads: sdk.threads.existing / sdk.messages.threads_of. Sidebar refreshes
live via a plugin.labels.changed WS event on new labels.
Account deletion: prune threads left empty after the cascade (they were user-scoped,
so only messages cascaded) + de-list the account from surviving cross-account threads.
Sync SPAM: opt-in per-account sync_spam (migration 0015, default off) — the engine
skips the Junk folder in backfill and poll unless enabled; wizard checkbox added.
Robustness / UX:
- WebSocket auto-reconnects with backoff and resyncs on (re)connect, and account
add/delete invalidates threads/labels — no more hard refresh for live updates.
- Backfill banner + sidebar ring also require done < total, so a completed account
can't leave a stuck "X of X".
- Scrub lone UTF-16 surrogates from parsed headers/bodies so bad messages no longer
crash the backfill on Postgres text/JSONB.
- Ultimo signet logo in the Mail/Calendar/Contacts sidebar header (and at the top of
the collapsed rail), via the shared plugin-ui.css + MailLayout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
Server-side deletions now propagate to the DB. Previously poll() detected new
messages and CONDSTORE flag deltas but never expunges (the old TODO), so mail
deleted in another client lingered as phantoms outside the small Drafts reconcile.
- provider: enable QRESYNC when advertised (_maybe_enable_qresync + qresync_enabled
property). poll() harvests VANISHED (expunged UIDs) and changed FLAGS in one pass
via a raw `SELECT <mbox> (QRESYNC (uidvalidity modseq))` — issued as a low-level
SELECT because aioimaplib only surfaces untagged VANISHED lines on a sync command,
not on an async UID FETCH. CONDSTORE-only servers (e.g. Gmail) keep the CHANGEDSINCE
flag delta and fall back to the engine reconcile.
- parsing: parse_vanished() for `VANISHED [(EARLIER)] <uid-set>` with range expansion.
- engine: apply_delta now drops expunged rows from the search index, un-links any
local draft mirroring a vanished UID, and publishes message.deleted for the UI.
- tests: parse_vanished unit cases + apply_delta expunge (removal, draft un-link,
modseq advance).
Verified live: the worker issues `SELECT "INBOX" (QRESYNC (uidvalidity modseq))`
against QRESYNC-capable accounts. ruff + mypy --strict clean; 296 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcj3TpydMTzTSmU7kunzf8
Search (PG backend):
- '@' terms match address substrings (from/to/cc), so a partial email like
"dyrektor@wfdif" resolves and narrows predictably as you type
- index each address split into parts, so a bare domain/local word ("wfdif")
finds the address (indexed as one lexeme otherwise); dotted terms fall back to
an address substring
- rank sender/recipient name+address matches above body/subject-only ones
(addr_hit tier, then recency); cursor extended to keep keyset pagination correct
- scripts/reindex_address_parts.py backfills address tokens into existing rows
Inline images:
- extract attachments via a full MIME walk instead of shallow iter_attachments(),
which missed inline images nested in mixed>related>alternative (cid: rendered
broken); has_attachments counts only real (non-cid) attachments
- scripts/backfill_inline_attachments.py stores previously-missed inline images
Folder-name i18n (SPEC §12 M2):
- accent/punctuation-insensitive, multi-delimiter (/ \ .) special-use inference
across ~13 languages; curated names (no substring false positives)
Link security (reading panel):
- links never navigate the message iframe: a parent-side click interceptor opens
http/https in a new tab (noopener) and mailto in the composer (recipient
pre-filled); all other schemes blocked
Backend 292 tests green, ruff + mypy --strict clean; frontend tsc/eslint/vitest clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
Search:
- Meilisearch backend (app/search/meili.py) + factory wiring; pinned SDK 2.7
- Search-as-you-type: debounced query + Postgres prefix matching, built from
the DB tokenizer so full email addresses match (email stays one lexeme)
- Saved searches API + sidebar virtual mailboxes; save affordance in the box
Account sync windows (SPEC §12 M2):
- Choose at setup: All (recent 30d first, then remainder), Last 3 months, New only
- Per-account sync_window + "Sync remaining mail" from settings (idempotent re-run)
- Provider windowed backfill (SINCE/BEFORE), message_count, newest_uid
- migration 0012: accounts.sync_window + backfill_total/backfill_done
Backfill progress UX:
- Floating progress card (auto-dissolves after 10s) + spinning sidebar ring that
fills with the account colour; driven by sync.progress over WS
Thread list pagination:
- Server-side offset + accurate grand total, with filter/order in SQL; working
prev/next across all synced mail
Reading panel:
- "Load images" one-off now actually renders images (?images=1) without trusting
the sender; envelope carries images_allowed so a trusted sender shows no prompt
Threading:
- Subject fallback no longer merges messages that declare an explicit (different)
parent; true siblings still group via a shared-parent reference match. Fixes
unrelated auto-replies with the same subject landing in one thread
Backend 257 tests green, ruff + mypy --strict clean; frontend tsc/eslint/vitest clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
Backend:
- Restore returns each message to its original folder (e.g. Sent/Archive) instead
of always the Inbox: new messages.restore_mailbox_id (migration 0010) +
restore_thread_job.
- Drafts: expunge reconciliation for the Drafts poll (provider.list_uids /
_reconcile_drafts) clears phantom "remote" drafts; send + discard drop the
mirrored DB row; adopting a remote draft carries its real attachments into the
compose flow.
- Thread detail/list are folder-aware: individually trashed/spam messages are
hidden outside their folder, Trash/Spam show only their own messages, and the
mail-list message count matches the view.
Frontend:
- Thread reading panel: fix newest-first card overlap/spacing (expand-all and
single expand), hide Reply/Forward on trashed messages, close the Actions menu
on message-body (iframe) clicks, scroll to the inline composer, and prompt
Save draft / Discard on the X close.
- Composer: remount when switching drafts, dirty-tracking to stop duplicate IMAP
mirrors, and close the From/More dropdowns on outside click.
- Sidebar: collapse/expand toggle (icons-only ~60px rail) and a log-out icon on
Sign out.
- Mail-standard printing in a new window with reliable page margins (lib/print.ts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
- Heartbeat JSON now carries {owned, sockets_open, tiers:{hot,warm,cold}, ts} per
§2 so peers and operators can see each worker's load and socket footprint.
- run.py runs the sync coordinator directly (not via a placeholder arq worker) so
SIGINT/SIGTERM can stop the claimer, cancel runners, and release every lease
explicitly within a 10s drain budget — peers adopt the accounts immediately
instead of waiting out the lease TTL. Removed the sync:control-queue hack and
the supervisor indirection (start/stop_supervisor, sync_on_startup/shutdown).
- Acceptance: graceful-shutdown test proves a peer adopts released accounts within
seconds under a 30s TTL (i.e. via release, not expiry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each owned account is tiered by the coordinator; the runner shapes its work to
match — correctness identical, only latency/socket cost differ.
- hot: persistent INBOX IDLE + secondary poll every POLL_WARM.
- warm: no persistent socket; transient INBOX+special poll every POLL_WARM.
- cold: same transient poll every POLL_COLD.
- engine: poll_all_loop (transient full poll); secondary_poll_loop interval is
now a parameter; POLL_INTERVAL removed in favor of POLL_WARM/POLL_COLD.
- coordinator: tier eval every 120s from session heat (sess:active:{user}) +
recent mail (max internal_date within TIER_HOT_ACTIVITY_H), with the
HOT_BUDGET_PER_WORKER guard shedding least-recently-active session-less hot
accounts; user.active event promotes a user's accounts to hot at once (≤5s).
- API: login + WS connect/ping publish user.active and refresh sess:active
(TTL TIER_SESSION_GRACE_MIN). New sync:tier:{id} key, default warm on claim.
- config: POLL_WARM/POLL_COLD/TIER_*/HOT_BUDGET_PER_WORKER.
- tests: 5 tier tests (session/recent-mail promotion, cold, user.active,
budget demotion) alongside the lease/lifecycle acceptance tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Coordinator subscribes to account.state_changed over Redis pub/sub: going
live marks the account claimable at once; disabled/deleted/error tears the
owning runner down immediately (≤30s) instead of waiting for the lease TTL.
- delete_account now publishes account.state_changed:deleted after dropping the
account from sync:live, so teardown is event-driven, not just fencing.
- AccountRunner gains a verify loop (defense in depth): it periodically confirms
the account is still live and flags itself dead if not; the coordinator reaps
dead runners and releases their lease. Zombie loops are structurally impossible.
- New core event user.active (SPEC §8.3), published on login and on every WS
connect; consumed for tier promotion in step 5.
- Acceptance: added event-driven teardown test (isolated from fencing/verify).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ownership of an account now moves through a Redis lease so any number of sync
workers can run against the same account set with no double-IDLE.
- leases.py: sync:live registry + sync:lease:{id} (SET NX EX) with owner-checked
Lua renew/release; mark_live / mark_not_live helpers.
- coordinator.py: SyncCoordinator claims from sync:live up to MAX_ACCOUNTS_PER_
WORKER, renews every LEASE_RENEW (fences the runner on lease loss), reconciles
sync:live against Postgres, heartbeats a worker key, and rebalances when it
holds well above a fair share.
- supervisor start/stop now drive a coordinator; engine._set_state and
delete_account maintain sync:live membership (+ lease drop = fencing teardown).
- config: MAX_ACCOUNTS_PER_WORKER, LEASE_TTL, LEASE_RENEW.
- tests: acceptance 1-3 (single ownership, crash failover, create/delete
lifecycle) driving the real coordinator + Redis with faked IMAP loops.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backend
- Sent copies: persist locally on send + periodic multi-folder CONDSTORE poll
(Sent/Archive/Trash/Junk) so they thread and show without an IDLE-only INBOX.
- Bcc: transmitted copy omits Bcc, Sent copy keeps it; store/display bcc_addr.
- Drafts: mirror local drafts to the IMAP Drafts folder (append/replace/expunge);
inbound sync of foreign drafts (un-threaded) with adopt-to-edit + dedup by UID.
- Per-user storage quota (5 GiB default) and manual "sync all folders" action.
- Account editing (display/color/server/password) and per-account HTML signature.
- Account view scoped to that account's inbox; new migrations 0006–0009.
Frontend
- Mail-list pagination, folder filters (All/Unread/Pinned), sort toggle, refresh.
- Aligned + widened mail-list margins; reading-pane bars fixed height.
- Rich-text composer: color palette, highlight, image; attachments; expand/pop-out.
- Signatures: auto-insert for new mail, replace on From change, reply insert.
- Restore/Delete actions per folder; drafts view merges local + remote, date-sorted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lukasandClaude Opus 5.5