mail UX: restore-to-folder, draft/attachment fixes, printing, sidebar collapse
Backend: - Restore returns each message to its original folder (e.g. Sent/Archive) instead of always the Inbox: new messages.restore_mailbox_id (migration 0010) + restore_thread_job. - Drafts: expunge reconciliation for the Drafts poll (provider.list_uids / _reconcile_drafts) clears phantom "remote" drafts; send + discard drop the mirrored DB row; adopting a remote draft carries its real attachments into the compose flow. - Thread detail/list are folder-aware: individually trashed/spam messages are hidden outside their folder, Trash/Spam show only their own messages, and the mail-list message count matches the view. Frontend: - Thread reading panel: fix newest-first card overlap/spacing (expand-all and single expand), hide Reply/Forward on trashed messages, close the Actions menu on message-body (iframe) clicks, scroll to the inline composer, and prompt Save draft / Discard on the X close. - Composer: remount when switching drafts, dirty-tracking to stop duplicate IMAP mirrors, and close the From/More dropdowns on outside click. - Sidebar: collapse/expand toggle (icons-only ~60px rail) and a log-out icon on Sign out. - Mail-standard printing in a new window with reliable page margins (lib/print.ts). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
This commit is contained in:
@@ -0,0 +1,103 @@
|
||||
How the worker functions (multi-user / multi-account)
|
||||
=====================================================
|
||||
|
||||
The worker wears two hats (in one process, one event loop)
|
||||
----------------------------------------------------------
|
||||
It is a single arq process (`arq workers.worker.WorkerSettings`) running one
|
||||
asyncio event loop. Inside it, two very different things run:
|
||||
|
||||
A) A job runner (the Redis queue side). The API enqueues discrete jobs; the
|
||||
worker pulls and runs them, up to `max_jobs = 10` concurrently,
|
||||
`job_timeout = 3600s`. The registered jobs:
|
||||
sync_bootstrap_account (backfill a new account), push_flags_job, move_job,
|
||||
move_thread_to_folder_job, delete_thread_job, send_job,
|
||||
push_draft_to_imap_job, remove_imap_draft_job, refresh_account_job.
|
||||
Each job builds a provider (which is where the mailbox credential is
|
||||
decrypted with the MASTER_KEY - worker-only), does the IMAP/SMTP op, and
|
||||
closes the connection. On failure they retry with backoff (~x5) and then
|
||||
revert or mark failed.
|
||||
|
||||
B) A live-sync supervisor (long-running loops). Started once in on_startup and
|
||||
is NOT part of the job queue.
|
||||
|
||||
|
||||
The per-account model (the key point)
|
||||
-------------------------------------
|
||||
On startup, start_supervisor reads every account with state = "live" and, for
|
||||
each account, spawns two persistent asyncio tasks:
|
||||
|
||||
1. INBOX IDLE loop (live_loop) - opens one IMAP connection and keeps it open,
|
||||
blocking on IMAP IDLE (server push). It reacts within seconds when mail
|
||||
arrives, and re-issues IDLE at least every 25 min. This is a *persistent*
|
||||
connection held 24/7.
|
||||
|
||||
2. Secondary poll loop (secondary_poll_loop) - every POLL_INTERVAL (120 s) it
|
||||
opens a fresh, transient connection, CONDSTORE-polls
|
||||
Sent/Drafts/Archive/Trash/Junk (cheap MODSEQ deltas), and closes it.
|
||||
|
||||
So the steady-state cost scales linearly with the total number of email
|
||||
accounts across all users:
|
||||
|
||||
- Persistent IMAP connections : ~1 per account (the INBOX IDLE) held
|
||||
continuously
|
||||
- asyncio tasks : 2 per account
|
||||
- Transient connections : one burst per account every 120 s (secondary
|
||||
poll) + per job
|
||||
|
||||
For 3 users x 3 accounts that is ~9 always-open IDLE connections and 18 tasks -
|
||||
trivial. For 500 accounts it is ~500 sockets open 24/7 plus 1000 tasks, and you
|
||||
will hit IMAP server per-user/global connection limits, file-descriptor limits,
|
||||
and memory pressure long before anything else.
|
||||
|
||||
|
||||
How updates reach the browsers
|
||||
------------------------------
|
||||
The worker never talks to browsers. When it persists a change it publishes to
|
||||
Redis pub/sub (evt:{event}); the API's WebSocket hub relays each event to the
|
||||
owning user's connected clients, which invalidate the relevant queries. So Redis
|
||||
is both the job queue and the fan-out bus.
|
||||
|
||||
|
||||
Honest scaling limitations
|
||||
--------------------------
|
||||
- Single process / single core. All those loops and jobs share one event loop.
|
||||
It is fine because IMAP work is I/O-bound (mostly waiting), but a heavy
|
||||
backfill or MIME parse briefly blocks everything else, and each raw message is
|
||||
loaded fully into memory to parse + blob-store - so messages x concurrency is
|
||||
your memory driver.
|
||||
|
||||
- The live-sync side does not horizontally scale. If you ran two workers for
|
||||
throughput, both would start IDLE loops for all live accounts -> duplicate
|
||||
connections and double-processing. The job side would shard fine across
|
||||
workers (that is what arq does), but the supervisor assumes exactly one
|
||||
instance.
|
||||
|
||||
- Account discovery is startup-only. New or re-configured accounts are not
|
||||
picked up as live until the worker restarts (the code says so explicitly). A
|
||||
new account does get its one-time backfill immediately (via the enqueued
|
||||
sync_bootstrap_account job), but its ongoing IDLE loop will not start until a
|
||||
restart. The same gap causes the "zombie loop" after account deletion.
|
||||
|
||||
- max_jobs = 10 is global, shared across all users - a burst of sends/moves
|
||||
queues behind that cap.
|
||||
|
||||
|
||||
What a real multi-tenant deployment would add
|
||||
---------------------------------------------
|
||||
- A pool of sync workers with account sharding: each account owned by exactly
|
||||
one worker (e.g., a Redis lease/consistent hashing), so IDLE loops do not
|
||||
duplicate and you can scale out.
|
||||
|
||||
- React to account.state_changed to start/stop loops at runtime (no restart on
|
||||
add/edit/delete).
|
||||
|
||||
- Connection budgeting - e.g., keep IDLE only for recently-active mailboxes and
|
||||
poll the long tail - plus per-server connection caps and backpressure.
|
||||
|
||||
- Move sync_bootstrap_account (a large backfill) onto a separate, higher-timeout
|
||||
queue from the fast interactive jobs (send/flags/move).
|
||||
|
||||
Net: the current design is a clean, correct single-node model - great for a
|
||||
handful of users with a few accounts each - but the per-account persistent IDLE
|
||||
loop and the startup-only, single-supervisor assumptions are the things you
|
||||
would re-architect before going multi-tenant at scale.
|
||||
Reference in New Issue
Block a user