Attachments stored once; nightly clean-up of unused files
Attachments are no longer stored apart from their message (they were kept twice: base64 inside the raw message and decoded on their own). The indexer records the MIME part (attachments.part_index, migration 0023); downloads, zip, inline images, thumbnails and forwarding take it out of the message; videos stream from a short-lived copy (kept a day, 1 GiB per user). A background pass on worker start removes the copies stored before, each only after checking it can be taken out again (spec App. D §D.6 item 1, criterion corrected to <= 60%). A nightly job (03:47, after the Trash/Spam purge) removes files nothing refers to any more — deleted or purged messages, previews of attachments gone, uploads no draft has used for a week, expired streaming copies. Files younger than an hour are left alone; a sent message waiting for its Sent copy is held until its job is done. Account removal keeps files a draft still uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -14,6 +14,7 @@ from typing import Any
|
||||
|
||||
from arq import func
|
||||
|
||||
from app.indexer.attachments import dedupe_existing_job
|
||||
from app.outbox.draft_sync import push_draft_to_imap_job, remove_imap_draft_job
|
||||
from app.outbox.jobs import file_sent_copy_job, send_job
|
||||
from app.push.service import push_new_mail_job
|
||||
@@ -62,4 +63,8 @@ INTERACTIVE_FUNCTIONS: list[Any] = [
|
||||
]
|
||||
|
||||
# Bulk work (§6): long timeout, low concurrency, isolated from interactive jobs.
|
||||
BULK_FUNCTIONS = [sync_bootstrap_account]
|
||||
BULK_FUNCTIONS = [
|
||||
sync_bootstrap_account,
|
||||
# Resumable (each batch is saved), so a long first run may span timeouts.
|
||||
func(dedupe_existing_job, timeout=6 * 3600),
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user