Attachments stored once; nightly clean-up of unused files
Attachments are no longer stored apart from their message (they were kept twice: base64 inside the raw message and decoded on their own). The indexer records the MIME part (attachments.part_index, migration 0023); downloads, zip, inline images, thumbnails and forwarding take it out of the message; videos stream from a short-lived copy (kept a day, 1 GiB per user). A background pass on worker start removes the copies stored before, each only after checking it can be taken out again (spec App. D §D.6 item 1, criterion corrected to <= 60%). A nightly job (03:47, after the Trash/Spam purge) removes files nothing refers to any more — deleted or purged messages, previews of attachments gone, uploads no draft has used for a week, expired streaming copies. Files younger than an hour are left alone; a sent message waiting for its Sent copy is held until its job is done. Account removal keeps files a draft still uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,36 @@
|
||||
"""attachments are kept once: inside their message
|
||||
|
||||
An attachment used to be stored twice — inside the raw message blob, and again
|
||||
as a blob of its own. Now only the raw message is kept; ``part_index`` says
|
||||
which MIME part of it (its position in a walk of the message) holds the
|
||||
attachment, so it can be taken out when downloaded. ``blob_sha256`` stays: the
|
||||
attachment's content hash, which names its thumbnail and checks what is taken
|
||||
out. Rows stored before this have no index; they are found by that hash.
|
||||
|
||||
Existing separate copies are removed by a background pass on the worker
|
||||
(app/indexer/attachments.py ``dedupe_existing``), only after checking each one
|
||||
can be taken out of its message again.
|
||||
|
||||
Revision ID: 0023
|
||||
Revises: 0022
|
||||
Create Date: 2026-10-08
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0023"
|
||||
down_revision: str | None = "0022"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute("ALTER TABLE attachments ADD COLUMN part_index integer")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute("ALTER TABLE attachments DROP COLUMN IF EXISTS part_index")
|
||||
Reference in New Issue
Block a user