Gmail label sync + no-dup backfill, live label/UI updates, account-delete cleanup, Sync-SPAM toggle, sidebar logo
Gmail sync (SPEC: Gmail sync): - Provider fetches X-GM-LABELS when X-GM-EXT-1 is advertised; parses custom labels (system labels dropped, mUTF-7 decoded) onto RemoteMessage. - Engine syncs only system folders + All Mail for Gmail, skips every label folder, and dedupes the All Mail pass by Message-ID — kills the INBOX/All-Mail duplication. - Custom labels ride the message.new event; the labels plugin subscribes and maps them to chips as "<name> [Gmail]" (per-message + thread aggregate + known set), so labels fill in gradually with the sync window. Non-ASCII labels decode right. Labels self-healing (message-driven reconcile): - Per-message keys are the source of truth; t: aggregates and _labels are derived. Reconcile drops labels of deleted messages, regroups survivors by their current thread (fixing JWZ thread merges), rewrites changed t: keys, and rebuilds _labels from live messages — keeping user labels, dropping orphaned Gmail ones. - New SDK reads: sdk.threads.existing / sdk.messages.threads_of. Sidebar refreshes live via a plugin.labels.changed WS event on new labels. Account deletion: prune threads left empty after the cascade (they were user-scoped, so only messages cascaded) + de-list the account from surviving cross-account threads. Sync SPAM: opt-in per-account sync_spam (migration 0015, default off) — the engine skips the Junk folder in backfill and poll unless enabled; wizard checkbox added. Robustness / UX: - WebSocket auto-reconnects with backoff and resyncs on (re)connect, and account add/delete invalidates threads/labels — no more hard refresh for live updates. - Backfill banner + sidebar ring also require done < total, so a completed account can't leave a stuck "X of X". - Scrub lone UTF-16 surrogates from parsed headers/bodies so bad messages no longer crash the backfill on Postgres text/JSONB. - Ultimo signet logo in the Mail/Calendar/Contacts sidebar header (and at the top of the collapsed rail), via the shared plugin-ui.css + MailLayout. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U72FZiF1W1pb8XxyWBTREA
This commit is contained in:
@@ -37,6 +37,17 @@ def test_parse_list_plain_folder_has_no_special_use() -> None:
|
||||
assert parsed.special_use is None
|
||||
|
||||
|
||||
def test_parse_list_skips_noselect_containers() -> None:
|
||||
# \Noselect / \NonExistent hierarchy nodes can't be SELECTed (Gmail's [Gmail]
|
||||
# root, a parent label like "Serial numbers " that only groups children).
|
||||
assert parse_list_line(rb'(\HasChildren \Noselect) "/" "[Gmail]"') is None
|
||||
assert parse_list_line(rb'(\HasChildren \Noselect) "/" "Serial numbers "') is None
|
||||
assert parse_list_line(rb'(\NonExistent \HasChildren) "/" "Parent"') is None
|
||||
# A selectable child of that container is still parsed.
|
||||
child = parse_list_line(rb'(\HasNoChildren) "/" "Serial numbers / licencje"')
|
||||
assert child is not None and child.path == "Serial numbers / licencje"
|
||||
|
||||
|
||||
def test_parse_list_utf7_name() -> None:
|
||||
# Modified UTF-7: "&AOk-" decodes to "é" -> folder "Réponses"
|
||||
parsed = parse_list_line(rb'(\HasNoChildren) "/" "R&AOk-ponses"')
|
||||
@@ -44,6 +55,35 @@ def test_parse_list_utf7_name() -> None:
|
||||
assert parsed.path == "Réponses"
|
||||
|
||||
|
||||
def test_parse_gm_labels() -> None:
|
||||
from app.providers.imap_smtp.parsing import parse_gm_labels
|
||||
|
||||
# System labels (backslash-prefixed) are dropped; custom labels kept + decoded.
|
||||
header = rb'1 FETCH (UID 5 X-GM-LABELS (\Inbox \Important "Konkurs" "Projekty/2026") FLAGS ())'
|
||||
assert parse_gm_labels(header) == ("Konkurs", "Projekty/2026")
|
||||
# Unquoted single-word custom label.
|
||||
assert parse_gm_labels(rb"1 FETCH (UID 5 X-GM-LABELS (\Sent PUP))") == ("PUP",)
|
||||
# Modified UTF-7 decoding of a non-ASCII label ("Zg&AUI-oszenia" -> "Zgłoszenia").
|
||||
assert parse_gm_labels(rb'1 FETCH (X-GM-LABELS ("Zg&AUI-oszenia"))') == ("Zgłoszenia",)
|
||||
# No X-GM-LABELS at all.
|
||||
assert parse_gm_labels(rb"1 FETCH (UID 5 FLAGS (\Seen))") == ()
|
||||
# Only system labels -> empty.
|
||||
assert parse_gm_labels(rb"1 FETCH (X-GM-LABELS (\Inbox \Starred))") == ()
|
||||
|
||||
|
||||
def test_parse_fetch_lines_carries_gm_labels() -> None:
|
||||
lines = [
|
||||
rb'1 FETCH (UID 9 X-GM-LABELS (\Inbox "Work") FLAGS (\Seen) '
|
||||
rb'INTERNALDATE "27-May-2026 19:08:09 +0200" RFC822.SIZE 3 BODY[] {3}',
|
||||
b"abc",
|
||||
b")",
|
||||
]
|
||||
out = parse_fetch_lines(lines)
|
||||
assert len(out) == 1
|
||||
assert out[0].gmail_labels == ("Work",)
|
||||
assert out[0].raw == b"abc"
|
||||
|
||||
|
||||
def test_parse_internaldate() -> None:
|
||||
dt = parse_internaldate("27-May-2026 19:08:09 +0200")
|
||||
assert dt == datetime.fromisoformat("2026-05-27T19:08:09+02:00")
|
||||
|
||||
Reference in New Issue
Block a user