Antispam C2: learn from what the user does
Explicit teaching shipped with C1 (the Spam / Not spam buttons). This adds the two halves that need no buttons at all. Passive learning (§C.5). Moving mail into Spam says "this is spam" more honestly than any button, because the user is doing it for their own reasons rather than to train a filter; moving it back out says the opposite. Only those two transitions teach — Spam to Trash is someone tidying up, Inbox to Archive says nothing about spam. Teaching the same message the same thing twice is skipped, since users move things back and forth, but reversing a lesson is allowed. If the engine is down nothing is recorded, so a later move can teach it again. This needed the transition, not just the destination: `message.changed` carried only where a message landed, so `move_job` now also publishes previous_mailbox_id and both special_use values. Provider bootstrap (§C.5). An account going live enqueues a job that teaches Bayes from mail the user already owns — up to 500 messages from the provider's Junk folder as spam, and up to 500 Inbox messages older than 30 days as ham. This is the constructive use of the history that classification deliberately refuses to act on: the same backfill, read instead of judged. It sleeps between messages so live classification never queues behind training, stops rather than grinding if the engine disappears, and marks the account done in KV so it runs once. That needed `PluginMessages.sample`: the SDK could fetch a message by id but not enumerate a folder. Ids only — bodies are pulled one at a time, so a 500-message sample never materialises 500 bodies. Verified against the running engine: teaching moved its `learned` counter 0 -> 2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
This commit is contained in:
@@ -13,6 +13,7 @@ spam message into Junk without dragging its thread along.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
import uuid
|
||||
from collections.abc import Iterable
|
||||
from email import message_from_bytes
|
||||
@@ -65,6 +66,42 @@ class PluginMessages:
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
async def sample(
|
||||
self,
|
||||
user_id: uuid.UUID,
|
||||
*,
|
||||
account_id: uuid.UUID,
|
||||
special_use: str,
|
||||
limit: int = 500,
|
||||
older_than_days: int | None = None,
|
||||
) -> list[uuid.UUID]:
|
||||
"""Message ids from one account folder, newest first, capped at ``limit``.
|
||||
|
||||
For building a training corpus out of mail the user already has — the
|
||||
provider's own Junk folder is a labelled spam set, and aged Inbox mail
|
||||
that was never moved is a reasonable ham set. ``older_than_days`` skips
|
||||
recent arrivals, which are the ones a user has not triaged yet.
|
||||
|
||||
Ids only: the caller fetches bodies through :meth:`raw` one at a time,
|
||||
so a 500-message sample never materialises 500 message bodies at once.
|
||||
"""
|
||||
stmt = (
|
||||
select(Message.id)
|
||||
.join(Mailbox, Mailbox.id == Message.mailbox_id)
|
||||
.where(
|
||||
Message.user_id == user_id,
|
||||
Message.account_id == account_id,
|
||||
Mailbox.special_use == special_use,
|
||||
)
|
||||
.order_by(Message.internal_date.desc())
|
||||
.limit(max(0, min(limit, 5000)))
|
||||
)
|
||||
if older_than_days is not None:
|
||||
cutoff = dt.datetime.now(dt.UTC) - dt.timedelta(days=older_than_days)
|
||||
stmt = stmt.where(Message.internal_date < cutoff)
|
||||
async with session_scope() as session:
|
||||
return list(await session.scalars(stmt))
|
||||
|
||||
async def threads_of(
|
||||
self, user_id: uuid.UUID, message_ids: Iterable[str]
|
||||
) -> dict[str, str | None]:
|
||||
|
||||
@@ -126,6 +126,10 @@ async def move_job(
|
||||
return
|
||||
src_path = src.path
|
||||
uid, dst_path, user_id = message.uid, dst.path, message.user_id
|
||||
# Carried into the event below: a consumer usually cares about the
|
||||
# transition, not the destination. "Now in Spam" and "moved out of Spam"
|
||||
# are different facts, and only the pair distinguishes them.
|
||||
src_special, dst_special = src.special_use, dst.special_use
|
||||
|
||||
provider = build_provider(account)
|
||||
try:
|
||||
@@ -145,7 +149,14 @@ async def move_job(
|
||||
message.mailbox_id = dst_id
|
||||
await get_event_bus().publish(
|
||||
"message.changed",
|
||||
{"user_id": str(user_id), "message_id": str(mid), "mailbox_id": str(dst_id)},
|
||||
{
|
||||
"user_id": str(user_id),
|
||||
"message_id": str(mid),
|
||||
"mailbox_id": str(dst_id),
|
||||
"previous_mailbox_id": src_mailbox_id,
|
||||
"special_use": dst_special,
|
||||
"previous_special_use": src_special,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user