Antispam: bootstrap every account, not just the ones we happen to see go live

Warming Bayes hung on a single event — `account.state_changed` reaching "live".
That covers an account added from now on and nothing else. An account that went
live before this plugin existed never gets one; neither does one whose Junk
folder was empty at the time and filled up later, or one whose owner enabled
spam-folder sync afterwards. A worker restarting at the wrong moment loses it
too. The result was a feature that quietly required someone to run it by hand
per account, which is not a feature.

An hourly sweep now walks every live account and offers each a bootstrap. That
works because the per-account job was already cheap to repeat: it returns at once
for an account already done, and declines — without marking it done — when the
corpus is still too small to learn from. So the sweep simply keeps asking until
there is something worth learning, then does it once.

`s.accounts.live()` is the new SDK surface it needs: cross-user by design,
because a sweep runs for the instance rather than a request. Ids and state only,
so reading anything still goes through the user-scoped accessors.

One failing account cannot stop the sweep — the mailbox that raises is logged and
the rest still get their turn.

docs/plugins.md now says this in general terms next to `JobSpec`: a state-change
hook is one-shot, so pair it with a sweep when the work must eventually happen
for everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
This commit is contained in:
2026-09-17 21:04:38 +02:00
co-authored by Claude Opus 5
parent 566f02a3f4
commit 8eb699a05f
6 changed files with 127 additions and 5 deletions
@@ -25,7 +25,7 @@ from webmail_plugin_antispam.hook import (
on_message_changed,
on_message_new,
)
from webmail_plugin_antispam.jobs import bootstrap_account
from webmail_plugin_antispam.jobs import bootstrap_account, bootstrap_sweep
from webmail_plugin_antispam.router import router
_HERE = Path(__file__).parent
@@ -41,9 +41,16 @@ plugin = Plugin(
# An account going live is the cue to warm Bayes from its history (§C.5).
"account.state_changed": on_account_state_changed,
},
# Enqueue-on-demand (no schedule); singleton so a flapping account cannot
# start two training passes over the same mailbox.
jobs=[JobSpec(func=bootstrap_account, schedule=None, singleton=True)],
jobs=[
# Enqueue-on-demand; singleton so a flapping account cannot start two
# training passes over the same mailbox.
JobSpec(func=bootstrap_account, schedule=None, singleton=True),
# And hourly, because the state-change hook is a one-shot event: it
# cannot help an account that went live before this plugin existed, or
# one whose Junk folder only became worth reading later. Offset from the
# hour so it does not pile onto every other scheduled job.
JobSpec(func=bootstrap_sweep, schedule="17 * * * *", singleton=True),
],
router=router,
migrations=str(_HERE / "migrations" / "versions"),
ui_manifest=UiManifest(
@@ -134,3 +134,33 @@ async def bootstrap_account(
"ham": ham_taught,
},
)
async def bootstrap_sweep(ctx: dict[str, Any]) -> None:
"""Find accounts that still need a warm start, and give them one.
The `account.state_changed` hook catches an account the moment it finishes
importing, which covers accounts added from now on. This covers everything
else, and it is the reason bootstrapping is not a thing anyone has to
remember to do:
* accounts that existed before this plugin did — their transition to `live`
happened long ago and will not happen again;
* accounts whose Junk folder was empty at the time and has since filled, or
which only started syncing the provider's spam folder later;
* a transition missed because the worker was restarting.
Cheap to repeat: `bootstrap_account` returns immediately for an account
already marked done, and declines (without marking) one whose corpus is still
too small to learn from — so this simply retries until there is something
worth learning.
"""
for user_id, account_id in await _SDK.accounts.live():
try:
await bootstrap_account(
ctx, user_id=str(user_id), account_id=str(account_id)
)
except Exception as exc: # noqa: BLE001 - one bad account must not stop the sweep
logger.warning(
"antispam bootstrap sweep failed for %s: %s", account_id, exc
)