Antispam: bootstrap every account, not just the ones we happen to see go live
Warming Bayes hung on a single event — `account.state_changed` reaching "live". That covers an account added from now on and nothing else. An account that went live before this plugin existed never gets one; neither does one whose Junk folder was empty at the time and filled up later, or one whose owner enabled spam-folder sync afterwards. A worker restarting at the wrong moment loses it too. The result was a feature that quietly required someone to run it by hand per account, which is not a feature. An hourly sweep now walks every live account and offers each a bootstrap. That works because the per-account job was already cheap to repeat: it returns at once for an account already done, and declines — without marking it done — when the corpus is still too small to learn from. So the sweep simply keeps asking until there is something worth learning, then does it once. `s.accounts.live()` is the new SDK surface it needs: cross-user by design, because a sweep runs for the instance rather than a request. Ids and state only, so reading anything still goes through the user-scoped accessors. One failing account cannot stop the sweep — the mailbox that raises is logged and the rest still get their turn. docs/plugins.md now says this in general terms next to `JobSpec`: a state-change hook is one-shot, so pair it with a sweep when the work must eventually happen for everything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
This commit is contained in:
@@ -25,7 +25,7 @@ from webmail_plugin_antispam.hook import (
|
||||
on_message_changed,
|
||||
on_message_new,
|
||||
)
|
||||
from webmail_plugin_antispam.jobs import bootstrap_account
|
||||
from webmail_plugin_antispam.jobs import bootstrap_account, bootstrap_sweep
|
||||
from webmail_plugin_antispam.router import router
|
||||
|
||||
_HERE = Path(__file__).parent
|
||||
@@ -41,9 +41,16 @@ plugin = Plugin(
|
||||
# An account going live is the cue to warm Bayes from its history (§C.5).
|
||||
"account.state_changed": on_account_state_changed,
|
||||
},
|
||||
# Enqueue-on-demand (no schedule); singleton so a flapping account cannot
|
||||
# start two training passes over the same mailbox.
|
||||
jobs=[JobSpec(func=bootstrap_account, schedule=None, singleton=True)],
|
||||
jobs=[
|
||||
# Enqueue-on-demand; singleton so a flapping account cannot start two
|
||||
# training passes over the same mailbox.
|
||||
JobSpec(func=bootstrap_account, schedule=None, singleton=True),
|
||||
# And hourly, because the state-change hook is a one-shot event: it
|
||||
# cannot help an account that went live before this plugin existed, or
|
||||
# one whose Junk folder only became worth reading later. Offset from the
|
||||
# hour so it does not pile onto every other scheduled job.
|
||||
JobSpec(func=bootstrap_sweep, schedule="17 * * * *", singleton=True),
|
||||
],
|
||||
router=router,
|
||||
migrations=str(_HERE / "migrations" / "versions"),
|
||||
ui_manifest=UiManifest(
|
||||
|
||||
@@ -134,3 +134,33 @@ async def bootstrap_account(
|
||||
"ham": ham_taught,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
async def bootstrap_sweep(ctx: dict[str, Any]) -> None:
|
||||
"""Find accounts that still need a warm start, and give them one.
|
||||
|
||||
The `account.state_changed` hook catches an account the moment it finishes
|
||||
importing, which covers accounts added from now on. This covers everything
|
||||
else, and it is the reason bootstrapping is not a thing anyone has to
|
||||
remember to do:
|
||||
|
||||
* accounts that existed before this plugin did — their transition to `live`
|
||||
happened long ago and will not happen again;
|
||||
* accounts whose Junk folder was empty at the time and has since filled, or
|
||||
which only started syncing the provider's spam folder later;
|
||||
* a transition missed because the worker was restarting.
|
||||
|
||||
Cheap to repeat: `bootstrap_account` returns immediately for an account
|
||||
already marked done, and declines (without marking) one whose corpus is still
|
||||
too small to learn from — so this simply retries until there is something
|
||||
worth learning.
|
||||
"""
|
||||
for user_id, account_id in await _SDK.accounts.live():
|
||||
try:
|
||||
await bootstrap_account(
|
||||
ctx, user_id=str(user_id), account_id=str(account_id)
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 - one bad account must not stop the sweep
|
||||
logger.warning(
|
||||
"antispam bootstrap sweep failed for %s: %s", account_id, exc
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user