Commit Graph
4 Commits
Author SHA1 Message Date
lukasandClaude Opus 5 059fa0ade3 Antispam: the app could never authenticate to the learning engine
Rspamd answers on two ports and only one of them asks for a password:
the scan worker takes /checkv2 anonymously, the controller demands
`Password:` for /stat and /learnspam. `webmail.sh init` generates
RSPAMD_CONTROLLER_PASSWORD and docker-compose hands it to the rspamd
container — but the shared x-backend-env block that feeds api and
worker-* never carried it. The plugin reads it from the environment,
found "", and sent no header.

So classification worked and learning never did. Every learn returned
401 and the plugin swallowed it, as it is built to: an engine problem
must not lose mail. Bootstrap logged "engine unavailable", passive
learning from Inbox->Spam moves taught nothing, and the single visible
trace was `"bayes": null` on GET /api/plugins/antispam/stats, on an
install whose verdict counts were ticking up normally.

Verified against a live engine: with no password /stat and /learnspam
both 401 and rspamd.learn() returns False; with it, stats returns and
learn returns True.

  - pass RSPAMD_CONTROLLER_PASSWORD to the app containers, in both the
    production and dev compose files
  - doctor now makes the real call from inside the api container, with
    the password that container actually holds, and says plainly that
    nothing will be learned if it fails
  - a test reads both compose files, because this is a missing line that
    review already let through once

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-21 22:17:29 +02:00
lukasandClaude Opus 5 8eb699a05f Antispam: bootstrap every account, not just the ones we happen to see go live
Warming Bayes hung on a single event — `account.state_changed` reaching "live".
That covers an account added from now on and nothing else. An account that went
live before this plugin existed never gets one; neither does one whose Junk
folder was empty at the time and filled up later, or one whose owner enabled
spam-folder sync afterwards. A worker restarting at the wrong moment loses it
too. The result was a feature that quietly required someone to run it by hand
per account, which is not a feature.

An hourly sweep now walks every live account and offers each a bootstrap. That
works because the per-account job was already cheap to repeat: it returns at once
for an account already done, and declines — without marking it done — when the
corpus is still too small to learn from. So the sweep simply keeps asking until
there is something worth learning, then does it once.

`s.accounts.live()` is the new SDK surface it needs: cross-user by design,
because a sweep runs for the instance rather than a request. Ids and state only,
so reading anything still goes through the user-scoped accessors.

One failing account cannot stop the sweep — the mailbox that raises is logged and
the rest still get their turn.

docs/plugins.md now says this in general terms next to `JobSpec`: a state-change
hook is one-shot, so pair it with a sweep when the work must eventually happen
for everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 21:04:38 +02:00
lukasandClaude Opus 5 5aa133e18c Antispam: plugin docs, the learning acceptance test, and a bootstrap that refuses bad corpora
docs/plugins.md described an SDK that no longer exists. It now covers
messages.raw/sample, actions.move_to_special, the envelope's account_id /
mailbox_special_use / account_state, the message.changed transition payload, the
message.actions.menu slot, Slot.title and .wm-menu-item — with the reasons, not
just the signatures. M3's acceptance is that a plugin can be written from this
file alone, and it had drifted.

Acceptance 4 (§C.8): a test that teaches 20 spam + 20 ham to a real engine and
checks the classifier generalises to held-out mail, keeps ham under the banner
threshold, and does not leak between users. It skips when no engine is reachable.
It does NOT assert the literal move_threshold — a synthetic fixture cannot reach
12.0 once the SMTP-time symbols are zeroed, which is the whole point of the
tuning. Recorded as OQ-8 rather than quietly asserting something weaker.

Bayes ignores training until min_learns, stock 200. For a personal mailbox that
means every correction appears to be ignored for months, so it is now 20.

The bootstrap would have made this filter worse. Accounts do not sync their
provider's spam folder by default, so the real corpus here is 3 junk beside 1,294
aged inbox — teaching both as found trains "everything is ham". It now refuses
below 10 spam samples (leaving the account unmarked so a later run can do it
properly) and otherwise caps ham at 3x the spam count.

Run against the live account, it correctly declined:
  "only 3 spam samples (need 10); enable spam-folder sync to build a corpus"

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 20:34:50 +02:00
lukasandClaude Opus 5 149c11581e Antispam C2: learn from what the user does
Explicit teaching shipped with C1 (the Spam / Not spam buttons). This adds the
two halves that need no buttons at all.

Passive learning (§C.5). Moving mail into Spam says "this is spam" more honestly
than any button, because the user is doing it for their own reasons rather than
to train a filter; moving it back out says the opposite. Only those two
transitions teach — Spam to Trash is someone tidying up, Inbox to Archive says
nothing about spam. Teaching the same message the same thing twice is skipped,
since users move things back and forth, but reversing a lesson is allowed. If
the engine is down nothing is recorded, so a later move can teach it again.

This needed the transition, not just the destination: `message.changed` carried
only where a message landed, so `move_job` now also publishes
previous_mailbox_id and both special_use values.

Provider bootstrap (§C.5). An account going live enqueues a job that teaches
Bayes from mail the user already owns — up to 500 messages from the provider's
Junk folder as spam, and up to 500 Inbox messages older than 30 days as ham.
This is the constructive use of the history that classification deliberately
refuses to act on: the same backfill, read instead of judged. It sleeps between
messages so live classification never queues behind training, stops rather than
grinding if the engine disappears, and marks the account done in KV so it runs
once.

That needed `PluginMessages.sample`: the SDK could fetch a message by id but not
enumerate a folder. Ids only — bodies are pulled one at a time, so a 500-message
sample never materialises 500 bodies.

Verified against the running engine: teaching moved its `learned` counter 0 -> 2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
2026-09-17 08:36:51 +02:00