- The "online reputation checks" rspamd profile used groups_enabled,
which means ONLY those groups: with the switch on, a message got the
RBL lookups alone (1 rule instead of 19, score 0.00) and Bayes never
ran, so every spam lesson was ignored. It now disables nothing.
- A reply or reply-all teaches the answered message as ham (not
forwards or new mail, not one's own Sent/Drafts, never over an
explicit spam mark; once per message). message.sent now carries the
draft's kind and reply_to_message_id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosted mailboxes (SPEC App. D), behind HOSTED_MAILBOXES (off: unchanged):
- Stalwart mail server, provisioned by `python -m app.hosted.provision`;
directory read from a Postgres view, app passwords only for mail apps
- native account per user, the worker signs in as master for the user
- raw messages of hosted mailboxes only cached, fetched back on demand
- migrations 0025 (accounts.kind, directory view), 0026 (IMAP ids bigint)
Mail:
- dark message mode switch in the reading pane (Thunderbird's algorithm)
- moves and folder creation quote the folder name ("Deleted Items"):
swipe-to-delete / Trash no longer bounce back
- thread unread counters: lock before recount, sorted lock order,
UIDVALIDITY re-import recounts, nightly repair_threads
- sidebar accounts sorted alphabetically, more space above headings
Settings: Security headings renamed; antispam accounts are explicit
(none ticked = none filtered), input widths, Activity hidden.
Plugin bundles served with Cache-Control: no-cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found on the NAS, where nothing had been learned in weeks while the UI said
"Marked as spam": the source tree predated v1.3.2, so api/worker-* had no
controller password and every lesson got 401 — and the trail of state that
left behind kept it broken after the password was fixed.
- /learn records a lesson only when rspamd accepted it. Recording refused ones
made the menu claim "Marked as spam" and its dedup blocked every retry.
- Bootstrap: done-markers move to a new key. The old one may have been written
by the "-0" bug (a total outage read as a clean run of zero), which left an
account marked done with nothing taught and never retried; every older
install re-runs the bootstrap once, harmlessly (rspamd skips what it knows).
- "Learn from my Spam folder now" in Settings > Antispam filtering queues the
bootstrap for the user's accounts at once, instead of the hourly sweep.
- webmail.sh update now recreates rspamd: it reads the controller password only
when created, so a changed .env reached the app but never the engine.
- doctor's advice for the 401 names both real causes (a pre-v1.3.2 compose
file, or an rspamd started with an old password) and the fix for each; the
old "update v1.3.2" advice could not help.
- docs/nas-install.md: how to refresh the source tree on a tarball install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`_teach` signalled failure by negating the count taught so far. That
works for every case but the one that matters: when the FIRST message is
refused nothing has been taught yet, and `-0` is `0` — not negative. The
caller's `< 0` test then read a total engine outage as a clean run of
zero, recorded `_bootstrapped` for the account, and the hourly sweep
skipped it from then on. The account most in need of a warm start was
the one guaranteed never to get one.
Which is precisely what happened alongside the missing controller
password: every learn was refused 401, and each account was quietly
retired on its first sweep.
Return None for "the engine stopped accepting these" so failure cannot
be confused with a count, and decline to mark an account whose samples
were all unreadable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
Rspamd answers on two ports and only one of them asks for a password:
the scan worker takes /checkv2 anonymously, the controller demands
`Password:` for /stat and /learnspam. `webmail.sh init` generates
RSPAMD_CONTROLLER_PASSWORD and docker-compose hands it to the rspamd
container — but the shared x-backend-env block that feeds api and
worker-* never carried it. The plugin reads it from the environment,
found "", and sent no header.
So classification worked and learning never did. Every learn returned
401 and the plugin swallowed it, as it is built to: an engine problem
must not lose mail. Bootstrap logged "engine unavailable", passive
learning from Inbox->Spam moves taught nothing, and the single visible
trace was `"bayes": null` on GET /api/plugins/antispam/stats, on an
install whose verdict counts were ticking up normally.
Verified against a live engine: with no password /stat and /learnspam
both 401 and rspamd.learn() returns False; with it, stats returns and
learn returns True.
- pass RSPAMD_CONTROLLER_PASSWORD to the app containers, in both the
production and dev compose files
- doctor now makes the real call from inside the api container, with
the password that container actually holds, and says plainly that
nothing will be learned if it fails
- a test reads both compose files, because this is a missing line that
review already let through once
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
Warming Bayes hung on a single event — `account.state_changed` reaching "live".
That covers an account added from now on and nothing else. An account that went
live before this plugin existed never gets one; neither does one whose Junk
folder was empty at the time and filled up later, or one whose owner enabled
spam-folder sync afterwards. A worker restarting at the wrong moment loses it
too. The result was a feature that quietly required someone to run it by hand
per account, which is not a feature.
An hourly sweep now walks every live account and offers each a bootstrap. That
works because the per-account job was already cheap to repeat: it returns at once
for an account already done, and declines — without marking it done — when the
corpus is still too small to learn from. So the sweep simply keeps asking until
there is something worth learning, then does it once.
`s.accounts.live()` is the new SDK surface it needs: cross-user by design,
because a sweep runs for the instance rather than a request. Ids and state only,
so reading anything still goes through the user-scoped accessors.
One failing account cannot stop the sweep — the mailbox that raises is logged and
the rest still get their turn.
docs/plugins.md now says this in general terms next to `JobSpec`: a state-change
hook is one-shot, so pair it with a sweep when the work must eventually happen
for everything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
docs/plugins.md described an SDK that no longer exists. It now covers
messages.raw/sample, actions.move_to_special, the envelope's account_id /
mailbox_special_use / account_state, the message.changed transition payload, the
message.actions.menu slot, Slot.title and .wm-menu-item — with the reasons, not
just the signatures. M3's acceptance is that a plugin can be written from this
file alone, and it had drifted.
Acceptance 4 (§C.8): a test that teaches 20 spam + 20 ham to a real engine and
checks the classifier generalises to held-out mail, keeps ham under the banner
threshold, and does not leak between users. It skips when no engine is reachable.
It does NOT assert the literal move_threshold — a synthetic fixture cannot reach
12.0 once the SMTP-time symbols are zeroed, which is the whole point of the
tuning. Recorded as OQ-8 rather than quietly asserting something weaker.
Bayes ignores training until min_learns, stock 200. For a personal mailbox that
means every correction appears to be ignored for months, so it is now 20.
The bootstrap would have made this filter worse. Accounts do not sync their
provider's spam folder by default, so the real corpus here is 3 junk beside 1,294
aged inbox — teaching both as found trains "everything is ham". It now refuses
below 10 spam samples (leaving the account unmarked so a later run can do it
properly) and otherwise caps ham at 3x the spam count.
Run against the live account, it correctly declined:
"only 3 spam samples (need 10); enable spam-folder sync to build a corpus"
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
Explicit teaching shipped with C1 (the Spam / Not spam buttons). This adds the
two halves that need no buttons at all.
Passive learning (§C.5). Moving mail into Spam says "this is spam" more honestly
than any button, because the user is doing it for their own reasons rather than
to train a filter; moving it back out says the opposite. Only those two
transitions teach — Spam to Trash is someone tidying up, Inbox to Archive says
nothing about spam. Teaching the same message the same thing twice is skipped,
since users move things back and forth, but reversing a lesson is allowed. If
the engine is down nothing is recorded, so a later move can teach it again.
This needed the transition, not just the destination: `message.changed` carried
only where a message landed, so `move_job` now also publishes
previous_mailbox_id and both special_use values.
Provider bootstrap (§C.5). An account going live enqueues a job that teaches
Bayes from mail the user already owns — up to 500 messages from the provider's
Junk folder as spam, and up to 500 Inbox messages older than 30 days as ham.
This is the constructive use of the history that classification deliberately
refuses to act on: the same backfill, read instead of judged. It sleeps between
messages so live classification never queues behind training, stops rather than
grinding if the engine disappears, and marks the account done in KV so it runs
once.
That needed `PluginMessages.sample`: the SDK could fetch a message by id but not
enumerate a folder. Ids only — bodies are pulled one at a time, so a 500-message
sample never materialises 500 bodies.
Verified against the running engine: teaching moved its `learned` counter 0 -> 2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SqJYt7jD7oUEkoY68sBZo
lukasandClaude Opus 5.5