lukas/webmail-client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Antispam: lessons refused by the engine no longer pass for learned

Browse Source 
Found on the NAS, where nothing had been learned in weeks while the UI said
"Marked as spam": the source tree predated v1.3.2, so api/worker-* had no
controller password and every lesson got 401 — and the trail of state that
left behind kept it broken after the password was fixed.

- /learn records a lesson only when rspamd accepted it. Recording refused ones
  made the menu claim "Marked as spam" and its dedup blocked every retry.
- Bootstrap: done-markers move to a new key. The old one may have been written
  by the "-0" bug (a total outage read as a clean run of zero), which left an
  account marked done with nothing taught and never retried; every older
  install re-runs the bootstrap once, harmlessly (rspamd skips what it knows).
- "Learn from my Spam folder now" in Settings > Antispam filtering queues the
  bootstrap for the user's accounts at once, instead of the hourly sweep.
- webmail.sh update now recreates rspamd: it reads the controller password only
  when created, so a changed .env reached the app but never the engine.
- doctor's advice for the 401 names both real causes (a pre-v1.3.2 compose
  file, or an rspamd started with an old password) and the fix for each; the
  old "update v1.3.2" advice could not help.
- docs/nas-install.md: how to refresh the source tree on a tarball install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
lukas
2026-10-02 20:25:14 +02:00
co-authored by Claude Opus 5.5
parent 165442d897
commit befa1c2404
7 changed files with 276 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plugins/webmail_plugin_antispam/webmail_plugin_antispam/jobs.py
+16 -5
View File
@@ -48,7 +48,12 @@ PAUSE_SECONDS = 0.05
MIN_SPAM_CORPUS = 10
MAX_HAM_RATIO = 3
_DONE_KEY = "_bootstrapped"
# Accounts whose warm start is done. A new key on purpose: markers under the old
# one ("_bootstrapped") may have been written by the `-0` bug described in
# _teach — an account marked done after a total engine outage, with nothing
# taught, and never retried. Ignoring them costs every older install one repeat
# bootstrap, which is harmless: rspamd skips messages it has already learned.
_DONE_KEY = "bootstrap_done"
async def _teach(user_id: uuid.UUID, ids: list[uuid.UUID], *, spam: bool) -> int | None:
@@ -75,9 +80,14 @@ async def _teach(user_id: uuid.UUID, ids: list[uuid.UUID], *, spam: bool) -> int
async def bootstrap_account(
ctx: dict[str, Any], *, user_id: str, account_id: str
ctx: dict[str, Any], *, user_id: str, account_id: str, force: bool = False
) -> None:
"""Teach Bayes from one account's existing mail. Safe to call more than once."""
"""Teach Bayes from one account's existing mail. Safe to call more than once.
``force`` runs it even for an account already marked done — the "learn from
my Spam folder now" button. The provider-Junk setting still applies: the
user turned that off for a reason.
"""
uid, acc = uuid.UUID(user_id), uuid.UUID(account_id)
settings = await policy.load_settings(uid)
@@ -85,7 +95,7 @@ async def bootstrap_account(
return
done = await _SDK.kv(uid).get(_DONE_KEY, [])
if account_id in done:
if account_id in done and not force:
return
spam_ids = await _SDK.messages.sample(
@@ -133,7 +143,8 @@ async def bootstrap_account(
)
return
await _SDK.kv(uid).set(_DONE_KEY, [*done, account_id])
if account_id not in done:
await _SDK.kv(uid).set(_DONE_KEY, [*done, account_id])
logger.info(
"antispam bootstrap for %s: learned %d spam, %d ham",
account_id,
plugins/webmail_plugin_antispam/webmail_plugin_antispam/router.py
+29 -7
View File
@@ -218,7 +218,10 @@ async def learn(body: LearnIn, user_id: CurrentUserId) -> dict[str, object]:
"""Teach this user's Bayes, then file the message accordingly (§C.5).
The move is what the user actually asked for, so it happens even when the
engine is unreachable — teaching is best-effort, filing is not.
engine is unreachable — teaching is best-effort, filing is not. But only a
lesson the engine accepted is recorded: recording a refused one made the menu
say "Marked as spam" for nothing, and its dedup then blocked every retry.
Left unrecorded, the move below lets the passive hook try again.
"""
spam = body.verdict == "spam"
raw = await _SDK.messages.raw(user_id, body.message_id)
@@ -231,12 +234,13 @@ async def learn(body: LearnIn, user_id: CurrentUserId) -> dict[str, object]:
# the very lesson we just taught. Writing `learned` first makes that hook's
# dedup see it and stand down. An upsert, because a message the provider
# filed as spam has no verdict row of ours to update.
await _remember_learned(
user_id,
body.message_id,
verdict=body.verdict,
action="moved" if spam else "none",
)
if taught:
await _remember_learned(
user_id,
body.message_id,
verdict=body.verdict,
action="moved" if spam else "none",
)
moved = await _SDK.actions.move_to_special(
user_id, body.message_id, "junk" if spam else "inbox"
@@ -244,6 +248,24 @@ async def learn(body: LearnIn, user_id: CurrentUserId) -> dict[str, object]:
return {"status": "ok", "taught": taught, "moved": moved}
@router.post("/bootstrap", dependencies=[Depends(require_csrf)])
async def bootstrap_now(user_id: CurrentUserId) -> dict[str, object]:
""" "Learn from my Spam folder now": queue the warm start for each of this
user's accounts, even ones already done, instead of waiting for the hourly
sweep. The job applies the same rules (setting on, enough spam samples)."""
from webmail_plugin_antispam.jobs import bootstrap_account
accounts = [acc for uid, acc in await _SDK.accounts.live() if uid == user_id]
for account_id in accounts:
await _SDK.enqueue(
bootstrap_account,
user_id=str(user_id),
account_id=str(account_id),
force=True,
)
return {"queued": len(accounts)}
@router.get("/stats", response_model=StatsOut)
async def stats(user_id: CurrentUserId) -> StatsOut:
async with _SDK.session(user_id) as s:
plugins/webmail_plugin_antispam/webmail_plugin_antispam/ui/settings.js
+17
View File
@@ -10,6 +10,7 @@ export default function mount(el, ctx) {
let rules = [];
let accounts = [];
let stats = null;
let bootstrapNote = ""; // outcome of the last "learn from my Spam folder now" click
let disposed = false;
async function load() {
@@ -115,6 +116,8 @@ export default function mount(el, ctx) {
<label class="wm-row"><input type="checkbox" data-f="network_checks"${settings.network_checks ? " checked" : ""}> Allow online reputation checks</label>
<p class="wm-faint">Off by default. When on, the filter asks <strong>Spamhaus</strong> whether the sending server or a linked domain is known for spam, and <strong>DNSWL</strong> whether the sender is known-good. That reveals those addresses and domains to them — and to whichever DNS resolver you use. The message itself is never sent, and no fingerprint of it leaves your server.</p>
<label class="wm-row"><input type="checkbox" data-f="learn_from_provider_junk"${settings.learn_from_provider_junk ? " checked" : ""}> Learn from mail your provider already marked as spam</label>
<div class="wm-row"><button class="wm-btn" data-bootstrap${settings.learn_from_provider_junk ? "" : " disabled"}>Learn from my Spam folder now</button><span class="wm-faint" data-bootstrap-note>${esc(bootstrapNote)}</span></div>
<p class="wm-faint">Otherwise this runs on its own once an hour, for accounts with at least 10 messages in Spam. Running it again is harmless: messages the filter already learned are skipped.</p>
</div>
<div class="wm-card"><strong>Activity</strong>${statsBlock()}</div>`;
@@ -133,6 +136,20 @@ export default function mount(el, ctx) {
patch({ enabled_accounts: ids });
}),
);
const boot = el.querySelector("[data-bootstrap]");
if (boot)
boot.addEventListener("click", async () => {
boot.disabled = true;
try {
const res = await ctx.api.pluginFetch("/bootstrap", { method: "POST" });
bootstrapNote = res.queued
? "Started — it learns in the background over the next few minutes."
: "No account has finished its first import yet.";
} catch {
bootstrapNote = "Could not start it — try again in a moment.";
}
render();
});
el.querySelectorAll("[data-add]").forEach((b) =>
b.addEventListener("click", async () => {
const kind = b.getAttribute("data-add");