Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -2,6 +2,12 @@
|
||||
input/
|
||||
output/
|
||||
|
||||
# Number normalizer: its own venv, and 1.1 GB of gated model weights that must
|
||||
# be fetched with the user's own HF token. Recreate with normalizer/setup.sh.
|
||||
normalizer/venv/
|
||||
normalizer/model/
|
||||
normalizer/models/
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
@@ -9,6 +15,9 @@ __pycache__/
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# Server credentials — never commit these
|
||||
gitea-server-credentials.md
|
||||
|
||||
# Machine-local Claude Code permissions (intentionally not shared)
|
||||
.claude/settings.local.json
|
||||
|
||||
|
||||
@@ -0,0 +1,253 @@
|
||||
# Build brief — Polish number normalizer (offline `transformers` component)
|
||||
|
||||
## Scope of this document
|
||||
|
||||
This brief covers **only the normalizer component**: installing it, running it
|
||||
offline with plain `transformers`, and what it does. **Integration is out of
|
||||
scope** — the calling app (`lektor_ai`) handles SRT parsing, decides which
|
||||
subtitle blocks to normalize, and invokes this component. The user will specify
|
||||
that wiring separately.
|
||||
|
||||
Assume the caller sends **one sentence / subtitle block at a time**, and only
|
||||
sends blocks that already contain at least one digit. This component's job is to
|
||||
return that text with numbers converted to spoken Polish words, per the rules
|
||||
below.
|
||||
|
||||
## Purpose
|
||||
|
||||
The downstream TTS (Higgs Audio v3) mispronounces raw digits — especially long
|
||||
numbers — sometimes emitting garbled non-words. This component removes that
|
||||
failure by converting digits to fully spelled-out Polish words **before** the
|
||||
text reaches the TTS, with correct grammatical case (the reason a neural
|
||||
normalizer is used rather than a lookup table: Polish numerals decline by case,
|
||||
gender, and the noun they govern).
|
||||
|
||||
Behaviour rules:
|
||||
|
||||
1. **Skip single-digit numbers.** A standalone one-digit integer (e.g. `3` in
|
||||
`Mam 3 koty`) is read correctly by the TTS and must be **left as a digit**.
|
||||
2. **Expand two-or-more-digit numbers** to words (`25`, `2020`, `1 350 000`).
|
||||
3. Complex numeric expressions — decimals (`3,5`), times (`14:30`), dates
|
||||
(`3 maja`), ratios/addresses (`12/45`), currency, units — are handled by the
|
||||
model. They are not "single-digit numbers" and are not skipped.
|
||||
4. A block that (after removing single digits) contains no multi-digit number is
|
||||
returned unchanged.
|
||||
|
||||
## The model
|
||||
|
||||
- **Repo:** `Folx/qwen3-0.6b-pl-text-normalization` (Hugging Face). ~1.1 GB,
|
||||
bfloat16, 600M params, `Qwen3ForCausalLM`. Fine-tuned specifically for Polish
|
||||
TTS text normalization (numbers, dates, times, units, abbreviations).
|
||||
- **GATED REPO.** The user must accept the terms on the model page and provide an
|
||||
`HF_TOKEN` for the **one-time** download. The build agent cannot accept the
|
||||
licence. Load the model from a **local path / local cache** at run time — never
|
||||
re-download.
|
||||
- **CRITICAL version requirement:** `transformers >= 5.0.0` and `torch >= 2.1.0`.
|
||||
Older transformers (e.g. 4.57) tokenize the Qwen3 chat template differently and
|
||||
silently produce wrong output even under greedy decode. Pin `transformers>=5.0`.
|
||||
- **System prompt (fixed, required):** `Zamieniasz tekst na formę mówioną.`
|
||||
- **Context:** max 512 tokens; quality degrades past ~300 characters. Input is one
|
||||
short block per call, so this is fine — warn (don't truncate) on longer inputs.
|
||||
- **Decoding:** greedy (`do_sample=False`), with **constrained decoding** that
|
||||
bans digit tokens from the output so no un-normalized number can leak through.
|
||||
|
||||
## Environment & installation
|
||||
|
||||
Install into a **dedicated, isolated venv/conda env**, separate from the ComfyUI
|
||||
/ Higgs TTS environment. Rationale: this model hard-requires `transformers>=5.0`,
|
||||
and installing that into the existing working TTS environment risks disturbing
|
||||
its pinned dependencies. Keep them apart; this component is invoked as its own
|
||||
process.
|
||||
|
||||
Pinned dependencies:
|
||||
|
||||
- `transformers>=5.0.0`
|
||||
- `torch>=2.1.0` — CUDA build matching the host (target: RTX 3090, 24 GB)
|
||||
- `accelerate` — for `device_map`
|
||||
- `num2words` — used only by the fallback path
|
||||
|
||||
One-time model fetch (network + token required):
|
||||
|
||||
```bash
|
||||
HF_TOKEN=... huggingface-cli download Folx/qwen3-0.6b-pl-text-normalization \
|
||||
--local-dir ./model
|
||||
```
|
||||
|
||||
Run time is **offline**. Set `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`, and
|
||||
load the model from `./model` (or the local HF cache). Add a check that a full run
|
||||
succeeds with networking disabled.
|
||||
|
||||
Load the model, tokenizer, and banned-token list **once** (module level) and reuse
|
||||
across calls — model load dominates cost.
|
||||
|
||||
## Normalization logic
|
||||
|
||||
For a given input string:
|
||||
|
||||
1. **Protect single-digit numbers.** Replace each standalone one-digit integer
|
||||
with a **digit-free sentinel**, remembering the mapping. The boundary set
|
||||
excludes adjacent digits and numeric punctuation so decimals/times/dates are
|
||||
not caught:
|
||||
|
||||
```python
|
||||
KEEP_RE = re.compile(r'(?<![\d.,:/\-])\d(?![\d.,:/\-])')
|
||||
```
|
||||
|
||||
Sentinels must contain **no digits** (digit tokens are banned in output, and
|
||||
the check in step 2 looks for leftover digits). Use letter-indexed sentinels,
|
||||
e.g. `KEEPNUMA`, `KEEPNUMB`, … .
|
||||
|
||||
2. **Decide.** If the protected text still contains a digit, that digit belongs
|
||||
to a 2+ digit / complex number → call the model. If no digit remains, skip the
|
||||
model and return the text with sentinels restored (only single digits were
|
||||
present).
|
||||
|
||||
3. **Model call.** Build the chat prompt (fixed system prompt + protected text),
|
||||
greedy decode with digit tokens banned, decode the completion.
|
||||
|
||||
4. **Restore** sentinels to their original single digits.
|
||||
|
||||
5. On any generation error, use the fallback instead of raising.
|
||||
|
||||
### Reference: model wrapper (adapt, then verify against the gated model)
|
||||
|
||||
```python
|
||||
import re, torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
MODEL = "./model" # local, offline
|
||||
SYS = "Zamieniasz tekst na formę mówioną."
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(MODEL)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
MODEL, torch_dtype=torch.bfloat16, device_map="auto"
|
||||
).eval()
|
||||
|
||||
def _banned_ids(tok):
|
||||
banned = set()
|
||||
for tid in range(tok.vocab_size):
|
||||
d = tok.decode([tid]).strip()
|
||||
if d and d.isdigit():
|
||||
banned.add(tid)
|
||||
for sym in ["$","%","€","£","¥","°","§","@","#","&","×","÷","±",
|
||||
"©","®","™","→","←","↑","↓","√","∞","≈","≠","≤","≥"]:
|
||||
ids = tok.encode(sym, add_special_tokens=False)
|
||||
if len(ids) == 1:
|
||||
banned.add(ids[0])
|
||||
return sorted(banned)
|
||||
|
||||
BANNED = _banned_ids(tok)
|
||||
|
||||
def _model_normalize(text: str) -> str:
|
||||
messages = [{"role": "system", "content": SYS},
|
||||
{"role": "user", "content": text}]
|
||||
prompt = tok.apply_chat_template(messages, tokenize=False,
|
||||
add_generation_prompt=True)
|
||||
inputs = tok(prompt, return_tensors="pt").to(model.device)
|
||||
n = inputs["input_ids"].shape[1]
|
||||
with torch.no_grad():
|
||||
out = model.generate(**inputs, max_new_tokens=400, do_sample=False,
|
||||
pad_token_id=tok.pad_token_id,
|
||||
suppress_tokens=BANNED)
|
||||
return tok.decode(out[0][n:], skip_special_tokens=True).strip()
|
||||
```
|
||||
|
||||
### Reference: skip / decide / restore + public function
|
||||
|
||||
```python
|
||||
KEEP_RE = re.compile(r'(?<![\d.,:/\-])\d(?![\d.,:/\-])')
|
||||
_LETTERS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
||||
|
||||
def _sentinel(i: int) -> str:
|
||||
s = ""; i += 1
|
||||
while i:
|
||||
i, r = divmod(i - 1, 26); s = _LETTERS[r] + s
|
||||
return f"KEEPNUM{s}"
|
||||
|
||||
def normalize(text: str) -> str:
|
||||
"""Return `text` with 2+ digit numbers spelled out in Polish; single-digit
|
||||
numbers left as digits. Safe to call on any string."""
|
||||
mapping = {}
|
||||
def protect(m):
|
||||
key = _sentinel(len(mapping)); mapping[key] = m.group(0); return key
|
||||
protected = KEEP_RE.sub(protect, text)
|
||||
|
||||
if re.search(r'\d', protected):
|
||||
try:
|
||||
result = _model_normalize(protected)
|
||||
except Exception:
|
||||
result = _fallback(protected)
|
||||
else:
|
||||
result = protected
|
||||
|
||||
for key, digit in mapping.items():
|
||||
result = result.replace(key, digit)
|
||||
return result
|
||||
```
|
||||
|
||||
### Reference: fallback (model unavailable / generation error)
|
||||
|
||||
Guarantees the component never raises. Degraded: nominative case only, bare
|
||||
multi-digit integers only (leaves decimals/times/dates untouched rather than
|
||||
mangle them). Log a warning when used.
|
||||
|
||||
```python
|
||||
from num2words import num2words
|
||||
|
||||
_INT_RE = re.compile(r'(?<![\d.,:/\-])\d{2,}(?:[\u00a0 .]\d{3})*(?![\d.,:/\-])')
|
||||
|
||||
def _fallback(text: str) -> str:
|
||||
def repl(m):
|
||||
digits = re.sub(r'[\u00a0 .]', '', m.group(0))
|
||||
return num2words(int(digits), lang="pl")
|
||||
return _INT_RE.sub(repl, text)
|
||||
```
|
||||
|
||||
## Error handling
|
||||
|
||||
- Input longer than ~300 chars: log a warning (quality drops), still process,
|
||||
never truncate silently.
|
||||
- Model load failure: fall back to `num2words` and log clearly that the neural
|
||||
normalizer is unavailable — never crash.
|
||||
- Per-call generation error: use the fallback for that call only.
|
||||
|
||||
## Testing / acceptance
|
||||
|
||||
The model produces valid case variants, so assert on **properties**, not exact
|
||||
strings, except where noted.
|
||||
|
||||
- **Single digit preserved:** `Mam 3 koty.` → still contains `3`.
|
||||
- **Multi-digit expanded:** `Wygrał 1247 złotych.` → no `1247`; contains `tysiąc`.
|
||||
- **Large number:** `populacja 1 350 000 osób` → no digits from that number;
|
||||
contains `milion`.
|
||||
- **Mixed (key sentinel round-trip):** `Mam 3 koty i 25 psów.` → keeps `3`,
|
||||
expands `25` → `dwadzieścia pięć`.
|
||||
- **Only single digits:** model is not called; text returned unchanged (assert
|
||||
via a spy on `_model_normalize`).
|
||||
- **No digits leak:** expanded output contains no digit characters except
|
||||
restored single digits.
|
||||
- **Sentinel round-trip:** `KEEPNUM…` never appears in final output. **If the
|
||||
model is found to alter/inflect the sentinel, flag it** — the fix is a more
|
||||
inert sentinel, or dropping protection and documenting that single digits
|
||||
co-occurring with multi-digit numbers get expanded.
|
||||
- **Offline:** the suite passes with networking disabled, `HF_HUB_OFFLINE=1`,
|
||||
`TRANSFORMERS_OFFLINE=1`.
|
||||
|
||||
## Deliverables
|
||||
|
||||
- `normalizer.py` — model/tokenizer/banned-list loaded once at import; public
|
||||
`normalize(text: str) -> str`; `_fallback`; skip/decide/restore logic.
|
||||
- `requirements.txt` — pinned as above.
|
||||
- `README.md` — env creation, one-time gated download with `HF_TOKEN`, offline
|
||||
run variables. (No caller/integration details — those are provided separately.)
|
||||
- `tests/` — the checks above with sample inputs.
|
||||
|
||||
## Design decisions (defaults — confirm if you disagree)
|
||||
|
||||
1. **Single digit inside a date/time.** `3 maja` keeps its `3` (space-separated
|
||||
standalone single digit), while `9:30`, `3,5`, `12/45` go to the model and are
|
||||
expanded. If you want `3 maja` → `trzeciego maja`, that's a one-line change to
|
||||
the boundary set.
|
||||
2. **Sentinel robustness.** Small residual risk the model alters the placeholder
|
||||
instead of passing it through. Encoded as an explicit test with a documented
|
||||
fallback so it's caught, not shipped silently.
|
||||
@@ -0,0 +1,49 @@
|
||||
# lektor_ai_comfy — replacement dictionary
|
||||
#
|
||||
# Words and phrases rewritten in the subtitles before they are sent to the TTS
|
||||
# engine, for anything the model reads out wrong: abbreviations, units, symbols,
|
||||
# names spelled in a way it mispronounces.
|
||||
#
|
||||
# One rule per line:
|
||||
#
|
||||
# pattern = replacement
|
||||
#
|
||||
# * Matching is CASE-SENSITIVE and matches whole words only: "np." leaves
|
||||
# "znp." alone, and "Joshua" leaves "joshua" alone.
|
||||
# * The replacement is inserted exactly as written — nothing is re-capitalised
|
||||
# for you. Note that the first letter of every subtitle block is already
|
||||
# capitalised before these rules run, so if a word can start a block, add a
|
||||
# second rule for the capitalised form.
|
||||
# * Longer patterns win: "km/h" below is used even though a plain "km" rule
|
||||
# would also fit, whatever order the two are written in.
|
||||
# * The replacement is never re-scanned, so rules cannot feed into each other.
|
||||
# * A pattern wrapped in slashes is a regular expression, and \1 \2 … in the
|
||||
# replacement refer to its groups.
|
||||
# * A line starting with # or ; is a comment. An empty replacement deletes the
|
||||
# phrase.
|
||||
# * "->" and "=>" work as separators too, if you prefer them to "=".
|
||||
#
|
||||
# The changes are visible in clips/tts_input.srt after a run — check there if a
|
||||
# rule is not doing what you expect.
|
||||
|
||||
|
||||
# ── Units ──────────────────────────────────────────────────────────────────
|
||||
km/h = kilometrów na godzinę
|
||||
m/s = metrów na sekundę
|
||||
|
||||
# ── Abbreviations ──────────────────────────────────────────────────────────
|
||||
np. = na przykład
|
||||
tzn. = to znaczy
|
||||
# These two almost always close a sentence, so the dot is kept; drop it if you
|
||||
# see an unwanted pause mid-sentence.
|
||||
itp. = i tym podobne.
|
||||
itd. = i tak dalej.
|
||||
m.in. = między innymi
|
||||
24/7 = dwadzieścia cztery na siedem
|
||||
|
||||
# ── Symbols ────────────────────────────────────────────────────────────────
|
||||
/(\d+)\s*%/ = \1 procent
|
||||
/(\d+)\s*°C/ = \1 stopni Celsjusza
|
||||
& = i
|
||||
|
||||
# ── Your own rules below ───────────────────────────────────────────────────
|
||||
+13
-4
@@ -17,6 +17,7 @@ stream-copied; only the new voiceover track is encoded.
|
||||
| [usage.md](usage.md) | Every CLI flag, the working modes, and practical recipes |
|
||||
| [architecture.md](architecture.md) | How the pipeline works internally, end to end |
|
||||
| [troubleshooting.md](troubleshooting.md) | Diagnosed failure modes and their fixes |
|
||||
| [../normalizer/README.md](../normalizer/README.md) | The Polish number normalizer, its own component |
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -35,10 +36,11 @@ python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs
|
||||
The result is `output/film_lektor.mkv` with two audio tracks — **Original** and
|
||||
**Polish** — plus every subtitle track from the source.
|
||||
|
||||
## The two halves of this project
|
||||
## The three parts of this project
|
||||
|
||||
The pieces are deliberately decoupled and it helps to keep them apart when
|
||||
diagnosing anything:
|
||||
The pieces are deliberately decoupled — each in its own environment, talking
|
||||
over a process boundary — and it helps to keep them apart when diagnosing
|
||||
anything:
|
||||
|
||||
**ComfyUI + Higgs Audio V3** — a local HTTP service on port 8189 that turns text
|
||||
into speech. Heavy: ~22 GB on disk, needs an NVIDIA GPU, lives in its own conda
|
||||
@@ -49,8 +51,15 @@ ComfyUI over HTTP, and does all timing and audio/video assembly with ffmpeg. It
|
||||
is pure Python standard library, runs on the system Python, and needs no conda
|
||||
environment or pip packages of its own.
|
||||
|
||||
**`normalizer/`** — an optional component that spells digits out as Polish words
|
||||
before they reach the TTS. Its own venv (~6.9 GB, mostly CUDA libraries) and
|
||||
its own 1.1 GB model, because it needs `transformers >= 5` and the ComfyUI
|
||||
environment must not be disturbed. Driven as a subprocess for one stage of each
|
||||
run, then closed.
|
||||
|
||||
So a TTS problem (bad pronunciation, hallucination, mumbling) belongs to the
|
||||
first half, and a timing, mixing, or muxing problem belongs to the second.
|
||||
first part, a timing, mixing or muxing problem to the second, and a number read
|
||||
in the wrong case to the third.
|
||||
|
||||
## Conventions in these docs
|
||||
|
||||
|
||||
+96
-2
@@ -37,23 +37,117 @@ must repeat the same `--start`/`--end`.
|
||||
and `<...>` HTML tags (`<i>`, `<font …>`).
|
||||
2. **Detect dialogue.** Lines beginning `- ` mean two speakers and are kept
|
||||
separate; ordinary wrapped lines are joined with a space.
|
||||
3. **Sentence-case ALL-CAPS blocks**, which the model would otherwise shout.
|
||||
3. **Title-case ALL-CAPS blocks**, which the model would otherwise shout.
|
||||
Every word is capitalised rather than lowercasing all but the first letter,
|
||||
because the obvious version also lowercases the proper nouns inside a
|
||||
caption: `NASA – MARGO MADISON` became `Nasa – margo madison`, which is wrong
|
||||
to read aloud and invites the number normalizer to "correct" the name (it
|
||||
produced `marga`). A/B clips showed the TTS reads both casings the same, so
|
||||
keeping the names costs nothing.
|
||||
4. **Capitalise a lowercase first letter**, since a fragment continuing from the
|
||||
previous subtitle confuses the model.
|
||||
5. **Add terminal punctuation** so the model produces a natural falling tail.
|
||||
5. **Apply the replacement dictionary** — `km/h` → `kilometrów na godzinę` and
|
||||
anything else the model reads out wrong (see below).
|
||||
6. **Add terminal punctuation** so the model produces a natural falling tail.
|
||||
|
||||
Step 1 must come first. `<i>ALL CAPS</i>` looks mixed-case while the tag is
|
||||
still present — the `i` in `<i>` is a lowercase letter — so the caps detection
|
||||
in step 3 would silently miss it.
|
||||
|
||||
Step 5 must come after step 3 for the same reason in reverse: a lowercase
|
||||
replacement dropped into `JEDZIE 200 KM/H` would make the block look mixed-case
|
||||
and the shouting would survive. It comes before step 6 so a rule can supply
|
||||
punctuation of its own.
|
||||
|
||||
### The replacement dictionary
|
||||
|
||||
`Dictionary.load()` reads `dictionary.txt` (`[paths] dictionary`, or
|
||||
`--dictionary`) into `pattern = replacement` rules — literal and
|
||||
case-insensitive, or a regular expression when the pattern is wrapped in
|
||||
slashes. Every rule is compiled into **one alternation** applied in a single
|
||||
`re.sub` pass, which buys three properties worth keeping:
|
||||
|
||||
- Replacement text is never rescanned, so rules cannot chain or loop.
|
||||
- Literals are ordered longest-first inside the alternation, so `km/h` beats a
|
||||
plain `km` rule regardless of the order in the file.
|
||||
- Rules cost one pass over the text no matter how many there are.
|
||||
|
||||
Literal patterns get `(?<!\w)` / `(?!\w)` guards only on edges that are word
|
||||
characters, which is what lets `km/h` and `np.` match on their own while `znp.`
|
||||
does not — a guard on a `/` or `.` edge would stop them matching at all.
|
||||
|
||||
Matching is case-sensitive and the replacement is verbatim. The alternative —
|
||||
match loosely, then infer how to re-case the replacement — was tried and
|
||||
removed: it silently rewrote `NASA` as `Nasa`, and a rule that fires on a case
|
||||
you did not write is hard to reason about. The cost is that the first letter of
|
||||
a block is capitalised before the dictionary runs, so a lowercase rule cannot
|
||||
fire there; on the seven subtitle files here that changed nothing, but it is why
|
||||
a word that can open a subtitle needs both forms.
|
||||
|
||||
A missing file, a line without a separator and an invalid regex are all
|
||||
warnings, never fatal — the same rule as `settings.ini`, for the same reason: a
|
||||
typo should not cost a long run.
|
||||
|
||||
`split_dialogue_blocks()` then divides a two-speaker block into one clip per
|
||||
line, apportioning the block's duration by character count.
|
||||
|
||||
### Number normalization (last stage)
|
||||
|
||||
`normalize_block_numbers()` rewrites digits as Polish words — `1247` →
|
||||
`tysiąc dwieście czterdzieści siedem` — because Higgs Audio misreads long digit
|
||||
strings and sometimes emits garbled non-words. It is **last** for two reasons:
|
||||
the dictionary may have produced the words a numeral has to agree with (`16%` →
|
||||
`16 procent` → `szesnaście procent`), and dialogue splitting decides which clip
|
||||
a number ends up in.
|
||||
|
||||
The model that does it needs `transformers>=5`, which cannot be installed beside
|
||||
ComfyUI's pinned dependencies, so it lives in `normalizer/` with its own venv and
|
||||
runs as a **separate process**. `NumberNormalizer` is the client half:
|
||||
|
||||
- One worker per run, not per block — the 1.1 GB load dominates everything else.
|
||||
It is started only if some block actually contains a number worth expanding,
|
||||
and closed before step 2, so it is not holding VRAM while Higgs generates.
|
||||
- **ComfyUI is asked to unload first** (`free_comfy_vram`). Higgs stays resident
|
||||
between runs and leaves too little VRAM; a normalizer that does not fit is
|
||||
spilled to RAM by accelerate *silently*, at about 5× the time per block. A
|
||||
`POST /free` is cheaper than a restart — the server stays warm — and the wait
|
||||
is on `torch_vram_total` in `/system_stats` rather than a fixed sleep, because
|
||||
the flag is consumed by ComfyUI's worker loop rather than by the request.
|
||||
- JSON lines over stdin/stdout, strictly one request per response. stdout is the
|
||||
protocol; logs go to stderr and are kept in a 40-line ring for diagnostics.
|
||||
- Reads are on a queue fed by a reader thread, which is what makes a per-block
|
||||
timeout possible at all — a wedged worker cannot block the run.
|
||||
- **Nothing here is fatal.** Missing venv, dead worker, timeout, out-of-order
|
||||
reply: each prints a warning, disables the stage, and lets the digits through
|
||||
untouched. A number read badly is a bad clip; a crash here is a lost film.
|
||||
|
||||
- **Two models behind one interface.** Plain integers go to num2words for the
|
||||
value and a general Polish LLM for the case; anything with a time, date,
|
||||
decimal or identifier goes to the 0.6B fine-tune. Measured on 40 real subtitle
|
||||
lines, each is clearly better than the other on its half and clearly worse on
|
||||
the other's. The worker reports which leg answered, and the run prints the
|
||||
split.
|
||||
- **Guards, not trust.** Both models can emit something worse than nothing — a
|
||||
repetition loop to the token ceiling, a dropped minus that inverts a
|
||||
temperature, a vanished noun, a number changed from 8250 to 8205. Each answer
|
||||
is checked against the input and falls back to num2words when it fails.
|
||||
|
||||
`needs_number_normalization()` duplicates the worker's own selection rule so
|
||||
blocks that need nothing never cross the process boundary. The two must stay in
|
||||
step; the worker's copy is the authoritative one, and a test in the normalizer
|
||||
suite imports the caller and compares both on a shared sample.
|
||||
|
||||
The result is written to `tts_input.srt` in that video's job folder
|
||||
(`output/clips/<video name>/`). It is the source of truth for block
|
||||
numbering: block *N* there is `clip_NNNN.flac` beside it and the
|
||||
argument to `--regen N`.
|
||||
|
||||
Which is why `--regen`, `--rebuild` **and `--resume-from`** all read that file
|
||||
rather than the original SRT. Everything upstream of it — the dictionary, the
|
||||
normalizer, dialogue splitting — can produce different text than it did on the
|
||||
first run, and any difference breaks the pairing with the clips already on disk.
|
||||
Step 1 is skipped entirely in those modes.
|
||||
|
||||
## 2. Speech generation
|
||||
|
||||
For each block the script POSTs a workflow graph to `/prompt`, polls
|
||||
|
||||
+28
-1
@@ -184,6 +184,27 @@ conda install -n comfy-tts -c conda-forge ffmpeg
|
||||
Note that this puts ffmpeg inside the conda env, so the script must then be run
|
||||
with that env active.
|
||||
|
||||
## Number normalizer (optional, recommended)
|
||||
|
||||
Digits reach the TTS as words rather than figures, in the correct Polish case.
|
||||
It has its own venv on purpose — the model needs `transformers >= 5`, which must
|
||||
not be forced on the ComfyUI environment installed above.
|
||||
|
||||
```bash
|
||||
cd ~/Programy/lektor_ai
|
||||
normalizer/setup.sh # venv + torch (cu128) + transformers
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model # the weights, ~1.1 GB
|
||||
```
|
||||
|
||||
The weights are gated: accept the terms at
|
||||
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> while signed in,
|
||||
then take a token from <https://huggingface.co/settings/tokens>. This is the only
|
||||
step that touches the network; runs are offline afterwards.
|
||||
|
||||
Skipping this is fine — the pipeline falls back to `num2words` (nominative case
|
||||
only) and says so. Turn the whole stage off with `numbers = no` under
|
||||
`[normalize]` in `settings.ini`. See [normalizer/README.md](../normalizer/README.md).
|
||||
|
||||
## Verifying
|
||||
|
||||
```bash
|
||||
@@ -198,7 +219,11 @@ python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
|
||||
# 3. Model present
|
||||
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors
|
||||
|
||||
# 4. End-to-end on 60 seconds of a real file
|
||||
# 4. Number normalizer, if installed
|
||||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||||
# -> "mode": "model" ("fallback" means the weights are missing)
|
||||
|
||||
# 5. End-to-end on 60 seconds of a real file
|
||||
cd ~/Programy/lektor_ai
|
||||
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00
|
||||
```
|
||||
@@ -213,5 +238,7 @@ Nothing was installed system-wide, so removal is just deleting directories:
|
||||
```bash
|
||||
conda env remove -n comfy-tts
|
||||
rm -rf ~/comfy-tts-app
|
||||
rm -rf ~/Programy/lektor_ai/normalizer/venv # number normalizer env
|
||||
rm -rf ~/Programy/lektor_ai/normalizer/model # and its weights
|
||||
rm -rf ~/miniforge3 # only if nothing else uses it
|
||||
```
|
||||
|
||||
+32
-4
@@ -6,7 +6,7 @@
|
||||
|---|---|---|
|
||||
| GPU | NVIDIA, 12 GB VRAM | Developed on an RTX 4070 Ti SUPER (16 GB) |
|
||||
| Driver | 525+ | Needed by the CUDA 12.8 wheels |
|
||||
| Disk | ~35 GB free | ~22 GB installed, plus working room for output |
|
||||
| Disk | ~35 GB free | ~22 GB installed (+8 GB with the number normalizer) |
|
||||
| RAM | 16 GB | The final mux briefly peaks around 2.6 GB |
|
||||
|
||||
The CUDA **toolkit** is not required. The PyTorch cu128 wheels bundle their own
|
||||
@@ -25,6 +25,9 @@ feature-length film.
|
||||
└── models/TTS 8.8 GB Higgs Audio V3 weights
|
||||
------
|
||||
21.7 GB
|
||||
|
||||
lektor_ai/normalizer/venv 6.9 GB optional: number normalizer env
|
||||
lektor_ai/normalizer/model 1.1 GB optional: its weights
|
||||
```
|
||||
|
||||
Output is roughly the size of the source video, since video is stream-copied. A
|
||||
@@ -49,13 +52,13 @@ regenerated without redoing a whole run (see
|
||||
## Dependency surface
|
||||
|
||||
This is the point most worth internalising, because it makes debugging much
|
||||
simpler: **the two halves have almost nothing in common.**
|
||||
simpler: **the three parts have almost nothing in common.**
|
||||
|
||||
### `lektor_ai_comfy.py` — standard library only
|
||||
|
||||
```python
|
||||
argparse json random re shutil subprocess sys threading time uuid
|
||||
pathlib.Path urllib.request
|
||||
argparse configparser json os queue random re shlex shutil subprocess
|
||||
sys threading time uuid pathlib.Path urllib.request
|
||||
```
|
||||
|
||||
No third-party packages. No conda environment. No virtualenv. It runs on the
|
||||
@@ -72,6 +75,25 @@ reachable ComfyUI instance. It talks to ComfyUI over plain HTTP with
|
||||
> **`requirements.txt` in the project root does not apply here.** It belongs to
|
||||
> the older `lektor_ai.py`, which used the hosted Boson AI API and needed
|
||||
> `openai>=1.0.0`. `lektor_ai_comfy.py` requires nothing from it.
|
||||
> `normalizer/requirements.txt` is a different file and does apply — to the
|
||||
> normalizer's own venv, never to the system Python.
|
||||
|
||||
### `normalizer/` — optional third environment
|
||||
|
||||
```
|
||||
torch 2.11.0+cu128 transformers 5.15.0 accelerate 1.14.0
|
||||
num2words 0.5.x (fallback path only)
|
||||
```
|
||||
|
||||
Its own venv at `normalizer/venv` (~6.9 GB — the cu128 wheels bundle their
|
||||
own CUDA runtime, again) plus `normalizer/model` (1.1 GB).
|
||||
Separate from ComfyUI's environment because it hard-requires `transformers >= 5`
|
||||
— 4.x tokenizes the Qwen3 chat template differently and silently produces wrong
|
||||
output. `lektor_ai_comfy.py` never imports it; it starts it as a subprocess and
|
||||
talks JSON lines over a pipe, so the version conflict never has to be resolved.
|
||||
|
||||
Entirely optional: without it the pipeline uses a `num2words` fallback, and with
|
||||
`numbers = no` it is not started at all.
|
||||
|
||||
### ComfyUI — a large, isolated environment
|
||||
|
||||
@@ -130,3 +152,9 @@ API keys, no accounts, no per-use cost.
|
||||
The model is published under a research / non-commercial licence and may require
|
||||
accepting its terms on huggingface.co, then `hf auth login`, before it will
|
||||
download.
|
||||
|
||||
The number normalizer's model is **gated** in the same way and needs an
|
||||
`HF_TOKEN` for its one-time download; see
|
||||
[installation.md](installation.md#number-normalizer-optional-recommended). Runs
|
||||
set `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`, so it never reaches the
|
||||
network afterwards.
|
||||
|
||||
@@ -203,6 +203,93 @@ Handled by `_clean_text()`, but only after tags are removed. `<i>HELLO</i>`
|
||||
looked mixed-case while the tag was present, because `i` is a lowercase letter.
|
||||
If shouting persists, check for an unhandled tag type.
|
||||
|
||||
Such blocks are converted to Title Case (`Nasa – Margo Madison`), not to
|
||||
sentence case. Lowercasing everything but the first letter also destroys the
|
||||
proper nouns in a caption, and the number normalizer then tries to repair the
|
||||
lowercase name — it turned `margo` into `marga`. Paired clips in
|
||||
`output/clips/all-caps/` show the TTS reads both casings identically, so the
|
||||
names are kept.
|
||||
|
||||
### A clip of an ALL-CAPS caption is cut off mid-word
|
||||
|
||||
Check whether the block still contains digits. Higgs truncates on raw digits far
|
||||
more often than on words: the same caption generated eight times, four with
|
||||
`21 MAJA 1983` and four with the date spelled out, was cut in 3 of 4 digit
|
||||
versions and 0 of 4 word versions — and the digit clips ran barely half as long
|
||||
as the speech required. Casing made no difference in that test.
|
||||
|
||||
That is the number normalizer's job, so if you see it, the block reached the TTS
|
||||
without being normalized: check `numbers = yes` and the `Normalizer ready` line
|
||||
in step 1.
|
||||
|
||||
### An abbreviation, unit or name is read out wrong
|
||||
|
||||
Put it in `dictionary.txt`: `km/h = kilometrów na godzinę`. The replacement
|
||||
happens before the text reaches the model, and the result is visible in the job
|
||||
folder's `tts_input.srt`. See [usage.md](usage.md#replacement-dictionary).
|
||||
|
||||
If a rule appears to do nothing, check the run's `Dictionary:` line — a rule
|
||||
that failed to parse was reported there and skipped, and a rule cannot match
|
||||
across a line break because each dialogue line is treated separately.
|
||||
|
||||
`--regen` and `--rebuild` read `tts_input.srt`, which already has the
|
||||
replacements baked in, so adding a rule afterwards needs either a fresh run
|
||||
from the SRT or a manual edit of `tts_input.srt` before `--regen`.
|
||||
|
||||
### Numbers are read as garbled non-words
|
||||
|
||||
That is what the number normalizer is for; check what the run printed in step 1.
|
||||
|
||||
- `Normalizer ready (model on cuda:0)` — it ran. Look at `tts_input.srt`: if the
|
||||
digits are still there, the block may hold only standalone single digits,
|
||||
which are left alone deliberately.
|
||||
- `Normalizer ready (fallback)` — the model is missing or failed to load, and
|
||||
`num2words` is standing in: bare integers only, nominative case only, times
|
||||
and decimals untouched. Install the model, see
|
||||
[usage.md](usage.md#number-normalization).
|
||||
- `normalizer interpreter not found` — run `normalizer/setup.sh`.
|
||||
- Nothing at all — the stage is off (`numbers = no`, or
|
||||
`--no-normalize-numbers`).
|
||||
|
||||
For the underlying cause of a bad load, run the component directly; it prints
|
||||
the real error instead of a summary:
|
||||
|
||||
```bash
|
||||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||||
normalizer/venv/bin/python normalizer/normalizer.py "Wygrał 1247 złotych."
|
||||
```
|
||||
|
||||
### A number is expanded in the wrong grammatical case
|
||||
|
||||
Nominative everywhere (`dwadzieścia pięć psach` instead of `dwudziestu pięciu
|
||||
psach`) means the `num2words` fallback is running, not the model — see above.
|
||||
The neural model is the only path that declines correctly.
|
||||
|
||||
### The normalizer is unexpectedly slow
|
||||
|
||||
Check which device the run reported. `Normalizer ready (model on cpu)` means it
|
||||
never got the GPU. If it says `cuda:0` and is still slow, it may have only got
|
||||
part of it: with Higgs resident there is not enough VRAM, and `accelerate`
|
||||
spills the rest to RAM without raising anything — about 5× slower.
|
||||
|
||||
The script prevents this by asking ComfyUI to unload before normalizing. If you
|
||||
do not see `Freed N GB` in step 1, either `free_comfy_vram = no`, or ComfyUI had
|
||||
nothing loaded (normal on a freshly started backend), or the call failed and
|
||||
said so.
|
||||
|
||||
### The normalizer times out on every block
|
||||
|
||||
`timeout` under `[normalize]` is per block, and a block takes well under a second
|
||||
on a GPU. Timing out means the model landed on the CPU or the worker is wedged.
|
||||
Check `torch.cuda.is_available()` in that venv:
|
||||
|
||||
```bash
|
||||
normalizer/venv/bin/python -c "import torch; print(torch.cuda.is_available())"
|
||||
```
|
||||
|
||||
The stage disables itself after the first timeout rather than repeating the wait
|
||||
on every remaining block, so a run costs one `timeout` at worst.
|
||||
|
||||
### `--embedded-subs` shows no usable tracks
|
||||
|
||||
The video only carries image-based subtitles (`hdmv_pgs_subtitle`,
|
||||
@@ -226,6 +313,58 @@ ComfyUI is not running. Start it and wait for the model to load:
|
||||
curl -s http://127.0.0.1:8189/system_stats | head -c 200
|
||||
```
|
||||
|
||||
### ComfyUI dies partway through a long queue
|
||||
|
||||
The run recovers by itself now, and says so:
|
||||
|
||||
```
|
||||
[2035/2073] 02:22:59 'Joshua.'
|
||||
ComfyUI went away ([Errno 104] Connection reset by peer) [1/3 for this clip].
|
||||
Stopping ComfyUI (pid 530039, suspended)…
|
||||
Starting ComfyUI in a new gnome-terminal window…
|
||||
ComfyUI died on this clip (1/3) — retrying with a different seed.
|
||||
```
|
||||
|
||||
The stale process is stopped before the restart, because a wedged or suspended
|
||||
backend still holds the port and the GPU and the new one then cannot start. A
|
||||
process in state `T` is killed outright — it never runs its signal handlers, so
|
||||
SIGTERM is accepted and ignored.
|
||||
|
||||
**Each retry uses a different seed**, which matters: these aborts are
|
||||
seed-dependent, not text-dependent. The block that killed ComfyUI three times in
|
||||
a 2073-clip run generated perfectly on its own with another seed.
|
||||
|
||||
After three deaths on the same clip the clip is skipped and the queue continues,
|
||||
with a report at the end:
|
||||
|
||||
```
|
||||
2 clip(s) crashed ComfyUI and were skipped — the voiceover is silent there.
|
||||
Edit their text in tts_input.srt, then:
|
||||
--regen 2035 4180
|
||||
```
|
||||
|
||||
Skipping only happens when the backend comes back healthy. If it does not, the
|
||||
run waits for it instead — otherwise a backend that is simply not running would
|
||||
cause every remaining clip to be "skipped" and produce a silent film.
|
||||
|
||||
### The clip that crashes ComfyUI
|
||||
|
||||
The abort looks like this in `comfyui.log`:
|
||||
|
||||
```
|
||||
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1515: indexSelectSmallIndex:
|
||||
Assertion `srcIndex < srcSelectDimSize` failed.
|
||||
Aborted (core dumped)
|
||||
```
|
||||
|
||||
An out-of-range index in an embedding lookup inside the Higgs audio tokenizer.
|
||||
It is a CUDA device-side assert, so it cannot be caught in Python — the process
|
||||
dies outright, which is why it looks like ComfyUI "will not start" when a client
|
||||
is resubmitting the same clip on every restart.
|
||||
|
||||
`--regen N` with a different seed is the fix; rewording the block helps if it
|
||||
persists.
|
||||
|
||||
### ComfyUI crashed and the GPU is stuck
|
||||
|
||||
```bash
|
||||
@@ -244,6 +383,11 @@ python3 lektor_ai_comfy.py film.mkv --resume-from 312
|
||||
Nothing else should be using the GPU during a run. Restart ComfyUI to clear
|
||||
fragmentation. The 4B model needs roughly 10–12 GB.
|
||||
|
||||
The number normalizer adds ~1.5 GB, but only during step 1 — its process exits
|
||||
before TTS generation starts, so the two never hold VRAM at the same time. If it
|
||||
cannot fit anyway, it logs the CUDA error and finishes the run on the `num2words`
|
||||
fallback rather than failing.
|
||||
|
||||
### `torch.cuda.is_available()` is False
|
||||
|
||||
```bash
|
||||
|
||||
+256
@@ -111,6 +111,25 @@ python3 lektor_ai_comfy.py input/film.mkv --resume-from 47
|
||||
The console prints `[47/693]` style progress, so the number to resume from is
|
||||
the last one that completed.
|
||||
|
||||
**It does not reprocess the subtitles.** The blocks come from the job folder's
|
||||
existing `tts_input.srt`, so the dictionary and the number normalizer are not
|
||||
run again:
|
||||
|
||||
```
|
||||
[1/4] Resuming from block 47 — reusing tts_input.srt…
|
||||
693 blocks, subtitles not reprocessed — the numbering still matches the clips on disk.
|
||||
```
|
||||
|
||||
That is a correctness requirement, not an optimisation. `--resume-from` pairs
|
||||
block *N* with `clip_NNNN.flac`, and re-deriving the text can change it — a
|
||||
dictionary rule added since, a different normalizer result, a changed `--start`.
|
||||
Same block count with different words is the dangerous case, because nothing
|
||||
would look wrong until you listened.
|
||||
|
||||
It also means `--start`/`--end` need not be repeated for the block list, since
|
||||
the saved file already holds the adjusted timings. If the job folder has no
|
||||
`tts_input.srt`, the run says so and falls back to parsing the SRT.
|
||||
|
||||
### `--skip-tts`
|
||||
|
||||
Uses a pre-assembled audio file and goes straight to the mux.
|
||||
@@ -143,6 +162,14 @@ tts_volume = 1.8
|
||||
|
||||
[paths]
|
||||
output_dir = output
|
||||
dictionary = dictionary.txt
|
||||
|
||||
[normalize]
|
||||
numbers = yes
|
||||
python = normalizer/venv/bin/python
|
||||
llm = normalizer/models/PLLuM-4B-instruct-2512
|
||||
timeout = 60
|
||||
free_comfy_vram = yes
|
||||
|
||||
[comfy]
|
||||
comfy_url = http://127.0.0.1:8189
|
||||
@@ -251,8 +278,237 @@ Typical starting point for dialogue-heavy material: `--orig-volume 0.5
|
||||
| Flag | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `--output-dir DIR` | `output` | Where everything is written |
|
||||
| `--dictionary FILE` | `dictionary.txt` | Word/phrase replacements, `""` disables |
|
||||
| `--normalize-numbers` / `--no-normalize-numbers` | on | Spell digits out in Polish |
|
||||
| `--comfy-url URL` | `http://127.0.0.1:8189` | Backend address |
|
||||
|
||||
## Replacement dictionary
|
||||
|
||||
`dictionary.txt` next to the script rewrites words and phrases in the subtitles
|
||||
**before** they reach the TTS engine. It exists for everything the model reads
|
||||
out wrong on its own: abbreviations, units, symbols, names with an unusual
|
||||
spelling.
|
||||
|
||||
One rule per line:
|
||||
|
||||
```
|
||||
km/h = kilometrów na godzinę
|
||||
np. = na przykład
|
||||
m.in. = między innymi
|
||||
/(\d+)\s*%/ = \1 procent
|
||||
```
|
||||
|
||||
| Rule | Behaviour |
|
||||
|---|---|
|
||||
| Matching | **Case-sensitive**, whole words only — `np.` does not match inside `znp.`, and `Joshua` does not match `joshua` |
|
||||
| Capitals | The replacement goes in exactly as written; nothing is re-cased for you |
|
||||
| Priority | Longer literals win: `km/h` beats a plain `km` rule whichever order they are written in |
|
||||
| Chaining | Replacement text is never rescanned, so rules cannot feed each other or loop |
|
||||
| Regex | A pattern in slashes is a regular expression; `\1 \2 …` refer to its groups |
|
||||
| Comments | Lines starting with `#` or `;`; `->` and `=>` also work as separators |
|
||||
| Deleting | An empty replacement removes the phrase |
|
||||
|
||||
Each run reports what it loaded and how much it changed:
|
||||
|
||||
```
|
||||
Dictionary: 14 rule(s) from /home/lukas/Programy/lektor_ai/dictionary.txt
|
||||
6 dictionary replacement(s) applied.
|
||||
```
|
||||
|
||||
The replaced text is what lands in the job folder's `tts_input.srt` — check
|
||||
there when a rule is not doing what you expect. Because `--regen` and
|
||||
`--rebuild` work from that file, they do **not** re-apply the dictionary: edit
|
||||
`tts_input.srt` and `--regen` the affected blocks, or rerun from the SRT.
|
||||
|
||||
A relative name is looked for in the current directory first and then next to
|
||||
the script, so one film can keep its own list without hiding the shared one.
|
||||
A missing file, a line with no separator and an invalid regex are all warnings,
|
||||
never fatal.
|
||||
|
||||
### The queue dictionary
|
||||
|
||||
`dictionary.txt` holds rules that are true of Polish generally. A batch often
|
||||
needs its own — a character's name the model mangles, a recurring
|
||||
mistranslation — and those do not belong in the shared file.
|
||||
|
||||
The GUI's **Queue dictionary** button opens a plain text editor with the same
|
||||
format, one rule per line:
|
||||
|
||||
```
|
||||
Joshua = Dżoszua
|
||||
P12 = pozycja dwunasta
|
||||
# comments work here too
|
||||
```
|
||||
|
||||
Saving writes `queue_dictionary.txt` into the **first queued video's job
|
||||
folder**, and every video in the queue is run with it. The dialog reports how
|
||||
many rules parsed, so a typo shows up when you save rather than as a rule that
|
||||
silently does nothing during the run. **Clear** empties the editor only —
|
||||
nothing is written until you press Save.
|
||||
|
||||
The two files are **merged into one rule set**, not chained: a replacement made
|
||||
by one still cannot be rewritten by the other, and longest-literal-wins applies
|
||||
across both. A run lists them separately:
|
||||
|
||||
```
|
||||
Dictionary: 10 rule(s) from /home/lukas/Programy/lektor_ai/dictionary.txt
|
||||
Dictionary: 2 rule(s) from output/clips/S02E01/queue_dictionary.txt
|
||||
3 dictionary replacement(s) applied.
|
||||
```
|
||||
|
||||
On the command line it is `--dictionary-extra FILE`, which works with any file
|
||||
you like.
|
||||
|
||||
## Number normalization
|
||||
|
||||
Higgs Audio misreads long digit strings and sometimes emits garbled non-words,
|
||||
so the **last** subtitle stage rewrites numbers as Polish words:
|
||||
|
||||
```
|
||||
Wygrał 1247 złotych. → Wygrał tysiąc dwieście czterdzieści siedem złotych.
|
||||
Mówimy o 25 psach. → Mówimy o dwudziestu pięciu psach.
|
||||
Mam 3 koty. → Mam 3 koty.
|
||||
```
|
||||
|
||||
The case matters — `25 psów` / `o 25 psach` / `z 25 psami` are three different
|
||||
forms of the same numeral — which is why this is a small fine-tuned model rather
|
||||
than a lookup table. Standalone single digits are left as digits: the TTS reads
|
||||
those correctly, and expanding them only makes the clip longer than the subtitle
|
||||
window it has to fit.
|
||||
|
||||
It runs **after** the dictionary, so a rule that turns `16%` into `16 procent`
|
||||
feeds `szesnaście procent`, and after dialogue splitting, so the numbers are
|
||||
normalized in the clip that actually says them.
|
||||
|
||||
| Flag | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `--normalize-numbers` | on | Spell digits out |
|
||||
| `--no-normalize-numbers` | | Leave digits alone for this run |
|
||||
|
||||
or in `settings.ini`:
|
||||
|
||||
```ini
|
||||
[normalize]
|
||||
numbers = no
|
||||
```
|
||||
|
||||
### Install
|
||||
|
||||
The model needs `transformers >= 5`, which must not go into the ComfyUI
|
||||
environment, so it lives in `normalizer/` with its own venv:
|
||||
|
||||
```bash
|
||||
normalizer/setup.sh # venv + torch + transformers
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model # the weights
|
||||
```
|
||||
|
||||
The weights are **gated**: accept the terms at
|
||||
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> with your Hugging
|
||||
Face account, then create a token at <https://huggingface.co/settings/tokens>.
|
||||
That is the only step that needs the network; runs are offline afterwards.
|
||||
|
||||
Full details, including the JSON-lines protocol and the deviations from the
|
||||
build brief, are in [normalizer/README.md](../normalizer/README.md).
|
||||
|
||||
### Two models, and which one gets a block
|
||||
|
||||
Numbers fail in two different ways and no single model handles both:
|
||||
|
||||
| block contains | goes to | because |
|
||||
|---|---|---|
|
||||
| plain integers only (74%) | num2words → inflection model | num2words cannot miscount; the model only fixes the case |
|
||||
| times, dates, decimals, `P10`, `LO 3842` (26%) | the 0.6B fine-tune | it was trained on those formats; the other leg destroys them |
|
||||
|
||||
The split is why `Mówili o 250000 uchodźcach.` comes out as *o dwustu
|
||||
pięćdziesięciu tysiącach uchodźców* — the fine-tune alone renders that as
|
||||
*o pięciu tysiącach*, losing the number — while `Teraz na P12.` still comes out
|
||||
as *P dwanaście* rather than *Pdziesięć*.
|
||||
|
||||
Every answer is then checked before use: runaway length, repetition loops,
|
||||
invalid characters, leftover digits, a lost minus sign, a changed number, a lost
|
||||
word. A block that fails falls back to num2words — nominative, but never wrong
|
||||
about the value. The run reports how often that happened.
|
||||
|
||||
Set `llm =` (empty) to use the fine-tune for everything.
|
||||
|
||||
### What a run prints
|
||||
|
||||
```
|
||||
5 block(s) contain numbers — normalizing…
|
||||
Asking ComfyUI to unload its models (7.9 GB)…
|
||||
Freed 7.9 GB.
|
||||
Normalizer ready (model on cuda:0, two models).
|
||||
5 of 5 block(s) rewritten (2 complex, 3 plain).
|
||||
1 block(s) failed a safety check and fell back (word lost x1).
|
||||
```
|
||||
|
||||
`Normalizer ready (fallback)` means the model is missing or failed to load and
|
||||
`num2words` is standing in — bare integers in the nominative case only, with
|
||||
times, dates and decimals left as digits. The run says so and continues.
|
||||
|
||||
### Sharing the GPU with ComfyUI
|
||||
|
||||
Higgs stays resident between runs — about 8.4 GB of a 16 GB card — which leaves
|
||||
too little for the normalizer. That failure is silent rather than loud:
|
||||
`accelerate` answers a shortage by spilling layers to RAM, so the run continues
|
||||
with no error and roughly **5× the time per block** (measured: 1.19 s → 5.96 s).
|
||||
|
||||
So before starting the normalizer the script asks ComfyUI to unload:
|
||||
|
||||
```
|
||||
5 block(s) contain numbers — normalizing…
|
||||
Asking ComfyUI to unload its models (7.9 GB)…
|
||||
Freed 7.9 GB.
|
||||
Normalizer ready (model on cuda:0).
|
||||
```
|
||||
|
||||
ComfyUI reloads Higgs on the first clip of step 2 — 20–30 s, once per run. It is
|
||||
a `POST /free` on ComfyUI's own API, not a restart: the server stays up, with
|
||||
its custom nodes loaded. The unload is asynchronous, so the script waits until
|
||||
`torch_vram_total` in `/system_stats` actually drops before continuing.
|
||||
|
||||
`--regen` and `--rebuild` work from `tts_input.srt`, which is already
|
||||
normalized, so they never normalize and never pay the reload. On a freshly
|
||||
started ComfyUI the call is a no-op and prints nothing.
|
||||
|
||||
Set `free_comfy_vram = no` to skip it — worth doing only if you have VRAM to
|
||||
spare, or if you have turned the neural normalizer off.
|
||||
|
||||
Nothing about this stage is fatal. A missing venv, a worker that dies, a block
|
||||
that takes longer than `timeout` — each is a warning, and the affected text goes
|
||||
to the TTS with its digits intact. The worker is started once per run and exits
|
||||
before TTS generation begins, so it is not holding VRAM while Higgs works.
|
||||
|
||||
## The backend's lifecycle
|
||||
|
||||
ComfyUI is normally started once and left running. The script manages it when it
|
||||
has to:
|
||||
|
||||
| situation | what happens |
|
||||
|---|---|
|
||||
| not running when a clip is submitted | started via `start_comfyui.sh`, waited for, the clip resubmitted |
|
||||
| running but wedged or suspended | stopped (SIGKILL if suspended) before the restart |
|
||||
| dies on a clip | up to 3 retries, each with a **different seed** |
|
||||
| dies 3 times on the same clip | that clip is skipped, the queue continues, all skips reported at the end |
|
||||
| queue finished | models unloaded, backend left running — it reloads on the next request |
|
||||
| GUI closed | backend stopped, so no idle process is left holding the GPU |
|
||||
|
||||
Its output goes to `comfyui.log` next to the script as well as to the terminal
|
||||
window, so a crash can be diagnosed after the fact.
|
||||
|
||||
The GUI has a **ComfyUI** button showing the same state: `Start ComfyUI` when it
|
||||
is down, `ComfyUI ●` when it is answering, `ComfyUI ◍` when it holds the port but
|
||||
does not answer. Pressing it starts or restarts.
|
||||
|
||||
Quitting the GUI stops the backend, including one you started by hand in a
|
||||
terminal — it is found by port, not by remembering what was launched. Between
|
||||
runs it stays up on purpose: the queue only unloads its models, so the next job
|
||||
does not pay for a cold start.
|
||||
|
||||
`start_comfyui.sh` ends with `exec python main.py`, so the script's PID *is*
|
||||
ComfyUI's. Without that the launcher only ever sees a wrapper shell, and killing
|
||||
it orphans the python still holding 8 GB of VRAM.
|
||||
|
||||
## Output
|
||||
|
||||
Everything belonging to one video is kept together in a **job folder** named
|
||||
|
||||
+890
-19
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,179 @@
|
||||
# Polish number normalizer
|
||||
|
||||
Rewrites digits as spoken Polish words before the text reaches the TTS:
|
||||
|
||||
```
|
||||
Wygrał 1247 złotych. → Wygrał tysiąc dwieście czterdzieści siedem złotych.
|
||||
Mówimy o 25 psach. → Mówimy o dwudziestu pięciu psach.
|
||||
Mam 3 koty. → Mam 3 koty. (nothing worth sending)
|
||||
Brakuje nam 15000 podpisów. → Brakuje nam piętnastu tysięcy podpisów.
|
||||
```
|
||||
|
||||
Higgs Audio v3 mispronounces raw digits — especially long ones — and sometimes
|
||||
emits garbled non-words. A lookup table cannot fix that, because Polish numerals
|
||||
decline by case, gender and the noun they govern (`25 psów` / `o 25 psach` /
|
||||
`z 25 psami`), so this uses a small fine-tuned model.
|
||||
|
||||
It runs in **its own venv**, separate from ComfyUI: the models require
|
||||
`transformers >= 5`, and forcing that on the working TTS install would risk its
|
||||
pinned dependencies. `lektor_ai_comfy.py` drives it as a subprocess, and asks
|
||||
ComfyUI to unload Higgs first so there is VRAM to work with.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
normalizer/setup.sh # venv + torch (cu128) + transformers + num2words
|
||||
```
|
||||
|
||||
Then the model, which is **gated** — accept the terms at
|
||||
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> with your Hugging
|
||||
Face account first, then:
|
||||
|
||||
```bash
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model
|
||||
```
|
||||
|
||||
The inflection model is separate and **not gated**:
|
||||
|
||||
```bash
|
||||
normalizer/venv/bin/hf download CYFRAGOVPL/PLLuM-4B-instruct-2512 \
|
||||
--local-dir normalizer/models/PLLuM-4B-instruct-2512
|
||||
```
|
||||
|
||||
Those are the only steps that need the network. Everything afterwards runs
|
||||
offline (`HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`, both set by the caller).
|
||||
Without the inflection model every block goes to the fine-tune, which is what
|
||||
the pipeline did before it existed.
|
||||
|
||||
Without the model the component still works, on a `num2words` fallback that
|
||||
handles bare integers in the nominative case only. It says so in the log, and
|
||||
`lektor_ai_comfy.py` prints a warning for the run.
|
||||
|
||||
## Check it
|
||||
|
||||
```bash
|
||||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||||
normalizer/venv/bin/python normalizer/normalizer.py "Wygrał 1247 złotych."
|
||||
normalizer/venv/bin/python -m unittest discover -s normalizer/tests -v
|
||||
```
|
||||
|
||||
The tests assert on properties (`no 1247 left`, `contains tysiąc`) rather than
|
||||
exact strings, because several case variants are correct. The ones that need the
|
||||
model skip themselves when it is not downloaded.
|
||||
|
||||
## What it does
|
||||
|
||||
1. **Skip what needs nothing.** A block whose only numbers are standalone single
|
||||
digits is never sent: the TTS says "pięć minut" for "5 minut" by itself, so
|
||||
there is nothing to gain and a whole sentence of rewriting to risk.
|
||||
2. **Route.** Two models with opposite weaknesses, measured on 40 real subtitle
|
||||
lines and a 32-case battery:
|
||||
|
||||
| | good at | bad at |
|
||||
|---|---|---|
|
||||
| fine-tune (0.6B) | `6:05`, `P10`, `LO 3842`, `95,7`, dates | loses a large number's value while declining it |
|
||||
| num2words + LLM | the value, always; the case | formats — a flight number becomes a quantity |
|
||||
|
||||
So a block goes to the second leg only when **every** number in it is a plain
|
||||
integer — 74% of numbered blocks on the sample. Everything else goes to the
|
||||
fine-tune.
|
||||
3. **Generate.** The fine-tune gets the text as-is. The other leg gets text with
|
||||
no digits at all — num2words has already produced correct nominative words —
|
||||
so its only job is inflection, and it cannot get a value wrong because it
|
||||
never reads a number. Greedy decoding, digit tokens banned, either way.
|
||||
4. **Guard.** The answer is checked before it is used (below). A block that
|
||||
fails falls back to num2words: dull, nominative, never wrong about the value.
|
||||
|
||||
## The guards
|
||||
|
||||
Both models can produce output worse than doing nothing. Each of these fired on
|
||||
a real failure during development:
|
||||
|
||||
| check | the failure it caught |
|
||||
|---|---|
|
||||
| runaway length | a 10-word line came back as 200 words |
|
||||
| repetition loop | `łącznie łącznie łącznie…` to the token ceiling |
|
||||
| invalid characters | a wall of `�` replacement bytes |
|
||||
| digits left | an un-normalized number slipped through |
|
||||
| minus sign lost | `-12 stopni` → *dwanaście stopni*, inverting a temperature |
|
||||
| numbers changed | `Z 8250 osobami` → *osiem tysięcy **dwustu pięciu*** (8205), and `20 lat` → *dwadzieścia **pięć** lat* (25) |
|
||||
| word lost | `38 000 000 mieszkańców` → *trzydzieści osiem milionów* |
|
||||
|
||||
The value check compares against what num2words would say, **in both
|
||||
directions** — a missing numeral and an added one are equally wrong, and a
|
||||
subset check passed `dwadzieścia` → `dwadzieścia pięć` while turning 20 into 25.
|
||||
Numerals are matched through a written-out canonical map rather than string
|
||||
similarity — `dwieście`
|
||||
and `dwustu` are one number and must match, `trzy` and `trzysta` differ by a
|
||||
hundred and must not, and ordinals like `piąty` count as `pięć`. Consonant
|
||||
folding was tried first and rejected good output.
|
||||
|
||||
It is skipped for identifiers: `LO 3842` is correctly read digit by digit, and
|
||||
demanding num2words' rendering would reject a right answer.
|
||||
|
||||
### Deviations from the build brief, and why
|
||||
|
||||
- **Single digits are no longer shielded.** The brief asks for standalone
|
||||
one-digit numbers to be hidden behind `KEEPNUM…` placeholders. Measured, that
|
||||
scored *worse* — 6/9 against 7/9 on the model card's own examples — and it
|
||||
produced an outright pronunciation error, because `3 maja` left as a digit is
|
||||
read *"trzy maja"* where Polish wants *"trzeciego maja"*. It also bought
|
||||
nothing on length: the TTS speaks `5` as *pięć* either way. What survives is
|
||||
the cheaper half — a block containing **only** standalone single digits is
|
||||
never sent at all.
|
||||
- **`KEEP_RE` has extra guards.** The brief's version protects the leading `1`
|
||||
of `1 350 000` as if it were standalone, and its boundary set `(?![\d.,:/-])`
|
||||
rejects any number followed by a comma or full stop — on real subtitles that
|
||||
skipped most of them (`Przód 85, tył 75.`). Numeric punctuation now only
|
||||
continues a number when a digit follows it.
|
||||
- **Two models, not one.** The brief scopes a single normalizer. The 0.6B
|
||||
fine-tune performs to its advertised 85.7%, but its tail includes losing a
|
||||
number's value while declining it, which no amount of prompting fixes. See
|
||||
the routing table above.
|
||||
- **The model loads lazily, not at import.** Same "load once, reuse" effect, but
|
||||
`--help`, `--status` and the non-neural half of the suite do not pay for a
|
||||
1.1 GB load they never use.
|
||||
- **`num2words` errors are caught broadly.** It raises `KeyError`, not
|
||||
`ValueError`, past its largest scale word.
|
||||
|
||||
### Known limits
|
||||
|
||||
Measured, and not fixable at this layer:
|
||||
|
||||
| input | output | |
|
||||
|---|---|---|
|
||||
| `Mieszka pod 12/45.` | *sto czternaście łamane przez trzy* | an address has no reliable reading without context |
|
||||
| `Zadzwoń pod 601 234 567.` | read as one 9-digit number | a phone number is indistinguishable from a grouped number |
|
||||
| `Zyskują 0,3 sekundy` | *trzy setne* | should be *dziesiąte*; the value guard cannot see inside a decimal |
|
||||
| `To dzieło z XIX wieku.` | unchanged | Roman numerals are not handled — use `dictionary.txt` |
|
||||
| `Prędkość 0,75 macha.` | unchanged | the model hallucinated a unit conversion; the guard caught it and fell back |
|
||||
|
||||
## Files
|
||||
|
||||
| File | |
|
||||
|---|---|
|
||||
| `normalizer.py` | The component: routing, both backends, guards, `normalize()` |
|
||||
| `worker.py` | JSON-lines service `lektor_ai_comfy.py` talks to (models loaded once per run) |
|
||||
| `bench.py` | Compares models on the same battery — `--compare a.json b.json` |
|
||||
| `setup.sh` | venv, dependencies, one-time gated model download |
|
||||
| `tests/` | Routing, numerals, guards, and the brief's acceptance checks |
|
||||
| `model/`, `models/`, `venv/` | Weights and environment — not in git |
|
||||
|
||||
## Interface
|
||||
|
||||
```python
|
||||
from normalizer import normalize, needs_normalization
|
||||
|
||||
needs_normalization("Mam 3 koty.") # False — nothing worth sending
|
||||
normalize("Wygrał 1247 złotych.") # never raises, never returns None
|
||||
```
|
||||
|
||||
`worker.py` speaks one JSON object per line on stdin/stdout, with logs on
|
||||
stderr:
|
||||
|
||||
```
|
||||
← {"ready": true, "mode": "model", "device": "cuda:0", "llm": true}
|
||||
→ {"id": 1, "text": "Wygrał 1247 złotych."}
|
||||
← {"id": 1, "text": "Wygrał tysiąc dwieście…", "changed": true,
|
||||
"leg": "complex", "guard": "", "error": null}
|
||||
```
|
||||
@@ -0,0 +1,251 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare normalizer models on the same sentences.
|
||||
|
||||
The shipped model is a 0.6B task-specific fine-tune that needs no prompt but
|
||||
miscounts large numbers and occasionally drops a word. This runs any causal LM
|
||||
over the same battery so the trade is measurable rather than argued about:
|
||||
|
||||
python bench.py # the fine-tune
|
||||
python bench.py --model models/PLLuM-4B-instruct-2512 --prompted
|
||||
python bench.py --model … --prompted --save pllum.json
|
||||
python bench.py --compare fine-tune.json pllum.json
|
||||
|
||||
`--prompted` switches from the fine-tune's bare "text in, text out" contract to
|
||||
an instruction + few-shot prompt, which a general instruct model needs. Digit
|
||||
tokens are banned from the output either way, and decoding is greedy, so a
|
||||
difference between two runs is the model and not the sampling.
|
||||
|
||||
Automatic flags are cheap heuristics, not a grade: `digits` means an
|
||||
un-normalized digit survived, `lost` means a word from the input has no
|
||||
plausible inflected match in the output. Read the text before believing either.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import normalizer as nz # noqa: E402 (reuses its protect/restore and banned list)
|
||||
|
||||
SYSTEM_PROMPTED = (
|
||||
"Jesteś narzędziem do normalizacji tekstu dla syntezatora mowy. "
|
||||
"Zamieniasz wszystkie liczby zapisane cyframi na słowa, w poprawnej formie "
|
||||
"gramatycznej wynikającej z kontekstu zdania. Nie zmieniasz, nie dodajesz "
|
||||
"ani nie usuwasz żadnych innych słów. Zwracasz wyłącznie przekształcone "
|
||||
"zdanie, bez komentarza."
|
||||
)
|
||||
|
||||
# Few-shot examples chosen to pin down the three things a general model gets
|
||||
# wrong without them: the case must follow the governing word, the rest of the
|
||||
# sentence must survive verbatim, and the answer must be the sentence alone.
|
||||
FEWSHOT = [
|
||||
("Koncert dla 8250 osób.",
|
||||
"Koncert dla ośmiu tysięcy dwustu pięćdziesięciu osób."),
|
||||
("Populacja Polski to 38 000 000 mieszkańców.",
|
||||
"Populacja Polski to trzydzieści osiem milionów mieszkańców."),
|
||||
("Mówimy o 25 psach.",
|
||||
"Mówimy o dwudziestu pięciu psach."),
|
||||
("Wygrał 1247 złotych.",
|
||||
"Wygrał tysiąc dwieście czterdzieści siedem złotych."),
|
||||
]
|
||||
|
||||
BATTERY = [
|
||||
"Sprawa dotyczy 1 350 000 ludzi.",
|
||||
"Koncert dla 8250 osób.",
|
||||
"Budżet wynosi 12 500 000 złotych.",
|
||||
"Populacja Polski to 38 000 000 mieszkańców.",
|
||||
"Zapłacili 1 500 000 000 dolarów.",
|
||||
"Zostało 999 999 sztuk.",
|
||||
"Numer seryjny 1234567.",
|
||||
"Ponad 100 000 widzów na stadionie.",
|
||||
"Mówimy o 1 350 000 ludziach.",
|
||||
"Z 8250 osobami nie da się dyskutować.",
|
||||
"Firma zatrudnia 2 500 pracowników.",
|
||||
"Dystans 42 195 metrów.",
|
||||
"Cena spadła z 15 700 000 do 8 250 000.",
|
||||
"Było tam 25 psów.",
|
||||
"Mówimy o 25 psach.",
|
||||
"Przyjechał z 25 psami.",
|
||||
"Przyglądał się 25 psom.",
|
||||
"Nie ma tu 25 psów.",
|
||||
"Urodził się 3 maja 1791 roku.",
|
||||
"Wzrost o 16 procent w 2026 roku.",
|
||||
"Spotkanie o 14:30.",
|
||||
"Godzina 23:56.",
|
||||
"Radio 95,7 FM.",
|
||||
"Około 3,5 metra.",
|
||||
"Mieszka pod 12/45.",
|
||||
"Wygrał 1247 złotych.",
|
||||
"To kosztuje 1999 dolarów i 99 centów.",
|
||||
"Mam 3 koty i 25 psów.",
|
||||
"Zostało 5 minut.",
|
||||
"Temperatura opon w porządku. Przód 85, tył 75.",
|
||||
"Louis Chiron miał 57 lat, kiedy wygrał Grand Prix Monako.",
|
||||
"Zwykłe zdanie bez żadnych liczb.",
|
||||
]
|
||||
|
||||
_WORD_RE = re.compile(r'[^\W\d_]+', re.UNICODE)
|
||||
|
||||
|
||||
def lost_words(src: str, out: str) -> list:
|
||||
"""Input words with no plausible match in the output.
|
||||
|
||||
Polish inflects, so an exact comparison is useless — `ludziach` legitimately
|
||||
becomes `ludziach` and nothing else, but `psach`/`psami` share a stem. A
|
||||
five-character prefix is a crude stand-in for a lemma and is good enough to
|
||||
notice a noun that vanished entirely, which is the failure worth catching.
|
||||
"""
|
||||
def stems(text):
|
||||
return {w.lower()[:5] for w in _WORD_RE.findall(text)}
|
||||
missing = stems(src) - stems(out)
|
||||
return sorted(w for w in missing if len(w) >= 4)
|
||||
|
||||
|
||||
def load_prompted(model_dir: str, gpu_gib: float = None,
|
||||
system: str = None, fewshot: list = None):
|
||||
"""A general instruct model: system prompt, few-shot, greedy, digits banned.
|
||||
|
||||
`gpu_gib` caps GPU use and spills the rest to RAM. Needed for the int4
|
||||
"awq" builds: transformers decompresses compressed-tensors weights back to
|
||||
bfloat16 at load, so an 11B model wants ~22 GB whatever the file size says.
|
||||
Offloading is slow but it is the difference between measuring a model and
|
||||
guessing about it.
|
||||
"""
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(model_dir)
|
||||
kwargs = {"dtype": torch.bfloat16, "device_map": "auto"}
|
||||
if gpu_gib == 0:
|
||||
# Everything in RAM. Slow, but the only way to run a compressed-tensors
|
||||
# int4 build in transformers: its decompression hook expands weights to
|
||||
# bfloat16 in place, and it cannot read the meta tensors that accelerate
|
||||
# leaves behind when max_memory forces a partial offload.
|
||||
kwargs["device_map"] = {"": "cpu"}
|
||||
elif gpu_gib:
|
||||
kwargs["max_memory"] = {0: f"{gpu_gib}GiB", "cpu": "80GiB"}
|
||||
model = AutoModelForCausalLM.from_pretrained(model_dir, **kwargs).eval()
|
||||
|
||||
banned = set()
|
||||
for tid in range(len(tok)):
|
||||
piece = tok.decode([tid]).strip()
|
||||
if piece and piece.isdigit():
|
||||
banned.add(tid)
|
||||
banned = sorted(banned)
|
||||
print(f" {len(banned)} digit tokens banned; device "
|
||||
f"{next(model.parameters()).device}", file=sys.stderr)
|
||||
|
||||
def run(text: str) -> str:
|
||||
messages = [{"role": "system", "content": system or SYSTEM_PROMPTED}]
|
||||
for user, assistant in (fewshot or FEWSHOT):
|
||||
messages.append({"role": "user", "content": user})
|
||||
messages.append({"role": "assistant", "content": assistant})
|
||||
messages.append({"role": "user", "content": text})
|
||||
prompt = tok.apply_chat_template(messages, tokenize=False,
|
||||
add_generation_prompt=True)
|
||||
inputs = tok(prompt, return_tensors="pt").to(model.device)
|
||||
n = inputs["input_ids"].shape[1]
|
||||
with torch.no_grad():
|
||||
out = model.generate(**inputs, max_new_tokens=nz.MAX_NEW_TOKENS,
|
||||
do_sample=False,
|
||||
pad_token_id=tok.pad_token_id or tok.eos_token_id,
|
||||
suppress_tokens=banned)
|
||||
answer = tok.decode(out[0][n:], skip_special_tokens=True).strip()
|
||||
# Instruct models like to introduce themselves; keep the last non-empty
|
||||
# line, which is the sentence in every observed case.
|
||||
lines = [l.strip() for l in answer.splitlines() if l.strip()]
|
||||
return lines[-1] if lines else answer
|
||||
|
||||
return run
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--model", default=None,
|
||||
help="Model directory (default: the shipped fine-tune)")
|
||||
ap.add_argument("--prompted", action="store_true",
|
||||
help="Use an instruction + few-shot prompt (general models)")
|
||||
ap.add_argument("--gpu-gib", type=float, default=None, metavar="N",
|
||||
help="Cap GPU memory and spill the rest to RAM (slow, but "
|
||||
"the only way to run a model that does not fit)")
|
||||
ap.add_argument("--no-protect", action="store_true",
|
||||
help="Do not shield standalone single digits — normalize the "
|
||||
"text exactly as the model card does")
|
||||
ap.add_argument("--save", metavar="FILE", help="Write results as JSON")
|
||||
ap.add_argument("--compare", nargs=2, metavar=("A", "B"),
|
||||
help="Print two saved runs side by side and exit")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.compare:
|
||||
a, b = (json.loads(Path(f).read_text(encoding="utf-8")) for f in args.compare)
|
||||
agree = 0
|
||||
for ra, rb in zip(a["results"], b["results"]):
|
||||
if ra["out"] == rb["out"]:
|
||||
agree += 1
|
||||
continue
|
||||
print(f" in : {ra['in']}")
|
||||
print(f" A : {ra['out']}")
|
||||
print(f" B : {rb['out']}\n")
|
||||
print(f"identical: {agree}/{len(a['results'])} "
|
||||
f"A={a['model']} B={b['model']}")
|
||||
return 0
|
||||
|
||||
if args.prompted:
|
||||
if not args.model:
|
||||
ap.error("--prompted needs --model")
|
||||
run = load_prompted(args.model, args.gpu_gib)
|
||||
label = args.model
|
||||
else:
|
||||
if args.model:
|
||||
nz.MODEL_DIR = Path(args.model)
|
||||
if not nz.load_model():
|
||||
print("model unavailable; refusing to benchmark the fallback",
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
run = nz.normalize
|
||||
label = str(nz.MODEL_DIR)
|
||||
if args.no_protect:
|
||||
nz.KEEP_RE = re.compile(r'(?!x)x') # matches nothing
|
||||
label += " [no single-digit protection]"
|
||||
|
||||
results, t0 = [], time.time()
|
||||
for text in BATTERY:
|
||||
t = time.time()
|
||||
out = run(text)
|
||||
results.append({"in": text, "out": out, "sec": round(time.time() - t, 2),
|
||||
"digits": bool(nz.DIGIT_RE.search(nz.KEEP_RE.sub("", out))),
|
||||
"lost": lost_words(text, out)})
|
||||
total = time.time() - t0
|
||||
|
||||
for r in results:
|
||||
flags = []
|
||||
if r["digits"]:
|
||||
flags.append("DIGITS LEFT")
|
||||
if r["lost"]:
|
||||
flags.append("LOST: " + ", ".join(r["lost"]))
|
||||
print(f" in : {r['in']}")
|
||||
print(f" out: {r['out']}" + (f" [{'; '.join(flags)}]" if flags else ""))
|
||||
print()
|
||||
|
||||
n_dig = sum(r["digits"] for r in results)
|
||||
n_lost = sum(bool(r["lost"]) for r in results)
|
||||
print(f"{label}\n {len(results)} sentences in {total:.1f}s "
|
||||
f"({total/len(results):.2f}s each)\n"
|
||||
f" {n_dig} with digits left, {n_lost} with a word apparently lost")
|
||||
|
||||
if args.save:
|
||||
Path(args.save).write_text(json.dumps(
|
||||
{"model": label, "prompted": args.prompted, "seconds": round(total, 1),
|
||||
"results": results}, ensure_ascii=False, indent=1), encoding="utf-8")
|
||||
print(f" saved to {args.save}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+645
@@ -0,0 +1,645 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Polish number normalizer — digits to spoken Polish words, for TTS.
|
||||
|
||||
Higgs Audio v3 mispronounces raw digits, especially long ones, and sometimes
|
||||
emits garbled non-words. This component rewrites them as fully spelled-out
|
||||
Polish, in the right grammatical case — which is why it is a neural model and
|
||||
not a lookup table: Polish numerals decline by case, gender and the noun they
|
||||
govern.
|
||||
|
||||
>>> normalize("Wygrał 1247 złotych.")
|
||||
'Wygrał tysiąc dwieście czterdzieści siedem złotych.'
|
||||
>>> normalize("Mam 3 koty.") # nothing worth sending: left alone
|
||||
'Mam 3 koty.'
|
||||
|
||||
Model: Folx/qwen3-0.6b-pl-text-normalization, loaded from a local directory and
|
||||
run offline. If it is unavailable the component degrades to a num2words
|
||||
fallback and says so in the log — it never raises and never crashes the caller.
|
||||
|
||||
Run `python normalizer.py "some text"` for a quick check, or `python worker.py`
|
||||
for the JSON-lines service that lektor_ai_comfy.py talks to.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
LOG = logging.getLogger("pl_normalizer")
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
MODEL_DIR = Path(os.environ.get("PL_NORMALIZER_MODEL", HERE / "model"))
|
||||
|
||||
# "auto" uses the GPU when one is free, "cpu" forces RAM. Forcing CPU is a real
|
||||
# use case, not a fallback: ComfyUI keeps ~8.4 GB of VRAM resident between runs,
|
||||
# so a normalizer that insists on the GPU either fights it or has to wait.
|
||||
DEVICE = os.environ.get("PL_NORMALIZER_DEVICE", "auto").strip().lower()
|
||||
|
||||
# The second model. The 0.6B fine-tune reads odd formats well — clock times,
|
||||
# race positions, call signs — but loses the value of a large number while
|
||||
# declining it ("z 8250 osobami" came back as 8205). A general Polish LLM has
|
||||
# the opposite profile. Neither is asked to do the other's job; see route().
|
||||
LLM_DIR = Path(os.environ.get("PL_NORMALIZER_LLM",
|
||||
HERE / "models" / "PLLuM-4B-instruct-2512"))
|
||||
|
||||
# Fixed by the fine-tune — the model was trained with this exact system prompt.
|
||||
SYSTEM_PROMPT = "Zamieniasz tekst na formę mówioną."
|
||||
|
||||
MAX_NEW_TOKENS = 400
|
||||
LONG_INPUT_CHARS = 300 # quality degrades past this; warn, never truncate
|
||||
|
||||
# Where a number begins and ends. Numeric punctuation only continues a number
|
||||
# when a digit follows it, which is what separates "3,5" and "14:30" (one number
|
||||
# each, for the model to handle) from "Przód 85, tył 75." (two, each ending at
|
||||
# ordinary sentence punctuation).
|
||||
#
|
||||
# The build brief writes these as (?<![\d.,:/\-]) and (?![\d.,:/\-]), which also
|
||||
# rejects every number followed by a comma or a full stop — on real subtitles
|
||||
# that silently skipped most of them.
|
||||
_NUM_START = r'(?<!\d)(?<!\d[.,:/\-])'
|
||||
_NUM_END = r'(?!\d)(?![.,:/\-]\d)'
|
||||
|
||||
# A standalone one-digit integer is pronounced correctly by the TTS, so it stays
|
||||
# a digit; anything longer or more complex goes to the model.
|
||||
#
|
||||
# The two space guards are not in the brief either, and are there because
|
||||
# without them "1 350 000" has its leading 1 protected as if it were standalone:
|
||||
# the model then sees "KEEPNUMA 350 000" and returns "1 trzysta pięćdziesiąt
|
||||
# tysięcy" — the million silently lost. A digit that sits next to a group of
|
||||
# exactly three is part of a grouped number, not a number of its own.
|
||||
KEEP_RE = re.compile(_NUM_START + r'(?<!\d[\u00a0\u202f ])'
|
||||
r'\d'
|
||||
+ _NUM_END + r'(?![\u00a0\u202f ]\d{3}(?!\d))')
|
||||
DIGIT_RE = re.compile(r'\d')
|
||||
|
||||
# Fallback path only: integers of 2+ digits, or any number written in groups of
|
||||
# three. Decimals, times and dates are deliberately left alone rather than
|
||||
# mangled without the model's grammar.
|
||||
_INT_RE = re.compile(_NUM_START +
|
||||
r'(?:\d{1,3}(?:[\u00a0\u202f .]\d{3})+|\d{2,})'
|
||||
+ _NUM_END)
|
||||
|
||||
# Banned alongside digits so the model cannot emit a symbol the TTS would read
|
||||
# as a word of the wrong language, or not read at all.
|
||||
_BANNED_SYMBOLS = ["$", "%", "€", "£", "¥", "°", "§", "@", "#", "&", "×", "÷", "±",
|
||||
"©", "®", "™", "→", "←", "↑", "↓", "√", "∞", "≈", "≠", "≤", "≥"]
|
||||
|
||||
# The instruction the general model gets. It never sees a digit — num2words has
|
||||
# already produced correct nominative words — so its only job is to inflect
|
||||
# them. That is why it cannot get a value wrong: it is not reading numbers.
|
||||
SYSTEM_INFLECT = (
|
||||
"Otrzymujesz polskie zdanie, w którym liczebniki są już zapisane słowami, "
|
||||
"ale w mianowniku. Popraw wyłącznie formę gramatyczną tych liczebników tak, "
|
||||
"aby pasowały do zdania (przypadek, rodzaj, liczba). Nie zmieniaj, nie "
|
||||
"dodawaj ani nie usuwaj żadnych innych słów. Zwróć samo zdanie."
|
||||
)
|
||||
|
||||
# Pins down the three things it gets wrong unprompted: the case must follow the
|
||||
# governing word, everything else must survive verbatim, and the answer is the
|
||||
# sentence alone. The last example is a no-op on purpose — without it the model
|
||||
# invents changes when none are needed.
|
||||
FEWSHOT_INFLECT = [
|
||||
("Mówili o dwieście pięćdziesiąt tysięcy uchodźcach.",
|
||||
"Mówili o dwustu pięćdziesięciu tysiącach uchodźców."),
|
||||
("Koncert dla osiem tysięcy dwieście pięćdziesiąt osób.",
|
||||
"Koncert dla ośmiu tysięcy dwustu pięćdziesięciu osób."),
|
||||
("Przyjechał z dwadzieścia pięć psami.",
|
||||
"Przyjechał z dwudziestoma pięcioma psami."),
|
||||
("Kosztuje tysiąc dwieście czterdzieści siedem złotych.",
|
||||
"Kosztuje tysiąc dwieście czterdzieści siedem złotych."),
|
||||
]
|
||||
|
||||
_state = {"loaded": False, "tok": None, "model": None, "banned": None, "error": None}
|
||||
_llm = {"loaded": False, "tok": None, "model": None, "banned": None, "error": None}
|
||||
|
||||
# Counted per process and reported by the worker, so a run can say which leg did
|
||||
# the work and how often a guard had to step in.
|
||||
STATS = {"plain": 0, "complex": 0, "guarded": 0, "reasons": {}}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _banned_ids(tok, cache_dir: Path = None) -> list:
|
||||
"""Token ids that may never appear in the output.
|
||||
|
||||
Scanning the whole vocabulary means one decode call per token — a few
|
||||
seconds for Qwen3's 150k entries — so the result is cached beside the model.
|
||||
"""
|
||||
cache = (cache_dir or MODEL_DIR) / "banned_tokens.json"
|
||||
size = len(tok)
|
||||
if cache.is_file():
|
||||
try:
|
||||
data = json.loads(cache.read_text(encoding="utf-8"))
|
||||
if data.get("vocab_size") == size:
|
||||
return data["ids"]
|
||||
except (OSError, ValueError, KeyError):
|
||||
pass # stale or corrupt — just recompute
|
||||
|
||||
banned = set()
|
||||
for tid in range(size):
|
||||
piece = tok.decode([tid]).strip()
|
||||
if piece and piece.isdigit():
|
||||
banned.add(tid)
|
||||
for sym in _BANNED_SYMBOLS:
|
||||
ids = tok.encode(sym, add_special_tokens=False)
|
||||
if len(ids) == 1:
|
||||
banned.add(ids[0])
|
||||
ids = sorted(banned)
|
||||
try:
|
||||
cache.write_text(json.dumps({"vocab_size": size, "ids": ids}), encoding="utf-8")
|
||||
except OSError:
|
||||
pass # cache is an optimisation, not a requirement
|
||||
return ids
|
||||
|
||||
|
||||
def load_model() -> bool:
|
||||
"""Load model, tokenizer and banned-token list once, and keep them.
|
||||
|
||||
Returns True when the neural path is available. A failure here is not an
|
||||
error: it is logged and the component runs on the fallback for the rest of
|
||||
the process. Loading is lazy rather than at import so that tests, --help and
|
||||
fallback-only runs do not pay for a 1.1 GB model they never use.
|
||||
"""
|
||||
if _state["loaded"]:
|
||||
return _state["model"] is not None
|
||||
_state["loaded"] = True
|
||||
|
||||
if not (MODEL_DIR / "config.json").is_file():
|
||||
_state["error"] = (f"no model in {MODEL_DIR} — see README.md for the "
|
||||
f"one-time download")
|
||||
LOG.warning("Neural normalizer unavailable (%s). Falling back to num2words: "
|
||||
"numbers will be spelled out in the nominative case only.",
|
||||
_state["error"])
|
||||
return False
|
||||
|
||||
try:
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(str(MODEL_DIR))
|
||||
placement = {"": "cpu"} if DEVICE == "cpu" else "auto"
|
||||
try:
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
str(MODEL_DIR), dtype=torch.bfloat16, device_map=placement)
|
||||
except TypeError:
|
||||
# transformers < 5 spells the same argument torch_dtype.
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
str(MODEL_DIR), torch_dtype=torch.bfloat16, device_map=placement)
|
||||
model.eval()
|
||||
_state["tok"], _state["model"] = tok, model
|
||||
_state["banned"] = _banned_ids(tok)
|
||||
LOG.info("Model loaded from %s on %s (%d banned tokens)",
|
||||
MODEL_DIR, next(model.parameters()).device, len(_state["banned"]))
|
||||
return True
|
||||
except Exception as exc: # missing torch, no VRAM, corrupt download…
|
||||
_state["tok"] = _state["model"] = None
|
||||
_state["error"] = f"{type(exc).__name__}: {exc}"
|
||||
LOG.warning("Could not load the neural normalizer (%s). Falling back to "
|
||||
"num2words: numbers will be spelled out in the nominative "
|
||||
"case only.", _state["error"])
|
||||
return False
|
||||
|
||||
|
||||
def model_available() -> bool:
|
||||
return load_model()
|
||||
|
||||
|
||||
def status() -> dict:
|
||||
"""What the caller needs to report: which path is live, and why."""
|
||||
ok = load_model()
|
||||
device = ""
|
||||
if ok:
|
||||
try:
|
||||
device = str(next(_state["model"].parameters()).device)
|
||||
except Exception:
|
||||
device = "unknown"
|
||||
return {"mode": "model" if ok else "fallback",
|
||||
"model_dir": str(MODEL_DIR),
|
||||
"device": device,
|
||||
"error": _state["error"]}
|
||||
|
||||
|
||||
def _model_normalize(text: str) -> str:
|
||||
import torch
|
||||
tok, model = _state["tok"], _state["model"]
|
||||
messages = [{"role": "system", "content": SYSTEM_PROMPT},
|
||||
{"role": "user", "content": text}]
|
||||
prompt = tok.apply_chat_template(messages, tokenize=False,
|
||||
add_generation_prompt=True)
|
||||
inputs = tok(prompt, return_tensors="pt").to(model.device)
|
||||
n = inputs["input_ids"].shape[1]
|
||||
with torch.no_grad():
|
||||
out = model.generate(**inputs,
|
||||
max_new_tokens=MAX_NEW_TOKENS,
|
||||
do_sample=False,
|
||||
pad_token_id=tok.pad_token_id or tok.eos_token_id,
|
||||
suppress_tokens=_state["banned"])
|
||||
return tok.decode(out[0][n:], skip_special_tokens=True).strip()
|
||||
|
||||
|
||||
def load_llm() -> bool:
|
||||
"""Load the general Polish LLM used for the inflection leg.
|
||||
|
||||
Optional: without it every block goes to the fine-tune, which is what the
|
||||
pipeline did before. Reported, never fatal.
|
||||
"""
|
||||
if _llm["loaded"]:
|
||||
return _llm["model"] is not None
|
||||
_llm["loaded"] = True
|
||||
|
||||
if not (LLM_DIR / "config.json").is_file():
|
||||
_llm["error"] = f"no model in {LLM_DIR}"
|
||||
LOG.warning("Inflection model unavailable (%s) — large numbers will be "
|
||||
"normalized by the fine-tune alone.", _llm["error"])
|
||||
return False
|
||||
try:
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(str(LLM_DIR))
|
||||
placement = {"": "cpu"} if DEVICE == "cpu" else "auto"
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
str(LLM_DIR), dtype=torch.bfloat16, device_map=placement).eval()
|
||||
_llm["tok"], _llm["model"] = tok, model
|
||||
_llm["banned"] = _banned_ids(tok, cache_dir=LLM_DIR)
|
||||
LOG.info("Inflection model loaded from %s on %s",
|
||||
LLM_DIR, next(model.parameters()).device)
|
||||
return True
|
||||
except Exception as exc:
|
||||
_llm["tok"] = _llm["model"] = None
|
||||
_llm["error"] = f"{type(exc).__name__}: {exc}"
|
||||
LOG.warning("Could not load the inflection model (%s) — large numbers "
|
||||
"will be normalized by the fine-tune alone.", _llm["error"])
|
||||
return False
|
||||
|
||||
|
||||
def _llm_inflect(text: str) -> str:
|
||||
import torch
|
||||
tok, model = _llm["tok"], _llm["model"]
|
||||
messages = [{"role": "system", "content": SYSTEM_INFLECT}]
|
||||
for user, assistant in FEWSHOT_INFLECT:
|
||||
messages.append({"role": "user", "content": user})
|
||||
messages.append({"role": "assistant", "content": assistant})
|
||||
messages.append({"role": "user", "content": text})
|
||||
prompt = tok.apply_chat_template(messages, tokenize=False,
|
||||
add_generation_prompt=True)
|
||||
inputs = tok(prompt, return_tensors="pt").to(model.device)
|
||||
n = inputs["input_ids"].shape[1]
|
||||
with torch.no_grad():
|
||||
out = model.generate(**inputs, max_new_tokens=MAX_NEW_TOKENS,
|
||||
do_sample=False,
|
||||
pad_token_id=tok.pad_token_id or tok.eos_token_id,
|
||||
suppress_tokens=_llm["banned"])
|
||||
answer = tok.decode(out[0][n:], skip_special_tokens=True).strip()
|
||||
# Instruct models like to introduce their answer; the sentence is the last
|
||||
# non-empty line in every case observed.
|
||||
lines = [l.strip() for l in answer.splitlines() if l.strip()]
|
||||
return lines[-1] if lines else answer
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fallback
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _fallback(text: str) -> str:
|
||||
"""Model unavailable. Nominative case only, bare integers only — wrong case
|
||||
is still far better than the TTS garbling a long digit string."""
|
||||
try:
|
||||
from num2words import num2words
|
||||
except ImportError:
|
||||
LOG.warning("num2words is not installed — leaving %r unchanged", text)
|
||||
return text
|
||||
|
||||
# A minus that stays a dash is a minus the TTS will not read, and -12
|
||||
# spoken as 12 is the wrong temperature.
|
||||
text = re.sub(r'(?<![\w])-(?=\d)', 'minus ', text)
|
||||
|
||||
def repl(m):
|
||||
digits = re.sub(r'[\u00a0\u202f .]', '', m.group(0))
|
||||
try:
|
||||
return num2words(int(digits), lang="pl")
|
||||
except Exception:
|
||||
# num2words raises KeyError, not ValueError, past its largest scale
|
||||
# word (10^66). Leaving the digits is bad; crashing a run is worse.
|
||||
LOG.warning("num2words could not spell out %r — leaving it", m.group(0))
|
||||
return m.group(0)
|
||||
return _INT_RE.sub(repl, text)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Routing
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# The two models fail in opposite directions, measured on 40 real subtitle lines
|
||||
# and a 32-case battery:
|
||||
#
|
||||
# fine-tune good: 6:05, P10, LO 3842, 95,7, 21.00, dates
|
||||
# bad: loses the value of a large number while declining it
|
||||
# ("z 8250 osobami" -> 8205), drops words
|
||||
# num2words good: the value, always, whatever the digit grouping
|
||||
# + LLM bad: destroys formats — a flight number becomes a quantity,
|
||||
# "P10" becomes "Pdziesięć", times fall apart
|
||||
#
|
||||
# So a block goes to the second leg only when every number in it is a plain
|
||||
# integer. On the sample, that is 77% of numbered blocks.
|
||||
|
||||
_MONTHS = ("stycznia|lutego|marca|kwietnia|maja|czerwca|lipca|sierpnia|"
|
||||
"września|października|listopada|grudnia")
|
||||
|
||||
_COMPLEX_RE = re.compile(
|
||||
r'\d[.,:/]\d' # 3,5 14:30 12/45 1.350
|
||||
r'|\d\s*[-–—]\s*\d' # 10-15
|
||||
r'|[^\W\d_]\d|\d[^\W\d_]' # P10, 1998cm3 — an identifier, not a count
|
||||
r'|\b[A-ZĄĆĘŁŃÓŚŹŻ]{2,3}\s\d{2,}\b' # LO 3842 — call sign, read digit by digit
|
||||
r'|\d+\s+(?i:' + _MONTHS + r')\b' # 13 maja — ordinal
|
||||
r'|\d{4}\s+(?i:r(?:oku|\.))', # 2026 roku — ordinal
|
||||
re.UNICODE)
|
||||
# Case matters here, which is why IGNORECASE is scoped to the two alternatives
|
||||
# that need it. Applied globally it turned the call-sign rule into "any short
|
||||
# word before a number", routing "za 150 milionów" and "Przód 85" to the wrong
|
||||
# leg — a third of the sample.
|
||||
|
||||
|
||||
def route(text: str) -> str:
|
||||
""""plain" when every number is a plain integer, else "complex"."""
|
||||
return "complex" if _COMPLEX_RE.search(text) else "plain"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Guards
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Both models can produce output that is worse than doing nothing: a repetition
|
||||
# loop that runs to the token ceiling, a dropped minus sign that inverts a
|
||||
# temperature, a vanished noun. These are cheap structural checks against the
|
||||
# input — no second model pass — and a block that fails one falls back to
|
||||
# num2words, which is dull but never wrong about the value.
|
||||
|
||||
_WORD_RE = re.compile(r'[^\W\d_]+', re.UNICODE)
|
||||
_VOWELS = str.maketrans("", "", "aeiouyąęó")
|
||||
_FOLD = str.maketrans("ąćęłńóśźż", "acelnoszz")
|
||||
|
||||
|
||||
def _skeleton(word: str) -> str:
|
||||
"""Consonants of a word, diacritics folded — a poor man's lemma, used to ask
|
||||
whether a word survived generation at all. Three consonants is deliberate:
|
||||
ludziach/ludzi and uchodźcach/uchodźców are the same word inflected, and a
|
||||
longer key would report them as lost."""
|
||||
return word.lower().translate(_FOLD).translate(_VOWELS)[:3]
|
||||
|
||||
|
||||
# Numerals need identity, not similarity: "dwieście" and "dwustu" are the same
|
||||
# number and must match, while "trzy" and "trzysta" differ by a factor of a
|
||||
# hundred and must not. No consonant rule does both, so the closed class is
|
||||
# written out. Patterns are tried in order, most specific first, against the
|
||||
# lowercased and diacritic-folded word.
|
||||
_NUMERAL_CANON = [
|
||||
(r'zero', "0"),
|
||||
(r'dwadziesc|dwudziest', "20"), (r'dwanasc|dwunast', "12"),
|
||||
(r'dwiesc|dwust', "200"), (r'dw', "2"),
|
||||
(r'trzynasc|trzynast', "13"), (r'trzydzies', "30"),
|
||||
(r'trzyst', "300"), (r'trz', "3"),
|
||||
(r'czternasc|czternast', "14"), (r'czterdzies', "40"),
|
||||
(r'czteryst|czterest', "400"), (r'czter', "4"),
|
||||
(r'pietnasc|pietnast', "15"), (r'piecdziesi', "50"),
|
||||
(r'piecset', "500"), (r'pie[cs]', "5"),
|
||||
(r'szesnasc|szesnast', "16"), (r'szescdziesi', "60"),
|
||||
(r'szescset', "600"), (r'szes', "6"),
|
||||
(r'siedemnasc|siedemnast', "17"), (r'siedemdziesi', "70"),
|
||||
(r'siedemset', "700"), (r'siedem|siedmi', "7"),
|
||||
(r'osiemnasc|osiemnast', "18"), (r'osiemdziesi', "80"),
|
||||
(r'osiemset', "800"), (r'osiem|osm', "8"),
|
||||
(r'dziewietnasc|dziewietnast', "19"), (r'dziewiecdziesi', "90"),
|
||||
(r'dziewiecset', "900"), (r'dziewi', "9"),
|
||||
(r'dziesi', "10"), (r'jedenasc|jedenast', "11"), (r'jed', "1"),
|
||||
# Ordinals denote the same number and are often the better reading —
|
||||
# "na pięćdziesiątym piątym okrążeniu". Without them the guard rejects
|
||||
# correct output.
|
||||
(r'pierwsz', "1"), (r'drug', "2"), (r'czwart', "4"), (r'piat', "5"),
|
||||
(r'szost', "6"), (r'siodm', "7"), (r'setn', "100"),
|
||||
(r'st[ou]$', "100"), (r'tysi', "1000"),
|
||||
(r'miliard', "1e9"), (r'milion', "1e6"),
|
||||
]
|
||||
_NUMERAL_CANON = [(re.compile("^(?:" + pat + ")"), value)
|
||||
for pat, value in _NUMERAL_CANON]
|
||||
|
||||
|
||||
def _numeral(word: str) -> str:
|
||||
"""The number a word denotes, whatever case it is in, or "" if it is not a
|
||||
numeral."""
|
||||
folded = word.lower().translate(_FOLD)
|
||||
for pattern, value in _NUMERAL_CANON:
|
||||
if pattern.match(folded):
|
||||
return value
|
||||
return ""
|
||||
|
||||
|
||||
# Same as _INT_RE but from one digit up. Only ever applied to blocks the router
|
||||
# called "plain", where every number is a bare integer — and there the model
|
||||
# expands single digits too, so the expectation has to include them.
|
||||
_ANY_INT_RE = re.compile(_NUM_START +
|
||||
r'(?:\d{1,3}(?:[\u00a0\u202f .]\d{3})+|\d+)'
|
||||
+ _NUM_END)
|
||||
|
||||
|
||||
def _expected_numerals(text: str) -> list:
|
||||
"""The numerals num2words would produce for `text`, canonicalised."""
|
||||
try:
|
||||
from num2words import num2words
|
||||
except ImportError:
|
||||
return []
|
||||
|
||||
def repl(m):
|
||||
digits = re.sub(r'[\u00a0\u202f .]', '', m.group(0))
|
||||
try:
|
||||
return num2words(int(digits), lang="pl")
|
||||
except Exception:
|
||||
return m.group(0)
|
||||
|
||||
spelled = _ANY_INT_RE.sub(repl, text)
|
||||
return sorted(n for n in map(_numeral, _WORD_RE.findall(spelled)) if n)
|
||||
|
||||
|
||||
def _repeats(text: str) -> bool:
|
||||
"""A word five times over, or a phrase three times — a generation loop.
|
||||
PLLuM produced 'łącznie' eighty times on one input; the fine-tune produced
|
||||
a wall of replacement characters on another."""
|
||||
words = text.lower().split()
|
||||
run = 1
|
||||
for a, b in zip(words, words[1:]):
|
||||
run = run + 1 if a == b else 1
|
||||
if run >= 4:
|
||||
return True
|
||||
for size in (2, 3):
|
||||
seen = {}
|
||||
for i in range(len(words) - size + 1):
|
||||
gram = tuple(words[i:i + size])
|
||||
seen[gram] = seen.get(gram, 0) + 1
|
||||
if seen[gram] >= 3:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _guard(original: str, candidate: str, check_value: bool) -> str:
|
||||
"""Return the reason `candidate` is unusable, or "" if it passes."""
|
||||
if not candidate.strip():
|
||||
return "empty"
|
||||
if len(candidate) > 3 * len(original) + 60:
|
||||
return "runaway length"
|
||||
if _repeats(candidate):
|
||||
return "repetition loop"
|
||||
if "�" in candidate:
|
||||
return "invalid characters"
|
||||
if DIGIT_RE.search(candidate):
|
||||
return "digits left in output"
|
||||
if re.search(r'(?<![\w])-\s?\d', original) and "minus" not in candidate.lower():
|
||||
return "minus sign lost"
|
||||
|
||||
if check_value:
|
||||
# The plain leg's model is given text with no digits in it and asked
|
||||
# only to inflect, so the numerals it returns must be exactly the ones
|
||||
# num2words put there — same values, same count. A one-directional
|
||||
# check is not enough: "dwadzieścia" -> "dwadzieścia pięć" keeps every
|
||||
# expected word and still turns 20 into 25.
|
||||
#
|
||||
# The expectation comes from the full num2words rendering rather than
|
||||
# from the digits alone, so numerals already written as words in the
|
||||
# subtitle ("350 tysięcy") are counted too.
|
||||
want = _expected_numerals(original)
|
||||
got = sorted(n for n in map(_numeral, _WORD_RE.findall(candidate)) if n)
|
||||
if want != got:
|
||||
missing = [n for n in want if n not in got]
|
||||
extra = [n for n in got if n not in want]
|
||||
detail = (f"missing {','.join(missing)}" if missing else "") + \
|
||||
(" " if missing and extra else "") + \
|
||||
(f"added {','.join(extra)}" if extra else "reordered")
|
||||
return f"numbers changed ({detail})"
|
||||
|
||||
# A word that vanished entirely. Short words are skipped because the
|
||||
# dictionary legitimately expands them (ul. -> ulicy, zł -> złote).
|
||||
have = [_skeleton(w) for w in _WORD_RE.findall(candidate)]
|
||||
for word in _WORD_RE.findall(original):
|
||||
if len(word) >= 5 and _skeleton(word) not in have:
|
||||
return f"word lost ({word})"
|
||||
return ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Decide / normalize
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def needs_normalization(text: str) -> bool:
|
||||
"""True when `text` holds a number the TTS would get wrong — anything with
|
||||
a digit, once standalone single digits are discounted.
|
||||
|
||||
A block whose only numbers are standalone single digits is not sent at all:
|
||||
the TTS says "pięć minut" for "5 minut" by itself, so there is nothing to
|
||||
gain and a whole sentence of rewriting to risk. Inside a block that *is*
|
||||
sent, single digits are normalized along with everything else.
|
||||
|
||||
lektor_ai_comfy.py applies the same rule before sending a block, so blocks
|
||||
that need nothing never cross the process boundary. Keep the two in step.
|
||||
"""
|
||||
return bool(DIGIT_RE.search(KEEP_RE.sub("", text)))
|
||||
|
||||
|
||||
def normalize(text: str) -> str:
|
||||
"""Return `text` with its numbers spelled out in Polish. Never raises."""
|
||||
return normalize_detail(text)[0]
|
||||
|
||||
|
||||
def normalize_detail(text: str) -> tuple:
|
||||
"""(result, leg, guard_reason) — the caller reports which leg did the work.
|
||||
|
||||
Plain integers go to num2words for the value and the LLM for the grammar,
|
||||
which is the only arrangement measured to get both right. Everything else
|
||||
goes to the fine-tune, which understands the formats. Either way the result
|
||||
is checked before it is returned.
|
||||
"""
|
||||
if not text or not DIGIT_RE.search(text):
|
||||
return text, "skipped", ""
|
||||
if not needs_normalization(text):
|
||||
return text, "skipped", "" # only standalone single digits
|
||||
if len(text) > LONG_INPUT_CHARS:
|
||||
LOG.warning("Input is %d characters; quality degrades past %d. "
|
||||
"Processing it anyway.", len(text), LONG_INPUT_CHARS)
|
||||
|
||||
leg = route(text)
|
||||
if leg == "plain" and load_llm():
|
||||
STATS["plain"] += 1
|
||||
nominative = _fallback(text) # deterministic: the value cannot be wrong
|
||||
try:
|
||||
candidate = _llm_inflect(nominative)
|
||||
except Exception as exc:
|
||||
LOG.warning("Inflection failed (%s: %s)", type(exc).__name__, exc)
|
||||
candidate = nominative
|
||||
safe, check_value = nominative, True
|
||||
else:
|
||||
leg = "complex" # no inflection model: the fine-tune takes it
|
||||
STATS["complex"] += 1
|
||||
if load_model():
|
||||
try:
|
||||
candidate = _model_normalize(text)
|
||||
except Exception as exc:
|
||||
LOG.warning("Generation failed (%s: %s)", type(exc).__name__, exc)
|
||||
candidate = _fallback(text)
|
||||
else:
|
||||
candidate = _fallback(text)
|
||||
# The value check needs a num2words expectation, which only exists for
|
||||
# plain integers — "LO 3842" is correctly read digit by digit, and
|
||||
# demanding "trzy tysiące osiemset" of it would reject a good answer.
|
||||
safe, check_value = _fallback(text), False
|
||||
|
||||
reason = _guard(text, candidate, check_value)
|
||||
if not reason:
|
||||
return candidate, leg, ""
|
||||
|
||||
STATS["guarded"] += 1
|
||||
STATS["reasons"][reason.split(" (")[0]] = \
|
||||
STATS["reasons"].get(reason.split(" (")[0], 0) + 1
|
||||
LOG.warning("Rejected %r -> %r (%s); using %r", text, candidate, reason, safe)
|
||||
return (safe if safe.strip() else text), leg, reason
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def main(argv=None) -> int:
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Spell out Polish numbers for TTS.",
|
||||
epilog="With no TEXT, reads lines from stdin and writes them normalized.")
|
||||
parser.add_argument("text", nargs="*", metavar="TEXT")
|
||||
parser.add_argument("--status", action="store_true",
|
||||
help="Report which path is live and exit")
|
||||
parser.add_argument("--fallback-only", action="store_true",
|
||||
help="Skip the model, use num2words (for testing)")
|
||||
parser.add_argument("-v", "--verbose", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
logging.basicConfig(level=logging.INFO if args.verbose else logging.WARNING,
|
||||
format="%(levelname)s: %(message)s", stream=sys.stderr)
|
||||
if args.fallback_only:
|
||||
_state.update(loaded=True, model=None, tok=None,
|
||||
error="--fallback-only requested")
|
||||
|
||||
if args.status:
|
||||
print(json.dumps(status(), ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
if args.text:
|
||||
print(normalize(" ".join(args.text)))
|
||||
return 0
|
||||
|
||||
for line in sys.stdin:
|
||||
print(normalize(line.rstrip("\n")), flush=True)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,13 @@
|
||||
# Polish number normalizer — isolated environment.
|
||||
#
|
||||
# Deliberately NOT installed into the ComfyUI / Higgs Audio environment: the
|
||||
# model needs transformers >= 5, and forcing that on the working TTS install
|
||||
# risks its pinned dependencies. The two run as separate processes.
|
||||
|
||||
# 4.x tokenizes the Qwen3 chat template differently and silently produces wrong
|
||||
# output even under greedy decode. This is a hard requirement, not a preference.
|
||||
transformers>=5.0.0
|
||||
|
||||
torch>=2.1.0 # install the CUDA build matching the host, see README
|
||||
accelerate>=0.26.0 # device_map="auto"
|
||||
num2words>=0.5.13 # fallback path only
|
||||
Executable
+98
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env bash
|
||||
# Create the isolated environment for the Polish number normalizer.
|
||||
#
|
||||
# ./setup.sh venv + dependencies
|
||||
# ./setup.sh --model also download the model (needs HF_TOKEN)
|
||||
# ./setup.sh --cpu CPU-only torch, for a machine without an NVIDIA GPU
|
||||
#
|
||||
# Kept separate from the ComfyUI environment on purpose: this needs
|
||||
# transformers >= 5, and the TTS install must not be disturbed.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
VENV="$HERE/venv"
|
||||
MODEL_DIR="$HERE/model"
|
||||
MODEL_REPO="Folx/qwen3-0.6b-pl-text-normalization"
|
||||
TORCH_INDEX="https://download.pytorch.org/whl/cu128" # same wheels as ComfyUI
|
||||
|
||||
WANT_MODEL=0
|
||||
CPU_ONLY=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--model) WANT_MODEL=1 ;;
|
||||
--cpu) CPU_ONLY=1 ;;
|
||||
-h|--help) sed -n '2,10p' "${BASH_SOURCE[0]}" | sed 's/^# \?//'; exit 0 ;;
|
||||
*) echo "Unknown option: $arg" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
say() { printf '\n\033[1;36m==> %s\033[0m\n' "$1"; }
|
||||
|
||||
say "Creating $VENV"
|
||||
python3 -m venv "$VENV"
|
||||
"$VENV/bin/pip" install --upgrade pip --quiet
|
||||
|
||||
say "Installing torch"
|
||||
if [ "$CPU_ONLY" = 1 ]; then
|
||||
"$VENV/bin/pip" install torch --index-url https://download.pytorch.org/whl/cpu
|
||||
else
|
||||
"$VENV/bin/pip" install torch --index-url "$TORCH_INDEX"
|
||||
fi
|
||||
|
||||
say "Installing transformers, accelerate, num2words"
|
||||
"$VENV/bin/pip" install "transformers>=5.0.0" "accelerate>=0.26.0" "num2words>=0.5.13"
|
||||
|
||||
if [ "$WANT_MODEL" = 1 ]; then
|
||||
say "Downloading $MODEL_REPO"
|
||||
"$VENV/bin/pip" install --quiet "huggingface_hub[cli]"
|
||||
|
||||
# Either an HF_TOKEN in the environment or a stored `hf auth login` works;
|
||||
# the CLI picks up both on its own. Only say something when there is
|
||||
# neither, because the failure then is a confusing "access denied".
|
||||
if [ -z "${HF_TOKEN:-}" ] && ! "$VENV/bin/hf" auth whoami >/dev/null 2>&1; then
|
||||
cat >&2 <<EOF
|
||||
No Hugging Face credentials. The repo is gated (approval is automatic, but it
|
||||
has to be your account that asks):
|
||||
|
||||
1. Open https://huggingface.co/$MODEL_REPO while signed in and accept the terms
|
||||
2. Either $VENV/bin/hf auth login
|
||||
or HF_TOKEN=hf_... $0 --model
|
||||
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
"$VENV/bin/hf" download "$MODEL_REPO" --local-dir "$MODEL_DIR"
|
||||
fi
|
||||
|
||||
say "Checking the installation"
|
||||
"$VENV/bin/python" - <<'PY'
|
||||
import sys
|
||||
try:
|
||||
import torch, transformers
|
||||
print(f" torch {torch.__version__} cuda_available={torch.cuda.is_available()}")
|
||||
if torch.cuda.is_available():
|
||||
print(f" device: {torch.cuda.get_device_name(0)}")
|
||||
print(f" transformers {transformers.__version__}")
|
||||
if int(transformers.__version__.split('.')[0]) < 5:
|
||||
print(" WARNING: transformers < 5 produces wrong output with this model",
|
||||
file=sys.stderr)
|
||||
except ImportError as exc:
|
||||
print(f" {exc}", file=sys.stderr); sys.exit(1)
|
||||
PY
|
||||
|
||||
"$VENV/bin/python" "$HERE/normalizer.py" --status
|
||||
|
||||
if [ ! -f "$MODEL_DIR/config.json" ]; then
|
||||
cat <<EOF
|
||||
|
||||
The environment is ready, but the model is not downloaded yet, so the
|
||||
normalizer will run on the degraded num2words fallback.
|
||||
|
||||
1. Accept the terms at https://huggingface.co/$MODEL_REPO
|
||||
2. HF_TOKEN=hf_... $HERE/setup.sh --model
|
||||
|
||||
EOF
|
||||
else
|
||||
say "Ready. Try: $VENV/bin/python $HERE/normalizer.py 'Wygrał 1247 złotych.'"
|
||||
fi
|
||||
@@ -0,0 +1,375 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Acceptance checks for the Polish number normalizer.
|
||||
|
||||
The model produces valid case variants, so these assert on *properties* — a
|
||||
digit is gone, a word stem is present — never on an exact string, except where
|
||||
the behaviour genuinely is exact (model-not-called, pass-through).
|
||||
|
||||
Run them in the normalizer venv:
|
||||
|
||||
normalizer/venv/bin/python -m pytest normalizer/tests -v
|
||||
|
||||
They pass with networking disabled. Tests that need the neural path are skipped
|
||||
automatically when the model has not been downloaded; the rest exercise the
|
||||
fallback and the protect/decide/restore logic, which is where the integration
|
||||
bugs actually live.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
os.environ.setdefault("HF_HUB_OFFLINE", "1")
|
||||
os.environ.setdefault("TRANSFORMERS_OFFLINE", "1")
|
||||
|
||||
import normalizer # noqa: E402
|
||||
|
||||
HAVE_MODEL = normalizer.model_available()
|
||||
needs_model = unittest.skipUnless(
|
||||
HAVE_MODEL, "model not downloaded — see normalizer/README.md")
|
||||
|
||||
DIGITS = re.compile(r'\d')
|
||||
|
||||
|
||||
class Selection(unittest.TestCase):
|
||||
"""Which blocks are worth sending at all. lektor_ai_comfy.py mirrors this
|
||||
rule to keep blocks that need nothing from crossing the process boundary."""
|
||||
|
||||
def test_single_digit_is_not_worth_normalizing(self):
|
||||
self.assertFalse(normalizer.needs_normalization("Mam 3 koty."))
|
||||
|
||||
def test_no_digits_at_all(self):
|
||||
self.assertFalse(normalizer.needs_normalization("Zwykłe zdanie."))
|
||||
|
||||
def test_multi_digit(self):
|
||||
self.assertTrue(normalizer.needs_normalization("Wygrał 1247 złotych."))
|
||||
|
||||
def test_complex_expressions_are_not_single_digits(self):
|
||||
for text in ("Spotkanie o 9:30.", "Około 3,5 metra.", "Mieszka pod 12/45.",
|
||||
"O 14:30 w środę."):
|
||||
with self.subTest(text=text):
|
||||
self.assertTrue(normalizer.needs_normalization(text))
|
||||
|
||||
def test_thousands_with_spaces(self):
|
||||
self.assertTrue(normalizer.needs_normalization("populacja 1 350 000 osób"))
|
||||
|
||||
def test_single_digit_before_punctuation_is_still_single(self):
|
||||
for text in ("Mam 3, potem 4.", "Zostało 5.", "Tylko 7!"):
|
||||
with self.subTest(text=text):
|
||||
self.assertFalse(normalizer.needs_normalization(text))
|
||||
|
||||
def test_multi_digit_before_punctuation_is_caught(self):
|
||||
self.assertTrue(normalizer.needs_normalization("Przód 85, tył 75."))
|
||||
|
||||
|
||||
class ClientRuleInSync(unittest.TestCase):
|
||||
"""lektor_ai_comfy.py keeps its own copy of the selection rule so blocks
|
||||
that need nothing never start a subprocess. A copy that drifts sends the
|
||||
wrong blocks — or silently stops sending any — so it is checked here, where
|
||||
the authoritative rule lives. The caller is pure standard library, so this
|
||||
venv can import it."""
|
||||
|
||||
SAMPLES = [
|
||||
"Mam 3 koty.", "Zwykłe zdanie.", "Wygrał 1247 złotych.",
|
||||
"Przód 85, tył 75.", "populacja 1 350 000 osób", "Radio 95,7 FM.",
|
||||
"Spotkanie o 14:30.", "Mieszka pod 12/45.", "Około 3,5 metra.",
|
||||
"Formuła 1", "Masz F2, masz F3.", "Zostało 5.", "Mam 3, potem 4.",
|
||||
"W 2026 roku.", "Tylko 7!", "1 pies, 2 koty, 300 myszy.",
|
||||
]
|
||||
|
||||
def test_same_verdict_on_every_sample(self):
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
try:
|
||||
import lektor_ai_comfy
|
||||
except ImportError as exc: # pragma: no cover - only if moved
|
||||
self.skipTest(f"caller not importable: {exc}")
|
||||
for text in self.SAMPLES:
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(lektor_ai_comfy.needs_number_normalization(text),
|
||||
normalizer.needs_normalization(text))
|
||||
|
||||
|
||||
class Boundaries(unittest.TestCase):
|
||||
"""KEEP_RE no longer shields anything — it only decides whether a block is
|
||||
worth sending. These pin down where a number starts and ends."""
|
||||
|
||||
def test_grouped_thousands_are_not_split(self):
|
||||
"""The leading 1 of "1 350 000" is not a standalone single digit; if it
|
||||
counted as one the block might not be sent at all."""
|
||||
protected = normalizer.KEEP_RE.sub("X", "populacja 1 350 000 osób")
|
||||
self.assertIn("1 350 000", protected)
|
||||
|
||||
def test_complex_numbers_are_not_single_digits(self):
|
||||
protected = normalizer.KEEP_RE.sub("X", "Mam 3 koty, 25 psów i 3,5 kg.")
|
||||
self.assertNotIn(" 3 ", protected) # standalone single digit
|
||||
self.assertIn("25", protected) # multi-digit untouched
|
||||
self.assertIn("3,5", protected) # decimal untouched
|
||||
|
||||
|
||||
class Routing(unittest.TestCase):
|
||||
"""Which leg a block goes to. The two models fail in opposite directions,
|
||||
so a misrouted block gets the worse of the two."""
|
||||
|
||||
PLAIN = ["Wygrał 1247 złotych.", "Mówili o 250000 uchodźcach.",
|
||||
"Przód 85, tył 75.", "Małpi gaj za 150 milionów dolarów.",
|
||||
"Bardzo. Ma z 80 lat.", "populacja 1 350 000 osób",
|
||||
# A range written in words is two plain integers; only a dash
|
||||
# between digits makes it a compound the fine-tune reads better.
|
||||
"Ma od 15 do 20 lat."]
|
||||
COMPLEX = ["Spotkanie o 14:30.", "Radio 95,7 FM.", "Mieszka pod 12/45.",
|
||||
"Teraz na P12.", "Lot LO 3842 jest opóźniony.",
|
||||
"Gdzie 13 maja 1950 roku się zaczęło.", "Ma od 10-20 lat.",
|
||||
"Silnik ma 1998 cm3 pojemności.", "W 2026 roku."]
|
||||
|
||||
def test_plain(self):
|
||||
for text in self.PLAIN:
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(normalizer.route(text), "plain")
|
||||
|
||||
def test_complex(self):
|
||||
for text in self.COMPLEX:
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(normalizer.route(text), "complex")
|
||||
|
||||
def test_case_sensitivity_of_the_call_sign_rule(self):
|
||||
"""Applied case-insensitively this rule matched any short word before a
|
||||
number and misrouted a third of the sample."""
|
||||
self.assertEqual(normalizer.route("Lot LO 3842 wystartował."), "complex")
|
||||
self.assertEqual(normalizer.route("Zapłacił za 150 złotych."), "plain")
|
||||
|
||||
|
||||
class Numerals(unittest.TestCase):
|
||||
"""The value guard needs numeral identity across inflection, which no
|
||||
consonant rule provides: dwieście/dwustu are one number, trzy/trzysta are
|
||||
not."""
|
||||
|
||||
SAME = [("dwieście", "dwustu"), ("pięćdziesiąt", "pięćdziesięciu"),
|
||||
("trzydzieści", "trzydziestu"), ("czterdzieści", "czterdziestu"),
|
||||
("tysiąc", "tysiącach"), ("milion", "milionach"),
|
||||
("osiem", "ośmiu"), ("pięć", "piątym"), ("dwadzieścia", "dwudziesty")]
|
||||
DIFFERENT = [("trzy", "trzysta"), ("pięć", "pięćdziesiąt"),
|
||||
("dwa", "dwieście"), ("milion", "miliard"),
|
||||
("sto", "sześćset"), ("cztery", "czternaście")]
|
||||
|
||||
def test_same_number(self):
|
||||
for a, b in self.SAME:
|
||||
with self.subTest(pair=(a, b)):
|
||||
self.assertEqual(normalizer._numeral(a), normalizer._numeral(b))
|
||||
self.assertTrue(normalizer._numeral(a))
|
||||
|
||||
def test_different_numbers(self):
|
||||
for a, b in self.DIFFERENT:
|
||||
with self.subTest(pair=(a, b)):
|
||||
self.assertNotEqual(normalizer._numeral(a), normalizer._numeral(b))
|
||||
|
||||
def test_ordinary_words_are_not_numerals(self):
|
||||
for word in ("koty", "uchodźcach", "okrążenie", "prędkość"):
|
||||
self.assertEqual(normalizer._numeral(word), "")
|
||||
|
||||
|
||||
class Guards(unittest.TestCase):
|
||||
"""Every check here fired on a real failure during development. The
|
||||
false-positive cases matter just as much: a guard that rejects good output
|
||||
silently downgrades the run."""
|
||||
|
||||
def test_accepts_correct_output(self):
|
||||
for original, candidate in [
|
||||
("Mówili o 250000 uchodźcach.",
|
||||
"Mówili o dwustu pięćdziesięciu tysiącach uchodźców."),
|
||||
("To wypis sprzed 30 lat.", "To wypis sprzed trzydziestu lat."),
|
||||
("kolizja na 55 okrążeniu", "kolizja na pięćdziesiątym piątym okrążeniu"),
|
||||
("misja to Apollo 25", "misja to Apollo dwudziesty piąty"),
|
||||
]:
|
||||
with self.subTest(candidate=candidate):
|
||||
self.assertEqual(normalizer._guard(original, candidate, True), "")
|
||||
|
||||
def test_value_changed(self):
|
||||
reason = normalizer._guard("Z 8250 osobami.",
|
||||
"Z osiem tysięcy dwustu pięciu osobami.", True)
|
||||
self.assertIn("numbers changed", reason)
|
||||
|
||||
def test_added_numeral(self):
|
||||
"""The check must run both ways. "dwadzieścia" -> "dwadzieścia pięć"
|
||||
keeps every expected word and still turns 20 into 25 — a one-directional
|
||||
subset check passed this and shipped 25 years of service as 20."""
|
||||
reason = normalizer._guard("Nagroda Za 20 Lat Pracy.",
|
||||
"Nagroda Za dwadzieścia Pięć Lat Pracy.", True)
|
||||
self.assertIn("added", reason)
|
||||
|
||||
def test_single_digits_count_toward_the_expectation(self):
|
||||
"""num2words skips one-digit numbers but the model expands them, so the
|
||||
expectation must cover them or "trzy" looks like an invention."""
|
||||
self.assertEqual(normalizer._guard("Mam 3 koty i 25 psów.",
|
||||
"Mam trzy koty i dwadzieścia pięć psów.",
|
||||
True), "")
|
||||
|
||||
def test_word_lost(self):
|
||||
reason = normalizer._guard("Populacja to 38000000 mieszkańców.",
|
||||
"Populacja to trzydzieści osiem milionów.", True)
|
||||
self.assertIn("word lost", reason)
|
||||
|
||||
def test_minus_lost(self):
|
||||
reason = normalizer._guard("Na dworze jest -12 stopni.",
|
||||
"Na dworze jest dwanaście stopni.", True)
|
||||
self.assertIn("minus", reason)
|
||||
|
||||
def test_repetition_loop(self):
|
||||
reason = normalizer._guard("Zapłaciłem 1234,56 zł.",
|
||||
"Zapłaciłem " + "łącznie " * 40 + "tysiąc.", False)
|
||||
self.assertTrue(reason)
|
||||
|
||||
def test_invalid_characters(self):
|
||||
self.assertIn("invalid", normalizer._guard("Dnia 15 września 1631 roku.",
|
||||
"Dnia " + "�" * 30, False))
|
||||
|
||||
def test_digits_left(self):
|
||||
self.assertIn("digits", normalizer._guard("Ma 25 lat.", "Ma 25 lat.", False))
|
||||
|
||||
def test_value_check_is_skipped_for_identifiers(self):
|
||||
""""LO 3842" is correctly read digit by digit, so demanding num2words'
|
||||
rendering of it would reject a good answer."""
|
||||
self.assertEqual(
|
||||
normalizer._guard("Lot LO 3842 jest opóźniony.",
|
||||
"Lot LO trzy osiem cztery dwa jest opóźniony.",
|
||||
False), "")
|
||||
|
||||
|
||||
class Fallback(unittest.TestCase):
|
||||
"""Degraded path: nominative only, bare integers only. It must still never
|
||||
lose a number or raise."""
|
||||
|
||||
def test_expands_multi_digit(self):
|
||||
out = normalizer._fallback("Wygrał 1247 złotych.")
|
||||
self.assertNotIn("1247", out)
|
||||
self.assertIn("tysiąc", out)
|
||||
|
||||
def test_keeps_single_digit(self):
|
||||
self.assertEqual(normalizer._fallback("Mam 3 koty."), "Mam 3 koty.")
|
||||
|
||||
def test_large_number_with_separators(self):
|
||||
out = normalizer._fallback("populacja 1 350 000 osób")
|
||||
self.assertNotIn("350", out)
|
||||
self.assertIn("milion", out)
|
||||
|
||||
def test_leaves_decimals_and_times_alone(self):
|
||||
for text in ("Około 3,5 metra.", "Spotkanie o 9:30.", "Radio 95,7 FM."):
|
||||
self.assertEqual(normalizer._fallback(text), text)
|
||||
|
||||
def test_number_followed_by_sentence_punctuation(self):
|
||||
"""A comma or full stop ends a number; it does not disqualify it. Real
|
||||
subtitles are full of "Przód 85, tył 75." and the brief's boundary set
|
||||
skipped every one of them."""
|
||||
out = normalizer._fallback("Przód 85, tył 75.")
|
||||
self.assertNotIn("85", out)
|
||||
self.assertNotIn("75", out)
|
||||
self.assertIn("osiemdziesiąt pięć", out)
|
||||
self.assertIn("siedemdziesiąt pięć", out)
|
||||
|
||||
def test_never_raises(self):
|
||||
for text in ("", "———", "9" * 400, "1/0", "0000"):
|
||||
with self.subTest(text=text[:20]):
|
||||
normalizer._fallback(text)
|
||||
|
||||
|
||||
class NormalizeContract(unittest.TestCase):
|
||||
"""Properties that hold on both paths — this is what the caller relies on."""
|
||||
|
||||
def test_no_digits_returns_input_unchanged(self):
|
||||
text = "Zwykłe zdanie bez liczb."
|
||||
self.assertEqual(normalizer.normalize(text), text)
|
||||
|
||||
def test_only_single_digits_skips_the_model(self):
|
||||
calls = []
|
||||
original = normalizer._model_normalize
|
||||
normalizer._model_normalize = lambda t: calls.append(t) or t
|
||||
try:
|
||||
self.assertEqual(normalizer.normalize("Mam 3 koty."), "Mam 3 koty.")
|
||||
finally:
|
||||
normalizer._model_normalize = original
|
||||
self.assertEqual(calls, [], "model must not be called for single digits")
|
||||
|
||||
def test_block_of_only_single_digits_is_returned_untouched(self):
|
||||
"""The cheap half of the brief's idea, kept: nothing to gain from
|
||||
sending it, a whole sentence of rewriting to risk."""
|
||||
for text in ("Mam 3 koty.", "Zostało 5 minut.", "Tylko 7!"):
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(normalizer.normalize(text), text)
|
||||
|
||||
def test_empty_and_odd_inputs(self):
|
||||
self.assertEqual(normalizer.normalize(""), "")
|
||||
self.assertEqual(normalizer.normalize(" "), " ")
|
||||
|
||||
def test_generation_error_falls_back(self):
|
||||
original = normalizer._model_normalize
|
||||
def boom(_):
|
||||
raise RuntimeError("CUDA out of memory")
|
||||
normalizer._model_normalize = boom
|
||||
try:
|
||||
out = normalizer.normalize("Wygrał 1247 złotych.")
|
||||
finally:
|
||||
normalizer._model_normalize = original
|
||||
self.assertNotIn("1247", out)
|
||||
self.assertIn("tysiąc", out)
|
||||
|
||||
|
||||
@needs_model
|
||||
class Neural(unittest.TestCase):
|
||||
"""The acceptance list from the build brief, against the real model."""
|
||||
|
||||
def test_multi_digit_expanded(self):
|
||||
out = normalizer.normalize("Wygrał 1247 złotych.")
|
||||
self.assertNotIn("1247", out)
|
||||
self.assertIn("tysiąc", out.lower())
|
||||
|
||||
def test_large_number(self):
|
||||
out = normalizer.normalize("Miasto ma populację 1 350 000 osób.")
|
||||
self.assertNotIn("350", out)
|
||||
self.assertIn("milion", out.lower())
|
||||
|
||||
def test_mixed_single_and_multi(self):
|
||||
"""A block sent for its multi-digit number gets its single digits
|
||||
normalized too — that is the point of dropping the placeholders, and
|
||||
it is what makes "3 maja" come out as "trzeciego maja"."""
|
||||
out = normalizer.normalize("Mam 3 koty i 25 psów.").lower()
|
||||
self.assertNotIn("3", out)
|
||||
self.assertIn("trzy", out)
|
||||
self.assertIn("dwadzieścia pięć", out)
|
||||
|
||||
def test_leg_is_reported(self):
|
||||
self.assertEqual(normalizer.normalize_detail("Mam 3 koty.")[1], "skipped")
|
||||
self.assertEqual(normalizer.normalize_detail("Jest dopiero 21.00.")[1],
|
||||
"complex")
|
||||
|
||||
def test_ordinal_date_is_fixed(self):
|
||||
"""The reason the placeholders had to go: left as a digit, the TTS
|
||||
reads "3 maja" as "trzy maja"."""
|
||||
out = normalizer.normalize("Urodził się 3 maja 1791 roku.").lower()
|
||||
self.assertIn("trzeciego maja", out)
|
||||
|
||||
def test_no_digits_leak(self):
|
||||
out = normalizer.normalize("Pędzi 300 kilometrów na godzinę.")
|
||||
self.assertFalse(DIGITS.search(out))
|
||||
|
||||
def test_case_declension(self):
|
||||
"""The whole reason for a neural normalizer: 'o 25 psach' is locative,
|
||||
so the numeral must not come out nominative."""
|
||||
out = normalizer.normalize("Mówimy o 25 psach.").lower()
|
||||
self.assertNotIn("25", out)
|
||||
self.assertNotIn("dwadzieścia pięć psach", out)
|
||||
|
||||
def test_time(self):
|
||||
out = normalizer.normalize("Spotkanie o 14:30.")
|
||||
self.assertFalse(DIGITS.search(out))
|
||||
|
||||
def test_offline(self):
|
||||
self.assertEqual(os.environ.get("HF_HUB_OFFLINE"), "1")
|
||||
self.assertFalse(DIGITS.search(normalizer.normalize("Ma 99 lat.")))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(f"model available: {HAVE_MODEL}", file=sys.stderr)
|
||||
unittest.main(verbosity=2)
|
||||
Executable
+74
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python3
|
||||
"""JSON-lines front end for normalizer.py.
|
||||
|
||||
The normalizer needs `transformers>=5`, which must not be installed into the
|
||||
ComfyUI/Higgs environment, so it lives in its own venv and is driven as a
|
||||
separate process. Loading a 1.1 GB model per subtitle block would be absurd, so
|
||||
this is a long-lived worker: the caller starts it once per run, streams blocks
|
||||
through it, and closes stdin when done.
|
||||
|
||||
Protocol — one JSON object per line, in both directions:
|
||||
|
||||
← {"ready": true, "mode": "model", "device": "cuda:0", "llm": true, …}
|
||||
→ {"id": 1, "text": "Wygrał 1247 złotych."}
|
||||
← {"id": 1, "text": "Wygrał tysiąc dwieście…", "changed": true,
|
||||
"leg": "plain", "guard": "", "error": null}
|
||||
→ {"quit": true} (or just close stdin)
|
||||
|
||||
`leg` is which model answered — "plain" for num2words plus the inflection
|
||||
model, "complex" for the fine-tune, "skipped" when the block needed nothing.
|
||||
`guard` names the check that rejected the model's answer, empty when it passed.
|
||||
|
||||
stdout carries the protocol and nothing else; every log line, warning and
|
||||
traceback goes to stderr. A request that fails is answered with the original
|
||||
text and an `error` field — the worker does not die on one bad block.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
|
||||
import normalizer
|
||||
|
||||
|
||||
def _send(obj) -> None:
|
||||
sys.stdout.write(json.dumps(obj, ensure_ascii=False) + "\n")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s",
|
||||
stream=sys.stderr)
|
||||
|
||||
state = normalizer.status() # loads the model, or reports why not
|
||||
state["llm"] = normalizer.load_llm() # the inflection leg is optional
|
||||
state["llm_dir"] = str(normalizer.LLM_DIR)
|
||||
_send({"ready": True, **state})
|
||||
|
||||
for line in sys.stdin:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
req = json.loads(line)
|
||||
except ValueError as exc:
|
||||
_send({"id": None, "text": None, "error": f"bad request: {exc}"})
|
||||
continue
|
||||
if req.get("quit"):
|
||||
break
|
||||
|
||||
text = req.get("text") or ""
|
||||
leg, guard = "", ""
|
||||
try:
|
||||
result, leg, guard = normalizer.normalize_detail(text)
|
||||
error = None
|
||||
except Exception as exc: # normalize_detail() is not meant to raise
|
||||
logging.exception("normalize_detail() raised")
|
||||
result, error = text, f"{type(exc).__name__}: {exc}"
|
||||
_send({"id": req.get("id"), "text": result, "changed": result != text,
|
||||
"leg": leg, "guard": guard, "error": error})
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+70
-3
@@ -30,7 +30,7 @@ Settings
|
||||
--------
|
||||
Defaults live in settings.ini next to the script:
|
||||
voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
|
||||
orig_volume, tts_volume, output_dir, comfy_url
|
||||
orig_volume, tts_volume, output_dir, dictionary, numbers, comfy_url
|
||||
|
||||
Precedence: command-line flag > settings.ini > built-in default.
|
||||
Use --settings FILE to select a different profile.
|
||||
@@ -59,10 +59,12 @@ Both external SRT files and embedded subtitle tracks go through the same cleanin
|
||||
|
||||
- HTML tags stripped: <i>, </i>, <b>, <font color="…">, etc.
|
||||
- ASS override codes stripped: {\an8}, {\pos(x,y)}, {\c&H…&}, etc.
|
||||
- ALL-CAPS blocks lowercased and converted to sentence case
|
||||
- ALL-CAPS blocks converted to Title Case (keeps proper nouns readable)
|
||||
- Sentence fragments starting with a lowercase letter are capitalised
|
||||
- Missing terminal punctuation added (… or . depending on context)
|
||||
- Dictionary replacements applied (see below)
|
||||
- Dialogue lines (prefixed with -) are split into separate clips with proportional timing
|
||||
- Numbers spelled out as Polish words, in the right case (see below)
|
||||
|
||||
With --embedded-subs the extracted subtitles are saved to the job folder as embedded_sub.srt
|
||||
so you can inspect them before or after the run.
|
||||
@@ -72,6 +74,63 @@ The final MKV preserves all subtitle tracks from the source video — text-based
|
||||
stream copy with no conversion.
|
||||
|
||||
|
||||
Replacement Dictionary
|
||||
----------------------
|
||||
dictionary.txt next to the script rewrites words and phrases before they reach
|
||||
the TTS engine — abbreviations, units, symbols, names the model mispronounces.
|
||||
One rule per line:
|
||||
|
||||
km/h = kilometrów na godzinę
|
||||
np. = na przykład
|
||||
/(\d+)\s*%/ = \1 procent
|
||||
|
||||
Matching is case-sensitive and whole-word only, so "np." leaves "znp." alone and
|
||||
"Joshua" leaves "joshua" alone. The replacement is inserted exactly as written.
|
||||
The first letter of each block is capitalised before these rules run, so add a
|
||||
capitalised variant for words that can start a subtitle.
|
||||
Longer patterns win over shorter ones regardless of the order they are written
|
||||
in, and replacement text is never rescanned, so rules cannot feed each other.
|
||||
A pattern wrapped in slashes is a regular expression, with \1 \2 … referring to
|
||||
its groups. Lines starting with # or ; are comments.
|
||||
|
||||
The result is visible in the job folder's tts_input.srt — check there if a rule
|
||||
is not doing what you expect. --regen and --rebuild work from that file, so they
|
||||
do not re-apply the dictionary; edit tts_input.srt directly, or rerun.
|
||||
|
||||
Use another list with --dictionary FILE (a relative name is looked for in the
|
||||
current directory first, then next to the script) or turn it off with
|
||||
--dictionary "". A missing file is not an error.
|
||||
|
||||
A second list can be merged in with --dictionary-extra FILE. The GUI writes one
|
||||
per queue — the "Queue dictionary" button — into the first queued video's job
|
||||
folder, for rules that belong to one batch rather than to Polish in general.
|
||||
|
||||
|
||||
Number Normalization
|
||||
--------------------
|
||||
The TTS misreads long digit strings and sometimes turns them into non-words, so
|
||||
after the dictionary every block containing a number is rewritten with the
|
||||
numbers spelled out in Polish — in the correct grammatical case, which is why
|
||||
this is a small neural model and not a lookup table:
|
||||
|
||||
Wygrał 1247 złotych. -> Wygrał tysiąc dwieście czterdzieści siedem złotych.
|
||||
Mówimy o 25 psach. -> Mówimy o dwudziestu pięciu psach.
|
||||
Mam 3 koty. -> Mam 3 koty. (single digits are read fine)
|
||||
|
||||
It runs in its own environment (transformers>=5, which must not be installed
|
||||
next to ComfyUI's pinned versions). One-time install:
|
||||
|
||||
normalizer/setup.sh # venv + dependencies
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model # gated weights, accept terms first
|
||||
|
||||
Terms: https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization
|
||||
Everything after that runs offline. See normalizer/README.md.
|
||||
|
||||
Turn it off with numbers = no under [normalize] in settings.ini, or with
|
||||
--no-normalize-numbers for one run. If the environment or the model is missing
|
||||
the run continues with the digits untouched — it is never fatal.
|
||||
|
||||
|
||||
Output Files
|
||||
------------
|
||||
Everything belonging to one video is kept together in a job folder named after
|
||||
@@ -130,6 +189,13 @@ All Options
|
||||
--max-tokens N Max audio tokens the model may generate per clip (default: 4096)
|
||||
Raise this if long clips are cut off mid-word.
|
||||
|
||||
--dictionary FILE Word/phrase replacements applied to the subtitles before
|
||||
TTS (default: dictionary.txt). Pass "" to disable.
|
||||
|
||||
--normalize-numbers Spell digits out as Polish words before TTS (default: on)
|
||||
--no-normalize-numbers
|
||||
Leave digits as digits. The TTS often misreads them.
|
||||
|
||||
--orig-volume 0-1 Volume of the original audio in the Polish lektor mix (default: 0.60)
|
||||
--tts-volume 0-2 Volume of the TTS voiceover in the Polish lektor mix (default: 1.8)
|
||||
Note: these affect Track 1 only. Track 0 is always the raw original.
|
||||
@@ -151,7 +217,8 @@ All Options
|
||||
Recovery and Fixing Bad Clips
|
||||
------------------------------
|
||||
|
||||
Resume a crashed run from block N (reuses existing clips 1..N-1):
|
||||
Resume a crashed run from block N (reuses existing clips 1..N-1 and the job
|
||||
folder's tts_input.srt — subtitles are not reprocessed):
|
||||
|
||||
python3 lektor_ai_comfy.py input/film.srt input/film.mkv --resume-from 47
|
||||
|
||||
|
||||
+42
-1
@@ -23,7 +23,7 @@ speed = 1.2
|
||||
# Sampling temperature. 1.0 makes the model occasionally recite the reference
|
||||
# transcript instead of the subtitle; 0.9 largely prevents that. Lower is more
|
||||
# deterministic but flatter.
|
||||
temperature = 0.6
|
||||
temperature = 0.8
|
||||
|
||||
# Ceiling on audio tokens per clip. Raise only if long clips are cut off
|
||||
# mid-word.
|
||||
@@ -55,6 +55,47 @@ tts_volume = 1.4
|
||||
# Relative paths are resolved from wherever you run the script.
|
||||
output_dir = output
|
||||
|
||||
# Word and phrase replacements applied to the subtitles before they are spoken,
|
||||
# e.g. "km/h = kilometrów na godzinę". One rule per line in that file; see the
|
||||
# comments at the top of it. A relative name is looked for in the current
|
||||
# directory first, then next to lektor_ai_comfy.py, so a single film can have
|
||||
# its own list. A missing file is not an error.
|
||||
dictionary = dictionary.txt
|
||||
|
||||
|
||||
[normalize]
|
||||
# Spell digits out as Polish words before the text reaches the TTS, in the right
|
||||
# grammatical case: "Wygrał 1247 złotych" -> "Wygrał tysiąc dwieście czterdzieści
|
||||
# siedem złotych". Higgs Audio misreads long digit strings and sometimes turns
|
||||
# them into non-words. Standalone single digits are read correctly and are left
|
||||
# alone. Runs last, on the text that goes into tts_input.srt. yes/no.
|
||||
numbers = yes
|
||||
|
||||
# Interpreter of the normalizer's own environment — it needs transformers>=5,
|
||||
# which must not be installed next to ComfyUI's pinned versions. Created by
|
||||
# normalizer/setup.sh. Relative paths are resolved next to lektor_ai_comfy.py.
|
||||
# If it is missing the run continues with the digits untouched.
|
||||
python = normalizer/venv/bin/python
|
||||
|
||||
# Second model, used for the grammar of plain integers. num2words supplies the
|
||||
# value (it cannot miscount) and this model puts the numeral into the right
|
||||
# case: "o 250000 uchodzcach" -> "o dwustu piecdziesieciu tysiacach uchodzcow".
|
||||
# The fine-tune alone gets that wrong, and sometimes loses the number itself.
|
||||
# Blocks with times, dates, decimals or identifiers skip it — it is worse than
|
||||
# the fine-tune on those. Leave empty to use the fine-tune for everything.
|
||||
llm = normalizer/models/PLLuM-4B-instruct-2512
|
||||
|
||||
# Seconds to wait for one block. On a GPU a block takes well under a second;
|
||||
# this only exists so a wedged worker cannot stall the whole run.
|
||||
timeout = 60
|
||||
|
||||
# Ask ComfyUI to unload Higgs before normalizing, so the normalizer gets the
|
||||
# GPU. Higgs stays resident between runs and leaves too little VRAM; a
|
||||
# normalizer that does not fit spills to RAM silently and runs about 5x slower.
|
||||
# ComfyUI reloads on the first clip of step 2 (20-30s, once per run). --regen
|
||||
# and --rebuild never normalize, so they never pay it. yes/no.
|
||||
free_comfy_vram = yes
|
||||
|
||||
|
||||
[comfy]
|
||||
# Address of the ComfyUI backend running Higgs Audio V3.
|
||||
|
||||
+4
-1
@@ -1,4 +1,7 @@
|
||||
#!/bin/bash
|
||||
source ~/miniforge3/etc/profile.d/conda.sh
|
||||
conda activate comfy-tts
|
||||
cd ~/comfy-tts-app/ComfyUI && python main.py --port 8189
|
||||
# exec: the shell is replaced by ComfyUI, so this script's PID is
|
||||
# ComfyUI's PID. Without it the launcher only ever sees a wrapper,
|
||||
# and killing that orphans the python still holding the GPU.
|
||||
cd ~/comfy-tts-app/ComfyUI && exec python main.py --port 8189
|
||||
|
||||
@@ -228,11 +228,18 @@ SETTING_LABELS = {
|
||||
"orig_volume": "sound volume",
|
||||
"tts_volume": "voiceover volume",
|
||||
"output_dir": "output folder",
|
||||
"dictionary": "replacement dictionary",
|
||||
"normalize_numbers": "spell out numbers",
|
||||
"normalizer_python": "normalizer interpreter",
|
||||
"normalizer_llm": "inflection model",
|
||||
"normalizer_timeout": "normalizer timeout (s)",
|
||||
"free_comfy_vram": "unload Higgs to normalize",
|
||||
"comfy_url": "ComfyUI URL",
|
||||
"comfy_autostart": "start ComfyUI if down",
|
||||
"comfy_start_command": "start script",
|
||||
}
|
||||
SECTION_TITLES = {"tts": "Speech", "mix": "Mix", "paths": "Paths", "comfy": "Backend"}
|
||||
SECTION_TITLES = {"tts": "Speech", "mix": "Mix", "paths": "Paths",
|
||||
"normalize": "Numbers", "comfy": "Backend"}
|
||||
|
||||
|
||||
def raw_settings(path):
|
||||
@@ -315,6 +322,100 @@ def save_settings(path, values):
|
||||
tmp.replace(path) # atomic, so a crash cannot truncate it
|
||||
|
||||
|
||||
class QueueDictionaryDialog(tk.Toplevel):
|
||||
"""Plain text editor for the queue's own replacement rules.
|
||||
|
||||
Deliberately a text box rather than a table: the file format is one
|
||||
`pattern = replacement` per line, and typing that is faster than filling in
|
||||
two columns. The same parser reads it as the shared dictionary, so regex
|
||||
rules and `#` comments work here too.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
super().__init__(app)
|
||||
self.app = app
|
||||
F = app.font_family
|
||||
self.title("Queue dictionary")
|
||||
self.configure(bg=BG)
|
||||
self.transient(app)
|
||||
|
||||
path = app.queue_dictionary_path()
|
||||
where = str(path) if path else "(add a video to the queue first)"
|
||||
|
||||
tk.Label(self, text="Replacements applied to every video in this queue,",
|
||||
bg=BG, fg=FG, font=(F, 10)).pack(anchor="w", padx=22, pady=(16, 0))
|
||||
tk.Label(self, text="in addition to dictionary.txt. One rule per line, "
|
||||
"case-sensitive, whole words:",
|
||||
bg=BG, fg=FG, font=(F, 10)).pack(anchor="w", padx=22)
|
||||
tk.Label(self, text=" Joshua = Dżoszua", bg=BG, fg="#8fb8d6",
|
||||
font=(F, 9)).pack(anchor="w", padx=22, pady=(6, 0))
|
||||
tk.Label(self, text=where, bg=BG, fg=MUTED, font=(F, 8)
|
||||
).pack(anchor="w", padx=22, pady=(8, 4))
|
||||
|
||||
frame = tk.Frame(self, bg=BG)
|
||||
frame.pack(fill="both", expand=True, padx=22)
|
||||
bar = tk.Scrollbar(frame)
|
||||
bar.pack(side="right", fill="y")
|
||||
self.text = tk.Text(frame, width=64, height=16, bg=FIELD, fg=FIELD_FG,
|
||||
insertbackground=FIELD_FG, relief="flat", bd=0,
|
||||
font=(F, 10), yscrollcommand=bar.set, undo=True)
|
||||
self.text.pack(side="left", fill="both", expand=True)
|
||||
bar.config(command=self.text.yview)
|
||||
|
||||
if path is not None and path.is_file():
|
||||
try:
|
||||
self.text.insert("1.0", path.read_text(encoding="utf-8"))
|
||||
except OSError as exc:
|
||||
messagebox.showerror("Queue dictionary", str(exc), parent=self)
|
||||
|
||||
self.status = tk.Label(self, text="", bg=BG, fg=MUTED, font=(F, 8))
|
||||
self.status.pack(anchor="w", padx=22, pady=(6, 0))
|
||||
|
||||
buttons = tk.Frame(self, bg=BG)
|
||||
buttons.pack(fill="x", padx=22, pady=16)
|
||||
RoundButton(buttons, "Close", self.destroy, bg="#6f6f6f", fg="#f0f0f0",
|
||||
hover="#828282", font=(F, 9, "bold"), padx=16, pady=6
|
||||
).pack(side="right")
|
||||
RoundButton(buttons, "Save", self._save, font=(F, 9, "bold"),
|
||||
padx=22, pady=6).pack(side="right", padx=(0, 8))
|
||||
RoundButton(buttons, "Clear", self._clear, bg="#4a4a4a", fg=MUTED,
|
||||
hover="#5c5c5c", font=(F, 9), padx=14, pady=6).pack(side="left")
|
||||
|
||||
self.bind("<Escape>", lambda _: self.destroy())
|
||||
self.update_idletasks()
|
||||
x = app.winfo_rootx() + (app.winfo_width() - self.winfo_width()) // 2
|
||||
self.geometry(f"+{max(0, x)}+{app.winfo_rooty() + 60}")
|
||||
self.grab_set()
|
||||
|
||||
def _clear(self):
|
||||
"""Empties the box only. Nothing is written until Save, so a misclick
|
||||
costs a Ctrl+Z rather than the file."""
|
||||
self.text.delete("1.0", "end")
|
||||
self.status.config(text="Cleared — press Save to write the empty file.")
|
||||
|
||||
def _save(self):
|
||||
path = self.app.queue_dictionary_path()
|
||||
if path is None:
|
||||
messagebox.showinfo(
|
||||
"Queue dictionary",
|
||||
"Add a video to the queue first — the file is written to the "
|
||||
"first video's job folder.", parent=self)
|
||||
return
|
||||
body = self.text.get("1.0", "end").rstrip() + "\n"
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(body, encoding="utf-8")
|
||||
except OSError as exc:
|
||||
messagebox.showerror("Queue dictionary", str(exc), parent=self)
|
||||
return
|
||||
|
||||
# Report what the pipeline will actually make of it, so a typo shows up
|
||||
# now rather than as a rule that silently does nothing during the run.
|
||||
loaded = core.Dictionary.load(path)
|
||||
self.status.config(text=f"Saved — {len(loaded)} rule(s) parsed.")
|
||||
self.app.log(f"Queue dictionary saved to {path} ({len(loaded)} rules).")
|
||||
|
||||
|
||||
class SettingsDialog(tk.Toplevel):
|
||||
"""Editor for settings.ini — the defaults new job blocks start from."""
|
||||
|
||||
@@ -342,7 +443,7 @@ class SettingsDialog(tk.Toplevel):
|
||||
if core.VOICE_PROMPTS_DIR.is_dir() else []
|
||||
|
||||
row = 0
|
||||
for section in ("tts", "mix", "paths", "comfy"):
|
||||
for section in ("tts", "mix", "paths", "normalize", "comfy"):
|
||||
keys = [k for k, spec in core._DEFAULTS.items() if spec[0] == section]
|
||||
if not keys:
|
||||
continue
|
||||
@@ -774,6 +875,16 @@ class JobBlock(tk.Frame):
|
||||
|
||||
cmd += self.common_args()
|
||||
|
||||
# The whole queue shares one extra dictionary, kept with the first
|
||||
# video. --regen and --rebuild work purely from tts_input.srt, so the
|
||||
# flag would do nothing there. A resume normally reuses that file too,
|
||||
# but falls back to parsing when it is missing — and then the queue's
|
||||
# rules must still apply, so it is passed.
|
||||
if mode in ("generate", "resume"):
|
||||
extra = self.app.queue_dictionary_path()
|
||||
if extra is not None and extra.is_file():
|
||||
cmd += ["--dictionary-extra", str(extra)]
|
||||
|
||||
if mode == "rebuild":
|
||||
cmd.append("--rebuild")
|
||||
elif mode == "regen":
|
||||
@@ -985,6 +1096,14 @@ class LektorUI(tk.Tk):
|
||||
RoundButton(top, "Settings", self.open_settings, bg="#3a3a3a", fg=MUTED,
|
||||
hover="#4a4a4a", font=(F, 9), padx=13, pady=6
|
||||
).pack(side="right")
|
||||
RoundButton(top, "Queue dictionary", self.open_queue_dictionary,
|
||||
bg="#3a3a3a", fg=MUTED, hover="#4a4a4a", font=(F, 9),
|
||||
padx=13, pady=6).pack(side="right", padx=(0, 8))
|
||||
self.comfy_btn = RoundButton(top, "ComfyUI", self.toggle_comfy,
|
||||
bg="#3a3a3a", fg=MUTED, hover="#4a4a4a",
|
||||
font=(F, 9), padx=13, pady=6)
|
||||
self.comfy_btn.pack(side="right", padx=(0, 8))
|
||||
self.after(500, self._refresh_comfy_button)
|
||||
|
||||
# scrollable list of job blocks
|
||||
mid = tk.Frame(self, bg=BG)
|
||||
@@ -1053,6 +1172,89 @@ class LektorUI(tk.Tk):
|
||||
self.stop_btn.pack(side="right")
|
||||
self.stop_btn.set_enabled(False)
|
||||
|
||||
# ── queue dictionary ────────────────────────────────────────────────────
|
||||
#
|
||||
# Replacements that apply to this batch only — a character's name the model
|
||||
# mangles, a recurring mistranslation — without editing the shared
|
||||
# dictionary.txt, which holds the rules that are true of Polish generally.
|
||||
# It lives in the first queued video's job folder so it sits with the
|
||||
# material it belongs to, and every job in the queue is passed it.
|
||||
|
||||
QUEUE_DICT_NAME = "queue_dictionary.txt"
|
||||
|
||||
def queue_dictionary_path(self):
|
||||
"""Path in the first queued video's job folder, or None if the queue is
|
||||
empty — there is nowhere to put it until a video is chosen."""
|
||||
for blk in self.blocks:
|
||||
folder = blk.job_folder()
|
||||
if folder is not None:
|
||||
return folder / self.QUEUE_DICT_NAME
|
||||
return None
|
||||
|
||||
def open_queue_dictionary(self):
|
||||
QueueDictionaryDialog(self)
|
||||
|
||||
# ── ComfyUI lifecycle ───────────────────────────────────────────────────
|
||||
#
|
||||
# The backend is normally started by hand in a terminal and left running,
|
||||
# which is fine until it crashes or gets suspended: it then holds the GPU
|
||||
# while answering nothing, and a queue started overnight goes nowhere. All
|
||||
# of the work is in lektor_ai_comfy so the terminal script and this button
|
||||
# behave identically.
|
||||
|
||||
def _comfy_state(self):
|
||||
"""(running, healthy) — separate, because a backend can hold the port
|
||||
and the GPU while no longer answering."""
|
||||
pid = core.comfy_pid()
|
||||
if pid is None:
|
||||
return False, False
|
||||
try:
|
||||
core._comfy_get("/system_stats")
|
||||
return True, True
|
||||
except Exception:
|
||||
return True, False
|
||||
|
||||
def _refresh_comfy_button(self):
|
||||
running, healthy = self._comfy_state()
|
||||
if healthy:
|
||||
self.comfy_btn.set_text("ComfyUI ●")
|
||||
elif running:
|
||||
self.comfy_btn.set_text("ComfyUI ◍") # up but not answering
|
||||
else:
|
||||
self.comfy_btn.set_text("Start ComfyUI")
|
||||
self.after(5000, self._refresh_comfy_button)
|
||||
|
||||
def toggle_comfy(self):
|
||||
"""Start it, or restart it when it is up but wedged."""
|
||||
core.COMFY_URL = self.defaults["comfy_url"]
|
||||
running, healthy = self._comfy_state()
|
||||
if healthy:
|
||||
if not messagebox.askyesno(
|
||||
"ComfyUI", "ComfyUI is running and answering.\n\n"
|
||||
"Restart it anyway?", parent=self):
|
||||
return
|
||||
self.comfy_btn.set_enabled(False)
|
||||
threading.Thread(target=self._comfy_worker, args=(running,),
|
||||
daemon=True).start()
|
||||
|
||||
def _comfy_worker(self, running):
|
||||
try:
|
||||
if running:
|
||||
self.log("Stopping ComfyUI…")
|
||||
core.stop_comfy_backend()
|
||||
self.log("Starting ComfyUI…")
|
||||
if not core.start_comfy_backend(core.COMFY_START_SCRIPT):
|
||||
self.log("Could not start ComfyUI — see the terminal window.")
|
||||
return
|
||||
if core.wait_for_backend(180.0):
|
||||
self.log("ComfyUI is up.")
|
||||
else:
|
||||
self.log(f"ComfyUI did not answer within 180s. "
|
||||
f"Its output is in {core.COMFY_LOG}.")
|
||||
finally:
|
||||
self.after(0, lambda: self.comfy_btn.set_enabled(True))
|
||||
self.after(0, self._refresh_comfy_button)
|
||||
|
||||
def open_settings(self):
|
||||
SettingsDialog(self)
|
||||
|
||||
@@ -1322,8 +1524,27 @@ class LektorUI(tk.Tk):
|
||||
return
|
||||
if self.proc and self.proc.poll() is None:
|
||||
self.proc.kill()
|
||||
|
||||
# Leaving ComfyUI behind means ~8 GB of VRAM held by a backend nobody is
|
||||
# driving. It is stopped by port, so one started by hand in a terminal
|
||||
# goes too — which is the case that actually strands the GPU.
|
||||
self._stop_comfy_on_exit()
|
||||
self.destroy()
|
||||
|
||||
def _stop_comfy_on_exit(self):
|
||||
"""Best effort: quitting must not be blocked by a backend that will not
|
||||
die, so failures are reported and swallowed."""
|
||||
core.COMFY_URL = self.defaults["comfy_url"]
|
||||
try:
|
||||
if core.comfy_pid() is None:
|
||||
return
|
||||
self.log("Stopping ComfyUI…")
|
||||
self.update_idletasks() # let the message paint before we block
|
||||
if not core.stop_comfy_backend():
|
||||
self.log("ComfyUI did not stop — check for a leftover process.")
|
||||
except Exception as exc: # noqa: BLE001
|
||||
self.log(f"Could not stop ComfyUI: {exc}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if not CLI.is_file():
|
||||
|
||||
Reference in New Issue
Block a user