Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,179 @@
|
||||
# Polish number normalizer
|
||||
|
||||
Rewrites digits as spoken Polish words before the text reaches the TTS:
|
||||
|
||||
```
|
||||
Wygrał 1247 złotych. → Wygrał tysiąc dwieście czterdzieści siedem złotych.
|
||||
Mówimy o 25 psach. → Mówimy o dwudziestu pięciu psach.
|
||||
Mam 3 koty. → Mam 3 koty. (nothing worth sending)
|
||||
Brakuje nam 15000 podpisów. → Brakuje nam piętnastu tysięcy podpisów.
|
||||
```
|
||||
|
||||
Higgs Audio v3 mispronounces raw digits — especially long ones — and sometimes
|
||||
emits garbled non-words. A lookup table cannot fix that, because Polish numerals
|
||||
decline by case, gender and the noun they govern (`25 psów` / `o 25 psach` /
|
||||
`z 25 psami`), so this uses a small fine-tuned model.
|
||||
|
||||
It runs in **its own venv**, separate from ComfyUI: the models require
|
||||
`transformers >= 5`, and forcing that on the working TTS install would risk its
|
||||
pinned dependencies. `lektor_ai_comfy.py` drives it as a subprocess, and asks
|
||||
ComfyUI to unload Higgs first so there is VRAM to work with.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
normalizer/setup.sh # venv + torch (cu128) + transformers + num2words
|
||||
```
|
||||
|
||||
Then the model, which is **gated** — accept the terms at
|
||||
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> with your Hugging
|
||||
Face account first, then:
|
||||
|
||||
```bash
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model
|
||||
```
|
||||
|
||||
The inflection model is separate and **not gated**:
|
||||
|
||||
```bash
|
||||
normalizer/venv/bin/hf download CYFRAGOVPL/PLLuM-4B-instruct-2512 \
|
||||
--local-dir normalizer/models/PLLuM-4B-instruct-2512
|
||||
```
|
||||
|
||||
Those are the only steps that need the network. Everything afterwards runs
|
||||
offline (`HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`, both set by the caller).
|
||||
Without the inflection model every block goes to the fine-tune, which is what
|
||||
the pipeline did before it existed.
|
||||
|
||||
Without the model the component still works, on a `num2words` fallback that
|
||||
handles bare integers in the nominative case only. It says so in the log, and
|
||||
`lektor_ai_comfy.py` prints a warning for the run.
|
||||
|
||||
## Check it
|
||||
|
||||
```bash
|
||||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||||
normalizer/venv/bin/python normalizer/normalizer.py "Wygrał 1247 złotych."
|
||||
normalizer/venv/bin/python -m unittest discover -s normalizer/tests -v
|
||||
```
|
||||
|
||||
The tests assert on properties (`no 1247 left`, `contains tysiąc`) rather than
|
||||
exact strings, because several case variants are correct. The ones that need the
|
||||
model skip themselves when it is not downloaded.
|
||||
|
||||
## What it does
|
||||
|
||||
1. **Skip what needs nothing.** A block whose only numbers are standalone single
|
||||
digits is never sent: the TTS says "pięć minut" for "5 minut" by itself, so
|
||||
there is nothing to gain and a whole sentence of rewriting to risk.
|
||||
2. **Route.** Two models with opposite weaknesses, measured on 40 real subtitle
|
||||
lines and a 32-case battery:
|
||||
|
||||
| | good at | bad at |
|
||||
|---|---|---|
|
||||
| fine-tune (0.6B) | `6:05`, `P10`, `LO 3842`, `95,7`, dates | loses a large number's value while declining it |
|
||||
| num2words + LLM | the value, always; the case | formats — a flight number becomes a quantity |
|
||||
|
||||
So a block goes to the second leg only when **every** number in it is a plain
|
||||
integer — 74% of numbered blocks on the sample. Everything else goes to the
|
||||
fine-tune.
|
||||
3. **Generate.** The fine-tune gets the text as-is. The other leg gets text with
|
||||
no digits at all — num2words has already produced correct nominative words —
|
||||
so its only job is inflection, and it cannot get a value wrong because it
|
||||
never reads a number. Greedy decoding, digit tokens banned, either way.
|
||||
4. **Guard.** The answer is checked before it is used (below). A block that
|
||||
fails falls back to num2words: dull, nominative, never wrong about the value.
|
||||
|
||||
## The guards
|
||||
|
||||
Both models can produce output worse than doing nothing. Each of these fired on
|
||||
a real failure during development:
|
||||
|
||||
| check | the failure it caught |
|
||||
|---|---|
|
||||
| runaway length | a 10-word line came back as 200 words |
|
||||
| repetition loop | `łącznie łącznie łącznie…` to the token ceiling |
|
||||
| invalid characters | a wall of `�` replacement bytes |
|
||||
| digits left | an un-normalized number slipped through |
|
||||
| minus sign lost | `-12 stopni` → *dwanaście stopni*, inverting a temperature |
|
||||
| numbers changed | `Z 8250 osobami` → *osiem tysięcy **dwustu pięciu*** (8205), and `20 lat` → *dwadzieścia **pięć** lat* (25) |
|
||||
| word lost | `38 000 000 mieszkańców` → *trzydzieści osiem milionów* |
|
||||
|
||||
The value check compares against what num2words would say, **in both
|
||||
directions** — a missing numeral and an added one are equally wrong, and a
|
||||
subset check passed `dwadzieścia` → `dwadzieścia pięć` while turning 20 into 25.
|
||||
Numerals are matched through a written-out canonical map rather than string
|
||||
similarity — `dwieście`
|
||||
and `dwustu` are one number and must match, `trzy` and `trzysta` differ by a
|
||||
hundred and must not, and ordinals like `piąty` count as `pięć`. Consonant
|
||||
folding was tried first and rejected good output.
|
||||
|
||||
It is skipped for identifiers: `LO 3842` is correctly read digit by digit, and
|
||||
demanding num2words' rendering would reject a right answer.
|
||||
|
||||
### Deviations from the build brief, and why
|
||||
|
||||
- **Single digits are no longer shielded.** The brief asks for standalone
|
||||
one-digit numbers to be hidden behind `KEEPNUM…` placeholders. Measured, that
|
||||
scored *worse* — 6/9 against 7/9 on the model card's own examples — and it
|
||||
produced an outright pronunciation error, because `3 maja` left as a digit is
|
||||
read *"trzy maja"* where Polish wants *"trzeciego maja"*. It also bought
|
||||
nothing on length: the TTS speaks `5` as *pięć* either way. What survives is
|
||||
the cheaper half — a block containing **only** standalone single digits is
|
||||
never sent at all.
|
||||
- **`KEEP_RE` has extra guards.** The brief's version protects the leading `1`
|
||||
of `1 350 000` as if it were standalone, and its boundary set `(?![\d.,:/-])`
|
||||
rejects any number followed by a comma or full stop — on real subtitles that
|
||||
skipped most of them (`Przód 85, tył 75.`). Numeric punctuation now only
|
||||
continues a number when a digit follows it.
|
||||
- **Two models, not one.** The brief scopes a single normalizer. The 0.6B
|
||||
fine-tune performs to its advertised 85.7%, but its tail includes losing a
|
||||
number's value while declining it, which no amount of prompting fixes. See
|
||||
the routing table above.
|
||||
- **The model loads lazily, not at import.** Same "load once, reuse" effect, but
|
||||
`--help`, `--status` and the non-neural half of the suite do not pay for a
|
||||
1.1 GB load they never use.
|
||||
- **`num2words` errors are caught broadly.** It raises `KeyError`, not
|
||||
`ValueError`, past its largest scale word.
|
||||
|
||||
### Known limits
|
||||
|
||||
Measured, and not fixable at this layer:
|
||||
|
||||
| input | output | |
|
||||
|---|---|---|
|
||||
| `Mieszka pod 12/45.` | *sto czternaście łamane przez trzy* | an address has no reliable reading without context |
|
||||
| `Zadzwoń pod 601 234 567.` | read as one 9-digit number | a phone number is indistinguishable from a grouped number |
|
||||
| `Zyskują 0,3 sekundy` | *trzy setne* | should be *dziesiąte*; the value guard cannot see inside a decimal |
|
||||
| `To dzieło z XIX wieku.` | unchanged | Roman numerals are not handled — use `dictionary.txt` |
|
||||
| `Prędkość 0,75 macha.` | unchanged | the model hallucinated a unit conversion; the guard caught it and fell back |
|
||||
|
||||
## Files
|
||||
|
||||
| File | |
|
||||
|---|---|
|
||||
| `normalizer.py` | The component: routing, both backends, guards, `normalize()` |
|
||||
| `worker.py` | JSON-lines service `lektor_ai_comfy.py` talks to (models loaded once per run) |
|
||||
| `bench.py` | Compares models on the same battery — `--compare a.json b.json` |
|
||||
| `setup.sh` | venv, dependencies, one-time gated model download |
|
||||
| `tests/` | Routing, numerals, guards, and the brief's acceptance checks |
|
||||
| `model/`, `models/`, `venv/` | Weights and environment — not in git |
|
||||
|
||||
## Interface
|
||||
|
||||
```python
|
||||
from normalizer import normalize, needs_normalization
|
||||
|
||||
needs_normalization("Mam 3 koty.") # False — nothing worth sending
|
||||
normalize("Wygrał 1247 złotych.") # never raises, never returns None
|
||||
```
|
||||
|
||||
`worker.py` speaks one JSON object per line on stdin/stdout, with logs on
|
||||
stderr:
|
||||
|
||||
```
|
||||
← {"ready": true, "mode": "model", "device": "cuda:0", "llm": true}
|
||||
→ {"id": 1, "text": "Wygrał 1247 złotych."}
|
||||
← {"id": 1, "text": "Wygrał tysiąc dwieście…", "changed": true,
|
||||
"leg": "complex", "guard": "", "error": null}
|
||||
```
|
||||
Reference in New Issue
Block a user