lukas/lektor-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control

Browse Source 
Text processing before TTS:

- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
  into one alternation applied in a single pass, so replacements never chain
  and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
  folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
  destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
  the normalizer to "correct" the name. Paired clips in output/clips/all-caps
  showed the TTS reads both casings the same.

normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:

- Two backends. Plain integers go to num2words for the value and a general
  Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
  fine-tune. Measured on 40 real subtitle lines, each is clearly better than
  the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
  leftover digits, a lost minus sign, a changed number, a lost word. A failure
  falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
  in VRAM is spilled to RAM silently at 5x the cost.

Backend lifecycle:

- ComfyUI is found by port through /proc, so a backend started by hand is
  managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
  outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
  seed (the aborts are seed-dependent, not text-dependent), then skipped and
  reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.

--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-12 21:55:31 +02:00
co-authored by Claude Opus 5
parent 7e2cee02e5
commit e201aadc3e
21 changed files with 3759 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/README.md
+13 -4
View File
@@ -17,6 +17,7 @@ stream-copied; only the new voiceover track is encoded.
| [usage.md](usage.md) | Every CLI flag, the working modes, and practical recipes |
| [architecture.md](architecture.md) | How the pipeline works internally, end to end |
| [troubleshooting.md](troubleshooting.md) | Diagnosed failure modes and their fixes |
| [../normalizer/README.md](../normalizer/README.md) | The Polish number normalizer, its own component |
## Quick start
@@ -35,10 +36,11 @@ python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs
The result is `output/film_lektor.mkv` with two audio tracks — **Original** and
**Polish** — plus every subtitle track from the source.
## The two halves of this project
## The three parts of this project
The pieces are deliberately decoupled and it helps to keep them apart when
diagnosing anything:
The pieces are deliberately decoupled — each in its own environment, talking
over a process boundary — and it helps to keep them apart when diagnosing
anything:
**ComfyUI + Higgs Audio V3** — a local HTTP service on port 8189 that turns text
into speech. Heavy: ~22 GB on disk, needs an NVIDIA GPU, lives in its own conda
@@ -49,8 +51,15 @@ ComfyUI over HTTP, and does all timing and audio/video assembly with ffmpeg. It
is pure Python standard library, runs on the system Python, and needs no conda
environment or pip packages of its own.
**`normalizer/`** — an optional component that spells digits out as Polish words
before they reach the TTS. Its own venv (~6.9 GB, mostly CUDA libraries) and
its own 1.1 GB model, because it needs `transformers >= 5` and the ComfyUI
environment must not be disturbed. Driven as a subprocess for one stage of each
run, then closed.
So a TTS problem (bad pronunciation, hallucination, mumbling) belongs to the
first half, and a timing, mixing, or muxing problem belongs to the second.
first part, a timing, mixing or muxing problem to the second, and a number read
in the wrong case to the third.
## Conventions in these docs
docs/architecture.md
+96 -2
View File
@@ -37,23 +37,117 @@ must repeat the same `--start`/`--end`.
and `<...>` HTML tags (`<i>`, `<font …>`).
2. **Detect dialogue.** Lines beginning `- ` mean two speakers and are kept
separate; ordinary wrapped lines are joined with a space.
3. **Sentence-case ALL-CAPS blocks**, which the model would otherwise shout.
3. **Title-case ALL-CAPS blocks**, which the model would otherwise shout.
Every word is capitalised rather than lowercasing all but the first letter,
because the obvious version also lowercases the proper nouns inside a
caption: `NASA – MARGO MADISON` became `Nasa – margo madison`, which is wrong
to read aloud and invites the number normalizer to "correct" the name (it
produced `marga`). A/B clips showed the TTS reads both casings the same, so
keeping the names costs nothing.
4. **Capitalise a lowercase first letter**, since a fragment continuing from the
previous subtitle confuses the model.
5. **Add terminal punctuation** so the model produces a natural falling tail.
5. **Apply the replacement dictionary** — `km/h` → `kilometrów na godzinę` and
anything else the model reads out wrong (see below).
6. **Add terminal punctuation** so the model produces a natural falling tail.
Step 1 must come first. `<i>ALL CAPS</i>` looks mixed-case while the tag is
still present — the `i` in `<i>` is a lowercase letter — so the caps detection
in step 3 would silently miss it.
Step 5 must come after step 3 for the same reason in reverse: a lowercase
replacement dropped into `JEDZIE 200 KM/H` would make the block look mixed-case
and the shouting would survive. It comes before step 6 so a rule can supply
punctuation of its own.
### The replacement dictionary
`Dictionary.load()` reads `dictionary.txt` (`[paths] dictionary`, or
`--dictionary`) into `pattern = replacement` rules — literal and
case-insensitive, or a regular expression when the pattern is wrapped in
slashes. Every rule is compiled into **one alternation** applied in a single
`re.sub` pass, which buys three properties worth keeping:
- Replacement text is never rescanned, so rules cannot chain or loop.
- Literals are ordered longest-first inside the alternation, so `km/h` beats a
plain `km` rule regardless of the order in the file.
- Rules cost one pass over the text no matter how many there are.
Literal patterns get `(?<!\w)` / `(?!\w)` guards only on edges that are word
characters, which is what lets `km/h` and `np.` match on their own while `znp.`
does not — a guard on a `/` or `.` edge would stop them matching at all.
Matching is case-sensitive and the replacement is verbatim. The alternative —
match loosely, then infer how to re-case the replacement — was tried and
removed: it silently rewrote `NASA` as `Nasa`, and a rule that fires on a case
you did not write is hard to reason about. The cost is that the first letter of
a block is capitalised before the dictionary runs, so a lowercase rule cannot
fire there; on the seven subtitle files here that changed nothing, but it is why
a word that can open a subtitle needs both forms.
A missing file, a line without a separator and an invalid regex are all
warnings, never fatal — the same rule as `settings.ini`, for the same reason: a
typo should not cost a long run.
`split_dialogue_blocks()` then divides a two-speaker block into one clip per
line, apportioning the block's duration by character count.
### Number normalization (last stage)
`normalize_block_numbers()` rewrites digits as Polish words — `1247` →
`tysiąc dwieście czterdzieści siedem` — because Higgs Audio misreads long digit
strings and sometimes emits garbled non-words. It is **last** for two reasons:
the dictionary may have produced the words a numeral has to agree with (`16%` →
`16 procent` → `szesnaście procent`), and dialogue splitting decides which clip
a number ends up in.
The model that does it needs `transformers>=5`, which cannot be installed beside
ComfyUI's pinned dependencies, so it lives in `normalizer/` with its own venv and
runs as a **separate process**. `NumberNormalizer` is the client half:
- One worker per run, not per block — the 1.1 GB load dominates everything else.
It is started only if some block actually contains a number worth expanding,
and closed before step 2, so it is not holding VRAM while Higgs generates.
- **ComfyUI is asked to unload first** (`free_comfy_vram`). Higgs stays resident
between runs and leaves too little VRAM; a normalizer that does not fit is
spilled to RAM by accelerate *silently*, at about 5× the time per block. A
`POST /free` is cheaper than a restart — the server stays warm — and the wait
is on `torch_vram_total` in `/system_stats` rather than a fixed sleep, because
the flag is consumed by ComfyUI's worker loop rather than by the request.
- JSON lines over stdin/stdout, strictly one request per response. stdout is the
protocol; logs go to stderr and are kept in a 40-line ring for diagnostics.
- Reads are on a queue fed by a reader thread, which is what makes a per-block
timeout possible at all — a wedged worker cannot block the run.
- **Nothing here is fatal.** Missing venv, dead worker, timeout, out-of-order
reply: each prints a warning, disables the stage, and lets the digits through
untouched. A number read badly is a bad clip; a crash here is a lost film.
- **Two models behind one interface.** Plain integers go to num2words for the
value and a general Polish LLM for the case; anything with a time, date,
decimal or identifier goes to the 0.6B fine-tune. Measured on 40 real subtitle
lines, each is clearly better than the other on its half and clearly worse on
the other's. The worker reports which leg answered, and the run prints the
split.
- **Guards, not trust.** Both models can emit something worse than nothing — a
repetition loop to the token ceiling, a dropped minus that inverts a
temperature, a vanished noun, a number changed from 8250 to 8205. Each answer
is checked against the input and falls back to num2words when it fails.
`needs_number_normalization()` duplicates the worker's own selection rule so
blocks that need nothing never cross the process boundary. The two must stay in
step; the worker's copy is the authoritative one, and a test in the normalizer
suite imports the caller and compares both on a shared sample.
The result is written to `tts_input.srt` in that video's job folder
(`output/clips/<video name>/`). It is the source of truth for block
numbering: block *N* there is `clip_NNNN.flac` beside it and the
argument to `--regen N`.
Which is why `--regen`, `--rebuild` **and `--resume-from`** all read that file
rather than the original SRT. Everything upstream of it — the dictionary, the
normalizer, dialogue splitting — can produce different text than it did on the
first run, and any difference breaks the pairing with the clips already on disk.
Step 1 is skipped entirely in those modes.
## 2. Speech generation
For each block the script POSTs a workflow graph to `/prompt`, polls
docs/installation.md
+28 -1
View File
@@ -184,6 +184,27 @@ conda install -n comfy-tts -c conda-forge ffmpeg
Note that this puts ffmpeg inside the conda env, so the script must then be run
with that env active.
## Number normalizer (optional, recommended)
Digits reach the TTS as words rather than figures, in the correct Polish case.
It has its own venv on purpose — the model needs `transformers >= 5`, which must
not be forced on the ComfyUI environment installed above.
```bash
cd ~/Programy/lektor_ai
normalizer/setup.sh # venv + torch (cu128) + transformers
HF_TOKEN=hf_... normalizer/setup.sh --model # the weights, ~1.1 GB
```
The weights are gated: accept the terms at
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> while signed in,
then take a token from <https://huggingface.co/settings/tokens>. This is the only
step that touches the network; runs are offline afterwards.
Skipping this is fine — the pipeline falls back to `num2words` (nominative case
only) and says so. Turn the whole stage off with `numbers = no` under
`[normalize]` in `settings.ini`. See [normalizer/README.md](../normalizer/README.md).
## Verifying
```bash
@@ -198,7 +219,11 @@ python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
# 3. Model present
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors
# 4. End-to-end on 60 seconds of a real file
# 4. Number normalizer, if installed
normalizer/venv/bin/python normalizer/normalizer.py --status
# -> "mode": "model" ("fallback" means the weights are missing)
# 5. End-to-end on 60 seconds of a real file
cd ~/Programy/lektor_ai
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00
```
@@ -213,5 +238,7 @@ Nothing was installed system-wide, so removal is just deleting directories:
```bash
conda env remove -n comfy-tts
rm -rf ~/comfy-tts-app
rm -rf ~/Programy/lektor_ai/normalizer/venv # number normalizer env
rm -rf ~/Programy/lektor_ai/normalizer/model # and its weights
rm -rf ~/miniforge3 # only if nothing else uses it
```
docs/requirements.md
+32 -4
View File
@@ -6,7 +6,7 @@
|---|---|---|
| GPU | NVIDIA, 12 GB VRAM | Developed on an RTX 4070 Ti SUPER (16 GB) |
| Driver | 525+ | Needed by the CUDA 12.8 wheels |
| Disk | ~35 GB free | ~22 GB installed, plus working room for output |
| Disk | ~35 GB free | ~22 GB installed (+8 GB with the number normalizer) |
| RAM | 16 GB | The final mux briefly peaks around 2.6 GB |
The CUDA **toolkit** is not required. The PyTorch cu128 wheels bundle their own
@@ -25,6 +25,9 @@ feature-length film.
└── models/TTS 8.8 GB Higgs Audio V3 weights
------
21.7 GB
lektor_ai/normalizer/venv 6.9 GB optional: number normalizer env
lektor_ai/normalizer/model 1.1 GB optional: its weights
```
Output is roughly the size of the source video, since video is stream-copied. A
@@ -49,13 +52,13 @@ regenerated without redoing a whole run (see
## Dependency surface
This is the point most worth internalising, because it makes debugging much
simpler: **the two halves have almost nothing in common.**
simpler: **the three parts have almost nothing in common.**
### `lektor_ai_comfy.py` — standard library only
```python
argparse json random re shutil subprocess sys threading time uuid
pathlib.Path urllib.request
argparse configparser json os queue random re shlex shutil subprocess
sys threading time uuid pathlib.Path urllib.request
```
No third-party packages. No conda environment. No virtualenv. It runs on the
@@ -72,6 +75,25 @@ reachable ComfyUI instance. It talks to ComfyUI over plain HTTP with
> **`requirements.txt` in the project root does not apply here.** It belongs to
> the older `lektor_ai.py`, which used the hosted Boson AI API and needed
> `openai>=1.0.0`. `lektor_ai_comfy.py` requires nothing from it.
> `normalizer/requirements.txt` is a different file and does apply — to the
> normalizer's own venv, never to the system Python.
### `normalizer/` — optional third environment
```
torch 2.11.0+cu128 transformers 5.15.0 accelerate 1.14.0
num2words 0.5.x (fallback path only)
```
Its own venv at `normalizer/venv` (~6.9 GB — the cu128 wheels bundle their
own CUDA runtime, again) plus `normalizer/model` (1.1 GB).
Separate from ComfyUI's environment because it hard-requires `transformers >= 5`
— 4.x tokenizes the Qwen3 chat template differently and silently produces wrong
output. `lektor_ai_comfy.py` never imports it; it starts it as a subprocess and
talks JSON lines over a pipe, so the version conflict never has to be resolved.
Entirely optional: without it the pipeline uses a `num2words` fallback, and with
`numbers = no` it is not started at all.
### ComfyUI — a large, isolated environment
@@ -130,3 +152,9 @@ API keys, no accounts, no per-use cost.
The model is published under a research / non-commercial licence and may require
accepting its terms on huggingface.co, then `hf auth login`, before it will
download.
The number normalizer's model is **gated** in the same way and needs an
`HF_TOKEN` for its one-time download; see
[installation.md](installation.md#number-normalizer-optional-recommended). Runs
set `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`, so it never reaches the
network afterwards.
docs/troubleshooting.md
+144
View File
@@ -203,6 +203,93 @@ Handled by `_clean_text()`, but only after tags are removed. `<i>HELLO</i>`
looked mixed-case while the tag was present, because `i` is a lowercase letter.
If shouting persists, check for an unhandled tag type.
Such blocks are converted to Title Case (`Nasa – Margo Madison`), not to
sentence case. Lowercasing everything but the first letter also destroys the
proper nouns in a caption, and the number normalizer then tries to repair the
lowercase name — it turned `margo` into `marga`. Paired clips in
`output/clips/all-caps/` show the TTS reads both casings identically, so the
names are kept.
### A clip of an ALL-CAPS caption is cut off mid-word
Check whether the block still contains digits. Higgs truncates on raw digits far
more often than on words: the same caption generated eight times, four with
`21 MAJA 1983` and four with the date spelled out, was cut in 3 of 4 digit
versions and 0 of 4 word versions — and the digit clips ran barely half as long
as the speech required. Casing made no difference in that test.
That is the number normalizer's job, so if you see it, the block reached the TTS
without being normalized: check `numbers = yes` and the `Normalizer ready` line
in step 1.
### An abbreviation, unit or name is read out wrong
Put it in `dictionary.txt`: `km/h = kilometrów na godzinę`. The replacement
happens before the text reaches the model, and the result is visible in the job
folder's `tts_input.srt`. See [usage.md](usage.md#replacement-dictionary).
If a rule appears to do nothing, check the run's `Dictionary:` line — a rule
that failed to parse was reported there and skipped, and a rule cannot match
across a line break because each dialogue line is treated separately.
`--regen` and `--rebuild` read `tts_input.srt`, which already has the
replacements baked in, so adding a rule afterwards needs either a fresh run
from the SRT or a manual edit of `tts_input.srt` before `--regen`.
### Numbers are read as garbled non-words
That is what the number normalizer is for; check what the run printed in step 1.
- `Normalizer ready (model on cuda:0)` — it ran. Look at `tts_input.srt`: if the
digits are still there, the block may hold only standalone single digits,
which are left alone deliberately.
- `Normalizer ready (fallback)` — the model is missing or failed to load, and
`num2words` is standing in: bare integers only, nominative case only, times
and decimals untouched. Install the model, see
[usage.md](usage.md#number-normalization).
- `normalizer interpreter not found` — run `normalizer/setup.sh`.
- Nothing at all — the stage is off (`numbers = no`, or
`--no-normalize-numbers`).
For the underlying cause of a bad load, run the component directly; it prints
the real error instead of a summary:
```bash
normalizer/venv/bin/python normalizer/normalizer.py --status
normalizer/venv/bin/python normalizer/normalizer.py "Wygrał 1247 złotych."
```
### A number is expanded in the wrong grammatical case
Nominative everywhere (`dwadzieścia pięć psach` instead of `dwudziestu pięciu
psach`) means the `num2words` fallback is running, not the model — see above.
The neural model is the only path that declines correctly.
### The normalizer is unexpectedly slow
Check which device the run reported. `Normalizer ready (model on cpu)` means it
never got the GPU. If it says `cuda:0` and is still slow, it may have only got
part of it: with Higgs resident there is not enough VRAM, and `accelerate`
spills the rest to RAM without raising anything — about 5× slower.
The script prevents this by asking ComfyUI to unload before normalizing. If you
do not see `Freed N GB` in step 1, either `free_comfy_vram = no`, or ComfyUI had
nothing loaded (normal on a freshly started backend), or the call failed and
said so.
### The normalizer times out on every block
`timeout` under `[normalize]` is per block, and a block takes well under a second
on a GPU. Timing out means the model landed on the CPU or the worker is wedged.
Check `torch.cuda.is_available()` in that venv:
```bash
normalizer/venv/bin/python -c "import torch; print(torch.cuda.is_available())"
```
The stage disables itself after the first timeout rather than repeating the wait
on every remaining block, so a run costs one `timeout` at worst.
### `--embedded-subs` shows no usable tracks
The video only carries image-based subtitles (`hdmv_pgs_subtitle`,
@@ -226,6 +313,58 @@ ComfyUI is not running. Start it and wait for the model to load:
curl -s http://127.0.0.1:8189/system_stats | head -c 200
```
### ComfyUI dies partway through a long queue
The run recovers by itself now, and says so:
```
[2035/2073] 02:22:59 'Joshua.'
ComfyUI went away ([Errno 104] Connection reset by peer) [1/3 for this clip].
Stopping ComfyUI (pid 530039, suspended)…
Starting ComfyUI in a new gnome-terminal window…
ComfyUI died on this clip (1/3) — retrying with a different seed.
```
The stale process is stopped before the restart, because a wedged or suspended
backend still holds the port and the GPU and the new one then cannot start. A
process in state `T` is killed outright — it never runs its signal handlers, so
SIGTERM is accepted and ignored.
**Each retry uses a different seed**, which matters: these aborts are
seed-dependent, not text-dependent. The block that killed ComfyUI three times in
a 2073-clip run generated perfectly on its own with another seed.
After three deaths on the same clip the clip is skipped and the queue continues,
with a report at the end:
```
2 clip(s) crashed ComfyUI and were skipped — the voiceover is silent there.
Edit their text in tts_input.srt, then:
--regen 2035 4180
```
Skipping only happens when the backend comes back healthy. If it does not, the
run waits for it instead — otherwise a backend that is simply not running would
cause every remaining clip to be "skipped" and produce a silent film.
### The clip that crashes ComfyUI
The abort looks like this in `comfyui.log`:
```
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1515: indexSelectSmallIndex:
Assertion `srcIndex < srcSelectDimSize` failed.
Aborted (core dumped)
```
An out-of-range index in an embedding lookup inside the Higgs audio tokenizer.
It is a CUDA device-side assert, so it cannot be caught in Python — the process
dies outright, which is why it looks like ComfyUI "will not start" when a client
is resubmitting the same clip on every restart.
`--regen N` with a different seed is the fix; rewording the block helps if it
persists.
### ComfyUI crashed and the GPU is stuck
```bash
@@ -244,6 +383,11 @@ python3 lektor_ai_comfy.py film.mkv --resume-from 312
Nothing else should be using the GPU during a run. Restart ComfyUI to clear
fragmentation. The 4B model needs roughly 10–12 GB.
The number normalizer adds ~1.5 GB, but only during step 1 — its process exits
before TTS generation starts, so the two never hold VRAM at the same time. If it
cannot fit anyway, it logs the CUDA error and finishes the run on the `num2words`
fallback rather than failing.
### `torch.cuda.is_available()` is False
```bash
docs/usage.md
+256
View File
@@ -111,6 +111,25 @@ python3 lektor_ai_comfy.py input/film.mkv --resume-from 47
The console prints `[47/693]` style progress, so the number to resume from is
the last one that completed.
**It does not reprocess the subtitles.** The blocks come from the job folder's
existing `tts_input.srt`, so the dictionary and the number normalizer are not
run again:
```
[1/4] Resuming from block 47 — reusing tts_input.srt…
693 blocks, subtitles not reprocessed — the numbering still matches the clips on disk.
```
That is a correctness requirement, not an optimisation. `--resume-from` pairs
block *N* with `clip_NNNN.flac`, and re-deriving the text can change it — a
dictionary rule added since, a different normalizer result, a changed `--start`.
Same block count with different words is the dangerous case, because nothing
would look wrong until you listened.
It also means `--start`/`--end` need not be repeated for the block list, since
the saved file already holds the adjusted timings. If the job folder has no
`tts_input.srt`, the run says so and falls back to parsing the SRT.
### `--skip-tts`
Uses a pre-assembled audio file and goes straight to the mux.
@@ -143,6 +162,14 @@ tts_volume = 1.8
[paths]
output_dir = output
dictionary = dictionary.txt
[normalize]
numbers = yes
python = normalizer/venv/bin/python
llm = normalizer/models/PLLuM-4B-instruct-2512
timeout = 60
free_comfy_vram = yes
[comfy]
comfy_url = http://127.0.0.1:8189
@@ -251,8 +278,237 @@ Typical starting point for dialogue-heavy material: `--orig-volume 0.5
| Flag | Default | Meaning |
|---|---|---|
| `--output-dir DIR` | `output` | Where everything is written |
| `--dictionary FILE` | `dictionary.txt` | Word/phrase replacements, `""` disables |
| `--normalize-numbers` / `--no-normalize-numbers` | on | Spell digits out in Polish |
| `--comfy-url URL` | `http://127.0.0.1:8189` | Backend address |
## Replacement dictionary
`dictionary.txt` next to the script rewrites words and phrases in the subtitles
**before** they reach the TTS engine. It exists for everything the model reads
out wrong on its own: abbreviations, units, symbols, names with an unusual
spelling.
One rule per line:
```
km/h = kilometrów na godzinę
np. = na przykład
m.in. = między innymi
/(\d+)\s*%/ = \1 procent
```
| Rule | Behaviour |
|---|---|
| Matching | **Case-sensitive**, whole words only — `np.` does not match inside `znp.`, and `Joshua` does not match `joshua` |
| Capitals | The replacement goes in exactly as written; nothing is re-cased for you |
| Priority | Longer literals win: `km/h` beats a plain `km` rule whichever order they are written in |
| Chaining | Replacement text is never rescanned, so rules cannot feed each other or loop |
| Regex | A pattern in slashes is a regular expression; `\1 \2 …` refer to its groups |
| Comments | Lines starting with `#` or `;`; `->` and `=>` also work as separators |
| Deleting | An empty replacement removes the phrase |
Each run reports what it loaded and how much it changed:
```
Dictionary: 14 rule(s) from /home/lukas/Programy/lektor_ai/dictionary.txt
6 dictionary replacement(s) applied.
```
The replaced text is what lands in the job folder's `tts_input.srt` — check
there when a rule is not doing what you expect. Because `--regen` and
`--rebuild` work from that file, they do **not** re-apply the dictionary: edit
`tts_input.srt` and `--regen` the affected blocks, or rerun from the SRT.
A relative name is looked for in the current directory first and then next to
the script, so one film can keep its own list without hiding the shared one.
A missing file, a line with no separator and an invalid regex are all warnings,
never fatal.
### The queue dictionary
`dictionary.txt` holds rules that are true of Polish generally. A batch often
needs its own — a character's name the model mangles, a recurring
mistranslation — and those do not belong in the shared file.
The GUI's **Queue dictionary** button opens a plain text editor with the same
format, one rule per line:
```
Joshua = Dżoszua
P12 = pozycja dwunasta
# comments work here too
```
Saving writes `queue_dictionary.txt` into the **first queued video's job
folder**, and every video in the queue is run with it. The dialog reports how
many rules parsed, so a typo shows up when you save rather than as a rule that
silently does nothing during the run. **Clear** empties the editor only —
nothing is written until you press Save.
The two files are **merged into one rule set**, not chained: a replacement made
by one still cannot be rewritten by the other, and longest-literal-wins applies
across both. A run lists them separately:
```
Dictionary: 10 rule(s) from /home/lukas/Programy/lektor_ai/dictionary.txt
Dictionary: 2 rule(s) from output/clips/S02E01/queue_dictionary.txt
3 dictionary replacement(s) applied.
```
On the command line it is `--dictionary-extra FILE`, which works with any file
you like.
## Number normalization
Higgs Audio misreads long digit strings and sometimes emits garbled non-words,
so the **last** subtitle stage rewrites numbers as Polish words:
```
Wygrał 1247 złotych. → Wygrał tysiąc dwieście czterdzieści siedem złotych.
Mówimy o 25 psach. → Mówimy o dwudziestu pięciu psach.
Mam 3 koty. → Mam 3 koty.
```
The case matters — `25 psów` / `o 25 psach` / `z 25 psami` are three different
forms of the same numeral — which is why this is a small fine-tuned model rather
than a lookup table. Standalone single digits are left as digits: the TTS reads
those correctly, and expanding them only makes the clip longer than the subtitle
window it has to fit.
It runs **after** the dictionary, so a rule that turns `16%` into `16 procent`
feeds `szesnaście procent`, and after dialogue splitting, so the numbers are
normalized in the clip that actually says them.
| Flag | Default | Meaning |
|---|---|---|
| `--normalize-numbers` | on | Spell digits out |
| `--no-normalize-numbers` | | Leave digits alone for this run |
or in `settings.ini`:
```ini
[normalize]
numbers = no
```
### Install
The model needs `transformers >= 5`, which must not go into the ComfyUI
environment, so it lives in `normalizer/` with its own venv:
```bash
normalizer/setup.sh # venv + torch + transformers
HF_TOKEN=hf_... normalizer/setup.sh --model # the weights
```
The weights are **gated**: accept the terms at
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> with your Hugging
Face account, then create a token at <https://huggingface.co/settings/tokens>.
That is the only step that needs the network; runs are offline afterwards.
Full details, including the JSON-lines protocol and the deviations from the
build brief, are in [normalizer/README.md](../normalizer/README.md).
### Two models, and which one gets a block
Numbers fail in two different ways and no single model handles both:
| block contains | goes to | because |
|---|---|---|
| plain integers only (74%) | num2words → inflection model | num2words cannot miscount; the model only fixes the case |
| times, dates, decimals, `P10`, `LO 3842` (26%) | the 0.6B fine-tune | it was trained on those formats; the other leg destroys them |
The split is why `Mówili o 250000 uchodźcach.` comes out as *o dwustu
pięćdziesięciu tysiącach uchodźców* — the fine-tune alone renders that as
*o pięciu tysiącach*, losing the number — while `Teraz na P12.` still comes out
as *P dwanaście* rather than *Pdziesięć*.
Every answer is then checked before use: runaway length, repetition loops,
invalid characters, leftover digits, a lost minus sign, a changed number, a lost
word. A block that fails falls back to num2words — nominative, but never wrong
about the value. The run reports how often that happened.
Set `llm =` (empty) to use the fine-tune for everything.
### What a run prints
```
5 block(s) contain numbers — normalizing…
Asking ComfyUI to unload its models (7.9 GB)…
Freed 7.9 GB.
Normalizer ready (model on cuda:0, two models).
5 of 5 block(s) rewritten (2 complex, 3 plain).
1 block(s) failed a safety check and fell back (word lost x1).
```
`Normalizer ready (fallback)` means the model is missing or failed to load and
`num2words` is standing in — bare integers in the nominative case only, with
times, dates and decimals left as digits. The run says so and continues.
### Sharing the GPU with ComfyUI
Higgs stays resident between runs — about 8.4 GB of a 16 GB card — which leaves
too little for the normalizer. That failure is silent rather than loud:
`accelerate` answers a shortage by spilling layers to RAM, so the run continues
with no error and roughly **5× the time per block** (measured: 1.19 s → 5.96 s).
So before starting the normalizer the script asks ComfyUI to unload:
```
5 block(s) contain numbers — normalizing…
Asking ComfyUI to unload its models (7.9 GB)…
Freed 7.9 GB.
Normalizer ready (model on cuda:0).
```
ComfyUI reloads Higgs on the first clip of step 2 — 20–30 s, once per run. It is
a `POST /free` on ComfyUI's own API, not a restart: the server stays up, with
its custom nodes loaded. The unload is asynchronous, so the script waits until
`torch_vram_total` in `/system_stats` actually drops before continuing.
`--regen` and `--rebuild` work from `tts_input.srt`, which is already
normalized, so they never normalize and never pay the reload. On a freshly
started ComfyUI the call is a no-op and prints nothing.
Set `free_comfy_vram = no` to skip it — worth doing only if you have VRAM to
spare, or if you have turned the neural normalizer off.
Nothing about this stage is fatal. A missing venv, a worker that dies, a block
that takes longer than `timeout` — each is a warning, and the affected text goes
to the TTS with its digits intact. The worker is started once per run and exits
before TTS generation begins, so it is not holding VRAM while Higgs works.
## The backend's lifecycle
ComfyUI is normally started once and left running. The script manages it when it
has to:
| situation | what happens |
|---|---|
| not running when a clip is submitted | started via `start_comfyui.sh`, waited for, the clip resubmitted |
| running but wedged or suspended | stopped (SIGKILL if suspended) before the restart |
| dies on a clip | up to 3 retries, each with a **different seed** |
| dies 3 times on the same clip | that clip is skipped, the queue continues, all skips reported at the end |
| queue finished | models unloaded, backend left running — it reloads on the next request |
| GUI closed | backend stopped, so no idle process is left holding the GPU |
Its output goes to `comfyui.log` next to the script as well as to the terminal
window, so a crash can be diagnosed after the fact.
The GUI has a **ComfyUI** button showing the same state: `Start ComfyUI` when it
is down, `ComfyUI ●` when it is answering, `ComfyUI ◍` when it holds the port but
does not answer. Pressing it starts or restarts.
Quitting the GUI stops the backend, including one you started by hand in a
terminal — it is found by port, not by remembering what was launched. Between
runs it stays up on purpose: the queue only unloads its models, so the next job
does not pay for a cold start.
`start_comfyui.sh` ends with `exec python main.py`, so the script's PID *is*
ComfyUI's. Without that the launcher only ever sees a wrapper shell, and killing
it orphans the python still holding 8 GB of VRAM.
## Output
Everything belonging to one video is kept together in a **job folder** named