Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control

Text processing before TTS:

- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
  into one alternation applied in a single pass, so replacements never chain
  and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
  folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
  destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
  the normalizer to "correct" the name. Paired clips in output/clips/all-caps
  showed the TTS reads both casings the same.

normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:

- Two backends. Plain integers go to num2words for the value and a general
  Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
  fine-tune. Measured on 40 real subtitle lines, each is clearly better than
  the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
  leftover digits, a lost minus sign, a changed number, a lost word. A failure
  falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
  in VRAM is spilled to RAM silently at 5x the cost.

Backend lifecycle:

- ComfyUI is found by port through /proc, so a backend started by hand is
  managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
  outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
  seed (the aborts are seed-dependent, not text-dependent), then skipped and
  reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.

--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-12 21:55:31 +02:00
co-authored by Claude Opus 5
parent 7e2cee02e5
commit e201aadc3e
21 changed files with 3759 additions and 52 deletions
+96 -2
View File
@@ -37,23 +37,117 @@ must repeat the same `--start`/`--end`.
and `<...>` HTML tags (`<i>`, `<font …>`).
2. **Detect dialogue.** Lines beginning `- ` mean two speakers and are kept
separate; ordinary wrapped lines are joined with a space.
3. **Sentence-case ALL-CAPS blocks**, which the model would otherwise shout.
3. **Title-case ALL-CAPS blocks**, which the model would otherwise shout.
Every word is capitalised rather than lowercasing all but the first letter,
because the obvious version also lowercases the proper nouns inside a
caption: `NASA – MARGO MADISON` became `Nasa – margo madison`, which is wrong
to read aloud and invites the number normalizer to "correct" the name (it
produced `marga`). A/B clips showed the TTS reads both casings the same, so
keeping the names costs nothing.
4. **Capitalise a lowercase first letter**, since a fragment continuing from the
previous subtitle confuses the model.
5. **Add terminal punctuation** so the model produces a natural falling tail.
5. **Apply the replacement dictionary** — `km/h` → `kilometrów na godzinę` and
anything else the model reads out wrong (see below).
6. **Add terminal punctuation** so the model produces a natural falling tail.
Step 1 must come first. `<i>ALL CAPS</i>` looks mixed-case while the tag is
still present — the `i` in `<i>` is a lowercase letter — so the caps detection
in step 3 would silently miss it.
Step 5 must come after step 3 for the same reason in reverse: a lowercase
replacement dropped into `JEDZIE 200 KM/H` would make the block look mixed-case
and the shouting would survive. It comes before step 6 so a rule can supply
punctuation of its own.
### The replacement dictionary
`Dictionary.load()` reads `dictionary.txt` (`[paths] dictionary`, or
`--dictionary`) into `pattern = replacement` rules — literal and
case-insensitive, or a regular expression when the pattern is wrapped in
slashes. Every rule is compiled into **one alternation** applied in a single
`re.sub` pass, which buys three properties worth keeping:
- Replacement text is never rescanned, so rules cannot chain or loop.
- Literals are ordered longest-first inside the alternation, so `km/h` beats a
plain `km` rule regardless of the order in the file.
- Rules cost one pass over the text no matter how many there are.
Literal patterns get `(?<!\w)` / `(?!\w)` guards only on edges that are word
characters, which is what lets `km/h` and `np.` match on their own while `znp.`
does not — a guard on a `/` or `.` edge would stop them matching at all.
Matching is case-sensitive and the replacement is verbatim. The alternative —
match loosely, then infer how to re-case the replacement — was tried and
removed: it silently rewrote `NASA` as `Nasa`, and a rule that fires on a case
you did not write is hard to reason about. The cost is that the first letter of
a block is capitalised before the dictionary runs, so a lowercase rule cannot
fire there; on the seven subtitle files here that changed nothing, but it is why
a word that can open a subtitle needs both forms.
A missing file, a line without a separator and an invalid regex are all
warnings, never fatal — the same rule as `settings.ini`, for the same reason: a
typo should not cost a long run.
`split_dialogue_blocks()` then divides a two-speaker block into one clip per
line, apportioning the block's duration by character count.
### Number normalization (last stage)
`normalize_block_numbers()` rewrites digits as Polish words — `1247` →
`tysiąc dwieście czterdzieści siedem` — because Higgs Audio misreads long digit
strings and sometimes emits garbled non-words. It is **last** for two reasons:
the dictionary may have produced the words a numeral has to agree with (`16%` →
`16 procent` → `szesnaście procent`), and dialogue splitting decides which clip
a number ends up in.
The model that does it needs `transformers>=5`, which cannot be installed beside
ComfyUI's pinned dependencies, so it lives in `normalizer/` with its own venv and
runs as a **separate process**. `NumberNormalizer` is the client half:
- One worker per run, not per block — the 1.1 GB load dominates everything else.
It is started only if some block actually contains a number worth expanding,
and closed before step 2, so it is not holding VRAM while Higgs generates.
- **ComfyUI is asked to unload first** (`free_comfy_vram`). Higgs stays resident
between runs and leaves too little VRAM; a normalizer that does not fit is
spilled to RAM by accelerate *silently*, at about 5× the time per block. A
`POST /free` is cheaper than a restart — the server stays warm — and the wait
is on `torch_vram_total` in `/system_stats` rather than a fixed sleep, because
the flag is consumed by ComfyUI's worker loop rather than by the request.
- JSON lines over stdin/stdout, strictly one request per response. stdout is the
protocol; logs go to stderr and are kept in a 40-line ring for diagnostics.
- Reads are on a queue fed by a reader thread, which is what makes a per-block
timeout possible at all — a wedged worker cannot block the run.
- **Nothing here is fatal.** Missing venv, dead worker, timeout, out-of-order
reply: each prints a warning, disables the stage, and lets the digits through
untouched. A number read badly is a bad clip; a crash here is a lost film.
- **Two models behind one interface.** Plain integers go to num2words for the
value and a general Polish LLM for the case; anything with a time, date,
decimal or identifier goes to the 0.6B fine-tune. Measured on 40 real subtitle
lines, each is clearly better than the other on its half and clearly worse on
the other's. The worker reports which leg answered, and the run prints the
split.
- **Guards, not trust.** Both models can emit something worse than nothing — a
repetition loop to the token ceiling, a dropped minus that inverts a
temperature, a vanished noun, a number changed from 8250 to 8205. Each answer
is checked against the input and falls back to num2words when it fails.
`needs_number_normalization()` duplicates the worker's own selection rule so
blocks that need nothing never cross the process boundary. The two must stay in
step; the worker's copy is the authoritative one, and a test in the normalizer
suite imports the caller and compares both on a shared sample.
The result is written to `tts_input.srt` in that video's job folder
(`output/clips/<video name>/`). It is the source of truth for block
numbering: block *N* there is `clip_NNNN.flac` beside it and the
argument to `--regen N`.
Which is why `--regen`, `--rebuild` **and `--resume-from`** all read that file
rather than the original SRT. Everything upstream of it — the dictionary, the
normalizer, dialogue splitting — can produce different text than it did on the
first run, and any difference breaks the pairing with the clips already on disk.
Step 1 is skipped entirely in those modes.
## 2. Speech generation
For each block the script POSTs a workflow graph to `/prompt`, polls