Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control

Text processing before TTS:

- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
  into one alternation applied in a single pass, so replacements never chain
  and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
  folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
  destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
  the normalizer to "correct" the name. Paired clips in output/clips/all-caps
  showed the TTS reads both casings the same.

normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:

- Two backends. Plain integers go to num2words for the value and a general
  Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
  fine-tune. Measured on 40 real subtitle lines, each is clearly better than
  the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
  leftover digits, a lost minus sign, a changed number, a lost word. A failure
  falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
  in VRAM is spilled to RAM silently at 5x the cost.

Backend lifecycle:

- ComfyUI is found by port through /proc, so a backend started by hand is
  managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
  outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
  seed (the aborts are seed-dependent, not text-dependent), then skipped and
  reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.

--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-12 21:55:31 +02:00
co-authored by Claude Opus 5
parent 7e2cee02e5
commit e201aadc3e
21 changed files with 3759 additions and 52 deletions
+42 -1
View File
@@ -23,7 +23,7 @@ speed = 1.2
# Sampling temperature. 1.0 makes the model occasionally recite the reference
# transcript instead of the subtitle; 0.9 largely prevents that. Lower is more
# deterministic but flatter.
temperature = 0.6
temperature = 0.8
# Ceiling on audio tokens per clip. Raise only if long clips are cut off
# mid-word.
@@ -55,6 +55,47 @@ tts_volume = 1.4
# Relative paths are resolved from wherever you run the script.
output_dir = output
# Word and phrase replacements applied to the subtitles before they are spoken,
# e.g. "km/h = kilometrów na godzinę". One rule per line in that file; see the
# comments at the top of it. A relative name is looked for in the current
# directory first, then next to lektor_ai_comfy.py, so a single film can have
# its own list. A missing file is not an error.
dictionary = dictionary.txt
[normalize]
# Spell digits out as Polish words before the text reaches the TTS, in the right
# grammatical case: "Wygrał 1247 złotych" -> "Wygrał tysiąc dwieście czterdzieści
# siedem złotych". Higgs Audio misreads long digit strings and sometimes turns
# them into non-words. Standalone single digits are read correctly and are left
# alone. Runs last, on the text that goes into tts_input.srt. yes/no.
numbers = yes
# Interpreter of the normalizer's own environment — it needs transformers>=5,
# which must not be installed next to ComfyUI's pinned versions. Created by
# normalizer/setup.sh. Relative paths are resolved next to lektor_ai_comfy.py.
# If it is missing the run continues with the digits untouched.
python = normalizer/venv/bin/python
# Second model, used for the grammar of plain integers. num2words supplies the
# value (it cannot miscount) and this model puts the numeral into the right
# case: "o 250000 uchodzcach" -> "o dwustu piecdziesieciu tysiacach uchodzcow".
# The fine-tune alone gets that wrong, and sometimes loses the number itself.
# Blocks with times, dates, decimals or identifiers skip it — it is worse than
# the fine-tune on those. Leave empty to use the fine-tune for everything.
llm = normalizer/models/PLLuM-4B-instruct-2512
# Seconds to wait for one block. On a GPU a block takes well under a second;
# this only exists so a wedged worker cannot stall the whole run.
timeout = 60
# Ask ComfyUI to unload Higgs before normalizing, so the normalizer gets the
# GPU. Higgs stays resident between runs and leaves too little VRAM; a
# normalizer that does not fit spills to RAM silently and runs about 5x slower.
# ComfyUI reloads on the first clip of step 2 (20-30s, once per run). --regen
# and --rebuild never normalize, so they never pay it. yes/no.
free_comfy_vram = yes
[comfy]
# Address of the ComfyUI backend running Higgs Audio V3.