Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control

Text processing before TTS:

- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
  into one alternation applied in a single pass, so replacements never chain
  and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
  folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
  destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
  the normalizer to "correct" the name. Paired clips in output/clips/all-caps
  showed the TTS reads both casings the same.

normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:

- Two backends. Plain integers go to num2words for the value and a general
  Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
  fine-tune. Measured on 40 real subtitle lines, each is clearly better than
  the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
  leftover digits, a lost minus sign, a changed number, a lost word. A failure
  falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
  in VRAM is spilled to RAM silently at 5x the cost.

Backend lifecycle:

- ComfyUI is found by port through /proc, so a backend started by hand is
  managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
  outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
  seed (the aborts are seed-dependent, not text-dependent), then skipped and
  reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.

--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-12 21:55:31 +02:00
co-authored by Claude Opus 5
parent 7e2cee02e5
commit e201aadc3e
21 changed files with 3759 additions and 52 deletions
+70 -3
View File
@@ -30,7 +30,7 @@ Settings
--------
Defaults live in settings.ini next to the script:
voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
orig_volume, tts_volume, output_dir, comfy_url
orig_volume, tts_volume, output_dir, dictionary, numbers, comfy_url
Precedence: command-line flag > settings.ini > built-in default.
Use --settings FILE to select a different profile.
@@ -59,10 +59,12 @@ Both external SRT files and embedded subtitle tracks go through the same cleanin
- HTML tags stripped: <i>, </i>, <b>, <font color="…">, etc.
- ASS override codes stripped: {\an8}, {\pos(x,y)}, {\c&H…&}, etc.
- ALL-CAPS blocks lowercased and converted to sentence case
- ALL-CAPS blocks converted to Title Case (keeps proper nouns readable)
- Sentence fragments starting with a lowercase letter are capitalised
- Missing terminal punctuation added (… or . depending on context)
- Dictionary replacements applied (see below)
- Dialogue lines (prefixed with -) are split into separate clips with proportional timing
- Numbers spelled out as Polish words, in the right case (see below)
With --embedded-subs the extracted subtitles are saved to the job folder as embedded_sub.srt
so you can inspect them before or after the run.
@@ -72,6 +74,63 @@ The final MKV preserves all subtitle tracks from the source video — text-based
stream copy with no conversion.
Replacement Dictionary
----------------------
dictionary.txt next to the script rewrites words and phrases before they reach
the TTS engine — abbreviations, units, symbols, names the model mispronounces.
One rule per line:
km/h = kilometrów na godzinę
np. = na przykład
/(\d+)\s*%/ = \1 procent
Matching is case-sensitive and whole-word only, so "np." leaves "znp." alone and
"Joshua" leaves "joshua" alone. The replacement is inserted exactly as written.
The first letter of each block is capitalised before these rules run, so add a
capitalised variant for words that can start a subtitle.
Longer patterns win over shorter ones regardless of the order they are written
in, and replacement text is never rescanned, so rules cannot feed each other.
A pattern wrapped in slashes is a regular expression, with \1 \2 … referring to
its groups. Lines starting with # or ; are comments.
The result is visible in the job folder's tts_input.srt — check there if a rule
is not doing what you expect. --regen and --rebuild work from that file, so they
do not re-apply the dictionary; edit tts_input.srt directly, or rerun.
Use another list with --dictionary FILE (a relative name is looked for in the
current directory first, then next to the script) or turn it off with
--dictionary "". A missing file is not an error.
A second list can be merged in with --dictionary-extra FILE. The GUI writes one
per queue — the "Queue dictionary" button — into the first queued video's job
folder, for rules that belong to one batch rather than to Polish in general.
Number Normalization
--------------------
The TTS misreads long digit strings and sometimes turns them into non-words, so
after the dictionary every block containing a number is rewritten with the
numbers spelled out in Polish — in the correct grammatical case, which is why
this is a small neural model and not a lookup table:
Wygrał 1247 złotych. -> Wygrał tysiąc dwieście czterdzieści siedem złotych.
Mówimy o 25 psach. -> Mówimy o dwudziestu pięciu psach.
Mam 3 koty. -> Mam 3 koty. (single digits are read fine)
It runs in its own environment (transformers>=5, which must not be installed
next to ComfyUI's pinned versions). One-time install:
normalizer/setup.sh # venv + dependencies
HF_TOKEN=hf_... normalizer/setup.sh --model # gated weights, accept terms first
Terms: https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization
Everything after that runs offline. See normalizer/README.md.
Turn it off with numbers = no under [normalize] in settings.ini, or with
--no-normalize-numbers for one run. If the environment or the model is missing
the run continues with the digits untouched — it is never fatal.
Output Files
------------
Everything belonging to one video is kept together in a job folder named after
@@ -130,6 +189,13 @@ All Options
--max-tokens N Max audio tokens the model may generate per clip (default: 4096)
Raise this if long clips are cut off mid-word.
--dictionary FILE Word/phrase replacements applied to the subtitles before
TTS (default: dictionary.txt). Pass "" to disable.
--normalize-numbers Spell digits out as Polish words before TTS (default: on)
--no-normalize-numbers
Leave digits as digits. The TTS often misreads them.
--orig-volume 0-1 Volume of the original audio in the Polish lektor mix (default: 0.60)
--tts-volume 0-2 Volume of the TTS voiceover in the Polish lektor mix (default: 1.8)
Note: these affect Track 1 only. Track 0 is always the raw original.
@@ -151,7 +217,8 @@ All Options
Recovery and Fixing Bad Clips
------------------------------
Resume a crashed run from block N (reuses existing clips 1..N-1):
Resume a crashed run from block N (reuses existing clips 1..N-1 and the job
folder's tts_input.srt — subtitles are not reprocessed):
python3 lektor_ai_comfy.py input/film.srt input/film.mkv --resume-from 47