Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+70
-3
@@ -30,7 +30,7 @@ Settings
|
||||
--------
|
||||
Defaults live in settings.ini next to the script:
|
||||
voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
|
||||
orig_volume, tts_volume, output_dir, comfy_url
|
||||
orig_volume, tts_volume, output_dir, dictionary, numbers, comfy_url
|
||||
|
||||
Precedence: command-line flag > settings.ini > built-in default.
|
||||
Use --settings FILE to select a different profile.
|
||||
@@ -59,10 +59,12 @@ Both external SRT files and embedded subtitle tracks go through the same cleanin
|
||||
|
||||
- HTML tags stripped: <i>, </i>, <b>, <font color="…">, etc.
|
||||
- ASS override codes stripped: {\an8}, {\pos(x,y)}, {\c&H…&}, etc.
|
||||
- ALL-CAPS blocks lowercased and converted to sentence case
|
||||
- ALL-CAPS blocks converted to Title Case (keeps proper nouns readable)
|
||||
- Sentence fragments starting with a lowercase letter are capitalised
|
||||
- Missing terminal punctuation added (… or . depending on context)
|
||||
- Dictionary replacements applied (see below)
|
||||
- Dialogue lines (prefixed with -) are split into separate clips with proportional timing
|
||||
- Numbers spelled out as Polish words, in the right case (see below)
|
||||
|
||||
With --embedded-subs the extracted subtitles are saved to the job folder as embedded_sub.srt
|
||||
so you can inspect them before or after the run.
|
||||
@@ -72,6 +74,63 @@ The final MKV preserves all subtitle tracks from the source video — text-based
|
||||
stream copy with no conversion.
|
||||
|
||||
|
||||
Replacement Dictionary
|
||||
----------------------
|
||||
dictionary.txt next to the script rewrites words and phrases before they reach
|
||||
the TTS engine — abbreviations, units, symbols, names the model mispronounces.
|
||||
One rule per line:
|
||||
|
||||
km/h = kilometrów na godzinę
|
||||
np. = na przykład
|
||||
/(\d+)\s*%/ = \1 procent
|
||||
|
||||
Matching is case-sensitive and whole-word only, so "np." leaves "znp." alone and
|
||||
"Joshua" leaves "joshua" alone. The replacement is inserted exactly as written.
|
||||
The first letter of each block is capitalised before these rules run, so add a
|
||||
capitalised variant for words that can start a subtitle.
|
||||
Longer patterns win over shorter ones regardless of the order they are written
|
||||
in, and replacement text is never rescanned, so rules cannot feed each other.
|
||||
A pattern wrapped in slashes is a regular expression, with \1 \2 … referring to
|
||||
its groups. Lines starting with # or ; are comments.
|
||||
|
||||
The result is visible in the job folder's tts_input.srt — check there if a rule
|
||||
is not doing what you expect. --regen and --rebuild work from that file, so they
|
||||
do not re-apply the dictionary; edit tts_input.srt directly, or rerun.
|
||||
|
||||
Use another list with --dictionary FILE (a relative name is looked for in the
|
||||
current directory first, then next to the script) or turn it off with
|
||||
--dictionary "". A missing file is not an error.
|
||||
|
||||
A second list can be merged in with --dictionary-extra FILE. The GUI writes one
|
||||
per queue — the "Queue dictionary" button — into the first queued video's job
|
||||
folder, for rules that belong to one batch rather than to Polish in general.
|
||||
|
||||
|
||||
Number Normalization
|
||||
--------------------
|
||||
The TTS misreads long digit strings and sometimes turns them into non-words, so
|
||||
after the dictionary every block containing a number is rewritten with the
|
||||
numbers spelled out in Polish — in the correct grammatical case, which is why
|
||||
this is a small neural model and not a lookup table:
|
||||
|
||||
Wygrał 1247 złotych. -> Wygrał tysiąc dwieście czterdzieści siedem złotych.
|
||||
Mówimy o 25 psach. -> Mówimy o dwudziestu pięciu psach.
|
||||
Mam 3 koty. -> Mam 3 koty. (single digits are read fine)
|
||||
|
||||
It runs in its own environment (transformers>=5, which must not be installed
|
||||
next to ComfyUI's pinned versions). One-time install:
|
||||
|
||||
normalizer/setup.sh # venv + dependencies
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model # gated weights, accept terms first
|
||||
|
||||
Terms: https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization
|
||||
Everything after that runs offline. See normalizer/README.md.
|
||||
|
||||
Turn it off with numbers = no under [normalize] in settings.ini, or with
|
||||
--no-normalize-numbers for one run. If the environment or the model is missing
|
||||
the run continues with the digits untouched — it is never fatal.
|
||||
|
||||
|
||||
Output Files
|
||||
------------
|
||||
Everything belonging to one video is kept together in a job folder named after
|
||||
@@ -130,6 +189,13 @@ All Options
|
||||
--max-tokens N Max audio tokens the model may generate per clip (default: 4096)
|
||||
Raise this if long clips are cut off mid-word.
|
||||
|
||||
--dictionary FILE Word/phrase replacements applied to the subtitles before
|
||||
TTS (default: dictionary.txt). Pass "" to disable.
|
||||
|
||||
--normalize-numbers Spell digits out as Polish words before TTS (default: on)
|
||||
--no-normalize-numbers
|
||||
Leave digits as digits. The TTS often misreads them.
|
||||
|
||||
--orig-volume 0-1 Volume of the original audio in the Polish lektor mix (default: 0.60)
|
||||
--tts-volume 0-2 Volume of the TTS voiceover in the Polish lektor mix (default: 1.8)
|
||||
Note: these affect Track 1 only. Track 0 is always the raw original.
|
||||
@@ -151,7 +217,8 @@ All Options
|
||||
Recovery and Fixing Bad Clips
|
||||
------------------------------
|
||||
|
||||
Resume a crashed run from block N (reuses existing clips 1..N-1):
|
||||
Resume a crashed run from block N (reuses existing clips 1..N-1 and the job
|
||||
folder's tts_input.srt — subtitles are not reprocessed):
|
||||
|
||||
python3 lektor_ai_comfy.py input/film.srt input/film.mkv --resume-from 47
|
||||
|
||||
|
||||
Reference in New Issue
Block a user