claudeandClaude Opus 5
e201aadc3e
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
113 lines
4.6 KiB
INI
113 lines
4.6 KiB
INI
# lektor_ai_comfy — default settings
|
|
#
|
|
# These are the values used when a flag is not given on the command line.
|
|
# Precedence: command-line flag > this file > built-in defaults
|
|
#
|
|
# Delete a line (or the whole file) to fall back to the built-in default.
|
|
# A malformed value is reported and ignored, never fatal.
|
|
#
|
|
# Not configurable here, because they belong to a single run rather than to
|
|
# your setup: --start --end --seed --regen --rebuild --resume-from
|
|
# --embedded-subs --skip-tts
|
|
|
|
|
|
[tts]
|
|
# Voice reference. Either a name in voices/ (expects NAME.wav + NAME.txt),
|
|
# or a full path to a .wav file.
|
|
voice = gudowski_short_3
|
|
|
|
# Per-clip speed-up applied with atempo, before assembly.
|
|
# Raise it if speech regularly overruns its subtitle window; 1.0 = no change.
|
|
speed = 1.2
|
|
|
|
# Sampling temperature. 1.0 makes the model occasionally recite the reference
|
|
# transcript instead of the subtitle; 0.9 largely prevents that. Lower is more
|
|
# deterministic but flatter.
|
|
temperature = 0.8
|
|
|
|
# Ceiling on audio tokens per clip. Raise only if long clips are cut off
|
|
# mid-word.
|
|
max_tokens = 4096
|
|
|
|
# Roughly one clip in twelve comes back cut off mid-word. Those are detected and
|
|
# asked for again with a different seed, up to this many extra attempts. 0 to
|
|
# switch retrying off.
|
|
truncation_retries = 2
|
|
|
|
# Trim this many ms from the start of every clip, to remove an inhale artifact.
|
|
# Try 80-150 if you hear one. Too high clips the first consonant.
|
|
strip_leading_ms = 0
|
|
|
|
# Fade out the last N ms of every clip to suppress tail pops. 0 disables.
|
|
trailing_fade_ms = 80
|
|
|
|
|
|
[mix]
|
|
# Levels inside the Polish lektor track only. Track 0 is always the untouched
|
|
# original, so mixing aggressively here costs you nothing.
|
|
# Dialogue-heavy sources: orig 0.5 / tts 2.0. Quiet sources: orig 1.0 / tts 1.4.
|
|
orig_volume = 0.70
|
|
tts_volume = 1.4
|
|
|
|
|
|
[paths]
|
|
# Where clips, assembled audio and the finished video are written.
|
|
# Relative paths are resolved from wherever you run the script.
|
|
output_dir = output
|
|
|
|
# Word and phrase replacements applied to the subtitles before they are spoken,
|
|
# e.g. "km/h = kilometrów na godzinę". One rule per line in that file; see the
|
|
# comments at the top of it. A relative name is looked for in the current
|
|
# directory first, then next to lektor_ai_comfy.py, so a single film can have
|
|
# its own list. A missing file is not an error.
|
|
dictionary = dictionary.txt
|
|
|
|
|
|
[normalize]
|
|
# Spell digits out as Polish words before the text reaches the TTS, in the right
|
|
# grammatical case: "Wygrał 1247 złotych" -> "Wygrał tysiąc dwieście czterdzieści
|
|
# siedem złotych". Higgs Audio misreads long digit strings and sometimes turns
|
|
# them into non-words. Standalone single digits are read correctly and are left
|
|
# alone. Runs last, on the text that goes into tts_input.srt. yes/no.
|
|
numbers = yes
|
|
|
|
# Interpreter of the normalizer's own environment — it needs transformers>=5,
|
|
# which must not be installed next to ComfyUI's pinned versions. Created by
|
|
# normalizer/setup.sh. Relative paths are resolved next to lektor_ai_comfy.py.
|
|
# If it is missing the run continues with the digits untouched.
|
|
python = normalizer/venv/bin/python
|
|
|
|
# Second model, used for the grammar of plain integers. num2words supplies the
|
|
# value (it cannot miscount) and this model puts the numeral into the right
|
|
# case: "o 250000 uchodzcach" -> "o dwustu piecdziesieciu tysiacach uchodzcow".
|
|
# The fine-tune alone gets that wrong, and sometimes loses the number itself.
|
|
# Blocks with times, dates, decimals or identifiers skip it — it is worse than
|
|
# the fine-tune on those. Leave empty to use the fine-tune for everything.
|
|
llm = normalizer/models/PLLuM-4B-instruct-2512
|
|
|
|
# Seconds to wait for one block. On a GPU a block takes well under a second;
|
|
# this only exists so a wedged worker cannot stall the whole run.
|
|
timeout = 60
|
|
|
|
# Ask ComfyUI to unload Higgs before normalizing, so the normalizer gets the
|
|
# GPU. Higgs stays resident between runs and leaves too little VRAM; a
|
|
# normalizer that does not fit spills to RAM silently and runs about 5x slower.
|
|
# ComfyUI reloads on the first clip of step 2 (20-30s, once per run). --regen
|
|
# and --rebuild never normalize, so they never pay it. yes/no.
|
|
free_comfy_vram = yes
|
|
|
|
|
|
[comfy]
|
|
# Address of the ComfyUI backend running Higgs Audio V3.
|
|
comfy_url = http://127.0.0.1:8189
|
|
|
|
# Start ComfyUI automatically when it stops answering. A crash loses its queue,
|
|
# so the clip in flight is resubmitted once the backend is back. At most 3
|
|
# attempts per run, 90s apart, so a backend that dies on startup cannot spawn an
|
|
# endless stream of terminal windows. yes/no.
|
|
autostart = yes
|
|
|
|
# The script used to start it. Relative paths are resolved next to
|
|
# lektor_ai_comfy.py.
|
|
start_command = start_comfyui.sh
|