Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Poll interval
wait_for_completion() polled every 3 s, which quantised every clip to a 3 s
boundary — across 2075 existing clips, 91% of the gaps landed exactly on a
multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so
short clips are picked up immediately without hammering the server on long
ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll,
a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real
generation time, e.g. "done (4.2s)".
A WebSocket would remove polling entirely but needs a third-party client,
which would cost this script its dependency-free property.
settings.ini
voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next
to the script. Precedence is CLI flag > settings.ini > built-in default,
wired by feeding loaded values in as argparse defaults. --settings FILE
selects an alternative profile.
INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or
JSON (no comments). A missing, partial, malformed or corrupt file degrades to
built-in defaults with a warning instead of killing a long run.
Run-specific flags are deliberately excluded: --start, --end, --seed,
--regen, --rebuild, --resume-from, --embedded-subs, --skip-tts.
Per-job folders
All state for one video now lives in <output_dir>/clips/<video name>/ —
clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and
video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level.
tts_input.srt moved with the clips because they are a matched pair: block N
in the SRT is clip_NNNN.flac. Previously both sat at the top level and each
new job overwrote the last one's, so rebuilding an older film silently
paired its clips with the newest film's text. Any past job can now be
rebuilt. A run against the old flat layout gets an explanatory message.
gudowski/ -> voices/
VOICE_PROMPTS_DIR is now resolved relative to the script rather than
hardcoded, so moving the project no longer breaks it. Voice filenames are
unchanged, so --voice gudowski_short still works.
Docs updated throughout for the new layout, settings file and poll behaviour.
Known issue, not addressed here: with a non-zero --start the voiceover plays
early. extract_video_fragment() stream-copies, so it can only cut on a
keyframe and seeks backwards to it, while TTS timestamps are relative to the
requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is
458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with
--start 00:00 are unaffected, since 0 is always a keyframe.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Converts SRT subtitles into a Polish lektor-style voiceover using a local
ComfyUI instance running Higgs Audio V3, then muxes the result back into the
source video as a second audio track. Video and original audio are always
stream-copied; only the new voiceover track is encoded.
Contents:
lektor_ai_comfy.py main pipeline (standard library only, needs ffmpeg)
lektor_ai_comfy_v2.py batched-context experiment (timing unresolved)
lektor_ai.py earlier version using the hosted Boson AI API
install_comfyui.sh reproduces the ComfyUI environment, versions pinned
docs/ requirements, installation, usage, architecture,
troubleshooting
gudowski/ voice references (.wav) with their transcripts (.txt)
Source media and generated video are gitignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
claudeandClaude Opus 5