claudeandClaude Opus 5
3dde6e58e4
Poll interval wait_for_completion() polled every 3 s, which quantised every clip to a 3 s boundary — across 2075 existing clips, 91% of the gaps landed exactly on a multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so short clips are picked up immediately without hammering the server on long ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll, a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real generation time, e.g. "done (4.2s)". A WebSocket would remove polling entirely but needs a third-party client, which would cost this script its dependency-free property. settings.ini voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms, orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next to the script. Precedence is CLI flag > settings.ini > built-in default, wired by feeding loaded values in as argparse defaults. --settings FILE selects an alternative profile. INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or JSON (no comments). A missing, partial, malformed or corrupt file degrades to built-in defaults with a warning instead of killing a long run. Run-specific flags are deliberately excluded: --start, --end, --seed, --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts. Per-job folders All state for one video now lives in <output_dir>/clips/<video name>/ — clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level. tts_input.srt moved with the clips because they are a matched pair: block N in the SRT is clip_NNNN.flac. Previously both sat at the top level and each new job overwrote the last one's, so rebuilding an older film silently paired its clips with the newest film's text. Any past job can now be rebuilt. A run against the old flat layout gets an explanatory message. gudowski/ -> voices/ VOICE_PROMPTS_DIR is now resolved relative to the script rather than hardcoded, so moving the project no longer breaks it. Voice filenames are unchanged, so --voice gudowski_short still works. Docs updated throughout for the new layout, settings file and poll behaviour. Known issue, not addressed here: with a non-zero --start the voiceover plays early. extract_video_fragment() stream-copies, so it can only cut on a keyframe and seeks backwards to it, while TTS timestamps are relative to the requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is 458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with --start 00:00 are unaffected, since 0 is always a keyframe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lektor_ai_comfy — Documentation
Converts SRT subtitles into a Polish AI voiceover in the lektor style (a single narrator read over the quietened original soundtrack) using a local ComfyUI instance running the Higgs Audio V3 TTS model, then muxes the result back into the video as a second audio track.
The source video is never re-encoded. Video and the original audio are stream-copied; only the new voiceover track is encoded.
Documentation map
| Document | What it covers |
|---|---|
| requirements.md | Hardware, software, and the exact dependency surface |
| installation.md | Installing ComfyUI + the TTS engine, and verifying it |
| usage.md | Every CLI flag, the working modes, and practical recipes |
| architecture.md | How the pipeline works internally, end to end |
| troubleshooting.md | Diagnosed failure modes and their fixes |
Quick start
Assuming ComfyUI is already installed (if not, see installation.md):
# 1. Start the TTS backend and leave it running
~/comfy-tts-app/start_comfyui.sh
# 2. In another terminal, generate the voiceover
cd ~/Programy/lektor_ai
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs
The result is output/film_lektor.mkv with two audio tracks — Original and
Polish — plus every subtitle track from the source.
The two halves of this project
The pieces are deliberately decoupled and it helps to keep them apart when diagnosing anything:
ComfyUI + Higgs Audio V3 — a local HTTP service on port 8189 that turns text into speech. Heavy: ~22 GB on disk, needs an NVIDIA GPU, lives in its own conda environment. It knows nothing about subtitles or video.
lektor_ai_comfy.py — a single-file script that parses subtitles, drives
ComfyUI over HTTP, and does all timing and audio/video assembly with ffmpeg. It
is pure Python standard library, runs on the system Python, and needs no conda
environment or pip packages of its own.
So a TTS problem (bad pronunciation, hallucination, mumbling) belongs to the first half, and a timing, mixing, or muxing problem belongs to the second.
Conventions in these docs
Paths are written as they exist on the machine this was developed on:
~/Programy/lektor_ai the project
~/comfy-tts-app/ComfyUI the TTS backend
~/miniforge3 the conda distribution
If yours differ, update the constants at the top of lektor_ai_comfy.py — see
installation.md.