Files

T

claudeandClaude Opus 5 3dde6e58e4 Settings file, per-job clip folders, faster polling, voices/ rename

Poll interval
  wait_for_completion() polled every 3 s, which quantised every clip to a 3 s
  boundary — across 2075 existing clips, 91% of the gaps landed exactly on a
  multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so
  short clips are picked up immediately without hammering the server on long
  ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll,
  a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real
  generation time, e.g. "done (4.2s)".

  A WebSocket would remove polling entirely but needs a third-party client,
  which would cost this script its dependency-free property.

settings.ini
  voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
  orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next
  to the script. Precedence is CLI flag > settings.ini > built-in default,
  wired by feeding loaded values in as argparse defaults. --settings FILE
  selects an alternative profile.

  INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or
  JSON (no comments). A missing, partial, malformed or corrupt file degrades to
  built-in defaults with a warning instead of killing a long run.

  Run-specific flags are deliberately excluded: --start, --end, --seed,
  --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts.

Per-job folders
  All state for one video now lives in <output_dir>/clips/<video name>/ —
  clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and
  video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level.

  tts_input.srt moved with the clips because they are a matched pair: block N
  in the SRT is clip_NNNN.flac. Previously both sat at the top level and each
  new job overwrote the last one's, so rebuilding an older film silently
  paired its clips with the newest film's text. Any past job can now be
  rebuilt. A run against the old flat layout gets an explanatory message.

gudowski/ -> voices/
  VOICE_PROMPTS_DIR is now resolved relative to the script rather than
  hardcoded, so moving the project no longer breaks it. Voice filenames are
  unchanged, so --voice gudowski_short still works.

Docs updated throughout for the new layout, settings file and poll behaviour.

Known issue, not addressed here: with a non-zero --start the voiceover plays
early. extract_video_fragment() stream-copies, so it can only cut on a
keyframe and seeks backwards to it, while TTS timestamps are relative to the
requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is
458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with
--start 00:00 are unaffected, since 0 is always a keyframe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-09 21:56:42 +02:00

5.4 KiB

Raw Blame History

Requirements

Hardware

	Minimum	Notes
GPU	NVIDIA, 12 GB VRAM	Developed on an RTX 4070 Ti SUPER (16 GB)
Driver	525+	Needed by the CUDA 12.8 wheels
Disk	~35 GB free	~22 GB installed, plus working room for output
RAM	16 GB	The final mux briefly peaks around 2.6 GB

The CUDA toolkit is not required. The PyTorch cu128 wheels bundle their own CUDA runtime, so the system nvcc version is irrelevant — this machine has CUDA 11.5 installed system-wide while PyTorch runs cu128 perfectly well. Only the driver version matters.

CPU-only inference technically works but is slow enough to be impractical for a feature-length film.

Disk footprint

~/miniforge3/envs/comfy-tts    9.7 GB   conda env (PyTorch and ~233 packages)
~/comfy-tts-app/ComfyUI        3.2 GB   application code and dependencies
  └── models/TTS               8.8 GB   Higgs Audio V3 weights
                              ------
                              21.7 GB

Output is roughly the size of the source video, since video is stream-copied. A 60-minute 10 Mbit/s episode produces about 5 GB. Intermediate FLAC clips add a few hundred MB per film and are kept deliberately, so individual clips can be regenerated without redoing a whole run (see usage.md).

Software

Component	Version here	Notes
OS	Linux (Ubuntu 22.04)	Only Linux is supported by the installer
ComfyUI	0.26.0 (`7cb784e0`)	Pinned
TTS-Audio-Suite	v5.3.0 (`09df1681`)	The only custom node installed
Higgs Audio V3	`bosonai/higgs-audio-v3-tts-4b`	8.7 GB, auto-downloaded
Python (ComfyUI)	3.12.13	conda-forge, in the `comfy-tts` env
PyTorch	2.11.0+cu128	With torchvision 0.26.0, torchaudio 2.11.0
Python (script)	3.8+	System Python; 3.10.12 here
ffmpeg / ffprobe	4.4+	System install; 4.4.2 here

Dependency surface

This is the point most worth internalising, because it makes debugging much simpler: the two halves have almost nothing in common.

`lektor_ai_comfy.py` — standard library only

argparse  json  random  re  shutil  subprocess  sys  threading  time  uuid
pathlib.Path            urllib.request

No third-party packages. No conda environment. No virtualenv. It runs on the system Python:

/usr/bin/python3 lektor_ai_comfy.py --help    # works, nothing installed

Its only external requirements are ffmpeg and ffprobe on PATH, and a reachable ComfyUI instance. It talks to ComfyUI over plain HTTP with urllib.request — there is no ComfyUI client library involved.

requirements.txt in the project root does not apply here. It belongs to the older lektor_ai.py, which used the hosted Boson AI API and needed openai>=1.0.0. lektor_ai_comfy.py requires nothing from it.

ComfyUI — a large, isolated environment

233 pip packages in the comfy-tts conda env. The important ones:

torch 2.11.0+cu128      torchaudio 2.11.0+cu128     torchvision 0.26.0+cu128
transformers 5.12.1     tokenizers 0.22.2           accelerate 1.14.0
numpy 2.2.6             scipy 1.18.0                librosa 0.11.0
soundfile 0.14.0        safetensors 0.8.0           einops 0.8.2

The conda environment itself holds only the 27 base conda-forge packages that make up the Python runtime. Everything else arrives via pip. In particular, portaudio, libsamplerate, ffmpeg and espeak-ng are not installed in the environment despite appearing in the original planning notes — the system ffmpeg is used and espeak-ng is absent entirely. Everything works without them.

TTS-Audio-Suite installs its dependencies through its own install.py, not through its requirements.txt. That file deliberately omits packages needing special handling — librosa, descript-audio-codec, onnxruntime, opencv-python and others are installed with --no-deps because pip would otherwise drag numpy backwards and break the rest of the environment. Never substitute pip install -r requirements.txt for python install.py.

Input requirements

Video — anything ffmpeg can demux. MKV and MP4 are the usual cases. The container must support what gets muxed back in; output is always MKV, which accepts every codec involved.

Subtitles — either an external .srt, or a text-based track inside the video (subrip, ass, ssa, webvtt, mov_text). Image-based tracks (hdmv_pgs_subtitle from Blu-ray, dvd_subtitle) contain bitmaps rather than text and cannot drive TTS without OCR; the --embedded-subs picker marks them as unusable. They are still copied to the output untouched.

Voice reference — a .wav plus a .txt holding its exact transcript, both in voices/. Three are bundled:

Voice	Length	Format
`gudowski`	2:10	44.1 kHz stereo
`gudowski_short` (default)	0:52	44.1 kHz stereo
`gudowski_short_2`	0:28	44.1 kHz stereo

The transcript matters as much as the audio — voice cloning quality drops noticeably without an accurate one. The script warns if the .txt is missing.

Network

Needed only at install time, for the ComfyUI and TTS-Audio-Suite clones, the pip downloads, and the 8.7 GB model. Generation afterwards is entirely local; no API keys, no accounts, no per-use cost.

The model is published under a research / non-commercial licence and may require accepting its terms on huggingface.co, then hf auth login, before it will download.

5.4 KiB Raw Blame History