claudeandClaude Opus 5
3dde6e58e4
Poll interval wait_for_completion() polled every 3 s, which quantised every clip to a 3 s boundary — across 2075 existing clips, 91% of the gaps landed exactly on a multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so short clips are picked up immediately without hammering the server on long ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll, a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real generation time, e.g. "done (4.2s)". A WebSocket would remove polling entirely but needs a third-party client, which would cost this script its dependency-free property. settings.ini voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms, orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next to the script. Precedence is CLI flag > settings.ini > built-in default, wired by feeding loaded values in as argparse defaults. --settings FILE selects an alternative profile. INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or JSON (no comments). A missing, partial, malformed or corrupt file degrades to built-in defaults with a warning instead of killing a long run. Run-specific flags are deliberately excluded: --start, --end, --seed, --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts. Per-job folders All state for one video now lives in <output_dir>/clips/<video name>/ — clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level. tts_input.srt moved with the clips because they are a matched pair: block N in the SRT is clip_NNNN.flac. Previously both sat at the top level and each new job overwrote the last one's, so rebuilding an older film silently paired its clips with the newest film's text. Any past job can now be rebuilt. A run against the old flat layout gets an explanatory message. gudowski/ -> voices/ VOICE_PROMPTS_DIR is now resolved relative to the script rather than hardcoded, so moving the project no longer breaks it. Voice filenames are unchanged, so --voice gudowski_short still works. Docs updated throughout for the new layout, settings file and poll behaviour. Known issue, not addressed here: with a non-zero --start the voiceover plays early. extract_video_fragment() stream-copies, so it can only cut on a keyframe and seeks backwards to it, while TTS timestamps are relative to the requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is 458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with --start 00:00 are unaffected, since 0 is always a keyframe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
133 lines
5.4 KiB
Markdown
133 lines
5.4 KiB
Markdown
# Requirements
|
|
|
|
## Hardware
|
|
|
|
| | Minimum | Notes |
|
|
|---|---|---|
|
|
| GPU | NVIDIA, 12 GB VRAM | Developed on an RTX 4070 Ti SUPER (16 GB) |
|
|
| Driver | 525+ | Needed by the CUDA 12.8 wheels |
|
|
| Disk | ~35 GB free | ~22 GB installed, plus working room for output |
|
|
| RAM | 16 GB | The final mux briefly peaks around 2.6 GB |
|
|
|
|
The CUDA **toolkit** is not required. The PyTorch cu128 wheels bundle their own
|
|
CUDA runtime, so the system `nvcc` version is irrelevant — this machine has CUDA
|
|
11.5 installed system-wide while PyTorch runs cu128 perfectly well. Only the
|
|
driver version matters.
|
|
|
|
CPU-only inference technically works but is slow enough to be impractical for a
|
|
feature-length film.
|
|
|
|
### Disk footprint
|
|
|
|
```
|
|
~/miniforge3/envs/comfy-tts 9.7 GB conda env (PyTorch and ~233 packages)
|
|
~/comfy-tts-app/ComfyUI 3.2 GB application code and dependencies
|
|
└── models/TTS 8.8 GB Higgs Audio V3 weights
|
|
------
|
|
21.7 GB
|
|
```
|
|
|
|
Output is roughly the size of the source video, since video is stream-copied. A
|
|
60-minute 10 Mbit/s episode produces about 5 GB. Intermediate FLAC clips add a
|
|
few hundred MB per film and are kept deliberately, so individual clips can be
|
|
regenerated without redoing a whole run (see
|
|
[usage.md](usage.md#working-modes)).
|
|
|
|
## Software
|
|
|
|
| Component | Version here | Notes |
|
|
|---|---|---|
|
|
| OS | Linux (Ubuntu 22.04) | Only Linux is supported by the installer |
|
|
| ComfyUI | 0.26.0 (`7cb784e0`) | Pinned |
|
|
| TTS-Audio-Suite | v5.3.0 (`09df1681`) | The only custom node installed |
|
|
| Higgs Audio V3 | `bosonai/higgs-audio-v3-tts-4b` | 8.7 GB, auto-downloaded |
|
|
| Python (ComfyUI) | 3.12.13 | conda-forge, in the `comfy-tts` env |
|
|
| PyTorch | 2.11.0+cu128 | With torchvision 0.26.0, torchaudio 2.11.0 |
|
|
| Python (script) | 3.8+ | System Python; 3.10.12 here |
|
|
| ffmpeg / ffprobe | 4.4+ | System install; 4.4.2 here |
|
|
|
|
## Dependency surface
|
|
|
|
This is the point most worth internalising, because it makes debugging much
|
|
simpler: **the two halves have almost nothing in common.**
|
|
|
|
### `lektor_ai_comfy.py` — standard library only
|
|
|
|
```python
|
|
argparse json random re shutil subprocess sys threading time uuid
|
|
pathlib.Path urllib.request
|
|
```
|
|
|
|
No third-party packages. No conda environment. No virtualenv. It runs on the
|
|
system Python:
|
|
|
|
```bash
|
|
/usr/bin/python3 lektor_ai_comfy.py --help # works, nothing installed
|
|
```
|
|
|
|
Its only external requirements are **ffmpeg** and **ffprobe** on `PATH`, and a
|
|
reachable ComfyUI instance. It talks to ComfyUI over plain HTTP with
|
|
`urllib.request` — there is no ComfyUI client library involved.
|
|
|
|
> **`requirements.txt` in the project root does not apply here.** It belongs to
|
|
> the older `lektor_ai.py`, which used the hosted Boson AI API and needed
|
|
> `openai>=1.0.0`. `lektor_ai_comfy.py` requires nothing from it.
|
|
|
|
### ComfyUI — a large, isolated environment
|
|
|
|
233 pip packages in the `comfy-tts` conda env. The important ones:
|
|
|
|
```
|
|
torch 2.11.0+cu128 torchaudio 2.11.0+cu128 torchvision 0.26.0+cu128
|
|
transformers 5.12.1 tokenizers 0.22.2 accelerate 1.14.0
|
|
numpy 2.2.6 scipy 1.18.0 librosa 0.11.0
|
|
soundfile 0.14.0 safetensors 0.8.0 einops 0.8.2
|
|
```
|
|
|
|
The conda environment itself holds only the 27 base conda-forge packages that
|
|
make up the Python runtime. Everything else arrives via pip. In particular,
|
|
`portaudio`, `libsamplerate`, `ffmpeg` and `espeak-ng` are **not** installed in
|
|
the environment despite appearing in the original planning notes — the system
|
|
ffmpeg is used and espeak-ng is absent entirely. Everything works without them.
|
|
|
|
TTS-Audio-Suite installs its dependencies through its own `install.py`, not
|
|
through its `requirements.txt`. That file deliberately omits packages needing
|
|
special handling — `librosa`, `descript-audio-codec`, `onnxruntime`,
|
|
`opencv-python` and others are installed with `--no-deps` because pip would
|
|
otherwise drag numpy backwards and break the rest of the environment. Never
|
|
substitute `pip install -r requirements.txt` for `python install.py`.
|
|
|
|
## Input requirements
|
|
|
|
**Video** — anything ffmpeg can demux. MKV and MP4 are the usual cases. The
|
|
container must support what gets muxed back in; output is always MKV, which
|
|
accepts every codec involved.
|
|
|
|
**Subtitles** — either an external `.srt`, or a text-based track inside the
|
|
video (`subrip`, `ass`, `ssa`, `webvtt`, `mov_text`). Image-based tracks
|
|
(`hdmv_pgs_subtitle` from Blu-ray, `dvd_subtitle`) contain bitmaps rather than
|
|
text and cannot drive TTS without OCR; the `--embedded-subs` picker marks them
|
|
as unusable. They are still copied to the output untouched.
|
|
|
|
**Voice reference** — a `.wav` plus a `.txt` holding its exact transcript, both
|
|
in `voices/`. Three are bundled:
|
|
|
|
| Voice | Length | Format |
|
|
|---|---|---|
|
|
| `gudowski` | 2:10 | 44.1 kHz stereo |
|
|
| `gudowski_short` (default) | 0:52 | 44.1 kHz stereo |
|
|
| `gudowski_short_2` | 0:28 | 44.1 kHz stereo |
|
|
|
|
The transcript matters as much as the audio — voice cloning quality drops
|
|
noticeably without an accurate one. The script warns if the `.txt` is missing.
|
|
|
|
## Network
|
|
|
|
Needed only at install time, for the ComfyUI and TTS-Audio-Suite clones, the pip
|
|
downloads, and the 8.7 GB model. Generation afterwards is entirely local; no
|
|
API keys, no accounts, no per-use cost.
|
|
|
|
The model is published under a research / non-commercial licence and may require
|
|
accepting its terms on huggingface.co, then `hf auth login`, before it will
|
|
download.
|