claudeandClaude Opus 5
3dde6e58e4
Poll interval wait_for_completion() polled every 3 s, which quantised every clip to a 3 s boundary — across 2075 existing clips, 91% of the gaps landed exactly on a multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so short clips are picked up immediately without hammering the server on long ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll, a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real generation time, e.g. "done (4.2s)". A WebSocket would remove polling entirely but needs a third-party client, which would cost this script its dependency-free property. settings.ini voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms, orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next to the script. Precedence is CLI flag > settings.ini > built-in default, wired by feeding loaded values in as argparse defaults. --settings FILE selects an alternative profile. INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or JSON (no comments). A missing, partial, malformed or corrupt file degrades to built-in defaults with a warning instead of killing a long run. Run-specific flags are deliberately excluded: --start, --end, --seed, --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts. Per-job folders All state for one video now lives in <output_dir>/clips/<video name>/ — clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level. tts_input.srt moved with the clips because they are a matched pair: block N in the SRT is clip_NNNN.flac. Previously both sat at the top level and each new job overwrote the last one's, so rebuilding an older film silently paired its clips with the newest film's text. Any past job can now be rebuilt. A run against the old flat layout gets an explanatory message. gudowski/ -> voices/ VOICE_PROMPTS_DIR is now resolved relative to the script rather than hardcoded, so moving the project no longer breaks it. Voice filenames are unchanged, so --voice gudowski_short still works. Docs updated throughout for the new layout, settings file and poll behaviour. Known issue, not addressed here: with a non-zero --start the voiceover plays early. extract_video_fragment() stream-copies, so it can only cut on a keyframe and seeks backwards to it, while TTS timestamps are relative to the requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is 458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with --start 00:00 are unaffected, since 0 is always a keyframe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
218 lines
6.8 KiB
Markdown
218 lines
6.8 KiB
Markdown
# Installation
|
||
|
||
Two things need to exist: the **ComfyUI backend** (large, isolated, GPU-bound)
|
||
and the **script** (which needs nothing installed at all).
|
||
|
||
## Automated install
|
||
|
||
`install_comfyui.sh` in the project root reproduces the exact working
|
||
environment. It is idempotent — safe to re-run, and it skips any step already
|
||
satisfied.
|
||
|
||
```bash
|
||
cd ~/Programy/lektor_ai
|
||
./install_comfyui.sh
|
||
```
|
||
|
||
Roughly 30–60 minutes, mostly downloading. Expect ~22 GB.
|
||
|
||
### Options
|
||
|
||
| Flag | Effect |
|
||
|---|---|
|
||
| *(none)* | Pinned versions — ComfyUI `7cb784e0`, suite `v5.3.0`. Recommended. |
|
||
| `--latest` | Newest ComfyUI and suite instead. You are on your own for breakage. |
|
||
| `--skip-model` | Skip the 8.7 GB download; it happens on first generation anyway. |
|
||
| `--lock FILE` | Restore an exact package set from a `pip freeze` lockfile. |
|
||
| `--help` | Usage summary. |
|
||
|
||
### Changing where things go
|
||
|
||
```bash
|
||
CONDA_ROOT=/opt/miniforge3 COMFY_ROOT=/data/comfy ./install_comfyui.sh
|
||
```
|
||
|
||
| Variable | Default |
|
||
|---|---|
|
||
| `CONDA_ROOT` | `~/miniforge3` |
|
||
| `COMFY_ROOT` | `~/comfy-tts-app` |
|
||
| `ENV_NAME` | `comfy-tts` |
|
||
| `COMFY_PORT` | `8189` |
|
||
| `PYTHON_VERSION` | `3.12` |
|
||
|
||
If you change `COMFY_ROOT` or `COMFY_PORT`, update the script constants too —
|
||
see [Configuring the script](#configuring-the-script).
|
||
|
||
### Exact reproduction
|
||
|
||
`comfyui-requirements.lock.txt` pins all 232 packages. To rebuild a
|
||
byte-identical environment:
|
||
|
||
```bash
|
||
./install_comfyui.sh --lock comfyui-requirements.lock.txt
|
||
```
|
||
|
||
Regenerate it after intentional changes:
|
||
|
||
```bash
|
||
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
|
||
pip freeze > comfyui-requirements.lock.txt
|
||
```
|
||
|
||
## What the installer does
|
||
|
||
Useful to know when something fails midway, since each step can be repeated on
|
||
its own.
|
||
|
||
1. **Preflight** — checks Linux, git, a downloader, GPU, disk space, ffmpeg.
|
||
Only missing git or a downloader is fatal; the rest warn.
|
||
2. **Miniforge** → `~/miniforge3`, batch mode. `conda init` is deliberately not
|
||
run, so your `.bashrc` is never touched.
|
||
3. **Conda env** `comfy-tts` with Python 3.12, then a guard that refuses to
|
||
continue if the wrong Python somehow ended up active.
|
||
4. **PyTorch** from the cu128 index, pinned to 2.11.0.
|
||
5. **ComfyUI** cloned and checked out at the pinned commit (detached HEAD — this
|
||
is expected, not an error), then its `requirements.txt` installed.
|
||
6. **TTS-Audio-Suite** cloned, checked out at `v5.3.0`, then **`python
|
||
install.py`** — its own resolver, which must not be replaced with a plain
|
||
pip install (see [requirements.md](requirements.md#dependency-surface)).
|
||
7. **Model** — `bosonai/higgs-audio-v3-tts-4b` into
|
||
`ComfyUI/models/TTS/higgs_audio_v3/`.
|
||
8. **Launcher** written to `$COMFY_ROOT/start_comfyui.sh`.
|
||
9. **Verification** — imports torch, torchaudio, transformers, safetensors,
|
||
soundfile, librosa and reports CUDA availability.
|
||
|
||
## Manual install
|
||
|
||
The equivalent by hand, if you would rather not use the script:
|
||
|
||
```bash
|
||
# Miniforge, self-contained, no shell integration
|
||
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
|
||
bash Miniforge3-Linux-x86_64.sh -b -p ~/miniforge3
|
||
source ~/miniforge3/etc/profile.d/conda.sh
|
||
|
||
conda create -n comfy-tts python=3.12 -y
|
||
conda activate comfy-tts
|
||
|
||
# PyTorch with the CUDA 12.8 runtime bundled
|
||
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
|
||
--index-url https://download.pytorch.org/whl/cu128
|
||
|
||
# ComfyUI, pinned
|
||
mkdir -p ~/comfy-tts-app && cd ~/comfy-tts-app
|
||
git clone https://github.com/comfyanonymous/ComfyUI.git
|
||
cd ComfyUI && git checkout 7cb784e0f48784bb6ed588912e186e5ee1e9ee68
|
||
pip install -r requirements.txt
|
||
|
||
# The TTS node, pinned — note install.py, NOT requirements.txt
|
||
cd custom_nodes
|
||
git clone https://github.com/diodiogod/TTS-Audio-Suite.git
|
||
cd TTS-Audio-Suite && git checkout tags/v5.3.0
|
||
python install.py
|
||
```
|
||
|
||
The model downloads itself on first generation. To fetch it ahead of time:
|
||
|
||
```python
|
||
from huggingface_hub import snapshot_download
|
||
snapshot_download(
|
||
repo_id="bosonai/higgs-audio-v3-tts-4b",
|
||
local_dir="~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b",
|
||
)
|
||
```
|
||
|
||
## Running the backend
|
||
|
||
```bash
|
||
~/comfy-tts-app/start_comfyui.sh
|
||
```
|
||
|
||
which is just:
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
source ~/miniforge3/etc/profile.d/conda.sh
|
||
conda activate comfy-tts
|
||
cd ~/comfy-tts-app/ComfyUI && python main.py --port 8189
|
||
```
|
||
|
||
Leave it running in its own terminal for the whole session — model load takes
|
||
tens of seconds and would otherwise be repeated. The web UI at
|
||
<http://127.0.0.1:8189> is useful for watching progress and reading errors, but
|
||
is not needed; the script drives everything over HTTP.
|
||
|
||
## Configuring the script
|
||
|
||
Most settings live in **`settings.ini`** next to the script — see
|
||
[usage.md](usage.md#settings-file). The backend address in particular:
|
||
|
||
```ini
|
||
[comfy]
|
||
comfy_url = http://127.0.0.1:8189
|
||
```
|
||
|
||
Two paths remain constants in `lektor_ai_comfy.py`, because the script touches
|
||
ComfyUI's own directories directly rather than over HTTP:
|
||
|
||
```python
|
||
COMFY_OUTPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
|
||
COMFY_INPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/input")
|
||
```
|
||
|
||
Generated audio is read straight out of the output directory and voice
|
||
references are copied into the input directory, so both must be real paths
|
||
reachable from wherever the script runs. **If you installed ComfyUI anywhere
|
||
other than `~/comfy-tts-app`, edit these two lines.** They are also what would
|
||
have to change first to drive a ComfyUI instance on another machine.
|
||
|
||
A third path, `VOICE_PROMPTS_DIR`, resolves to `voices/` beside the script and
|
||
needs no editing as long as the project stays intact.
|
||
|
||
The script itself needs no installation. Ensure ffmpeg is present:
|
||
|
||
```bash
|
||
ffmpeg -version && ffprobe -version
|
||
```
|
||
|
||
If missing, and you would rather not use sudo:
|
||
|
||
```bash
|
||
conda install -n comfy-tts -c conda-forge ffmpeg
|
||
```
|
||
|
||
Note that this puts ffmpeg inside the conda env, so the script must then be run
|
||
with that env active.
|
||
|
||
## Verifying
|
||
|
||
```bash
|
||
# 1. Backend responds
|
||
curl -s http://127.0.0.1:8189/system_stats | head -c 200
|
||
|
||
# 2. GPU visible to PyTorch
|
||
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
|
||
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
|
||
# -> 2.11.0+cu128 True
|
||
|
||
# 3. Model present
|
||
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors
|
||
|
||
# 4. End-to-end on 60 seconds of a real file
|
||
cd ~/Programy/lektor_ai
|
||
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00
|
||
```
|
||
|
||
The short run is the real test — it exercises subtitle parsing, TTS, assembly
|
||
and muxing, and finishes in a couple of minutes.
|
||
|
||
## Uninstalling
|
||
|
||
Nothing was installed system-wide, so removal is just deleting directories:
|
||
|
||
```bash
|
||
conda env remove -n comfy-tts
|
||
rm -rf ~/comfy-tts-app
|
||
rm -rf ~/miniforge3 # only if nothing else uses it
|
||
```
|