claudeandClaude Opus 5
97866e6623
install_comfyui.sh becomes install_backend.sh and now builds everything the app needs, not just the speech half: - Step 7 runs normalizer/setup.sh --model, which fetches both models: the open inflection model first, so the install gets as far as it can before anything can ask for credentials, then the gated fine-tune. - A missing Hugging Face account is no longer fatal. Accepting a licence is the one thing an installer cannot do, so the rest installs, the verification reports "fallback only (gated model missing)", and the summary prints the three commands that finish it. - --skip-normalizer, disk requirement 35 -> 55 GB, removal instructions and the step list updated to match. Queue logging — queue.log in the first queued video's job folder, which for a single-video run is its own: - Records what went wrong rather than what happened: warnings, tracebacks, ffmpeg errors, guard rejections, clips cut off mid-word or that crashed the backend, and the --regen line carrying their numbers. - ComfyUI failures *and* restarts, so the log shows the recovery rather than reading as though the run simply stopped. GUI-initiated backend actions go in too; they never pass through a job's pipe. - A failure also dumps the job's last 15 lines. Keyword matching alone is not enough — ffmpeg says "Invalid data found when processing input" and nothing that looks like an error. UI: - The output pane follows new output only while already at the bottom. Scroll up and it stays there; scroll back down and it resumes. Reading an earlier warning during a run was impossible while every ffmpeg progress frame yanked the view to the end. - The ComfyUI button's status dot is drawn as its own canvas item, so it can be green (answering) or amber (holds the port, does not answer). A "." inside the label could only ever be the same grey as the words. comfyui.log is capped at 120,000 lines, trimmed on backend start and every five minutes. Rewritten in place, never replaced: tee holds it open in append mode, so swapping the inode would leave the backend writing to an unlinked file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
258 lines
8.7 KiB
Markdown
258 lines
8.7 KiB
Markdown
# Installation
|
||
|
||
Two things need to exist: the **ComfyUI backend** (large, isolated, GPU-bound)
|
||
and the **script** (which needs nothing installed at all).
|
||
|
||
## Automated install
|
||
|
||
`install_backend.sh` in the project root reproduces the exact working
|
||
environment. It is idempotent — safe to re-run, and it skips any step already
|
||
satisfied.
|
||
|
||
```bash
|
||
cd ~/Programy/lektor_ai
|
||
./install_backend.sh
|
||
```
|
||
|
||
Roughly 45–90 minutes, almost all of it downloading. Expect ~39 GB: ~22 GB for
|
||
the speech backend and ~17 GB for the normalizer's own environment and models.
|
||
|
||
### Options
|
||
|
||
| Flag | Effect |
|
||
|---|---|
|
||
| *(none)* | Pinned versions — ComfyUI `7cb784e0`, suite `v5.3.0`. Recommended. |
|
||
| `--latest` | Newest ComfyUI and suite instead. You are on your own for breakage. |
|
||
| `--skip-model` | Skip the 8.7 GB TTS download; it happens on first generation anyway. |
|
||
| `--skip-normalizer` | ComfyUI only. Digits then reach the TTS as digits, which it misreads. |
|
||
| `--lock FILE` | Restore an exact package set from a `pip freeze` lockfile. |
|
||
| `--help` | Usage summary. |
|
||
|
||
### Changing where things go
|
||
|
||
```bash
|
||
CONDA_ROOT=/opt/miniforge3 COMFY_ROOT=/data/comfy ./install_backend.sh
|
||
```
|
||
|
||
| Variable | Default |
|
||
|---|---|
|
||
| `CONDA_ROOT` | `~/miniforge3` |
|
||
| `COMFY_ROOT` | `~/comfy-tts-app` |
|
||
| `ENV_NAME` | `comfy-tts` |
|
||
| `COMFY_PORT` | `8189` |
|
||
| `PYTHON_VERSION` | `3.12` |
|
||
|
||
If you change `COMFY_ROOT` or `COMFY_PORT`, update the script constants too —
|
||
see [Configuring the script](#configuring-the-script).
|
||
|
||
### Exact reproduction
|
||
|
||
`comfyui-requirements.lock.txt` pins all 232 packages. To rebuild a
|
||
byte-identical environment:
|
||
|
||
```bash
|
||
./install_backend.sh --lock comfyui-requirements.lock.txt
|
||
```
|
||
|
||
Regenerate it after intentional changes:
|
||
|
||
```bash
|
||
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
|
||
pip freeze > comfyui-requirements.lock.txt
|
||
```
|
||
|
||
## What the installer does
|
||
|
||
Useful to know when something fails midway, since each step can be repeated on
|
||
its own.
|
||
|
||
1. **Preflight** — checks Linux, git, a downloader, GPU, disk space, ffmpeg.
|
||
Only missing git or a downloader is fatal; the rest warn.
|
||
2. **Miniforge** → `~/miniforge3`, batch mode. `conda init` is deliberately not
|
||
run, so your `.bashrc` is never touched.
|
||
3. **Conda env** `comfy-tts` with Python 3.12, then a guard that refuses to
|
||
continue if the wrong Python somehow ended up active.
|
||
4. **PyTorch** from the cu128 index, pinned to 2.11.0.
|
||
5. **ComfyUI** cloned and checked out at the pinned commit (detached HEAD — this
|
||
is expected, not an error), then its `requirements.txt` installed.
|
||
6. **TTS-Audio-Suite** cloned, checked out at `v5.3.0`, then **`python
|
||
install.py`** — its own resolver, which must not be replaced with a plain
|
||
pip install (see [requirements.md](requirements.md#dependency-surface)).
|
||
7. **Model** — `bosonai/higgs-audio-v3-tts-4b` into
|
||
`ComfyUI/models/TTS/higgs_audio_v3/`.
|
||
8. **Number normalizer** — `normalizer/setup.sh --model`: its own venv with
|
||
`transformers>=5`, the open inflection model, and the gated fine-tune if you
|
||
have credentials. Failure here is reported, not fatal.
|
||
9. **Launcher** written to `$COMFY_ROOT/start_comfyui.sh`.
|
||
10. **Verification** — imports torch, torchaudio, transformers, safetensors,
|
||
soundfile, librosa, reports CUDA availability, and prints whether the
|
||
normalizer came up neural or on the fallback.
|
||
|
||
## Manual install
|
||
|
||
The equivalent by hand, if you would rather not use the script:
|
||
|
||
```bash
|
||
# Miniforge, self-contained, no shell integration
|
||
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
|
||
bash Miniforge3-Linux-x86_64.sh -b -p ~/miniforge3
|
||
source ~/miniforge3/etc/profile.d/conda.sh
|
||
|
||
conda create -n comfy-tts python=3.12 -y
|
||
conda activate comfy-tts
|
||
|
||
# PyTorch with the CUDA 12.8 runtime bundled
|
||
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
|
||
--index-url https://download.pytorch.org/whl/cu128
|
||
|
||
# ComfyUI, pinned
|
||
mkdir -p ~/comfy-tts-app && cd ~/comfy-tts-app
|
||
git clone https://github.com/comfyanonymous/ComfyUI.git
|
||
cd ComfyUI && git checkout 7cb784e0f48784bb6ed588912e186e5ee1e9ee68
|
||
pip install -r requirements.txt
|
||
|
||
# The TTS node, pinned — note install.py, NOT requirements.txt
|
||
cd custom_nodes
|
||
git clone https://github.com/diodiogod/TTS-Audio-Suite.git
|
||
cd TTS-Audio-Suite && git checkout tags/v5.3.0
|
||
python install.py
|
||
```
|
||
|
||
The model downloads itself on first generation. To fetch it ahead of time:
|
||
|
||
```python
|
||
from huggingface_hub import snapshot_download
|
||
snapshot_download(
|
||
repo_id="bosonai/higgs-audio-v3-tts-4b",
|
||
local_dir="~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b",
|
||
)
|
||
```
|
||
|
||
## Running the backend
|
||
|
||
```bash
|
||
~/comfy-tts-app/start_comfyui.sh
|
||
```
|
||
|
||
which is just:
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
source ~/miniforge3/etc/profile.d/conda.sh
|
||
conda activate comfy-tts
|
||
cd ~/comfy-tts-app/ComfyUI && python main.py --port 8189
|
||
```
|
||
|
||
Leave it running in its own terminal for the whole session — model load takes
|
||
tens of seconds and would otherwise be repeated. The web UI at
|
||
<http://127.0.0.1:8189> is useful for watching progress and reading errors, but
|
||
is not needed; the script drives everything over HTTP.
|
||
|
||
## Configuring the script
|
||
|
||
Most settings live in **`settings.ini`** next to the script — see
|
||
[usage.md](usage.md#settings-file). The backend address in particular:
|
||
|
||
```ini
|
||
[comfy]
|
||
comfy_url = http://127.0.0.1:8189
|
||
```
|
||
|
||
Two paths remain constants in `lektor_ai_comfy.py`, because the script touches
|
||
ComfyUI's own directories directly rather than over HTTP:
|
||
|
||
```python
|
||
COMFY_OUTPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
|
||
COMFY_INPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/input")
|
||
```
|
||
|
||
Generated audio is read straight out of the output directory and voice
|
||
references are copied into the input directory, so both must be real paths
|
||
reachable from wherever the script runs. **If you installed ComfyUI anywhere
|
||
other than `~/comfy-tts-app`, edit these two lines.** They are also what would
|
||
have to change first to drive a ComfyUI instance on another machine.
|
||
|
||
A third path, `VOICE_PROMPTS_DIR`, resolves to `voices/` beside the script and
|
||
needs no editing as long as the project stays intact.
|
||
|
||
The script itself needs no installation. Ensure ffmpeg is present:
|
||
|
||
```bash
|
||
ffmpeg -version && ffprobe -version
|
||
```
|
||
|
||
If missing, and you would rather not use sudo:
|
||
|
||
```bash
|
||
conda install -n comfy-tts -c conda-forge ffmpeg
|
||
```
|
||
|
||
Note that this puts ffmpeg inside the conda env, so the script must then be run
|
||
with that env active.
|
||
|
||
## Number normalizer
|
||
|
||
`install_backend.sh` installs this too — its own venv, the 8 GB inflection
|
||
model, and the 1.1 GB fine-tune. Nothing further is needed unless the script
|
||
says otherwise.
|
||
|
||
It gets its own venv on purpose: it needs `transformers >= 5`, which must not be
|
||
forced on the ComfyUI environment installed above. They talk over a pipe.
|
||
|
||
**The one thing the installer cannot do for you** is accept a licence. The
|
||
fine-tune is gated, so if you have no Hugging Face credentials the script
|
||
installs everything else, reports `fallback only (gated model missing)`, and
|
||
prints these three steps:
|
||
|
||
```bash
|
||
# 1. accept the terms while signed in:
|
||
# https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization
|
||
normalizer/venv/bin/hf auth login
|
||
normalizer/setup.sh --model
|
||
```
|
||
|
||
`HF_TOKEN=hf_...` in the environment works instead of `hf auth login`, and the
|
||
installer picks up either automatically if you set it up beforehand.
|
||
|
||
Until then the pipeline runs with `num2words` for times, dates and identifiers,
|
||
and says so on every run. Turn the stage off entirely with `numbers = no` under
|
||
`[normalize]` in `settings.ini`. See [normalizer/README.md](../normalizer/README.md).
|
||
|
||
## Verifying
|
||
|
||
```bash
|
||
# 1. Backend responds
|
||
curl -s http://127.0.0.1:8189/system_stats | head -c 200
|
||
|
||
# 2. GPU visible to PyTorch
|
||
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
|
||
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
|
||
# -> 2.11.0+cu128 True
|
||
|
||
# 3. Model present
|
||
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors
|
||
|
||
# 4. Number normalizer, if installed
|
||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||
# -> "mode": "model" ("fallback" means the weights are missing)
|
||
|
||
# 5. End-to-end on 60 seconds of a real file
|
||
cd ~/Programy/lektor_ai
|
||
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00
|
||
```
|
||
|
||
The short run is the real test — it exercises subtitle parsing, TTS, assembly
|
||
and muxing, and finishes in a couple of minutes.
|
||
|
||
## Uninstalling
|
||
|
||
Nothing was installed system-wide, so removal is just deleting directories:
|
||
|
||
```bash
|
||
conda env remove -n comfy-tts
|
||
rm -rf ~/comfy-tts-app
|
||
rm -rf ~/Programy/lektor_ai/normalizer/venv # number normalizer env
|
||
rm -rf ~/Programy/lektor_ai/normalizer/model # and its weights
|
||
rm -rf ~/miniforge3 # only if nothing else uses it
|
||
```
|