claudeandClaude Opus 5
e201aadc3e
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
245 lines
8.0 KiB
Markdown
245 lines
8.0 KiB
Markdown
# Installation
|
||
|
||
Two things need to exist: the **ComfyUI backend** (large, isolated, GPU-bound)
|
||
and the **script** (which needs nothing installed at all).
|
||
|
||
## Automated install
|
||
|
||
`install_comfyui.sh` in the project root reproduces the exact working
|
||
environment. It is idempotent — safe to re-run, and it skips any step already
|
||
satisfied.
|
||
|
||
```bash
|
||
cd ~/Programy/lektor_ai
|
||
./install_comfyui.sh
|
||
```
|
||
|
||
Roughly 30–60 minutes, mostly downloading. Expect ~22 GB.
|
||
|
||
### Options
|
||
|
||
| Flag | Effect |
|
||
|---|---|
|
||
| *(none)* | Pinned versions — ComfyUI `7cb784e0`, suite `v5.3.0`. Recommended. |
|
||
| `--latest` | Newest ComfyUI and suite instead. You are on your own for breakage. |
|
||
| `--skip-model` | Skip the 8.7 GB download; it happens on first generation anyway. |
|
||
| `--lock FILE` | Restore an exact package set from a `pip freeze` lockfile. |
|
||
| `--help` | Usage summary. |
|
||
|
||
### Changing where things go
|
||
|
||
```bash
|
||
CONDA_ROOT=/opt/miniforge3 COMFY_ROOT=/data/comfy ./install_comfyui.sh
|
||
```
|
||
|
||
| Variable | Default |
|
||
|---|---|
|
||
| `CONDA_ROOT` | `~/miniforge3` |
|
||
| `COMFY_ROOT` | `~/comfy-tts-app` |
|
||
| `ENV_NAME` | `comfy-tts` |
|
||
| `COMFY_PORT` | `8189` |
|
||
| `PYTHON_VERSION` | `3.12` |
|
||
|
||
If you change `COMFY_ROOT` or `COMFY_PORT`, update the script constants too —
|
||
see [Configuring the script](#configuring-the-script).
|
||
|
||
### Exact reproduction
|
||
|
||
`comfyui-requirements.lock.txt` pins all 232 packages. To rebuild a
|
||
byte-identical environment:
|
||
|
||
```bash
|
||
./install_comfyui.sh --lock comfyui-requirements.lock.txt
|
||
```
|
||
|
||
Regenerate it after intentional changes:
|
||
|
||
```bash
|
||
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
|
||
pip freeze > comfyui-requirements.lock.txt
|
||
```
|
||
|
||
## What the installer does
|
||
|
||
Useful to know when something fails midway, since each step can be repeated on
|
||
its own.
|
||
|
||
1. **Preflight** — checks Linux, git, a downloader, GPU, disk space, ffmpeg.
|
||
Only missing git or a downloader is fatal; the rest warn.
|
||
2. **Miniforge** → `~/miniforge3`, batch mode. `conda init` is deliberately not
|
||
run, so your `.bashrc` is never touched.
|
||
3. **Conda env** `comfy-tts` with Python 3.12, then a guard that refuses to
|
||
continue if the wrong Python somehow ended up active.
|
||
4. **PyTorch** from the cu128 index, pinned to 2.11.0.
|
||
5. **ComfyUI** cloned and checked out at the pinned commit (detached HEAD — this
|
||
is expected, not an error), then its `requirements.txt` installed.
|
||
6. **TTS-Audio-Suite** cloned, checked out at `v5.3.0`, then **`python
|
||
install.py`** — its own resolver, which must not be replaced with a plain
|
||
pip install (see [requirements.md](requirements.md#dependency-surface)).
|
||
7. **Model** — `bosonai/higgs-audio-v3-tts-4b` into
|
||
`ComfyUI/models/TTS/higgs_audio_v3/`.
|
||
8. **Launcher** written to `$COMFY_ROOT/start_comfyui.sh`.
|
||
9. **Verification** — imports torch, torchaudio, transformers, safetensors,
|
||
soundfile, librosa and reports CUDA availability.
|
||
|
||
## Manual install
|
||
|
||
The equivalent by hand, if you would rather not use the script:
|
||
|
||
```bash
|
||
# Miniforge, self-contained, no shell integration
|
||
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
|
||
bash Miniforge3-Linux-x86_64.sh -b -p ~/miniforge3
|
||
source ~/miniforge3/etc/profile.d/conda.sh
|
||
|
||
conda create -n comfy-tts python=3.12 -y
|
||
conda activate comfy-tts
|
||
|
||
# PyTorch with the CUDA 12.8 runtime bundled
|
||
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
|
||
--index-url https://download.pytorch.org/whl/cu128
|
||
|
||
# ComfyUI, pinned
|
||
mkdir -p ~/comfy-tts-app && cd ~/comfy-tts-app
|
||
git clone https://github.com/comfyanonymous/ComfyUI.git
|
||
cd ComfyUI && git checkout 7cb784e0f48784bb6ed588912e186e5ee1e9ee68
|
||
pip install -r requirements.txt
|
||
|
||
# The TTS node, pinned — note install.py, NOT requirements.txt
|
||
cd custom_nodes
|
||
git clone https://github.com/diodiogod/TTS-Audio-Suite.git
|
||
cd TTS-Audio-Suite && git checkout tags/v5.3.0
|
||
python install.py
|
||
```
|
||
|
||
The model downloads itself on first generation. To fetch it ahead of time:
|
||
|
||
```python
|
||
from huggingface_hub import snapshot_download
|
||
snapshot_download(
|
||
repo_id="bosonai/higgs-audio-v3-tts-4b",
|
||
local_dir="~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b",
|
||
)
|
||
```
|
||
|
||
## Running the backend
|
||
|
||
```bash
|
||
~/comfy-tts-app/start_comfyui.sh
|
||
```
|
||
|
||
which is just:
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
source ~/miniforge3/etc/profile.d/conda.sh
|
||
conda activate comfy-tts
|
||
cd ~/comfy-tts-app/ComfyUI && python main.py --port 8189
|
||
```
|
||
|
||
Leave it running in its own terminal for the whole session — model load takes
|
||
tens of seconds and would otherwise be repeated. The web UI at
|
||
<http://127.0.0.1:8189> is useful for watching progress and reading errors, but
|
||
is not needed; the script drives everything over HTTP.
|
||
|
||
## Configuring the script
|
||
|
||
Most settings live in **`settings.ini`** next to the script — see
|
||
[usage.md](usage.md#settings-file). The backend address in particular:
|
||
|
||
```ini
|
||
[comfy]
|
||
comfy_url = http://127.0.0.1:8189
|
||
```
|
||
|
||
Two paths remain constants in `lektor_ai_comfy.py`, because the script touches
|
||
ComfyUI's own directories directly rather than over HTTP:
|
||
|
||
```python
|
||
COMFY_OUTPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
|
||
COMFY_INPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/input")
|
||
```
|
||
|
||
Generated audio is read straight out of the output directory and voice
|
||
references are copied into the input directory, so both must be real paths
|
||
reachable from wherever the script runs. **If you installed ComfyUI anywhere
|
||
other than `~/comfy-tts-app`, edit these two lines.** They are also what would
|
||
have to change first to drive a ComfyUI instance on another machine.
|
||
|
||
A third path, `VOICE_PROMPTS_DIR`, resolves to `voices/` beside the script and
|
||
needs no editing as long as the project stays intact.
|
||
|
||
The script itself needs no installation. Ensure ffmpeg is present:
|
||
|
||
```bash
|
||
ffmpeg -version && ffprobe -version
|
||
```
|
||
|
||
If missing, and you would rather not use sudo:
|
||
|
||
```bash
|
||
conda install -n comfy-tts -c conda-forge ffmpeg
|
||
```
|
||
|
||
Note that this puts ffmpeg inside the conda env, so the script must then be run
|
||
with that env active.
|
||
|
||
## Number normalizer (optional, recommended)
|
||
|
||
Digits reach the TTS as words rather than figures, in the correct Polish case.
|
||
It has its own venv on purpose — the model needs `transformers >= 5`, which must
|
||
not be forced on the ComfyUI environment installed above.
|
||
|
||
```bash
|
||
cd ~/Programy/lektor_ai
|
||
normalizer/setup.sh # venv + torch (cu128) + transformers
|
||
HF_TOKEN=hf_... normalizer/setup.sh --model # the weights, ~1.1 GB
|
||
```
|
||
|
||
The weights are gated: accept the terms at
|
||
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> while signed in,
|
||
then take a token from <https://huggingface.co/settings/tokens>. This is the only
|
||
step that touches the network; runs are offline afterwards.
|
||
|
||
Skipping this is fine — the pipeline falls back to `num2words` (nominative case
|
||
only) and says so. Turn the whole stage off with `numbers = no` under
|
||
`[normalize]` in `settings.ini`. See [normalizer/README.md](../normalizer/README.md).
|
||
|
||
## Verifying
|
||
|
||
```bash
|
||
# 1. Backend responds
|
||
curl -s http://127.0.0.1:8189/system_stats | head -c 200
|
||
|
||
# 2. GPU visible to PyTorch
|
||
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
|
||
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
|
||
# -> 2.11.0+cu128 True
|
||
|
||
# 3. Model present
|
||
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors
|
||
|
||
# 4. Number normalizer, if installed
|
||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||
# -> "mode": "model" ("fallback" means the weights are missing)
|
||
|
||
# 5. End-to-end on 60 seconds of a real file
|
||
cd ~/Programy/lektor_ai
|
||
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00
|
||
```
|
||
|
||
The short run is the real test — it exercises subtitle parsing, TTS, assembly
|
||
and muxing, and finishes in a couple of minutes.
|
||
|
||
## Uninstalling
|
||
|
||
Nothing was installed system-wide, so removal is just deleting directories:
|
||
|
||
```bash
|
||
conda env remove -n comfy-tts
|
||
rm -rf ~/comfy-tts-app
|
||
rm -rf ~/Programy/lektor_ai/normalizer/venv # number normalizer env
|
||
rm -rf ~/Programy/lektor_ai/normalizer/model # and its weights
|
||
rm -rf ~/miniforge3 # only if nothing else uses it
|
||
```
|