Files
lektor-ai/docs/installation.md
T
claudeandClaude Opus 5 e201aadc3e Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:

- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
  into one alternation applied in a single pass, so replacements never chain
  and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
  folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
  destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
  the normalizer to "correct" the name. Paired clips in output/clips/all-caps
  showed the TTS reads both casings the same.

normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:

- Two backends. Plain integers go to num2words for the value and a general
  Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
  fine-tune. Measured on 40 real subtitle lines, each is clearly better than
  the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
  leftover digits, a lost minus sign, a changed number, a lost word. A failure
  falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
  in VRAM is spilled to RAM silently at 5x the cost.

Backend lifecycle:

- ComfyUI is found by port through /proc, so a backend started by hand is
  managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
  outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
  seed (the aborts are seed-dependent, not text-dependent), then skipped and
  reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.

--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 21:55:31 +02:00

8.0 KiB
Raw Blame History

Installation

Two things need to exist: the ComfyUI backend (large, isolated, GPU-bound) and the script (which needs nothing installed at all).

Automated install

install_comfyui.sh in the project root reproduces the exact working environment. It is idempotent — safe to re-run, and it skips any step already satisfied.

cd ~/Programy/lektor_ai
./install_comfyui.sh

Roughly 30–60 minutes, mostly downloading. Expect ~22 GB.

Options

Flag Effect
(none) Pinned versions — ComfyUI 7cb784e0, suite v5.3.0. Recommended.
--latest Newest ComfyUI and suite instead. You are on your own for breakage.
--skip-model Skip the 8.7 GB download; it happens on first generation anyway.
--lock FILE Restore an exact package set from a pip freeze lockfile.
--help Usage summary.

Changing where things go

CONDA_ROOT=/opt/miniforge3 COMFY_ROOT=/data/comfy ./install_comfyui.sh
Variable Default
CONDA_ROOT ~/miniforge3
COMFY_ROOT ~/comfy-tts-app
ENV_NAME comfy-tts
COMFY_PORT 8189
PYTHON_VERSION 3.12

If you change COMFY_ROOT or COMFY_PORT, update the script constants too — see Configuring the script.

Exact reproduction

comfyui-requirements.lock.txt pins all 232 packages. To rebuild a byte-identical environment:

./install_comfyui.sh --lock comfyui-requirements.lock.txt

Regenerate it after intentional changes:

source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
pip freeze > comfyui-requirements.lock.txt

What the installer does

Useful to know when something fails midway, since each step can be repeated on its own.

  1. Preflight — checks Linux, git, a downloader, GPU, disk space, ffmpeg. Only missing git or a downloader is fatal; the rest warn.
  2. Miniforge → ~/miniforge3, batch mode. conda init is deliberately not run, so your .bashrc is never touched.
  3. Conda env comfy-tts with Python 3.12, then a guard that refuses to continue if the wrong Python somehow ended up active.
  4. PyTorch from the cu128 index, pinned to 2.11.0.
  5. ComfyUI cloned and checked out at the pinned commit (detached HEAD — this is expected, not an error), then its requirements.txt installed.
  6. TTS-Audio-Suite cloned, checked out at v5.3.0, then python install.py — its own resolver, which must not be replaced with a plain pip install (see requirements.md).
  7. Model — bosonai/higgs-audio-v3-tts-4b into ComfyUI/models/TTS/higgs_audio_v3/.
  8. Launcher written to $COMFY_ROOT/start_comfyui.sh.
  9. Verification — imports torch, torchaudio, transformers, safetensors, soundfile, librosa and reports CUDA availability.

Manual install

The equivalent by hand, if you would rather not use the script:

# Miniforge, self-contained, no shell integration
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p ~/miniforge3
source ~/miniforge3/etc/profile.d/conda.sh

conda create -n comfy-tts python=3.12 -y
conda activate comfy-tts

# PyTorch with the CUDA 12.8 runtime bundled
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
    --index-url https://download.pytorch.org/whl/cu128

# ComfyUI, pinned
mkdir -p ~/comfy-tts-app && cd ~/comfy-tts-app
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI && git checkout 7cb784e0f48784bb6ed588912e186e5ee1e9ee68
pip install -r requirements.txt

# The TTS node, pinned — note install.py, NOT requirements.txt
cd custom_nodes
git clone https://github.com/diodiogod/TTS-Audio-Suite.git
cd TTS-Audio-Suite && git checkout tags/v5.3.0
python install.py

The model downloads itself on first generation. To fetch it ahead of time:

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="bosonai/higgs-audio-v3-tts-4b",
    local_dir="~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b",
)

Running the backend

~/comfy-tts-app/start_comfyui.sh

which is just:

#!/bin/bash
source ~/miniforge3/etc/profile.d/conda.sh
conda activate comfy-tts
cd ~/comfy-tts-app/ComfyUI && python main.py --port 8189

Leave it running in its own terminal for the whole session — model load takes tens of seconds and would otherwise be repeated. The web UI at http://127.0.0.1:8189 is useful for watching progress and reading errors, but is not needed; the script drives everything over HTTP.

Configuring the script

Most settings live in settings.ini next to the script — see usage.md. The backend address in particular:

[comfy]
comfy_url = http://127.0.0.1:8189

Two paths remain constants in lektor_ai_comfy.py, because the script touches ComfyUI's own directories directly rather than over HTTP:

COMFY_OUTPUT_DIR  = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
COMFY_INPUT_DIR   = Path("/home/lukas/comfy-tts-app/ComfyUI/input")

Generated audio is read straight out of the output directory and voice references are copied into the input directory, so both must be real paths reachable from wherever the script runs. If you installed ComfyUI anywhere other than ~/comfy-tts-app, edit these two lines. They are also what would have to change first to drive a ComfyUI instance on another machine.

A third path, VOICE_PROMPTS_DIR, resolves to voices/ beside the script and needs no editing as long as the project stays intact.

The script itself needs no installation. Ensure ffmpeg is present:

ffmpeg -version && ffprobe -version

If missing, and you would rather not use sudo:

conda install -n comfy-tts -c conda-forge ffmpeg

Note that this puts ffmpeg inside the conda env, so the script must then be run with that env active.

Digits reach the TTS as words rather than figures, in the correct Polish case. It has its own venv on purpose — the model needs transformers >= 5, which must not be forced on the ComfyUI environment installed above.

cd ~/Programy/lektor_ai
normalizer/setup.sh                             # venv + torch (cu128) + transformers
HF_TOKEN=hf_... normalizer/setup.sh --model     # the weights, ~1.1 GB

The weights are gated: accept the terms at https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization while signed in, then take a token from https://huggingface.co/settings/tokens. This is the only step that touches the network; runs are offline afterwards.

Skipping this is fine — the pipeline falls back to num2words (nominative case only) and says so. Turn the whole stage off with numbers = no under [normalize] in settings.ini. See normalizer/README.md.

Verifying

# 1. Backend responds
curl -s http://127.0.0.1:8189/system_stats | head -c 200

# 2. GPU visible to PyTorch
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
# -> 2.11.0+cu128 True

# 3. Model present
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors

# 4. Number normalizer, if installed
normalizer/venv/bin/python normalizer/normalizer.py --status
# -> "mode": "model"   ("fallback" means the weights are missing)

# 5. End-to-end on 60 seconds of a real file
cd ~/Programy/lektor_ai
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00

The short run is the real test — it exercises subtitle parsing, TTS, assembly and muxing, and finishes in a couple of minutes.

Uninstalling

Nothing was installed system-wide, so removal is just deleting directories:

conda env remove -n comfy-tts
rm -rf ~/comfy-tts-app
rm -rf ~/Programy/lektor_ai/normalizer/venv    # number normalizer env
rm -rf ~/Programy/lektor_ai/normalizer/model   # and its weights
rm -rf ~/miniforge3        # only if nothing else uses it