lektor-ai/docs/README.md

# lektor_ai_comfy — Documentation

Converts SRT subtitles into a Polish AI voiceover in the *lektor* style (a single
narrator read over the quietened original soundtrack) using a local ComfyUI
instance running the Higgs Audio V3 TTS model, then muxes the result back into
the video as a second audio track.

The source video is never re-encoded. Video and the original audio are
stream-copied; only the new voiceover track is encoded.

## Documentation map

| Document | What it covers |
|---|---|
| [requirements.md](requirements.md) | Hardware, software, and the exact dependency surface |
| [installation.md](installation.md) | Installing ComfyUI + the TTS engine, and verifying it |
| [usage.md](usage.md) | Every CLI flag, the working modes, and practical recipes |
| [architecture.md](architecture.md) | How the pipeline works internally, end to end |
| [troubleshooting.md](troubleshooting.md) | Diagnosed failure modes and their fixes |
| [../normalizer/README.md](../normalizer/README.md) | The Polish number normalizer, its own component |

## Quick start

Assuming ComfyUI is already installed (if not, see
[installation.md](installation.md)):

```bash
# 1. Start the TTS backend and leave it running
~/comfy-tts-app/start_comfyui.sh

# 2. In another terminal, generate the voiceover
cd ~/Programy/lektor_ai
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs
```

The result is `output/film_lektor.mkv` with two audio tracks — **Original** and
**Polish** — plus every subtitle track from the source.

## The three parts of this project

The pieces are deliberately decoupled — each in its own environment, talking
over a process boundary — and it helps to keep them apart when diagnosing
anything:

**ComfyUI + Higgs Audio V3** — a local HTTP service on port 8189 that turns text
into speech. Heavy: ~22 GB on disk, needs an NVIDIA GPU, lives in its own conda
environment. It knows nothing about subtitles or video.

**`lektor_ai_comfy.py`** — a single-file script that parses subtitles, drives
ComfyUI over HTTP, and does all timing and audio/video assembly with ffmpeg. It
is pure Python standard library, runs on the system Python, and needs no conda
environment or pip packages of its own.

**`normalizer/`** — an optional component that spells digits out as Polish words
before they reach the TTS. Its own venv (~6.9 GB, mostly CUDA libraries) and
its own 1.1 GB model, because it needs `transformers >= 5` and the ComfyUI
environment must not be disturbed. Driven as a subprocess for one stage of each
run, then closed.

So a TTS problem (bad pronunciation, hallucination, mumbling) belongs to the
first part, a timing, mixing or muxing problem to the second, and a number read
in the wrong case to the third.

## Conventions in these docs

Paths are written as they exist on the machine this was developed on:

```
~/Programy/lektor_ai      the project
~/comfy-tts-app/ComfyUI   the TTS backend
~/miniforge3              the conda distribution
```

If yours differ, update the constants at the top of `lektor_ai_comfy.py` — see
[installation.md](installation.md#configuring-the-script).