67 lines
2.5 KiB
Markdown
67 lines
2.5 KiB
Markdown
 
|
# lektor_ai_comfy — Documentation
|
||
|
|
|
||
|
|
Converts SRT subtitles into a Polish AI voiceover in the *lektor* style (a single
|
||
|
|
narrator read over the quietened original soundtrack) using a local ComfyUI
|
||
|
|
instance running the Higgs Audio V3 TTS model, then muxes the result back into
|
||
|
|
the video as a second audio track.
|
||
|
|
|
||
|
|
The source video is never re-encoded. Video and the original audio are
|
||
|
|
stream-copied; only the new voiceover track is encoded.
|
||
|
|
|
||
|
|
## Documentation map
|
||
|
|
|
||
|
|
| Document | What it covers |
|
||
|
|
|---|---|
|
||
|
|
| [requirements.md](requirements.md) | Hardware, software, and the exact dependency surface |
|
||
|
|
| [installation.md](installation.md) | Installing ComfyUI + the TTS engine, and verifying it |
|
||
|
|
| [usage.md](usage.md) | Every CLI flag, the working modes, and practical recipes |
|
||
|
|
| [architecture.md](architecture.md) | How the pipeline works internally, end to end |
|
||
|
|
| [troubleshooting.md](troubleshooting.md) | Diagnosed failure modes and their fixes |
|
||
|
|
|
||
|
|
## Quick start
|
||
|
|
|
||
|
|
Assuming ComfyUI is already installed (if not, see
|
||
|
|
[installation.md](installation.md)):
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# 1. Start the TTS backend and leave it running
|
||
|
|
~/comfy-tts-app/start_comfyui.sh
|
||
|
|
|
||
|
|
# 2. In another terminal, generate the voiceover
|
||
|
|
cd ~/Programy/lektor_ai
|
||
|
|
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs
|
||
|
|
```
|
||
|
|
|
||
|
|
The result is `output/film_lektor.mkv` with two audio tracks — **Original** and
|
||
|
|
**Polish** — plus every subtitle track from the source.
|
||
|
|
|
||
|
|
## The two halves of this project
|
||
|
|
|
||
|
|
The pieces are deliberately decoupled and it helps to keep them apart when
|
||
|
|
diagnosing anything:
|
||
|
|
|
||
|
|
**ComfyUI + Higgs Audio V3** — a local HTTP service on port 8189 that turns text
|
||
|
|
into speech. Heavy: ~22 GB on disk, needs an NVIDIA GPU, lives in its own conda
|
||
|
|
environment. It knows nothing about subtitles or video.
|
||
|
|
|
||
|
|
**`lektor_ai_comfy.py`** — a single-file script that parses subtitles, drives
|
||
|
|
ComfyUI over HTTP, and does all timing and audio/video assembly with ffmpeg. It
|
||
|
|
is pure Python standard library, runs on the system Python, and needs no conda
|
||
|
|
environment or pip packages of its own.
|
||
|
|
|
||
|
|
So a TTS problem (bad pronunciation, hallucination, mumbling) belongs to the
|
||
|
|
first half, and a timing, mixing, or muxing problem belongs to the second.
|
||
|
|
|
||
|
|
## Conventions in these docs
|
||
|
|
|
||
|
|
Paths are written as they exist on the machine this was developed on:
|
||
|
|
|
||
|
|
```
|
||
|
|
~/Programy/lektor_ai the project
|
||
|
|
~/comfy-tts-app/ComfyUI the TTS backend
|
||
|
|
~/miniforge3 the conda distribution
|
||
|
|
```
|
||
|
|
|
||
|
|
If yours differ, update the constants at the top of `lektor_ai_comfy.py` — see
|
||
|
|
[installation.md](installation.md#configuring-the-script).
|