lektor-ai/docs/usage.md

# Usage

ComfyUI must be running before any command that generates speech
(`~/comfy-tts-app/start_comfyui.sh`). `--rebuild` is the exception — it only
touches ffmpeg and works with the backend stopped.

## Invocation

```
python3 lektor_ai_comfy.py [SRT] VIDEO [options]
```

Both positionals are optional in the argparse sense, because several modes do
not need an SRT file. The rule:

```bash
# External subtitle file — pass both, SRT first
python3 lektor_ai_comfy.py input/film.pl.srt input/film.mkv

# Subtitles inside the video — pass the video alone
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs

# --rebuild and --regen also need only the video
python3 lektor_ai_comfy.py input/film.mkv --rebuild
```

When a single positional is given in a mode that needs no SRT, the script moves
it into the video slot automatically. Always quote paths containing spaces or
parentheses:

```bash
python3 lektor_ai_comfy.py "input/Star City (2026)/S01_E01 The Eyes.mkv" --embedded-subs
```

## Working modes

### Full generation (default)

Parses subtitles, generates one TTS clip per block, assembles, muxes.

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs
```

Roughly real-time-ish on a 4070 Ti SUPER: a 60-minute episode of ~700 subtitle
blocks takes on the order of an hour. Each clip is saved as it completes, so a
crash never loses more than the clip in flight.

### `--embedded-subs`

Lists the video's subtitle tracks and prompts for one:

```
  Available subtitle tracks in S01_E01 The Eyes.mkv:
    [1] Stream #2: subrip [eng] — Forced
    [2] Stream #3: subrip [eng]
    [3] Stream #4: subrip [eng] — SDH
   ...
    [30] Stream #31: subrip [pol]
  Select track number: 30
```

The chosen track is extracted to `output/embedded_sub.srt` and then processed
exactly like an external file. Image-based tracks are listed but marked
unusable — they hold bitmaps, not text.

### `--rebuild`

Reassembles audio from existing clips and rebuilds the video, with no TTS at
all. This is the fast iteration loop — seconds to minutes rather than an hour.

```bash
python3 lektor_ai_comfy.py input/film.mkv --rebuild
python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0
```

Use it after `--regen`, or to adjust the mix. It requires `output/tts_input.srt`
and a complete set of `output/clips/clip_NNNN.flac`, and reports which are
missing if any are.

If the original run used `--end`, pass it again so the same video fragment is
used.

### `--regen`

Regenerates specific clips by block number, leaving everything else alone.

```bash
python3 lektor_ai_comfy.py input/film.mkv --regen 12 22 41
python3 lektor_ai_comfy.py input/film.mkv --rebuild        # then apply
```

Block numbers match `output/tts_input.srt` and the `clip_NNNN.flac` filenames.
Only the selected FLAC files are replaced — `--rebuild` afterwards is what
actually updates the video.

This is the intended fix for a bad clip. There is no automatic retry; when the
model hallucinates, mumbles, or truncates, note the number and regenerate it. A
different seed usually resolves it, and `--seed 0` (the default) randomises
every run.

### `--resume-from`

Continues a crashed run, loading clips `1..N-1` from disk and generating from
`N` onward.

```bash
python3 lektor_ai_comfy.py input/film.mkv --resume-from 47
```

The console prints `[47/693]` style progress, so the number to resume from is
the last one that completed.

### `--skip-tts`

Uses a pre-assembled audio file and goes straight to the mux.

```bash
python3 lektor_ai_comfy.py input/film.mkv --skip-tts output/tts_assembled.flac
```

## Options

### Range

| Flag | Default | Meaning |
|---|---|---|
| `--start MM:SS` | `00:00` | Start of the range to process |
| `--end MM:SS` | *(end)* | End of the range |

With `--end`, a matching fragment is extracted to `output/video_fragment.mkv`
and all TTS timestamps become relative to `--start`. Pass the same values to any
later `--rebuild`, or the audio will not line up.

`--end 01:00` is the quickest way to sanity-check a new voice or speed setting.

### Voice and model

| Flag | Default | Meaning |
|---|---|---|
| `--voice NAME\|PATH` | `gudowski_short` | Name in `gudowski/`, or a path to a WAV |
| `--seed N` | `0` | Base seed; `0` randomises each run |
| `--temperature T` | `0.9` | Sampling temperature |
| `--max-tokens N` | `4096` | Token ceiling per clip |

`--temperature` is worth understanding. At `1.0` the model sometimes recites the
reference transcript instead of the subtitle. `0.9` largely prevents that;
lower is more deterministic but flatter. A per-clip seed of `base_seed + index`
means a fixed `--seed` reproduces a run exactly.

Raise `--max-tokens` only if long clips are cut off mid-word.

### Timing and cleanup

| Flag | Default | Meaning |
|---|---|---|
| `--speed FACTOR` | `1.2` | Per-clip speed-up via `atempo` |
| `--strip-leading-ms MS` | `0` | Trim the start of each clip |
| `--trailing-fade-ms MS` | `80` | Fade the end of each clip |

`--speed` compresses speech so a clip is more likely to fit its subtitle window.
It is applied **per clip, before assembly** — never to the assembled track,
which would shift every timestamp by speeding up the silences too.

`--strip-leading-ms` removes an inhale artifact at clip starts; try 80–150 if
you hear one. `--trailing-fade-ms` suppresses tail pops; `0` disables it.

### Mixing

| Flag | Default | Meaning |
|---|---|---|
| `--orig-volume 0-1` | `0.60` | Original audio level inside the lektor mix |
| `--tts-volume 0-2` | `1.8` | Voiceover level inside the lektor mix |

These affect **Track 1 only**. Track 0 is always the untouched original, so
nothing is lost by mixing aggressively. Changing them needs only `--rebuild`.

Typical starting point for dialogue-heavy material: `--orig-volume 0.5
--tts-volume 2.0`. For quiet sources, `--orig-volume 1.0 --tts-volume 1.4`.

### Paths

| Flag | Default | Meaning |
|---|---|---|
| `--output-dir DIR` | `output` | Where everything is written |
| `--comfy-url URL` | `http://127.0.0.1:8189` | Backend address |

## Output

```
output/
  tts_input.srt         processed text actually sent to TTS — inspect this
  embedded_sub.srt      extracted embedded subtitle (--embedded-subs only)
  clips/clip_NNNN.flac  one lossless clip per block, kept for --regen
  tts_assembled.flac    all clips at their final positions, lossless
  video_fragment.mkv    extracted segment (--end only)
  {stem}_lektor.mkv     the finished video
```

The finished file contains:

| Track | Content |
|---|---|
| Video | Stream-copied, untouched |
| Audio 0 | **Original** — stream-copied, full 5.1 if the source had it |
| Audio 1 | **Polish** — lektor mix, AAC 192k stereo, `language=pol` |
| Subtitles | Every track from the source, stream-copied |

The lektor track is stereo deliberately; see
[troubleshooting.md](troubleshooting.md#the-lektor-track-is-silent-in-the-player).

FLAC is used for every intermediate so repeated processing stays lossless. Only
the final voiceover track is ever encoded.

## Recipes

**Check a voice before committing to a full run**

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --end 02:00 --voice gudowski
```

**Tune the mix without regenerating speech**

```bash
python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.45 --tts-volume 2.1
```

**Fix a handful of bad clips**

```bash
# read output/tts_input.srt, note the offending block numbers
python3 lektor_ai_comfy.py input/film.mkv --regen 88 91 104
python3 lektor_ai_comfy.py input/film.mkv --rebuild
```

**Reproduce a run exactly**

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --seed 123456
```

**Speech running long and overlapping the next line**

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --speed 1.35
```

## Reading the console

```
[1/4] Parsing subtitles 00:00–end…
      693 entries after dialogue splitting.
[2/4] Generating TTS (693 clips, speed 1.2×)…
  [12/693] 00:01:34,120  'Nie wiem, o czym mówisz.'
    Waiting for a3f8b201… done.
    → placed at 00:01:34,120, duration 1.84s
```

`→ placed at` reveals timing drift. A clip that overruns its window pushes the
next one forward, shown as `[+1.20s behind SRT]`. Small values are normal and
self-correct at the next natural gap in the subtitles; values that keep growing
mean `--speed` is too low for the material.

Assembly then shows a progress bar per batch (clips are assembled 500 at a time
to stay under the open-file limit), followed by the two-pass mux.