lektor-ai/docs/usage.md

# Usage

ComfyUI must be running before any command that generates speech
(`~/comfy-tts-app/start_comfyui.sh`). `--rebuild` is the exception — it only
touches ffmpeg and works with the backend stopped.

## Invocation

```
python3 lektor_ai_comfy.py [SRT] VIDEO [options]
```

Both positionals are optional in the argparse sense, because several modes do
not need an SRT file. The rule:

```bash
# External subtitle file — pass both, SRT first
python3 lektor_ai_comfy.py input/film.pl.srt input/film.mkv

# Subtitles inside the video — pass the video alone
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs

# --rebuild and --regen also need only the video
python3 lektor_ai_comfy.py input/film.mkv --rebuild
```

When a single positional is given in a mode that needs no SRT, the script moves
it into the video slot automatically. Always quote paths containing spaces or
parentheses:

```bash
python3 lektor_ai_comfy.py "input/Star City (2026)/S01_E01 The Eyes.mkv" --embedded-subs
```

## Working modes

### Full generation (default)

Parses subtitles, generates one TTS clip per block, assembles, muxes.

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs
```

Roughly real-time-ish on a 4070 Ti SUPER: a 60-minute episode of ~700 subtitle
blocks takes on the order of an hour. Each clip is saved as it completes, so a
crash never loses more than the clip in flight.

### `--embedded-subs`

Lists the video's subtitle tracks and prompts for one:

```
  Available subtitle tracks in S01_E01 The Eyes.mkv:
    [1] Stream #2: subrip [eng] — Forced
    [2] Stream #3: subrip [eng]
    [3] Stream #4: subrip [eng] — SDH
   ...
    [30] Stream #31: subrip [pol]
  Select track number: 30
```

The chosen track is extracted to `embedded_sub.srt` in the job folder and processed
exactly like an external file. Image-based tracks are listed but marked
unusable — they hold bitmaps, not text.

### `--rebuild`

Reassembles audio from existing clips and rebuilds the video, with no TTS at
all. This is the fast iteration loop — seconds to minutes rather than an hour.

```bash
python3 lektor_ai_comfy.py input/film.mkv --rebuild
python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0
```

Use it after `--regen`, or to adjust the mix. It reads that video's job folder
(`output/clips/<video name>/`) and needs its `tts_input.srt` plus a complete set
of `clip_NNNN.flac`, reporting which are missing if any are.

If the original run used `--end`, pass it again so the same video fragment is
used.

### `--regen`

Regenerates specific clips by block number, leaving everything else alone.

```bash
python3 lektor_ai_comfy.py input/film.mkv --regen 12 22 41
python3 lektor_ai_comfy.py input/film.mkv --rebuild        # then apply
```

Block numbers match that job's `tts_input.srt` and its `clip_NNNN.flac` files.
Only the selected FLAC files are replaced — `--rebuild` afterwards is what
actually updates the video.

This is the intended fix for a bad clip. There is no automatic retry; when the
model hallucinates, mumbles, or truncates, note the number and regenerate it. A
different seed usually resolves it, and `--seed 0` (the default) randomises
every run.

### `--resume-from`

Continues a crashed run, loading clips `1..N-1` from disk and generating from
`N` onward.

```bash
python3 lektor_ai_comfy.py input/film.mkv --resume-from 47
```

The console prints `[47/693]` style progress, so the number to resume from is
the last one that completed.

### `--skip-tts`

Uses a pre-assembled audio file and goes straight to the mux.

```bash
python3 lektor_ai_comfy.py input/film.mkv --skip-tts "output/clips/film/tts_assembled.flac"
```

## Settings file

Defaults live in **`settings.ini`** next to the script, so the values you always
want no longer need to be typed on every run. Precedence:

```
command-line flag   >   settings.ini   >   built-in default
```

```ini
[tts]
voice = gudowski_short
speed = 1.2
temperature = 0.9
max_tokens = 4096
strip_leading_ms = 0
trailing_fade_ms = 80

[mix]
orig_volume = 0.60
tts_volume = 1.8

[paths]
output_dir = output

[comfy]
comfy_url = http://127.0.0.1:8189
```

Every one of these has a matching flag that overrides it for a single run:

```bash
# settings.ini says tts_volume = 1.8; this run uses 2.5
python3 lektor_ai_comfy.py film.mkv --embedded-subs --tts-volume 2.5
```

Each run prints which file it used, so there is never any doubt:

```
  Settings: /home/lukas/Programy/lektor_ai/settings.ini
```

**Alternative profiles** — keep per-show settings side by side and select one:

```bash
python3 lektor_ai_comfy.py film.mkv --embedded-subs --settings profiles/docu.ini
```

**Behaviour when things are missing or wrong**, none of which is fatal:

| Situation | Result |
|---|---|
| No `settings.ini` | Built-in defaults; the run says so |
| Key absent from the file | That key falls back to its built-in default |
| Value unparseable (`speed = fast`) | Warning, that key falls back, run continues |
| File is not valid INI | Warning, all built-in defaults, run continues |

Comments use `#` or `;`, inline comments included. Deleting a line is a valid
way to return it to its built-in default.

**Not in the file, by design**: `--start`, `--end`, `--seed`, `--regen`,
`--rebuild`, `--resume-from`, `--embedded-subs`, `--skip-tts`. These describe a
particular run rather than your setup, so pinning them in config would be a
trap.

## Options

Every option below can also be set in `settings.ini`, except those noted as
run-specific.

### Range

| Flag | Default | Meaning |
|---|---|---|
| `--start MM:SS` | `00:00` | Start of the range to process |
| `--end MM:SS` | *(end)* | End of the range |

With `--end`, a matching fragment is extracted to `video_fragment.mkv` in the job folder
and all TTS timestamps become relative to `--start`. Pass the same values to any
later `--rebuild`, or the audio will not line up.

`--end 01:00` is the quickest way to sanity-check a new voice or speed setting.

### Voice and model

| Flag | Default | Meaning |
|---|---|---|
| `--voice NAME\|PATH` | `gudowski_short` | Name in `voices/`, or a path to a WAV |
| `--seed N` | `0` | Base seed; `0` randomises each run |
| `--temperature T` | `0.9` | Sampling temperature |
| `--max-tokens N` | `4096` | Token ceiling per clip |

`--temperature` is worth understanding. At `1.0` the model sometimes recites the
reference transcript instead of the subtitle. `0.9` largely prevents that;
lower is more deterministic but flatter. A per-clip seed of `base_seed + index`
means a fixed `--seed` reproduces a run exactly.

Raise `--max-tokens` only if long clips are cut off mid-word.

### Timing and cleanup

| Flag | Default | Meaning |
|---|---|---|
| `--speed FACTOR` | `1.2` | Per-clip speed-up via `atempo` |
| `--strip-leading-ms MS` | `0` | Trim the start of each clip |
| `--trailing-fade-ms MS` | `80` | Fade the end of each clip |

`--speed` compresses speech so a clip is more likely to fit its subtitle window.
It is applied **per clip, before assembly** — never to the assembled track,
which would shift every timestamp by speeding up the silences too.

`--strip-leading-ms` removes an inhale artifact at clip starts; try 80–150 if
you hear one. `--trailing-fade-ms` suppresses tail pops; `0` disables it.

### Mixing

| Flag | Default | Meaning |
|---|---|---|
| `--orig-volume 0-1` | `0.60` | Original audio level inside the lektor mix |
| `--tts-volume 0-2` | `1.8` | Voiceover level inside the lektor mix |

These affect **Track 1 only**. Track 0 is always the untouched original, so
nothing is lost by mixing aggressively. Changing them needs only `--rebuild`.

Typical starting point for dialogue-heavy material: `--orig-volume 0.5
--tts-volume 2.0`. For quiet sources, `--orig-volume 1.0 --tts-volume 1.4`.

### Paths

| Flag | Default | Meaning |
|---|---|---|
| `--output-dir DIR` | `output` | Where everything is written |
| `--comfy-url URL` | `http://127.0.0.1:8189` | Backend address |

## Output

Everything belonging to one video is kept together in a **job folder** named
after the video file. Only the finished video is written to the top level.

```
output/
  clips/
    S01_E01 The Eyes/          job folder, named after the video file
      clip_NNNN.flac           one lossless clip per block, kept for --regen
      tts_input.srt            processed text actually sent to TTS
      tts_assembled.flac       all clips at their final positions, lossless
      embedded_sub.srt         extracted subtitle (--embedded-subs only)
      video_fragment.mkv       extracted segment (--end only)
    F1 The Movie 2025/
      ...
  S01_E01 The Eyes_lektor.mkv  the finished video
  F1 The Movie 2025_lektor.mkv
```

Each run prints the folder it is using:

```
  Job folder: output/clips/S01_E01 The Eyes
```

This is what makes `--rebuild` and `--regen` safe across films. `tts_input.srt`
and the clips are a matched pair — block *N* in the SRT is `clip_NNNN.flac` —
so both live in the job folder. They previously sat at the top level and every
new job overwrote the last one's, which meant rebuilding an older film paired
its clips with the newest film's text. Any past job can now be rebuilt at any
time.

The folder is derived from the video filename, so `--rebuild` and `--regen`
locate it automatically; just pass the same video path.

The finished file contains:

| Track | Content |
|---|---|
| Video | Stream-copied, untouched |
| Audio 0 | **Original** — stream-copied, full 5.1 if the source had it |
| Audio 1 | **Polish** — lektor mix, AAC 192k stereo, `language=pol` |
| Subtitles | Every track from the source, stream-copied |

The lektor track is stereo deliberately; see
[troubleshooting.md](troubleshooting.md#the-lektor-track-is-silent-in-the-player).

FLAC is used for every intermediate so repeated processing stays lossless. Only
the final voiceover track is ever encoded.

## Recipes

**Check a voice before committing to a full run**

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --end 02:00 --voice gudowski
```

**Tune the mix without regenerating speech**

```bash
python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.45 --tts-volume 2.1
```

**Fix a handful of bad clips**

```bash
# read output/clips/<video name>/tts_input.srt, note the offending block numbers
python3 lektor_ai_comfy.py input/film.mkv --regen 88 91 104
python3 lektor_ai_comfy.py input/film.mkv --rebuild
```

**Reproduce a run exactly**

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --seed 123456
```

**Speech running long and overlapping the next line**

```bash
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --speed 1.35
```

## Reading the console

```
[1/4] Parsing subtitles 00:00–end…
      693 entries after dialogue splitting.
[2/4] Generating TTS (693 clips, speed 1.2×)…
  [12/693] 00:01:34,120  'Nie wiem, o czym mówisz.'
    Waiting for a3f8b201… done.
    → placed at 00:01:34,120, duration 1.84s
```

`→ placed at` reveals timing drift. A clip that overruns its window pushes the
next one forward, shown as `[+1.20s behind SRT]`. Small values are normal and
self-correct at the next natural gap in the subtitles; values that keep growing
mean `--speed` is too low for the material.

Assembly then shows a progress bar per batch (clips are assembled 500 at a time
to stay under the open-file limit), followed by the two-pass mux.