lektor-ai/docs/usage.md

Usage

ComfyUI must be running before any command that generates speech (~/comfy-tts-app/start_comfyui.sh). --rebuild is the exception — it only touches ffmpeg and works with the backend stopped.

Invocation

python3 lektor_ai_comfy.py [SRT] VIDEO [options]

Both positionals are optional in the argparse sense, because several modes do not need an SRT file. The rule:

# External subtitle file — pass both, SRT first
python3 lektor_ai_comfy.py input/film.pl.srt input/film.mkv

# Subtitles inside the video — pass the video alone
python3 lektor_ai_comfy.py input/film.mkv --embedded-subs

# --rebuild and --regen also need only the video
python3 lektor_ai_comfy.py input/film.mkv --rebuild

When a single positional is given in a mode that needs no SRT, the script moves it into the video slot automatically. Always quote paths containing spaces or parentheses:

python3 lektor_ai_comfy.py "input/Star City (2026)/S01_E01 The Eyes.mkv" --embedded-subs

Working modes

Full generation (default)

Parses subtitles, generates one TTS clip per block, assembles, muxes.

python3 lektor_ai_comfy.py input/film.mkv --embedded-subs

Roughly real-time-ish on a 4070 Ti SUPER: a 60-minute episode of ~700 subtitle blocks takes on the order of an hour. Each clip is saved as it completes, so a crash never loses more than the clip in flight.

--embedded-subs

Lists the video's subtitle tracks and prompts for one:

  Available subtitle tracks in S01_E01 The Eyes.mkv:
    [1] Stream #2: subrip [eng] — Forced
    [2] Stream #3: subrip [eng]
    [3] Stream #4: subrip [eng] — SDH
   ...
    [30] Stream #31: subrip [pol]
  Select track number: 30

The chosen track is extracted to output/embedded_sub.srt and then processed exactly like an external file. Image-based tracks are listed but marked unusable — they hold bitmaps, not text.

--rebuild

Reassembles audio from existing clips and rebuilds the video, with no TTS at all. This is the fast iteration loop — seconds to minutes rather than an hour.

python3 lektor_ai_comfy.py input/film.mkv --rebuild
python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0

Use it after --regen, or to adjust the mix. It requires output/tts_input.srt and a complete set of output/clips/clip_NNNN.flac, and reports which are missing if any are.

If the original run used --end, pass it again so the same video fragment is used.

--regen

Regenerates specific clips by block number, leaving everything else alone.

python3 lektor_ai_comfy.py input/film.mkv --regen 12 22 41
python3 lektor_ai_comfy.py input/film.mkv --rebuild        # then apply

Block numbers match output/tts_input.srt and the clip_NNNN.flac filenames. Only the selected FLAC files are replaced — --rebuild afterwards is what actually updates the video.

This is the intended fix for a bad clip. There is no automatic retry; when the model hallucinates, mumbles, or truncates, note the number and regenerate it. A different seed usually resolves it, and --seed 0 (the default) randomises every run.

--resume-from

Continues a crashed run, loading clips 1..N-1 from disk and generating from N onward.

python3 lektor_ai_comfy.py input/film.mkv --resume-from 47

The console prints [47/693] style progress, so the number to resume from is the last one that completed.

--skip-tts

Uses a pre-assembled audio file and goes straight to the mux.

python3 lektor_ai_comfy.py input/film.mkv --skip-tts output/tts_assembled.flac

Options

Range

Flag	Default	Meaning
--start MM:SS	00:00	Start of the range to process
--end MM:SS	(end)	End of the range

With --end, a matching fragment is extracted to output/video_fragment.mkv and all TTS timestamps become relative to --start. Pass the same values to any later --rebuild, or the audio will not line up.

--end 01:00 is the quickest way to sanity-check a new voice or speed setting.

Voice and model

Flag	Default	Meaning
--voice NAME|PATH	gudowski_short	Name in gudowski/, or a path to a WAV
--seed N	0	Base seed; 0 randomises each run
--temperature T	0.9	Sampling temperature
--max-tokens N	4096	Token ceiling per clip

--temperature is worth understanding. At 1.0 the model sometimes recites the reference transcript instead of the subtitle. 0.9 largely prevents that; lower is more deterministic but flatter. A per-clip seed of base_seed + index means a fixed --seed reproduces a run exactly.

Raise --max-tokens only if long clips are cut off mid-word.

Timing and cleanup

Flag	Default	Meaning
--speed FACTOR	1.2	Per-clip speed-up via atempo
--strip-leading-ms MS	0	Trim the start of each clip
--trailing-fade-ms MS	80	Fade the end of each clip

--speed compresses speech so a clip is more likely to fit its subtitle window. It is applied per clip, before assembly — never to the assembled track, which would shift every timestamp by speeding up the silences too.

--strip-leading-ms removes an inhale artifact at clip starts; try 80–150 if you hear one. --trailing-fade-ms suppresses tail pops; 0 disables it.

Mixing

Flag	Default	Meaning
--orig-volume 0-1	0.60	Original audio level inside the lektor mix
--tts-volume 0-2	1.8	Voiceover level inside the lektor mix

These affect Track 1 only. Track 0 is always the untouched original, so nothing is lost by mixing aggressively. Changing them needs only --rebuild.

Typical starting point for dialogue-heavy material: --orig-volume 0.5 --tts-volume 2.0. For quiet sources, --orig-volume 1.0 --tts-volume 1.4.

Paths

Flag	Default	Meaning
--output-dir DIR	output	Where everything is written
--comfy-url URL	http://127.0.0.1:8189	Backend address

Output

output/
  tts_input.srt         processed text actually sent to TTS — inspect this
  embedded_sub.srt      extracted embedded subtitle (--embedded-subs only)
  clips/clip_NNNN.flac  one lossless clip per block, kept for --regen
  tts_assembled.flac    all clips at their final positions, lossless
  video_fragment.mkv    extracted segment (--end only)
  {stem}_lektor.mkv     the finished video

The finished file contains:

Track	Content
Video	Stream-copied, untouched
Audio 0	Original — stream-copied, full 5.1 if the source had it
Audio 1	Polish — lektor mix, AAC 192k stereo, language=pol
Subtitles	Every track from the source, stream-copied

The lektor track is stereo deliberately; see troubleshooting.md.

FLAC is used for every intermediate so repeated processing stays lossless. Only the final voiceover track is ever encoded.

Recipes

Check a voice before committing to a full run

python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --end 02:00 --voice gudowski

Tune the mix without regenerating speech

python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.45 --tts-volume 2.1

Fix a handful of bad clips

# read output/tts_input.srt, note the offending block numbers
python3 lektor_ai_comfy.py input/film.mkv --regen 88 91 104
python3 lektor_ai_comfy.py input/film.mkv --rebuild

Reproduce a run exactly

python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --seed 123456

Speech running long and overlapping the next line

python3 lektor_ai_comfy.py input/film.mkv --embedded-subs --speed 1.35

Reading the console

[1/4] Parsing subtitles 00:00–end…
      693 entries after dialogue splitting.
[2/4] Generating TTS (693 clips, speed 1.2×)…
  [12/693] 00:01:34,120  'Nie wiem, o czym mówisz.'
    Waiting for a3f8b201… done.
    → placed at 00:01:34,120, duration 1.84s

→ placed at reveals timing drift. A clip that overruns its window pushes the next one forward, shown as [+1.20s behind SRT]. Small values are normal and self-correct at the next natural gap in the subtitles; values that keep growing mean --speed is too low for the material.

Assembly then shows a progress bar per batch (clips are assembled 500 at a time to stay under the open-file limit), followed by the two-pass mux.