lektor-ai/docs/troubleshooting.md

# Troubleshooting

Failure modes that have actually occurred here, with the diagnosis that
identified each one. Where a symptom is misleading, that is called out — several
of these point somewhere other than their cause.

## Audio problems in the finished file

### The lektor track is silent in the player

The single most misleading failure in this project. It has **two distinct
causes**, and volume measurements cannot tell them apart — both produce a track
that measures perfectly loud.

Diagnose by checking the declared channel layout and the interleaving, in that
order.

**Cause 1 — unknown channel layout**

```bash
ffprobe -v error -select_streams a -show_entries stream=index,codec_name,channels,channel_layout -of csv=p=0 out.mkv
```

```
1,eac3,6,5.1(side)     ← original, fine
2,aac,6,unknown        ← broken: 6 channels, no routing information
```

ffmpeg's AAC encoder does not signal a 5.1 layout, in any container. Players
cannot map the channels and output silence. The fix is the stereo lektor track
now used by default (`aformat=channel_layouts=stereo` on both inputs). A correct
track reports `2,stereo`.

**Cause 2 — broken interleaving**

Compare byte positions for the same timestamp:

```bash
ffprobe -read_intervals 00:20:00%+1 -select_streams v   -show_entries packet=pos -of csv=p=0 out.mkv | head -1
ffprobe -read_intervals 00:20:00%+1 -select_streams a:1 -show_entries packet=pos -of csv=p=0 out.mkv | head -1
```

They should be within a few MB. Observed on a broken file:

| time | drift from video |
|---|---|
| 02:00 | 477 MB |
| 10:00 | **2315 MB** |
| 50:00 | 740 MB |

Sparse subtitle tracks make the muxer give up interleaving after its default
10-second window. ffmpeg reads the file fine, so every level measurement looks
healthy — but players read MKV sequentially and never reach that audio.

The fix is `-max_interleave_delta 0` as an **output** option. An existing file
can be repaired without regenerating anything:

```bash
ffmpeg -i broken.mkv -map 0 -c copy -max_interleave_delta 0 fixed.mkv
```

14 seconds and ~2.6 GB of transient RAM for a 60-minute file.

> Placement is the trap. Before `-i` the flag configures the demuxer, changes
> nothing, and produces byte-identical output — which is easily mistaken for
> "this fix doesn't work".

### The voiceover is late by a constant offset

The source audio starts at a non-zero PTS. Confirm:

```bash
ffprobe -v error -select_streams a:0 -show_entries stream=start_time -of csv=p=0 film.mkv
# 4.043000  ← not 0
```

`amix` cannot emit output until both inputs have frames, so the voiceover is
delayed by that amount. Handled automatically now
(`asetpts=PTS-STARTPTS,adelay=<offset>`); the run prints:

```
Audio PTS offset: 4.043s — normalising orig_mix for lektor track
```

Note that this affects **only** the mixed track. Track 0 is stream-copied and
was never wrong, so "the original audio is offset" and "the original audio
inside the lektor mix is offset" are different problems with different fixes.

### Voiceover drifts progressively out of sync

Not a mixing fault — clips are overrunning their windows and pushing each other
forward. The console shows it:

```
→ placed at 00:12:03,400, duration 3.10s [+2.40s behind SRT]
```

Raise `--speed` (try 1.3–1.4) and rebuild. Small drift is expected and
self-corrects at the next gap in the subtitles; only sustained growth is a
problem.

### The whole voiceover sounds too quiet or too loud

Adjust and rebuild — no regeneration needed:

```bash
python3 lektor_ai_comfy.py film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0
```

Track 0 is unaffected by these, always.

## Muxing failures

### `Failed to inject frame into filter network: Invalid argument`

```
Error while filtering: Invalid argument
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:1
```

Aborts partway through, at a **different timestamp each run**. The message
blames audio; audio is not the cause. It appears when filtering and
subtitle muxing happen in one ffmpeg command on a source with many subtitle
tracks — the sparse streams stall the muxer while the filter graph keeps
producing.

The varying failure point is the tell. A genuinely corrupt frame fails at the
same place every time.

Fixed by the two-pass structure in `merge_voiceover()`. If you are editing that
function, do not merge the passes back together.

### The mux fails on DTS or TrueHD audio

Only an issue if output is changed to MP4, which cannot carry those codecs. The
default MKV output handles them. Keep it.

## TTS problems

### A clip recites the reference transcript

The model occasionally speaks the contents of `gudowski.txt` instead of the
subtitle. Most likely at `--temperature 1.0`; `0.9` is the default for this
reason.

```bash
python3 lektor_ai_comfy.py film.mkv --regen 88
python3 lektor_ai_comfy.py film.mkv --rebuild
```

`--seed 0` randomises each run, so a regeneration gets a different sample.

### A clip is cut off mid-word

The token ceiling was reached. Raise it and regenerate just that clip:

```bash
python3 lektor_ai_comfy.py film.mkv --regen 214 --max-tokens 8192
```

### Every clip returns identical audio

`enable_audio_cache` is being set to `True` somewhere in
`build_text_workflow()`. It must stay `False` or ComfyUI serves cached results
and `--regen` does nothing.

### Stiff pauses at every comma

Baked into the model weights. There is no SSML support and no parameter for it.
Rephrasing the subtitle text is the only lever.

### Audible breath at the start of clips

```bash
python3 lektor_ai_comfy.py film.mkv --regen 5 --strip-leading-ms 120
```

Values of 80–150 ms usually suffice. Too high clips the first consonant.

### Voice cloning quality is poor

Check that the reference `.txt` exists and matches the `.wav` exactly. The run
prints:

```
Reference text loaded (930 chars)
```

or warns if it is missing. An inaccurate transcript degrades cloning noticeably.

## Subtitle problems

### Formatting tags are being spoken

`<i>`, `</i>` and `{\an8}` are stripped by `_strip_sub_tags()` before anything
else. If new tags appear, extend that function — and keep the stripping first,
because tags interfere with the ALL-CAPS detection that follows.

### ALL-CAPS text is being shouted

Handled by `_clean_text()`, but only after tags are removed. `<i>HELLO</i>`
looked mixed-case while the tag was present, because `i` is a lowercase letter.
If shouting persists, check for an unhandled tag type.

### `--embedded-subs` shows no usable tracks

The video only carries image-based subtitles (`hdmv_pgs_subtitle`,
`dvd_subtitle`). These are bitmaps and need OCR. Supply an external `.srt`
instead — the image tracks are still copied to the output.

### Wrong language selected

The picker lists language tags but they are only as good as the source's
metadata. `embedded_sub.srt` in the job folder holds exactly what was extracted; check it
before committing to a full run.

## Backend problems

### `Connection refused` on port 8189

ComfyUI is not running. Start it and wait for the model to load:

```bash
~/comfy-tts-app/start_comfyui.sh
curl -s http://127.0.0.1:8189/system_stats | head -c 200
```

### ComfyUI crashed and the GPU is stuck

```bash
sudo fuser -k /dev/nvidia*
sudo nvidia-smi --gpu-reset
```

Then restart and resume from the last completed block:

```bash
python3 lektor_ai_comfy.py film.mkv --resume-from 312
```

### Out of VRAM

Nothing else should be using the GPU during a run. Restart ComfyUI to clear
fragmentation. The 4B model needs roughly 10–12 GB.

### `torch.cuda.is_available()` is False

```bash
source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
```

Expect `2.11.0+cu128 True`. If False, check `nvidia-smi` works and the driver is
525 or newer. The CUDA **toolkit** version is irrelevant — the wheels bundle
their own runtime, and this machine runs cu128 wheels against a system CUDA
11.5 without issue.

## Script problems

### `Provide an SRT subtitle file, or use --embedded-subs`

A mode that needs subtitles was given only a video. Either pass the SRT first,
or add `--embedded-subs`.

### `No such file or directory` on the video

Almost always an unquoted path with spaces or parentheses, or a wrong extension
(`.mp4` where the file is `.mkv`). Quote it and check with `ls`.

### `Missing clip files — regenerate them first`

`--rebuild` needs a complete set. The listed clips must be regenerated with
`--regen`, or `--resume-from` used to fill the gap.

### `No tts_input.srt found`

`--rebuild` and `--regen` require a previous full run in the same
`--output-dir`.

## General diagnostic method

The lesson from the silent-track investigation, which took several wrong turns:

1. **Measure, don't assume.** Level, layout, and byte position each reveal a
   different class of fault.
2. **Isolate by subtraction.** Removing subtitles from the failing command is
   what proved they were the cause, not the audio the error named.
3. **A varying failure point means state, not data.** Corrupt input fails in the
   same place every time; buffering and interleaving problems move around.
4. **"ffmpeg reads it fine" is not "a player can play it."** Verify position, not
   just content.
5. **Check flag placement.** Several ffmpeg options mean different things before
   `-i` versus before the output path, and the wrong position often fails
   silently rather than erroring.