lektor-ai/docs/troubleshooting.md

Troubleshooting

Failure modes that have actually occurred here, with the diagnosis that identified each one. Where a symptom is misleading, that is called out — several of these point somewhere other than their cause.

Audio problems in the finished file

The lektor track is silent in the player

The single most misleading failure in this project. It has two distinct causes, and volume measurements cannot tell them apart — both produce a track that measures perfectly loud.

Diagnose by checking the declared channel layout and the interleaving, in that order.

Cause 1 — unknown channel layout

ffprobe -v error -select_streams a -show_entries stream=index,codec_name,channels,channel_layout -of csv=p=0 out.mkv

1,eac3,6,5.1(side)     ← original, fine
2,aac,6,unknown        ← broken: 6 channels, no routing information

ffmpeg's AAC encoder does not signal a 5.1 layout, in any container. Players cannot map the channels and output silence. The fix is the stereo lektor track now used by default (aformat=channel_layouts=stereo on both inputs). A correct track reports 2,stereo.

Cause 2 — broken interleaving

Compare byte positions for the same timestamp:

ffprobe -read_intervals 00:20:00%+1 -select_streams v   -show_entries packet=pos -of csv=p=0 out.mkv | head -1
ffprobe -read_intervals 00:20:00%+1 -select_streams a:1 -show_entries packet=pos -of csv=p=0 out.mkv | head -1

They should be within a few MB. Observed on a broken file:

time	drift from video
02:00	477 MB
10:00	2315 MB
50:00	740 MB

Sparse subtitle tracks make the muxer give up interleaving after its default 10-second window. ffmpeg reads the file fine, so every level measurement looks healthy — but players read MKV sequentially and never reach that audio.

The fix is -max_interleave_delta 0 as an output option. An existing file can be repaired without regenerating anything:

ffmpeg -i broken.mkv -map 0 -c copy -max_interleave_delta 0 fixed.mkv

14 seconds and ~2.6 GB of transient RAM for a 60-minute file.

Placement is the trap. Before -i the flag configures the demuxer, changes nothing, and produces byte-identical output — which is easily mistaken for "this fix doesn't work".

The voiceover is late by a constant offset

The source audio starts at a non-zero PTS. Confirm:

ffprobe -v error -select_streams a:0 -show_entries stream=start_time -of csv=p=0 film.mkv
# 4.043000  ← not 0

amix cannot emit output until both inputs have frames, so the voiceover is delayed by that amount. Handled automatically now (asetpts=PTS-STARTPTS,adelay=<offset>); the run prints:

Audio PTS offset: 4.043s — normalising orig_mix for lektor track

Note that this affects only the mixed track. Track 0 is stream-copied and was never wrong, so "the original audio is offset" and "the original audio inside the lektor mix is offset" are different problems with different fixes.

Voiceover drifts progressively out of sync

Not a mixing fault — clips are overrunning their windows and pushing each other forward. The console shows it:

→ placed at 00:12:03,400, duration 3.10s [+2.40s behind SRT]

Raise --speed (try 1.3–1.4) and rebuild. Small drift is expected and self-corrects at the next gap in the subtitles; only sustained growth is a problem.

The whole voiceover sounds too quiet or too loud

Adjust and rebuild — no regeneration needed:

python3 lektor_ai_comfy.py film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0

Track 0 is unaffected by these, always.

Muxing failures

Failed to inject frame into filter network: Invalid argument

Error while filtering: Invalid argument
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:1

Aborts partway through, at a different timestamp each run. The message blames audio; audio is not the cause. It appears when filtering and subtitle muxing happen in one ffmpeg command on a source with many subtitle tracks — the sparse streams stall the muxer while the filter graph keeps producing.

The varying failure point is the tell. A genuinely corrupt frame fails at the same place every time.

Fixed by the two-pass structure in merge_voiceover(). If you are editing that function, do not merge the passes back together.

The mux fails on DTS or TrueHD audio

Only an issue if output is changed to MP4, which cannot carry those codecs. The default MKV output handles them. Keep it.

TTS problems

A clip recites the reference transcript

The model occasionally speaks the contents of gudowski.txt instead of the subtitle. Most likely at --temperature 1.0; 0.9 is the default for this reason.

python3 lektor_ai_comfy.py film.mkv --regen 88
python3 lektor_ai_comfy.py film.mkv --rebuild

--seed 0 randomises each run, so a regeneration gets a different sample.

A clip is cut off mid-word

The token ceiling was reached. Raise it and regenerate just that clip:

python3 lektor_ai_comfy.py film.mkv --regen 214 --max-tokens 8192

Every clip returns identical audio

enable_audio_cache is being set to True somewhere in build_text_workflow(). It must stay False or ComfyUI serves cached results and --regen does nothing.

Stiff pauses at every comma

Baked into the model weights. There is no SSML support and no parameter for it. Rephrasing the subtitle text is the only lever.

Audible breath at the start of clips

python3 lektor_ai_comfy.py film.mkv --regen 5 --strip-leading-ms 120

Values of 80–150 ms usually suffice. Too high clips the first consonant.

Voice cloning quality is poor

Check that the reference .txt exists and matches the .wav exactly. The run prints:

Reference text loaded (930 chars)

or warns if it is missing. An inaccurate transcript degrades cloning noticeably.

Subtitle problems

Formatting tags are being spoken

<i>, </i> and {\an8} are stripped by _strip_sub_tags() before anything else. If new tags appear, extend that function — and keep the stripping first, because tags interfere with the ALL-CAPS detection that follows.

ALL-CAPS text is being shouted

Handled by _clean_text(), but only after tags are removed. <i>HELLO</i> looked mixed-case while the tag was present, because i is a lowercase letter. If shouting persists, check for an unhandled tag type.

--embedded-subs shows no usable tracks

The video only carries image-based subtitles (hdmv_pgs_subtitle, dvd_subtitle). These are bitmaps and need OCR. Supply an external .srt instead — the image tracks are still copied to the output.

Wrong language selected

The picker lists language tags but they are only as good as the source's metadata. output/embedded_sub.srt holds exactly what was extracted; check it before committing to a full run.

Backend problems

Connection refused on port 8189

ComfyUI is not running. Start it and wait for the model to load:

~/comfy-tts-app/start_comfyui.sh
curl -s http://127.0.0.1:8189/system_stats | head -c 200

ComfyUI crashed and the GPU is stuck

sudo fuser -k /dev/nvidia*
sudo nvidia-smi --gpu-reset

Then restart and resume from the last completed block:

python3 lektor_ai_comfy.py film.mkv --resume-from 312

Out of VRAM

Nothing else should be using the GPU during a run. Restart ComfyUI to clear fragmentation. The 4B model needs roughly 10–12 GB.

torch.cuda.is_available() is False

source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

Expect 2.11.0+cu128 True. If False, check nvidia-smi works and the driver is 525 or newer. The CUDA toolkit version is irrelevant — the wheels bundle their own runtime, and this machine runs cu128 wheels against a system CUDA 11.5 without issue.

Script problems

Provide an SRT subtitle file, or use --embedded-subs

A mode that needs subtitles was given only a video. Either pass the SRT first, or add --embedded-subs.

No such file or directory on the video

Almost always an unquoted path with spaces or parentheses, or a wrong extension (.mp4 where the file is .mkv). Quote it and check with ls.

Missing clip files — regenerate them first

--rebuild needs a complete set. The listed clips must be regenerated with --regen, or --resume-from used to fill the gap.

No tts_input.srt found

--rebuild and --regen require a previous full run in the same --output-dir.

General diagnostic method

The lesson from the silent-track investigation, which took several wrong turns:

1. Measure, don't assume. Level, layout, and byte position each reveal a different class of fault.
2. Isolate by subtraction. Removing subtitles from the failing command is what proved they were the cause, not the audio the error named.
3. A varying failure point means state, not data. Corrupt input fails in the same place every time; buffering and interleaving problems move around.
4. "ffmpeg reads it fine" is not "a player can play it." Verify position, not just content.
5. Check flag placement. Several ffmpeg options mean different things before -i versus before the output path, and the wrong position often fails silently rather than erroring.