# Troubleshooting Failure modes that have actually occurred here, with the diagnosis that identified each one. Where a symptom is misleading, that is called out — several of these point somewhere other than their cause. ## Audio problems in the finished file ### The lektor track is silent in the player The single most misleading failure in this project. It has **two distinct causes**, and volume measurements cannot tell them apart — both produce a track that measures perfectly loud. Diagnose by checking the declared channel layout and the interleaving, in that order. **Cause 1 — unknown channel layout** ```bash ffprobe -v error -select_streams a -show_entries stream=index,codec_name,channels,channel_layout -of csv=p=0 out.mkv ``` ``` 1,eac3,6,5.1(side) ← original, fine 2,aac,6,unknown ← broken: 6 channels, no routing information ``` ffmpeg's AAC encoder does not signal a 5.1 layout, in any container. Players cannot map the channels and output silence. The fix is the stereo lektor track now used by default (`aformat=channel_layouts=stereo` on both inputs). A correct track reports `2,stereo`. **Cause 2 — broken interleaving** Compare byte positions for the same timestamp: ```bash ffprobe -read_intervals 00:20:00%+1 -select_streams v -show_entries packet=pos -of csv=p=0 out.mkv | head -1 ffprobe -read_intervals 00:20:00%+1 -select_streams a:1 -show_entries packet=pos -of csv=p=0 out.mkv | head -1 ``` They should be within a few MB. Observed on a broken file: | time | drift from video | |---|---| | 02:00 | 477 MB | | 10:00 | **2315 MB** | | 50:00 | 740 MB | Sparse subtitle tracks make the muxer give up interleaving after its default 10-second window. ffmpeg reads the file fine, so every level measurement looks healthy — but players read MKV sequentially and never reach that audio. The fix is `-max_interleave_delta 0` as an **output** option. An existing file can be repaired without regenerating anything: ```bash ffmpeg -i broken.mkv -map 0 -c copy -max_interleave_delta 0 fixed.mkv ``` 14 seconds and ~2.6 GB of transient RAM for a 60-minute file. > Placement is the trap. Before `-i` the flag configures the demuxer, changes > nothing, and produces byte-identical output — which is easily mistaken for > "this fix doesn't work". ### The voiceover is late by a constant offset The source audio starts at a non-zero PTS. Confirm: ```bash ffprobe -v error -select_streams a:0 -show_entries stream=start_time -of csv=p=0 film.mkv # 4.043000 ← not 0 ``` `amix` cannot emit output until both inputs have frames, so the voiceover is delayed by that amount. Handled automatically now (`asetpts=PTS-STARTPTS,adelay=`); the run prints: ``` Audio PTS offset: 4.043s — normalising orig_mix for lektor track ``` Note that this affects **only** the mixed track. Track 0 is stream-copied and was never wrong, so "the original audio is offset" and "the original audio inside the lektor mix is offset" are different problems with different fixes. ### Voiceover drifts progressively out of sync Not a mixing fault — clips are overrunning their windows and pushing each other forward. The console shows it: ``` → placed at 00:12:03,400, duration 3.10s [+2.40s behind SRT] ``` Raise `--speed` (try 1.3–1.4) and rebuild. Small drift is expected and self-corrects at the next gap in the subtitles; only sustained growth is a problem. ### The whole voiceover sounds too quiet or too loud Adjust and rebuild — no regeneration needed: ```bash python3 lektor_ai_comfy.py film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0 ``` Track 0 is unaffected by these, always. ## Muxing failures ### `Failed to inject frame into filter network: Invalid argument` ``` Error while filtering: Invalid argument Failed to inject frame into filter network: Invalid argument Error while processing the decoded data for stream #0:1 ``` Aborts partway through, at a **different timestamp each run**. The message blames audio; audio is not the cause. It appears when filtering and subtitle muxing happen in one ffmpeg command on a source with many subtitle tracks — the sparse streams stall the muxer while the filter graph keeps producing. The varying failure point is the tell. A genuinely corrupt frame fails at the same place every time. Fixed by the two-pass structure in `merge_voiceover()`. If you are editing that function, do not merge the passes back together. ### The mux fails on DTS or TrueHD audio Only an issue if output is changed to MP4, which cannot carry those codecs. The default MKV output handles them. Keep it. ## TTS problems ### A clip recites the reference transcript The model occasionally speaks the contents of `gudowski.txt` instead of the subtitle. Most likely at `--temperature 1.0`; `0.9` is the default for this reason. ```bash python3 lektor_ai_comfy.py film.mkv --regen 88 python3 lektor_ai_comfy.py film.mkv --rebuild ``` `--seed 0` randomises each run, so a regeneration gets a different sample. ### A clip is cut off mid-word The token ceiling was reached. Raise it and regenerate just that clip: ```bash python3 lektor_ai_comfy.py film.mkv --regen 214 --max-tokens 8192 ``` ### Every clip returns identical audio `enable_audio_cache` is being set to `True` somewhere in `build_text_workflow()`. It must stay `False` or ComfyUI serves cached results and `--regen` does nothing. ### Stiff pauses at every comma Baked into the model weights. There is no SSML support and no parameter for it. Rephrasing the subtitle text is the only lever. ### Audible breath at the start of clips ```bash python3 lektor_ai_comfy.py film.mkv --regen 5 --strip-leading-ms 120 ``` Values of 80–150 ms usually suffice. Too high clips the first consonant. ### Voice cloning quality is poor Check that the reference `.txt` exists and matches the `.wav` exactly. The run prints: ``` Reference text loaded (930 chars) ``` or warns if it is missing. An inaccurate transcript degrades cloning noticeably. ## Subtitle problems ### Formatting tags are being spoken ``, `` and `{\an8}` are stripped by `_strip_sub_tags()` before anything else. If new tags appear, extend that function — and keep the stripping first, because tags interfere with the ALL-CAPS detection that follows. ### ALL-CAPS text is being shouted Handled by `_clean_text()`, but only after tags are removed. `HELLO` looked mixed-case while the tag was present, because `i` is a lowercase letter. If shouting persists, check for an unhandled tag type. ### `--embedded-subs` shows no usable tracks The video only carries image-based subtitles (`hdmv_pgs_subtitle`, `dvd_subtitle`). These are bitmaps and need OCR. Supply an external `.srt` instead — the image tracks are still copied to the output. ### Wrong language selected The picker lists language tags but they are only as good as the source's metadata. `output/embedded_sub.srt` holds exactly what was extracted; check it before committing to a full run. ## Backend problems ### `Connection refused` on port 8189 ComfyUI is not running. Start it and wait for the model to load: ```bash ~/comfy-tts-app/start_comfyui.sh curl -s http://127.0.0.1:8189/system_stats | head -c 200 ``` ### ComfyUI crashed and the GPU is stuck ```bash sudo fuser -k /dev/nvidia* sudo nvidia-smi --gpu-reset ``` Then restart and resume from the last completed block: ```bash python3 lektor_ai_comfy.py film.mkv --resume-from 312 ``` ### Out of VRAM Nothing else should be using the GPU during a run. Restart ComfyUI to clear fragmentation. The 4B model needs roughly 10–12 GB. ### `torch.cuda.is_available()` is False ```bash source ~/miniforge3/etc/profile.d/conda.sh && conda activate comfy-tts python -c "import torch; print(torch.__version__, torch.cuda.is_available())" ``` Expect `2.11.0+cu128 True`. If False, check `nvidia-smi` works and the driver is 525 or newer. The CUDA **toolkit** version is irrelevant — the wheels bundle their own runtime, and this machine runs cu128 wheels against a system CUDA 11.5 without issue. ## Script problems ### `Provide an SRT subtitle file, or use --embedded-subs` A mode that needs subtitles was given only a video. Either pass the SRT first, or add `--embedded-subs`. ### `No such file or directory` on the video Almost always an unquoted path with spaces or parentheses, or a wrong extension (`.mp4` where the file is `.mkv`). Quote it and check with `ls`. ### `Missing clip files — regenerate them first` `--rebuild` needs a complete set. The listed clips must be regenerated with `--regen`, or `--resume-from` used to fill the gap. ### `No tts_input.srt found` `--rebuild` and `--regen` require a previous full run in the same `--output-dir`. ## General diagnostic method The lesson from the silent-track investigation, which took several wrong turns: 1. **Measure, don't assume.** Level, layout, and byte position each reveal a different class of fault. 2. **Isolate by subtraction.** Removing subtitles from the failing command is what proved they were the cause, not the audio the error named. 3. **A varying failure point means state, not data.** Corrupt input fails in the same place every time; buffering and interleaving problems move around. 4. **"ffmpeg reads it fine" is not "a player can play it."** Verify position, not just content. 5. **Check flag placement.** Several ffmpeg options mean different things before `-i` versus before the output path, and the wrong position often fails silently rather than erroring.