7e2cee02e53a2cde8f6407bc0b7b867f8a37bfb5
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  claudeandClaude Opus 5
|
7e2cee02e5 |
Survive a ComfyUI crash, retry clips the model cuts short
Crash recovery ComfyUI's queue and history do not persist across a restart, so a prompt submitted before a crash exists nowhere afterwards. The old poll loop waited on that dead prompt for ever, and its error branch skipped the backoff, so it hammered the closed port four times a second printing "Poll error" on every attempt. run_workflow() replaces wait_for_completion(). It reports once when the backend goes away and once when it returns, polls every 3s in between, and on reconnect checks both /history and /queue: if the prompt is in neither, the queue did not survive and the clip is submitted again. When it cannot tell, it assumes the job is still alive — resubmitting a running job would waste a generation. It also starts ComfyUI itself, via the script named by [comfy] start_command, in a new terminal window. Guarded by a 90s cooldown and 3 attempts per run so a backend that dies during startup cannot spawn an endless stream of windows; the window pauses on exit so a failed start leaves its error on screen. Falls back to a background launch with a log file when there is no display. Set [comfy] autostart = no, or pass --no-comfy-autostart, to turn it off. Clips cut short mid-word About one clip in twelve comes back truncated: the model stops before the line is finished. Measured on raw ComfyUI output, 6 of 80 recent clips, and 15 of 472 in one episode. It is invisible in the text but obvious in the audio — a finished clip decays to about -65 dB, a cut one stops at about -16 dB. Truncated clips are now regenerated with a distant seed, up to [tts] truncation_retries extra attempts, and a run that still has bad clips ends by printing them as a ready-made --regen list. --check-clips reports the same for a job already on disk. Worth recording what this is NOT: the trailing-silence trimming added earlier was suspected, and cleared. It only cuts silence that runs to end of file, so a pause between words never triggers it — verified against six synthetic cases including a 1.2s mid-clip pause, and against real clips containing pauses. The truncation is present in ComfyUI's own output before this script touches it. settings.ini New: [tts] truncation_retries, [comfy] autostart and start_command, all documented in place. The editor renders yes/no keys as a dropdown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
200d221816 |
Add graphical queue (ui.py), fix runaway-clip padding
ui.py — graphical front-end Queue several videos and run them one after another. Each job shells out to lektor_ai_comfy.py with ordinary command-line arguments, so the terminal script stays the single source of truth and remains usable on its own. Videos chosen together in one multi-select form a linked set whose settings move together; blocks added with + stay independent. Subtitles are found beside the video automatically, picked manually, or chosen from the video's embedded tracks. REGENERATE / RESUME / RE-BUILD map to --regen, --resume-from and --rebuild; RESUME pre-fills the last clip on disk, since an interrupted run can leave that one half-written. A Settings dialog edits settings.ini in place, rewriting only the value lines so the file's comments survive — configparser would have dropped all 29 of them. Closing asks first, and names the queued jobs that would be discarded. tkinter, so no new dependencies. It needs a Tk with real fonts: the system one after `apt install python3-tk`. Under miniforge's Python it still runs but falls back to X11 bitmap fonts, which the UI detects and reports. Reading the child's pipe deliberately avoids text=True. Text mode enables universal newlines, which rewrites a lone carriage return as a newline — so ffmpeg's progress bars redrew as a new line per update instead of animating. Wheel events are bound per widget rather than with bind_all, so scrolling stops at a widget's limit instead of spilling into whatever is behind it, and can no longer change a dropdown's value in passing. start_lektor_ui.sh Picks a Python that has tkinter, preferring the system one for its fonts. Runaway clip padding When the model fails to emit a stop token it generates until --max-tokens is exhausted: a two-second line becomes a 136s clip. Placement uses a push-forward cursor, so that padding delayed every clip after it — measured on one film, 1807 of 2075 clips were pushed late, the worst by 981 seconds, which is why the voiceover fell silent over later dialogue. Clips are now trimmed before they are saved. Two padding modes occur and both are handled: digital silence, found with silencedetect; and a steady tone that sits above any sane silence threshold, found by its flatness instead — tone tails spread 3-5 dB across their windows where speech spreads 43-61 dB. --trim-silence repairs jobs generated before this, in place and without regenerating anything. Applied to the six existing jobs it removed 77 clips' worth of padding, about 1h55m, leaving every job within a few seconds of its subtitle timing. --embedded-sub-index Selects an embedded subtitle track by ffmpeg stream index instead of prompting, for the UI and for scripted runs. Interactive behaviour is unchanged when it is omitted. settings.ini — voice gudowski_short_3, temperature 0.6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
3dde6e58e4 |
Settings file, per-job clip folders, faster polling, voices/ rename
Poll interval wait_for_completion() polled every 3 s, which quantised every clip to a 3 s boundary — across 2075 existing clips, 91% of the gaps landed exactly on a multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so short clips are picked up immediately without hammering the server on long ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll, a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real generation time, e.g. "done (4.2s)". A WebSocket would remove polling entirely but needs a third-party client, which would cost this script its dependency-free property. settings.ini voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms, orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next to the script. Precedence is CLI flag > settings.ini > built-in default, wired by feeding loaded values in as argparse defaults. --settings FILE selects an alternative profile. INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or JSON (no comments). A missing, partial, malformed or corrupt file degrades to built-in defaults with a warning instead of killing a long run. Run-specific flags are deliberately excluded: --start, --end, --seed, --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts. Per-job folders All state for one video now lives in <output_dir>/clips/<video name>/ — clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level. tts_input.srt moved with the clips because they are a matched pair: block N in the SRT is clip_NNNN.flac. Previously both sat at the top level and each new job overwrote the last one's, so rebuilding an older film silently paired its clips with the newest film's text. Any past job can now be rebuilt. A run against the old flat layout gets an explanatory message. gudowski/ -> voices/ VOICE_PROMPTS_DIR is now resolved relative to the script rather than hardcoded, so moving the project no longer breaks it. Voice filenames are unchanged, so --voice gudowski_short still works. Docs updated throughout for the new layout, settings file and poll behaviour. Known issue, not addressed here: with a non-zero --start the voiceover plays early. extract_video_fragment() stream-copies, so it can only cut on a keyframe and seeks backwards to it, while TTS timestamps are relative to the requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is 458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with --start 00:00 are unaffected, since 0 is always a keyframe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
fc4b761f52 |
Initial commit: Polish AI dubbing pipeline (ComfyUI + Higgs Audio V3)
Converts SRT subtitles into a Polish lektor-style voiceover using a local
ComfyUI instance running Higgs Audio V3, then muxes the result back into the
source video as a second audio track. Video and original audio are always
stream-copied; only the new voiceover track is encoded.
Contents:
lektor_ai_comfy.py main pipeline (standard library only, needs ffmpeg)
lektor_ai_comfy_v2.py batched-context experiment (timing unresolved)
lektor_ai.py earlier version using the hosted Boson AI API
install_comfyui.sh reproduces the ComfyUI environment, versions pinned
docs/ requirements, installation, usage, architecture,
troubleshooting
gudowski/ voice references (.wav) with their transcripts (.txt)
Source media and generated video are gitignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|