1dbe8625319f243f361406225b38eca9d24ce511
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  claudeandClaude Opus 5
|
1dbe862531 |
Add the Knapik voice and a fourth Gudowski sample
Tomasz_Knapik.wav is a second lektor voice to try against Gudowski. It has no reference transcript yet, so it needs one written before it can be used the way the others are. gudowski_short_4.wav is a fourth reference sample with its transcript, and gudowski_generated.wav is TTS output kept beside the samples for comparison, not an input. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
e541ef7ee9 |
Give a wedged ComfyUI ten restarts per clip instead of three
Three was not enough in practice: a backend that dies during model load can burn all three before it ever comes up healthy, and the clip is then skipped — or, when the backend is simply gone, the run stalls waiting for it. Both caps move together. COMFY_AUTOSTART_MAX is what actually gates a relaunch, so raising CLIP_CRASH_LIMIT alone would still have stopped at three restarts. Counting is unchanged: both are per clip, not per run. The cost of a truly dead backend goes up accordingly — up to ten rounds of a 90s cooldown plus the 180s health timeout before a clip is given up on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
39452a4281 |
Mux the spoken subtitles into the output, and fix MP4 subtitle copying
--include-subs adds the subtitles the voiceover was made from to the finished video as one more track, alongside whatever the source already carried. Useful where the source has no Polish subtitles of its own, and for checking the voiceover against the text it came from. Every run now writes clips/<job>/voiceover_subs.srt: the cue text verbatim, timestamps shifted by --start and cues outside --start..--end dropped. Verbatim because this is what a viewer reads, not what the model was handed — dictionary rewrites and spelled-out numbers must not leak into it. It is written whether or not --include-subs was passed, because --rebuild --include-subs is given only the video and would otherwise have nothing to mux; runs made before this existed fall back to tts_input.srt with a note that the wording is the model's. Copying subtitle tracks from an MP4 source was broken, and silently: MP4 stores them as mov_text (tx3g), which the Matroska muxer rejects — "Subtitle codec 94213 is not supported" — and then no output file is written at all. Those tracks are now decoded to SubRip in a temporary file and muxed from there, while image formats (PGS, VOBSUB) are still stream-copied since they cannot become text. Two details that follow from the detour: ffmpeg's tx3g decoder emits the format's style records as <font face="Serif"><font size="18"> markup, which players either honour (subtitles rendered at whatever pixel size the MP4 specified, usually unreadably small) or print literally — so the tags are stripped; and a standalone .srt carries no language, so the source track's language and title tags are re-applied by hand or the output reads "undefined". An empty placeholder track, which rips often carry, is dropped rather than muxed as an empty one. UI: an "include the subtitles" checkbox next to the embedded-subtitles one. It is cleared and greyed out when the embedded track is chosen, since those subtitles already travel with the video, and it mirrors across linked jobs like the other per-job settings. RoundCheck grew a set_enabled() for that. The remove button moves into a column of its own spanning both rows, so it stays centred now that the second row is taller. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
d350b314ad |
Give the app an icon and a place in the application menu
icon.svg is the source; make_icons.py rasterises it into assets/icon-<size>.png at the six sizes a window manager asks for. Tk 8.6 reads PNG but not SVG, so the icon has to ship pre-rendered, and the glyph is laid on a rounded plate in the app's own colours — light on dark stays legible against both a light and a dark dock, which slate grey on nothing does not. ui.py loads whichever of those files exist and passes them all to iconphoto(), letting the WM pick per context (title bar, alt-tab, dock). The images are kept on the instance because Tk holds only a weak reference to a PhotoImage and the icon blanks out when it is collected. A missing or unreadable file is ignored rather than allowed to fail a launch. It also passes className="lektor-ai", which lands in WM_CLASS. Without it Tk calls itself "Tk" and that is what the dock shows on hover; the readable name comes from the desktop entry, which matches the window through StartupWMClass. install_desktop.sh writes ~/.local/share/applications/lektor-ai.desktop and copies the icons into the hicolor theme, so the app appears in the menu and can be pinned. Everything it installs lives under ~/.local and comes back out with --uninstall. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
4a9b05de66 |
Add two standalone tools: prose to speech, TMPlayer to SRT
tts_txt.py reads a plain text file aloud with the lektor voice. The pipeline is built around subtitles — every clip is placed at a timestamp — so prose needs the one piece that is genuinely missing, a chunker that splits text into sentence-sized pieces (240 chars, on sentence boundaries, long sentences split at clause breaks). Everything else is lektor_ai_comfy's: the dictionary, the Polish number normalizer, the ComfyUI call with its truncation retries and crash recovery, the per-clip trim and speed-up. Clips land in output/clips/<stem>/ like a film's, so --resume-from and re-assembly work the same way, and --dry-run prints the chunks without generating anything. tmp2srt.py converts TMPlayer subtitles (HH:MM:SS:text, '|' for a line break) to SRT. TMPlayer stores only a start time, so an end is inferred: the next cue's start less a small gap, or sooner when 16 chars/second says the line can be read in less. The separator after the timestamp is optional in the wild — some files use ':', some a space, some nothing — so all three parse. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
97866e6623 |
Install the whole backend from one script; queue log; UI fixes
install_comfyui.sh becomes install_backend.sh and now builds everything the app needs, not just the speech half: - Step 7 runs normalizer/setup.sh --model, which fetches both models: the open inflection model first, so the install gets as far as it can before anything can ask for credentials, then the gated fine-tune. - A missing Hugging Face account is no longer fatal. Accepting a licence is the one thing an installer cannot do, so the rest installs, the verification reports "fallback only (gated model missing)", and the summary prints the three commands that finish it. - --skip-normalizer, disk requirement 35 -> 55 GB, removal instructions and the step list updated to match. Queue logging — queue.log in the first queued video's job folder, which for a single-video run is its own: - Records what went wrong rather than what happened: warnings, tracebacks, ffmpeg errors, guard rejections, clips cut off mid-word or that crashed the backend, and the --regen line carrying their numbers. - ComfyUI failures *and* restarts, so the log shows the recovery rather than reading as though the run simply stopped. GUI-initiated backend actions go in too; they never pass through a job's pipe. - A failure also dumps the job's last 15 lines. Keyword matching alone is not enough — ffmpeg says "Invalid data found when processing input" and nothing that looks like an error. UI: - The output pane follows new output only while already at the bottom. Scroll up and it stays there; scroll back down and it resumes. Reading an earlier warning during a run was impossible while every ffmpeg progress frame yanked the view to the end. - The ComfyUI button's status dot is drawn as its own canvas item, so it can be green (answering) or amber (holds the port, does not answer). A "." inside the label could only ever be the same grey as the words. comfyui.log is capped at 120,000 lines, trimmed on backend start and every five minutes. Rewritten in place, never replaced: tee holds it open in append mode, so swapping the inode would leave the backend writing to an unlinked file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
e201aadc3e |
Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
claudeandClaude Opus 5
|
7e2cee02e5 |
Survive a ComfyUI crash, retry clips the model cuts short
Crash recovery ComfyUI's queue and history do not persist across a restart, so a prompt submitted before a crash exists nowhere afterwards. The old poll loop waited on that dead prompt for ever, and its error branch skipped the backoff, so it hammered the closed port four times a second printing "Poll error" on every attempt. run_workflow() replaces wait_for_completion(). It reports once when the backend goes away and once when it returns, polls every 3s in between, and on reconnect checks both /history and /queue: if the prompt is in neither, the queue did not survive and the clip is submitted again. When it cannot tell, it assumes the job is still alive — resubmitting a running job would waste a generation. It also starts ComfyUI itself, via the script named by [comfy] start_command, in a new terminal window. Guarded by a 90s cooldown and 3 attempts per run so a backend that dies during startup cannot spawn an endless stream of windows; the window pauses on exit so a failed start leaves its error on screen. Falls back to a background launch with a log file when there is no display. Set [comfy] autostart = no, or pass --no-comfy-autostart, to turn it off. Clips cut short mid-word About one clip in twelve comes back truncated: the model stops before the line is finished. Measured on raw ComfyUI output, 6 of 80 recent clips, and 15 of 472 in one episode. It is invisible in the text but obvious in the audio — a finished clip decays to about -65 dB, a cut one stops at about -16 dB. Truncated clips are now regenerated with a distant seed, up to [tts] truncation_retries extra attempts, and a run that still has bad clips ends by printing them as a ready-made --regen list. --check-clips reports the same for a job already on disk. Worth recording what this is NOT: the trailing-silence trimming added earlier was suspected, and cleared. It only cuts silence that runs to end of file, so a pause between words never triggers it — verified against six synthetic cases including a 1.2s mid-clip pause, and against real clips containing pauses. The truncation is present in ComfyUI's own output before this script touches it. settings.ini New: [tts] truncation_retries, [comfy] autostart and start_command, all documented in place. The editor renders yes/no keys as a dropdown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
200d221816 |
Add graphical queue (ui.py), fix runaway-clip padding
ui.py — graphical front-end Queue several videos and run them one after another. Each job shells out to lektor_ai_comfy.py with ordinary command-line arguments, so the terminal script stays the single source of truth and remains usable on its own. Videos chosen together in one multi-select form a linked set whose settings move together; blocks added with + stay independent. Subtitles are found beside the video automatically, picked manually, or chosen from the video's embedded tracks. REGENERATE / RESUME / RE-BUILD map to --regen, --resume-from and --rebuild; RESUME pre-fills the last clip on disk, since an interrupted run can leave that one half-written. A Settings dialog edits settings.ini in place, rewriting only the value lines so the file's comments survive — configparser would have dropped all 29 of them. Closing asks first, and names the queued jobs that would be discarded. tkinter, so no new dependencies. It needs a Tk with real fonts: the system one after `apt install python3-tk`. Under miniforge's Python it still runs but falls back to X11 bitmap fonts, which the UI detects and reports. Reading the child's pipe deliberately avoids text=True. Text mode enables universal newlines, which rewrites a lone carriage return as a newline — so ffmpeg's progress bars redrew as a new line per update instead of animating. Wheel events are bound per widget rather than with bind_all, so scrolling stops at a widget's limit instead of spilling into whatever is behind it, and can no longer change a dropdown's value in passing. start_lektor_ui.sh Picks a Python that has tkinter, preferring the system one for its fonts. Runaway clip padding When the model fails to emit a stop token it generates until --max-tokens is exhausted: a two-second line becomes a 136s clip. Placement uses a push-forward cursor, so that padding delayed every clip after it — measured on one film, 1807 of 2075 clips were pushed late, the worst by 981 seconds, which is why the voiceover fell silent over later dialogue. Clips are now trimmed before they are saved. Two padding modes occur and both are handled: digital silence, found with silencedetect; and a steady tone that sits above any sane silence threshold, found by its flatness instead — tone tails spread 3-5 dB across their windows where speech spreads 43-61 dB. --trim-silence repairs jobs generated before this, in place and without regenerating anything. Applied to the six existing jobs it removed 77 clips' worth of padding, about 1h55m, leaving every job within a few seconds of its subtitle timing. --embedded-sub-index Selects an embedded subtitle track by ffmpeg stream index instead of prompting, for the UI and for scripted runs. Interactive behaviour is unchanged when it is omitted. settings.ini — voice gudowski_short_3, temperature 0.6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
3dde6e58e4 |
Settings file, per-job clip folders, faster polling, voices/ rename
Poll interval wait_for_completion() polled every 3 s, which quantised every clip to a 3 s boundary — across 2075 existing clips, 91% of the gaps landed exactly on a multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so short clips are picked up immediately without hammering the server on long ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll, a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real generation time, e.g. "done (4.2s)". A WebSocket would remove polling entirely but needs a third-party client, which would cost this script its dependency-free property. settings.ini voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms, orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next to the script. Precedence is CLI flag > settings.ini > built-in default, wired by feeding loaded values in as argparse defaults. --settings FILE selects an alternative profile. INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or JSON (no comments). A missing, partial, malformed or corrupt file degrades to built-in defaults with a warning instead of killing a long run. Run-specific flags are deliberately excluded: --start, --end, --seed, --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts. Per-job folders All state for one video now lives in <output_dir>/clips/<video name>/ — clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level. tts_input.srt moved with the clips because they are a matched pair: block N in the SRT is clip_NNNN.flac. Previously both sat at the top level and each new job overwrote the last one's, so rebuilding an older film silently paired its clips with the newest film's text. Any past job can now be rebuilt. A run against the old flat layout gets an explanatory message. gudowski/ -> voices/ VOICE_PROMPTS_DIR is now resolved relative to the script rather than hardcoded, so moving the project no longer breaks it. Voice filenames are unchanged, so --voice gudowski_short still works. Docs updated throughout for the new layout, settings file and poll behaviour. Known issue, not addressed here: with a non-zero --start the voiceover plays early. extract_video_fragment() stream-copies, so it can only cut on a keyframe and seeks backwards to it, while TTS timestamps are relative to the requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is 458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with --start 00:00 are unaffected, since 0 is always a keyframe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
fc4b761f52 |
Initial commit: Polish AI dubbing pipeline (ComfyUI + Higgs Audio V3)
Converts SRT subtitles into a Polish lektor-style voiceover using a local
ComfyUI instance running Higgs Audio V3, then muxes the result back into the
source video as a second audio track. Video and original audio are always
stream-copied; only the new voiceover track is encoded.
Contents:
lektor_ai_comfy.py main pipeline (standard library only, needs ffmpeg)
lektor_ai_comfy_v2.py batched-context experiment (timing unresolved)
lektor_ai.py earlier version using the hosted Boson AI API
install_comfyui.sh reproduces the ComfyUI environment, versions pinned
docs/ requirements, installation, usage, architecture,
troubleshooting
gudowski/ voice references (.wav) with their transcripts (.txt)
Source media and generated video are gitignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|