39452a4281bb1c13c02b124d8f314a2836324b3a
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  claudeandClaude Opus 5
|
39452a4281 |
Mux the spoken subtitles into the output, and fix MP4 subtitle copying
--include-subs adds the subtitles the voiceover was made from to the finished video as one more track, alongside whatever the source already carried. Useful where the source has no Polish subtitles of its own, and for checking the voiceover against the text it came from. Every run now writes clips/<job>/voiceover_subs.srt: the cue text verbatim, timestamps shifted by --start and cues outside --start..--end dropped. Verbatim because this is what a viewer reads, not what the model was handed — dictionary rewrites and spelled-out numbers must not leak into it. It is written whether or not --include-subs was passed, because --rebuild --include-subs is given only the video and would otherwise have nothing to mux; runs made before this existed fall back to tts_input.srt with a note that the wording is the model's. Copying subtitle tracks from an MP4 source was broken, and silently: MP4 stores them as mov_text (tx3g), which the Matroska muxer rejects — "Subtitle codec 94213 is not supported" — and then no output file is written at all. Those tracks are now decoded to SubRip in a temporary file and muxed from there, while image formats (PGS, VOBSUB) are still stream-copied since they cannot become text. Two details that follow from the detour: ffmpeg's tx3g decoder emits the format's style records as <font face="Serif"><font size="18"> markup, which players either honour (subtitles rendered at whatever pixel size the MP4 specified, usually unreadably small) or print literally — so the tags are stripped; and a standalone .srt carries no language, so the source track's language and title tags are re-applied by hand or the output reads "undefined". An empty placeholder track, which rips often carry, is dropped rather than muxed as an empty one. UI: an "include the subtitles" checkbox next to the embedded-subtitles one. It is cleared and greyed out when the embedded track is chosen, since those subtitles already travel with the video, and it mirrors across linked jobs like the other per-job settings. RoundCheck grew a set_enabled() for that. The remove button moves into a column of its own spanning both rows, so it stays centred now that the second row is taller. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
d350b314ad |
Give the app an icon and a place in the application menu
icon.svg is the source; make_icons.py rasterises it into assets/icon-<size>.png at the six sizes a window manager asks for. Tk 8.6 reads PNG but not SVG, so the icon has to ship pre-rendered, and the glyph is laid on a rounded plate in the app's own colours — light on dark stays legible against both a light and a dark dock, which slate grey on nothing does not. ui.py loads whichever of those files exist and passes them all to iconphoto(), letting the WM pick per context (title bar, alt-tab, dock). The images are kept on the instance because Tk holds only a weak reference to a PhotoImage and the icon blanks out when it is collected. A missing or unreadable file is ignored rather than allowed to fail a launch. It also passes className="lektor-ai", which lands in WM_CLASS. Without it Tk calls itself "Tk" and that is what the dock shows on hover; the readable name comes from the desktop entry, which matches the window through StartupWMClass. install_desktop.sh writes ~/.local/share/applications/lektor-ai.desktop and copies the icons into the hicolor theme, so the app appears in the menu and can be pinned. Everything it installs lives under ~/.local and comes back out with --uninstall. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NkWDx9dVB7xBpqntvjvm6A |
||
|
claudeandClaude Opus 5
|
97866e6623 |
Install the whole backend from one script; queue log; UI fixes
install_comfyui.sh becomes install_backend.sh and now builds everything the app needs, not just the speech half: - Step 7 runs normalizer/setup.sh --model, which fetches both models: the open inflection model first, so the install gets as far as it can before anything can ask for credentials, then the gated fine-tune. - A missing Hugging Face account is no longer fatal. Accepting a licence is the one thing an installer cannot do, so the rest installs, the verification reports "fallback only (gated model missing)", and the summary prints the three commands that finish it. - --skip-normalizer, disk requirement 35 -> 55 GB, removal instructions and the step list updated to match. Queue logging — queue.log in the first queued video's job folder, which for a single-video run is its own: - Records what went wrong rather than what happened: warnings, tracebacks, ffmpeg errors, guard rejections, clips cut off mid-word or that crashed the backend, and the --regen line carrying their numbers. - ComfyUI failures *and* restarts, so the log shows the recovery rather than reading as though the run simply stopped. GUI-initiated backend actions go in too; they never pass through a job's pipe. - A failure also dumps the job's last 15 lines. Keyword matching alone is not enough — ffmpeg says "Invalid data found when processing input" and nothing that looks like an error. UI: - The output pane follows new output only while already at the bottom. Scroll up and it stays there; scroll back down and it resumes. Reading an earlier warning during a run was impossible while every ffmpeg progress frame yanked the view to the end. - The ComfyUI button's status dot is drawn as its own canvas item, so it can be green (answering) or amber (holds the port, does not answer). A "." inside the label could only ever be the same grey as the words. comfyui.log is capped at 120,000 lines, trimmed on backend start and every five minutes. Rewritten in place, never replaced: tee holds it open in append mode, so swapping the inode would leave the backend writing to an unlinked file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
e201aadc3e |
Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
claudeandClaude Opus 5
|
7e2cee02e5 |
Survive a ComfyUI crash, retry clips the model cuts short
Crash recovery ComfyUI's queue and history do not persist across a restart, so a prompt submitted before a crash exists nowhere afterwards. The old poll loop waited on that dead prompt for ever, and its error branch skipped the backoff, so it hammered the closed port four times a second printing "Poll error" on every attempt. run_workflow() replaces wait_for_completion(). It reports once when the backend goes away and once when it returns, polls every 3s in between, and on reconnect checks both /history and /queue: if the prompt is in neither, the queue did not survive and the clip is submitted again. When it cannot tell, it assumes the job is still alive — resubmitting a running job would waste a generation. It also starts ComfyUI itself, via the script named by [comfy] start_command, in a new terminal window. Guarded by a 90s cooldown and 3 attempts per run so a backend that dies during startup cannot spawn an endless stream of windows; the window pauses on exit so a failed start leaves its error on screen. Falls back to a background launch with a log file when there is no display. Set [comfy] autostart = no, or pass --no-comfy-autostart, to turn it off. Clips cut short mid-word About one clip in twelve comes back truncated: the model stops before the line is finished. Measured on raw ComfyUI output, 6 of 80 recent clips, and 15 of 472 in one episode. It is invisible in the text but obvious in the audio — a finished clip decays to about -65 dB, a cut one stops at about -16 dB. Truncated clips are now regenerated with a distant seed, up to [tts] truncation_retries extra attempts, and a run that still has bad clips ends by printing them as a ready-made --regen list. --check-clips reports the same for a job already on disk. Worth recording what this is NOT: the trailing-silence trimming added earlier was suspected, and cleared. It only cuts silence that runs to end of file, so a pause between words never triggers it — verified against six synthetic cases including a 1.2s mid-clip pause, and against real clips containing pauses. The truncation is present in ComfyUI's own output before this script touches it. settings.ini New: [tts] truncation_retries, [comfy] autostart and start_command, all documented in place. The editor renders yes/no keys as a dropdown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
claudeandClaude Opus 5
|
200d221816 |
Add graphical queue (ui.py), fix runaway-clip padding
ui.py — graphical front-end Queue several videos and run them one after another. Each job shells out to lektor_ai_comfy.py with ordinary command-line arguments, so the terminal script stays the single source of truth and remains usable on its own. Videos chosen together in one multi-select form a linked set whose settings move together; blocks added with + stay independent. Subtitles are found beside the video automatically, picked manually, or chosen from the video's embedded tracks. REGENERATE / RESUME / RE-BUILD map to --regen, --resume-from and --rebuild; RESUME pre-fills the last clip on disk, since an interrupted run can leave that one half-written. A Settings dialog edits settings.ini in place, rewriting only the value lines so the file's comments survive — configparser would have dropped all 29 of them. Closing asks first, and names the queued jobs that would be discarded. tkinter, so no new dependencies. It needs a Tk with real fonts: the system one after `apt install python3-tk`. Under miniforge's Python it still runs but falls back to X11 bitmap fonts, which the UI detects and reports. Reading the child's pipe deliberately avoids text=True. Text mode enables universal newlines, which rewrites a lone carriage return as a newline — so ffmpeg's progress bars redrew as a new line per update instead of animating. Wheel events are bound per widget rather than with bind_all, so scrolling stops at a widget's limit instead of spilling into whatever is behind it, and can no longer change a dropdown's value in passing. start_lektor_ui.sh Picks a Python that has tkinter, preferring the system one for its fonts. Runaway clip padding When the model fails to emit a stop token it generates until --max-tokens is exhausted: a two-second line becomes a 136s clip. Placement uses a push-forward cursor, so that padding delayed every clip after it — measured on one film, 1807 of 2075 clips were pushed late, the worst by 981 seconds, which is why the voiceover fell silent over later dialogue. Clips are now trimmed before they are saved. Two padding modes occur and both are handled: digital silence, found with silencedetect; and a steady tone that sits above any sane silence threshold, found by its flatness instead — tone tails spread 3-5 dB across their windows where speech spreads 43-61 dB. --trim-silence repairs jobs generated before this, in place and without regenerating anything. Applied to the six existing jobs it removed 77 clips' worth of padding, about 1h55m, leaving every job within a few seconds of its subtitle timing. --embedded-sub-index Selects an embedded subtitle track by ffmpeg stream index instead of prompting, for the UI and for scripted runs. Interactive behaviour is unchanged when it is omitted. settings.ini — voice gudowski_short_3, temperature 0.6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |