Add graphical queue (ui.py), fix runaway-clip padding
ui.py — graphical front-end Queue several videos and run them one after another. Each job shells out to lektor_ai_comfy.py with ordinary command-line arguments, so the terminal script stays the single source of truth and remains usable on its own. Videos chosen together in one multi-select form a linked set whose settings move together; blocks added with + stay independent. Subtitles are found beside the video automatically, picked manually, or chosen from the video's embedded tracks. REGENERATE / RESUME / RE-BUILD map to --regen, --resume-from and --rebuild; RESUME pre-fills the last clip on disk, since an interrupted run can leave that one half-written. A Settings dialog edits settings.ini in place, rewriting only the value lines so the file's comments survive — configparser would have dropped all 29 of them. Closing asks first, and names the queued jobs that would be discarded. tkinter, so no new dependencies. It needs a Tk with real fonts: the system one after `apt install python3-tk`. Under miniforge's Python it still runs but falls back to X11 bitmap fonts, which the UI detects and reports. Reading the child's pipe deliberately avoids text=True. Text mode enables universal newlines, which rewrites a lone carriage return as a newline — so ffmpeg's progress bars redrew as a new line per update instead of animating. Wheel events are bound per widget rather than with bind_all, so scrolling stops at a widget's limit instead of spilling into whatever is behind it, and can no longer change a dropdown's value in passing. start_lektor_ui.sh Picks a Python that has tkinter, preferring the system one for its fonts. Runaway clip padding When the model fails to emit a stop token it generates until --max-tokens is exhausted: a two-second line becomes a 136s clip. Placement uses a push-forward cursor, so that padding delayed every clip after it — measured on one film, 1807 of 2075 clips were pushed late, the worst by 981 seconds, which is why the voiceover fell silent over later dialogue. Clips are now trimmed before they are saved. Two padding modes occur and both are handled: digital silence, found with silencedetect; and a steady tone that sits above any sane silence threshold, found by its flatness instead — tone tails spread 3-5 dB across their windows where speech spreads 43-61 dB. --trim-silence repairs jobs generated before this, in place and without regenerating anything. Applied to the six existing jobs it removed 77 clips' worth of padding, about 1h55m, leaving every job within a few seconds of its subtitle timing. --embedded-sub-index Selects an embedded subtitle track by ffmpeg stream index instead of prompting, for the UI and for scripted runs. Interactive behaviour is unchanged when it is omitted. settings.ini — voice gudowski_short_3, temperature 0.6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
+166
-1
@@ -373,6 +373,91 @@ def strip_leading_audio(input_path: Path, output_path: Path, strip_ms: float):
|
||||
], f"Strip {strip_ms:.0f}ms leading audio")
|
||||
|
||||
|
||||
# A clip's trailing silence must not count towards its duration: placement uses
|
||||
# a push-forward cursor, so padding at the end of one clip delays every clip
|
||||
# after it. Usually that padding is a fraction of a second, but when the model
|
||||
# fails to emit a stop token it generates until --max-tokens runs out — a short
|
||||
# line then yields a clip of speech followed by two minutes of digital silence,
|
||||
# which shunts the rest of the film out of sync.
|
||||
SILENCE_DB = -45.0 # anything quieter counts as silence
|
||||
SILENCE_MIN = 0.35 # ignore shorter gaps; commas and beats are not padding
|
||||
SILENCE_KEEP_MS = 120 # leave this much tail so speech does not end abruptly
|
||||
|
||||
|
||||
def trailing_silence_start(path: Path) -> float:
|
||||
"""Time at which the clip's final run of silence begins, or None."""
|
||||
result = subprocess.run(
|
||||
["ffmpeg", "-hide_banner", "-i", str(path),
|
||||
"-af", f"silencedetect=noise={SILENCE_DB}dB:d={SILENCE_MIN}",
|
||||
"-f", "null", "-"],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
try:
|
||||
duration = get_audio_duration(path)
|
||||
except RuntimeError:
|
||||
return None
|
||||
|
||||
open_at = None
|
||||
for kind, value in re.findall(r"silence_(start|end):\s*([0-9.]+)", result.stderr):
|
||||
t = float(value)
|
||||
if kind == "start":
|
||||
open_at = t
|
||||
elif t < duration - 0.05:
|
||||
open_at = None # that silence ended mid-clip, so it is a pause
|
||||
return open_at
|
||||
|
||||
|
||||
# Some runaway generations pad with a steady tone instead of digital silence.
|
||||
# It sits above any sane silence threshold, so silencedetect cannot see it —
|
||||
# but real speech swings tens of dB between windows while a tone does not.
|
||||
# Measured here: tone tails spread 3-5 dB, speech tails 43-61 dB.
|
||||
FLAT_TAIL_DB = 15.0 # spread below this is not speech
|
||||
FLAT_TAIL_MIN = 5.0 # the flat run must last at least this long
|
||||
RUNAWAY_SECONDS = 20.0 # only look for tone padding on clips this long
|
||||
|
||||
|
||||
def flat_tail_start(path: Path, window: float = 0.5) -> float:
|
||||
"""Start of a long, unnaturally constant tail, or None."""
|
||||
result = subprocess.run(
|
||||
["ffmpeg", "-hide_banner", "-i", str(path),
|
||||
"-af", f"astats=metadata=1:reset={window},"
|
||||
"ametadata=print:key=lavfi.astats.Overall.RMS_level",
|
||||
"-f", "null", "-"],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
blob = result.stderr + result.stdout
|
||||
values = [(-120.0 if "inf" in m else float(m))
|
||||
for m in re.findall(r"RMS_level=(-?[0-9.]+|-?inf)", blob)]
|
||||
need = int(FLAT_TAIL_MIN / window)
|
||||
if len(values) < need + 1:
|
||||
return None
|
||||
for i in range(len(values) - need):
|
||||
tail = values[i:]
|
||||
if max(tail) - min(tail) <= FLAT_TAIL_DB:
|
||||
return i * window
|
||||
return None
|
||||
|
||||
|
||||
def trim_trailing_silence(input_path: Path, output_path: Path) -> float:
|
||||
"""Cut the final silence off a clip. Returns the seconds removed."""
|
||||
duration = get_audio_duration(input_path)
|
||||
start = trailing_silence_start(input_path)
|
||||
if start is None and duration > RUNAWAY_SECONDS:
|
||||
start = flat_tail_start(input_path)
|
||||
keep = None if start is None else start + SILENCE_KEEP_MS / 1000
|
||||
if keep is None or keep >= duration - 0.02 or keep <= 0.05:
|
||||
if input_path != output_path:
|
||||
shutil.copy2(input_path, output_path)
|
||||
return 0.0
|
||||
_run([
|
||||
"ffmpeg", "-y",
|
||||
"-i", str(input_path),
|
||||
"-t", f"{keep:.3f}",
|
||||
str(output_path),
|
||||
], f"Trim {duration - keep:.1f}s trailing silence")
|
||||
return duration - keep
|
||||
|
||||
|
||||
def _assemble_batch(clips: list, output_path: str, label: str,
|
||||
total_seconds: float = 0.0, start_offset: float = 0.0):
|
||||
"""Mix a batch of (start_sec, path) clips with adelay into output_path.
|
||||
@@ -771,8 +856,22 @@ def _generate_one(text: str, voice_filename: str, voice_ref_text: str,
|
||||
strip_leading_audio(raw_clip, stripped, strip_leading_ms)
|
||||
raw_clip = stripped
|
||||
|
||||
raw_duration = get_audio_duration(raw_clip)
|
||||
trimmed = out_dir / f"{label}_t.flac"
|
||||
removed = trim_trailing_silence(raw_clip, trimmed)
|
||||
if removed > 0:
|
||||
raw_clip = trimmed
|
||||
# Losing most of the clip to silence means the model never stopped and
|
||||
# ran to the token ceiling. The speech is usually fine, but say so —
|
||||
# it can also mean the line was cut short, which needs a --regen.
|
||||
if removed > 20 and removed > raw_duration * 0.5:
|
||||
print(f" ! {removed:.0f}s of trailing silence removed "
|
||||
f"({raw_duration:.0f}s generated for {raw_duration - removed:.1f}s of "
|
||||
f"speech) — the model did not stop on its own; check this clip")
|
||||
|
||||
fast_clip = out_dir / f"{label}.flac"
|
||||
speed_up_audio(raw_clip, fast_clip, speed, fade_out_ms=trailing_fade_ms)
|
||||
trimmed.unlink(missing_ok=True)
|
||||
|
||||
return fast_clip, get_audio_duration(fast_clip)
|
||||
|
||||
@@ -924,6 +1023,45 @@ def _rebuild_video(args, out_dir: Path):
|
||||
print(f"\nDone! Output: {final_output}")
|
||||
|
||||
|
||||
def _trim_job_clips(args, out_dir: Path):
|
||||
"""Trim trailing silence from every clip already on disk for this video.
|
||||
|
||||
Repairs a job generated before trimming existed, or one where the model ran
|
||||
to the token ceiling. Only silence is removed, so the speech is untouched
|
||||
and nothing needs regenerating — follow with --rebuild.
|
||||
"""
|
||||
job = job_dir(out_dir, args.video)
|
||||
clips = sorted(job.glob("clip_*.flac"))
|
||||
if not clips:
|
||||
sys.exit(f"No clips found in {job}." + _legacy_hint(out_dir, job))
|
||||
|
||||
print(f"\n Scanning {len(clips)} clips in {job}…")
|
||||
total_removed, changed, runaways = 0.0, 0, []
|
||||
for i, clip in enumerate(clips, 1):
|
||||
before = get_audio_duration(clip)
|
||||
tmp = clip.with_suffix(".trim.flac")
|
||||
removed = trim_trailing_silence(clip, tmp)
|
||||
if removed > 0.02:
|
||||
tmp.replace(clip)
|
||||
total_removed += removed
|
||||
changed += 1
|
||||
if removed > 20:
|
||||
runaways.append((clip.name, before, removed))
|
||||
print(f" {clip.name} {before:7.2f}s -> {before - removed:6.2f}s "
|
||||
f"(-{removed:.2f}s)")
|
||||
else:
|
||||
tmp.unlink(missing_ok=True)
|
||||
|
||||
print(f"\n Trimmed {changed}/{len(clips)} clips, {total_removed:.0f}s of silence removed.")
|
||||
if runaways:
|
||||
print(f" {len(runaways)} clip(s) had run to the token ceiling:")
|
||||
for name, before, removed in runaways:
|
||||
print(f" {name}: {before:.0f}s generated for {before - removed:.1f}s of speech")
|
||||
print(" Their speech is intact, but listen to them — if any line is cut "
|
||||
"short, regenerate it with --regen.")
|
||||
print("\n Now run --rebuild to reassemble with the corrected timing.")
|
||||
|
||||
|
||||
def _regen_clips(args, out_dir: Path):
|
||||
"""Regenerate specific clips by tts_input block number.
|
||||
Only replaces FLAC files — run --rebuild afterwards to reassemble audio and video.
|
||||
@@ -1050,6 +1188,14 @@ def main():
|
||||
"(same number as in clips/clip_NNNN.flac and tts_input.srt). "
|
||||
"Only replaces the selected FLAC files; run --rebuild "
|
||||
"afterwards to reassemble audio and rebuild the video.")
|
||||
parser.add_argument("--trim-silence", action="store_true",
|
||||
help="Trim trailing silence from the clips already generated for "
|
||||
"this video, then stop. Repairs runs where the model failed "
|
||||
"to stop and padded a clip out to --max-tokens, which pushes "
|
||||
"every later clip out of sync. Follow with --rebuild.")
|
||||
parser.add_argument("--embedded-sub-index", type=int, default=None, metavar="N",
|
||||
help="With --embedded-subs, pick this ffmpeg stream index instead "
|
||||
"of prompting. Used by ui.py and for scripted runs.")
|
||||
parser.add_argument("--embedded-subs", action="store_true",
|
||||
help="Pick a subtitle track embedded in the video file instead of "
|
||||
"providing an external SRT. Lists available tracks and prompts "
|
||||
@@ -1068,7 +1214,8 @@ def main():
|
||||
# When the user writes `script.py video.mkv --embedded-subs` (or --rebuild /
|
||||
# --regen without an SRT), argparse puts the lone positional into args.srt.
|
||||
# Detect this and move it to args.video.
|
||||
needs_srt = not (args.embedded_subs or args.rebuild or bool(args.regen))
|
||||
needs_srt = not (args.embedded_subs or args.rebuild or args.trim_silence
|
||||
or bool(args.regen))
|
||||
if args.video is None and args.srt is not None and not needs_srt:
|
||||
args.video = args.srt
|
||||
args.srt = None
|
||||
@@ -1081,6 +1228,11 @@ def main():
|
||||
out_dir = Path(args.output_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# ── Maintenance: trim silence from existing clips, then stop ─────────────
|
||||
if args.trim_silence:
|
||||
_trim_job_clips(args, out_dir)
|
||||
return
|
||||
|
||||
# ── Rebuild mode: reassemble + remux from existing clips, no TTS ─────────
|
||||
if args.rebuild:
|
||||
_rebuild_video(args, out_dir)
|
||||
@@ -1107,6 +1259,19 @@ def main():
|
||||
sub_list = list_subtitle_streams(args.video)
|
||||
if not sub_list:
|
||||
sys.exit(f"No subtitle tracks found in {Path(args.video).name}.")
|
||||
if args.embedded_sub_index is not None:
|
||||
# Non-interactive selection by ffmpeg stream index, for the GUI and
|
||||
# for scripted batch runs. Interactive behaviour is unchanged when
|
||||
# this is omitted.
|
||||
chosen = next((s for s in sub_list
|
||||
if s["index"] == args.embedded_sub_index), None)
|
||||
if chosen is None:
|
||||
avail = ", ".join(str(s["index"]) for s in sub_list)
|
||||
sys.exit(f"No subtitle stream #{args.embedded_sub_index} in "
|
||||
f"{Path(args.video).name}. Available: {avail}")
|
||||
print(f" Using subtitle stream #{chosen['index']} "
|
||||
f"({chosen['codec_name']}{' ' + chosen['language'] if chosen['language'] else ''})")
|
||||
else:
|
||||
print(f" Available subtitle tracks in {Path(args.video).name}:")
|
||||
for i, s in enumerate(sub_list):
|
||||
lang = f" [{s['language']}]" if s['language'] else ""
|
||||
|
||||
+2
-2
@@ -14,7 +14,7 @@
|
||||
[tts]
|
||||
# Voice reference. Either a name in voices/ (expects NAME.wav + NAME.txt),
|
||||
# or a full path to a .wav file.
|
||||
voice = gudowski_short
|
||||
voice = gudowski_short_3
|
||||
|
||||
# Per-clip speed-up applied with atempo, before assembly.
|
||||
# Raise it if speech regularly overruns its subtitle window; 1.0 = no change.
|
||||
@@ -23,7 +23,7 @@ speed = 1.2
|
||||
# Sampling temperature. 1.0 makes the model occasionally recite the reference
|
||||
# transcript instead of the subtitle; 0.9 largely prevents that. Lower is more
|
||||
# deterministic but flatter.
|
||||
temperature = 0.7
|
||||
temperature = 0.6
|
||||
|
||||
# Ceiling on audio tokens per clip. Raise only if long clips are cut off
|
||||
# mid-word.
|
||||
|
||||
Executable
+49
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# start_lektor_ui.sh — launch the lektor_ai voiceover queue UI.
|
||||
#
|
||||
# Picks a Python that actually has tkinter. The system one is preferred: its Tk
|
||||
# is built against fontconfig and renders real fonts, while the miniforge build
|
||||
# only carries legacy X11 bitmap fonts and looks dated. Either will run.
|
||||
#
|
||||
# ComfyUI needs to be running for GENERATE to do anything — start_comfyui.sh.
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$(readlink -f "$0")")"
|
||||
|
||||
if [[ ! -f ui.py ]]; then
|
||||
echo "ui.py not found next to this script ($PWD)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Preference order, best-looking first.
|
||||
CANDIDATES=(
|
||||
/usr/bin/python3
|
||||
python3
|
||||
"$HOME/miniforge3/bin/python"
|
||||
"$HOME/miniforge3/envs/comfy-tts/bin/python"
|
||||
)
|
||||
|
||||
PY=""
|
||||
for c in "${CANDIDATES[@]}"; do
|
||||
if command -v "$c" >/dev/null 2>&1 && "$c" -c "import tkinter" >/dev/null 2>&1; then
|
||||
PY="$c"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ -z "$PY" ]]; then
|
||||
cat >&2 <<'EOF'
|
||||
No Python with tkinter was found.
|
||||
|
||||
Install it for the system Python (recommended — gives proper fonts):
|
||||
sudo apt install python3-tk
|
||||
|
||||
Miniforge's Python also ships tkinter if you have it:
|
||||
~/miniforge3/bin/python ui.py
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exec "$PY" ui.py "$@"
|
||||
@@ -0,0 +1,2 @@
|
||||
Sprawował dyplomatyczną misję w trudnym okresie, w czasie wojen bałkańskich i na początku Wielkiej Wojny. W roku tysiąc dziewięćset szesnastym prosto z Sofii udał się do Stanów Zjednoczonych, by objąć funkcję ambasadora Austro-Węgier w tym kraju. Nie otrzymał jednak od aliantów dyplomatycznego listu żelaznego. Podróżował zatem do Ameryki okrężną drogą. Zjawił się w Waszyngtonie dopiero pod koniec stycznia tysiąc dziewięćset siedemnastego roku, kiedy Niemcy zaczynały swoją nieograniczoną wojnę podwodną na Atlantyku. Tarnowski tłumaczył amerykańskiemu sekretarzowi stanu Robertowi Lansingowi, że Austro-Węgry nie mają nic wspólnego z atakowaniem i zatapianiem przez okręty podwodne statków pasażerskich i handlowych. Kiedy okazało się, że prawda jest inna, Tarnowski wyjaśniał amerykańskim politykom, że to Niemcy. Zwykle w czwartki po południu jestem w Armfilmie. Jest tu bardzo fajnie.
|
||||
|
||||
Binary file not shown.
Reference in New Issue
Block a user