lukas/lektor-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Settings file, per-job clip folders, faster polling, voices/ rename

Browse Source 
Poll interval
  wait_for_completion() polled every 3 s, which quantised every clip to a 3 s
  boundary — across 2075 existing clips, 91% of the gaps landed exactly on a
  multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so
  short clips are picked up immediately without hammering the server on long
  ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll,
  a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real
  generation time, e.g. "done (4.2s)".

  A WebSocket would remove polling entirely but needs a third-party client,
  which would cost this script its dependency-free property.

settings.ini
  voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
  orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next
  to the script. Precedence is CLI flag > settings.ini > built-in default,
  wired by feeding loaded values in as argparse defaults. --settings FILE
  selects an alternative profile.

  INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or
  JSON (no comments). A missing, partial, malformed or corrupt file degrades to
  built-in defaults with a warning instead of killing a long run.

  Run-specific flags are deliberately excluded: --start, --end, --seed,
  --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts.

Per-job folders
  All state for one video now lives in <output_dir>/clips/<video name>/ —
  clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and
  video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level.

  tts_input.srt moved with the clips because they are a matched pair: block N
  in the SRT is clip_NNNN.flac. Previously both sat at the top level and each
  new job overwrote the last one's, so rebuilding an older film silently
  paired its clips with the newest film's text. Any past job can now be
  rebuilt. A run against the old flat layout gets an explanatory message.

gudowski/ -> voices/
  VOICE_PROMPTS_DIR is now resolved relative to the script rather than
  hardcoded, so moving the project no longer breaks it. Voice filenames are
  unchanged, so --voice gudowski_short still works.

Docs updated throughout for the new layout, settings file and poll behaviour.

Known issue, not addressed here: with a non-zero --start the voiceover plays
early. extract_video_fragment() stream-copies, so it can only cut on a
keyframe and seeks backwards to it, while TTS timestamps are relative to the
requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is
458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with
--start 00:00 are unaffected, since 0 is always a keyframe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-09 21:56:42 +02:00
co-authored by Claude Opus 5
parent fc4b761f52
commit 3dde6e58e4
15 changed files with 430 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lektor_ai_comfy.py
+182 -47
View File
@@ -9,6 +9,7 @@ Requires ComfyUI with TTS-Audio-Suite running at http://127.0.0.1:8189
"""
import argparse
import configparser
import json
import random
import re
@@ -25,10 +26,63 @@ from urllib.request import urlopen, Request
# Config
# ---------------------------------------------------------------------------
COMFY_URL = "http://127.0.0.1:8189"
COMFY_OUTPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
COMFY_INPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/input")
VOICE_PROMPTS_DIR = Path("/home/lukas/Programy/lektor_ai/gudowski")
VOICE_PROMPTS_DIR = Path(__file__).resolve().parent / "voices"
SETTINGS_FILE = Path(__file__).resolve().parent / "settings.ini"
# Fallback values, used when settings.ini is absent or omits a key. Keeping them
# here means the script still runs correctly with no settings file at all.
# Each entry is (ini_section, ini_key, type, value).
_DEFAULTS = {
"voice": ("tts", "voice", str, "gudowski_short"),
"speed": ("tts", "speed", float, 1.2),
"temperature": ("tts", "temperature", float, 0.9),
"max_tokens": ("tts", "max_tokens", int, 4096),
"strip_leading_ms": ("tts", "strip_leading_ms", float, 0.0),
"trailing_fade_ms": ("tts", "trailing_fade_ms", float, 80.0),
"orig_volume": ("mix", "orig_volume", float, 0.60),
"tts_volume": ("mix", "tts_volume", float, 1.8),
"output_dir": ("paths", "output_dir", str, "output"),
"comfy_url": ("comfy", "comfy_url", str, "http://127.0.0.1:8189"),
}
COMFY_URL = _DEFAULTS["comfy_url"][3] # rebound from settings/CLI in main()
def load_settings(path: Path) -> dict:
"""Read settings.ini, falling back to _DEFAULTS for anything missing.
Values here become argparse's defaults, so precedence ends up as
CLI flag > settings.ini > _DEFAULTS without any extra bookkeeping.
A malformed or unreadable file is reported and then ignored rather than
being fatal — a typo in a config file should not cost you a long run.
"""
values = {key: spec[3] for key, spec in _DEFAULTS.items()}
if not path.is_file():
return values
parser = configparser.ConfigParser(inline_comment_prefixes=("#", ";"))
try:
parser.read(path, encoding="utf-8")
except (configparser.Error, OSError) as exc:
print(f" WARNING: could not read {path.name} ({exc}) — using built-in defaults",
file=sys.stderr)
return values
for key, (section, option, cast, fallback) in _DEFAULTS.items():
if not parser.has_option(section, option):
continue
raw = parser.get(section, option).strip()
if not raw:
continue
try:
values[key] = cast(raw)
except ValueError:
print(f" WARNING: {path.name} [{section}] {option}={raw!r} is not a valid "
f"{cast.__name__} — using {fallback!r}", file=sys.stderr)
return values
# Subtitle codecs that carry real text and can therefore be extracted and fed
# to the TTS engine. Used only by the --embedded-subs picker; image-based
@@ -642,14 +696,31 @@ def submit_workflow(workflow: dict) -> str:
return resp["prompt_id"]
def wait_for_completion(prompt_id: str, poll_interval: int = 3) -> dict:
def wait_for_completion(prompt_id: str, poll_interval: float = 0.25,
max_interval: float = 2.0, backoff_after: float = 10.0) -> dict:
"""Block until ComfyUI reports the job finished, then return its history entry.
The poll interval, not the GPU, governs perceived generation speed: a clip
that finishes 100 ms after a poll still waits for the next one. This used to
be a flat 3 s, which quantised every clip to a 3 s boundary — measured across
2075 real clips, 91% of the gaps landed exactly on a multiple of 3 s, and
roughly a third of each run was spent sleeping on already-completed work.
Polling starts tight so the common case (most clips finish in 3-6 s) is
picked up almost immediately, then backs off once a job looks slow so a
stuck or very long generation does not sit at 4 requests/second forever.
"""
print(f" Waiting for {prompt_id[:8]}…", end="", flush=True)
start = time.monotonic()
interval = poll_interval
next_dot = 1.0
while True:
try:
history = _comfy_get(f"/history/{prompt_id}")
except Exception as e:
print(f"\n Poll error: {e}", file=sys.stderr)
time.sleep(poll_interval)
time.sleep(min(interval, 1.0))
continue
if prompt_id in history:
@@ -660,11 +731,17 @@ def wait_for_completion(prompt_id: str, poll_interval: int = 3) -> dict:
print(f"\nComfyUI error: {msg_data}", file=sys.stderr)
sys.exit(1)
if status.get("completed"):
print(" done.")
print(f" done ({time.monotonic() - start:.1f}s).")
return entry
print(".", end="", flush=True)
time.sleep(poll_interval)
elapsed = time.monotonic() - start
# One dot per second regardless of poll rate, or the line fills with dots.
if elapsed >= next_dot:
print(".", end="", flush=True)
next_dot += 1.0
time.sleep(interval)
if elapsed > backoff_after:
interval = min(interval * 1.5, max_interval)
def extract_output_audio(history_entry: dict) -> Path:
@@ -762,18 +839,45 @@ def generate_clips(blocks: list, voice_filename: str, voice_ref_text: str,
# Rebuild / Regen helpers
# ---------------------------------------------------------------------------
def job_dir(out_dir: Path, video_path: str) -> Path:
"""Per-job working directory: <output_dir>/clips/<video filename without extension>/
Everything that belongs to one video lives here — the clips, the processed
tts_input.srt, the assembled audio, any extracted subtitle or video
fragment. Only the finished <stem>_lektor.mkv is written to <output_dir>.
Without this separation every job overwrote the previous one's clip_NNNN.flac
and tts_input.srt, so --rebuild and --regen were only ever safe on the most
recent run. The clips and tts_input.srt are a matched pair — block N in the
SRT is clip_NNNN.flac — which is why the SRT lives here too rather than at
the top level.
"""
return out_dir / "clips" / Path(video_path).stem
def _legacy_hint(out_dir: Path, job: Path) -> str:
"""Point at the pre-per-job layout if that is what the user actually has."""
if (out_dir / "tts_input.srt").is_file():
return (f"\n\nA tts_input.srt exists at {out_dir} in the old shared layout. "
f"Clips are now kept per job in {job}. Move that run's files there, "
f"or regenerate it.")
return ""
def _rebuild_video(args, out_dir: Path):
"""Re-assemble audio from existing clips and rebuild the final video.
No TTS generation — use this to change volumes or fix assembly without
waiting for the model to run again.
"""
tts_srt = out_dir / "tts_input.srt"
job = job_dir(out_dir, args.video)
tts_srt = job / "tts_input.srt"
if not tts_srt.exists():
sys.exit(f"No tts_input.srt found in {out_dir} — run a full generation first.")
sys.exit(f"No tts_input.srt found in {job} — run a full generation first."
+ _legacy_hint(out_dir, job))
blocks = load_tts_input_srt(tts_srt)
total = len(blocks)
clips_dir = out_dir / "clips"
clips_dir = job
missing = [
clips_dir / f"clip_{i+1:04d}.flac"
@@ -793,7 +897,7 @@ def _rebuild_video(args, out_dir: Path):
clips.append((actual_start, clip_path))
cursor = actual_start + duration
assembled_tts = out_dir / "tts_assembled.flac"
assembled_tts = job / "tts_assembled.flac"
print(f"\n Assembling {total} clips…")
assemble_timed_audio(clips, str(assembled_tts))
print(f" Assembled: {assembled_tts} ({assembled_tts.stat().st_size // 1024} KB)")
@@ -803,7 +907,7 @@ def _rebuild_video(args, out_dir: Path):
if args.end:
start_sec = _parse_mmss(args.start)
end_sec = _parse_mmss(args.end)
video_fragment = out_dir / "video_fragment.mkv"
video_fragment = job / "video_fragment.mkv"
if not video_fragment.exists():
extract_video_fragment(args.video, start_sec, end_sec, str(video_fragment))
video_src = str(video_fragment)
@@ -824,15 +928,17 @@ def _regen_clips(args, out_dir: Path):
"""Regenerate specific clips by tts_input block number.
Only replaces FLAC files — run --rebuild afterwards to reassemble audio and video.
"""
tts_srt = out_dir / "tts_input.srt"
job = job_dir(out_dir, args.video)
tts_srt = job / "tts_input.srt"
if not tts_srt.exists():
sys.exit(f"No tts_input.srt found in {out_dir} — run a full generation first.")
sys.exit(f"No tts_input.srt found in {job} — run a full generation first."
+ _legacy_hint(out_dir, job))
blocks = load_tts_input_srt(tts_srt)
total = len(blocks)
regen_set = set(args.regen)
clips_dir = out_dir / "clips"
clips_dir.mkdir(exist_ok=True)
clips_dir = job
clips_dir.mkdir(parents=True, exist_ok=True)
bad = [n for n in regen_set if n < 1 or n > total]
if bad:
@@ -874,9 +980,23 @@ def _regen_clips(args, out_dir: Path):
def main():
global COMFY_URL
# --settings is resolved before the main parser is built, because the file's
# contents become that parser's defaults.
pre = argparse.ArgumentParser(add_help=False)
pre.add_argument("--settings", default=str(SETTINGS_FILE), metavar="FILE")
pre_args, _ = pre.parse_known_args()
settings_path = Path(pre_args.settings)
cfg = load_settings(settings_path)
parser = argparse.ArgumentParser(
description="Generate AI voiceover via ComfyUI + Higgs Audio V3, per-subtitle."
description="Generate AI voiceover via ComfyUI + Higgs Audio V3, per-subtitle.",
epilog=f"Defaults are read from {SETTINGS_FILE.name} next to this script; "
f"any flag given on the command line overrides it.",
)
parser.add_argument("--settings", default=str(SETTINGS_FILE), metavar="FILE",
help=f"Settings file to read defaults from "
f"(default: {SETTINGS_FILE.name})")
parser.add_argument("srt", nargs="?", default=None,
help="Input SRT subtitle file (omit with --embedded-subs)")
parser.add_argument("video", nargs="?", default=None,
@@ -885,33 +1005,39 @@ def main():
help="Fragment start time (default: 00:00)")
parser.add_argument("--end", default=None, metavar="MM:SS",
help="Fragment end time")
parser.add_argument("--voice", default="gudowski_short",
help="Voice prompt name or WAV path (default: gudowski_short)")
parser.add_argument("--speed", type=float, default=1.2, metavar="FACTOR",
help="Per-clip playback speed via atempo (default: 1.2)")
parser.add_argument("--voice", default=cfg["voice"],
help=f"Voice prompt name or WAV path (default: {cfg['voice']})")
parser.add_argument("--speed", type=float, default=cfg["speed"], metavar="FACTOR",
help=f"Per-clip playback speed via atempo (default: {cfg['speed']})")
parser.add_argument("--seed", type=int, default=0,
help="Base generation seed (default: 0 = random each run)")
parser.add_argument("--orig-volume", type=float, default=0.60, metavar="0-1",
help="Original audio volume in mix (default: 0.60)")
parser.add_argument("--tts-volume", type=float, default=1.8, metavar="0-2",
help="Voiceover volume in mix (default: 1.8)")
parser.add_argument("--output-dir", default="output",
help="Output directory (default: output/)")
parser.add_argument("--comfy-url", default=COMFY_URL,
help=f"ComfyUI URL (default: {COMFY_URL})")
parser.add_argument("--orig-volume", type=float, default=cfg["orig_volume"], metavar="0-1",
help=f"Original audio volume in mix (default: {cfg['orig_volume']})")
parser.add_argument("--tts-volume", type=float, default=cfg["tts_volume"], metavar="0-2",
help=f"Voiceover volume in mix (default: {cfg['tts_volume']})")
parser.add_argument("--output-dir", default=cfg["output_dir"],
help=f"Output directory (default: {cfg['output_dir']}/)")
parser.add_argument("--comfy-url", default=cfg["comfy_url"],
help=f"ComfyUI URL (default: {cfg['comfy_url']})")
parser.add_argument("--skip-tts", metavar="AUDIO_FILE",
help="Skip TTS and use this pre-assembled audio file")
parser.add_argument("--temperature", type=float, default=0.9, metavar="T",
help="Sampling temperature (default: 0.9; lower = more deterministic)")
parser.add_argument("--strip-leading-ms", type=float, default=0.0, metavar="MS",
help="Strip N ms from the start of each TTS clip to remove "
"leading breath/noise artifact (default: 0, try 80–150)")
parser.add_argument("--trailing-fade-ms", type=float, default=80.0, metavar="MS",
help="Fade out the last N ms of each TTS clip to suppress "
"tail pops/noise (default: 80, set 0 to disable)")
parser.add_argument("--max-tokens", type=int, default=4096, metavar="N",
help="Max audio tokens the TTS model may generate per clip "
"(default: 4096; raise if long clips are cut off mid-word)")
parser.add_argument("--temperature", type=float, default=cfg["temperature"], metavar="T",
help=f"Sampling temperature (default: {cfg['temperature']}; "
f"lower = more deterministic)")
parser.add_argument("--strip-leading-ms", type=float, default=cfg["strip_leading_ms"],
metavar="MS",
help=f"Strip N ms from the start of each TTS clip to remove "
f"leading breath/noise artifact "
f"(default: {cfg['strip_leading_ms']}, try 80–150)")
parser.add_argument("--trailing-fade-ms", type=float, default=cfg["trailing_fade_ms"],
metavar="MS",
help=f"Fade out the last N ms of each TTS clip to suppress "
f"tail pops/noise (default: {cfg['trailing_fade_ms']}, "
f"set 0 to disable)")
parser.add_argument("--max-tokens", type=int, default=cfg["max_tokens"], metavar="N",
help=f"Max audio tokens the TTS model may generate per clip "
f"(default: {cfg['max_tokens']}; raise if long clips are "
f"cut off mid-word)")
parser.add_argument("--resume-from", type=int, default=0, metavar="N",
help="Resume a crashed run: skip blocks 1..(N-1) by loading "
"their existing clip files, generate from block N onwards.")
@@ -933,6 +1059,11 @@ def main():
COMFY_URL = args.comfy_url
if settings_path.is_file():
print(f" Settings: {settings_path}")
else:
print(f" Settings: {settings_path.name} not found — using built-in defaults")
# ── Resolve positional ambiguity for modes that don't need an SRT file ────
# When the user writes `script.py video.mkv --embedded-subs` (or --rebuild /
# --regen without an SRT), argparse puts the lone positional into args.srt.
@@ -947,10 +1078,8 @@ def main():
parser.error("Provide an SRT subtitle file, or use --embedded-subs to pick one "
"from the video.")
out_dir = Path(args.output_dir)
out_dir = Path(args.output_dir)
out_dir.mkdir(parents=True, exist_ok=True)
clips_dir = out_dir / "clips"
clips_dir.mkdir(exist_ok=True)
# ── Rebuild mode: reassemble + remux from existing clips, no TTS ─────────
if args.rebuild:
@@ -962,6 +1091,12 @@ def main():
_regen_clips(args, out_dir)
return
# Everything belonging to this video lives together, so successive jobs
# cannot overwrite each other's clips or tts_input.srt.
clips_dir = job_dir(out_dir, args.video)
clips_dir.mkdir(parents=True, exist_ok=True)
print(f" Job folder: {clips_dir}")
start_sec = _parse_mmss(args.start)
end_sec = _parse_mmss(args.end) if args.end else None
@@ -991,7 +1126,7 @@ def main():
if chosen['codec_name'].lower() not in _TEXT_SUB_CODECS:
sys.exit(f"Stream #{chosen['index']} ({chosen['codec_name']}) is image-based "
"and cannot be converted to text for TTS.")
embedded_srt = out_dir / "embedded_sub.srt"
embedded_srt = clips_dir / "embedded_sub.srt"
extract_subtitle_track(args.video, chosen['index'], str(embedded_srt))
srt_path = str(embedded_srt)
else:
@@ -1005,7 +1140,7 @@ def main():
blocks = split_dialogue_blocks(blocks)
print(f" {len(blocks)} entries after dialogue splitting.")
srt_preview = out_dir / "tts_input.srt"
srt_preview = clips_dir / "tts_input.srt"
srt_preview.write_text(blocks_to_srt(blocks), encoding="utf-8")
print(f" Saved to {srt_preview} for inspection.")
@@ -1035,7 +1170,7 @@ def main():
trailing_fade_ms=args.trailing_fade_ms,
max_new_tokens=args.max_tokens)
assembled_tts = out_dir / "tts_assembled.flac"
assembled_tts = clips_dir / "tts_assembled.flac"
assemble_timed_audio(clips, str(assembled_tts))
print(f" Assembled: {assembled_tts} ({assembled_tts.stat().st_size // 1024} KB)")
print(f" Clips saved to {clips_dir}")
@@ -1044,7 +1179,7 @@ def main():
print(f"\n[3/4] Extracting video fragment…")
real_end = _parse_mmss(args.end) if args.end else None
if real_end is not None:
video_fragment = out_dir / "video_fragment.mkv"
video_fragment = clips_dir / "video_fragment.mkv"
extract_video_fragment(args.video, start_sec, real_end, str(video_fragment))
else:
video_fragment = Path(args.video)