lukas/lektor-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Settings file, per-job clip folders, faster polling, voices/ rename

Browse Source 
Poll interval
  wait_for_completion() polled every 3 s, which quantised every clip to a 3 s
  boundary — across 2075 existing clips, 91% of the gaps landed exactly on a
  multiple of 3 s. It now starts at 0.25 s and backs off to 2 s after 10 s, so
  short clips are picked up immediately without hammering the server on long
  ones. Measured over 34 clips: 4.75 s actual vs 6.35 s under the old poll,
  a 25% cut in TTS wait (~18 min on a 693-clip episode). Runs now print real
  generation time, e.g. "done (4.2s)".

  A WebSocket would remove polling entirely but needs a third-party client,
  which would cost this script its dependency-free property.

settings.ini
  voice, speed, temperature, max_tokens, strip_leading_ms, trailing_fade_ms,
  orig_volume, tts_volume, output_dir and comfy_url moved to an INI file next
  to the script. Precedence is CLI flag > settings.ini > built-in default,
  wired by feeding loaded values in as argparse defaults. --settings FILE
  selects an alternative profile.

  INI rather than TOML (tomllib is 3.11+, this runs on system Python 3.10) or
  JSON (no comments). A missing, partial, malformed or corrupt file degrades to
  built-in defaults with a warning instead of killing a long run.

  Run-specific flags are deliberately excluded: --start, --end, --seed,
  --regen, --rebuild, --resume-from, --embedded-subs, --skip-tts.

Per-job folders
  All state for one video now lives in <output_dir>/clips/<video name>/ —
  clips, tts_input.srt, tts_assembled.flac, embedded_sub.srt and
  video_fragment.mkv. Only <stem>_lektor.mkv stays at the top level.

  tts_input.srt moved with the clips because they are a matched pair: block N
  in the SRT is clip_NNNN.flac. Previously both sat at the top level and each
  new job overwrote the last one's, so rebuilding an older film silently
  paired its clips with the newest film's text. Any past job can now be
  rebuilt. A run against the old flat layout gets an explanatory message.

gudowski/ -> voices/
  VOICE_PROMPTS_DIR is now resolved relative to the script rather than
  hardcoded, so moving the project no longer breaks it. Voice filenames are
  unchanged, so --voice gudowski_short still works.

Docs updated throughout for the new layout, settings file and poll behaviour.

Known issue, not addressed here: with a non-zero --start the voiceover plays
early. extract_video_fragment() stream-copies, so it can only cut on a
keyframe and seeks backwards to it, while TTS timestamps are relative to the
requested start. On S01_E08 at --start 07:40 the nearest earlier keyframe is
458.375 s, putting the voiceover 1.625 s ahead of the picture. Runs with
--start 00:00 are unaffected, since 0 is always a keyframe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-09 21:56:42 +02:00
co-authored by Claude Opus 5
parent fc4b761f52
commit 3dde6e58e4
15 changed files with 430 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+32 -6
View File
@@ -8,13 +8,13 @@ the straightforward version was tried first and broke.
SRT / embedded track
│ parse, clean, split dialogue
▼
subtitle blocks ──────────────► output/tts_input.srt
subtitle blocks ──────────────► <job>/tts_input.srt
│ one HTTP job per block
▼
ComfyUI + Higgs Audio V3
│ per-clip atempo + fade
▼
clips/clip_NNNN.flac
<job>/clip_NNNN.flac
│ adelay + amix, batched
▼
tts_assembled.flac
@@ -49,15 +49,41 @@ in step 3 would silently miss it.
`split_dialogue_blocks()` then divides a two-speaker block into one clip per
line, apportioning the block's duration by character count.
The result is written to `output/tts_input.srt`. That file is the source of
truth for block numbering: block *N* there is `clips/clip_NNNN.flac` and the
The result is written to `tts_input.srt` in that video's job folder
(`output/clips/<video name>/`). It is the source of truth for block
numbering: block *N* there is `clip_NNNN.flac` beside it and the
argument to `--regen N`.
## 2. Speech generation
For each block the script POSTs a workflow graph to `/prompt`, polls
`/history/{id}` every 3 s, and reads the resulting file directly from ComfyUI's
output directory.
`/history/{id}` until it completes, and reads the resulting file directly from
ComfyUI's output directory.
**Polling cadence is a performance decision, not a detail.** A clip that
finishes just after a poll waits for the next one, so the interval — not the
GPU — sets the floor on throughput. It was originally a flat 3 s, which
quantised every clip to a 3 s boundary: measured across 2075 real clips, 91% of
the gaps landed exactly on a multiple of 3 s.
`wait_for_completion()` now starts at 0.25 s and backs off to 2 s once a job
passes 10 s, so the common case is picked up almost immediately while a stuck
generation does not sit at 4 requests/second indefinitely. Measured over 34
clips:
| | mean | median | 34 clips |
|---|---|---|---|
| Actual generation | 4.75 s | 4.60 s | 162 s |
| Old flat 3 s poll | 6.35 s | 6.00 s | 216 s |
That is 1.60 s wasted per clip, a **25% reduction in TTS wait** — about 18
minutes on a 693-clip episode. The run also now prints real generation time
(`done (4.2s)`), which is what makes this measurable at all.
ComfyUI also pushes completion events over a WebSocket, which would remove
polling entirely. It is not used because Python has no standard-library
WebSocket client, and that would cost this script its only real virtue as a
dependency-free file. Polling at 0.25 s recovers most of the benefit.
The graph has five nodes:
docs/installation.md
+21 -11
View File
@@ -144,20 +144,30 @@ is not needed; the script drives everything over HTTP.
## Configuring the script
`lektor_ai_comfy.py` has four constants at the top. If you installed anywhere
other than the defaults, edit them:
Most settings live in **`settings.ini`** next to the script — see
[usage.md](usage.md#settings-file). The backend address in particular:
```python
COMFY_URL = "http://127.0.0.1:8189"
COMFY_OUTPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
COMFY_INPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/input")
VOICE_PROMPTS_DIR = Path("/home/lukas/Programy/lektor_ai/gudowski")
```ini
[comfy]
comfy_url = http://127.0.0.1:8189
```
`COMFY_URL` can also be overridden per-run with `--comfy-url`. The two directory
constants cannot — the script reads generated audio directly from ComfyUI's
output directory and copies voice references into its input directory, so both
must be real paths on the same filesystem.
Two paths remain constants in `lektor_ai_comfy.py`, because the script touches
ComfyUI's own directories directly rather than over HTTP:
```python
COMFY_OUTPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/output")
COMFY_INPUT_DIR = Path("/home/lukas/comfy-tts-app/ComfyUI/input")
```
Generated audio is read straight out of the output directory and voice
references are copied into the input directory, so both must be real paths
reachable from wherever the script runs. **If you installed ComfyUI anywhere
other than `~/comfy-tts-app`, edit these two lines.** They are also what would
have to change first to drive a ComfyUI instance on another machine.
A third path, `VOICE_PROMPTS_DIR`, resolves to `voices/` beside the script and
needs no editing as long as the project stays intact.
The script itself needs no installation. Ensure ffmpeg is present:
docs/requirements.md
+1 -1
View File
@@ -110,7 +110,7 @@ text and cannot drive TTS without OCR; the `--embedded-subs` picker marks them
as unusable. They are still copied to the output untouched.
**Voice reference** — a `.wav` plus a `.txt` holding its exact transcript, both
in `gudowski/`. Three are bundled:
in `voices/`. Three are bundled:
| Voice | Length | Format |
|---|---|---|
docs/troubleshooting.md
+1 -1
View File
@@ -212,7 +212,7 @@ instead — the image tracks are still copied to the output.
### Wrong language selected
The picker lists language tags but they are only as good as the source's
metadata. `output/embedded_sub.srt` holds exactly what was extracted; check it
metadata. `embedded_sub.srt` in the job folder holds exactly what was extracted; check it
before committing to a full run.
## Backend problems
docs/usage.md
+107 -15
View File
@@ -60,7 +60,7 @@ Lists the video's subtitle tracks and prompts for one:
Select track number: 30
```
The chosen track is extracted to `output/embedded_sub.srt` and then processed
The chosen track is extracted to `embedded_sub.srt` in the job folder and processed
exactly like an external file. Image-based tracks are listed but marked
unusable — they hold bitmaps, not text.
@@ -74,9 +74,9 @@ python3 lektor_ai_comfy.py input/film.mkv --rebuild
python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.5 --tts-volume 2.0
```
Use it after `--regen`, or to adjust the mix. It requires `output/tts_input.srt`
and a complete set of `output/clips/clip_NNNN.flac`, and reports which are
missing if any are.
Use it after `--regen`, or to adjust the mix. It reads that video's job folder
(`output/clips/<video name>/`) and needs its `tts_input.srt` plus a complete set
of `clip_NNNN.flac`, reporting which are missing if any are.
If the original run used `--end`, pass it again so the same video fragment is
used.
@@ -90,7 +90,7 @@ python3 lektor_ai_comfy.py input/film.mkv --regen 12 22 41
python3 lektor_ai_comfy.py input/film.mkv --rebuild # then apply
```
Block numbers match `output/tts_input.srt` and the `clip_NNNN.flac` filenames.
Block numbers match that job's `tts_input.srt` and its `clip_NNNN.flac` files.
Only the selected FLAC files are replaced — `--rebuild` afterwards is what
actually updates the video.
@@ -116,11 +116,79 @@ the last one that completed.
Uses a pre-assembled audio file and goes straight to the mux.
```bash
python3 lektor_ai_comfy.py input/film.mkv --skip-tts output/tts_assembled.flac
python3 lektor_ai_comfy.py input/film.mkv --skip-tts "output/clips/film/tts_assembled.flac"
```
## Settings file
Defaults live in **`settings.ini`** next to the script, so the values you always
want no longer need to be typed on every run. Precedence:
```
command-line flag > settings.ini > built-in default
```
```ini
[tts]
voice = gudowski_short
speed = 1.2
temperature = 0.9
max_tokens = 4096
strip_leading_ms = 0
trailing_fade_ms = 80
[mix]
orig_volume = 0.60
tts_volume = 1.8
[paths]
output_dir = output
[comfy]
comfy_url = http://127.0.0.1:8189
```
Every one of these has a matching flag that overrides it for a single run:
```bash
# settings.ini says tts_volume = 1.8; this run uses 2.5
python3 lektor_ai_comfy.py film.mkv --embedded-subs --tts-volume 2.5
```
Each run prints which file it used, so there is never any doubt:
```
Settings: /home/lukas/Programy/lektor_ai/settings.ini
```
**Alternative profiles** — keep per-show settings side by side and select one:
```bash
python3 lektor_ai_comfy.py film.mkv --embedded-subs --settings profiles/docu.ini
```
**Behaviour when things are missing or wrong**, none of which is fatal:
| Situation | Result |
|---|---|
| No `settings.ini` | Built-in defaults; the run says so |
| Key absent from the file | That key falls back to its built-in default |
| Value unparseable (`speed = fast`) | Warning, that key falls back, run continues |
| File is not valid INI | Warning, all built-in defaults, run continues |
Comments use `#` or `;`, inline comments included. Deleting a line is a valid
way to return it to its built-in default.
**Not in the file, by design**: `--start`, `--end`, `--seed`, `--regen`,
`--rebuild`, `--resume-from`, `--embedded-subs`, `--skip-tts`. These describe a
particular run rather than your setup, so pinning them in config would be a
trap.
## Options
Every option below can also be set in `settings.ini`, except those noted as
run-specific.
### Range
| Flag | Default | Meaning |
@@ -128,7 +196,7 @@ python3 lektor_ai_comfy.py input/film.mkv --skip-tts output/tts_assembled.flac
| `--start MM:SS` | `00:00` | Start of the range to process |
| `--end MM:SS` | *(end)* | End of the range |
With `--end`, a matching fragment is extracted to `output/video_fragment.mkv`
With `--end`, a matching fragment is extracted to `video_fragment.mkv` in the job folder
and all TTS timestamps become relative to `--start`. Pass the same values to any
later `--rebuild`, or the audio will not line up.
@@ -138,7 +206,7 @@ later `--rebuild`, or the audio will not line up.
| Flag | Default | Meaning |
|---|---|---|
| `--voice NAME\|PATH` | `gudowski_short` | Name in `gudowski/`, or a path to a WAV |
| `--voice NAME\|PATH` | `gudowski_short` | Name in `voices/`, or a path to a WAV |
| `--seed N` | `0` | Base seed; `0` randomises each run |
| `--temperature T` | `0.9` | Sampling temperature |
| `--max-tokens N` | `4096` | Token ceiling per clip |
@@ -187,16 +255,40 @@ Typical starting point for dialogue-heavy material: `--orig-volume 0.5
## Output
Everything belonging to one video is kept together in a **job folder** named
after the video file. Only the finished video is written to the top level.
```
output/
tts_input.srt processed text actually sent to TTS — inspect this
embedded_sub.srt extracted embedded subtitle (--embedded-subs only)
clips/clip_NNNN.flac one lossless clip per block, kept for --regen
tts_assembled.flac all clips at their final positions, lossless
video_fragment.mkv extracted segment (--end only)
{stem}_lektor.mkv the finished video
clips/
S01_E01 The Eyes/ job folder, named after the video file
clip_NNNN.flac one lossless clip per block, kept for --regen
tts_input.srt processed text actually sent to TTS
tts_assembled.flac all clips at their final positions, lossless
embedded_sub.srt extracted subtitle (--embedded-subs only)
video_fragment.mkv extracted segment (--end only)
F1 The Movie 2025/
...
S01_E01 The Eyes_lektor.mkv the finished video
F1 The Movie 2025_lektor.mkv
```
Each run prints the folder it is using:
```
Job folder: output/clips/S01_E01 The Eyes
```
This is what makes `--rebuild` and `--regen` safe across films. `tts_input.srt`
and the clips are a matched pair — block *N* in the SRT is `clip_NNNN.flac` —
so both live in the job folder. They previously sat at the top level and every
new job overwrote the last one's, which meant rebuilding an older film paired
its clips with the newest film's text. Any past job can now be rebuilt at any
time.
The folder is derived from the video filename, so `--rebuild` and `--regen`
locate it automatically; just pass the same video path.
The finished file contains:
| Track | Content |
@@ -229,7 +321,7 @@ python3 lektor_ai_comfy.py input/film.mkv --rebuild --orig-volume 0.45 --tts-vol
**Fix a handful of bad clips**
```bash
# read output/tts_input.srt, note the offending block numbers
# read output/clips/<video name>/tts_input.srt, note the offending block numbers
python3 lektor_ai_comfy.py input/film.mkv --regen 88 91 104
python3 lektor_ai_comfy.py input/film.mkv --rebuild
```