Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+32
-4
@@ -6,7 +6,7 @@
|
||||
|---|---|---|
|
||||
| GPU | NVIDIA, 12 GB VRAM | Developed on an RTX 4070 Ti SUPER (16 GB) |
|
||||
| Driver | 525+ | Needed by the CUDA 12.8 wheels |
|
||||
| Disk | ~35 GB free | ~22 GB installed, plus working room for output |
|
||||
| Disk | ~35 GB free | ~22 GB installed (+8 GB with the number normalizer) |
|
||||
| RAM | 16 GB | The final mux briefly peaks around 2.6 GB |
|
||||
|
||||
The CUDA **toolkit** is not required. The PyTorch cu128 wheels bundle their own
|
||||
@@ -25,6 +25,9 @@ feature-length film.
|
||||
└── models/TTS 8.8 GB Higgs Audio V3 weights
|
||||
------
|
||||
21.7 GB
|
||||
|
||||
lektor_ai/normalizer/venv 6.9 GB optional: number normalizer env
|
||||
lektor_ai/normalizer/model 1.1 GB optional: its weights
|
||||
```
|
||||
|
||||
Output is roughly the size of the source video, since video is stream-copied. A
|
||||
@@ -49,13 +52,13 @@ regenerated without redoing a whole run (see
|
||||
## Dependency surface
|
||||
|
||||
This is the point most worth internalising, because it makes debugging much
|
||||
simpler: **the two halves have almost nothing in common.**
|
||||
simpler: **the three parts have almost nothing in common.**
|
||||
|
||||
### `lektor_ai_comfy.py` — standard library only
|
||||
|
||||
```python
|
||||
argparse json random re shutil subprocess sys threading time uuid
|
||||
pathlib.Path urllib.request
|
||||
argparse configparser json os queue random re shlex shutil subprocess
|
||||
sys threading time uuid pathlib.Path urllib.request
|
||||
```
|
||||
|
||||
No third-party packages. No conda environment. No virtualenv. It runs on the
|
||||
@@ -72,6 +75,25 @@ reachable ComfyUI instance. It talks to ComfyUI over plain HTTP with
|
||||
> **`requirements.txt` in the project root does not apply here.** It belongs to
|
||||
> the older `lektor_ai.py`, which used the hosted Boson AI API and needed
|
||||
> `openai>=1.0.0`. `lektor_ai_comfy.py` requires nothing from it.
|
||||
> `normalizer/requirements.txt` is a different file and does apply — to the
|
||||
> normalizer's own venv, never to the system Python.
|
||||
|
||||
### `normalizer/` — optional third environment
|
||||
|
||||
```
|
||||
torch 2.11.0+cu128 transformers 5.15.0 accelerate 1.14.0
|
||||
num2words 0.5.x (fallback path only)
|
||||
```
|
||||
|
||||
Its own venv at `normalizer/venv` (~6.9 GB — the cu128 wheels bundle their
|
||||
own CUDA runtime, again) plus `normalizer/model` (1.1 GB).
|
||||
Separate from ComfyUI's environment because it hard-requires `transformers >= 5`
|
||||
— 4.x tokenizes the Qwen3 chat template differently and silently produces wrong
|
||||
output. `lektor_ai_comfy.py` never imports it; it starts it as a subprocess and
|
||||
talks JSON lines over a pipe, so the version conflict never has to be resolved.
|
||||
|
||||
Entirely optional: without it the pipeline uses a `num2words` fallback, and with
|
||||
`numbers = no` it is not started at all.
|
||||
|
||||
### ComfyUI — a large, isolated environment
|
||||
|
||||
@@ -130,3 +152,9 @@ API keys, no accounts, no per-use cost.
|
||||
The model is published under a research / non-commercial licence and may require
|
||||
accepting its terms on huggingface.co, then `hf auth login`, before it will
|
||||
download.
|
||||
|
||||
The number normalizer's model is **gated** in the same way and needs an
|
||||
`HF_TOKEN` for its one-time download; see
|
||||
[installation.md](installation.md#number-normalizer-optional-recommended). Runs
|
||||
set `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`, so it never reaches the
|
||||
network afterwards.
|
||||
|
||||
Reference in New Issue
Block a user