Add subtitle dictionaries, Polish number normalizer, ComfyUI lifecycle control
Text processing before TTS:
- dictionary.txt rewrites words and phrases (km/h, np., units). Rules compile
into one alternation applied in a single pass, so replacements never chain
and longest-literal wins. Case-sensitive, whole-word.
- A per-queue second dictionary, written by the GUI into the first video's job
folder and merged with the shared one (--dictionary-extra).
- ALL-CAPS blocks become Title Case rather than sentence case: lowercasing
destroyed proper nouns inside captions ("NASA - MARGO MADISON") and invited
the normalizer to "correct" the name. Paired clips in output/clips/all-caps
showed the TTS reads both casings the same.
normalizer/ — digits to spoken Polish, in its own venv because it needs
transformers>=5, which must not touch ComfyUI's pinned versions:
- Two backends. Plain integers go to num2words for the value and a general
Polish LLM for the case; times, dates, decimals and identifiers go to a 0.6B
fine-tune. Measured on 40 real subtitle lines, each is clearly better than
the other on its half. 74% of numbered blocks take the first leg.
- Guards on every answer: runaway length, repetition loops, invalid bytes,
leftover digits, a lost minus sign, a changed number, a lost word. A failure
falls back to num2words, which is nominative but never wrong about the value.
- ComfyUI is asked to unload before normalizing; a normalizer that does not fit
in VRAM is spilled to RAM silently at 5x the cost.
Backend lifecycle:
- ComfyUI is found by port through /proc, so a backend started by hand is
managed too. SIGTERM escalates to SIGKILL, and a suspended process is killed
outright since it never runs its handlers.
- A clip that kills the backend is retried three times, each with a different
seed (the aborts are seed-dependent, not text-dependent), then skipped and
reported so an overnight queue keeps moving.
- start_comfyui.sh execs python, so the script's PID is ComfyUI's.
- GUI gains ComfyUI and Queue dictionary buttons; quitting stops the backend.
--resume-from now reuses the job folder's tts_input.srt instead of reparsing
the subtitles, so block N still pairs with clip_NNNN.flac.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+28
-1
@@ -184,6 +184,27 @@ conda install -n comfy-tts -c conda-forge ffmpeg
|
||||
Note that this puts ffmpeg inside the conda env, so the script must then be run
|
||||
with that env active.
|
||||
|
||||
## Number normalizer (optional, recommended)
|
||||
|
||||
Digits reach the TTS as words rather than figures, in the correct Polish case.
|
||||
It has its own venv on purpose — the model needs `transformers >= 5`, which must
|
||||
not be forced on the ComfyUI environment installed above.
|
||||
|
||||
```bash
|
||||
cd ~/Programy/lektor_ai
|
||||
normalizer/setup.sh # venv + torch (cu128) + transformers
|
||||
HF_TOKEN=hf_... normalizer/setup.sh --model # the weights, ~1.1 GB
|
||||
```
|
||||
|
||||
The weights are gated: accept the terms at
|
||||
<https://huggingface.co/Folx/qwen3-0.6b-pl-text-normalization> while signed in,
|
||||
then take a token from <https://huggingface.co/settings/tokens>. This is the only
|
||||
step that touches the network; runs are offline afterwards.
|
||||
|
||||
Skipping this is fine — the pipeline falls back to `num2words` (nominative case
|
||||
only) and says so. Turn the whole stage off with `numbers = no` under
|
||||
`[normalize]` in `settings.ini`. See [normalizer/README.md](../normalizer/README.md).
|
||||
|
||||
## Verifying
|
||||
|
||||
```bash
|
||||
@@ -198,7 +219,11 @@ python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
|
||||
# 3. Model present
|
||||
ls -la ~/comfy-tts-app/ComfyUI/models/TTS/higgs_audio_v3/higgs-audio-v3-tts-4b/model.safetensors
|
||||
|
||||
# 4. End-to-end on 60 seconds of a real file
|
||||
# 4. Number normalizer, if installed
|
||||
normalizer/venv/bin/python normalizer/normalizer.py --status
|
||||
# -> "mode": "model" ("fallback" means the weights are missing)
|
||||
|
||||
# 5. End-to-end on 60 seconds of a real file
|
||||
cd ~/Programy/lektor_ai
|
||||
python3 lektor_ai_comfy.py "input/film.mkv" --embedded-subs --end 01:00
|
||||
```
|
||||
@@ -213,5 +238,7 @@ Nothing was installed system-wide, so removal is just deleting directories:
|
||||
```bash
|
||||
conda env remove -n comfy-tts
|
||||
rm -rf ~/comfy-tts-app
|
||||
rm -rf ~/Programy/lektor_ai/normalizer/venv # number normalizer env
|
||||
rm -rf ~/Programy/lektor_ai/normalizer/model # and its weights
|
||||
rm -rf ~/miniforge3 # only if nothing else uses it
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user