Survive a ComfyUI crash, retry clips the model cuts short
Crash recovery ComfyUI's queue and history do not persist across a restart, so a prompt submitted before a crash exists nowhere afterwards. The old poll loop waited on that dead prompt for ever, and its error branch skipped the backoff, so it hammered the closed port four times a second printing "Poll error" on every attempt. run_workflow() replaces wait_for_completion(). It reports once when the backend goes away and once when it returns, polls every 3s in between, and on reconnect checks both /history and /queue: if the prompt is in neither, the queue did not survive and the clip is submitted again. When it cannot tell, it assumes the job is still alive — resubmitting a running job would waste a generation. It also starts ComfyUI itself, via the script named by [comfy] start_command, in a new terminal window. Guarded by a 90s cooldown and 3 attempts per run so a backend that dies during startup cannot spawn an endless stream of windows; the window pauses on exit so a failed start leaves its error on screen. Falls back to a background launch with a log file when there is no display. Set [comfy] autostart = no, or pass --no-comfy-autostart, to turn it off. Clips cut short mid-word About one clip in twelve comes back truncated: the model stops before the line is finished. Measured on raw ComfyUI output, 6 of 80 recent clips, and 15 of 472 in one episode. It is invisible in the text but obvious in the audio — a finished clip decays to about -65 dB, a cut one stops at about -16 dB. Truncated clips are now regenerated with a distant seed, up to [tts] truncation_retries extra attempts, and a run that still has bad clips ends by printing them as a ready-made --regen list. --check-clips reports the same for a job already on disk. Worth recording what this is NOT: the trailing-silence trimming added earlier was suspected, and cleared. It only cuts silence that runs to end of file, so a pause between words never triggers it — verified against six synthetic cases including a 1.2s mid-clip pause, and against real clips containing pauses. The truncation is present in ComfyUI's own output before this script touches it. settings.ini New: [tts] truncation_retries, [comfy] autostart and start_command, all documented in place. The editor renders yes/no keys as a dropdown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -29,6 +29,11 @@ temperature = 0.6
|
||||
# mid-word.
|
||||
max_tokens = 4096
|
||||
|
||||
# Roughly one clip in twelve comes back cut off mid-word. Those are detected and
|
||||
# asked for again with a different seed, up to this many extra attempts. 0 to
|
||||
# switch retrying off.
|
||||
truncation_retries = 2
|
||||
|
||||
# Trim this many ms from the start of every clip, to remove an inhale artifact.
|
||||
# Try 80-150 if you hear one. Too high clips the first consonant.
|
||||
strip_leading_ms = 0
|
||||
@@ -54,3 +59,13 @@ output_dir = output
|
||||
[comfy]
|
||||
# Address of the ComfyUI backend running Higgs Audio V3.
|
||||
comfy_url = http://127.0.0.1:8189
|
||||
|
||||
# Start ComfyUI automatically when it stops answering. A crash loses its queue,
|
||||
# so the clip in flight is resubmitted once the backend is back. At most 3
|
||||
# attempts per run, 90s apart, so a backend that dies on startup cannot spawn an
|
||||
# endless stream of terminal windows. yes/no.
|
||||
autostart = yes
|
||||
|
||||
# The script used to start it. Relative paths are resolved next to
|
||||
# lektor_ai_comfy.py.
|
||||
start_command = start_comfyui.sh
|
||||
|
||||
Reference in New Issue
Block a user