The pause pass in vad::tighten_words (#3347) cuts a word's span at a
pause of 0.2 s or more. A start that the alignment places late inside
continuous speech has no pause to cut at, so the word before it keeps
the beginning of the next one; the audio level gives no boundary to
move it to. Words placed a whole sentence away are not moved back
either. Neither limit was written down; both go under Limits.
On a 60 s interview probe (whisper-small) one pair has this shape:
"that?" spans 0.6 s and "Talk" 60 ms of continuous speech, with no
dip in the level longer than 20 ms between them.
AI assistance: Mystic Intelligence for MysticArt Pictures.
Whisper's alignment makes words contiguous and places a start that
follows a pause late, so the previous word's span held the pause and
the next word's beginning; vad::tighten_words only trimmed silence at
the edges, which were voiced. A run of 200 ms or more of silence with
voice on both sides inside a word's span now ends the word, and the
next word, when contiguous, starts after it. When the voice before the
pause is too short to be the word (under half of 50 ms per letter) the
word starts after the pause, and a contiguous previous word takes the
scrap only when no pause separates its voice from it. Several pauses in
one span cascade through the following words.
On a 60 s probe (whisper-small) the 4 pauses that sat inside words are
all between words afterwards and no word keeps silence at an edge; on
a 48-minute interview the pauses inside a word's span go from 336 of
530 to 0, words longer than 1.5 s from 103 to 28, and the word after a
pause starts within 50 ms of the voiced onset 488 times instead of 192.
AI assistance: Mystic Intelligence for MysticArt Pictures.
Co-authored-by: Brandon Thomas <echelon@gmail.com>
- transcript.generate is enabled only with a recogniser: one installed in
Session::transcriber, or the build's own (feature `whisper`). Otherwise
describe, command_list enabled_only and the menu report it disabled with
"speech-to-text is not available in this build" as the reason (#97).
Transcribe Sequence, which runs it, follows.
- transcript.downloadModel is enabled only with `speech-download` (#98).
- file.import with Automatically transcribe clips on no longer skips
transcription silently when it can't run: the disabled command's reason
lands in the import's errors as "transcription: ..." (#89).
- docs/transcripts.md describes the disabled state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
whisper-tiny measures 12.2% WER over 797 words on local samples.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aiongg