mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-27 19:48:38 -06:00
Brandon ThomasandClaude Opus 5.5
fc18a6962e
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned, SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto), pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD word tightening, MFCC speaker clustering. Both features off by default and for wasm. - project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9 migration and a v8 fixture. - edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs, word ranges, pause/filler detection, ripple delete of many ranges, caption layout. - engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift, renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel); Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the desktop and CLI apps). Without whisper, transcript.generate fails with a clear error. - ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words, Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with text.transcript.* automation ids; headless UI test. - docs/transcripts.md, crates/speech/README.md, architecture and project-files updates. whisper-tiny measures 12.2% WER over 797 words on local samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
22 lines
1.2 KiB
Markdown
22 lines
1.2 KiB
Markdown
# filmcraft-speech
|
|
|
|
Speech-to-text for FilmCraft's Text panel ▸ Transcript (L2). User-facing behaviour, commands and
|
|
model handling: [docs/transcripts.md](../../docs/transcripts.md).
|
|
|
|
- `Transcriber` trait (mono 16 kHz in, word-timed `filmcraft_project::Transcript` out) and
|
|
`FixedTranscriber` for tests and agents.
|
|
- `models`: Whisper model catalogue (pinned revisions, SHA-256, sizes, licences) and, with feature
|
|
`download`, the verified downloader (ureq + rustls + RustCrypto). Weights are never bundled.
|
|
- `whisper` (feature `whisper`): Whisper inference on candle (CPU, pure Rust).
|
|
- `mel`, `vad`, `diarize`: log-mel front end, word-bound tightening, MFCC-clustering speaker labels.
|
|
|
|
Both features are off by default and are never enabled for wasm (`cargo xtask wasm` checks the
|
|
crate without them).
|
|
|
|
## References
|
|
|
|
Implemented from the published description of the model: A. Radford et al., "Robust Speech
|
|
Recognition via Large-Scale Weak Supervision" (OpenAI, 2022), the model cards and configuration
|
|
files published with the weights (`config.json`, `generation_config.json`, `tokenizer.json`), and
|
|
the candle tensor library's public API. Weights: OpenAI Whisper, MIT licence.
|