Media and proxy paths read from a project file, a project template or an
interchange file are opened only when they name a local file or a file on
the network share the document itself is on (craft-docpath); a relative
path is taken from the document's folder. Other paths count as missing:
the item renders the offline slate with the reason, the open says how many
were not opened, and Link Media lists them. This covers the open-time
scan, rendering and proxies, the growing-files look and interchange
import. Files imported or linked in this session are opened as given.
The Scratch Disks folders use the same check, and Captured Audio and Video
(voice-over takes, pasted images, the default proxy folder) and Audio
Previews follow the rule Auto Save and Video Previews already follow.
Problem: .dng files were not importable, and a DNG (a TIFF) would have
gone to the TIFF decoder, which reads at best the embedded preview. A
CinemaDNG shot (a folder of numbered DNG frames) could not be edited.
Fix: "dng" joins the importable still extensions. filmcraft-media
develops DNGs with filmcraft-dng into scene-linear Rec. 709 float
frames (transfer Linear, like OpenEXR), so highlights above 1.0 reach
the compositor. A single DNG is a still; a numbered folder imports
through the existing image-sequence path as one clip, whose frame rate
and start time code come from the first frame's CinemaDNG FrameRate and
TimeCodes tags (else Settings > Media > Indeterminate Media Timebase).
Lossy and JPEG XL DNGs are refused as unsupported by name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
APNG and animated GIF files opened as stills showing only their first
frame. They now open as movie clips (crates/media/src/animated.rs):
- every frame is decoded and composited onto the full canvas by the
image crate (APNG blend/dispose ops, GIF disposal), keeping alpha
- each frame lasts its own delay; GIF delays <= 10 ms play as 100 ms
and APNG delays < 10 ms as 10 ms, as browsers do
- consecutive identical frames are merged; the clip is one pass of the
animation and its timebase is the most common frame length
- hostile input: at most 10,000 frames / 1 GiB of decoded pictures; a
damaged frame ends the clip at the frames before it; files with one
frame (or none that decode) still open as stills, as before
Tests: media unit tests (timing, transparency, merging, single-frame
files stay stills, truncated/bit-flipped files never panic) and an
engine test that imports an animated GIF and reopens it from a saved
project.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QkQ2PDrNFjs9Vm8T44tdjA
Transport and program streams listed a single audio stream, so file.import
and New Sequence From Clip kept one clip. Probe every audio PID (capped),
decode each independently, and place linked clips per stream. MP4/MKV
already did this; MXF same-rate PCM is still interleaved as one stream.
- filmcraft-tts (feature kokoro): Kokoro-82M on candle, implemented from
the StyleTTS 2 / iSTFTNet / ALBERT papers, config.json and the
checkpoint's tensor shapes; official .pth read with candle's restricted
pickle reader; voice packs from their zip. Neural Voice: tts-text
phonemize -> Kokoro per chunk -> pauses -> pitch via audio-dsp.
- catalog: 9 US English voices (model card grade C+ or better) + CMUdict,
every file pinned to a revision and SHA-256 (336 MB).
- speech: generic download_files (same verified downloader).
- engine: tts.render (background synthesis into Session::tts_cache),
tts.downloadVoices (background job, cancellable), voices list with
package state; feature neural-voices (default on in the desktop app).
- ui: natural/basic voice groups, consent box (size, sources, licences),
download progress + Cancel, synthesis never blocks the window;
ui.set {tts:{...}} for agents.
Measured: Whisper-base WER on 50 original sentences 2.1-3.6 % for all 9
voices (built-in robotic voice 96.6 %); 2.2-2.3x real time on CPU.
Gates: fmt, clippy, layers, assets, wasm pass; tests pass except
shortcuts_tests::assign_reassigns_conflicts_and_undoes (fails on main)
and the frame pool test (flaky under load, passes alone).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Write what a narrator says, pick a voice, pitch and pace, and add it to
the timeline. Selecting a narration clip loads it back into the panel;
Save regenerates it in place as one undo step, keeping the clip's start
and length (longer speech kept whole in the file, shorter speech padded
with silence).
- filmcraft-tts (L2): Voice trait, voice catalogue, pause markers
([pause 1s]), and an original formant synthesizer (Basic Female/Male;
robotic placeholder until the neural voices land).
- project: Narration + Project::narrations (schema v13, no-op migration).
- engine: tts.voices/preview/create/edit/inspect; narrations never
overwrite clips (free targeted track, free track, else a new track).
- ui: Text to Speech panel beside Properties, preview playback,
click-to-edit, automation ids.
Gates: fmt, clippy -D warnings, layers, assets and wasm (rustup
toolchain) pass. Tests: all pass except three that fail identically on
main (61238bf) on this machine: shortcuts_tests::assign_reassigns_
conflicts_and_undoes, gpu fx_tests::gpu_effects_match_cpu_exactly, and
the frame pool test, which passes when run alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
whisper-tiny measures 12.2% WER over 797 words on local samples.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- filmcraft-media: ByteReader/ReaderOpener (open containers without reading the whole
file) and the pending flag for asynchronous (web Blob) reads.
- filmcraft-codecs: MP4/MOV and Matroska open from a ByteReader; registry of reader
openers (platform sources first).
- filmcraft-engine: Services::reader / export_in_memory; streamed imports; media pool
never caches a still-loading file as offline; Session::pump_jobs advances stepped
exports on hosts without threads.
- filmcraft-export: Exporter (one batch per step, re-renders on pending reads) used by
export(); OutputSink for in-memory output.
- filmcraft-ui-egui: FrameServer::pump (time-budgeted cooperative frame jobs on wasm,
retries jobs whose media is loading); screenshots as base64 on wasm; web-time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- media.createProxies: background job (jobs.list progress) transcoding with our own
encoders — ProRes 422 Proxy ¼/½, ProRes LT ½, H.264 ¼/½ (audio included) — into
<media dir>/Proxies or a chosen folder; attached when the job finishes
(Session::poll_persistence applies finished media jobs).
- media.attachProxies (duration ±1 frame and frame rate must match; size/aspect may
differ), media.detachProxies, media.reconnectFullRes, media.toggleProxies
(Preferences ▸ Media ▸ enableProxies; bumps the view revision without dirtying the
project), media.proxyPresets.
- Proxy frames are smaller; ProxySource reports the original size and the compositor
derives its pixel scale from the frame, so Motion and effects render the same
picture (44 dB PSNR vs full res with Motion + Gaussian Blur, 48 dB at ½).
Export always renders full resolution.
- ExportSettings::prores_profile (proxy / lt / standard / hq).
- project.ingestSettings + file.import: copy (fingerprint-verified), transcode,
create proxies, copy and create proxies.
- UI: Toggle Proxies button on both monitors (lit when on), Create Proxies dialog,
Project panel Proxy submenu and P badges.
- Measured (one core, 4K H.264 testsrc2): CPU composite 201 → 35 ms/frame at ¼ with
ProRes Proxy ¼, 277 → 131 ms at ½ with H.264 ¼; decode 9.2 → 0.7 ms with H.264 ¼.
- Tests: create+attach+toggle same composite modulo scale, attach checks, export
ignores proxies, ingest copy + proxies, ignored 4K benchmark, headless UI test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- project::essential: Dialogue/Music/SFX/Ambience settings (Loudness, Repair, Clarity, Creative,
Ducking, Pan, Clip Volume, Mute), our own EQ/reverb/type presets, and apply(): settings become
ordinary clip effects (EffectInstance::essential) in canonical order; only changed parameters
are written, so keyframes survive; auto-match gain → clip gain, Clip Volume → Volume offset,
Pan → Panner
- effect defs: DeEsser, DeReverb (params), Enhance Speech, Stereo Width; render mappings for them,
and DeNoise amount / DeHummer gain now reach the DSP; Mute; render::audio::clip_signal
- engine essentialSound.* (inspect, setType, clearType, set, applyPreset, savePreset,
deletePreset, autoMatch, generateDucking), undoable; per-type loudness targets and user presets
in preferences
- tests: auto-match within 0.001 LU of target (−23 / −16, with and without a chain), ducking
keyframes within 20 ms on a synthetic dialogue+music project and −15.00 dB in the mix, repair
metrics (hum −39 dB, rumble −22 dB, noise floor −13 dB, sibilance −19 dB, reverb tail −10 dB),
render chunking bit-identical and WAV export = render; full Dialogue chain 22× realtime (release)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- project: ItemKind::Graphic (transparent canvas source, hidden from the Project panel,
unlimited duration); graphic layers are hidden effect instances graphic_text /
graphic_shape on the clip (serde for free, keyframes on every animatable property,
Source Text holds), with appearance (fill, two strokes outer/centre/inner, background
box with radius, drop shadow) and transform (position, anchor, scale, rotation,
opacity); eval_layer gives a plain LayerSpec. No schema migration: all additive.
- render: graphic_clip rasterises each layer straight to the output transform (crisp at
any Motion scale/rotation), caches layer rasters, composes a tight image for the GPU
plan; standard effects on a graphic run at source resolution first.
- engine: graphics.newText (Cmd+T) / newShape (new clip above the footage at the playhead
or a layer in an existing clip), setText (typing coalesces into one undo step), set,
selectLayer, deleteLayer, arrangeLayer, align (frame / selection), distribute,
graphics.list, fonts.list. EditorState.graphic_layers holds the layer selection.
- goldens: graphic_title, graphic_shapes (CPU and GPU parity).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New crate filmcraft-format (L3): files are an envelope {format, schema_version, generator,
project}; a chain of single-step JSON migrations (v1 bare Project → v2 envelope) upgrades old
files on load; newer-than-supported files are refused with a clear "update FilmCraft" error;
damaged/foreign files report as such.
- atomic_write: temp file in the same dir + fsync + rename + dir fsync; used for every project
write (FsServices), so a crash mid-save never corrupts the project.
- Engine: file.save/saveAs/open go through the format crate; new file.saveCopy (Save a Copy…,
Cmd+Alt+S) and file.revert; the first save over an upgraded file keeps
"<name> (schema vN backup).fcproj". Project.version dropped (the envelope carries it).
- Hand-made v1 fixtures (minimal + edit without later serde-default fields) in
crates/format/tests/fixtures; 2,000-clip encode/decode timing test.
- docs/project-files.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
