diff --git a/crates/ac3/README.md b/crates/ac3/README.md new file mode 100644 index 000000000..53d2f3247 --- /dev/null +++ b/crates/ac3/README.md @@ -0,0 +1,70 @@ +# filmcraft-ac3 + +Clean-room AC-3 (Dolby Digital) audio decoder in pure Rust. Layer L0: depends only on +`thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg, +liba52) was consulted; FFmpeg is used only as an external fixture generator and test oracle. + +## Specification + +ATSC A/52:2012 "Digital Audio Compression (AC-3, E-AC-3) Standard" (17 December 2012), the +publicly available standard: §5 bit stream syntax and semantics, §6 decoding overview, §7.1 +exponents, §7.2 parametric bit allocation (Tables 7.6-7.16 extracted mechanically from the +standard's text), §7.3 mantissas and dither, §7.4 coupling, §7.5 rematrixing, §7.7.1 dynamic +range, §7.9 transforms and block switching. The window is computed as the Kaiser-Bessel derived +window (α = 5) and checked against Table 7.33 to 6·10⁻⁶. + +## API + +```rust +let mut dec = filmcraft_ac3::Decoder::new(); +let h = filmcraft_ac3::parse_header(frame)?; // rate, channels, frame size +let out = dec.decode(frame)?; // 1536 samples per channel, WAV channel order +dec.reset(); // before decoding from another position +``` + +## Decoder + +- Every audio coding mode (1+1, 1/0, 2/0, 3/0, 2/1, 3/1, 2/2, 3/2) with or without LFE; 32 / + 44.1 / 48 kHz and every frame size; bsid ≤ 8 (9 and 10, the half / quarter-rate variants, are + accepted). E-AC-3 (bsid 11-16) is refused with `Error::Unsupported`. +- D15 / D25 / D45 exponents with reuse across blocks; the parametric bit allocation in the + standard's fixed-point steps, delta bit allocation, the all-zero SNR offset special case; + grouped 3-, 5- and 11-level mantissas (groups shared across exponent sets), 7- and 15-level + and asymmetric mantissas up to 16 bits. +- Channel coupling (coupling bands, master coordinates, phase flags in 2/0), rematrixing (all + four banding cases), dynamic range control (`dynrng`, `dynrng2`; applied at full scale), dither + for zero-bit mantissas (uniform ±0.707, §7.3.4). +- 512-sample IMDCT and the pair of 256-sample IMDCTs for block-switched blocks, with radix-2 + inverse FFTs, KBD window and overlap-add. +- Output: f32 at full scale ±1; channels reordered from the coded order (L C R Ls Rs LFE) to the + WAV / SMPTE order (L R C LFE Ls Rs). No downmix (the mixer handles layouts). +- Dither is reseeded per syncframe from the frame's CRC words, so a frame decodes to the same + samples however it was reached (seeking, render caches). +- Errors (bad exponents, reserved codes, blocks running past the frame) return `Err`; garbage + never panics. + +## Accuracy (vs FFmpeg as an external oracle) + +AC-3 output is not bit-exact across decoders: zero-bit mantissas are filled with +decoder-specific dither. `tests/oracle.rs` decodes FFmpeg-encoded streams and compares with +FFmpeg's decode (worst channel SNR): + +| Stream | SNR vs FFmpeg | SNR between two of our dither seeds | +|---|---|---| +| 2/0 48 kHz 192 kb/s (tones, chirp) | 66.1 dB | 76.0 dB | +| 2/0 44.1 kHz 96 kb/s with coupling and rematrixing | 84.5 dB | 81.7 dB | +| 1/0 32 kHz 96 kb/s | 94.1 dB | 95.7 dB | +| 3/2 + LFE 48 kHz 448 kb/s | 81.4 dB | 90.2 dB | +| 2/0 pink noise 128 kb/s (many zero-bit mantissas) | 27.7 dB | 27.9 dB | + +The noise row shows the difference is dither: two of our own decodes with different dither +sequences differ as much as we differ from FFmpeg. In MPEG-2 TS (AVCHD-style `.mts`) and VOB the +maximum sample difference to FFmpeg is 2.3·10⁻³. + +## Gaps + +- FFmpeg's AC-3 encoder never block-switches, so the 256-sample transforms are implemented from + §7.9.4.2 but not checked against an oracle. +- No downmixing, no heavy compression (`compr`) or dialogue normalisation (both are metadata + for playback systems), no CRC check (corrupt frames are caught by consistency checks). +- E-AC-3 is not decoded. diff --git a/crates/codecs/Cargo.toml b/crates/codecs/Cargo.toml index 90e1c9ca8..c713bb663 100644 --- a/crates/codecs/Cargo.toml +++ b/crates/codecs/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "filmcraft-codecs" -description = "Container + codec hub: MP4/MOV, Matroska, MXF and Ogg sources, image sequences, GOP-aware seeking, decoder registry, audio decoding" +description = "Container + codec hub: MP4/MOV, Matroska, MXF, Ogg and MPEG TS/PS sources, image sequences, GOP-aware seeking, decoder registry, audio decoding" version.workspace = true edition.workspace = true license.workspace = true diff --git a/crates/codecs/src/lib.rs b/crates/codecs/src/lib.rs index a51b0b9f3..0666bd326 100644 --- a/crates/codecs/src/lib.rs +++ b/crates/codecs/src/lib.rs @@ -7,9 +7,12 @@ //! `filmcraft-isobmff`: GOP-aware random access (seek to the preceding sync sample and decode //! forward, caching every decoded frame of the GOP), sequential fast path for playback, and //! packet-cached audio decoding. -//! - [`MkvSource`], [`MxfSource`] (OP1a / OP-Atom: AVC, VC-3, ProRes, PCM; MPEG-2 reported as -//! unsupported) and [`OggSource`] (Ogg Opus with granule-position seeking, Ogg Vorbis): the same +//! - [`MkvSource`], [`MxfSource`] (OP1a / OP-Atom: AVC, MPEG-2 incl. D-10 / XDCAM, VC-3, ProRes, +//! PCM) and [`OggSource`] (Ogg Opus with granule-position seeking, Ogg Vorbis): the same //! GOP-aware video access and packet-cached audio. +//! - [`MpegSource`]: MPEG-2 transport streams (`.ts`, `.m2ts`, `.mts`), program streams (`.mpg`, +//! `.vob`, `.mod`) and MPEG-1/2 video elementary streams: MPEG-1/2, H.264 and HEVC video; MPEG +//! audio, AAC (ADTS / LATM), AC-3 and LPCM. //! - [`AudioFileSource`]: standalone compressed audio files (MP3, FLAC, AIFF, …). //! - [`openers`]: the openers to register with the engine's media pool. diff --git a/crates/mpeg2v/README.md b/crates/mpeg2v/README.md new file mode 100644 index 000000000..0562ffed3 --- /dev/null +++ b/crates/mpeg2v/README.md @@ -0,0 +1,130 @@ +# filmcraft-mpeg2v + +Clean-room MPEG-2 video and MPEG-1 video decoder in pure Rust. Layer L0: depends only on +`thiserror` and (optional, feature `threads`, default on) `rayon`. Builds for +`wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg, libmpeg2, libavcodec) was +consulted; FFmpeg is used only as an external fixture generator and test oracle. + +## Specifications + +| Document | Edition | Used for | +|---|---|---| +| ITU-T H.262 \| ISO/IEC 13818-2 | 02/2000 (second edition, with Amd. 1 and Cor. 1/2 as incorporated in the 2000 text) | sequence / GOP / picture headers and extensions, slices, macroblocks, VLC tables (Annex B), inverse quantisation and mismatch control (§7.4), IDCT (Annex A), motion vectors and prediction (§7.6) | +| ISO/IEC 11172-2 | 1993 | MPEG-1 video: slices spanning rows, MPEG-1 escape codes, oddification, full-pel vectors, D pictures | +| IEEE Std 1180 | 1990 | IDCT accuracy test procedure (run as a unit test) | +| ISO/IEC TR 13818-5 (MPEG Software Simulation Group decoder behaviour) | 1997 | two decoder behaviours the normative text leaves open (see "Interpretations") | + +## API + +```rust +let mut dec = filmcraft_mpeg2v::Decoder::new(); +for (i, au) in filmcraft_mpeg2v::access_units(&es).into_iter().enumerate() { + for pic in dec.decode(&es[au], i as i64)? { // display order + // pic.y / pic.cb / pic.cr: tight 8-bit planes; pic.chroma: 4:2:0 / 4:2:2 + // pic.field_order(), pic.picture_type, pic.pts, pic.info (SequenceInfo) + } +} +let rest = dec.flush(); +let info = filmcraft_mpeg2v::probe(&es); // sequence header + extensions +let au = filmcraft_mpeg2v::scan_access_unit(&es[..]); // picture types without decoding +``` + +`decode` takes any chunk holding whole pictures (an access unit from a TS/PS/MXF/MOV sample). +Pictures come out in display order (anchors are held until the next anchor) and carry the `pts` +of the chunk their first field came in. `reset` forgets pictures but keeps the sequence +parameters, so a random-access point without its own sequence header still decodes. + +## Decoder + +- Main Profile at Low/Main/High-1440/High Level, 4:2:2 Profile (IMX / D-10, XDCAM HD422), + Simple Profile, MPEG-1 (including D pictures). 4:4:4 is refused (no profile in use for it). +- Frame pictures with frame and field motion compensation, dual prime, frame and field DCT; + field pictures with field, 16x8 and dual-prime motion compensation, the second field of a + P frame predicted from the first; open and closed GOPs; B pictures without a forward + reference after a broken link are dropped (closed GOP: decoded from the backward reference). +- Both intra VLC tables, zigzag and alternate scan, linear and non-linear quantiser scale, + quantiser matrices from the sequence header and the quant matrix extension (chroma matrices + for 4:2:2), intra DC precision 8-11 bits, concealment motion vectors, saturation and + mismatch control. +- IDCT: the separable 2-D IDCT in `f32` with even/odd symmetry, the DC term added exactly, + row skipping and a DC-only path; rounded and saturated to [-256, 255]. `ieee_1180_accuracy` + runs the IEEE 1180 procedure (10 000 blocks × 3 ranges × 2 signs): peak error ≤ 1 and all mean + / mean-square limits met. +- Motion compensation clamps reference coordinates to the picture (only corrupt streams point + outside it); a missing reference predicts from mid-gray. +- Slices of different macroblock rows decode in parallel on rayon (each row into its own chunk + of the frame); MPEG-1 slices, which may span rows, decode in order. +- Corrupt slices are concealed (the rest of the picture still decodes) and counted + (`Decoder::errors`, `last_error`); nothing panics on truncated or mutated input. + +### Interpretations + +Two behaviours follow the MPEG Software Simulation Group reference decoder (and FFmpeg), which +the synthetic-stream tests confirmed against FFmpeg: + +- Skipped macroblocks in field pictures predict from the field of the same parity, in B + pictures as well as P pictures. +- Motion vector reconstruction (§7.6.3.1) brings the vector back into range only on the side the + delta moves it, and keeps the prediction for a zero delta. This differs from the literal + formula only when the prediction is out of range: a field vector's doubled vertical + component predicting a following frame vector. + +## Accuracy (vs FFmpeg as an external oracle) + +MPEG-2's IDCT is not bit-exact by definition (any IEEE 1180-accurate IDCT conforms, and the +encoder's reconstruction drifts from every decoder's). FFmpeg uses an integer IDCT, so frames +are compared within a tolerance: **per frame max |Δ| ≤ 4 and PSNR ≥ 58 dB, at most 3 % of +samples differing**. Frame count, display order, picture types, interlacing and field order are +exact (ffprobe). + +`tests/oracle.rs` (FFmpeg-encoded elementary streams): + +| Fixture | Content | Max Δ | Differing | PSNR | +|---|---|---|---|---| +| m2v_cif_ibbp | 352×288 progressive IBBP, Main@Main | 2 | 0.60 % | 70.3 dB | +| m2v_576i_tff | 720×576 interlaced TFF, field MC/DCT, alternate scan | 2 | 0.33 % | 73.0 dB | +| m2v_480i_bff_vlc1 | 720×480 BFF, intra VLC table 1, non-linear q, 10-bit DC | 2 | 0.38 % | 72.3 dB | +| m2v_imx_422_intra | 720×608 4:2:2 intra (IMX / D-10 style) | 1 | 0.23 % | 74.6 dB | +| m2v_hd422_longgop | 1920×1080 4:2:2 interlaced long GOP (4:2:2@High) | 2 | 0.20 % | 75.1 dB | +| m2v_1080i | 1920×1080 4:2:0 interlaced, 25 Mb/s (Main@High) | 3 | 0.25 % | 74.2 dB | +| m2v_escapes_qmatrix | q = 1 (escape codes), custom intra/inter matrices | 2 | 1.80 % | 65.6 dB | +| m2v_ipp | no B pictures | 2 | 1.67 % | 65.9 dB | +| m1v_sif | MPEG-1 352×240 IBBP | 2 | 0.59 % | 70.4 dB | +| m1v_q1 | MPEG-1 q = 1 (8/16-bit escapes) | 2 | 1.93 % | 65.2 dB | + +FFmpeg's encoder never writes field pictures, 16x8 or dual-prime prediction, so +`src/synth_tests.rs` writes random but valid streams with them (random macroblock types, +vectors, field selects, quantiser scales and coefficients; skipped macroblocks; concealment +vectors; 4:2:0 and 4:2:2; TFF and BFF) and FFmpeg decodes the same streams with `-xerror`: + +| Stream | Max Δ | PSNR | +|---|---|---| +| field pictures I/P/B + 16x8, TFF 4:2:0 / BFF 4:2:2 | 2 | 65.7–66.0 dB | +| field pictures + dual prime, TFF 4:2:0 / BFF 4:2:2 | 2 | 66.5–67.5 dB | +| interlaced frame pictures + field MC + dual prime, TFF / BFF | 2 | 64.8–65.7 dB | +| frame pictures with B, 4:2:2, alternate scan, non-linear q | 1 | 66.8 dB | +| seed sweep: 8 seeds × 4 configurations (160 streams checked during development) | ≤ 2 | ≥ 64.7 dB | + +Residuals are kept realistic (dequantised coefficients within ±300): with extreme residuals +FFmpeg's integer IDCT saturates differently from the exact one. + +## Performance + +`cargo test --release -p filmcraft-mpeg2v --test perf -- --ignored --nocapture` decodes the +1080i fixture (1920×1080 4:2:0, 25 Mb/s, IBBP, field MC/DCT; FFmpeg-encoded) on an M4 Pro +(14 cores) that was heavily loaded by other jobs (load average ≈ 190), best of five runs: +**271 fps on one thread, 274 fps with slice rows on rayon** (runs under heavier momentary load +fell to 77–150 fps). These are lower bounds. + +## Tests + +- `src/vlc.rs`: every table builds without prefix clashes; Kraft sums equal 1 minus exactly the + codes the standard reserves; every coded_block_pattern and motion_code value appears once; + motion codes are the address-increment codes plus a sign; 111 run/level pairs per DCT table. +- `src/idct.rs`: IEEE 1180; DC-only path equals the full transform for every DC value. +- `src/headers.rs`, `src/mc.rs`, `src/bits.rs`: scans, start codes, frame rates, half-sample + rounding, field addressing, edge clamping. +- `src/tests.rs`: a hand-built intra picture decodes to its DC; truncation at every byte and + 3000 random mutations never panic. +- `src/synth_tests.rs`, `tests/oracle.rs`: see above (single-threaded decode identical to the + threaded one). diff --git a/crates/mpegts/README.md b/crates/mpegts/README.md new file mode 100644 index 000000000..15bb9101e --- /dev/null +++ b/crates/mpegts/README.md @@ -0,0 +1,80 @@ +# filmcraft-mpegts + +Clean-room MPEG-2 Systems demuxer: transport streams and program streams. Layer L0: depends only +on `thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg, +libavformat) was consulted; FFmpeg is used only as an external fixture generator and test +oracle. + +## Specifications + +| Document | Edition | Used for | +|---|---|---| +| ITU-T H.222.0 \| ISO/IEC 13818-1 | 2000 (second edition) | TS packets, adaptation fields, PCR, PSI (PAT / PMT, CRC-32 of Annex A), PES headers, PTS/DTS, program streams (pack, system header, program stream map), stream types | +| ISO/IEC 11172-1 | 1993 | MPEG-1 system streams (pack header, PES header stuffing / STD buffer / timestamps) | +| Blu-ray / AVCHD BDAV MPEG-2 TS layout (192-byte source packets with a 4-byte TP_extra_header; registration `HDMV`; stream types 0x80-0x92) | public descriptions | `.m2ts` / `.mts` | +| DVD-Video private stream 1 sub-stream layout (AC-3 / DTS 0x80-0x8F with a 3-byte header, LPCM 0xA0-0xAF with a 6-byte header, sub-pictures 0x20-0x3F) | public descriptions | `.vob` | +| ETSI EN 300 468 (descriptors 0x56, 0x59, 0x6A, 0x7A, 0x7B), ATSC A/52 Annex A (AC-3 in TS) | — | identifying private PES streams | +| ISO/IEC 11172-3, 13818-3 (frame headers), ISO/IEC 13818-7 (ADTS), 14496-3 (LOAS), ATSC A/52 (syncinfo, E-AC-3 frmsiz) | — | audio frame lengths for splitting | + +## What it does + +```rust +let file = filmcraft_mpegts::open(&bytes)?; // any ByteSource (file, memory) +for (i, s) in file.streams.iter().enumerate() { + println!("{:?} {} {} units", s.id, s.codec.name(), s.units.len()); +} +let v = file.find(Kind::Video, |_| true).unwrap(); +let au: Vec = file.read_unit(&bytes, v, 0)?; // the first access unit +``` + +- **Transport streams**: 188-byte packets, 192-byte BDAV packets (sync byte after a 4-byte + arrival timestamp) and 204-byte packets; the packet size is sniffed from five consecutive sync + bytes. Lost sync resynchronises on the next aligned run. PSI sections are reassembled across + packets and checked with the MPEG CRC-32; the first program of the PAT is demuxed. Continuity + counter errors, CRC errors and resyncs are reported as warnings. +- **Program streams**: MPEG-2 and MPEG-1 packs, system headers, the program stream map (stream + types for elementary stream ids), padding, private stream 2 (DVD navigation) skipped, private + stream 1 split by sub-stream id. Garbage resynchronises on the next pack or PES start code. +- **Streams**: MPEG-1/2 video, H.264, HEVC, MPEG audio, AAC (ADTS, LATM/LOAS), AC-3, E-AC-3, + DTS, TrueHD, Blu-ray LPCM, DVD LPCM (format from the sub-stream header), PGS / DVB / teletext + subtitles; ISO 639 language from the PMT. +- **Access units**, found while the file is scanned (one pass, streaming; only a few bytes of + each stream are buffered): + - MPEG-1/2 video: a coded frame — a frame picture, or two field pictures (paired by + picture_structure) — with the sequence / GOP headers before it; picture type, temporal + reference, field structure, top_field_first, repeat_first_field, progressive_frame, closed + GOP / broken link. A PTS belongs to the unit whose picture start code is in that PES. + - H.264 / HEVC: PES packets with a timestamp (a PES without one continues the previous unit); + random access from IDR / IRAP NAL types; disposable when every slice is a non-reference. + - Audio: one frame per unit from the frame headers (sync re-hunted after garbage; a header + found while hunting is confirmed by the next one); a PTS belongs to the first frame starting + in the PES. + - LPCM and everything else: one PES packet per unit. +- **Timestamps**: 33-bit PTS/DTS unwrapped across the wrap (and back, for B pictures shown + before a wrap that came earlier); PCR / SCR range. +- **Reading**: a unit records only the position of its first packet and its offset in that + packet's payload; `read_unit` re-reads from there (the index costs a few dozen bytes per + unit, not per packet). + +## Tests + +- `src/tests.rs`: a small TS muxer (PAT, PMT with CRC, PES split into 188- or 192-byte packets + with adaptation-field stuffing) and PS writer (packs ≤ 2000 bytes, so frames span PES packets; + AC-3 in private stream 1): every access unit reads back byte-identical with its PTS, key and + disposable flags, field pairs as one unit, audio frames with the first-in-PES timestamp; + 33-bit unwrap; truncation every 37 bytes and 400 random mutations never panic. +- `src/es.rs`, `src/pes.rs`: frame sizes (MPEG audio layers I-III, MPEG-2 LSF, AC-3 at all + rates), MPEG-2 and MPEG-1 PES headers. +- `crates/codecs/tests/mpeg_oracle.rs`: FFmpeg-written TS / M2TS / VOB / MPEG-1 system files — + frame counts and every frame's PTS equal ffprobe's, decoded media against FFmpeg (see + `filmcraft-codecs`). + +## Gaps + +- One program per TS (the first in the PAT); no program switching, no section-based + metadata (SDT / EIT), no scrambled streams. +- H.264 / HEVC access units follow PES packets: streams that put several pictures in one PES + packet (rare in practice; not seen in broadcast, AVCHD or FFmpeg output) would be indexed as + one unit. +- DVD navigation (IFO / NAV packets), multiple VOB concatenation and angle interleaving are not + handled; each file opens on its own. diff --git a/crates/mxf/README.md b/crates/mxf/README.md index 8c6ab6a47..7d833b318 100644 --- a/crates/mxf/README.md +++ b/crates/mxf/README.md @@ -15,7 +15,7 @@ Implemented from the SMPTE documents (editions used): | SMPTE ST 378 | 2004 | OP1a | | SMPTE ST 390 | 2011 | OP-Atom | | SMPTE ST 379-1 / 379-2 | 2009 / 2010 | generic container: content packages, element keys, frame / clip wrapping | -| SMPTE ST 381-1 | 2005 | MPEG video mapping (MPEG-2 identified, MPEG video descriptor) | +| SMPTE ST 381-1 | 2005 | MPEG video mapping (MPEG-2 identified, MPEG video descriptor; decoded by `filmcraft-mpeg2v` in `filmcraft-codecs`) | | SMPTE ST 381-3 | 2013 | AVC byte-stream mapping and AVC sub-descriptor | | SMPTE ST 382 | 2007 | AES3 and Broadcast Wave audio mapping (wave / AES3 descriptors) | | SMPTE ST 331 | 2011 | element data of 8-channel AES3 sound (D-10) | @@ -60,7 +60,7 @@ let pcm = mxf.read_pcm(&file, a, 48_000, 1920)?; // planar f32, sample-exa pictures but has no temporal offsets (FFmpeg-written AVC), `needs_reorder` is set: the caller orders by the bitstream (`filmcraft-codecs` uses the AVC picture order counts). - **Codecs.** From the picture essence coding label, then the essence container label, then the - essence bytes: AVC / AVC-Intra, VC-3, ProRes (profile from the label), MPEG-2 (identified only), + essence bytes: AVC / AVC-Intra, VC-3, ProRes (profile from the label), MPEG-2 (decoded in `filmcraft-codecs`), MPEG-4 visual, JPEG 2000, DV, uncompressed. - **Sound.** PCM chunks (frame- or clip-wrapped) with sample counts; `read_pcm` decodes little-endian 8/16/24/32-bit PCM (wave / AES3 descriptors) and ST 331 AES3 elements (D-10). @@ -79,6 +79,6 @@ encrypted essence (ST 429-6), and descriptive metadata (parsed sets are ignored) 600 random mutations never panic and only list fully-present samples. - `crates/codecs/tests/mxf_oracle.rs` (FFmpeg-written fixtures, FFmpeg as the decode oracle): H.264 long-GOP with B pictures (bit-exact every frame + 25 random seeks), H.264 29.97 DF - timecode, DNxHR LB (±2), ProRes 422 (±1), MPEG-2 (reported unsupported, audio exact), + timecode, DNxHR LB (±2), ProRes 422 (±1), MPEG-2 long GOP, XDCAM HD422 and D-10 (±4), OP-Atom VC-3 (clip-wrapped) and PCM, D-10 AES3 audio; PCM sample-exact in every file; truncated and corrupted files. `cargo xtask fixtures codecs` pre-generates them. diff --git a/docs/architecture.md b/docs/architecture.md index b41865ace..4ec5ae4d8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -27,7 +27,7 @@ Design principles: L2 edit · codecs · interchange · captions · speech L1 frame · media · project · audio-dsp · text L0 foundation: time · geom · color · bitstream · testkit (dev-dependency only) - codecs/containers: isobmff · matroska · mxf · ogg · h264 · h264enc · hevc · vp9 · av1 · prores · dnx · aac · opus + codecs/containers: isobmff · matroska · mxf · mpegts · ogg · h264 · h264enc · hevc · vp9 · av1 · mpeg2v · prores · dnx · aac · ac3 · opus ``` Crates are named `filmcraft-` (`crates/time` is `filmcraft-time`). The apps are `filmcraft` @@ -42,13 +42,16 @@ and `filmcraft-cli`. | `isobmff` | L0 | MP4/MOV demux and mux | | `matroska` | L0 | MKV/WebM demux | | `mxf` | L0 | MXF demux (SMPTE ST 377-1): OP1a / OP-Atom, index tables, AVC / VC-3 / ProRes / MPEG-2 identification, PCM / AES3 sound, timecode ([README](../crates/mxf/README.md)) | +| `mpegts` | L0 | MPEG-2 Systems demux (H.222.0): transport streams (188 / 192-byte BDAV / 204), program streams (MPEG-2 / MPEG-1, VOB, MOD), PSI, PES, PTS / PCR, access-unit index built on open ([README](../crates/mpegts/README.md)) | | `ogg` | L0 | Ogg demux (RFC 3533), Ogg Opus timing (RFC 7845 granules, pre-skip, end trimming), Vorbis headers | | `h264`, `h264enc` | L0 | H.264 decoder; H.264 encoder | | `hevc` | L0 | H.265 Main/Main 10 decoder | | `prores` | L0 | ProRes decoder and encoder | | `dnx` | L0 | DNxHD / DNxHR (SMPTE ST 2019-1 VC-3) decoder and DNxHR encoder | | `av1` | L0 | AV1 decoder (Main profile; bit-exact with libdav1d; see its README for the stage table) | +| `mpeg2v` | L0 | MPEG-2 video (Main / 4:2:2 profile; frame and field pictures, dual prime) and MPEG-1 video decoder ([README](../crates/mpeg2v/README.md)) | | `aac` | L0 | AAC-LC decoder and encoder | +| `ac3` | L0 | AC-3 (ATSC A/52) decoder ([README](../crates/ac3/README.md)) | | `testkit` | L0 | test-only helpers, used only as a dev-dependency: ffmpeg/ffprobe discovery, fixture dirs, golden images ([testing.md](testing.md)) | | `frame` | L1 | `VideoFrame` (planar YUV / RGBA8 / linear RGBA f32, colour metadata), `AudioBuffer` | | `media` | L1 | `MediaSource` trait, probing/openers, frame cache, generators, stills, image sequences, WAV / Broadcast WAV | @@ -57,7 +60,7 @@ and `filmcraft-cli`. | `text` | L1 | text engine: font database (bundled OFL fonts + system fonts), shaping (harfrust), bidi, line breaking, paragraph layout, glyph/path rasteriser, strokes ([crates/text/README.md](../crates/text/README.md)) | | `edit` | L2 | pure edit algebra (insert, overwrite, razor, ripple, roll, slip, slide, rate stretch…; text-based editing: `edit::transcript`) | | `speech` | L2 | speech-to-text: `Transcriber` trait, Whisper model catalogue + verified downloader (feature `download`), pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), speaker labelling ([transcripts.md](transcripts.md)) | -| `codecs` | L2 | container + codec hub: MP4/MOV, MKV, MXF and Ogg sources, GOP-aware seeking, decoder registry, audio decoding | +| `codecs` | L2 | container + codec hub: MP4/MOV, MKV, MXF, Ogg and MPEG TS / PS / video elementary stream sources, GOP-aware seeking, decoder registry, audio decoding | | `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) | | `render` | L3 | sequence evaluation, CPU compositor, video effects (`effects`, `vfx`; effects needing other frames or tracks read them through `vfx::FxEnv`), transitions, audio mix | | `gpu` | L3 | wgpu compositor (WGSL) | diff --git a/docs/testing.md b/docs/testing.md index 71a67c3aa..7fb9b68d5 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -60,7 +60,7 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2). - `cargo xtask fixtures [crate…]` pre-generates the whole fixture matrix up front (useful before a parallel test run or on a fresh machine). It runs each crate's ignored `generate_fixtures` test — the same generators the oracle tests call — for `h264`, `hevc`, `isobmff`, `matroska`, `dnx`, - `prores` and `codecs` (MXF and Ogg), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The + `prores`, `mpeg2v`, `ac3` and `codecs` (MXF, Ogg and MPEG TS / PS), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The `aac`, `opus` and `h264enc` oracles generate small per-test signals on demand and are not part of the matrix. - Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones. @@ -84,7 +84,10 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2). | `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target | | `audio-dsp` loudness | `tests/loudness_oracle.rs`: signals generated in Rust (997 Hz sine, pink noise at 48/96 kHz, speech-like bursts at 48/44.1 kHz, stereo with silence and sub-gate passages, an fs/4 inter-sample-peak tone) written as float WAV and measured with `ffmpeg -af ebur128=peak=true:metadata=1` | momentary and short-term every 100 ms **±0.1 LU**, integrated **±0.1 LU**, LRA **±0.5 LU**, true peak **±0.2 dB** (against the analytic value when one exists). Measured: ΔM/ΔS ≤ 0.0005 LU, ΔI ≤ 0.007 LU, ΔLRA ≤ 0.04 LU, ΔTP ≤ 0.045 dB against ffmpeg. On the fs/4 tone ffmpeg's own true peak is +0.6 dB high (−0.32 vs the analytic −0.92 dBTP); ours is −0.05 dB | | `audio-dsp` effects | `src/effects/premiere_tests.rs`, `src/design.rs`, registry tests in `src/effects/mod.rs`: generated tones, noise and impulses | every effect: neutral settings = delayed identity (≤ 2·10⁻⁴), bit-exact determinism, latency constant and equal to where an impulse lands, block-size independence, finite / denormal-free tails at extreme settings, > 1× realtime; per effect a level or frequency-response check (measured tone vs analytic `response_db` within 0.1–0.2 dB, Butterworth vs closed form 10⁻⁶ dB, Chebyshev/elliptic ripple and stop-band bounds, LR4 crossover sum flat within 0.02 dB, comb-notch depth, echo positions to the sample, Schroeder RT60 of generated impulses within 25 %, click repair −20 dB) | -| `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 reported unsupported; truncation/mutation never panics | +| `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 long GOP, XDCAM HD422 (1080i 4:2:2) and D-10 / IMX (720×608 4:2:2) within ±4 (`mpeg2v` criterion); truncation/mutation never panics | +| `mpeg2v` | ffmpeg-encoded MPEG-1/2 elementary streams (progressive, TFF/BFF interlaced, 4:2:2 intra and long GOP, 1080i, escapes + custom matrices, MPEG-1) and our synthetic streams with field pictures / 16x8 / dual prime / concealment vectors, decoded by ffmpeg (`-xerror`) | per frame max ±4 and PSNR ≥ 58 dB, ≤ 3 % of samples differing (ffmpeg's integer IDCT vs our IEEE 1180-accurate one; measured max ±1-3, 65-75 dB); frame count, display order, picture types, interlacing and field order **exact** (ffprobe); IEEE 1180 procedure passes | +| MPEG TS / PS (via `codecs/tests/mpeg_oracle.rs`) | ffmpeg-written TS (MPEG-2 + MP2, H.264 + AAC, HEVC + LATM AAC), BDAV `.m2ts` / `.mts` (H.264 + Blu-ray LPCM / AC-3), VOB (MPEG-2 + MP2 / DVD LPCM 16 and 24-bit / AC-3), MPEG-1 system streams, `.m2v`; MPEG-2 in MOV (XDCAM `xd5c`), MP4 (`mp4v`), MKV | frame count and every frame's **PTS exact** (ffprobe); H.264 / HEVC **bit-exact**, MPEG-2 within the `mpeg2v` criterion; random seeks = sequential decode; LPCM **sample-exact**; AAC ≤ 2.1·10⁻⁷, MP2 (symphonia) ≤ 3.2·10⁻⁵, AC-3 ≤ 5·10⁻³ (decoder-specific dither; measured 2.3·10⁻³); random-access audio = sequential | +| `ac3` | ffmpeg AC-3 encodes (2/0 with coupling and rematrixing, 1/0 32 kHz, 3/2+LFE 448 kb/s, noise) decoded by ffmpeg | SNR per channel ≥ 40-60 dB on tonal content (measured 66-94 dB), ≥ 18 dB on noise (27.7 dB, the same as between two of our own dither seeds) | | `ogg` (via `codecs/tests/ogg_oracle.rs`) | ffmpeg (libopus, Vorbis) files decoded by libopus / ffmpeg | length **exact** (pre-skip, end trimming); SNR ≥ 50 dB CELT (measured 77.7), ≥ 10 dB SILK, Vorbis 139 dB; random seeks within 6·10⁻⁶ of the continuous decode | | image sequences / BWF (`codecs/tests/media_oracle.rs`) | ffmpeg-written PNG/TIFF/BMP/JPEG stills read by ffmpeg's image2 demuxer; ffmpeg BWF (`-write_bext`) | PNG/TIFF/BMP **exact**, JPEG ±12 (measured 3); missing frames hold the previous one; BWF TimeReference = ffprobe's, start timecode, samples exact | | `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count | diff --git a/xtask/src/main.rs b/xtask/src/main.rs index 02106e42d..fedffaac1 100644 --- a/xtask/src/main.rs +++ b/xtask/src/main.rs @@ -382,6 +382,7 @@ const FIXTURE_GENERATORS: &[(&str, &str)] = &[ ("codecs", "mxf_oracle"), ("codecs", "mpeg_oracle"), ("mpeg2v", "oracle"), + ("ac3", "oracle"), ("codecs", "ogg_oracle"), ];