M9.15: READMEs for mpeg2v / mpegts / ac3 (specs, accuracy, benchmark); architecture and testing docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas
2026-10-03 04:14:23 -04:00
co-authored by Claude Opus 5.5
parent 78c263a238
commit 263a39132f
9 changed files with 300 additions and 10 deletions
+70
View File
@@ -0,0 +1,70 @@
# filmcraft-ac3
Clean-room AC-3 (Dolby Digital) audio decoder in pure Rust. Layer L0: depends only on
`thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg,
liba52) was consulted; FFmpeg is used only as an external fixture generator and test oracle.
## Specification
ATSC A/52:2012 "Digital Audio Compression (AC-3, E-AC-3) Standard" (17 December 2012), the
publicly available standard: §5 bit stream syntax and semantics, §6 decoding overview, §7.1
exponents, §7.2 parametric bit allocation (Tables 7.6-7.16 extracted mechanically from the
standard's text), §7.3 mantissas and dither, §7.4 coupling, §7.5 rematrixing, §7.7.1 dynamic
range, §7.9 transforms and block switching. The window is computed as the Kaiser-Bessel derived
window (α = 5) and checked against Table 7.33 to 6·10⁻⁶.
## API
```rust
let mut dec = filmcraft_ac3::Decoder::new();
let h = filmcraft_ac3::parse_header(frame)?; // rate, channels, frame size
let out = dec.decode(frame)?; // 1536 samples per channel, WAV channel order
dec.reset(); // before decoding from another position
```
## Decoder
- Every audio coding mode (1+1, 1/0, 2/0, 3/0, 2/1, 3/1, 2/2, 3/2) with or without LFE; 32 /
44.1 / 48 kHz and every frame size; bsid ≤ 8 (9 and 10, the half / quarter-rate variants, are
accepted). E-AC-3 (bsid 11-16) is refused with `Error::Unsupported`.
- D15 / D25 / D45 exponents with reuse across blocks; the parametric bit allocation in the
standard's fixed-point steps, delta bit allocation, the all-zero SNR offset special case;
grouped 3-, 5- and 11-level mantissas (groups shared across exponent sets), 7- and 15-level
and asymmetric mantissas up to 16 bits.
- Channel coupling (coupling bands, master coordinates, phase flags in 2/0), rematrixing (all
four banding cases), dynamic range control (`dynrng`, `dynrng2`; applied at full scale), dither
for zero-bit mantissas (uniform ±0.707, §7.3.4).
- 512-sample IMDCT and the pair of 256-sample IMDCTs for block-switched blocks, with radix-2
inverse FFTs, KBD window and overlap-add.
- Output: f32 at full scale ±1; channels reordered from the coded order (L C R Ls Rs LFE) to the
WAV / SMPTE order (L R C LFE Ls Rs). No downmix (the mixer handles layouts).
- Dither is reseeded per syncframe from the frame's CRC words, so a frame decodes to the same
samples however it was reached (seeking, render caches).
- Errors (bad exponents, reserved codes, blocks running past the frame) return `Err`; garbage
never panics.
## Accuracy (vs FFmpeg as an external oracle)
AC-3 output is not bit-exact across decoders: zero-bit mantissas are filled with
decoder-specific dither. `tests/oracle.rs` decodes FFmpeg-encoded streams and compares with
FFmpeg's decode (worst channel SNR):
| Stream | SNR vs FFmpeg | SNR between two of our dither seeds |
|---|---|---|
| 2/0 48 kHz 192 kb/s (tones, chirp) | 66.1 dB | 76.0 dB |
| 2/0 44.1 kHz 96 kb/s with coupling and rematrixing | 84.5 dB | 81.7 dB |
| 1/0 32 kHz 96 kb/s | 94.1 dB | 95.7 dB |
| 3/2 + LFE 48 kHz 448 kb/s | 81.4 dB | 90.2 dB |
| 2/0 pink noise 128 kb/s (many zero-bit mantissas) | 27.7 dB | 27.9 dB |
The noise row shows the difference is dither: two of our own decodes with different dither
sequences differ as much as we differ from FFmpeg. In MPEG-2 TS (AVCHD-style `.mts`) and VOB the
maximum sample difference to FFmpeg is 2.3·10⁻³.
## Gaps
- FFmpeg's AC-3 encoder never block-switches, so the 256-sample transforms are implemented from
§7.9.4.2 but not checked against an oracle.
- No downmixing, no heavy compression (`compr`) or dialogue normalisation (both are metadata
for playback systems), no CRC check (corrupt frames are caught by consistency checks).
- E-AC-3 is not decoded.
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "filmcraft-codecs"
description = "Container + codec hub: MP4/MOV, Matroska, MXF and Ogg sources, image sequences, GOP-aware seeking, decoder registry, audio decoding"
description = "Container + codec hub: MP4/MOV, Matroska, MXF, Ogg and MPEG TS/PS sources, image sequences, GOP-aware seeking, decoder registry, audio decoding"
version.workspace = true
edition.workspace = true
license.workspace = true
+5 -2
View File
@@ -7,9 +7,12 @@
//! `filmcraft-isobmff`: GOP-aware random access (seek to the preceding sync sample and decode
//! forward, caching every decoded frame of the GOP), sequential fast path for playback, and
//! packet-cached audio decoding.
//! - [`MkvSource`], [`MxfSource`] (OP1a / OP-Atom: AVC, VC-3, ProRes, PCM; MPEG-2 reported as
//! unsupported) and [`OggSource`] (Ogg Opus with granule-position seeking, Ogg Vorbis): the same
//! - [`MkvSource`], [`MxfSource`] (OP1a / OP-Atom: AVC, MPEG-2 incl. D-10 / XDCAM, VC-3, ProRes,
//! PCM) and [`OggSource`] (Ogg Opus with granule-position seeking, Ogg Vorbis): the same
//! GOP-aware video access and packet-cached audio.
//! - [`MpegSource`]: MPEG-2 transport streams (`.ts`, `.m2ts`, `.mts`), program streams (`.mpg`,
//! `.vob`, `.mod`) and MPEG-1/2 video elementary streams: MPEG-1/2, H.264 and HEVC video; MPEG
//! audio, AAC (ADTS / LATM), AC-3 and LPCM.
//! - [`AudioFileSource`]: standalone compressed audio files (MP3, FLAC, AIFF, …).
//! - [`openers`]: the openers to register with the engine's media pool.
+130
View File
@@ -0,0 +1,130 @@
# filmcraft-mpeg2v
Clean-room MPEG-2 video and MPEG-1 video decoder in pure Rust. Layer L0: depends only on
`thiserror` and (optional, feature `threads`, default on) `rayon`. Builds for
`wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg, libmpeg2, libavcodec) was
consulted; FFmpeg is used only as an external fixture generator and test oracle.
## Specifications
| Document | Edition | Used for |
|---|---|---|
| ITU-T H.262 \| ISO/IEC 13818-2 | 02/2000 (second edition, with Amd. 1 and Cor. 1/2 as incorporated in the 2000 text) | sequence / GOP / picture headers and extensions, slices, macroblocks, VLC tables (Annex B), inverse quantisation and mismatch control (§7.4), IDCT (Annex A), motion vectors and prediction (§7.6) |
| ISO/IEC 11172-2 | 1993 | MPEG-1 video: slices spanning rows, MPEG-1 escape codes, oddification, full-pel vectors, D pictures |
| IEEE Std 1180 | 1990 | IDCT accuracy test procedure (run as a unit test) |
| ISO/IEC TR 13818-5 (MPEG Software Simulation Group decoder behaviour) | 1997 | two decoder behaviours the normative text leaves open (see "Interpretations") |
## API
```rust
let mut dec = filmcraft_mpeg2v::Decoder::new();
for (i, au) in filmcraft_mpeg2v::access_units(&es).into_iter().enumerate() {
for pic in dec.decode(&es[au], i as i64)? { // display order
// pic.y / pic.cb / pic.cr: tight 8-bit planes; pic.chroma: 4:2:0 / 4:2:2
// pic.field_order(), pic.picture_type, pic.pts, pic.info (SequenceInfo)
}
}
let rest = dec.flush();
let info = filmcraft_mpeg2v::probe(&es); // sequence header + extensions
let au = filmcraft_mpeg2v::scan_access_unit(&es[..]); // picture types without decoding
```
`decode` takes any chunk holding whole pictures (an access unit from a TS/PS/MXF/MOV sample).
Pictures come out in display order (anchors are held until the next anchor) and carry the `pts`
of the chunk their first field came in. `reset` forgets pictures but keeps the sequence
parameters, so a random-access point without its own sequence header still decodes.
## Decoder
- Main Profile at Low/Main/High-1440/High Level, 4:2:2 Profile (IMX / D-10, XDCAM HD422),
Simple Profile, MPEG-1 (including D pictures). 4:4:4 is refused (no profile in use for it).
- Frame pictures with frame and field motion compensation, dual prime, frame and field DCT;
field pictures with field, 16x8 and dual-prime motion compensation, the second field of a
P frame predicted from the first; open and closed GOPs; B pictures without a forward
reference after a broken link are dropped (closed GOP: decoded from the backward reference).
- Both intra VLC tables, zigzag and alternate scan, linear and non-linear quantiser scale,
quantiser matrices from the sequence header and the quant matrix extension (chroma matrices
for 4:2:2), intra DC precision 8-11 bits, concealment motion vectors, saturation and
mismatch control.
- IDCT: the separable 2-D IDCT in `f32` with even/odd symmetry, the DC term added exactly,
row skipping and a DC-only path; rounded and saturated to [-256, 255]. `ieee_1180_accuracy`
runs the IEEE 1180 procedure (10 000 blocks × 3 ranges × 2 signs): peak error ≤ 1 and all mean
/ mean-square limits met.
- Motion compensation clamps reference coordinates to the picture (only corrupt streams point
outside it); a missing reference predicts from mid-gray.
- Slices of different macroblock rows decode in parallel on rayon (each row into its own chunk
of the frame); MPEG-1 slices, which may span rows, decode in order.
- Corrupt slices are concealed (the rest of the picture still decodes) and counted
(`Decoder::errors`, `last_error`); nothing panics on truncated or mutated input.
### Interpretations
Two behaviours follow the MPEG Software Simulation Group reference decoder (and FFmpeg), which
the synthetic-stream tests confirmed against FFmpeg:
- Skipped macroblocks in field pictures predict from the field of the same parity, in B
pictures as well as P pictures.
- Motion vector reconstruction (§7.6.3.1) brings the vector back into range only on the side the
delta moves it, and keeps the prediction for a zero delta. This differs from the literal
formula only when the prediction is out of range: a field vector's doubled vertical
component predicting a following frame vector.
## Accuracy (vs FFmpeg as an external oracle)
MPEG-2's IDCT is not bit-exact by definition (any IEEE 1180-accurate IDCT conforms, and the
encoder's reconstruction drifts from every decoder's). FFmpeg uses an integer IDCT, so frames
are compared within a tolerance: **per frame max |Δ| ≤ 4 and PSNR ≥ 58 dB, at most 3 % of
samples differing**. Frame count, display order, picture types, interlacing and field order are
exact (ffprobe).
`tests/oracle.rs` (FFmpeg-encoded elementary streams):
| Fixture | Content | Max Δ | Differing | PSNR |
|---|---|---|---|---|
| m2v_cif_ibbp | 352×288 progressive IBBP, Main@Main | 2 | 0.60 % | 70.3 dB |
| m2v_576i_tff | 720×576 interlaced TFF, field MC/DCT, alternate scan | 2 | 0.33 % | 73.0 dB |
| m2v_480i_bff_vlc1 | 720×480 BFF, intra VLC table 1, non-linear q, 10-bit DC | 2 | 0.38 % | 72.3 dB |
| m2v_imx_422_intra | 720×608 4:2:2 intra (IMX / D-10 style) | 1 | 0.23 % | 74.6 dB |
| m2v_hd422_longgop | 1920×1080 4:2:2 interlaced long GOP (4:2:2@High) | 2 | 0.20 % | 75.1 dB |
| m2v_1080i | 1920×1080 4:2:0 interlaced, 25 Mb/s (Main@High) | 3 | 0.25 % | 74.2 dB |
| m2v_escapes_qmatrix | q = 1 (escape codes), custom intra/inter matrices | 2 | 1.80 % | 65.6 dB |
| m2v_ipp | no B pictures | 2 | 1.67 % | 65.9 dB |
| m1v_sif | MPEG-1 352×240 IBBP | 2 | 0.59 % | 70.4 dB |
| m1v_q1 | MPEG-1 q = 1 (8/16-bit escapes) | 2 | 1.93 % | 65.2 dB |
FFmpeg's encoder never writes field pictures, 16x8 or dual-prime prediction, so
`src/synth_tests.rs` writes random but valid streams with them (random macroblock types,
vectors, field selects, quantiser scales and coefficients; skipped macroblocks; concealment
vectors; 4:2:0 and 4:2:2; TFF and BFF) and FFmpeg decodes the same streams with `-xerror`:
| Stream | Max Δ | PSNR |
|---|---|---|
| field pictures I/P/B + 16x8, TFF 4:2:0 / BFF 4:2:2 | 2 | 65.7–66.0 dB |
| field pictures + dual prime, TFF 4:2:0 / BFF 4:2:2 | 2 | 66.5–67.5 dB |
| interlaced frame pictures + field MC + dual prime, TFF / BFF | 2 | 64.8–65.7 dB |
| frame pictures with B, 4:2:2, alternate scan, non-linear q | 1 | 66.8 dB |
| seed sweep: 8 seeds × 4 configurations (160 streams checked during development) | ≤ 2 | ≥ 64.7 dB |
Residuals are kept realistic (dequantised coefficients within ±300): with extreme residuals
FFmpeg's integer IDCT saturates differently from the exact one.
## Performance
`cargo test --release -p filmcraft-mpeg2v --test perf -- --ignored --nocapture` decodes the
1080i fixture (1920×1080 4:2:0, 25 Mb/s, IBBP, field MC/DCT; FFmpeg-encoded) on an M4 Pro
(14 cores) that was heavily loaded by other jobs (load average ≈ 190), best of five runs:
**271 fps on one thread, 274 fps with slice rows on rayon** (runs under heavier momentary load
fell to 77–150 fps). These are lower bounds.
## Tests
- `src/vlc.rs`: every table builds without prefix clashes; Kraft sums equal 1 minus exactly the
codes the standard reserves; every coded_block_pattern and motion_code value appears once;
motion codes are the address-increment codes plus a sign; 111 run/level pairs per DCT table.
- `src/idct.rs`: IEEE 1180; DC-only path equals the full transform for every DC value.
- `src/headers.rs`, `src/mc.rs`, `src/bits.rs`: scans, start codes, frame rates, half-sample
rounding, field addressing, edge clamping.
- `src/tests.rs`: a hand-built intra picture decodes to its DC; truncation at every byte and
3000 random mutations never panic.
- `src/synth_tests.rs`, `tests/oracle.rs`: see above (single-threaded decode identical to the
threaded one).
+80
View File
@@ -0,0 +1,80 @@
# filmcraft-mpegts
Clean-room MPEG-2 Systems demuxer: transport streams and program streams. Layer L0: depends only
on `thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg,
libavformat) was consulted; FFmpeg is used only as an external fixture generator and test
oracle.
## Specifications
| Document | Edition | Used for |
|---|---|---|
| ITU-T H.222.0 \| ISO/IEC 13818-1 | 2000 (second edition) | TS packets, adaptation fields, PCR, PSI (PAT / PMT, CRC-32 of Annex A), PES headers, PTS/DTS, program streams (pack, system header, program stream map), stream types |
| ISO/IEC 11172-1 | 1993 | MPEG-1 system streams (pack header, PES header stuffing / STD buffer / timestamps) |
| Blu-ray / AVCHD BDAV MPEG-2 TS layout (192-byte source packets with a 4-byte TP_extra_header; registration `HDMV`; stream types 0x80-0x92) | public descriptions | `.m2ts` / `.mts` |
| DVD-Video private stream 1 sub-stream layout (AC-3 / DTS 0x80-0x8F with a 3-byte header, LPCM 0xA0-0xAF with a 6-byte header, sub-pictures 0x20-0x3F) | public descriptions | `.vob` |
| ETSI EN 300 468 (descriptors 0x56, 0x59, 0x6A, 0x7A, 0x7B), ATSC A/52 Annex A (AC-3 in TS) | — | identifying private PES streams |
| ISO/IEC 11172-3, 13818-3 (frame headers), ISO/IEC 13818-7 (ADTS), 14496-3 (LOAS), ATSC A/52 (syncinfo, E-AC-3 frmsiz) | — | audio frame lengths for splitting |
## What it does
```rust
let file = filmcraft_mpegts::open(&bytes)?; // any ByteSource (file, memory)
for (i, s) in file.streams.iter().enumerate() {
println!("{:?} {} {} units", s.id, s.codec.name(), s.units.len());
}
let v = file.find(Kind::Video, |_| true).unwrap();
let au: Vec<u8> = file.read_unit(&bytes, v, 0)?; // the first access unit
```
- **Transport streams**: 188-byte packets, 192-byte BDAV packets (sync byte after a 4-byte
arrival timestamp) and 204-byte packets; the packet size is sniffed from five consecutive sync
bytes. Lost sync resynchronises on the next aligned run. PSI sections are reassembled across
packets and checked with the MPEG CRC-32; the first program of the PAT is demuxed. Continuity
counter errors, CRC errors and resyncs are reported as warnings.
- **Program streams**: MPEG-2 and MPEG-1 packs, system headers, the program stream map (stream
types for elementary stream ids), padding, private stream 2 (DVD navigation) skipped, private
stream 1 split by sub-stream id. Garbage resynchronises on the next pack or PES start code.
- **Streams**: MPEG-1/2 video, H.264, HEVC, MPEG audio, AAC (ADTS, LATM/LOAS), AC-3, E-AC-3,
DTS, TrueHD, Blu-ray LPCM, DVD LPCM (format from the sub-stream header), PGS / DVB / teletext
subtitles; ISO 639 language from the PMT.
- **Access units**, found while the file is scanned (one pass, streaming; only a few bytes of
each stream are buffered):
- MPEG-1/2 video: a coded frame — a frame picture, or two field pictures (paired by
picture_structure) — with the sequence / GOP headers before it; picture type, temporal
reference, field structure, top_field_first, repeat_first_field, progressive_frame, closed
GOP / broken link. A PTS belongs to the unit whose picture start code is in that PES.
- H.264 / HEVC: PES packets with a timestamp (a PES without one continues the previous unit);
random access from IDR / IRAP NAL types; disposable when every slice is a non-reference.
- Audio: one frame per unit from the frame headers (sync re-hunted after garbage; a header
found while hunting is confirmed by the next one); a PTS belongs to the first frame starting
in the PES.
- LPCM and everything else: one PES packet per unit.
- **Timestamps**: 33-bit PTS/DTS unwrapped across the wrap (and back, for B pictures shown
before a wrap that came earlier); PCR / SCR range.
- **Reading**: a unit records only the position of its first packet and its offset in that
packet's payload; `read_unit` re-reads from there (the index costs a few dozen bytes per
unit, not per packet).
## Tests
- `src/tests.rs`: a small TS muxer (PAT, PMT with CRC, PES split into 188- or 192-byte packets
with adaptation-field stuffing) and PS writer (packs ≤ 2000 bytes, so frames span PES packets;
AC-3 in private stream 1): every access unit reads back byte-identical with its PTS, key and
disposable flags, field pairs as one unit, audio frames with the first-in-PES timestamp;
33-bit unwrap; truncation every 37 bytes and 400 random mutations never panic.
- `src/es.rs`, `src/pes.rs`: frame sizes (MPEG audio layers I-III, MPEG-2 LSF, AC-3 at all
rates), MPEG-2 and MPEG-1 PES headers.
- `crates/codecs/tests/mpeg_oracle.rs`: FFmpeg-written TS / M2TS / VOB / MPEG-1 system files —
frame counts and every frame's PTS equal ffprobe's, decoded media against FFmpeg (see
`filmcraft-codecs`).
## Gaps
- One program per TS (the first in the PAT); no program switching, no section-based
metadata (SDT / EIT), no scrambled streams.
- H.264 / HEVC access units follow PES packets: streams that put several pictures in one PES
packet (rare in practice; not seen in broadcast, AVCHD or FFmpeg output) would be indexed as
one unit.
- DVD navigation (IFO / NAV packets), multiple VOB concatenation and angle interleaving are not
handled; each file opens on its own.
+3 -3
View File
@@ -15,7 +15,7 @@ Implemented from the SMPTE documents (editions used):
| SMPTE ST 378 | 2004 | OP1a |
| SMPTE ST 390 | 2011 | OP-Atom |
| SMPTE ST 379-1 / 379-2 | 2009 / 2010 | generic container: content packages, element keys, frame / clip wrapping |
| SMPTE ST 381-1 | 2005 | MPEG video mapping (MPEG-2 identified, MPEG video descriptor) |
| SMPTE ST 381-1 | 2005 | MPEG video mapping (MPEG-2 identified, MPEG video descriptor; decoded by `filmcraft-mpeg2v` in `filmcraft-codecs`) |
| SMPTE ST 381-3 | 2013 | AVC byte-stream mapping and AVC sub-descriptor |
| SMPTE ST 382 | 2007 | AES3 and Broadcast Wave audio mapping (wave / AES3 descriptors) |
| SMPTE ST 331 | 2011 | element data of 8-channel AES3 sound (D-10) |
@@ -60,7 +60,7 @@ let pcm = mxf.read_pcm(&file, a, 48_000, 1920)?; // planar f32, sample-exa
pictures but has no temporal offsets (FFmpeg-written AVC), `needs_reorder` is set: the caller
orders by the bitstream (`filmcraft-codecs` uses the AVC picture order counts).
- **Codecs.** From the picture essence coding label, then the essence container label, then the
essence bytes: AVC / AVC-Intra, VC-3, ProRes (profile from the label), MPEG-2 (identified only),
essence bytes: AVC / AVC-Intra, VC-3, ProRes (profile from the label), MPEG-2 (decoded in `filmcraft-codecs`),
MPEG-4 visual, JPEG 2000, DV, uncompressed.
- **Sound.** PCM chunks (frame- or clip-wrapped) with sample counts; `read_pcm` decodes
little-endian 8/16/24/32-bit PCM (wave / AES3 descriptors) and ST 331 AES3 elements (D-10).
@@ -79,6 +79,6 @@ encrypted essence (ST 429-6), and descriptive metadata (parsed sets are ignored)
600 random mutations never panic and only list fully-present samples.
- `crates/codecs/tests/mxf_oracle.rs` (FFmpeg-written fixtures, FFmpeg as the decode oracle):
H.264 long-GOP with B pictures (bit-exact every frame + 25 random seeks), H.264 29.97 DF
timecode, DNxHR LB (±2), ProRes 422 (±1), MPEG-2 (reported unsupported, audio exact),
timecode, DNxHR LB (±2), ProRes 422 (±1), MPEG-2 long GOP, XDCAM HD422 and D-10 (±4),
OP-Atom VC-3 (clip-wrapped) and PCM, D-10 AES3 audio; PCM sample-exact in every file;
truncated and corrupted files. `cargo xtask fixtures codecs` pre-generates them.
+5 -2
View File
@@ -27,7 +27,7 @@ Design principles:
L2 edit · codecs · interchange · captions · speech
L1 frame · media · project · audio-dsp · text
L0 foundation: time · geom · color · bitstream · testkit (dev-dependency only)
codecs/containers: isobmff · matroska · mxf · ogg · h264 · h264enc · hevc · vp9 · av1 · prores · dnx · aac · opus
codecs/containers: isobmff · matroska · mxf · mpegts · ogg · h264 · h264enc · hevc · vp9 · av1 · mpeg2v · prores · dnx · aac · ac3 · opus
```
Crates are named `filmcraft-<dir>` (`crates/time` is `filmcraft-time`). The apps are `filmcraft`
@@ -42,13 +42,16 @@ and `filmcraft-cli`.
| `isobmff` | L0 | MP4/MOV demux and mux |
| `matroska` | L0 | MKV/WebM demux |
| `mxf` | L0 | MXF demux (SMPTE ST 377-1): OP1a / OP-Atom, index tables, AVC / VC-3 / ProRes / MPEG-2 identification, PCM / AES3 sound, timecode ([README](../crates/mxf/README.md)) |
| `mpegts` | L0 | MPEG-2 Systems demux (H.222.0): transport streams (188 / 192-byte BDAV / 204), program streams (MPEG-2 / MPEG-1, VOB, MOD), PSI, PES, PTS / PCR, access-unit index built on open ([README](../crates/mpegts/README.md)) |
| `ogg` | L0 | Ogg demux (RFC 3533), Ogg Opus timing (RFC 7845 granules, pre-skip, end trimming), Vorbis headers |
| `h264`, `h264enc` | L0 | H.264 decoder; H.264 encoder |
| `hevc` | L0 | H.265 Main/Main 10 decoder |
| `prores` | L0 | ProRes decoder and encoder |
| `dnx` | L0 | DNxHD / DNxHR (SMPTE ST 2019-1 VC-3) decoder and DNxHR encoder |
| `av1` | L0 | AV1 decoder (Main profile; bit-exact with libdav1d; see its README for the stage table) |
| `mpeg2v` | L0 | MPEG-2 video (Main / 4:2:2 profile; frame and field pictures, dual prime) and MPEG-1 video decoder ([README](../crates/mpeg2v/README.md)) |
| `aac` | L0 | AAC-LC decoder and encoder |
| `ac3` | L0 | AC-3 (ATSC A/52) decoder ([README](../crates/ac3/README.md)) |
| `testkit` | L0 | test-only helpers, used only as a dev-dependency: ffmpeg/ffprobe discovery, fixture dirs, golden images ([testing.md](testing.md)) |
| `frame` | L1 | `VideoFrame` (planar YUV / RGBA8 / linear RGBA f32, colour metadata), `AudioBuffer` |
| `media` | L1 | `MediaSource` trait, probing/openers, frame cache, generators, stills, image sequences, WAV / Broadcast WAV |
@@ -57,7 +60,7 @@ and `filmcraft-cli`.
| `text` | L1 | text engine: font database (bundled OFL fonts + system fonts), shaping (harfrust), bidi, line breaking, paragraph layout, glyph/path rasteriser, strokes ([crates/text/README.md](../crates/text/README.md)) |
| `edit` | L2 | pure edit algebra (insert, overwrite, razor, ripple, roll, slip, slide, rate stretch…; text-based editing: `edit::transcript`) |
| `speech` | L2 | speech-to-text: `Transcriber` trait, Whisper model catalogue + verified downloader (feature `download`), pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), speaker labelling ([transcripts.md](transcripts.md)) |
| `codecs` | L2 | container + codec hub: MP4/MOV, MKV, MXF and Ogg sources, GOP-aware seeking, decoder registry, audio decoding |
| `codecs` | L2 | container + codec hub: MP4/MOV, MKV, MXF, Ogg and MPEG TS / PS / video elementary stream sources, GOP-aware seeking, decoder registry, audio decoding |
| `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) |
| `render` | L3 | sequence evaluation, CPU compositor, video effects (`effects`, `vfx`; effects needing other frames or tracks read them through `vfx::FxEnv`), transitions, audio mix |
| `gpu` | L3 | wgpu compositor (WGSL) |
+5 -2
View File
@@ -60,7 +60,7 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
- `cargo xtask fixtures [crate…]` pre-generates the whole fixture matrix up front (useful before a
parallel test run or on a fresh machine). It runs each crate's ignored `generate_fixtures` test —
the same generators the oracle tests call — for `h264`, `hevc`, `isobmff`, `matroska`, `dnx`,
`prores` and `codecs` (MXF and Ogg), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The
`prores`, `mpeg2v`, `ac3` and `codecs` (MXF, Ogg and MPEG TS / PS), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The
`aac`, `opus` and `h264enc` oracles generate small per-test signals on demand and are not part of
the matrix.
- Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones.
@@ -84,7 +84,10 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
| `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target |
| `audio-dsp` loudness | `tests/loudness_oracle.rs`: signals generated in Rust (997 Hz sine, pink noise at 48/96 kHz, speech-like bursts at 48/44.1 kHz, stereo with silence and sub-gate passages, an fs/4 inter-sample-peak tone) written as float WAV and measured with `ffmpeg -af ebur128=peak=true:metadata=1` | momentary and short-term every 100 ms **±0.1 LU**, integrated **±0.1 LU**, LRA **±0.5 LU**, true peak **±0.2 dB** (against the analytic value when one exists). Measured: ΔM/ΔS ≤ 0.0005 LU, ΔI ≤ 0.007 LU, ΔLRA ≤ 0.04 LU, ΔTP ≤ 0.045 dB against ffmpeg. On the fs/4 tone ffmpeg's own true peak is +0.6 dB high (−0.32 vs the analytic −0.92 dBTP); ours is −0.05 dB |
| `audio-dsp` effects | `src/effects/premiere_tests.rs`, `src/design.rs`, registry tests in `src/effects/mod.rs`: generated tones, noise and impulses | every effect: neutral settings = delayed identity (≤ 2·10⁻⁴), bit-exact determinism, latency constant and equal to where an impulse lands, block-size independence, finite / denormal-free tails at extreme settings, > 1× realtime; per effect a level or frequency-response check (measured tone vs analytic `response_db` within 0.1–0.2 dB, Butterworth vs closed form 10⁻⁶ dB, Chebyshev/elliptic ripple and stop-band bounds, LR4 crossover sum flat within 0.02 dB, comb-notch depth, echo positions to the sample, Schroeder RT60 of generated impulses within 25 %, click repair −20 dB) |
| `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 reported unsupported; truncation/mutation never panics |
| `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 long GOP, XDCAM HD422 (1080i 4:2:2) and D-10 / IMX (720×608 4:2:2) within ±4 (`mpeg2v` criterion); truncation/mutation never panics |
| `mpeg2v` | ffmpeg-encoded MPEG-1/2 elementary streams (progressive, TFF/BFF interlaced, 4:2:2 intra and long GOP, 1080i, escapes + custom matrices, MPEG-1) and our synthetic streams with field pictures / 16x8 / dual prime / concealment vectors, decoded by ffmpeg (`-xerror`) | per frame max ±4 and PSNR ≥ 58 dB, ≤ 3 % of samples differing (ffmpeg's integer IDCT vs our IEEE 1180-accurate one; measured max ±1-3, 65-75 dB); frame count, display order, picture types, interlacing and field order **exact** (ffprobe); IEEE 1180 procedure passes |
| MPEG TS / PS (via `codecs/tests/mpeg_oracle.rs`) | ffmpeg-written TS (MPEG-2 + MP2, H.264 + AAC, HEVC + LATM AAC), BDAV `.m2ts` / `.mts` (H.264 + Blu-ray LPCM / AC-3), VOB (MPEG-2 + MP2 / DVD LPCM 16 and 24-bit / AC-3), MPEG-1 system streams, `.m2v`; MPEG-2 in MOV (XDCAM `xd5c`), MP4 (`mp4v`), MKV | frame count and every frame's **PTS exact** (ffprobe); H.264 / HEVC **bit-exact**, MPEG-2 within the `mpeg2v` criterion; random seeks = sequential decode; LPCM **sample-exact**; AAC ≤ 2.1·10⁻⁷, MP2 (symphonia) ≤ 3.2·10⁻⁵, AC-3 ≤ 5·10⁻³ (decoder-specific dither; measured 2.3·10⁻³); random-access audio = sequential |
| `ac3` | ffmpeg AC-3 encodes (2/0 with coupling and rematrixing, 1/0 32 kHz, 3/2+LFE 448 kb/s, noise) decoded by ffmpeg | SNR per channel ≥ 40-60 dB on tonal content (measured 66-94 dB), ≥ 18 dB on noise (27.7 dB, the same as between two of our own dither seeds) |
| `ogg` (via `codecs/tests/ogg_oracle.rs`) | ffmpeg (libopus, Vorbis) files decoded by libopus / ffmpeg | length **exact** (pre-skip, end trimming); SNR ≥ 50 dB CELT (measured 77.7), ≥ 10 dB SILK, Vorbis 139 dB; random seeks within 6·10⁻⁶ of the continuous decode |
| image sequences / BWF (`codecs/tests/media_oracle.rs`) | ffmpeg-written PNG/TIFF/BMP/JPEG stills read by ffmpeg's image2 demuxer; ffmpeg BWF (`-write_bext`) | PNG/TIFF/BMP **exact**, JPEG ±12 (measured 3); missing frames hold the previous one; BWF TimeReference = ffprobe's, start timecode, samples exact |
| `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count |
+1
View File
@@ -382,6 +382,7 @@ const FIXTURE_GENERATORS: &[(&str, &str)] = &[
("codecs", "mxf_oracle"),
("codecs", "mpeg_oracle"),
("mpeg2v", "oracle"),
("ac3", "oracle"),
("codecs", "ogg_oracle"),
];