mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-24 10:10:05 -06:00
M9.15: READMEs for mpeg2v / mpegts / ac3 (specs, accuracy, benchmark); architecture and testing docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
78c263a238
commit
263a39132f
@@ -0,0 +1,70 @@
|
||||
# filmcraft-ac3
|
||||
|
||||
Clean-room AC-3 (Dolby Digital) audio decoder in pure Rust. Layer L0: depends only on
|
||||
`thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg,
|
||||
liba52) was consulted; FFmpeg is used only as an external fixture generator and test oracle.
|
||||
|
||||
## Specification
|
||||
|
||||
ATSC A/52:2012 "Digital Audio Compression (AC-3, E-AC-3) Standard" (17 December 2012), the
|
||||
publicly available standard: §5 bit stream syntax and semantics, §6 decoding overview, §7.1
|
||||
exponents, §7.2 parametric bit allocation (Tables 7.6-7.16 extracted mechanically from the
|
||||
standard's text), §7.3 mantissas and dither, §7.4 coupling, §7.5 rematrixing, §7.7.1 dynamic
|
||||
range, §7.9 transforms and block switching. The window is computed as the Kaiser-Bessel derived
|
||||
window (α = 5) and checked against Table 7.33 to 6·10⁻⁶.
|
||||
|
||||
## API
|
||||
|
||||
```rust
|
||||
let mut dec = filmcraft_ac3::Decoder::new();
|
||||
let h = filmcraft_ac3::parse_header(frame)?; // rate, channels, frame size
|
||||
let out = dec.decode(frame)?; // 1536 samples per channel, WAV channel order
|
||||
dec.reset(); // before decoding from another position
|
||||
```
|
||||
|
||||
## Decoder
|
||||
|
||||
- Every audio coding mode (1+1, 1/0, 2/0, 3/0, 2/1, 3/1, 2/2, 3/2) with or without LFE; 32 /
|
||||
44.1 / 48 kHz and every frame size; bsid ≤ 8 (9 and 10, the half / quarter-rate variants, are
|
||||
accepted). E-AC-3 (bsid 11-16) is refused with `Error::Unsupported`.
|
||||
- D15 / D25 / D45 exponents with reuse across blocks; the parametric bit allocation in the
|
||||
standard's fixed-point steps, delta bit allocation, the all-zero SNR offset special case;
|
||||
grouped 3-, 5- and 11-level mantissas (groups shared across exponent sets), 7- and 15-level
|
||||
and asymmetric mantissas up to 16 bits.
|
||||
- Channel coupling (coupling bands, master coordinates, phase flags in 2/0), rematrixing (all
|
||||
four banding cases), dynamic range control (`dynrng`, `dynrng2`; applied at full scale), dither
|
||||
for zero-bit mantissas (uniform ±0.707, §7.3.4).
|
||||
- 512-sample IMDCT and the pair of 256-sample IMDCTs for block-switched blocks, with radix-2
|
||||
inverse FFTs, KBD window and overlap-add.
|
||||
- Output: f32 at full scale ±1; channels reordered from the coded order (L C R Ls Rs LFE) to the
|
||||
WAV / SMPTE order (L R C LFE Ls Rs). No downmix (the mixer handles layouts).
|
||||
- Dither is reseeded per syncframe from the frame's CRC words, so a frame decodes to the same
|
||||
samples however it was reached (seeking, render caches).
|
||||
- Errors (bad exponents, reserved codes, blocks running past the frame) return `Err`; garbage
|
||||
never panics.
|
||||
|
||||
## Accuracy (vs FFmpeg as an external oracle)
|
||||
|
||||
AC-3 output is not bit-exact across decoders: zero-bit mantissas are filled with
|
||||
decoder-specific dither. `tests/oracle.rs` decodes FFmpeg-encoded streams and compares with
|
||||
FFmpeg's decode (worst channel SNR):
|
||||
|
||||
| Stream | SNR vs FFmpeg | SNR between two of our dither seeds |
|
||||
|---|---|---|
|
||||
| 2/0 48 kHz 192 kb/s (tones, chirp) | 66.1 dB | 76.0 dB |
|
||||
| 2/0 44.1 kHz 96 kb/s with coupling and rematrixing | 84.5 dB | 81.7 dB |
|
||||
| 1/0 32 kHz 96 kb/s | 94.1 dB | 95.7 dB |
|
||||
| 3/2 + LFE 48 kHz 448 kb/s | 81.4 dB | 90.2 dB |
|
||||
| 2/0 pink noise 128 kb/s (many zero-bit mantissas) | 27.7 dB | 27.9 dB |
|
||||
|
||||
The noise row shows the difference is dither: two of our own decodes with different dither
|
||||
sequences differ as much as we differ from FFmpeg. In MPEG-2 TS (AVCHD-style `.mts`) and VOB the
|
||||
maximum sample difference to FFmpeg is 2.3·10⁻³.
|
||||
|
||||
## Gaps
|
||||
|
||||
- FFmpeg's AC-3 encoder never block-switches, so the 256-sample transforms are implemented from
|
||||
§7.9.4.2 but not checked against an oracle.
|
||||
- No downmixing, no heavy compression (`compr`) or dialogue normalisation (both are metadata
|
||||
for playback systems), no CRC check (corrupt frames are caught by consistency checks).
|
||||
- E-AC-3 is not decoded.
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "filmcraft-codecs"
|
||||
description = "Container + codec hub: MP4/MOV, Matroska, MXF and Ogg sources, image sequences, GOP-aware seeking, decoder registry, audio decoding"
|
||||
description = "Container + codec hub: MP4/MOV, Matroska, MXF, Ogg and MPEG TS/PS sources, image sequences, GOP-aware seeking, decoder registry, audio decoding"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
@@ -7,9 +7,12 @@
|
||||
//! `filmcraft-isobmff`: GOP-aware random access (seek to the preceding sync sample and decode
|
||||
//! forward, caching every decoded frame of the GOP), sequential fast path for playback, and
|
||||
//! packet-cached audio decoding.
|
||||
//! - [`MkvSource`], [`MxfSource`] (OP1a / OP-Atom: AVC, VC-3, ProRes, PCM; MPEG-2 reported as
|
||||
//! unsupported) and [`OggSource`] (Ogg Opus with granule-position seeking, Ogg Vorbis): the same
|
||||
//! - [`MkvSource`], [`MxfSource`] (OP1a / OP-Atom: AVC, MPEG-2 incl. D-10 / XDCAM, VC-3, ProRes,
|
||||
//! PCM) and [`OggSource`] (Ogg Opus with granule-position seeking, Ogg Vorbis): the same
|
||||
//! GOP-aware video access and packet-cached audio.
|
||||
//! - [`MpegSource`]: MPEG-2 transport streams (`.ts`, `.m2ts`, `.mts`), program streams (`.mpg`,
|
||||
//! `.vob`, `.mod`) and MPEG-1/2 video elementary streams: MPEG-1/2, H.264 and HEVC video; MPEG
|
||||
//! audio, AAC (ADTS / LATM), AC-3 and LPCM.
|
||||
//! - [`AudioFileSource`]: standalone compressed audio files (MP3, FLAC, AIFF, …).
|
||||
//! - [`openers`]: the openers to register with the engine's media pool.
|
||||
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
# filmcraft-mpeg2v
|
||||
|
||||
Clean-room MPEG-2 video and MPEG-1 video decoder in pure Rust. Layer L0: depends only on
|
||||
`thiserror` and (optional, feature `threads`, default on) `rayon`. Builds for
|
||||
`wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg, libmpeg2, libavcodec) was
|
||||
consulted; FFmpeg is used only as an external fixture generator and test oracle.
|
||||
|
||||
## Specifications
|
||||
|
||||
| Document | Edition | Used for |
|
||||
|---|---|---|
|
||||
| ITU-T H.262 \| ISO/IEC 13818-2 | 02/2000 (second edition, with Amd. 1 and Cor. 1/2 as incorporated in the 2000 text) | sequence / GOP / picture headers and extensions, slices, macroblocks, VLC tables (Annex B), inverse quantisation and mismatch control (§7.4), IDCT (Annex A), motion vectors and prediction (§7.6) |
|
||||
| ISO/IEC 11172-2 | 1993 | MPEG-1 video: slices spanning rows, MPEG-1 escape codes, oddification, full-pel vectors, D pictures |
|
||||
| IEEE Std 1180 | 1990 | IDCT accuracy test procedure (run as a unit test) |
|
||||
| ISO/IEC TR 13818-5 (MPEG Software Simulation Group decoder behaviour) | 1997 | two decoder behaviours the normative text leaves open (see "Interpretations") |
|
||||
|
||||
## API
|
||||
|
||||
```rust
|
||||
let mut dec = filmcraft_mpeg2v::Decoder::new();
|
||||
for (i, au) in filmcraft_mpeg2v::access_units(&es).into_iter().enumerate() {
|
||||
for pic in dec.decode(&es[au], i as i64)? { // display order
|
||||
// pic.y / pic.cb / pic.cr: tight 8-bit planes; pic.chroma: 4:2:0 / 4:2:2
|
||||
// pic.field_order(), pic.picture_type, pic.pts, pic.info (SequenceInfo)
|
||||
}
|
||||
}
|
||||
let rest = dec.flush();
|
||||
let info = filmcraft_mpeg2v::probe(&es); // sequence header + extensions
|
||||
let au = filmcraft_mpeg2v::scan_access_unit(&es[..]); // picture types without decoding
|
||||
```
|
||||
|
||||
`decode` takes any chunk holding whole pictures (an access unit from a TS/PS/MXF/MOV sample).
|
||||
Pictures come out in display order (anchors are held until the next anchor) and carry the `pts`
|
||||
of the chunk their first field came in. `reset` forgets pictures but keeps the sequence
|
||||
parameters, so a random-access point without its own sequence header still decodes.
|
||||
|
||||
## Decoder
|
||||
|
||||
- Main Profile at Low/Main/High-1440/High Level, 4:2:2 Profile (IMX / D-10, XDCAM HD422),
|
||||
Simple Profile, MPEG-1 (including D pictures). 4:4:4 is refused (no profile in use for it).
|
||||
- Frame pictures with frame and field motion compensation, dual prime, frame and field DCT;
|
||||
field pictures with field, 16x8 and dual-prime motion compensation, the second field of a
|
||||
P frame predicted from the first; open and closed GOPs; B pictures without a forward
|
||||
reference after a broken link are dropped (closed GOP: decoded from the backward reference).
|
||||
- Both intra VLC tables, zigzag and alternate scan, linear and non-linear quantiser scale,
|
||||
quantiser matrices from the sequence header and the quant matrix extension (chroma matrices
|
||||
for 4:2:2), intra DC precision 8-11 bits, concealment motion vectors, saturation and
|
||||
mismatch control.
|
||||
- IDCT: the separable 2-D IDCT in `f32` with even/odd symmetry, the DC term added exactly,
|
||||
row skipping and a DC-only path; rounded and saturated to [-256, 255]. `ieee_1180_accuracy`
|
||||
runs the IEEE 1180 procedure (10 000 blocks × 3 ranges × 2 signs): peak error ≤ 1 and all mean
|
||||
/ mean-square limits met.
|
||||
- Motion compensation clamps reference coordinates to the picture (only corrupt streams point
|
||||
outside it); a missing reference predicts from mid-gray.
|
||||
- Slices of different macroblock rows decode in parallel on rayon (each row into its own chunk
|
||||
of the frame); MPEG-1 slices, which may span rows, decode in order.
|
||||
- Corrupt slices are concealed (the rest of the picture still decodes) and counted
|
||||
(`Decoder::errors`, `last_error`); nothing panics on truncated or mutated input.
|
||||
|
||||
### Interpretations
|
||||
|
||||
Two behaviours follow the MPEG Software Simulation Group reference decoder (and FFmpeg), which
|
||||
the synthetic-stream tests confirmed against FFmpeg:
|
||||
|
||||
- Skipped macroblocks in field pictures predict from the field of the same parity, in B
|
||||
pictures as well as P pictures.
|
||||
- Motion vector reconstruction (§7.6.3.1) brings the vector back into range only on the side the
|
||||
delta moves it, and keeps the prediction for a zero delta. This differs from the literal
|
||||
formula only when the prediction is out of range: a field vector's doubled vertical
|
||||
component predicting a following frame vector.
|
||||
|
||||
## Accuracy (vs FFmpeg as an external oracle)
|
||||
|
||||
MPEG-2's IDCT is not bit-exact by definition (any IEEE 1180-accurate IDCT conforms, and the
|
||||
encoder's reconstruction drifts from every decoder's). FFmpeg uses an integer IDCT, so frames
|
||||
are compared within a tolerance: **per frame max |Δ| ≤ 4 and PSNR ≥ 58 dB, at most 3 % of
|
||||
samples differing**. Frame count, display order, picture types, interlacing and field order are
|
||||
exact (ffprobe).
|
||||
|
||||
`tests/oracle.rs` (FFmpeg-encoded elementary streams):
|
||||
|
||||
| Fixture | Content | Max Δ | Differing | PSNR |
|
||||
|---|---|---|---|---|
|
||||
| m2v_cif_ibbp | 352×288 progressive IBBP, Main@Main | 2 | 0.60 % | 70.3 dB |
|
||||
| m2v_576i_tff | 720×576 interlaced TFF, field MC/DCT, alternate scan | 2 | 0.33 % | 73.0 dB |
|
||||
| m2v_480i_bff_vlc1 | 720×480 BFF, intra VLC table 1, non-linear q, 10-bit DC | 2 | 0.38 % | 72.3 dB |
|
||||
| m2v_imx_422_intra | 720×608 4:2:2 intra (IMX / D-10 style) | 1 | 0.23 % | 74.6 dB |
|
||||
| m2v_hd422_longgop | 1920×1080 4:2:2 interlaced long GOP (4:2:2@High) | 2 | 0.20 % | 75.1 dB |
|
||||
| m2v_1080i | 1920×1080 4:2:0 interlaced, 25 Mb/s (Main@High) | 3 | 0.25 % | 74.2 dB |
|
||||
| m2v_escapes_qmatrix | q = 1 (escape codes), custom intra/inter matrices | 2 | 1.80 % | 65.6 dB |
|
||||
| m2v_ipp | no B pictures | 2 | 1.67 % | 65.9 dB |
|
||||
| m1v_sif | MPEG-1 352×240 IBBP | 2 | 0.59 % | 70.4 dB |
|
||||
| m1v_q1 | MPEG-1 q = 1 (8/16-bit escapes) | 2 | 1.93 % | 65.2 dB |
|
||||
|
||||
FFmpeg's encoder never writes field pictures, 16x8 or dual-prime prediction, so
|
||||
`src/synth_tests.rs` writes random but valid streams with them (random macroblock types,
|
||||
vectors, field selects, quantiser scales and coefficients; skipped macroblocks; concealment
|
||||
vectors; 4:2:0 and 4:2:2; TFF and BFF) and FFmpeg decodes the same streams with `-xerror`:
|
||||
|
||||
| Stream | Max Δ | PSNR |
|
||||
|---|---|---|
|
||||
| field pictures I/P/B + 16x8, TFF 4:2:0 / BFF 4:2:2 | 2 | 65.7–66.0 dB |
|
||||
| field pictures + dual prime, TFF 4:2:0 / BFF 4:2:2 | 2 | 66.5–67.5 dB |
|
||||
| interlaced frame pictures + field MC + dual prime, TFF / BFF | 2 | 64.8–65.7 dB |
|
||||
| frame pictures with B, 4:2:2, alternate scan, non-linear q | 1 | 66.8 dB |
|
||||
| seed sweep: 8 seeds × 4 configurations (160 streams checked during development) | ≤ 2 | ≥ 64.7 dB |
|
||||
|
||||
Residuals are kept realistic (dequantised coefficients within ±300): with extreme residuals
|
||||
FFmpeg's integer IDCT saturates differently from the exact one.
|
||||
|
||||
## Performance
|
||||
|
||||
`cargo test --release -p filmcraft-mpeg2v --test perf -- --ignored --nocapture` decodes the
|
||||
1080i fixture (1920×1080 4:2:0, 25 Mb/s, IBBP, field MC/DCT; FFmpeg-encoded) on an M4 Pro
|
||||
(14 cores) that was heavily loaded by other jobs (load average ≈ 190), best of five runs:
|
||||
**271 fps on one thread, 274 fps with slice rows on rayon** (runs under heavier momentary load
|
||||
fell to 77–150 fps). These are lower bounds.
|
||||
|
||||
## Tests
|
||||
|
||||
- `src/vlc.rs`: every table builds without prefix clashes; Kraft sums equal 1 minus exactly the
|
||||
codes the standard reserves; every coded_block_pattern and motion_code value appears once;
|
||||
motion codes are the address-increment codes plus a sign; 111 run/level pairs per DCT table.
|
||||
- `src/idct.rs`: IEEE 1180; DC-only path equals the full transform for every DC value.
|
||||
- `src/headers.rs`, `src/mc.rs`, `src/bits.rs`: scans, start codes, frame rates, half-sample
|
||||
rounding, field addressing, edge clamping.
|
||||
- `src/tests.rs`: a hand-built intra picture decodes to its DC; truncation at every byte and
|
||||
3000 random mutations never panic.
|
||||
- `src/synth_tests.rs`, `tests/oracle.rs`: see above (single-threaded decode identical to the
|
||||
threaded one).
|
||||
@@ -0,0 +1,80 @@
|
||||
# filmcraft-mpegts
|
||||
|
||||
Clean-room MPEG-2 Systems demuxer: transport streams and program streams. Layer L0: depends only
|
||||
on `thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg,
|
||||
libavformat) was consulted; FFmpeg is used only as an external fixture generator and test
|
||||
oracle.
|
||||
|
||||
## Specifications
|
||||
|
||||
| Document | Edition | Used for |
|
||||
|---|---|---|
|
||||
| ITU-T H.222.0 \| ISO/IEC 13818-1 | 2000 (second edition) | TS packets, adaptation fields, PCR, PSI (PAT / PMT, CRC-32 of Annex A), PES headers, PTS/DTS, program streams (pack, system header, program stream map), stream types |
|
||||
| ISO/IEC 11172-1 | 1993 | MPEG-1 system streams (pack header, PES header stuffing / STD buffer / timestamps) |
|
||||
| Blu-ray / AVCHD BDAV MPEG-2 TS layout (192-byte source packets with a 4-byte TP_extra_header; registration `HDMV`; stream types 0x80-0x92) | public descriptions | `.m2ts` / `.mts` |
|
||||
| DVD-Video private stream 1 sub-stream layout (AC-3 / DTS 0x80-0x8F with a 3-byte header, LPCM 0xA0-0xAF with a 6-byte header, sub-pictures 0x20-0x3F) | public descriptions | `.vob` |
|
||||
| ETSI EN 300 468 (descriptors 0x56, 0x59, 0x6A, 0x7A, 0x7B), ATSC A/52 Annex A (AC-3 in TS) | — | identifying private PES streams |
|
||||
| ISO/IEC 11172-3, 13818-3 (frame headers), ISO/IEC 13818-7 (ADTS), 14496-3 (LOAS), ATSC A/52 (syncinfo, E-AC-3 frmsiz) | — | audio frame lengths for splitting |
|
||||
|
||||
## What it does
|
||||
|
||||
```rust
|
||||
let file = filmcraft_mpegts::open(&bytes)?; // any ByteSource (file, memory)
|
||||
for (i, s) in file.streams.iter().enumerate() {
|
||||
println!("{:?} {} {} units", s.id, s.codec.name(), s.units.len());
|
||||
}
|
||||
let v = file.find(Kind::Video, |_| true).unwrap();
|
||||
let au: Vec<u8> = file.read_unit(&bytes, v, 0)?; // the first access unit
|
||||
```
|
||||
|
||||
- **Transport streams**: 188-byte packets, 192-byte BDAV packets (sync byte after a 4-byte
|
||||
arrival timestamp) and 204-byte packets; the packet size is sniffed from five consecutive sync
|
||||
bytes. Lost sync resynchronises on the next aligned run. PSI sections are reassembled across
|
||||
packets and checked with the MPEG CRC-32; the first program of the PAT is demuxed. Continuity
|
||||
counter errors, CRC errors and resyncs are reported as warnings.
|
||||
- **Program streams**: MPEG-2 and MPEG-1 packs, system headers, the program stream map (stream
|
||||
types for elementary stream ids), padding, private stream 2 (DVD navigation) skipped, private
|
||||
stream 1 split by sub-stream id. Garbage resynchronises on the next pack or PES start code.
|
||||
- **Streams**: MPEG-1/2 video, H.264, HEVC, MPEG audio, AAC (ADTS, LATM/LOAS), AC-3, E-AC-3,
|
||||
DTS, TrueHD, Blu-ray LPCM, DVD LPCM (format from the sub-stream header), PGS / DVB / teletext
|
||||
subtitles; ISO 639 language from the PMT.
|
||||
- **Access units**, found while the file is scanned (one pass, streaming; only a few bytes of
|
||||
each stream are buffered):
|
||||
- MPEG-1/2 video: a coded frame — a frame picture, or two field pictures (paired by
|
||||
picture_structure) — with the sequence / GOP headers before it; picture type, temporal
|
||||
reference, field structure, top_field_first, repeat_first_field, progressive_frame, closed
|
||||
GOP / broken link. A PTS belongs to the unit whose picture start code is in that PES.
|
||||
- H.264 / HEVC: PES packets with a timestamp (a PES without one continues the previous unit);
|
||||
random access from IDR / IRAP NAL types; disposable when every slice is a non-reference.
|
||||
- Audio: one frame per unit from the frame headers (sync re-hunted after garbage; a header
|
||||
found while hunting is confirmed by the next one); a PTS belongs to the first frame starting
|
||||
in the PES.
|
||||
- LPCM and everything else: one PES packet per unit.
|
||||
- **Timestamps**: 33-bit PTS/DTS unwrapped across the wrap (and back, for B pictures shown
|
||||
before a wrap that came earlier); PCR / SCR range.
|
||||
- **Reading**: a unit records only the position of its first packet and its offset in that
|
||||
packet's payload; `read_unit` re-reads from there (the index costs a few dozen bytes per
|
||||
unit, not per packet).
|
||||
|
||||
## Tests
|
||||
|
||||
- `src/tests.rs`: a small TS muxer (PAT, PMT with CRC, PES split into 188- or 192-byte packets
|
||||
with adaptation-field stuffing) and PS writer (packs ≤ 2000 bytes, so frames span PES packets;
|
||||
AC-3 in private stream 1): every access unit reads back byte-identical with its PTS, key and
|
||||
disposable flags, field pairs as one unit, audio frames with the first-in-PES timestamp;
|
||||
33-bit unwrap; truncation every 37 bytes and 400 random mutations never panic.
|
||||
- `src/es.rs`, `src/pes.rs`: frame sizes (MPEG audio layers I-III, MPEG-2 LSF, AC-3 at all
|
||||
rates), MPEG-2 and MPEG-1 PES headers.
|
||||
- `crates/codecs/tests/mpeg_oracle.rs`: FFmpeg-written TS / M2TS / VOB / MPEG-1 system files —
|
||||
frame counts and every frame's PTS equal ffprobe's, decoded media against FFmpeg (see
|
||||
`filmcraft-codecs`).
|
||||
|
||||
## Gaps
|
||||
|
||||
- One program per TS (the first in the PAT); no program switching, no section-based
|
||||
metadata (SDT / EIT), no scrambled streams.
|
||||
- H.264 / HEVC access units follow PES packets: streams that put several pictures in one PES
|
||||
packet (rare in practice; not seen in broadcast, AVCHD or FFmpeg output) would be indexed as
|
||||
one unit.
|
||||
- DVD navigation (IFO / NAV packets), multiple VOB concatenation and angle interleaving are not
|
||||
handled; each file opens on its own.
|
||||
@@ -15,7 +15,7 @@ Implemented from the SMPTE documents (editions used):
|
||||
| SMPTE ST 378 | 2004 | OP1a |
|
||||
| SMPTE ST 390 | 2011 | OP-Atom |
|
||||
| SMPTE ST 379-1 / 379-2 | 2009 / 2010 | generic container: content packages, element keys, frame / clip wrapping |
|
||||
| SMPTE ST 381-1 | 2005 | MPEG video mapping (MPEG-2 identified, MPEG video descriptor) |
|
||||
| SMPTE ST 381-1 | 2005 | MPEG video mapping (MPEG-2 identified, MPEG video descriptor; decoded by `filmcraft-mpeg2v` in `filmcraft-codecs`) |
|
||||
| SMPTE ST 381-3 | 2013 | AVC byte-stream mapping and AVC sub-descriptor |
|
||||
| SMPTE ST 382 | 2007 | AES3 and Broadcast Wave audio mapping (wave / AES3 descriptors) |
|
||||
| SMPTE ST 331 | 2011 | element data of 8-channel AES3 sound (D-10) |
|
||||
@@ -60,7 +60,7 @@ let pcm = mxf.read_pcm(&file, a, 48_000, 1920)?; // planar f32, sample-exa
|
||||
pictures but has no temporal offsets (FFmpeg-written AVC), `needs_reorder` is set: the caller
|
||||
orders by the bitstream (`filmcraft-codecs` uses the AVC picture order counts).
|
||||
- **Codecs.** From the picture essence coding label, then the essence container label, then the
|
||||
essence bytes: AVC / AVC-Intra, VC-3, ProRes (profile from the label), MPEG-2 (identified only),
|
||||
essence bytes: AVC / AVC-Intra, VC-3, ProRes (profile from the label), MPEG-2 (decoded in `filmcraft-codecs`),
|
||||
MPEG-4 visual, JPEG 2000, DV, uncompressed.
|
||||
- **Sound.** PCM chunks (frame- or clip-wrapped) with sample counts; `read_pcm` decodes
|
||||
little-endian 8/16/24/32-bit PCM (wave / AES3 descriptors) and ST 331 AES3 elements (D-10).
|
||||
@@ -79,6 +79,6 @@ encrypted essence (ST 429-6), and descriptive metadata (parsed sets are ignored)
|
||||
600 random mutations never panic and only list fully-present samples.
|
||||
- `crates/codecs/tests/mxf_oracle.rs` (FFmpeg-written fixtures, FFmpeg as the decode oracle):
|
||||
H.264 long-GOP with B pictures (bit-exact every frame + 25 random seeks), H.264 29.97 DF
|
||||
timecode, DNxHR LB (±2), ProRes 422 (±1), MPEG-2 (reported unsupported, audio exact),
|
||||
timecode, DNxHR LB (±2), ProRes 422 (±1), MPEG-2 long GOP, XDCAM HD422 and D-10 (±4),
|
||||
OP-Atom VC-3 (clip-wrapped) and PCM, D-10 AES3 audio; PCM sample-exact in every file;
|
||||
truncated and corrupted files. `cargo xtask fixtures codecs` pre-generates them.
|
||||
|
||||
@@ -27,7 +27,7 @@ Design principles:
|
||||
L2 edit · codecs · interchange · captions · speech
|
||||
L1 frame · media · project · audio-dsp · text
|
||||
L0 foundation: time · geom · color · bitstream · testkit (dev-dependency only)
|
||||
codecs/containers: isobmff · matroska · mxf · ogg · h264 · h264enc · hevc · vp9 · av1 · prores · dnx · aac · opus
|
||||
codecs/containers: isobmff · matroska · mxf · mpegts · ogg · h264 · h264enc · hevc · vp9 · av1 · mpeg2v · prores · dnx · aac · ac3 · opus
|
||||
```
|
||||
|
||||
Crates are named `filmcraft-<dir>` (`crates/time` is `filmcraft-time`). The apps are `filmcraft`
|
||||
@@ -42,13 +42,16 @@ and `filmcraft-cli`.
|
||||
| `isobmff` | L0 | MP4/MOV demux and mux |
|
||||
| `matroska` | L0 | MKV/WebM demux |
|
||||
| `mxf` | L0 | MXF demux (SMPTE ST 377-1): OP1a / OP-Atom, index tables, AVC / VC-3 / ProRes / MPEG-2 identification, PCM / AES3 sound, timecode ([README](../crates/mxf/README.md)) |
|
||||
| `mpegts` | L0 | MPEG-2 Systems demux (H.222.0): transport streams (188 / 192-byte BDAV / 204), program streams (MPEG-2 / MPEG-1, VOB, MOD), PSI, PES, PTS / PCR, access-unit index built on open ([README](../crates/mpegts/README.md)) |
|
||||
| `ogg` | L0 | Ogg demux (RFC 3533), Ogg Opus timing (RFC 7845 granules, pre-skip, end trimming), Vorbis headers |
|
||||
| `h264`, `h264enc` | L0 | H.264 decoder; H.264 encoder |
|
||||
| `hevc` | L0 | H.265 Main/Main 10 decoder |
|
||||
| `prores` | L0 | ProRes decoder and encoder |
|
||||
| `dnx` | L0 | DNxHD / DNxHR (SMPTE ST 2019-1 VC-3) decoder and DNxHR encoder |
|
||||
| `av1` | L0 | AV1 decoder (Main profile; bit-exact with libdav1d; see its README for the stage table) |
|
||||
| `mpeg2v` | L0 | MPEG-2 video (Main / 4:2:2 profile; frame and field pictures, dual prime) and MPEG-1 video decoder ([README](../crates/mpeg2v/README.md)) |
|
||||
| `aac` | L0 | AAC-LC decoder and encoder |
|
||||
| `ac3` | L0 | AC-3 (ATSC A/52) decoder ([README](../crates/ac3/README.md)) |
|
||||
| `testkit` | L0 | test-only helpers, used only as a dev-dependency: ffmpeg/ffprobe discovery, fixture dirs, golden images ([testing.md](testing.md)) |
|
||||
| `frame` | L1 | `VideoFrame` (planar YUV / RGBA8 / linear RGBA f32, colour metadata), `AudioBuffer` |
|
||||
| `media` | L1 | `MediaSource` trait, probing/openers, frame cache, generators, stills, image sequences, WAV / Broadcast WAV |
|
||||
@@ -57,7 +60,7 @@ and `filmcraft-cli`.
|
||||
| `text` | L1 | text engine: font database (bundled OFL fonts + system fonts), shaping (harfrust), bidi, line breaking, paragraph layout, glyph/path rasteriser, strokes ([crates/text/README.md](../crates/text/README.md)) |
|
||||
| `edit` | L2 | pure edit algebra (insert, overwrite, razor, ripple, roll, slip, slide, rate stretch…; text-based editing: `edit::transcript`) |
|
||||
| `speech` | L2 | speech-to-text: `Transcriber` trait, Whisper model catalogue + verified downloader (feature `download`), pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), speaker labelling ([transcripts.md](transcripts.md)) |
|
||||
| `codecs` | L2 | container + codec hub: MP4/MOV, MKV, MXF and Ogg sources, GOP-aware seeking, decoder registry, audio decoding |
|
||||
| `codecs` | L2 | container + codec hub: MP4/MOV, MKV, MXF, Ogg and MPEG TS / PS / video elementary stream sources, GOP-aware seeking, decoder registry, audio decoding |
|
||||
| `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) |
|
||||
| `render` | L3 | sequence evaluation, CPU compositor, video effects (`effects`, `vfx`; effects needing other frames or tracks read them through `vfx::FxEnv`), transitions, audio mix |
|
||||
| `gpu` | L3 | wgpu compositor (WGSL) |
|
||||
|
||||
+5
-2
@@ -60,7 +60,7 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
|
||||
- `cargo xtask fixtures [crate…]` pre-generates the whole fixture matrix up front (useful before a
|
||||
parallel test run or on a fresh machine). It runs each crate's ignored `generate_fixtures` test —
|
||||
the same generators the oracle tests call — for `h264`, `hevc`, `isobmff`, `matroska`, `dnx`,
|
||||
`prores` and `codecs` (MXF and Ogg), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The
|
||||
`prores`, `mpeg2v`, `ac3` and `codecs` (MXF, Ogg and MPEG TS / PS), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The
|
||||
`aac`, `opus` and `h264enc` oracles generate small per-test signals on demand and are not part of
|
||||
the matrix.
|
||||
- Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones.
|
||||
@@ -84,7 +84,10 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
|
||||
| `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target |
|
||||
| `audio-dsp` loudness | `tests/loudness_oracle.rs`: signals generated in Rust (997 Hz sine, pink noise at 48/96 kHz, speech-like bursts at 48/44.1 kHz, stereo with silence and sub-gate passages, an fs/4 inter-sample-peak tone) written as float WAV and measured with `ffmpeg -af ebur128=peak=true:metadata=1` | momentary and short-term every 100 ms **±0.1 LU**, integrated **±0.1 LU**, LRA **±0.5 LU**, true peak **±0.2 dB** (against the analytic value when one exists). Measured: ΔM/ΔS ≤ 0.0005 LU, ΔI ≤ 0.007 LU, ΔLRA ≤ 0.04 LU, ΔTP ≤ 0.045 dB against ffmpeg. On the fs/4 tone ffmpeg's own true peak is +0.6 dB high (−0.32 vs the analytic −0.92 dBTP); ours is −0.05 dB |
|
||||
| `audio-dsp` effects | `src/effects/premiere_tests.rs`, `src/design.rs`, registry tests in `src/effects/mod.rs`: generated tones, noise and impulses | every effect: neutral settings = delayed identity (≤ 2·10⁻⁴), bit-exact determinism, latency constant and equal to where an impulse lands, block-size independence, finite / denormal-free tails at extreme settings, > 1× realtime; per effect a level or frequency-response check (measured tone vs analytic `response_db` within 0.1–0.2 dB, Butterworth vs closed form 10⁻⁶ dB, Chebyshev/elliptic ripple and stop-band bounds, LR4 crossover sum flat within 0.02 dB, comb-notch depth, echo positions to the sample, Schroeder RT60 of generated impulses within 25 %, click repair −20 dB) |
|
||||
| `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 reported unsupported; truncation/mutation never panics |
|
||||
| `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 long GOP, XDCAM HD422 (1080i 4:2:2) and D-10 / IMX (720×608 4:2:2) within ±4 (`mpeg2v` criterion); truncation/mutation never panics |
|
||||
| `mpeg2v` | ffmpeg-encoded MPEG-1/2 elementary streams (progressive, TFF/BFF interlaced, 4:2:2 intra and long GOP, 1080i, escapes + custom matrices, MPEG-1) and our synthetic streams with field pictures / 16x8 / dual prime / concealment vectors, decoded by ffmpeg (`-xerror`) | per frame max ±4 and PSNR ≥ 58 dB, ≤ 3 % of samples differing (ffmpeg's integer IDCT vs our IEEE 1180-accurate one; measured max ±1-3, 65-75 dB); frame count, display order, picture types, interlacing and field order **exact** (ffprobe); IEEE 1180 procedure passes |
|
||||
| MPEG TS / PS (via `codecs/tests/mpeg_oracle.rs`) | ffmpeg-written TS (MPEG-2 + MP2, H.264 + AAC, HEVC + LATM AAC), BDAV `.m2ts` / `.mts` (H.264 + Blu-ray LPCM / AC-3), VOB (MPEG-2 + MP2 / DVD LPCM 16 and 24-bit / AC-3), MPEG-1 system streams, `.m2v`; MPEG-2 in MOV (XDCAM `xd5c`), MP4 (`mp4v`), MKV | frame count and every frame's **PTS exact** (ffprobe); H.264 / HEVC **bit-exact**, MPEG-2 within the `mpeg2v` criterion; random seeks = sequential decode; LPCM **sample-exact**; AAC ≤ 2.1·10⁻⁷, MP2 (symphonia) ≤ 3.2·10⁻⁵, AC-3 ≤ 5·10⁻³ (decoder-specific dither; measured 2.3·10⁻³); random-access audio = sequential |
|
||||
| `ac3` | ffmpeg AC-3 encodes (2/0 with coupling and rematrixing, 1/0 32 kHz, 3/2+LFE 448 kb/s, noise) decoded by ffmpeg | SNR per channel ≥ 40-60 dB on tonal content (measured 66-94 dB), ≥ 18 dB on noise (27.7 dB, the same as between two of our own dither seeds) |
|
||||
| `ogg` (via `codecs/tests/ogg_oracle.rs`) | ffmpeg (libopus, Vorbis) files decoded by libopus / ffmpeg | length **exact** (pre-skip, end trimming); SNR ≥ 50 dB CELT (measured 77.7), ≥ 10 dB SILK, Vorbis 139 dB; random seeks within 6·10⁻⁶ of the continuous decode |
|
||||
| image sequences / BWF (`codecs/tests/media_oracle.rs`) | ffmpeg-written PNG/TIFF/BMP/JPEG stills read by ffmpeg's image2 demuxer; ffmpeg BWF (`-write_bext`) | PNG/TIFF/BMP **exact**, JPEG ±12 (measured 3); missing frames hold the previous one; BWF TimeReference = ffprobe's, start timecode, samples exact |
|
||||
| `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count |
|
||||
|
||||
@@ -382,6 +382,7 @@ const FIXTURE_GENERATORS: &[(&str, &str)] = &[
|
||||
("codecs", "mxf_oracle"),
|
||||
("codecs", "mpeg_oracle"),
|
||||
("mpeg2v", "oracle"),
|
||||
("ac3", "oracle"),
|
||||
("codecs", "ogg_oracle"),
|
||||
];
|
||||
|
||||
|
||||
Reference in New Issue
Block a user