filmcraft/crates/h264

filmcraft-h264

Clean-room, pure-Rust (no unsafe) H.264 / AVC decoder, implemented from the public ITU-T Rec. H.264 (ISO/IEC 14496-10) specification. It is an L0 crate: it depends only on filmcraft-bitstream, thiserror, and optionally rayon (default feature threads), and builds for wasm32-unknown-unknown.

All syntax tables (CAVLC code tables, the 1024 CABAC (m, n) context initialisation pairs, rangeTabLPS, scaling defaults, deblocking thresholds, the 8x8 significance-map tables) were extracted from the specification text (the extraction scripts only read the ITU-T PDF). No code was taken or ported from FFmpeg, x264, JM or openh264. ffmpeg/libx264 are only used as external test oracles and fixture generators.

Supported

• NAL units: Annex-B byte streams and length-prefixed (avcC) samples; SPS (all fields incl. VUI/HRD, scaling lists with fall-back rules A/B, cropping, high-profile fields), PPS (incl. transform_8x8_mode, scaling lists, slice groups parsed), SEI / AUD / filler skipped.
• Slice header: all fields incl. reference list modification, prediction weight table and decoded reference picture marking. POC types 0, 1, 2. frame_num gap handling ("non-existing" frames).
• Entropy coding: CAVLC and CABAC (all syntax elements needed for frame coding, 4:2:0).
• Macroblocks: I (Intra 4x4 / 8x8 / 16x16, I_PCM), P (all partitions, P_8x8ref0, P_Skip), B (all partitions and sub-partitions, B_Skip, B_Direct_16x16, B_Direct_8x8), spatial and temporal direct prediction with/without direct_8x8_inference, constrained intra prediction.
• Transforms / quantisation: 4x4, 8x8, Intra16x16 DC Hadamard, 2x2 chroma DC; flat and custom scaling matrices (SPS and PPS level).
• Inter prediction: quarter-sample 6-tap luma, eighth-sample chroma, explicit and implicit weighted prediction, multiple reference frames, long-term references.
• DPB: sliding window, all MMCOs (1-6), IDR / no_output_of_prior_pics, bumping with DPB size from the level (or VUI max_dec_frame_buffering) and max_num_reorder_frames.
• Deblocking filter: full bS derivation, disable_deblocking_filter_idc 0/1/2, slice offsets, 8x8-transform edges.
• Output: cropped 8-bit planar 4:2:0 pictures in output order with the caller's pts, POC, key flag, VUI colour description (range, primaries, transfer, matrix) and sample aspect ratio.
• Frame-level multithreading (see below).

API

use filmcraft_h264::Decoder;

let mut dec = Decoder::new(); // or Decoder::with_threads(n), Decoder::from_avcc(&avcc)?
for (pts, access_unit) in access_units {
    for pic in dec.decode(access_unit, pts)? {
        // pic.width / pic.height (cropped), pic.y / pic.u / pic.v, pic.y_stride / pic.uv_stride,
        // pic.pts, pic.poc, pic.key, pic.color, pic.sar
    }
}
for pic in dec.flush() { /* ... */ }
if let Some(err) = dec.take_error() { /* error reported by a decoding thread */ }

decode expects whole access units (a buffer with several complete access units — e.g. an entire Annex-B file — also works). Decoder::stats() returns counters of the coding tools seen (macroblock types, slice types, MMCOs, long-term marks, frame_num gaps, ...).

Threading model

The calling thread parses headers and performs all POC / DPB / reference-list bookkeeping (it only needs header data). The macroblock layer of each picture is decoded as a job on a rayon pool. A job reconstructs into private buffers, deblocks each macroblock row once the row below it is reconstructed, and publishes finished rows (samples + motion data for co-located lookups) into the picture's shared Frame (OnceLock per macroblock row). Jobs of later pictures block per row on exactly the reference rows they read, so many pictures decode concurrently while the output stays bit-exact. Without the threads feature (or on wasm, or with with_threads(1)) jobs run inline. With threads, decode returns pictures once their job has finished, so output lags input by up to roughly the number of threads.

Tests

cargo test -p filmcraft-h264 (fixtures are generated on first use into target/fixtures/h264/ with ffmpeg + libx264; tests print a message and skip when /opt/homebrew/bin/ffmpeg is absent).

• Unit tests: CAVLC tables (prefix-freeness / Kraft sums, spec-style residual example), CABAC engine (context init formula, round trip against an encoder model), intra predictors with hand-computed values, transforms, luma interpolation against a straightforward reference implementation, weighting, macroblock type tables, scaling-list parsing.
• Synthetic streams (src/synth_tests.rs): hand-built CAVLC and CABAC bitstreams covering what libx264 never produces — I_PCM, long-term references, MMCO 1-6, frame_num gaps, reference list modification (short- and long-term), explicit weighted prediction in P and B slices, B_Skip with explicit bi-prediction. The expected output is known exactly and is also checked against ffmpeg.
• Robustness: avcC input with pts round trip; randomly corrupted, truncated and garbage input never panics or deadlocks (single- and multi-threaded).
• Conformance (tests/conformance.rs): every fixture is decoded single-threaded and frame-threaded and compared byte-for-byte with ffmpeg -f rawvideo -pix_fmt yuv420p; on mismatch the first differing frame, plane, sample position and macroblock are reported. Output pts/POC order is checked too.

Fixture matrix (all bit-exact, single- and multi-threaded)

libx264 fixtures unless noted; the VideoToolbox (hardware encoder) fixtures are skipped where that encoder is unavailable.

fixture	source / size	configuration
intra_cavlc	testsrc2 176x144	Baseline, keyint=1, no deblock
intra_cavlc_noise	mandelbrot+noise 352x288	Baseline, keyint=1, no deblock
intra_cavlc_8x8	testsrc2+noise 352x288	High CAVLC, 8x8 transform, intra only
intra_cavlc_cqm	testsrc2+noise 352x288	High CAVLC, cqm=jvt, intra only
p_cavlc_nodeblock	testsrc2+noise 352x288	Baseline, ref=3, no deblock
baseline_qcif	testsrc2 176x144	Baseline, keyint=15
baseline_cif_noise	mandelbrot+noise 352x288	Baseline
cavlc_b	testsrc2+noise 352x288	High, cabac=0, bframes=3
cavlc_b_temporal	testsrc2+noise 352x288	Main, cabac=0, direct=temporal, weightb
cavlc_weightp	testsrc2 fade 352x288	Main, cabac=0, weightp=2, weightb
slices4_cavlc	testsrc2+noise 352x288	Baseline, slices=4
qp1_cavlc	mandelbrot+noise 176x144	Main, CAVLC, QP 1
main_cabac_b	testsrc2+noise 352x288	Main (CABAC, B-frames)
high_720p	smptehdbars+noise 1280x720	High
high_1080p	testsrc2 1920x1080	High
crop_1918x1078	testsrc2+noise 1918x1078	High, frame cropping
bpyramid	testsrc2+noise 352x288	bframes=3, b-pyramid=normal (MMCO)
weightp2	testsrc2 fade 352x288	weightp=2
weightb	testsrc2 fade 352x288	weightb=1, bframes=3 (implicit)
direct_temporal	testsrc2+noise 352x288	direct=temporal
direct_spatial	mandelbrot+noise 352x288	direct=spatial
ref4	testsrc2+noise 352x288	ref=4
no_deblock	testsrc2+noise 352x288	no-deblock
deblock_m2_2	testsrc2+noise 352x288	deblock=-2,2
cqm_jvt	testsrc2+noise 352x288	cqm=jvt
slices4	testsrc2+noise 352x288	slices=4 (CABAC)
keyint10	testsrc2+noise 352x288	keyint=10
open_gop	testsrc2+noise 352x288	open-gop, keyint=12, bframes=3
constrained_intra	testsrc2+noise 352x288	constrained-intra
no8x8dct	testsrc2+noise 352x288	8x8dct=0
qp50	testsrc2+noise 352x288	QP 50
qp1	mandelbrot+noise 352x288	QP 1
bench_1080p	testsrc2+noise 1920x1080, 120 frames	preset medium, CRF 20 (8.8 Mbit/s)
vt_baseline	testsrc2+noise 320x240	Apple VideoToolbox, Baseline (macOS only)
vt_main	mandelbrot+noise 640x360	VideoToolbox, Main
vt_high	testsrc2+noise 640x360	VideoToolbox, High
vt_high_1080p	testsrc2 1920x1080	VideoToolbox, High 8 Mbit/s

cargo test --release -p filmcraft-h264 --test conformance coverage_report -- --ignored --nocapture prints which coding tools each fixture exercises.

Performance

cargo test --release -p filmcraft-h264 --test perf -- --ignored --nocapture (1080p High profile, CABAC, B-pyramid, 120 frames, 8.8 Mbit/s; bit-exactness is verified first). Measured on an Apple M4 Pro (14 cores) while the machine was heavily shared (load average 40-180), so these are lower bounds:

threads	fps
1	~105-122 (≈ 33 Mcycles per frame)
14	~500-600

The example cargo run --release -p filmcraft-h264 --example h264dec -- in.h264 out.yuv decodes a file; H264_BENCH_ITERS=n (and H264_THREADS=t) turns it into a benchmark, H264_STATS=1 prints the coverage counters.

Known gaps

• Interlaced coding (field pictures, PAFF, MBAFF): streams with frame_mbs_only_flag = 0 return Error::Unsupported.
• Only 8-bit 4:2:0. High 10 / 4:2:2 / 4:4:4 / monochrome and lossless (qpprime_y_zero_transform_bypass) return Error::Unsupported.
• FMO (slice groups), SP/SI slices, data partitioning (Extended profile), MVC/SVC NAL units.
• Error concealment is minimal: undecodable slices leave the prediction/previous content; missing references are replaced by other references.
• Single-threaded speed has no SIMD-specific code yet (the hot loops are written to auto-vectorise).

Extending to High 10 and 4:2:2

The chroma format and bit depth are isolated behind Sps::check_supported. The parts that are format-specific today: Planes/FrameRow sample type (u8; a u16 variant or a generic sample type is needed for bit depths > 8, together with Clip1 ranges, QpBdOffset in the QP derivations and the scaled tC0/alpha/beta in deblocking), chroma block geometry (MbWidthC / MbHeightC = 8x8 in slicedec, intra::pred_chroma, the chroma DC transform and the CABAC/CAVLC chroma DC parsing — the 2x4 total_zeros tables and ctxBlockCat 3 NumC8x8 handling are already in place), chroma motion-vector vertical scaling and the chroma deblocking edges (4:2:2 has four horizontal chroma edges).