mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-24 10:10:05 -06:00
aiongg
0a881266c8
* Tests: one integration-test binary per crate Each crate with several tests/*.rs files keeps them as modules of tests/it/, so cargo links one test executable per crate instead of one per file. Files that change process-wide state or time a workload stay binaries of their own. * Profiles: line tables for workspace crates, no debug info for dependencies * Windows ARM64: nightly and on release pushes instead of every push to main * xtask ci: clippy and tests in the dev profile The release profile's thin LTO and single codegen unit made linking the test binaries take most of the gate. The codec and audio crates stay at opt-level 3 in the dev profile, so their tests run at release speed. * xtask ci: nextest, doctests, no examples, one wasm check Tests run with cargo nextest when it is installed (cargo test otherwise) and build with --lib --bins --tests, so examples aren't linked; doctests run separately. The wasm check keeps going past the first failing crate. VP8 fixture generation locks across processes, since nextest runs each test in its own. * VP9 synthetic fixtures: write aside and rename A test in another process could read a fixture ffmpeg was still writing. * Fuzz tests: a third of each sweep by default, all of it with FILMCRAFT_FUZZ_ITERS Every seed, mutation kind and thread count still runs. Header bytes each get one overwrite value and two bit flips in rotation, so every value and bit index still occurs across the region. FILMCRAFT_FUZZ_ITERS=N runs the whole sweep plus N random rounds per seed, as before. * Platform fallback tests: one failure point per hybrid path The failure-anywhere test injected a failure at four points around every restart point; the hybrid's paths differ only in the first GOP, around the second and last restart points, at the end and never. The replay test needs the first two GOPs. * Export tests: shorter ranges with the same checks The bitrate test measures 0.5–1.5 s, where each preset reaches its own target (on 0–3 s the simple demo left two presets at the same rate). The every-preset test exports three frames per preset. * Audio tests: effects in parallel, a table-driven FFT for the Opus quality metric The extreme-settings test keeps both settings and the 40 s tail for every effect, one thread each. The Opus quality metric's FFT builds its twiddles once and allocates nothing per transform (same algorithm and summation order). * CI: dev-profile gate with nextest xtask ci no longer builds with --release, so the release-profile overrides go; nextest is installed for the test step. * CI: apt packages through the cached install-apt-packages action * Review fixes: fuzz tests with a panic hook in their own binaries, loud fixture lock The fuzz tests of av1, h264, hevc, mpeg2v, vp8 and vp9 count panics through a process-wide hook, so a panic in another test of the same process would fail them; they are binaries of their own again. The VP8 fixture lock fails the test instead of skipping it silently, and the full-sweep doc comments say how many truncation points the full run samples. * Fuzz tests: rotate the default subset per run (FILMCRAFT_FUZZ_SEED replays one) * platform tests: decode-switch tests in their own binaries, one NVENC lock
5.2 KiB
5.2 KiB
filmcraft-mpegts
Clean-room MPEG-2 Systems demuxer: transport streams and program streams. Layer L0: depends only
on thiserror. Builds for wasm32-unknown-unknown; no unsafe. No GPL/LGPL code (FFmpeg,
libavformat) was consulted; FFmpeg is used only as an external fixture generator and test
oracle.
Specifications
| Document | Edition | Used for |
|---|---|---|
| ITU-T H.222.0 | ISO/IEC 13818-1 | 2000 (second edition) | TS packets, adaptation fields, PCR, PSI (PAT / PMT, CRC-32 of Annex A), PES headers, PTS/DTS, program streams (pack, system header, program stream map), stream types |
| ISO/IEC 11172-1 | 1993 | MPEG-1 system streams (pack header, PES header stuffing / STD buffer / timestamps) |
Blu-ray / AVCHD BDAV MPEG-2 TS layout (192-byte source packets with a 4-byte TP_extra_header; registration HDMV; stream types 0x80-0x92) |
public descriptions | .m2ts / .mts |
| DVD-Video private stream 1 sub-stream layout (AC-3 / DTS 0x80-0x8F with a 3-byte header, LPCM 0xA0-0xAF with a 6-byte header, sub-pictures 0x20-0x3F) | public descriptions | .vob |
| ETSI EN 300 468 (descriptors 0x56, 0x59, 0x6A, 0x7A, 0x7B), ATSC A/52 Annex A (AC-3 in TS) | — | identifying private PES streams |
| ISO/IEC 11172-3, 13818-3 (frame headers), ISO/IEC 13818-7 (ADTS), 14496-3 (LOAS), ATSC A/52 (syncinfo, E-AC-3 frmsiz) | — | audio frame lengths for splitting |
What it does
let file = filmcraft_mpegts::open(&bytes)?; // any ByteSource (file, memory)
for (i, s) in file.streams.iter().enumerate() {
println!("{:?} {} {} units", s.id, s.codec.name(), s.units.len());
}
let v = file.find(Kind::Video, |_| true).unwrap();
let au: Vec<u8> = file.read_unit(&bytes, v, 0)?; // the first access unit
- Transport streams: 188-byte packets, 192-byte BDAV packets (sync byte after a 4-byte arrival timestamp) and 204-byte packets; the packet size is sniffed from five consecutive sync bytes. Lost sync resynchronises on the next aligned run. PSI sections are reassembled across packets and checked with the MPEG CRC-32; the first program of the PAT is demuxed. Continuity counter errors, CRC errors and resyncs are reported as warnings.
- Program streams: MPEG-2 and MPEG-1 packs, system headers, the program stream map (stream types for elementary stream ids), padding, private stream 2 (DVD navigation) skipped, private stream 1 split by sub-stream id. Garbage resynchronises on the next pack or PES start code.
- Streams: MPEG-1/2 video, H.264, HEVC, MPEG audio, AAC (ADTS, LATM/LOAS), AC-3, E-AC-3, DTS, TrueHD, Blu-ray LPCM, DVD LPCM (format from the sub-stream header), PGS / DVB / teletext subtitles; ISO 639 language from the PMT.
- Access units, found while the file is scanned (one pass, streaming; only a few bytes of
each stream are buffered):
- MPEG-1/2 video: a coded frame — a frame picture, or two field pictures (paired by picture_structure) — with the sequence / GOP headers before it; picture type, temporal reference, field structure, top_field_first, repeat_first_field, progressive_frame, closed GOP / broken link. A PTS belongs to the unit whose picture start code is in that PES.
- H.264 / HEVC: PES packets with a timestamp (a PES without one continues the previous unit); random access from IDR / IRAP NAL types; disposable when every slice is a non-reference.
- Audio: one frame per unit from the frame headers (sync re-hunted after garbage; a header found while hunting is confirmed by the next one); a PTS belongs to the first frame starting in the PES.
- LPCM and everything else: one PES packet per unit.
- Timestamps: 33-bit PTS/DTS unwrapped across the wrap (and back, for B pictures shown before a wrap that came earlier); PCR / SCR range.
- Reading: a unit records only the position of its first packet and its offset in that
packet's payload;
read_unitre-reads from there (the index costs a few dozen bytes per unit, not per packet).
Tests
src/tests.rs: a small TS muxer (PAT, PMT with CRC, PES split into 188- or 192-byte packets with adaptation-field stuffing) and PS writer (packs ≤ 2000 bytes, so frames span PES packets; AC-3 in private stream 1): every access unit reads back byte-identical with its PTS, key and disposable flags, field pairs as one unit, audio frames with the first-in-PES timestamp; 33-bit unwrap; truncation every 37 bytes and 400 random mutations never panic.src/es.rs,src/pes.rs: frame sizes (MPEG audio layers I-III, MPEG-2 LSF, AC-3 at all rates), MPEG-2 and MPEG-1 PES headers.crates/codecs/tests/it/mpeg_oracle.rs: FFmpeg-written TS / M2TS / VOB / MPEG-1 system files — frame counts and every frame's PTS equal ffprobe's, decoded media against FFmpeg (seefilmcraft-codecs).
Gaps
- One program per TS (the first in the PAT); no program switching, no section-based metadata (SDT / EIT), no scrambled streams.
- H.264 / HEVC access units follow PES packets: streams that put several pictures in one PES packet (rare in practice; not seen in broadcast, AVCHD or FFmpeg output) would be indexed as one unit.
- DVD navigation (IFO / NAV packets), multiple VOB concatenation and angle interleaving are not handled; each file opens on its own.