insideTheFluxandClaude Opus 5.5
24ee45c7f8
From CodeRabbit's review of #2607: when the blocks before the first image descriptor could not be walked, check_fillable fell back to treating the whole file as pixel data and the whole screen as covered, so padding could let a large canvas through the bound. A GIF with no first frame has nothing to decode, so it is now Malformed ("no readable first frame"). The image crate's GIF decoder also stops at such files when it opens them, so this was not reachable through read_header or decode in the cases tried; the bound no longer relies on the two walks agreeing. Unit test a_gif_with_no_readable_first_frame_is_refused calls check_fillable on a GIF whose comment runs to the end of the file; the old fallback returned Ok for its 400 MP screen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PtjHGt5gnCHXgLt4NbGorf
lightcraft-codecs (L1)
Standard image formats for LightCraft: format sniffing, decoding to linear light with colour
management, fast thumbnails, and encoders with metadata embedding. Pure Rust, no C dependencies,
builds for wasm32-unknown-unknown.
API at a glance
let fmt: Option<Format> = sniff(&bytes); // JPEG/PNG/TIFF/WebP/AVIF/HEIF/JXL/GIF/BMP/PSD/raw
let d: Decoded = decode(&bytes, DecodeOptions::default())?;
// d.image: Rgb32f, linear light in the source primaries (d.space: SourceSpace)
// d.alpha, d.bit_depth, d.float, d.orientation (EXIF value, NOT applied), d.exif/xmp/icc (raw blobs)
let working: Rgb32f = to_working(&d); // linear Rec.2020 D65 (Bradford), alpha flattened onto white
let fast = decode(&bytes, DecodeOptions::fit(2048, 2048))?; // JPEG: DCT-domain 1/2..1/8 scaling
let t: Thumbnail = decode_thumbnail(&bytes, 256)?; // embedded EXIF/MPF preview or scaled decode
let h: Header = read_header(&bytes)?; // stored width/height + orientation, no pixels decoded
let icc = icc::write_named(NamedSpace::DisplayP3); // or icc::write_matrix_trc(&space, &Trc::…)
let meta = EncodeMeta { icc: Some(&icc), exif: Some(&exif), xmp: Some(&xmp) };
encode_jpeg(&EncodeImage::rgba8(&img), 90, ChromaSubsampling::S444, &meta)?;
encode_png(&EncodeImage::new(w, h, 4, Samples::U16(&px)), &meta)?;
encode_tiff(&EncodeImage::new(w, h, 3, Samples::F32(&px)), TiffCompression::Deflate, &meta)?;
encode_webp_lossless(&img, &meta)?;
encode_avif(&img, 70, 6, &meta)?; // native + feature `avif`
encode_jxl(&img, &JxlOptions { quality: 90, ..Default::default() }, &meta)?; // native + feature `jxl`
// HDR gain map JPEG (ISO 21496-1): SDR + HDR renditions → base + gain map, and back
let (map, gm) = gainmap::compute(&sdr_linear, &hdr_linear, w, h, luma, &GainMapOptions::default())?;
gainmap::encode_jpeg(&EncodeImage::new(w, h, 3, Samples::U8(&base8)), &map, &gm, 90, ChromaSubsampling::S444, &meta)?;
let found = gainmap::read_jpeg(&bytes); // ours, cameras', Lightroom's
gainmap::apply(&mut base_linear, w, h, &found.gain, &found.meta, 1.0)?;
Thumbnail fallback decoding limits the source to 64 million pixels by default.
Non-JPEG decoders allocate at source resolution before resizing, even for a tiny
thumbnail. Use decode_thumbnail_with and ThumbnailOptions::max_pixels to set
a different source budget. Oversized sources return Error::TooLarge; embedded
JPEG previews can still be used without decoding the full source image.
Format::RawTiffLike (DNG, CR2, NEF, ARW, PEF, ORF, RW2, SRW, …) and Format::RawOther (CR3, RAF,
CRW, MRW, X3F) are detected so the engine can route them to lightcraft-raw; decode returns
Error::Unsupported for them.
Formats
| Format | Decode | Encode | Notes |
|---|---|---|---|
| JPEG | yes (zune-jpeg; jpeg-decoder for DCT scaling, CMYK/YCCK, 12-bit, non-interleaved scans) | yes (jpeg-encoder, baseline, 4:4:4/4:2:2/4:2:0) | EXIF (APP1), XMP (APP1, ≤ 64 KiB), ICC (APP2, multi-chunk), Adobe APP14, MPF previews |
| PNG | yes, 1–16-bit, palette, tRNS, interlaced | 8/16-bit, gray/GA/RGB/RGBA | iCCP, eXIf, iTXt XMP; sRGB/cICP/cHRM/gAMA honoured when no ICC |
| TIFF | 8/16-bit int (32/64-bit int kept to 16 bits), 16/32/64-bit float, gray/RGB/CMYK/palette, alpha (assoc./unassoc.), planar, LZW/Deflate/PackBits/JPEG | 8/16-bit, 32-bit float; none/LZW/Deflate/PackBits | ICC (34675), XMP (700), Orientation; first IFD only; EXIF not written (use lightcraft-tiff) |
| WebP | lossy + lossless, alpha, first frame of animations | lossless only | ICCP/EXIF/XMP chunks |
| GIF / BMP | yes (first GIF frame) | — | via image |
| PSD / PSB | merged composite: 8/16/32-bit gray, RGB, CMYK, indexed, duotone (as gray); raw/RLE | — | ICC (1039), EXIF (1058), XMP (1060); ZIP-compressed composite, Lab and 1-bit unsupported |
| JPEG XL | yes (jxl-oxide, feature jxl, default on) |
yes (jixel, encode_jxl, feature jxl, native only): lossy VarDCT / lossless Modular, 8/16-bit, effort (fast / normal), progressive; HDR as 16-bit Rec. 2020 PQ |
decode: enum colour → rendered straight to linear Rec.2020 (XYB via linear sRGB primaries); ICC → our ICC path; orientation applied by the decoder (reported as 1). Encode: lossy files declare an enum encoding (sRGB, Display P3, Rec. 2020 BT.709 curve, Rec. 2020 PQ), lossless ones embed the ICC profile; EXIF / XMP in Exif / xml boxes |
| AVIF | no | yes (ravif/rav1e, native only, feature avif) |
8/10-bit sRGB, EXIF; no ICC in the muxer. HDR: encode_avif_hdr writes 10-bit BT.2020 PQ (SDR white at 203 cd/m², CICP nclx + clli) or HLG (hlg_signal, SDR white at 0.75) |
| HEIC/HEIF | yes with feature heif (lightcraft-heif: heic-rs + libheif-matching colour; 4:2:0/mono, 8–10-bit, grids, irot/imir/clap, ICC/nclx/VUI, EXIF/XMP, thumbnail item) |
— | 4:2:2/4:4:4 (Canon/Sony HIF) unsupported |
| Gain map JPEG (HDR) | gain map found and decoded (gainmap::read_jpeg): ISO 21496-1 metadata, else Adobe hdrgm XMP |
yes (gainmap::encode_jpeg) |
CIPA DC-007 MPF index (gain map typed 0x050000), ISO 21496-1 APP2 on both images, hdrgm + Container XMP for Android/Chrome, Apple HDRGainMap/HDRToneMap XMP; gain maps are never used as thumbnails |
Colour
Decoders never guess silently; SourceSpace::origin says where the interpretation came from:
IccMatrixTrc (profile applied exactly: TRC linearization + colorants → XYZ D50), IccCms (LUT-based
or CMYK profile converted by the CMS straight to linear Rec.2020), Container (PNG chunks, JXL enum,
EXIF R03 Adobe RGB hint), Untagged (assumed sRGB; untagged float data assumed linear),
IccUnsupported (profile present but unusable → sRGB fallback), Naive (CMYK without profile).
decode_jpeg_with_fallback(bytes, opts, space) lets an enclosing RAW container supply the colour
space of a JPEG with no ICC or EXIF metadata. It applies the transfer curve before linear-light
resizing; a JPEG carrying its own metadata retains its existing interpretation.
ICC profiles are parsed with moxcms (v2/v4, curv/para TRCs, chad); sRGB, Display P3,
Adobe RGB (1998), ProPhoto (ROMM) and Rec.2020 are recognised by colorants. icc::write_matrix_trc
emits v4 display profiles (D50 PCS, Bradford chad) for export embedding.
Before colour conversion, positive-infinity float samples become 1.0; NaN and negative infinity become 0.0. This also applies to alpha. Finite samples keep their existing range: matrix/TRC paths preserve values above 1.0 and below 0.0, while CMS inputs and alpha remain bounded to 0..1.
Dependencies and licences
All pure Rust, verified with cargo info / cargo tree:
| Crate | Licence | Use |
|---|---|---|
| zune-jpeg, zune-core | MIT OR Apache-2.0 OR Zlib | JPEG decode |
| jpeg-decoder | MIT OR Apache-2.0 | scaled / CMYK JPEG decode |
| jpeg-encoder | (MIT OR Apache-2.0) AND IJG | JPEG encode (IJG: permissive, attribution in docs) |
| png | MIT OR Apache-2.0 | PNG |
| tiff | MIT | TIFF |
| image-webp | MIT OR Apache-2.0 | WebP |
| image (gif, bmp only) | MIT OR Apache-2.0 | GIF/BMP |
| moxcms | BSD-3-Clause OR Apache-2.0 | ICC parse/write, CMS |
| jxl-oxide | MIT OR Apache-2.0 | JPEG XL decode |
| jixel | BSD-3-Clause OR Apache-2.0 | JPEG XL encode (native only) |
| ravif (+ rav1e, avif-serialize) | BSD-3-Clause / BSD-2-Clause | AVIF encode (native only; heavy to compile) |
Deliberately not used: anything AGPL from imazen (heic, zenjpeg, jxl-encoder, rav1d-safe), jpegxl-rs / imagequant (GPL / C), dav1d / libheif (C).
Performance (Apple M-series, release, synthetic 24 MP JPEG q90 4:2:0)
Run cargo test --release -p lightcraft-codecs --test it bench:: -- --ignored --nocapture.
| Operation | Time |
|---|---|
full decode → linear Rgb32f |
~92 ms (≈ 260 MP/s) |
| decode fit 2048 (DCT 1/2) | ~81 ms |
| thumbnail 256, scaled decode (DCT 1/8) | ~54 ms (entropy decoding bound) |
| thumbnail 256 from embedded EXIF/MPF preview | ~0.6 ms |
to_working matrix |
~23 ms |
| PNG / TIFF-deflate encode (fast levels) | ~430 ms / ~900 ms |
Gaps / limitations
- AVIF decode: no permissively licensed, pure-Rust AV1 decoder exists today (dav1d is C;
rav1d-safe is AGPL). Files are sniffed and rejected with
Error::Unsupported. - HEIC needs the
heiffeature (off by default: HEVC patents are the distributor's call); without it,Error::Unsupportedsays the build has no HEIC support. 4:2:2 and 4:4:4 HEIF (Canon/Sony HIF) is refused: heic-rs 0.1.1 loses sync on those bitstreams. - JPEG: arithmetic coding unsupported (as in both decoders); extended XMP (> 64 KiB) neither read nor written; EXIF > 64 KiB not written. The encoder uses standard Huffman tables (interleaved baseline) for maximum compatibility.
- Thumbnails under 15 ms need an embedded preview ≥ the requested edge (EXIF IFD1 or MPF). Most
cameras store a 160 px EXIF thumbnail: use
decode_thumbnail_withwithmin_embedded_edge: 160for an instant placeholder, otherwise the DCT-scaled path takes ~50 ms for 24 MP. - TIFF: first image only; EXIF is not re-emitted on encode; Lab/uncompressed-YCbCr unsupported.
- PSD: only the merged composite (no layers); ZIP-compressed composites unsupported.
- Panics inside third-party decoders are caught (
catch_unwind) on unwinding targets; on wasm32 (panic = abort) we rely on the property tests of our own code plus the decoders' own hardening.