mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-26 11:09:23 -06:00
Alex ChambersandClaude Opus 5.5
ac677a2624
Sony ARWs record shading, lateral chromatic aberration and distortion as a camera setting plus a radial table in the raw image IFD (0x7031-0x7037), with copies in the encrypted SR2SubIFD (0x797c-0x7982). They are now decoded for every body that records them and carried through RawInfo and RawImage as `lens_metadata.sony` (types in crates/raw/src/lens_metadata.rs, decoder in crates/raw/src/vendor/arw_lens.rs): - Raw-IFD fields are the ones the TIFF reader parsed, which is what the image itself uses. A scan of the directory adds diagnostics only: repeated tags, and why an entry the reader skipped was dropped (values outside the file, an unknown type, past the reader's limits). SR2SubIFD fields are read from their first entry. Only the exact layouts are accepted: SHORT[1] or non-negative SSHORT[1] settings, SSHORT[17] headed by 16 for shading and distortion, SSHORT[33] headed by 32 for CA. Nothing is allocated from a count in the file. - The raw IFD's copy wins. The SR2SubIFD copy fills in only for a tag the raw IFD lacks, never for a malformed one, and the two copies' agreement is recorded. - Settings keep their number and their documented meaning, Unknown(value) otherwise (never Off). - Framing evidence (stored frame, Sony and default crops, reference rectangle, coding, 3:2) and a per-component status. Only Applied components become opcodes. Applied corrections are unchanged: the ILCE-7RM4A distortion conversion receives exactly the table it received before, and no other body or component gains a correction. The ILCE-7CR distortion misses its camera JPEG at 24 mm, and shading and CA semantics are not validated on any body; see docs/sony-lens-corrections.md. Locating the SR2SubIFD, now done for every ARW that has one, selects the SR2Private offset, length and key with the TIFF reader's own rules, which lightcraft-tiff now exposes and builds its reader on (directory_entries, entry_span, ParseOptions::budget_for, decode_value). Only each field's first value is decoded, so the directory's other entries cost no memory sized by the file: a 16 MiB file whose SR2Private holds eight fields aliasing it peaks at 37 kB. DirectoryEntries keeps its layout private, and its positions() iterator uses checked arithmetic. Tests: synthetic decoder tests; a differential test of the SR2Private selection against the TIFF reader's in both byte orders (0 to 8,193 filler entries, unreadable and unusual first occurrences of each field, a used-up value budget); a dhat peak-heap test; public-corpus checks against the values ExifTool prints, with pinned and anchored SHA-256s (LIGHTCRAFT_CORPUS_REQUIRED=1 makes absent files fail); and an opt-in local inventory check (crates/raw/tests/sony_lens_inventory.rs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
80 lines
4.0 KiB
Rust
80 lines
4.0 KiB
Rust
//! Peak heap of reading Sony's `SR2Private` directory (`crates/raw/src/vendor/sr2.rs`), measured with dhat's testing
|
||
//! mode. Every ARW with `DNGPrivateData` has its `SR2SubIFD` located and decrypted, so the directory that points to
|
||
//! it must not cost memory sized by its other entries: a hostile file can give it fields that each alias most of the
|
||
//! file. Only the block's offset, length and key may be read. One test per binary (dhat's profiler is global).
|
||
|
||
use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, Tiff, TiffWriter, Value, tags as t};
|
||
|
||
#[global_allocator]
|
||
static ALLOC: dhat::Alloc = dhat::Alloc;
|
||
|
||
/// A 32 × 24 Sony CFA file of `size` bytes whose `SR2Private` directory holds the block's offset, length and key plus
|
||
/// `aliases` UNDEFINED fields that each cover the first `size` bytes of the file. Plain black level and white balance
|
||
/// are present, so nothing but the lens record needs the `SR2SubIFD`.
|
||
fn hostile_arw(size: usize, aliases: u16) -> Vec<u8> {
|
||
let o = ByteOrder::Little;
|
||
let mut raw = IfdBuilder::new();
|
||
raw.set(t::MAKE, Value::Ascii("SONY".into()));
|
||
raw.set(t::MODEL, Value::Ascii("ILCE-7CR".into()));
|
||
raw.set(t::IMAGE_WIDTH, Value::Long(vec![32]));
|
||
raw.set(t::IMAGE_LENGTH, Value::Long(vec![24]));
|
||
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![16]));
|
||
raw.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![1]));
|
||
raw.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
|
||
raw.set(t::COMPRESSION, Value::Short(vec![1]));
|
||
raw.set(t::WHITE_LEVEL, Value::Long(vec![16383]));
|
||
raw.set(0x7310, Value::Short(vec![512; 4]));
|
||
raw.set(0x7313, Value::Short(vec![2048, 1024, 1024, 1536]));
|
||
// a distortion setting, so the file has a lens record that says whether the SR2SubIFD was read
|
||
raw.set(0x7036, Value::Short(vec![0]));
|
||
raw.set(t::DNG_PRIVATE_DATA, Value::Byte(vec![0; 4]));
|
||
raw.set_image(ImageData::Strips { rows_per_strip: 24, strips: vec![vec![0; 32 * 24 * 2]] });
|
||
let mut file = TiffWriter::new(o, false).write(&[raw]).unwrap();
|
||
let pointer = Tiff::parse(&file).unwrap().ifds[0].get(t::DNG_PRIVATE_DATA).unwrap().offset as usize;
|
||
file.resize(size, 0);
|
||
let private = file.len();
|
||
let entries = 3 + aliases;
|
||
let block_at = private + 2 + 12 * usize::from(entries) + 4;
|
||
let block_len = 64u32;
|
||
o.put_u16(&mut file, entries);
|
||
for (tag, kind, count, value) in [(0x7200u16, 4u16, 1u32, block_at as u32), (0x7201, 4, 1, block_len)] {
|
||
o.put_u16(&mut file, tag);
|
||
o.put_u16(&mut file, kind);
|
||
o.put_u32(&mut file, count);
|
||
o.put_u32(&mut file, value);
|
||
}
|
||
o.put_u16(&mut file, 0x7221);
|
||
o.put_u16(&mut file, 7);
|
||
o.put_u32(&mut file, 4);
|
||
file.extend([0x11, 0x22, 0x33, 0x44]);
|
||
for i in 0..aliases {
|
||
o.put_u16(&mut file, 0x7230 + i);
|
||
o.put_u16(&mut file, 7);
|
||
o.put_u32(&mut file, size as u32);
|
||
o.put_u32(&mut file, 0);
|
||
}
|
||
o.put_u32(&mut file, 0);
|
||
file.resize(block_at + block_len as usize, 0);
|
||
let mut at = Vec::new();
|
||
o.put_u32(&mut at, private as u32);
|
||
file[pointer..pointer + 4].copy_from_slice(&at);
|
||
file
|
||
}
|
||
|
||
#[test]
|
||
fn sr2_private_fields_cost_no_memory_sized_by_the_file() {
|
||
// eight fields of 16 MiB each: 128 MiB if they were decoded, within the TIFF reader's default value budget
|
||
let file = hostile_arw(16 << 20, 8);
|
||
let _profiler = dhat::Profiler::builder().testing().build();
|
||
let header = lightcraft_raw::probe_info(&file).unwrap();
|
||
let full = lightcraft_raw::decode(&file).unwrap();
|
||
let peak = dhat::HeapStats::get().max_bytes;
|
||
// the block was still found and decrypted (its content is noise: no SR2SubIFD copies)
|
||
for info in [&header, &full.info()] {
|
||
let sony = info.lens_metadata.sony.as_ref().expect("a lens record");
|
||
assert!(sony.sr2_read, "the SR2SubIFD was not read");
|
||
}
|
||
eprintln!("peak heap {peak} bytes decoding a {} byte file", file.len());
|
||
assert!(peak < 2 << 20, "peak heap {peak} bytes decoding a {} byte file", file.len());
|
||
}
|