LR-IMP-FORMATS: ARW6 module docs cite docs/arw6-compression.md

docs/superpowers/ held the working spec and plan, which pointed at the
local plan/ folder. The ARW6 module docs now cite the sections of
docs/arw6-compression.md, which gains the facts they relied on from the
spec: the NP field of word 0, the stream header word layout and how the
first half's residual lines are counted.

Co-Authored-By: Claude <noreply@anthropic.com>

crates/raw/src/vendor/arw6.rs

@@ -2,8 +2,8 @@
 //!
 //! Clean-room. The layout comes from SMPTE RDD 34 (Picture_info in word 0) and from Phase 0 black-box
 //! inspection of real ILCE-7RM6 files (tile table, group size words, stream headers and index entries), each rule
-//! checked by re-serialising parsed tiles byte-for-byte; see `docs/superpowers/specs/2026-10-09-sony-arw6-decoder-design.md`
-//! (*Container (Sony)*). No other raw decoder's source was consulted.
+//! checked by re-serialising parsed tiles byte-for-byte. The format is described in `docs/arw6-compression.md`; section
+//! names in italics below refer to it. No other raw decoder's source was consulted.
 
 use crate::llvc::{Bands3, Plane, band_rows, dequant, reconstruct3, vld_decode_line};
 use crate::vendor::arw6_curve::{ARW6_CURVE, CURVE_KNEE};
@@ -48,7 +48,7 @@ pub(crate) struct TileEntry {
 /// Most tiles a strip may hold (real files have 1 or 4).
 const MAX_TILES: usize = 16;
 
-/// Parse the tile table at the start of the strip (file structure, spec *Container*): `u32` count, `u32` 0, then
+/// Parse the tile table at the start of the strip (file structure, *Container*): `u32` count, `u32` 0, then
 /// 24-byte entries `u64` offset (relative to the strip), `u32` x, y, width, height, all little-endian.
 pub(crate) fn parse_tile_table(strip: &[u8], width: usize, height: usize) -> Result<Vec<TileEntry>> {
     let count = u32_le(strip, 0).ok_or_else(|| corrupt("truncated tile table"))?;
@@ -136,7 +136,7 @@ fn field(w: &[u8], pos: usize, n: usize) -> Option<u32> {
 const GROUP_STREAMS: [usize; 5] = [3, 3, 3, 3, 1];
 
 /// Parse word 0 (RDD 34 `Picture_info`; every real file has HS = tile width, VS = tile height / 2, BBD 16, NC 3, NW 3,
-/// NP 1000000000b), the group size words 3..=7 and each stream's header word (spec *Container*, Phase 0).
+/// NP 1000000000b), the group size words 3..=7 and each stream's header word (*Container*, Phase 0).
 pub(crate) fn parse_tile_header(tile: &[u8], tw: usize, th: usize) -> Result<TileHeader> {
     let w0 = tile.get(..16).ok_or_else(|| corrupt("tile shorter than its picture header"))?;
     let f = |pos, n| field(w0, pos, n).map(|v| v as usize);
@@ -218,10 +218,10 @@ pub(crate) fn stream_halves(tile: &[u8], s: &StreamRef, ntu: usize) -> Result<Ve
     Ok(halves)
 }
 
-// ---- sensor-anchored frame geometry and tile reconstruction (spec *Frame geometry*) ----
+// ---- sensor-anchored frame geometry and tile reconstruction (*Frame geometry*) ----
 
 /// Halves of `vs` plane rows are anchored to the sensor: half `n` covers plane rows `8n - 6 + s .. 8n + 1 + s`, so
-/// `ceil((vs - 2 - s) / 8) + 1` halves are needed, two per TU (controller ruling, spec *Frame geometry*).
+/// `ceil((vs - 2 - s) / 8) + 1` halves are needed, two per TU (*Container*).
 pub(crate) fn tu_count(vs: usize, s: u8) -> usize {
     let halves = vs.saturating_sub(2usize.saturating_add(usize::from(s))).div_ceil(8).saturating_add(1);
     halves.div_ceil(2)
@@ -246,7 +246,7 @@ pub(crate) fn half_rows(n: usize, s: u8, rows: usize) -> Range<usize> {
 
 /// First half's frame rows `lines - 2`, read before the TU count is known from g4's first index entry alone
 /// (index table at word `g4.word + 1`, entry 0's `A` = big-endian u16; its bytes start after the index table).
-/// Residual lines are decoded while at least 8 bits remain and fewer than 8 lines were read (spec *Frame geometry*).
+/// Residual lines are decoded while at least 8 bits remain and fewer than 8 lines were read (*Frame geometry*).
 pub(crate) fn first_half_shift(tile: &[u8], g4: &StreamRef, width2: usize) -> Result<u8> {
     let idx = g4.word.checked_add(1).and_then(|w| w.checked_mul(16)).ok_or_else(|| corrupt("stream position"))?;
     let data = g4.index_words.checked_mul(16).and_then(|l| l.checked_add(idx)).ok_or_else(|| corrupt("stream position"))?;
@@ -365,7 +365,7 @@ fn put(plane: &mut Plane, row: usize, line: &[i32], mut f: impl FnMut(i32) -> i3
     Ok(())
 }
 
-/// Lay a half's lines into (band, rows, QI) in order: the natural clipped assignment of spec *Frame geometry*.
+/// Lay a half's lines into (band, rows, QI) in order: the natural clipped assignment of *Frame geometry*.
 fn fill(set: [(&mut Plane, &[usize], u32); 3], lines: &[Vec<i32>]) -> Result<()> {
     let mut it = lines.iter();
     for (plane, rows, qi) in set {
@@ -398,7 +398,8 @@ fn component(c: usize, streams: &[Vec<HalfLines>], layouts: &[HalfRows], vs: usi
     let grp = |g: usize, n: usize| streams.get(3 * g + c).and_then(|s| s.get(n)).ok_or_else(|| corrupt("missing half"));
     for (n, r) in layouts.iter().enumerate() {
         let (h0, l0) = grp(0, n)?;
-        // LL3: the nibble must be 0 (Phase 0: never seen otherwise); lines are DPCM from 2048 for the green mean (c = 0) and from 0 for the base-free chroma planes (spec *Container*).
+        // LL3: the nibble must be 0 (Phase 0: never seen otherwise); lines are DPCM from 2048 for the green mean (c = 0) and
+        // from 0 for the base-free chroma planes (*Entropy coding and dequantisation*).
         if h0.qi[0] != 0 {
             return Err(unsupported("LL3 quantiser index"));
         }
@@ -464,7 +465,7 @@ pub(crate) fn curve(code12: i32) -> u16 {
     }
 }
 
-/// Colour reconstruction (spec *Colour reconstruction*, checked against Phase 0 oracle output): the planes give a
+/// Colour reconstruction (*Colour reconstruction*, checked against Phase 0 oracle output): the planes give a
 /// `(2 W2) x (2 H2)` mosaic with R at (0,0), G1 (0,1), G2 (1,0), B (1,1) of every 2x2 cell, each value passed through
 /// [`curve`]. G1 comes from the green mean minus a residual average, G2 is predicted from the (unclipped) G1 plus the
 /// residual, and R/B are chroma plus the mean of the greens clamped to 0..=4095.

crates/raw/src/vendor/arw6_testenc.rs

@@ -1,4 +1,5 @@
 //! Test-only ARW6 synthetic encoder pieces: forward 5/3 transforms (exact inverse of `llvc::inverse_*`) and `quantize`.
+//! Section names in italics refer to `docs/arw6-compression.md`.
 
 use crate::llvc::{Bands3, Plane, band_rows};
 use lightcraft_tiff::tags as t;
@@ -217,11 +218,11 @@ pub(crate) struct Qis {
 
 impl Qis {
     pub(crate) const ZERO: Qis = Qis { l1: [[0; 3]; 3], l2: [[0; 3]; 3], l3: [[0; 3]; 3], res: 0 };
-    /// The values seen in every real file (spec *Container*).
+    /// The values seen in every real file (*Container*).
     pub(crate) const REAL: Qis = Qis { l1: [[1, 1, 2], [2, 2, 3], [2, 2, 3]], l2: [[0, 0, 1]; 3], l3: [[0; 3]; 3], res: 2 };
 }
 
-/// One tile to encode: `s` is the anchor offset (spec *Frame geometry*), planes are W/2 x H/2.
+/// One tile to encode: `s` is the anchor offset (*Frame geometry*), planes are W/2 x H/2.
 pub(crate) struct TileSpec {
     pub s: u8,
     pub qi: Qis,
@@ -287,7 +288,7 @@ fn build_stream(halves: &[(Vec<u8>, Vec<u32>)]) -> Vec<u8> {
 }
 
 /// Encode one tile: forward 3-level 5/3 per component with the phases of `s`, quantise, pack every half in natural
-/// clipped order (spec *Frame geometry*), then add index tables, stream headers and the tile header.
+/// clipped order (*Frame geometry*), then add index tables, stream headers and the tile header.
 pub(crate) fn encode_tile(t: &TileSpec) -> Vec<u8> {
     let (vs, s) = (t.m.height, i32::from(t.s));
     let ntu = crate::vendor::arw6::tu_count(vs, t.s);

docs/arw6-compression.md

@@ -12,8 +12,8 @@ The codec is Sony's use of SMPTE RDD 34 (LLVC, Low Latency Video Codec). `crates
 
 - Raw SubIFD: Compression 32766, 14 bits per sample, BlackLevel 512 x4 (`0x7310`), WB `0x7313`, crop `0x74c7/0x74c8` (full frame: origin 12,8, size 9984x6656; APS-C: origin 16,10, size 6528x4352), CFA RGGB. The Sony tag-based white level (15360) is not read; the DNG `WhiteLevel` is (16383).
 - The single strip starts with a tile table: `u32` count, `u32` 0, then per tile a `u64` offset (relative to the strip) and `u32` x, y, width, height. Seen: full frame 4 tiles of 5008x3336 (HQ) or 5024 / 4992 wide (Compressed); APS-C (6592x4372) one tile. A 480-byte block follows the table (not interpreted, not needed).
-- A tile is a sequence of 16-byte words. Word 0 is RDD 34 `Picture_info` (HS = tile width, VS = tile height / 2, BBD = 16, NC = 3, NW = 3). Word 1 holds five `u24` group byte-sums, word 2 is zero, words 3-7 describe the groups (a count byte, then `u24` stream sizes in words). Groups: g0 = LL3 (3 streams, one per component), g1 / g2 / g3 = level-3 / 2 / 1 bands (3 streams each), g4 = green residual (1 stream). Streams follow in that order from word 8.
-- A stream is a header word, an index table and data, each padded to a word. The index has one entry per transform unit (TU): `A(16) X B(16) X`, where A and B are the byte lengths of the TU's two halves and X are quantiser-index (QI) nibbles (one for g0 and g4, three for g1-g3: HL, LH, HH). Halves per tile = ceil((VS - 2 - s) / 8) + 1 with s the first-half shift (below), TUs = ceil(halves / 2): 105 and 137 for the sample sizes. Halves are numbered n = 2 * TU + h; each is byte-aligned and consecutive halves are contiguous.
+- A tile is a sequence of 16-byte words. Word 0 is RDD 34 `Picture_info` (HS = tile width, VS = tile height / 2, BBD = 16, NC = 3, NW = 3, NP = `1000000000b`; other values are refused as unsupported). Word 1 holds five `u24` group byte-sums, word 2 is zero, words 3-7 describe the groups (a count byte, then `u24` stream sizes in words). Groups: g0 = LL3 (3 streams, one per component), g1 / g2 / g3 = level-3 / 2 / 1 bands (3 streams each), g4 = green residual (1 stream). Streams follow in that order from word 8.
+- A stream is a header word (a `u16` count of index words, a `u24` count of data words, then 11 bytes that are not interpreted), an index table and data, each padded to a word. The index has one entry per transform unit (TU): `A(16) X B(16) X`, where A and B are the byte lengths of the TU's two halves and X are quantiser-index (QI) nibbles (one for g0 and g4, three for g1-g3: HL, LH, HH). Halves per tile = ceil((VS - 2 - s) / 8) + 1 with s the first-half shift (below), TUs = ceil(halves / 2): 105 and 137 for the sample sizes. Halves are numbered n = 2 * TU + h; each is byte-aligned and consecutive halves are contiguous.
 - QI values seen, identical in Compressed and HQ files: level 1 component 0 (1,1,2), components 1-2 (2,2,3); level 2 (0,0,1); level 3 (0,0,0); LL3 0; residual 2. Compressed and HQ differ only in the coefficient data.
 
 ### Entropy coding and dequantisation
@@ -27,7 +27,7 @@ The codec is Sony's use of SMPTE RDD 34 (LLVC, Low Latency Video Codec). `crates
 ### Frame geometry
 
 - Each tile holds three W/2 x H/2 planes (W, H = tile size): component 0 = M, the green mean; components 1-2 = chroma c1, c2; plus a W/2 x H/2 residual plane from g4. A three-level 5/3 wavelet is inverted (vertical, then horizontal) with whole-sample symmetric extension at every edge and integer arithmetic. Tiles are independent.
-- The TU grid and the lifting phases are anchored to the full sensor, not to the tile. Let s = (lines in g4's first half) - 2, read from the stream: 0 on full frame, 3 on the APS-C crop (the crop's sensor-row offset 573 is 5 mod 8). Half n then covers plane rows 8n - 6 + s .. 8n + 1 + s, clipped to the frame. The lifting phases follow from s: full frame (0,1,1), APS-C (1,0,0). Each half holds, in clipped natural order, its rows of HL1, LH1, HH1, HL2, LH2, HH2, LL3, HL3, LH3, HH3 and, in g4, 8 residual rows; edge halves are shorter. With the right s every line of every file is used.
+- The TU grid and the lifting phases are anchored to the full sensor, not to the tile. Let s = (lines in g4's first half) - 2, read from the stream (residual lines are decoded while at least 8 bits of the half remain, at most 8): 0 on full frame, 3 on the APS-C crop (the crop's sensor-row offset 573 is 5 mod 8). Half n then covers plane rows 8n - 6 + s .. 8n + 1 + s, clipped to the frame. The lifting phases follow from s: full frame (0,1,1), APS-C (1,0,0). Each half holds, in clipped natural order, its rows of HL1, LH1, HH1, HL2, LH2, HH2, LL3, HL3, LH3, HH3 and, in g4, 8 residual rows; edge halves are shorter. With the right s every line of every file is used.
 
 ### Colour reconstruction