Commit Graph
100 Commits
Author SHA1 Message Date
aiongg 035d4d5dcb Merge PR #2977: LR-META-EXIF-FULL: reject malformed EXIF clocks
docs/gaps.md, docs/parity-checklist.md, docs/whats-new.md: both sides kept; the PR's sentences added to main's rows and entries.
docs/xmp-interop.md: capture-time paragraph combines both sides.
The PR's test file crates/engine/src/tests_import.rs landed in crates/engine/src/tests/import.rs (moved on main).
2026-10-31 02:08:25 +00:00
aiongg e967200f0d Merge PR #2912: Reject incomplete ISO timestamps in metadata and track logs
crates/meta/src/xmp.rs: kept main's 'mod tests;'; the PR's two tests ported to xmp/tests.rs.
docs/gaps.md, docs/parity-checklist.md, docs/whats-new.md: both sides kept; newer date in headers; XMP row and header carry the PR's sentence.
2026-10-31 02:07:55 +00:00
aiongg 020622aadf Merge PR #2776: Reject signed numeric character references in XML entity decoding
crates/meta/src/xmp.rs: kept main's 'mod tests;' (tests moved to xmp/tests.rs); PR's new test ported there.
2026-10-31 02:06:49 +00:00
aiongg 6c8bcf8a29 Merge PR #2771: Read Canon maker-note lens when LensModel is missing 2026-10-31 02:06:36 +00:00
aiongg 8c96803360 Merge PR #2717: Never crash: rational tag lists show 16 values like every other list (#2579 item 7) 2026-10-31 02:06:36 +00:00
aiongg 9a98d40575 Merge PR #2605: Write capture time UTC offset and sub-seconds in write_exif 2026-10-31 02:06:35 +00:00
aiongg 0c264a6a49 Merge PR #2290: #2284: read the Exif and XMP items of HEIF files 2026-10-31 02:06:35 +00:00
Brandon Thomas bdb146be43 Merge PR #2833 2026-10-30 15:24:30 -04:00
eliets2 3833c37a75 review: read Canon LensModel (0x0095) for the maker-note lens fallback
- tag 0x0001 is CameraSettings (numeric LensType at index 22), not an ASCII name; real EOS 5D files never carry a lens name there
- read the lens name from 0x0095 (LensModel) and yield none when only the numeric LensType exists
- fixture now builds a representative maker note (CameraSettings table + LensType 178) and covers both byte orders plus the nameless case
2026-10-28 07:56:50 +02:00
eliets2 9db9811759 Read Canon maker-note lens when LensModel is missing
LightCraft already fills a missing lens_model from the Panasonic and Nikon maker notes in try_read_exif, but never from the Canon maker note, so Canon bodies such as the EOS 5D that omit EXIF LensModel cannot match any lens profile even when the maker note records the lens.

Add a canon_maker_note_lens fallback in try_read_exif, parsed with lightcraft_tiff::makernote::parse_makernote (Canon notes are the Plain kind), mirroring panasonic_maker_note_lens; add a unit test in exif.rs that feeds an Exif block with a Canon maker note but no LensModel and asserts lens_model is populated.

Test: canon_maker_note_lens_fills_lens_model failed before the change and passes after.
2026-10-28 07:56:50 +02:00
Dylan M. Taylor 38c3232214 refactor(meta): isolate descriptive metadata merging 2026-10-28 03:24:18 +00:00
tecari0andClaude Sonnet 5.5 3ae66b2d2f Merge main into fix-2284-heif-exif
Resolve the conflict in the crates/meta/src/lib.rs doc comment: list
both PSD/PSB (from main) and HEIF/HEIC/AVIF (from this branch).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 15:15:45 -07:00
aiongg d151df0149 style: cargo fmt the moved test modules 2026-10-26 18:30:36 +00:00
aiongg 55c9eac7c7 refactor(lightcraft-meta): move focus.rs tests to crates/meta/src/focus/tests.rs 2026-10-26 18:15:50 +00:00
aiongg 03e3b947dc refactor(lightcraft-meta): move xmp.rs tests to crates/meta/src/xmp/tests.rs 2026-10-26 18:15:50 +00:00
Dylan M. Taylor 3022baa861 security(raw,meta): follow-ups from review
An oversized CR3 candidate is skipped without ending the search, repeated child IFD references count once toward the pending cap, deflated chunks lay their rows out as the unpacker reads them (16-bit words, 32-bit word alignment), the tile grid's nominal size is checked against the image before anything is decoded, chunk grouping saturates, and the XMP flattened-key budget is document-wide. The JPEG-header test now parses a real header.
2026-10-25 16:12:03 -04:00
Dylan M. Taylor d8ceb3a6af security/meta,merge,raw,tiff,denoise: bound memory sized by attacker-shaped inputs
XMP flattening, sparse-focus EXIF decode (plus the maker-note and TIFF value
budgets), lossy-DNG chunk staging, panorama stitch frames, the reference ONNX
executor's tensors and the denoise product reader all now reject or cap what a
malformed file can make them allocate: each result is checked before the
memory behind it is reserved, every regression carrying a test that fails
without its bound.
2026-10-25 15:42:19 -04:00
ronibalthazar a1e1b4cc0e LR-META-EXIF-FULL: reject malformed EXIF clocks 2026-10-25 19:03:33 +00:00
ronibalthazar 07d1f0ad5d LR-META-XMP: reject incomplete ISO timestamps 2026-10-25 06:35:43 +00:00
NetterSilverandClaude Haiku 5.5 4ef14cc4b3 Strip trailing NULs from non-UTF-8 IPTC text
Found while testing crates/meta/src/iptc.rs:8 (decode)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 20:38:36 +02:00
NetterSilverandClaude Haiku 5.5 e0519d4300 Reject signed numeric character references in XML entity decoding
Found while testing crates/meta/src/xmp.rs:184 (entity); the same pattern is in crates/lensdb/src/xml.rs:45 and crates/engine/src/preset_luminar.rs:90

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 06:02:56 +00:00
francoisgnandClaude Opus 5.5 144d900c21 Merge upstream/main (integration batch 43, #2729, #2737, #2412) into round3a-integration
Conflicts:
- apps/lightcraft/src/native_menu.rs: upstream's item swapping (macOS keys
  given up for real, Help menu not swapped, Settings / Quit tracked) with
  KEY-GRID's keymap-aware rows. `Item.label` stays the text the row shows
  (native_item_presentation), so `Item::native_label` reuses it instead of
  formatting it again; Settings / Quit store theirs through
  display_item_label, as upstream's native_label did. No early return after
  a rebuild (upstream).
- crates/ui-egui/src/i18n.rs: upstream's tests (#2729: one test per check
  across languages, gaps reported, never a failure) plus the batch's two new
  ones (dropped-photos notice, Polish counts); production code as merged.
- grid.rs, keyword_list.rs, masking.rs: the translation tests #2729 removed
  stay removed; the batch's new keyword and segmenter tests stay.
- cmd/library.rs, ui-egui/src/lib.rs: both test modules, each with its
  #[cfg(test)].
- xtask/src/main.rs: #2302's test_selected and upstream's translation_gaps.
- docs: tracker / gap rows take both sides' edits, headers list both sides'
  changes, history rows of both sides newest-first. What's New: one section
  per date, newest first (the batch had 10-22 above 10-23, and second 10-21
  and 10-19 sections); no line lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 02:26:48 +03:00
Brandon Thomas 1acd2938ea Merge PR #2546
# Conflicts:
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/whats-new.md
2026-10-23 17:21:18 -04:00
Brandon Thomas bbf9b713f2 Merge PR #2526
# Conflicts:
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/whats-new.md
2026-10-23 17:21:17 -04:00
francoisgnandClaude Opus 5.5 f74178cdae Merge upstream/main (#2710 one test binary per crate, #2713) into round3a-integration
Conflicts:
- xtask/src/main.rs: upstream's runners (cargo test or nextest, no
  examples, doctests apart, one wasm `cargo check --keep-going`) with
  #2302's `ci --quick` on top: test_selected builds the same targets
  (--lib --bins --tests), the wasm check takes the changed crates in one
  call, the full test step is upstream's.
- apps/lightcraft/src/control_server.rs: upstream's `before` closure, with
  the batch's no-delay sockets kept.
- docs/parity-checklist.md: LR-BEHAV-ACCESS from main (tests/it path),
  LR-BEHAV-LOCALE / LR-BEHAV-LOCALIZE from the batch (Polish).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:06:33 +03:00
aiongg f3bb7a9f30 CI: one test binary per crate, line tables only, one WASM check (#2710)
* CI: dev profile keeps line tables for workspace crates and no debug info for dependencies

* Control server tests: persistent_requests_and_slow_replies_outlive_the_idle_read_timeout no longer races the idle timeout

Each request (and the closing half-shutdown) reaches the socket before the
worker reads, so on a slow machine only the reply waits outlast the 100 ms
idle timeout, as the test intends. The idle test sends its input the same
way, and the no-frames test allows 5 s instead of 2 s to give up.

* CI: one integration-test binary per crate

Each crate's tests/*.rs become modules of tests/it/main.rs (51 test files
into 13 binaries, plus 6 that stay their own). Inner #![cfg] attributes
move to the mod lines. Files that change state for the whole process keep
a binary of their own: codecs jxl_tail_heap and raw sr2_heap (dhat global
allocator), gpu shutdown, shutdown_first and memory, ui-egui shutdown.
Docs, comments and the GPU probe name the new paths and targets
(--test it <module>::); the CI report names an it binary's skips after
the crate.

* CI: tests under nextest, doctests apart, one wasm check

xtask ci runs the tests with cargo nextest run --lib --bins --tests when
cargo-nextest is installed, else cargo test with the same targets and a
one-line notice; doctests run after them with cargo test --workspace --doc.
The heif and hardware GPU steps use the same runner. Examples stay out of
the test build (clippy --all-targets still checks them). Concurrent tests
under nextest follow the RUST_TEST_THREADS cap (NEXTEST_TEST_THREADS).
The wasm check is one cargo check --keep-going over the same crates, and
clippy runs with --keep-going.

* CI: tests run under cargo test by default, nextest with LIGHTCRAFT_NEXTEST=1 (per-process scene cache)
2026-10-23 13:34:30 -04:00
darthnutandClaude Opus 5.5 84951ff579 Never crash: rational tag lists show 16 values like every other list (#2579)
The tag panel writes a list as its first 16 values and a count (`list`), but RATIONAL and
SRATIONAL arrays were formatted and joined whole, so a tag holding millions of fractions
became a string as long. They now go through the same cap (`list_with`, which formats only
the values it shows). Item 7 of #2579.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:07:47 -07:00
NetterSilverandClaude Haiku 5.5 7fcd2b8c99 Write capture time UTC offset and sub-seconds in write_exif
Found while testing crates/meta/src/exif.rs:462 (write_exif; compare the reader in from_tiff at crates/meta/src/exif.rs:96-107)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 17:32:10 +00:00
ronibalthazar 36d6762f08 #2519: reject invalid UTF-8 in PSD XMP resources 2026-10-22 16:15:39 +01:00
ronibalthazar 26a24eb22e Fail closed on malformed PSD XMP and duplicate resources 2026-10-22 15:48:40 +01:00
ronibalthazar d18fb06aa8 Fail closed on malformed PSD EXIF during extraction 2026-10-22 14:59:39 +01:00
ronibalthazar c8390afbd2 Fail closed on malformed PSD metadata during original export 2026-10-22 14:21:32 +01:00
ronibalthazar 6413bccbfd #2519: read PSD and PSB metadata 2026-10-22 11:20:53 +01:00
NetterSilverandClaude Opus 5.5 0cdbc6c15a Write exact 1/N EXIF ExposureTime for fast shutter speeds
Found while testing crates/meta/src/exif.rs:483 (write_exif, the `r` helper; via lightcraft_tiff::writer::rational at crates/tiff/src/writer.rs:261)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 09:25:00 +00:00
ronibalthazar 8d4e6e1e01 LR-META-XMP: preserve foreign namespace collisions 2026-10-22 06:42:11 +01:00
aiongg 8daa25c4c9 Merge track r2-t04 into next
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/target-app-parity.md
#	docs/whats-new.md
2026-10-22 02:53:42 +00:00
aiongg 70e9338085 Merge PR #1040: LR-IMP-FORMATS: decode Minolta MRW
docs/parity-checklist.md: header keeps the newer date; LR-IMP-FORMATS row keeps both sides' changes.
docs/raw-parity.md: header change note names both sides' changes.
docs/raw/coverage.tsv: merged cleanly; same rows as both sides, no duplicates (regenerated on the combined branch).
2026-10-22 02:42:05 +00:00
aiongg b1d36f6b36 Merge PR #2417: Carry rounded GPS minutes into degrees in XMP writer 2026-10-22 02:41:08 +00:00
aiongg dbf4b2b428 Merge PR #2409: Drop non-finite elevation values in the GPX parser 2026-10-22 02:41:08 +00:00
aiongg 6e7d733ab5 Merge PR #2309: Fix trailing dot in All Metadata rational formatter 2026-10-22 02:40:48 +00:00
aiongg 5cd029a40c Merge PR #1628: #1617: placeholder GPS block (NUL refs, 0/0 rationals) reads as no location 2026-10-22 02:39:34 +00:00
Dylan M. Taylor bfb23db2ca Use chrono instead of hand-rolled UTC conversion (#2457)
* Use chrono for UTC/calendar conversion instead of hand-rolled copies

Replaces the hand-rolled Howard Hinnant civil/days-from-civil copies and the
leap-year/days-in-month table with chrono as a workspace dependency:
- apps/lightcraft/src/logging.rs: timestamp (log line time)
- crates/catalog/src/dates.rs: civil + weekday
- crates/catalog/src/stacks.rs: iso_seconds (date -> seconds)
- crates/engine/src/cmd/face_recognize.rs: capture_secs (native only)
- crates/meta/src/datetime.rs: DateTime::unix_seconds and the valid() day check
Output formats and semantics are unchanged (pre-epoch clamp, day rollover that
normalize_iso relies on, leap-second tolerance, EXIF/XMP parsing). chrono uses
only std, so wasm stays safe. AGENTS.md/CLAUDE.md now say time goes through chrono.

Follow-up to storytold/cadcraft#1785

* Never panic on a clock outside chrono's range; keep 5-digit years unprefixed

chrono's From<SystemTime> unwraps, so a clock past year 262143 panicked
inside the logger. timestamp now uses the fallible DateTime::from_timestamp
and reads such a clock as the epoch, keeping the old pre-epoch clamp.

timestamp and catalog's civil write years past 9999 with {:04} like the old
{y:04} did, because %Y would add a '+'. Tests cover the out-of-range and
5-digit-year cases.

* Don't build an unrepresentable SystemTime in the far-future test

UNIX_EPOCH + Duration::from_secs(u64::MAX / 2) overflows Windows'
SystemTime and panicked before timestamp() ran, failing the Windows
test job. Build it with checked_add and only assert where it exists.

* time: preserve old out-of-range date conversion (no epoch fallback)

`DateTime::from_timestamp(...).unwrap_or_default()` silently reset any instant
past chrono's year range to 1970-01-01, changing output the old hand-rolled
Howard-Hinnant code converted exactly (e.g. 10_000_000_000_000 s was
318857-05-20 17:46:40, not the epoch). Restore the old conversion domain with a
cycle-based helper that routes the calendar through chrono (the calendar
repeats every 400 years = 146_097 days, so month/day/time come from chrono and
only the year is carried in i128, keeping every i64 exact).

- apps/lightcraft/src/logging.rs: `timestamp` via `utc_parts`; keep the
  pre-1970 epoch clamp; u64 secs past i64::MAX saturate, not epoch.
- crates/catalog/src/dates.rs: `civil` cycle-based (the callers stay inside
  0000-9999, but a far clock no longer resets to the epoch).

Pinned old outputs in tests:
  logging u64::MAX/2 s -> 292277026596-12-04T15:30:07.000Z (was epoch),
          10_000_000_000_000 s -> 318857-05-20T17:46:40.000Z
  civil 10_000_000_000_000 -> 318857-05-20T17:46:40,
        -10_000_000_000_000 -> -314918-08-13T06:13:20, i64 extremes

* Don't build a year-318857 SystemTime unchecked in the logging test

Windows' SystemTime only reaches about year 30828, so
UNIX_EPOCH + Duration::from_secs(10_000_000_000_000) panicked before
timestamp() ran and failed the Windows test job. Build it with
checked_add and assert where the platform can hold it. The year-10000
case fits everywhere and stays unconditional.

* Name the 400-year range extension as the one exception to the time rule

The rule said never to hand-roll calendar math, but the range fix
(shifting by whole 400-year Gregorian cycles so dates past chrono's year
range keep their old output) is cycle arithmetic, so the rule and the code
disagreed. The rule now names that exception: the library still computes
month, day and time, and only whole cycles are carried in the year.

* Say the library computes month and day; time-of-day is plain arithmetic

Narrows the range-extension wording: the date library computes the
calendar fields, and splitting seconds into hours, minutes and seconds
is ordinary arithmetic, which is what the code does.
2026-10-21 19:36:36 -04:00
NetterSilverandClaude Haiku 5.5 8fe440c759 Carry rounded GPS minutes into degrees in XMP writer
Found while testing crates/meta/src/xmp.rs:458 (fmt_gps_coord)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 19:59:03 +00:00
NetterSilverandClaude Haiku 5.5 81b8f0e598 Drop non-finite elevation values in the GPX parser
Found while testing crates/meta/src/gpx.rs:150

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 16:48:29 +00:00
Andrew FrahnandClaude Sonnet 5.5 5e1bac1264 Merge upstream/main into pr/longtail-raws
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 09:58:17 +11:00
NetterSilverandClaude Haiku 5.5 af0146c5e8 Fix trailing dot in All Metadata rational formatter
Found while testing crates/meta/src/tags.rs:148 (rational)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 13:33:41 +02:00
NetterSilverandClaude Haiku 5.5 a3d00301cf Write ISOSpeed extension tag for ISO above 65534 in write_exif
Found while testing crates/meta/src/exif.rs:455 (write_exif)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 05:16:03 +00:00
tecari0andClaude Opus 5 60843a0c66 #2284: document the HEIF reader's test helpers and the shared box readers
The pre-merge docstring check counts every function the diff touches. The
`be16` / `be32` / `be64` readers became crate-public here, and the HEIF tests
added helpers and cases without a line saying what each one covers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-18 21:15:49 -07:00
tecari0andClaude Opus 5 6c2ff558d9 #2284: read the Exif and XMP items of HEIF files
A HEIC imported with no capture time. `container::embedded` knew JPEG, PNG,
WebP, TIFF-based files, RAF and CR3, but no ISO base media still image, so
every HEIC produced an empty `Embedded` and carried no Exif at all (#1233).
With no date in the file and none in a sidecar, the import falls back to the
file's modification time, which LightCraft writes as UTC: a photo taken at
13:00 in UTC-5 was catalogued as 18:00, and one taken after 19:00 local time
landed in the next day's folder.

`meta::heif` reads the `meta` box's item information (`iinf`) and item
location (`iloc`) boxes and returns the `Exif` item's TIFF stream and the XMP
`mime` item. It parses the container only, so it is in every build and the
HEVC decoder stays behind the `heif` feature. The box walker `cr3` already
had is now shared instead of written twice.

Camera, lens, ISO, GPS and keywords come through with the date. HEIF puts
orientation in the container and its Exif tag only mirrors that, so the probe
still takes the orientation from the container; a test checks that a HEIC
whose Exif says 6 imports upright.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-18 20:57:13 -07:00
Brandon Thomas dee8358c38 Merge PR #1838: Drop out-of-range GPS coordinates read from XMP 2026-10-18 09:20:40 -04:00