aleleba/lightcraft
1
0
Fork 0
mirror of https://github.com/storytold/lightcraft.git synced 2026-11-01 05:57:09 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into fix/merge-preview-after-failure-2708

Browse Source 
# Conflicts:
#	docs/parity-checklist.md
This commit is contained in:
Bruno-Alumn
2026-10-30 21:22:36 +01:00
parent 2d15ae7718 8aabf61449
commit 8d59ca335e
238 changed files with 8940 additions and 5252 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/packaging-lint.yml
+24
View File
@@ -37,3 +37,27 @@ jobs:
run: |
./packaging/windows/test-install-folder.ps1 -Arch x64
./packaging/windows/test-install-folder.ps1 -Arch x86
windows-install-policy:
timeout-minutes: 10
name: Windows installation policy (no app build)
runs-on: windows-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install the pinned packaging toolchain
shell: pwsh
run: |
$tools = Join-Path $env:RUNNER_TEMP 'craft-install-policy-wix'
dotnet tool install --tool-path $tools wix --version 5.0.2
if ($LASTEXITCODE -ne 0) { throw 'WiX installation failed.' }
$tools >> $env:GITHUB_PATH
$policy = Get-Content -Raw -LiteralPath packaging/windows/install-policy.json | ConvertFrom-Json
foreach ($extension in @($policy.extensions)) {
& (Join-Path $tools 'wix.exe') extension add -g "WixToolset.$extension.wixext/5.0.2"
if ($LASTEXITCODE -ne 0) { throw "WiX extension $extension installation failed." }
}
- name: Build and check non-executable installer fixtures
shell: pwsh
run: ./packaging/windows/test-install-policy.ps1 -OutputDirectory (Join-Path $env:RUNNER_TEMP 'craft-install-policy-tests')
.github/workflows/windows-arm64.yml
+2
View File
@@ -32,3 +32,5 @@ jobs:
with:
app-name: lightcraft
app-dir: LightCraft
install-relative-path: Programs/ArtCraft/LightCraft
install-scope: perUser
AGENTS.md
+6 -1
View File
@@ -51,7 +51,7 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Last-resort guard:** a panic hook plus `catch_unwind` around command dispatch and import/export turns an escaped
panic into an error dialog and keeps the document. It's a safety net, not a licence; keep `panic = "unwind"` on native.
- **Prove it:** every crash fix lands with a small synthetic regression test that panicked before the fix.
- Enforced by clippy: root `clippy.toml` allows unwrap/expect/panic/indexing in tests only, and every production crate
- Enforced by clippy: root `clippy.toml` allows unwrap/expect/panic in tests only; indexing is not denied. Every production crate
root (`lib.rs`, each binary's `main.rs`) carries `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic,
clippy::unimplemented, clippy::todo, clippy::unreachable)]`. Not `[workspace.lints]`: those would also hit
integration tests, examples and benches. New crates start with the attribute.
@@ -156,3 +156,8 @@ evaluates it (`.github/workflows/nix.yml`), so the build may lag behind the work
GitHub profiles). Never hand-edit `contributors.json`.
- Maintainers refresh the stats with `python3 ../../craftrules/scripts/contributors.py .` (it also
re-verifies who wrote each `people.toml` entry; `--check` only verifies).
## CI workflows
- Workflows in `.github/workflows/` call the shared workflows in [`storytold/craft-common-actions`](https://github.com/storytold/craft-common-actions) at `@main`, with their inputs and secrets. No other jobs: if no shared workflow does what's needed, it doesn't go here.
- Don't add or change CI in a feature or fix PR. If the existing workflows can't build or test a change, link an existing craft-common-actions issue or PR, or open a new issue, or a PR only for a small fix that is safe for every app and does not increase CI time.
- Some workflow files here still run their own jobs. Don't add to them. Moving a job into craft-common-actions is its own task, never part of another PR, and the job stays here until a shared workflow runs it.
CLAUDE.md
+12 -4
View File
@@ -52,10 +52,13 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Last-resort guard:** a panic hook plus `catch_unwind` around command dispatch and import/export turns an escaped
panic into an error dialog and keeps the document. It's a safety net, not a licence; keep `panic = "unwind"` on native.
- **Prove it:** every crash fix lands with a small synthetic regression test that panicked before the fix.
- Enforced by clippy: root `clippy.toml` allows unwrap/expect/panic/indexing in tests only, and every production crate
root (`lib.rs`, each binary's `main.rs`) carries `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic,
clippy::unimplemented, clippy::todo, clippy::unreachable)]`. Not `[workspace.lints]`: those would also hit
integration tests, examples and benches. New crates start with the attribute.
- Enforced by clippy: root `clippy.toml` keeps unwrap/expect/panic allowances scoped to tests; indexing is not denied. The workspace
denies `cognitive_complexity`, `too_many_arguments` and `wildcard_imports`; members inherit them with
`[lints] workspace = true`, and `sysmem` repeats those denies in its local lint table. Every production crate root
(`lib.rs`, each binary's `main.rs`) carries `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic,
clippy::unimplemented, clippy::todo, clippy::unreachable)]`. Every new workspace crate must include
`[lints] workspace = true` in `Cargo.toml` to inherit `[workspace.lints]`, and new production crate roots include
the attribute before modules or other crate items; crate-level `//!` documentation may precede it.
## Non-negotiables
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, profiles (DCP), lens profiles (LCP), camera matrices, fonts. Observation of the installed Lightroom is read-only (it syncs the user's personal library: never import/edit/rate/delete there). Never copy GPL/LGPL/AGPL code (darktable, RawTherapee, ART, LibRaw, rawspeed, rawloader, rawler, lensfun, dcraw-derived GPL code…).
@@ -149,3 +152,8 @@ photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`,
`nix/package.nix` = the Nix package (`nix build` builds both binaries with the craft-fonts input, installs the
desktop file/icons/AppStream metadata and runs `cargo test --workspace`; `nix develop` = dev shell). Community-maintained; CI only
evaluates it (`.github/workflows/nix.yml`), so the build may lag behind the workspace; see README → Quick start.
## CI workflows
- Workflows in `.github/workflows/` call the shared workflows in [`storytold/craft-common-actions`](https://github.com/storytold/craft-common-actions) at `@main`, with their inputs and secrets. No other jobs: if no shared workflow does what's needed, it doesn't go here.
- Don't add or change CI in a feature or fix PR. If the existing workflows can't build or test a change, link an existing craft-common-actions issue or PR, or open a new issue, or a PR only for a small fix that is safe for every app and does not increase CI time.
- Some workflow files here still run their own jobs. Don't add to them. Moving a job into craft-common-actions is its own task, never part of another PR, and the job stays here until a shared workflow runs it.
Cargo.lock
Generated
+54 -51
View File
@@ -888,7 +888,7 @@ dependencies = [
"safetensors 0.7.0",
"thiserror 2.0.21",
"yoke 0.8.3",
"zip 7.2.0",
"zip 7.4.0",
]
[[package]]
@@ -1238,13 +1238,14 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0"
source = "git+https://github.com/storytold/craft-libs?rev=04aa1703b32d299e54f78fcde4182f2b393a9116#04aa1703b32d299e54f78fcde4182f2b393a9116"
dependencies = [
"chrono",
"dunce",
"egui",
"flate2",
"getrandom 0.3.4",
"image",
"log",
"plist",
"semver",
@@ -1257,7 +1258,7 @@ dependencies = [
"verifysign",
"winreg",
"winsafe",
"zip 9.1.0",
"zip 9.1.3",
]
[[package]]
@@ -1870,7 +1871,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -3535,7 +3536,7 @@ dependencies = [
[[package]]
name = "lightcraft"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"chrono",
"craft-update",
@@ -3562,7 +3563,7 @@ dependencies = [
[[package]]
name = "lightcraft-catalog"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"chrono",
"crc32fast",
@@ -3570,17 +3571,19 @@ dependencies = [
"lightcraft-geom",
"lightcraft-meta",
"lightcraft-raw",
"lightcraft-testkit",
"log",
"proptest",
"serde",
"serde_json",
"thiserror 2.0.21",
"web-time",
"winapi-util",
]
[[package]]
name = "lightcraft-cli"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"dhat",
"lightcraft-codecs",
@@ -3596,7 +3599,7 @@ dependencies = [
[[package]]
name = "lightcraft-codecs"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"avif-serialize",
"bytemuck",
@@ -3627,7 +3630,7 @@ dependencies = [
[[package]]
name = "lightcraft-color"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-geom",
"proptest",
@@ -3636,7 +3639,7 @@ dependencies = [
[[package]]
name = "lightcraft-denoise"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"faer-core",
"lightcraft-denoise-core",
@@ -3653,7 +3656,7 @@ dependencies = [
[[package]]
name = "lightcraft-denoise-core"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"dhat",
"lightcraft-raster",
@@ -3669,7 +3672,7 @@ dependencies = [
[[package]]
name = "lightcraft-develop"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-color",
"lightcraft-geom",
@@ -3681,7 +3684,7 @@ dependencies = [
[[package]]
name = "lightcraft-engine"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"ab_glyph",
"chrono",
@@ -3725,7 +3728,7 @@ dependencies = [
[[package]]
name = "lightcraft-faces"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-testkit",
"matrixmultiply",
@@ -3740,7 +3743,7 @@ dependencies = [
[[package]]
name = "lightcraft-fetch"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"log",
"rustls",
@@ -3751,7 +3754,7 @@ dependencies = [
[[package]]
name = "lightcraft-geocode"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"miniz_oxide 0.9.1",
"serde",
@@ -3759,7 +3762,7 @@ dependencies = [
[[package]]
name = "lightcraft-geom"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"proptest",
"serde",
@@ -3767,7 +3770,7 @@ dependencies = [
[[package]]
name = "lightcraft-gpu"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"bytemuck",
"lightcraft-color",
@@ -3787,7 +3790,7 @@ dependencies = [
[[package]]
name = "lightcraft-heif"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"heic-rs",
"rayon",
@@ -3795,14 +3798,14 @@ dependencies = [
[[package]]
name = "lightcraft-inpaint"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"rayon",
]
[[package]]
name = "lightcraft-lensdb"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-develop",
"lightcraft-geom",
@@ -3811,7 +3814,7 @@ dependencies = [
[[package]]
name = "lightcraft-mcp"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-codecs",
"lightcraft-engine",
@@ -3822,7 +3825,7 @@ dependencies = [
[[package]]
name = "lightcraft-merge"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-codecs",
"lightcraft-color",
@@ -3838,7 +3841,7 @@ dependencies = [
[[package]]
name = "lightcraft-meta"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"chrono",
"lightcraft-geom",
@@ -3851,7 +3854,7 @@ dependencies = [
[[package]]
name = "lightcraft-pipeline"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"dhat",
"image",
@@ -3870,7 +3873,7 @@ dependencies = [
[[package]]
name = "lightcraft-preview"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"jpeg-encoder",
"lightcraft-raster",
@@ -3882,7 +3885,7 @@ dependencies = [
[[package]]
name = "lightcraft-raster"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"libc",
"lightcraft-color",
@@ -3894,7 +3897,7 @@ dependencies = [
[[package]]
name = "lightcraft-raw"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"dhat",
"jpeg-encoder",
@@ -3918,7 +3921,7 @@ dependencies = [
[[package]]
name = "lightcraft-scenes"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"image",
"lightcraft-color",
@@ -3929,7 +3932,7 @@ dependencies = [
[[package]]
name = "lightcraft-segment"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"candle-core",
"candle-nn",
@@ -3943,7 +3946,7 @@ dependencies = [
[[package]]
name = "lightcraft-slideshow"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"lightcraft-raster",
"serde",
@@ -3952,15 +3955,15 @@ dependencies = [
[[package]]
name = "lightcraft-sysmem"
version = "0.14.0"
version = "0.15.0"
[[package]]
name = "lightcraft-testkit"
version = "0.14.0"
version = "0.15.0"
[[package]]
name = "lightcraft-tiff"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"proptest",
"serde",
@@ -3969,7 +3972,7 @@ dependencies = [
[[package]]
name = "lightcraft-ui-egui"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"craft-ui",
"craft-update",
@@ -4000,7 +4003,7 @@ dependencies = [
[[package]]
name = "lightcraft-web"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"crc32fast",
"eframe",
@@ -4881,7 +4884,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d8fae84b431384b68627d0f9b3b1245fcf9f46f6c0e3dc902e9dce64edd1967"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -5840,7 +5843,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys 0.12.1",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -5853,7 +5856,7 @@ dependencies = [
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki 0.103.15",
"rustls-webpki 0.103.16",
"subtle",
"zeroize",
]
@@ -5893,11 +5896,11 @@ dependencies = [
"rustls",
"rustls-native-certs",
"rustls-platform-verifier-android",
"rustls-webpki 0.103.15",
"rustls-webpki 0.103.16",
"security-framework",
"security-framework-sys",
"webpki-root-certs",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -5949,9 +5952,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.15"
version = "0.103.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2"
checksum = "4443b1b6d39d7c663139942cc4558262562d2c84123bb6282ef177009ca8b281"
dependencies = [
"ring",
"rustls-pki-types",
@@ -6476,10 +6479,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.3",
"getrandom 0.3.4",
"once_cell",
"rustix 1.1.5",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -7402,7 +7405,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -7817,7 +7820,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"brotli",
"flate2",
@@ -8084,9 +8087,9 @@ dependencies = [
[[package]]
name = "zip"
version = "7.2.0"
version = "7.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c42e33efc22a0650c311c2ef19115ce232583abbe80850bc8b66509ebef02de0"
checksum = "cc12baa6db2b15a140161ce53d72209dacea594230798c24774139b54ecaa980"
dependencies = [
"crc32fast",
"indexmap",
@@ -8096,9 +8099,9 @@ dependencies = [
[[package]]
name = "zip"
version = "9.1.0"
version = "9.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2886b00003a72c4ccee3f42cfc34908d33b2508a74d827dc2a4244706475c704"
checksum = "1156b39d142a34b999c2310a50455e0cb3760bf7efad0122f796d813bee26ccb"
dependencies = [
"crc32fast",
"flate2",
Cargo.toml
+7 -3
View File
@@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.14.0"
version = "0.15.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -12,7 +12,8 @@ repository = "https://github.com/storytold/lightcraft"
[workspace.dependencies]
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0", features = ["serde"] }
craft-docpath = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0" }
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0", features = ["egui"] }
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "04aa1703b32d299e54f78fcde4182f2b393a9116", features = ["egui"] }
craft-window = { git = "https://github.com/storytold/craft-libs", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0", features = ["egui"] }
# internal
lightcraft-denoise-core = { path = "crates/denoise-core" }
@@ -54,6 +55,7 @@ log = "0.4"
# All date/time conversion goes through chrono (no hand-rolled calendar code beyond the 400-year range extension; AGENTS.md).
chrono = { version = "0.4.45", default-features = false, features = ["std"] }
rayon = "1"
winapi-util = "0.1" # lightcraft-catalog reads a file's volume serial number and file index with it on Windows
proptest = "1"
bytemuck = { version = "1", features = ["derive"] }
crc32fast = "1"
@@ -121,9 +123,11 @@ web-sys = "0.3"
unsafe_code = "deny"
[workspace.lints.clippy]
cognitive_complexity = "deny"
too_many_arguments = "deny"
wildcard_imports = "deny"
dbg_macro = "warn"
todo = "warn"
too_many_arguments = "allow"
needless_range_loop = "allow"
type_complexity = "allow"
excessive_precision = "allow"
README.md
+13 -2
View File
@@ -563,9 +563,20 @@ Questions, ideas or a bug you'd like to talk through first? Bring them to [Disco
| arm64 (Snapdragon and other ARM PCs) | `lightcraft-<ver>-windows-arm64.msi` | `lightcraft-<ver>-windows-arm64-portable.zip` |
| x86 (32-bit) | `lightcraft-<ver>-windows-x86.msi` | `lightcraft-<ver>-windows-x86-portable.zip` |
Installers and executables are code-signed. The installer asks where to install (`C:\Program Files\LightCraft` by
default; upgrades stay in the folder you chose) and ends on a page confirming LightCraft was installed, with an
Installers and executables are code-signed. The installer asks who to install for and where (for your account only, in
`%LOCALAPPDATA%\Programs\ArtCraft\LightCraft`, by default; "Everyone on this computer" uses
`C:\Program Files\ArtCraft\LightCraft` (`Program Files (x86)` for the 32-bit build); upgrades stay in the folder you chose) and ends on a page confirming LightCraft was installed, with an
option to start it. For unattended installs: `msiexec /i lightcraft-<ver>-windows-x64.msi /qn INSTALLFOLDER="D:\Apps\LightCraft\"`.
A silent install is per-user by default. `INSTALLFOLDER` chooses the folder, not who it is installed for;
add `ALLUSERS=1`, from an elevated terminal or deployment tool, for an all-users install.
**Update or uninstall asks for an old `.msi`, or stops with error 1612 or 1714?** Windows removes the installed version
with its own cached copy of that version's installer; if the copy is gone, it asks for the original file and stops
(an update then ends with error 1603). Download the release MSI of the version and build you have installed from
[all releases](https://github.com/storytold/lightcraft/releases) and run
`msiexec /fv "<path to that .msi>" /qb+ /norestart`, approving the permission prompt if Windows shows one. That restores
the cached copy; then retry the update or uninstall. Opening the MSI, or choosing Repair or Uninstall on it, may still
fail. Error 1603 has other causes too; if this doesn't help, open an issue with the error message.
### macOS
README.zh-hans.md
+4 -2
View File
@@ -484,9 +484,11 @@ egui 前端是一个可替换的 crate；它下面没有任何东西知道 UI
| arm64（骁龙及其他 ARM PC） | `lightcraft-<ver>-windows-arm64.msi` | `lightcraft-<ver>-windows-arm64-portable.zip` |
| x86（32 位） | `lightcraft-<ver>-windows-x86.msi` | `lightcraft-<ver>-windows-x86-portable.zip` |
安装程序和可执行文件都有代码签名。安装程序会询问安装位置（默认
`C:\Program Files\LightCraft`；升级会留在你选择的文件夹），结束时有一页确认 LightCraft 已安装，
安装程序和可执行文件都有代码签名。安装程序会询问为谁安装以及安装位置（默认仅为当前账户，安装到
`%LOCALAPPDATA%\Programs\ArtCraft\LightCraft`；选择“这台计算机上的所有人”则安装到
`C:\Program Files\ArtCraft\LightCraft`（32 位版本为 `Program Files (x86)`）；升级会留在你选择的文件夹），结束时有一页确认 LightCraft 已安装，
并可选择启动它。无人值守安装：`msiexec /i lightcraft-<ver>-windows-x64.msi /qn INSTALLFOLDER="D:\Apps\LightCraft\"`。
静默安装默认仅为当前用户安装。`INSTALLFOLDER` 只决定安装文件夹，不决定为谁安装；若要为所有用户安装，请在提升权限的终端或部署工具中加上 `ALLUSERS=1`。
### macOS
README.zh-hant.md
+4 -2
View File
@@ -514,9 +514,11 @@ egui 前端是一個可替換的 crate；它之下沒有任何東西知道 UI
| arm64（Snapdragon 及其他 ARM PC） | `lightcraft-<ver>-windows-arm64.msi` | `lightcraft-<ver>-windows-arm64-portable.zip` |
| x86（32 位元） | `lightcraft-<ver>-windows-x86.msi` | `lightcraft-<ver>-windows-x86-portable.zip` |
安裝程式與執行檔都有程式碼簽章。安裝程式會詢問安裝位置（預設為
`C:\Program Files\LightCraft`；升級時會留在你選擇的資料夾），結束時會有一頁確認 LightCraft 已安裝，
安裝程式與執行檔都有程式碼簽章。安裝程式會詢問為誰安裝以及安裝位置（預設僅為目前帳戶，安裝到
`%LOCALAPPDATA%\Programs\ArtCraft\LightCraft`；選擇「這台電腦上的所有人」則安裝到
`C:\Program Files\ArtCraft\LightCraft`（32 位元版本為 `Program Files (x86)`）；升級時會留在你選擇的資料夾），結束時會有一頁確認 LightCraft 已安裝，
並可選擇立即啟動。無訊息安裝：`msiexec /i lightcraft-<ver>-windows-x64.msi /qn INSTALLFOLDER="D:\Apps\LightCraft\"`。
無訊息安裝預設僅為目前使用者安裝。`INSTALLFOLDER` 只決定安裝資料夾，不決定為誰安裝；若要為所有使用者安裝，請在提升權限的終端機或部署工具中加上 `ALLUSERS=1`。
### macOS
ROADMAP.md
+3 -2
View File
@@ -2,7 +2,7 @@
**Stage: alpha** · next: beta, ~14 points and ~340–600 h away
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (Print: Japanese photo and postcard paper sizes — L判, 2L判, KG判, はがき, 往復はがき, 六切, 四切, 名刺; Minolta MRW decoded, 6 bodies verified; RAW: 83% of Adobe's models decode, 38% verified after the camera-coverage sweep #1327; German catalog 91% → 97%: interface strings and What's New lines translated) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-24 · **Change:** minor (the lens profile database priority links its tracking issue #329; Print: Japanese photo and postcard paper sizes — L判, 2L判, KG判, はがき, 往復はがき, 六切, 四切, 名刺; Minolta MRW decoded, 6 bodies verified; RAW: 83% of Adobe's models decode, 38% verified after the camera-coverage sweep #1327; German catalog 91% → 97%: interface strings and What's New lines translated) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
One-page summary of how close LightCraft is to Adobe Lightroom. Agents: read this, then pick work
from [docs/gaps.md](docs/gaps.md). The full assessment, method and evidence are in
@@ -108,7 +108,7 @@ Ranked toward beta; detail in [docs/roadmap.md](docs/roadmap.md), gaps in [docs/
4. ~~Full-resolution zoom~~ (done 2026-10-12)
5. AI masks: Subject / Sky / People (40–70 h + model decision)
6. Compressed ORF and HE NEF (28–55 h)
7. Lens profile database of our own (40–80 h)
7. Lens profile database of our own (40–80 h, #329)
8. Tests in CI on macOS, Windows and Linux (6–12 h)
9. Per-model raw verification (20–35 h)
@@ -191,6 +191,7 @@ Ranked toward beta; detail in [docs/roadmap.md](docs/roadmap.md), gaps in [docs/
| Date | Change | Summary |
|---|---|---|
| 2026-10-24 | minor | Priority 7 (lens profile database) links its tracking issue #329; headline numbers unchanged |
| 2026-10-22 | minor | Print: Japanese photo and postcard paper sizes (L判, 2L判, KG判, はがき, 往復はがき, 六切, 四切, 名刺) in the page setup, Print dialog and PDF output; headline numbers unchanged |
| 2026-10-19 | minor | Indonesian interface (`id`): every plain and formats message, plus 69 command labels, locale, menu command and tests; What's New left in English; Localization ~55% → ~64% (mean of the 12 key languages), 8 of 12 present |
| 2026-10-12 | minor | Italian interface (`it`, #1192): 2,201 of 2,398 messages, locale, menu command and tests; headline numbers unchanged |
apps/lightcraft-web/src/web.rs
+1
View File
@@ -175,6 +175,7 @@ fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen
let (backup_backend, backup_files) = (backend.clone(), files);
let restore_backend = backend.clone();
Services {
local_places: None,
picker: None,
pick_display_profile: None,
screen_profile: None,
apps/lightcraft/src/gpu_crash.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use super::prefs::config_dir;
use super::*;
use super::window_gpu;
/// The window's renderer (egui-wgpu): eframe's defaults, with LightCraft's backend choice — DX12
/// alone on Windows unless `LIGHTCRAFT_GPU_BACKEND` / `WGPU_BACKEND` say otherwise (issue #136:
apps/lightcraft/src/main.rs
+1 -1
View File
@@ -136,7 +136,7 @@ const EXIT_WAIT: std::time::Duration = std::time::Duration::from_secs(2);
fn main() -> eframe::Result {
// Windows: the update helper is a copy of this program started with `--finish-update`.
#[cfg(windows)]
if let Some(result) = craft_update::self_update::finish(&craft_update::suite::LIGHTCRAFT, &std::env::args_os().skip(1).collect::<Vec<_>>()) {
if let Some(result) = craft_update::self_update::finish(&lightcraft_ui_egui::UPDATE_APP, &std::env::args_os().skip(1).collect::<Vec<_>>()) {
std::process::exit(i32::from(result.is_err()));
}
// First, so every start-up record is kept for the log file (`logging`).
apps/lightcraft/src/native_menu.rs
+31 -5
View File
@@ -295,12 +295,20 @@ fn label_text(s: &str) -> String {
lightcraft_ui_egui::i18n::tr(s).replace('&', "&&")
}
/// Drop a trailing separator (left behind when an item moved to the app menu).
fn tidy_separators(mut nodes: Vec<MenuNode>) -> Vec<MenuNode> {
while matches!(nodes.last(), Some(MenuNode::Separator)) {
nodes.pop();
/// Drop the separators left behind when an item moved to the app menu: a leading one, a trailing
/// one and the second of two in a row.
fn tidy_separators(nodes: Vec<MenuNode>) -> Vec<MenuNode> {
let mut out: Vec<MenuNode> = Vec::with_capacity(nodes.len());
for node in nodes {
if node == MenuNode::Separator && out.last().is_none_or(|last| *last == MenuNode::Separator) {
continue;
}
out.push(node);
}
nodes
while out.last() == Some(&MenuNode::Separator) {
out.pop();
}
out
}
impl NativeMenu {
@@ -570,6 +578,24 @@ impl NativeMenu {
mod tests {
use super::*;
/// An item that moves to the app menu (Settings…) can sit between two separators, or open or
/// close the menu's list: the rows left behind keep at most one separator between them.
#[test]
fn rows_left_after_a_move_keep_single_separators() {
let item = |id: &str| MenuNode::Item {
id: id.to_string(),
label: id.to_string(),
shortcut: None,
enabled: true,
checked: None,
params: Value::Null,
};
let sep = || MenuNode::Separator;
let tidy = tidy_separators(vec![sep(), item("a"), sep(), sep(), item("b"), sep(), sep()]);
assert_eq!(tidy, vec![item("a"), sep(), item("b")]);
assert_eq!(tidy_separators(vec![sep(), sep()]), Vec::<MenuNode>::new());
}
/// The menu model's titles stay English, so only the language can tell the native menu to
/// rebuild its titles (and the macOS app menu): every language switch — also between two CJK
/// languages — must change the structure key.
apps/lightcraft/src/prefs.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{LibraryProblem, LightcraftApp, UiState};
/// Where the system keeps monitor profiles, for the Choose Profile dialog to start in: the user's
/// own profiles first (Linux: colord / DisplayCAL; macOS: ColorSync), then the system's.
apps/lightcraft/src/session.rs
+1
View File
@@ -11,6 +11,7 @@ pub(crate) fn services(cc: &eframe::CreationContext<'_>, log_file: Option<&std::
#[cfg(any(target_os = "linux", target_os = "freebsd"))]
let _ = DIALOG_PARENT.set(rfd::FileDialog::new().set_parent(cc));
Services {
local_places: None,
// Commands' file dialogs run on their own thread (#191): a dialog on the UI thread
// stopped the window answering the compositor, which then reported the app as hung.
picker: Some(Box::new(move |req: lightcraft_ui_egui::pick::PickRequest| {
clippy.toml
+1
View File
@@ -5,3 +5,4 @@ allow-indexing-slicing-in-tests = true
disallowed-types = [
{ path = "egui::ComboBox", reason = "use craft_ui::ComboBox: egui's combo box menu can't grow back while open (egui#5225)" },
]
cognitive-complexity-threshold = 30
contributors/people.toml
+3
View File
@@ -74,3 +74,6 @@ sync_github_name = true
[people.dotCooCoo]
real_name = "Robert Lee"
sync_github_name = true
[people.dylanmtaylor]
sync_github_name = true
crates/catalog/Cargo.toml
+4
View File
@@ -20,6 +20,10 @@ log = { workspace = true }
chrono = { workspace = true }
web-time = { workspace = true }
[target.'cfg(windows)'.dependencies]
winapi-util = { workspace = true }
[dev-dependencies]
lightcraft-geom = { workspace = true }
lightcraft-testkit = { workspace = true }
proptest = { workspace = true }
crates/catalog/examples/persist_bench.rs
+1 -1
View File
@@ -14,7 +14,7 @@
use std::sync::Arc;
use std::time::Instant;
use lightcraft_catalog::*;
use lightcraft_catalog::{Catalog, DEFAULT_FORGET_DAYS, FsStore, HistoryStep, Journal, MediaKind, Op, Photo, PhotoId, Source, folder_of};
use lightcraft_develop::DevelopSettings;
fn ms(t: Instant) -> f64 {
crates/catalog/src/query.rs
+13 -11
View File
@@ -20,7 +20,7 @@ pub enum RatingOp {
#[serde(default, rename_all = "camelCase")]
pub struct Filter {
/// Free-text search: every token must match filename, title, caption, keywords, camera, lens,
/// location or format. Tokens like `rating:3`, `flag:pick`, `iso:>800`, `camera:x2` are fielded.
/// location or format. Tokens like `rating:3`, `rating:>=3`, `flag:pick`, `iso:>800`, `camera:x2` are fielded.
pub text: String,
pub rating: u8,
pub rating_op: RatingOp,
@@ -179,23 +179,25 @@ fn token_matches(p: &Photo, tok: &str) -> bool {
// a token that starts with a quote is a phrase, even with a colon in it
if let Some((field, quoted)) = t.split_once(':').filter(|_| !t.starts_with('"')) {
let val = unquote(quoted);
let num = |s: &str| -> Option<(char, f64)> {
let (op, rest) = match s.chars().next()? {
c @ ('>' | '<' | '=') => (c, &s[1..]),
_ => ('=', s),
};
rest.parse().ok().map(|v| (op, v))
// `>=` / `<=` before `>` / `<`: otherwise `rating:>=3` reads `=3` as the number and matches nothing
let num = |s: &str| -> Option<(&str, f64)> {
let op = [">=", "<=", ">", "<", "="].into_iter().find(|op| s.starts_with(op)).unwrap_or("");
s.get(op.len()..)?.parse().ok().map(|v| (op, v))
};
let cmp = |x: f64, s: &str| match num(s) {
Some(('>', v)) => x > v,
Some(('<', v)) => x < v,
Some((">", v)) => x > v,
Some(("<", v)) => x < v,
Some((">=", v)) => x >= v,
Some(("<=", v)) => x <= v,
Some((_, v)) => (x - v).abs() < 1e-9,
None => false,
};
// Fields stored as `f32` compare at f32 precision: 2.8f32 widened to f64 is not 2.8.
let cmp32 = |x: f32, s: &str| match num(s) {
Some(('>', v)) => x > v as f32,
Some(('<', v)) => x < v as f32,
Some((">", v)) => x > v as f32,
Some(("<", v)) => x < v as f32,
Some((">=", v)) => x >= v as f32,
Some(("<=", v)) => x <= v as f32,
Some((_, v)) => x == v as f32,
None => false,
};
crates/catalog/src/safe_file.rs
+108 -6
View File
@@ -23,6 +23,7 @@
//! replacing anything** (a hard link, which fails on a taken name; where the volume has no hard
//! links, an exclusive create that claims the name, then a rename over that placeholder).
//! - [`same_file`] tells whether two paths name the same file on disk (device + inode on Unix,
//! volume serial number + file index on Windows,
//! resolved paths elsewhere), for "is this target an original?" checks.
use std::cell::Cell;
@@ -267,26 +268,75 @@ fn same_content(path: &Path, data: &[u8]) -> io::Result<bool> {
}
}
/// Do `a` and `b` name the same existing file (through links, `..`, case-insensitive names)?
/// Do `a` and `b` name the same existing file (through hard links, symlinks, `..`,
/// case-insensitive names)?
pub fn same_file(a: &Path, b: &Path) -> bool {
let (Ok(ma), Ok(mb)) = (fs::metadata(a), fs::metadata(b)) else { return false };
compare_files(a, b, false).unwrap_or(false)
}
/// [`same_file`] for a check that must not let an unknown identity pass: `Err` when the metadata
/// of either path cannot be read for a reason other than the path not existing (for example a
/// folder it cannot enter), or when both paths exist with the same length but the file identity
/// of either cannot be read (on Windows, when either file cannot be opened or reports file index 0). A path that does not
/// exist is still `Ok(false)`.
pub fn same_file_strict(a: &Path, b: &Path) -> io::Result<bool> {
compare_files(a, b, true)
}
fn compare_files(a: &Path, b: &Path, strict: bool) -> io::Result<bool> {
let missing = |e: &io::Error| matches!(e.kind(), io::ErrorKind::NotFound | io::ErrorKind::NotADirectory);
let (ma, mb) = match (fs::metadata(a), fs::metadata(b)) {
(Ok(ma), Ok(mb)) => (ma, mb),
(Err(e), _) | (_, Err(e)) if strict && !missing(&e) => return Err(e),
_ => return Ok(false),
};
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
ma.dev() == mb.dev() && ma.ino() == mb.ino()
Ok(ma.dev() == mb.dev() && ma.ino() == mb.ino())
}
#[cfg(not(unix))]
{
if ma.len() != mb.len() {
return false;
return Ok(false);
}
match (fs::canonicalize(a), fs::canonicalize(b)) {
// A hard link has its own path but shares the volume serial number and file index with
// its original. A file index of 0 produces an error in strict mode. Lenient mode compares resolved paths.
// On ReFS two files can share the 64-bit file index, so the lengths are compared first.
#[cfg(windows)]
{
match file_ids(a, b).and_then(|(ia, ib)| same_file_id(ia, ib, strict)) {
Ok(true) => return Ok(true),
Ok(_) => {}
Err(e) if strict => return Err(e),
Err(_) => {}
}
}
Ok(match (fs::canonicalize(a), fs::canonicalize(b)) {
(Ok(x), Ok(y)) => x == y || x.to_string_lossy().to_lowercase() == y.to_string_lossy().to_lowercase(),
_ => false,
}
})
}
}
/// The volume serial number and file index of `a` and `b`, read while both files are open.
#[cfg(windows)]
fn file_ids(a: &Path, b: &Path) -> io::Result<((u64, u64), (u64, u64))> {
use winapi_util::{Handle, file::information};
let (ha, hb) = (Handle::from_path_any(a)?, Handle::from_path_any(b)?);
let (ia, ib) = (information(&ha)?, information(&hb)?);
Ok(((ia.volume_serial_number(), ia.file_index()), (ib.volume_serial_number(), ib.file_index())))
}
/// The comparison uses each file's volume serial number and file index. Strict mode returns an error for file index 0.
#[cfg(any(windows, test))]
fn same_file_id(a: (u64, u64), b: (u64, u64), strict: bool) -> io::Result<bool> {
if strict && (a.1 == 0 || b.1 == 0) {
return Err(io::Error::other("file identity is unavailable"));
}
Ok(a.1 != 0 && b.1 != 0 && a == b)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -397,6 +447,29 @@ mod tests {
let _ = fs::remove_dir_all(&d);
}
#[test]
fn same_file_sees_a_hard_link_under_another_name() {
let d = temp("same-link");
let p = d.join("IMG_1.jpg");
fs::write(&p, b"original").unwrap();
let linked = d.join("linked.png");
match fs::hard_link(&p, &linked) {
Ok(()) => {}
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => panic!("{}: {e}", linked.display()),
// no hard links on this volume (FAT/exFAT, some shares), as in `publish_new`
Err(e) => {
let _ = fs::remove_dir_all(&d);
lightcraft_testkit::skip!(System, "no hard links in {}: {e}", d.display());
return;
}
}
fs::write(d.join("copy.png"), b"original").unwrap();
assert!(same_file(&p, &linked), "a hard link is the same file");
assert!(same_file(&linked, &p));
assert!(!same_file(&p, &d.join("copy.png")), "a copy with the same content and length is another file");
let _ = fs::remove_dir_all(&d);
}
#[test]
fn same_file_sees_through_other_spellings() {
let d = temp("same");
@@ -409,4 +482,33 @@ mod tests {
assert!(!same_file(&p, &d.join("missing.jpg")));
let _ = fs::remove_dir_all(&d);
}
#[test]
fn strict_comparison_reports_a_path_it_cannot_query() {
let d = temp("strict-query");
let p = d.join("x.jpg");
fs::write(&p, b"x").unwrap();
// A NUL byte makes the metadata lookup fail with an error other than "not found" on
// every platform, as a folder the process cannot enter does.
let unreadable = Path::new("IMG_1\0.jpg");
assert!(same_file_strict(unreadable, &p).is_err());
assert!(same_file_strict(&p, unreadable).is_err());
assert!(!same_file(unreadable, &p), "the lenient comparison still answers false");
assert!(!same_file_strict(&d.join("missing.jpg"), &p).unwrap(), "a missing original is another file");
assert!(!same_file_strict(&p.join("under-a-file.jpg"), &p).unwrap());
let _ = fs::remove_dir_all(&d);
}
#[test]
fn a_zero_file_index_is_not_a_file_identity() {
for strict in [false, true] {
assert!(same_file_id((7, 42), (7, 42), strict).unwrap());
assert!(!same_file_id((7, 42), (7, 43), strict).unwrap());
assert!(!same_file_id((7, 42), (8, 42), strict).unwrap());
}
for (a, b) in [((7, 0), (7, 0)), ((0, 0), (0, 0)), ((7, 42), (7, 0)), ((7, 0), (7, 42))] {
assert!(same_file_id(a, b, true).is_err(), "strict comparison must reject an unavailable identity: {a:?}, {b:?}");
assert!(!same_file_id(a, b, false).unwrap(), "lenient comparison can fall back to resolved paths");
}
}
}
crates/catalog/src/tests.rs
+52 -27
View File
@@ -163,6 +163,42 @@ fn search_tokens_match_f32_aperture_and_focal() {
assert!(q("f:>2.8").is_empty(), "f:>2.8");
assert!(q("focal:<5.6").is_empty(), "focal:<5.6");
assert_eq!(q("f:>2.7"), vec![a], "f:>2.7");
assert_eq!(q("f:>=2.8"), vec![a], "f:>=2.8 counts the photo's own value");
assert_eq!(q("focal:<=5.6"), vec![a], "focal:<=5.6");
assert!(q("f:>=2.9").is_empty(), "f:>=2.9");
assert!(q("focal:<=5.5").is_empty(), "focal:<=5.5");
}
/// `>=` and `<=` in a fielded token are the inclusive comparisons. They used to read the `=` as
/// part of the number, so `rating:>=3` or `iso:<=400` silently matched no photo at all.
#[test]
fn search_tokens_compare_inclusively() {
let mut c = Catalog::new();
let (two, three, four) = (
photo(&mut c, "two.jpg", "2026-05-01T10:00:00"),
photo(&mut c, "three.jpg", "2026-05-02T10:00:00"),
photo(&mut c, "four.jpg", "2026-05-03T10:00:00"),
);
for (id, rating, iso) in [(two, 2, 200), (three, 3, 400), (four, 4, 800)] {
c.apply(Op::SetRating { id, rating }).unwrap();
let mut m = c.photo(id).unwrap().meta.clone();
m.iso = Some(iso);
c.apply(Op::SetMeta { id, meta: Box::new(m) }).unwrap();
}
let q = |t: &str| {
let mut ids = c.query(&Filter { text: t.into(), ..Default::default() }, &Sort::default());
ids.sort();
ids
};
assert_eq!(q("rating:>=3"), vec![three, four]);
assert_eq!(q("stars:<=3"), vec![two, three]);
assert_eq!(q("iso:>=400"), vec![three, four]);
assert_eq!(q("iso:<=400"), vec![two, three]);
assert_eq!(q("rating:>3"), vec![four], "strict comparisons are unchanged");
assert_eq!(q("rating:=3"), vec![three]);
assert_eq!(q("rating:3"), vec![three]);
assert!(q("rating:>=").is_empty(), "an operator without a number matches nothing");
assert!(q("rating:=>3").is_empty(), "not an operator");
}
/// A `captured` string that is not ASCII (the Lightroom importer copies `captureTime` into an
@@ -904,17 +940,9 @@ fn search_text_finds_people_and_quoted_values() {
/// What the catalog remembers of its latest ops: which photos each changed and what of them, as long as that can be
/// said photo by photo.
#[test]
fn the_catalog_remembers_which_photos_its_latest_ops_changed_and_what_of_them() {
fn photo_ops_record_touched_ids_and_changed_aspects() {
let mut c = Catalog::default();
let photo = |c: &mut Catalog, name: &str| {
let id = c.alloc_photo_id();
c.apply(Op::AddPhoto {
photo: Box::new(Photo::new(id, Source::File { path: format!("/p/{name}") }, name, "JPEG", 60, 40, "2026-01-01T10:00:00")),
})
.unwrap();
id
};
let (a, b) = (photo(&mut c, "a.jpg"), photo(&mut c, "b.jpg"));
let (a, b) = (photo(&mut c, "a.jpg", "2026-01-01T10:00:00"), photo(&mut c, "b.jpg", "2026-01-01T10:00:00"));
let start = c.revision;
assert_eq!(c.touched_since(start), Some(vec![]));
assert_eq!(c.aspects_since(start), Some(Aspects::NONE));
@@ -940,28 +968,25 @@ fn the_catalog_remembers_which_photos_its_latest_ops_changed_and_what_of_them()
assert_eq!(c.touched_since(before), None);
assert_eq!(c.aspects_since(start), None);
assert_eq!(c.touched_since(c.revision), Some(vec![]), "from after it, it is known again");
// a revision that is not this catalog's past
assert_eq!(c.touched_since(c.revision + 5), None);
// a revision set from outside (a library just loaded): nothing is known of how the catalog got there, and after
// the next op nothing of before it
let old = c.revision;
}
#[test]
fn history_handles_loaded_revisions_failed_batches_and_retention() {
let mut c = Catalog::default();
let a = photo(&mut c, "a.jpg", "2026-01-01T10:00:00");
let album = c.alloc_album_id();
c.apply(Op::AddAlbum { album: Album::new(album, "Trip") }).unwrap();
// a batch that fails midway is undone as a whole; a batch in a batch is remembered photo by photo; a batch
// with an op that changes more than photos is not
let before = c.revision;
c.revision = 7 << 32;
assert_eq!(c.touched_since(old), None);
assert_eq!(c.touched_since(before), None);
assert_eq!(c.aspects_since(c.revision), None);
let loaded = c.revision;
c.apply(Op::SetRating { id: a, rating: 1 }).unwrap();
assert_eq!(c.touched_since(loaded), Some(vec![a]));
assert_eq!(c.touched_since(old), None);
// an op that fails changes nothing and is not remembered
let before = c.revision;
assert!(c.apply(Op::SetRating { id: PhotoId(9_999), rating: 1 }).is_err());
assert_eq!(c.revision, before);
assert_eq!(c.touched_since(before), Some(vec![]));
c.apply(Op::SetFlag { id: a, flag: Flag::Pick }).unwrap();
assert_eq!(c.touched_since(before), Some(vec![a]));
assert!(c.aspects_since(before).is_some_and(|x| x.any_of(Aspects::FLAG)));
// a batch that fails midway is undone as a whole; a batch in a batch is remembered photo by photo; a batch
// with an op that changes more than photos is not
assert_eq!(c.touched_since(before), None);
let before = c.revision;
assert!(c.apply(Op::Batch { ops: vec![Op::SetRating { id: a, rating: 5 }, Op::SetRating { id: PhotoId(9_999), rating: 1 }] }).is_err());
assert_eq!((c.revision, c.photo(a).map(|p| p.rating)), (before, Some(1)));
crates/codecs/src/encode.rs
+82 -44
View File
@@ -227,7 +227,7 @@ pub enum TiffCompression {
/// Encode TIFF: 8/16-bit integer or 32-bit float, gray/RGB with optional (unassociated) alpha.
/// ICC and XMP are embedded; EXIF is not (write it through the TIFF writer in `lightcraft-tiff`).
pub fn encode_tiff(img: &EncodeImage, compression: TiffCompression, meta: &EncodeMeta) -> Result<Vec<u8>> {
use tiff::encoder::{Compression, DeflateLevel, TiffEncoder, colortype};
use tiff::encoder::{Compression, DeflateLevel, TiffEncoder};
use tiff::tags::Predictor;
img.validate()?;
let e = |e: tiff::TiffError| Error::Encode(e.to_string());
@@ -243,54 +243,92 @@ pub fn encode_tiff(img: &EncodeImage, compression: TiffCompression, meta: &Encod
if !float && matches!(compression, TiffCompression::Lzw | TiffCompression::Deflate) { Predictor::Horizontal } else { Predictor::None };
let mut enc = TiffEncoder::new(&mut cur).map_err(e)?.with_compression(comp).with_predictor(predictor);
let (w, h) = (img.width, img.height);
let n = w as usize * h as usize;
// Gray + alpha is stored as RGBA (the encoder has no GrayA colour type).
let ch = img.channels as usize;
macro_rules! write {
($ct:ty, $data:expr, $alpha:expr) => {{
let mut im = enc.new_image::<$ct>(w, h).map_err(e)?;
if let Some(ppi) = meta.ppi.filter(|p| *p > 0) {
im.resolution(tiff::tags::ResolutionUnit::Inch, tiff::encoder::Rational { n: ppi as u32, d: 1 });
}
if let Some(icc) = meta.icc {
im.encoder().write_tag(tiff::tags::Tag::IccProfile, icc).map_err(e)?;
}
if let Some(xmp) = meta.xmp {
im.encoder().write_tag(tiff::tags::Tag::from_u16_exhaustive(700), xmp.as_bytes()).map_err(e)?;
}
if $alpha {
im.encoder().write_tag(tiff::tags::Tag::ExtraSamples, &[2u16][..]).map_err(e)?;
}
im.write_data($data).map_err(e)?;
}};
}
fn expand_ga<T: Copy>(s: &[T], n: usize) -> Vec<T> {
s.as_chunks::<2>().0.iter().take(n).flat_map(|c| [c[0], c[0], c[0], c[1]]).collect()
}
let pixels = w as usize * h as usize;
match img.samples {
Samples::U8(s) => match ch {
1 => write!(colortype::Gray8, &s[..n], false),
2 => write!(colortype::RGBA8, &expand_ga(s, n), true),
3 => write!(colortype::RGB8, &s[..n * 3], false),
_ => write!(colortype::RGBA8, &s[..n * 4], true),
},
Samples::U16(s) => match ch {
1 => write!(colortype::Gray16, &s[..n], false),
2 => write!(colortype::RGBA16, &expand_ga(s, n), true),
3 => write!(colortype::RGB16, &s[..n * 3], false),
_ => write!(colortype::RGBA16, &s[..n * 4], true),
},
Samples::F32(s) => match ch {
1 => write!(colortype::Gray32Float, &s[..n], false),
2 => write!(colortype::RGBA32Float, &expand_ga(s, n), true),
3 => write!(colortype::RGB32Float, &s[..n * 3], false),
_ => write!(colortype::RGBA32Float, &s[..n * 4], true),
},
Samples::U8(samples) => write_tiff_u8(&mut enc, samples, pixels, img.channels as usize, img, meta)?,
Samples::U16(samples) => write_tiff_u16(&mut enc, samples, pixels, img.channels as usize, img, meta)?,
Samples::F32(samples) => write_tiff_f32(&mut enc, samples, pixels, img.channels as usize, img, meta)?,
}
Ok(cur.into_inner())
}
macro_rules! write_tiff_image {
($encoder:expr, $image:expr, $meta:expr, $color:ty, $data:expr, $alpha:expr) => {{
let mut image = $encoder.new_image::<$color>($image.width, $image.height).map_err(|e| Error::Encode(e.to_string()))?;
if let Some(ppi) = $meta.ppi.filter(|ppi| *ppi > 0) {
image.resolution(tiff::tags::ResolutionUnit::Inch, tiff::encoder::Rational { n: ppi as u32, d: 1 });
}
if let Some(icc) = $meta.icc {
image.encoder().write_tag(tiff::tags::Tag::IccProfile, icc).map_err(|e| Error::Encode(e.to_string()))?;
}
if let Some(xmp) = $meta.xmp {
image.encoder().write_tag(tiff::tags::Tag::from_u16_exhaustive(700), xmp.as_bytes()).map_err(|e| Error::Encode(e.to_string()))?;
}
if $alpha {
image.encoder().write_tag(tiff::tags::Tag::ExtraSamples, &[2u16][..]).map_err(|e| Error::Encode(e.to_string()))?;
}
image.write_data($data).map_err(|e| Error::Encode(e.to_string()))?;
}};
}
fn expanded_gray_alpha<T: Copy>(samples: &[T], pixels: usize) -> Vec<T> {
samples.as_chunks::<2>().0.iter().take(pixels).flat_map(|channel| [channel[0], channel[0], channel[0], channel[1]]).collect()
}
fn write_tiff_u8<W: std::io::Write + std::io::Seek>(
encoder: &mut tiff::encoder::TiffEncoder<W>,
samples: &[u8],
pixels: usize,
channels: usize,
image: &EncodeImage,
meta: &EncodeMeta,
) -> Result<()> {
use tiff::encoder::colortype;
match channels {
1 => write_tiff_image!(encoder, image, meta, colortype::Gray8, &samples[..pixels], false),
2 => write_tiff_image!(encoder, image, meta, colortype::RGBA8, &expanded_gray_alpha(samples, pixels), true),
3 => write_tiff_image!(encoder, image, meta, colortype::RGB8, &samples[..pixels * 3], false),
_ => write_tiff_image!(encoder, image, meta, colortype::RGBA8, &samples[..pixels * 4], true),
}
Ok(())
}
fn write_tiff_u16<W: std::io::Write + std::io::Seek>(
encoder: &mut tiff::encoder::TiffEncoder<W>,
samples: &[u16],
pixels: usize,
channels: usize,
image: &EncodeImage,
meta: &EncodeMeta,
) -> Result<()> {
use tiff::encoder::colortype;
match channels {
1 => write_tiff_image!(encoder, image, meta, colortype::Gray16, &samples[..pixels], false),
2 => write_tiff_image!(encoder, image, meta, colortype::RGBA16, &expanded_gray_alpha(samples, pixels), true),
3 => write_tiff_image!(encoder, image, meta, colortype::RGB16, &samples[..pixels * 3], false),
_ => write_tiff_image!(encoder, image, meta, colortype::RGBA16, &samples[..pixels * 4], true),
}
Ok(())
}
fn write_tiff_f32<W: std::io::Write + std::io::Seek>(
encoder: &mut tiff::encoder::TiffEncoder<W>,
samples: &[f32],
pixels: usize,
channels: usize,
image: &EncodeImage,
meta: &EncodeMeta,
) -> Result<()> {
use tiff::encoder::colortype;
match channels {
1 => write_tiff_image!(encoder, image, meta, colortype::Gray32Float, &samples[..pixels], false),
2 => write_tiff_image!(encoder, image, meta, colortype::RGBA32Float, &expanded_gray_alpha(samples, pixels), true),
3 => write_tiff_image!(encoder, image, meta, colortype::RGB32Float, &samples[..pixels * 3], false),
_ => write_tiff_image!(encoder, image, meta, colortype::RGBA32Float, &samples[..pixels * 4], true),
}
Ok(())
}
/// Encode a flat PSD (8/16-bit gray or RGB, no layers; alpha dropped): the hand-off an image
/// editor such as PhotoCraft saves back to in place. ICC, EXIF, XMP and resolution are embedded.
pub fn encode_psd(img: &EncodeImage, meta: &EncodeMeta) -> Result<Vec<u8>> {
crates/codecs/src/jpeg_par.rs
+76 -45
View File
@@ -255,9 +255,20 @@ fn fdct_quant(block: &[f32; 64], recip: &[f32; 64], out: &mut [i16; 64]) {
aan_1d(&mut d, c, 8);
}
for i in 0..64 {
out[i] = (d[i] * recip[i]).round() as i16;
out[i] = round_i16(d[i] * recip[i]);
}
}
/// `x.round() as i16` (half away from zero, saturating, NaN to 0) for every `x` (checked for all of them), in plain
/// arithmetic: without SSE4.1 `f32::round` is a libm call, one per coefficient, that keeps the loop from vectorising.
#[inline(always)]
fn round_i16(x: f32) -> i16 {
// (below 2^24 the truncation is exact and so is the remainder)
let c = x.clamp(-65536.0, 65536.0);
let t = c as i32;
let f = c - t as f32;
let r = t + i32::from(f >= 0.5) - i32::from(f <= -0.5);
r.clamp(i32::from(i16::MIN), i32::from(i16::MAX)) as i16
}
#[inline]
fn magnitude(v: i32) -> (u32, u32) {
@@ -395,31 +406,30 @@ fn encode_with(
(true, _) | (false, ChromaSubsampling::S444) => (1usize, 1usize),
(false, _) => (2, 2),
};
// colour conversion (JFIF YCbCr, full range, level-shifted) in one pass; chroma averaged 2×2 for 4:2:0
let n = width * height;
#[cfg(feature = "parallel")]
use rayon::prelude::*;
// 4:4:4: Y, Cb, Cr at full resolution in one pass; 4:2:0: Y here, chroma below straight from 2×2 RGB means
// colour conversion (JFIF YCbCr, full range, level-shifted) of the pixel rows `rows`, one band's worth at a
// time (in the band's task: the planes of the whole image would be ~290 MB at 24 MP, written and read back
// once): 4:4:4 Y, Cb, Cr at full resolution in one pass; 4:2:0 Y, and chroma (rows `crows` of the
// half-size planes) straight from 2×2 RGB means (conversion is linear: average first, then convert once)
let ncomp = if gray || hs == 2 { 1 } else { 3 };
let mut full: Vec<Vec<f32>> = (0..ncomp).map(|_| vec![0f32; n]).collect();
{
let rows: Vec<(usize, Vec<&mut [f32]>)> = {
let mut its: Vec<_> = full.iter_mut().map(|p| p.chunks_mut(width)).collect();
(0..height).map_while(|y| Some((y, its.iter_mut().map(|it| it.next()).collect::<Option<_>>()?))).collect()
};
let conv = |(y, mut out): (usize, Vec<&mut [f32]>)| {
let (cw, ch) = (width.div_ceil(2), height.div_ceil(2));
let convert = |rows: std::ops::Range<usize>, crows: std::ops::Range<usize>| -> Vec<Plane> {
let mut full: Vec<Vec<f32>> = (0..ncomp).map(|_| vec![0f32; rows.len() * width]).collect();
for (k, y) in rows.clone().enumerate() {
let src = &data[y * width * channels..(y + 1) * width * channels];
let mut out: Vec<&mut [f32]> = full.iter_mut().map(|p| &mut p[k * width..(k + 1) * width]).collect();
if gray {
for (o, p) in out[0].iter_mut().zip(src.chunks_exact(channels)) {
*o = p[0] as f32 - 128.0;
}
return;
continue;
}
if out.len() == 1 {
for (o, p) in out[0].iter_mut().zip(src.chunks_exact(channels)) {
*o = 0.299 * p[0] as f32 + 0.587 * p[1] as f32 + 0.114 * p[2] as f32 - 128.0;
}
return;
continue;
}
for (x, p) in src.chunks_exact(channels).enumerate() {
let (r, g, b) = (p[0] as f32, p[1] as f32, p[2] as f32);
@@ -427,37 +437,26 @@ fn encode_with(
out[1][x] = -0.168_736 * r - 0.331_264 * g + 0.5 * b;
out[2][x] = 0.5 * r - 0.418_688 * g - 0.081_312 * b;
}
};
#[cfg(feature = "parallel")]
rows.into_par_iter().for_each(conv);
#[cfg(not(feature = "parallel"))]
rows.into_iter().for_each(conv);
}
let mut it = full.into_iter();
let mut planes = vec![Plane { w: width, h: height, data: it.next().unwrap_or_default() }];
planes.extend(it.map(|c| Plane { w: width, h: height, data: c }));
if !gray && hs == 2 {
// conversion is linear: average RGB over 2×2 first, then convert once
let (cw, ch) = (width.div_ceil(2), height.div_ceil(2));
let (mut cb, mut cr) = (vec![0f32; cw * ch], vec![0f32; cw * ch]);
let down = |(cy, (ob, or)): (usize, (&mut [f32], &mut [f32]))| {
let (y0, y1) = (2 * cy, (2 * cy + 1).min(height - 1));
let (r0, r1) = (&data[y0 * width * channels..], &data[y1 * width * channels..]);
for cx in 0..cw {
let (x0, x1) = (2 * cx * channels, (2 * cx + 1).min(width - 1) * channels);
let s = |k: usize| (r0[x0 + k] as f32 + r0[x1 + k] as f32 + r1[x0 + k] as f32 + r1[x1 + k] as f32) * 0.25;
let (r, g, b) = (s(0), s(1), s(2));
ob[cx] = -0.168_736 * r - 0.331_264 * g + 0.5 * b;
or[cx] = 0.5 * r - 0.418_688 * g - 0.081_312 * b;
}
let mut planes: Vec<Plane> = full.into_iter().map(|data| Plane { w: width, h: rows.len(), data }).collect();
if !gray && hs == 2 {
let (mut cb, mut cr) = (vec![0f32; cw * crows.len()], vec![0f32; cw * crows.len()]);
for (cy, (ob, or)) in crows.clone().zip(cb.chunks_mut(cw).zip(cr.chunks_mut(cw))) {
let (y0, y1) = (2 * cy, (2 * cy + 1).min(height - 1));
let (r0, r1) = (&data[y0 * width * channels..], &data[y1 * width * channels..]);
for cx in 0..cw {
let (x0, x1) = (2 * cx * channels, (2 * cx + 1).min(width - 1) * channels);
let s = |k: usize| (r0[x0 + k] as f32 + r0[x1 + k] as f32 + r1[x0 + k] as f32 + r1[x1 + k] as f32) * 0.25;
let (r, g, b) = (s(0), s(1), s(2));
ob[cx] = -0.168_736 * r - 0.331_264 * g + 0.5 * b;
or[cx] = 0.5 * r - 0.418_688 * g - 0.081_312 * b;
}
}
};
#[cfg(feature = "parallel")]
cb.par_chunks_mut(cw).zip(cr.par_chunks_mut(cw)).enumerate().for_each(down);
#[cfg(not(feature = "parallel"))]
cb.chunks_mut(cw).zip(cr.chunks_mut(cw)).enumerate().for_each(down);
planes.push(Plane { w: cw, h: ch, data: cb });
planes.push(Plane { w: cw, h: ch, data: cr });
}
planes.push(Plane { w: cw, h: crows.len(), data: cb });
planes.push(Plane { w: cw, h: crows.len(), data: cr });
}
planes
};
let ql = scaled_quant(&Q_LUMA, quality);
let qc = scaled_quant(&Q_CHROMA, quality);
@@ -473,7 +472,12 @@ fn encode_with(
let (mut blk, mut q) = ([0f32; 64], [0i16; 64]);
let (mut tokens, mut freq) = (Vec::new(), [[0u64; 256]; 4]);
let mut pred = [0i32; 3];
for mcu_y in band * rows_per_band..((band + 1) * rows_per_band).min(my) {
let (m0, m1) = (band * rows_per_band, ((band + 1) * rows_per_band).min(my));
// the band's planes, its MCU row `m0` at their top (blocks past the image's bottom repeat its last row,
// which is the band's last row then)
let planes = convert(m0 * mcu_h..(m1 * mcu_h).min(height), m0 * 8..(m1 * 8).min(ch));
for mcu_y in m0..m1 {
let mcu_y = mcu_y - m0;
for mcu_x in 0..mx {
for vy in 0..vs {
for hx in 0..hs {
@@ -497,7 +501,6 @@ fn encode_with(
let tokenized: Vec<(Vec<Token>, [Freq; 4])> = bands.par_iter().map(tokenize_band).collect();
#[cfg(not(feature = "parallel"))]
let tokenized: Vec<(Vec<Token>, [Freq; 4])> = bands.iter().map(tokenize_band).collect();
drop(planes);
// the tables: DC luma, AC luma, DC chroma, AC chroma
let tables: [([u8; 16], Vec<u8>); 4] = if optimize {
let mut total = [[0u64; 256]; 4];
@@ -746,6 +749,34 @@ mod tests {
assert_eq!(jpeg_decoder::Decoder::new(&std[..]).decode().unwrap(), jpeg_decoder::Decoder::new(&opt[..]).decode().unwrap());
}
/// The bytes written for 4:4:4, 4:2:0, grayscale and RGBA input at a size cut into many bands, with partial MCUs
/// at the right and bottom edges, pinned: each band converts its own rows to YCbCr, which must not change a bit.
#[test]
fn encoded_bytes_are_pinned() {
let fnv = |b: &[u8]| b.iter().fold(0xcbf2_9ce4_8422_2325u64, |h, &x| (h ^ u64::from(x)).wrapping_mul(0x100_0000_01b3));
let (w, h) = (1203, 1037);
let rgb = test_rgb(w, h);
let gray: Vec<u8> = rgb.as_chunks::<3>().0.iter().map(|p| p[1]).collect();
let rgba: Vec<u8> = rgb.as_chunks::<3>().0.iter().flat_map(|p| [p[0], p[1], p[2], 255]).collect();
let got = [
fnv(&encode(&rgb, w, h, 3, 90, ChromaSubsampling::S444, &[])),
fnv(&encode(&rgb, w, h, 3, 75, ChromaSubsampling::S420, &[])),
fnv(&encode(&gray, w, h, 1, 90, ChromaSubsampling::S444, &[])),
fnv(&encode(&rgba, w, h, 4, 85, ChromaSubsampling::S444, &[])),
];
assert_eq!(got, PINNED, "{got:#x?}");
}
/// [`round_i16`] against `f32::round` at the halves, the ends of `i16`, NaN and infinities, and a spread of every
/// magnitude (all 2^32 inputs were compared once, offline).
#[test]
fn round_i16_is_round_as_i16() {
let special = [0.5f32, -0.5, 1.5, -1.5, 2.5, -2.5, 0.499_999_97, -0.499_999_97, 32767.5, -32768.5, 32766.5, 1e9, -1e9];
let all = special.into_iter().chain([f32::NAN, f32::INFINITY, f32::NEG_INFINITY]).chain((0..=u32::MAX).step_by(9973).map(f32::from_bits));
for x in all {
assert_eq!(round_i16(x), x.round() as i16, "{x:e}");
}
}
const PINNED: [u64; 4] = [0x52ba_772a_589c_1430, 0x013f_8364_821f_1dde, 0xb2a3_b5e5_aef3_7402, 0x08df_d6dc_aa2f_1c78];
#[test]
fn grayscale_and_quality_monotone() {
let (w, h) = (300, 200);
crates/color/src/perceptual.rs
+10 -1
View File
@@ -132,7 +132,9 @@ fn hue_of(c: [f32; 3], max: f32, d: f32) -> f32 {
} else {
(c[0] - c[1]) / d + 4.0
};
h * 60.0
let deg = h * 60.0;
// rem_euclid can round up to exactly 6.0 in f32; keep the range [0, 360).
if deg >= 360.0 { 0.0 } else { deg }
}
#[cfg(test)]
@@ -173,4 +175,11 @@ mod tests {
}
assert_eq!(rgb_to_hsv([0.0, 1.0, 0.0])[0], 120.0);
}
#[test]
fn hue_below_360_for_tiny_blue() {
let c = [1.0, 0.0, 1e-9];
assert!(rgb_to_hsv(c)[0] < 360.0, "hsv hue {}", rgb_to_hsv(c)[0]);
assert!(rgb_to_hsl(c)[0] < 360.0, "hsl hue {}", rgb_to_hsl(c)[0]);
}
}
crates/color/src/transfer.rs
+6
View File
@@ -49,6 +49,7 @@ pub fn hlg_encode(e: f64) -> f64 {
}
pub fn hlg_decode(s: f64) -> f64 {
let s = s.max(0.0);
if s <= 0.5 { s * s / 3.0 } else { (((s - HLG_C) / HLG_A).exp() + HLG_B) / 12.0 }
}
@@ -112,6 +113,11 @@ mod tests {
}
}
#[test]
fn hlg_decode_clamps_negative() {
assert_eq!(hlg_decode(-0.5), 0.0);
}
#[test]
fn pq_hlg_roundtrip() {
for i in 0..=50 {
crates/denoise-core/src/bayer.rs
+18 -7
View File
@@ -14,6 +14,14 @@ pub struct Layout {
pub dy: usize,
}
/// The mosaic dimensions and cell origin for one square denoiser input tile.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct TileRegion {
pub mosaic_size: (usize, usize),
pub cell_origin: (isize, isize),
pub tile_size: usize,
}
impl Layout {
/// The layout of a 2 × 2 colour pattern, row-major, colours 0 = red, 1 = green, 2 = blue; `None` when
/// it is not a Bayer cell (one red, one blue, the greens on the other diagonal).
@@ -65,7 +73,10 @@ pub fn reflect(i: isize, n: usize) -> usize {
/// Pack the `tile × tile` cells whose top-left cell is `(cx0, cy0)` into `out` as four planes `[R, G1, G2, B]`
/// (`4 · tile²` values, NCHW). G1 is the green beside the red sample, G2 the green under it. Samples outside
/// the picture are mirrored. Returns `false` (and writes nothing) when the sizes do not fit.
pub fn pack_tile(mosaic: &[f32], width: usize, height: usize, layout: Layout, cx0: isize, cy0: isize, tile: usize, out: &mut [f32]) -> bool {
pub fn pack_tile(mosaic: &[f32], layout: Layout, region: TileRegion, out: &mut [f32]) -> bool {
let (width, height) = region.mosaic_size;
let (cx0, cy0) = region.cell_origin;
let tile = region.tile_size;
let Some(plane) = tile.checked_mul(tile) else { return false };
let Some(values) = plane.checked_mul(4) else { return false };
if width == 0
@@ -113,7 +124,7 @@ mod tests {
let layout = Layout::from_cell([0, 1, 1, 2]).unwrap();
let mut out = [7.0; 4];
for (w, h, tile, x) in [(usize::MAX, 2, 1, 0), (1, 1, usize::MAX, 0), (1, 1, 0, 0), (1, 1, 1, isize::MAX), (1, 1, 1, isize::MIN)] {
assert!(!pack_tile(&[0.5], w, h, layout, x, 0, tile, &mut out));
assert!(!pack_tile(&[0.5], layout, TileRegion { mosaic_size: (w, h), cell_origin: (x, 0), tile_size: tile }, &mut out));
assert_eq!(out, [7.0; 4]);
}
for n in [1, 2, isize::MAX as usize, usize::MAX] {
@@ -160,7 +171,7 @@ mod tests {
let (nx, ny) = layout.cells(w, h);
let tile = nx.max(ny);
let mut out = vec![0.0; 4 * tile * tile];
assert!(pack_tile(&mosaic, w, h, layout, 0, 0, tile, &mut out));
assert!(pack_tile(&mosaic, layout, TileRegion { mosaic_size: (w, h), cell_origin: (0, 0), tile_size: tile }, &mut out));
let plane = tile * tile;
for (p, want) in [10.0, 20.0, 20.0, 30.0].into_iter().enumerate() {
assert!(out[p * plane..(p + 1) * plane].iter().all(|&v| v == want), "layout {cell:?} plane {p}");
@@ -174,7 +185,7 @@ mod tests {
let (w, h) = (4, 4);
let mosaic: Vec<f32> = (0..16).map(|i| i as f32).collect();
let mut out = vec![0.0; 4 * 4];
assert!(pack_tile(&mosaic, w, h, Layout { dx: 0, dy: 0 }, 0, 0, 2, &mut out));
assert!(pack_tile(&mosaic, Layout { dx: 0, dy: 0 }, TileRegion { mosaic_size: (w, h), cell_origin: (0, 0), tile_size: 2 }, &mut out));
// cell (0,0): R = site (0,0)=0, G1 = site (1,0)=1, G2 = site (0,1)=4, B = site (1,1)=5
assert_eq!([out[0], out[4], out[8], out[12]], [0.0, 1.0, 4.0, 5.0]);
// cell (1,0): sites (2,0)=2, (3,0)=3, (2,1)=6, (3,1)=7
@@ -184,8 +195,8 @@ mod tests {
#[test]
fn packing_rejects_wrong_sizes() {
let mut out = vec![0.0; 16];
assert!(!pack_tile(&[0.0; 15], 4, 4, Layout { dx: 0, dy: 0 }, 0, 0, 2, &mut out));
assert!(!pack_tile(&[0.0; 16], 4, 4, Layout { dx: 0, dy: 0 }, 0, 0, 3, &mut out));
assert!(!pack_tile(&[], 0, 0, Layout { dx: 0, dy: 0 }, 0, 0, 2, &mut out));
assert!(!pack_tile(&[0.0; 15], Layout { dx: 0, dy: 0 }, TileRegion { mosaic_size: (4, 4), cell_origin: (0, 0), tile_size: 2 }, &mut out));
assert!(!pack_tile(&[0.0; 16], Layout { dx: 0, dy: 0 }, TileRegion { mosaic_size: (4, 4), cell_origin: (0, 0), tile_size: 3 }, &mut out));
assert!(!pack_tile(&[], Layout { dx: 0, dy: 0 }, TileRegion { mosaic_size: (0, 0), cell_origin: (0, 0), tile_size: 2 }, &mut out));
}
}
crates/denoise-core/src/run.rs
+7 -2
View File
@@ -9,7 +9,7 @@ use std::sync::atomic::{AtomicBool, Ordering};
use lightcraft_raster::Rgb32f;
use rayon::prelude::*;
use crate::bayer::{Layout, pack_tile};
use crate::bayer::{Layout, TileRegion, pack_tile};
use crate::manifest::Gain;
use crate::tiles::{starts, weight};
@@ -139,7 +139,12 @@ pub fn denoise_bayer(
.par_iter()
.map(|&(x0, y0, bx, ax, by, ay)| {
let mut input = allocate(4 * p.tile * p.tile, 0f32)?;
if !pack_tile(mosaic, width, height, layout, x0 as isize, y0 as isize, p.tile, &mut input) {
if !pack_tile(
mosaic,
layout,
TileRegion { mosaic_size: (width, height), cell_origin: (x0 as isize, y0 as isize), tile_size: p.tile },
&mut input,
) {
return Err(Error::Input("a tile could not be packed".into()));
}
let mut out = runner.run(&input)?;
crates/engine/examples/export_bench.rs
+81
View File
@@ -0,0 +1,81 @@
//! Single-photo export benchmark (median and minimum of N runs):
//! `cargo run --release -p lightcraft-engine --example export_bench -- corpus/raw/arw-sony-a7m3-compressed.arw`
//!
//! Imports the files given and runs the export path the app, CLI and MCP share
//! ([`lightcraft_engine::export::export_photo`]: decode the original, render it full size, encode)
//! once per file per run, forgetting the decoded original between runs so each export reads and
//! decodes it like a batch export does. `N=9` runs (default), `OPT='{"format":"png"}'` for other
//! export options (JSON as `app.export` takes them), `LIGHTCRAFT_GPU=0` for the CPU pipeline,
//! `PHASES=1` to time render and encode separately, `OUT=dir` to write each file's export there
//! (for byte comparisons between builds).
use std::time::Instant;
use lightcraft_engine::Session;
use lightcraft_engine::catalog::PhotoId;
use lightcraft_engine::export::{ExportOptions, encode_rendered, export_metadata, export_photo};
fn stats(mut v: Vec<f64>) -> (f64, f64) {
v.sort_by(f64::total_cmp);
(v[v.len() / 2], v[0])
}
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
assert!(!args.is_empty(), "usage: export_bench FILE...");
let n: usize = std::env::var("N").ok().and_then(|v| v.parse().ok()).unwrap_or(9).max(1);
let opt: serde_json::Value =
std::env::var("OPT").ok().map(|s| serde_json::from_str(&s).expect("OPT is JSON")).unwrap_or_else(|| serde_json::json!({}));
let o = ExportOptions::from_params(&opt).expect("export options");
let mut s = Session::new().with_fs();
let paths: Vec<String> = args.iter().map(|p| std::fs::canonicalize(p).expect("input file").to_string_lossy().to_string()).collect();
let r = s.execute("library.import", &serde_json::json!({"paths": paths})).expect("import");
let ids: Vec<PhotoId> = r["imported"].as_array().expect("ids").iter().filter_map(|v| v.as_u64()).map(PhotoId).collect();
assert_eq!(ids.len(), args.len(), "every file imports");
println!("{} file(s), {:?}, {n} runs, gpu {}", ids.len(), o.format, lightcraft_engine::gpu::available());
let mut times = Vec::new();
for i in 0..=n {
let t = Instant::now();
for (k, &id) in ids.iter().enumerate() {
let e = export_photo(&mut s, id, &o, k + 1).expect("export");
if i == 0
&& let Some(dir) = std::env::var_os("OUT")
{
std::fs::write(std::path::Path::new(&dir).join(&e.file_name), &e.bytes).expect("write");
println!(" {} {}×{} {} bytes", e.file_name, e.width, e.height, e.bytes.len());
}
s.media.forget(id);
}
// run 0 warms up (GPU device, allocator, page cache)
if i > 0 {
times.push(t.elapsed().as_secs_f64() * 1e3);
}
}
let (med, min) = stats(times.clone());
println!("export: median {med:.1} ms, min {min:.1} ms (runs: {})", times.iter().map(|t| format!("{t:.0}")).collect::<Vec<_>>().join(" "));
if std::env::var_os("PHASES").is_none() {
return;
}
let (mut render, mut encode) = (Vec::new(), Vec::new());
for _ in 0..n {
let (mut tr, mut te) = (0.0, 0.0);
for &id in &ids {
let p = s.catalog.photo(id).expect("photo").clone();
let (w, h) = lightcraft_engine::export::output_size(&p, &o);
let meta = export_metadata(&p, &s.catalog, &o);
let job = s.export_job(id, w, h, o.effective_space(), o.effective_depth()).expect("job");
let t = Instant::now();
let r = job.run().rendered.expect("render");
tr += t.elapsed().as_secs_f64() * 1e3;
let t = Instant::now();
std::hint::black_box(encode_rendered(&r, &o, meta.as_ref()).expect("encode"));
te += t.elapsed().as_secs_f64() * 1e3;
s.media.forget(id);
}
render.push(tr);
encode.push(te);
}
let (rm, rn) = stats(render);
let (em, en) = stats(encode);
println!(" decode+render: median {rm:.1} ms, min {rn:.1} ms");
println!(" encode: median {em:.1} ms, min {en:.1} ms");
}
crates/engine/examples/render_bench.rs
+100 -88
View File
@@ -91,107 +91,119 @@ fn diff(a: &lightcraft_raster::Rgba8, b: &lightcraft_raster::Rgba8) -> (u8, f64)
(max, sum as f64 / (a.data.len() * 3).max(1) as f64)
}
fn benchmark_jpeg(args: &[String], n: usize) {
// a JPEG (e.g. an embedded preview pulled out with exiftool) decoded to fit a few boxes
for path in args {
let bytes = std::fs::read(path).expect("read");
for edge in [256u32, 512, 1600, 2560] {
let o = lightcraft_codecs::DecodeOptions::fit(edge, edge);
let d = lightcraft_codecs::decode(&bytes, o).expect("decode");
println!(
" {path} fit {edge}: {}×{} {}",
d.image.width,
d.image.height,
best(n, || drop(lightcraft_codecs::decode(&bytes, o).expect("decode")))
);
}
}
}
fn benchmark_load(args: &[String], n: usize, only: &str) {
// opening a photo in the loupe, step by step: what the app's loaders cost (per file)
for path in args {
let bytes = std::fs::read(path).expect("read");
println!("{path}");
if std::env::var("LOADONLY").is_ok() {
// only the preview load (for a profiler)
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
continue;
}
println!(" {:<36} {}", "file read:", best(n, || drop(std::fs::read(path).expect("read"))));
println!(" {:<36} {}", "raw decode:", best(n, || drop(lightcraft_raw::decode(&bytes).expect("decode"))));
let mut raw = lightcraft_raw::decode(&bytes).expect("decode");
if let lightcraft_raw::RawData::U16(d) = &raw.data {
let sum = d.iter().enumerate().fold(0u64, |a, (i, &v)| a.wrapping_mul(31).wrapping_add(v as u64 ^ i as u64));
println!(" decode checksum: {sum:016x}");
}
if only == "load1" {
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
continue;
}
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
if let Some(k) = lightcraft_engine::files::bin_factor(&raw, 2560) {
println!(" {:<36} {}", format!("binned ×{k} masked:"), best(n, || drop(raw.develop_binned_masked(k, 0.99))));
}
for edge in [2560usize, 1600] {
println!(
" {:<36} {}",
format!("embedded srgb, whole file ({edge}):"),
best(n, || drop(lightcraft_engine::files::embedded_preview_srgb(&bytes, edge)))
);
let quick = lightcraft_engine::files::fs_preview_loader();
println!(" {:<36} {}", format!("fs_preview_loader ({edge}):"), best(n, || drop(quick(path, edge))));
}
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
let (loader, _) = lightcraft_engine::files::fs_hooks();
println!(" {:<36} {}", "fs loader (2560, read+load):", best(n, || drop(loader(path, 2560).expect("load"))));
}
}
fn benchmark_source(args: &[String], n: usize) {
// opening a photo: where the time goes before the first sharp loupe (per file)
for path in args {
let bytes = std::fs::read(path).expect("read");
println!("{path}");
let mut raw = lightcraft_raw::decode(&bytes).expect("decode");
// as the loader does: lens corrections are applied by the pipeline, not in the raw develop
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
println!(" {}×{} cpp {} cfa {:?}", raw.width, raw.height, raw.cpp, raw.cfa.as_ref().map(|c| c.name()));
println!(" {:<36} {}", "raw decode:", best(n, || drop(lightcraft_raw::decode(&bytes).expect("decode"))));
println!(" {:<36} {}", "normalize:", best(n, || drop(raw.normalized().expect("norm"))));
let norm = raw.normalized().expect("norm");
println!(" {:<36} {}", "demosaic AHD:", best(n, || drop(lightcraft_raw::demosaic(&norm, lightcraft_raw::Method::Ahd))));
if let Some(k) = lightcraft_engine::files::bin_factor(&raw, 2560) {
println!(" {:<36} {}", format!("binned ×{k} (2560 preview):"), best(n, || drop(raw.develop_binned(k, 0.99))));
let img = raw.develop_binned(k, 0.99).expect("bin").expect("binnable");
let t = lightcraft_raw::color::camera_transform(&raw, lightcraft_raw::color::as_shot_white_xy(&raw));
println!(
" {:<36} {}",
"highlight reconstruct (binned):",
best(n, || {
let mut i = img.clone();
lightcraft_raw::highlight::reconstruct(&mut i, t.wb, 0.99);
})
);
println!(" {:<36} {}", "clone (binned):", best(n, || drop(img.clone())));
println!(" {:<36} {}", "fit 2560 (binned):", best(n, || drop(fit(&img, 2560, 2560, Filter::Box))));
}
println!(" {:<36} {}", "embedded preview (2560):", best(n, || drop(lightcraft_engine::files::load_embedded_preview(&bytes, 2560))));
for edge in [512usize, 2560, usize::MAX] {
println!(
" {:<36} {}",
format!("load_bytes({edge}):"),
best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, edge).expect("load")))
);
}
}
}
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
let n: usize = std::env::var("N").ok().and_then(|v| v.parse().ok()).unwrap_or(5);
let only = std::env::var("ONLY").unwrap_or_default();
let run = |name: &str| only.is_empty() || only.split(',').any(|o| name.contains(o));
if only == "jpeg" {
// a JPEG (e.g. an embedded preview pulled out with exiftool) decoded to fit a few boxes
for path in &args {
let bytes = std::fs::read(path).expect("read");
for edge in [256u32, 512, 1600, 2560] {
let o = lightcraft_codecs::DecodeOptions::fit(edge, edge);
let d = lightcraft_codecs::decode(&bytes, o).expect("decode");
println!(
" {path} fit {edge}: {}×{} {}",
d.image.width,
d.image.height,
best(n, || drop(lightcraft_codecs::decode(&bytes, o).expect("decode")))
);
}
}
benchmark_jpeg(&args, n);
return;
}
if only == "load" || only == "load1" {
// opening a photo in the loupe, step by step: what the app's loaders cost (per file)
for path in &args {
let bytes = std::fs::read(path).expect("read");
println!("{path}");
if std::env::var("LOADONLY").is_ok() {
// only the preview load (for a profiler)
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
continue;
}
println!(" {:<36} {}", "file read:", best(n, || drop(std::fs::read(path).expect("read"))));
println!(" {:<36} {}", "raw decode:", best(n, || drop(lightcraft_raw::decode(&bytes).expect("decode"))));
let mut raw = lightcraft_raw::decode(&bytes).expect("decode");
if let lightcraft_raw::RawData::U16(d) = &raw.data {
let sum = d.iter().enumerate().fold(0u64, |a, (i, &v)| a.wrapping_mul(31).wrapping_add(v as u64 ^ i as u64));
println!(" decode checksum: {sum:016x}");
}
if only == "load1" {
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
continue;
}
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
if let Some(k) = lightcraft_engine::files::bin_factor(&raw, 2560) {
println!(" {:<36} {}", format!("binned ×{k} masked:"), best(n, || drop(raw.develop_binned_masked(k, 0.99))));
}
for edge in [2560usize, 1600] {
println!(
" {:<36} {}",
format!("embedded srgb, whole file ({edge}):"),
best(n, || drop(lightcraft_engine::files::embedded_preview_srgb(&bytes, edge)))
);
let quick = lightcraft_engine::files::fs_preview_loader();
println!(" {:<36} {}", format!("fs_preview_loader ({edge}):"), best(n, || drop(quick(path, edge))));
}
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
let (loader, _) = lightcraft_engine::files::fs_hooks();
println!(" {:<36} {}", "fs loader (2560, read+load):", best(n, || drop(loader(path, 2560).expect("load"))));
}
benchmark_load(&args, n, &only);
return;
}
if only == "source" {
// opening a photo: where the time goes before the first sharp loupe (per file)
for path in &args {
let bytes = std::fs::read(path).expect("read");
println!("{path}");
let mut raw = lightcraft_raw::decode(&bytes).expect("decode");
// as the loader does: lens corrections are applied by the pipeline, not in the raw develop
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
println!(" {}×{} cpp {} cfa {:?}", raw.width, raw.height, raw.cpp, raw.cfa.as_ref().map(|c| c.name()));
println!(" {:<36} {}", "raw decode:", best(n, || drop(lightcraft_raw::decode(&bytes).expect("decode"))));
println!(" {:<36} {}", "normalize:", best(n, || drop(raw.normalized().expect("norm"))));
let norm = raw.normalized().expect("norm");
println!(" {:<36} {}", "demosaic AHD:", best(n, || drop(lightcraft_raw::demosaic(&norm, lightcraft_raw::Method::Ahd))));
if let Some(k) = lightcraft_engine::files::bin_factor(&raw, 2560) {
println!(" {:<36} {}", format!("binned ×{k} (2560 preview):"), best(n, || drop(raw.develop_binned(k, 0.99))));
let img = raw.develop_binned(k, 0.99).expect("bin").expect("binnable");
let t = lightcraft_raw::color::camera_transform(&raw, lightcraft_raw::color::as_shot_white_xy(&raw));
println!(
" {:<36} {}",
"highlight reconstruct (binned):",
best(n, || {
let mut i = img.clone();
lightcraft_raw::highlight::reconstruct(&mut i, t.wb, 0.99);
})
);
println!(" {:<36} {}", "clone (binned):", best(n, || drop(img.clone())));
println!(" {:<36} {}", "fit 2560 (binned):", best(n, || drop(fit(&img, 2560, 2560, Filter::Box))));
}
println!(" {:<36} {}", "embedded preview (2560):", best(n, || drop(lightcraft_engine::files::load_embedded_preview(&bytes, 2560))));
for edge in [512usize, 2560, usize::MAX] {
println!(
" {:<36} {}",
format!("load_bytes({edge}):"),
best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, edge).expect("load")))
);
}
}
benchmark_source(&args, n);
return;
}
let run = |name: &str| only.is_empty() || only.split(',').any(|o| name.contains(o));
let (full, info) = match args.first() {
Some(path) => {
let bytes = std::fs::read(path).expect("read");
crates/engine/src/camera_preview/colour.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::look_fit::{fit_dng_tone, fit_preview};
use super::mono::{fit_dark_tone, monochrome_look};
use super::proxy::{LensGeometry, proxies};
use super::*;
use super::{CameraLook, CameraTone, CameraTransform, ColorData, D50, D65, Mat3, ProfileLook, REC2020, RawFormat, RawImage, ToneCurve, Xy, bradford};
/// Long edge of the sensor/JPEG proxy a single photo's look is fitted on (the acceptance gates
/// below were set at this size).
crates/engine/src/camera_preview/hue_sat.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::mono::to_prophoto;
use super::pairs::{displayed, luma};
use super::*;
use super::{CameraLook, CameraTone, HsvTable, Mat3, ToneMap, luminance_2020};
/// [`HueSat::apply`] restores the luminance in full only if the table kept at least this share of it. A colour
/// the table darkens to [`NO_LUMINANCE_KEPT`] or less (only near-black noise, whose channels go negative) keeps
crates/engine/src/camera_preview/look_fit.rs
+1 -1
View File
@@ -3,7 +3,7 @@ use super::mono::{fit_tone_only_with, monochrome};
use super::pairs::{Attempt, TONE_FITS, fit_attempt, fit_partial, search_ordered};
use super::proxy::{CLIP_TOLERANCE, LensGeometry, MIN_PREVIEW_FRACTION, proxies, structure_agrees};
use super::tone::{HighlightCheck, highlights};
use super::*;
use super::{CameraLook, CameraTone, CameraTransform, HsvTable, Mat3, RawImage, Rgb32f, ToneMap, luminance_2020};
/// The look fitted to the file's camera JPEG, and whether that JPEG is black and white ([`monochrome`]).
pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, lens: Option<&LensGeometry>) -> (Option<CameraLook>, bool) {
crates/engine/src/camera_preview/mono.rs
+1 -1
View File
@@ -7,7 +7,7 @@ use super::pairs::{
};
use super::proxy::{LensGeometry, proxies};
use super::tone::{HighlightCheck, ToneFit, fit_tone, highlights, highlights_hold};
use super::*;
use super::{CameraLook, CameraTransform, D50, D65, HsvTable, Mat3, PROPHOTO, REC2020, RawImage, Rgb32f, ToneMap, bradford, luminance_2020};
/// A camera JPEG pixel's channels may differ by this much (linear) and still count as grey.
const GREY_SPREAD: f32 = 0.002;
crates/engine/src/camera_preview/pairs.rs
+1 -1
View File
@@ -3,7 +3,7 @@ use super::proxy::DARK_FLOOR;
#[cfg(test)]
use super::tone::highlights;
use super::tone::{HighlightCheck, ToneFit, highlights_hold};
use super::*;
use super::{CameraLook, CameraTone, HsvTable, Mat3, Rgb32f, ToneMap, luminance_2020};
/// A fit must cut the held-out squared error to below this share of the fallback's.
pub(super) const MIN_IMPROVEMENT: f64 = 0.7;
crates/engine/src/camera_preview/proxy.rs
+1 -1
View File
@@ -2,7 +2,7 @@ use super::colour::{PROFILE_PROXY, PROXY, file_local_look};
use super::hue_sat::fit_hue_sat;
use super::look_fit::{FIT_PREVIEW_MAX_PIXELS, SENSOR_CLIP, clip_mask, without_clipped};
use super::pairs::{collect_pairs, fit_matrix};
use super::*;
use super::{CameraTransform, Filter, HsvTable, Mat3, RawFormat, RawImage, Rgb32f, fit, luminance_2020, resize};
/// The embedded preview's long edge must be at least this share of the raw's (and 384 px): a DNG whose preview is a
/// small thumbnail is not fitted. The fit itself runs on a proxy of at most 384 px, so the 1024 px preview
crates/engine/src/camera_preview/tone.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::look_fit::dro_off;
use super::pairs::{Pairs, displayed, luma};
use super::*;
use super::{CameraTone, Rgb32f, ToneMap, luminance_2020};
/// How the per-photo tone curve is fitted ([`fit_pairs_ordered`] tries them in order).
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
crates/engine/src/cmd/browse.rs
+2 -1
View File
@@ -35,8 +35,9 @@ fn browse(s: &mut Session, p: &Value) -> Result<Value> {
let mut v: Vec<String> = std::fs::read_dir(&dir)
.map_err(|e| bad(C, format!("{path}: {e}")))?
.flatten()
.filter(|e| !crate::walk::is_hidden_entry(e))
.map(|e| e.path())
.filter(|f| f.is_file() && is_supported(f) && !f.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')))
.filter(|f| f.is_file() && is_supported(f))
.map(|f| f.to_string_lossy().to_string())
.collect();
v.sort();
crates/engine/src/cmd/denoise.rs
+22 -9
View File
@@ -129,16 +129,18 @@ fn identify(path: &Path, c: &str, sha_known: Option<&str>) -> Result<Source> {
/// Install the model in `src`. A model that passes its self-test becomes the one in use (`activate`, the default): installing
/// a model is how someone says they want it. `verified` is for a download whose hash was already checked.
fn install_file(
dir: &Path,
loader: &crate::denoise::Loader,
struct InstallFile<'a> {
dir: &'a Path,
run_on: RunOn,
accepted_at: String,
c: &str,
src: &Path,
command: &'a str,
src: &'a Path,
acknowledged: bool,
verified: bool,
) -> Result<(Value, Arc<dyn crate::denoise::Model>)> {
}
fn install_file(loader: &crate::denoise::Loader, request: InstallFile<'_>) -> Result<(Value, Arc<dyn crate::denoise::Model>)> {
let InstallFile { dir, run_on, accepted_at, command: c, src, acknowledged, verified } = request;
let ident = identify(src, c, None)?;
let m = ident.manifest.clone();
manifest::validate(&m).map_err(|e| bad(c, e.to_string()))?;
@@ -262,7 +264,8 @@ fn start_install(s: &mut Session, src: PathBuf, id: String, activate: bool, veri
.spawn(move || {
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let _held = crate::memory::work_gate().acquire(512 * 1024 * 1024);
install_file(&dir, &loader, run_on, at, C, &src, true, verified).map_err(|e| plain(&e))
install_file(&loader, InstallFile { dir: &dir, run_on, accepted_at: at, command: C, src: &src, acknowledged: true, verified })
.map_err(|e| plain(&e))
}))
.unwrap_or_else(|_| Err("the model installation stopped unexpectedly".into()));
if verified {
@@ -290,8 +293,18 @@ fn install(s: &mut Session, p: &Value) -> Result<Value> {
if s.denoise.install.is_some() {
return Err(bad(C, "another model is being installed"));
}
let (result, model) =
install_file(&models_dir(s, C)?, &s.denoise.loader, s.denoise.settings.run_on(), (s.clock)(), C, Path::new(path), acknowledged, false)?;
let (result, model) = install_file(
&s.denoise.loader,
InstallFile {
dir: &models_dir(s, C)?,
run_on: s.denoise.settings.run_on(),
accepted_at: (s.clock)(),
command: C,
src: Path::new(path),
acknowledged,
verified: false,
},
)?;
activate_install(s, &result, model, activate)
}
crates/engine/src/cmd/export.rs
+1 -1
View File
@@ -38,7 +38,7 @@ impl Session {
}
// the preset's depth is judged against the format written, known only now (`path`); a
// depth the call passes itself is the caller's, and validated as such
if !p.get("bitDepth").is_some_and(|d| !d.is_null()) {
if p.get("bitDepth").is_none_or(|d| d.is_null()) {
crate::export::ExportOptions::drop_unfit_depth(&mut out, fallback);
}
Ok(out)
crates/engine/src/cmd/geocode.rs
+4 -1
View File
@@ -188,7 +188,10 @@ mod download {
use lightcraft_geocode::{GEONAMES_BASE, GEONAMES_ZIP};
use super::*;
use super::{
Cancel, DownloadOutcome, GEONAMES_ATTRIBUTION, GEONAMES_FILES, GeoNames, PoisonError, Result, Session, Value, bad, dataset_dir, json,
str_param,
};
use crate::activity::Unit;
const MAX_ZIP: u64 = 64 * 1024 * 1024;
crates/engine/src/cmd/library/filters.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{AlbumId, LibrarySource, PhotoId, Result, Session, Value, bad};
/// `base` with a partial Filter (JSON) merged on top.
/// `patch` merged onto `base`, its rules brought up to date (`RuleSet::upgrade`), unchecked (see
crates/engine/src/cmd/library/import.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use super::selection::strs;
use super::*;
use super::{AlbumId, PhotoId, Result, Selection, Session, Value, bad, bool_or, json, str_param};
/// The auto-import folder's visible files with their sizes (the file-system half of
/// `library.autoImportScan`; the app lists on a worker thread and passes `listing`).
crates/engine/src/cmd/library/selection.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{AlbumId, Flag, Op, PhotoId, Result, Selection, Session, Value, bad, bool_or, json, ok};
/// The selection after photos left the view (deleted, or rated, flagged or labelled out of it).
/// Its callers end an open edit (a slider drag) before their own change, as moving to the next
crates/engine/src/cmd/library/specs.rs
+4 -1
View File
@@ -5,7 +5,10 @@ use super::selection::{
album_param, album_targets, album_targets_or_err, change_flag, change_rating, cull, flip, for_targets, has_library, mark_deleted,
opt_album_param, rotate, row_step, select_after_deletion, select_by, step, strs,
};
use super::*;
use super::{
Album, AlbumId, ColorLabel, CommandSpec, CopyrightStatus, Filter, Flag, GroupBy, LibrarySource, Op, PhotoId, Result, Selection, Sort, SortKey,
Value, always, bad, bool_or, cmd, has_active, has_library_selection, has_selection, json, ok, str_param,
};
/// Seeds stay exactly representable as a JSON double, so web and agent clients read back what they wrote.
const MAX_SEED: u64 = 1 << 53;
crates/engine/src/crs/tests.rs
+17 -1
View File
@@ -37,9 +37,16 @@ fn props(x: &str) -> Props {
#[test]
fn maps_common_fields() {
let p = props(SIDECAR);
assert!(has_adjustments(&p));
let partial = to_partial(&p, Target::RawAbsolute);
let s = apply_partial(&DevelopSettings::for_raw(5000.0, 0.0), &partial, 1.0);
assert!(has_adjustments(&p));
assert_common_tone_fields(&s);
assert_common_color_fields(&s);
assert_common_detail_fields(&s);
assert_common_geometry_fields(&s);
}
fn assert_common_tone_fields(s: &DevelopSettings) {
assert_eq!(s.wb.mode, WbMode::Custom);
assert_eq!((s.wb.temp, s.wb.tint), (5150.0, 12.0));
assert_eq!(s.light.exposure, 0.65);
@@ -54,15 +61,24 @@ fn maps_common_fields() {
assert_eq!(s.curve.refine_saturation, 40.0);
assert!((s.curve.master[1].x - 64.0 / 255.0).abs() < 1e-9 && (s.curve.master[1].y - 52.0 / 255.0).abs() < 1e-9);
assert!(s.curve.red.is_empty(), "identity channel curve maps to no curve");
}
fn assert_common_color_fields(s: &DevelopSettings) {
assert_eq!((s.grading.shadows.hue, s.grading.shadows.sat), (210.0, 14.0));
assert_eq!((s.grading.highlights.hue, s.grading.highlights.sat), (40.0, 9.0));
assert_eq!((s.grading.midtones.hue, s.grading.midtones.sat), (120.0, 6.0));
assert_eq!((s.grading.balance, s.grading.blending), (20.0, 70.0));
}
fn assert_common_detail_fields(s: &DevelopSettings) {
assert_eq!((s.detail.sharpen_amount, s.detail.sharpen_radius, s.detail.sharpen_detail, s.detail.sharpen_masking), (55.0, 1.2, 30.0, 12.0));
assert_eq!((s.detail.nr_luminance, s.detail.nr_color), (18.0, 25.0));
assert_eq!(s.vignette.amount, -21.0);
assert_eq!(s.vignette.style, VignetteStyle::ColorPriority);
assert_eq!((s.grain.amount, s.grain.size, s.grain.roughness), (15.0, 30.0, 60.0));
}
fn assert_common_geometry_fields(s: &DevelopSettings) {
assert_eq!(s.geometry.upright, Upright::Level);
assert_eq!(s.geometry.vertical, -10.0);
assert!(s.optics.remove_ca);
crates/engine/src/denoise/gpu.rs
+1 -1
View File
@@ -1,7 +1,7 @@
#[cfg(feature = "denoise")]
use super::model::{GpuSide, STACK};
#[cfg(feature = "denoise")]
use super::*;
use super::{Arc, Duration, Mutex, Path, PathBuf, PoisonError, SystemTime, TileRunner};
/// Tiles run at once on a graphics card: enough to keep it fed while others are packed and blended.
#[cfg(feature = "denoise")]
crates/engine/src/denoise/model.rs
+3 -2
View File
@@ -4,8 +4,9 @@ use super::gpu::{GPU_CRASHED, GPU_MAX_FALLBACKS, GPU_PARALLEL, GPU_SLOW, GPU_STU
use super::settings::{installed_models, read_settings};
#[cfg(feature = "denoise")]
use super::state::Pace;
use super::*;
use super::{Arc, DenoiserManifest, Mutex, Path, TileRunner, Value, json};
#[cfg(feature = "denoise")]
use super::{AtomicBool, AtomicUsize, OnceLock, Ordering, PathBuf, PoisonError, RunError};
/// Stack of the threads that run the model (the runtime recurses through its graph).
pub(super) const STACK: usize = 16 << 20;
crates/engine/src/denoise/products.rs
+4 -1
View File
@@ -1,6 +1,9 @@
use super::model::{Loader, Model, ModelSlot, RunOn, STACK};
use super::settings::EXT;
use super::*;
use super::{
Arc, AtomicBool, AtomicUsize, ByteReader, Condvar, Control, DenoiserManifest, HashSet, Hasher128, Layout, Mutex, OnceLock, Ordering, Params,
Path, PathBuf, Photo, PoisonError, RunError, Source, SystemTime, denoise_bayer, product,
};
/// Samples at or above this (white = 1) are clipped: the model leaves them as the camera recorded them.
const CLIP: f32 = 0.99;
crates/engine/src/denoise/session.rs
+4 -1
View File
@@ -4,7 +4,10 @@ use super::model::STACK;
use super::products::{Active, JobSpec, MakeError, Progress, eligible, make_product, product_key, product_path, products_in};
use super::settings::{DIR, SETTINGS_TTL, installed_models, read_settings};
use super::state::{AUTO_LOOK, Failure, MAX_QUEUE, Outcome, Pace, PhotoState, Running};
use super::*;
use super::{
Arc, DenoiseSpec, DevelopSettings, HashMap, HashSet, MakeProduct, Mutex, Ordering, PairLoader, Path, PathBuf, Photo, PhotoId, PoisonError,
Selection, Session, Source, Value, json, product,
};
impl Session {
/// Where the active model's tiles run (see [`Model::device`]); `{kind: "none"}` while no model is loaded.
crates/engine/src/denoise/settings.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::gpu::GPU_SETUP_MARKER;
use super::model::RunOn;
use super::*;
use super::{DenoiserManifest, Deserialize, Duration, Path, PathBuf, Serialize, Value};
/// Extension of a product file.
pub const EXT: &str = "lcdn";
crates/engine/src/denoise/state.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::model::{Loader, default_loader};
use super::products::{Active, Progress};
use super::settings::Settings;
use super::*;
use super::{Arc, Downloads, HashMap, Mutex, PathBuf, PhotoId, Serialize, VecDeque};
/// Most photos waiting in the queue.
pub(super) const MAX_QUEUE: usize = 5000;
crates/engine/src/export/batch.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::ExportOptions;
use super::place::Placer;
use super::prepare::{Destination, Exported, PreparedExport};
use super::*;
/// Run prepared exports in order: pick each one's path, and hand the bytes (and sidecars) to
/// `write`. `exists` tells whether a path is taken. The conflict policy applies to a file and its
crates/engine/src/export/encoding.rs
+34 -11
View File
@@ -1,9 +1,16 @@
use super::options::{ExportFormat, MetadataPolicy, SharpenAmount, SharpenFor};
use super::watermark::{Watermark, draw_watermark, draw_watermark_deep};
use super::*;
use super::{
ChromaSubsampling, DateTime, DeepImage, DeepSamples, EncodeImage, EncodeMeta, ExportOptions, Gps, Metadata, NamedSpace, OutputDepth, OutputSpace,
Rgba8, Samples, encode, icc,
};
use std::borrow::Cow;
/// Unsharp mask on an 8-bit image (separable box-blur approximation of a small Gaussian).
pub fn output_sharpen(img: &mut Rgba8, target: SharpenFor, amount: SharpenAmount) {
if !sharpens(img.width, img.height, target) {
return;
}
let src: Vec<[f32; 3]> = img.data.iter().map(|p| [p[0] as f32, p[1] as f32, p[2] as f32]).collect();
if let Some(out) = unsharp(img.width, img.height, &src, target, amount) {
for (p, v) in img.data.iter_mut().zip(out) {
@@ -17,6 +24,9 @@ pub fn output_sharpen(img: &mut Rgba8, target: SharpenFor, amount: SharpenAmount
/// [`output_sharpen`] on a high-bit-depth image (float images are sharpened in linear light).
pub fn output_sharpen_deep(img: &mut DeepImage, target: SharpenFor, amount: SharpenAmount) {
let (w, h) = (img.width, img.height);
if !sharpens(w, h, target) {
return;
}
match &mut img.samples {
DeepSamples::U16(v) => {
let src: Vec<[f32; 3]> = v.as_chunks::<3>().0.iter().map(|c| [c[0] as f32, c[1] as f32, c[2] as f32]).collect();
@@ -37,6 +47,11 @@ pub fn output_sharpen_deep(img: &mut DeepImage, target: SharpenFor, amount: Shar
}
}
/// Whether output sharpening for `target` changes a `w × h` image ([`unsharp`] returns `None` otherwise).
fn sharpens(w: usize, h: usize, target: SharpenFor) -> bool {
target != SharpenFor::None && w >= 3 && h >= 3
}
/// The sharpened values of `src` (`w × h`), or `None` when nothing is to be done.
fn unsharp(w: usize, h: usize, src: &[[f32; 3]], target: SharpenFor, amount: SharpenAmount) -> Option<Vec<[f32; 3]>> {
let (radius, base) = match target {
@@ -57,8 +72,9 @@ fn unsharp(w: usize, h: usize, src: &[[f32; 3]], target: SharpenFor, amount: Sha
let r = radius as isize;
let box_pass = |inp: &[[f32; 3]], horizontal: bool| -> Vec<[f32; 3]> {
let mut out = vec![[0.0; 3]; inp.len()];
for y in 0..h {
for x in 0..w {
// rows in parallel: each output pixel is computed on its own, so the bits don't depend on the split
lightcraft_raster::par_rows(&mut out, w, |y, row| {
for (x, o) in row.iter_mut().enumerate() {
let mut acc = [0.0f32; 3];
for d in -r..=r {
let (sx, sy) = if horizontal {
@@ -70,9 +86,9 @@ fn unsharp(w: usize, h: usize, src: &[[f32; 3]], target: SharpenFor, amount: Sha
acc = [acc[0] + p[0], acc[1] + p[1], acc[2] + p[2]];
}
let n = (2 * r + 1) as f32;
out[y * w + x] = acc.map(|v| v / n);
*o = acc.map(|v| v / n);
}
}
});
out
};
let blur = box_pass(&box_pass(src, true), false);
@@ -142,8 +158,12 @@ fn encode_jpeg_with_limit(o: &ExportOptions, jpeg: impl Fn(u8) -> lightcraft_cod
best.ok_or_else(|| format!("cannot fit within {kb} KB"))
}
/// Output sharpening and the watermark on an 8-bit render in `o.effective_space()`.
fn finish_8bit(img: &Rgba8, o: &ExportOptions) -> Rgba8 {
/// Output sharpening and the watermark on an 8-bit render in `o.effective_space()`: the render itself when
/// there is neither (a copy is ~100 MB at 24 MP).
fn finish_8bit<'a>(img: &'a Rgba8, o: &ExportOptions) -> Cow<'a, Rgba8> {
if o.watermark.is_none() && !sharpens(img.width, img.height, o.sharpen) {
return Cow::Borrowed(img);
}
let mut img = img.clone();
output_sharpen(&mut img, o.sharpen, o.sharpen_amount);
let space = o.effective_space();
@@ -151,18 +171,21 @@ fn finish_8bit(img: &Rgba8, o: &ExportOptions) -> Rgba8 {
let wm = Watermark { color: srgb8_in(space, wm.color), target: Some(space), ..wm.clone() };
draw_watermark(&mut img, &wm);
}
img
Cow::Owned(img)
}
/// Output sharpening and the watermark on a high-bit-depth render.
fn finish_deep(img: &DeepImage, o: &ExportOptions) -> DeepImage {
/// Output sharpening and the watermark on a high-bit-depth render (the render itself when there is neither).
fn finish_deep<'a>(img: &'a DeepImage, o: &ExportOptions) -> Cow<'a, DeepImage> {
if o.watermark.is_none() && !sharpens(img.width, img.height, o.sharpen) {
return Cow::Borrowed(img);
}
let mut img = img.clone();
output_sharpen_deep(&mut img, o.sharpen, o.sharpen_amount);
if let Some(wm) = &o.watermark {
let wm = Watermark { color: srgb8_in(img.space, wm.color), target: Some(img.space), ..wm.clone() };
draw_watermark_deep(&mut img, &wm);
}
img
Cow::Owned(img)
}
/// Encode an ISO 21496-1 gain map JPEG from the SDR rendition `sdr` (8-bit, in the output space)
crates/engine/src/export/options.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{Deserialize, Serialize};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
crates/engine/src/export/params.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::options::{Conflict, DestinationMode, ExportFormat, HdrCurve, MetadataPolicy, Resize, ResizeMode, SharpenAmount, SharpenFor};
use super::prepare::DngCompression;
use super::watermark::{Anchor, Watermark};
use super::*;
use super::{ExportOptions, JxlColour, JxlEffort, JxlOptions, OutputDepth, OutputSpace, TiffCompression};
impl Default for ExportOptions {
fn default() -> Self {
crates/engine/src/export/place.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::ExportOptions;
use super::options::{Conflict, DestinationMode};
use super::prepare::{Destination, Exported};
use super::*;
/// Write an exported or rendered file on disk: its folder is created if needed, and the file is
/// replaced atomically ([`lightcraft_catalog::safe_file::write_atomic_nosync`]: a temp file
crates/engine/src/export/prepare.rs
+1 -1
View File
@@ -2,7 +2,7 @@ use super::batch::run_batch;
use super::encoding::{encode_gain_map_jpeg, encode_rendered, export_metadata, removes_location};
use super::options::ExportFormat;
use super::params::apple_photos_unsupported;
use super::*;
use super::{ExportOptions, Metadata, OutputDepth};
/// One exported file.
pub struct Exported {
crates/engine/src/export/tests.rs
+46
View File
@@ -209,6 +209,42 @@ fn sharpen_increases_edge_contrast() {
assert!(img.data[3][0] < 80 && img.data[4][0] > 170);
}
/// Output sharpening's pixels for every target and amount, pinned (its box blurs run row-parallel, which must not
/// change a bit), on 8- and 16-bit images; with no target the image is left as it is.
#[test]
fn sharpened_pixels_are_pinned() {
let fnv = |b: &[u8]| b.iter().fold(0xcbf2_9ce4_8422_2325u64, |h, &x| (h ^ u64::from(x)).wrapping_mul(0x100_0000_01b3));
let mut got = Vec::new();
for target in [SharpenFor::Screen, SharpenFor::Matte, SharpenFor::Glossy] {
for amount in [SharpenAmount::Low, SharpenAmount::High] {
let mut img = test_image();
output_sharpen(&mut img, target, amount);
got.push(fnv(img.data.as_flattened()));
}
let src = test_image();
let samples = src.data.iter().flat_map(|p| [p[0], p[1], p[2]].map(|v| u16::from(v) * 257)).collect();
let mut deep = DeepImage { width: src.width, height: src.height, samples: DeepSamples::U16(samples), space: OutputSpace::Srgb };
output_sharpen_deep(&mut deep, target, SharpenAmount::Standard);
let DeepSamples::U16(v) = &deep.samples else { unreachable!() };
got.push(fnv(&v.iter().flat_map(|s| s.to_le_bytes()).collect::<Vec<u8>>()));
}
assert_eq!(got, SHARPENED, "{got:#x?}");
let mut img = test_image();
output_sharpen(&mut img, SharpenFor::None, SharpenAmount::High);
assert_eq!(img, test_image());
}
const SHARPENED: [u64; 9] = [
0x1494_273e_0bdc_a61c,
0x3fbb_40c4_74f4_b0c3,
0xa27f_8ff5_82a0_99f9,
0x6550_6571_a38d_4d9f,
0x5e28_f4f3_fc8a_9685,
0xdc2b_6f65_9138_3b1d,
0x8ce6_f0b8_d8ed_8026,
0x92d8_12e7_4b2e_81fe,
0x139f_6d78_b1f2_a910,
];
/// Exported files carry keywords as Lightroom Classic writes them: names flat in `dc:subject`
/// (with the keywords containing them), full paths in `lr:hierarchicalSubject`, and nothing of
/// a keyword left out of export. They used to carry the `a|b` paths in `dc:subject`.
@@ -1031,6 +1067,11 @@ fn avif_ten_bit() {
/// unknown keys (naming the nearest known one) and values of the wrong kind or range.
#[test]
fn export_params_are_validated() {
assert_invalid_export_params();
assert_export_options_round_trip();
}
fn assert_invalid_export_params() {
use serde_json::json;
let err = |p: serde_json::Value| ExportOptions::from_params(&p).err().map(|e| e.to_string()).unwrap_or_default();
let e = err(json!({"longEdgee": 400}));
@@ -1056,6 +1097,11 @@ fn export_params_are_validated() {
.to_string()
.starts_with("invalid parameters for `export.savePreset`")
);
}
fn assert_export_options_round_trip() {
use serde_json::json;
let err = |p: serde_json::Value| ExportOptions::from_params(&p).err().map(|e| e.to_string()).unwrap_or_default();
// everything the presets, the dialog, Export with Previous and `render --opt` send passes
for p in builtin_presets() {
ExportOptions::from_params(&p.params).unwrap_or_else(|e| panic!("{}: {e}", p.name));
crates/engine/src/export/watermark.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use super::encoding::srgb8_in;
use super::*;
use super::{DeepImage, DeepSamples, Deserialize, OutputSpace, Rgba8, Serialize};
/// Where a watermark sits.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
crates/engine/src/faces_worker.rs
+6 -8
View File
@@ -13,7 +13,7 @@ use std::sync::{Arc, Condvar, Mutex, PoisonError};
use std::time::Duration;
use lightcraft_catalog::PhotoId;
use lightcraft_faces::align::{Rgb, align_to_template, crop_box};
use lightcraft_faces::align::{FaceCrop, Rgb, align_to_template, crop_box};
use lightcraft_faces::runtime::Embedder;
use lightcraft_faces::yunet::{Detector, Face, Options};
use lightcraft_geom::Rect;
@@ -221,13 +221,11 @@ pub(crate) fn process(p: Prepared, embedder: &Embedder) -> Done {
Some((_, f)) => align_to_template(&img, &scaled(f), iw, ih),
None => crop_box(
&img,
rect.x0 as f32 * w as f32,
rect.y0 as f32 * h as f32,
rect.x1 as f32 * w as f32,
rect.y1 as f32 * h as f32,
BOX_PAD,
iw,
ih,
FaceCrop {
bounds: (rect.x0 as f32 * w as f32, rect.y0 as f32 * h as f32, rect.x1 as f32 * w as f32, rect.y1 as f32 * h as f32),
padding: BOX_PAD,
output_size: (iw, ih),
},
),
};
// shown by the detector's box where it found this face: every face then has the same tightness
crates/engine/src/files/fs.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::load::{load_vec, load_vec_pair, load_vec_reduced};
use super::preview::{fit_box, fs_preview_loader, preview_orientation};
use super::probe::probe_bytes;
use super::*;
use super::{Arc, DenoiseSpec, FileLoader, FileProbe, Filter, PairLoader, ReducedLoader, fit};
/// The file's first [`lightcraft_raw::PREVIEW_PREFIX`] bytes (or the whole file when it is shorter),
/// which is what every format [`lightcraft_raw::bounded_preview`] covers keeps its pointer in.
crates/engine/src/files/load.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::colour::bin_factor;
use super::preview::{fit_box, load_embedded_preview};
use super::probe::preview_reason;
use super::*;
use super::{Arc, CameraColourModel, DenoiseSpec, Filter, HIGHLIGHT_CLIP, Orientation, Rgb32f, SourceInfo, fit, xy_to_temp_tint};
/// Decode a file into a linear Rec.2020 image no larger than `max_edge`, oriented.
///
crates/engine/src/files/preview.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use super::fs::{quick_image_srgb, raf_preview_extent, read_prefix, read_prefix_at};
use super::*;
use super::{Arc, Filter, Orientation, PreviewLoader, Rgb32f, SourceInfo, fit};
/// Orientation for an embedded preview: its own EXIF orientation when it has one, else the raw file's.
pub(super) fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
crates/engine/src/files/probe.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::load::WbScaleOf;
use super::preview::embedded_preview_size;
use super::*;
use super::{MediaKind, Meta, Orientation, ProbeInfo};
pub(super) fn meta_of(m: &lightcraft_meta::Metadata) -> (Meta, Option<String>) {
let shutter = m.exposure_time.map(|t| if t >= 1.0 { format!("{t:.0}") } else { format!("1/{:.0}", 1.0 / t) }).unwrap_or_default();
crates/engine/src/import/commit.rs
+1 -1
View File
@@ -2,7 +2,7 @@ use super::defaults::apply_import_defaults;
use super::files::copy_into;
use super::job::{Prepared, PreparedItem, ReadyFile};
use super::types::{Duplicate, ImportOptions, ImportReport, Kept, Moved, OnDeleted};
use super::*;
use super::{HashMap, Op, Path, PathBuf, Photo, PhotoId, ProbeInfo, Session, Source};
/// A batch's items in file order while its transfers are pending ([`ImportJob::run_plan`]).
#[derive(Default)]
crates/engine/src/import/defaults.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{Deserialize, Photo, Serialize, Session};
/// The develop settings a photo gets on import: raws start from their as-shot white balance with
/// default sharpening / colour noise reduction, and file-embedded lens corrections on (as the
crates/engine/src/import/files.rs
+2 -1
View File
@@ -5,7 +5,7 @@ use super::job::{ImportJob, Prepared, PreparedItem, ReadyFile};
use super::scan::ScanProgress;
#[cfg(test)]
use super::types::{ImportMode, ImportOptions};
use super::*;
use super::{Path, PathBuf, ProbeInfo, Session};
/// File extensions offered by import discovery and file pickers (lower case).
/// This includes preview-only raws; the file probe decides whether sensor decoding is available.
@@ -253,6 +253,7 @@ pub(super) fn copy_into(dir: &Path, src: &str, name: Option<&str>, probe_hash: O
#[cfg(test)]
mod prepared_tests {
use super::*;
use lightcraft_catalog::{Op, Photo, PhotoId, Source};
/// Extensions the picker may offer: every one in [`EXTENSIONS`], minus the
/// files this build cannot import (see [`accept_list`]).
crates/engine/src/import/job.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::commit::{How, Plan, Slot, Transfer, commit_prepared, name_family, run_transfers, system_clock};
use super::files::{expand, probe_all};
use super::types::{ImportCandidate, ImportMode, ImportOptions, ImportReport, Kept, OnDeleted};
use super::*;
use super::{HashMap, MediaKind, Path, PathBuf, Photo, PhotoId, ProbeInfo, Session, Source};
/// Import files/folders. See the module docs.
pub fn import(s: &mut Session, paths: &[String], mode: ImportMode) -> crate::Result<ImportReport> {
crates/engine/src/import/scan.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::files::{expand, probe_all};
use super::types::ImportCandidate;
use super::*;
use super::{HashMap, Path, PathBuf, PhotoId, ProbeInfo, Session, Source};
/// What a [`scan`] needs from the session, so it can run on another thread (a folder on a network
/// share can take minutes to read).
crates/engine/src/import/types.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{Deserialize, MediaKind, Photo, Serialize};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum ImportMode {
crates/engine/src/lightroom_catalog.rs
+79 -67
View File
@@ -858,6 +858,83 @@ pub(crate) struct ApplyPrepared<'a> {
pub(crate) undo0: usize,
}
fn stage_collections(
s: &mut crate::Session,
data: &CatalogImport,
index: &mut ImportIndex,
ids: &HashMap<i64, PhotoId>,
ops: &mut Vec<Op>,
warnings: &mut Vec<String>,
) -> usize {
let mut albums = HashMap::new();
for collection in &data.collections {
if number(collection, "systemOnly") != 0 {
continue;
}
let collection_key = format!("{}:{}", key(&data.source), number(collection, "id_local"));
let id = index
.collections
.get(&collection_key)
.copied()
.map(lightcraft_catalog::AlbumId)
.filter(|id| s.catalog.album(*id).is_some())
.unwrap_or_else(|| s.catalog.alloc_album_id());
index.collections.insert(collection_key, id.0);
albums.insert(number(collection, "id_local"), id);
}
let parents: HashMap<_, _> = data.collections.iter().map(|r| (number(r, "id_local"), number(r, "parent"))).collect();
let mut grouped_members: HashMap<i64, Vec<&Row>> = HashMap::new();
for member in &data.members {
grouped_members.entry(number(member, "collection")).or_default().push(member);
}
for members in grouped_members.values_mut() {
members.sort_by(|a, b| text(a, "positionInCollection").cmp(text(b, "positionInCollection")));
}
let mut ordered: Vec<_> = data.collections.iter().collect();
ordered.sort_by_key(|r| {
let mut depth = 0;
let mut at = number(r, "parent");
while at != 0 && depth < 64 {
depth += 1;
at = parents.get(&at).copied().unwrap_or(0);
}
depth
});
for collection in ordered {
let Some(id) = albums.get(&number(collection, "id_local")).copied() else { continue };
let mut album = Album::new(id, text(collection, "name"));
album.parent = albums.get(&number(collection, "parent")).copied();
album.folder = text(collection, "creationId").contains("group");
if text(collection, "creationId").contains("smart") {
warnings.push(format!("{}: smart rules preserved in archive; imported current membership", album.name));
}
if !album.folder {
album.photos = grouped_members
.get(&number(collection, "id_local"))
.into_iter()
.flatten()
.filter_map(|r| ids.get(&number(r, "image")).copied())
.collect();
}
if s.catalog.album(id).is_some() {
// Reimport preserves the user's album name/hierarchy, merges only new membership.
if let Some(old) = s.catalog.album(id).filter(|a| !a.folder && !a.is_smart()) {
let mut photos = old.photos.clone();
let mut seen: HashSet<_> = photos.iter().copied().collect();
for photo in album.photos {
if seen.insert(photo) {
photos.push(photo);
}
}
ops.push(Op::SetAlbumPhotos { id, photos });
}
} else {
ops.push(Op::AddAlbum { album });
}
}
albums.len()
}
/// Turn the read catalog into library records in one undo step and return the import report
/// (added, missing, failed, warnings and skipped images).
pub(crate) fn apply_prepared(s: &mut crate::Session, mut data: CatalogImport, context: ApplyPrepared<'_>) -> crate::Result<AppliedImport> {
@@ -1022,78 +1099,13 @@ pub(crate) fn apply_prepared(s: &mut crate::Session, mut data: CatalogImport, co
}
by_path.entry(key(&src.path)).or_insert(photo_id);
}
let mut albums = HashMap::new();
for collection in &data.collections {
if number(collection, "systemOnly") != 0 {
continue;
}
let collection_key = format!("{}:{}", key(&data.source), number(collection, "id_local"));
let id = index
.collections
.get(&collection_key)
.copied()
.map(lightcraft_catalog::AlbumId)
.filter(|id| s.catalog.album(*id).is_some())
.unwrap_or_else(|| s.catalog.alloc_album_id());
index.collections.insert(collection_key, id.0);
albums.insert(number(collection, "id_local"), id);
}
let parents: HashMap<_, _> = data.collections.iter().map(|r| (number(r, "id_local"), number(r, "parent"))).collect();
let mut grouped_members: HashMap<i64, Vec<&Row>> = HashMap::new();
for member in &data.members {
grouped_members.entry(number(member, "collection")).or_default().push(member);
}
for members in grouped_members.values_mut() {
members.sort_by(|a, b| text(a, "positionInCollection").cmp(text(b, "positionInCollection")));
}
let mut ordered: Vec<_> = data.collections.iter().collect();
ordered.sort_by_key(|r| {
let mut depth = 0;
let mut at = number(r, "parent");
while at != 0 && depth < 64 {
depth += 1;
at = parents.get(&at).copied().unwrap_or(0);
}
depth
});
for collection in ordered {
let Some(id) = albums.get(&number(collection, "id_local")).copied() else { continue };
let mut album = Album::new(id, text(collection, "name"));
album.parent = albums.get(&number(collection, "parent")).copied();
album.folder = text(collection, "creationId").contains("group");
if text(collection, "creationId").contains("smart") {
warnings.push(format!("{}: smart rules preserved in archive; imported current membership", album.name));
}
if !album.folder {
album.photos = grouped_members
.get(&number(collection, "id_local"))
.into_iter()
.flatten()
.filter_map(|r| ids.get(&number(r, "image")).copied())
.collect();
}
if s.catalog.album(id).is_some() {
// Reimport preserves the user's album name/hierarchy, merges only new membership.
if let Some(old) = s.catalog.album(id).filter(|a| !a.folder && !a.is_smart()) {
let mut photos = old.photos.clone();
let mut seen: HashSet<_> = photos.iter().copied().collect();
for photo in album.photos {
if seen.insert(photo) {
photos.push(photo);
}
}
ops.push(Op::SetAlbumPhotos { id, photos });
}
} else {
ops.push(Op::AddAlbum { album });
}
}
let album_count = stage_collections(s, &data, index, &ids, &mut ops, &mut warnings);
if !ops.is_empty() {
s.commit("Import Lightroom Catalog", Op::Batch { ops })?;
}
s.merge_undo(s.undo.len().saturating_sub(undo0), "Import Lightroom Catalog");
Ok(AppliedImport {
report: json!({"source":data.source,"photos":ids.len(),"imported":imported,"library":s.library.as_ref().filter(|l| l.on_disk).map(|l| l.dir.to_string_lossy()),"collections":albums.len(),"preservedExistingEdits":preserved,"missing":missing,"failed":files.failed,"warnings":warnings,"skipped":data.skipped,"unmapped":unmapped,"archive":archive_path,"mapping":ids}),
report: json!({"source":data.source,"photos":ids.len(),"imported":imported,"library":s.library.as_ref().filter(|l| l.on_disk).map(|l| l.dir.to_string_lossy()),"collections":album_count,"preservedExistingEdits":preserved,"missing":missing,"failed":files.failed,"warnings":warnings,"skipped":data.skipped,"unmapped":unmapped,"archive":archive_path,"mapping":ids}),
index: index.clone(),
})
}
crates/engine/src/media/cache.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::settings_hash::SettingsHashes;
use super::source::{DecodedSource, DenoiseSpec, PairLoader, RENDERED_MEM_BYTES, SourceLevel, SourceRef, THUMB_SOURCE_BYTES};
use super::*;
use super::{Arc, DevelopSettings, Hasher128, Lru, MediaCache, Photo, PhotoId, PreviewCache, Rgb32f, Source, SourceInfo};
/// The photos that have a cached denoised picture for the denoise model in use. Only memory: nothing here touches the
/// file system, so it can be asked on the UI thread.
crates/engine/src/media/quick.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::render::{RenderJob, RenderResult};
use super::source::{PreviewLoader, SourceLevel};
use super::*;
use super::{Arc, Deserialize, Hash128, Histogram, Photo, PhotoId, PreviewCache, Rendered, Rgba8, Serialize, Source, SourceInfo};
/// Where a [`QuickJob`]'s stand-in image came from.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
crates/engine/src/media/render.rs
+3 -1
View File
@@ -1,6 +1,8 @@
use super::quick::QuickSource;
use super::source::{DecodedSource, SourceLevel, SourceRef};
use super::*;
use super::{
Arc, DevelopSettings, Hash128, Histogram, PhotoId, PreviewCache, Quality, RenderRequest, Rendered, Rgb32f, Rgba8, Source, SourceInfo, StageCache,
};
/// Everything needed to render one photo, detached from the session.
#[derive(Clone)]
crates/engine/src/media/session.rs
+1 -1
View File
@@ -1,7 +1,7 @@
use super::quick::{QuickJob, content_key, source_info};
use super::render::{RenderJob, RenderResult};
use super::source::{PreviewLoader, RENDER_CACHE_VERSION, SourceLevel, SourceRef, THUMB_SIZES, face_edge, source_edge_needed, thumb_bucket};
use super::*;
use super::{Arc, DevelopSettings, Hash128, Hasher128, MediaKind, Op, Photo, PhotoId, RenderRequest, Rendered, Rgb32f, Source, SourceInfo};
impl crate::Session {
/// Build a render job for `id` fitting `max_w × max_h`. `before` renders the unedited look.
crates/engine/src/media/settings_hash.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{Arc, DevelopSettings, HashMap, PhotoId, VecDeque, Weak};
pub(super) const SETTINGS_HASH_ENTRIES: usize = 1024;
crates/engine/src/media/source.rs
+2 -2
View File
@@ -1,7 +1,7 @@
use super::*;
use super::{Arc, Deserialize, DevelopSettings, MediaKind, Photo, Rgb32f, Rgba8, Serialize, SourceInfo};
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 70;
pub const RENDER_CACHE_VERSION: u64 = 71;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
crates/engine/src/originals.rs
+52 -13
View File
@@ -9,7 +9,7 @@ use std::path::{Path, PathBuf};
#[cfg(not(any(unix, windows)))]
use std::time::SystemTime;
use lightcraft_catalog::safe_file::same_file;
use lightcraft_catalog::safe_file::{same_file, same_file_strict};
use lightcraft_catalog::{Catalog, Source};
/// The directory entry a screenshot is about to write, pinned at request admission.
@@ -147,14 +147,30 @@ impl OriginalGuard {
if let Some(n) = std::fs::canonicalize(target).ok().and_then(|c| c.file_name().map(|n| n.to_string_lossy().to_lowercase())) {
names.push(n);
}
// The identity check refuses a target whose file identity cannot be read, so an unreadable
// hard link to an original is not taken for an ordinary file.
let refuse_if_same = |p: &Path, what: &str| -> Result<(), String> {
let same = if hard_links {
same_file_strict(p, target).map_err(|e| {
format!(
"{}: LightCraft cannot tell whether this is {what} in the library ({e}) and never writes over an original (choose another folder or file name)",
target.display()
)
})?
} else {
same_file(p, target)
};
if same {
return Err(format!(
"{} is {what} in the library: LightCraft never writes over an original (choose another folder or file name)",
target.display()
));
}
Ok(())
};
for n in &names {
for (p, what) in self.by_name.get(n).into_iter().flatten() {
if same_file(p, target) {
return Err(format!(
"{} is {what} in the library: LightCraft never writes over an original (choose another folder or file name)",
target.display()
));
}
refuse_if_same(p, what)?;
}
}
// An imported hard link or symlink can have a different name from its original. A write
@@ -166,12 +182,7 @@ impl OriginalGuard {
if !hard_links && std::fs::read_link(p).is_err() {
continue;
}
if same_file(p, target) {
return Err(format!(
"{} is {what} in the library: LightCraft never writes over an original (choose another folder or file name)",
target.display()
));
}
refuse_if_same(p, what)?;
}
Ok(())
}
@@ -339,4 +350,32 @@ mod tests {
}
let _ = std::fs::remove_dir_all(&d);
}
#[cfg(windows)]
#[test]
fn a_hard_link_whose_identity_cannot_be_read_is_refused() {
use std::os::windows::fs::OpenOptionsExt;
let d = std::env::temp_dir().join(format!("lc-unreadable-link-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
let mut s = crate::Session::new();
let original = d.join("IMG_1.jpg");
std::fs::write(&original, b"original").unwrap();
let photo = Photo::new(PhotoId(1), Source::File { path: original.to_string_lossy().to_string() }, "IMG_1.jpg", "JPEG", 10, 10, "2026-01-01");
s.catalog.apply(Op::AddPhoto { photo: Box::new(photo) }).unwrap();
let linked = d.join("shot.png");
std::fs::hard_link(&original, &linked).unwrap();
// A handle that shares nothing makes every other open of the file fail, so the file
// identity of either name cannot be read while it is held.
let held = std::fs::OpenOptions::new().read(true).share_mode(0).open(&original).unwrap();
let refused = s.pin_write_target(&linked.to_string_lossy()).unwrap_err();
assert!(refused.contains("cannot tell whether"), "{refused}");
drop(held);
let refused = s.pin_write_target(&linked.to_string_lossy()).unwrap_err();
assert!(refused.contains("original"), "{refused}");
assert!(!refused.contains("cannot tell whether"), "{refused}");
let _ = std::fs::remove_dir_all(&d);
}
}
crates/engine/src/preset_import.rs
+23 -2
View File
@@ -502,12 +502,16 @@ pub fn group_from_dir(dir: &str) -> Option<String> {
}
pub(crate) fn build(props: &Props, values: &crate::crs_masks::Values, name: String, group: Option<String>, from_photo: bool) -> Option<Imported> {
let (mut settings, unmapped) = crate::crs::to_partial_report(props, Some(values), crate::crs::Target::Any, crate::crs_masks::DEFAULT_ASPECT);
let (mut settings, mut unmapped) = crate::crs::to_partial_report(props, Some(values), crate::crs::Target::Any, crate::crs_masks::DEFAULT_ASPECT);
if from_photo && let Some(o) = settings.as_object_mut() {
// a photo's own framing and absolute white balance don't belong in a look
o.remove("crop");
o.remove("geometry");
if o.get("wb").and_then(|w| w.get("mode")).and_then(Value::as_str).is_none_or(|m| m == "custom" || m == "asShot") {
let mode = o.get("wb").and_then(|w| w.get("mode")).and_then(Value::as_str);
if mode == Some("custom") {
unmapped.push("Temperature, Tint (custom white balance of a photo: not kept in a look)".into());
}
if mode.is_none_or(|m| m == "custom" || m == "asShot") {
o.remove("wb");
}
}
@@ -750,6 +754,23 @@ text]], z = ZSTR "loc" }"#,
assert!(read_presets("plain.jpg", b"\xff\xd8 no xmp", None).is_err());
}
#[test]
fn photo_custom_white_balance_left_out_is_reported() {
// issue #2938: a photo's custom Kelvin that the look leaves out is reported
let photo = |wb: &str| {
let mut file = b"\xff\xd8\xff\xe1 http://ns.adobe.com/xap/1.0/\0".to_vec();
file.extend_from_slice(XMP.replace(r#"crs:WhiteBalance="As Shot" crs:Temperature="5600""#, wb).as_bytes());
read_presets("/x/repro.jpg", &file, None).unwrap().remove(0)
};
let custom = photo(r#"crs:WhiteBalance="Custom" crs:Temperature="8000" crs:Tint="+16""#);
assert!(custom.unmapped.iter().any(|u| u.starts_with("Temperature, Tint (")), "{:?}", custom.unmapped);
// a named white balance stays, unreported; As Shot is no loss
let daylight = photo(r#"crs:WhiteBalance="Daylight""#);
assert_eq!(daylight.preset.settings["wb"]["mode"], json!("daylight"));
assert!(daylight.unmapped.is_empty(), "{:?}", daylight.unmapped);
assert!(photo(r#"crs:WhiteBalance="As Shot""#).unmapped.is_empty());
}
#[test]
fn zip_bombs_are_refused() {
// an entry unpacking past the limit (3 MB of zeros pack into a few kB): refused, named
crates/engine/src/print/compose.rs
+15 -4
View File
@@ -255,6 +255,15 @@ pub struct Image16 {
pub rgb: Vec<u16>,
}
/// A rectangular image region used as a source or destination for `Image16::blit`.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct BlitRect {
pub x: usize,
pub y: usize,
pub w: usize,
pub h: usize,
}
impl Image16 {
pub fn filled(w: usize, h: usize, c: [u16; 3]) -> Result<Self, String> {
let n = w.checked_mul(h).and_then(|n| n.checked_mul(3)).ok_or("image too large")?;
@@ -284,9 +293,11 @@ impl Image16 {
}
/// Copies the `w` x `h` block of `src` at (`sx`, `sy`) to (`dx`, `dy`), clipped to both images.
pub fn blit(&mut self, src: &Image16, sx: usize, sy: usize, dx: usize, dy: usize, w: usize, h: usize) {
let w = w.min(src.w.saturating_sub(sx)).min(self.w.saturating_sub(dx));
let h = h.min(src.h.saturating_sub(sy)).min(self.h.saturating_sub(dy));
pub fn blit(&mut self, src: &Image16, source: BlitRect, destination: BlitRect) {
let BlitRect { x: sx, y: sy, .. } = source;
let BlitRect { x: dx, y: dy, .. } = destination;
let w = source.w.min(destination.w).min(src.w.saturating_sub(sx)).min(self.w.saturating_sub(dx));
let h = source.h.min(destination.h).min(src.h.saturating_sub(sy)).min(self.h.saturating_sub(dy));
for row in 0..h {
let from = src.rgb.get(((sy + row) * src.w + sx) * 3..((sy + row) * src.w + sx + w) * 3);
let to = self.rgb.get_mut(((dy + row) * self.w + dx) * 3..((dy + row) * self.w + dx + w) * 3);
@@ -424,7 +435,7 @@ pub fn draw_cell(page: &mut Image16, photo: &Image16, cell: PxRect, style: &Cell
if border > 0 {
page.fill_rect(dx - border, dy - border, w + 2 * border, h + 2 * border, style.border);
}
page.blit(&img, sx, sy, dx, dy, w, h);
page.blit(&img, BlitRect { x: sx, y: sy, w, h }, BlitRect { x: dx, y: dy, w, h });
let s = style.stroke_px.min(w.min(h) / 2);
if s > 0 {
page.fill_rect(dx, dy, w, s, style.stroke);
crates/engine/src/print/compose/tests.rs
+2 -2
View File
@@ -182,9 +182,9 @@ fn rotate_and_blit_are_clipped() {
let r = img.rotated_cw().unwrap();
assert_eq!((r.w, r.h), (2, 3));
let mut p = page();
p.blit(&img, 0, 0, 99, 99, 10, 10);
p.blit(&img, BlitRect { x: 0, y: 0, w: 10, h: 10 }, BlitRect { x: 99, y: 99, w: 10, h: 10 });
assert_eq!(at(&p, 99, 99), at(&img, 0, 0));
p.blit(&img, 5, 5, 0, 0, 3, 3);
p.blit(&img, BlitRect { x: 5, y: 5, w: 3, h: 3 }, BlitRect { x: 0, y: 0, w: 3, h: 3 });
p.fill_rect(95, 95, 50, 50, RED);
assert_eq!(at(&p, 99, 99), RED);
p.fill_rect(usize::MAX, usize::MAX, usize::MAX, usize::MAX, RED);
crates/engine/src/rename/fs.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use super::sidecar::PacketVersion;
use super::*;
use super::{Path, PathBuf, SidecarNaming};
/// Every sidecar that may belong to `path` (both naming conventions, `.xmp` and `.XMP`).
pub(super) fn sidecars(path: &str) -> Vec<(PathBuf, SidecarNaming, bool)> {
crates/engine/src/rename/moves.rs
+1 -1
View File
@@ -1,6 +1,6 @@
use super::fs::{MoveFs, RealFs, folded_path, is_respelling, move_one, sidecars};
use super::sidecar::{RetainedPacket, SidecarMove, advance_packets, note_companions, packet_aliases};
use super::*;
use super::{HashMap, HashSet, Op, Path, PathBuf, SidecarNaming, Source};
/// Why [`move_file`] failed, and what it left behind.
#[derive(Clone, Debug, PartialEq)]
crates/engine/src/rename/plan.rs
+1 -1
View File
@@ -2,7 +2,7 @@ use super::fs::{MoveFs, RealFs, folded_path};
use super::moves::{BatchMoveError, move_all, set_file_ops_to};
use super::sidecar::{PacketVersion, retry_sidecars, retry_sidecars_for_paths};
use super::tokens::{expand, split_ext};
use super::*;
use super::{EngineError, HashMap, HashSet, Op, Path, PathBuf, Photo, PhotoId, RenamePlan, Result, Session, Source};
/// The photos among `ids` a rename handles, in order: photos sharing one file (virtual copies)
/// rename once. No file-system calls.
crates/engine/src/rename/sidecar.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use super::fs::{MoveFs, is_respelling, move_one};
use super::*;
use super::{Path, PathBuf};
/// An unfinished sidecar operation. Keep its exact paths and any recorded copy requirement: the photo may
/// already have followed its new path, so a retry cannot infer this work from the catalog.
crates/engine/src/rename/tokens.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use super::*;
use super::{MONTHS, Path, Photo, PhotoId, Serialize, Source};
pub(super) fn split_ext(name: &str) -> (&str, &str) {
match name.rsplit_once('.') {
crates/engine/src/sidecar/tests_repair_companions.rs
+73 -73
View File
@@ -73,85 +73,85 @@ fn partial_move_companions_allow_save_and_autowrite_without_weakening_collisions
for upper_full in [false, true] {
for naming in [SidecarNaming::Stem, SidecarNaming::Full] {
for auto in [false, true] {
let dir = crate::tests_xmp::temp_dir(&format!("r3-companions-{upper_stem}-{upper_full}-{naming:?}-{auto}"));
let a = dir.join("a.jpg");
let b = dir.join("Trip.jpg");
std::fs::write(&a, b"physical fixture; Session rename does not decode").unwrap();
let path = |photo: &Path, style, upper| {
let path = sidecar_path(&photo.to_string_lossy(), style);
if upper { path.with_extension("XMP") } else { path }
};
let stem_a = path(&a, SidecarNaming::Stem, upper_stem);
let full_a = path(&a, SidecarNaming::Full, upper_full);
let stem_b = path(&b, SidecarNaming::Stem, upper_stem);
let full_b = path(&b, SidecarNaming::Full, upper_full);
let stem_bytes = packet("stem-keep");
let full_bytes = packet("full-keep");
std::fs::write(&stem_a, &stem_bytes).unwrap();
std::fs::write(&full_a, &full_bytes).unwrap();
let mut s = session(&a);
s.xmp.naming = naming;
s.xmp.auto_write = auto;
let fs = Faults(vec![full_a.clone(), b.clone()]);
let plans = s.plan_rename_with(&fs, &[PhotoId(1)], "Trip", 1);
assert!(s.execute_fn("photo.rename", |s| Ok(json!({"renamed": s.apply_rename_with(&fs, &plans)?}))).is_err());
assert_eq!(s.catalog.photo(PhotoId(1)).unwrap().source, Source::File { path: b.to_string_lossy().into_owned() });
let pending = queues(&s);
assert_eq!(pending.len(), 1);
assert_eq!(std::fs::read_to_string(&stem_b).unwrap(), stem_bytes);
// Missing packet and an unrelated packet at either case spelling remain protected.
let held = dir.join("held.xmp");
std::fs::rename(&full_a, &held).unwrap();
s.execute("photo.setMeta", &json!({"title": "Fresh", "addKeywords": ["café"]})).unwrap();
assert_eq!(save(&mut s)["failed"].as_array().unwrap().len(), 1);
assert_eq!(queues(&s), pending);
assert_eq!(std::fs::read_to_string(&stem_b).unwrap(), stem_bytes);
std::fs::rename(&held, &full_a).unwrap();
for collision in [full_b.clone(), full_b.with_extension(if upper_full { "xmp" } else { "XMP" })] {
std::fs::write(&collision, packet("unrelated-packet")).unwrap();
let occupied = std::fs::read(&collision).unwrap();
assert_eq!(save(&mut s)["failed"].as_array().unwrap().len(), 1);
assert_eq!(queues(&s), pending);
assert_eq!(std::fs::read(&collision).unwrap(), occupied);
assert_eq!(std::fs::read_to_string(&full_a).unwrap(), full_bytes);
assert_eq!(std::fs::read_to_string(&stem_b).unwrap(), stem_bytes);
std::fs::remove_file(collision).unwrap();
}
// AutoWrite is the first successful repair when enabled; explicit save otherwise.
s.execute("photo.rate", &json!({"rating": 4})).unwrap();
if !auto {
assert_eq!(save(&mut s)["failed"].as_array().unwrap().len(), 0);
}
assert!(queues(&s).is_empty());
assert!(!full_a.exists());
let (written, untouched, expected) =
if naming == SidecarNaming::Stem { (&stem_b, &full_b, &full_bytes) } else { (&full_b, &stem_b, &stem_bytes) };
assert_fresh(written);
assert_eq!(std::fs::read_to_string(untouched).unwrap(), *expected);
assert_eq!(parse_sidecar(&std::fs::read_to_string(written).unwrap(), crate::crs::Target::Rendered).unwrap().rating, Some(4));
for (command, rating) in [("edit.undo", 0), ("edit.redo", 4)] {
s.execute(command, &json!({})).unwrap();
if !auto {
save(&mut s);
}
assert_fresh(written);
assert_eq!(
parse_sidecar(&std::fs::read_to_string(written).unwrap(), crate::crs::Target::Rendered).unwrap().rating,
Some(rating)
);
assert_eq!(std::fs::read_to_string(untouched).unwrap(), *expected);
}
assert_eq!(s.xmp.naming, naming);
assert_eq!(s.xmp.auto_write, auto);
drop(s);
std::fs::remove_dir_all(dir).unwrap();
check_companion_scenario(upper_stem, upper_full, naming, auto);
}
}
}
}
}
fn check_companion_scenario(upper_stem: bool, upper_full: bool, naming: SidecarNaming, auto: bool) {
let dir = crate::tests_xmp::temp_dir(&format!("r3-companions-{upper_stem}-{upper_full}-{naming:?}-{auto}"));
let a = dir.join("a.jpg");
let b = dir.join("Trip.jpg");
std::fs::write(&a, b"physical fixture; Session rename does not decode").unwrap();
let path = |photo: &Path, style, upper| {
let path = sidecar_path(&photo.to_string_lossy(), style);
if upper { path.with_extension("XMP") } else { path }
};
let stem_a = path(&a, SidecarNaming::Stem, upper_stem);
let full_a = path(&a, SidecarNaming::Full, upper_full);
let stem_b = path(&b, SidecarNaming::Stem, upper_stem);
let full_b = path(&b, SidecarNaming::Full, upper_full);
let stem_bytes = packet("stem-keep");
let full_bytes = packet("full-keep");
std::fs::write(&stem_a, &stem_bytes).unwrap();
std::fs::write(&full_a, &full_bytes).unwrap();
let mut s = session(&a);
s.xmp.naming = naming;
s.xmp.auto_write = auto;
let fs = Faults(vec![full_a.clone(), b.clone()]);
let plans = s.plan_rename_with(&fs, &[PhotoId(1)], "Trip", 1);
assert!(s.execute_fn("photo.rename", |s| Ok(json!({"renamed": s.apply_rename_with(&fs, &plans)?}))).is_err());
assert_eq!(s.catalog.photo(PhotoId(1)).unwrap().source, Source::File { path: b.to_string_lossy().into_owned() });
let pending = queues(&s);
assert_eq!(pending.len(), 1);
assert_eq!(std::fs::read_to_string(&stem_b).unwrap(), stem_bytes);
// Missing packet and an unrelated packet at either case spelling remain protected.
let held = dir.join("held.xmp");
std::fs::rename(&full_a, &held).unwrap();
s.execute("photo.setMeta", &json!({"title": "Fresh", "addKeywords": ["café"]})).unwrap();
assert_eq!(save(&mut s)["failed"].as_array().unwrap().len(), 1);
assert_eq!(queues(&s), pending);
assert_eq!(std::fs::read_to_string(&stem_b).unwrap(), stem_bytes);
std::fs::rename(&held, &full_a).unwrap();
for collision in [full_b.clone(), full_b.with_extension(if upper_full { "xmp" } else { "XMP" })] {
std::fs::write(&collision, packet("unrelated-packet")).unwrap();
let occupied = std::fs::read(&collision).unwrap();
assert_eq!(save(&mut s)["failed"].as_array().unwrap().len(), 1);
assert_eq!(queues(&s), pending);
assert_eq!(std::fs::read(&collision).unwrap(), occupied);
assert_eq!(std::fs::read_to_string(&full_a).unwrap(), full_bytes);
assert_eq!(std::fs::read_to_string(&stem_b).unwrap(), stem_bytes);
std::fs::remove_file(collision).unwrap();
}
// AutoWrite is the first successful repair when enabled; explicit save otherwise.
s.execute("photo.rate", &json!({"rating": 4})).unwrap();
if !auto {
assert_eq!(save(&mut s)["failed"].as_array().unwrap().len(), 0);
}
assert!(queues(&s).is_empty());
assert!(!full_a.exists());
let (written, untouched, expected) =
if naming == SidecarNaming::Stem { (&stem_b, &full_b, &full_bytes) } else { (&full_b, &stem_b, &stem_bytes) };
assert_fresh(written);
assert_eq!(std::fs::read_to_string(untouched).unwrap(), *expected);
assert_eq!(parse_sidecar(&std::fs::read_to_string(written).unwrap(), crate::crs::Target::Rendered).unwrap().rating, Some(4));
for (command, rating) in [("edit.undo", 0), ("edit.redo", 4)] {
s.execute(command, &json!({})).unwrap();
if !auto {
save(&mut s);
}
assert_fresh(written);
assert_eq!(parse_sidecar(&std::fs::read_to_string(written).unwrap(), crate::crs::Target::Rendered).unwrap().rating, Some(rating));
assert_eq!(std::fs::read_to_string(untouched).unwrap(), *expected);
}
assert_eq!(s.xmp.naming, naming);
assert_eq!(s.xmp.auto_write, auto);
drop(s);
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn rollback_repair_and_shared_copy_keep_known_companions() {
for rollback in [false, true] {
crates/engine/src/sidecar/tests_repair_provenance.rs
+59 -53
View File
@@ -348,64 +348,12 @@ fn detached_history_entry_is_checked_before_unrelated_global_retirement() {
// A private-method contract, not a UI reproduction: application reachability of two
// simultaneous repair owners is unproven. Native move failures and follow_stuck seed
// both records; the second owner's normal apply_rename preflight is deliberately bypassed.
struct NativePartial {
id: PhotoId,
original: PathBuf,
moved: PathBuf,
stem: PathBuf,
full: PathBuf,
companion: PathBuf,
target: PathBuf,
}
let root = crate::tests_xmp::temp_dir("detached-history-preflight-contract");
let image = lightcraft_raster::Rgba8::from_fn(4, 3, |_, _| [40, 80, 120, 255]);
let png = lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(&image), &Default::default()).unwrap();
let mut s = Session::new();
s.xmp.naming = SidecarNaming::Stem;
let mut native = Vec::new();
for (i, name) in ["first", "second"].into_iter().enumerate() {
let dir = root.join(name);
std::fs::create_dir_all(&dir).unwrap();
let original = dir.join("a.png");
let moved = dir.join("Trip.png");
std::fs::write(&original, &png).unwrap();
let stem = path(&original, SidecarNaming::Stem, false);
let full = path(&original, SidecarNaming::Full, true);
for (p, marker) in [(&stem, format!("{name}-stem")), (&full, format!("{name}-full"))] {
std::fs::write(p, packet(&marker)).unwrap();
assert_eq!(parse_sidecar(&std::fs::read_to_string(p).unwrap(), crate::crs::Target::Rendered).unwrap().rating, Some(5));
}
let id = PhotoId(i as u64 + 1);
let photo = Photo::new(id, Source::File { path: original.to_string_lossy().into_owned() }, "a.png", "PNG", 4, 3, "");
s.catalog.apply(Op::AddPhoto { photo: Box::new(photo) }).unwrap();
let fs = Faults(vec![full.clone(), moved.clone()]);
if i == 0 {
let plans = s.plan_rename_with(&fs, &[id], "Trip", 1);
assert!(s.apply_rename_with(&fs, &plans).is_err());
} else {
// Register the genuine second partial move without draining the first repair.
let moves = vec![(original.to_string_lossy().into_owned(), moved.to_string_lossy().into_owned())];
let err = crate::rename::move_all(&fs, &moves).unwrap_err();
assert_eq!(err.stuck, moves);
assert_eq!(err.pending_sidecars.len(), 1);
let op = Op::SetFile { id, source: Source::File { path: moved.to_string_lossy().into_owned() }, file_name: "Trip.png".into() };
s.follow_stuck(&op, err, true);
}
assert_eq!(s.undo.len(), i + 1);
assert_eq!(queued(&s).len(), i + 1);
assert_eq!(s.catalog.photo(id).unwrap().source, Source::File { path: moved.to_string_lossy().into_owned() });
let decoded = lightcraft_codecs::decode(&std::fs::read(&moved).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), (4, 3));
native.push(NativePartial {
id,
original,
companion: path(&moved, SidecarNaming::Stem, false),
target: path(&moved, SidecarNaming::Full, true),
moved,
stem,
full,
});
}
let native = create_native_partials(&root, &png, &mut s);
let first = &native[0];
let second = &native[1];
let first_stem = std::fs::read(&first.companion).unwrap();
@@ -463,3 +411,61 @@ fn detached_history_entry_is_checked_before_unrelated_global_retirement() {
drop(s);
std::fs::remove_dir_all(root).unwrap();
}
struct NativePartial {
id: PhotoId,
original: PathBuf,
moved: PathBuf,
stem: PathBuf,
full: PathBuf,
companion: PathBuf,
target: PathBuf,
}
fn create_native_partials(root: &Path, png: &[u8], s: &mut Session) -> Vec<NativePartial> {
let mut native = Vec::new();
for (i, name) in ["first", "second"].into_iter().enumerate() {
let dir = root.join(name);
std::fs::create_dir_all(&dir).unwrap();
let original = dir.join("a.png");
let moved = dir.join("Trip.png");
std::fs::write(&original, png).unwrap();
let stem = path(&original, SidecarNaming::Stem, false);
let full = path(&original, SidecarNaming::Full, true);
for (p, marker) in [(&stem, format!("{name}-stem")), (&full, format!("{name}-full"))] {
std::fs::write(p, packet(&marker)).unwrap();
assert_eq!(parse_sidecar(&std::fs::read_to_string(p).unwrap(), crate::crs::Target::Rendered).unwrap().rating, Some(5));
}
let id = PhotoId(i as u64 + 1);
let photo = Photo::new(id, Source::File { path: original.to_string_lossy().into_owned() }, "a.png", "PNG", 4, 3, "");
s.catalog.apply(Op::AddPhoto { photo: Box::new(photo) }).unwrap();
let fs = Faults(vec![full.clone(), moved.clone()]);
if i == 0 {
let plans = s.plan_rename_with(&fs, &[id], "Trip", 1);
assert!(s.apply_rename_with(&fs, &plans).is_err());
} else {
// Register the genuine second partial move without draining the first repair.
let moves = vec![(original.to_string_lossy().into_owned(), moved.to_string_lossy().into_owned())];
let err = crate::rename::move_all(&fs, &moves).unwrap_err();
assert_eq!(err.stuck, moves);
assert_eq!(err.pending_sidecars.len(), 1);
let op = Op::SetFile { id, source: Source::File { path: moved.to_string_lossy().into_owned() }, file_name: "Trip.png".into() };
s.follow_stuck(&op, err, true);
}
assert_eq!(s.undo.len(), i + 1);
assert_eq!(queued(s).len(), i + 1);
assert_eq!(s.catalog.photo(id).unwrap().source, Source::File { path: moved.to_string_lossy().into_owned() });
let decoded = lightcraft_codecs::decode(&std::fs::read(&moved).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), (4, 3));
native.push(NativePartial {
id,
original,
companion: path(&moved, SidecarNaming::Stem, false),
target: path(&moved, SidecarNaming::Full, true),
moved,
stem,
full,
});
}
native
}
crates/engine/src/sidecar/tests_repair_saves.rs
+110 -106
View File
@@ -58,118 +58,122 @@ fn autowrite_repairs_partial_rename_before_merging_foreign_xmp_across_library_sw
for save_naming in [SidecarNaming::Stem, SidecarNaming::Full] {
for upper in [false, true] {
for switch in [false, true] {
let root = crate::tests_xmp::temp_dir(&format!("repair-save-{naming:?}-{save_naming:?}-{upper}-{switch}"));
let a = root.join("a.png");
let b = root.join("Trip.png");
png(&a);
let mut old = sidecar_path(&a.to_string_lossy(), naming);
if upper {
old = old.with_extension("XMP");
}
std::fs::write(&old, FOREIGN).unwrap();
let mut s = Session::new().with_fs();
s.open_library(root.join("library-a"), false).unwrap();
s.execute("library.import", &json!({"paths": [a]})).unwrap();
let mut id = s.catalog.photos().next().unwrap().id;
s.selection = crate::Selection::single(id);
s.undo.clear();
let fs = PartialRename { sidecar: old.clone(), photo: b.clone() };
let plans = s.plan_rename_with(&fs, &[id], "Trip", 1);
assert!(s.execute_fn("photo.rename", |s| Ok(json!({"renamed": s.apply_rename_with(&fs, &plans)?}))).is_err());
assert_eq!(queued(&s).len(), 1);
assert_eq!(s.catalog.photo(id).unwrap().source, Source::File { path: b.to_string_lossy().into_owned() });
let pending = queued(&s);
let held = root.join("temporarily-unavailable.xmp");
std::fs::rename(&old, &held).unwrap();
if switch {
s.close_library().unwrap();
s.open_library(root.join("library-b"), false).unwrap();
assert_eq!(s.pending_sidecar_repairs, pending);
s.execute("library.import", &json!({"paths": [b]})).unwrap();
id = s.catalog.photos().next().unwrap().id;
s.selection = crate::Selection::single(id);
}
s.xmp.naming = save_naming;
s.xmp.auto_write = true;
s.execute("photo.rate", &json!({"rating": 4})).unwrap();
let preferred = sidecar_path(&b.to_string_lossy(), save_naming);
assert!(!preferred.exists() && !preferred.with_extension("XMP").exists());
assert!(!pending[0].to.exists());
assert_eq!(queued(&s), pending, "unavailable repair stays queued after AutoWrite");
let saved = s.execute("photo.saveMetadataToFile", &json!({})).unwrap();
assert!(saved["written"].as_array().unwrap().is_empty());
assert!(saved["failed"][0]["error"].as_str().unwrap().contains("unfinished XMP repair"));
assert_eq!(std::fs::read(&held).unwrap(), FOREIGN);
assert_eq!(queued(&s), pending);
// A missing sidecar for another photo does not prevent this healthy save.
let healthy = root.join("healthy.png");
png(&healthy);
let healthy_id = PhotoId(999);
s.commit(
"Healthy Photo",
Op::AddPhoto {
photo: Box::new(Photo::new(
healthy_id,
Source::File { path: healthy.to_string_lossy().into_owned() },
"healthy.png",
"PNG",
4,
3,
"",
)),
},
)
.unwrap();
s.execute("photo.setMeta", &json!({"ids": [healthy_id.0], "title": "Healthy save"})).unwrap();
let packet = std::fs::read_to_string(sidecar_path(&healthy.to_string_lossy(), save_naming)).unwrap();
assert_eq!(parse_sidecar(&packet, crate::crs::Target::Rendered).unwrap().title.as_deref(), Some("Healthy save"));
assert_eq!(queued(&s), pending);
// An existing destination is never overwritten and the original packet remains.
std::fs::rename(&held, &old).unwrap();
let destination = pending[0].to.clone();
let collision = String::from_utf8(FOREIGN.to_vec()).unwrap().replace("Rating=\"2\"", "Rating=\"5\"").into_bytes();
let collision_path = if upper { preferred.clone() } else { destination.clone() };
std::fs::write(&collision_path, &collision).unwrap();
s.execute("photo.setMeta", &json!({"ids": [id.0], "title": "Fresh title", "addKeywords": ["café"]})).unwrap();
assert_eq!(std::fs::read(&old).unwrap(), FOREIGN);
assert_eq!(std::fs::read(&collision_path).unwrap(), collision);
assert_eq!(queued(&s), pending);
s.selection = crate::Selection::single(id);
assert_eq!(s.execute("photo.saveMetadataToFile", &json!({})).unwrap()["failed"].as_array().unwrap().len(), 1);
assert_eq!(std::fs::read(&collision_path).unwrap(), collision);
std::fs::remove_file(&collision_path).unwrap(); // remove only this test's collision
// The next successful AutoWrite runs the real repair and production XMP merge.
s.execute("photo.setMeta", &json!({"title": "Final title"})).unwrap();
assert!(queued(&s).is_empty());
assert!(!old.exists());
let packet = std::fs::read_to_string(&destination).unwrap();
assert!(packet.contains("fixture:Marker=\"keep-me\""));
let parsed = parse_sidecar(&packet, crate::crs::Target::Rendered).unwrap();
assert_eq!(parsed.rating, Some(4));
assert_eq!(parsed.title.as_deref(), Some("Final title"));
assert!(parsed.keywords.as_ref().unwrap().contains(&"café".to_string()));
if upper {
// by listing: on a case-insensitive volume `preferred` answers for the upper-case packet
let listed = std::fs::read_dir(&root).unwrap().any(|e| Some(e.unwrap().file_name().as_os_str()) == preferred.file_name());
assert!(!listed, "merge uses the repaired uppercase packet");
}
for (step, title) in [("edit.undo", "Fresh title"), ("edit.redo", "Final title")] {
s.execute(step, &json!({})).unwrap();
let packet = std::fs::read_to_string(&destination).unwrap();
assert!(packet.contains("fixture:Marker=\"keep-me\""));
assert_eq!(parse_sidecar(&packet, crate::crs::Target::Rendered).unwrap().title.as_deref(), Some(title));
}
drop(s);
std::fs::remove_dir_all(root).unwrap();
check_autowrite_repair_scenario(naming, save_naming, upper, switch);
}
}
}
}
}
fn check_autowrite_repair_scenario(naming: SidecarNaming, save_naming: SidecarNaming, upper: bool, switch: bool) {
let root = crate::tests_xmp::temp_dir(&format!("repair-save-{naming:?}-{save_naming:?}-{upper}-{switch}"));
let a = root.join("a.png");
let b = root.join("Trip.png");
png(&a);
let mut old = sidecar_path(&a.to_string_lossy(), naming);
if upper {
old = old.with_extension("XMP");
}
std::fs::write(&old, FOREIGN).unwrap();
let mut s = Session::new().with_fs();
s.open_library(root.join("library-a"), false).unwrap();
s.execute("library.import", &json!({"paths": [a]})).unwrap();
let mut id = s.catalog.photos().next().unwrap().id;
s.selection = crate::Selection::single(id);
s.undo.clear();
let fs = PartialRename { sidecar: old.clone(), photo: b.clone() };
let plans = s.plan_rename_with(&fs, &[id], "Trip", 1);
assert!(s.execute_fn("photo.rename", |s| Ok(json!({"renamed": s.apply_rename_with(&fs, &plans)?}))).is_err());
assert_eq!(queued(&s).len(), 1);
assert_eq!(s.catalog.photo(id).unwrap().source, Source::File { path: b.to_string_lossy().into_owned() });
let pending = queued(&s);
let held = root.join("temporarily-unavailable.xmp");
std::fs::rename(&old, &held).unwrap();
if switch {
s.close_library().unwrap();
s.open_library(root.join("library-b"), false).unwrap();
assert_eq!(s.pending_sidecar_repairs, pending);
s.execute("library.import", &json!({"paths": [b]})).unwrap();
id = s.catalog.photos().next().unwrap().id;
s.selection = crate::Selection::single(id);
}
s.xmp.naming = save_naming;
s.xmp.auto_write = true;
s.execute("photo.rate", &json!({"rating": 4})).unwrap();
let preferred = sidecar_path(&b.to_string_lossy(), save_naming);
assert!(!preferred.exists() && !preferred.with_extension("XMP").exists());
assert!(!pending[0].to.exists());
assert_eq!(queued(&s), pending, "unavailable repair stays queued after AutoWrite");
let saved = s.execute("photo.saveMetadataToFile", &json!({})).unwrap();
assert!(saved["written"].as_array().unwrap().is_empty());
assert!(saved["failed"][0]["error"].as_str().unwrap().contains("unfinished XMP repair"));
assert_eq!(std::fs::read(&held).unwrap(), FOREIGN);
assert_eq!(queued(&s), pending);
// A missing sidecar for another photo does not prevent this healthy save.
let healthy = root.join("healthy.png");
png(&healthy);
let healthy_id = PhotoId(999);
s.commit(
"Healthy Photo",
Op::AddPhoto {
photo: Box::new(Photo::new(healthy_id, Source::File { path: healthy.to_string_lossy().into_owned() }, "healthy.png", "PNG", 4, 3, "")),
},
)
.unwrap();
s.execute("photo.setMeta", &json!({"ids": [healthy_id.0], "title": "Healthy save"})).unwrap();
let packet = std::fs::read_to_string(sidecar_path(&healthy.to_string_lossy(), save_naming)).unwrap();
assert_eq!(parse_sidecar(&packet, crate::crs::Target::Rendered).unwrap().title.as_deref(), Some("Healthy save"));
assert_eq!(queued(&s), pending);
// An existing destination is never overwritten and the original packet remains.
std::fs::rename(&held, &old).unwrap();
let destination = pending[0].to.clone();
let collision = String::from_utf8(FOREIGN.to_vec()).unwrap().replace("Rating=\"2\"", "Rating=\"5\"").into_bytes();
let collision_path = if upper { preferred.clone() } else { destination.clone() };
std::fs::write(&collision_path, &collision).unwrap();
s.execute("photo.setMeta", &json!({"ids": [id.0], "title": "Fresh title", "addKeywords": ["café"]})).unwrap();
assert_eq!(std::fs::read(&old).unwrap(), FOREIGN);
assert_eq!(std::fs::read(&collision_path).unwrap(), collision);
assert_eq!(queued(&s), pending);
s.selection = crate::Selection::single(id);
assert_eq!(s.execute("photo.saveMetadataToFile", &json!({})).unwrap()["failed"].as_array().unwrap().len(), 1);
assert_eq!(std::fs::read(&collision_path).unwrap(), collision);
std::fs::remove_file(&collision_path).unwrap(); // remove only this test's collision
// The next successful AutoWrite runs the real repair and production XMP merge.
s.execute("photo.setMeta", &json!({"title": "Final title"})).unwrap();
assert!(queued(&s).is_empty());
assert!(!old.exists());
let packet = std::fs::read_to_string(&destination).unwrap();
assert!(packet.contains("fixture:Marker=\"keep-me\""));
let parsed = parse_sidecar(&packet, crate::crs::Target::Rendered).unwrap();
assert_eq!(parsed.rating, Some(4));
assert_eq!(parsed.title.as_deref(), Some("Final title"));
assert!(parsed.keywords.as_ref().unwrap().contains(&"café".to_string()));
assert_repaired_case_spelling(&root, &preferred, upper);
assert_undo_redo_preserve_foreign_packet(&mut s, &destination);
drop(s);
std::fs::remove_dir_all(root).unwrap();
}
fn assert_repaired_case_spelling(root: &Path, preferred: &Path, upper: bool) {
if !upper {
return;
}
// By listing, a case-insensitive volume answers for the uppercase packet.
let listed = std::fs::read_dir(root).unwrap().any(|e| Some(e.unwrap().file_name().as_os_str()) == preferred.file_name());
assert!(!listed, "merge uses the repaired uppercase packet");
}
fn assert_undo_redo_preserve_foreign_packet(s: &mut Session, destination: &Path) {
for (step, title) in [("edit.undo", "Fresh title"), ("edit.redo", "Final title")] {
s.execute(step, &json!({})).unwrap();
let packet = std::fs::read_to_string(destination).unwrap();
assert!(packet.contains("fixture:Marker=\"keep-me\""));
assert_eq!(parse_sidecar(&packet, crate::crs::Target::Rendered).unwrap().title.as_deref(), Some(title));
}
}
#[test]
fn unowned_shared_stem_repair_does_not_block_healthy_jpeg_full_sidecar_save() {
let root = crate::tests_xmp::temp_dir("repair-save-shared-stem");
crates/engine/src/tests/import.rs
+116 -82
View File
@@ -1,7 +1,11 @@
//! Import: recursive folders, duplicate detection by content, add vs copy into the library.
use std::path::Path;
use std::{
path::{Path, PathBuf},
sync::atomic::AtomicUsize,
};
use lightcraft_catalog::PhotoId;
use serde_json::{Value, json};
use crate::Session;
@@ -1711,109 +1715,139 @@ fn smart_previews_from_an_older_look_fit_are_refreshed() {
}
fn smart_look_refresh_for(format: &str) {
use std::sync::atomic::{AtomicUsize, Ordering};
let src = temp_dir(&format!("smartlook-src-{format}"));
let lib = temp_dir(&format!("smartlook-lib-{format}"));
write_png(&src.join("a.png"), 3);
write_png(&src.join("b.png"), 9);
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
s.execute("library.import", &json!({"paths": [src.join("a.png").to_string_lossy(), src.join("b.png").to_string_lossy()]})).unwrap();
let find = |s: &Session, name: &str| s.catalog.photos().find(|p| p.file_name == name).unwrap().id;
let (id, png) = (find(&s, "a.png"), find(&s, "b.png"));
// `a` stands for a raw that takes the per-file camera look (the test loader decodes its pixels)
let mut raw = (**s.catalog.photo(id).unwrap()).clone();
raw.format = format.into();
raw.kind = lightcraft_catalog::MediaKind::Raw;
s.catalog.apply(lightcraft_catalog::Op::RemovePhoto { id }).unwrap();
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(raw) }).unwrap();
assert_eq!(s.catalog.photo(id).unwrap().relative_wb(), format == "ARW");
assert!(!s.catalog.photo(png).unwrap().relative_wb());
s.execute("library.select", &json!({"ids": [id.0, png.0], "active": id.0})).unwrap();
let dir = s.media.smart_dir.clone().unwrap();
let file = crate::smart::sharded_file_path(&dir, s.catalog.photo(id).unwrap());
let png_file = crate::smart::sharded_file_path(&dir, s.catalog.photo(png).unwrap());
let (old, new) = (test_tone(1.5), test_tone(3.5));
let calls = std::sync::Arc::new(AtomicUsize::new(0));
let plain = s.media.file_loader.clone().unwrap();
s.media.file_loader = Some(look_loader(plain.clone(), old, calls.clone()));
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
let mut case = SmartLookCase::new(format);
assert_current_smart_look_is_not_rebuilt(&mut case);
let (old_bytes, png_bytes) = keep_offline_previews(&mut case);
refresh_online_previews(&mut case, &old_bytes, &png_bytes);
let _ = std::fs::remove_dir_all(&case.src);
let _ = std::fs::remove_dir_all(&case.lib);
}
struct SmartLookCase {
src: PathBuf,
lib: PathBuf,
session: Session,
id: PhotoId,
dir: PathBuf,
file: PathBuf,
png_file: PathBuf,
old: lightcraft_pipeline::tone::CameraTone,
new: lightcraft_pipeline::tone::CameraTone,
calls: std::sync::Arc<std::sync::atomic::AtomicUsize>,
plain_loader: crate::media::FileLoader,
}
impl SmartLookCase {
fn new(format: &str) -> Self {
let src = temp_dir(&format!("smartlook-src-{format}"));
let lib = temp_dir(&format!("smartlook-lib-{format}"));
write_png(&src.join("a.png"), 3);
write_png(&src.join("b.png"), 9);
let mut session = Session::new().with_fs();
session.open_library(&lib, false).unwrap();
session.execute("library.import", &json!({"paths": [src.join("a.png").to_string_lossy(), src.join("b.png").to_string_lossy()]})).unwrap();
let find = |s: &Session, name: &str| s.catalog.photos().find(|p| p.file_name == name).unwrap().id;
let (id, png_id) = (find(&session, "a.png"), find(&session, "b.png"));
// `a` stands for a raw that takes the per-file camera look (the test loader decodes its pixels)
let mut raw = (**session.catalog.photo(id).unwrap()).clone();
raw.format = format.into();
raw.kind = lightcraft_catalog::MediaKind::Raw;
session.catalog.apply(lightcraft_catalog::Op::RemovePhoto { id }).unwrap();
session.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(raw) }).unwrap();
assert_eq!(session.catalog.photo(id).unwrap().relative_wb(), format == "ARW");
assert!(!session.catalog.photo(png_id).unwrap().relative_wb());
session.execute("library.select", &json!({"ids": [id.0, png_id.0], "active": id.0})).unwrap();
let dir = session.media.smart_dir.clone().unwrap();
let file = crate::smart::sharded_file_path(&dir, session.catalog.photo(id).unwrap());
let png_file = crate::smart::sharded_file_path(&dir, session.catalog.photo(png_id).unwrap());
let (old, new) = (test_tone(1.5), test_tone(3.5));
let calls = std::sync::Arc::new(AtomicUsize::new(0));
let plain_loader = session.media.file_loader.clone().unwrap();
session.media.file_loader = Some(look_loader(plain_loader.clone(), old, calls.clone()));
Self { src, lib, session, id, dir, file, png_file, old, new, calls, plain_loader }
}
}
fn assert_current_smart_look_is_not_rebuilt(case: &mut SmartLookCase) {
let r = case.session.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["refreshed"].as_u64()), (Some(2), Some(0)), "{r}");
assert_eq!(smart_header(&file)["look_version"], json!(crate::camera_preview::LOOK_VERSION), "the writer stamps the fit version");
assert!(!crate::smart::is_stale(&file));
assert_eq!(smart_header(&case.file)["look_version"], json!(crate::camera_preview::LOOK_VERSION), "the writer stamps the fit version");
assert!(!crate::smart::is_stale(&case.file));
// current header: kept as it is, the original is not even read
let before = std::fs::read(&file).unwrap();
calls.store(0, Ordering::SeqCst);
s.media.file_loader = Some(look_loader(plain.clone(), new, calls.clone()));
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
let before = std::fs::read(&case.file).unwrap();
case.calls.store(0, std::sync::atomic::Ordering::SeqCst);
case.session.media.file_loader = Some(look_loader(case.plain_loader.clone(), case.new, case.calls.clone()));
let r = case.session.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["refreshed"].as_u64()), (Some(2), Some(0)), "{r}");
assert!(std::fs::read(&file).unwrap() == before, "a current proxy is not rebuilt");
assert_eq!(calls.load(Ordering::SeqCst), 0);
assert_eq!(std::fs::read(&case.file).unwrap(), before, "a current proxy is not rebuilt");
assert_eq!(case.calls.load(std::sync::atomic::Ordering::SeqCst), 0);
}
fn keep_offline_previews(case: &mut SmartLookCase) -> (Vec<u8>, Vec<u8>) {
// old headers (no field), originals offline: kept unchanged, and still marked stale
for f in [&file, &png_file] {
for f in [&case.file, &case.png_file] {
edit_smart_header(f, |h| {
h.as_object_mut().unwrap().remove("look_version");
});
}
assert!(crate::smart::is_stale(&file), "no field = version 0");
let old_bytes = std::fs::read(&file).unwrap();
let png_bytes = std::fs::read(&png_file).unwrap();
std::fs::rename(&src, src.with_extension("offline")).unwrap();
s.media.forget(id);
s.media.forget(png);
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert!(crate::smart::is_stale(&case.file), "no field = version 0");
let old_bytes = std::fs::read(&case.file).unwrap();
let png_bytes = std::fs::read(&case.png_file).unwrap();
std::fs::rename(&case.src, case.src.with_extension("offline")).unwrap();
case.session.media.forget(case.id);
let r = case.session.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["refreshed"].as_u64(), r["staleKept"].as_u64()), (Some(2), Some(0), Some(1)), "{r}");
assert!(r["failed"].as_array().unwrap().is_empty(), "{r}");
assert!(std::fs::read(&file).unwrap() == old_bytes, "offline: the stored preview and curve are untouched");
assert!(crate::smart::is_stale(&file), "and it stays marked for the next time");
assert_eq!(crate::smart::decode(&old_bytes).unwrap().1, Some(old), "the old look is still there");
assert_eq!(std::fs::read(&case.file).unwrap(), old_bytes, "offline: the stored preview and curve are untouched");
assert!(crate::smart::is_stale(&case.file), "and it stays marked for the next time");
assert_eq!(crate::smart::decode(&old_bytes).unwrap().1, Some(case.old), "the old look is still there");
// ... and the opening refresh keeps it too, then marks the folder checked
assert!(!crate::smart::look_checked(&dir));
crate::cmd::previews::refresh_stale_smart_previews(&mut s).unwrap().join().unwrap();
assert!(std::fs::read(&file).unwrap() == old_bytes);
assert!(crate::smart::look_checked(&dir));
assert!(!crate::smart::look_checked(&case.dir));
crate::cmd::previews::refresh_stale_smart_previews(&mut case.session).unwrap().join().unwrap();
assert_eq!(std::fs::read(&case.file).unwrap(), old_bytes);
assert!(crate::smart::look_checked(&case.dir));
(old_bytes, png_bytes)
}
fn refresh_online_previews(case: &mut SmartLookCase, old_bytes: &[u8], png_bytes: &[u8]) {
// the original comes back: the scan has run at this version, so the next opening leaves it;
// Build Smart Previews rebuilds it with the current look
std::fs::rename(src.with_extension("offline"), &src).unwrap();
calls.store(0, Ordering::SeqCst);
assert!(crate::cmd::previews::refresh_stale_smart_previews(&mut s).is_none(), "checked once per version");
assert!(std::fs::read(&file).unwrap() == old_bytes);
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["refreshed"].as_u64(), calls.load(Ordering::SeqCst)), (Some(1), 1), "{r}");
assert_eq!(smart_header(&file)["look_version"], json!(crate::camera_preview::LOOK_VERSION));
assert_eq!(crate::smart::decode(&std::fs::read(&file).unwrap()).unwrap().1, Some(new), "rebuilt with the current look");
assert!(!crate::smart::is_stale(&file));
assert!(std::fs::read(&png_file).unwrap() == png_bytes, "a PNG's proxy is never rebuilt for the look");
std::fs::rename(case.src.with_extension("offline"), &case.src).unwrap();
case.calls.store(0, std::sync::atomic::Ordering::SeqCst);
assert!(crate::cmd::previews::refresh_stale_smart_previews(&mut case.session).is_none(), "checked once per version");
assert_eq!(std::fs::read(&case.file).unwrap(), old_bytes);
let r = case.session.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["refreshed"].as_u64(), case.calls.load(std::sync::atomic::Ordering::SeqCst)), (Some(1), 1), "{r}");
assert_eq!(smart_header(&case.file)["look_version"], json!(crate::camera_preview::LOOK_VERSION));
assert_eq!(crate::smart::decode(&std::fs::read(&case.file).unwrap()).unwrap().1, Some(case.new), "rebuilt with the current look");
assert!(!crate::smart::is_stale(&case.file));
assert_eq!(std::fs::read(&case.png_file).unwrap(), png_bytes, "a PNG's proxy is never rebuilt for the look");
// a folder checked at an older version (or not at all) is scanned once at the next opening
edit_smart_header(&file, |h| h["look_version"] = json!(0));
std::fs::write(dir.join("look-version"), b"0\n").unwrap();
assert!(!crate::smart::look_checked(&dir));
s.media.forget(id);
calls.store(0, Ordering::SeqCst);
crate::cmd::previews::refresh_stale_smart_previews(&mut s).unwrap().join().unwrap();
assert_eq!(calls.load(Ordering::SeqCst), 1, "only the raw's original is read");
assert!(!crate::smart::is_stale(&file));
assert!(std::fs::read(&png_file).unwrap() == png_bytes);
assert!(crate::smart::look_checked(&dir));
edit_smart_header(&case.file, |h| h["look_version"] = json!(0));
std::fs::write(case.dir.join("look-version"), b"0\n").unwrap();
assert!(!crate::smart::look_checked(&case.dir));
case.session.media.forget(case.id);
case.calls.store(0, std::sync::atomic::Ordering::SeqCst);
crate::cmd::previews::refresh_stale_smart_previews(&mut case.session).unwrap().join().unwrap();
assert_eq!(case.calls.load(std::sync::atomic::Ordering::SeqCst), 1, "only the raw's original is read");
assert!(!crate::smart::is_stale(&case.file));
assert_eq!(std::fs::read(&case.png_file).unwrap(), png_bytes);
assert!(crate::smart::look_checked(&case.dir));
// once only
let rebuilt = std::fs::read(&file).unwrap();
assert!(crate::cmd::previews::refresh_stale_smart_previews(&mut s).is_none());
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["refreshed"].as_u64(), calls.load(Ordering::SeqCst)), (Some(0), 1), "{r}");
assert!(std::fs::read(&file).unwrap() == rebuilt);
let rebuilt = std::fs::read(&case.file).unwrap();
assert!(crate::cmd::previews::refresh_stale_smart_previews(&mut case.session).is_none());
let r = case.session.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["refreshed"].as_u64(), case.calls.load(std::sync::atomic::Ordering::SeqCst)), (Some(0), 1), "{r}");
assert_eq!(std::fs::read(&case.file).unwrap(), rebuilt);
// Build Smart Previews refreshes a stale one the same way
edit_smart_header(&file, |h| h["look_version"] = json!(0));
s.media.file_loader = Some(plain);
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
edit_smart_header(&case.file, |h| h["look_version"] = json!(0));
case.session.media.file_loader = Some(case.plain_loader.clone());
let r = case.session.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!(r["refreshed"], 1, "{r}");
assert_eq!(crate::smart::decode(&std::fs::read(&file).unwrap()).unwrap().1, None, "this fit gives no curve: none is kept");
let _ = std::fs::remove_dir_all(&src);
let _ = std::fs::remove_dir_all(&lib);
assert_eq!(crate::smart::decode(&std::fs::read(&case.file).unwrap()).unwrap().1, None, "this fit gives no curve: none is kept");
}
/// Issue #1534 (review): a proxy's encoding version is migrated independently of the per-file
crates/engine/src/walk.rs
+60 -3
View File
@@ -43,8 +43,37 @@ pub struct Walked {
pub entries: usize,
}
fn hidden(p: &Path) -> bool {
p.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.'))
/// Whether the entry has a dot-prefixed name or a Windows or macOS hidden attribute.
pub fn is_hidden_entry(entry: &std::fs::DirEntry) -> bool {
hidden_with_metadata(&entry.path(), || entry.metadata())
}
/// Whether the path has a dot-prefixed name or a Windows or macOS hidden attribute.
pub fn is_hidden_path(path: &Path) -> bool {
hidden_with_metadata(path, || std::fs::symlink_metadata(path))
}
fn hidden_with_metadata(path: &Path, metadata: impl FnOnce() -> std::io::Result<std::fs::Metadata>) -> bool {
if path.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
return true;
}
#[cfg(windows)]
{
use std::os::windows::fs::MetadataExt;
const FILE_ATTRIBUTE_HIDDEN: u32 = 0x2;
metadata().is_ok_and(|m| m.file_attributes() & FILE_ATTRIBUTE_HIDDEN != 0)
}
#[cfg(target_os = "macos")]
{
use std::os::darwin::fs::MetadataExt;
const UF_HIDDEN: u32 = 0x0000_8000;
metadata().is_ok_and(|m| m.st_flags() & UF_HIDDEN != 0)
}
#[cfg(not(any(windows, target_os = "macos")))]
{
let _ = metadata;
false
}
}
/// What a folder's listing says an entry is.
@@ -120,7 +149,7 @@ fn walk(root: &Path, skip: Option<&Path>, limits: Limits, mut keep: impl FnMut(&
let p = e.path();
// (a hidden entry's kind is only asked for when the listing is wanted; every other
// entry's is asked once)
let is_hidden = hidden(&p);
let is_hidden = is_hidden_entry(&e);
if is_hidden && listed.is_none() {
continue;
}
@@ -178,6 +207,34 @@ mod tests {
d
}
#[test]
fn hidden_names_and_platform_attributes_are_excluded_from_walks() {
let root = tmp("hidden-attributes");
for folder in [".dot", "Hidden"] {
std::fs::create_dir_all(root.join(folder)).unwrap();
std::fs::write(root.join(folder).join("inside.jpg"), b"x").unwrap();
}
for file in ["Shown.jpg", ".dot.jpg", "hidden.jpg"] {
std::fs::write(root.join(file), b"x").unwrap();
}
for name in ["Hidden", "hidden.jpg"] {
#[cfg(windows)]
assert!(std::process::Command::new("attrib").arg("+h").arg(root.join(name)).status().unwrap().success());
#[cfg(target_os = "macos")]
assert!(std::process::Command::new("chflags").arg("hidden").arg(root.join(name)).status().unwrap().success());
assert_eq!(is_hidden_path(&root.join(name)), cfg!(any(windows, target_os = "macos")));
}
let mut found = files_in(&root, None, Limits::default(), |_| true).files;
found.sort();
let mut expected = vec![root.join("Shown.jpg")];
if !cfg!(any(windows, target_os = "macos")) {
expected.extend([root.join("Hidden/inside.jpg"), root.join("hidden.jpg")]);
}
expected.sort();
assert_eq!(found, expected);
std::fs::remove_dir_all(root).unwrap();
}
/// Given a folder with files, a folder inside it, a hidden file and a link, when it is walked
/// with its listings noted, then the walk finds what it finds without them, and each folder's
/// listing holds every entry by name with its kind (hidden ones too, a link as neither file nor
crates/faces/src/align.rs
+19 -8
View File
@@ -19,6 +19,14 @@ pub struct Rgb<'a> {
pub height: usize,
}
/// Face bounds, padding factor, and output dimensions for box-based alignment.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct FaceCrop {
pub bounds: (f32, f32, f32, f32),
pub padding: f32,
pub output_size: (usize, usize),
}
impl Rgb<'_> {
fn valid(&self) -> bool {
self.width > 0 && self.height > 0 && self.width.checked_mul(self.height).and_then(|n| n.checked_mul(3)) == Some(self.data.len())
@@ -145,7 +153,10 @@ pub fn align_to_template(img: &Rgb, landmarks: &[(f32, f32); 5], out_w: usize, o
}
/// A square around a face box (pixels of `img`), `pad` times its larger side, as `out_w` × `out_h`.
pub fn crop_box(img: &Rgb, x0: f32, y0: f32, x1: f32, y1: f32, pad: f32, out_w: usize, out_h: usize) -> Option<Vec<u8>> {
pub fn crop_box(img: &Rgb, crop: FaceCrop) -> Option<Vec<u8>> {
let (x0, y0, x1, y1) = crop.bounds;
let pad = crop.padding;
let (out_w, out_h) = crop.output_size;
if ![x0, y0, x1, y1, pad].iter().all(|v| v.is_finite()) {
return None;
}
@@ -247,16 +258,16 @@ mod tests {
}
}
let img = Rgb { data: &data, width: w, height: h };
let out = crop_box(&img, 90.0, 40.0, 110.0, 60.0, 2.0, 64, 64).unwrap();
let out = crop_box(&img, FaceCrop { bounds: (90.0, 40.0, 110.0, 60.0), padding: 2.0, output_size: (64, 64) }).unwrap();
// the 20 px block fills the middle half of the 64 px output
let at = |x: usize, y: usize| out[(y * 64 + x) * 3 + 1];
assert!(at(32, 32) > 250 && at(20, 20) > 250 && at(44, 44) > 250 && at(5, 5) < 5 && at(60, 32) < 5);
assert!(crop_box(&img, 5.0, 5.0, 5.0, 5.0, 2.0, 64, 64).is_none(), "an empty box");
assert!(crop_box(&img, f32::NAN, 0.0, 10.0, 10.0, 2.0, 64, 64).is_none());
assert!(crop_box(&img, 0.0, 0.0, 10.0, 10.0, 2.0, 0, 64).is_none());
assert!(crop_box(&img, 0.0, 0.0, 10.0, 10.0, 2.0, 100_000, 64).is_none());
assert!(crop_box(&img, FaceCrop { bounds: (5.0, 5.0, 5.0, 5.0), padding: 2.0, output_size: (64, 64) }).is_none(), "an empty box");
assert!(crop_box(&img, FaceCrop { bounds: (f32::NAN, 0.0, 10.0, 10.0), padding: 2.0, output_size: (64, 64) }).is_none());
assert!(crop_box(&img, FaceCrop { bounds: (0.0, 0.0, 10.0, 10.0), padding: 2.0, output_size: (0, 64) }).is_none());
assert!(crop_box(&img, FaceCrop { bounds: (0.0, 0.0, 10.0, 10.0), padding: 2.0, output_size: (100_000, 64) }).is_none());
let bad = Rgb { data: &data[..10], width: w, height: h };
assert!(crop_box(&bad, 0.0, 0.0, 10.0, 10.0, 2.0, 64, 64).is_none(), "wrong data length");
assert!(crop_box(&bad, FaceCrop { bounds: (0.0, 0.0, 10.0, 10.0), padding: 2.0, output_size: (64, 64) }).is_none(), "wrong data length");
}
#[test]
@@ -264,7 +275,7 @@ mod tests {
let data = vec![80u8; 50 * 50 * 3];
let img = Rgb { data: &data, width: 50, height: 50 };
// a box mostly outside the picture is padded by clamping, not an error
assert!(crop_box(&img, -30.0, -30.0, 20.0, 20.0, 1.5, 32, 32).is_some());
assert!(crop_box(&img, FaceCrop { bounds: (-30.0, -30.0, 20.0, 20.0), padding: 1.5, output_size: (32, 32) }).is_some());
let far: [(f32, f32); 5] = [(1e6, 1e6), (1e6 + 5.0, 1e6), (1e6 + 2.0, 1e6 + 3.0), (1e6, 1e6 + 6.0), (1e6 + 5.0, 1e6 + 6.0)];
let _ = align_to_template(&img, &far, 112, 112);
let tiny: [(f32, f32); 5] = [(1.0, 1.0), (1.01, 1.0), (1.0, 1.01), (1.01, 1.01), (1.005, 1.005)];
crates/faces/src/net/kernels.rs
+12 -8
View File
@@ -1,5 +1,5 @@
//! Safe CPU kernels shared by detection and recognition. No inner thread pool.
use super::*;
use super::{ConvJob, Graph, MAX_TENSOR, NetError, Node, Result, Tensor, chw, conv_direct, pads, pair, program, shape, shape_err};
use ndarray::{ArrayView2, ArrayViewMut2, linalg::general_mat_mul};
const BLOCK: usize = 256;
#[derive(Default)]
@@ -30,7 +30,16 @@ fn multiply(w: &[f32], x: &[f32], y: &mut [f32], m: usize, k: usize, n: usize) -
}
/// For equal-width, unit-stride maps a shifted plane is one contiguous copy. Only horizontal
/// border stripes need masking; clipping the flat source range also removes vertical padding.
fn shifted_plane(plane: &[f32], width: usize, ky: usize, kx: usize, pt: usize, pl: usize, start: usize, row: &mut [f32]) -> Result<()> {
#[derive(Clone, Copy)]
struct PlaneShift {
width: usize,
kernel: (usize, usize),
padding: (usize, usize),
start: usize,
}
fn shifted_plane(plane: &[f32], row: &mut [f32], shift: PlaneShift) -> Result<()> {
let PlaneShift { width, kernel: (ky, kx), padding: (pt, pl), start } = shift;
let end = start + row.len();
let delta = (ky as i128 - pt as i128) * width as i128 + kx as i128 - pl as i128;
let lo = (-delta).clamp(start as i128, end as i128);
@@ -145,13 +154,8 @@ pub(super) fn conv(n: &Node, x: &Tensor, g: &Graph, scratch: &mut Scratch) -> Re
if sh == 1 && sw == 1 && ow == w {
shifted_plane(
plane,
w,
ky,
kx,
pt,
pl,
start,
row.get_mut(..len).ok_or_else(|| NetError::Shape("short lower panel".into()))?,
PlaneShift { width: w, kernel: (ky, kx), padding: (pt, pl), start },
)?;
continue;
}
crates/faces/src/net/program.rs
+1 -1
View File
@@ -9,7 +9,7 @@
use std::borrow::Cow;
use super::*;
use super::{ConvJob, Data, HashMap, Net, NetError, Result, kernels, pads, pair, shape, shape_err, unsupported};
/// Value 0 is the network's input.
pub const INPUT: usize = 0;
crates/faces/src/net/shape.rs
+1 -1
View File
@@ -1,5 +1,5 @@
//! Bounded, allocation-free (apart from small shape lists) graph checking.
use super::*;
use super::{Data, Graph, HashMap, MAX_TENSOR, NetError, Node, Result, pads, pair, shape_err, unsupported};
use crate::graph::{Attr, Weight};
pub(super) fn elements(s: &[usize]) -> Result<usize> {
crates/geocode/src/unzip.rs
+18 -1
View File
@@ -61,7 +61,13 @@ pub fn extract(zip: &[u8], name: &str, max: usize) -> Result<Vec<u8>, GeocodeErr
return match method {
0 if data.len() <= max => Ok(data.to_vec()),
0 => Err(GeocodeError::Invalid(format!("{name} is larger than {max} bytes"))),
8 => miniz_oxide::inflate::decompress_to_vec_with_limit(data, max).map_err(|e| bad(&format!("deflate: {e:?}"))),
8 => {
let decoded = miniz_oxide::inflate::decompress_to_vec_with_limit(data, max).map_err(|e| bad(&format!("deflate: {e:?}")))?;
if decoded.len() != usize_ {
return Err(bad("uncompressed size"));
}
Ok(decoded)
}
m => Err(bad(&format!("compression method {m}"))),
};
}
@@ -120,4 +126,15 @@ mod tests {
let _ = extract(&bad, "cities15000.txt", 100);
}
}
#[test]
fn rejects_deflated_entry_with_inconsistent_directory_size() {
let mut z = make("cities15000.txt", b"hello world");
let eocd = z.len() - 22;
let cd = u32::from_le_bytes([z[eocd + 16], z[eocd + 17], z[eocd + 18], z[eocd + 19]]) as usize;
let declared_size = cd + 24;
z[declared_size..declared_size + 4].copy_from_slice(&2u32.to_le_bytes());
assert!(extract(&z, "cities15000.txt", 100).is_err());
}
}
crates/gpu/src/ctx.rs
+56 -6
View File
@@ -499,11 +499,31 @@ impl Gpu {
self.over_limit(bytes.len() as u64);
return Buf { buf: Some(Tracked::new(self.placeholder())), len };
}
let buf = self.device.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: None,
contents: bytes,
usage: wgpu::BufferUsages::STORAGE | wgpu::BufferUsages::COPY_SRC | wgpu::BufferUsages::COPY_DST,
});
let usage = wgpu::BufferUsages::STORAGE | wgpu::BufferUsages::COPY_SRC | wgpu::BufferUsages::COPY_DST;
if let Some(size) = wgpu::BufferSize::new(bytes.len() as u64).filter(|s| s.get() % wgpu::COPY_BUFFER_ALIGNMENT == 0) {
// through the queue: the bytes are copied once, into a staging buffer the next submission copies from, in
// parallel parts (`create_buffer_init` zeroes a staging buffer first and copies over that, two passes on
// one thread over a full-size photo's ~290 MB)
let buf = self.device.create_buffer(&wgpu::BufferDescriptor { label: None, size: size.get(), usage, mapped_at_creation: false });
if let Some(mut view) = self.queue.write_buffer_with(&buf, 0, size) {
const PART: usize = 1 << 20;
{
let (whole, mut tail) = view.slice(..).into_chunks::<PART>();
let (src, src_tail) = bytes.as_chunks::<PART>();
let mut parts: Vec<_> = whole.into_iter().zip(src).map(Some).collect();
lightcraft_raster::par_rows(&mut parts, 1, |_, part| {
for (d, s) in part.iter_mut().filter_map(Option::take) {
wgpu::WriteOnly::<[u8]>::from(d).copy_from_slice(s);
}
});
tail.copy_from_slice(src_tail);
}
// (dropping the view schedules the copy)
drop(view);
return Buf { buf: Some(Tracked::new(buf)), len };
}
}
let buf = self.device.create_buffer_init(&wgpu::util::BufferInitDescriptor { label: None, contents: bytes, usage });
Buf { buf: Some(Tracked::new(buf)), len }
}
@@ -612,7 +632,17 @@ impl Gpu {
Ok(d) => d,
Err(e) => return failed(format!("{e:?}")),
};
let out: Vec<T> = bytemuck::cast_slice(&data).to_vec();
// copied out in parallel parts: one thread reads mapped memory slowly (~2 GB/s, so ~45 ms for a 24 MP
// export's pixels), and the new vector's pages are first touched there too
let src: &[u8] = &data;
// (`len` counts 32-bit words: as many `T` as they fill)
let mut out = vec![<T as bytemuck::Zeroable>::zeroed(); src.len() / std::mem::size_of::<T>().max(1)];
const PART: usize = 1 << 20;
lightcraft_raster::par_rows(bytemuck::cast_slice_mut::<T, u8>(&mut out), PART, |k, part| {
if let Some(s) = src.get(k * PART..k * PART + part.len()) {
part.copy_from_slice(s);
}
});
drop(data);
staging.unmap();
out
@@ -811,6 +841,26 @@ mod tests {
Some(stdout)
}
/// An upload reaches the device whole: it is copied into staging memory in parallel parts, the last one
/// shorter than the others (and a small upload is that last part alone).
#[test]
fn uploads_read_back_exactly() {
let Some(g) = crate::device() else {
lightcraft_testkit::skip!(Gpu, "no GPU device");
return;
};
for len in [1usize, 1000, (3 << 20) / 4 + 777] {
let data: Vec<u32> = (0..len as u32).map(|i| i.wrapping_mul(2_654_435_761)).collect();
let b = g.upload(&data);
let back: Vec<u32> = g.finish_and_read(g.encoder(), &b, len);
assert!(back == data, "{len} values");
}
// read back as wider values (`len` counts 32-bit words), as `read_rgb` does
let data: Vec<u32> = (0..3 << 20).collect();
let b = g.upload(&data);
let back: Vec<[u32; 3]> = g.finish_and_read(g.encoder(), &b, data.len());
assert!(back.len() == 1 << 20 && back.as_flattened() == data.as_slice());
}
/// Issue #1858: once a Vulkan or DX12 device is lost, wgpu 30 panics in `Device::poll` instead of
/// returning an error. A render's readback panicked; while it unwound, a buffer's way back to
/// the pool polls again from a destructor, and that second panic aborted the app.
crates/gpu/src/nn.rs
+17 -14
View File
@@ -235,32 +235,35 @@ fn round_up(v: usize, to: usize) -> usize {
v.div_ceil(to) * to
}
/// Work out the steps, the buffers and the arranged weights for `net` on `tile × tile` input cells.
fn plan(net: &Net, tile: usize, max_binding: u64) -> Result<(Layout, Vec<Layer>), String> {
fn validate_plan(net: &Net, tile: usize, ops: &[Op]) -> Result<(), String> {
if !net.fits_tile(tile) {
return unsupported("the tile does not fit the network");
}
if tile > MAX_TILE {
return unsupported("the tile is larger than the kernels' dispatches reach");
}
if !net.in_channels.is_multiple_of(4) {
return unsupported("the input channel count is not a multiple of 4");
}
let count = ops.len();
let ends_right = matches!(
(count.checked_sub(2).and_then(|i| ops.get(i)), ops.last()),
(Some(Op::Conv(conv)), Some(Op::DepthToSpace2 { src }))
if conv.k == 1 && conv.src2.is_none() && *src == count - 1 && conv.cout == 12 && conv.leaky.is_none()
);
if ends_right { Ok(()) } else { unsupported("the network does not end in a 1 × 1 convolution to 12 channels and depth-to-space") }
}
/// Work out the steps, the buffers and the arranged weights for `net` on `tile × tile` input cells.
fn plan(net: &Net, tile: usize, max_binding: u64) -> Result<(Layout, Vec<Layer>), String> {
let ops = net.ops();
validate_plan(net, tile, ops)?;
let side = |t: usize| -> usize {
let level = net.shape(t).map_or(0, |s| s.level);
if level >= 0 { tile >> level } else { tile << (-level) }
};
let chan = |t: usize| net.shape(t).map_or(0, |s| s.channels);
if !net.in_channels.is_multiple_of(4) {
return unsupported("the input channel count is not a multiple of 4");
}
let ops = net.ops();
// the network must end in a 1 × 1 convolution and a depth-to-space (they run as one kernel)
let n_ops = ops.len();
let ends_right = matches!(
(n_ops.checked_sub(2).and_then(|i| ops.get(i)), ops.last()),
(Some(Op::Conv(c)), Some(Op::DepthToSpace2 { src })) if c.k == 1 && c.src2.is_none() && *src == n_ops - 1 && c.cout == 12 && c.leaky.is_none()
);
if !ends_right {
return unsupported("the network does not end in a 1 × 1 convolution to 12 channels and depth-to-space");
}
// the last layer that reads each tensor
let mut last_use = vec![0usize; n_ops + 1];
for (i, op) in ops.iter().enumerate() {

Some files were not shown because too many files have changed in this diff Show More