On an Intel Mac with an AMD Radeon Pro 5300M the Metal compiler crashes
on candle's mlx_gemm kernel, candle-metal-kernels panics on the compile
error, and the worker reported "the AI model failed unexpectedly" and
loaded the model on Metal again for every next job, so AI masks never
worked there.
The worker now retries a job that fails on the GPU (a panic or a candle
error) once on the CPU, after freeing the model. When that works, the
model stays on the CPU for the session and the job's outcome carries a
notice that segment_poll shows once; when the CPU fails too, the GPU
was not the cause and the next job uses it again. Missing or damaged
model files, render and remote-worker errors are not retried. Only
builds with a GPU path (macOS) retry; LIGHTCRAFT_GPU=0 and
LIGHTCRAFT_GPU_BACKEND=off now keep SAM 3 on the CPU from the start
(lightcraft_gpu::env_off_reason), like AI Denoise and the face models.
lightcraft-segment: Sam3::load_cpu, GPU_BUILD; text_logits returns the
crate's Error so Describe failures are classified like clicks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The crate ports Apache-2.0-only Hugging Face transformers code: it is
now `license = "Apache-2.0"`, and every ported file carries the
modification notice Apache-2.0 §4(b) asks for.
- Weights::open checks every tensor's byte range (and the header)
against the file's length: a truncated or damaged download is an
error at load, never a short read later; regression test with
hostile offsets.
- fetch/: the model download, only when asked for. An ordered mirror
list (env LIGHTCRAFT_SAM3_MIRRORS, a mirrors file, DEFAULT_MIRRORS —
empty until LightCraft's CDN exists, TODO for the maintainer),
fallback to the next mirror, resume via Range, redirects, connect and
stall timeouts, cancel, size caps, exact size + SHA-256 (pinned from
tools/install-sam3.sh) before an atomic rename; damaged files are
deleted, HTTP error bodies never written.
- HTTP/1.1 client over std sockets + rustls with the RustCrypto provider
(rustls-rustcrypto) and webpki-roots: no ring / aws-lc-rs, nothing
compiled from C (no new `cc` users in the app's build graph).
- Tests against a local TcpListener server (no internet): fallback,
redirects, chunked bodies, resume, 416, hash mismatch, too much data,
stalls, cancel.
- *.safetensors are gitignored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI masks from Meta's SAM 3, run in LightCraft itself: a new
`lightcraft-segment` crate ports the Hugging Face transformers
implementation (Apache-2.0) to candle, on the GPU through Metal on macOS
(CPU elsewhere), with no C/C++ and no `unsafe`.
- Object: click to include, ⌥-click to leave a part out (SAM 3's
interactive prompt encoder + two-way mask decoder).
- Describe: a text prompt selects every instance ("sky", "the red car";
"car, road" merges several) via the CLIP text encoder and DETR detector.
- Both as new masks and as Add / Subtract / Intersect components; + / −
buttons under the selected mask combine selections.
- The segmentation is stored with the mask (288² quantized logits, ~10-30
KB), so renders, exports and the web build never need the model.
- A zoomed-in detail pass (background, ~5 s on an M4 Pro) re-segments the
area around a selection from a sharper source: 5-10x finer edges on
small objects.
- Per-selection Edge slider (hard ↔ soft); hovering a mask in the list
shows it in red (the always-on overlay is now off by default, O toggles).
- The engine feature `sam` (on in the desktop app) gates the model; the
image is encoded once per photo in the background, clicks take ~35 ms.
- Weights are not bundled (SAM License): tools/install-sam3.sh / .ps1
download and verify facebook/sam3; docs/ai-masks.md.
- Clearer errors (missing file, missing model) and a stderr logger in the
app (LIGHTCRAFT_LOG=info).
candle is pinned at 0.9.2: later releases make candle-core depend on
tokenizers with the Oniguruma C library. Every stage is checked against
the Python reference (tests/reference.rs, tools/sam3_reference.py): on
Metal the backbone matches to 7e-5 relative error, masks to ~5e-5.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C8BkS7JDqa1EMcjW6spDGA
