Conflict in crates/track/src/camtrack/bundle.rs (imports): upstream replaced
the linalg glob import with an explicit list; kept that list and added the
branch's SolveError import (use super::{Distortion, SolveError}). Input
validation, size budget and Result-returning adjust/cost are unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflict in crates/track/src/face.rs: upstream moved the feature-blob search
into FaceTracker::feature_blobs. Re-applied the branch there: the per-pixel
point-in-polygon loop is replaced by non_skin_inside (one crossing pass per
window row) and the roll rotation is hoisted with face_mapper.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Upstream split lib.rs files into modules and moved engine commands/ to cmd/; this branch's change merges without textual conflicts.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reuse consecutive caller-owned pyramids for regular Warp motion and trajectory tracking. Preserve the original three-level motion search by setting aside the trajectory-only extra level, without cloning planes. Keep detailed analysis at its existing distinct resolutions and test bitwise motion and complete JSON equality against separate pyramids.
Fixes#2322
Share the unchanged Jacobi factors between full SVD and null-vector callers. Select the same right singular vector directly instead of normalizing U and sorting copies that the caller discards.
Fixes#2329
Return errors for invalid bundle observations and oversized workspaces, reject non-finite projections and mismatched correspondence slices, and prevent track frame overflow. Cover the reported failures with synthetic regression tests.
Fixes#2330
Use a bounded inline array for the current camera parameters and preserve the existing accumulation order. Extra parameters spill safely if the parameter set grows.
Fixes#2327
step() cloned the whole RGBA frame to remember it (33 MB at 1080p,
133 MB at 4K) and then cropped that clone, although only the region
around the outline is ever read. The tracker now stores the previous
frame already cropped to the region the next step will use, computed
from the updated outline with the same margin and rounding as before,
so the crops, and therefore the tracking result, are unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Store the two four-element DLT rows in arrays while preserving arithmetic and accumulation order. This removes two heap allocations for every triangulated view.
Fixes#2326
The feature mask tested every non-skin window pixel against the 48-edge
face outline (about 25 M edge tests per frame). A scanline hits the same
edges for every pixel of a row, so the crossings are now computed once
per row with the same expression and comparisons as inside_polygon, and
a pixel's parity follows from how many lie to its right. The face-frame
rotation is also worked out once per frame instead of once per blob
pixel.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The nightly FreeBSD run measured 119 ms/frame against a 100 ms limit: the mean included
Tracker::new and any scheduling stalls on the two-core VM. The best single step with a
250 ms bound still catches an order-of-magnitude regression.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: with a trained model installed, Object Matte's optional box reached
MobileSAM only as background points along the box's edges (an approximation),
although the model's prompt encoder takes a box prompt directly.
Fix: a new Roto Brush stroke kind, Box (two opposite corners), that
stroke_prompt turns into the model prompt's bbox (scaled to the matte grid,
corners in any order, degenerate or non-finite boxes ignored). The graph cut
never rasterizes it. Object Matte emits it instead of the edge strokes when a
model segments; the classical fallback keeps the edge strokes. A prior matte's
box no longer replaces a box the user drew on that frame.
Tests: effects object_matte::the_box_reaches_the_model_as_a_box_prompt (a
stand-in model sees the box, at full and half resolution, and no edge points);
track roto::tests::box_stroke_is_the_model_box_prompt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- effectcraft-segment: a FaceModel interface (find a face in a region, follow it; a Topology
maps the model's points to the tracker's landmarks and outline), a TensorFlow Lite reader and
interpreter (float ops, float16 weights folded; matches ai-edge-litert to ~1e-6), and
MediaPipe Face Landmarker (Google, Apache-2.0: BlazeFace + Face Mesh V2, 3.8 MB, pinned
SHA-256). The registry gains a task and authors per model; installed weights get a NOTICE.
- effectcraft-track: FaceTracker::new_with uses the model, falls back to the classical fitter
when it finds no face; chin and jaw come from the outline either way so measurements agree.
- Engine: roto_models becomes models (both tasks); face.models / face.model.* commands,
Settings ▸ Face Tracking, `wait` on select/install/download for scripts; track.mask reports
faceModel.
- Credits: ATTRIBUTION.md lists the optional models and the Apache-2.0 specs the code follows.
- Docs: README, ROADMAP (G7), gaps, parity (TRK-3 score unchanged pending G1), agents,
preferences, architecture, segment README.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New L1 crate effectcraft-segment: the MaskModel interface (prompts → foreground probability),
a registry of open-licensed models (licence must be MIT/Apache/BSD/…; URL, size, SHA-256),
a PyTorch checkpoint reader (zip + pickle subset), dependency-free SHA-256, and MobileSAM
(TinyViT-5M + SAM prompt encoder / two-way transformer / mask decoder) on rayon and ndarray's
safe GEMM. Loads the official mobile_sam.pt as published; checked against candle
(embedding 0.3% RMS, 99.9% equal mask pixels); ~0.7 s/frame encoder at 32 threads, cached
per frame, ~0.1 s decoder.
- track::roto: segment_with / propagate_with prompt a model with the strokes or the flow-warped
matte (box, core points, mask); its probability is a strong, unblurred prior for the same
graph cut (strokes stay hard constraints); a failing model or one disagreeing with the flow
(IoU < 0.5) is ignored. Prior gets a weight.
- effects::roto: Version 2.0 / 3.0 use the model chosen in Settings (set_model / model_for);
the model id is part of the chain seed.
- engine: roto_models (roto.models, roto.model.select / install / download / remove), weights in
the models folder (host::config_dir()/models, shared by the app, CLI and MCP), background
loading, downloads through the system curl, verified; Settings ▸ Roto Brush page (prefs
roto.model). host::config_dir moved from the desktop app.
- Tests: segment (kernels vs direct sums, checkpoint reader, SHA-256 vectors, registry), track
(fake model: prior shapes the cut, failing / disagreeing models ignored; real weights with
EFFECTCRAFT_MOBILESAM: base IoU 0.989, propagated ≥ 0.980 vs classic 0.973), engine (registry,
verification, selection; real install + load), MCP.
- Docs: README, ROADMAP, parity (MSK-4 done: 90/2/0), preferences, agents, architecture,
crates/segment/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The camera solver, KLT pyramid and subspace (warp stabilizer) tracker expect()ed data-structure
invariants: a track point seen in a frame, a solved camera, a pyramid level. A missing value now
skips that point/track (or, for finish(), returns a SolveError: "no frame was solved") instead of
panicking. Denies the panic lints in the crate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Viewer frames render in long-lived Web Workers holding a project replica
(engine::remote: structural JSON diff/patch, Mirror, FrameServer); the
frame cache hands jobs to a RemoteFrames renderer (UI-thread fallback).
Roto Brush propagation runs in a job worker and streams segmentations
back (FrameSeg binary form), relayed to frame workers. Control-channel
wait:true commands whose job is offloaded reply when it ends. The Media
Browser browses a pluggable Browser: the file system on the desktop;
browser storage and File System Access folders on the web.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Structure from motion for the 3D Camera Tracker in effectcraft-track
(camtrack): long KLT feature tracks with re-detection, parallax keyframes,
two-view initialisation (normalised 8-point essential matrix in RANSAC or
planar homography decomposition), incremental resection + triangulation,
sparse Levenberg-Marquardt bundle adjustment with the Schur complement and
Huber loss, focal length search (Fixed / Variable Zoom / Specify Angle of
View), tripod pan solve, Auto Detect, ground plane / target / world mapping.
Synthetic-scene tests (path, focal, error, tripod, flat, rendered frames).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Library groundwork for mask tracking (MSK-3) and the Warp Stabilizer (TRK-2),
clean-room from published methods:
- track::klt: Shi-Tomasi corners, pyramidal Lucas-Kanade with forward-backward check
- track::fit: RANSAC + least-squares translation/scale/similarity/affine/homography
- track:😷 MaskTracker following the pixels inside a mask outline per frame
- track::stabilize: incremental clip analysis (serde WarpAnalysis), Gaussian path
smoothing (Matsushita 2006), No Motion, preserve scale, crop / auto-scale with
max-scale relaxation
- path::interp: Mask Interpolation correspondence (arc-length vertex insertion,
shape-context matching) and rigid/linear in-betweens
Engine commands, UI and the effect itself are not wired yet (stubs unchanged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tracker panel (Window > Tracker): Track Camera / Warp Stabilizer (disabled),
Track Motion / Stabilize Motion, Motion Source, Current Track, Track Type,
Position/Rotation/Scale, Motion Target with Edit Target..., Options...
(Motion Tracker Options dialog), Analyze (1 frame back, back, forward,
1 frame forward; stop while running) with progress, Reset and Apply (Apply
Dimensions dialog). Track point widgets in the viewer: feature and search
regions with corner handles, attach point crosshair and track path, all
draggable through track.setPoint (one undo step per drag). Automation ids for
every widget; headless UI test.
Tracker: featureless regions no longer drift (ties keep the prediction, only
confident matches predict motion). docs/agents.md: tracking example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L2 crate effectcraft-track: per-point feature/search region tracking with
pyramid normalized cross-correlation and Lucas-Kanade (similarity + gain/bias)
sub-pixel refinement, confidence, adapt/stop/extrapolate/continue actions,
RGB/luminance/saturation channels, blur/enhance pre-processing, and
homography/affine/similarity solves.
Project: Motion Trackers > Tracker n > Track Point n property tree
(GroupKind::Tracker with type, target and options). Engine: track.* commands
(new/motion/stabilize/property/select/setType/setTarget/options/setPoint/
analyze/stop/apply/reset/delete/status), background or blocking analysis
job with progress and cancel (one undo step), apply to Transform, Stabilize
and Corner Pin. Renderer::layer_source for the tracker's frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BFlat