Gradient meshes: a grid whose point count overflows is malformed

GradientMesh::is_valid and the two mesh envelope checks (envelope_grid
and the envelope's surface) compare the number of points with
(rows + 1) × (cols + 1), multiplying the document's u32 rows and
columns unchecked. With u32::MAX rows and columns the product is 2^64:
debug builds panic on the overflow and release builds wrap it to 0, so
a mesh or a mesh envelope without points passes as valid. Selecting
it, Outline view, hit testing and export then index points that aren't
there (the mesh lines, outline and edges), tessellating it asks for an
impossible capacity, and a mesh envelope with handles to fill in loops
2^64 times making them until memory runs out. On wasm32, where usize
has 32 bits, 65,535 rows and columns are enough, and a panic there
aborts the web app.

The three checks now share grid_len, which counts the points with
checked arithmetic and gives None when the count doesn't fit in a
usize: a grid that can't be counted matches no point list, so it is
malformed like any other wrong count.

The test checks that a u32::MAX × u32::MAX mesh without points isn't
valid and has no lines, outline or quads, that envelope_grid refuses
those counts, and that a mesh envelope with them leaves art in place.

Reported-by: BS.ehlas.cz

crates/doc/src/live.rs

@@ -587,12 +587,18 @@ fn even_steps(n: u32) -> Vec<f64> {
     (0..=n).map(|i| i as f64 / n as f64).collect()
 }
 
+/// The number of points of a `(rows+1)×(cols+1)` grid, `None` past `usize`: counts come from
+/// files, and u32::MAX rows and columns (65,535 on 32-bit targets) would wrap to an empty grid.
+fn grid_len(rows: u32, cols: u32) -> Option<usize> {
+    (rows as usize).checked_add(1)?.checked_mul((cols as usize).checked_add(1)?)
+}
+
 impl GradientMesh {
     pub fn idx(&self, r: usize, c: usize) -> usize {
         r * (self.cols as usize + 1) + c
     }
     pub fn is_valid(&self) -> bool {
-        self.rows >= 1 && self.cols >= 1 && self.points.len() == (self.rows as usize + 1) * (self.cols as usize + 1)
+        self.rows >= 1 && self.cols >= 1 && grid_len(self.rows, self.cols) == Some(self.points.len())
     }
 
     /// A 1×1 mesh from its four corners (bottom-left, top-left, top-right, bottom-right in u/v
@@ -1158,7 +1164,7 @@ fn surface_of<'a>(kind: &'a EnvelopeKind, src: Rect, frame: Affine) -> Box<dyn F
             {
                 return Box::new(move |u: f64, v: f64| mesh_at(&m, u, v));
             }
-            if *rows == 0 || *cols == 0 || points.len() as u64 != (u64::from(*rows) + 1) * (u64::from(*cols) + 1) {
+            if *rows == 0 || *cols == 0 || grid_len(*rows, *cols) != Some(points.len()) {
                 return Box::new(flat);
             }
             let (rows, cols) = (*rows as usize, *cols as usize);
@@ -1210,8 +1216,7 @@ pub fn smooth_handles(rows: u32, cols: u32, points: &[Point]) -> Vec<[Vec2; 4]>
 /// the smooth ones, see [`smooth_handles`]) as a gradient mesh with black points: what the
 /// canvas draws and the Mesh tool and Direct Selection edit. `None` for a malformed grid.
 pub fn envelope_grid(rows: u32, cols: u32, points: &[Point], handles: &[[Vec2; 4]]) -> Option<GradientMesh> {
-    // (Counts come from files: no overflow on 32-bit targets.)
-    if rows == 0 || cols == 0 || points.len() as u64 != (u64::from(rows) + 1) * (u64::from(cols) + 1) {
+    if rows == 0 || cols == 0 || grid_len(rows, cols) != Some(points.len()) {
         return None;
     }
     let smooth;
@@ -1829,6 +1834,20 @@ mod tests {
         assert!((p.x - 37.0).abs() < 1e-6 && (p.y - 81.0).abs() < 1e-6, "{p:?}");
     }
 
+    /// A file's rows and columns can be anything: a point count too large to compute is no match
+    /// for the points (u32::MAX × u32::MAX wrapped to 0 and passed for a grid without points).
+    #[test]
+    fn a_grid_too_large_to_count_is_malformed() {
+        let m = GradientMesh { rows: u32::MAX, cols: u32::MAX, points: vec![] };
+        assert!(!m.is_valid());
+        assert_eq!((m.lines(), m.outline(), m.quads(8).len()), (PathData::default(), PathData::default(), 0));
+        assert!(envelope_grid(u32::MAX, u32::MAX, &[], &[]).is_none());
+        // A mesh envelope without handles maps through its points: malformed, it leaves art as it is.
+        let kind = EnvelopeKind::Mesh { rows: u32::MAX, cols: u32::MAX, points: vec![], handles: vec![] };
+        let p = Point::new(25.0, 50.0);
+        assert_eq!(envelope_mapper(&kind, Rect::new(0.0, 0.0, 100.0, 100.0))(p), p);
+    }
+
     #[test]
     fn mesh_insert_and_remove_lines() {
         let mut m =