Three defects, the first found by inspecting the user's Penpot file live
while the gate reported something else.
The audit resolved the page root with penpotUtils.findShapeById(rootId).
Every new Penpot page shares the same root frame id,
00000000-0000-0000-0000-000000000000, and findShapeById searches globally,
so that lookup always returned the root of the FIRST page in the file - the
user's empty one - rather than the page the gate had just created. That is
why the baseline reported shapeCount 0 on all ten prompts while the file
actually held 28 shapes and 20 texts. It resolves by pageId now, which is
unique.
With that fixed the baseline reproduces the reported symptom exactly rather
than something worse: the button gets 4 shapes, all grey; the navbar gets
10 shapes and 6 real texts with a single distinct fill colour. Production
does create structure and text. What it never manages is to apply colour -
it creates the shapes in a call that works, then sets fills in a later call
using Figma syntax, that call throws, and the shapes keep the #B1B2B5
default. So "creates grey boxes" was accurate and "creates nothing" was my
measurement error.
Two metrics now capture that directly, since neither distinct-colour counts
nor placeholder-grey counts see it - #FFFFFF and #000000 are not mid greys,
so a design where every fill is a default passes both. explicitFillShare is
the share of filled shapes whose colour is not one of Penpot's three
defaults, thresholded by difficulty (0.80 high, 0.65 medium, 0.50 low,
since a small artefact's structural neutrals weigh heavily in a ratio). And
onlyDefaultColors is a veto: true when the whole palette is those three.
White and black stay legitimate when chosen - the distinction is
co-presence, not the hex. They are only suspicious when they are all there
is; alongside chosen brand colours they also count as neutrals in
paletteStructured.
resumen() crashed adding None scores from failed prompts. Beyond the crash,
an unmeasured prompt must not average in as a zero: "could not measure" and
"the model did it badly" are different, and averaging them would have made
a mid-run plugin outage look like cheap quality. They are reported
separately and any missing prompt makes the verdict invalid, because a
baseline with 5 of 10 measured is not a baseline.
The gate now empties its own page after exporting the PNG. Across three
runs the plugin reliably handled 5 or 6 heavy prompts and then degraded to
30-second timeouts on createPage - that is not random flakiness, it tracks
the file growing by one page per prompt, so the gate was manufacturing its
own failure. The evidence that matters is the PNG plus the JSON metrics,
not the live page. A page is kept only when its PNG failed, so there is
something to inspect. Exit code 3 now distinguishes "plugin degraded
mid-run, reload it and resume these ids" from "plugin not connected".
aleleba
f63ff6830d