aleleba 869b00c924 Phase 6.3.17: make gate 5 measure all ten prompts in one resilient run
The Penpot MCP keeps going down and each window where it works is
expensive - three of the user's windows were spent on runs that aborted
partway. The run is now built to finish rather than to be correct about
why it stopped.

One prompt failing no longer ends the run. Failures are collected and
retried on a later pass, up to three passes, with a clean handshake and a
probe between them. Even an unexpected exception is caught per prompt,
because one bug in one prompt must not take down the other nine. The
opening probe no longer aborts either: a plugin that does not answer now
may answer on pass two, and aborting there throws away the whole window
for a transient state.

Two time budgets bound it: four minutes per prompt, checked before every
turn of the agent loop, and forty-five minutes of wall clock. Both are env
vars. A hang used to mean waiting forever; now it costs one prompt and the
run continues.

Results are written after every prompt, so a crash costs at most the
prompt in flight, and the JSON is only marked complete when all ten have a
score.

The run also inventories the accumulated gate5 pages at the end and writes
the list to disk. It does not delete them: the pages this run created are
already emptied after their PNG is exported, but the ones from earlier runs
are in the user's own file, so that is offered rather than assumed.
2026-07-30 20:27:46 +00:00
S
Description
No description provided
465 MiB
Languages
Python 92.8%
Jinja 7.2%