Phase 6.4.25b: measure generalization probe on v2b-bf16 (ADDITIONAL, not official gate 5)

Browse Source

4 prompts in domains absent from both seeds and the official gate5_prompts.jsonl (ferreteria,
gimnasio, tienda de mascotas, panaderia). score medio 32.0, veto limpio 3/4, 0/4 prompts >=60.

gp-03-tienda-mascotas violates the veto twice over: it hits both import_image variants the
model has ever tried (penpot.importImage on turn 1, penpotUtils.importImage on turn 2) plus a
numeric fontSize assignment -- the uploadMediaUrl reinforcement seeds from this correction
round did not transfer to an unseen domain. It is also flagged via GATE5_KNOWN_OVERLAP_IDS
(shares a 6-gram with the same seeds as g5-03), so its score is a double caution case.

gp-02-gimnasio hit the same transient Penpot MCP transport drop as g5-06/g5-07 on its first
pass, but the harness's own automatic retry (pass 2/3) recovered cleanly within the default
240s -- no manual timeout override was needed here.

...

This commit is contained in:

aleleba

2026-08-04 06:43:26 +00:00

parent bf795bd01c

commit 3496051f72

6 changed files with 2214 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

data/gate5_png_v2b-bf16-generalization/gp-02-gimnasio.png

BIN

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 1.6 KiB