cc54edc39dc859b63e78419c549128e96931458e
New offline script re-scans the code payloads already saved in the gate 5 result JSONs (holdout, full, generalization) using the corrected forbidden- behavior list -- no live Penpot needed, reuses scan_forbidden/score_prompt from 35_gate5_penpot_design.py as the single source of truth. Holdout result changes materially: 3 of the 5 flagged prompts (pd-008, pd-009, pd-035) were flagged only for flex.appendChild. With that pattern removed, forbidden-API rate drops from 8.3% (5/60) to 3.3% (2/60), which is now under the <=5% fallback threshold -- the holdout gate now passes. Full mode and the generalization probe are unaffected: neither run's transcripts ever triggered the flex.appendChild pattern, so their scores are identical before and after. Original result files are kept untouched; corrected results are written alongside as *-corrected.json.
Description
No description provided
465 MiB
Languages
Python
92.8%
Jinja
7.2%