Phase 6.4.25: point vllm-eval at v2-bf16 and record gate 2/5-holdout results

Browse Source

Gate 2 on v2-bf16: PASS, 192/200 = 96.0% (penpot 60/60 = 100.0%).
Gate 5 holdout mode (60 prompts, fallback trigger): FAIL, forbidden-API
rate 15.0% (9/60) vs threshold <=5% -- model still occasionally invents
penpot.importImage(...) for photo/URL prompts.

...

This commit is contained in:

aleleba

2026-08-03 07:31:04 +00:00

parent 363648fb5f

commit 0cf8bb1df7

3 changed files with 3751 additions and 2 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

data/gate5_holdout_v2-bf16-holdout.json

+887

File diff suppressed because one or more lines are too long