Phase 6.4.25b: measure gate 5 full mode on v2b-bf16 (post-correction)

Browse Source

10/10 prompts measured. VEREDICTO: NO APRUEBA (score medio 39.8 vs threshold 65, veto limpio
8/10, 3/10 prompts >=60). The flagship g5-06-landing-pizzeria now clears its own threshold
(69.6 >= 60) once a transient Penpot MCP transport drop is ruled out (see g5-06/g5-07 rerun
below) -- a real improvement over v2-bf16's 34.8 on that prompt, but the aggregate gate
criteria still fail. g5-03-card-producto is flagged via GATE5_KNOWN_OVERLAP_IDS: it shares a
6-gram with seeds 134-137 already baked into v2b-bf16's training, so its result should be
read with caution rather than as a clean generalization signal.

g5-06/g5-07 were re-measured once with GATE5_ONLY + GATE5_PROMPT_TIMEOUT_S=420 (default 240s
untouched) after their first pass hit a genuine Penpot MCP transport drop (ConnectionError +
handshake rebuild) that ate into the turn budget and caused a false-negative timeout. g5-04
and g5-08 show the same 'no board created' signature but their raw turn-by-turn transcripts
have zero MCP/transport/handshake evidence -- both are the known post-error tool-call-format
bug (literal '<tool_call>' text instead of a structured call), so they are left as genuine
failures with no rerun.

...

This commit is contained in:

aleleba

2026-08-04 06:43:10 +00:00

parent c7b1311928

commit bf795bd01c

10 changed files with 4526 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

data/gate5_results_v2b-bf16-full.json

+4461

File diff suppressed because one or more lines are too long