aleleba/qwen3-6-lora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 6.4.25: point vllm-eval at v2-bf16 and record gate 2/5-holdout results

Browse Source 
Gate 2 on v2-bf16: PASS, 192/200 = 96.0% (penpot 60/60 = 100.0%).
Gate 5 holdout mode (60 prompts, fallback trigger): FAIL, forbidden-API
rate 15.0% (9/60) vs threshold <=5% -- model still occasionally invents
penpot.importImage(...) for photo/URL prompts.
This commit is contained in:
aleleba
2026-08-03 07:31:04 +00:00
parent 363648fb5f
commit 0cf8bb1df7
3 changed files with 3751 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/gate2_results_v2bf16.json
+2859
View File
File diff suppressed because it is too large Load Diff
data/gate5_holdout_v2-bf16-holdout.json
+887
View File
File diff suppressed because one or more lines are too long
docker-compose.eval.yml
+5 -2
View File
@@ -7,10 +7,13 @@ services:
ports: ports:
- "8001:8000" - "8001:8000"
volumes: volumes:
- /home/aleleba/ft-models/Qwen3.6-35B-A3B-mcp-bf16:/model:ro # Fase 6.4.25: apunta al candidato v2-bf16 (base + LoRA #1 + LoRA #2), no al
# bf16 de Fase 4 (que solo tiene el LoRA #1). Es el checkpoint contra el que
# corren las puertas 5 y 2 del disparador de fallback antes de cuantizar.
- /home/aleleba/ft-models/Qwen3.6-35B-A3B-mcp-v2-bf16:/model:ro
command: command:
- "--model=/model" - "--model=/model"
- "--served-model-name=qwen3.6-35b-a3b-mcp-bf16" - "--served-model-name=qwen3.6-35b-a3b-mcp-v2-bf16"
- "--tensor-parallel-size=1" - "--tensor-parallel-size=1"
- "--max-model-len=32768" - "--max-model-len=32768"
- "--enable-auto-tool-choice" - "--enable-auto-tool-choice"