Phase 6.4.25b: parametrize gate5 prompts path, add generalization probe

GATE5_PROMPTS_PATH env var lets the same 17-metric harness (veto system,
MCP retry/page handling) run against a different prompts file. Unset
behavior is unchanged -- the official gate 5 (data/gate5_prompts.jsonl,
10 frozen prompts) stays untouched.

data/gate5_generalization_probe.jsonl: 4 prompts in rubros confirmed
absent from the 138-seed corpus, the 10 gate5 prompts, and the 60-prompt
holdout -- ferreteria, gimnasio, tienda de mascotas, panaderia. Swapped
out "veterinaria" (originally planned) after finding seed 83 is a
substantial design-repair seed for a vet clinic; "tienda de mascotas"
verified clean instead. This is an additional check, separate from the
official gate 5, meant to test whether the correction generalizes across
business types rather than overfitting to the pizzeria case.
This commit is contained in:
2026-08-03 22:53:07 +00:00
parent 20493a99b6
commit 959404823a
2 changed files with 10 additions and 1 deletions
+6 -1
View File
@@ -61,7 +61,12 @@ import requests
REPO_ROOT = Path(__file__).resolve().parent.parent
SCHEMAS_DIR = REPO_ROOT / "data" / "schemas"
PROMPTS_PATH = REPO_ROOT / "data" / "gate5_prompts.jsonl"
# GATE5_PROMPTS_PATH (Fase 6.4.25b): permite apuntar el mismo harness (17 metricas, veto,
# manejo de paginas/reintentos MCP) a un archivo de prompts distinto de los 10 oficiales de
# la puerta 5 -- p.ej. la sonda de generalizacion de rubros nunca vistos. La puerta 5 oficial
# (data/gate5_prompts.jsonl) queda intacta: sin la env var, el comportamiento es identico.
PROMPTS_PATH = Path(os.environ["GATE5_PROMPTS_PATH"]) if os.environ.get("GATE5_PROMPTS_PATH") \
else REPO_ROOT / "data" / "gate5_prompts.jsonl"
HOLDOUT_PATH = REPO_ROOT / "data" / "holdout_penpot_design.jsonl"
PENPOT_TOOLS_PATH = SCHEMAS_DIR / "penpot.json"
SYSTEM_PROMPT_PATH = SCHEMAS_DIR / "penpot_system_prompt.md"