Phase 6.4.25b: parametrize gate5 prompts path, add generalization probe
GATE5_PROMPTS_PATH env var lets the same 17-metric harness (veto system, MCP retry/page handling) run against a different prompts file. Unset behavior is unchanged -- the official gate 5 (data/gate5_prompts.jsonl, 10 frozen prompts) stays untouched. data/gate5_generalization_probe.jsonl: 4 prompts in rubros confirmed absent from the 138-seed corpus, the 10 gate5 prompts, and the 60-prompt holdout -- ferreteria, gimnasio, tienda de mascotas, panaderia. Swapped out "veterinaria" (originally planned) after finding seed 83 is a substantial design-repair seed for a vet clinic; "tienda de mascotas" verified clean instead. This is an additional check, separate from the official gate 5, meant to test whether the correction generalizes across business types rather than overfitting to the pizzeria case.
This commit is contained in:
@@ -61,7 +61,12 @@ import requests
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
SCHEMAS_DIR = REPO_ROOT / "data" / "schemas"
|
||||
PROMPTS_PATH = REPO_ROOT / "data" / "gate5_prompts.jsonl"
|
||||
# GATE5_PROMPTS_PATH (Fase 6.4.25b): permite apuntar el mismo harness (17 metricas, veto,
|
||||
# manejo de paginas/reintentos MCP) a un archivo de prompts distinto de los 10 oficiales de
|
||||
# la puerta 5 -- p.ej. la sonda de generalizacion de rubros nunca vistos. La puerta 5 oficial
|
||||
# (data/gate5_prompts.jsonl) queda intacta: sin la env var, el comportamiento es identico.
|
||||
PROMPTS_PATH = Path(os.environ["GATE5_PROMPTS_PATH"]) if os.environ.get("GATE5_PROMPTS_PATH") \
|
||||
else REPO_ROOT / "data" / "gate5_prompts.jsonl"
|
||||
HOLDOUT_PATH = REPO_ROOT / "data" / "holdout_penpot_design.jsonl"
|
||||
PENPOT_TOOLS_PATH = SCHEMAS_DIR / "penpot.json"
|
||||
SYSTEM_PROMPT_PATH = SCHEMAS_DIR / "penpot_system_prompt.md"
|
||||
|
||||
Reference in New Issue
Block a user