Phase 6.3.17: measure the gate 5 baseline against production

Ran the full agent loop against vllm-qwen36 on port 8000 (read-only HTTP)
with the Penpot plugin live, before asking for any downtime. Without this
file "it improved" would be a claim rather than a measurement.

Result over the 8 graded prompts: mean score 14.9, zero prompts at or above
60, veto violated on 2 of 8, 66% of execute_code calls raised, and 7 of 8
prompts burned all 14 turns without producing a final message.

The plan predicted production would score near zero on distinct colours and
style richness while producing a high shape count - grey boxes. The shape
count is also zero. On a fresh page it creates nothing at all, so the
failure sits upstream of the grey boxes: the model invents a Figma-shaped
API wholesale and every call throws. From the captured turns:

  penpot.currentPage()                     is a property, not a function
  penpot.createRectangle(page, 200, 56)    takes no arguments
  penpot.createText(page, ...)             takes one, the text
  penpot.getPageById(...)                  lives on penpotUtils
  fills   = [{type:'solid', color:{r,g,b,a}}]   is {fillColor, fillOpacity}
  shadows = [{type:'drop', x, y, blur, ...}]    is {style, offsetX, offsetY}

It then spends the remaining turns querying penpot_api_info without
recovering. So the reported symptom understated it.

Two robustness fixes the run itself forced, both after losing a completed
run to them:
- A ConnectionError does not just drop the request, it can drop the MCP
  session, so retrying the same tools/call against a dead session fails
  identically every time - which is exactly what the first attempt showed,
  four retries and four identical ConnectionErrors. The client now redoes
  the handshake before retrying, and that recovered two drops in this run.
- Results are written after every prompt. The first attempt died on prompt
  4 and lost the three already measured, which is the expensive data
  precisely because it requires production to be up.

data/gate5_prod_baseline_run.log

@@ -0,0 +1,77 @@
+[OK] los 8 prompts son disjuntos de train/eval/seeds (7472 shingles de referencia)
+[MCP] handshake ok con 'penpot-mcp-server'; tools: execute_code, high_level_overview, penpot_api_info, export_shape
+[MCP] sonda de conectividad del plugin: {
+  "result": {
+    "gate5Probe": true
+  },
+  "log": ""
+}
+
+--- g5-01-boton-primario (baja) -> pagina gate5/prod-baseline/g5-01-boton-primario/20260730T175220Z
+[MCP] fallo transitorio en 'tools/call' (intento 1/5): error de transporte contra el MCP de Penpot: ConnectionError. Reintento en 4s
+[MCP] rehaciendo el handshake (la sesion pudo haber muerto)
+[MCP] handshake ok con 'penpot-mcp-server'; tools: execute_code, high_level_overview, penpot_api_info, export_shape
+[WARN] g5-01-boton-primario: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-01-boton-primario: export_shape no devolvio binario
+    score=0.0 (2/14) grises=0 prohibido=7 shapes=0 colores=0 textos=0
+
+--- g5-02-navbar-flex (baja) -> pagina gate5/prod-baseline/g5-02-navbar-flex/20260730T175644Z
+[WARN] g5-02-navbar-flex: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-02-navbar-flex: export_shape no devolvio binario
+    score=17.6 (3/17) grises=0 prohibido=0 shapes=0 colores=0 textos=0
+
+--- g5-03-card-producto (media) -> pagina gate5/prod-baseline/g5-03-card-producto/20260730T175753Z
+[WARN] g5-03-card-producto: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-03-card-producto: export_shape no devolvio binario
+    score=20.0 (3/15) grises=0 prohibido=0 shapes=0 colores=0 textos=0
+
+--- g5-04-tokens-biblioteca (media) -> pagina gate5/prod-baseline/g5-04-tokens-biblioteca/20260730T175855Z
+[WARN] g5-04-tokens-biblioteca: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-04-tokens-biblioteca: export_shape no devolvio binario
+    score=20.0 (3/15) grises=0 prohibido=0 shapes=0 colores=0 textos=0
+
+--- g5-05-grid-tres-diferenciales (media) -> pagina gate5/prod-baseline/g5-05-grid-tres-diferenciales/20260730T175957Z
+[WARN] g5-05-grid-tres-diferenciales: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-05-grid-tres-diferenciales: export_shape no devolvio binario
+    score=17.6 (3/17) grises=0 prohibido=0 shapes=0 colores=0 textos=0
+
+--- g5-06-landing-pizzeria (alta) -> pagina gate5/prod-baseline/g5-06-landing-pizzeria/20260730T180102Z
+[MCP] fallo transitorio en 'tools/call' (intento 1/5): error de transporte contra el MCP de Penpot: ConnectionError. Reintento en 4s
+[MCP] rehaciendo el handshake (la sesion pudo haber muerto)
+[MCP] handshake ok con 'penpot-mcp-server'; tools: execute_code, high_level_overview, penpot_api_info, export_shape
+[WARN] g5-06-landing-pizzeria: export_shape no devolvio binario
+    score=0.0 (3/17) grises=0 prohibido=5 shapes=0 colores=0 textos=0
+
+--- g5-07-onboarding-mobile (alta) -> pagina gate5/prod-baseline/g5-07-onboarding-mobile/20260730T180525Z
+[WARN] g5-07-onboarding-mobile: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-07-onboarding-mobile: export_shape no devolvio binario
+    score=17.6 (3/17) grises=0 prohibido=0 shapes=0 colores=0 textos=0
+
+--- g5-08-reparacion-grises (alta) -> pagina gate5/prod-baseline/g5-08-reparacion-grises/20260730T180627Z
+[WARN] g5-08-reparacion-grises: se agotaron los 14 turnos sin mensaje final
+[WARN] g5-08-reparacion-grises: export_shape no devolvio binario
+    score=26.7 (4/15) grises=0 prohibido=0 shapes=0 colores=0 textos=0
+
+=== Puerta 5 -- calidad de diseno en Penpot (tag=prod-baseline) ===
+
+  prompt                           dif     score    pasa  grises  prohib  shapes  colores  textos
+  g5-01-boton-primario             baja      0.0   2/14        0       7       0        0       0
+  g5-02-navbar-flex                baja     17.6   3/17        0       0       0        0       0
+  g5-03-card-producto              media    20.0   3/15        0       0       0        0       0
+  g5-04-tokens-biblioteca          media    20.0   3/15        0       0       0        0       0
+  g5-05-grid-tres-diferenciales    media    17.6   3/17        0       0       0        0       0
+  g5-06-landing-pizzeria           alta      0.0   3/17        0       5       0        0       0
+  g5-07-onboarding-mobile          alta     17.6   3/17        0       0       0        0       0
+  g5-08-reparacion-grises          alta     26.7   4/15        0       0       0        0       0
+
+  condiciones de aprobacion:
+    [NO ] score medio >= 65                        -> 14.9
+    [NO ] veto limpio en 8/8                       -> 6/8
+    [NO ] >= 6/8 prompts >= 60                     -> 0/8
+    [NO ] landing de pizzeria >= 60                -> 0.0
+
+  VEREDICTO GLOBAL: NO APRUEBA
+
+  tiempo total: 15.1 min
+  resultados en data/gate5_results_prod-baseline.json
+  PNGs en data/gate5_png_prod-baseline/ (la puerta no borra nada: las paginas quedan para inspeccion humana)

scripts/35_gate5_penpot_design.py

@@ -205,6 +205,7 @@ class PenpotMCP:
         self.session_id = None
         self.tool_names = []
         self._next_id = 0
+        self._in_handshake = False
         self._http = requests.Session()
 
     # -- transporte -------------------------------------------------------------------
@@ -306,10 +307,21 @@ class PenpotMCP:
                 ultimo = e
                 if intento == MCP_MAX_RETRIES:
                     break
-                espera = MCP_RETRY_BACKOFF * intento
+                espera = MCP_RETRY_BACKOFF * (2 ** (intento - 1))
                 print(f"[MCP] fallo transitorio en '{method}' (intento {intento}/"
                       f"{MCP_MAX_RETRIES}): {e}. Reintento en {espera:.0f}s")
                 time.sleep(espera)
+                # Un ConnectionError no solo tira la request: puede tirar la SESION. Reintentar
+                # el mismo tools/call contra una sesion muerta falla siempre igual, que es lo
+                # que se vio en la primera corrida (4 intentos identicos, 4 ConnectionError).
+                # Hay que rehacer el handshake antes de reintentar.
+                if not self._in_handshake and method != "initialize":
+                    try:
+                        print("[MCP] rehaciendo el handshake (la sesion pudo haber muerto)")
+                        self.session_id = None
+                        self.handshake()
+                    except Gate5Error as e2:
+                        print(f"[MCP] el re-handshake tambien fallo: {e2}")
         raise Gate5Error(f"el MCP de Penpot fallo {MCP_MAX_RETRIES} veces en '{method}': {ultimo}")
 
     def _notify(self, method, params=None):
@@ -320,6 +332,13 @@ class PenpotMCP:
 
     # -- protocolo --------------------------------------------------------------------
     def handshake(self):
+        self._in_handshake = True
+        try:
+            return self._handshake_inner()
+        finally:
+            self._in_handshake = False
+
+    def _handshake_inner(self):
         result = self._request("initialize", {
             "protocolVersion": self.PROTOCOL_VERSION,
             "capabilities": {},
@@ -1280,29 +1299,54 @@ def run_full(cfg, tools, system_prompt):
     print(f"[MCP] sonda de conectividad del plugin: {sonda[:120]}")
 
     png_dir = REPO_ROOT / "data" / f"gate5_png_{cfg['GATE5_TAG']}"
-    t0 = time.time()
-    resultados = [run_prompt(cfg, mcp, tools, system_prompt, fila, png_dir) for fila in prompts]
-    dt = time.time() - t0
-
-    aprueba, veredicto = resumen(resultados, cfg["GATE5_TAG"])
-    print(f"\n  tiempo total: {dt / 60:.1f} min")
-
     salida = REPO_ROOT / "data" / f"gate5_results_{cfg['GATE5_TAG']}.json"
+
+    def volcar(resultados_parciales, dt_parcial, veredicto_parcial=None, completo=False):
+        """Escribe el JSON de resultados. Se llama DESPUES DE CADA PROMPT.
+
+        La primera corrida del baseline murio en el prompt 4 por una caida de conexion del MCP
+        y se perdio la medicion de los tres que ya habian terminado -- que era justamente el
+        dato caro, porque exige tener produccion arriba. Un fallo de transporte no puede costar
+        el trabajo ya hecho.
+        """
         salida.write_text(json.dumps({
-        "modo": "completo",
+            "modo": "completo" if completo else "parcial",
             "tag": cfg["GATE5_TAG"],
             "modelo": cfg["GATE5_MODEL"],
             "max_turnos": MAX_TURNS,
             "max_tokens": MAX_TOKENS,
             "timestamp": datetime.now(timezone.utc).isoformat(),
-        "duracion_s": round(dt, 1),
+            "duracion_s": round(dt_parcial, 1),
+            "prompts_completados": len(resultados_parciales),
+            "prompts_totales": len(prompts),
             "metricas": [{"clave": k, "etiqueta": e, "comparador": c} for k, e, c in METRICS],
             "veto": list(VETO_METRICS),
             "umbrales": {pid: {k: (list(v) if isinstance(v, tuple) else v) for k, v in u.items()}
                          for pid, u in THRESHOLDS.items()},
-        "veredicto": veredicto,
+            "veredicto": veredicto_parcial,
-        "prompts": resultados,
+            "prompts": resultados_parciales,
         }, ensure_ascii=False, indent=2), encoding="utf-8")
+
+    t0 = time.time()
+    resultados = []
+    for fila in prompts:
+        try:
+            resultados.append(run_prompt(cfg, mcp, tools, system_prompt, fila, png_dir))
+        except PluginNotConnected:
+            volcar(resultados, time.time() - t0)
+            raise
+        except Gate5Error as e:
+            print(f"[ERROR] {fila['id']}: {e}")
+            print(f"[INFO] se guardan los {len(resultados)} prompt(s) ya medidos y se sigue con "
+                  f"el siguiente; el JSON queda marcado como parcial")
+            resultados.append({"id": fila["id"], "error": str(e), "score": None})
+        volcar(resultados, time.time() - t0)
+    dt = time.time() - t0
+
+    aprueba, veredicto = resumen(resultados, cfg["GATE5_TAG"])
+    print(f"\n  tiempo total: {dt / 60:.1f} min")
+
+    volcar(resultados, dt, veredicto_parcial=veredicto, completo=True)
     print(f"  resultados en {salida.relative_to(REPO_ROOT)}")
     print(f"  PNGs en {png_dir.relative_to(REPO_ROOT)}/ (la puerta no borra nada: las paginas "
           f"quedan para inspeccion humana)")