Phase 6.4.25b: gate 1 on v2b-bf16 -- full PASS on all buckets
Global weighted loss 0.2725 (vs baseline 0.2750, previous v2-bf16 was 0.2734). All 6/6 non-penpot buckets pass the 0.10 regression threshold, including delegacion_subagentes which had narrowly failed on v2-bf16 (+0.1019 -> now +0.0882). The 13 corrective seeds didn't hurt forgetting. Also points vllm-eval at the new v2b-bf16 checkpoint for the gate 2/5 re-measurement.
This commit is contained in:
@@ -0,0 +1,47 @@
|
|||||||
|
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||||||
|
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||||||
|
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||||||
|
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||||||
|
dispatch key: ADInplaceOrView
|
||||||
|
previous kernel: no debug info
|
||||||
|
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||||||
|
self.m.impl(
|
||||||
|
[transformers] The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
|
||||||
|
[INFO] cargando checkpoint mergeado desde /workspace/ft-models/Qwen3.6-35B-A3B-mcp-v2b-bf16
|
||||||
|
␍Loading weights: 0%| | 0/693 [00:00<?, ?it/s]␍Loading weights: 66%|██████▌ | 457/693 [00:00<00:00, 4566.97it/s]␍Loading weights: 100%|██████████| 693/693 [00:00<00:00, 6103.32it/s]
|
||||||
|
[INFO] modelo cargado en 298.1s
|
||||||
|
[INFO] 147 ejemplos en data/eval.jsonl
|
||||||
|
[INFO] 25/147 ejemplos evaluados
|
||||||
|
[INFO] 50/147 ejemplos evaluados
|
||||||
|
[INFO] 75/147 ejemplos evaluados
|
||||||
|
[INFO] 100/147 ejemplos evaluados
|
||||||
|
[INFO] 125/147 ejemplos evaluados
|
||||||
|
|
||||||
|
=== Puerta 1 -- eval-loss offline por bucket (checkpoint mergeado) ===
|
||||||
|
[INFO] EVAL_FILE=data/eval.jsonl
|
||||||
|
[INFO] BASELINE_EVAL_LOSS=0.2750
|
||||||
|
[INFO] tiempo de eval: 138.7s, memoria pico: 69.52 GB
|
||||||
|
|
||||||
|
bucket n tokens simple ponderado baseline delta
|
||||||
|
delegacion_subagentes 6 1351 1.3580 1.1657 1.0775 +0.0882
|
||||||
|
manejo_errores 6 1691 0.6403 0.6350 0.5579 +0.0771
|
||||||
|
negativos 7 920 1.7187 1.7455 1.7138 +0.0317
|
||||||
|
otros_mcps 29 5614 0.3849 0.4088 0.3973 +0.0115
|
||||||
|
penpot 11 2075 0.8814 0.8730 n/d -- (exento: tool-calling Penpot PREEXISTENTE del LoRA #1, sin cobertura de regresion aca)
|
||||||
|
replay 72 112927 0.1807 0.1833 0.1733 +0.0100
|
||||||
|
skills_adherencia 16 4271 1.4482 1.4177 1.3304 +0.0873
|
||||||
|
|
||||||
|
bucket=replay (aislado) n= 72 loss_avg_simple=0.1807 loss_avg_weighted=0.1833
|
||||||
|
|
||||||
|
loss_avg GLOBAL simple (por ejemplo) = 0.5514
|
||||||
|
loss_avg GLOBAL ponderado (por token) = 0.2725
|
||||||
|
baseline (ponderado por token) = 0.2750
|
||||||
|
diferencia absoluta (ponderado vs baseline) = 0.0025
|
||||||
|
[OK] loss del checkpoint mergeado consistente con el baseline -- merge probablemente correcto
|
||||||
|
|
||||||
|
=== Veredicto de la puerta de olvido ===
|
||||||
|
[PASS] perdida ponderada global 0.2725 <= 0.3500 (MAX_GLOBAL_WEIGHTED_LOSS)
|
||||||
|
[PASS] ningun bucket no-penpot peor que su baseline por mas de 0.1000 (MAX_BUCKET_REGRESSION) -- umbral evaluado sobre 6/7 buckets
|
||||||
|
|
||||||
|
VEREDICTO PUERTA 1: PASS
|
||||||
|
EXIT_CODE:0
|
||||||
@@ -7,13 +7,15 @@ services:
|
|||||||
ports:
|
ports:
|
||||||
- "8001:8000"
|
- "8001:8000"
|
||||||
volumes:
|
volumes:
|
||||||
# Fase 6.4.25: apunta al candidato v2-bf16 (base + LoRA #1 + LoRA #2), no al
|
# Fase 6.4.25b: apunta al candidato v2b-bf16 (base + LoRA #1 + LoRA #2
|
||||||
# bf16 de Fase 4 (que solo tiene el LoRA #1). Es el checkpoint contra el que
|
# corregido con los 13 seeds del diagnostico de causa raiz), no al v2-bf16
|
||||||
# corren las puertas 5 y 2 del disparador de fallback antes de cuantizar.
|
# original (que fallo la puerta 5 modo full por el bug de formato de
|
||||||
- /home/aleleba/ft-models/Qwen3.6-35B-A3B-mcp-v2-bf16:/model:ro
|
# tool-call). Es el checkpoint contra el que se remide la puerta 5 tras la
|
||||||
|
# iteracion correctiva, antes de decidir NVFP4 vs fallback.
|
||||||
|
- /home/aleleba/ft-models/Qwen3.6-35B-A3B-mcp-v2b-bf16:/model:ro
|
||||||
command:
|
command:
|
||||||
- "--model=/model"
|
- "--model=/model"
|
||||||
- "--served-model-name=qwen3.6-35b-a3b-mcp-v2-bf16"
|
- "--served-model-name=qwen3.6-35b-a3b-mcp-v2b-bf16"
|
||||||
- "--tensor-parallel-size=1"
|
- "--tensor-parallel-size=1"
|
||||||
- "--max-model-len=32768"
|
- "--max-model-len=32768"
|
||||||
- "--enable-auto-tool-choice"
|
- "--enable-auto-tool-choice"
|
||||||
|
|||||||
Reference in New Issue
Block a user