aleleba
20493a99b6
Global weighted loss 0.2725 (vs baseline 0.2750, previous v2-bf16 was 0.2734). All 6/6 non-penpot buckets pass the 0.10 regression threshold, including delegacion_subagentes which had narrowly failed on v2-bf16 (+0.1019 -> now +0.0882). The 13 corrective seeds didn't hurt forgetting. Also points vllm-eval at the new v2b-bf16 checkpoint for the gate 2/5 re-measurement.
48 lines
3.3 KiB
Plaintext
48 lines
3.3 KiB
Plaintext
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
|
Overriding a previously registered kernel for the same operator and the same dispatch key
|
|
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
|
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
|
dispatch key: ADInplaceOrView
|
|
previous kernel: no debug info
|
|
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
|
self.m.impl(
|
|
[transformers] The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
|
|
[INFO] cargando checkpoint mergeado desde /workspace/ft-models/Qwen3.6-35B-A3B-mcp-v2b-bf16
|
|
|
|
Loading weights: 0%| | 0/693 [00:00<?, ?it/s]
|
|
Loading weights: 66%|██████▌ | 457/693 [00:00<00:00, 4566.97it/s]
|
|
Loading weights: 100%|██████████| 693/693 [00:00<00:00, 6103.32it/s]
|
|
[INFO] modelo cargado en 298.1s
|
|
[INFO] 147 ejemplos en data/eval.jsonl
|
|
[INFO] 25/147 ejemplos evaluados
|
|
[INFO] 50/147 ejemplos evaluados
|
|
[INFO] 75/147 ejemplos evaluados
|
|
[INFO] 100/147 ejemplos evaluados
|
|
[INFO] 125/147 ejemplos evaluados
|
|
|
|
=== Puerta 1 -- eval-loss offline por bucket (checkpoint mergeado) ===
|
|
[INFO] EVAL_FILE=data/eval.jsonl
|
|
[INFO] BASELINE_EVAL_LOSS=0.2750
|
|
[INFO] tiempo de eval: 138.7s, memoria pico: 69.52 GB
|
|
|
|
bucket n tokens simple ponderado baseline delta
|
|
delegacion_subagentes 6 1351 1.3580 1.1657 1.0775 +0.0882
|
|
manejo_errores 6 1691 0.6403 0.6350 0.5579 +0.0771
|
|
negativos 7 920 1.7187 1.7455 1.7138 +0.0317
|
|
otros_mcps 29 5614 0.3849 0.4088 0.3973 +0.0115
|
|
penpot 11 2075 0.8814 0.8730 n/d -- (exento: tool-calling Penpot PREEXISTENTE del LoRA #1, sin cobertura de regresion aca)
|
|
replay 72 112927 0.1807 0.1833 0.1733 +0.0100
|
|
skills_adherencia 16 4271 1.4482 1.4177 1.3304 +0.0873
|
|
|
|
bucket=replay (aislado) n= 72 loss_avg_simple=0.1807 loss_avg_weighted=0.1833
|
|
|
|
loss_avg GLOBAL simple (por ejemplo) = 0.5514
|
|
loss_avg GLOBAL ponderado (por token) = 0.2725
|
|
baseline (ponderado por token) = 0.2750
|
|
diferencia absoluta (ponderado vs baseline) = 0.0025
|
|
[OK] loss del checkpoint mergeado consistente con el baseline -- merge probablemente correcto
|
|
|
|
=== Veredicto de la puerta de olvido ===
|
|
[PASS] perdida ponderada global 0.2725 <= 0.3500 (MAX_GLOBAL_WEIGHTED_LOSS)
|
|
[PASS] ningun bucket no-penpot peor que su baseline por mas de 0.1000 (MAX_BUCKET_REGRESSION) -- umbral evaluado sobre 6/7 buckets
|
|
|
|
VEREDICTO PUERTA 1: PASS
|
|
EXIT_CODE:0
|