Phase 2 v1 (reduced volume) of the Qwen3.6-35B-A3B LoRA fine-tuning pipeline: builds the training dataset from sanitized sources plus authored synthetic content, ready for end-to-end validation before investing in the full volume (~2500-3000 examples planned for a later pass).
Sanitizes 77 source files from ~/.claude/ (skills, agents, plans) into data/raw/sanitized/, removing secrets with detect-secrets plus 8 project-specific regexes.
Authors 294 synthetic training seeds across 6 buckets, grounded in the real MCP schemas and sanitized skills/agents.
Assembles and validates the final 1470-example dataset (750 new + 720 Phase 1 replay), split 90/10 stratified by bucket, and confirms it tokenizes cleanly end-to-end with the real production chat template.
Changes
File
Change
scripts/04_sanitize.py
Secret scrubbing over 77 source files (5 SKILL.md, 7 agents, 65 plans) into data/raw/sanitized/; 10 unique secrets stably substituted; gate green
data/raw/sanitized/**
Sanitized copies of skills, agents, and plans used as grounding material for the seeds
data/raw/seeds/*.jsonl
294 training seeds across 6 buckets: penpot (41), other MCPs combined - gitea/github-personal/docmost/atlassian (102), skill adherence (55), subagent delegation (25), negatives/no-tool (46), error handling (25); structured messages+tools+tool_calls format with dict arguments; spark-ssh skill deliberately held out (0 mentions, verified)
data/schemas/SEED_FORMAT.md
Documents the seed format used to author the synthetic examples
scripts/05_build_dataset.py
Deterministic-variation assembly (seed 42) of the 294 seeds + 720 Phase 1 replay examples into 1470 combined examples, 90/10 stratified split by bucket
data/train.jsonl
1323 training examples produced by the assembly script
scripts/06_validate_dataset.py
Validates the dataset with the real tokenizer/chat_template (transformers 5.14.1) inside the qwen-lora-train container on spark
data/chat_template_train.jinja
Training fallback chat template: exact copy of the production template plus {% generation %} tags around assistant turns only (needed for loss masking), production template untouched
Test Plan
scripts/04_sanitize.py gate passes green with 0 residual secrets across 77 sanitized files
scripts/05_build_dataset.py produces 1470 combined examples with a 90/10 stratified split by bucket (1323 train / 147 eval)
scripts/06_validate_dataset.py run against the real tokenizer/chat_template inside the qwen-lora-train container on spark: 1470/1470 examples ok, 0 exceptions, 0 filtered by length (8192 token limit), 0 surviving secrets
End-to-end LoRA training smoke run on this v1 dataset before committing to the full Phase 2 volume
## Summary
- Phase 2 v1 (reduced volume) of the Qwen3.6-35B-A3B LoRA fine-tuning pipeline: builds the training dataset from sanitized sources plus authored synthetic content, ready for end-to-end validation before investing in the full volume (~2500-3000 examples planned for a later pass).
- Sanitizes 77 source files from `~/.claude/` (skills, agents, plans) into `data/raw/sanitized/`, removing secrets with `detect-secrets` plus 8 project-specific regexes.
- Authors 294 synthetic training seeds across 6 buckets, grounded in the real MCP schemas and sanitized skills/agents.
- Assembles and validates the final 1470-example dataset (750 new + 720 Phase 1 replay), split 90/10 stratified by bucket, and confirms it tokenizes cleanly end-to-end with the real production chat template.
## Changes
| File | Change |
|------|--------|
| `scripts/04_sanitize.py` | Secret scrubbing over 77 source files (5 SKILL.md, 7 agents, 65 plans) into `data/raw/sanitized/`; 10 unique secrets stably substituted; gate green |
| `data/raw/sanitized/**` | Sanitized copies of skills, agents, and plans used as grounding material for the seeds |
| `data/raw/seeds/*.jsonl` | 294 training seeds across 6 buckets: penpot (41), other MCPs combined - gitea/github-personal/docmost/atlassian (102), skill adherence (55), subagent delegation (25), negatives/no-tool (46), error handling (25); structured `messages`+`tools`+`tool_calls` format with dict arguments; `spark-ssh` skill deliberately held out (0 mentions, verified) |
| `data/schemas/SEED_FORMAT.md` | Documents the seed format used to author the synthetic examples |
| `scripts/05_build_dataset.py` | Deterministic-variation assembly (seed 42) of the 294 seeds + 720 Phase 1 replay examples into 1470 combined examples, 90/10 stratified split by bucket |
| `data/train.jsonl` | 1323 training examples produced by the assembly script |
| `scripts/06_validate_dataset.py` | Validates the dataset with the real tokenizer/chat_template (transformers 5.14.1) inside the `qwen-lora-train` container on spark |
| `data/chat_template_train.jinja` | Training fallback chat template: exact copy of the production template plus `{% generation %}` tags around assistant turns only (needed for loss masking), production template untouched |
## Test Plan
- [x] `scripts/04_sanitize.py` gate passes green with 0 residual secrets across 77 sanitized files
- [x] `scripts/05_build_dataset.py` produces 1470 combined examples with a 90/10 stratified split by bucket (1323 train / 147 eval)
- [x] `scripts/06_validate_dataset.py` run against the real tokenizer/chat_template inside the `qwen-lora-train` container on spark: 1470/1470 examples ok, 0 exceptions, 0 filtered by length (8192 token limit), 0 surviving secrets
- [ ] End-to-end LoRA training smoke run on this v1 dataset before committing to the full Phase 2 volume
El chat_template.jinja de produccion no tiene tags {% generation %}, por lo que
return_assistant_tokens_mask salia vacio para el 100% de los ejemplos en el primer run.
Se genero data/chat_template_train.jinja (copia exacta del template real, con {%- generation -%}
envolviendo solo el contenido/tool_calls/im_end de cada turno assistant) para el fallback
de masking ya anticipado en la Decision de diseno #4 del plan principal -- el chat_template.jinja
original no se toca, sigue siendo el que sirve produccion.
aleleba
merged commit 5f912a15db into master2026-07-28 19:54:40 -06:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
~/.claude/(skills, agents, plans) intodata/raw/sanitized/, removing secrets withdetect-secretsplus 8 project-specific regexes.Changes
scripts/04_sanitize.pydata/raw/sanitized/; 10 unique secrets stably substituted; gate greendata/raw/sanitized/**data/raw/seeds/*.jsonlmessages+tools+tool_callsformat with dict arguments;spark-sshskill deliberately held out (0 mentions, verified)data/schemas/SEED_FORMAT.mdscripts/05_build_dataset.pydata/train.jsonlscripts/06_validate_dataset.pyqwen-lora-traincontainer on sparkdata/chat_template_train.jinja{% generation %}tags around assistant turns only (needed for loss masking), production template untouchedTest Plan
scripts/04_sanitize.pygate passes green with 0 residual secrets across 77 sanitized filesscripts/05_build_dataset.pyproduces 1470 combined examples with a 90/10 stratified split by bucket (1323 train / 147 eval)scripts/06_validate_dataset.pyrun against the real tokenizer/chat_template inside theqwen-lora-traincontainer on spark: 1470/1470 examples ok, 0 exceptions, 0 filtered by length (8192 token limit), 0 surviving secretsEl chat_template.jinja de produccion no tiene tags {% generation %}, por lo que return_assistant_tokens_mask salia vacio para el 100% de los ejemplos en el primer run. Se genero data/chat_template_train.jinja (copia exacta del template real, con {%- generation -%} envolviendo solo el contenido/tool_calls/im_end de cada turno assistant) para el fallback de masking ya anticipado en la Decision de diseno #4 del plan principal -- el chat_template.jinja original no se toca, sigue siendo el que sirve produccion.