387b237ee5efbff7af63df4029c8e2012a164423
1
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c9792c5c40
|
Phase 6.3: rewrite the Penpot seed corpus and build the LoRA #2 mix
Replaces the 41 old Penpot seeds with 105 new ones. The old set taught three API forms that do not exist - findShapeById(page, id), shape.layout, and createText() with no argument - and 21 of the 41 used the first one, so patching was never an option: keeping them would mean fine-tuning against the correction. The valuable lessons were re-founded on the real API instead (the reversed children array in flex, persisting ids in storage, never logging what you also return). 105 rather than 96 because nine multi-section compositions are split into two trajectories each: the first builds the skeleton and persists ids, palette, scale and helper functions in storage, the second recovers them and fills the sections. That was forced by the 3000-token ceiling, but it is better pedagogy anyway, and it is what execute_code's own description asks for. It also paid for itself: the helpers cost ~600 chars once instead of twice, and the skeleton call needs no export_shape, which freed the budget to carry the verbatim system block. Measured against the linter with the corpus-wide thresholds active: 143 unique code payloads (the old set had 36), 32% of seeds carrying the server's system block verbatim (the old set had none), and every coverage category met where the old set had zero addGridLayout, zero shadows, zero uploadMediaUrl and zero layoutChild. The flagship changed domain from pizzeria to an empanada shop. Gate 5's prompt 6 is the exact production failure, and the seed had been written to the same wording - a literal substring of the gate prompt, sharing two 6-gram shingles where the gate's disjointness check fails on one. Training on the prompt we then evaluate would make gate 5 measure memorisation. The real pizzeria prompt still runs in the human acceptance test. The same check also caught an onboarding seed sitting too close to gate prompt 7. Mix: 1000 examples split 90/10, giving 901 train and 99 eval. The mix is 1000 rather than 900 because 900 has to be the post-split train size: 900/16 = 56.25 steps per epoch x 3 = 168 steps, deliberately paired with phase 3's 166 so the optimiser trajectory length is comparable. Building 900 and then carving out eval would have left 811 and 152 steps, silently breaking that pairing. Two guards in the builder had to be corrected against the real corpus: - The forbidden-pattern scan now reads only `code` payloads. Scanning the serialised example flagged the corrective seeds themselves - the one that opens with the user asking "Importá esta imagen con import_image", the one quoting the overview line that mentions import_image verbatim, the one explaining that board.layout does not exist. They name the wrong API precisely in order to teach against it. - The exemption for error-recovery seeds is derived from content, not from the mix portion: a forbidden pattern is allowed only where its tool result is a real error string and a later payload does the same thing without it. Keying on the portion broke as soon as an error-recovery seed lived in group A1, where the findShapeById arity seed naturally belongs. Validated with the production tokenizer at preserve_thinking=true: p50 2466, p90 2988, max 3250 tokens over the 105 seeds. Ten sit just above 3000, so MAX_TOKENS goes to 3300 for the run and the smoke run will train on the 32 longest examples specifically, turning the peak-memory question into a ten-minute measurement instead of a risk discovered hours in. |