Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/paper.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -288,10 +288,11 @@ The Arabic client line has since been extended to a complete, confidence-bracket
|layerdrop (enc 12→6, Muon, 6 ep) |190M |5.78 |[3.03, 3.49]
|2.0 rung (Muon, 3 ep) |300M |4.82 |[2.36, 2.82]
|2.1 rung (Muon, 6 ep) |300M |4.57 |[1.91, 2.35]
|2.1 + full Tashkeela (G2b) |300M |4.82 |[2.19, 2.55]
|teacher r7 |580M |2.29 |—
|===

Intervals are disjoint between adjacent rungs end to end: the frontier's separations are statistically real, not seed noise. Two structural findings sit on it. First, *pretrained width is load-bearing; depth is the compressible axis*. SVD width-stitching of the pretrained ByT5-small fails at both tested ratios (3.8× narrow: 82.96 — worse than the from-scratch collapse; 2× narrow: 78.23), while a verbatim layer copy of encoder layers 12→6 trains to 5.78 full-set — a shippable rung at 63% of the parameters whose 1.21pp depth cost at 6 epochs is CI-separated from the full-depth peer. Width surgery destroys the pretrained representation; depth surgery spends it. Second, the epochs lever is real but secondary: doubling 3→6 epochs moves the rung 4.82→4.57 (−0.25pp) — most of the residual is not undertraining. A pre-registered causal test of the remaining domain-coverage attribution swapped 8k news-domain units for classical-register Tashkeela at constant 30k budget and came back *negative* (5.81, −0.98pp vs control): register mix at constant budget does not fix the residual. The add-direction variant — 48k total with the full cleaned Tashkeela corpus (5× classical coverage, all other levers held) — is the live test of the domain-coverage attribution.
Intervals are disjoint between adjacent rungs end to end: the frontier's separations are statistically real, not seed noise. Two structural findings sit on it. First, *pretrained width is load-bearing; depth is the compressible axis*. SVD width-stitching of the pretrained ByT5-small fails at both tested ratios (3.8× narrow: 82.96 — worse than the from-scratch collapse; 2× narrow: 78.23), while a verbatim layer copy of encoder layers 12→6 trains to 5.78 full-set — a shippable rung at 63% of the parameters whose 1.21pp depth cost at 6 epochs is CI-separated from the full-depth peer. Width surgery destroys the pretrained representation; depth surgery spends it. Second, the epochs lever is real but secondary: doubling 3→6 epochs moves the rung 4.82→4.57 (−0.25pp) — most of the residual is not undertraining. Two pre-registered causal tests closed the domain-coverage attribution. The swap direction — 8k news-domain units replaced by classical-register Tashkeela at constant 30k budget — came back *negative* (5.81, −0.98pp vs control). The add direction — 48k total with the full cleaned Tashkeela corpus (5× classical coverage, all other levers held, 39,018 steps) — came back *flat-negative*: 4.8231 full-set, delta vs teacher 2.3717 [2.194, 2.554], statistically indistinguishable from the 2.1 rung (4.5701 [1.91, 2.35]) with the point estimate 0.25pp worse. Both directions of the classical-corpus lever fail: the residual is *not* a classical-domain coverage deficit. It reframes as a teacher–student interaction the corpus cannot reach — candidate levers are on-policy distillation (in flight) and label-distribution effects, not more in-domain text.

== The decode protocol is part of the measurement
[[section-decode]]
Expand Down
Loading