From 0e3f126542972d7b05ff428bbeac3a4c0aee7d79 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Fri, 4 Sep 2026 13:44:05 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20Paper=20B=20=E2=80=94=20G2b=20cell=20cl?= =?UTF-8?q?oses=20the=20domain-coverage=20attribution?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Frontier table gains the G2b rung (4.8231 [2.194,2.554], CI-overlapping with 2.1's [1.91,2.35]); the causal-tests paragraph reframes: both directions of the classical-corpus lever fail, the residual is a teacher-student interaction the corpus cannot reach, and on-policy distillation is the live lever. --- docs/paper.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/paper.adoc b/docs/paper.adoc index cfeb568..a59d26a 100644 --- a/docs/paper.adoc +++ b/docs/paper.adoc @@ -288,10 +288,11 @@ The Arabic client line has since been extended to a complete, confidence-bracket |layerdrop (enc 12→6, Muon, 6 ep) |190M |5.78 |[3.03, 3.49] |2.0 rung (Muon, 3 ep) |300M |4.82 |[2.36, 2.82] |2.1 rung (Muon, 6 ep) |300M |4.57 |[1.91, 2.35] +|2.1 + full Tashkeela (G2b) |300M |4.82 |[2.19, 2.55] |teacher r7 |580M |2.29 |— |=== -Intervals are disjoint between adjacent rungs end to end: the frontier's separations are statistically real, not seed noise. Two structural findings sit on it. First, *pretrained width is load-bearing; depth is the compressible axis*. SVD width-stitching of the pretrained ByT5-small fails at both tested ratios (3.8× narrow: 82.96 — worse than the from-scratch collapse; 2× narrow: 78.23), while a verbatim layer copy of encoder layers 12→6 trains to 5.78 full-set — a shippable rung at 63% of the parameters whose 1.21pp depth cost at 6 epochs is CI-separated from the full-depth peer. Width surgery destroys the pretrained representation; depth surgery spends it. Second, the epochs lever is real but secondary: doubling 3→6 epochs moves the rung 4.82→4.57 (−0.25pp) — most of the residual is not undertraining. A pre-registered causal test of the remaining domain-coverage attribution swapped 8k news-domain units for classical-register Tashkeela at constant 30k budget and came back *negative* (5.81, −0.98pp vs control): register mix at constant budget does not fix the residual. The add-direction variant — 48k total with the full cleaned Tashkeela corpus (5× classical coverage, all other levers held) — is the live test of the domain-coverage attribution. +Intervals are disjoint between adjacent rungs end to end: the frontier's separations are statistically real, not seed noise. Two structural findings sit on it. First, *pretrained width is load-bearing; depth is the compressible axis*. SVD width-stitching of the pretrained ByT5-small fails at both tested ratios (3.8× narrow: 82.96 — worse than the from-scratch collapse; 2× narrow: 78.23), while a verbatim layer copy of encoder layers 12→6 trains to 5.78 full-set — a shippable rung at 63% of the parameters whose 1.21pp depth cost at 6 epochs is CI-separated from the full-depth peer. Width surgery destroys the pretrained representation; depth surgery spends it. Second, the epochs lever is real but secondary: doubling 3→6 epochs moves the rung 4.82→4.57 (−0.25pp) — most of the residual is not undertraining. Two pre-registered causal tests closed the domain-coverage attribution. The swap direction — 8k news-domain units replaced by classical-register Tashkeela at constant 30k budget — came back *negative* (5.81, −0.98pp vs control). The add direction — 48k total with the full cleaned Tashkeela corpus (5× classical coverage, all other levers held, 39,018 steps) — came back *flat-negative*: 4.8231 full-set, delta vs teacher 2.3717 [2.194, 2.554], statistically indistinguishable from the 2.1 rung (4.5701 [1.91, 2.35]) with the point estimate 0.25pp worse. Both directions of the classical-corpus lever fail: the residual is *not* a classical-domain coverage deficit. It reframes as a teacher–student interaction the corpus cannot reach — candidate levers are on-policy distillation (in flight) and label-distribution effects, not more in-domain text. == The decode protocol is part of the measurement [[section-decode]]