From 4af74b92f005ba70d1f341b96e2ab66dda31de00 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sat, 5 Sep 2026 06:05:21 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20Hebrew=20layerdrop=20verdict=20?= =?UTF-8?q?=E2=80=94=20depth-cut=20does=20not=20transfer=20(77.48=20DER)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Single-variable replication of the Arabic depth-cut on the Hebrew logit-KD lineage collapsed (+53.77pp vs full-depth's 30.38) despite normal training convergence. The compressible-depth finding is scoped: recipe- and language-dependent, not a universal property. Confound noted: Arabic used sequence-KD + Muon at 6ep; Hebrew used logit-KD at 3ep. --- docs/RESULTS.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/RESULTS.md b/docs/RESULTS.md index e9ce4d4..38dd6a6 100644 --- a/docs/RESULTS.md +++ b/docs/RESULTS.md @@ -521,3 +521,30 @@ levers: on-policy distillation — GKD in flight; label-distribution effects). G2a (2.1) remains the best student; G2b stands as the closing negative of the data-side program. Provenance: labels sha256 b59e2f56 (235.0MB). + +## heb-diac-small-s46-layerdrop — the depth-cut does NOT transfer: 77.48 DER (2026-09-05) + +Item 04 (TODO.publish-client): the Arabic width/depth finding +replicated on Hebrew — encoder 12->6 verbatim layer copy from +pretrained ByT5-small, single variable vs run-002-s46 (same s46 +teacher, hebrew-v4 corpus, 3 epochs, logit-KD recipe). Training +converged normally (val_loss 0.550); the full Nakdimon gate did not: + +| Model | DER (n=1864) | +|---|---| +| teacher s46 | 23.72 (reproduces exactly) | +| full-depth student (run-002, 1.1) | 30.38 | +| **layerdrop student (run-003)** | **77.48** — collapse | + +Paired delta +53.77pp [51.64, 55.92], p=0. + +Verdict: the "depth is the compressible axis" finding does NOT +transfer as-is. Confound, stated honestly: the Arabic rung used +sequence-KD (teacher labels, Muon, 6 epochs); this run used the Hebrew +lineage's logit-KD (alpha-KL + CE) at its native 3 epochs — so the +collapse may be recipe-dependent (depth-cut + logit-KD), not purely +linguistic. Either way the cross-lingual generalization claim is +closed as a negative: depth-compressibility is NOT a universal +property of pretrained ByT5-small; it held under one recipe on one +language. Paper B's depth paragraph is scoped accordingly (this entry +is its counterexample).