Repository navigation
perf(hermitcrab): run the required-slot look-ahead on the shared trailing-slot template walk - #577
johnml1135 wants to merge 6 commits into
Conversation
Rows 43 and 44 move from in progress to done. R1 on #546 is draft PR #577 (commit bbb81a0, shadow 0 violations); R3 ordered look-ahead is draft PR #576 (shadow 0 violations). Adds the round-doc section for both and the PanGloss port issues (sillsdev/PanGloss #65, #66). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #577 +/- ##
==========================================
+ Coverage 74.57% 74.81% +0.23%
==========================================
Files 453 455 +2
Lines 38342 38749 +407
Branches 5284 5375 +91
==========================================
+ Hits 28593 28989 +396
- Misses 8583 8593 +10
- Partials 1166 1167 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| Walk(child, word, state, fired); | ||
| } | ||
| if (child.Optional) | ||
| Walk(child, input, state, fired); |
There was a problem hiding this comment.
Minor: F1. Skipping an optional slot re-runs the look-ahead on the same word. When child.Optional, Walk(child, input, ...) checks every live template from slot Count - 1 - child.Depth (line 127), but the parent already scanned input from the child's slot through that same slot. If the skipped slot is pure, the result is the same, so the repeat is pure cost. The per-template ApplySlots does not re-check on a skip. That may explain the 1.07% (Mbugwe) and 1.40% (Sena) MORE unification calls than #546 alone. Re-check only when the skipped slot is impure. Unverified: I read the code but did not count unifications (a scratch test was not permitted).
| input.Freeze(); | ||
| Word[] expected = own.Apply(input).ToArray(); | ||
| Word[] actual = shared.Apply(input).ToArray(); | ||
| Assert.That(actual, Is.EqualTo(expected).Using(FreezableEqualityComparer<Word>.Default), text); |
There was a problem hiding this comment.
FYI: F2. This exhaustive test pins shared-On only to per-template-On, so a bad prune in both paths (for example a wrong slot index) would still pass. It never compares against Off, which is the PR's soundness claim. It also compares with FreezableEqualityComparer, while the sibling test (lines 66-69) uses AnalysisMergeKey, the identity analysis folding uses. Adding shared(On) == own(Off) under AnalysisMergeKey costs one more Apply per input.
Public API, target framework, package, and Findings: F1 new (unverified); F2 new. Reviewed at bbb81a0 |
Adds the round-doc section "Combined stack and R2's cost": the full stack against master (work, unification, matcher per grammar), additivity (near-additive on Mbugwe, overlapping on Sena, super-additive on Aweti), R2's root cause (its own probe, low prune rate) and the witness-reuse fix folded into #572 @ 01bdf5a, the 1-4 point residual, and the landing decision: #572 is a draft and lands only on top of #577 and #574. Stress grammars are PR #578 (opt-in, [Explicit], not run in CI). Ledger: row 40 (R2) Outcome records the fold and the landing decision; new row 47 records the combined stack measurement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k guard, avoid per-node closures The shared walk is only taken when MaxAlternatives is 0, so its limit check could never fire; the guard comment now names all five fallback conditions; Walk checks admitted templates with a loop instead of a capturing lambda. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#522 removed AnalysisStateKey along with the analysis-cascade memo and kept the same identity as AnalysisStratumRule.AnalysisMergeKey, so the shared-walk equivalence test now compares outputs under that key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erial An analysis template partial now stops as soon as a required slot can no longer find the segments its affix inserts, instead of paying for every optional slot before the required one. The test is used only while every slot walked since is a pure copy-and-insert rule, so parses do not change. A count-only shadow mode and a null-by-default check observer let a harness verify soundness and cost on real grammars. Closes #561 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… walk Each template under a trie node gets its own look-ahead and a dead template stops collecting below it while its siblings go on. Closes #566 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bbb81a0 to
fcb65d7
Compare
Start here:
src/SIL.Machine.Morphology.HermitCrab/AnalysisAffixTemplatesRule.cs, theWalkmethod -- the per-template look-ahead mask on the shared trie walk. Review only the top commit (fcb65d7); the branch also carries #546 and #575.Status: draft, stacked on #546 (shared trailing-slot trie, open) and #575 (required-slot look-ahead, open). Its base is master, so the PR diff also shows both. CI green. Closes #566.
What it does. The shared trailing-slot walk from #546 now runs #575's required-slot look-ahead at every trie node, for each admitted template below it. A template whose required slot cannot find its material is dead for that subtree while the templates sharing the node carry on; a child whose templates are all dead is skipped before its slot rule runs. On the benchmark grammars, adding the look-ahead to the shared walk cut a further 8.26% (Mbugwe), 25.97% (Sena) and 18.75% (Aweti) of work.
The unknown you arrive with: is it sound with sharing? Dead templates cannot complete (the #575 argument, applied per template), so outputs equal the shared walk without the look-ahead. The mask is one
boolper template per call: no cache, noInterlocked. The sameRequiredSlotLookaheadOff/Shadow/On switch as #575 applies; shadow reports fired, violated and completed-under-fired counts per template.Where to look
LookaheadPrunesOnlyTheTemplateWhoseRequiredSlotIsGone.LookaheadKeepsEveryAttemptWhenBothTemplatesAreFeasible.ShadowLookaheadOnTheSharedWalkNeverViolatesOrChangesTheOutput.LookaheadSharedWalkMatchesPerTemplateWalkOnEveryShortInput.Not here. The shared walk still runs only at
Morpher.MaxDegreeOfParallelism == 1, as in #546; default callers get throughput from #582 (open). The look-ahead adds unification calls: 1.07% (Mbugwe) and 1.40% (Sena) more than #546 alone. Later look-ahead refinements are #576, #579 and #580.Stack and verification. Merge order: #546, #575, then this. Ran earlier: HermitCrab test project, parity against master, #546, #575 and the combination on every benchmark word; shadow-mode violations 0. Not rerun after the rebase onto #522: the measurements below. The four truncate-morphotactic conformance failures are known.
Next: land #546 and #575, then review the top commit.
Reading this a year from now -- start here
This is the join of two independent optimisations: #546 removes repeated slot batches across templates, #575 prunes a partial parse as soon as a required slot's material is gone. Each alone leaves work the other removes, so the join is measured on its own (#566).
Evidence
Analysis non-template events plus synthesis steps, and unification calls including the look-ahead's own checks, master versus each variant. Measured before #522 removed the analysis memo; not rerun.
Added to #546: 8.26%, 25.97% and 18.75% less work on Mbugwe, Sena and Aweti, with 1.07% and 1.40% MORE unification calls on Mbugwe and Sena and 11.18% less on Aweti. Of the shared-slot work (about 65% of events before), 5.0% (Mbugwe), 20.2% (Sena) and 11.4% (Aweti) of events remain after both.
Shadow: 0 violations and 0 completions under a fired check on Mbugwe, Sena, Aweti, Amharic, Indonesian, conformance and the stress fixtures. Parses equal master, #546, #575 and the combination on every word.
Decisions and why
🤖 Generated with Claude Code
This change is