Skip to content

perf(hermitcrab): run the required-slot look-ahead on the shared trailing-slot template walk - #577

Draft
johnml1135 wants to merge 6 commits into
masterfrom
perf/hc-lookahead-on-546
Draft

johnml1135 wants to merge 6 commits into
masterfrom
perf/hc-lookahead-on-546

Conversation

@johnml1135

@johnml1135 johnml1135 commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Start here: src/SIL.Machine.Morphology.HermitCrab/AnalysisAffixTemplatesRule.cs, the Walk method -- the per-template look-ahead mask on the shared trie walk. Review only the top commit (fcb65d7); the branch also carries #546 and #575.

Status: draft, stacked on #546 (shared trailing-slot trie, open) and #575 (required-slot look-ahead, open). Its base is master, so the PR diff also shows both. CI green. Closes #566.

What it does. The shared trailing-slot walk from #546 now runs #575's required-slot look-ahead at every trie node, for each admitted template below it. A template whose required slot cannot find its material is dead for that subtree while the templates sharing the node carry on; a child whose templates are all dead is skipped before its slot rule runs. On the benchmark grammars, adding the look-ahead to the shared walk cut a further 8.26% (Mbugwe), 25.97% (Sena) and 18.75% (Aweti) of work.

The unknown you arrive with: is it sound with sharing? Dead templates cannot complete (the #575 argument, applied per template), so outputs equal the shared walk without the look-ahead. The mask is one bool per template per call: no cache, no Interlocked. The same RequiredSlotLookahead Off/Shadow/On switch as #575 applies; shadow reports fired, violated and completed-under-fired counts per template.

Where to look

  • Pruning one template without losing its neighbour: LookaheadPrunesOnlyTheTemplateWhoseRequiredSlotIsGone.
  • No extra attempts when nothing is pruned: LookaheadKeepsEveryAttemptWhenBothTemplatesAreFeasible.
  • Shadow mode on the shared walk: ShadowLookaheadOnTheSharedWalkNeverViolatesOrChangesTheOutput.
  • Equality with the per-template walk on every input up to length 5: LookaheadSharedWalkMatchesPerTemplateWalkOnEveryShortInput.

Not here. The shared walk still runs only at Morpher.MaxDegreeOfParallelism == 1, as in #546; default callers get throughput from #582 (open). The look-ahead adds unification calls: 1.07% (Mbugwe) and 1.40% (Sena) more than #546 alone. Later look-ahead refinements are #576, #579 and #580.

Stack and verification. Merge order: #546, #575, then this. Ran earlier: HermitCrab test project, parity against master, #546, #575 and the combination on every benchmark word; shadow-mode violations 0. Not rerun after the rebase onto #522: the measurements below. The four truncate-morphotactic conformance failures are known.

Next: land #546 and #575, then review the top commit.


Reading this a year from now -- start here

This is the join of two independent optimisations: #546 removes repeated slot batches across templates, #575 prunes a partial parse as soon as a required slot's material is gone. Each alone leaves work the other removes, so the join is measured on its own (#566).

Evidence

Analysis non-template events plus synthesis steps, and unification calls including the look-ahead's own checks, master versus each variant. Measured before #522 removed the analysis memo; not rerun.

grammar #575 alone #546 alone both
Mbugwe 7.07% less / 0.33% less 26.05% less / 16.09% less 32.15% less / 15.19% less
Sena 34.30% less / 1.56% less 40.09% less / 6.13% less 55.64% less / 4.81% less
Aweti 7.90% less / 4.50% less 1.61% less / 1.14% less 20.06% less / 12.18% less
Amharic 9.10% less / 0.63% MORE 0.00% equal 9.10% less / 0.63% MORE
Indonesian equal equal equal

Added to #546: 8.26%, 25.97% and 18.75% less work on Mbugwe, Sena and Aweti, with 1.07% and 1.40% MORE unification calls on Mbugwe and Sena and 11.18% less on Aweti. Of the shared-slot work (about 65% of events before), 5.0% (Mbugwe), 20.2% (Sena) and 11.4% (Aweti) of events remain after both.

Shadow: 0 violations and 0 completions under a fired check on Mbugwe, Sena, Aweti, Amharic, Indonesian, conformance and the stress fixtures. Parses equal master, #546, #575 and the combination on every word.

Decisions and why

🤖 Generated with Claude Code


This change is Reviewable

johnml1135 added a commit that referenced this pull request Oct 28, 2026
Rows 43 and 44 move from in progress to done. R1 on #546 is draft PR #577
(commit bbb81a0, shadow 0 violations); R3 ordered look-ahead is draft
PR #576 (shadow 0 violations). Adds the round-doc section for both and
the PanGloss port issues (sillsdev/PanGloss #65, #66).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.25234% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.81%. Comparing base (c245476) to head (fcb65d7).

Files with missing lines Patch % Lines
...Morphology.HermitCrab/AnalysisAffixTemplateRule.cs 78.43% 16 Missing and 6 partials ⚠️
...hine.Morphology.HermitCrab/RequiredSlotMaterial.cs 90.00% 9 Missing and 6 partials ⚠️
...orphology.HermitCrab/AnalysisAffixTemplatesRule.cs 94.70% 7 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #577      +/-   ##
==========================================
+ Coverage   74.57%   74.81%   +0.23%     
==========================================
  Files         453      455       +2     
  Lines       38342    38749     +407     
  Branches     5284     5375      +91     
==========================================
+ Hits        28593    28989     +396     
- Misses       8583     8593      +10     
- Partials     1166     1167       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Walk(child, word, state, fired);
}
if (child.Optional)
Walk(child, input, state, fired);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: F1. Skipping an optional slot re-runs the look-ahead on the same word. When child.Optional, Walk(child, input, ...) checks every live template from slot Count - 1 - child.Depth (line 127), but the parent already scanned input from the child's slot through that same slot. If the skipped slot is pure, the result is the same, so the repeat is pure cost. The per-template ApplySlots does not re-check on a skip. That may explain the 1.07% (Mbugwe) and 1.40% (Sena) MORE unification calls than #546 alone. Re-check only when the skipped slot is impure. Unverified: I read the code but did not count unifications (a scratch test was not permitted).

input.Freeze();
Word[] expected = own.Apply(input).ToArray();
Word[] actual = shared.Apply(input).ToArray();
Assert.That(actual, Is.EqualTo(expected).Using(FreezableEqualityComparer<Word>.Default), text);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: F2. This exhaustive test pins shared-On only to per-template-On, so a bad prune in both paths (for example a wrong slot index) would still pass. It never compares against Off, which is the PR's soundness claim. It also compares with FreezableEqualityComparer, while the sibling test (lines 66-69) uses AnalysisMergeKey, the identity analysis folding uses. Adding shared(On) == own(Off) under AnalysisMergeKey costs one more Apply per input.

@claude

claude Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor
  1. Verdict: approve with fixes. The shared walk keeps outputs sound: dead templates are marked on entry and cleared on return, and the slot index Count - 1 - node.Depth agrees with the per-template walk.
  2. Most important: F1 (src/SIL.Machine.Morphology.HermitCrab/AnalysisAffixTemplatesRule.cs:170). An optional-slot skip re-runs the look-ahead the parent node just ran on the same word. That plausibly accounts for the extra unification calls reported against perf(hermitcrab): share trailing analysis template slot batches #546.
  3. Counts: Critical 0, Important 1, Low 1.
  4. Ran: gh pr checks 577 shows both builds (ubuntu-22.04, windows-latest), the NuGet package and comment hygiene passing, with claude-review pending. I read bbb81a00 against AnalysisAffixTemplateRule.RequiredSlotCannotUnapply and ApplySlots.
  5. Not verified: the unification overhead in F1, because a scratch test was not permitted here. I also did not check the reported grammar measurements. Per the PR description, I scoped this round to bbb81a00; the perf(hermitcrab): share trailing analysis template slot batches #546 and Prune affix template partials whose required slot cannot find its material #575 code it carries belongs to those reviews.

Public API, target framework, package, and sillsdev/machine.py parity: none verified changed. The new members are internal, and HermitCrab has no Python counterpart.

Findings: F1 new (unverified); F2 new.

Reviewed at bbb81a0

johnml1135 added a commit that referenced this pull request Oct 28, 2026
Adds the round-doc section "Combined stack and R2's cost": the full stack
against master (work, unification, matcher per grammar), additivity
(near-additive on Mbugwe, overlapping on Sena, super-additive on Aweti),
R2's root cause (its own probe, low prune rate) and the witness-reuse fix
folded into #572 @ 01bdf5a, the 1-4 point residual, and the landing
decision: #572 is a draft and lands only on top of #577 and #574. Stress
grammars are PR #578 (opt-in, [Explicit], not run in CI).

Ledger: row 40 (R2) Outcome records the fold and the landing decision;
new row 47 records the combined stack measurement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 requested a review from jtmaxwell3 October 28, 2026 18:08
johnml1135 and others added 6 commits October 28, 2026 21:42
…k guard, avoid per-node closures

The shared walk is only taken when MaxAlternatives is 0, so its limit check
could never fire; the guard comment now names all five fallback conditions;
Walk checks admitted templates with a loop instead of a capturing lambda.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#522 removed AnalysisStateKey along with the analysis-cascade memo and
kept the same identity as AnalysisStratumRule.AnalysisMergeKey, so the
shared-walk equivalence test now compares outputs under that key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erial

An analysis template partial now stops as soon as a required slot can no
longer find the segments its affix inserts, instead of paying for every
optional slot before the required one. The test is used only while every slot
walked since is a pure copy-and-insert rule, so parses do not change. A
count-only shadow mode and a null-by-default check observer let a harness
verify soundness and cost on real grammars.

Closes #561

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… walk

Each template under a trie node gets its own look-ahead and a dead template stops collecting below it while its siblings go on.

Closes #566

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HermitCrab analysis: measure required-slot look-ahead on the #546 shared walk

3 participants