[johnml1135]

Start here: conformance-stress/README.md -- it lists the 18 grammars, what each exercises and which optimization it was built for.

Status: stacked on #480 — merge in order #556 → #500 → #480 → #578.

Status: draft. Stacked on #480 (open); merge #480 first, and the base integrate-conformance-framework has moved since this was cut. CI is red: the Build jobs (ubuntu and windows) fail EveryConformanceFixturePassesSelfCheck, and the Conformance suite job fails 3 truncate-morphotactic fixtures (truncate-morphotactic-copy-u, -identity, -subsumption). Both name the truncate-morphotactic family that #480 already treats as nondeterministic, not these grammars; not compared against #480's own CI.

Adds 18 HermitCrab stress grammars in a new conformance-stress/ directory, a sibling of conformance/, for optimization work. They are not conformance fixtures and do not run by default. Each models a language type where one analysis or synthesis optimization matters: Bantu slot templates, agglutinative early-lexicon roots, dense co-occurrence, MPR concord, environment-conditioned allomorphs, and others. Expected parses are master's output, so a fixture is a parity check: an optimization must leave parses unchanged. Work is compared in step counts, not time.

The unknown a reviewer arrives with is "does this change CI or the conformance numbers?". It should not: nothing that enumerates fixtures scans conformance-stress/, and no workflow selects it.

Where to look

• tests/SIL.Machine.Morphology.HermitCrab.Tests/StressFixtureParityTests.cs: [Explicit] and [Category("Stress")], so a plain dotnet test reports it not executed. Opt in with dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests --filter TestCategory=Stress.
• Runner: opt in with --fixtures conformance-stress --include-pathological; the conformance runner needs --fixtures <path> and is otherwise pointed at conformance/.
• Isolation: GrammarCoverageGate, ledgers, manifests and counterfactual coverage read conformance/ only; no workflow (ci.yml, conformance.yml, counterfactual-coverage.yml) mentions conformance-stress.
• Marking: each of the 18 words.yaml files starts with # kind: stress.

Not here: polundanasak in early-lexicon-agglutinative (engines before #548, merged, trace an extra phonological rule for it); a trace-memory fixture with up to 864 parses per word; any marking of the flaky truncate fixtures (#480 has expect_fail, expect_skip and expect_crash, and no known-nondeterministic state).

Verification: parity runs and the plain dotnet test and runner comparisons against #480's head were done on the earlier 16-grammar version; they were not re-run for the 18-grammar head, so no pass counts are claimed here. csharpier and comment hygiene were clean then, and the comment-hygiene jobs pass now.

Next: confirm conformance-stress/ as an opt-in directory here, and settle #480 so this can leave draft.

---

Reading this a year from now -- start here

These grammars exist so an analysis or synthesis optimization can be checked for unchanged parses and measured in steps, without adding to the conformance suite's runtime or its coverage ledgers. The first 16 were added together; bantu-tam-fusion (built for the impure-slot look-ahead stacked on #576) and truncating-slot-chain (the worst case for the supply-set look-ahead stacked on #579) were added after. Both new grammars are synthetic, not real languages.

Decisions and why

• Separate directory, not a flag inside conformance/: the coverage gate, ledgers, manifests and counterfactual coverage all enumerate conformance/, so nothing there has to learn to skip these.
• [Explicit] plus a category: the test reuses Fixture.DiscoverAll and Runner.RunSelfCheck from the conformance runner, so no new harness.
• Expected parses come from master, three identical runs required (per the README).

Paths not taken

Marking the flaky truncate fixtures was left out because #480 has no known-nondeterministic state. A trace-memory fixture was left out because its words have up to 864 parses each.

Deferred

Whether the red Build and Conformance jobs match #480's own was not checked against #480's CI. A known-nondeterministic state in the conformance framework would let the truncate family stop failing CI; that belongs to #480.

🤖 Generated with Claude Code

---

This change is 

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.80%. Comparing base (e9569ec) to head (c343892).

Additional details and impacted files

@@                       Coverage Diff                        @@
##           integrate-conformance-framework     #578   +/-   ##
================================================================
  Coverage                            74.80%   74.80%           
================================================================
  Files                                  556      556           
  Lines                                56544    56544           
  Branches                              7722     7722           
================================================================
  Hits                                 42297    42297           
  Misses                               12354    12354           
  Partials                              1893     1893           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[claude]

Minor: F7. This slot rule still sets outputPartOfSpeech, but words.yaml:19 marks the fixture fieldworks_producible: true. 1992485 removed this attribute from four other grammars because FieldWorks never writes it, and conformance/docs/engine-only-fixture-retirement.md says LoadInflAffixProcessRule never sets it. Seven more producible grammars have the same issue on their slot rules (mrS1a, rPl, rCls, mrFV1, mrTA1, ...). Please drop the attribute there and rerun parity, or mark those fixtures not producible.

[claude]

1. Verdict: approve with fixes. F1 through F5 are fixed, and all 16 stress fixtures pass at the head.
2. Most important: F7, conformance-stress/edge-cases/required-template-slot/grammar.xml:40. Eight grammars marked fieldworks_producible: true still put outputPartOfSpeech on slot rules, which FieldWorks never writes. 1992485 removed that attribute from the other four slot-template grammars for this reason.
3. Counts: Critical 0, Important 1, Low 0.
4. Ran dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests --filter TestCategory=Stress on the merge checkout: 16 passed, 0 failed, in 22 s. Ran git grep over conformance-stress/ for the provenance names F4 cited: no matches. Checked <Slot morphologicalRules> members against outputPartOfSpeech in every stress grammar. The compare against the last reviewed commit reports diverged, so I reviewed the whole PR again. Per gh pr checks 578: Conformance suite and Comment hygiene pass; both Builds and claude-review are pending.
5. Not verified: whether dropping outputPartOfSpeech changes any parse in those eight grammars; the F5 deletion experiment, which I did not rerun; and Build results, which are still pending.

Public API, target framework, package, machine.py parity: None verified. The PR adds one [Explicit] test class and data under conformance-stress/.

Findings: F1 addressed, F2 addressed, F3 addressed, F4 addressed, F5 addressed, F6 addressed, F7 new.

Reviewed at c343892