Skip to content

decider: reuse request prefixes with conservative auto selection - #113

Merged
hsliuustc0106 merged 49 commits into
InferMatrixLab:mainfrom
Levius-Fubuki:codex/decider-shared-prefix
Oct 24, 2026
Merged

hsliuustc0106 merged 49 commits into
InferMatrixLab:mainfrom
Levius-Fubuki:codex/decider-shared-prefix

Conversation

@Levius-Fubuki

@Levius-Fubuki Levius-Fubuki commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Conflict resolution against main — 2026-10-24

Head f285b8081300906cb62fc1e95a21ce46e57ec08b normally merges main 67b867b04b23acfe735c381559f7b2510f1e784e, retaining the original branch history. The only manual conflict resolution is the Qwen test manifest: Graph, prefix, constructor/stream-ownership, image-decoding and device-feature targets are all retained without duplicate registrations. ABI remains 9.

Current-head local macOS checks passed: formatting; strict workspace/all-target Clippy; workspace tests (310 passed, 0 failed, 65 ignored); workspace release build; 22 Decider Python tests; CUDA text contract (0 errors, existing Laya ABI warning); strict MkDocs; and whitespace/merge checks. Device/checkpoint cases were skipped; no new CUDA compilation, GPU parity or performance qualification was performed. Historical device evidence remains pinned to its original source. Hosted checks for this new head are reported by GitHub separately. The original PR remains open.


Current-main integration and A800 validation — 2026-10-23

Current head b3cd06a27f0995cd7719c8148b8b479e11e44a53, tree 608ccac86a6f61377d22940fc252934a3e4d20d9, integrated with main 5ba6a3ba668cb33028eb0bce2c6d17cdc53aebe7 through normal merges. Original history is retained. Includes refreshed #112 and canonical #99 profile/ownership changes. Current main owns all20 CUDA build inputs; ABI remains9.

Source review found a stale constructor-test error substring after #99's guard wording change; both assertions were repaired with the production guard retained. The documented Graph selector and duplicate Decider navigation were also fixed. Initial failure and focused/full repaired results are preserved.

Current-source local checks: 269 Rust workspace tests passed,0 failed,57 optional ignored; all format/strict Clippy/release/Python/contracts/smoke/strict-docs/whitespace stages passed. Four Python suites total163 tests on this head. Across both heads all26 configured local stages passed; optional workspace skips are not device passes.

**Fresh A80080GB/sm80 qualification:**13 selected GPU Rust executions plus2 GPU-hidden contract executions on this head. Default and cuDNN libraries were freshly compiled as ABI9, capability/dependency receipts checked, and consumers rebuilt separately. Fresh pinned reference20 requests/56 rows, all six modes120 parity records on this head, original protocol2 numerical/schema gates unchanged. Maximum probability/per-level-fit drift0.0138, Score drift0.04. Shared/fixed logits+responses and long-auto/fixed are exact with real sharing; matched packed4/Graph4 responses are exact with positive captures/replays and zero unexpected fallbacks. No equality across differently packed controls or performance claim.

Six-mode HTTP162 logical checks,12 concurrent requests per mode with observed client overlap/direct-proxy body parity; six projection-failure retirement modes with503/unavailable health/two later refusals and sampled model memory release;18 capture-fallback requests all passed on this head. The combined serial campaign has60 successful stages; every process stage ends at0MiB device use.

Independent agent source review inspected both complete effective diffs and all repairs, with no unresolved actionable source finding. The separate offline evidence review recomputed retained raw results and checked full source identity. This is author-side agent review, not maintainer approval or a new human attestation. Existing contributor-intent declarations and self-review history below are preserved verbatim; earlier sections remain historical evidence for their own source pins.

CI success · Docs success at this exact current head.

Full report, protocol, raw source/CPU/GPU/HTTP evidence, independent reviews and portable audit. All21 public assets and their checksums were downloaded and verified; downloaded audit passed. The GPU evidence was downloaded, hashed and audited before platform shutdown; fresh SSH became unavailable after shutdown. Both original PRs remain open and unmerged; main was not changed. Scope is bounded correctness on one A800, not all optional tests, broad task quality, other models/hardware, throughput, latency or peak-memory profiling.


Compatibility with PR149 verified — 2026-10-22

This PR remains 5e340cab24b3e55704bfc674a725b5ed82238710. Compatibility is implemented in the original #149 branch at 428f834fad970f7e943180fb749871e04cbdb3e0, which retains this113 head as a parent. No source change here and no GitHub PR merge. Graph/fixed/shared/auto ABI9 modes and runtime admission/deadline controls are preserved together.

The exact combined head passed local305 tests (57 optional ignored), all standard build/docs checks and80 Python tests. A800 qualification passed20 reference/120 mode comparisons, six-mode HTTP,42 lifecycle scenarios, six failure retirements and18 capture-fallback cases; corrected server CPU/doc71 +startup4 and15 targeted device/checkpoint executions also passed. CI · Docs are green.

New combined-head evidence, scope, raw records and portable independent audit. All five downloaded assets and11,886 public files verified; numerical gates unchanged, no performance claim. This targeted combined-head campaign does not relabel the earlier57-case qualification below as rerun, and does not extend old approvals to the new integration. Existing human declarations remain preserved.


ABI9 A800 qualification completed — 2026-10-22

Qualified source remains 5e340cab24b3e55704bfc674a725b5ed82238710; no source change was needed. One A80080GB/sm80, driver595.71.05, CUDA13.0.88. All14 follow-up cases and scope · complete additive evidence and portable audit.

  • Both default/cuDNN ABI9 builds passed the fixed-GEMM device gate. Twenty fresh reference requests/56 rows passed all six eager1/packed4/Graph4/fixed4/shared4/auto4 controls. Maximum probability/per-level-fit drift0.0138, Score drift0.04, no Choice disagreements under unchanged protocol2 gates. Shared/fixed raw logits and full responses were exact; long auto also matched fixed and actually shared. Graph replay was observed with no fallback.
  • All162 HTTP checks across six modes passed;72 concurrent response bodies and client overlap were retained. All six real projection-failure retirement modes, including auto-selected shared, passed with503/unavailable health/two later refusals; capture fallback passed18 requests. Inherited vision capture/copy drain controls also passed.
  • The frozen optional Rust inventory is now 57/57 unique cases (43 GPU +14 CPU), 64 pass executions. The remaining14 have all been run: CLM5, Laya3, OmniJev2, Jev-VL1 CPU cases plus Open-Jev tokenization, JEMM BF16 label-head GPU and Open-Jev packed/Graph GPU. No execution blocker remains. The complete pinned Open-Jev27B export exercises actual packing; its extra trace replication observed10 captures/4 replays/0 fallback. This follow-up adds15 executions:14 new cases and one diagnostic replication.
  • FP32 GDN14 shapes and JEMM11 reference operators/replay passed in the initial campaign. Operator checks retain their separately identified matching-cuDNN9.20 variant and original scope. CPU-only head/tokenizer/prefix subsets do not claim a full-model export; detailed scope and genuine pinned-input provenance are in the linked pack.

The complete additive archive is 568,456 bytes, SHA256 c48bba3b167f2791bb93f72b70a3c265300708cb6d5688502ccfe07b738548f1. It includes323 public files, the prior225-file pack verbatim and all210 earlier sealed raw records. All five downloaded assets were byte/checksum verified; the downloaded portable audit and independent review passed, including636 source hashes. Source remains unchanged and GPU memory returned to0 MiB. Preparation and auditor errors/corrections are retained. Large external model/tokenizer/package inputs are omitted with provenance, not measurement logs.

The initial ABI9 campaign and its immutable archive preserve the earlier43/57 snapshot. There are zero benchmark samples and no performance, broad model-quality or other-hardware claim. Offline checks review recorded author execution; they do not independently authenticate GPU execution or rerun device math. Existing human declarations below are preserved without adding a new human attestation or maintainer approval.


Current-main fixed/shared integration — 2026-10-22

Current head: 5e340cab24b3e55704bfc674a725b5ed82238710. Normal merge 1f25992f preserves original PR #113 head a631b8b9 as first parent and maintained #112 3de420cc as second parent, including main 32bdd2aade0e0f3f0452d04ca5022fcaaa4783c9. The original #98/#99 commits remain ancestors. No other contributor PR was changed or closed.

The integration reconciles fixed/shared GEMM with main's reference GEMM, unmerged LoRA, FP32 GDN and multimodal Graph paths. The combined Qwen CUDA contract is ABI 9: rebuild the matching library and all Qwen consumers together. Header, manifest, loader, fake libraries and current deployment recipes agree. Fixed/shared profiles reject unsupported reference/FP32/adapter combinations before resource allocation. Model-owned prefix and fixed handles preserve synchronization and lifetime rules, together with #112's explicit Graph controls and single final-field stream owner.

A full workspace run exposed a pre-existing Cua test fixture collision when parallel tests received the same clock tick. A separate focused test-only commit adds a process-local sequence to temporary directory names and an actual eight-thread, identical-timestamp filesystem regression. The original failure and subsequent passing results are retained; production Cua behavior is unchanged.

Executed validation

cargo fmt --all --check
cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings
cargo test --workspace --locked --features omni-laya/serve
cargo build --workspace --release --locked --features omni-laya/serve
python -m unittest discover -s tests/benchmarks -p 'test_*.py' -v
python -m unittest discover -s tests/decider -p 'test_*.py' -v
python src/backends/cuda/check_contract.py --repo-root .
python -m mkdocs build --strict
git diff --check

Final macOS arm64 source: 268 workspace tests passed / 0 failed / 57 ignored. Formatting, strict Clippy, release build, strict docs and whitespace checks passed. Python benchmark/Decider/CUDA-checker/backend suites passed 58/22/68/8 cases, and smoke validated four requests. Targeted Qwen host tests passed 24 cases with 29 device cases ignored. Source contract has zero errors and one existing Laya declaration warning. A fresh CPU-only Docker CUDA 13 default build for sm89 passed; symbol/dependency inspection and the CPU ABI probe confirmed ABI 9. This compiles CUDA source but executes no GPU kernels.

Independent full-diff review and a follow-up review of the fixture repair found no unresolved actionable findings. Final effective diff against main: 53 files, +3025/-242 (3267 total changed lines; 2303 source/test lines). Reviewed diff SHA256: 1e773225c702f530c5544016e259ec74427fd2187f1a525dabe77b1947bcc8ea. Final commit tree exactly matches the validated tree.

Updated device qualification — 2026-10-22: the fresh A800/sm80 ABI9 campaign above now validates the bounded fixed/shared/auto/Graph, HTTP and lifecycle matrix with default and matching cuDNN builds. The initial43/57 snapshot has now been completed by the follow-up above:57/57 unique optional Rust cases (43GPU +14CPU), with the complete pinned Open-Jev27B packed/Graph regression included. Other GPUs/toolchains, broader full-model and Hopper qualification, and broad quality/performance remain outside this campaign. Historical standalone ABI7 and main/combined ABI8 results below keep their original source scope. Rebuild the matching ABI9 library and all Qwen consumers together. Current-head hosted CI and Docs passed; PR documentation deployment is intentionally skipped.

Existing contributor declarations below are preserved; this update reports agent execution and review without adding human attestation or maintainer approval. Historical base/ABI/hold statements below retain their named revisions and are superseded by this heading for the current integration state.


CPU maintenance and merge handoff — 2026-10-15

Current head: a631b8b95cb4a2853d821f6dde576717d479e461.
Actual target: main at f2d3d32897c1fe190fecf853a82a9456d3791462; the PR target was not changed.

This branch contains the original shared changes d717e497 (#98) and 2ff1396 (#99), not the latest #98/#99 tips. Preferred handoff: coordinate #98 → #99, then refresh this branch while preserving its Decider Graph/cache/lifetime changes and recheck the resulting diff; #112 can land independently first. A maintainer may instead choose this cumulative branch first and coordinate duplicate PR closure with their author. No other contributor's PR was modified or closed.

The standalone branch remains ABI 7; main/#112 remain ABI 6, and the separate shared-vision stack uses ABI 8. Run fixed_gemm_rows_do_not_depend_on_m on each deployment GPU. The documented HTTP503/unavailable-health alert and rebuild/restart/warmup recovery also apply if auto mode selects a shared plan that subsequently fails. These are documented requirements, not new device passes.

No source was changed in this pass. Inspected the current source/dependency history and completed existing self-review record; previously observed current-head hosted checks were successful. Prior personally-run Rust/device records below keep their original source and scope; no Rust suite or GPU execution was repeated in this body-only update.

This pass follows the contributor's 2026-10-15 instruction to maintain these existing PRs, complete CPU-only fixes and merge preparation, and keep #117 unmerged. Existing contributor declarations are preserved; this update adds no new human attestation or maintainer approval. No PR was merged and no GPU/model/performance run was performed. Source-pinned historical results remain historical. Current-head hosted workflow links are recorded separately below when available.

Historical records below

Earlier maintenance/review/test sections retain their named revisions and environments. Their old ABI or build commands are historical reproduction instructions, not commands for deploying this current head. Use the current branch recipes for builds; the CPU checks above do not qualify skipped device tests.

Historical maintenance — 2026-10-15

Current head: a631b8b95cb4a2853d821f6dde576717d479e461.

Synced onto maintained #112 at 03bbb61, including main after #111's merge. Added the per-device fixed_gemm_rows_do_not_depend_on_m command and its exact one-test acceptance rule, the actual HTTP503/unavailable-health alert condition, same-ABI rebuild guidance and restart/warmup recovery procedure. Source inspection confirmed automatic mode also shares the failure/retirement contract when it selects a shared plan; the documentation now states that rather than treating auto as an execution-error fallback. Production retirement behavior and all original numerical gates are unchanged. #98/#99 remain separate open dependencies: their merge ordering is recorded for maintainer coordination, and no other contributor's PR was modified or merged.

Personally run: format, strict workspace Clippy, workspace tests (206 passed, 0 failed, 45 ignored), release build, benchmark tests, CUDA checker/backend tests, source contract check, smoke validation and strict MkDocs all passed. Device-dependent tests were compiled and skipped on this macOS host. No new GPU, numerical-parity or performance pass is claimed.

Current-head hosted workflows: ci.yml, docs.yml. Linked hosted results are separate from personally executed local checks. Earlier evidence sections retain their named original revisions and scope; they are not new qualification claims.

No PR merge was performed. #117 remains on its explicit merge hold.

Purpose

Add default-off request-local shared-prefix execution and its independent fixed-GEMM control. Reuse aligned request/question prefixes while preserving branch state, positions, final-token readout and original response order. Keep main’s separate text/multimodal Graph lifetimes, both external GPU regression modules, and the ABI 7 fixed-GEMM interface. Prefix buffers persist, while their contents are valid only within one request.

#94/#110/#111 prerequisites are now merged into main. Review the remaining #112 → #113 increments in order; #113 also carries the #98/#99 shared-executor dependencies, whose merge order remains a maintainer coordination item.

Required human contributor answers

These answers reuse the contributor's explicit 2026-10-12 maintenance instruction:
keep the existing PRs mergeable and CI passing; eliminate conflicts and review
issues; serve maintainers who need clear, verified changes that are easy to merge;
repair, validate and push to the existing PRs. The source is the contributor's
local maintenance conversation on that date; private conversation details are
not published.

  1. What outcome are you trying to achieve?

    Keep the existing PRs mergeable and CI passing.

  2. What is the highest-leverage variable for that outcome?

    Eliminate merge conflicts and resolve review issues.

  3. How will this work move that variable rather than create busywork?

    Repair the identified conflicts and review issues, validate the affected
    integration, and push the verified changes to the existing PR branches.

  4. Who is the customer or user, and what do they actually need?

    Repository maintainers, who need clear, verified changes that are easy to merge.

After answering the four questions: is a PR needed at all to achieve this outcome?

The contributor explicitly directed updates to the existing PRs. Maintain these
PRs; no new PR is needed for the maintenance work.

Historical Current integration

Head: e790c3cfc4bdc1106fe1aa9cfe118d045ab63c41 (documentation-only follow-up to runtime source 4c7d437023c47c777afc7b82e6cb95a399468785). Target: main at 3e9cc33a61ddf110d97512fb6f296cf540377c61, refreshed during this maintenance. Main and published prior heads remain ancestors through normal merges.

Target diff: 52 files, 2566 authored changed lines / 3436 total text changed lines (additions plus deletions). Counts exclude Markdown, licenses/notices, lockfiles, static fixtures and the example request from authored lines; the ABI manifest is included.

The predecessor merge conflict was reproduced before repair. The resolution retains persistent shared-prefix state and fixed-GEMM handle destruction together with both Graph caches and both root test modules. The current-main worker contracts and GPU tests remain present. Workspace recipe commands include omni-laya/serve.

Current main and the independent #111/#112 consumers require ABI 6. This complete #113 branch requires ABI 7, including cs1_gemm_create_fixed. Rebuild the library and every consumer together. The #113 recipe’s ABI 7 command applies to that branch; it does not imply ABI 7 is already in main. Combining this chain with #135’s ABI 8 changes is a separate cumulative integration and validation task; the standalone evidence here does not qualify it.

Runtime-source CPU checks and documentation follow-up

The following CPU-only macOS checks passed on runtime-source predecessor 4c7d437023c47c777afc7b82e6cb95a399468785 (Rust 1.98.0):

cargo fmt --all --check
cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings
cargo test --workspace --locked --features omni-laya/serve
cargo build --workspace --release --locked --features omni-laya/serve
python -m unittest discover -s tests/decider -p 'test_*.py' -v

Workspace: 201 passed, 0 failed, 45 ignored; Decider Python helpers: 22 passed. The workspace ignored device/checkpoint cases; separate evidence below states which were later executed. Release builds for #112/#113 were rebuilt after clearing only affected package artifacts to ensure the exact worktree code was compiled. Python 3.12 with pinned documentation/benchmark requirements was used for broader host checks. Strict documentation build passed for #111 and #113; the shared benchmark suite passed 23 tests and validated the four-request smoke manifest. #113 also passed 68 CUDA checker tests, 8 backend helper tests and the ABI manifest check (0 errors; the existing Laya declaration warning remains).

The current upstream PR checks for published runtime-source predecessor 4c7d437023c47c777afc7b82e6cb95a399468785 completed successfully:

PR documentation deployment is intentionally skipped. The CUDA-contract CI job runs host checks and does not compile CUDA or execute GPU tests. These are upstream checks for the published runtime-source predecessor; earlier CI URLs in the prior description are historical. The new documentation-only head has passed local strict MkDocs and git diff --check; its hosted checks passed at the new head, linked below.

Demo / evidence and limits

Fresh device qualification — 2026-10-13

The designated GPU runner executed qualified runtime source 4c7d437023c47c777afc7b82e6cb95a399468785 with the freshly compiled ABI 7 library on one NVIDIA GeForce RTX 4090, sm_89. Toolchain: CUDA 13.0.88, Rust 1.98.0, Python 3.12.3, Torch 2.12.1+cu130, Transformers 5.17.0. Checkpoint/tokenizer/calibration: Mapika/decider-2b 533964dae8be954c5b5e19fa4948e48408094c1e; author reference: 50d0be0d7cb43d2066965ce5fa7f3fe4e489a60f. All five model files and five imported reference files were hash-checked. Optional FLA, causal-conv1d and flash-attention packages were absent; the reference used the documented Transformers fallbacks. The existing Python environment was not changed.

The frozen synthetic corpus has 5 requests / 20 independent rows / 18,394 processed tokens, with a longest reference-compared row of 3,541 tokens: short mixed Choice/Score/Noul, long shared mixed, reordered short questions, short input after long shared work, and empty questions. The original gates were fixed before execution: probability/per-level-fit drift ≤0.02, Score drift ≤0.1, matching Choice decision at reference margin ≥0.05, exact token/candidate/readout/usage/identity comparisons, and validation protocol version 2 finite typed-schema/formula/rounding checks.

  • Native/reference: 5/5 requests passed; maximum probability/per-level-fit drift 0.0119, maximum Score drift 0.01, no Choice disagreements in this corpus.
  • Eager, packed4, Graph-packed4, fixed control and shared-prefix-packed4: all passed the same gates; maximum probability drift across modes 0.012. Fixed/shared raw logits and complete responses were equal for every stored parity/warmup/repeated record. Actual Graph capture/replay and actual prefix token savings were required and observed.
  • Real head-failure retirement: eager, packed, Graph-packed and prefix-packed passed. Each genuine worker warmed up, returned 503 on its second projection, became unavailable, refused two later requests and released more than 3,000 MiB of sampled model residency. These are before/after lifecycle samples, not peak-memory measurements.
  • Capture-failure fallback: 5/5 requests passed, with one capture failure, permanent eager fallback and unchanged decision fidelity.
  • Selected GPU regressions: 7 passed—two BF16 head microtests, complete eager/packed/shared request checks, ordered Graph shape/growth/cache checks, and fixed-GEMM row independence.

Native diagnostic SHA-256: 936d9cd20ec5881d8d89ea7d1ce7db6d15d2f5ada885f07371a6bb5bab56f2ea; ABI 7 library: 2e6d15a1726314440e8547986fb7cd56a45fdc5931dcd56694d78883a20dff28; frozen reference records: 965e57ba64275f6db5ec0d6597da73111309e812070149a79b14dd434b67ac36. A 31-path pre-GPU inventory, complete raw records, protocols, logs and failure wrappers are retained in the maintenance evidence archive. Both local and server CPU reviews recomputed the raw responses with the unchanged validator. Fresh raw evidence is retained in the local maintenance archive; no new public asset has been published.

Reproduction uses recipe/decider/download_weights.py, src/backends/cuda/qwen3_5/build.sh ... 89, the pinned author reference, tests/decider/verify_reference.py --cases <frozen-cases.json>, verify_modes.py --plan <frozen-plan.json>, verify_retirement.py and verify_graph_fallback.py. The frozen case/plan files and hashes are included in the retained archive. Every native consumer must be rebuilt with the matching library.

Only two diagnostic roundtrip timing samples per mode were recorded; they are descriptive and do not establish a speedup. No frontend latency, throughput, task-quality, other-GPU, auto-prefix, 36,864-token limit or cumulative ABI 8 claim is made by this campaign. The five-request reference corpus and selected tests delimit this qualification.

Historical evidence

The original Decider followups archive and recipe/decider/validation.md retain the earlier 2026-10-07 campaigns and their own implementation/ABI provenance. Those results are historical; the fresh results above are tied to the repaired head and current environment. Historical Torch 2.14.0 measurements are not presented as measurements of the current Torch 2.12.1 campaign.

Review map and self-review

Trace prepared complete rows through batching and response reconstruction, then review Graph/scratch/prefix lifetimes, synchronization, error retirement and Rust/CUDA ABI declarations. Processing and adapters stay model-owned, shared device operations stay under Qwen/CUDA, admission stays per complete request, and tests remain under root tests/. Generated measurement output and agent plans are excluded from source changes.

The documentation follow-up changes only recipe/decider/README.md and validation.md to date the fresh qualification, preserve historical provenance and distinguish standalone ABI 7 from main ABI 6 and cumulative ABI 8. src and tests tree hashes match the qualified runtime source exactly (b97905de9812612fafe32b9fcc1b9bd316cf1364 and ecdc18b8e31e6e9628bc54075b3fa03d8fd354ee); manifests, CI, navigation, dependencies and download helper also remain byte-identical. The fresh device evidence therefore applies to this unchanged runtime, while the new source head is explicitly a documentation-only qualification note.

The complete current-target diff and predecessor increments received local self-review and independent review; no semantic findings remained in that review. Device evidence has the scope stated above and does not replace contributor or maintainer review. The contributor explicitly confirmed completion of self-review and authorized this checklist/template update on 2026-10-13 in the maintenance conversation. This records the human confirmation; it does not turn unverified measurements into passes or replace maintainer review.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on the stated purpose.
  • I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
  • I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Documentation-head hosted verification

Exact documentation head e790c3cfc4bdc1106fe1aa9cfe118d045ab63c41: CI and Docs passed. Source/test/build-input identity to the qualified runtime head is recorded above.

Cumulative ABI 8 qualification — 2026-10-13

Combined integration branch, final head 18e1a5a6f68b9d11d3e11c7ea7b939af87272f53, integrates the twelve source-qualified maintenance heads frozen for the earlier 2026-10-13 campaign through normal merges. The later #117 ancestry-only follow-up leaves its complete source tree unchanged. This is separate review material, with no new PR and no main merge. It includes a substantive cross-feature fix: model-owned transient vision output survives its borrow, then retires only after both producer and consumer streams drain. A production-entry injected-copy failure confirms the matching output allocation is released before error return and subsequent requests recover.

262 Rust CPU tests (59 ignored), 163 Python tests, strict Clippy/format, release, docs and ABI contracts pass. Fresh cuDNN ABI8 GPU qualification passes 17 Rust device invocations, all eleven fixed-seed JEMM reference operators with exact Graph replay, and five Decider controls (five requests/20 rows each); maximum probability drift 0.012, Score drift 0.01. Fixed/shared logits and complete responses are exact; actual Graph replay and shared-prefix savings are observed. Runtime validation source is b909398d; the final head changes only active recipe ABI wording, with source/test/build identity verified. No new performance or full 27B/Hopper claim is added.

Exact-head cumulative CI and Docs passed. Raw protocols, responses, failures and hashes are retained in the local maintenance evidence archive.

Levius-Fubuki and others added 8 commits October 7, 2026 21:11
Add CUDA operations that continue a sequence after a shared prefix,
for request-local prefix reuse (InferMatrixLab#85): the Gated DeltaNet conv with the
inputs of the three positions before it, the chunked prefill from and
to a float32 state, gated attention for queries after cached keys, and
a pitched device-to-device row copy. The existing conv, prefill and
gated attention are these operations without history, state or cached
positions. Bump the ABI to 6 and add the Rust bindings and GPU tests
against the unsplit calls.
cuBLASLt's heuristic picks an algorithm per M, so a row's result can
change with the number of rows in the call, and a prefix and its branch
run separately round differently from one pass over the same tokens.
Add cs1_gemm_create_fixed: one algorithm per weight shape (N, K, ldy),
the heuristic's first choice at a reference M among algorithms without
split-K, for every M. The default handle is unchanged. Bump the ABI to 7
and add a GPU test that each projection's rows match across M.
@twu3202

twu3202 commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Rebased #97 to #99 onto main (after #102 and #92). #99 now splits the layer loop the way your merge does, packed sequences plus single-sequence spans, so #113 can build on its head directly.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Levius-Fubuki

Copy link
Copy Markdown
Collaborator Author

Checked the rebased #99: the packed/span loop is compatible, and the CUDA kernels/FFI match #113. I’ll sync #113 onto the new head while preserving its Graph stats, explicit mode and lifetime handling.

@Levius-Fubuki Levius-Fubuki changed the title feat(decider): request-local shared prefix reuse decider: reuse request prefixes with conservative auto selection Oct 8, 2026
@Levius-Fubuki

Copy link
Copy Markdown
Collaborator Author

@hsliuustc0106 Follow-up to your latest review.

The contributor has explicitly confirmed completion of self-review, and all four self-review declarations are now checked in the PR description. The four human contributor answers and PR-necessity decision remain recorded.

This follow-up updates the PR description only; the source head remains e790c3cfc4bdc1106fe1aa9cfe118d045ab63c41. The reported validation results and unverified checks remain documented with their original scope.

Please re-check the updated description and reassess the checklist-only readiness item, then re-review this PR.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head e790c3cfc4: one new docs commit since my last check — "Document current Decider ABI 7 device qualification" — and, decisively, the four-item self-review checklist is now checked (4/4), completing the post-#138 template gate that was the sole blocker (human answers and PR-necessity were already recorded).

The new commit is reviewed and it improves honesty rather than weakening it: it records the 2026-10-13 ABI-7 device qualification (RTX 4090, five frozen requests, 20 rows, eager/packed/Graph/fixed fidelity controls, retirement and capture-fallback checks) with explicit limits stated — "two timing samples per mode are only descriptive", "neither campaign establishes task quality, every supported GPU or universal bit-identical agreement with Transformers", and rebuild-together requirements per consumer generation. Fork CI at this exact head: rust/cuda-contract/benchmarks/docs all pass.

No actionable findings remain. APPROVED. Unverified checks: the 2026-10-13 qualification itself is author-run (no GPU on this host); the historical 2026-10-07 campaign remains archive-based; Rust suite covered by the exact-head CI run; broader HTTP/performance coverage stays outside the campaign as the docs state. Merge in stack order (#94 → #110 → #111 → #112 → #113).

@twu3202

twu3202 commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

@hsliuustc0106 one note on merge order: this branch also carries #98 (d717e49, the fixed-algorithm GEMM handle) and #99 (2ff1396, the shared-prefix executor), which I opened for #85. If it works for you, merging those two first would leave #113's own diff as just the Decider side. Both are green at d8cd04f and 18cbd99. If you'd rather merge #113 as it is, I'll close #98 and #99 once it's in.

@MrDongsls

Copy link
Copy Markdown
Collaborator

Findings

P3 — Forced modes fail hard with a request 503; prep ops before enabling them in production
recipe/decider/validation.md:289 @ e790c3cf

  • Under forced modes (DECIDER_PREFIX=1/DECIDER_FIXED=1) a length the selected fixed algorithm cannot serve fails hard and surfaces as a request 503 — declared conservative design, documented at validation.md:289 and summarized in recipe/decider/README.md:101.
    Fix: prep an alert for "fixed algorithm cannot serve this length" plus the rebuild+restart plan before enabling forced modes in production; auto mode is unaffected.

P3 — M-independence of the fixed algorithm rests on one GPU check
src/backends/cuda/qwen3_5/README.md:23 @ e790c3cf

  • M-independence is qualified on one author-run RTX 4090 check.
    Fix: run fixed_gemm_rows_do_not_depend_on_m on each new GPU before relying on fixed/shared modes, as the README recommends.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head a631b8b95c after the rebase wave (prior review: APPROVED at e790c3cfc). Wave-verification note (this pass): the rebase wave was verified systematically — every own-commit series is patch-identical per range-diff; every post-pin merge resolution was diffed against both parents with zero dropped lines; approved-content markers verified present in every tree; the key new merge 3be46fa (shared-vision line × jemm v2 tip) carries #143\u2019s new test files intact and is ABI 8/8 consistent at every descendant head.

The three merge resolutions were verified: the approved decider docs (recipe/decider/README.md + validation.md) survive intact with pure additions from the #111-line ancestry; the laya test files are unioned, not dropped; ABI 7 remains consistent (ops.h = manifest) at this head; the prefix-workloads consumer and tests/decider/data/README.md are present. Upstream CI (full suite incl. rust/build/benchmarks) green at this exact head.

APPROVED (verdict carries). Unverified: the 2026-10-13 device qualification remains author-run (no GPU on this host); Rust suite covered by exact-head CI.

Preserve graph-loading ownership and diagnostics, combine fixed and reference GEMM profiles under ABI 9, and retain model-specific numerical paths and deployment recovery guidance.
Add a per-process sequence to the PID and clock nonce and cover simultaneous fixture creation with a shared timestamp. The full workspace run observed AlreadyExists with the original helper.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head 5e340cab24 after the rebase/merge wave onto main@32bdd2a (prior review: APPROVED at a631b8b95c). Own-commit series patch-identical (git range-diff, named refs, cat-file verified). New since that approval: one test-only commit 5e340cab "test(cua-s1): prevent concurrent export fixture directory collisions" — reviewed: prevents parallel test runs from colliding on the shared export fixture directory; no production code touched. The merge resolutions carry the absorbed jemm/decider lineage, consistent with the pass-57 wave verification. Upstream CI at this exact head: full suite incl. rust/build/benchmarks green.

APPROVED (verdict carries). Unverified: the 2026-10-13 device qualification remains author-run (no GPU on this host); Rust suite covered by exact-head CI.

Copy link
Copy Markdown
Collaborator Author

Fresh ABI 9 A800 GPU qualification — 2026-10-22

Ran the integrated source 5e340cab24b3e55704bfc674a725b5ed82238710 on one NVIDIA A800 80GB/sm80, driver 595.71.05, CUDA 13.0.88. Source, five pinned Decider model files, six consuming binaries and three fresh CUDA-library variants were rehashed unchanged after execution. No PR source change was needed.

Evidence release and all five assets · verification guide

Archive: 433,822 bytes, SHA256 cdafe3ed43a62e48a2dae323a85ea1070dd027ed5b03e772060923c33b8f9584. All 210 sealed raw records plus their original manifest are retained byte-identically, including setup failures, original responses and logs. The public package also includes source-pinned audit inputs, component notices and a portable auditor. All five published assets were downloaded again, byte/checksum verified, and the downloaded package's offline audit passed with all 636 source files checked.

  • Both default/cuDNN ABI9 builds passed the required fixed-GEMM gate, exactly one executed pass each. The main cuDNN9.24 library SHA256 is 899e851433303a251e4e851ac05cedd77a3fe754698ce9292234de4edc9547fe.
  • 20 fresh reference requests / 56 independent rows, six modes: eager1, packed4, Graph4, fixed4, shared4, auto4; all 120 per-mode comparisons passed the unchanged protocol2 gates. Maximum probability/per-level-fit drift 0.0138, Score drift 0.04, zero qualified or lower-margin Choice disagreements. Shared/fixed raw logits and complete responses were exact; long auto was also exact to fixed and recorded real sharing. Actual Graph replay had zero fallback.
  • 162 HTTP checks across six modes passed, with exact worker/frontend bodies, all 72 concurrent response bodies and overlapping client intervals retained. Six real projection-failure retirement paths passed, including auto-selected shared: 503, unavailable health, two later refusals, and helper-recorded process-memory drops above3000MiB. Capture-start fallback passed all18 default requests with one fallback and zero captures/replays. Vision capture failure and queued-copy drain/recovery also passed.
  • 49 Rust pass-executions cover43/57 unique optional cases (41 GPU + 2 CPU), including all16 kernels, five Decider GPU cases (36,864-token boundary and auto isolation included), Graph/current-input/cache growth, prefix isolation and Cua compatibility controls. FP32 GDN's14 shapes passed; JEMM's11 reference operators and replay were exact on the separately rebuilt matching-cuDNN9.20 library (58bf7de0320d88107d2445d0fd165b84e6c3590b2a8bdc1e75f1eb210e281035).

Extract and run python3 public_audit.py with Python3.9+ standard library; no GPU/packages/network needed. The independent final audit found no unresolved actionable evidence finding. It recomputes recorded response/wire/hash checks and inspects operator/test records; it does not independently authenticate GPU execution or recreate tokenization/GPU tensors. Process-memory pairs are recorded helper samples, not peak-memory or raw allocator traces.

Scope: this A800/ABI9 campaign supersedes the previous device-validation handoff for the bounded matrix above. The remaining14 optional cases are explicitly listed, and full other-model/27B/Hopper/other-GPU qualification remains unverified. There were zero selected measured timing samples and no speedup, throughput, latency, peak-memory or broad model-quality claim. Original source and numerical gates were unchanged; launcher/cache/auditor setup failures and their corrections are retained. All campaign workers exited and final recorded GPU use was0MiB. This is execution evidence, not a new human attestation or maintainer approval.

Copy link
Copy Markdown
Collaborator Author

ABI9 A800 follow-up: all remaining14 optional cases completed — 2026-10-22

Exact source is unchanged: 5e340cab24b3e55704bfc674a725b5ed82238710. On the same A80080GB/sm80, all14 previously unverified optional cases now pass: 12 CPU +2 GPU, with no remaining execution blocker. No source, fixture, numerical gate or tolerance was changed.

Family Exact case Result
clm every_tensor_conversion_matches_the_oracle PASS
clm decisions_match_the_reference_implementation PASS
clm to_text_matches_the_reference_byte_for_byte PASS
clm state_text_and_candidates_match_the_reference_byte_for_byte PASS
clm raw_requests_match_the_reference_byte_for_byte PASS
jemm bf16_label_head_gemm_matches_exact_constant_rows PASS
jev_vl manifest_suffix_split_matches_full_expand PASS
laya official_packing_parity PASS
laya every_weight_conversion_matches_torch PASS
laya checkpoint_inventory_covers_legacy_temperature PASS
omnijev heads_match_reference PASS
omnijev tokenization_matches_reference PASS
open_jev packed_candidates_preserve_isolation_order_and_graph_shapes PASS
open_jev tokenization_matches_reference_and_rejects_oversize_prompts PASS

CLM used genuine pinned weights and the documented16-tensor export oracle; fresh text-oracle regeneration was byte-identical. Laya's three tests are CPU-only: pinned206-tensor conversion, official17-case packing and legacy-temperature inventory all pass. OmniJev used genuine heads/tokenizer against committed references. Jev-VL checked all12 frozen R1 image entries with the original restored manifest; this is a tokenizer/prefix check, not a full-VL-model export.

Open-Jev used the complete pinned27B export, because the supported9B contract disables packing. All18 original base shards and six trained checkpoint files were verified. The maintained exporter produced17 files/51,269,477,178 bytes; before/after hashes were unchanged across the main GPU test. The regression passed candidate isolation/order/state/shape checks; a separately planned trace replication additionally observed 10 actual captures +4 replays, zero fallback. Both GPU tests used ABI9 library SHA256 899e851433303a251e4e851ac05cedd77a3fe754698ce9292234de4edc9547fe. The compatible isolated exporter environment and its initial tokenizers-version rejection are retained; global packages were unchanged.

Cumulative frozen optional inventory at this head: 57/57 unique cases (43 GPU +14 CPU), 64 Rust pass executions. The follow-up contributes15 executions:14 new cases plus one Graph trace replication. This is functional regression coverage, not blanket full-model quality or other-hardware qualification. Zero benchmark samples; no performance/speedup claim. All636 recorded source files remain unchanged; GPU memory returned to0 MiB.

Complete additive evidence, exact14-case catalog, raw commands/logs/oracles/pins, portable audit and independent review. The prior release and its210 raw records remain immutable and are included verbatim under previous/. The new pack has323 public files (322 hashed entries +manifest). All five downloaded release assets were verified, and the downloaded pack passed the offline audit including636 source hashes. Large external model/tokenizer/wheel inputs are omitted with provenance and an explicit omission ledger; all measurement and preparation-error logs are retained. Independent audit reviews the recorded evidence; it does not authenticate device execution or independently rerun GPU math.

Copy link
Copy Markdown
Collaborator Author

Compatibility with #149 is now resolved downstream at #149 428f834fad970f7e943180fb749871e04cbdb3e0 (tree 1d68e3c9c338141aa3efe35740f996e5b22de8ad). This PR's source remains 5e340cab24b3e55704bfc674a725b5ed82238710; no commit was pushed here and no GitHub PR was merged. The integrated149 head retains this exact113 head as a parent, preserving ABI9, Graph and fixed/shared/auto behavior while adding runtime admission/deadline controls to all modes.

  • Local exact source: workspace 305 passed /57 optional ignored, formatting, Clippy, release build and strict docs passed;58 benchmark +22 Decider Python tests passed.
  • A80080GB/sm80: corrected CPU/doc 71 passed, separate startup 4 passed, and 15 targeted device/checkpoint Rust pass executions (mixed GPU and CPU checkpoint cases).
  • Fresh reference 20 cases, six modes 120 comparison records; fixed/shared logits+responses exact, long auto exact to fixed with positive sharing, positive Graph capture/replay. Original protocol2 gates unchanged.
  • Six-mode direct/frontend HTTP: 390 saved response bodies, including294 successful POST occurrences with exact same-mode native parity. Runtime: 42 scenarios /3,170 literal observations covering permanent422/no Retry-After, exact-ceiling200, request/token contention503+Retry-After1, upload/active/queue504, positive retained depth/capacity and recovery.
  • All six real projection-failure retirements passed (unavailable health, depth0, later refusals, released GPU state); Graph capture-failure fallback passed18 cases. Initial/final644 source hashes, model-directory entries, two ABI9 libraries and three fresh Rust consumers were rechecked; GPU memory returned0 MiB. Existing CUDA libraries are reused with exact20-file build-input identity, not claimed freshly rebuilt.

Exact combined-head CI · Docs are green. This is targeted qualification of the combined head, not a rerun/relabeling of this unchanged PR's earlier57-case qualification; that historical evidence and source pin remain intact. The new integrated source needs fresh maintainer review; previous approvals are not extended by this comment.

Immutable evidence, reconstruction patch, review map, raw records, independent review and portable audit. Archive 1,297,304 bytes, SHA256 36d4f60af0a405eea6266187e934267583296844866dd1283878196876baae6f;11,886 public files. All five release assets were downloaded and byte/checksum verified; every manifest entry and the downloaded offline audit passed. Both failed harness attempts (early queue probe200; token probe taking capacity before holder503) and environment corrections are retained, not counted as passes. Readiness was corrected without changing source, numerical gates,64-row workload,500ms deadlines or required outcomes. Large reproduction inputs are explicitly omitted with hashes/provenance. Zero performance samples; offline audit reviews recorded author execution, not independent GPU authentication.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

State correction: my re-approval of this PR at this exact head (5e340cab24, pass 73) was posted with the wrong review event type (COMMENT instead of APPROVE) due to a scripting error in my posting script — the full re-approval verdict and its verification are in that review body directly below and are unchanged: zero actionable findings, upstream full-suite CI green at this head. This review corrects the recorded state to match that verdict. Unverified checks as listed there: 2026-10-13 device qualification author-run; Rust suite via exact-head CI.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #113 incremental review — verdict carried — pass 84 (head b3cd06a27)

**Verdict: APPROVED (carried).

Pinned state (re-verified)

  • PR head: b3cd06a27f0995cd7719c8148b8b479e11e44a53 (git rev-parse origin/pr/113 == gh pr view 113 headRefOid). Non-draft, base main, title "decider: reuse request prefixes with conservative auto selection".
  • Previously reviewed head: 5e340cab2 (my APPROVE of 2026-10-22T10:07Z, a state correction of the pass-73 re-approval; zero actionable findings on record — re-read both review bodies via the reviews API, confirmed).
  • First-parent chain (verified with rev-parse ^1/^2, all SHAs cat-file -e OK):
    b3cd06a2 (authored) ← 782db6f4 (merge) ← 267614bf (merge) ← 5e340cab2.
  • git merge-base --is-ancestor 5e340cab2 origin/pr/113 → yes. No own-series commit was rewritten (the previously approved head is the direct first-parent ancestor); range-diff gate therefore trivially satisfied, not run.
  • origin/main tip 5ba6a3ba6 is the merge-base; merge 267614bf absorbs 3a289bd1 (codex/pr112-main-20261023, contains main) and merge 782db6f4 absorbs f54b3cdf (prefix-shared-executor, contains main). Tree delta 5e340cab2..head: 12 files, +592/−36.

Merge hygiene (both merges had conflicts; every resolution hunk read)

  • 267614bf (5e340ca × 3a289bd): synthetic git merge-tree --write-tree = 232801baa, actual = c26126a36 → differs in 4 conflicted files. Resolutions are real side-choices, no novel content: recipe/open_jev/native.md keeps PR-side ABI-9 wording; qwen3_5/README.md hunk 1 takes side-branch ABI-9 paragraph, hunk 2 takes PR-side shared-executor text; gemm.cu dedups a duplicated reference_m field and takes the side-branch's plan_for scheme logic; tests/qwen3_5/kernels.rs keeps the PR-side shape list (2B/4B/9B/27B).
  • 782db6f4 (267614b × f54b3cd): synthetic 14582ba2f, actual 790d43945 → differs in 5 files. docs/supported-models.md keeps the Decider-2B row (correct for this PR); recipe/open_jev/native.md and qwen3_5/README.md keep the fuller PR-side wording; Cargo.toml final state is byte-identical to 5e340cab2 (union incl. both graph and prefix test registrations — nothing lost); model.rs takes the canonical ensure_fixed_profile()? helper in forward_fixed/forward_shared and keeps the already-approved PR-side Drop (unconditional scratch/shared clear) and field order.
  • Tree-wide conflict-marker grep at head (.rs .md .toml .py .cu .c): clean.

Authored commit b3cd06a2 ("Align profile rejection regression and Graph recipe with canonical executor")

2 files, test+docs only:

  • tests/qwen3_5/graph_loading.rs (4 lines): assertion text "require merged weights" → "only merged weights", matching the canonical guard message at src/models/qwen3_5/native/src/model.rs:1658 ("fixed-algorithm GEMMs and shared prefixes support only merged weights with the default numerics"). Not vacuous: the test still asserts both forward_fixed and forward_shared return errors and that the loading_live allocation metric is unchanged across the failed calls.
  • recipe/decider/validation.md (3 lines): pins the single named test ordered_shape_replay_current_ids_growth_and_bounded_cache in the --test graph invocation. The test exists (tests/qwen3_5/graph.rs:7) and its body matches the doc's description (ordered-shape replay, fresh-ID upload before replay, shape growth, FIFO eviction beyond 64 cached shapes, 0 fallbacks).
  • No behavior change smuggled in: the 5e340cab2..head delta of model.rs is a pure refactor — inline ensure! (identical predicate lora.is_none() && reference.is_none() && fp32_gdn.is_none()) moved into the ensure_fixed_profile helper plus a restored doc comment; the production guard is retained.

12-file delta (full new state read)

  • tests/qwen3_5/model.rs (+226, new): substantive profile-rejection regression. Generates a CPU fake CUDA library whose stubs are derived from cuda.rs's api! block at build time (cannot drift from the ABI), compiles it with cc (#[cfg(unix)]), loads a tiny 2-layer LoRA checkpoint, and asserts forward_fixed/forward_shared are refused with zero CUDA calls (call counter unchanged) and zero buffers (scratch none, fixed_gemm null, shared empty) across LoRA, FP32-GDN and reference profiles, and that the guard passes with default numerics. Wired via #[cfg(test)] #[path] at src/model.rs:2402.
  • tests/qwen3_5/profile_fake.c (+20, new): counting CPU fake for the above; PROFILE_FAKE_ABI injected at compile time from the parsed source constant.
  • benchmarks/bench.py (+68) and benchmarks/README.md (+26): --score-decimals {2,3,4} (default 4), score_allowance worst-case bound 0.5·10^-sd + 0.5·10^-4·Σk + 1e-9 (math checks out for independently rounded probabilities), exact score level-range check, complete-consistent zero-fit exception for Decider's all-zero Score, config.json records score_decimals + validation_protocol. Every README claim ("exact level range", "records it for reproducibility", accepted by read_answers/request_one/replay) matches the code.
  • tests/benchmarks/test_score_wire.py (+225, new): 8 tests incl. a randomized property test (counts 2–10 × decimals 2/3/4 × 32 samples), zero-fit completeness/negatives, transport integration via MockTransport, and end-to-end bench.run config recording. In CI via .github/workflows/ci.yml:44.
  • src/backends/cuda/qwen3_5/gemm.cu (14): head state is byte-identical to canonical f54b3cdf; reference scheme exemption moved from first_choice into plan_for — semantically identical on the plan_for path; side effect: fixed_choice's explicit NONE (line 98) is no longer overridden to MASK for reference handles, which is strictly more correct for a function that then verifies splits==1 && scheme==NONE. Informational; upstream-validated content.
  • src/backends/cuda/contract.md (2): example-manifest abi_version 7→4 inside a block the doc explicitly labels "illustrative, not the current Qwen manifest" — reduces impersonation of a real ABI. Informational.
  • docs/architecture.md (7): new sentence "refuse an unmerged adapter, reference numerics or FP32 GDN before any device work" matches ensure_fixed_profile exactly (and is now machine-checked by the new profile test).
  • tests/qwen3_5/graph_loading.rs, recipe/decider/validation.md: as above.
  • Decider runtime admission / Graph lifetime production code: unchanged in this delta (authored commit is test+docs; merges carried canonical upstream executor content, resolved as verified above).

PR body (template + evidence pins)

  • New 2026-10-23 section pins head b3cd06a27… and tree 608ccac86a6f… — verified equal to git rev-parse b3cd06a2^{tree}. Evidence explicitly re-pinned "on this head"; CI/Docs links match the runs observed below. Not stale.
  • Body explicitly discloses the authored repair ("stale constructor-test error substring after #99's guard wording change; both assertions repaired with the production guard retained") — consistent with the commit.
  • Four human contributor answers + the "is a PR needed" follow-up present; 4/4 self-review checkboxes checked; scope limits and author-run disclaimers preserved.

CI at the exact head (observed)

  • gh pr checks 113: rust 2m22s pass, benchmarks pass, cuda-contract pass, build pass, deploy skipping (intentional, documented).
  • Actions API head_sha=b3cd06a2… on both repos returns the same two runs, both completed/success: CI 39886014613, Docs 39886014628 (InferMatrixLab URLs; ThinkFlowLab API lists the same runs).

Host checks I ran (no cargo/GPU per host rule; /Users/hsliu/miniconda3/bin/python on a detached worktree at b3cd06a2, removed after)

  • python -m unittest discover -s tests/benchmarks -p 'test_*.py' → 65 tests OK (includes the new score-wire suite).
  • python -m unittest discover -s tests/decider -p 'test_*.py' → 22 tests OK.
  • python -m mkdocs build --strict → clean build, no warnings.

Findings

Zero P0–P3 findings. Informational notes only:

  1. src/backends/cuda/qwen3_5/gemm.cu:98 — fixed_choice's explicit CUBLASLT_REDUCTION_SCHEME_NONE is now honored for reference handles (previously overridden to MASK by first_choice); strictly more correct given the following splits==1 && scheme==NONE verification. Content is byte-identical to the canonical upstream branch.
  2. src/backends/cuda/contract.md:54 — example manifest now says abi_version: 4; self-disclaimed as illustrative.

Prior-finding status

No open findings carried: my last two reviews (COMMENTED 2026-10-22T02:04Z; state-correction APPROVE 2026-10-22T10:07Z) recorded zero actionable findings at 5e340cab2, and the two merges since then introduced no unresolved resolution issue. Standing unverified items unchanged in kind: device/A800 qualification remains author-reported (no GPU on this host); the Rust suite is covered by exact-head hosted CI (green), not run locally.

Verdict rationale

Non-draft + zero findings + template answers present + evidence re-pinned to the exact new head (tree hash verified) + exact-head CI green on both workflows. No downgrade trigger; prior APPROVE carries to b3cd06a27f0995cd7719c8148b8b479e11e44a53.

@psv666

psv666 commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Additional independent review at b3cd06a27f0995cd7719c8148b8b479e11e44a53: no actionable defect found in the inspected prefix planning/mode integration.

I checked complete-row validation, exact token-prefix grouping, nonempty suffixes, question/candidate reconstruction, auto-selection's saved-work threshold, mode exclusivity, health snapshots and error synchronization. Personally run: 21 Decider CPU tests passed / 9 ignored. The two shared-prefix GPU regressions also passed using this branch's own compiled Qwen consumer, with bit-identical results to independent fixed-algorithm full prompts.

The generic GPU input was derived from already-cached Qwen/Qwen3.5-0.8B@2fc06364715b967f1860aea9cf38778875588b17: language BF16 bytes were preserved and the 36 FP32 language scalar/norm tensors were rounded to BF16; configuration unchanged. Model-file SHA-256: 0b7ab8ba01f46922e58132ef2846cf1b1cf8d8813efa6169e9f45ef76116cbe1. This checks shared execution behavior, not the trained model/adapter or its final quality.

Strict docs passed. The native backend build inputs match the compiled ABI9 library used in the GPU run. I did not load the pinned trained Decider checkpoint or repeat its HTTP, task-output, full-model retirement or performance campaigns; generic-backbone results do not re-certify those author records. This is a review/validation comment, not an approval.

Review provenance and scope
  • Head b3cd06a27f0995cd7719c8148b8b479e11e44a53; PR API base 5ba6a3ba668cb33028eb0bce2c6d17cdc53aebe7; diff merge-base 5ba6a3ba668cb33028eb0bce2c6d17cdc53aebe7. Head checked unchanged before publication. Tests used isolated source archives and per-PR Rust build directories.
  • Code/test diff +2157/−204; total text diff +3064/−219; 45 changed files. Code counts exclude generated JSON fixtures/docs.
  • Repository ID 1386454853. Loaded trusted .agents/skills/system1-omni-review/SKILL.md from target main@5ba6a3ba668cb33028eb0bce2c6d17cdc53aebe7, SHA-256 ae2dfac116376ae2ec41f65122cc1bf5aea10751b8b907198a56154e5d67230e, plus architecture, repository contracts, CONTRIBUTING, self-review artifact/demo rules, current workflows and affected component docs. Observed CI is recorded separately from personally run checks.

Copy link
Copy Markdown
Collaborator Author

@hsliuustc0106 Conflicts with main are resolved and pushed; local checks pass. Could you please re-review? Thanks!

@hsliuustc0106
hsliuustc0106 merged commit 3b3e9f4 into InferMatrixLab:main Oct 24, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants