Skip to content

decider: add verified eager native CUDA worker - #110

Merged
hsliuustc0106 merged 15 commits into
InferMatrixLab:mainfrom
Levius-Fubuki:codex/decider-native-worker
Oct 12, 2026
Merged

hsliuustc0106 merged 15 commits into
InferMatrixLab:mainfrom
Levius-Fubuki:codex/decider-native-worker

Conversation

@Levius-Fubuki

@Levius-Fubuki Levius-Fubuki commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

Decider-2B v11 currently has CPU request compilation/response assembly in #94 but no checkpoint inference or native HTTP worker. Add the complete eager Rust/CUDA path: verify the immutable released checkpoint/config/tokenizer, run independent Qwen rows, project all 255 tied output labels on CUDA with BF16 output rounding, and serve Choice/Noul/isolated Score through the existing frontend.

One SerialScheduler admission covers the complete request and GPU head. Errors synchronize before releasing admission, retire failed execution and return unavailable health. Real warmup completes before serving. Depends on #94; the main-based diff includes that CPU contract.

Resolution — 2026-10-10

The reference comparator rejects NaN/infinity, booleans, malformed answer/calibration schemas, invalid rounding and inconsistent confidence/certainty/Score summaries. Auxiliary checks derive their bounds from probability wire rounding; the original probability/Score/winner gates remain 0.02/0.1/0.05. Revised protocol v2 is saved before launching workers.

Current head: ebd572b9cd885a67949624f9c37c253ddfe9de4e. Actual PR target: main at 7cbce19f117f23214c8acd8d874c2ab5f10c683e; diff merge base: 7cbce19f117f23214c8acd8d874c2ab5f10c683e. Main-target diff: 32 files, 2,756 authored changed lines / 2,982 total text changed lines (additions plus deletions). Increment against repaired #94 a2c836f91e5e0390d7bf7223c009ea20d751e8ae: 1,625 authored / 1,638 total changed lines. The actual PR base remains main. Authored counts include source, tests, build/validation scripts, Cargo/backend manifests and workflow code; documentation, licenses/notices, static fixtures, lockfiles and generated output are excluded.

Dependency/review order: #94 → #110 → #111 → #112 → #113. Normal merges preserve published history. The cumulative diff includes prerequisite processing, execution and response reconstruction; review the repaired increments separately, then their model/executor interfaces and device resource lifetimes together.

Test Plan / result

System1-Omni Version / Commit: ebd572b9cd885a67949624f9c37c253ddfe9de4e.

python -m unittest discover -s tests/decider -p "test_*.py" -v: 12 passed on this stage. The reference tests include corrupted calibrated fields and nonfinite input cases. cargo test --locked -p omni-decider-native: 10 passed, 0 failed, 5 ignored.

Fresh checks run locally on CPU-only macOS. GPU compilation, current-head checkpoint inference, HTTP numerical parity, device-failure retirement and performance are unverified. ABI6 remains required at this stage; rebuild CUDA library and worker together. Skipped GPU/checkpoint tests are not passes.

Demo / evidence

Host regression bodies and fixtures are retained under root tests/. Protocol v2 distinguishes helper validation from real inference. Historical 2026-10-07 device evidence and 2026-10-08 verification archive retain their recorded source/library/workload scope; they do not validate this integrated head. No new numerical or performance claim is made.

Agent-assisted review and fixes are contributor preparation; maintainer review and dependency merges remain outstanding.

Current-head host validation

cargo fmt --all --check, cargo clippy --workspace --locked --all-targets -- -D warnings, cargo test --workspace --locked, cargo build --workspace --release --locked and mkdocs build --strict pass independently on this repaired stage. Workspace: 149 passed, 0 failed, 30 ignored; Decider Python helpers: 12 passed. Both pinned CPU golden contract tests also pass, and the frontend CI command passes 12 tests. Hardware-dependent ignored tests remain unexecuted.

Exact-head hosted verification

GitHub Actions ran in Levius-Fubuki/system1-omni on the identical PR head ebd572b9cd885a67949624f9c37c253ddfe9de4e. These are fork workflow-dispatch runs, separate from upstream pull-request merge checks.

CI rust covers formatting, all-target Clippy, frontend API tests, workspace tests and release build. Benchmark validation/tests also pass. Docs build is strict; deployment is intentionally skipped on the fork. The published Fork CI / rust, Fork CI / benchmarks and Fork Docs / build statuses all report success for this SHA.

Review feedback resolution

  • Normal merge ebd572b9 carries the repaired parent and current main without force-pushing or changing Decider numerical/device source. Root Cargo.toml retains both Decider and OmniJev workspace members.
  • 365462ce retains the reviewed correction in tests/decider/verify_reference.py: The comparator rejects nonfinite/boolean/schema/calibration errors and preserves probability/Score/winner gates 0.02/0.1/0.05. Host regressions remain under root tests/decider/.
  • Current-head CPU/host checks pass. Historical GPU evidence remains bound to its original source and library; no current merge-head device parity or performance claim is made.

@Levius-Fubuki
Levius-Fubuki marked this pull request as ready for review October 8, 2026 01:11
Copilot AI balanced review requested due to automatic review settings October 8, 2026 01:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 5d1302aac3 (7 commits; true delta 4d4c504e..5d1302aac — the GitHub files-changed view also carries #94 because #94 sits on the older main tip). No actionable findings.

The delta adds checkpoint.rs with real identity pinning (size + SHA-256 of config/tokenizer/weights, sharded-index refusal, backbone-shape assertion, BF16 finite check, 256-row zero-padded tied-head extraction), executor.rs with a dedicated head stream and cuBLASLt GEMM handle, catch_unwind around device work, both streams synchronized before admission release, and retire-on-error setting health unavailable; serve.rs maps 415/422/500-else-503 with an 8 MiB body limit consistent with the frontend limits, and prepares requests outside admission via spawn_blocking. Error retirement and health semantics match the runtime contract pattern in docs/architecture.md, and GPU-touching unit tests are correctly #[ignore]d (fail_head.c is an LD_PRELOAD interposer for an opt-in failure test, not a CI stub).

Template, self-review checklist, and stated counts (2,481 authored / 2,707 total — both match my independent count) are in order. One P3: the body's "Full self-review completed" section still pins superseded head 31acb29b6 with 2477/3116 counts, while the Validation section correctly describes the current head — please update or drop the stale section (same staleness exists in #111–#113).

CI (rust/build/benchmarks) passes on this head (observed). The GPU campaign (RTX 4090, pinned-reference parity with max probability drift 0.0119 and zero Choice disagreements, direct+frontend HTTP parity) is author-reported at snapshot heads with SHA-pinned archives; not independently executed by this review — no GPU was used and no Rust toolchain exists on this host.

Provenance: canonical .agents/skills/system1-omni-review/SKILL.md (SHA-256 58be3bc6…dde108) and its repository map read at trusted base 4a79980d8a75, plus CONTRIBUTING.md and docs/architecture.md. Remote head rechecked before posting.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head 365462cead after the branch rebase (prior review: COMMENTED at 5d1302aac3, 2026-10-08). Series identity: git range-diff of the previously reviewed head vs the new head shows every commit patch-identical and none added or dropped — a rebase, not new work. Provenance: review skill at main@4a79980d (SHA-256 58be3bc6…dde108); remote head re-pinned immediately before posting.

Prior finding status — still open (no content change): the "Full self-review completed" section still pins superseded head 31acb29b6 with stale counts; please update or drop it. No CI has reported on the rebased head yet (identical series passed rust/build/benchmarks at the prior head). Unverified (unchanged): GPU evidence author-reported; no GPU/Rust toolchain on this host.

Verdict: COMMENTED (one open P3).

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head ebd572b9cd — the branch moved again after my review earlier today at 365462cead. git range-diff against that reviewed head: every commit patch-identical, none added or dropped — another rebase, no content change. The one open P3 (stale "Full self-review completed" section pinning superseded head 31acb29b6) is unchanged and still open. No CI on this head yet. Verdict: COMMENTED (unchanged).

@Levius-Fubuki

Copy link
Copy Markdown
Collaborator Author

@hsliuustc0106 The stale “Full self-review completed” section has been removed from this PR description, including the superseded head SHA and line counts referenced in your latest review. The description now identifies the current head ebd572b9cd885a67949624f9c37c253ddfe9de4e and its validation scope.

CI has also completed successfully on this exact head:

I verified that the linked fork Actions runs use the exact PR head SHA above. These are fork workflow runs, separate from upstream PR merge checks; they do not establish fresh GPU numerical parity, device-failure retirement, or performance validation.

Please re-check the current description and exact-head CI evidence, and reassess the remaining P3 about the stale self-review section.

@hsliuustc0106
hsliuustc0106 merged commit 3e9cc33 into InferMatrixLab:main Oct 12, 2026
3 checks passed
@Levius-Fubuki Levius-Fubuki mentioned this pull request Oct 13, 2026
2 of 4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants