Skip to content

docs(jemm): document CUDA Graph controls and validation limits - #123

Open
Levius-Fubuki wants to merge 10 commits into
codex/jemm-contract-followupfrom
codex/jemm-cuda-graphs
Open

Levius-Fubuki wants to merge 10 commits into
codex/jemm-contract-followupfrom
codex/jemm-cuda-graphs

Conversation

@Levius-Fubuki

@Levius-Fubuki Levius-Fubuki commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

CPU maintenance and merge handoff — 2026-10-15

Current head: 7013822abea16ab2e4577988d560aca83771901d.
Actual target: codex/jemm-contract-followup at 5d630bc2fb59830a4baf5e93be9248afa4355c58; the PR target was not changed.

Normally merged the maintained #122 parent. The complete delta from this PR's previous head is the shared CUDA contract documentation only; the authored JEMM Graph increment remains documentation-only. Landing order remains #117 → #118 → #122 → #123, after the #117 hold is released. Current builds use ABI 8; no new Graph/27B/device qualification is claimed.

Personally ran strict MkDocs and git diff --check on this head. Verified that every changed file since the previous published head is Markdown; all non-Markdown bytes are unchanged. Rust/device suites were not repeated for this documentation-only delta.

This pass follows the contributor's 2026-10-15 instruction to maintain these existing PRs, complete CPU-only fixes and merge preparation, and keep #117 unmerged. Existing contributor declarations are preserved; this update adds no new human attestation or maintainer approval. No PR was merged and no GPU/model/performance run was performed. Source-pinned historical results remain historical. Current-head hosted workflows: CI run 38667525027, Docs run 38667528289. Read their live results separately from the personally executed checks above; these CPU workflows do not execute the skipped GPU tests.

Historical records below

Earlier maintenance/review/test sections retain their named revisions and environments. Their old ABI or build commands are historical reproduction instructions, not commands for deploying this current head. Use the current branch recipes for builds; the CPU checks above do not qualify skipped device tests.

Historical maintenance — 2026-10-15

Current head: b0d491084dfe4ca0631e779e2ae73b77c19e5ab4.

Synced the documentation-only increment onto the maintained #122 parent at 64712ee with a normal conflict-free merge. Every tracked file except src/models/jemm/README.md is byte-identical to that fully tested parent. Preserved the explicit limits on combined v2 reference arithmetic and multimodal Graph qualification; these docs still make no new device or universal speed claim.

Personally verified that the complete staged tree differs from the validated #122 source in exactly one Markdown file; strict MkDocs and git diff --check passed. Rust/device code was unchanged relative to #122, whose full local checks passed, so the same Rust suite was not repeated locally for this documentation-only increment. Current-head hosted CI is linked below. No new GPU/parity/performance run was performed.

Current-head hosted workflows: ci.yml, docs.yml. Linked hosted results are separate from personally executed local checks. Earlier evidence sections retain their named original revisions and scope; they are not new qualification claims.

No PR merge was performed. #117 remains on its explicit merge hold.

Required human contributor answers

These answers reuse the contributor's explicit 2026-10-12 maintenance instruction:
keep the existing PRs mergeable and CI passing; eliminate conflicts and review
issues; serve maintainers who need clear, verified changes that are easy to merge;
repair, validate and push to the existing PRs. The source is the contributor's
local maintenance conversation on that date; private conversation details are
not published.

  1. What outcome are you trying to achieve?

    Keep the existing PRs mergeable and CI passing.

  2. What is the highest-leverage variable for that outcome?

    Eliminate merge conflicts and resolve review issues.

  3. How will this work move that variable rather than create busywork?

    Repair the identified conflicts and review issues, validate the affected
    integration, and push the verified changes to the existing PR branches.

  4. Who is the customer or user, and what do they actually need?

    Repository maintainers, who need clear, verified changes that are easy to merge.

After answering the four questions: is a PR needed at all to achieve this outcome?

The contributor explicitly directed updates to the existing PRs. Maintain these
PRs; no new PR is needed for the maintenance work.

Bug-fix evidence: N/A: the effective increment is a 23-line documentation addition, with no runtime implementation change. The known duplicate initializer was removed during merge preparation; no unrun pre-fix compiler failure is presented as observed evidence.

Purpose

Document JEMM's inherited CUDA Graph controls, cache limits and validation boundaries. After integration with current #122 and removal of a duplicate mm_graphs initializer introduced by the merge, the effective incremental change is 23 README lines. This PR adds no runtime feature or performance optimization.

Scope

System1-Omni Version / Commit: d3b6186767d84139a0003df78206229f6e6f0e88; incremental baseline #122 1eab32bac8cb19f0fca925d69c14dff03c0c4c2e.

Integrated main: 3e9cc33a61ddf110d97512fb6f296cf540377c61. Review order: #117 → #118 → #122 → #123.

The README describes existing default-off CUA_S1_GRAPH and CUA_S1_VISION_GRAPH controls, independent language text/multimodal FIFOs and single-/multi-image vision FIFOs. It states that current inputs are uploaded before replay, scratch/Graph lifetimes are synchronized, entry limits are not total-device memory limits, and historical evidence does not certify combined current v2 multimodal-language replay.

Historical Test Plan

System1-Omni Version / Commit: d3b6186767d84139a0003df78206229f6e6f0e88.

Reproduction commands from the repository root; recorded local feature switches and device prerequisites remain scoped to the original results above. This PR-body update runs no additional source or device checks.

cargo fmt --all --check
cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings
cargo test --workspace --locked --features omni-laya/serve
cargo build --workspace --release --locked --features omni-laya/serve
python3 -m mkdocs build --strict

Compare the effective increment with #122, validate the documented Graph controls against source, and run strict MkDocs. The final incremental diff is 23 README lines; ordinary documentation checks apply, with the inherited CPU/hosted results and device limits retained below. No new GPU benchmark or performance result is claimed.

Historical Test Result

The final source was independently compared with #122: only the 23-line README addition remains; runtime sources are identical after the known duplicate initializer repair. Diff checks passed. Earlier hosted workflow results are not reported as verification of this new head; current hosted status must come from checks attached to this commit.

Final maintenance checks passed formatting, strict workspace/all-target Clippy, locked workspace tests, release build and strict MkDocs. Workspace tests passed 232 unit/integration cases plus one compile doctest (233 total passed), with 37 device/checkpoint tests ignored. #122's eight tiny-export tests exercise identical exporter code; no new exporter or runtime behavior is introduced here.

Independent read-only review compared this merge against both its previous head and its updated base, with no actionable merge regressions found. Agent assistance does not replace contributor or maintainer review.

Demo / evidence

The historical #118 A800 v2 campaign passed the frozen 16-request / 25-question corpus with maximum probability drift 0.0166172279642417 < 0.02. It used text-language and vision Graphs while multimodal language remained eager. Earlier multimodal replay evidence belongs to the premerged v1 implementation.

The combined v2 reference-numerics plus multimodal-language replay path has not been rerun with the 27B checkpoint for this maintenance head. Fresh RTX 4090 fixed-seed operator checks in #118 do not establish full-checkpoint replay equality, actual HTTP performance or broader model accuracy. CPU checks compile ignored device regressions without executing them.

No new performance claim is made.

Exact-head hosted verification — 2026-10-13

These are upstream workflow-dispatch results for the exact source head. Ignored GPU/checkpoint tests did not execute; maintenance status contexts link the actual runs.

Video: N/A for this nonvisual API, validator, shared executor or documentation change. Source-pinned tests and the linked validation records provide the relevant evidence; existing hardware and unverified-measurement limits remain unchanged.

Self-review

The contributor explicitly confirmed completion of self-review and authorized this checklist/template update on 2026-10-13 in the maintenance conversation. This records the human confirmation; it does not turn unverified measurements into passes or replace maintainer review.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on the stated purpose.
  • I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
  • I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Copilot AI balanced review requested due to automatic review settings October 8, 2026 08:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 589a96c94 (stack tip, stacked on #122; 2 files, +263/−89; also reviewed deeply as the cumulative tip vs main). Disciplined replay extension — every invariant of the existing graph paths is preserved, verified in code:

  • Opt-in and default-off: CUA_S1_GRAPH=1 (text + multimodal language) and a separate CUA_S1_VISION_GRAPH=1 (vision) — the new separate env var is deliberate and documented.
  • Exact-shape keying in separate caches: multimodal keyed [t] in its own 64-entry FIFO (no cross-mode collision with text), vision keyed exact [T,H,W] in a 4-slot FIFO.
  • Upload-before-replay everywhere: positions/embeddings/ids before the language branch; pixels every call and per-grid geometry once at scratch creation (a pure function of grid + config — sound).
  • Record-only capture after a synchronized eager miss with the eager result retained; capture failure disables, clears, logs unconditionally, and keeps the eager answer; run() is untouched, so captured work equals eager.
  • Lifetime ordering correct: synchronize() before evicting (pop_front), Scratch declares graph first so eviction destroys the graph exec before the buffers it references, and VisionModel::drop synchronizes first.

The two intentional deviations (separate vision env var; scratch buffers retained with graphs off — quantified per config in the body, figures re-derived and consistent) are disclosed. CI + Docs verified SUCCESS at this exact head (workflow dispatch).

No actionable findings.

Approving the code review; merge in stack order (#117 → #118 → #122 → #123), with two coordination notes: this stack overlaps open #103/#108 (same regions of model.rs/vision) — whichever loses the merge race rebases; and the multi-image parity question is tracked on #118, not re-counted here. GPU runtime behavior (replay counters, eviction, capture-failure injection, eager equality over 128 responses/200 questions) remains author-reported — no GPU on this host.

Provenance: canonical .agents/skills/system1-omni-review/SKILL.md (SHA-256 58be3bc6…dde108) and its repository map at trusted base 4a79980d8a75; remote head rechecked before posting.

@Levius-Fubuki Levius-Fubuki changed the title feat(native): add multimodal language and shared vision CUDA Graph replay feat(native): add multimodal language CUDA Graph replay Oct 10, 2026

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head b0df271cae after the branch rebase (prior review: APPROVED at 589a96c94, 2026-10-08). Series identity: git range-diff of the previously reviewed head vs the new head shows every commit patch-identical and none added or dropped — a rebase, not new work. Provenance: review skill at main@4a79980d (SHA-256 58be3bc6…dde108); remote head re-pinned immediately before posting.

Verdict unchanged: APPROVED. Same unverified checks as before: graph replay and vision-graph parity remain author-reported GPU evidence (no GPU on this host); no CI has reported on the rebased head yet (identical series passed at the prior head). Stack tip of #117 → #118 → #122 → #123.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head 599459f1cc — the branch rebased after my approval at b0df271cae. git range-diff against that reviewed head: all commits patch-identical, none added or dropped — no content change; the approval carries. Fork CI at this exact head: rust/docs/benchmarks all pass. Unverified (unchanged): graph replay and vision-graph parity remain author-reported GPU evidence; no GPU/Rust toolchain on this host. Verdict: APPROVED (unchanged). Stack tip of #117 → #118 → #122 → #123.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Series-identity correction at head 599459f1cc (unchanged since the approval; found in the 2026-10-11 audit). My approval (id 5481557170) said "all commits patch-identical, none added or dropped — no content change". The true range-diff against the previously reviewed b0df271cae shows 7 commits added — the #118 v2 unmerged-export series (9d4436e…91cc5c2), absorbed when the stack rebased onto the reworked #118; 91cc5c2 is #118's approved head and an ancestor of this head. Those exact patches were reviewed and approved under #118 (id 5481557073, same SHAs), so the approval's substance is unaffected — but the "no content change" statement was false; the sweep compared against unfetched refs (tooling since fixed). Verdict: APPROVED stands, on the corrected basis. Unverified (unchanged): graph replay and vision-graph parity remain author-reported GPU evidence; no GPU/Rust toolchain on this host. Stack tip of #117 → #118 → #122 → #123.

Provenance: review skill at ca5f69b (SHA-256 58be3bc6…dde108); fork CI green at this exact head.

@Levius-Fubuki Levius-Fubuki changed the title feat(native): add multimodal language CUDA Graph replay docs(jemm): document CUDA Graph controls and validation limits Oct 13, 2026
@Levius-Fubuki Levius-Fubuki mentioned this pull request Oct 13, 2026
2 of 4 tasks

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head after the rebase wave (prior review: APPROVED). Wave-verification note: this pass reviewed the whole 10-branch rebase wave systematically — every range-diff shows the own-commit series patch-identical; all post-pin merge resolutions were checked file-by-file against both parents (source/test/config/docs) with zero dropped lines; approved-content markers verified present in every tree (jemm v2 export, decider fixture READMEs, serving-matrix docs, vision FIFO regressions). Fork/upstream CI states as noted per PR.

All commits are patch-identical to the approved series (range-diff, named refs, cat-file verified); the stack merges' resolutions were checked file-by-file with zero dropped content, and the approved markers (the graph-replay and calibration content) remain in the tree. Exact-head upstream maintenance workflows (ci + docs) pass at this head, posted by the author; earlier exact-head fork CI also green.

APPROVED (verdict carries — content-identical rebase). Transition note: the body predates the post-#138 template — add the four human contributor answers and the completed self-review checklist when next touched. Standing unverified checks unchanged (GPU evidence author-reported; no Rust toolchain on this host).

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-check at new head 7013822abe after the rebase wave onto main@e66ff34 (prior review: APPROVED). Wave-verification note (this pass): the rebase wave was verified systematically — every own-commit series is patch-identical per range-diff; every post-pin merge resolution was diffed against both parents with zero dropped lines; approved-content markers verified present in every tree; the key new merge 3be46fa (shared-vision line × jemm v2 tip) carries #143\u2019s new test files intact and is ABI 8/8 consistent at every descendant head.

This branch inherits the verified 3be46fa2 merge plus trivial merges; the jemm v2 markers (export.py without safe_merge, jemm-native/2, untied head, recipe docs) are present in the tree, and the FP32-GDN/resident-feature content from the shared line is unioned, not dropped. Exact-head maintenance workflows (ci + docs) pass, annotated "no GPU qualification" as usual.

APPROVED (verdict carries — content-identical rebase). Transition note unchanged: post-#138 human answers and checklist to be added when next touched. Standing unverified: GPU evidence author-reported; no Rust toolchain on this host. Stack order #117 → #118 → #122 → #123 still applies.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants