Skip to content

ONNX op wishlist: typed Where, pair compares, copy-free matmul_f32, gather_rows_f32, float accumulation order - #400

Merged
AdaWorldAPI merged 11 commits into
masterfrom
ccr-45a41fb4-0j3i0l
Oct 30, 2026
Merged

AdaWorldAPI merged 11 commits into
masterfrom
ccr-45a41fb4-0j3i0l

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

This adds the ndarray functions that lance-graph's ONNX registry probe asked for in its wishlist. That is D-TRACT-HARVEST-0; on Jina v5 small, 3,474 of 7,620 nodes are still unregistered. Each item was first checked against master to confirm it was a real gap. This session pushes to one branch only, so the items come as one commit each rather than one PR each.

1. A typed Where: blend_f32, blend_f32_scalar, blend_i64, blend_i64_scalar

  • All four share one generic body with blend_i32. The select moves whole elements and never computes on them, so f32 NaN payloads and -0.0 come through bit for bit; the test checks both.
  • The scalar variants cover Where(mask, scores, -inf) without filling a -inf vector.

2. Comparing two arrays into a mask, plus i64 compares

  • {eq,lt,le,gt,ge}_{f32,i32,i64}_pair_to_mask(a, b, out) cover ONNX Equal, Less, Greater and the rest. The f32 versions follow IEEE rules, so NaN compares false.
  • Adds {eq,ne,lt,le,gt,ge}_i64_to_mask, comparing an i64 array against a scalar. No backend has a signed i64 compare, so these go through U64x8::cmpgt_mask / cmpeq_mask after XOR-ing the sign bit, which maps signed order onto unsigned order exactly.
  • Every function is checked against a scalar loop across 10 lengths, with MIN/MAX, ±0, ±inf and NaN.

3. matmul_f32 without copies, and matmul_f32_batched

  • matmul_f32 used to make two packing copies and allocate a result buffer. It now calls matrixmultiply::sgemm, the same kernel, with each view's own strides, and writes straight into out.
  • The result is bit-identical to the old path for row-major, column-major, transposed, stepped and reversed inputs. out is overwritten without being read, and k = 0 gives zeros.
  • out must still be row-major: the existing NonContiguousOutput check is unchanged.
  • matmul_f32_batched takes [b,m,k] × [b,k,n]. A rhs with batch size 1 is shared by every entry; any other batch-size mismatch is refused.

4. Float accumulation order across CPU tiers

examples/float_accum_probe.rs hashes the bits of a fixed gemm_f32 and a fixed mean_axis / sum. I ran it on five tiers, using qemu -cpu to choose matrixmultiply's kernel:

AVX-512 FMA+AVX2 NEON AVX, no FMA SSE2
sum, mean_axis A A A A A
gemm_f32 F F F N N
  • sum and mean_axis give the same bits on every tier. They use a fixed eight-way partial-sum order, which is written in Rust source, with no FMA and no reassociation.
  • gemm_f32 gives two sets of results, not five. The FMA kernels agree with each other across x86 and ARM, and so do the non-FMA kernels. The order over k is therefore the same in all five; only fused versus separate rounding differs.
  • So the gap against ONNX Runtime comes from summation order, not from the backend, and one tolerance holds on every ndarray tier.
  • Both facts are pinned by tests, so a matrixmultiply upgrade that moved either class fails. The full write-up is in .claude/knowledge/float-accumulation-order.md.

5. hpc::gather::gather_rows_f32

  • Applies the ONNX index rule [-rows, rows). A bad index returns GatherError instead of panicking or clamping.
  • Every index is checked before anything is written, so out is unchanged on an error.

Also: the simd_stride module doc now names lut9's second input as the DR512 presence/polarity plane, not a sign plane.

Evidence

  • Disable runs. Each guard was broken in turn and restored from the commit; each one turned its test red:
    • the i64 sign flip,
    • the pair-compare tail mask,
    • the blend's else-side index,
    • the matmul input strides,
    • the gather negative bound,
    • the gather validate-before-write.
  • New tests for items 1, 2, 3 and 5 on every tier.
    • AVX-512 on the host: pass.
    • AVX2 (config-v3): 14/14.
    • AVX without AVX2 (qemu, SandyBridge): 14/14.
    • aarch64 NEON (qemu): 10/10; the matmul tests are x86-only.
  • The item-4 tests are running on the four other tiers now; I'll report the result on this PR.
  • Full lib suite: 2,687 passed, 0 failed.
  • cargo clippy --features std --all-targets -- -D warnings and cargo fmt are clean, the new doctests pass, and the facet-stride guard is clean on the diff.
  • Not covered: cargo check for wasm32 fails in getrandom on master as well.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY

…Where); one generic body for the blend family

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
…and i64 scalar compares

ONNX Equal/Less/Greater take two tensors. i64 compares go through
U64x8 cmpgt/cmpeq with the sign bit flipped, which every backend has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
…king, no result buffer); matmul_f32_batched

Same matrixmultiply kernel, bit-identical to the packed path on row-major,
column-major, transposed, stepped and reversed inputs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
…ws) index rule, refusing instead of panicking

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
…ce/polarity plane, not a sign plane

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
…ype_complexity)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a204a3fb-36d6-4632-bbdc-fd550ee57332
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

… comparing CPU tiers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
sum/mean_axis are bit-identical on AVX-512, FMA+AVX2, AVX, SSE2 and NEON
(fixed eight-way order); gemm_f32 splits into an FMA class and a non-FMA
class with the same k order. Both facts are pinned by tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review October 30, 2026 10:04
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T10:06:44.733716Z e057579 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@AdaWorldAPI AdaWorldAPI changed the title ONNX op wishlist: typed Where, pair compares, copy-free matmul_f32, gather_rows_f32 ONNX op wishlist: typed Where, pair compares, copy-free matmul_f32, gather_rows_f32, float accumulation order Oct 30, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e0575799bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/numeric_util.rs Resolved
Comment thread src/hpc/amx_matmul.rs Resolved
…f32_batched validates inner shapes even for an empty batch

Both from the Codex review on #400. On 64-bit the fixture values are
unchanged, so the pinned gemm hash still holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
@AdaWorldAPI
AdaWorldAPI merged commit f932a1c into master Oct 30, 2026
30 checks passed
AdaWorldAPI pushed a commit that referenced this pull request Oct 30, 2026
Records #402, plus the rows #400 and #401 moved. New dots N22-N26: the
adders as facade words with no consumer yet, the exp speed-up reaching
every exp-based word, AVX2 select mask packing as a second array-backed
cost, NEON scalar fcmp/csel in simd_exp_f32, and the transactional
Abacus against the non-transactional eager counter. Fruit L20, L21, H12,
H13; PR C not planned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants