Skip to content

Add hoist_invariants() directive for semiring reductions - #9190

Open
alexreinking wants to merge 6 commits into
mainfrom
alexreinking/rfactor-hoisting
Open

Add hoist_invariants() directive for semiring reductions#9190
alexreinking wants to merge 6 commits into
mainfrom
alexreinking/rfactor-hoisting

Conversation

@alexreinking

@alexreinking alexreinking commented Jul 4, 2026

Copy link
Copy Markdown
Member

This PR adds a scheduling directive similar to rfactor that hoists invariant factors according to a detected distributive (semiring) law into the write-back part of the accumulation step. This enables writing straightforward quantized kernels in the algorithm language and factoring them into efficient kernels.

This is meant to work towards optimizing quantized mat-mul and mat-vec kernels (especially those from GGML).

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Python bindings updated (if public API changed)
  • Benchmarks are included here if the change is intended to affect performance.
  • Commits include AI attribution where applicable (see Code of Conduct)

@alexreinking alexreinking added the release_notes For changes that may warrant a note in README for official releases. label Jul 4, 2026
@alexreinking alexreinking changed the title Alexreinking/rfactor hoisting Allow hoisting invariant factors in rfactor Jul 4, 2026
@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.86842% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.14%. Comparing base (4620569) to head (e588356).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/Func.cpp 84.86% 8 Missing and 15 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9190      +/-   ##
==========================================
+ Coverage   70.02%   70.14%   +0.12%     
==========================================
  Files         258      259       +1     
  Lines       78702    79272     +570     
  Branches    19164    19344     +180     
==========================================
+ Hits        55110    55609     +499     
- Misses      17884    17899      +15     
- Partials     5708     5764      +56     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking
alexreinking marked this pull request as draft July 6, 2026 14:22
@alexreinking alexreinking added the dev_meeting Topic to be discussed at the next dev meeting label Jul 6, 2026
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 2 times, most recently from 9e244ff to 328fd0a Compare July 20, 2026 15:28
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 3 times, most recently from f2cbdc3 to f271649 Compare July 29, 2026 19:07
@alexreinking
alexreinking changed the base branch from main to alexreinking/change-type July 29, 2026 19:07
@alexreinking alexreinking changed the title Allow hoisting invariant factors in rfactor Add hoist_invariants() directive Jul 29, 2026
@alexreinking alexreinking changed the title Add hoist_invariants() directive Add hoist_invariants() directive for semiring reductions Jul 29, 2026
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from f271649 to 9660010 Compare July 29, 2026 19:33
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 9660010 to 12834ac Compare July 29, 2026 19:59
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 12834ac to d049c0c Compare July 30, 2026 12:45
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from d049c0c to 460d780 Compare July 30, 2026 13:35
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 2 times, most recently from 8f48f71 to 25be6cb Compare July 30, 2026 15:20
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 3 times, most recently from a26996d to 9787945 Compare July 30, 2026 16:57
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from c8dc9c6 to 7e94a45 Compare August 7, 2026 07:52
Comment thread test/performance/separable_downsample.cpp Outdated
Comment thread test/performance/separable_downsample.cpp Outdated
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 7e94a45 to 7a912b0 Compare August 8, 2026 19:31
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 3 times, most recently from 519d92d to bd73073 Compare August 8, 2026 20:37
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from bd73073 to 0c7be4f Compare August 9, 2026 14:09
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 0c7be4f to 1a19df0 Compare August 9, 2026 14:10

@abadams abadams left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly need to delete the separable downsample test, because it doesn't do what we hoped, but LGTM for the rest of it.

@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 3 times, most recently from 3cbcddf to fb002b3 Compare August 13, 2026 14:23
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from fb002b3 to e0bc603 Compare August 13, 2026 14:25
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from e0bc603 to 79fbc49 Compare August 15, 2026 18:26
Base automatically changed from alexreinking/change-type to main August 16, 2026 03:41
alexreinking and others added 6 commits August 15, 2026 23:41
Introduce Stage::hoist_invariants(), which applies a semiring distributive law to hoist loop-invariant factors out of associative reductions. The returned intermediate accumulates the factor-free reduction over the original RDom, while the write-back applies the factor once.

Support +/*, min/+, max/+, or/&&, and and/|| laws, including factors nested within associative chains. Add reusable binary-operator helpers, the Python binding, and correctness tests.
Add a quantized (int8 x int8 -> f32) mat-vec performance test targeting ARM's
SDOT instruction, scheduled by composing eager_inline(), hoist_invariants(),
rfactor(), and change_type(). eager_inline() folds the Wt and VecDq producers
into Acc's update so their scale factors -- WtScale(i) and VecScale -- surface
as explicit leaves that hoist_invariants() can then lift out of the reduction as
the invariant product WtScale(i) * VecScale. rfactor() splits the now scale-free
reduction by block, and change_type(Int(32)) retypes the innermost per-block dot
product so CodeGen_ARM matches it to SDOT.

The generated assembly contains sdot instructions, the Hoisted/PlainRfactor
numerical cross-check passes, and the composed schedule is several times faster
than the non-hoisting variant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…elf_reference

A prior rebase silently merged this call site as a 3-argument Call::make()
after the Function overload gained a required follow_global_wrappers
parameter. This is a Func's self-reference to the rfactor intermediate, so
it must not follow global wrappers, matching the other self-reference call
sites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 79fbc49 to e588356 Compare August 16, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release_notes For changes that may warrant a note in README for official releases.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants