Add hoist_invariants() directive for semiring reductions - #9190
Open
alexreinking wants to merge 6 commits into
Open
Add hoist_invariants() directive for semiring reductions#9190alexreinking wants to merge 6 commits into
alexreinking wants to merge 6 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9190 +/- ##
==========================================
+ Coverage 70.02% 70.14% +0.12%
==========================================
Files 258 259 +1
Lines 78702 79272 +570
Branches 19164 19344 +180
==========================================
+ Hits 55110 55609 +499
- Misses 17884 17899 +15
- Partials 5708 5764 +56 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alexreinking
marked this pull request as draft
July 6, 2026 14:22
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
2 times, most recently
from
July 20, 2026 15:28
9e244ff to
328fd0a
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
3 times, most recently
from
July 29, 2026 19:07
f2cbdc3 to
f271649
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
July 29, 2026 19:33
f271649 to
9660010
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
July 29, 2026 19:59
9660010 to
12834ac
Compare
4 tasks
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
July 30, 2026 12:45
12834ac to
d049c0c
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
July 30, 2026 13:35
d049c0c to
460d780
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
2 times, most recently
from
July 30, 2026 15:20
8f48f71 to
25be6cb
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
3 times, most recently
from
July 30, 2026 16:57
a26996d to
9787945
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 7, 2026 07:52
c8dc9c6 to
7e94a45
Compare
abadams
reviewed
Aug 7, 2026
abadams
reviewed
Aug 7, 2026
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 8, 2026 19:31
7e94a45 to
7a912b0
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
3 times, most recently
from
August 8, 2026 20:37
519d92d to
bd73073
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 9, 2026 14:09
bd73073 to
0c7be4f
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 9, 2026 14:10
0c7be4f to
1a19df0
Compare
abadams
approved these changes
Aug 10, 2026
abadams
left a comment
Member
There was a problem hiding this comment.
Sadly need to delete the separable downsample test, because it doesn't do what we hoped, but LGTM for the rest of it.
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
3 times, most recently
from
August 13, 2026 14:23
3cbcddf to
fb002b3
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 13, 2026 14:25
fb002b3 to
e0bc603
Compare
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 15, 2026 18:26
e0bc603 to
79fbc49
Compare
Introduce Stage::hoist_invariants(), which applies a semiring distributive law to hoist loop-invariant factors out of associative reductions. The returned intermediate accumulates the factor-free reduction over the original RDom, while the write-back applies the factor once. Support +/*, min/+, max/+, or/&&, and and/|| laws, including factors nested within associative chains. Add reusable binary-operator helpers, the Python binding, and correctness tests.
Add a quantized (int8 x int8 -> f32) mat-vec performance test targeting ARM's SDOT instruction, scheduled by composing eager_inline(), hoist_invariants(), rfactor(), and change_type(). eager_inline() folds the Wt and VecDq producers into Acc's update so their scale factors -- WtScale(i) and VecScale -- surface as explicit leaves that hoist_invariants() can then lift out of the reduction as the invariant product WtScale(i) * VecScale. rfactor() splits the now scale-free reduction by block, and change_type(Int(32)) retypes the innermost per-block dot product so CodeGen_ARM matches it to SDOT. The generated assembly contains sdot instructions, the Hoisted/PlainRfactor numerical cross-check passes, and the composed schedule is several times faster than the non-hoisting variant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…elf_reference A prior rebase silently merged this call site as a 3-argument Call::make() after the Function overload gained a required follow_global_wrappers parameter. This is a Func's self-reference to the rfactor intermediate, so it must not follow global wrappers, matching the other self-reference call sites. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alexreinking
force-pushed
the
alexreinking/rfactor-hoisting
branch
from
August 16, 2026 03:41
79fbc49 to
e588356
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a scheduling directive similar to
rfactorthat hoists invariant factors according to a detected distributive (semiring) law into the write-back part of the accumulation step. This enables writing straightforward quantized kernels in the algorithm language and factoring them into efficient kernels.This is meant to work towards optimizing quantized mat-mul and mat-vec kernels (especially those from GGML).
Checklist