Skip to content

Fixed #1111 Add branching logic for sdp #1200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
NimaSarajpoor wants to merge 6 commits into
base: main
Choose a base branch
from
+102 −10
Open

Fixed #1111 Add branching logic for sdp #1200

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b2221b1
add branching logic to sdp func in core
NimaSarajpoor Aug 25, 2026
5450557
add test function to improve coverage
NimaSarajpoor Aug 25, 2026
61bd5b7
Merge branch 'stumpy-dev:main' into sdp_branching_logic
NimaSarajpoor Sep 5, 2026
d01573d
Use closure to allow user to set the values of some variables
NimaSarajpoor Sep 6, 2026
a3538d1
avoid using hard-coded value
NimaSarajpoor Sep 6, 2026
6a48340
improve docstrings
NimaSarajpoor Sep 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions stumpy/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"STUMPY_MAX_P_NORM_DISTANCE": np.finfo(np.float64).max,
"STUMPY_MAX_DISTANCE": np.sqrt(np.finfo(np.float64).max),
"STUMPY_EXCL_ZONE_DENOM": 4,
"STUMPY_NJIT_SDP_Q_LENGTH": 128, # 2 ** 7
"STUMPY_FASTMATH_TRUE": True,
"STUMPY_FASTMATH_FLAGS": {"nsz", "arcp", "contract", "afn", "reassoc"},
"STUMPY_FASTMATH_FASTMATH._ADD_ASSOC": True,
Expand All @@ -38,6 +39,7 @@
STUMPY_MAX_P_NORM_DISTANCE = _STUMPY_DEFAULTS["STUMPY_MAX_P_NORM_DISTANCE"]
STUMPY_MAX_DISTANCE = _STUMPY_DEFAULTS["STUMPY_MAX_DISTANCE"]
STUMPY_EXCL_ZONE_DENOM = _STUMPY_DEFAULTS["STUMPY_EXCL_ZONE_DENOM"]
STUMPY_NJIT_SDP_Q_LENGTH = _STUMPY_DEFAULTS["STUMPY_NJIT_SDP_Q_LENGTH"]
STUMPY_FASTMATH_TRUE = _STUMPY_DEFAULTS["STUMPY_FASTMATH_TRUE"]
STUMPY_FASTMATH_FLAGS = _STUMPY_DEFAULTS["STUMPY_FASTMATH_FLAGS"]

Expand Down
100 changes: 90 additions & 10 deletions stumpy/core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -648,24 +648,104 @@ def check_window_size(m, max_size=None, n=None):
warnings.warn(msg)


def sliding_dot_product(Q, T):
def make_sliding_dot_product(boundaries=None, default_func=None):
"""
Calculate the sliding window dot product.
A closure to compute the sliding dot product that allows users
to use different methods in different cases

Parameters
----------
Q : numpy.ndarray
Query array or subsequence
boundaries : nested list, default None
A list of items, where each item is a list of 3 elements:
* index 0: (LB_m, UB_m)
* index 1: (LB_n, UB_n)
* index 2: func
where, fucn is the sdp function for computing

@seanlaw seanlaw Sep 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

func not fucn

the sliding dot product of ``Q`` and ``T``, when
m=len(Q) falls into range [LB_m, UB_m], and n=len(T)
falls into range [LB_n, UB_n].
When this is None (default), it will automatically be set to
the following value:
[
[
(3, config.STUMPY_NJIT_SDP_Q_LENGTH),
(3, np.inf),
sdp._njit_sliding_dot_product,
]
]

T : numpy.ndarray
Time series or sequence
default_func : class 'function', default None
A callable object that is used for computing
the sliding dot product between a query `Q` and
time series `T`. This is used when len(Q) and len(T)
values do not fall into any boundary provided
in `boundaries`. When None (default), this will
automatically be set to `sdp._pyfftw_sliding_dot_product`
if available. If not, it will be automatically set to
`sdp._sliding_dot_product`.

Returns
-------
output : numpy.ndarray
Sliding dot product between `Q` and `T`.
"""
return sdp._sliding_dot_product(Q, T)
sliding_dot_product : callable
A callable object that computes the sliding dot product between ``Q``
and ``T`` using different methods based on len(Q) and len(T). It
internally checks the boundary in `boundaries` and choose a function

@seanlaw seanlaw Sep 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"chooseS"

"""
stumpy_default_func = sdp._sliding_dot_product
if sdp.PYFFTW_IS_AVAILABLE: # pragma: no cover
stumpy_default_func = sdp._pyfftw_sliding_dot_product

if default_func is None:
default_func = stumpy_default_func

if boundaries is None:
boundaries = [
# [
# (LB_m, UB_m),
# (LB_n, UB_n),
# func
# ]
[
(3, config.STUMPY_NJIT_SDP_Q_LENGTH),
(3, np.inf),
sdp._njit_sliding_dot_product,
],
]

def sliding_dot_product(Q, T):
"""
Compute the sliding dot product between ``Q`` and ``T``
by using different methods according to len(Q) and len(T)

Parameters
----------
Q : numpy.ndarray
Query array or subsequence.

T : numpy.ndarray
Time series or sequence.

Returns
-------
out : numpy.ndarray
Sliding dot product between ``Q`` and ``T``.
"""
m = len(Q)
n = len(T)
if m == n:
return np.array([np.dot(Q, T)])

for item in boundaries:

@seanlaw seanlaw Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's just something about this that doesn't sit right with me. Here's what I am seeing:

  1. I get that when we do sliding_dot_product = make_sliding_dot_product() (let's refer to this left side as sdp1), you are setting sliding_dot_product to the sliding_dot_product function that is being returned from inside of the closure (let's refer to this inside-of-the-closure-function as sdp2)
  2. However, the contents of the sliding_dot_product function inside of the closure (sdp2) are never executed until you call sdp1.
  3. This means that if you have many Q and T pairs, it is performing boundary comparisons every time rather than performing an O(1) lookup
  4. Similarly, if you have many boundaries to check, then this would be slow

So, while the logic is sound, this point:

Doesn't this mean that every time somebody passes in the boundaries then the sets of bounds get loaded every time you call this function?

is still relevant. Yes, the sets of boundaries are supplied/embedded once into sdp2 BUT each call to sdp2 triggers a slow iterative search. Is there a cheap time AND space efficient way to create an O(1) function lookup given m and n? This for loop isn't the right approach. If we have an O(1) function lookup then it's really lightweight and I wouldn't mind going the config route because 99.9% of users will use our default.

Having said that, are we going to have a way for users to run a function and come up with boundaries and functions that are best suited for their hardware?

@NimaSarajpoor NimaSarajpoor Sep 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, while the logic is sound, this point:

Doesn't this mean that every time somebody passes in the boundaries then the sets of bounds get loaded every time you call this function?

is still relevant. Yes, the sets of boundaries are supplied/embedded once into sdp2 BUT each call to sdp2 triggers a slow iterative search.

Right, I misunderstood your point before.

Is there a cheap time AND space efficient way to create an O(1) function lookup given m and n? This for loop isn't the right approach. If we have an O(1) function lookup then it's really lightweight

Let's replace that for-loop with choose_sdp_func, a function that gets m and n as inputs and return a sdp function as output, and also let's get rid of closure. So, we can have a regular function like this:

def sliding_dot_product(Q, T, choose_sdp_func=None):
    if m == n:
        return np.array([np.dot(Q, T)])
 
    if choose_sdp_func is None: 
        # STUMPY DEFAULT
    
    sdp_func = choose_sdp_func(m, n)  # branching logic
    return sdp_func(Q, T)

Is it a bad design? This allows [advanced] users to use different ways to implement branching logic for their choose_sdp_func function:

  • A dictionary, keyed by (m,n), and the values are sdp functions. It is O(1)
  • Or, if-else logic
  • Or, boundaries.
  • etc

We can always add a caching mechanism on top of the function choose_sdp_func to make sure it returns output in O(1) for a repeated input (m, n). Regarding STUMPY DEFAULT for choose_sdp_func, we can go with a simple if-else logic.

As a side, regarding the following question you raised before in another comment:

the question remains, how do we allow the flexibility to swap out or modify the if/else logic in that case as well? We should think about this..

I think the proposal should address that

Having said that, are we going to have a way for users to run a function and come up with boundaries and functions that are best suited for their hardware?

Finding "boundaries" is tricky. I have a narrower scope (and, tbh, simpler solution 😅) in mind... a function that gets (m, n) and a list of sdp functions as input, and returns the best one for the provided (m, n).

@NimaSarajpoor NimaSarajpoor Sep 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finding "boundaries" is tricky

Okay, I think it might be tricky for most users too! So, maybe they appreciate if we just come up with a function that helps them identify boundaries approximately. For instance, we just explore cases where m and n are exact power of two.

@seanlaw seanlaw Sep 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am mostly interested in what the STUMPY_DEFAULT choose_sdp_func would look like and how efficient it would be?

(LB_m, UB_m), (LB_n, UB_n), func = item
if LB_m <= m <= UB_m and LB_n <= n <= UB_n:
return func(Q, T)

return default_func(Q, T)

return sliding_dot_product


sliding_dot_product = make_sliding_dot_product()


@njit(
Expand Down
10 changes: 10 additions & 0 deletions tests/test_core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,16 @@ def test_sliding_dot_product(Q, T):
npt.assert_allclose(cmp_mp, ref_mp, atol=1.5e-07)


def test_sliding_dot_product_large_Q():
# Set len(T) > len(Q) > config.STUMPY_NJIT_SDP_Q_LENGTH,
# so that it triggers a certain code flow
Q = rng.RNG.rand(config.STUMPY_NJIT_SDP_Q_LENGTH + 1)
T = rng.RNG.rand(config.STUMPY_NJIT_SDP_Q_LENGTH + 2)
ref_mp = naive.rolling_window_dot_product(Q, T)
cmp_mp = core.sliding_dot_product(Q, T)
npt.assert_allclose(cmp_mp, ref_mp, atol=1.5e-07)


def test_welford_nanvar():
T = rng.RNG.rand(64)
m = 10
Expand Down
Loading