Skip to content

Add Hypothesis property-based tests, fix 3 bugs found - #555

Merged
aarmey merged 2 commits into
mainfrom
add-property-based-tests
Sep 12, 2026
Merged

aarmey merged 2 commits into
mainfrom
add-property-based-tests

Conversation

@aarmey

@aarmey aarmey commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds ddmc/tests/test_properties.py: property-based tests (via Hypothesis) for the core sequence-distance and preprocessing helpers in ddmc.pam250, ddmc.binomial, ddmc.motifs, and ddmc.logistic_regression. Instead of hand-picked examples, each test generates many random-but-valid inputs and checks an invariant that should hold for all of them (symmetry, round-tripping, matching a reference implementation, etc.).

Adds hypothesis to the dev dependency group.

Running these tests turned up three real bugs, all now fixed in this PR:

  1. get_pam250_scores silently overflowed int8. The pairwise PAM250 similarity matrix was accumulated into an int8 array. A length-11 peptide's self-similarity score can exceed int8's 127-value ceiling for highly self-similar residues — e.g. 11 tryptophans score 11 * 17 = 187, which wrapped around to -69. This meant highly conserved/repetitive motifs (poly-W, poly-C) could be scored as less similar to themselves than to unrelated sequences, silently corrupting the PAM250 distance metric used by DDMC(distance_method="PAM250").

    >>> get_pam250_scores(["WWWWWWWWWWW", "CCCCCCCCCCC"])
    array([[ -69,  -88],
           [ -88, -124]])   # before the fix

    Fix: widen the accumulator from int8 to int32 (ddmc/pam250.py).

  2. compute_control_pssm crashed on the exact input its real caller produces. It looked up each residue with AAlist.index(aa), which raises ValueError for a lowercase letter. But ddmc.binomial.BackgroundSeqs — the only place compute_control_pssm is actually called from, via DDMC.get_pssms(PsP_background=True) (ddmc/clustering.py:281-282) — always lowercases the phosphoacceptor of the sequences it returns. So DDMC.get_pssms(PsP_background=True) was broken end-to-end, raising ValueError: 'y' is not in list (or 's'/'t').
    Fix: upper-case each residue before the AAlist lookup (ddmc/motifs.py).

  3. normalize_cluster_centers centered along the wrong axis. Its docstring says it zero-means each cluster (column) across samples (rows) — the usual per-feature standardization before classification. But the implementation wrapped StandardScaler(with_std=False).fit_transform in centers.T ... .T, which instead zero-meaned each sample across that sample's clusters, leaving the per-cluster mean across samples unchanged (and generally nonzero).
    Fix: call StandardScaler(with_std=False).fit_transform(centers) directly, without the transposes (ddmc/logistic_regression.py).

The other seven property tests passed from the start and pin down invariants worth protecting going forward — fast_position_weight_matrix numerically matching the Biopython reference PWM, GenerateBinarySeqID's one-hot encoding round-tripping to the original sequence, CountPsiteTypes accounting for every input sequence, and PAM250-score symmetry. All ten tests, including the three that exposed bugs, now pass as ordinary regression tests (no xfail markers needed).

Test plan

  • uv run pytest ddmc/tests/test_properties.py -v — 10 passed
  • uv run pytest ddmc/tests/ -q --deselect ddmc/tests/test_datasets.py — 54 passed (no regressions; test_datasets.py deselected as it needs network-fetched data not available in this environment)
  • uv run ruff check ddmc/ — clean

🤖 Generated with Claude Code

aarmey and others added 2 commits September 12, 2026 08:08
Adds ddmc/tests/test_properties.py with property-based tests (via
Hypothesis) covering the core sequence-distance and preprocessing
helpers in ddmc.pam250, ddmc.binomial, ddmc.motifs, and
ddmc.logistic_regression.

These tests found three real bugs, kept as strict xfail so CI stays
green while documenting them (and will loudly flag a regression if the
underlying bug is fixed without updating the test):

- get_pam250_scores accumulates pairwise scores in an int8 array, which
  silently overflows for highly self-similar peptides (e.g. poly-W or
  poly-C), producing wrong (sometimes negative) similarity scores.
- compute_control_pssm crashes with ValueError on a lowercase
  phosphoacceptor character, which is exactly what its real caller
  chain produces: BackgroundSeqs lowercases the phosphoacceptor, and
  DDMC.get_pssms(PsP_background=True) passes that output straight in.
- normalize_cluster_centers centers along the wrong axis: the
  docstring says it zero-means each cluster across samples, but the
  double-transpose around StandardScaler actually zero-means each
  sample across clusters instead.

The remaining property tests pass and pin down invariants (PWM
symmetry/equivalence to the Biopython reference, one-hot encoding
round-trips, phosphosite-type counting) that should keep holding as
this code changes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- get_pam250_scores: widen the pairwise-score accumulator from int8 to
  int32 so it no longer silently overflows (and wraps to a negative
  score) for highly self-similar peptides such as poly-tryptophan or
  poly-cysteine motifs.
- compute_control_pssm: upper-case each residue before the AAlist
  lookup, so it no longer crashes on the lowercased phosphoacceptor
  that BackgroundSeqs (its actual caller, via
  DDMC.get_pssms(PsP_background=True)) always produces.
- normalize_cluster_centers: drop the erroneous transpose-around-
  StandardScaler, which was centering each sample across its clusters
  instead of centering each cluster across samples as documented.

Turns the corresponding xfail(strict=True) tests in
ddmc/tests/test_properties.py into ordinary passing regression tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@aarmey aarmey changed the title Add Hypothesis property-based tests; found 3 bugs Add Hypothesis property-based tests, fix 3 bugs found Sep 12, 2026
@aarmey
aarmey merged commit b1db60f into main Sep 12, 2026
2 checks passed
@aarmey
aarmey deleted the add-property-based-tests branch September 12, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant