Skip to content

Add benchmarking test framework and improve featurization performance - #48

Open
d33bs wants to merge 19 commits into
WayScience:mainfrom
d33bs:hortuscute
Open

Add benchmarking test framework and improve featurization performance#48
d33bs wants to merge 19 commits into
WayScience:mainfrom
d33bs:hortuscute

Conversation

@d33bs

@d33bs d33bs commented Aug 5, 2026

Copy link
Copy Markdown
Member

Description

This PR adds a benchmarking test-based framework and improves feature extraction performance generally. I focused on making relatively boring or safe changes which don't interfere with the static output we already have achieved in this project. Below is a table of what we're seeing.

Synthetic data:

┌───────────────────┬─────────┬─────────┬─────────┐
│      feature      │  main   │ branch  │ speedup │
├───────────────────┼─────────┼─────────┼─────────┤
│ neighbors         │ 0.162 s │ 0.009 s │ ~18×    │
├───────────────────┼─────────┼─────────┼─────────┤
│ intensity         │ 0.337 s │ 0.071 s │ ~4.7×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ volume_size_shape │ 0.082 s │ 0.043 s │ ~1.9×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ texture           │ 0.802 s │ 0.555 s │ ~1.4×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ colocalization    │ 0.458 s │ 0.383 s │ ~1.2×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ granularity       │ 0.113 s │ 0.104 s │ ~1.08×  │
└───────────────────┴─────────┴─────────┴─────────┘

Real-world (small CP tutorial) data:

┌───────────────────┬─────────┬─────────┬─────────┐
│      feature      │  main   │ branch  │ speedup │
├───────────────────┼─────────┼─────────┼─────────┤
│ intensity         │ 0.579 s │ 0.087 s │ ~6.7×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ neighbors         │ 0.286 s │ 0.035 s │ ~8.2×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ texture           │ 0.551 s │ 0.288 s │ ~1.9×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ volume_size_shape │ 0.186 s │ 0.087 s │ ~2.2×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ colocalization    │ 0.457 s │ 0.315 s │ ~1.45×  │
├───────────────────┼─────────┼─────────┼─────────┤
│ granularity       │ 1.756 s │ ~1.77 s │ —       │
├───────────────────┼─────────┼─────────┼─────────┤
│ TOTAL             │ 3.82 s  │ 2.57 s  │ ~1.48×  │
└───────────────────┴─────────┴─────────┴─────────┘

What kind of change(s) are included?

  • Documentation (changes docs or other related content)
  • Bug fix (fixes an issue).
  • Enhancement (adds functionality).
  • Breaking change (these changes would cause existing functionality to not work as expected).

Checklist

Please ensure that all boxes are checked before indicating that this pull request is ready for review.

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have searched for existing content to ensure this is not a duplicate.
  • I have performed a self-review of these additions (including spelling, grammar, and related).
  • These changes pass all pre-commit checks.
  • I have added comments to my code to help provide understanding
  • I have added a test which covers the code changes found within this PR
  • I have deleted all non-relevant text in this pull request template.

📚 Documentation preview 📚: https://zedprofiler--48.org.readthedocs.build/en/48/

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@7d76edb). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #48   +/-   ##
=======================================
  Coverage        ?   93.38%           
=======================================
  Files           ?       15           
  Lines           ?     1376           
  Branches        ?        0           
=======================================
  Hits            ?     1285           
  Misses          ?       91           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@d33bs
d33bs marked this pull request as ready for review August 5, 2026 22:29
@d33bs
d33bs requested review from MikeLippincott and gwaybio August 5, 2026 22:29

@MikeLippincott MikeLippincott left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM I left some minor concerns

Comment thread docs/src/benchmarking.md Outdated
Comment thread src/zedprofiler/featurization/intensity.py
Comment thread src/zedprofiler/featurization/intensity.py
Comment thread src/zedprofiler/featurization/neighbors.py
Comment thread tests/featurization/test_neighbors.py
Comment thread tests/benchmarking.py Outdated
Comment thread tests/benchmarking.py Outdated
d33bs and others added 6 commits August 6, 2026 10:56
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants