Skip to content

fix(tests): CI-safe timing threshold for partial-cache latency test - #982

Merged
paddymul merged 1 commit into
mainfrom
fix/ci-partial-cache-timing-threshold
Sep 28, 2026
Merged

paddymul merged 1 commit into
mainfrom
fix/ci-partial-cache-timing-threshold

Conversation

@paddymul

Copy link
Copy Markdown
Collaborator

Summary

  • CI's Python / Test (Max Versions) (3.13) job failed on main (run 36148490455): test_huge_dataframe_partial_cache_scenario asserted initial_time < 0.5 and got 0.7584092617034912.
  • That assertion measures wall-clock widget-construction latency, which is sensitive to shared CI-runner scheduling variance, not the cache behavior under test. Locally the same code path consistently finishes in well under 0.1s across a dozen runs (traced with the executor's own logging - the only variable-cost step, one-time mp_timeout calibration, is paid by an earlier test in the same file and was already warm by the time this test ran).
  • No commit between the last green Checks run and the failing one touched buckaroo/file_cache/ or the widget construction path, so this isn't a regression - it's a hardcoded absolute-time bound that doesn't hold up under runner load.
  • Widened the bound to 3.0s, matching the CI_TIMEOUT headroom this same test directory (tests/unit/file_cache/mp_test_utils.py) already uses for the same class of fork/scheduling-bootstrap variance.

Test plan

  • uv run pytest tests/unit/file_cache/test_partial_cache_computation.py -vv (all 3 tests pass)
  • uv run pytest tests/unit/ -m "not slow" (1171 passed, 5 skipped)
  • CI green on this PR

🤖 Generated with Claude Code

… timing threshold

CI's Python / Test (Max Versions) (3.13) job failed on main
(run 36148490455) with initial_time=0.758s against a 0.5s bound. The
assertion measures wall-clock widget-construction time, which is
sensitive to shared-runner scheduling variance rather than the cache
behavior itself - locally the same path consistently finishes in well
under 0.1s. No file_cache logic changed between the last green Checks
run and this one, so the failure isn't a regression. Widen the bound to
3.0s, matching the CI_TIMEOUT headroom this test directory already uses
for the same class of scheduling-bootstrap variance (mp_test_utils.py).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📦 TestPyPI package published

pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.8.dev36458802452

or with uv:

uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.8.dev36458802452

MCP server for Claude Code

claude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.15.8.dev36458802452" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table

📖 Docs preview

🎨 Storybook preview

@paddymul
paddymul added this pull request to the merge queue Sep 28, 2026
Merged via the queue into main with commit 3564f60 Sep 28, 2026
28 checks passed

This branch was successfully deployed

1 active deployment
testpypi — b3347388 Deployed Sep 28, 2026 by paddymul via Publish to TestPyPI #1585
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant