Skip to content

fix: sum rollups in Postgres instead of in the application - #361

Merged
AKogut merged 2 commits into
mainfrom
perf/358-rollups-in-sql
Sep 30, 2026
Merged

AKogut merged 2 commits into
mainfrom
perf/358-rollups-in-sql

Conversation

@AKogut

@AKogut AKogut commented Sep 30, 2026

Copy link
Copy Markdown
Owner

Closes #358

getFlakinessCost, getTestLeaderboards, getDailyTrend and getProjectHealthKpis loaded every per-test daily rollup row in their window and summed them in JavaScript. Each now asks Postgres for what it shows:

Now
KPIs one row of sums
daily trend sums grouped by day
leaderboards the top limit rows of each board, joined to their identities
cost sums by day, from which the totals follow; the tests with reruns or flaky occurrences (the only possible offenders), grouped before joining; the quarantined identities' rollups through the primary key. Offenders are still priced and ranked in the application, with the same rounding

Numbers

packages/queries/bench (ADR-0005), p95 ms:

query 2k tests, before → after 20k tests, before → after
cost of flakiness 186 → 8 2,433 → 104
leaderboards 96 → 9 1,301 → 146
daily trend 66 → 6 586 → 61
health KPIs 40 → 8 373 → 74

All four are under the 300 ms target at 20,000 tests.

Same answers

rollup-aggregates-db.test.ts seeds 120 tests over 20 days with random counts, some quarantined, plus a noisy row in another project. It compares each function with a reference that adds the raw rows up the old way.

The first version of that test failed intermittently on cost offenders. Two tests tied on both spend and rerun time, and the reference kept them in insertion order while the query returned them by id. The old code had the same problem: ties came out in whatever order the rows arrived. Rankings now break ties on test id, explicitly in both. Eight consecutive runs pass, and the existing cost tests, the full queries suite (293) and the web suite pass.

The cost panel, the leaderboards, the daily trend and the KPIs fetched
one daily_test_stats row per test per day and added them up in
JavaScript, so their cost grew with suite size times days: 600,000 rows
for 20,000 tests over a month.

Each now asks Postgres for exactly what it shows: sums per day, the top
rows of each leaderboard, and for cost only the tests with reruns or
flaky occurrences, ranked in the application as before. The quarantine
figures start from the quarantined identities rather than joining every
row. Ties in the rankings break on test id instead of on whatever order
the rows arrived in.

p95 at 20,000 tests: cost 2,433 -> 104 ms, leaderboards 1,301 -> 146 ms,
daily trend 586 -> 61 ms, KPIs 373 -> 74 ms. At 2,000 tests the cost
panel went from 186 to 8 ms.

Closes #358
@AKogut AKogut added area:db Schema & migrations type:bug Defect in existing behaviour labels Sep 30, 2026
@AKogut AKogut self-assigned this Sep 30, 2026
@AKogut
AKogut merged commit 8211bc7 into main Sep 30, 2026
6 checks passed
@AKogut
AKogut deleted the perf/358-rollups-in-sql branch September 30, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db Schema & migrations type:bug Defect in existing behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trend and cost panels sum per-test rollups in the application

1 participant