Skip to content

fix: keep each run's status counts on the run - #362

Merged
AKogut merged 2 commits into
mainfrom
perf/357-run-counts
Sep 30, 2026
Merged

AKogut merged 2 commits into
mainfrom
perf/357-run-counts

Conversation

@AKogut

@AKogut AKogut commented Sep 30, 2026

Copy link
Copy Markdown
Owner

Closes #357

Why the counts can live on the run

A run is exactly one ingest batch: the processor upserts it by idempotency key and replaces its executions by ordinal, deleting any left over. So a run's counts are the statuses of the batch that last wrote it. The worker computes them from the batch and sets them in the same run.upsert (create and update). A redelivery therefore replaces the counts. Identity merges and splits move executions between identities, never between runs, so they cannot change a run's counts.

Changes

  • run gains passed_count, failed_count, skipped_count and flaky_count; the migration backfills them from existing executions.
  • The processor sets them from the batch.
  • listRuns reads them instead of groupBy over test_execution.
  • getRun counts the executions it already loads, one query fewer.
  • @flakemetry/db exports countStatuses and runCountFields, used by the processor, and refreshRunCounts, which recounts a project's runs. The demo seed and test fixtures that write executions directly use it.
  • docs/configuration.md: a run whose raw executions retention has pruned keeps its counts on the runs list, where it used to show zeros. Its detail page no longer lists executions, as before.

Verified

  • Benchmark (ADR-0005), runs list p95: 4,760 → 1.9 ms at 20,000 tests, 23.5 → 0.9 ms at 2,000. Run detail 141 → 123 ms at 20,000 tests.

  • Migration on the benchmark databases: 1 s at 1.5 M executions, 4 s at 15 M. 0 runs differ from a recount by status.

  • Processor tests:

    • counts equal the stored executions' statuses;
    • redelivering the same run twice leaves them unchanged, and redelivering it with different results replaces them.

    Dropping the counts from the upsert's create fails the first test, and dropping them from update fails the second.

  • pnpm demo seed: 24 runs, counts summing to all 201 executions, no mismatch.

  • The read-paths listRuns test named "with counts" never checked them; it now does.

  • Suites: queries 289, worker 125, api 121, db 19 and web 47 pass with the database; lint and typecheck pass; schema drift clean.

The runs list counted the executions of every run on the page, by
status, on each request: suite size times page size. At 20,000 tests
that was 4.8 s at p95.

A run is exactly one ingest batch, and reprocessing its idempotency key
replaces its executions, so its counts are the statuses of the batch.
The worker now writes them on the run in the same upsert, which makes a
redelivery replace them rather than add to them. Moving executions
between identities never moves them between runs, so a restitch leaves
them alone. The migration backfills existing runs from their executions:
4 s for 15 M executions, and every run matched a recount.

The runs list reads the stored counts: 1.9 ms p95 at 20,000 tests, 0.9
ms at 2,000. The run page counts the executions it already loads rather
than querying them twice. A run whose executions retention has pruned
now keeps its counts instead of showing zeros.

Closes #357
@AKogut AKogut added area:db Schema & migrations type:bug Defect in existing behaviour labels Sep 30, 2026
@AKogut AKogut self-assigned this Sep 30, 2026
@AKogut
AKogut merged commit 11857b9 into main Sep 30, 2026
8 checks passed
@AKogut
AKogut deleted the perf/357-run-counts branch September 30, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db Schema & migrations type:bug Defect in existing behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runs list counts every execution of the page's runs on each request

1 participant