Repository navigation
fix: keep each run's status counts on the run - #362
Merged
Merged
Conversation
The runs list counted the executions of every run on the page, by status, on each request: suite size times page size. At 20,000 tests that was 4.8 s at p95. A run is exactly one ingest batch, and reprocessing its idempotency key replaces its executions, so its counts are the statuses of the batch. The worker now writes them on the run in the same upsert, which makes a redelivery replace them rather than add to them. Moving executions between identities never moves them between runs, so a restitch leaves them alone. The migration backfills existing runs from their executions: 4 s for 15 M executions, and every run matched a recount. The runs list reads the stored counts: 1.9 ms p95 at 20,000 tests, 0.9 ms at 2,000. The run page counts the executions it already loads rather than querying them twice. A run whose executions retention has pruned now keeps its counts instead of showing zeros. Closes #357
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #357
Why the counts can live on the run
A run is exactly one ingest batch: the processor upserts it by idempotency key and replaces its executions by ordinal, deleting any left over. So a run's counts are the statuses of the batch that last wrote it. The worker computes them from the batch and sets them in the same
run.upsert(create and update). A redelivery therefore replaces the counts. Identity merges and splits move executions between identities, never between runs, so they cannot change a run's counts.Changes
rungainspassed_count,failed_count,skipped_countandflaky_count; the migration backfills them from existing executions.listRunsreads them instead ofgroupByovertest_execution.getRuncounts the executions it already loads, one query fewer.@flakemetry/dbexportscountStatusesandrunCountFields, used by the processor, andrefreshRunCounts, which recounts a project's runs. The demo seed and test fixtures that write executions directly use it.docs/configuration.md: a run whose raw executions retention has pruned keeps its counts on the runs list, where it used to show zeros. Its detail page no longer lists executions, as before.Verified
Benchmark (ADR-0005), runs list p95: 4,760 → 1.9 ms at 20,000 tests, 23.5 → 0.9 ms at 2,000. Run detail 141 → 123 ms at 20,000 tests.
Migration on the benchmark databases: 1 s at 1.5 M executions, 4 s at 15 M. 0 runs differ from a recount by status.
Processor tests:
Dropping the counts from the upsert's
createfails the first test, and dropping them fromupdatefails the second.pnpm demoseed: 24 runs, counts summing to all 201 executions, no mismatch.The
read-pathslistRuns test named "with counts" never checked them; it now does.Suites: queries 289, worker 125, api 121, db 19 and web 47 pass with the database; lint and typecheck pass; schema drift clean.