Repository navigation
selection-misses: tell a likely flake from a likely miss by cifail's history (#60) - #70
Merged
Merged
Conversation
…history (#60) Each miss is looked up in the store analyze records into. One that failed before on other commits comes after those that never did, marked with how often and whether it is flaky; --json has history. The commit under test (--commit, HEAD by default) is not history, --repo scopes it, --no-history skips it, and a store that can't be read is a warning. The action passes GITHUB_SHA and shows the history on each miss. Also: CliHarness gives each run an empty stdin. A command given no path reads stdin when redirected, and under a shell that left the test host's pipe open the suite hung. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #60.
Every candidate miss used to be equally suspect, and ruling out a flake was left to the reader, even though cifail records every failing test it analyses.
SelectionMisses.WithHistory(misses, rows, repoId, commit)attaches each miss's earlier failures (TestStat, fromTestFlakeComputer) and orders the misses with likely misses (no earlier failure) first, then by how often each failed before.selection-missesnow takes the store options (StoreSettings, the same storeanalyzerecords into), plus:--commit(default HEAD) and--repo(default the current repo).--no-history.--jsonaddshistory: {failures, distinctDays, flaky, lastSeen}.--commit "$GITHUB_SHA"and appends ", failed Nx on D day(s) before, likely flaky" to each miss. It needsCIFAIL_DB_*for a shared store, which the action already passes into the container. The README says so.Also fixed here: the Cli test suite hanging locally.
CliHarnessreplaced stdout and stderr but not stdin. A command given no path reads stdin whenever it is redirected, so under a shell that left the test host's stdin pipe open,ReadToEnd()blocked and the suite hung. It happened three times in this session; CI passes/dev/null, so it never hangs there. Each run now gets an empty stdin of its own. Three runs with stdin inherited from that shell all passed.Tests: a Core test for history, ordering, commit exclusion, repo filtering and arguments, and a CLI test that seeds the store with
analyzeand checks the text, the JSON and--no-history. Locally Core passes 544/544 and Cli 204/204.🤖 Generated with Claude Code