Skip to content

Everything merged since v0.20.0 went in unreviewed, and the check reads green #293

Description

@LKSNDRTMLKV

Asked to investigate the security findings sitting on main before the 0.21.0 cut. There are none — and the reason is worse than a list of findings would have been.

Nothing on main was reviewed

Every pull request merged since the v0.20.0 cut reports the same CodeRabbit status:

CodeRabbit    pass    0    Review skipped: excluded by label configuration

Twelve of thirteen. The exception is #283, and only because it was labelled a few hours before merging.

merged since v0.20.0 reviewed
#271, #272, #273, #276, #281, #284, #285, #286, #288, #289, #290, #291 ❌ none
#287, #292, #283 ✅ labelled and read

So the honest answer to "what security findings went in" is: unknown, because nothing looked. A green CodeRabbit check on those twelve means "no review ran", not "a review found nothing" — and the two render identically in the checks list.

This is the configured behaviour, and .coderabbit.yaml predicted it

auto_review.enabled: false with an opt-in review-ready label. The config's own comment says what happens when nobody labels:

when no review happens at all, those checks are skipped with it and the CodeRabbit status passes vacuously, which is worse than not having them

That is the state. The setting is defensible — the open-source plan rate-limits reviews and one-per-push would spend the budget on typo fixes — but it depends on a human remembering a label, and over thirteen merges that held once.

What is going into 0.21.0 unread

git diff --stat v0.20.0 main76 files, +3924 / −607.

Several meet CLAUDE.md's own "worth a review" bar:

None of that is an allegation that something is wrong. It is that nobody has looked, at the things the repo's own rules say to look at.

Proposed remedy, before the cut

A merged PR cannot be reviewed retroactively, but the accumulated diff can:

  1. branch at the tag — git branch audit/v0-20-0-baseline v0.20.0 and push it;
  2. open a PR with base audit/v0-20-0-baseline, head main, so the diff is everything 0.21.0 adds;
  3. label it review-ready, take the one pass over all 76 files;
  4. act on whatever it finds, then close without merging — it exists to be read, not landed;
  5. delete the baseline branch.

One review, one rate-limit slot, covering everything in the release.

And the smaller fix that stops the recurrence

The label is load-bearing and invisible when forgotten. Options, cheapest first:

  • a required status check, or a simple CI step, that fails a PR carrying neither review-ready nor an explicit @coderabbitai review — turning a silent skip into a visible one;
  • or review_status: true, so the skip at least posts a comment instead of only a green tick.

🚨 Whatever is chosen, the thing to fix is that "not reviewed" and "reviewed clean" look the same in the checks list. Three separate variants of that cost time today: this one, a stacked PR whose base disabled reviews (#292), and a PR whose findings were all stale but still displayed (#280).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    securitySecurity-relevant issuesurface/docs-onlyContained to documentation; no code depends on ittype/decisionNeeds a call; analysis alone will not close iturgency/nextBlocks work already scheduled

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions