fix(code-review): post the review comment from the workflow, not the model - #27
Closed
rathorevaibhav wants to merge 1 commit into
Closed
rathorevaibhav wants to merge 1 commit into
rathorevaibhav wants to merge 1 commit into
Conversation
…model
The action auto-detects "agent" mode on pull_request events. Agent mode posts
nothing to the PR on its own, and use_sticky_comment is inert there. The review
only reached the PR when the model happened to call `gh pr comment` itself,
which it did on 4 of the last 14 runs. The other 10 finished green with the
full review sitting in the job summary and no comment on the PR.
Take the model out of the delivery path. A new step reads the action's
execution_file output, pulls the closing message, and posts it. The prompt now
tells the model to make its final message the review and to post nothing, and
`gh pr comment` is off the default tool list.
The step also honours use_sticky_comment for real by updating its own earlier
comment, matched on a hidden marker and a bot author. It uses GITHUB_TOKEN
because the action revokes its app token in its own final step.
Also fix guidelines_dir, which has never reached the model. It was interpolated
inside the default value of another workflow_call input, where GitHub cannot
resolve it, so every prompt so far has read "Check if the directory `` exists".
The default prompt now carries a {{GUIDELINES_DIR}} placeholder that a job step
substitutes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Closing for now. The team's call is that the missed comments are infrequent enough not to warrant this change yet. Leaving the branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Two bugs in
claude-code-review.yml, both silent. The workflow goes green either way.1. The review was often never posted
anthropics/claude-code-actionauto-detects agent mode on apull_requestevent. Agent mode posts nothing to the PR by itself, anduse_sticky_commentdoes nothing there. The review reached the PR only when the model chose to rungh pr commenton its own.It usually did not. Across the last 14 labelled runs on
coloredcow-admin/coloredcow, only 4 calledgh pr comment:gh pr comment?The 10 misses all finished
subtype: successwith the complete review in the run'sresultfield, visible only in the Actions job summary. PR #1470 still has no review on it. On PR #1462 a teammate copy-pasted the summary in by hand three minutes after the run.Fix: take the model out of the delivery path. A new
Post review commentstep reads the action'sexecution_fileoutput, pulls the closing message withjq, and posts it. The prompt now tells the model to make its final message the review and to post nothing itself, andgh pr commentis off the default tool list.The step also makes
use_sticky_commentmean something again. It finds its own earlier comment by a hidden<!-- claude-code-review -->marker plus a bot author and updates that, so re-running the label does not stack duplicates.It posts with
GITHUB_TOKEN, so comments come fromgithub-actions[bot]rather thanclaude[bot]. That is deliberate: the action revokes its own app token in its last internal step, sosteps.claude-review.outputs.github_tokenis already dead by the time a following step runs.2.
guidelines_dirnever reached the modelThe default
promptinterpolated${{ inputs.guidelines_dir }}inside the default value of anotherworkflow_callinput. GitHub resolves input defaults before the caller's inputs exist, so it rendered empty. Every prompt ever sent has said:No review has read a project's guidelines. Not once.
Fix: the default prompt carries a literal
{{GUIDELINES_DIR}}placeholder, and a newBuild review promptstep substitutes it where inputs do resolve. Callers passing a custompromptcan use the same placeholder.Also corrected a stale path in
automated-code-review/README.md, which pointed teams atdocs/code-review-guidelines.md.setup.shwritesdocs/code-review/review-guidelines.md.Testing
Replayed the real PR #1505 execution output (39 turns, rebuilt from the run log) through the new step with
ghstubbed:Also checked the sticky lookup against a comment with a
nullbody and against a human who pasted the marker. The old expression crashed onnull; the hardened one skips both and matches only the bot's comment.Note for reviewers
Sanitize inputsstill interpolates${{ inputs.allowed_tools }}straight into a shell script. That is a script-injection vector and predates this change. I left it alone to keep this PR to the two diagnosed bugs. Worth a follow-up.🤖 Generated with Claude Code