[Dong1017]

Refs #215

Fix the two runtime identity mismatches observed in runs 40683638660 and 40683653311. Both planned successfully but skipped execution; no provider calls or reservations occurred.

• Compare the trusted implementation with authenticated current main, independently of the PR's historical comparison base, before reservation and each provider request.
• Validate pull_request_target run.head_sha against the PR head; retain repository/workflow checks and the full original source-plan fingerprint check before importing results.
• Add sanitized actual run metadata fixtures and regression tests. No budget, provider, credential, label or workflow-permission change.

flowchart LR
 A[Authenticated main] --> B[Trusted implementation]
 C[PR base and head] --> D[Frozen comparison]
 E[Actions run PR head] --> D
 B --> F[Reservation and request checks]
 D --> F
 F --> G[Original plan authenticated before publication]

Loading

Patch independently reviewed before refresh; latest-main refresh has no overlapping upstream changes. Refreshed-base full suite: 2,310 passed, 3 skipped, 183 subtests. Focused suite: 291 passed; independent integration rerun: 69 passed. Whole-repository Ruff and diff checks passed. Independent review verified all 12 published file hashes against the reviewed patch. No live model/hardware result is claimed by these offline tests.

AI assistance: OpenAI Codex. Exact underlying model/effort identifiers were not exposed by this runtime; not inferred. This corrective change touches tooling, tests and documentation, not knowledge-source authoring.

[github-actions]

Skill evaluation

Automatic execution / cumulative PR budget: {"reason":"Audited PR ledger missing, unavailable or unsafe","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}

Advisory: obtain missing evidence

Retained evidence availability: planned / unexecuted. No approval or merge authorization is issued.

• Retained evidence is planned / unexecuted.
• No supported comparison cases were selected.
• Uncovered/unsupported scope needs a case or explicit reviewer disposition.

PLANNED: execution is not authorized by knowing this hash.
Outcomes: {"fix":0,"inconclusive":0,"missing_pair":0,"regression":0,"unchanged_correct":0,"unchanged_incorrect":0}
Earlier revision results are stale for this plan.

Selected comparisons: 0 · Estimated task calls: 0.
Uncovered/unsupported changes: 12. Gaps never count as passes.

Applicable only while these revisions and inputs match the current PR.
Paid execution requires trusted label authorization and cumulative PR reservation. Unknown accounting blocks new paid calls; planning is not execution.
Pipeline evidence only; offline fixed outputs do not establish skill efficacy.
Evidence and logs · Retention: 30 days.
Outcomes are historical; verifiable only before Unix time 1795787108 while the linked artifact is available. Expired evidence is unverified.

Plan bindings, budget, frozen exam and advisory details

Unchanged-correct is valid neutral evidence, not incremental benefit.
Content/source review and authorship admission remain separate.
Trusted regrading checks retained answers, not model/hardware provenance or a hostile executor; this is not an efficacy certification.

Per-case outcomes (discovery excluded; regressions and gaps first):

[]

Showing at most 20 of 0 pairs; the evidence link contains all outcomes, including omitted regressions/gaps.

Plan SHA-256: 899a345d12d65236ceff124ad48f811b3e2fc0d80dcdd995de2e15ed16ecbf6f
Base: 2c30fbbbc12af2eb086b23340089e3371139c245 · Head: 74379aa3ae85380a75ef5d6be6a45dd2002a59bd
Trusted implementation: 1f471680a81a650b75b4067f9f4db2c77b192e3b

Model/settings: {"effort":"not_applicable","name":"claude-opus-5-5"}
Budget: {"comparisons":0,"estimated_calls":0,"limits_per_comparison":{"campaign_seconds":600,"max_task_calls":2,"repeats":1,"timeout_seconds":180,"token_stop_new_trials":200000},"provider_config_sha256":"253cf88a7ff04311fd1dae897cb68a8c986f136e837dd00dbe5f8872bd094b5f","reservation":null}

Skill	Comparison	Cases	Estimated calls

Provider/settings: {"client":{"api_version":"2023-06-01","endpoint":"https://yibuapi.com/v1/messages","kind":"anthropic-messages-agent-v1","max_output_tokens":4096,"max_read_bytes":32768,"max_request_bytes":131072,"max_requests":6,"max_tool_calls":12,"provider":"yibuapi"},"limits":{"token_stop_new_trials":200000},"model":{"effort":"not_applicable","name":"claude-opus-5-5"},"pricing":{"currency":"USD","expires_at":1793271228,"group":"default","max_input_tokens":1000000,"model":"claude-opus-5-5","rate_evidence":"https://github.com/InferMatrixLab/InferRSI/issues/215#issuecomment-6467531341","rates":{"cache_read":"0.4","cache_write":"10","input":"8","output":"40"},"request_fee":"0","unit":"per_million_tokens"},"spend_limit":{"amount":"200","currency":"USD"}}

Uncovered / unsupported paths (12)

• uncovered: "docs/skill-eval-pr.md"
• uncovered: "tests/fixtures/skill_eval_live_identity/README.md"
• uncovered: "tests/fixtures/skill_eval_live_identity/plan-identity-40683638660.json"
• uncovered: "tests/fixtures/skill_eval_live_identity/plan-identity-40683653311.json"
• uncovered: "tests/fixtures/skill_eval_live_identity/run-40683638660.json"
• uncovered: "tests/fixtures/skill_eval_live_identity/run-40683653311.json"
• uncovered: "tests/fixtures/skill_eval_pr_support.py"
• uncovered: "tests/test_skill_eval_auto.py"
• uncovered: "tests/test_skill_eval_planning_boundaries.py"
• uncovered: "tests/test_skill_eval_pr.py"
• uncovered: "tools/skill_eval_auto.py"
• uncovered: "tools/skill_eval_pr.py"

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-28T13:45:48.636656Z	74379aa	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Dong1017]

Independent review found no blocking defects at 74379aa. All 12 published file hashes match the reviewed patch. Full local suite: 2,310 passed, 3 skipped; independent integration rerun: 69 passed. Prospective integration with current main 1f47168 passed 291 controller/security and 468 package/index tests.

Ready CI 40699199811 passed all four jobs; declaration run 40699457197 passed. The separate automated code review completed without findings. Runtime metadata regression checks cover both the historical comparison base and Actions PR-head identities while preserving original-plan authentication. Actual paid execution and report delivery still require the approved rollout; this comment records software verification, not efficacy or issue acceptance.