Refresh pinned BC artifacts after full verification

[Sun Haoran (haoranpb)]

Summary

• Follow-up to Pin exact BC container artifact URLs (step 1) #893: resolve the latest public BC artifact once per dataset version, then use that exact candidate in every bug-fix verification matrix job.
• Only after a full successful scheduled run on main, push a pins-only branch if the verified URLs differ from the saved pins; failed and partial runs retain the existing pins. BC Insider URLs are unchanged.
• A PR for the branch must be created manually and undergo the required code-owner review. The workflow neither opens a PR nor requests auto-merge; repository rules prevent a direct update to main.

Notes

• This PR is stacked on Pin exact BC container artifact URLs (step 1) #893 and intentionally excludes the initial pinning change.
• Local tests exercise candidate resolution, URL/cache consistency, config merging, and workflow promotion gating.

[Sun Haoran (haoranpb)]

Copilot this PR should follow the design of the previous #893

The part about take the stored artifact URLs is completed, now focus on leveraging the weekly run workflow to refresh them

Keep the code changes simple and small

[Copilot]

Copilot this PR should follow the design of the previous #893

The part about take the stored artifact URLs is completed, now focus on leveraging the weekly run workflow to refresh the...

Addressed in 42a3e94. The PR now preserves #893’s hardcoded pin design and changes only the weekly dataset workflow: it resolves refreshed pins once, verifies that exact content across the full matrix, and proposes the verified update afterward.

[Copilot]

Copilot review overview

🟡 Changes recommended

Promotion tests are missing and the implemented manual-PR flow contradicts the PR description.

Review effort: Balanced
Findings: 3

Open (3)

• Add tests for schedule-only promotion and pin propagation · New
• Correct PR description to reflect manual PR creation · New
• Correct “resiliant” spelling to “resilient” · New

What changed in this PR

Moves public BC artifact pins into configuration and adds scheduled refresh verification.

Changes:

• Resolves and verifies latest public artifact URLs.
• Promotes successful results to a branch for manual PR creation.
• Documents the refresh process.

File	Description
scripts/​BCBenchUtils.psm1	Loads and refreshes artifact pins.
scripts/​BCBenchArtifactPins.json	Stores pinned URLs.
.github/​workflows/​dataset-validation.yml	Verifies and proposes refreshed pins.
CONTRIBUTING.md	Documents artifact refresh behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.