Skip to content

fix: 有界部分扣分 trace / bounded partial-credit traces - #71

Merged
yyuan308 merged 1 commit into
mainfrom
codex/bounded-trace-partial-credit
Aug 29, 2026
Merged

yyuan308 merged 1 commit into
mainfrom
codex/bounded-trace-partial-credit

Conversation

@yyuan308

Copy link
Copy Markdown
Collaborator

摘要 / Summary

  • 执行型 rubric 的 criterion 分值改为“可扣分上限”:有作答证据时允许按 score step 的部分扣分,但不得超过 criterion 上限、不得重复 criterion、不得使用未声明 criterion。
  • Treats execution-rubric criterion points as deduction caps: evidence-supported partial deductions are allowed on the score step, but may not exceed a cap, repeat a criterion, or use an undeclared criterion.
  • 仍严格要求每题 deduction total 等于分差。仅当唯一一个已选 criterion 可在上限内修复加法笔误时,才做不改变 score 的私有归一化;有多个可能修复时仍拒绝并重试。
  • Deduction totals still must match the score gap. Only a uniquely determined, cap-bounded arithmetic repair is normalized privately without changing the score; ambiguous cases remain rejected and retried.

验证 / Verification

  • python -m unittest tests.benchmark.core.test_execution_contract_rubric tests.benchmark.core.test_deduction_trace_contract tests.benchmark.core.test_model_runner — 41 passed
  • python scripts/run_ci_tests.py core — 486 passed
  • 新增覆盖:部分扣分、超上限拒绝、唯一修复、歧义修复拒绝。

隐私、限制与下一步 / Privacy, limitations, and next step

  • 只包含公共 runner 和合成测试;未暂存或提交 Data/、学生内容、成绩、转录、gold 或模型输出。
  • Contains only public runner code and synthetic tests; no Data/, student content, scores, transcripts, gold, or model output is staged or committed.
  • 不报告模型指标。合并后会从全新目录重启三条已对齐的开发集路线;此前所有部分输出仍不可报告。

@yyuan308
yyuan308 merged commit f52833a into main Aug 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant