feat(data): add access labels and authorization cases as dataset v2, with a stricter security gate

[poppycoderr]

Background

The decision table is enforced in SQL, but the demo corpus had no access labels. The evaluation security gate could therefore only exercise tenant isolation. This PR delivers milestone item 2.7: labelled data, hand-written visibility, authorization negatives and a stricter gate. The corpus changes, so the result is published as dataset v2.

Dataset v2

 data/
 ├── corpus/northstar/        # 17 documents
+│                            # + 7 restricted look-alikes; sales-pricing-guide has two versions
 ├── manifests/*.yaml
+│                            # classification, allowed_departments, required_projects, history
 ├── principals.yaml
+│                            # + dave-contractor: clearance public, no department, no projects
-└── eval/v1/                 # 70 cases
+└── eval/v2/                 # 108 cases: the 70 unchanged, 14 authorized, 24 new authorization negatives

• Existing documents stay visible. The 17 Northstar documents become internal, and two of them stay public. Alice, Bob and Carol therefore see them exactly as before.
• Restrictions come from new documents. Each new document is written to resemble an existing one: confidential compensation bands that repeat the on-call allowance topic, two project documents, two department-only documents, and one restricted document that no demo principal may read.
• A document whose labels change between versions. sales-pricing-guide version 1 was readable by everyone internal. Version 2 changes the prices and restricts the document to sales and support.
• Authorization negatives. There are 30: 7 cross-tenant and 23 inside the tenant (clearance 10, project 6, department 7). The validator requires at least 25.
• Independent visibility labels. visibility.yaml was written by reading the labels and the principals' attributes, not by running the policy compiler.

The gate

before any query, per principal
  listed = every chunk the principal can list
  a listed document outside the visible set, or a chunk of a replaced version   → violation
  a visible document that cannot be listed                                      → run aborts (labels and system disagree)
per case and strategy
  every returned chunk: document in the visible set and version == current version, else violation
any violation → exit code 2, CI fails

• Version check. A chunk of a replaced version is a violation even when the document is visible, because labels belong to versions.
• Listing check. It covers documents that no query happens to retrieve.
• Loading histories. ga-eval load ingests a document's history oldest version first, so version numbers match the labels.
• Stale evidence. The validator rejects evidence that cites a replaced version.

To confirm the gate catches real mistakes, the restricted document was relabelled as public on a running stack. The run failed with exit code 2, the listing check reported the document for all four Northstar principals, and the affected cases reported violations.

Evaluation result

These figures come from the CI evaluation run of this PR (Linux x86_64, dataset v2, test split, 57 answerable cases).

Strategy	Recall@10	MRR@10	nDCG@10	Violations
sparse-only	0.930 [0.86, 0.98]	0.640 [0.54, 0.74]	0.711 [0.63, 0.79]	0
dense-only	0.965 [0.91, 1.00]	0.856 [0.78, 0.93]	0.884 [0.81, 0.94]	0
hybrid-rrf	0.965 [0.91, 1.00]	0.797 [0.71, 0.88]	0.837 [0.77, 0.90]	0
bm25-reference	0.921 [0.84, 0.98]	0.689 [0.59, 0.78]	0.744 [0.66, 0.82]	0

• Security. There are zero violations. The listings of all five principals match the hand-written visibility: 19, 19, 21, 2 and 4 documents. None of the 96 authorization-negative runs returned anything unauthorized.
• Ranking conclusions are unchanged. Dense beats full-text search on MRR@10 by +0.216 [+0.12, +0.31]. Hybrid shows no detectable difference from dense: −0.059 [−0.13, +0.01].
• Not comparable with v1 reports. The 70 original cases are unchanged, but 21 of their 280 case and strategy results moved. Principals with access to the new documents now have more candidates to rank, and BM25's corpus statistics changed.

The published v2 report from the main CI run, with the README numbers, follows in a separate PR.

---

背景

决策表已经在 SQL 里执行，但 demo 语料没有访问标签，所以评测的安全门禁此前只能检验租户隔离。本 PR 完成里程碑的 2.7 项：带标签的数据、人工编写的可见性标注、授权负例，以及更严格的门禁。语料发生了变化，所以结果作为数据集 v2 发布。

数据集 v2

目录变化见英文部分的 diff。

• 已有文档保持可见。 原有的 17 篇 Northstar 文档改为 internal，其中 2 篇保持 public。Alice、Bob、Carol 看到的内容和之前完全一样。
• 限制来自新文档。 每篇新文档都写得和某篇已有文档相似：重复了 on-call 津贴话题的机密薪酬文档、两篇项目文档、两篇仅限部门的文档，以及一篇 demo 里没有任何身份可以读的 restricted 文档。
• 一篇在版本之间改变标签的文档。 sales-pricing-guide 的第 1 版对所有 internal 身份可见。第 2 版改了价格，并把文档限制给销售和支持部门。
• 授权负例。 共 30 条：7 条跨租户，23 条在租户内部（密级 10 条、项目 6 条、部门 7 条）。校验器要求至少 25 条。
• 独立的可见性标注。 visibility.yaml 是对照标签和身份属性手写的，不是运行 policy compiler 得到的。

门禁

门禁的逻辑见英文部分的伪代码。

• 版本检查。 即使文档本身可见，返回被替换版本的 chunk 也算越权，因为标签属于版本。
• 列表检查。 它覆盖了没有被任何查询检索到的文档。
• 加载历史版本。 ga-eval load 会按从旧到新的顺序入库文档的历史版本，这样版本号和标注一致。
• 过期证据。 校验器会拒绝引用了已被替换版本的证据。

为了确认门禁能抓住真实的错误，我在运行中的环境里把那篇 restricted 文档改标成 public。评测以退出码 2 失败，列表检查对 4 个 Northstar 身份都报出了这篇文档，受影响的用例也报出了越权。

评测结果

以下数字来自本 PR 的 CI 评测（Linux x86_64，数据集 v2，test 划分，57 条可回答用例）。指标见英文部分的表格。

• 安全。 越权结果为 0。5 个身份能列出的文档与手写的可见性标注一致，分别是 19、19、21、2、4 篇。96 次授权负例运行没有返回任何越权内容。
• 排序结论没有变。 dense 在 MRR@10 上比全文检索高 +0.216 [+0.12, +0.31]。hybrid 与 dense 之间没有可检测的差异：−0.059 [−0.13, +0.01]。
• 不能与 v1 的报告直接比较。 原有的 70 条用例没有改动，但它们的 280 个「用例 × 策略」结果里有 21 个发生了变化。原因是能访问新文档的身份多了候选，BM25 的语料统计也变了。

正式的 v2 报告会取自 main 上的 CI 运行，连同 README 里的数字，放在另一个 PR 里。