Skip to content

test(skills): the platform skill's evals ask for push notifications on os serve - #30101

Merged
os-zhuang merged 3 commits into
mainfrom
claude/project-thread-xxbrzx
Nov 4, 2026
Merged

os-zhuang merged 3 commits into
mainfrom
claude/project-thread-xxbrzx

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J

Requested by Irene · project thread

Part of #29969

Clause-②: no
Metadata-protocol: none

The maintainer waived the model-tier mandate for skills/** while that tier had no quota; this PR was developed below it.

维护者速读

  • 改了什么:skills/objectstack-platform/evals/config-plugins-ops.json 新增第 12 个 eval:提示要求给 os serve 部署开启推送;期望答案在栈根声明 requires: ['push']、写顶层 push: 键,且不内联任何密钥(APNs .p8 与 Firebase 服务账号 JSON 都按引用命名)。另加一个 '@objectstack/skills': patch changeset。
  • 为什么改:PR feat(push): requires push and the stack push key mount PushServicePlugin under os serve #29968 让 requires: ['push'] 与栈的 push 键在 os serve 下挂上 PushServicePlugin,其 changeset 记下了本卡这一评测欠账。
  • 风险与回滚:只加评测夹具与 changeset;eval 夹具只计量、不受 token 棘轮约束,无上限变动;回滚即 revert。
  • 先看:新 case 12 的 assertions。断言只覆盖 SKILL.md 现在教的内容(L42 顶层 push 键、L310 push token),不断言 transport / apns / fcm / relay,因为 SKILL.md 还没教 push 块的形状。卡片 Wanted (2)(钉住 transport: 'direct' 与被拒的 relay)移到 skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092:先在技能里教 push 块形状,再扩展本 case;故本 PR 是 Part of #29969。
  • 需要你做的:Tier H 受管面(skills/**),需要 GOVERNED_APPROVERS 的批准;PR 保持 draft。

What changes

skills/objectstack-platform/evals/config-plugins-ops.json gains case 12, in the same prompt / expected_output / files / assertions layout as cases 1–11:

  • Prompt: turn on push notifications for an os serve deployment whose iOS and Android apps are signed with the team's own APNs key and Firebase project.
  • Expected output: requires: ['push'] on the stack root, which mounts @objectstack/service-push, plus a top-level push: key. There is no host code and no PushServicePlugin in plugins:. The APNs signing key and the service-account JSON stay in the secrets layer and are named by reference (credentialRef). Nothing is written inline.
  • must_contain: requires, 'push', push:.
  • must_not_contain: BEGIN PRIVATE KEY, privateKey:, "private_key".

Why the case stays generic. skills/objectstack-platform/SKILL.md teaches only two things about push today: push among the stack's top-level keys (L42) and the push token in the requires table (L310). It does not yet teach the push block's shape: transport, the apns / fcm vendor blocks, credentialRef, or the boot refusing the hosted relay. So the case asserts the stack-level shape the skill teaches and the no-inline-secret rule, and does not yet pin the transport: as written it still passes a push: {} or transport: 'relay' answer, which the boot refuses. SKILL.md is not touched here: it is at 6000 of its 6011-token ceiling, and teaching the push block's shape there is its own change.

What moves to #30092. The card's Wanted (2), pinning transport: 'direct' and the refused relay, moves to #30092, which first teaches the push block's shape in the skill and then extends this case. Hence Part of #29969, not Fixes.

The facts were read on main from PR #29968: packages/spec/src/system/push-config.zod.ts (PushServiceConfigSchema, with credentialRef a secrets-layer reference and the inline privateKey / key / serviceAccount keys refused) and packages/services/service-push/src/capability-arg.ts.

.changeset/29969-platform-push-eval.md: '@objectstack/skills': patch, with both declaration lines.

Verification

Head 66bfe707, three commits on 61b487b2. Its tree is identical to 44939f51, the head the contract review read: e871e3cd pinned the transport in the case and 66bfe707 reverts it, because that pin moves to #30092.

  • The fixture parses as JSON (12 cases). For every case, each must_contain string appears in its expected_output and no must_not_contain string does.
  • node scripts/check-skills-token-ratchet.mjs: exit 0. Eval fixtures are measured, not ratcheted, so no ceiling row moved.
Local gates on 44939f5 and again on 66bfe70: 39 derived, 39 run with exit 0, 0 NOT MEASURED each time (`dispatch-gates --ran`)
  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 39 commands for the two changed files. All of them ran after pnpm install --frozen-lockfile and a full pnpm build (exit 0). Each had a 900 s timeout, and its exit code was captured before any pipe.
  • All 39 exited 0. They include check-adr-0087-registration.mjs --base origin/main, check-changeset-no-major.mjs, check-empty-changeset.mjs, check:changeset-gate-self-tests, check-comment-mask-corpus.mjs, check:doc-authoring, check:skill-compatibility, check:skill-frame-sync, check:skill-identifier-liveness, check:dts-closure and check:dual-build-cjs-loads.
  • The artifact rosters pnpm check:changeset-fixed and pnpm check:skill-top-level-keys both exited 0.
  • --ran printed: 39 derived famil(ies) accounted for — 39 run, 0 NOT-MEASURED.
  • node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD: exit 0 on both heads (three commits on 66bfe70).

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic. (Not touched.)
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran. (pnpm lint was not run: the diff is one JSON fixture and one changeset, no code.)
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset.
  • 6. packages/spec touched: its generated artifacts are regenerated and committed. (Not touched.)
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English.
  • The Before Submitting list in CONTRIBUTING.md is done.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T

…n os serve

Case 12 in objectstack-platform/evals/config-plugins-ops.json asks to turn on
push for an os serve deployment; the expected answer declares
requires: ['push'] with a top-level push: key and writes no secret inline.
The assertions stay with what SKILL.md teaches today.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T
@github-actions github-actions Bot added size/s documentation Improvements or additions to documentation tooling labels Nov 4, 2026
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J

Contract-review-claim: session_01J8VVAnMUWA9rufXzFaEh4J · head 44939f5 · 2026-11-04T13:45Z

The seat review runs at CONTRACT_REVIEW_TIER in an isolated subagent (the seat session is not at that tier). No review seat (lanes/review.md) is seated.


Generated by Claude Code

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 44939f5195637a68f600e0ace19038d3029ff2e5
Local-runs: probe — on a detached worktree of the head (merge-base with origin/main 61b487b; origin/main at f68b47c when fetched, five commits past the merge-base, none of which touches skills/objectstack-platform/**, packages/spec/src/stack.zod.ts, packages/spec/src/system/push-config.zod.ts or packages/services/service-push/**; dispatch-gates names scripts/codemod-split-engine.mjs as the only moved path under its surface, not an input): node scripts/check-skills-token-ratchet.mjs (exit 0: skills/objectstack-platform/SKILL.md 6000 / 6011, headroom 11, byte-identical to the merge-base at 24019 bytes; skills/objectstack-platform/evals/config-plugins-ops.json 2729 tokens, printed as "eval fixture, measured but not ratcheted"; 57 authored files within ceilings, 29 eval fixtures measured) and --self-test (exit 0, 97 cases); node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (39 commands, matching the dev record's 39; no eval-specific family in the list); node scripts/pm/check-skill-id-lint.mjs (110 files clean, exit 0) and --self-test (23 cases); node scripts/check-skill-identifier-liveness.mjs (486 citations over 71 published files, exit 0) and --self-test; check-skill-top-level-keys.mjs (exit 0: SKILL.md L33 enumerates all 53 authorable top-level keys), check-skill-compatibility-version.mjs, check-skill-frame-sync.mjs, check-role-word.mjs (all exit 0); node scripts/pm/check-governed-merges.mjs --branch HEAD (exit 3, GOVERNED, Tier H, 1 of 2 paths hit skills/**, 28 changed lines); node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD and --range 61b487b2e7..HEAD (exit 0, one commit, model-free trailer pair); node scripts/check-empty-changeset.mjs --base origin/main (exit 0, one declaring changeset added, none from the merge-base modified) and --self-test (165 assertions); node scripts/check-changeset-no-major.mjs --base origin/main (exit 0, level axis not applicable locally); python3 -I over the fixture (parses; 12 cases; every must_contain string in its own expected_output, no must_not_contain string in it, for all 12; one key order and one assertion key order across all cases); wc -c / wc -l on both trees; grep -i over skills/objectstack-platform/** for credentialRef, secrets layer, .p8, service-account, relay, transport: (none); git merge-base --is-ancestor and git log for PR #30071's squash commit; gh api REST reads of the PR, its files, commits, reviews and comments, card #29969 and its comments, PRs #30071 and #29968, card #29920, and the head's check-runs three times. No node_modules in this container, so no pnpm check:* wrapper ran; check:doc-authoring, check-adr-0087-registration and the dts/cjs families are read from CI. The worktree was removed after the runs and the main checkout was not touched.
Scope: skills/objectstack-platform/evals/config-plugins-ops.json (the governed hit: case 12, head L201–218) + read, not reviewed: .changeset/29969-platform-push-eval.md (the prose face, checked sentence by sentence here), skills/objectstack-platform/SKILL.md L33–42 (the top-level key list, push at L42) and L300–330 (the requires: table, push at L310, and the tier-gated / enterprise notes), packages/spec/src/stack.zod.ts L1135–1150 (the push key, its TSDoc and describe()), packages/spec/src/system/push-config.zod.ts in full (header L8–87, secretReference L114–121, the APNs and FCM guidance maps L133–182, the relay block L205–235, the direct block L242–260, the union L270–280), packages/services/service-push/src/capability-arg.ts L54–73 (resolvePushCapabilityArg) and push-plugin.ts L46–50 (PUSH_RELAY_NOT_SERVED_MESSAGE), packages/core/src/capability-providers.ts L206 and packages/spec/src/kernel/platform-capabilities.ts L66–70, L220 (the push token's provider), scripts/check-skills-token-ratchet.mjs L153–182 (population 1b) and L584 (the platform ceiling row), skills/README.md L72–75 and skills/objectstack-data/evals/README.md (the fixture shape), packages/skills/scripts/sync-catalog.mjs L5–50 and packages/skills/package.json (what @objectstack/skills ships), .github/PULL_REQUEST_TEMPLATE.md L22–30 (the Human Minimum list), scripts/check-commit-card-trailers.mjs L320 (the model-id shape), card #29969 body, comments 6612527491 (the carried should-fix), 6612799302 (triage) and 6631549725 (claim), PR #30101 body and its one comment 6632263193 (the seat's review claim on this head), PR #30071 and PR #29968 metadata, card #29920 state, the dev gate record pr-29969/gates-44939f51.txt (read as input)

Reviewer: an isolated contract reviewer at CONTRACT_REVIEW_TIER, read-only on GitHub (REST reads only; nothing written, pushed or commented), fed the card body and comments, the PR body, the net diff against the merge-base, the head's check-runs, AGENTS.md, the pm-dispatch SKILL.md contract-review section, contract-review.md, review-checklist.md, lanes/skills.md and lanes/review.md. The dev gate record was read as input, not as conclusions; every count and every claim about the spec, the push service and the skill below is this review's own reading of the head tree and of main at the merge-base. No environment variable value is printed anywhere in this record.

① Derived judgments

  1. File surface. One commit (44939f5, author Claude, PR author objectstack-fleet[bot], head repo = base repo, draft, base main, trailers Claude-Session: + Co-Authored-By: Claude, no model identifier). git diff --numstat 61b487b2 44939f51, the REST file listing (+28 / −0, 2 files) and the PR body agree on exactly two paths: skills/objectstack-platform/evals/config-plugins-ops.json +18/−0 (202 → 220 lines, 9967 → 10915 bytes) and .changeset/29969-platform-push-eval.md +10/−0 (new, 754 bytes). The claim (6631549725) named exactly these plus "if the fixture's index lists cases, its one-line entry" (there is no index: evals/ holds the one JSON). SKILL.md is byte-identical to the merge-base. Correct, and on surface.
  2. What the skill teaches today, read at the head (= merge-base for this file). Exactly two mentions of push: push closes the top-level key list at L42, and the requires: table row at L310 maps push (with analytics … sms) to @objectstack/service- + the token. grep -i over SKILL.md, references/*.md and rules/*.md for credentialRef, secrets layer, .p8, service-account, relay and transport: finds nothing. So the PR body's "L42" and "L310" reproduce, and its statement that the skill does not teach the push block's shape is true — and the three must_not_contain strings and the credentialRef sentence in expected_output already go beyond what the skill teaches (see ④ and F1).
  3. expected_output, fact by fact against the code. "requires: ['push'] on the stack root (the token mounts @objectstack/service-push)": capability-providers.ts L206 and platform-capabilities.ts L220 both bind the token to that package, and SKILL.md L310 says the same. "Plus a top-level push: key on the stack definition for the transport": stack.zod.ts L1145 (push: PushServiceConfigSchema.optional()), read by resolvePushCapabilityArg when requires names push (TSDoc L1138–1139). "No host code, no PushServicePlugin in plugins:": under objectstack serve the capability loop builds the plugin from the key (capability-arg.ts L8–17), so a second instance in plugins: is a duplicate — the shape case 3 already refuses for job. "The APNs signing key and the Firebase service-account JSON stay in the secrets layer (an environment variable in the open tier) and the config names them by reference (credentialRef)": push-config.zod.ts L59–64 and the two credentialRef fields (L153–154, L178–179). "No .p8 key, privateKey or service-account JSON written inline": L65–69, privateKey refused by name in both guidance maps (L140, L169), the PEM and JSON shapes refused by secretReference (L114–121). Every sentence is true; nothing in the expected output teaches something false. What it does not say is which transport — see ④.
  4. The must_contain strings. requires is an ordinary English word ("this requires an APNs key" passes it); case 3 carries the same string as precedent, and the pair 'push' (quoted, the array-literal spelling) and push: (the key spelling) are what the case actually discriminates on: an answer that writes plugins: [new PushServicePlugin(…)] with no requires and no push: key fails both. A prose answer that writes push in backticks without quotes or a colon fails too, which is the fixture's convention (cases 9 and 10 pin 'sharing' and overrides: the same way). Adequate; not every answer passes.
  5. The must_not_contain strings. An inlined APNs .p8 key is a PEM block whose first line is -----BEGIN PRIVATE KEY-----, caught by BEGIN PRIVATE KEY; an inlined FCM service-account JSON carries "private_key": "-----BEGIN PRIVATE KEY-----…", caught twice; a TypeScript privateKey: key, which the schema refuses by name whether its value is inline or process.env.X, is caught by privateKey:. The strings are absent from expected_output (which writes privateKey without the colon, the case-1 pattern of workflows: versus "no workflows keys"). Residual gap: serviceAccount: / key: with a path or an env reference pass, though the schema refuses those keys by name (L139, L166) — minor, since a file path is not the secret and the boot refuses the key loudly. The set discriminates the card's "puts no secret inline".
  6. Consistency with cases 1–11. Ids 1–12 consecutive; one key order (id, prompt, expected_output, files, assertions) and one assertion key order (must_contain, must_not_contain) across all twelve; files: [] like every sibling; two-space indentation, no tabs, trailing newline; the file parses. For all twelve cases every must_contain string is in its own expected_output and no must_not_contain string is (the eval self-consistency rule). Correct.
  7. The ratchet. Population 1b of check-skills-token-ratchet.mjs (header L174–182) classes skills/<name>/evals/** as "measured and printed in the bundle total, NOT ratcheted" by ruling, with a CEILINGS row on an eval path RED by design; the gate prints the fixture as "2729 tokens — eval fixture, measured but not ratcheted" and exits 0; the self-test pins the membership. So the dev's "eval fixtures are measured, not ratcheted, so no ceiling row moved" is the gate's own classification. SKILL.md is untouched, so its row (L584, 6011) did not move; its headroom is 11 tokens, which matters for ④. Correct.
  8. Changeset. .changeset/29969-platform-push-eval.md: frontmatter '@objectstack/skills': patch; line-initial Clause-②: no and Metadata-protocol: none; prose: "gains case 12" (true), "the prompt asks to turn on push notifications for an os serve deployment" (true, L203), "the expected answer declares requires: ['push'] with a top-level push: key … and writes no secret inline (the APNs signing key and the Firebase service-account JSON are named by reference)" (true of L204), "the assertions stay with what objectstack-platform/SKILL.md teaches today, the push token and the top-level key; the push block's transport and vendor shape are not asserted until the skill teaches them" (true as a description of the assertions; whether it is the right choice is ④). @objectstack/skills is published (package.json has no private, files is dist, README.md, CHANGELOG.md) and sync-catalog.mjs copies skills/** whole into dist/skills/<skill>/…, so the fixture ships (inert, per skills/README.md L74–75) and a patch changeset is owed; the triage note (6612799302) says the same and skip-changeset would have been wrong. Labels documentation, size/s, tooling: no skip-changeset. Check Changeset is success on the head. Correct.
  9. Gates re-run here. The 39 derived commands match the dev record's 39, and the list carries no eval-specific family (the eval self-consistency rule lives in the ratchet's population text and this review's own parse, not a derived gate). Of those re-run without node_modules: ratchet and self-test, skill-id-lint and self-test, skill-identifier-liveness and self-test, skill-top-level-keys, skill-compatibility, skill-frame-sync, role-word, commit-card-trailers (two ranges), empty-changeset and self-test, changeset-no-major all exit 0; governed-merges exit 3 by design. The pnpm check:* wrappers (doc-authoring, adr-0087-registration, dts-closure, dual-build-cjs-loads, …) could not run in this container and are read from CI, where Lint · repo & tooling gates, Lint · CI, release & engine gates, Lint · docs & code guards and Lint · PM dispatch-gates are success on the head. Nothing I could measure contradicts the dev record.
  10. PR body against the head. First content line Fixes #29969; the only other card numbers (feat(push): requires push and the stack push key mount PushServicePlugin under os serve #29968, docs(skills): the platform skill teaches the package customizations read's contributions #30071) are references and PR-body guards is success. Clause-②: no and Metadata-protocol: none line-initial and right (no spec, no key). "Head 44939f51, one commit on 61b487b2": reproduces. "The fixture parses as JSON (12 cases). For every case, each must_contain string appears … and no must_not_contain string does": reproduces. "39 derived, 39 run": reproduces. The model note ("The maintainer waived the model-tier mandate for skills/** while that tier had no quota; this PR was developed below it") names no model identifier and no family word. Human Minimum: eight items ticked; item 3 says pnpm lint was not run (Lint · ESLint is success on the head, so nothing is hidden); items 1 and 6 "not touched" are true. Two sentences do not hold: "SKILL.md is not touched here, because PR docs(skills): the platform skill teaches the package customizations read's contributions #30071 is editing it" and the claim's "open PR docs(skills): the platform skill teaches the package customizations read's contributions #30071" — PR docs(skills): the platform skill teaches the package customizations read's contributions #30071 was merged at 2026-11-04T13:11:04Z, its squash commit 89803c5 is in this PR's own merge-base (three commits below 61b487b), the head commit was authored at 2026-11-04T13:14Z and the PR opened at 2026-11-04T13:44:48Z; the L42 / L310 the body cites are read from the tree that already carries docs(skills): the platform skill teaches the package customizations read's contributions #30071. The real constraint on SKILL.md is its headroom (11 tokens), which the body does not name: S1 (③). The Fixes verb is judged in ④ / F1.
  11. Governance. check-governed-merges.mjs --branch HEAD: 1 of 2 paths hits skills/** (the changeset is not on the register), Tier H, 28 changed lines under the 3000 threshold, exit 3 as designed; the PR is a draft, no reviews, no assignee, no requested reviewer, auto_merge null, mergeable_state blocked. Landing is the maintainer's hand or an authorized approval; this record does not lift it, and with the verdict below it must not.

② Semver level

Patch, and the changeset says patch. @objectstack/skills publishes skills/** through dist/skills/<skill>/…, so a new eval fixture case is a user-visible (if inert) addition to a published package with no protocol change: Clause-②: no, Metadata-protocol: none, '@objectstack/skills': patch. No packages/spec/src/**, no TypeScript, no skip-changeset; Check Changeset is success on the head.

③ Boundary flags

  • Dev open_questions: none in the body. The body's acceptance notes amount to one: the case stays generic because the skill does not teach the push block's shape. Answered in ④ / F1: the stated principle is not what the case does (its own must_not_contain set and credentialRef sentence assert spec facts the skill never teaches), and the card's second Wanted item was implementable inside the claimed file surface.
  • S1, the SKILL.md premise: the body's "PR docs(skills): the platform skill teaches the package customizations read's contributions #30071 is editing it" and the claim's "open PR docs(skills): the platform skill teaches the package customizations read's contributions #30071" were both false when written (①.10). Not a blocker on its own — the file is untouched and the citations are from the right tree — but the sentence should say what actually holds: the platform ceiling has 11 tokens of headroom, so the carried should-fix (6612527491: teach one clause beside the eval) cannot be paid without a cut, and that is a separate card or a serial PR, not this one's reason.
  • Model note: the PR body carries no model identifier and no family word; the trailer pair is model-free (check-commit-card-trailers exit 0). Not this PR's: the claim comment's Container & model: line on the card names a model family word where the template asks for a tier word — a seat record on the card, outside this diff.
  • Independence: PR author objectstack-fleet[bot], commit author Claude; no off-relay author note is owed. The PR thread carries the seat's Contract-review-claim: for this head (6632263193) and no other record; no review seat is seated, as the claim says. No assignee or requested reviewer on the PR.
  • Tier H: this record does not lift the landing, and a FAIL record withholds it regardless; auto_merge is null, so no disarm is owed. Five check-runs were still in progress at the last read (below).

④ Design judgment

Per hits file, against the card's scenario and the code on main. The card (filed by the spec seat from PR #29968's Skills-docs: follow-up owed line) wants two things of the one case: (1) asked to turn on push for an os serve deployment, the agent declares requires: ['push'] and a push: { transport: 'direct', apns | fcm: { credentialRef, … } } block and puts no secret inline; (2) the case pins the relay status — a push: {} / transport: 'relay' answer is wrong until the hosted relay sender (#29920, still open) ships, and the right answer says so. The claim on the card (6631549725) repeats it: "the case pins transport: 'direct'".

  • skills/objectstack-platform/evals/config-plugins-ops.json, case 12. The prompt is right: "our own iOS and Android apps, signed with our APNs key and our Firebase project" is the white-label build push-config.zod.ts L31–35 reserves the direct transport for, so the one correct answer is transport: 'direct' with vendor blocks, and the spec says so in the key's own describe() (stack.zod.ts L1147–1149: "transport direct with apns and/or fcm blocks … Absent or transport relay is the hosted relay, which the boot refuses until the relay sender ships") and in the boot's refusal text (PUSH_RELAY_NOT_SERVED_MESSAGE, push-plugin.ts L46–50). The delivered case says "a top-level push: key … for the transport" and stops: transport, direct, relay and push: {} are all absent from expected_output and from both assertion lists. So an answer of requires: ['push'] plus push: {} — or push: { transport: 'relay' } — passes every assertion of case 12, and it is exactly the answer resolvePushCapabilityArg refuses before the kernel boots (capability-arg.ts L66–70). The case accepts, as right, the one wrong answer the card was filed to pin. The body's reason — "the assertions cover only what the skill teaches" — is not what the case does: the skill teaches nothing about credentialRef, the secrets layer, .p8 keys or service-account JSON (①.2), yet the case asserts all of them, correctly, from the spec; the transport is a spec fact of the same kind and the same source file, and transport: 'relay' as a must_not_contain string, with one clause in expected_output naming the direct transport and the refused relay, fits inside the claimed file surface with no SKILL.md edit. What was delivered teaches nothing false and is correctly built; what the card asked for is not in it, and Fixes #29969 would close the card with Wanted (2) undone and uncarried. Does not hold: F1.
  • .changeset/29969-platform-push-eval.md (prose face, checked here): every sentence true of the diff (①.8). Its last sentence records the narrowing as a choice; it follows F1's resolution.

Must-fix: F1 — case 12 accepts a boot-refused answer and leaves the card's Wanted (2) unpinned while the PR says Fixes #29969. Resolve one of two ways, both inside the claimed surface. (a) Pin it in the case: in expected_output, after "for the transport", name the transport — e.g. "transport: 'direct' with an apns and/or fcm block, each with its credentialRef; the hosted relay (transport relay, or an absent or empty push block) is refused at boot until its sender ships" — and add transport: 'relay' to must_not_contain (the expected_output must not carry that exact string, the case-1 pattern; 'direct' may join must_contain); then update the changeset's last sentence, which would no longer be true. (b) Keep the case generic: change Fixes #29969 to Part of #29969, and the seat files Wanted (2) (and the carried SKILL.md clause, which the 11-token headroom makes a cut) as the card the body names. (a) is about three lines and matches the card and the claim; (b) is a body edit plus a card. Either way the delta is reviewed on its head. Should-fix: S1 — the PR body's SKILL.md sentence (①.10, ③): replace "because PR #30071 is editing it" with the fact — #30071 is merged and in this PR's merge-base; SKILL.md has 11 tokens of headroom under its 6011 ceiling, so teaching the push block there is a separate change.

Check-runs at record time: 39 check-runs on 44939f5, 21 success, 13 skipped, 0 failure, 5 in progress on the third read (Test Core 1/8 and 2/8, Type Check · workspace, Type Check · consumer gates, Type Check · debt ledger); three reads went 38 → 38 → 39 runs and 15 → 19 → 21 success with no non-success completion at any read. Success: Lint & Repo Gates (a required floor), Lint · repo & tooling gates, Lint · CI, release & engine gates, Lint · PM dispatch-gates, Lint · docs & code guards, Lint · ESLint, Lint · scoped · tooling self-tests, Lint · scoped · slot lookup & corpus, Lint · scoped · query-options erasure, Type Check · source gates, Build Core, Governed Surface Queue Guard, Dogfood Regression Gate, Test UI, Plan Test Core shards, Live E2E UI, PR-body guards, Check Changeset, Check PR Size, Auto Label, filter. The other required floor, TypeScript Type Check, had no check-run yet at the last read: its three shards were still in progress, so it is pending, not red. Skipped by path filter or opt-in: Build Docs, Build UI Deps, Build & E2E UI, Lint UI, Type Check UI, UI Published Artifacts, Dogfood Verify CLI, Temporal Conformance, Schedule gate, 18.x upgrade harness, Packed-tarball smoke, and the matrix-templated placeholder rows for Test UI and the Dogfood shards. Combined status pending (statuses API, 0 statuses). mergeable_state blocked, draft, auto_merge null, labels documentation, size/s, tooling; Tier H (skills/**), landing by the maintainer's hand or an authorized approval, and not before the F1 delta is reviewed PASS on its head.

Implemented-by: claude/project-thread-xxbrzx
Reviewed-by: session_01J8VVAnMUWA9rufXzFaEh4J

VERDICT: FAIL


Generated by Claude Code

…s the relay

Case 12's expected answer names transport: 'direct' with apns / fcm blocks,
each with its credentialRef, and says the hosted relay (also an absent or
empty push block) is refused at boot until its sender ships; the case adds
transport: 'relay' to must_not_contain. The changeset says the same.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T
…d refuses the relay"

This reverts commit e871e3c. Pinning the
transport moves to the follow-up that first teaches the push block's shape
in the skill; the case stays as reviewed.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J

Contract-review-claim: session_01J8VVAnMUWA9rufXzFaEh4J · head 66bfe70 · 2026-11-04T14:15Z

Delta review against the FAIL at head 44939f5 (tree identical at 66bfe70; the delta is the PR body, option (b) of F1, and S1). The seat review runs at CONTRACT_REVIEW_TIER in an isolated subagent. No review seat (lanes/review.md) is seated.


Generated by Claude Code

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 66bfe707d27053ed0d7790d1f43d3c94a8bdb53c
Local-runs: probe — on a detached worktree of the head (merge-base with origin/main still 61b487b; origin/main at 1f13eb3 when fetched, seven commits past the merge-base, none of which touches skills/objectstack-platform/**, packages/spec/src/stack.zod.ts, packages/spec/src/system/push-config.zod.ts, packages/services/service-push/**, scripts/check-skills-token-ratchet.mjs or scripts/pm/dispatch-gates.mjs; dispatch-gates again names scripts/codemod-split-engine.mjs as the only moved path under its surface, not an input): git rev-parse <sha>^{tree} on both heads (one tree, 1af464ec), git diff --stat 44939f51 66bfe707 (empty), git diff --numstat 61b487b2 66bfe707 (+28 / −0, two files), git diff 44939f51 e871e3cd (the pin commit the head reverts), git merge-base --is-ancestor of PR #30071's squash commit against the merge-base and the head, git log of the merge-base..origin/main range; node scripts/check-skills-token-ratchet.mjs (exit 0: skills/objectstack-platform/SKILL.md 6000 / 6011, headroom 11; config-plugins-ops.json 2729 tokens, "eval fixture, measured but not ratcheted") and --self-test (97 cases); node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (39 commands; the sorted set is identical to the dev record's 39 for this head, which is itself identical to the 44939f5 record); node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD and --range 61b487b2e7..HEAD (exit 0, three commits, model-free trailer pairs); node scripts/pm/check-governed-merges.mjs --branch HEAD (exit 3, GOVERNED, 1 of 2 paths hit skills/**, 28 changed lines); node scripts/check-empty-changeset.mjs --base origin/main and node scripts/check-changeset-no-major.mjs --base origin/main (exit 0); node scripts/check-skill-top-level-keys.mjs (exit 0, 53 keys), node scripts/pm/check-skill-id-lint.mjs (110 files clean), node scripts/check-skill-identifier-liveness.mjs (486 citations over 71 files), check-skill-compatibility-version.mjs, check-skill-frame-sync.mjs, check-role-word.mjs (all exit 0); python3 -I over the fixture (parses; 12 cases; every must_contain string in its own expected_output, no must_not_contain string in it, for all 12; one key order and one assertion key order); wc -lc on the fixture and SKILL.md; grep over the live PR body for closing verbs, card numbers, ISO date-times, claude- ids, the family words the repo's recogniser lists as fixtures and "model"; gh api REST reads of the PR (metadata, files, commits, reviews, three comments), PR #30071, cards #29969 and #30092 with their comments, the carried should-fix 6612527491, the head's check-runs and combined status, the superseded head e871e3c's check-runs and its Lint & Type Check workflow run with its jobs, and the head's Lint & Type Check run. No node_modules in this container, so no pnpm check:* wrapper ran; those families are read from CI. The worktree was removed after the runs and the main checkout was not touched.
Scope: the PR body as it now reads (the only thing in the delta) + read, not reviewed: skills/objectstack-platform/evals/config-plugins-ops.json L201–218 (case 12, unchanged bytes), .changeset/29969-platform-push-eval.md (unchanged bytes), skills/objectstack-platform/SKILL.md L42 and L310 (unchanged, byte-identical to the merge-base at 24019 bytes), card #30092 body and its one triage comment 6632507548, card #29969 body and comments 6612527491 (the carried should-fix), 6612799302 and 6631549725, PR comments 6632263193 (the claim on 44939f5), 6632524557 (the FAIL being continued) and 6632942931 (the seat's claim on this head), PR #30071 metadata, the three PR commit messages, the dev gate record pr-29969/gates-66bfe707.txt (read as input)
Delta review of 6632524557 on 44939f5

Reviewer: the same isolated contract reviewer at CONTRACT_REVIEW_TIER, read-only on GitHub (REST reads only; nothing written, pushed or commented). The full review on 44939f5 (FAIL: must-fix F1, should-fix S1) stands as read for everything it measured on the tree, because the tree has not changed; this record judges only the delta — the two commits that cancel each other, and the PR body as it now reads against the head, card #29969 and card #30092. The dev gate record was read as input, not as conclusions. No environment variable value is printed anywhere in this record. Dates quoted from GitHub are written in the placeholder form, not typed.

① Derived judgments

  1. Delta surface. Two new commits on top of 44939f5, both author Claude, both with the model-free trailer pair (check-commit-card-trailers exit 0 over the three): e871e3c, "test(skills): the push eval case pins the direct transport and refuses the relay" (2 files, +4 / −3: transport: 'direct' with apns / fcm and credentialRef into expected_output, transport: 'relay' into must_not_contain, the changeset's prose re-worded — option (a) of F1), and 66bfe70, Revert "…", whose message says the pin "moves to the follow-up that first teaches the push block's shape in the skill; the case stays as reviewed". git rev-parse gives one tree for 44939f5 and 66bfe70 (1af464ec785db2b13ba3359ff86d6a9b91433395); git diff --stat between them is empty; the REST file listing is still two files, +28 / −0. So every figure of 6632524557 ①.1–①.9 and ①.11 holds on this head unchanged: the fixture at 220 lines / 10915 bytes with case 12 at L202, the twelve-case self-consistency, the ratchet at 6000 / 6011 with the fixture measured but not ratcheted, the changeset's two declaration lines, the 39 derived gates. The seat took option (b). Correct, and on surface.
  2. F1(b), the closing verb. The body's first content line is now Part of #29969. A grep of the live body for Fixes, Closes, Resolves followed by a card number finds nothing; Fixes appears once, inside the sentence "Hence Part of #29969, not Fixes", with no number after it. The only card numbers in the body are feat(push): requires push and the stack push key mount PushServicePlugin under os serve #29968 (twice, as the source PR), skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969 (three times) and skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092 (four times). PR-body guards (claim · duplicate · Part-of · single-writer path) is success on the head on both of its rows, the second of which ran after the body edit. Merging this PR will not close skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969. Resolved.
  3. F1(b), the card the body names. The body says, in English and in the 速读: "The card's Wanted (2), pinning transport: 'direct' and the refused relay, moves to skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092, which first teaches the push block's shape in the skill and then extends this case." Card skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092 (open, pm:queue, domain:skills, priority:p3, triaged by 6632507548; filed by this seat before the PR was opened, from the same session) reads, under "What the skill says": "Nothing teaches the block itself: the transport choice, the apns / fcm credentials by credentialRef (never inline), or which transports are served today. The hosted relay sender is service-push: send through the hosted relay transport (push-relay.zod.ts wire, cloud binding credential) #29920, still open." Its Wanted is three bullets: (i) "One short passage, in SKILL.md or a reference the skill routes to: what a push block carries, credentials by credentialRef only, and the relay transport's current status" — this is the carried should-fix 6612527491 (teach one clause beside the eval: the push row is the only requires row whose boot is refused without its config key); (ii) "Then extend skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969's eval case (config-plugins-ops.json) to assert the taught shape" — the eval half of skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969's Wanted (2), since the taught shape of (i) includes the transport and the relay status; (iii) "Pay within the check-skills-token-ratchet ceiling by same-file deletion. SKILL.md has about 11 tokens of headroom after PR docs(skills): the platform skill teaches the package customizations read's contributions #30071, so a reference file may be the better home" — the headroom fact S1 asked for, on the card as well. Both items F1(b) required are on the card and the card points back at skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969. Reading note, not a finding: skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092 does not quote skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969's Wanted (2) verbatim — the strings transport: 'direct', push: {} and transport: 'relay' are not on it; "the transport choice", "which transports are served today" and "the relay transport's current status", with service-push: send through the hosted relay transport (push-relay.zod.ts wire, cloud binding credential) #29920 named, carry the same content, and bullet (ii) binds the eval to whatever (i) teaches. The implementer of skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092 has the full review (6632524557 ④) and this PR body, both of which spell the pin out. Resolved.
  4. S1, the SKILL.md sentence. The sentence "SKILL.md is not touched here, because PR docs(skills): the platform skill teaches the package customizations read's contributions #30071 is editing it" is gone from the body. In its place: "SKILL.md is not touched here: it is at 6000 of its 6011-token ceiling, and teaching the push block's shape there is its own change." The ratchet on this head prints exactly 6000 tokens against a 6011 ceiling (headroom 11), so the sentence is true. PR docs(skills): the platform skill teaches the package customizations read's contributions #30071 is no longer mentioned anywhere in the body, so no claim about it remains to be wrong; for the record, docs(skills): the platform skill teaches the package customizations read's contributions #30071 is merged (merged at 2026-11-04T13:11Z), its squash commit 89803c5 is an ancestor of this PR's merge-base 61b487b (two commits below it) and therefore of the head, and the L42 / L310 the body cites are read from a tree that already carries it. Resolved.
  5. Every other body claim, re-read against 66bfe70. "Head 66bfe707, three commits on 61b487b2. Its tree is identical to 44939f51 … e871e3cd pinned the transport in the case and 66bfe707 reverts it": reproduces (①.1). "Why the case stays generic" now states plainly that "as written it still passes a push: {} or transport: 'relay' answer, which the boot refuses" — the fact the full review's ④ established, now acknowledged rather than hidden, with the pin routed to skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092. The What-changes bullets (prompt, expected output, the three must_contain and three must_not_contain strings) match case 12 byte for byte. "skills/objectstack-platform/SKILL.md teaches only two things about push today: … (L42) and … (L310)": sed -n '42p;310p' reproduces both lines on the head. "The fixture parses as JSON (12 cases). For every case, each must_contain string appears … and no must_not_contain string does": reproduces. "check-skills-token-ratchet.mjs: exit 0. Eval fixtures are measured, not ratcheted, so no ceiling row moved": reproduces. .changeset/29969-platform-push-eval.md: '@objectstack/skills': patch, Clause-②: no, Metadata-protocol: none line-initial, prose unchanged and still true of the diff (the last sentence, "not asserted until the skill teaches them", is now the body's stated position too). Body Clause-②: no and Metadata-protocol: none line-initial at L8–L9. "The facts were read on main from PR feat(push): requires push and the stack push key mount PushServicePlugin under os serve #29968": the two files named exist on the merge-base and are untouched by the seven newer main commits. Human Minimum: eight items ticked; item 3's "pnpm lint was not run" is still honest (Lint · ESLint success on the head); items 1 and 6 "not touched" true. Correct.
  6. Gate section. The <details> block now says "Local gates on 44939f5 and again on 66bfe70: 39 derived, 39 run with exit 0, 0 NOT MEASURED each time". The dev record for this head has 39 lines, every one :: exit 0; stripped of the exit suffix and sorted, its command set is identical to what dispatch-gates --commands prints on the head worktree here (39), and identical to the 44939f5 record. Of the 39, the plain node families I could run without node_modules all exit 0 here (①.1 and the Local-runs line); the pnpm check:* wrappers (doc-authoring, dts-closure, dual-build-cjs-loads, pm-governed-merges, published-files, …) and check-adr-0087-registration are read from CI, where Lint · repo & tooling gates, Lint · CI, release & engine gates, Lint · docs & code guards, Lint · PM dispatch-gates and Lint · scoped · tooling self-tests are all success on 66bfe70. "check-commit-card-trailers.mjs --range origin/main..HEAD: exit 0 on both heads (three commits on 66bfe70)": reproduces. "pnpm check:skill-top-level-keys exited 0": the underlying node scripts/check-skill-top-level-keys.mjs exits 0 here (53 keys at SKILL.md:33); check:changeset-fixed is read from Check Changeset success. Nothing I could measure contradicts the dev record.
  7. Model note and timestamps. The model note is unchanged: "The maintainer waived the model-tier mandate for skills/** while that tier had no quota; this PR was developed below it" — a tier phrase, no model identifier, no family word. A grep of the body for claude-, the four display-name spellings and four id forms the repo's recogniser lists as fixtures (check-commit-card-trailers.mjs L836–850), and "model" finds only the harness's <!-- claude-projects-attribution: … --> marker (not a model identifier; PR-body guards success), "model-tier mandate" and the "Claude Code" product name in the footer. A grep for YYYY-MM-DD followed by T or a space and hh:mm finds no date-time anywhere in the body, quoted or not. The three commit messages carry the model-free trailer pair (check-commit-card-trailers exit 0). Correct.
  8. The superseded head's red rows. On e871e3c the check-run rows Lint & Repo Gates and TypeScript Type Check read failure. Both belong to workflow run 41843010475 (Lint & Type Check, pull_request, created at 2026-11-04T13:58Z), whose conclusion is cancelled: its ten lane jobs (Lint · ESLint, Lint · repo & tooling gates, Lint · docs & code guards, Lint · PM dispatch-gates, Lint · CI, release & engine gates, the four Type Check · lanes) are cancelled, three scoped lint lanes finished success before the cancel, and the two aggregator jobs that needs the lanes report failure because their dependencies were cancelled, not because anything failed. The cancel coincides with the push of 66bfe70 (committed at 2026-11-04T13:59Z; its own Lint & Type Check run 41843247871 was created at 2026-11-04T14:00Z and finished success). On the full-battery run for e871e3c, Build Core, Plan Test Core shards, Live E2E UI, Check Changeset and the Test Core matrix row are likewise cancelled. In one line: the red TypeScript Type Check and Lint & Repo Gates rows on e871e3c are the cancelled run of a head that lived about ninety seconds and is not this head; on 66bfe70 both floors are success.
  9. Review state on this head. The PR thread carries the seat's Contract-review-claim: for 66bfe70 (6632942931, naming this as a delta review of the FAIL) and no record on this head; reviews: 0; draft, auto_merge null, mergeable_state clean (all check-runs complete), no assignee or requested reviewer, labels documentation, size/s, tooling. Tier H (skills/**) still; this record does not lift the landing.

② Semver level

Unchanged: patch, and the changeset still says patch. The delta is the PR body and two commits that leave the tree where it was; @objectstack/skills publishes skills/**, so the new eval case is a user-visible (inert) addition to a published package with no protocol change: Clause-②: no, Metadata-protocol: none on both the body and the changeset. No packages/spec/src/**, no skip-changeset; Check Changeset is success on the head on both of its rows and check-empty-changeset exits 0 here.

③ Boundary flags

④ Design judgment

Per hits file, for the delta only:

Must-fix: none. Should-fix: none remaining — F1 (by option (b)) and S1 of 6632524557 are both resolved on this head.

Check-runs at record time: 45 check-runs on 66bfe70, 30 success, 15 skipped, 0 failure, 0 in progress; combined status pending (statuses API, 0 statuses). Success includes both required floors (Lint & Repo Gates, TypeScript Type Check), every Lint · and Type Check · lane, Build Core, Plan Test Core shards, Test Core with its two run shards, Test UI, Live E2E UI, Dogfood Regression Gate, Governed Surface Queue Guard, Check Changeset (both rows), Check PR Size, Auto Label, filter and PR-body guards (both rows, the second after the body edit). Skipped by path filter or opt-in: Build Docs, Build UI Deps, Build & E2E UI, Lint UI, Type Check UI, UI Published Artifacts, Dogfood Verify CLI, Temporal Conformance, Schedule gate, 18.x upgrade harness, Packed-tarball smoke, the matrix-templated placeholder rows, and the Auto Label / Check PR Size rows of the body-edit run. The red rows on e871e3c are the superseded head's cancelled run (①.8), not this head. mergeable_state clean, draft, auto_merge null; Tier H (skills/**), landing by the maintainer's hand or an authorized approval, not by this record.

Implemented-by: claude/project-thread-xxbrzx
Reviewed-by: session_01J8VVAnMUWA9rufXzFaEh4J

VERDICT: PASS


Generated by Claude Code

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J

维护者速读

席位核对 — ACCEPT · domain:skills#1 · head 66bfe707d27053ed0d7790d1f43d3c94a8bdb53c

  1. 形态: draft → main,Part of #29969(无关闭关键字);作者 objectstack-fleet[bot];两个文件都在认领面内,附 @objectstack/skills patch changeset(⛔ 不挂 skip-changeset)。
  2. 前提实测: 复核对过 packages/spec/src/system/push-config.zod.ts 与 packages/services/service-push/src/capability-arg.ts;SKILL.md 6000 / 6011 由棘轮复现;head 的树与全量复核所读的 44939f51 一致(1af464ec)。
  3. 门禁(钉 head 66bfe707): check-runs success 30 / skipped 15,无失败、无进行中。派生门禁 39 条全过(--ran 记录在 PR 正文)。e871e3cd 上的两条红行是被取代 head 的已取消运行,不属本 head。
  4. 接受的偏差: 开发在低于规定档位的模型上进行,依维护者原话(无额度时豁免)。
  5. 范围外去向: 卡片 Wanted (2) 与 SKILL.md push 块教学由 skills(objectstack-platform): the skill names the push key and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092 承接;本 PR 合入后 skills(objectstack-platform): an eval case for the push capability and push stack key (follow-up of #29820) #29969 按部分落地处理。
  6. 落地条款: Tier H。保持 draft,挂 needs-user-decision,同轮请 os-zhuang、hotlong 审批;获批后本席过落地前检、清标、ready、auto-merge;维护者直合亦可。

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/s tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants