Repository navigation
test(skills): the platform skill's evals ask for push notifications on os serve - #30101
Conversation
…n os serve Case 12 in objectstack-platform/evals/config-plugins-ops.json asks to turn on push for an os serve deployment; the expected answer declares requires: ['push'] with a top-level push: key and writes no secret inline. The assertions stay with what SKILL.md teaches today. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T
|
domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J Contract-review-claim: session_01J8VVAnMUWA9rufXzFaEh4J · head 44939f5 · 2026-11-04T13:45Z The seat review runs at CONTRACT_REVIEW_TIER in an isolated subagent (the seat session is not at that tier). No review seat ( Generated by Claude Code |
|
domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J Contract reviewServed-tier: Reviewer: an isolated contract reviewer at CONTRACT_REVIEW_TIER, read-only on GitHub (REST reads only; nothing written, pushed or commented), fed the card body and comments, the PR body, the net diff against the merge-base, the head's check-runs, AGENTS.md, the pm-dispatch SKILL.md contract-review section, contract-review.md, review-checklist.md, lanes/skills.md and lanes/review.md. The dev gate record was read as input, not as conclusions; every count and every claim about the spec, the push service and the skill below is this review's own reading of the head tree and of ① Derived judgments
② Semver levelPatch, and the changeset says patch. ③ Boundary flags
④ Design judgmentPer hits file, against the card's scenario and the code on
Must-fix: F1 — case 12 accepts a boot-refused answer and leaves the card's Wanted (2) unpinned while the PR says Check-runs at record time: 39 check-runs on 44939f5, 21 success, 13 skipped, 0 failure, 5 in progress on the third read (Test Core 1/8 and 2/8, Type Check · workspace, Type Check · consumer gates, Type Check · debt ledger); three reads went 38 → 38 → 39 runs and 15 → 19 → 21 success with no non-success completion at any read. Success: Lint & Repo Gates (a required floor), Lint · repo & tooling gates, Lint · CI, release & engine gates, Lint · PM dispatch-gates, Lint · docs & code guards, Lint · ESLint, Lint · scoped · tooling self-tests, Lint · scoped · slot lookup & corpus, Lint · scoped · query-options erasure, Type Check · source gates, Build Core, Governed Surface Queue Guard, Dogfood Regression Gate, Test UI, Plan Test Core shards, Live E2E UI, PR-body guards, Check Changeset, Check PR Size, Auto Label, filter. The other required floor, Implemented-by: VERDICT: FAIL Generated by Claude Code |
…s the relay Case 12's expected answer names transport: 'direct' with apns / fcm blocks, each with its credentialRef, and says the hosted relay (also an absent or empty push block) is refused at boot until its sender ships; the case adds transport: 'relay' to must_not_contain. The changeset says the same. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T
…d refuses the relay" This reverts commit e871e3c. Pinning the transport moves to the follow-up that first teaches the push block's shape in the skill; the case stays as reviewed. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T
|
domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J Contract-review-claim: session_01J8VVAnMUWA9rufXzFaEh4J · head 66bfe70 · 2026-11-04T14:15Z Delta review against the FAIL at head 44939f5 (tree identical at 66bfe70; the delta is the PR body, option (b) of F1, and S1). The seat review runs at CONTRACT_REVIEW_TIER in an isolated subagent. No review seat ( Generated by Claude Code |
|
domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J Contract reviewServed-tier: Reviewer: the same isolated contract reviewer at CONTRACT_REVIEW_TIER, read-only on GitHub (REST reads only; nothing written, pushed or commented). The full review on 44939f5 (FAIL: must-fix F1, should-fix S1) stands as read for everything it measured on the tree, because the tree has not changed; this record judges only the delta — the two commits that cancel each other, and the PR body as it now reads against the head, card #29969 and card #30092. The dev gate record was read as input, not as conclusions. No environment variable value is printed anywhere in this record. Dates quoted from GitHub are written in the placeholder form, not typed. ① Derived judgments
② Semver levelUnchanged: patch, and the changeset still says patch. The delta is the PR body and two commits that leave the tree where it was; ③ Boundary flags
④ Design judgmentPer hits file, for the delta only:
Must-fix: none. Should-fix: none remaining — F1 (by option (b)) and S1 of 6632524557 are both resolved on this head. Check-runs at record time: 45 check-runs on 66bfe70, 30 success, 15 skipped, 0 failure, 0 in progress; combined status Implemented-by: VERDICT: PASS Generated by Claude Code |
|
domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J 维护者速读
席位核对 — ACCEPT · domain:skills#1 · head
|
domain:skills#1 · huangyiirene · session_01J8VVAnMUWA9rufXzFaEh4J
Requested by Irene · project thread
Part of #29969
Clause-②: no
Metadata-protocol: none
The maintainer waived the model-tier mandate for
skills/**while that tier had no quota; this PR was developed below it.维护者速读
skills/objectstack-platform/evals/config-plugins-ops.json新增第 12 个 eval:提示要求给os serve部署开启推送;期望答案在栈根声明requires: ['push']、写顶层push:键,且不内联任何密钥(APNs .p8 与 Firebase 服务账号 JSON 都按引用命名)。另加一个'@objectstack/skills': patchchangeset。requires: ['push']与栈的push键在os serve下挂上PushServicePlugin,其 changeset 记下了本卡这一评测欠账。assertions。断言只覆盖SKILL.md现在教的内容(L42 顶层push键、L310pushtoken),不断言 transport / apns / fcm / relay,因为SKILL.md还没教 push 块的形状。卡片 Wanted (2)(钉住transport: 'direct'与被拒的 relay)移到 skills(objectstack-platform): the skill names thepushkey and capability but never teaches the push block's shape (transport, apns / fcm, credentialRef) #30092:先在技能里教 push 块形状,再扩展本 case;故本 PR 是Part of #29969。skills/**),需要GOVERNED_APPROVERS的批准;PR 保持 draft。What changes
skills/objectstack-platform/evals/config-plugins-ops.jsongains case 12, in the sameprompt/expected_output/files/assertionslayout as cases 1–11:os servedeployment whose iOS and Android apps are signed with the team's own APNs key and Firebase project.requires: ['push']on the stack root, which mounts@objectstack/service-push, plus a top-levelpush:key. There is no host code and noPushServicePlugininplugins:. The APNs signing key and the service-account JSON stay in the secrets layer and are named by reference (credentialRef). Nothing is written inline.requires,'push',push:.BEGIN PRIVATE KEY,privateKey:,"private_key".Why the case stays generic.
skills/objectstack-platform/SKILL.mdteaches only two things about push today:pushamong the stack's top-level keys (L42) and thepushtoken in therequirestable (L310). It does not yet teach the push block's shape:transport, theapns/fcmvendor blocks,credentialRef, or the boot refusing the hosted relay. So the case asserts the stack-level shape the skill teaches and the no-inline-secret rule, and does not yet pin the transport: as written it still passes apush: {}ortransport: 'relay'answer, which the boot refuses.SKILL.mdis not touched here: it is at 6000 of its 6011-token ceiling, and teaching the push block's shape there is its own change.What moves to #30092. The card's Wanted (2), pinning
transport: 'direct'and the refused relay, moves to #30092, which first teaches the push block's shape in the skill and then extends this case. HencePart of #29969, notFixes.The facts were read on
mainfrom PR #29968:packages/spec/src/system/push-config.zod.ts(PushServiceConfigSchema, withcredentialRefa secrets-layer reference and the inlineprivateKey/key/serviceAccountkeys refused) andpackages/services/service-push/src/capability-arg.ts..changeset/29969-platform-push-eval.md:'@objectstack/skills': patch, with both declaration lines.Verification
Head
66bfe707, three commits on61b487b2. Its tree is identical to44939f51, the head the contract review read:e871e3cdpinned the transport in the case and66bfe707reverts it, because that pin moves to #30092.must_containstring appears in itsexpected_outputand nomust_not_containstring does.node scripts/check-skills-token-ratchet.mjs: exit 0. Eval fixtures are measured, not ratcheted, so no ceiling row moved.Local gates on 44939f5 and again on 66bfe70: 39 derived, 39 run with exit 0, 0 NOT MEASURED each time (`dispatch-gates --ran`)
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderived 39 commands for the two changed files. All of them ran afterpnpm install --frozen-lockfileand a fullpnpm build(exit 0). Each had a 900 s timeout, and its exit code was captured before any pipe.check-adr-0087-registration.mjs --base origin/main,check-changeset-no-major.mjs,check-empty-changeset.mjs,check:changeset-gate-self-tests,check-comment-mask-corpus.mjs,check:doc-authoring,check:skill-compatibility,check:skill-frame-sync,check:skill-identifier-liveness,check:dts-closureandcheck:dual-build-cjs-loads.pnpm check:changeset-fixedandpnpm check:skill-top-level-keysboth exited 0.--ranprinted:39 derived famil(ies) accounted for — 39 run, 0 NOT-MEASURED.node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD: exit 0 on both heads (three commits on 66bfe70).Human Minimum
packages/specholds no runtime logic. (Not touched.)pnpm lintwas not run: the diff is one JSON fixture and one changeset, no code.)main, one pull request.packages/spectouched: its generated artifacts are regenerated and committed. (Not touched.)content/docs/references/,packages/*/CHANGELOG.mdorcontent/docs/releases/.🤖 Generated with Claude Code
https://claude.ai/code/session_01J8VVAjFpbQxLtYHuw3di1T