Repository navigation
skills(automation): integrations are declared, not coded — rule section, per-archetype recipes, three evals - #27322
Conversation
…on, per-archetype recipes, three evals (WIP) Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U
|
epic:#27293#1 · sunhaolin · session_017xRBcVBKAFk1ntXSjvPt7U Contract reviewServed-tier: Reviewed at 2026-10-28T17:15Z, read-only: the card (body and all seven comments), the PR body, its file list, the net diff against Check-runs on the head (the gate verdicts): 42 runs, all completed — 28 ① Derived judgments
② Semver level
③ Boundary flagsEvery dev flag answered;
④ Design judgment
Observations outside this PR's claims (no finding, carrier noted): (a) package-authored scheduled work is switched off by default and listed as "disabled by deployment policy" ( Attempts to break it that did not land: a CEL envelope inside Implemented-by: VERDICT: PASS Must-fix: none. Should-fix: (1) SKILL.md five-condition sentence — say platform code is the platform's, filed as a card, the app writes none; (2) recipe 1 |
… conditions license, dereference every $ref offline, api row names secretRef, eval token clientSecret: Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U
|
epic:#27293#1 · sunhaolin · session_017xRBcVBKAFk1ntXSjvPt7U Contract reviewServed-tier: Delta review at 2026-10-28T17:48Z, read-only, of the one commit between the PASS head Check-runs on this head (the gate verdicts): 42 runs, all completed — 28 ① Derived judgments
② Semver levelUnchanged by the delta: ③ Boundary flagsThe delta carries no new dev flag and no
Escalations from the first record stand unchanged (the card's " ④ Design judgment
Tried to break it: an Must-fix: none. Should-fix: none. Implemented-by: VERDICT: PASS |
|
epic:#27293#1 · sunhaolin · session_017xRBcVBKAFk1ntXSjvPt7U 维护者速读
席位意见: 赞成合入。两轮达档契约复核 PASS( 席位核对ACCEPT at head
|
Fixes #27223
Clause-②: no
Metadata-protocol: none
维护者速读
objectstack-automation技能包新增规则节「Integrations are declared, not coded」(厂商系统走应用内声明的connectors:条目 +connector_action,回调走api流;五种情形才写平台代码;加载器暂不支持的文档是「待平台卡」的缺口,不是写插件的理由),connector_action节点行不再写 "pre-built";新增references/integration-recipes.md(六种原型各一份配方 + 作者/管理员凭据交接 + 两条「尚未支持」)和evals/integrations-declared.json(DocuSign / Stripe 读 / 无规格厂商三例,DocuSign 例对任何「写平台插件」的答案判失败)。skills/objectstack-automation/SKILL.md新节(约第 402 行起)——五种「写平台代码」的情形与缺口命名的措辞;②references/integration-recipes.md「Holds on every recipe」第二条(connectorConfig.input按原样发送,逐记录 payload 今天不可声明)——这是实测到的运行时事实,也是下方 Acceptance notes 第 1 条发现;③evals/integrations-declared.json第 1 例的 must_not_contain。node scripts/check-skills-token-ratchet.mjs && node scripts/pm/check-skill-id-lint.mjs——预期两条均 ✓,automation 五个计价文件 headroom 全为 0。What changes
skills/objectstack-automation/SKILL.md— theconnector_actionnode row no longer says "pre-built integration connector"; the "reserve custom code for edge-case integrations" sentence is gone; a new section Integrations are declared, not coded states the rule, the five cases that justify platform code, and that a document the loader cannot take yet (YAML, Swagger 2.0,$ref, OAuth, request signing) is a gap to name against its platform card, never a reason to write code. Thehttprow's auth-routing tail moved into the recipes. Paid for in the same file: net −1 token under the ratchet, headroom 0.skills/objectstack-automation/references/integration-recipes.md(new, 150 lines) — one recipe per archetype: OpenAPI with a static key (Stripe balance, nightly), remote MCP with a static header (GitHub), plain REST (provider: 'rest',actionId: 'request'), the vendor callback (apiflow withsecretRef), a hand-written OpenAPI 3.x document for a spec-less vendor, and the two not yet entries (OAuth-protected, signature-authenticated). Each states what the app author declares and what the customer's administrator supplies. It also carries three limits measured on this base:connectorConfig.inputis handed to the provider as written (no CEL envelope or{{ }}hole is evaluated —service-automation/src/builtin/connector-nodes.ts:108, andvalue-slot-map.tslists no connector slot), theopenapiprovider has noincludekey (connector-openapi/src/openapi-provider.ts:38-52— the trimmed document is the allowlist), and the inboundapidoor verifies onlyx-objectstack-signaturein the timestamped form (trigger-api/src/api-trigger.ts:163-178, "has no unsigned branch"), so a vendor signing its own header cannot post today.skills/objectstack-automation/evals/integrations-declared.json(new) — the DocuSign case (expected: a declarativeopenapiinstance over a trimmed, packaged 3.x document converted offline from Swagger 2.0, anapicallback flow, three named gaps, marked blocked on OAuth;must_not_containrefusesregisterConnector(,connector-esign,provider: 'docusign',implements Plugin,clientSecret,token:), the Stripe-read case (scheduled flow writingcrm_accountfields) and the vendor-without-spec case.scripts/check-skills-token-ratchet.mjs— the SKILL.md row re-locked at its measurement (6466 → 6465); two rows pin the new files at their landed counts (1193, 2177)..changeset/27223-integrations-declared.md—@objectstack/skillspatch (the catalog ships in that package'sdistviasync-catalog.mjs).Card numbers are not in the skill text. The card asks the recipes to name the lifting cards.
pnpm check:pm-skill-id-lintforbids#NNNin every.mdunderskills/by maintainer ruling (2026-08-12, widened to the published catalog on 2026-10-12) — measured on this tree: a probe lineWaits on #26709 for include.underreferences/reds the gate (exit 1, naming the file and line). The recipes therefore name each gap and say it is blocked on "its platform card"; the numbers live here:includeon theopenapiprovider and offline normalization → #26709;oauth2in the declarative auth union → #26702; request signing → #26710; the administrator-filled credential object → #27237 (in flight; the recipes teach today's env-varcredentialRefonly).Bundle growth, with its authorization. The published bundle grows by 3,370 tokens (two new files pinned at landed counts; SKILL.md −1). The ratchet lands growth only against a maintainer ruling quoted in the PR body; this one is the card's own directive, verbatim and untranslated: 「所以应该先处理 A/B 吧,并且要有 skills或者文档指导元数据开发者。前面agent还在建议我开发 DocuSign」 and 「hotclm 的开发agent 就搞错了,说明 skills还是不对,要立卡完善」(2026-10-28). The PM-set budget — SKILL.md net tokens ≤ 0, recipes ≤ 150 lines, one eval file — is met. The claim's "one entry in
references/_index.md" does not apply: that file is generator-owned (gen:skill-refswrites schema pointers only);check:skill-refspasses with the hand-written reference beside it, and SKILL.md links the recipes directly.Tests
No package source changed, so no
pnpm test/pnpm typecheckis owed; the edited gate script's own suite is its--self-test(79 cases pass, below).Skill evals — what exists and what was run. The root repo has no eval runner: no
pnpmscript consumesskills/*/evals/*.json, and the only eval gate in the tree,ui/scripts/check-skill-eval-tokens.mjs, scansui/skillsalone. The card's "pnpmskill evals green" therefore cannot be run as spelled. What was run instead, from a scratch replay (not committed) of that gate's whole-token rule — everymust_containtoken must occur as a whole token in the bundle's.mdcorpus — overskills/objectstack-automation/:Oracle, base bundle (a0aa8a1): the three new evals FAIL — exit 1
Oracle, this branch (2993d73): the three new evals PASS — exit 0
NOT MEASURED: a live LLM answering arm per eval (base text vs new text) — no runner exists and a self-graded run by the authoring seat is not evidence; the DocuSign case's refusal of a plugin answer is enforced by its
must_not_containlist, not by a measured model run.Acceptance notes
connectorConfig.inputon aconnector_actionnode is handed to the provider handler as written (packages/services/service-automation/src/builtin/connector-nodes.ts:108:handler((cfg.input ?? {}) …));builtin/value-slot-map.tsresolvesfields,subflow/mapinput,scriptinputs,screendefaults,notifytemplateDataandhttpbody, never connectorinput. The authoring door accepts a CEL envelope there:FlowSchema.safeParseof a flow whoseconnectorConfig.inputis{ path: { customer: { dialect: 'cel', source: 'record.stripe_customer_id' } } }answerssuccess = trueand keeps the envelope verbatim (measured withtsxagainstpackages/spec/src/automation/flow.zod.tson this tree). Theopenapihandler then interpolates the path fromString(value), so the vendor receives an object stringified. Reach:os validaterunsFlowSchema; producer: any app author's stored flow definition. Dedupe words:connector_action input value slot CEL envelope verbatim. The recipes state the limit; they do not work around it.pnpmskill evals" does not exist for the root catalog — the eval fixtures underskills/*/evals/have no mechanical consumer in this repository;ui/has one for its own bundle. Observation for the PM (the acceptance wording), not a card from this seat.evals/flows-triggers-approvals.jsoncarry untaught tokens on base as well as on this branch — eval 7 (excludeSubmitter: true) and eval 8 (type: 'screen',type: 'file',signed_copy). No root gate asserts the rule; carrier: none.#25747(the DocuSign instance the card cites as open) is closednot_plannedas of 2026-10-28 — no effect on the premise, which was re-verified onorigin/maina0aa8a1:pre-built integration connectoratSKILL.md:89;defineConnector|provider: 'openapi'→ 0 files underskills/*/referencesandskills/*/evals;Declaring a provider-bound instanceatcontent/docs/automation/connectors.mdx:40.references/examples-flows.md:49-55(the plugin-registeredslackconnector in aconnector_action) is outside the declared surface and was not edited; it stays a legitimate shape, and the rewrittenconnector_actionrow says so ("or a plugin's").runAs: 'system'paragraph in SKILL.md says a system sweep must pin its organization but names no column, and the catalog's own scheduled example carries no predicate; the Stripe recipe mirrors the taught shape and adds a one-line reminder. Read-only inference; noted, not filed.PREREQUISITE NOT MET: they read builtdist/, which this diff never builds) —check:doc-formula-expressions,check:dts-closure,check:dual-build-cjs-loads,check:lean-entry-closure,check:sourcemap-no-sources-content. They are pulled in by thescripts/touch, read none of the changed paths, and run in CI after the build; recorded as NOT MEASURED locally.Verification
Head
2993d737(tree identical to the readings below; no commit followed them).skills/** readings — the two required before/after counts (lines and tokens)
skills/objectstack-automation/SKILL.mdskills/objectstack-automation/references/integration-recipes.mdskills/objectstack-automation/evals/integrations-declared.jsonskills/*/SKILL.md)Token ratchet and id-lint (exit codes captured before any pipe)
Derived gate families — `dispatch-gates --commands` on the actual diff, every command run, reconciled with `--ran`
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderived 64 commands for the 5 changed paths; each ran asbash -c CMD > log 2>&1; code=$?and was recorded asCMD :: exit CODE.node scripts/pm/dispatch-gates.mjs --ran gates-ran-coded.txt --repo objectstack-ai/objectstack-> exit 0: "✓ dispatch-gates --ran: 64 derived famil(ies) accounted for — 59 run, 5 NOT-MEASURED (5 DERIVED from a recorded exit 3)."Exit 0 (59), among them:
check:skill-docs,check:skill-refs(@objectstack/spec),check:doc-authoring,check:nul-bytes,check:pm-skill-id-lint,check:skill-identifier-liveness,check:skill-frame-sync,check:skill-compatibility,check:ratchet-remedy-authority,check:pm-dispatch-gates(412 s),check:changeset-gate-self-tests,check-empty-changeset --base origin/main,check-adr-0087-registration --base origin/main,check-changeset-no-major --base origin/main,check:published-files,check:merge-driver,check:cross-package-test-inputs.Exit 3 (5,
PREREQUISITE NOT MET, NOT MEASURED locally):check:doc-formula-expressions,check:dts-closure,check:dual-build-cjs-loads,check:lean-entry-closure,check:sourcemap-no-sources-content.Not derived, run on their own:
check:skills-token-ratchetfamily is in the derived list; nopackages/specbuild was needed (no package source changed, andcheck:skill-docs/check:skill-refsrun from source).FlowSchema probe for Acceptance note 1
CI: the PR's check-runs are
in_progressat report time; the PM reads their convergence.Generated by Claude Code