From 205d191590d5fd1176e73632d8b789efda2d600e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 02:32:37 +0000 Subject: [PATCH 01/13] feat(scripts): add the engine.ts split codemod scripts/codemod-split-engine.mjs is the large-file programme's codemod for packages/objectql/src/engine.ts, modelled on the sql-driver pilot: whole sections of ObjectQL's method bodies move to section modules behind the hand series' host pattern, the top-level declarations ahead of the class move whole into a leaf, and the script proves byte-identity, determinism and the declaration-file bar. Sections are named by anchor member, and the import bindings only moved text read are pruned (noUnusedLocals), with the prune recomputed from the head in the proof. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- scripts/codemod-split-engine.mjs | 2163 ++++++++++++++++++++++++++++++ 1 file changed, 2163 insertions(+) create mode 100644 scripts/codemod-split-engine.mjs diff --git a/scripts/codemod-split-engine.mjs b/scripts/codemod-split-engine.mjs new file mode 100644 index 00000000000..a602bc589f8 --- /dev/null +++ b/scripts/codemod-split-engine.mjs @@ -0,0 +1,2163 @@ +#!/usr/bin/env node +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * codemod-split-engine — move whole sections of `ObjectQL`'s method bodies out + * of `packages/objectql/src/engine.ts` into section modules, move the + * top-level declarations ahead of the class into one leaf module, + * deterministically, and prove the move. + * + * node scripts/codemod-split-engine.mjs # plan: print the section table, write nothing + * node scripts/codemod-split-engine.mjs --write # apply the split, then verify it + * node scripts/codemod-split-engine.mjs --verify --base + * # re-prove a split on disk against engine.ts at + * node scripts/codemod-split-engine.mjs --dts + * # the declaration-file bar on two built dists + * node scripts/codemod-split-engine.mjs --self-test + * node scripts/codemod-split-engine.mjs --help # this usage, on any tree (split or not) + * + * ## Why a script, and what it repeats + * + * The large-file programme's ruling A (`6583399215`) splits each file by a + * deterministic codemod that applies the file's existing cut pattern to whole + * sections, regenerated on the latest `main` at landing instead of rebased. + * `scripts/codemod-split-sql-driver.mjs` is the pilot; this is the same + * programme for `engine.ts`, and it applies the cut pattern THIS file's hand + * series (21 cuts, 25 `engine-*.ts` modules) already uses, which is the + * pilot's pattern too: + * + * - The method stays on the class with its docblock, modifiers and signature + * byte-for-byte, so the published `.d.ts` keeps its names and positions and + * an instance spy or a subclass override is still the one callers reach. + * Only `async` is dropped from a stub, as every async stub of the series did. + * - The body moves, unedited, into an exported function of the same name in + * `engine-
.ts`, whose `this` is one named host: the `ObjectQL` + * members the body reads, typed AS the class members (`ObjectQL['x']`). + * A private member is reached the same way as a public one: the indexed + * access type names it, and the stub, inside the class, reads it. + * - The stub hands the body that host: a method as a closure that calls it on + * the engine at call time, a field or accessor as a GETTER that reads it + * when the body reads it. + * + * Ruling `6591753301` (option A) widened the pattern two ways, and this script + * applies both, as the pilot does: + * + * - (a) A section module may import VALUES from `engine.ts`. A staying + * top-level name a moved body reads is exported for it by one generated + * `export { … }` list (the declaration's own text is not touched), and + * `index.ts` re-exports by name, so none reaches the package. Every such + * use is inside a function body, so the cycle is never read at evaluation. + * - (b) The top-level declarations ahead of the class (`const` / `function` / + * `interface` / `type`) move WHOLE, with the comments above them, into one + * leaf module, `LEAF_MODULE`, in base order. The leaf never imports + * `engine.ts` or an `engine-*` module: a declaration that names the class, + * an `engine-*` import or a declaration that stays, stays itself. `engine.ts` + * imports back what it still reads and re-exports, by name, every moved + * declaration it exported, so every importer of `./engine.js` (`index.ts`, + * the hand-series modules, the tests) reads what it read. + * + * Two things this file needs that the pilot's did not: + * + * - SECTIONS BY ANCHOR. `engine.ts` has five banners in a 17,000-line class, + * so a section is named by its FIRST member, in `SECTION_ANCHORS` (the + * `protocol.ts` codemod's form): a member belongs to the last anchor at or + * above it. An anchor the class does not declare, or anchors out of class + * order, stop the script (exit 2): a new section is a decision someone + * writes here, never a name the script makes up. + * - AN IMPORT PRUNE. The package compiles with `noUnusedLocals`, so an import + * binding only moved text read is dropped from `engine.ts` + * (`pruneImports`). It is the one edit the undo cannot read back from the + * head; the proof recomputes it from the head and requires every pruned + * name to occur in a moved body or a moved declaration. + * + * ⛔ The moved text is not re-indented and not formatted: the body is the + * method's `{ … }` exactly as it stood, so byte-identity is equality of text, + * not equality after a transform. + * + * ## What stays, and why (each reason is mechanical) + * + * A method stays whole on the class when moving it would change what it does, + * what the published `.d.ts` says, or what a gate or test that reads + * `engine.ts` BY PATH sees. `STAY_REASONS` lists every reason for a method and + * `TOP_STAY_REASONS` every reason for a top-level declaration; the plan prints + * every one that stayed and why. The reasons that are tables rather than rules + * are `PATH_READER_PINS` (a name whose text a reader extracts from `engine.ts` + * by path), `PATH_READER_NEEDLES` (literal text a reader requires to find + * there) and `CLASS_PATH_READER_NEEDLES` (the same, for a reader that also + * reads the leaf, so only a method carrying it stays). Ruling `6597555122` + * item 3: a reader that can read the new modules gains them in its reading + * list in the same PR instead, and needs no row here. + * + * ## The proof (`--write` runs it on its own output; `--verify` re-runs it) + * + * 1. Byte-identity: every generated function's body, parameters and return + * type are the base method's text, exactly; every leaf declaration is its + * base statement's text with the comments above it, exactly; and putting + * every body back (and `async` back on its stub), taking the generated + * imports and exports out, and splicing the leaf back between the + * statements that stayed rebuilds the base `engine.ts` byte-for-byte once + * the base's import bindings the head no longer reads are pruned from it + * (the prune above), so nothing else in it moved. The leaf's relative + * VALUE imports are followed on disk: none may reach `engine.ts` or an + * `engine-*` module. + * 2. The declaration-file bar of ruling `6348891160` item 1, on the built + * `dist/index.d.ts` and `dist/index.d.mts` (`--dts`), as the pilot runs it. + * A `.d.ts` order-only difference is judged by ruling `6441032442` B's + * second normalisation (ruling `6597555122` item 2), a report reading on + * the two dists, never a `scripts/` file. + * + * ## Determinism + * + * The input is the text of `engine.ts`. It is parsed in an in-memory project + * (no `node_modules`, no build output, no clock), every collection is emitted + * in source order or sorted, and the output is a pure function of the input: + * two runs on one input write byte-identical trees (the plan prints the output + * tree's sha256). Only the proof's leaf-reach walk reads sibling files. + * + * ts-morph is a root devDependency, resolved from the root manifest, as the + * pilot takes it. + * + * Exit codes: 0 done · 1 a proof failed (stop: the generated split is never + * hand-edited to pass) · 2 the script could not run (unknown anchor, missing + * prerequisite, bad arguments) · 3 an input does not parse (`ts-parse.mjs`'s + * refusal, which names the parse errors). + */ + +import { createHash } from 'node:crypto'; +import { execFileSync } from 'node:child_process'; +import { existsSync, readFileSync, writeFileSync } from 'node:fs'; +import { createRequire } from 'node:module'; +import { dirname, join, relative, resolve } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +import { isEntrypoint } from './invoked-as.mjs'; +import { parseSourceFile } from './ts-parse.mjs'; + +const ROOT = join(fileURLToPath(new URL('.', import.meta.url)), '..'); +const SRC_REL = 'packages/objectql/src'; +const SRC_DIR = join(ROOT, SRC_REL); +const HOST_BASENAME = 'engine.ts'; +const HOST_PATH = join(SRC_DIR, HOST_BASENAME); +const CLASS_NAME = 'ObjectQL'; +const CARD = '#29986'; +const SCRIPT = 'scripts/codemod-split-engine.mjs'; + +/** The marker every generated file and stub carries; `--verify` keys on it. */ +const MARK = `[${CARD} codemod]`; + +/** A generated or hand-series section module of `engine.ts`, by its import specifier. */ +const SEAM_SPECIFIER = /^\.\/engine-[a-z0-9-]+\.js$/; + +/** + * The sections of `ObjectQL`, top to bottom, each named by its first member + * (a field or a method), mapped to the module its movable bodies go to. + * `null` keeps the section on the class. No module shares a file with a + * hand-written one: the script refuses to write over a file it did not + * generate, and the hand series' 25 module names are not reused. + */ +const SECTION_ANCHORS = Object.freeze([ + ['txStore', 'Class head: units, hooks, functions, body runners, hook dispatch', 'engine-hooks-and-functions.ts'], + ['registerAction', 'Action system and packaged-action activation', 'engine-actions.ts'], + ['_heldFileResolver', 'Registration seams, readable-ids security, formula and default permissions', 'engine-security-resolvers.ts'], + ['writeEpoch', 'Authz invalidation, middleware, session, driver options, posture, organization probe', 'engine-middleware-and-context.ts'], + ['resolveNowDefault', 'Field defaults and autonumbers', 'engine-defaults-and-autonumbers.ts'], + ['registrationConversionWarned', 'Package, app, plugin, driver and datasource registration', 'engine-registration.ts'], + ['setRealtimeService', 'Realtime, i18n, home amounts, validation messages, references, master-detail', 'engine-references-and-master-detail.ts'], + ['setCryptoProvider', 'Crypto, secrets, internal fields, schema and datasource routing, init', 'engine-secrets-and-routing.ts'], + ['objectHasMediaField', 'Migration flags, value-shape deviations, creation attestations, destroy', 'engine-migration-state.ts'], + ['MAX_EXPAND_DEPTH', 'Summary indexes, related-record expansion, file and reference hydration', 'engine-summaries-and-expansion.ts'], + ['resolveWhereTokens', 'Query lowering, search, find and findOne', 'engine-find.ts'], + ['staticReadonlyCreateStrip', 'Write preparation and insert', 'engine-insert.ts'], + ['update', 'Update', 'engine-update.ts'], + ['planCascadeAtomicity', 'Cascade and referential delete planning', 'engine-cascade-planning.ts'], + ['delete', 'Delete, count, aggregate and execute', 'engine-delete-and-aggregate.ts'], + ['transaction', 'Transactions', 'engine-transactions-section.ts'], + ['registerObject', 'Compatibility / convenience API', 'engine-compatibility-section.ts'], +]); + +/** + * The leaf the top-level declarations ahead of the class move to, whole + * (lever (b) of ruling `6591753301`). A leaf: it never imports `engine.ts` or + * any `engine-*` module, so it is evaluated before the class and forms no + * cycle with it. + */ +const LEAF_MODULE = 'engine-top-level.ts'; + +/** + * Names whose text a reader takes out of `engine.ts` BY PATH. Moving it would + * leave that reader reading a stub, or nothing. A class method or a top-level + * declaration of the name stays. Each row names the reader. A row naming + * nothing the file declares stops the script (exit 2), so the table cannot rot + * into names nobody reads. + */ +const PATH_READER_PINS = Object.freeze({ +}); + +/** + * Literal text a reader requires to find in `engine.ts` BY PATH. A class + * method or a top-level declaration whose text carries one stays, so the + * reader still finds it where it looks. Each row names the reader. A needle + * the file no longer carries stops the script (exit 2). + */ +const PATH_READER_NEEDLES = Object.freeze({ +}); + +/** + * Literal text a reader requires to find in `engine.ts` OR the leaf, but reads + * in no section module. A method whose text carries one stays on the class; a + * top-level declaration carrying one moves as any other, because the reader + * reads the leaf it moves to. The same stale-needle refusal applies. + */ +const CLASS_PATH_READER_NEEDLES = Object.freeze({ +}); + +/** + * The `ObjectQL` methods a host hands its body as a BOUND reference + * (`this.m.bind(this)`) rather than as a closure that calls it. The pilot + * needed one for a gate that refuses an unbound builder call; no gate reads + * `engine.ts` that way, so the set is empty and every method goes as a + * closure. + */ +const BOUND_HOST_METHODS = new Set(); + +/** Reserved words a function declaration may not be named. */ +const RESERVED = new Set(('break case catch class const continue debugger default delete do else enum export extends ' + + 'false finally for function if import in instanceof new null return super switch this throw true try typeof var ' + + 'void while with implements interface let package private protected public static yield await').split(' ')); + +/** Every reason a method stays, with the one-line why the plan prints. */ +const STAY_REASONS = Object.freeze({ + 'no-section': 'declared above the first section anchor', + 'section-kept': 'its section maps to no module', + 'static': 'a static member (the host is an instance)', + 'no-body': 'an overload signature or abstract member', + 'generator': 'a generator', + 'overloaded': 'an overloaded method', + 'decorated': 'a decorated method', + 'not-identifier-name': 'a computed, string or #private name', + 'reserved-name': 'its name is reserved for a function declaration', + 'inferred-return-type': 'a non-private method whose return type is inferred (the .d.ts would infer it through the module)', + 'type-parameters': 'a generic method (Function#call does not carry its type parameters)', + 'parameter-shape': 'a destructured, defaulted or `this` parameter (the stub cannot forward it unchanged)', + 'comment-in-parameters': 'a comment inside its parameter list (one-line parameters would drop it)', + 'path-reader': 'a reader takes its text out of engine.ts by path (PATH_READER_PINS / PATH_READER_NEEDLES / CLASS_PATH_READER_NEEDLES)', + 'arguments': 'its body reads `arguments`', + 'super': 'its body reads `super`', + 'new-target': 'its body reads `new.target`', + 'this-type': 'it names the polymorphic `this` type', + 'bare-this': 'it uses `this` other than as `this.member`', + 'unknown-member': 'it reads a `this.member` the class does not declare', + 'this-write': 'it assigns a `this.member` (a host getter cannot be written)', + 'method-as-value': 'it reads a method as a value instead of calling it', + 'field-call': 'it calls a field or accessor as a function (the host would change its receiver)', + 'overloaded-host-method': 'it calls an overloaded method (the host closure cannot be typed)', + 'non-public-static': 'it reads a private or protected static of ObjectQL (unreachable from a module function)', + 'module-write': 'it assigns a module-scope binding of engine.ts (an import is read-only)', + 'already-moved': 'its body is already a hand-series seam stub', + 'name-taken': 'its name is already a module-scope binding in engine.ts', + 'name-collision': 'its name collides with an import its module needs', + 'parameter-shadows': 'a parameter shadows the moved function or a stub local', + 'no-yield': 'its stub would be no shorter than its body (moving it would grow engine.ts)', +}); + +/** Every reason a top-level declaration ahead of the class stays in engine.ts. */ +const TOP_STAY_REASONS = Object.freeze({ + 'top-kind': 'not a const / function / interface / type declaration of plain names', + 'merged-declaration': 'its name is declared by more than one statement (an overload set or a merged declaration)', + 'path-reader': 'a reader takes its text out of engine.ts by path (PATH_READER_PINS / PATH_READER_NEEDLES)', + 'class-reference': 'it names a class engine.ts declares (the leaf would import engine.ts back)', + 'seam-import': 'it uses an import from an engine-* module (the leaf would reach engine.ts back through it)', + 'needs-stayer': 'it names a declaration that stays (the leaf would import engine.ts back)', +}); + +// ── Loading ts-morph ───────────────────────────────────────────────────────── + +let tsMorph = null; + +/** ts-morph from the root manifest's devDependencies; a missing install is exit 2, never a stack trace. */ +async function loadTsMorph() { + if (tsMorph) return tsMorph; + const manifest = join(ROOT, 'package.json'); + try { + const resolved = createRequire(manifest).resolve('ts-morph'); + const mod = await import(pathToFileURL(resolved).href); + tsMorph = mod.default ?? mod; + return tsMorph; + } catch (error) { + console.error( + `✗ codemod-split-engine: PREREQUISITE NOT MET — ts-morph does not resolve from ${manifest} ` + + `(${error && error.code ? error.code : 'error'}). Run \`pnpm install\` in this worktree.`, + ); + process.exit(2); + } +} + +/** + * One in-memory project per parse: no file system, no lib, no build output. + * The text goes through `ts-parse.mjs`'s checked parse first, which refuses a + * source with parse errors (its exit 3) instead of handing back a recovered + * partial tree; ts-morph then reads the same text, already vouched for. + */ +function parse(tm, fileName, text) { + parseSourceFile(fileName, text); + const project = new tm.Project({ + useInMemoryFileSystem: true, + compilerOptions: { target: tm.ts.ScriptTarget.ES2020, strict: true, noLib: true, allowJs: false }, + }); + project.getFileSystem().writeFileSync(fileName, text); + return project.addSourceFileAtPath(fileName); +} + +/** A reason the script cannot run at all (exit 2) — never a verdict on the split. */ +class CodemodRefusal extends Error {} + +function refuse(message) { + throw new CodemodRefusal(message); +} + +// ── Reading the sections ───────────────────────────────────────────────────── + +/** + * Each member's section: the last anchor at or above it. Refuses an anchor the + * class does not declare, a duplicate anchor, or anchors out of class order. + * Returns a function from a member node to its `[anchor, title, module]` row, + * or null above the first anchor. + */ +function readSections(cls, anchors) { + const firstStart = new Map(); + for (const m of cls.getMembers()) { + const name = typeof m.getName === 'function' ? m.getName() : null; + if (name && !firstStart.has(name)) firstStart.set(name, m.getStart()); + } + const rows = []; + const seen = new Set(); + for (const row of anchors) { + const [anchor] = row; + if (seen.has(anchor)) refuse(`SECTION_ANCHORS names ${anchor} twice`); + seen.add(anchor); + if (!firstStart.has(anchor)) refuse(`SECTION_ANCHORS names ${anchor}, which ${CLASS_NAME} does not declare`); + rows.push({ row, at: firstStart.get(anchor) }); + } + for (let i = 1; i < rows.length; i += 1) { + if (rows[i].at <= rows[i - 1].at) refuse(`SECTION_ANCHORS out of class order: ${rows[i].row[0]} is declared above ${rows[i - 1].row[0]}`); + } + return (member) => { + let hit = null; + for (const r of rows) if (r.at <= member.getStart()) hit = r.row; + return hit; + }; +} +// ── Analysis ───────────────────────────────────────────────────────────────── + +/** The names engine.ts binds at module scope, and which of them it exports. */ +function moduleScope(tm, sf) { + const { Node } = tm; + const bindings = new Map(); // name -> { kind: 'import' | 'local', specifier?, form?, imported?, typeOnly? } + const exported = new Set(); + const importOrder = new Map(); // specifier -> first position among the import declarations + for (const st of sf.getStatements()) { + if (Node.isImportDeclaration(st)) { + const specifier = st.getModuleSpecifierValue(); + const order = importOrder.has(specifier) ? importOrder.get(specifier) : importOrder.size; + importOrder.set(specifier, order); + const declTypeOnly = st.isTypeOnly(); + const def = st.getDefaultImport(); + if (def) bindings.set(def.getText(), { kind: 'import', specifier, form: 'default', typeOnly: declTypeOnly }); + const ns = st.getNamespaceImport(); + if (ns) bindings.set(ns.getText(), { kind: 'import', specifier, form: 'namespace', typeOnly: declTypeOnly }); + for (const spec of st.getNamedImports()) { + bindings.set((spec.getAliasNode() ?? spec.getNameNode()).getText(), { + kind: 'import', specifier, form: 'named', imported: spec.getNameNode().getText(), + typeOnly: declTypeOnly || spec.isTypeOnly(), + }); + } + continue; + } + if (Node.isExportDeclaration(st)) { + if (!st.getModuleSpecifier()) for (const spec of st.getNamedExports()) exported.add(spec.getNameNode().getText()); + continue; + } + const isExported = Node.isExportable(st) && st.isExported(); + const names = []; + if (Node.isVariableStatement(st)) { + for (const decl of st.getDeclarations()) { + for (const id of [decl.getNameNode(), ...decl.getNameNode().getDescendantsOfKind(tm.SyntaxKind.Identifier)]) { + if (Node.isIdentifier(id)) names.push(id.getText()); + } + } + } else if (typeof st.getName === 'function' && st.getName()) { + names.push(st.getName()); + } + for (const name of names) { + bindings.set(name, { kind: 'local' }); + if (isExported) exported.add(name); + } + } + return { bindings, exported, importOrder }; +} + +/** Is `node` inside a type node, below `stop`? */ +function inTypePosition(tm, node, stop) { + for (let n = node.getParent(); n && n !== stop; n = n.getParent()) { + if (tm.ts.isTypeNode(n.compilerNode)) return true; + } + return false; +} + +/** The node that binds the `this` a `this` keyword at `node` reads. */ +function thisContainer(tm, node) { + const { Node } = tm; + for (let n = node.getParent(); n; n = n.getParent()) { + if ( + Node.isFunctionDeclaration(n) || Node.isFunctionExpression(n) || Node.isMethodDeclaration(n) || + Node.isConstructorDeclaration(n) || Node.isGetAccessorDeclaration(n) || Node.isSetAccessorDeclaration(n) || + Node.isClassDeclaration(n) || Node.isClassExpression(n) || Node.isClassStaticBlockDeclaration(n) + ) return n; + } + return null; +} + +/** Is the property access `pa` written (assigned, updated, deleted, destructured into)? */ +function isWritten(tm, pa) { + const { Node, SyntaxKind } = tm; + let target = pa; + for (let p = target.getParent(); p; target = p, p = p.getParent()) { + if ( + Node.isParenthesizedExpression(p) || Node.isArrayLiteralExpression(p) || Node.isObjectLiteralExpression(p) || + Node.isPropertyAssignment(p) || Node.isShorthandPropertyAssignment(p) || Node.isSpreadElement(p) || + Node.isSpreadAssignment(p) + ) continue; + break; + } + const parent = target.getParent(); + if (!parent) return false; + if (Node.isBinaryExpression(parent) && parent.getLeft() === target) { + const op = parent.getOperatorToken().getKind(); + return op >= SyntaxKind.FirstAssignment && op <= SyntaxKind.LastAssignment; + } + if (target === pa && (Node.isPrefixUnaryExpression(parent) || Node.isPostfixUnaryExpression(parent))) { + const op = parent.getOperatorToken(); + return op === SyntaxKind.PlusPlusToken || op === SyntaxKind.MinusMinusToken; + } + if (target === pa && Node.isDeleteExpression(parent)) return true; + if ((Node.isForOfStatement(parent) || Node.isForInStatement(parent)) && parent.getInitializer() === target) return true; + return false; +} + +/** The class's instance members by name: `{ kind: 'method' | 'field' | 'getter' | 'setter', overloaded }`. */ +function instanceMembers(tm, cls) { + const { Node } = tm; + const members = new Map(); + const note = (name, kind, extra = {}) => { + const prev = members.get(name); + if (prev && prev.kind === 'setter' && kind === 'getter') { members.set(name, { kind: 'getter', ...extra }); return; } + if (prev && prev.kind === 'getter' && kind === 'setter') return; + members.set(name, { kind, ...extra }); + }; + for (const m of cls.getMembers()) { + if (Node.isConstructorDeclaration(m)) { + for (const p of m.getParameters()) if (p.isParameterProperty()) note(p.getName(), 'field'); + continue; + } + if (typeof m.isStatic === 'function' && m.isStatic()) continue; + const nameNode = typeof m.getNameNode === 'function' ? m.getNameNode() : null; + if (!nameNode || !Node.isIdentifier(nameNode)) continue; + const name = nameNode.getText(); + if (Node.isMethodDeclaration(m)) note(name, 'method', { overloaded: m.getOverloads().length > 0 }); + else if (Node.isPropertyDeclaration(m)) note(name, 'field'); + else if (Node.isGetAccessorDeclaration(m)) note(name, 'getter'); + else if (Node.isSetAccessorDeclaration(m)) note(name, 'setter'); + } + return members; +} + +/** + * Read one method: why it must stay (empty = it moves), what its body reads + * through the host, and which imports its module needs. + */ +function readMethod(tm, ctx, m) { + const { Node, SyntaxKind } = tm; + const reasons = []; + const add = (code, detail) => { + const tag = detail ? `${code}: ${detail}` : code; + if (!reasons.includes(tag)) reasons.push(tag); + }; + const host = new Map(); // member -> 'method' | 'getter' + const imports = new Map(); // local -> { value } + const moduleRefs = new Map(); // engine.ts top-level name -> { value } + + if (m.isStatic()) add('static'); + if (!m.getBody()) add('no-body'); + if (m.isAbstract && m.isAbstract()) add('no-body'); + if (m.isGenerator()) add('generator'); + if (m.getOverloads().length > 0) add('overloaded'); + if (m.getDecorators().length > 0) add('decorated'); + const nameNode = m.getNameNode(); + if (!Node.isIdentifier(nameNode)) { add('not-identifier-name'); return { reasons, host, imports, moduleRefs }; } + const name = nameNode.getText(); + if (RESERVED.has(name)) add('reserved-name'); + if (!m.getReturnTypeNode() && m.getScope() !== 'private') add('inferred-return-type'); + if (m.getTypeParameters().length > 0) add('type-parameters'); + for (const p of m.getParameters()) { + if (!Node.isIdentifier(p.getNameNode()) || p.hasInitializer() || p.getName() === 'this') add('parameter-shape', p.getName()); + } + const body = m.getBody(); + if (!body) return { reasons, host, imports, moduleRefs }; + + // A comment between the parentheses that no parameter's own text carries. + const params = m.getParameters(); + const open = m.getFirstChildByKindOrThrow(SyntaxKind.OpenParenToken); + const close = m.getFirstChildByKindOrThrow(SyntaxKind.CloseParenToken); + let between = ctx.text.slice(open.getEnd(), close.getStart()); + for (const p of [...params].reverse()) { + between = between.slice(0, p.getStart() - open.getEnd()) + between.slice(p.getEnd() - open.getEnd()); + } + if (/\/\/|\/\*/.test(between)) add('comment-in-parameters'); + + const methodText = m.getText(); + for (const needle of Object.keys(ctx.needles)) if (methodText.includes(needle)) add('path-reader', JSON.stringify(needle)); + for (const needle of Object.keys(ctx.classNeedles)) if (methodText.includes(needle)) add('path-reader', JSON.stringify(needle)); + if (Object.prototype.hasOwnProperty.call(ctx.pins, name)) add('path-reader', name); + + const regions = [...m.getTypeParameters(), ...params, m.getReturnTypeNode(), body].filter(Boolean); + const checker = ctx.sf.getProject().getTypeChecker().compilerObject; + for (const region of regions) { + const nodes = [region, ...region.getDescendants()]; + for (const node of nodes) { + const kind = node.getKind(); + if (kind === SyntaxKind.SuperKeyword) add('super'); + if (kind === SyntaxKind.ThisType) add('this-type'); + if (Node.isMetaProperty(node) && node.getKeywordToken() === SyntaxKind.NewKeyword) add('new-target'); + if (kind === SyntaxKind.ThisKeyword) { + if (thisContainer(tm, node) !== m) continue; + const parent = node.getParent(); + if (!Node.isPropertyAccessExpression(parent) || parent.getExpression() !== node) { add('bare-this'); continue; } + const memberNameNode = parent.getNameNode(); + if (!Node.isIdentifier(memberNameNode)) { add('bare-this'); continue; } + const member = memberNameNode.getText(); + const info = ctx.members.get(member); + if (!info) { add('unknown-member', member); continue; } + if (isWritten(tm, parent)) add('this-write', member); + const outer = parent.getParent(); + const called = Node.isCallExpression(outer) && outer.getExpression() === parent; + if (info.kind === 'method') { + if (!called) add('method-as-value', member); + if (info.overloaded) add('overloaded-host-method', member); + host.set(member, 'method'); + } else if (info.kind === 'setter') { + add('this-write', member); + } else { + if (called) add('field-call', member); + host.set(member, 'getter'); + } + continue; + } + if (kind !== SyntaxKind.Identifier) continue; + if (node.getText() === 'arguments') { + const parent = node.getParent(); + const isName = Node.isPropertyAccessExpression(parent) && parent.getNameNode() === node; + if (!isName) add('arguments'); + } + const parent = node.getParent(); + let symbol; + if (Node.isShorthandPropertyAssignment(parent) && parent.getNameNode() === node) { + symbol = checker.getShorthandAssignmentValueSymbol(parent.compilerNode); + } else { + symbol = checker.getSymbolAtLocation(node.compilerNode); + } + if (!symbol || !symbol.declarations) continue; + for (const decl of symbol.declarations) { + if (decl.getSourceFile() !== ctx.sf.compilerNode) continue; + const ts = tm.ts; + const typePos = inTypePosition(tm, node, m); + if (ts.isImportSpecifier(decl) || ts.isImportClause(decl) || ts.isNamespaceImport(decl)) { + const local = node.getText(); + const prev = imports.get(local); + imports.set(local, { value: Boolean(prev && prev.value) || !typePos }); + continue; + } + if (!isModuleScopeDecl(ts, decl)) continue; + // Lever (a) of ruling 6591753301: a module-scope name is imported — + // from the leaf when it moved there, from `engine.ts` (exported + // for it) when it stayed. Two things a module function cannot do: + // reach a non-public static of the class, and assign an import. + const declName = decl.name && ts.isIdentifier(decl.name) ? decl.name.text : node.getText(); + if (ts.isClassDeclaration(decl) && declName === CLASS_NAME && !typePos) { + const access = node.getParent(); + if (Node.isPropertyAccessExpression(access) && access.getExpression() === node) { + const member = access.getName(); + const scope = ctx.statics.get(member); + if (scope && scope !== 'public') add('non-public-static', member); + } + } + if (!typePos && isWritten(tm, node)) add('module-write', declName); + const prev = moduleRefs.get(declName); + moduleRefs.set(declName, { value: Boolean(prev && prev.value) || !typePos }); + } + } + } + + const binding = ctx.scope.bindings.get(name); + if (binding) { + if (binding.kind === 'import' && SEAM_SPECIFIER.test(binding.specifier)) add('already-moved', binding.specifier); + else add('name-taken'); + } + for (const p of params) { + const pn = p.getName(); + if (pn === name || pn === 'host' || /^read[A-Z]/.test(pn)) add('parameter-shadows', pn); + } + return { reasons, host, imports, moduleRefs }; +} + +function isModuleScopeDecl(ts, decl) { + if (ts.isVariableDeclaration(decl)) { + const statement = decl.parent && decl.parent.parent; + return Boolean(statement && ts.isVariableStatement(statement) && ts.isSourceFile(statement.parent)); + } + if (ts.isBindingElement(decl)) { + let n = decl; + while (n && !ts.isVariableDeclaration(n)) n = n.parent; + return Boolean(n && isModuleScopeDecl(ts, n)); + } + return Boolean(decl.parent && ts.isSourceFile(decl.parent)) && + (ts.isFunctionDeclaration(decl) || ts.isClassDeclaration(decl) || ts.isInterfaceDeclaration(decl) || + ts.isTypeAliasDeclaration(decl) || ts.isEnumDeclaration(decl) || ts.isModuleDeclaration(decl)); +} + +// ── The plan ───────────────────────────────────────────────────────────────── + +/** `engine-transactions-section.ts` -> `TransactionsSectionHost`. */ +function hostNameOf(moduleFile) { + const slug = moduleFile.replace(/^engine-/, '').replace(/\.ts$/, ''); + return `${slug.split('-').map((w) => w[0].toUpperCase() + w.slice(1)).join('')}Host`; +} + +/** Is `node` inside a JSDoc comment? A `{@link X}` there is prose, not a use. */ +function inJsDoc(tm, node) { + for (let n = node.compilerNode; n; n = n.parent) { + if (n.kind >= tm.SyntaxKind.FirstJSDocNode && n.kind <= tm.SyntaxKind.LastJSDocNode) return true; + } + return false; +} + +/** The statement of `sf` (a direct child of the source file) a declaration belongs to. */ +function topStatementOf(ts, decl) { + let n = decl; + while (n.parent && !ts.isSourceFile(n.parent)) n = n.parent; + return n.parent ? n : null; +} + +/** The names a top-level statement declares, or null when one is not a plain identifier. */ +function declaredNames(tm, st) { + const { Node } = tm; + if (Node.isVariableStatement(st)) { + const names = []; + for (const d of st.getDeclarations()) { + if (!Node.isIdentifier(d.getNameNode())) return null; + names.push(d.getName()); + } + return names; + } + if (Node.isFunctionDeclaration(st) || Node.isInterfaceDeclaration(st) || Node.isTypeAliasDeclaration(st)) { + return st.getName() ? [st.getName()] : null; + } + return null; +} + +/** + * Lever (b): read every top-level statement between the last import and the + * class, and decide which move whole into the leaf. A mover never names the + * class, an `engine-*` module's import, or a statement that stays — so the + * leaf it lands in never imports `engine.ts` back. Pure. + */ +function planTopLevel(tm, ctx) { + const { Node, SyntaxKind } = tm; + const ts = tm.ts; + const stmts = ctx.sf.getStatements(); + const clsIndex = stmts.indexOf(ctx.cls); + let lastImport = -1; + stmts.forEach((st, i) => { if (Node.isImportDeclaration(st)) lastImport = i; }); + const checker = ctx.sf.getProject().getTypeChecker().compilerObject; + const nameCount = new Map(); + for (const st of stmts) { + for (const n of declaredNames(tm, st) ?? []) nameCount.set(n, (nameCount.get(n) ?? 0) + 1); + } + const records = []; + const byStatement = new Map(); + for (let i = lastImport + 1; i < clsIndex; i += 1) { + const st = stmts[i]; + const names = declaredNames(tm, st); + const record = { + node: st, names: names ?? [], label: names ? names.join(', ') : `(${st.getKindName()})`, + exported: Node.isExportable(st) && st.isExported(), reasons: [], + deps: new Set(), imports: new Map(), + startLine: st.getStartLineNumber(true), lines: st.getEndLineNumber() - st.getStartLineNumber(true) + 1, + typeOnly: Node.isInterfaceDeclaration(st) || Node.isTypeAliasDeclaration(st), + }; + const add = (code, detail) => { + const tag = detail ? `${code}: ${detail}` : code; + if (!record.reasons.includes(tag)) record.reasons.push(tag); + }; + if (!names || (Node.isVariableStatement(st) && st.hasDeclareKeyword()) || + (Node.isFunctionDeclaration(st) && (!st.getBody() || st.hasDeclareKeyword()))) add('top-kind'); + for (const n of record.names) if ((nameCount.get(n) ?? 0) > 1) add('merged-declaration', n); + for (const n of record.names) if (Object.prototype.hasOwnProperty.call(ctx.pins, n)) add('path-reader', n); + const fullText = st.getFullText(); + for (const needle of Object.keys(ctx.needles)) if (fullText.includes(needle)) add('path-reader', JSON.stringify(needle)); + for (const node of st.getDescendantsOfKind(SyntaxKind.Identifier)) { + if (inJsDoc(tm, node)) continue; + const parent = node.getParent(); + const symbol = Node.isShorthandPropertyAssignment(parent) && parent.getNameNode() === node + ? checker.getShorthandAssignmentValueSymbol(parent.compilerNode) + : checker.getSymbolAtLocation(node.compilerNode); + if (!symbol || !symbol.declarations) continue; + for (const decl of symbol.declarations) { + if (decl.getSourceFile() !== ctx.sf.compilerNode) continue; + const typePos = inTypePosition(tm, node, st); + if (ts.isImportSpecifier(decl) || ts.isImportClause(decl) || ts.isNamespaceImport(decl)) { + const local = node.getText(); + const binding = ctx.scope.bindings.get(local); + if (binding && (SEAM_SPECIFIER.test(binding.specifier) || binding.specifier === './engine.js')) add('seam-import', local); + const prev = record.imports.get(local); + record.imports.set(local, { value: Boolean(prev && prev.value) || !typePos }); + continue; + } + if (!isModuleScopeDecl(ts, decl)) continue; + const owner = topStatementOf(ts, decl); + if (!owner || owner === st.compilerNode) continue; + if (owner === ctx.cls.compilerNode) { add('class-reference'); continue; } + record.deps.add(owner); + } + } + records.push(record); + byStatement.set(st.compilerNode, record); + } + // A mover may not name a statement that stays: close over the stayers until stable. + for (let changed = true; changed;) { + changed = false; + for (const r of records) { + if (r.reasons.length > 0) continue; + for (const owner of r.deps) { + const dep = byStatement.get(owner); + if (!dep || dep.reasons.length > 0) { + const depName = dep ? dep.label : (declaredNames(tm, ctx.sf.getStatements().find((s) => s.compilerNode === owner)) ?? ['?']).join(', '); + r.reasons.push(`needs-stayer: ${depName}`); + changed = true; + break; + } + } + } + } + const movers = records.filter((r) => r.reasons.length === 0); + const moverNames = new Map(); + for (const r of movers) for (const n of r.names) moverNames.set(n, r); + return { records, movers, moverNames }; +} + +/** + * Which moved top-level names the text that STAYS in `engine.ts` still + * reads, value or type: everything outside the moved statements and the moved + * method bodies, JSDoc excepted. Those are what `engine.ts` imports back + * from the leaf. + */ +function hostUsesOfMovers(tm, ctx, top, methodRecords) { + const { SyntaxKind } = tm; + const ts = tm.ts; + const checker = ctx.sf.getProject().getTypeChecker().compilerObject; + const skip = []; + for (const r of top.movers) skip.push([r.node.getFullStart(), r.node.getEnd()]); + for (const r of methodRecords) if (r.reasons.length === 0) skip.push([r.node.getBody().getStart(), r.node.getBody().getEnd()]); + const uses = new Map(); + for (const node of ctx.sf.getDescendantsOfKind(SyntaxKind.Identifier)) { + const at = node.getStart(); + if (skip.some(([a, b]) => at >= a && at < b)) continue; + if (inJsDoc(tm, node)) continue; + const name = node.getText(); + if (!top.moverNames.has(name)) continue; + const symbol = checker.getSymbolAtLocation(node.compilerNode); + if (!symbol || !symbol.declarations) continue; + const mover = top.moverNames.get(name); + if (!symbol.declarations.some((d) => topStatementOf(ts, d) === mover.node.compilerNode)) continue; + const typePos = inTypePosition(tm, node, ctx.sf); + const prev = uses.get(name); + uses.set(name, { value: Boolean(prev && prev.value) || !typePos }); + } + return uses; +} + +/** + * Read `text` (an `engine.ts`) and decide, per method, where its body goes, + * and per top-level declaration ahead of the class, whether it moves to the + * leaf. Pure: the same text gives the same plan. + */ +function plan(tm, text, { + anchors = SECTION_ANCHORS, pins = PATH_READER_PINS, needles = PATH_READER_NEEDLES, classNeedles = CLASS_PATH_READER_NEEDLES, +} = {}) { + const sf = parse(tm, HOST_PATH, text); + const cls = sf.getClass(CLASS_NAME); + if (!cls) refuse(`no class ${CLASS_NAME} in ${HOST_BASENAME}`); + const sectionOf = readSections(cls, anchors); + const statics = new Map(); + for (const m of cls.getMembers()) { + if (typeof m.isStatic === 'function' && m.isStatic() && typeof m.getName === 'function') { + statics.set(m.getName(), typeof m.getScope === 'function' ? m.getScope() : 'public'); + } + } + const ctx = { text, sf, cls, members: instanceMembers(tm, cls), scope: moduleScope(tm, sf), statics, pins, needles, classNeedles }; + const methodNames = new Set(cls.getMethods().map((m) => m.getName())); + const stalePins = Object.keys(pins).filter((n) => !methodNames.has(n) && !ctx.scope.bindings.has(n)); + if (stalePins.length > 0) refuse(`PATH_READER_PINS names what ${HOST_BASENAME} no longer declares: ${stalePins.join(', ')}`); + const staleNeedles = Object.keys(needles).filter((n) => !text.includes(n)); + if (staleNeedles.length > 0) refuse(`PATH_READER_NEEDLES names text ${HOST_BASENAME} no longer carries: ${staleNeedles.map((n) => JSON.stringify(n)).join(', ')}`); + const staleClassNeedles = Object.keys(classNeedles).filter((n) => !text.includes(n)); + if (staleClassNeedles.length > 0) refuse(`CLASS_PATH_READER_NEEDLES names text ${HOST_BASENAME} no longer carries: ${staleClassNeedles.map((n) => JSON.stringify(n)).join(', ')}`); + + const top = planTopLevel(tm, ctx); + + const records = []; + for (const m of cls.getMethods()) { + const startLine = m.getStartLineNumber(true); + const section = sectionOf(m); + const read = readMethod(tm, ctx, m); + const name = m.getName(); + if (!section) read.reasons.unshift('no-section'); + else if (section[2] === null) read.reasons.unshift('section-kept'); + const lineCount = m.getEndLineNumber() - startLine + 1; + const body = m.getBody(); + const bodyLines = body ? body.getEndLineNumber() - body.getStartLineNumber() + 1 : 0; + const record = { + name, node: m, section: section ? section[1] : null, module: section ? section[2] : null, + startLine, lineCount, bodyLines, ...read, + }; + if (record.reasons.length === 0) { + record.stubLines = stubBody(record, record.module, ' ').split('\n').length; + if (record.stubLines >= bodyLines) record.reasons.push('no-yield'); + } + records.push(record); + } + + // A function name its module's imports already use: drop it, recompute, until stable. + for (let changed = true; changed;) { + changed = false; + const byModule = groupMovers(records); + for (const [moduleFile, movers] of byModule) { + const taken = new Set([CLASS_NAME, hostNameOf(moduleFile), 'ThisParameterType', 'Pick']); + for (const r of movers) { + for (const local of r.imports.keys()) taken.add(local); + for (const local of r.moduleRefs.keys()) taken.add(local); + } + for (const r of movers) { + if (taken.has(r.name)) { r.reasons.push('name-collision'); changed = true; } + } + } + } + const modules = groupMovers(records); + const hostUses = hostUsesOfMovers(tm, ctx, top, records); + + // What each side imports across the cut: the section modules read moved + // names from the leaf and staying names from engine.ts (lever (a)); + // engine.ts reads moved names from the leaf. + const sectionLeafNames = new Set(); + const hostExportsForSections = new Map(); // staying name -> { value } + for (const movers of modules.values()) { + for (const r of movers) { + for (const [n, use] of r.moduleRefs) { + if (top.moverNames.has(n)) { sectionLeafNames.add(n); continue; } + if (n === CLASS_NAME || ctx.scope.exported.has(n)) continue; + const prev = hostExportsForSections.get(n); + hostExportsForSections.set(n, { value: Boolean(prev && prev.value) || use.value }); + } + } + } + const leafExports = []; + for (const r of top.movers) { + if (r.exported) continue; + for (const n of r.names) if (hostUses.has(n) || sectionLeafNames.has(n)) leafExports.push(n); + } + return { sf, cls, ctx, records, modules, top, hostUses, leafExports, hostExportsForSections }; +} + +/** Movers grouped by module, modules in the order their first member appears. */ +function groupMovers(records) { + const byModule = new Map(); + for (const r of records) { + if (r.reasons.length > 0 || !r.module) continue; + if (!byModule.has(r.module)) byModule.set(r.module, []); + byModule.get(r.module).push(r); + } + return byModule; +} + +// ── Generation ─────────────────────────────────────────────────────────────── + +const readerName = (member) => `read${member[0].toUpperCase()}${member.slice(1)}`; + +const byName = ([a], [b]) => (a < b ? -1 : a > b ? 1 : 0); +const leafSpecifier = () => `./${LEAF_MODULE.replace(/\.ts$/, '.js')}`; + +/** + * One `import` / `export` statement over `names`: on one line when it fits in + * 120 columns, else one name per wrapped run, two-space indented. + */ +function namedStatement(head, names, tail) { + const one = `${head} { ${names.join(', ')} }${tail}`; + if (one.length <= 120) return one; + const out = [`${head} {`]; + let line = ' '; + for (const n of names) { + if (line.length + n.length + 2 > 118 && line !== ' ') { out.push(line); line = ' '; } + line += ` ${n},`; + } + out.push(line); + out.push(`}${tail}`); + return out.join('\n'); +} + +/** `import { … }` and `import type { … }` from one specifier, values first. */ +function importPair(specifier, uses) { + const out = []; + const entries = [...uses.entries()].sort(byName); + const values = entries.filter(([, u]) => u.value).map(([n]) => n); + const types = entries.filter(([, u]) => !u.value).map(([n]) => n); + if (values.length > 0) out.push(namedStatement('import', values, ` from '${specifier}';`)); + if (types.length > 0) out.push(namedStatement('import type', types, ` from '${specifier}';`)); + return out; +} + +/** + * The import lines a module needs: engine.ts's own imports it uses, in + * engine.ts's import order; then the moved top-level names, from the leaf; + * then what stayed in engine.ts (lever (a)), including the class. + */ +function importLines(scope, users, { needsHost = false, moduleRefs = new Map(), leafNames = new Map() } = {}) { + const bySpecifier = new Map(); + for (const r of users) { + for (const [local, use] of r.imports) { + const b = scope.bindings.get(local); + if (!b || b.kind !== 'import') refuse(`import binding ${local} not found`); + if (!bySpecifier.has(b.specifier)) bySpecifier.set(b.specifier, new Map()); + const slot = bySpecifier.get(b.specifier); + const prev = slot.get(local); + slot.set(local, { ...b, value: Boolean(prev && prev.value) || (use.value && !b.typeOnly) }); + } + } + const out = []; + const specifiers = [...bySpecifier.keys()].sort((a, b) => scope.importOrder.get(a) - scope.importOrder.get(b)); + for (const specifier of specifiers) { + const entries = [...bySpecifier.get(specifier).entries()].sort(byName); + const named = (value) => entries + .filter(([, e]) => e.form === 'named' && e.value === value) + .map(([local, e]) => (e.imported === local ? local : `${e.imported} as ${local}`)); + const def = entries.find(([, e]) => e.form === 'default'); + const ns = entries.find(([, e]) => e.form === 'namespace'); + const valueNamed = named(true); + const defValue = def && def[1].value ? def[0] : null; + if (defValue || valueNamed.length > 0) { + const parts = [defValue, valueNamed.length > 0 ? `{ ${valueNamed.join(', ')} }` : null].filter(Boolean); + out.push(`import ${parts.join(', ')} from '${specifier}';`); + } + if (def && !def[1].value) out.push(`import type ${def[0]} from '${specifier}';`); + if (ns) out.push(`import ${ns[1].value ? '' : 'type '}* as ${ns[0]} from '${specifier}';`); + const typeNamed = named(false); + if (typeNamed.length > 0) out.push(`import type { ${typeNamed.join(', ')} } from '${specifier}';`); + } + const fromLeaf = new Map(); + const fromHost = new Map(); + for (const [n, use] of moduleRefs) (leafNames.has(n) ? fromLeaf : fromHost).set(n, use); + if (needsHost && !fromHost.has(CLASS_NAME)) fromHost.set(CLASS_NAME, { value: false }); + out.push(...importPair(leafSpecifier(), fromLeaf)); + out.push(...importPair('./engine.js', fromHost)); + return out; +} + +/** `text` wrapped into ` * ` comment lines no wider than `width`. */ +function wrapComment(words, width = 80) { + const lines = []; + let line = ' *'; + for (const w of words) { + if (line.length + 1 + w.length > width && line !== ' *') { lines.push(line); line = ' *'; } + line += ` ${w}`; + } + if (line !== ' *') lines.push(line); + return lines; +} + +function paramsText(m) { + return m.getParameters().map((p) => p.getText()); +} + +function forwardList(m) { + return m.getParameters().map((p) => (p.isRestParameter() ? `...${p.getName()}` : p.getName())); +} + +function moduleText(moduleFile, title, movers, scope, leafNames) { + const hostName = hostNameOf(moduleFile); + const hostMembers = new Map(); + for (const r of movers) for (const [member, kind] of r.host) hostMembers.set(member, kind); + const needsHost = hostMembers.size > 0; + const moduleRefs = new Map(); + for (const r of movers) { + for (const [n, use] of r.moduleRefs) { + const prev = moduleRefs.get(n); + moduleRefs.set(n, { value: Boolean(prev && prev.value) || use.value }); + } + } + const readsHostValues = [...moduleRefs].some(([n, use]) => use.value && !leafNames.has(n)); + const names = movers.map((r) => `\`${r.name}\`${r === movers[movers.length - 1] ? '' : ','}`); + const out = []; + out.push('// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.'); + out.push(''); + out.push('/**'); + out.push(...wrapComment([MARK, 'The', `"${title}"`, 'section', 'of', '`ObjectQL`:', 'the', 'bodies', 'of', + ...names, 'moved', 'out', 'of', '`engine.ts`', 'whole', 'by', `\`${SCRIPT}\`.`])); + out.push(' *'); + out.push(' * A MOVE, not a redesign. Each function below is the body of the `ObjectQL`'); + out.push(' * method of the same name exactly as it stood: not re-indented, not'); + out.push(' * formatted, not edited. The method stays on the class with its docblock and'); + out.push(' * signature, so the published `dist/*.d.ts`, an instance spy and a subclass'); + out.push(' * override are what they were; the docblock there is the contract.'); + out.push(' *'); + if (needsHost) { + out.push(` * The bodies reach the class through {@link ${hostName}}: exactly the`); + out.push(' * `ObjectQL` members they read, private ones included. Each method hands its'); + out.push(' * body one literal of the members that body reads, plus a `.call`: methods go'); + out.push(' * as closures that call them on the engine at call time, so a subclass'); + out.push(' * override or an instance spy is still the one reached; fields and accessors'); + out.push(' * go as GETTERS, read when the body reads them, as they were on the class.'); + } else { + out.push(' * No body reads an `ObjectQL` member, so each method calls its function'); + out.push(' * directly.'); + } + out.push(' *'); + if (readsHostValues) { + out.push(' * Values that stayed in `engine.ts` are imported from it (lever (a) of'); + out.push(' * ruling 6591753301); every use is inside a function body, so the import'); + out.push(' * cycle is never read at evaluation time. `index.ts` re-exports nothing from'); + out.push(' * this module.'); + } else { + out.push(' * The module imports `engine.ts` for types only, so the two modules form no'); + out.push(' * evaluation cycle. `index.ts` re-exports nothing from this module.'); + } + out.push(' */'); + out.push(''); + out.push(...importLines(scope, movers, { needsHost, moduleRefs, leafNames })); + if (needsHost) { + out.push(''); + out.push('/**'); + out.push(' * The `ObjectQL` members the functions below read — every one of them, and'); + out.push(' * no other. Each is typed AS the class member (an indexed access on the'); + out.push(' * class), so a change to a member\'s signature on the class is a change here'); + out.push(' * too. `readonly` members are the getters.'); + out.push(' */'); + out.push(`export interface ${hostName} {`); + for (const member of [...hostMembers.keys()].sort()) { + out.push(` ${hostMembers.get(member) === 'getter' ? 'readonly ' : ''}${member}: ObjectQL['${member}'];`); + } + out.push('}'); + } + for (const r of movers) { + const m = r.node; + const picks = [...r.host.keys()].sort().map((k) => `'${k}'`).join(' | '); + const params = [...(r.host.size > 0 ? [`this: Pick<${hostName}, ${picks}>`] : []), ...paramsText(m)]; + const ret = m.getReturnTypeNode() ? `: ${m.getReturnTypeNode().getText()}` : ''; + out.push(''); + out.push(`/** The body of \`ObjectQL#${r.name}\`; the method's docblock there is its contract. */`); + out.push(`export ${m.isAsync() ? 'async ' : ''}function ${r.name}(${params.join(', ')})${ret} ${m.getBody().getText()}`); + } + out.push(''); + return out.join('\n'); +} + +/** + * The leaf: the moved top-level statements, each its base text exactly (its + * leading comments and blank lines included), in base order, after the + * imports they use; then one `export { … }` of the moved names base did not + * export and something outside the leaf now imports. + */ +function leafText(scope, top, leafExports) { + // The header is `//` lines, not a `/** */` docblock: the bundled `.d.ts` + // keeps a JSDoc block that leads a module's first statement, and a leaf + // bundled first would publish text the base did not carry (the pilot's + // measurement: 14 added lines in dist/index.d.ts). + const out = []; + out.push('// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.'); + out.push(''); + out.push(...wrapComment([MARK, 'The', 'top-level', 'declarations', 'ahead', 'of', '`ObjectQL`', 'in', '`engine.ts`', + 'that', 'moved', 'out', 'whole', 'by', `\`${SCRIPT}\``, '(lever', '(b)', 'of', 'ruling', + '6591753301).']).map((l) => l.replace(/^ \*/, '//'))); + out.push('//'); + out.push('// A MOVE, not a redesign: each declaration below is its `engine.ts` text'); + out.push('// exactly, with the comments above it, in the order it stood there. A LEAF:'); + out.push('// it imports neither `engine.ts` nor any `engine-*` module, so it is'); + out.push('// evaluated before the class and forms no cycle with it. `engine.ts`'); + out.push('// re-exports, by name, every declaration here it exported before, so every'); + out.push('// importer of `./engine.js` reads what it read.'); + out.push(''); + out.push(...importLines(scope, top.movers)); + let text = out.join('\n'); + for (const r of top.movers) text += r.node.getFullText(); + if (leafExports.length > 0) { + text += `\n\n// ${MARK} Declared above without \`export\`, and read by \`engine.ts\` or a section module.\n`; + text += `${namedStatement('export', [...leafExports].sort(), ';')}`; + } + return `${text}\n`; +} + +function stubBody(r, moduleFile, closeIndent) { + const ind = `${closeIndent} `; + const hostName = hostNameOf(moduleFile); + const out = ['{']; + const args = forwardList(r.node).join(', '); + if (r.host.size === 0) { + out.push(`${ind}// ${MARK} The body moved to \`${moduleFile}\`.`); + out.push(`${ind}return ${r.name}(${args});`); + } else { + out.push(`${ind}// ${MARK} The body moved to \`${moduleFile}\` (see \`${hostName}\`).`); + const members = [...r.host.keys()].sort(); + for (const member of members) { + if (r.host.get(member) === 'getter') out.push(`${ind}const ${readerName(member)} = () => this.${member};`); + } + out.push(`${ind}const host: ThisParameterType = {`); + for (const member of members) { + if (r.host.get(member) === 'getter') out.push(`${ind} get ${member}() { return ${readerName(member)}(); },`); + else if (BOUND_HOST_METHODS.has(member)) out.push(`${ind} ${member}: this.${member}.bind(this),`); + else out.push(`${ind} ${member}: (...a) => this.${member}(...a),`); + } + out.push(`${ind}};`); + out.push(`${ind}return ${r.name}.call(host${args ? `, ${args}` : ''});`); + } + out.push(`${closeIndent}}`); + return out.join('\n'); +} + +/** + * Apply a plan to `text`. Returns `{ host, modules, pruned }`: the new + * `engine.ts`, a Map of module basename -> text (the leaf among them), and the + * import bindings the prune dropped. Pure. + */ +function generate(tm, text, planned) { + const { sf, ctx, modules, top, hostUses, leafExports, hostExportsForSections } = planned; + const edits = []; + const importBlock = []; + if (top.movers.length > 0) { + const leafUses = new Map([...hostUses].filter(([n]) => top.moverNames.has(n))); + const lines = importPair(leafSpecifier(), leafUses); + if (lines.length > 0) { + importBlock.push(`\n// ${MARK} The top-level declarations that moved whole to \`${LEAF_MODULE}\`, as this file still reads them.`); + for (const l of lines) importBlock.push(`\n${l}`); + } + const reexports = top.movers.filter((r) => r.exported) + .flatMap((r) => r.names.map((n) => [n, r.typeOnly ? `type ${n}` : n])) + .sort(byName).map(([, spelled]) => spelled); + if (reexports.length > 0) { + importBlock.push(`\n// ${MARK} Re-exported by name, so every importer of \`./engine.js\` reads what it read.`); + importBlock.push(`\n${namedStatement('export', reexports, ` from '${leafSpecifier()}';`)}`); + } + for (const r of top.movers) edits.push({ start: r.node.getFullStart(), end: r.node.getEnd(), text: '' }); + } + for (const [moduleFile, movers] of modules) { + const title = movers[0].section; + const hostName = hostNameOf(moduleFile); + const anyHost = movers.some((r) => r.host.size > 0); + importBlock.push(`\n// ${MARK} ${title}: the methods of the same names below hand the bodies` + + `${anyHost ? ` what they read (\`${hostName}\`)` : ' their arguments'}.`); + importBlock.push(`\n${namedStatement('import', movers.map((r) => r.name).sort(), ` from './${moduleFile.replace(/\.ts$/, '.js')}';`)}`); + for (const r of movers) { + const body = r.node.getBody(); + const lineStart = text.lastIndexOf('\n', body.getEnd() - 1) + 1; + const closeIndent = text.slice(lineStart, body.getEnd() - 1); + if (!/^ *$/.test(closeIndent)) refuse(`${r.name}: the body's closing brace does not open its own line`); + edits.push({ start: body.getStart(), end: body.getEnd(), text: stubBody(r, moduleFile, closeIndent) }); + const asyncKw = r.node.getModifiers().find((x) => x.getKind() === tm.SyntaxKind.AsyncKeyword); + if (asyncKw) edits.push({ start: asyncKw.getStart(), end: r.node.getNameNode().getStart(), text: '' }); + } + } + if (hostExportsForSections.size > 0) { + const spelled = [...hostExportsForSections.keys()].sort().map((n) => { + const st = ctx.sf.getStatements().find((s) => (declaredNames(tm, s) ?? []).includes(n)); + return st && (tm.Node.isInterfaceDeclaration(st) || tm.Node.isTypeAliasDeclaration(st)) ? `type ${n}` : n; + }); + importBlock.push(`\n// ${MARK} Exported for the section modules above, which import them (lever (a)); \`index.ts\` re-exports by name, so none reaches the package.`); + importBlock.push(`\n${namedStatement('export', spelled, ';')}`); + } + const lastImport = sf.getImportDeclarations().at(-1); + if (!lastImport) refuse('engine.ts has no import declaration to place the generated imports after'); + if (importBlock.length > 0) edits.push({ start: lastImport.getEnd(), end: lastImport.getEnd(), text: importBlock.join('') }); + edits.sort((a, b) => b.start - a.start || b.end - a.end); + let out = text; + let floor = Infinity; + for (const e of edits) { + if (e.end > floor) refuse('overlapping edits'); + out = out.slice(0, e.start) + e.text + out.slice(e.end); + floor = e.start; + } + // The imports only moved text read are dropped (the package compiles with + // `noUnusedLocals`); `verify` recomputes the same prune from the head. + const pruned = pruneImports(tm, out, usedImportNames(tm, out)); + const moduleTexts = new Map(); + const leafNames = top.moverNames; + if (top.movers.length > 0) moduleTexts.set(LEAF_MODULE, leafText(ctx.scope, top, leafExports)); + for (const [moduleFile, movers] of modules) { + moduleTexts.set(moduleFile, moduleText(moduleFile, movers[0].section, movers, ctx.scope, leafNames)); + } + return { host: pruned.text, modules: moduleTexts, pruned: pruned.removed }; +} + +// ── The import prune ───────────────────────────────────────────────────────── + +/** Is `node` (a compiler node) inside a JSDoc comment? */ +function inJsDocNode(ts, node) { + for (let n = node; n; n = n.parent) { + if (n.kind >= ts.SyntaxKind.FirstJSDocNode && n.kind <= ts.SyntaxKind.LastJSDocNode) return true; + } + return false; +} + +/** Is the identifier a property NAME (`a.NAME`, `{ NAME: … }`, a member name), not a reference? */ +function isPropertyNameIdentifier(ts, node) { + const p = node.parent; + if (!p) return false; + if (ts.isPropertyAccessExpression(p) && p.name === node) return true; + if ((ts.isPropertyAssignment(p) || ts.isPropertyDeclaration(p) || ts.isPropertySignature(p) || + ts.isMethodDeclaration(p) || ts.isMethodSignature(p) || ts.isGetAccessorDeclaration(p) || + ts.isSetAccessorDeclaration(p) || ts.isEnumMember(p)) && p.name === node) return true; + if (ts.isQualifiedName(p) && p.right === node) return true; + return false; +} + +/** + * The import bindings of an `engine.ts` that something outside its import + * declarations reads: a reference in code or a type, a local `export { … }`, + * or a JSDoc `{@link X}` / `@see X` name. Those JSDoc names count because the + * compiler's `noUnusedLocals` counts them; other JSDoc text and `//` comments + * do not. + */ +function usedImportNames(tm, text) { + const ts = tm.ts; + const sfm = parse(tm, HOST_PATH, text); + const sf = sfm.compilerNode; + const checker = sfm.getProject().getTypeChecker().compilerObject; + const used = new Set(); + const isImportDecl = (d) => d.getSourceFile() === sf && (ts.isImportSpecifier(d) || ts.isImportClause(d) || ts.isNamespaceImport(d)); + const leftmost = (name) => { + let n = name; + while (n && (ts.isQualifiedName(n) || ts.isPropertyAccessExpression(n) || n.kind === ts.SyntaxKind.JSDocMemberName)) { + n = ts.isQualifiedName(n) || n.kind === ts.SyntaxKind.JSDocMemberName ? n.left : n.expression; + } + return n && ts.isIdentifier(n) ? n : undefined; + }; + const visitDoc = (node) => { + const linkish = node.kind === ts.SyntaxKind.JSDocLink || node.kind === ts.SyntaxKind.JSDocLinkCode || + node.kind === ts.SyntaxKind.JSDocLinkPlain || node.kind === ts.SyntaxKind.JSDocNameReference; + if (linkish && node.name) { + const id = leftmost(node.name); + const symbol = id && checker.getSymbolAtLocation(id); + if (id && (symbol?.declarations ?? []).some(isImportDecl)) used.add(id.text); + } + if (Array.isArray(node.comment)) node.comment.forEach(visitDoc); + ts.forEachChild(node, visitDoc); + }; + const visit = (node) => { + if (ts.isImportDeclaration(node)) return; + for (const doc of node.jsDoc ?? []) visitDoc(doc); + if (ts.isExportDeclaration(node) && node.moduleSpecifier) return; + if (ts.isIdentifier(node) && !inJsDocNode(ts, node)) { + const parent = node.parent; + let symbol; + if (ts.isExportSpecifier(parent)) symbol = checker.getExportSpecifierLocalTargetSymbol(parent); + else if (ts.isShorthandPropertyAssignment(parent) && parent.name === node) symbol = checker.getShorthandAssignmentValueSymbol(parent); + else if (!isPropertyNameIdentifier(ts, node)) symbol = checker.getSymbolAtLocation(node); + if ((symbol?.declarations ?? []).some(isImportDecl)) used.add(node.text); + } + ts.forEachChild(node, visit); + }; + visit(sf); + return used; +} + +/** + * Drop every import binding `used` lacks. A declaration left with no binding + * goes whole (its own text and the line break after it; the comments above it + * stay); one left with some keeps them, a one-line `{ … }` on one line, a + * multi-line one as one binding per line. Pure. Returns `{ text, removed }`. + */ +function pruneImports(tm, text, used) { + const ts = tm.ts; + const sf = parseSourceFile(HOST_PATH, text); + const edits = []; + const removed = []; + for (const st of sf.statements) { + if (!ts.isImportDeclaration(st) || !st.importClause) continue; + const clause = st.importClause; + const nb = clause.namedBindings; + const named = nb && ts.isNamedImports(nb) ? [...nb.elements] : []; + const dropDefault = Boolean(clause.name && !used.has(clause.name.text)); + const dropNs = Boolean(nb && ts.isNamespaceImport(nb) && !used.has(nb.name.text)); + const dropNamed = named.filter((e) => !used.has(e.name.text)); + if (!dropDefault && !dropNs && dropNamed.length === 0) continue; + if (dropDefault) removed.push(clause.name.text); + if (dropNs) removed.push(nb.name.text); + for (const e of dropNamed) removed.push(e.name.text); + const keepDefault = Boolean(clause.name && !dropDefault); + const keepNs = Boolean(nb && ts.isNamespaceImport(nb) && !dropNs); + const keepNamed = named.filter((e) => used.has(e.name.text)); + if (!keepDefault && !keepNs && keepNamed.length === 0) { + const end = text[st.getEnd()] === '\n' ? st.getEnd() + 1 : st.getEnd(); + edits.push({ start: st.getStart(sf), end, text: '' }); + continue; + } + if (nb && ts.isNamedImports(nb) && dropNamed.length > 0) { + if (keepNamed.length === 0) { + edits.push({ start: clause.name.getEnd(), end: nb.getEnd(), text: '' }); + } else { + const one = !nb.getText(sf).includes('\n'); + const parts = keepNamed.map((e) => e.getText(sf)); + edits.push({ start: nb.getStart(sf), end: nb.getEnd(), text: one ? `{ ${parts.join(', ')} }` : `{\n${parts.map((x) => ` ${x},`).join('\n')}\n}` }); + } + } + if (dropNs) edits.push({ start: clause.name.getEnd(), end: nb.getEnd(), text: '' }); + if (dropDefault) edits.push({ start: clause.name.getStart(sf), end: nb.getStart(sf), text: '' }); + } + edits.sort((a, b) => b.start - a.start); + let out = text; + for (const e of edits) out = out.slice(0, e.start) + e.text + out.slice(e.end); + return { text: out, removed: removed.sort() }; +} + +// ── Proof 1: byte-identity ─────────────────────────────────────────────────── + +const sha256 = (text) => createHash('sha256').update(text).digest('hex'); + +/** + * Prove a split against its base, from the texts alone (it shares no state with + * `generate`): every generated function is its base method's text, every leaf + * declaration is its base statement's text, and undoing the split rebuilds the + * base `engine.ts` byte-for-byte, once the base's import bindings the head no + * longer reads are pruned from it. + * + * Undoing is: put each body back on its stub (and `async` back), drop the + * generated import / export statements, then splice the leaf's statements back + * between the statements that stayed — the interleaving read off the base by + * NAME only, so every byte of the result comes from the head or the leaf. + * + * The one edit the undo does not reverse by reading the head is the import + * prune: it is recomputed from the head (which import bindings does the head + * still read?) and applied to the base, and every binding it drops must occur + * in a moved body or a moved declaration, so a binding the head lost for any + * other reason fails. + * + * Returns `{ failures, moved, movedTop, pruned, linesBefore, linesAfter }`; no failure = proved. + */ +function verify(tm, baseText, headText, moduleTexts, { readSibling = null } = {}) { + const { Node } = tm; + const failures = []; + const moved = []; + const movedTop = []; + const baseSf = parse(tm, HOST_PATH, baseText); + const baseCls = baseSf.getClass(CLASS_NAME); + const headSf = parse(tm, HOST_PATH, headText); + const headCls = headSf.getClass(CLASS_NAME); + const none = { failures: [`no class ${CLASS_NAME} at base or head`], moved, movedTop, pruned: [], linesBefore: 0, linesAfter: 0 }; + if (!baseCls || !headCls) return none; + const prune = pruneImports(tm, baseText, usedImportNames(tm, headText)); + const target = prune.text; + + const modules = new Map(); + let leaf = null; + for (const [file, text] of moduleTexts) { + if (!text.includes(MARK)) continue; + const sf = parse(tm, join(SRC_DIR, file), text); + if (file === LEAF_MODULE) { + const statements = []; + const stmts = sf.getStatements(); + // The generated header is leading trivia of the first statement when no + // import precedes it; a moved statement's text starts after it. + const headerEnd = text.indexOf('\n\n', text.indexOf(MARK)) + 1; + stmts.forEach((st, i) => { + if (Node.isImportDeclaration(st)) { + if (st.getModuleSpecifierValue() === './engine.js' || SEAM_SPECIFIER.test(st.getModuleSpecifierValue())) { + failures.push(`${file}: imports ${st.getModuleSpecifierValue()} — a leaf never imports engine.ts or an engine-* module`); + } + return; + } + if (i === stmts.length - 1 && Node.isExportDeclaration(st) && !st.getModuleSpecifier() && st.getFullText().includes(MARK)) return; + const names = declaredNames(tm, st); + if (!names) { failures.push(`${file}: a statement that is not a moved declaration: ${st.getText().slice(0, 60)}`); return; } + statements.push({ names, text: text.slice(Math.max(st.getFullStart(), headerEnd), st.getEnd()), node: st }); + }); + leaf = { file, sf, statements }; + if (readSibling) { + // The leaf's relative VALUE imports, followed on disk: none may reach + // engine.ts or an engine-* module. A type-only import is erased, so it + // forms no evaluation cycle and is not followed. + const seen = new Set(); + const queue = valueSpecifiers(text).map((spec) => [file, spec]); + while (queue.length > 0) { + const [from, spec] = queue.shift(); + let rel = relative(SRC_DIR, resolve(dirname(join(SRC_DIR, from)), spec.replace(/\.js$/, '.ts'))); + if (seen.has(rel)) continue; + seen.add(rel); + if (rel === HOST_BASENAME || /^engine-[a-z0-9-]+\.ts$/.test(rel)) { + failures.push(`${file}: reaches ${rel} through its value imports (from ${from}) — not a leaf`); + continue; + } + let sibling = readSibling(rel); + if (sibling === null) { + const index = rel.replace(/\.ts$/, '/index.ts'); + sibling = readSibling(index); + if (sibling !== null) rel = index; + } + if (sibling === null) continue; + for (const next of valueSpecifiers(sibling)) queue.push([rel, next]); + } + } + continue; + } + const fns = new Map(); + for (const st of sf.getStatements()) { + if (Node.isFunctionDeclaration(st)) fns.set(st.getName(), st); + else if (!Node.isImportDeclaration(st) && !Node.isInterfaceDeclaration(st)) { + failures.push(`${file}: a top-level statement that is neither an import, the host interface nor a moved function: ${st.getText().slice(0, 60)}`); + } + } + modules.set(file, { sf, fns, claimed: new Set() }); + } + + const marker = new RegExp(`^\\{\\s*// ${MARK.replace(/[[\]]/g, '\\$&')} The body moved to \`(engine-[a-z0-9-]+\\.ts)\``); + const edits = []; + for (const m of headCls.getMethods()) { + const body = m.getBody(); + if (!body) continue; + const hit = marker.exec(body.getText()); + if (!hit) continue; + const name = m.getName(); + const mod = modules.get(hit[1]); + const fn = mod && mod.fns.get(name); + if (!fn) { failures.push(`${name}: its stub names ${hit[1]}, which has no function ${name}`); continue; } + mod.claimed.add(name); + const base = baseCls.getMethod(name); + if (!base || !base.getBody()) { failures.push(`${name}: no method of that name with a body at base`); continue; } + const fnParams = fn.getParameters().filter((p) => p.getName() !== 'this').map((p) => p.getText()); + const baseParams = base.getParameters().map((p) => p.getText()); + const same = { + body: fn.getBody().getText() === base.getBody().getText(), + params: JSON.stringify(fnParams) === JSON.stringify(baseParams), + returnType: (fn.getReturnTypeNode()?.getText() ?? null) === (base.getReturnTypeNode()?.getText() ?? null), + async: fn.isAsync() === base.isAsync(), + typeParameters: fn.getTypeParameters().length === 0 && base.getTypeParameters().length === 0, + }; + for (const [what, ok] of Object.entries(same)) if (!ok) failures.push(`${name}: the generated function's ${what} is not the base method's`); + moved.push({ + name, module: hit[1], bodySha: sha256(fn.getBody().getText()), baseBodySha: sha256(base.getBody().getText()), + lines: base.getBody().getEndLineNumber() - base.getBody().getStartLineNumber() + 1, text: fn.getText(), ...same, + }); + edits.push({ start: body.getStart(), end: body.getEnd(), text: fn.getBody().getText() }); + if (fn.isAsync()) edits.push({ start: m.getNameNode().getStart(), end: m.getNameNode().getStart(), text: 'async ' }); + } + for (const [file, mod] of modules) { + for (const name of mod.fns.keys()) if (!mod.claimed.has(name)) failures.push(`${file}: function ${name} has no stub on the class`); + } + const generatedSpecifier = (spec) => { + const hit = /^\.\/(engine-[a-z0-9-]+)\.js$/.exec(spec ?? ''); + return Boolean(hit && (modules.has(`${hit[1]}.ts`) || (leaf && `${hit[1]}.ts` === leaf.file))); + }; + for (const st of headSf.getStatements()) { + const drop = (Node.isImportDeclaration(st) && generatedSpecifier(st.getModuleSpecifierValue())) || + (Node.isExportDeclaration(st) && (st.getModuleSpecifier() + ? generatedSpecifier(st.getModuleSpecifierValue()) + : st.getFullText().includes(MARK))); + if (drop) edits.push({ start: st.getFullStart(), end: st.getEnd(), text: '' }); + } + edits.sort((a, b) => b.start - a.start || b.end - a.end); + let rebuilt = headText; + for (const e of edits) rebuilt = rebuilt.slice(0, e.start) + e.text + rebuilt.slice(e.end); + + if (leaf) { + // Splice the leaf back, in the order the base interleaves it by NAME. + const restSf = parse(tm, HOST_PATH, rebuilt); + const rest = restSf.getStatements(); + const queue = [...leaf.statements]; + const baseStmts = parse(tm, HOST_PATH, target).getStatements(); + let j = 0; + let assembled = rest.length > 0 ? rebuilt.slice(0, rest[0].getFullStart()) : ''; + for (const b of baseStmts) { + const names = declaredNames(tm, b); + const key = names ? names.join(',') : null; + if (queue.length > 0 && key !== null && queue[0].names.join(',') === key) { + const l = queue.shift(); + assembled += l.text; + const same = l.text === b.getFullText(); + if (!same) failures.push(`${leaf.file}: ${key} is not its base text (leading comments included)`); + movedTop.push({ + name: key, lines: b.getEndLineNumber() - b.getStartLineNumber(true) + 1, + sha: sha256(l.text), baseSha: sha256(b.getFullText()), same, exported: Node.isExportable(b) && b.isExported(), + }); + continue; + } + if (j >= rest.length) { failures.push(`undoing the split runs out of head statements at base ${key ?? b.getKindName()}`); break; } + assembled += rest[j].getFullText(); + j += 1; + } + if (queue.length > 0) failures.push(`${leaf.file}: ${queue.map((q) => q.names.join(',')).join('; ')} not placed — out of base order or not in the base`); + if (rest.length > 0) assembled += rebuilt.slice(rest[rest.length - 1].getEnd()); + rebuilt = assembled; + } + + const movedText = [...moved.map((m) => m.text), ...(leaf ? leaf.statements.map((s) => s.text) : [])].join('\n'); + for (const n of prune.removed) { + if (!new RegExp(`(^|[^\\w$])${n.replace(/\$/g, '\\$')}($|[^\\w$])`).test(movedText)) { + failures.push(`the head dropped the import \`${n}\`, which no moved body or declaration reads`); + } + } + if (rebuilt !== target) { + const a = rebuilt.split('\n'); + const b = target.split('\n'); + let i = 0; + while (i < Math.max(a.length, b.length) && a[i] === b[i]) i += 1; + failures.push(`undoing the split does not rebuild the base engine.ts (its unread imports pruned): first difference at line ${i + 1}` + + `\n rebuilt: ${JSON.stringify(a[i] ?? '')}\n base: ${JSON.stringify(b[i] ?? '')}`); + } + if (moved.length === 0 && movedTop.length === 0) failures.push('no moved method or declaration found: a proof over nothing is not a proof'); + return { failures, moved, movedTop, pruned: prune.removed, linesBefore: lineCount(baseText), linesAfter: lineCount(headText) }; +} + +/** Lines as `wc -l` counts them: newline characters. */ +function lineCount(text) { + return text.split('\n').length - 1; +} + +// ── Proof 2: the declaration-file bar (ruling 6348891160 item 1) ───────────── + +/** + * The ruling's one normalisation, verbatim: sort the members of each run of + * string-literal union members in place. Nothing else is normalised. + */ +const NORMALISER_ONE_LINER = + 'node -e \'const s=require("fs").readFileSync(process.argv[1],"utf8");process.stdout.write(s.replace(/"[^"\\n]*"(?:\\s*\\|\\s*"[^"\\n]*")+/g,r=>r.split(/\\s*\\|\\s*/).sort().join(" | ")))\' FILE | sha256sum'; + +function normaliseUnionOrder(text) { + return text.replace(/"[^"\n]*"(?:\s*\|\s*"[^"\n]*")+/g, (r) => r.split(/\s*\|\s*/).sort().join(' | ')); +} + +/** Export names of a bundled `.d.ts`: every `export { … }` list, plus `export Name`. */ +function dtsExportNames(text) { + const names = new Set(); + for (const m of text.matchAll(/^export\s*(?:type\s*)?\{([^}]*)\}/gm)) { + for (const part of m[1].split(',')) { + const t = part.trim().replace(/^type\s+/, ''); + if (!t) continue; + const as = /\bas\s+([\w$]+)$/.exec(t); + names.add(as ? as[1] : t); + } + } + for (const m of text.matchAll(/^export\s+(?:declare\s+)?(?:abstract\s+)?(?:class|interface|type|function|const|let|var|enum|namespace)\s+([\w$]+)/gm)) names.add(m[1]); + return [...names].sort(); +} + +/** One positive control: the first `private` member made public. */ +function injectVisibilityChange(text) { + const at = text.search(/^ {4}private /m); + return at < 0 ? null : text.slice(0, at) + ' ' + text.slice(at + ' private '.length); +} + +/** + * The number of lines deleted plus lines inserted between two line arrays — + * what `diff` prints as `<` and `>` lines (Myers' O((N+M)D) edit distance, so + * a handful of changed lines in a 9,000-line file costs nothing). + */ +function lineEditDistance(a, b) { + const n = a.length; + const m = b.length; + const max = n + m; + const off = max + 1; + const v = new Int32Array(2 * max + 3); + for (let d = 0; d <= max; d += 1) { + for (let k = -d; k <= d; k += 2) { + let x = (k === -d || (k !== d && v[off + k - 1] < v[off + k + 1])) ? v[off + k + 1] : v[off + k - 1] + 1; + let y = x - k; + while (x < n && y < m && a[x] === b[y]) { x += 1; y += 1; } + v[off + k] = x; + if (x >= n && y >= m) return d; + } + } + return max; +} + +/** + * The bar on one declaration file pair. Returns `{ failures, reading }`. + * + * `rawDiffLines` counts the lines that differ IN PLACE when both files have the + * same line count (one per swapped union, as the ruling reads it), and the + * lines `diff` would print (deleted plus inserted) when the counts differ; + * `rawDiffMethod` names which. Before, a differing line count read as 0. + */ +function dtsBarFor(file, before, after) { + const failures = []; + const nb = normaliseUnionOrder(before); + const na = normaliseUnionOrder(after); + const bl = before.split('\n'); + const al = after.split('\n'); + let rawDiffLines = 0; + const rawDiffMethod = bl.length === al.length ? 'in-place' : 'diff'; + if (bl.length !== al.length) { + rawDiffLines = lineEditDistance(bl, al); + failures.push(`${file}: ${bl.length} lines at base, ${al.length} at head — a line added or removed is a declaration change, outside the ruling`); + } else { + for (let i = 0; i < bl.length; i += 1) { + if (bl[i] === al[i]) continue; + rawDiffLines += 1; + if (normaliseUnionOrder(bl[i]) !== normaliseUnionOrder(al[i])) { + failures.push(`${file}:${i + 1} differs other than by union member order:\n base: ${bl[i].trim()}\n head: ${al[i].trim()}`); + } + } + } + if (nb !== na) failures.push(`${file}: normalised base and head differ`); + const eb = dtsExportNames(before); + const ea = dtsExportNames(after); + if (JSON.stringify(eb) !== JSON.stringify(ea)) failures.push(`${file}: the export list differs (${eb.length} at base, ${ea.length} at head)`); + const control = injectVisibilityChange(after); + const controlSha = control === null ? null : sha256(normaliseUnionOrder(control)); + if (control === null) failures.push(`${file}: no private member to make public — the positive control could not run (INCONCLUSIVE)`); + else if (controlSha === sha256(nb)) failures.push(`${file}: the positive control normalises to the base hash — the normaliser hides a visibility change (INCONCLUSIVE)`); + return { + failures, + reading: { + file, + rawBase: sha256(before), rawHead: sha256(after), rawDiffLines, rawDiffMethod, + normalisedBase: sha256(nb), normalisedHead: sha256(na), + exportsBase: eb.length, exportsHead: ea.length, + positiveControl: controlSha, + }, + }; +} + +function dtsBar(beforeDir, afterDir) { + const failures = []; + const readings = []; + for (const file of ['index.d.ts', 'index.d.mts']) { + const b = join(beforeDir, file); + const a = join(afterDir, file); + if (!existsSync(b) || !existsSync(a)) refuse(`--dts: ${file} missing under ${existsSync(b) ? afterDir : beforeDir}`); + const r = dtsBarFor(file, readFileSync(b, 'utf8'), readFileSync(a, 'utf8')); + failures.push(...r.failures); + readings.push(r.reading); + } + return { failures, readings }; +} + +// ── Reports ────────────────────────────────────────────────────────────────── + +function planReport(planned) { + const out = []; + const bySection = new Map(); + for (const r of planned.records) { + const key = r.section ?? '(above the first anchor)'; + if (!bySection.has(key)) bySection.set(key, []); + bySection.get(key).push(r); + } + let movedMethods = 0; + let movedLines = 0; + let movedSaved = 0; + const reasonCounts = new Map(); + for (const [section, rs] of bySection) { + const movers = rs.filter((r) => r.reasons.length === 0); + const lines = movers.reduce((n, r) => n + r.bodyLines, 0); + const saved = movers.reduce((n, r) => n + r.bodyLines - r.stubLines, 0); + movedMethods += movers.length; + movedLines += lines; + movedSaved += saved; + out.push(`§ ${section} -> ${rs[0].module ?? '(stays)'}: ${movers.length}/${rs.length} methods move ` + + `(${lines} body lines out, ${lines - saved} stub lines in)`); + for (const r of rs) { + if (r.reasons.length === 0) out.push(` move ${r.name} (body ${r.bodyLines} -> stub ${r.stubLines})`); + else out.push(` stay ${r.name} (${r.lineCount}) — ${r.reasons.join('; ')}`); + for (const reason of r.reasons) { + const code = reason.split(':')[0]; + reasonCounts.set(code, (reasonCounts.get(code) ?? 0) + 1); + } + } + } + out.push(''); + out.push(`total: ${movedMethods}/${planned.records.length} methods move into ${planned.modules.size} module(s): ` + + `${movedLines} body lines out, ${movedLines - movedSaved} stub lines in, plus ${2 * planned.modules.size} import lines`); + out.push(`stay reasons (a method may carry several): ${[...reasonCounts].sort((a, b) => b[1] - a[1]).map(([c, n]) => `${c} ${n}`).join(', ')}`); + const top = planned.top; + out.push(''); + out.push(`§ top-level declarations ahead of the class -> ${LEAF_MODULE}: ${top.movers.length}/${top.records.length} move`); + const topReasons = new Map(); + for (const r of top.records) { + if (r.reasons.length === 0) out.push(` move ${r.label} (${r.lines})${r.exported ? ' [exported, re-exported]' : ''}`); + else out.push(` stay ${r.label} (${r.lines}) — ${r.reasons.join('; ')}`); + for (const reason of r.reasons) { + const code = reason.split(':')[0]; + topReasons.set(code, (topReasons.get(code) ?? 0) + 1); + } + } + const topLines = top.movers.reduce((n, r) => n + r.lines, 0); + out.push(`top-level total: ${top.movers.length}/${top.records.length} declarations move (${topLines} lines with their docblocks); ` + + `stay reasons: ${[...topReasons].sort((a, b) => b[1] - a[1]).map(([c, n]) => `${c} ${n}`).join(', ') || 'none'}`); + out.push(`lever (a): ${planned.hostExportsForSections.size} staying name(s) exported for the section modules` + + `${planned.hostExportsForSections.size ? ` (${[...planned.hostExportsForSections.keys()].sort().join(', ')})` : ''}; ` + + `${planned.leafExports.length} leaf name(s) exported beside the base exports`); + return out.join('\n'); +} + +function treeHash(files) { + const h = createHash('sha256'); + for (const [name, text] of [...files].sort(([a], [b]) => (a < b ? -1 : 1))) h.update(`${name}\0${sha256(text)}\n`); + return h.digest('hex'); +} + +function printVerify(v) { + for (const f of v.failures) console.error(` ✗ ${f}`); + const identical = v.moved.filter((m) => m.body && m.params && m.returnType && m.async).length; + const identicalTop = v.movedTop.filter((m) => m.same).length; + console.log(`byte-identity: ${identical}/${v.moved.length} moved bodies identical to base (body, parameters, return type, async); ` + + `${identicalTop}/${v.movedTop.length} moved top-level declarations identical to base (leading comments included); ` + + `engine.ts ${v.linesBefore} -> ${v.linesAfter} lines; undoing the split ${v.failures.some((f) => f.startsWith('undoing')) ? 'does NOT' : 'does'} rebuild the base byte-for-byte ` + + `once the ${v.pruned.length} import binding(s) only moved text reads are pruned from the base (${v.pruned.join(', ') || 'none'})`); +} + +function gitShow(ref, path) { + return execFileSync('git', ['show', `${ref}:${path}`], { cwd: ROOT, encoding: 'utf8', maxBuffer: 1 << 28 }); +} + +/** A file under `engine.ts`'s directory by its relative path, or null when there is none. */ +function readSibling(rel) { + const p = join(SRC_DIR, rel); + return existsSync(p) ? readFileSync(p, 'utf8') : null; +} + +function generatedModulesOnDisk() { + const out = new Map(); + for (const file of [LEAF_MODULE, ...SECTION_ANCHORS.map((row) => row[2])]) { + if (!file) continue; + const p = join(SRC_DIR, file); + if (existsSync(p)) out.set(file, readFileSync(p, 'utf8')); + } + return out; +} + +// ── Entry ──────────────────────────────────────────────────────────────────── + +const USAGE = [ + 'usage: node scripts/codemod-split-engine.mjs [--write] plan (or apply) the split of an UNSPLIT engine.ts', + ' node scripts/codemod-split-engine.mjs --verify --base REF re-prove a split on disk against engine.ts at REF', + ' node scripts/codemod-split-engine.mjs --dts BEFORE_DIST AFTER_DIST the declaration-file bar on two built dists', + ' node scripts/codemod-split-engine.mjs --self-test', + ' node scripts/codemod-split-engine.mjs --help', +].join('\n'); + +async function main(argv) { + // Answered before any guard: `--help` reads no tree, so the "already split" + // refusal below must never be what a reader asking for usage gets. + if (argv.includes('--help') || argv.includes('-h')) { + console.log(USAGE); + return 0; + } + if (argv.includes('--self-test')) { + await selfTest(); + if (process.exitCode === 1) return 1; + if (!selfTestReachedVerdict) { + console.error('\n✗ codemod-split-engine self-test: selfTest() returned without reaching its verdict,\n' + + 'so no success line was printed. Exiting 0 here would report a self-test that never\n' + + 'finished as a self-test that passed.'); + return 1; + } + return 0; + } + if (argv.includes('--dts')) { + const i = argv.indexOf('--dts'); + const [before, after] = [argv[i + 1], argv[i + 2]]; + if (!before || !after) refuse('--dts needs '); + const { failures, readings } = dtsBar(before, after); + console.log(`normaliser (ruling 6348891160 item 2, verbatim): ${NORMALISER_ONE_LINER}`); + for (const r of readings) console.log(JSON.stringify(r)); + for (const f of failures) console.error(` ✗ ${f}`); + console.log(failures.length === 0 ? '✓ declaration-file bar: PASS' : `✗ declaration-file bar: FAIL (${failures.length})`); + return failures.length === 0 ? 0 : 1; + } + const tm = await loadTsMorph(); + if (argv.includes('--verify')) { + const i = argv.indexOf('--base'); + const ref = i >= 0 ? argv[i + 1] : null; + if (!ref) refuse('--verify needs --base '); + const v = verify(tm, gitShow(ref, `${SRC_REL}/${HOST_BASENAME}`), readFileSync(HOST_PATH, 'utf8'), generatedModulesOnDisk(), { readSibling }); + printVerify(v); + return v.failures.length === 0 ? 0 : 1; + } + const input = readFileSync(HOST_PATH, 'utf8'); + if (input.includes(MARK)) refuse(`${HOST_BASENAME} already carries ${MARK} stubs: run the script on an unsplit file (re-run on the latest main, never on its own output)`); + const planned = plan(tm, input); + console.log(planReport(planned)); + const out = generate(tm, input, planned); + const files = new Map([[HOST_BASENAME, out.host], ...out.modules]); + console.log(`output tree sha256: ${treeHash(files)} (input sha256 ${sha256(input)})`); + const v = verify(tm, input, out.host, out.modules, { readSibling }); + printVerify(v); + if (v.failures.length > 0) { + console.error('✗ the generated split fails byte-identity: nothing written. Report the fork; never hand-edit the split.'); + return 1; + } + if (!argv.includes('--write')) { + console.log('plan only: nothing written (pass --write to apply).'); + return 0; + } + for (const [file] of out.modules) { + const p = join(SRC_DIR, file); + if (existsSync(p) && !readFileSync(p, 'utf8').includes(MARK)) refuse(`${file} exists and was not written by this script`); + } + writeFileSync(HOST_PATH, out.host); + for (const [file, text] of out.modules) writeFileSync(join(SRC_DIR, file), text); + console.log(`✓ wrote ${HOST_BASENAME} and ${out.modules.size} module(s) under ${SRC_REL}`); + return 0; +} + + +/** + * The relative specifiers a module's VALUE imports and re-exports name: an + * `import type` / `export type` declaration, or a named list whose every + * member is `type`-marked, is erased by the compiler and is left out. + */ +function valueSpecifiers(src) { + const out = []; + for (const m of src.matchAll(/^[ \t]*(?:import|export)\s+(type\s+)?([^;'"]*?)\s*from\s*'(\.{1,2}\/[^']+)'/gm)) { + if (m[1]) continue; + const inner = /^\{([^}]*)\}$/.exec(m[2].trim()); + if (inner && inner[1].split(',').map((s) => s.trim()).filter(Boolean).every((s) => s.startsWith('type '))) continue; + out.push(m[3]); + } + for (const m of src.matchAll(/^[ \t]*import\s*'(\.{1,2}\/[^']+)'/gm)) out.push(m[1]); + return out; +} + +// ── Self-test ──────────────────────────────────────────────────────────────── +// +// The proofs above are this script's whole value, and a proof that cannot fail +// proves nothing, so the self-test feeds each one an input it must REFUSE. The +// roster pins battery NAMES with a floor each, the roster's own size is pinned, +// and the verdict sets a flag the dispatch reads (AGENTS.md, "Writing a +// `--self-test`"), so a battery that stops running, or a `return` above the +// verdict, cannot read as a pass. + +let selfTestReachedVerdict = false; + +const SELF_TEST_BATTERIES = Object.freeze({ + 'declaration bar': 7, + 'stay rules': 12, + 'top-level moves': 11, + 'byte-identity proof': 10, + 'import prune': 4, + 'refusals': 6, +}); +const SELF_TEST_BATTERY_FLOOR = 6; + +const FIXTURE_ANCHORS = Object.freeze([ + ['count', 'Lifecycle', 'engine-lifecycle.ts'], + ['keptSection', 'Kept', null], +]); +const FIXTURE_NEEDLES = Object.freeze({ 'PINNED_TEXT': 'a fixture reader', 'needleInMethod(': 'a fixture reader' }); +const FIXTURE_CLASS_NEEDLES = Object.freeze({ 'classReaderText(': 'a fixture reader that also reads the leaf' }); +const FIXTURE_PINS = Object.freeze({ pinnedMethod: 'a fixture reader' }); +const fixturePlan = (tm, text) => plan(tm, text, + { anchors: FIXTURE_ANCHORS, pins: FIXTURE_PINS, needles: FIXTURE_NEEDLES, classNeedles: FIXTURE_CLASS_NEEDLES }); + +function fixtureSource(extraMethods = '') { + const pad = Array.from({ length: 12 }, (_, i) => ` const p${i} = ${i};`).join('\n'); + return `import type { Knex } from 'knex'; +import { helperImported } from './other.js'; +import { onlyMovedReads } from './moved-only.js'; +import { seamHelper } from './engine-seam.js'; + +/** A value the leaf takes, read by a moved body. */ +const LOCAL_VALUE = 1; +type LocalType = { n: number }; +export interface ExportedShape { a: number } + +// A reader takes this one out of the file by path. +const PINNED = 'PINNED_TEXT'; +function dependsOnPinned(): string { + return PINNED; +} +function namesClass(d: ObjectQL): ObjectQL { + return d; +} +const VIA_SEAM = seamHelper(); +export const DERIVED = LOCAL_VALUE + 1; +function classReaderText(n: number): number { + return n; +} + +export class ObjectQL { + private count = 0; + private knex: any = null; + protected logger: { warn(m: string): void } = console; + private readonly fn = (x: string): string => x; + private static readonly SECRET = 1; + + /** Docblock that stays. */ + private async movable(object: string, trx?: Knex.Transaction): Promise { +${pad} + const a = helperImported(object) as ExportedShape | string; + this.logger.warn(String(a) + onlyMovedReads()); + return this.other(String(a)) + String(trx ? this.count : 0); + } + + protected other(x: string): string { + return x; + } + + protected writes(): void { +${pad} + this.count = 1; + } + + protected asValue(): string[] { +${pad} + return ['a'].map(this.other); + } + + protected readsLocal(): number { +${pad} + return LOCAL_VALUE + dependsOnPinned().length; + } + + protected namesLocalType(): LocalType | null { +${pad} + return null; + } + + public inferred() { +${pad} + return 1; + } + + protected scoped(object: string): string { +${pad} + return this.getBuilder(object); + } + + protected getBuilder(object: string): string { + return object; + } + + protected defaulted(x: string = 'a'): string { +${pad} + return x; + } + + protected callsField(): string { +${pad} + return this.fn('a'); + } + + protected readsStatic(): number { +${pad} + return ObjectQL.SECRET; + } + + protected pinnedMethod(): number { +${pad} + return 3; + } + + protected carriesNeedle(): number { +${pad} + return needleInMethod(1); + } + + protected carriesClassNeedle(): number { +${pad} + return classReaderText(1); + } +${extraMethods} + protected keptSection(): number { +${pad} + return 2 + VIA_SEAM + namesClass(this).count; + } +} + +function needleInMethod(n: number): number { + return n; +} +`; +} + +async function selfTest() { + const tm = await loadTsMorph(); + const batterySeen = new Map(); + let battery = null; + let failed = 0; + let cases = 0; + const open = (name) => { battery = name; }; + const check = (label, ok, detail = '') => { + batterySeen.set(battery, (batterySeen.get(battery) ?? 0) + 1); + cases += 1; + if (ok) console.log(` ✓ [${battery}] ${label}`); + else { failed += 1; console.error(` ✗ [${battery}] ${label}${detail ? ` — ${detail}` : ''}`); } + }; + const refused = (fn) => { try { fn(); return false; } catch (e) { return e instanceof CodemodRefusal; } }; + + open('declaration bar'); + { + const base = 'export declare class A {\n private x;\n t: "b" | "a";\n}\nexport { A };\n'; + const swapped = base.replace('"b" | "a"', '"a" | "b"'); + check('a union-member swap passes, and is counted as one raw diff line', + (() => { const r = dtsBarFor('f', base, swapped); return r.failures.length === 0 && r.reading.rawDiffLines === 1; })()); + check('the normaliser sorts union members and nothing else', normaliseUnionOrder('x: "b" | "a"; y: B | A') === 'x: "a" | "b"; y: B | A'); + check('a retyped member fails', dtsBarFor('f', base, base.replace('"b" | "a"', '"b" | "c"')).failures.length > 0); + check('a private member made public fails (the positive control the bar runs)', + dtsBarFor('f', base, base.replace(' private x;', ' x;')).failures.length > 0); + check('an added line fails', dtsBarFor('f', base, base.replace('}\n', ' y: number;\n}\n')).failures.length > 0); + check('a line-count change is counted as diff counts it, never as 0', + (() => { const r = dtsBarFor('f', base, base.replace('}\n', ' y: number;\n}\n')).reading; return r.rawDiffLines === 1 && r.rawDiffMethod === 'diff'; })() + && lineEditDistance(['a', 'b', 'c'], ['a', 'x', 'c', 'd']) === 3); + check('a changed export list fails', dtsBarFor('f', base, base.replace('export { A };', 'export { A as B };')).failures.length > 0); + } + + open('stay rules'); + { + const planned = fixturePlan(tm, fixtureSource()); + const why = (name) => (planned.records.find((r) => r.name === name)?.reasons ?? ['']).join('; '); + const stays = (name, code) => check(`${name} stays (${code})`, why(name).split('; ').some((r) => r.startsWith(code)), why(name)); + const moves = (name, because) => check(`${name} moves (${because})`, why(name) === '', why(name)); + moves('movable', 'the canonical body, private members read through the host'); + moves('readsLocal', 'lever (a)/(b): it imports the leaf value and the staying function'); + moves('namesLocalType', 'a non-exported type it names moves to the leaf'); + moves('scoped', 'a method it calls goes as a closure'); + stays('other', 'no-yield'); + stays('writes', 'this-write'); + stays('asValue', 'method-as-value'); + stays('inferred', 'inferred-return-type'); + stays('defaulted', 'parameter-shape'); + stays('readsStatic', 'non-public-static'); + check('a pinned method and a method carrying a needle stay (path-reader)', + why('pinnedMethod').startsWith('path-reader') && why('carriesNeedle').startsWith('path-reader'), + `${why('pinnedMethod')} | ${why('carriesNeedle')}`); + check('a method in a section that maps to no module stays (section-kept)', why('keptSection').startsWith('section-kept'), why('keptSection')); + } + + open('top-level moves'); + { + const input = fixtureSource(); + const planned = fixturePlan(tm, input); + const top = (label) => planned.top.records.find((r) => r.label === label); + const why = (label) => (top(label)?.reasons ?? ['']).join('; '); + check('a value, a local type and an exported interface move', ['LOCAL_VALUE', 'LocalType', 'ExportedShape', 'DERIVED'] + .every((n) => why(n) === ''), ['LOCAL_VALUE', 'LocalType', 'ExportedShape', 'DERIVED'].map(why).join(' | ')); + check('a declaration carrying a reader\'s needle stays (path-reader)', why('PINNED').startsWith('path-reader'), why('PINNED')); + check('a declaration carrying a class-scoped needle moves: its reader reads the leaf', why('classReaderText') === '', why('classReaderText')); + check('a declaration naming a stayer stays (needs-stayer)', why('dependsOnPinned').startsWith('needs-stayer'), why('dependsOnPinned')); + check('a declaration naming the class stays (class-reference)', why('namesClass').startsWith('class-reference'), why('namesClass')); + check('a declaration using an engine-* import stays (seam-import)', why('VIA_SEAM').startsWith('seam-import'), why('VIA_SEAM')); + check('a declaration after the class is not in the region', !top('needleInMethod')); + const out = generate(tm, input, planned); + const leaf = out.modules.get(LEAF_MODULE) ?? ''; + check('the leaf header is line comments, never a docblock the bundled .d.ts would publish', + leaf.includes(MARK) && !leaf.slice(0, leaf.indexOf('\n\n', leaf.indexOf(MARK))).includes('/*')); + check('the leaf imports neither engine.ts nor an engine-* module', leaf.length > 0 && !/from '\.\/engine[.-]/.test(leaf), leaf.slice(0, 200)); + check('engine.ts re-exports the exported movers by name, the interface as a type', + /export \{ DERIVED, type ExportedShape \} from '\.\/engine-top-level\.js';/.test(out.host)); + check('lever (a): the staying function a moved body reads is exported for its module, and the module imports it', + /export \{ dependsOnPinned \};/.test(out.host) && + /import \{ dependsOnPinned \} from '\.\/engine\.js';/.test(out.modules.get('engine-lifecycle.ts') ?? '')); + } + + open('byte-identity proof'); + { + const input = fixtureSource(); + const planned = fixturePlan(tm, input); + const out = generate(tm, input, planned); + const again = generate(tm, input, fixturePlan(tm, input)); + const v = verify(tm, input, out.host, out.modules); + check('the generated split proves against its base', v.failures.length === 0 && v.moved.length === 4 && v.movedTop.length === 5, + [...v.failures, `moved: ${v.moved.map((m) => m.name).join()}`, `top: ${v.movedTop.map((m) => m.name).join()}`].join(' | ')); + check('two runs on one input write byte-identical trees', + treeHash(new Map([['h', out.host], ...out.modules])) === treeHash(new Map([['h', again.host], ...again.modules]))); + const mod = out.modules.get('engine-lifecycle.ts'); + const edited = new Map([...out.modules, ['engine-lifecycle.ts', mod.replace('const p3 = 3;', 'const p3 = 4;')]]); + check('one edited character in a moved body fails', verify(tm, input, out.host, edited).failures.length > 0); + check('an edited stub signature fails', + verify(tm, input, out.host.replace('trx?: Knex.Transaction): Promise', 'trx?: Knex.Transaction): Promise'), out.modules).failures.length > 0); + check('an extra top-level statement in a module fails', + verify(tm, input, out.host, new Map([...out.modules, ['engine-lifecycle.ts', `${mod}\nexport const extra = 1;\n`]])).failures.length > 0); + check('a proof over no moved method fails', verify(tm, input, input, new Map()).failures.length > 0); + const leaf = out.modules.get(LEAF_MODULE); + const leafWith = (text) => new Map([...out.modules, [LEAF_MODULE, text]]); + check('one edited character in a moved declaration fails', + verify(tm, input, out.host, leafWith(leaf.replace('const LOCAL_VALUE = 1;', 'const LOCAL_VALUE = 2;'))).failures.length > 0); + check('an edited comment above a moved declaration fails', + verify(tm, input, out.host, leafWith(leaf.replace('/** A value the leaf takes', '/** A value the leaf holds'))).failures.length > 0); + check('a leaf that imports engine.ts back fails', + verify(tm, input, out.host, leafWith(leaf.replace('\nconst LOCAL_VALUE', "\nimport { ObjectQL } from './engine.js';\nconst LOCAL_VALUE"))).failures.length > 0); + const swapped = leaf.replace('type LocalType = { n: number };\nexport interface ExportedShape { a: number }', + 'export interface ExportedShape { a: number }\ntype LocalType = { n: number };'); + check('two moved declarations out of base order fail', swapped !== leaf && verify(tm, input, out.host, leafWith(swapped)).failures.length > 0); + } + + open('import prune'); + { + const input = fixtureSource(); + const out = generate(tm, input, fixturePlan(tm, input)); + check('an import only a moved body read is pruned from engine.ts', out.pruned.includes('onlyMovedReads') && + !out.host.includes('onlyMovedReads'), out.pruned.join()); + check('an import the staying text still reads is kept', out.host.includes("import { seamHelper } from './engine-seam.js';")); + const v = verify(tm, input, out.host, out.modules); + check('the proof recomputes the prune from the head and names it', v.failures.length === 0 && v.pruned.includes('onlyMovedReads'), v.failures.join(' | ')); + const lost = out.host.replace("import { seamHelper } from './engine-seam.js';\n", ''); + check('an import the head lost that no moved text reads fails', verify(tm, input, lost, out.modules).failures.length > 0); + } + + open('refusals'); + { + check('an anchor the class does not declare refuses', + refused(() => plan(tm, fixtureSource(), { anchors: [['gone', 'x', 'engine-x.ts']], pins: {}, needles: {}, classNeedles: {} }))); + check('anchors out of class order refuse', + refused(() => plan(tm, fixtureSource(), { anchors: [['keptSection', 'x', null], ['count', 'y', 'engine-y.ts']], pins: {}, needles: {}, classNeedles: {} }))); + check('a pin naming what the file lacks refuses', refused(() => plan(tm, fixtureSource(), { anchors: FIXTURE_ANCHORS, pins: { gone: 'x' }, needles: {}, classNeedles: {} }))); + check('a class-scoped needle the file no longer carries refuses', + refused(() => plan(tm, fixtureSource(), { anchors: FIXTURE_ANCHORS, pins: {}, needles: {}, classNeedles: { 'gone text': 'x' } }))); + check('a needle the file no longer carries refuses', refused(() => plan(tm, fixtureSource(), { anchors: FIXTURE_ANCHORS, pins: {}, needles: { 'gone text': 'x' }, classNeedles: {} }))); + // `--help` reads no tree: it answers before the "already split" guard, so + // it exits 0 whether engine.ts on disk is split (as on a branch) or not. + check('--help is answered, never refused, whatever engine.ts holds', (await main(['--help']).catch(() => 2)) === 0); + } + + const declared = Object.keys(SELF_TEST_BATTERIES); + if (declared.length < SELF_TEST_BATTERY_FLOOR) { + failed += 1; + console.error(`✗ self-test floor: ${declared.length} batteries declared, below the pinned ${SELF_TEST_BATTERY_FLOOR}`); + } + for (const [name, count] of batterySeen) { + if (!declared.includes(name)) { failed += 1; console.error(`✗ self-test floor: battery "${name}" ran ${count} case(s) but is not declared`); } + } + for (const name of declared) { + const count = batterySeen.get(name) ?? 0; + if (count < SELF_TEST_BATTERIES[name]) { + failed += 1; + console.error(`✗ self-test floor: battery "${name}" ran ${count} case(s), below its pinned ${SELF_TEST_BATTERIES[name]}`); + } + } + if (failed > 0) { + console.error(`\n✗ codemod-split-engine self-test: ${failed} failure(s) (cases and floor).`); + process.exitCode = 1; + selfTestReachedVerdict = true; + return; + } + console.log(`\n✓ codemod-split-engine self-test: ${cases} cases in ${declared.length} batteries pass.`); + selfTestReachedVerdict = true; +} + +if (isEntrypoint(import.meta.url)) { + main(process.argv.slice(2)).then( + (code) => process.exit(code), + (error) => { + if (error instanceof CodemodRefusal) { + console.error(`✗ codemod-split-engine: ${error.message}`); + process.exit(2); + } + console.error(error); + process.exit(2); + }, + ); +} From a33e087be311311afc5d3fd4939972f8622b630e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 02:35:33 +0000 Subject: [PATCH 02/13] refactor(objectql): generate the engine.ts split with the codemod Output of `node scripts/codemod-split-engine.mjs --write` on main 0b99c2b1, not edited by hand: 39 ObjectQL method bodies move into 13 section modules, the 146 top-level declarations ahead of the class move whole into engine-top-level.ts, and engine.ts drops the 85 import bindings only the moved text read. engine.ts goes from 23,245 to 16,869 lines. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- packages/objectql/src/engine-actions.ts | 70 + .../objectql/src/engine-cascade-planning.ts | 165 + .../src/engine-defaults-and-autonumbers.ts | 375 + .../src/engine-delete-and-aggregate.ts | 557 ++ packages/objectql/src/engine-find.ts | 180 + .../src/engine-hooks-and-functions.ts | 122 + packages/objectql/src/engine-insert.ts | 76 + .../src/engine-middleware-and-context.ts | 147 + .../engine-references-and-master-detail.ts | 81 + packages/objectql/src/engine-registration.ts | 348 + .../src/engine-secrets-and-routing.ts | 110 + .../objectql/src/engine-security-resolvers.ts | 120 + .../src/engine-summaries-and-expansion.ts | 190 + packages/objectql/src/engine-top-level.ts | 4976 +++++++++++ packages/objectql/src/engine.ts | 7456 ++--------------- 15 files changed, 8057 insertions(+), 6916 deletions(-) create mode 100644 packages/objectql/src/engine-actions.ts create mode 100644 packages/objectql/src/engine-cascade-planning.ts create mode 100644 packages/objectql/src/engine-defaults-and-autonumbers.ts create mode 100644 packages/objectql/src/engine-delete-and-aggregate.ts create mode 100644 packages/objectql/src/engine-find.ts create mode 100644 packages/objectql/src/engine-hooks-and-functions.ts create mode 100644 packages/objectql/src/engine-insert.ts create mode 100644 packages/objectql/src/engine-middleware-and-context.ts create mode 100644 packages/objectql/src/engine-references-and-master-detail.ts create mode 100644 packages/objectql/src/engine-registration.ts create mode 100644 packages/objectql/src/engine-secrets-and-routing.ts create mode 100644 packages/objectql/src/engine-security-resolvers.ts create mode 100644 packages/objectql/src/engine-summaries-and-expansion.ts create mode 100644 packages/objectql/src/engine-top-level.ts diff --git a/packages/objectql/src/engine-actions.ts b/packages/objectql/src/engine-actions.ts new file mode 100644 index 00000000000..9fe7841707f --- /dev/null +++ b/packages/objectql/src/engine-actions.ts @@ -0,0 +1,70 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Action system and packaged-action activation" section + * of `ObjectQL`: the bodies of `registerAction`, `listRegisteredActions` moved + * out of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link ActionsHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface ActionsHost { + readonly actions: ObjectQL['actions']; + readonly logger: ObjectQL['logger']; +} + +/** The body of `ObjectQL#registerAction`; the method's docblock there is its contract. */ +export function registerAction(this: Pick, objectName: string, actionName: string, handler: (ctx: any) => Promise | any, packageName?: string): void { + const key = `${objectName}:${actionName}`; + const replaced = this.actions.get(key); + if (replaced !== undefined && replaced.package !== packageName) { + const by = (name: string | undefined) => (name ? `by package '${name}'` : 'with no package'); + this.logger.warn( + `Action handler '${key}' registered ${by(packageName)} replaces the one registered ${by(replaced.package)}: ` + + `the action door now runs the handler registered ${by(packageName)}, whichever declaration object ` + + `'${objectName}' lists${packageName ? `, and withdrawing package '${packageName}' leaves the action with no handler` : ''}. ` + + `Two owners share one handler key only when both declare action '${actionName}' on object ` + + `'${objectName}' — rename one of them.`, + { objectName, actionName, package: packageName, replacedPackage: replaced.package }, + ); + } + this.actions.set(key, { handler, package: packageName }); + this.logger.debug('Registered action', { objectName, actionName, package: packageName }); + } + +/** The body of `ObjectQL#listRegisteredActions`; the method's docblock there is its contract. */ +export function listRegisteredActions(this: Pick): Array<{ objectName: string; actionName: string; package?: string }> { + const out: Array<{ objectName: string; actionName: string; package?: string }> = []; + for (const [key, entry] of this.actions.entries()) { + const sep = key.indexOf(':'); + if (sep < 0) continue; + out.push({ + objectName: key.slice(0, sep), + actionName: key.slice(sep + 1), + ...(entry.package ? { package: entry.package } : {}), + }); + } + return out; + } diff --git a/packages/objectql/src/engine-cascade-planning.ts b/packages/objectql/src/engine-cascade-planning.ts new file mode 100644 index 00000000000..91f259646cd --- /dev/null +++ b/packages/objectql/src/engine-cascade-planning.ts @@ -0,0 +1,165 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Cascade and referential delete planning" section of + * `ObjectQL`: the bodies of `warnCascadeNotAtomic`, `cascadeRelationBehavior`, + * `referenceProbeFilterOfSet`, `isTenantColumnSetNull`, + * `organizationCountIsDisclosable` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link CascadePlanningHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import type { ServiceObject } from '@objectstack/spec/data'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { ORGANIZATION_OBJECT, resolveTenantFieldName } from './tenancy/system-write-organization.js'; +import { isFederatedUnprovisionedInjectedColumn } from './federated-object.js'; +import { declaredMultiValued, deleteWalkTargetOf, statesDeleteSemantics } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface CascadePlanningHost { + readonly cascadeNotAtomicReported: ObjectQL['cascadeNotAtomicReported']; + count: ObjectQL['count']; + readonly defaultDriver: ObjectQL['defaultDriver']; + readonly logger: ObjectQL['logger']; + referenceProbeFilter: ObjectQL['referenceProbeFilter']; + resolveObjectName: ObjectQL['resolveObjectName']; +} + +/** The body of `ObjectQL#warnCascadeNotAtomic`; the method's docblock there is its contract. */ +export function warnCascadeNotAtomic(this: Pick, object: string): void { + if (this.cascadeNotAtomicReported.has(object)) return; + this.cascadeNotAtomicReported.add(object); + this.logger.warn( + `Cascade delete of '${object}' cannot run as one unit of work: the cascade reaches an object routed ` + + `to a datasource other than the default one ('${this.defaultDriver ?? ''}'), and a transaction ` + + "covers one driver's connection only (ADR-0119 D1 — no two-phase commit). The cascade therefore runs " + + 'UNWRAPPED, as every cascade did before a single-datasource cascade was made one transaction: if a ' + + 'later dependent refuses the delete, the rows already ' + + 'removed stay removed while the call rejects. Route the cascading objects to one datasource to get the ' + + 'atomic path. Reported once per object per engine instance.', + { object, defaultDatasource: this.defaultDriver ?? undefined }, + ); + } + +/** The body of `ObjectQL#cascadeRelationBehavior`; the method's docblock there is its contract. */ +export function cascadeRelationBehavior(this: Pick, object: string, child: ServiceObject, fieldName: string, fdef: any): string | undefined { + // [#22957] The family test is the shared predicate, so this reader, the + // two walks that read it and `planCascadeAtomicity` admit one family. + if (!statesDeleteSemantics(fdef)) return undefined; + // [#18550] Same arbiter, same absence-vs-unreadability split as + // {@link ObjectQL.planCascadeAtomicity} states above — and this is the + // seam where the silence was measurable end to end: an unreadable + // carrier made the relation invisible to the cascade, so `delete()` + // removed the parent, left a `master_detail` child behind, and + // reported success. No `restrict` refusal, no `set_null`, nothing + // logged. Absence still skips the field here exactly as before. + const ref = deleteWalkTargetOf(fdef, 'ObjectQL.cascadeDeleteRelations'); + if (!ref) return undefined; + // Match the target object by raw or resolved name. + let resolvedRef: string | undefined; + try { resolvedRef = this.resolveObjectName(ref); } catch { resolvedRef = undefined; } + if (ref !== object && resolvedRef !== object) return undefined; + + // [#21910, #21918] A lookup the registry INJECTED into a federated + // object, and the object does not provision, is not a reference to + // anything, so it is not a relation to probe. That is the tenant + // anchor `organization_id`, the ADR-0117 D1 anchor + // `owning_business_unit_id`, and the owner and audit lookups + // `owner_id` / `created_by` / `updated_by`. On a federated object each + // exists in the registered schema and nowhere else: the probe below + // was refused by the driver (`INVALID_FILTER`, no such column), its + // catch propagated the refusal as #8895 rules for a missing column, + // and deleting the organization, business unit or user it names was + // refused on a deployment with a federated object bound. + // `buildDriverOptions` and the related-record read already refuse this + // reading of the tenant column. + // {@link isFederatedUnprovisionedInjectedColumn} reads which columns + // those are from the registry's own provenance, never from a list of + // names, and {@link ObjectQL.planCascadeAtomicity} asks it too. + // + // ⛔ The probe's catch ({@link probeReferencingRows}) is deliberately NOT + // widened to pass a missing column as benign. That would invert #8895's + // discriminate or propagate for every object, not just these injected + // columns: a lookup the author declared on a federated object, including + // an author's own `organization_id` or `owner_id`, stays in the scan, and + // its probe failure still propagates. + if (isFederatedUnprovisionedInjectedColumn(child, fieldName)) return undefined; + + // A master-detail parent owns its children: cascade by default (the + // child FK is typically required, so set_null would be invalid). Only + // an explicit `restrict` deviates. A plain lookup honors its + // configured deleteBehavior (default set_null). + // + // [#22957] A `tree` reads like a `lookup`: the spec materializes the same + // `set_null` default on it, and an authored `cascade` / `restrict` is + // honoured as written. Its `cascade` is a subtree delete, and a cycle in + // the data (a corrupt row pointing at its own descendant) terminates on + // the cascade set, the same membership test any self-reference has. + // + // [#29253] A `user` field reaches here only with an authored value + // ({@link statesDeleteSemantics}), and reads it as a `lookup` reads its + // own: the `|| 'set_null'` fallback below is never what a `user` gets. + // + // [#9625] "Only an explicit `restrict` deviates" is the whole of it: + // every other value a master_detail can declare — including an + // explicit `deleteBehavior: 'set_null'` — resolves to `cascade` here, + // silently. Measured and pinned (`engine-cascade-delete.test.ts`). + return fdef.type === 'master_detail' + ? (fdef.deleteBehavior === 'restrict' ? 'restrict' : 'cascade') + : (fdef.deleteBehavior || 'set_null'); + } + +/** The body of `ObjectQL#referenceProbeFilterOfSet`; the method's docblock there is its contract. */ +export function referenceProbeFilterOfSet(this: Pick, fieldName: string, fdef: { type?: unknown; multiple?: unknown }, ids: ReadonlyArray): Record { + if (ids.length === 1) return this.referenceProbeFilter(fieldName, fdef, ids[0]); + if (!declaredMultiValued(fdef)) return { [fieldName]: { $in: [...ids] } }; + return { + $or: ids.flatMap((id) => { + const limb = this.referenceProbeFilter(fieldName, fdef, id); + return Array.isArray(limb.$or) ? (limb.$or as Array>) : [limb]; + }), + }; + } + +/** The body of `ObjectQL#isTenantColumnSetNull`; the method's docblock there is its contract. */ +export function isTenantColumnSetNull(object: string, child: ServiceObject, fieldName: string, behavior: string | undefined): boolean { + return ( + object === ORGANIZATION_OBJECT && + behavior === 'set_null' && + resolveTenantFieldName(child) === fieldName + ); + } + +/** The body of `ObjectQL#organizationCountIsDisclosable`; the method's docblock there is its contract. */ +export async function organizationCountIsDisclosable(this: Pick, childName: string, where: Record, count: number, context: ExecutionContext | undefined): Promise { + try { + return (await this.count(childName, { where, context })) === count; + } catch (error) { + // The refusal stands either way; only the number is withheld. + this.logger.warn('[cascade-delete] disclosure count failed; the refusal names the table without a count', { + object: childName, error: (error as Error)?.message ?? String(error), + }); + return false; + } + } diff --git a/packages/objectql/src/engine-defaults-and-autonumbers.ts b/packages/objectql/src/engine-defaults-and-autonumbers.ts new file mode 100644 index 00000000000..f85bb7a9459 --- /dev/null +++ b/packages/objectql/src/engine-defaults-and-autonumbers.ts @@ -0,0 +1,375 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Field defaults and autonumbers" section of `ObjectQL`: + * the bodies of `resolveNowDefault`, `resolveOptionDefault`, + * `applyFieldDefaults`, `defaultFieldEntries`, `applyAutonumbers`, + * `adoptExplicitAutonumber`, `isIssuedAutonumberCollision`, + * `warnAutonumberReadersUnserved`, `seedAutonumber` moved out of `engine.ts` + * whole by `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link DefaultsAndAutonumbersHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { isCurrentUserDefaultToken, isNowDefaultToken, missingFieldValues, parseAutonumberFormat, renderAutonumber, resolveAutonumberFormat } from '@objectstack/spec/data'; +import { isUniqueViolationError, keysetWalk, uniqueViolationColumn } from '@objectstack/types'; +import type { KeysetPageQuery } from '@objectstack/types'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { isMissingTableError } from '@objectstack/metadata/errors'; +import { ExpressionEngine } from '@objectstack/formula'; +import type { EvalPermissions } from '@objectstack/formula'; +import { + AUTONUMBER_SEED_PAGE_SIZE, declaredMultiValued, isExpressionDefault, readStoredAutonumberCounter, +} from './engine-top-level.js'; +import type { FormulaPlanEntry, IssuedAutonumber } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface DefaultsAndAutonumbersHost { + adoptExplicitAutonumber: ObjectQL['adoptExplicitAutonumber']; + readonly autonumberCounters: ObjectQL['autonumberCounters']; + readonly autonumberReadersUnservedReported: ObjectQL['autonumberReadersUnservedReported']; + defaultFieldEntries: ObjectQL['defaultFieldEntries']; + find: ObjectQL['find']; + getSchema: ObjectQL['getSchema']; + readonly logger: ObjectQL['logger']; + normalizeExpressionDefault: ObjectQL['normalizeExpressionDefault']; + resolveNowDefault: ObjectQL['resolveNowDefault']; + resolveOptionDefault: ObjectQL['resolveOptionDefault']; + seedAutonumber: ObjectQL['seedAutonumber']; +} + +/** The body of `ObjectQL#resolveNowDefault`; the method's docblock there is its contract. */ +export function resolveNowDefault(fieldType: unknown, instant: Date): string { + const iso = instant.toISOString(); // YYYY-MM-DDTHH:MM:SS.sssZ + if (fieldType === 'date') return iso.slice(0, 10); + if (fieldType === 'time') { + const timeOfDay = iso.slice(11, 23); // HH:MM:SS.fff + // Trim a zero-millisecond `.000` so a defaulted row is byte-canonical and + // still matches an equality filter against `'HH:MM:SS'` — the same trim + // the SQL driver's time DEFAULT and `canonicalTimeOfDay` apply. + return timeOfDay.endsWith('.000') ? timeOfDay.slice(0, 8) : timeOfDay; + } + return iso; + } + +/** The body of `ObjectQL#resolveOptionDefault`; the method's docblock there is its contract. */ +export function resolveOptionDefault(field: { type?: unknown; options?: unknown; multiple?: unknown }): unknown { + const options = field.options; + if (!Array.isArray(options)) return undefined; + const marked: unknown[] = []; + for (const o of options) { + if (!o || typeof o !== 'object') continue; + if ((o as { default?: unknown }).default !== true) continue; + const value = (o as { value?: unknown }).value; + // An option marked default but carrying no value declares nothing this + // method can apply — skipped rather than stored as `null`, which would + // be indistinguishable from "no default" downstream. + if (value === undefined || value === null) continue; + marked.push(value); + } + if (marked.length === 0) return undefined; + // [#18408] Was `field.multiple === true`. See {@link declaredMultiValued}. + return declaredMultiValued(field) ? marked : marked[0]; + } + +/** The body of `ObjectQL#applyFieldDefaults`; the method's docblock there is its contract. */ +export function applyFieldDefaults(this: Pick, object: string, record: Record, execCtx?: ExecutionContext, nowSnapshot?: Date, permissions?: EvalPermissions): Record { + const fieldEntries = this.defaultFieldEntries(object); + if (!fieldEntries) return record; + const out = { ...record }; + const now = nowSnapshot ?? new Date(); + for (const f of fieldEntries) { + // Apply the default when the field is either OMITTED (`undefined`) or an + // EXPLICIT `null` — both mean "no value supplied" on insert (#2706). A + // real value (including `''`) is respected. Insert-only path, so an + // intentional "set to null" on update is never touched here. + if (out[f.name] != null) continue; + if (f.defaultValue == null) { + // No `defaultValue` — fall back to the option marked `default: true`. + // + // Reached ONLY through this branch, which is the whole point of putting + // it here (#7246): an option's `value` is a plain literal by + // construction (`SystemIdentifierSchema`), never one of the runtime + // TOKENS below, so it must not be handed to `isCurrentUserDefaultToken` + // / `isNowDefaultToken`. An option literally spelled `current_user` + // stores those twelve characters — it is a picklist value, not an + // instruction — and that is pinned by test. + // + // The `dv == null` gate above is the presence test that decides + // precedence: `defaultValue: ''` is a REAL default (`'' == null` is + // false), so it wins and this fallback never fires for it, exactly as + // `''` supplied by a caller is respected as a real value. + const fromOption = this.resolveOptionDefault(f); + if (fromOption !== undefined) out[f.name] = fromOption; + continue; + } + const dv = f.defaultValue; + if (isExpressionDefault(dv)) { + const result = ExpressionEngine.evaluate(dv, { + now, + timezone: execCtx?.timezone, + user: execCtx?.userId ? { id: String(execCtx.userId), positions: execCtx?.positions ?? [] } : undefined, + org: execCtx?.tenantId ? { id: String(execCtx.tenantId) } : undefined, + permissions, + record: out, + extra: { object }, + }); + if (result.ok) { + // Normalized to the declared type's stored shape, never assigned + // verbatim: the temporal stdlib returns a `Date` and the contract + // names an ISO-8601 string (#7373 — {@link normalizeExpressionDefault}). + out[f.name] = this.normalizeExpressionDefault(f.type, result.value); + } else { + this.logger.warn('Failed to evaluate default expression', { + object, field: f.name, error: result.error, + }); + } + } else if (isCurrentUserDefaultToken(dv)) { + // `current_user` token → the acting user's id at insert time. Declarative + // counterpart to writing a beforeInsert hook; mirrors the 'NOW()' string + // convention and is resolved app-side per request (driver-agnostic), so + // `Field.user({ defaultValue: 'current_user' })` auto-fills the actor. + // When there is no authenticated user (system/anonymous), leave it unset + // and let required-validation decide — never stamp a bogus owner. + // + // The token spelling comes from `@objectstack/spec/data` + // (`DEFAULT_VALUE_TOKENS`), the one place the family is declared, so a + // driver's DDL reads the SAME set when deciding which `defaultValue`s + // may become a physical column DEFAULT. When the two sides disagreed, + // SQL emitted `DEFAULT 'current_user'` and the DATABASE overrode the + // "leave it unset" decision below with a literal non-id (#4560). + if (execCtx?.userId != null) out[f.name] = String(execCtx.userId); + } else if (isNowDefaultToken(dv)) { + // `NOW()` token → the insert-time clock, in the storage shape the + // field's declared type calls for ({@link resolveNowDefault}). + // + // The mirror of the `current_user` crack above (#4597 / #4560): this + // token was understood by the SQL driver and NOT by the engine, so + // `out[f.name] = dv` sent the literal string `'NOW()'` to every + // driver. SQL hid it — `formatInput`'s safety net swapped in a real + // timestamp before the wire — while memory/mongodb have no such net, + // so the same declaration behaved differently per datasource. That + // split surfaced two ways: a validation-visible field was REJECTED by + // the engine's own write validator ("must be a valid datetime"), and a + // `readonly`/`system` field — which `validateRecord` then skipped, i.e. + // the ~100 `created_at`/`updated_at` platform declarations — silently + // stored the four characters `NOW()`. (Since #21663 a readonly value's + // shape is judged too, so that literal would now be refused.) + // + // Resolved from the caller's `nowSnapshot`, so every defaulted field + // in one insert (and every row of one batch) carries the SAME instant. + // + // The driver's own now-handling is unchanged and stays as defence in + // depth: `SqlDriver.formatInput`'s safety net (now unreachable from + // this path) and the native column DEFAULT, which still serves writes + // that bypass the engine entirely — the same division of labour + // `current_user` has. + out[f.name] = this.resolveNowDefault(f.type, now); + } else { + out[f.name] = dv; + } + } + return out; + } + +/** The body of `ObjectQL#defaultFieldEntries`; the method's docblock there is its contract. */ +export function defaultFieldEntries(this: Pick, object: string): Array<{ + name: string; type?: unknown; defaultValue?: unknown; options?: unknown; multiple?: unknown; + }> | undefined { + const fieldsRaw = (this.getSchema(object) as any)?.fields; + if (!fieldsRaw || typeof fieldsRaw !== 'object') return undefined; + // `fields` may be a Record (canonical) or an array (legacy). + return Array.isArray(fieldsRaw) + ? fieldsRaw + : Object.entries(fieldsRaw).map(([name, def]) => ({ name, ...(def as object) })); + } + +/** The body of `ObjectQL#applyAutonumbers`; the method's docblock there is its contract. */ +export async function applyAutonumbers(this: Pick, object: string, record: Record, execCtx?: ExecutionContext, driverOwnsAutonumber?: boolean): Promise { + if (driverOwnsAutonumber) return []; // driver generates persistently in create() + const fields = (this.getSchema(object) as any)?.fields; + if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return []; + const now = new Date(); + const timezone = execCtx?.timezone; + const issued: IssuedAutonumber[] = []; + for (const [name, def] of Object.entries(fields)) { + if ((def as any)?.type !== 'autonumber') continue; + // The contract answers "which format?" — canonical `autonumberFormat`, + // then the `format` shorthand (#1603), then the declared default + // `{0000}`. One resolver, shared with the SQL driver, so a format-less + // field cannot render two shapes again (#6555; see the docstring above). + // `fmt` is kept only for the diagnostic below, which now names the format + // actually rendered with rather than `undefined`. + const fmt = resolveAutonumberFormat(def as never); + const tokens = parseAutonumberFormat(fmt); + const current = record[name]; + // Respect an explicit value — reachable only for an EXEMPT writer now + // (isSystem / preserveAudit / a hook stamp): #5503's strip removed every + // other caller's value before this method was called. Respecting it is + // unchanged; what is new is that the counter LEARNS from it (#6806). + if (current != null && current !== '') { + this.adoptExplicitAutonumber(object, name, tokens, record, String(current), now, timezone); + continue; + } + // Refuse to generate when an interpolated `{field}` is empty — it would + // render to an empty prefix and merge this record into the wrong counter + // scope. Mirror the SQL driver so both paths fail identically (#1603). + const missing = missingFieldValues(tokens, record); + if (missing.length > 0) { + throw new Error( + `Cannot generate autonumber "${object}.${name}" (format "${fmt}"): ` + + `referenced field(s) [${missing.join(', ')}] are empty on the record. ` + + `Fields interpolated into an autonumber format must be set before the record is created.`, + ); + } + // The counter scope is the rendered prefix (date/field tokens before the + // sequence slot); it is independent of the counter value, so a throwaway + // render with seq 0 yields the scope and the literal prefix to seed from. + const probe = renderAutonumber({ tokens, seq: 0, record, now, timezone }); + const counterKey = `${object}.${name}.${probe.scope}`; + let next = this.autonumberCounters.get(counterKey); + if (next == null) next = await this.seedAutonumber(object, name, probe.prefix, probe.suffix, execCtx); + next += 1; + this.autonumberCounters.set(counterKey, next); + record[name] = renderAutonumber({ tokens, seq: next, record, now, timezone }).value; + issued.push({ field: name, counterKey }); + } + return issued; + } + +/** The body of `ObjectQL#adoptExplicitAutonumber`; the method's docblock there is its contract. */ +export function adoptExplicitAutonumber(this: Pick, object: string, field: string, tokens: ReturnType, record: Record, value: string, now: Date, timezone?: string): void { + let probe: ReturnType; + try { + // An empty `{field}` would render an empty prefix and point at the wrong + // counter — the same hazard the generating branch throws on. Here it is a + // reason to learn nothing, never a reason to fail the caller's write. + if (missingFieldValues(tokens, record).length > 0) return; + probe = renderAutonumber({ tokens, seq: 0, record, now, timezone }); + } catch { + return; + } + const counterKey = `${object}.${field}.${probe.scope}`; + const seeded = this.autonumberCounters.get(counterKey); + if (seeded == null) return; // not seeded yet — the first seed scan will read this row + const supplied = readStoredAutonumberCounter(value, probe.prefix, probe.suffix); + if (supplied == null || supplied <= seeded) return; + this.autonumberCounters.set(counterKey, supplied); + this.logger.debug('Autonumber counter lifted to an externally supplied value', { + object, field, counterKey, from: seeded, to: supplied, + }); + } + +/** The body of `ObjectQL#isIssuedAutonumberCollision`; the method's docblock there is its contract. */ +export function isIssuedAutonumberCollision(error: unknown, issued: IssuedAutonumber[]): boolean { + if (issued.length === 0) return false; + if (!isUniqueViolationError(error)) return false; + const column = uniqueViolationColumn(error); + if (column === undefined) return true; + return issued.some((one) => one.field === column); + } + +/** The body of `ObjectQL#warnAutonumberReadersUnserved`; the method's docblock there is its contract. */ +export function warnAutonumberReadersUnserved(this: Pick, object: string, plan: readonly FormulaPlanEntry[], driver: unknown): void { + if (this.autonumberReadersUnservedReported.has(object)) return; + this.autonumberReadersUnservedReported.add(object); + const fields = plan.map((entry) => entry.name); + const driverName = (driver as { name?: unknown } | null)?.name; + this.logger.error( + `Stored field(s) [${fields.join(', ')}] on '${object}' read a record number that driver ` + + `'${String(driverName ?? 'unknown')}' issues only inside create(), so every insert stores them computed ` + + 'WITHOUT the number while the write reports success; an update by id recomputes them later. ' + + 'Use a driver that implements IDataDriver.issueAutonumbers (driver-sql and the drivers built on it do), ' + + 'or keep the record number out of the stored expression. Reported once per object.', + undefined, + { object, fields, driver: driverName }, + ); + } + +/** The body of `ObjectQL#seedAutonumber`; the method's docblock there is its contract. */ +export async function seedAutonumber(this: Pick, object: string, field: string, prefix: string, suffix: string, execCtx?: ExecutionContext): Promise { + try { + // Canonical `fields`, not the wire spelling `select` — this call sat on + // the exact #4371 silent drop (the projection never applied; the scan + // worked only because an unprojected row still carries `field`), and the + // catch below would have swallowed the guard's rejection into "seed + // from 0", i.e. duplicate autonumbers. + const walk = keysetWalk>( + (q: KeysetPageQuery) => this.find(object, { + ...q, + fields: ['id', field], + context: execCtx, + } as any), + { + where: prefix ? { [field]: { $startsWith: prefix } } : undefined, + pageSize: AUTONUMBER_SEED_PAGE_SIZE, + }, + ); + let max = 0; + for await (const page of walk.pages()) { + for (const r of page) { + const v = r?.[field]; + if (v == null) continue; + // The reading itself lives in `readStoredAutonumberCounter` (the + // section above describes it) because #6806's adopt-on-exempt-write + // resync must read a supplied value by the SAME rules this scan reads + // a stored one — two copies of it would drift into two different + // answers for one row, which is a duplicate record number. Its + // anchored half is spec's `readAutonumberCounter`, the one the SQL + // driver's own scan calls (#6560). + const counter = readStoredAutonumberCounter(String(v), prefix, suffix); + if (counter != null) max = Math.max(max, counter); + } + } + // The walk is unbounded (no `max`), so truncation here means the scan + // could not COMPLETE: a row carried no `id` to seek past, or the reader + // never applied the seek predicate. Either way rows were left unread, and + // the max over what was read is a floor, not the max. Answering with it + // is the "seed below the real MAX" defect this method was fixed for, so + // it fails loudly instead — the same disposition #6114 gave the read + // outage: allocate nothing, write nothing. + if (walk.truncated) { + throw new Error( + `Cannot seed the autonumber counter for "${object}.${field}": the seeding scan ` + + `could not visit every stored row (it stopped after ${walk.scanned} rows without ` + + `reaching the end). Seeding from a partial scan would issue record numbers that ` + + `collide with existing ones, so no number was allocated.`, + ); + } + return max; + } catch (error) { + // [#5979] Discriminate by error TYPE. Seeding from 0 is the truth for + // exactly ONE failure reason — the table has not been provisioned, so + // there are genuinely no rows and number 1 collides with nothing. + if (isMissingTableError(error, object)) return 0; + // Every other failure (connection drop, timeout, permission denial, + // query error) means the rows may well exist and simply were not seen. + // Answering 0 there restarts the sequence at 1 against a table already + // holding N rows and issues autonumbers that COLLIDE with existing ones + // — a value written wrong, which no retry and no restart repairs. So the + // read failure propagates and the caller allocates nothing: the write + // fails loudly instead of succeeding with a forged business identifier. + // This is the hazard the #4371 comment above the read already named. + throw error; + } + } diff --git a/packages/objectql/src/engine-delete-and-aggregate.ts b/packages/objectql/src/engine-delete-and-aggregate.ts new file mode 100644 index 00000000000..bc3f1ffe040 --- /dev/null +++ b/packages/objectql/src/engine-delete-and-aggregate.ts @@ -0,0 +1,557 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Delete, count, aggregate and execute" section of + * `ObjectQL`: the bodies of `aggregate` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link DeleteAndAggregateHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { normalizeFilterComparandTypes } from '@objectstack/spec/data'; +import type { EngineAggregateOptions, QueryAST } from '@objectstack/spec/data'; +import { assertFilterIsMaterializable, assertListComparandShapes, invalidFilterError } from './filter-comparand-shape.js'; +import { assertHavingTemporalComparandsInterpretable, assertTemporalComparandsInterpretable } from './temporal-comparand-door.js'; +import { assertTextOperatorTargetsAreStringCapable } from './text-operator-declared-type-door.js'; +import { narrowHavingNumberComparands, narrowNumberComparands } from './number-comparand-declared-type-door.js'; +import { rejectCredentialEvaluation } from './credential-evaluation-guard.js'; +import { rejectInternalFieldEvaluation } from './internal-field-evaluation-guard.js'; +import { assertGroupByNamesNoJsonStoredField } from './group-by-structured-json-door.js'; +import { assertAggregationFieldTypesAccepted } from './aggregate-field-type-door.js'; +import { assertRecordCurrencyAggregatesGrouped } from './record-currency-aggregate-door.js'; +import { assertAggregatedColumnsAreDistinct, assertDateBucketTargetsAreTemporal } from './aggregate-query-shape-door.js'; +import { applyInMemoryAggregation } from './in-memory-aggregation.js'; +import { aggregatedRowColumnClasses, aggregatedRowColumnTypes, aggregatedRowColumns, applyHaving, assertAggregationFilterIsEvaluable, assertHavingIsEvaluable, assertHavingIsFilterCondition } from './having-filter.js'; +import { + ENGINE_AGGREGATE_OPTION_KEYS, ENGINE_WHERE_SLOTS, declaredDatetimeLowering, describeNonFilterWhere, + foldEngineOptionAliases, isWhereFilterObject, lowerWhereFilterArray, mergeReadContext, rejectUnknownEngineOptions, + resolveThenLowerWhere, whereResolvedTokenJudge, +} from './engine-top-level.js'; +import type { EngineReadOptions, OperationContext } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface DeleteAndAggregateHost { + readonly _registry: ObjectQL['_registry']; + buildDriverOptions: ObjectQL['buildDriverOptions']; + composeReadableIdsAtBoundary: ObjectQL['composeReadableIdsAtBoundary']; + executeWithMiddleware: ObjectQL['executeWithMiddleware']; + expandSearchOnOptions: ObjectQL['expandSearchOnOptions']; + getDriver: ObjectQL['getDriver']; + readonly logger: ObjectQL['logger']; + rejectCredentialAggregation: ObjectQL['rejectCredentialAggregation']; + readonly relatedSchemaOf: ObjectQL['relatedSchemaOf']; + resolveObjectName: ObjectQL['resolveObjectName']; + resolveWhereTokens: ObjectQL['resolveWhereTokens']; +} + +/** The body of `ObjectQL#aggregate`; the method's docblock there is its contract. */ +export async function aggregate(this: Pick, object: string, query: EngineAggregateOptions, options?: EngineReadOptions): Promise { + object = this.resolveObjectName(object); + // Fold the `filter` alias into `where` (#4346) — the AST below reads + // `query.where` only, so an unfolded `{ filter }` aggregated every row. + query = foldEngineOptionAliases(object, 'aggregate', query, ENGINE_WHERE_SLOTS); + rejectUnknownEngineOptions(object, 'aggregate', query, ENGINE_AGGREGATE_OPTION_KEYS); + query = lowerWhereFilterArray(object, 'aggregate', query, this._registry.getObject(object), this.relatedSchemaOf); + // [ADR-0100] The filter half of the credential-column guard — `where` + // and every `aggregations[i].filter` — before the search term joins + // `where`; the aggregate half (`rejectCredentialAggregation`) follows. + // [internal: true] The flagged-column refusal goes first, over every + // position at once (`where`, `groupBy`, each aggregation's `field` and + // `filter`), so a non-system caller meets the data door's words here + // too rather than the aggregate guard's. + rejectInternalFieldEvaluation(object, query, mergeReadContext(query?.context, options?.context), this.relatedSchemaOf); + rejectCredentialEvaluation(object, 'aggregate', query, mergeReadContext(query?.context, options?.context), this.relatedSchemaOf); + // ADR-0061 `search` → the rows are searched BEFORE they are grouped, by + // the one expander `find` runs, at the same point in the sequence (after + // the `where` doors above, before the AST is built and tokens resolve). + query = await this.expandSearchOnOptions(object, query, options); + this.rejectCredentialAggregation(object, query); + // [#20783] …and a `groupBy` entry naming a structured-JSON field (`json`, + // `composite`, `address`, …) is refused `INVALID_FIELD` / 400 here, before + // any driver is asked: the drivers share no meaning for a JSON document as + // a group key (memory merged every row into one group, SQLite grouped each + // serialized document apart, PostgreSQL answered 500). After the + // credential refusal, which reads the same entries, so a protected field + // keeps that refusal's words. [#20808] The same door refuses a `groupBy` + // entry naming a MULTI-VALUE field (`isMultiValueField`: a list stored in + // a JSON column, split the same three ways), judged in one walk so the + // first offending position is the one named. + assertGroupByNamesNoJsonStoredField(object, this._registry.getObject(object), query.groupBy); + // [#20808] …and every aggregation's (function, declared field type) + // pair, asked of the spec table (`isAggregateCompatibleWithFieldType`) + // beside `isMultiValueField`, before any driver is asked. [#20808] took + // the `count_distinct` row (memory counted equal documents apart, SQLite + // compared serialized text, PostgreSQL answered 500); [#20914] took the + // other rows — `max` over a `json` field answered a document in memory, a + // string on SQLite and a 500 on PostgreSQL; `sum` over one answered `0`, + // `0` and a 500. Every row of the table is asked (see the door's header). + assertAggregationFieldTypesAccepted(object, this._registry.getObject(object), query.aggregations); + // [ADR-0144 D6] …and a `sum` / `avg` / `min` / `max` over a `record`-mode + // amount that does not group by its code field: the rows hold amounts in + // different currencies, which add into one number under one label. + // After the type door, so a type it refuses keeps that door's words. + assertRecordCurrencyAggregatesGrouped(object, this._registry.getObject(object), query.groupBy, query.aggregations); + // [#25818] …and the two shapes that answered a plausible wrong table: a + // `dateGranularity` over a field declared neither a date nor a datetime + // (every row in one null bucket), and two columns of the aggregated row + // under one name (the last value written won). After the two doors + // above, so a JSON field under a date bucket keeps their words. + assertDateBucketTargetsAreTemporal(object, this._registry.getObject(object), query.groupBy); + assertAggregatedColumnsAreDistinct(object, query.groupBy, query.aggregations); + // [#10576] The per-aggregation `filter` (`AggregationNodeSchema.filter`, + // the contract half of #10413) is a second filter position on this verb, + // so it walks through the same refusal doors `where` does at this seam: + // the comparand-shape gate (#5869 — a scalar `$in` etc. is refused, not + // silently matched against nothing) and the materializable-field gate + // (#8296 — a typo'd column would otherwise select ZERO rows for that one + // aggregation, silently, which is the same wrong-number shape #10413 + // measured). The path names which aggregation carries the offending key. + { + const aggregations = Array.isArray(query.aggregations) ? query.aggregations : []; + let typedAggregations: typeof aggregations | undefined; + for (const [i, agg] of aggregations.entries()) { + const aggFilter = (agg as { filter?: unknown })?.filter; + if (aggFilter == null) continue; + // [#20122] The SHAPE gate `where` takes first on its own seam + // (`lowerWhereFilterArray`, #20121) — first here too, with its + // object test (`isWhereFilterObject`) and its words, minus the + // array half (below). A filter that is not a filter object + // (a string, a number, a boolean, `''`, a `Map`, a `Date`) was + // stepped around by every door below, each of which walks a filter + // node's keys, and then dropped: the fork below reads it as "no + // filter", so the aggregation read every row of its group + // (`driver-sql`'s native aggregate answered a non-empty string + // with a 501 instead). The wire door already refuses these shapes + // through `AggregationNodeSchema`; this closes the in-process one. + // + // An ARRAY is refused too, `[]` included — unlike `where`, which + // lowers the condition-array sugar in its array branch. The slot + // is declared `FilterConditionSchema` (an object, no array form), + // the wire door refuses every array here (`VALIDATION_FAILED`), + // and in-process a condition array counted no row (the walker + // read its index positions as column names) while `[]` read as + // no filter. Same ruling as `having`'s condition check (#20099). + if (Array.isArray(aggFilter)) { + let shown: string; + try { + shown = JSON.stringify(aggFilter) ?? String(aggFilter); + } catch { + shown = String(aggFilter); + } + if (shown.length > 80) shown = `${shown.slice(0, 77)}...`; + throw invalidFilterError( + `aggregate('${object}'): 'aggregations[${i}].filter' must be a filter object, received ` + + `an array (${shown}). The condition-array form — [field, operator, value] tuples and ` + + `["and", …] groups — is input-only sugar lowered on 'where' alone; a per-aggregation ` + + `filter is declared as a filter condition object and does not take it, so an array here ` + + `was never applied as the filter it spells. Write the object form, ` + + `{ "FIELD": { "$gt": 100 } }, or omit 'filter' for no filter.`, + ); + } + if (!isWhereFilterObject(aggFilter)) { + throw invalidFilterError( + `aggregate('${object}'): 'aggregations[${i}].filter' must be a filter object, ` + + `received ${describeNonFilterWhere(aggFilter)}. It was not applied, and an ` + + `unapplied filter would have aggregated every row of each group for that aggregation.`, + ); + } + assertListComparandShapes(object, 'aggregate', aggFilter, `aggregations[${i}].filter`); + assertFilterIsMaterializable(object, 'aggregate', this._registry.getObject(object), aggFilter); + // [#15661] …and the declared-type door for the text operators: a + // `$contains` over a numeric column in ONE aggregation's filter is + // the same silent zero at a second filter position, and a door that + // spoke on `where` alone would answer one mistake two ways within a + // single verb. [#20334] Rooted at this position, as the shape door + // above is: the refusal said `at where.amount.$contains`, a `where` + // the author did not write. + assertTextOperatorTargetsAreStringCapable( + object, 'aggregate', this._registry.getObject(object), aggFilter, `aggregations[${i}].filter`, + ); + // [#20148] …and the TEMPORAL-comparand door, fourth here as it is + // fourth on `where`'s seam (#8690) — the same function, against the + // object's declared fields, since this filter reads the object's + // raw rows. A comparand a declared date / datetime / time field + // cannot interpret (`'not-a-date'`, the preset name + // `'last_30_days'`) was compared as written: no row counted, a + // `$between` with one such endpoint counted EVERY row, and the same + // bound in a `where` was refused 400 — measured on the base on + // driver-memory and driver-sql, through the engine and REST. + // Before the token resolver below, as there, so a `{placeholder}` + // is stepped around and resolved (or refused) a moment later. + // [#20334] Rooted at this position too: the refusal said + // `at where.placed_on.$gt`. + assertTemporalComparandsInterpretable( + object, 'aggregate', this._registry.getObject(object), aggFilter, `aggregations[${i}].filter`, + ); + // [#20351] …and the NUMBER-comparand door, fifth here as it is + // fifth on `where`'s seam: a non-numeric string against a declared + // numeric field counted no row (every row under `$ne`) where its + // `where` twin was a 500 on PostgreSQL, and a numeric string is + // narrowed to its number, copy-on-write, before the in-memory + // evaluator compares it. Rooted at this position. [#20546] Its + // walk's no-operator-object arm too: `{ amount: { a: 1 } }` here + // counted no row, silently, on every driver. [#20745] So did + // `{ owner: { region: 'NA' } }` beneath a lookup; a JSON object + // here is refused alike, one answer per filter at every position. + // [#21333] …and its boolean arm: `"true"` against a declared + // boolean field counted no row (every row under `$ne`) on both + // drivers; it is narrowed to `true` here, `"yes"` refused. + const numeric = narrowNumberComparands( + object, 'aggregate', this._registry.getObject(object), aggFilter, `aggregations[${i}].filter`, + ); + // [#20122] …and the two doors `having` took at its own entry + // (#20099), so a refusal here is the FILTER's, never the data's: + // 1. the comparand-TYPE door `where` takes in + // `lowerWhereFilterArray`, rooted at this position — a plain + // object, a `Map` or a function comparand was compared as it + // stood and counted no row (every row under `$ne`), and a + // `Symbol` under an ordering operator threw an uncoded + // `TypeError`, on a populated table only. An exact-range + // bigint is narrowed copy-on-write, as it is there; + // 2. the walker's own refusals, judged once + // (`assertAggregationFilterIsEvaluable`). The fallback below + // walks the filter per SOURCE row, so `{ amount: { $median: 1 } }` + // was a 400 on a populated table and a `200 []` on an empty one, + // and a `$or` whose first branch held counted every row. + // [#20148] …then, against the object's DECLARED fields, the two + // cross-field rules `where` gets from `driver-sql`'s compiler: + // a `{ $field }` names a declared field, and an `addDays` pair + // follows `FieldReferenceSchema.addDays`' class rule. Both were + // answered by the fallback — no row counted — where `where` is + // refused. Refused in `where`'s words for that comparison, which + // withhold the fields, the operator and the reason; the + // withheld half goes to this log, as the driver writes its own. + // [#21007] …and, last, `where`'s JSON-column rule: a scalar + // comparison (`$in`, `$nin`, `$eq`, an ordering, implicit + // equality, …) on a field declared JSON-stored is refused in + // `where`'s words — counted by the fallback, `$nin` counted the + // very rows it was asked to exclude. Same withholding, same log. + const typed = normalizeFilterComparandTypes(numeric, `aggregate('${object}')`, `aggregations[${i}].filter`); + assertAggregationFilterIsEvaluable(typed, i, { + object, + fields: (this._registry.getObject(object) as { fields?: unknown } | undefined)?.fields, + reportWithheld: (diagnostic) => this.logger.warn( + `aggregate('${object}'): INVALID_FILTER — refusal detail withheld from the response, as it ` + + `is for the same refusal in a where. Full diagnostic: ${diagnostic}`, + ), + }); + if (typed !== aggFilter) { + typedAggregations ??= [...aggregations]; + typedAggregations[i] = { ...(agg as object), filter: typed } as typeof agg; + } + } + if (typedAggregations) query = { ...query, aggregations: typedAggregations }; + } + // [#19974] `having` is this verb's THIRD filter position, and it walks + // through the same comparand-shape face the other two take above — + // called once, here, with the path seeded at `having`. The 2026-09-23 + // ruling on #19757 refuses an array in the equality slot at that face + // "for every driver at once", and the face refuses it for `where` on + // every driver. `having` never reaches a driver, though: the engine + // evaluates it itself (`applyHaving`, having-filter.ts), and that walker + // ANSWERED every shape the face refuses — `{ total: [5] }` by JS `==` + // coercion (`5 == [5]` is true), `{ total: { $eq: [5] } }` by `!=`, a + // scalar `$nin` or a malformed `$between` by keeping every group, a null + // `$lt` bound by keeping none. Measured on the base through this method, + // on driver-memory and driver-sqlite-wasm, on both doors below. + // + // ONE call covers BOTH `applyHaving` doors — the native + // `driver.aggregate()` path and the in-memory fallback — because it runs + // before either is chosen and before any driver is asked for a row. The + // shape of a comparand is a property of the FILTER, so it is judged once + // per query, never per aggregated row: an empty grouped set refuses the + // same `having` a populated one does. ⛔ Not a second face and not a + // walker-local check in having-filter.ts: the verdicts, the wording and + // the `INVALID_FILTER` / 400 envelope are the face's own, so a `having` + // refusal reads byte for byte as the `where` refusal of the same shape, + // path aside — and whatever arm the face gains next, `having` gains too. + // + // [#20099] …and the rest of the doors `where` takes, at the same seam, so + // the whole clause is judged once, before any row exists. In order: + // 1. `having` is a filter condition at all — `QuerySchema.having` and + // `EngineAggregateOptions.having` declare `FilterConditionSchema`, + // which refuses an array or a scalar. The walker read an array's index + // keys as columns (`[['total','>',100]]` kept no group) and a scalar + // as no condition (every group). ⛔ Not lowered: the `FilterArray` + // sugar is declared on `where` alone, and every later door steps + // silently around a non-node, so this runs first. + // 2. the comparand-shape face (above); + // 3. the comparand-TYPE door — the same `normalizeFilterComparandTypes` + // `lowerWhereFilterArray` runs on `where`, path rooted at `having`, so + // `{ total: { $eq: { v: 1 } } }` is the `where` refusal of that shape + // rather than a silent `[]`, and an exact-range bigint is narrowed + // copy-on-write exactly as it is there; + // 4. the walker's own refusals plus the `{ $field }` reference's + // position and name, judged against the aggregated row's column set + // read off THIS query (`assertHavingIsEvaluable`). The walker raised + // them per aggregated row, so an empty grouped set answered `200 []` + // for a `having` a populated one refused. [#20123] The same column + // set judges every KEY too: a key naming no column read "no value" + // in every group, so a typo for an alias kept no group (every group + // under a negation) with no error. + // A reference that passes is then RESOLVED against each aggregated row + // (having-filter.ts `compareWithReference`) on both doors below. + assertHavingIsFilterCondition(query.having); + assertListComparandShapes(object, 'aggregate', query.having, 'having'); + // [#20176] The object's declaration, read once: the per-aggregation + // `filter` reads a temporal comparand by its column's storage rule + // (`applyInMemoryAggregation` below), and `having` by its aggregated + // column's — the rule the drivers apply to the same comparand in a + // `where`. Before, both positions compared it as written, so an ISO + // instant against a `date` field counted 1 row where the `where` twin + // counted 3, over REST, on driver-memory and driver-sql alike. + const declaredFields = (this._registry.getObject(object) as { fields?: Record } | undefined)?.fields; + // [#20127] …and each aggregated column's class, read off the query and the + // object's declaration, so a `{ $field, addDays }` pair is judged by the + // rule `FieldReferenceSchema.addDays` declares (two temporal columns of + // one class) rather than answered by epoch-ms coercion. [#20176] Kept for + // both `applyHaving` doors below, which read a temporal column's + // comparands by its storage rule. + const havingColumnClasses = aggregatedRowColumnClasses(query.groupBy, query.aggregations, declaredFields); + { + const having = normalizeFilterComparandTypes(query.having, `aggregate('${object}')`, 'having'); + assertHavingIsEvaluable( + having, + aggregatedRowColumns(query.groupBy, query.aggregations), + havingColumnClasses, + // [#21255] …and each column's TYPE, which a plain `{ $field }` + // reference's class rule asks the spec's verdict of — the rule + // `where` applies to every reference, not only an `addDays` pair. + aggregatedRowColumnTypes(query.groupBy, query.aggregations, declaredFields), + ); + // [#20263] …and last, the TEMPORAL-comparand door `where` (#8690) and + // the per-aggregation `filter` (#20148) take: the same walk and the + // same `@objectstack/core` predicate, with each column's kind read + // from the class #20127 derived above (`min` / `max` of a temporal + // field keeps its kind, a `day` bucket is a date, `count` / `sum` / + // `avg` are not temporal). A comparand the column's storage rule + // cannot read (`'not-a-date'`, a `date` year outside 0..9999) was + // compared as written and kept no group or every group, on both + // `applyHaving` doors, while its `where` twin answered 400. After + // every other `having` door, so a clause one of them refuses keeps + // that refusal's words; on the caller's own clause, before the + // bigint narrowing, which is what `where`'s object form judges. + assertHavingTemporalComparandsInterpretable(object, query.having, havingColumnClasses, query); + // [#20351] …then the NUMBER-comparand door `where` and the + // per-aggregation `filter` take, over the columns #20127 classes + // `numeric`: a non-numeric string kept no group (every group under + // `$ne`) on both `applyHaving` doors, and a numeric string is narrowed + // to its number. On the bigint-narrowed clause, so the two + // narrowings compose. + // [#20546] The same walk's no-operator-object arm judges every column + // whose TYPE holds scalar values (hence the types, beside the + // classes): `{ total: { a: 1 } }` kept no group, silently, on every + // driver, where its `where` twin answered two ways. [#20745] A + // relation or JSON column's type is judged too (a lookup groupBy's + // nested-relation `having` kept no group on every driver). + // [#21333] …and, by the same types, the boolean arm: over a groupBy + // of a boolean field `"true"` kept no group (`$ne "true"` every + // group) on both drivers; it is narrowed to `true`, `"yes"` refused. + const numeric = narrowHavingNumberComparands( + object, having, havingColumnClasses, + aggregatedRowColumnTypes(query.groupBy, query.aggregations, declaredFields), + ); + if (numeric !== query.having) query = { ...query, having: numeric }; + } + const driver = this.getDriver(object); + this.logger.debug(`Aggregate on ${object} using ${driver.name}`, query); + + const opCtx: OperationContext = { + object, + operation: 'aggregate', + // On the opCtx so middleware-injected read filters (RLS/sharing) land + // in `ast.where` — same #2737 hole as count(): a locally-built AST + // aggregated over rows the caller may not see. + ast: { + object, + where: query.where, + groupBy: query.groupBy, + aggregations: query.aggregations, + // ENFORCED since #4286 (step 3). On the ast so the FLS predicate + // guard walks its references (predicate-guard.ts) and a future + // native pushdown has it in hand; the engine's post-aggregation + // applyHaving() below is authoritative either way. + having: query.having, + }, + options: query, + context: mergeReadContext(query?.context, options?.context), + }; + // [ADR-0053 D-D1, amended — #5930] Each of this verb's three filter + // positions resolves, then lowers, through `resolveThenLowerWhere`: `where` + // and `aggregations[i].filter` narrow the object's raw rows, so they read + // its declared field types; `having` narrows the aggregated row, so it + // reads each aggregated column's type (`min` / `max` of a `datetime` + // field is a `datetime`; a `count` is a number). + const rowLowering = declaredDatetimeLowering(this._registry.getObject(object)); + await this.resolveWhereTokens(opCtx.ast as QueryAST, opCtx.context, rowLowering, 'where', 'aggregate'); + // [#10576] Filter tokens (`{userId}`-style placeholders, #3810) resolve + // in per-aggregation filters exactly as they do in `where` — a filter + // position is a filter position, and an unresolved placeholder would be + // compared as a literal string, silently selecting zero rows for that one + // aggregation. Copy-on-write like `withResolvedWhere`: the entry objects + // belong to the caller (view metadata / flow config get reused), so a + // resolved filter lands on a copied entry, never written back through. + { + const astAggs = (opCtx.ast as QueryAST).aggregations; + if (Array.isArray(astAggs) && astAggs.some((a) => (a as { filter?: unknown })?.filter != null)) { + (opCtx.ast as QueryAST).aggregations = astAggs.map((a, i) => { + const f = (a as { filter?: unknown })?.filter; + if (f == null) return a; + const resolved = resolveThenLowerWhere( + f as any, opCtx.context, rowLowering, + whereResolvedTokenJudge(object, 'aggregate', this._registry.getObject(object), `aggregations[${i}].filter`), + ); + return resolved === f ? a : { ...(a as object), filter: resolved } as typeof a; + }); + } + } + // [#20334] …and in `having`, through the resolver `where` takes, so a + // relative-date token compares as the day it names rather than as its own + // text, and an unknown one is `FILTER_TOKEN_UNKNOWN` / 400 (it kept no + // group with a 200). Both `applyHaving` doors below read `ast.having`, so + // one call covers the native and the rows path, before any driver read. + // After every `having` door above, as `where`'s resolution follows its + // doors: the temporal door steps around a `{placeholder}` exactly as + // `where`'s does (so, as there, the resolved value is judged only for + // its year, by the resolution stage — #20844), and the other doors + // judged a string that resolves to a string. + { + const havingColumnTypes = aggregatedRowColumnTypes(query.groupBy, query.aggregations, declaredFields); + await this.resolveWhereTokens( + opCtx.ast as QueryAST, + opCtx.context, + { isDatetimeColumn: (column) => havingColumnTypes.get(column) === 'datetime' }, + 'having', + ); + } + + await this.executeWithMiddleware(opCtx, async () => { + // [ADR-0139 D2] The executor boundary, once for BOTH driver paths below + // (they are exclusive, and the rows path keeps this `where` by + // reference): the AST is final once the chain has run. + const ast = await this.composeReadableIdsAtBoundary(object, 'aggregate', opCtx.ast as QueryAST, opCtx.context); + + // Prefer driver.aggregate() when available — driver.find() in many + // drivers (e.g. driver-sql) does not honor `groupBy` / `aggregations` + // and would silently return ungrouped raw rows. Fall back to find() + // for drivers that handle aggregations through their query AST. + const drv = driver as any; + // Structured groupBy items ({field, dateGranularity}) require the + // driver to advertise per-granularity native bucket support via + // `supports.queryDateGranularity[g]`. If every structured item is + // supported we can push the aggregate down to the driver; otherwise + // we fall back to driver.find() + in-memory bucketing so the result + // remains correct on a face that buckets fewer (e.g. the Turso remote + // transport, which advertises none). + const groupByItems = Array.isArray(query.groupBy) ? query.groupBy : []; + const granularityCaps: Record | undefined = + drv?.supports?.queryDateGranularity; + const structuredItems = groupByItems.filter((g) => typeof g !== 'string'); + const allStructuredSupported = structuredItems.every((g) => { + if (!g?.dateGranularity) return true; // plain {field} object is fine + return granularityCaps?.[g.dateGranularity] === true; + }); + // ADR-0053 Phase 2 (D2): native driver date bucketing (driver-sql's + // `to_char` / `date_format` / `strftime`) is UTC-only — SQLite has no + // tz database and MySQL needs tz tables loaded, + // so pushing tz-aware bucketing down splits boundaries per dialect. When + // a non-UTC reference timezone is in play we therefore force the + // in-memory path: the date-range `where` still goes to the driver (only + // matching rows are fetched), but bucketing runs uniformly in JS so a + // row near a tz day-boundary lands identically on every driver. + const tz = query.timezone; + const hasDateBucket = structuredItems.some((g) => !!g?.dateGranularity); + const tzRequiresInMemory = !!tz && tz !== 'UTC' && hasDateBucket; + // [#10576] Per-aggregation filters (`aggregations[].filter`, the + // contract half of #10413) force the in-memory path: no driver compiles + // a conditional aggregate (SQL `FILTER (WHERE …)` / `CASE WHEN`) from + // this key today, and pushing it down would aggregate the UNFILTERED + // rows — the silent drop #10413 measured. Same correct-first two-tier + // shape as date bucketing and HAVING above: a driver that grows native + // support must advertise a capability flag, at which point this fork + // becomes its fallback tier. The driver faces also refuse the key + // loudly (NOT_IMPLEMENTED/501) if reached directly, so neither seam can + // drop it in silence. `{}` counts as no filter (the vacuous-filter + // convention `where` / `having` already follow). + const hasAggregationFilter = (Array.isArray(ast.aggregations) ? ast.aggregations : []) + .some((a) => { + const f = (a as { filter?: unknown })?.filter; + return f != null && typeof f === 'object' && Object.keys(f).length > 0; + }); + if (typeof drv.aggregate === 'function' && allStructuredSupported && !tzRequiresInMemory && !hasAggregationFilter) { + // HAVING is engine-owned (#4286): applied AFTER aggregation, over + // the aggregated row's own columns (aggregation aliases + groupBy + // projections), identically on both paths. No driver implements it + // natively today; one that grows native support must advertise a + // capability flag, at which point this post-filter becomes the + // fallback tier — the dateGranularity two-tier pattern. + const aggregated = await drv.aggregate(object, ast, this.buildDriverOptions(object, opCtx.context, undefined, { recordedVerdict: opCtx.tenantLayer0Verdict })); + return applyHaving(aggregated, ast.having, havingColumnClasses); + } + // In-memory fallback path: ask the driver for raw rows, then bucket + + // aggregate here. This guarantees `groupBy` (incl. structured items + // carrying `dateGranularity`) and `aggregations` always work even on + // drivers that have no native aggregation support (driver-rest, + // driver-memory, partial SQL drivers), and is the path that honours a + // non-UTC reference timezone. + // + // [#16642] The `find` call asks for ROWS, so the aggregation keys this + // path is about to evaluate ITSELF are stripped from the AST it sends + // down. `find()`'s contract says nothing about `groupBy` / + // `aggregations`, and the drivers disagree about them: `driver-sql` + // and `driver-rest` ignore both and return rows (which is the only + // reason this path has ever worked), while `driver-memory` honours + // them — its `find()` funnels straight into the same + // `performAggregation` its `aggregate(AST)` door uses, and + // `driver-mongodb` / `driver-turso` carry the same refusal on their + // own aggregation faces. Sending the keys to a driver of that kind + // made this ONE seam answer two different wrong things: + // + // * a per-aggregation `filter` (the key that ROUTED the call here) + // was refused NOT_IMPLEMENTED/501 by the driver's own #10413 + // guard — a refusal aimed at direct callers, raised against the + // engine's own lowering, so `service-analytics` answered 501 for + // a measure `filter` on the memory driver while sqlite answered + // the number (#16642); + // * a date-bucketed `groupBy` came back ALREADY grouped, on the raw + // timestamp, and `applyInMemoryAggregation` then aggregated those + // GROUP rows a second time — a count of buckets reported under + // the author's own measure name, i.e. a plausible wrong number + // rather than a refusal. + // + // Both driver refusals document themselves as "unreachable through + // `engine.aggregate`, which lowers in memory for every driver" + // (`driver-memory`'s `refusePerAggregationFilter`, `driver-sql`'s + // `unsupportedAggregationFilterError`); this is the line that makes + // that true. `having` goes with them: `applyHaving` below is the + // authority on it and no driver evaluates it in `find`. + const rowsAst: QueryAST = { ...ast }; + delete rowsAst.groupBy; + delete rowsAst.aggregations; + delete rowsAst.having; + const raw = await driver.find(object, rowsAst, this.buildDriverOptions(object, opCtx.context, undefined, { recordedVerdict: opCtx.tenantLayer0Verdict })); + return applyHaving(applyInMemoryAggregation(raw, ast, tz, declaredFields), ast.having, havingColumnClasses); + }); + + return opCtx.result as any[]; + } diff --git a/packages/objectql/src/engine-find.ts b/packages/objectql/src/engine-find.ts new file mode 100644 index 00000000000..865fe219813 --- /dev/null +++ b/packages/objectql/src/engine-find.ts @@ -0,0 +1,180 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Query lowering, search, find and findOne" section of + * `ObjectQL`: the bodies of `expandSearchOnAst`, `requireFindOnePredicate`, + * `reportFindFailure` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link FindHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { resolveSearchFieldResolution } from '@objectstack/spec/data'; +import type { QueryAST, ServiceObject } from '@objectstack/spec/data'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { isMissingTableError } from '@objectstack/metadata/errors'; +import { collectCredentialFields, collectInternalReadFields } from './secret-fields.js'; +import { expandSearchToFilter, normalizeSearch } from './search-filter.js'; +import { + emptySearchSetError, refuseSearchOverrideNamingNoField, refuseSearchOverrideNamingNoSearchableField, + writeFailureLogMeta, +} from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface FindHost { + readonly logger: ObjectQL['logger']; + resolveSearchQueryableFields: ObjectQL['resolveSearchQueryableFields']; +} + +/** The body of `ObjectQL#expandSearchOnAst`; the method's docblock there is its contract. */ +export async function expandSearchOnAst(this: Pick, ast: QueryAST, schema: ServiceObject | undefined, context: ExecutionContext | undefined): Promise { + // The `$search`/`$searchFields` OData spellings are NOT read here: the + // protocol layer normalizes them to the bare keys before the engine + // (protocol.ts findData), and a direct engine call carrying one is an + // unknown option — rejected at the entry point, not silently dropped + // (#4371). + const raw = (ast as any).search; + // [#27732] An override naming no field is refused, never read as absent. + // Judged before the expansion and so before any driver read, whether or + // not a search term rode along, on both slots, as the door does. + refuseSearchOverrideNamingNoField(ast.object, (ast as any).searchFields, 'searchFields'); + if (raw !== null && typeof raw === 'object') refuseSearchOverrideNamingNoField(ast.object, raw.fields, 'search.fields'); + if (raw != null && schema?.fields) { + const requestedFields = (ast as any).searchFields + ?? (typeof raw === 'object' ? raw?.fields : undefined); + // [#22646] A `search` never scans an `internal: true` column. The flag's + // promise ("the value is never returned on a generic data exit") is a + // withhold on the read path (the row strip) and a refusal everywhere the + // value would be revealed by evaluation; `$search` is such a position — + // a cross-field `$icontains` that confirms a guessed prefix of a withheld + // credential, matching the row only when the guess is right. The field is + // dropped from the set the expansion resolves over (a WITHHOLD, like the + // auto-default's `hidden`/credential-type exclusions one layer down), so + // the default set never scans it, for every verb that reaches here + // (`find` / `findOne` / `aggregate`, and `searchAll` through them). An + // EXPLICIT `$searchFields` naming one is refused loudly at the generic + // data door instead (`assertSearchFieldsAreSearchable`, + // `@objectstack/metadata-protocol`), the #4254 posture — an unhonoured + // narrowing must be a 400, not a silent widening. A direct engine caller + // naming one beside a searchable field has it intersected away here, the + // engine's existing tolerant posture for an unknown requested search + // field; naming only such fields is refused below (#28120). + // [ADR-0100] …and never a `secret` / `password` column either, by the + // collector the credential guards read: the auto-default already skips + // those types, and an author-declared `searchableFields` naming one is + // withheld here the same way, so a search term is never evaluated over + // a value the read path masks. + const internalFields = [...new Set([...collectInternalReadFields(schema), ...collectCredentialFields(schema)])]; + const scanFields = internalFields.length === 0 + ? (schema.fields as any) + : Object.fromEntries( + Object.entries(schema.fields as Record) + .filter(([name]) => !internalFields.includes(name)), + ); + const expandOptions = { + fields: scanFields, + searchableFields: (schema as any).searchableFields, + requestedFields, + // [ADR-0079] `nameField` is the canonical primary-title pointer; + // `displayNameField` is the deprecated alias (still honored). + displayField: (schema as any).nameField ?? (schema as any).displayNameField, + }; + // [#28120] An override naming none of the fields the expansion would + // scan is refused, never widened back to the whole set. + refuseSearchOverrideNamingNoSearchableField( + ast.object, requestedFields, (ast as any).searchFields != null ? 'searchFields' : 'search.fields', + resolveSearchFieldResolution(expandOptions).allowed, + ); + // [#25672] …and never a column the CALLER may not query. The field-read + // predicate guard (plugin-security) runs after this expansion and judges + // the expanded `$or` as the caller's own predicate, so one hidden column + // in the server-resolved set refused the whole search with 403 — + // `sys_user`'s `role` / `ban_reason` / `last_login_ip` broke every user + // picker for members. The set is narrowed to the security service's + // `getQueryableFields` answer, the published complement of what that + // guard refuses (one derivation, no second field-permission rule here), + // asked with the context the middleware will see. A field the caller + // NAMED in `searchFields` is not withheld, so the guard still refuses it + // loudly. The guard itself is unchanged, and still judges every + // predicate the caller wrote. + const queryableFields = normalizeSearch(raw).query.trim() + ? await this.resolveSearchQueryableFields(ast.object, context) + : undefined; + const searchFilter = expandSearchToFilter(raw, { ...expandOptions, queryableFields }); + if (searchFilter) { + ast.where = ast.where ? { $and: [ast.where, searchFilter] } : searchFilter; + } else if (normalizeSearch(raw).query.trim()) { + // [#24284] A non-blank term that expanded to nothing means the RESOLVED + // search set is empty: `expandSearchToFilter` answers `null` for + // exactly two reasons, and the other one (a blank term) is excluded + // here. Leaving `where` untouched would answer the search with EVERY + // row, indistinguishable from "everything matched" — the silent + // widening #4254 refuses on the explicit-override axis. Judged on the + // set the expansion itself resolved (no second resolver), so it holds + // for every verb that reaches here (`find` / `findOne` / `aggregate` / + // the count) and every door (REST, dispatcher, SDK, a hook's + // `ctx.api`). An explicit override intersected to nothing is refused + // before the expansion (#28120) unless the allowed set is empty too, so + // it reaches this only then. [#25672] A set the caller's + // queryable fields withheld down to nothing gets the same answer: the + // declared set expands without the narrowing, so the refusal says so. + const withheld = queryableFields !== undefined && expandSearchToFilter(raw, expandOptions) !== null; + throw emptySearchSetError(ast.object, schema, internalFields.length > 0, withheld); + } + } + delete (ast as any).search; + delete (ast as any).searchFields; + } + +/** The body of `ObjectQL#requireFindOnePredicate`; the method's docblock there is its contract. */ +export function requireFindOnePredicate(object: string, ast: QueryAST): void { + const where = ast.where as unknown; + const hasPredicate = + where != null && + (typeof where !== 'object' || Array.isArray(where) || Object.keys(where).length > 0); + if (hasPredicate) return; + if (Array.isArray(ast.orderBy) && ast.orderBy.length > 0) return; + throw new Error( + `findOne('${object}') selects no particular record: 'where' is absent or empty ` + + `and the query carries no 'orderBy'. findOne applies limit: 1, so this would return an ` + + `ARBITRARY row — a real, plausible-looking record unrelated to what was asked for, which ` + + `no caller's null-check can catch. Pass 'where' (or a 'search' that resolves to ` + + `one) to select the record; pass 'orderBy' if you mean "the first record in THIS order"; ` + + `or call find('${object}', { limit: 1 }) if any row will genuinely do.`, + ); + } + +/** The body of `ObjectQL#reportFindFailure`; the method's docblock there is its contract. */ +export function reportFindFailure(this: Pick, object: string, error: unknown): void { + if (isMissingTableError(error, object)) { + this.logger.debug('Find operation failed', { + object, + reason: 'table-not-provisioned', + error: (error as { message?: unknown } | null | undefined)?.message, + }); + return; + } + // [#17212] `warn`, not `error`: `find`'s `catch` rethrows, so the caller + // is told — see the docblock's last section. + this.logger.warn('Find operation failed', writeFailureLogMeta(error, { object })); + } diff --git a/packages/objectql/src/engine-hooks-and-functions.ts b/packages/objectql/src/engine-hooks-and-functions.ts new file mode 100644 index 00000000000..8ba54862892 --- /dev/null +++ b/packages/objectql/src/engine-hooks-and-functions.ts @@ -0,0 +1,122 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Class head: units, hooks, functions, body runners, hook dispatch" section of + * `ObjectQL`: the bodies of `unregisterHooksByPackage`, + * `unregisterFunctionsByPackage`, `dispatchUnscopedMultiWriteHooks` moved out + * of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link HooksAndFunctionsHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import type { HookContext } from '@objectstack/spec/data'; +import { hookStoppedRefusal } from './summary-errors.js'; +import { HookTargetRebindError } from './hook-target-rebind-errors.js'; +import { hookMatchesObject } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface HooksAndFunctionsHost { + readonly functions: ObjectQL['functions']; + readonly hooks: ObjectQL['hooks']; + readonly logger: ObjectQL['logger']; +} + +/** The body of `ObjectQL#unregisterHooksByPackage`; the method's docblock there is its contract. */ +export function unregisterHooksByPackage(this: Pick, packageId: string): number { + if (!packageId) return 0; + let removed = 0; + for (const [event, entries] of this.hooks.entries()) { + const before = entries.length; + const kept = entries.filter((e) => e.packageId !== packageId); + if (kept.length !== before) { + this.hooks.set(event, kept); + removed += before - kept.length; + } + } + if (removed > 0) { + this.logger.debug('Unregistered hooks by package', { packageId, removed }); + } + return removed; + } + +/** The body of `ObjectQL#unregisterFunctionsByPackage`; the method's docblock there is its contract. */ +export function unregisterFunctionsByPackage(this: Pick, packageId: string): number { + if (!packageId) return 0; + let removed = 0; + for (const [name, entry] of this.functions.entries()) { + if (entry.packageId === packageId) { + this.functions.delete(name); + removed += 1; + } + } + if (removed > 0) { + this.logger.debug('Unregistered functions by package', { packageId, removed }); + } + return removed; + } + +/** The body of `ObjectQL#dispatchUnscopedMultiWriteHooks`; the method's docblock there is its contract. */ +export async function dispatchUnscopedMultiWriteHooks(this: Pick, event: 'beforeUpdate' | 'beforeDelete', object: string, batchCtx: HookContext): Promise { + const entries = this.hooks.get(event); + if (!entries || entries.length === 0) return; + const flagged = entries.filter( + (entry) => entry.dispatchUnscopedMultiWrite && hookMatchesObject(entry, object), + ); + if (flagged.length === 0) return; + // Mirrors `triggerHooks`' opt-out rule: `skipAutomations` suppresses only + // metadata-bound entries. No metadata binding can set the flag today, so + // this is parity with the dispatch loop rather than a reachable branch — + // kept so the two rules cannot drift if that ever changes. + const skipAutomations = + (batchCtx.session as { skipAutomations?: boolean } | undefined)?.skipAutomations === true; + const wholeOpCtx = { + ...batchCtx, + dispatch: { + mode: 'record', + index: 0, + scope: (batchCtx.dispatch as { scope: Record }).scope, + }, + } as unknown as HookContext; + for (const entry of flagged) { + if (skipAutomations && entry.meta) continue; + // [#29422] As in `triggerHooks`. + try { + await entry.handler(wholeOpCtx); + } catch (e) { + throw hookStoppedRefusal(e, event, object, entry.hookName); + } + } + // `input` is identity-shared with the batch context, so the observation + // covers everything the handlers touched. + const observed = (wholeOpCtx.input as { id?: unknown }).id; + if (observed !== undefined) { + throw new HookTargetRebindError({ + object, + event, + path: 'unscoped-multi', + expectedId: undefined, + observedId: observed, + }); + } + } diff --git a/packages/objectql/src/engine-insert.ts b/packages/objectql/src/engine-insert.ts new file mode 100644 index 00000000000..49d954a46eb --- /dev/null +++ b/packages/objectql/src/engine-insert.ts @@ -0,0 +1,76 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Write preparation and insert" section of `ObjectQL`: + * the bodies of `staticReadonlyCreateStrip` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link InsertHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import type { EvalPermissions } from '@objectstack/formula'; +import { stripReadonlyFields } from './validation/rule-validator.js'; +import type { staticReadonlyInsertSubject } from './validation/rule-validator.js'; +import type { PermissionResolution } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface InsertHost { + applyFieldDefaults: ObjectQL['applyFieldDefaults']; + resolveDefaultPermissions: ObjectQL['resolveDefaultPermissions']; +} + +/** The body of `ObjectQL#staticReadonlyCreateStrip`; the method's docblock there is its contract. */ +export async function staticReadonlyCreateStrip(this: Pick, object: string, readonlySubject: NonNullable>, row: Record, supplied: Readonly>, context: ExecutionContext | undefined, pass: { + logger: Parameters[3]; + strictReadonlyWrites: boolean; + hookWrittenKeys: ReadonlySet | undefined; + nowSnap: Date; + permissionResolution: PermissionResolution | undefined; + }): Promise<{ row: Record; taken: string[]; error?: unknown }> { + const stripped = stripReadonlyFields( + readonlySubject as any, row, supplied, pass.logger, + { + strictReadonlyWrites: pass.strictReadonlyWrites, + hookWrittenKeys: pass.hookWrittenKeys, + verb: 'insert', + }, + ) as Record; + if (stripped === row) return { row, taken: [] }; + const taken = Object.keys(row).filter((k) => !(k in stripped)); + if (taken.length > 0) { + let redefaultPermissions: EvalPermissions | undefined; + try { + redefaultPermissions = (await this.resolveDefaultPermissions( + object, [stripped], context, pass.permissionResolution, taken, + ))(stripped); + } catch (err) { + return { row, taken, error: err }; + } + const redefaulted = this.applyFieldDefaults(object, stripped, context, pass.nowSnap, redefaultPermissions); + for (const k of taken) { + if (redefaulted[k] !== undefined) stripped[k] = redefaulted[k]; + } + } + return { row: stripped, taken }; + } diff --git a/packages/objectql/src/engine-middleware-and-context.ts b/packages/objectql/src/engine-middleware-and-context.ts new file mode 100644 index 00000000000..8b124c71aa9 --- /dev/null +++ b/packages/objectql/src/engine-middleware-and-context.ts @@ -0,0 +1,147 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Authz invalidation, middleware, session, driver options, posture, organization probe" + * section of `ObjectQL`: the bodies of `executeWithMiddleware`, + * `resolveSystemInsertOrganization` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link MiddlewareAndContextHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { redactPropagatedDriverFault } from '@objectstack/types'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { isWriteEpochOperation } from './write-epoch.js'; +import { ORGANIZATION_OBJECT, SystemWriteOrganizationRequiredError, carriesOrganization, resolveSystemWriteOrganization, resolveTenantFieldName } from './tenancy/system-write-organization.js'; +import { isFederatedObject } from './federated-object.js'; +import type { OperationContext } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface MiddlewareAndContextHost { + readonly _registry: ObjectQL['_registry']; + readonly middlewares: ObjectQL['middlewares']; + probeInstallOrganizations: ObjectQL['probeInstallOrganizations']; + resolveEnginePosture: ObjectQL['resolveEnginePosture']; + readonly writeEpoch: ObjectQL['writeEpoch']; +} + +/** The body of `ObjectQL#executeWithMiddleware`; the method's docblock there is its contract. */ +export async function executeWithMiddleware(this: Pick, ctx: OperationContext, executor: () => Promise): Promise { + // [#11968] Advance the write epoch FIRST — ahead of every middleware, and + // so ahead of any `isSystem` bypass one of them applies. The writes most + // likely to change what a caller may see are system ones: the platform + // seeder, a package publish, the auto-org-admin grant. A guard that only + // saw user writes would leave cached authorization standing across exactly + // the grants that matter. (`@objectstack/plugin-security` made the same + // choice inside its own middleware in #10757; hoisting it here is what + // makes the covered set identical BY CONSTRUCTION to "everything the + // middleware chain sees", instead of identical by two authors agreeing.) + // + // ⛔ Do not move this below the `applicable` filter: that filter is a + // per-object middleware selector, and an epoch that only advanced when some + // middleware happened to be registered for the written object would be a + // seam with holes in it. + if (isWriteEpochOperation(ctx.operation)) { + this.writeEpoch.bump('write'); + } + + const applicable = this.middlewares.filter(m => + !m.object || m.object === '*' || m.object === ctx.object + ); + + let index = 0; + const next = async (): Promise => { + if (index < applicable.length) { + const mw = applicable[index++]; + await mw.fn(ctx, next); + } else { + try { + ctx.result = await executor(); + } catch (e) { + // [#21274] The engine BOUNDARY for every operation this seam runs + // (find, findOne, count, aggregate, insert, update, delete): a driver + // error leaves the engine with its bound statement and the caller's + // values cut, from `message`, `stack`, the driver's own + // statement-bearing properties and `cause` — the class and the codes + // kept. Innermost, so a middleware that logs what it caught gets + // the cut too. The executors' own log lines ran before this, from + // the raw error, and are unchanged. See `redactPropagatedDriverFault`. + throw redactPropagatedDriverFault(e); + } + } + }; + + await next(); + return ctx.result; + } + +/** The body of `ObjectQL#resolveSystemInsertOrganization`; the method's docblock there is its contract. */ +export async function resolveSystemInsertOrganization(this: Pick, object: string, execCtx: ExecutionContext | undefined, rows: readonly Record[]): Promise { + // Already carrying an organization on the context — a session write, or a + // system write that threaded one. Nothing to resolve; this is the shape the + // ruling asks every system write to reach. + if (carriesOrganization(execCtx?.tenantId)) return undefined; + // [ADR-0131 D9, C8 stage 8b] Under every posture a platform object has no + // exemption: with a tenant column it is in scope like any other object, so + // its organization-less system insert is stamped with the derived + // organization or refused below — never left NULL. The walled half landed + // in 18.x, after the stages that gave the walled writers (identity mail, + // global audit, declared webhooks) an organization or a sibling of their + // own. ⛔ Never re-introduce a per-table or per-posture cut here: a table + // whose rows are deployment-level declares no organization column + // (`sys_secret` and `sys_flow_credential` since the steps 5+6 landing, + // #24582) and exits below, on the missing tenant field. + const objectSchema = this._registry.getObject(object) as any; + // A federated object's schema is the REMOTE's (ADR-0015); the platform's + // injected column says nothing about it, which is the same reason + // `buildDriverOptions` withholds `tenantId` there. + if (isFederatedObject(objectSchema)) return undefined; + const tenantField = resolveTenantFieldName(objectSchema); + if (!tenantField) return undefined; + // A row that names its own organization has carried one explicitly. Only a + // row still missing it can fork a counter, so a batch where every row is + // stamped resolves nothing — and a batch where some are not is decided by + // the ones that are not. + if (rows.every((row) => carriesOrganization(row?.[tenantField]))) return undefined; + + const posture = this.resolveEnginePosture(); + // [ADR-0131 D9] Zero organizations is refused under `single` only where + // the composition registers the organization object; the composition that + // registers none keeps today's branch (`no-organization-object`). The + // probe itself is unchanged and answers `[]` for both — this is the one + // fact that tells them apart, read from the same registry the probe reads. + const decision = await resolveSystemWriteOrganization({ + posture, + probeOrganizations: () => this.probeInstallOrganizations(), + organizationObjectRegistered: !!this._registry.getObject(ORGANIZATION_OBJECT), + }); + if (decision.kind === 'derived') return decision.organizationId; + if (decision.kind === 'no-organization-object') return undefined; + throw new SystemWriteOrganizationRequiredError( + object, + posture, + decision.reason, + decision.organizationCount, + ); + } diff --git a/packages/objectql/src/engine-references-and-master-detail.ts b/packages/objectql/src/engine-references-and-master-detail.ts new file mode 100644 index 00000000000..4a0aba2253c --- /dev/null +++ b/packages/objectql/src/engine-references-and-master-detail.ts @@ -0,0 +1,81 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Realtime, i18n, home amounts, validation messages, references, master-detail" + * section of `ObjectQL`: the bodies of `inspectDanglingReferences`, + * `inspectCascadeDelete` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link ReferencesAndMasterDetailHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { postureUsesUnionScope } from '@objectstack/spec/security'; +import { auditDanglingReferences } from './integrity/dangling-reference-audit.js'; +import type { AuditableObject, DanglingReferenceAuditOptions, DanglingReferenceReport } from './integrity/dangling-reference-audit.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface ReferencesAndMasterDetailHost { + readonly _registry: ObjectQL['_registry']; + collectCascadeDeleteSet: ObjectQL['collectCascadeDeleteSet']; + find: ObjectQL['find']; + readonly logger: ObjectQL['logger']; + referenceExists: ObjectQL['referenceExists']; + resolveEnginePosture: ObjectQL['resolveEnginePosture']; + resolveObjectName: ObjectQL['resolveObjectName']; +} + +/** The body of `ObjectQL#inspectDanglingReferences`; the method's docblock there is its contract. */ +export async function inspectDanglingReferences(this: Pick, options?: DanglingReferenceAuditOptions): Promise { + // Read once per run, live — the posture IN FORCE, by the engine's one reader. + const unionScope = postureUsesUnionScope(this.resolveEnginePosture()); + return auditDanglingReferences( + { + objects: () => this._registry.getAllObjects() as unknown as AuditableObject[], + find: (object, opts) => this.find(object, opts as any) as Promise>>, + probe: (target, id, organization) => this.referenceExists( + target, id, organization == null || unionScope ? undefined : ({ tenantId: organization } as ExecutionContext), + ), + warn: (msg, meta) => this.logger?.warn?.(msg, meta as any), + }, + options, + ); + } + +/** The body of `ObjectQL#inspectCascadeDelete`; the method's docblock there is its contract. */ +export async function inspectCascadeDelete(this: Pick, object: string, id: string | number): Promise> { + const resolved = this.resolveObjectName(object); + const set = await this.collectCascadeDeleteSet( + resolved, [id], undefined, this._registry.getAllObjects(), { fileElevations: false }, + ); + const root = { object: resolved, id: String(id) }; + const members = [root]; + for (const [memberObject, ids] of set.members) { + for (const memberId of ids) { + if (memberObject === root.object && memberId === root.id) continue; + members.push({ object: memberObject, id: memberId }); + } + } + return members; + } diff --git a/packages/objectql/src/engine-registration.ts b/packages/objectql/src/engine-registration.ts new file mode 100644 index 00000000000..86a0b1c68d1 --- /dev/null +++ b/packages/objectql/src/engine-registration.ts @@ -0,0 +1,348 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Package, app, plugin, driver and datasource registration" section of + * `ObjectQL`: the bodies of `registerPackageBody`, + * `registerMetadataCollections`, `supplyFileColumnsMovedResolver`, + * `registerDatasourceDef`, `applyExternalDriverFloor`, `assertWriteAllowed` + * moved out of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link RegistrationHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import type { ServiceObject } from '@objectstack/spec/data'; +import type { InstalledPackageSource } from '@objectstack/spec/kernel'; +import type { IDataDriver } from '@objectstack/core'; +import { ExternalWriteForbiddenError, pluralToSingular } from '@objectstack/spec/shared'; +import type { ArtifactInstallScope } from './registry.js'; +import { ASSEMBLED_VIEW_ITEMS_KEY, AssembledViewArtifactSchema, expandViewContainer, isAggregatedViewContainer } from '@objectstack/spec'; +import { createExternalDriverFloor, isExternalDriverFloor, needsExternalDriverFloor, unwrapExternalDriverFloor } from './external-driver-floor.js'; +import { + METADATA_ARRAY_KEYS, assembledViewItemRefusal, noteRegistrationStamp, registerAppNotingStamp, + resolveMetadataItemName, stampObjectPackage, stampPackage, +} from './engine-top-level.js'; +import type { DatasourceDef } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface RegistrationHost { + readonly _registry: ObjectQL['_registry']; + applyExternalDriverFloor: ObjectQL['applyExternalDriverFloor']; + readonly datasourceDefs: ObjectQL['datasourceDefs']; + readonly drivers: ObjectQL['drivers']; + haveFileColumnsMoved: ObjectQL['haveFileColumnsMoved']; + readonly logger: ObjectQL['logger']; + registerMetadataCollections: ObjectQL['registerMetadataCollections']; + registerPlugin: ObjectQL['registerPlugin']; + resolveNavObjectNames: ObjectQL['resolveNavObjectNames']; +} + +/** The body of `ObjectQL#registerPackageBody`; the method's docblock there is its contract. */ +export function registerPackageBody(this: Pick, manifest: any, id: string, namespace: string | undefined, version: string | undefined, source: Exclude, scope: ArtifactInstallScope | undefined): void { + // 1. Register the Package (manifest + lifecycle state). + // [ADR-0130 D1/D3] `scope` is the artifact's own package list, passed by + // the load path that read the artifact. It is what lets the install gate + // tell a CO-OWNER (another package from this same artifact) from a + // stranger — the question D1 corrected the gate to ask. Absent for every + // single-package caller, where the gate behaves exactly as before (D7). + this._registry.installPackage(manifest, source, undefined, scope); + this.logger.debug('Installed Package', { id: manifest.id, name: manifest.name, namespace }); + + // 2. Register owned objects + if (manifest.objects) { + if (Array.isArray(manifest.objects)) { + this.logger.debug('Registering objects from manifest (Array)', { id, objectCount: manifest.objects.length }); + for (const objDef of manifest.objects) { + const fqn = this._registry.registerObject(stampObjectPackage(objDef, id, version), id, namespace, 'own'); + this.logger.debug('Registered Object', { fqn, from: id }); + } + } else { + this.logger.debug('Registering objects from manifest (Map)', { id, objectCount: Object.keys(manifest.objects).length }); + // `manifest` is `any` (a raw authored manifest), so the map branch + // widens its values to `unknown`. State the contract once, on the + // entries, rather than casting the argument at the call: the map + // values ARE authored `ServiceObject`s — the INPUT shape, which is + // what `registerObject` takes since ADR-0122 phase 2 (commit 53068c130). + for (const [name, objDef] of Object.entries(manifest.objects) as [string, ServiceObject][]) { + // Ensure name in definition matches key + objDef.name = name; + const fqn = this._registry.registerObject(stampObjectPackage(objDef, id, version), id, namespace, 'own'); + this.logger.debug('Registered Object', { fqn, from: id }); + } + } + } + + // 2b. Register object extensions (fields added to objects owned by other packages) + if (Array.isArray(manifest.objectExtensions) && manifest.objectExtensions.length > 0) { + this.logger.debug('Registering object extensions', { id, count: manifest.objectExtensions.length }); + for (const ext of manifest.objectExtensions) { + const targetFqn = ext.extend; + const priority = ext.priority ?? 200; + // Create a partial object definition for the extension + const extDef = { + name: targetFqn, // Use the target FQN as name + fields: ext.fields, + // [ADR-0146 D10, second rule] The per-field display patches, + // the slot as `ObjectExtensionSchema.fieldPatches` declares + // it — applied by the registry's `mergeObjectDefinitions`, + // whose fold every object read shares. + fieldPatches: ext.fieldPatches, + label: ext.label, + pluralLabel: ext.pluralLabel, + description: ext.description, + validations: ext.validations, + indexes: ext.indexes, + }; + // Register as extension (namespace is undefined since we're targeting by FQN) + this._registry.registerObject(stampPackage(extDef, id, version), id, undefined, 'extend', priority); + this.logger.debug('Registered Object Extension', { target: targetFqn, priority, from: id }); + } + } + + // 3. Register apps (UI navigation definitions) as their own metadata type + // Resolve short objectName references in navigation to FQN so the + // Console UI can match them against the object registry. + if (Array.isArray(manifest.apps) && manifest.apps.length > 0) { + this.logger.debug('Registering apps from manifest', { id, count: manifest.apps.length }); + for (const app of manifest.apps) { + const appName = app.name || app.id; + if (appName) { + const resolved = namespace ? this.resolveNavObjectNames(app, namespace) : app; + registerAppNotingStamp(this._registry, resolved, id, version); + this.logger.debug('Registered App', { app: appName, from: id }); + } + } + } + + // 4. If manifest itself looks like an App (has navigation), also register as app + // This handles the case where the manifest IS the app definition (legacy/simple packages) + if (manifest.name && manifest.navigation && !manifest.apps?.length) { + const resolved = namespace ? this.resolveNavObjectNames(manifest, namespace) : manifest; + registerAppNotingStamp(this._registry, resolved, id, version); + this.logger.debug('Registered manifest-as-app', { app: manifest.name, from: id }); + } + + // (Navigation contributions, ADR-0029 D7, register in `registerApp` + // once this body has registered whole.) + + // 5. Register all other metadata types generically — the SAME seam the + // nested-plugin path uses (`registerMetadataCollections`), so the two + // entry points cannot answer differently for one collection (#7163). + this.registerMetadataCollections(manifest, id, 'manifest', version); + + // 6. Register seed data as metadata (keyed by target object name) + const seedData = (manifest as any).data; + if (Array.isArray(seedData) && seedData.length > 0) { + this.logger.debug('Registering seed data datasets', { id, count: seedData.length }); + for (const dataset of seedData) { + if (dataset.object) { + // [#28534] The collection loop's owner-stamp refusal for the + // one collection filed outside it was judged before anything + // registered ({@link packageBodySeedDataRefusal}). + const stampBefore = (dataset as { _packageId?: unknown })._packageId; + this._registry.registerItem('data', stampPackage(dataset, id, version), 'object' as any, id); + noteRegistrationStamp(dataset, stampBefore); + } + } + } + + // (Contributed kinds register in `registerApp` once this body has + // registered whole: they belong to no package, so no restore of a + // refused body could take them back.) + + // 7. Recursively register nested plugins + if (Array.isArray(manifest.plugins) && manifest.plugins.length > 0) { + this.logger.debug('Processing nested plugins', { id, count: manifest.plugins.length }); + for (const plugin of manifest.plugins) { + if (plugin && typeof plugin === 'object') { + const pluginName = plugin.name || plugin.id || 'unnamed-plugin'; + this.logger.debug('Registering nested plugin', { pluginName, parentId: id }); + this.registerPlugin(plugin, id, namespace, version); + } + } + } + } + +/** The body of `ObjectQL#registerMetadataCollections`; the method's docblock there is its contract. */ +export function registerMetadataCollections(this: Pick, source: any, ownerId: string, sourceLabel: string, ownerVersion?: string) { + // `ownerVersion` is the owning package's version, stamped like `ownerId`. + for (const key of METADATA_ARRAY_KEYS) { + const items = (source as any)?.[key]; + if (!Array.isArray(items) || items.length === 0) continue; + this.logger.debug(`Registering ${key} from ${sourceLabel}`, { id: ownerId, count: items.length }); + // A `picklistExtensions` entry is not an item: it has no name, only + // the list it adds to, and the registry merges it there — additive + // only, a repeated value refused loudly (`registerPicklistExtension`). + if (key === 'picklistExtensions') { + for (const extension of items) this._registry.registerPicklistExtension(extension, ownerId); + continue; + } + for (const item of items) { + const itemName = resolveMetadataItemName(key, item); + if (!itemName) { + this.logger.warn(`Skipping ${pluralToSingular(key)} without a derivable name`, { id: ownerId }); + continue; + } + // [#28697] Every refusal this seam used to raise here — the + // divergent view container, the permission set's owner, an + // item's own `_packageId`, a `views:` entry that is not a + // container — is judged from the body alone, so it is asked + // before anything of the package registers, with the reasons it + // exists: {@link packageBodyCollectionRefusal}. Both callers ask + // it first (`registerApp`, for the body and each nested plugin). + const toRegister = item.name === itemName ? item : { ...item, name: itemName }; + const stampBefore = (toRegister as { _packageId?: unknown })._packageId; + this._registry.registerItem(pluralToSingular(key), stampPackage(toRegister, ownerId, ownerVersion), 'name' as any, ownerId); + noteRegistrationStamp(toRegister, stampBefore); + // "Object has-many View" (ADR-0017): a `defineView` document + // aggregates an object's views. Register the container under the + // bare key (above, back-compat) AND expand it into + // independent ViewItems registered under ., so + // `getViewsByObject()` / `GET /meta/view?object=` surface the + // per-view `package` layer the switcher + Studio consume. + if (key === 'views' && isAggregatedViewContainer(toRegister)) { + for (const vi of expandViewContainer(itemName, toRegister)) { + for (const w of vi._diagnostics?.warnings ?? []) { + this.logger.warn(`View expansion warning for '${vi.name}': ${w.message}`, { from: ownerId }); + } + this._registry.registerItem('view', stampPackage(vi, ownerId, ownerVersion), 'name' as any, ownerId); + } + } + } + } + + // [#5320] The `viewItems:` channel — the declared entry for the + // NON-container view artifacts a runtime-ASSEMBLED manifest carries + // (tenant-authored standalone ViewItems, flattened overlays, expanded + // items a travelling container cannot re-derive). Written by the manifest + // assemblers (`partitionAssembledViewArtifacts` — package export, the + // artifact factories); refused at the authoring door (`defineStack` types + // the key `never`), so only machine-assembled manifests legitimately + // reach here carrying it. Strictly schema'd: each entry is judged by + // `AssembledViewArtifactSchema` and the PARSED body is what registers, so + // an undeclared bag neither passes nor rides through — declared = + // enforced, in both directions. + const assembledItems = (source as any)?.[ASSEMBLED_VIEW_ITEMS_KEY]; + if (Array.isArray(assembledItems) && assembledItems.length > 0) { + this.logger.debug(`Registering ${ASSEMBLED_VIEW_ITEMS_KEY} from ${sourceLabel}`, { id: ownerId, count: assembledItems.length }); + for (const item of assembledItems) { + // [#28697] The entry's parse refusal and its owner stamp were + // judged before anything registered ({@link packageBodyCollectionRefusal}); + // the parse is repeated for the body it registers. + const parsed = AssembledViewArtifactSchema.safeParse(item); + if (!parsed.success) throw assembledViewItemRefusal(item, parsed.error, sourceLabel, ownerId); + const body = parsed.data as Record; + const itemName = resolveMetadataItemName('views', body); + if (!itemName) { + this.logger.warn('Skipping viewItems entry without a derivable name', { id: ownerId }); + continue; + } + // No {@link noteRegistrationStamp} follows the `registerItem` below, on + // purpose: what it stamps is `parsed.data`, the parse's own new + // object, never the caller's entry, so this site leaves no in-place + // stamp on anything a later registration could be handed + // (`collection-item-owner-stamp.test.ts` pins the caller's entry + // unstamped). + const toRegister = body.name === itemName ? body : { ...body, name: itemName }; + this._registry.registerItem('view', stampPackage(toRegister, ownerId, ownerVersion), 'name' as any, ownerId); + } + } + } + +/** The body of `ObjectQL#supplyFileColumnsMovedResolver`; the method's docblock there is its contract. */ +export function supplyFileColumnsMovedResolver(this: Pick, driver: IDataDriver): void { + const sink = driver as unknown as { + setFileColumnsMovedResolver?: (resolve: () => Promise) => void; + }; + if (typeof sink.setFileColumnsMovedResolver !== 'function') return; + try { + sink.setFileColumnsMovedResolver(() => this.haveFileColumnsMoved()); + } catch (e: any) { + // Installing the question must never break a driver registration. A + // driver that did not take the resolver keeps its own default, which is + // the JSON arm — the same place every other way of not knowing lands. + this.logger.debug('Driver declined the ADR-0104 media-arm resolver', { + driverName: driver.name, + error: e?.message ?? String(e), + }); + } + } + +/** The body of `ObjectQL#registerDatasourceDef`; the method's docblock there is its contract. */ +export function registerDatasourceDef(this: Pick, def: DatasourceDef): void { + if (!def?.name) return; + // [#27277] A def that names no driver keeps the id stored before it: a + // caller registering only the write-gate facts after a connect must not + // take the floor back off, so an absent key never reads as "not a plugin + // driver". It is a keep, not a source: `unregisterDriver` drops the stored + // def, so a connect that follows an eviction has nothing to keep — the + // datasource-connection service names `driver` on every registration + // ([#27666], `EngineDatasourceDef.driver`). + const driver = def.driver ?? this.datasourceDefs.get(def.name)?.driver; + this.datasourceDefs.set(def.name, { driver, schemaMode: def.schemaMode, external: def.external }); + this.applyExternalDriverFloor(def.name); + } + +/** The body of `ObjectQL#applyExternalDriverFloor`; the method's docblock there is its contract. */ +export function applyExternalDriverFloor(this: Pick, name: string): void { + const current = this.drivers.get(name); + if (!current) return; + const floored = isExternalDriverFloor(current); + if (needsExternalDriverFloor(this.datasourceDefs.get(name))) { + if (floored) return; + this.drivers.set(name, createExternalDriverFloor(current, { + datasource: name, + fieldsOf: (object) => this._registry.getObject(object)?.fields, + // [#29762] Read from the def held NOW, on every read: a redefinition + // that keeps the floor returns above without re-wrapping the driver. + rowCap: () => this.datasourceDefs.get(name)?.external?.rowCap, + logger: this.logger, + })); + this.logger.info('Datasource served by a plugin driver: the engine applies each query in memory', { + datasource: name, + }); + } else if (floored) { + this.drivers.set(name, unwrapExternalDriverFloor(current)); + } + } + +/** The body of `ObjectQL#assertWriteAllowed`; the method's docblock there is its contract. */ +export function assertWriteAllowed(this: Pick, objectName: string, operation: 'insert' | 'update' | 'delete'): void { + const object = this._registry.getObject(objectName) as any; + const dsName = object?.datasource; + if (!dsName || dsName === 'default') return; + + const ds = this.datasourceDefs.get(dsName); + // No recorded definition, or an explicitly managed one ⇒ allow. + if (!ds || !ds.schemaMode || ds.schemaMode === 'managed') return; + + const dsAllows = ds.external?.allowWrites ?? false; + const objAllows = object?.external?.writable ?? false; + if (!(dsAllows && objAllows)) { + throw new ExternalWriteForbiddenError( + `Write '${operation}' blocked on object '${objectName}': datasource '${dsName}' is external ` + + `(schemaMode=${ds.schemaMode}). Requires datasource.external.allowWrites=true (got ${dsAllows}) ` + + `AND object.external.writable=true (got ${objAllows}).`, + ); + } + } diff --git a/packages/objectql/src/engine-secrets-and-routing.ts b/packages/objectql/src/engine-secrets-and-routing.ts new file mode 100644 index 00000000000..9b5d2835e39 --- /dev/null +++ b/packages/objectql/src/engine-secrets-and-routing.ts @@ -0,0 +1,110 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Crypto, secrets, internal fields, schema and datasource routing, init" + * section of `ObjectQL`: the bodies of `resolveDatasourceBinding` moved out of + * `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link SecretsAndRoutingHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * Values that stayed in `engine.ts` are imported from it (lever (a) of + * ruling 6591753301); every use is inside a function body, so the import + * cycle is never read at evaluation time. `index.ts` re-exports nothing from + * this module. + */ + +import { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface SecretsAndRoutingHost { + readonly _registry: ObjectQL['_registry']; + readonly defaultDriver: ObjectQL['defaultDriver']; + readonly drivers: ObjectQL['drivers']; + readonly manifests: ObjectQL['manifests']; + resolveDatasourceFromMapping: ObjectQL['resolveDatasourceFromMapping']; +} + +/** The body of `ObjectQL#resolveDatasourceBinding`; the method's docblock there is its contract. */ +export function resolveDatasourceBinding(this: Pick, objectName: string): { + datasource: string; + via: 'explicit' | 'mapping' | 'lifecycle' | 'package' | 'default'; + /** Owning package, on `via: 'package'` only — for the debug line. */ + packageId?: string; + } | undefined { + const object = this._registry.getObject(objectName); + + // 1. Object's explicit datasource field (highest priority) + if (object?.datasource && object.datasource !== 'default') { + return { datasource: object.datasource, via: 'explicit' }; + } + + // 2. Check datasourceMapping rules + // + // A rule that MATCHES is a routing decision, not a hint (#4462). It used to + // fall through to steps 3-5 whenever the named datasource had no live + // driver, which put an object's rows in the DEFAULT store while every + // signal said otherwise: boot succeeded, `/ready` answered 200, the + // datasource name appeared nowhere in the log, and the write returned 201. + // An operator who routes an object to Postgres and gets the URL wrong finds + // out by going to look in Postgres and finding it empty. + // + // `default` is the one name that legitimately resolves onward: the default + // driver keeps its NATURAL name (#3826), so `drivers.has('default')` is + // false by construction and step 5 is how routing to it works. + const mappedDatasource = this.resolveDatasourceFromMapping(objectName, object); + if (mappedDatasource && mappedDatasource !== 'default') { + return { datasource: mappedDatasource, via: 'mapping' }; + } + + // 3. Lifecycle-class separation (ADR-0057 §3.6): high-frequency + // platform-generated data (telemetry / event) and the audit ledger move + // to a dedicated 'telemetry' datasource when one is registered, so their + // growth can never again pollute the business DB. `transient` stays on + // the primary deliberately: those objects are user-session data, and + // some (e.g. better-auth's sys_device_code) are also accessed outside + // the engine — splitting their storage would split their brain. + const lifecycleClass = (object as { lifecycle?: { class?: string } } | undefined)?.lifecycle?.class; + if ( + lifecycleClass !== undefined && + ObjectQL.SYSTEM_LEDGER_LIFECYCLE_CLASSES.has(lifecycleClass) && + this.drivers.has(ObjectQL.LIFECYCLE_DATASOURCE) + ) { + return { datasource: ObjectQL.LIFECYCLE_DATASOURCE, via: 'lifecycle' }; + } + + // 4. Check package's defaultDatasource + // Use the object's FQN name (from getObject) for ownership lookup + const fqn = object?.name || objectName; + const owner = this._registry.getObjectOwner(fqn); + if (owner?.packageId) { + const manifest = this.manifests.get(owner.packageId); + const packageDatasource = manifest?.defaultDatasource; + if (packageDatasource && packageDatasource !== 'default' && this.drivers.has(packageDatasource)) { + return { datasource: packageDatasource, via: 'package', packageId: owner.packageId }; + } + } + + // 5. Fallback to global default driver + if (this.defaultDriver && this.drivers.has(this.defaultDriver)) { + return { datasource: this.defaultDriver, via: 'default' }; + } + + return undefined; + } diff --git a/packages/objectql/src/engine-security-resolvers.ts b/packages/objectql/src/engine-security-resolvers.ts new file mode 100644 index 00000000000..b4f5da6545e --- /dev/null +++ b/packages/objectql/src/engine-security-resolvers.ts @@ -0,0 +1,120 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Registration seams, readable-ids security, formula and default permissions" + * section of `ObjectQL`: the bodies of `refuseUncomposableReadableIdsClasses`, + * `resolveFormulaRelated` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link SecurityResolversHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { referenceTargetOf } from '@objectstack/spec/data'; +import type { EngineQueryOptions } from '@objectstack/spec/data'; +import { describeReadableIdsLeaf, readableIdsRefusalError } from './filter-comparand-shape.js'; +import type { ComposableReadableIdsSite } from './filter-comparand-shape.js'; +import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel'; +import { undeclaredRelatedFields } from './engine-top-level.js'; +import type { FormulaPlanEntry, FormulaRelated } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface SecurityResolversHost { + readonly _registry: ObjectQL['_registry']; + readonly formulaHopReads: ObjectQL['formulaHopReads']; + resolveOneHopRelated: ObjectQL['resolveOneHopRelated']; + uncomposableReadableIdsParent: ObjectQL['uncomposableReadableIdsParent']; +} + +/** The body of `ObjectQL#refuseUncomposableReadableIdsClasses`; the method's docblock there is its contract. */ +export function refuseUncomposableReadableIdsClasses(this: Pick, object: string, operation: string, sites: readonly ComposableReadableIdsSite[]): string[] { + const firstSite = new Map(); + for (const site of sites) if (!firstSite.has(site.leaf.object)) firstSite.set(site.leaf.object, site); + const refuse = (site: ComposableReadableIdsSite, why: string): Error => readableIdsRefusalError( + object, + `${operation}('${object}'): ${describeReadableIdsLeaf(site.at, site.leaf)}, and the engine composes no ` + + `such leaf ${why}. The query was NOT run and no driver was asked. Read this class through its bounded ` + + 'probe, chosen before the leaf is written.', + ); + const composable: string[] = []; + for (const [parent, site] of firstSite) { + // A parent the registry does not hold has no record anyone may read: + // its leaf is answered `$in: []` without asking anything, and it is in + // none of D5's classes. + if (!this._registry.getObject(parent)) continue; + const uncomposable = this.uncomposableReadableIdsParent(object, parent); + if (uncomposable?.class === 'object-scoped-read-middleware') { + throw refuse(site, `over '${parent}', whose rows an object-scoped read middleware narrows: that WHERE is ` + + `built at read time and is no part of its read scope, so the composed leaf would be wider than a read of '${parent}'`); + } + if (uncomposable?.class === 'other-datasource') { + throw refuse(site, `over '${parent}', which is bound to datasource '${String(uncomposable.parentDatasource)}' while '${object}' is ` + + `bound to '${String(uncomposable.objectDatasource)}': no driver evaluates a subquery across two datasources`); + } + composable.push(parent); + } + return composable; + } + +/** The body of `ObjectQL#resolveFormulaRelated`; the method's docblock there is its contract. */ +export async function resolveFormulaRelated(this: Pick, schema: any, plan: readonly FormulaPlanEntry[], rows: ReadonlyArray | undefined | null>, context: unknown, query?: EngineQueryOptions): Promise { + const wanted = new Map>(); + for (const entry of plan) { + if (entry.hopConflict) continue; + for (const [fk, named] of entry.hops ?? []) { + let set = wanted.get(fk); + if (!set) wanted.set(fk, (set = new Set())); + for (const name of named) set.add(name); + } + } + if (wanted.size === 0) return undefined; + if (query && this.formulaHopReads.has(query)) { + return `this object was itself read one hop through a reference by another object's formula, and a ` + + 'formula resolves ONE hop: a formula read this way does not read through a reference of its own ' + + '(a second hop). Store the value on this object as a field of its own to read it from there'; + } + // [#24410] The read set per reference is the UNION of every formula's + // columns, and the loader answers a whole reference `undeclared-field` when + // any column of it is undeclared — so one formula's misspelt column nulled + // every sibling reading through the same reference. The undeclared columns + // are taken out of the shared read here and judged per formula + // ({@link formulaHopScope}); the loader reads the declared rest. + const fields = (schema?.fields ?? {}) as Record; + const undeclared = new Map }>(); + for (const [fk, named] of wanted) { + const target = referenceTargetOf(fields[fk]); + // The loader's own verdicts stand for these: no target, or a hop into + // the stored-metadata-body family, which is never read. + if (!target || isStoredMetadataBodyObject(target)) continue; + const declared = (this._registry.getObject(target) as { fields?: Record } | undefined)?.fields; + const missing = undeclaredRelatedFields(declared, named); + if (missing.length === 0) continue; + undeclared.set(fk, { object: target, fields: new Set(missing) }); + for (const name of missing) named.delete(name); + if (named.size === 0) wanted.delete(fk); + } + const lookup = wanted.size > 0 + ? await this.resolveOneHopRelated(schema, wanted, rows, context, 'caller') + : () => undefined; + return { bindings: (row) => lookup(row), undeclared }; + } diff --git a/packages/objectql/src/engine-summaries-and-expansion.ts b/packages/objectql/src/engine-summaries-and-expansion.ts new file mode 100644 index 00000000000..11d64562504 --- /dev/null +++ b/packages/objectql/src/engine-summaries-and-expansion.ts @@ -0,0 +1,190 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Summary indexes, related-record expansion, file and reference hydration" + * section of `ObjectQL`: the bodies of `buildSummaryIndex`, + * `initializeSummaryFields` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class with its docblock and + * signature, so the published `dist/*.d.ts`, an instance spy and a subclass + * override are what they were; the docblock there is the contract. + * + * The bodies reach the class through {@link SummariesAndExpansionHost}: exactly the + * `ObjectQL` members they read, private ones included. Each method hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { referenceCarrierOf } from '@objectstack/spec/data'; +import type { ServiceObject } from '@objectstack/spec/data'; +import { summaryEmptySetValue } from './summary-aggregate.js'; +import type { SummaryDescriptor } from './summary-aggregate.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the functions below read — every one of them, and + * no other. Each is typed AS the class member (an indexed access on the + * class), so a change to a member's signature on the class is a change here + * too. `readonly` members are the getters. + */ +export interface SummariesAndExpansionHost { + readonly _registry: ObjectQL['_registry']; + getOwnedSummaryDescriptors: ObjectQL['getOwnedSummaryDescriptors']; + readonly logger: ObjectQL['logger']; +} + +/** The body of `ObjectQL#buildSummaryIndex`; the method's docblock there is its contract. */ +export function buildSummaryIndex(this: Pick): { + byChild: Map; + byParent: Map; + } { + const index = new Map(); + const byParent = new Map(); + const objects: ServiceObject[] = this._registry.getAllObjects(); + for (const parent of objects as any[]) { + const fields = parent?.fields; + if (!fields || typeof fields !== 'object' || Array.isArray(fields)) continue; + for (const [summaryField, def] of Object.entries(fields)) { + const d: any = def; + if (d?.type !== 'summary' || !d.summaryOperations) continue; + const so = d.summaryOperations; + const childObject = so.object; + const fn = so.function; + if (!childObject || !fn) continue; + // Resolve the FK on the child pointing back to this parent. + let fkField: string | undefined = so.relationshipField; + // [#19082] Candidate relation fields on the child whose `reference` + // carrier no reader can read, spelled `.`. Collected + // rather than thrown on — see the two comments below. + const unreadableCarriers: string[] = []; + if (!fkField) { + const child = this._registry.getObject(childObject) as any; + const cfields = child?.fields || {}; + for (const [cfName, cdef] of Object.entries(cfields)) { + const cd: any = cdef; + if (cd?.type !== 'master_detail' && cd?.type !== 'lookup') continue; + // [#19082] The carrier is read through the ONE arbiter, the same + // one {@link ObjectQL.planCascadeAtomicity} and + // {@link ObjectQL.cascadeDeleteRelations} were routed through — + // but its refusal is CAUGHT here instead of propagated, for a + // reason those two seams do not have: this is a SCAN over every + // relation field on the child, looking for the one that points + // back. Letting the first unreadable carrier throw would hide a + // perfectly readable sibling that IS the foreign key, turning a + // roll-up that works today into a hard failure of every write. + // The three answers stay distinct: + // + // - ABSENCE (`undefined` / `null` / `''`) is unchanged and still + // silent — `referenceCarrierOf` answers `undefined` for all + // three, this `continue` skips the field, and a field that + // names no target is a legal thing to declare. + // - A READABLE carrier compares exactly as `cd.reference === + // parent.name` compared it, so every roll-up that resolved + // before resolves now. + // - UNREADABILITY is recorded, and it is what the skip below + // stops being silent about. + let carrier: string | undefined; + try { + carrier = referenceCarrierOf(cd, 'ObjectQL.buildSummaryIndex'); + } catch { + unreadableCarriers.push(`${childObject}.${cfName}`); + continue; + } + if (carrier === parent.name) { + fkField = cfName; + break; + } + } + } + if (!fkField) { + // [#19082] The skip is unchanged; its SILENCE is what ends here. + // + // An unreadable carrier compares false against every name, so + // `fkField` stayed unset and this `continue` dropped a DECLARED + // roll-up out of both indexes with no diagnostic anywhere — the + // second way this one function invents "nothing to recompute" (the + // first, the registry read, is #9154 above). It is a durability + // degradation by the repo's own test: every write keeps reporting + // success while a persisted summary value silently stops tracking + // its children, so it is an `error`, not a `warn`, and it owes both + // the consequence and the fix in the line it prints. + // + // Said once per INDEX BUILD, never once per write: + // {@link ensureSummaryIndexes} memoises the result against the + // registry's `objectRevision`, which moves only on a metadata + // mutation. And only the unreadable case speaks — a roll-up whose + // child genuinely declares no relation field at all is a different + // (and much louder at authoring time) condition, left exactly as it + // was rather than widened into here. + if (unreadableCarriers.length > 0) { + const msg = + `[summary-index] ${parent.name}.${summaryField} is NOT indexed and will NOT recompute: ` + + `its roll-up over '${childObject}' has no resolvable foreign key, because ` + + `${unreadableCarriers.length} candidate relation field(s) on '${childObject}' carry a ` + + `\`reference\` no reader can read (${unreadableCarriers.join(', ')}). ` + + `\`FieldSchema.reference\` is an optional STRING — the target object's name — and a ` + + `non-string carrier is refused by \`ObjectSchema.safeParse\`, so such a definition ` + + `reached this registry around the parse seam (a raw \`registerObject\`, or a metadata ` + + `row stored before that tightening). CONSEQUENCE: ${parent.name}.${summaryField} keeps ` + + `whatever value it holds now through EVERY insert / update / delete of '${childObject}', ` + + `while every one of those writes reports success and nothing else reports this. FIX: ` + + `spell the carrier as the target object name (reference: '${parent.name}') on the ` + + `child's master_detail/lookup field, or name the foreign key explicitly with ` + + `summaryOperations.relationshipField on ${parent.name}.${summaryField}.`; + // Sanctioned logger shape (PR #9750): reach for `error`, fall back + // to `warn` — ⛔ never an optional call like `logger.error?.()`, + // which emits nothing against a sink that has no `error`. + if (typeof this.logger.error === 'function') this.logger.error(msg); + else this.logger.warn(msg); + } + continue; // can't resolve the relationship — skip + } + // Optional per-summary predicate: only child rows matching it are + // aggregated (e.g. sum receipts where { status: 'received' }). ANDed with + // the parent-FK match at recompute time. Ignore a non-object filter. + const filter = so.filter && typeof so.filter === 'object' && !Array.isArray(so.filter) + ? so.filter as Record + : undefined; + const descriptor: SummaryDescriptor = { + parentObject: parent.name, summaryField, childObject, fkField, fn, sourceField: so.field, filter, + }; + const list = index.get(childObject) ?? []; + list.push(descriptor); + index.set(childObject, list); + // Same descriptor, parent-side view. Only descriptors that made it this + // far are indexed either way, so "seeded at insert" and "maintained by + // recompute" are the same set by construction — a roll-up whose + // relationship could not be resolved (the `continue` above) is left + // untouched on both paths rather than seeded with a 0 nothing updates. + const owned = byParent.get(parent.name) ?? []; + owned.push(descriptor); + byParent.set(parent.name, owned); + } + } + return { byChild: index, byParent }; + } + +/** The body of `ObjectQL#initializeSummaryFields`; the method's docblock there is its contract. */ +export function initializeSummaryFields(this: Pick, object: string, record: any): any { + const descriptors = this.getOwnedSummaryDescriptors(object); + if (descriptors.length === 0) return record; + if (!record || typeof record !== 'object' || Array.isArray(record)) return record; + let out: Record = record; + for (const desc of descriptors) { + const seed = summaryEmptySetValue(desc.fn); + if (seed == null) continue; // min/max/avg — undefined on an empty set + if (out[desc.summaryField] != null) continue; // author supplied a value + if (out === record) out = { ...record }; + out[desc.summaryField] = seed; + } + return out; + } diff --git a/packages/objectql/src/engine-top-level.ts b/packages/objectql/src/engine-top-level.ts new file mode 100644 index 00000000000..86184945d2c --- /dev/null +++ b/packages/objectql/src/engine-top-level.ts @@ -0,0 +1,4976 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// [#29986 codemod] The top-level declarations ahead of `ObjectQL` in +// `engine.ts` that moved out whole by `scripts/codemod-split-engine.mjs` (lever +// (b) of ruling 6591753301). +// +// A MOVE, not a redesign: each declaration below is its `engine.ts` text +// exactly, with the comments above it, in the order it stood there. A LEAF: +// it imports neither `engine.ts` nor any `engine-*` module, so it is +// evaluated before the class and forms no cycle with it. `engine.ts` +// re-exports, by name, every declaration here it exported before, so every +// importer of `./engine.js` reads what it read. + +import { AUDIT_PROVENANCE_FIELDS, ENGINE_UPDATE_UPSERT_REMOVED, FILE_REFERENCE_TYPES, FullTextSearchSchema, QUERY_CURSOR_REMOVED, QUERY_DISTINCT_REMOVED, REFERENCE_VALUE_TYPES, RPC_QUERY_ALIAS_SLOTS, VALID_AST_OPERATORS, foldQueryAliasSlots, isFilterAST, isMultiValueField, lowerFilterCondition, normalizeFilterComparandTypes, parseFilterAST, readAutonumberCounter, referenceCarrierOf, referenceTargetOf, resolveInjectedColumnProvenance } from '@objectstack/spec/data'; +import type { DroppedFieldsEvent, FilterCondition, FilterLoweringOptions, HookContext, QueryAST, QueryAliasSlot, ServiceObject } from '@objectstack/spec/data'; +import type { EngineFilterJudgement, ISecurityService } from '@objectstack/spec/contracts'; +import { assertFilterIsMaterializable, assertListComparandShapes, collectComposableReadableIds, invalidFilterError } from './filter-comparand-shape.js'; +import { assertResolvedTemporalTokensInRange, assertTemporalComparandsInterpretable } from './temporal-comparand-door.js'; +import type { ResolvedTokenJudge } from './temporal-comparand-door.js'; +import { assertTextOperatorTargetsAreStringCapable } from './text-operator-declared-type-door.js'; +import { mapRelationConditions, narrowNumberComparands } from './number-comparand-declared-type-door.js'; +import { lowerRelationSite } from './relation-filter-lowering.js'; +import type { RelationFilterSite } from './relation-filter-lowering.js'; +import { SystemFieldName } from '@objectstack/spec/system'; +import { judgeCollectionItemOwnerStamp } from '@objectstack/spec/kernel'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import type { FlowFunctionEffect } from '@objectstack/spec/automation'; +import { filterTokenContextFrom, resolveFilterTokens } from '@objectstack/core'; +import type { IDataDriver } from '@objectstack/core'; +import { ORGANIZATION_OBJECT } from './tenancy/system-write-organization.js'; +import { judgePermissionSetOwnerBinding } from '@objectstack/spec/security'; +import type { TenantLayer0Verdict } from '@objectstack/spec/security'; +import { applyProtection, pluralToSingular } from '@objectstack/spec/shared'; +import type { SchemaRegistry } from './registry.js'; +import { SEARCHABLE_ENUM_TYPES, SEARCHABLE_TEXTUAL_TYPES } from './search-filter.js'; +import { ExpressionEngine, analyzeRelationshipTraversals, isUnparseableDate } from '@objectstack/formula'; +import type { EvalError, EvalOptionLabels, EvalPermissions } from '@objectstack/formula'; +import { declaredOptionLabels } from './formula-option-labels.js'; +import { ASSEMBLED_VIEW_ITEMS_KEY, AssembledViewArtifactSchema, isAggregatedViewContainer, isViewContainerShaped } from '@objectstack/spec'; +import type { Expression } from '@objectstack/spec'; +import { deriveViewContainerObject } from '@objectstack/metadata/view-container'; +import { viewContainerNameRefusal } from './view-container-name-refusal.js'; +import { ValidationError } from './validation/record-validator.js'; +import type { FieldValidationError } from './validation/record-validator.js'; +import { collectReferenceTraversals, referenceTraversalConflicts, staticReadonlyInsertSubject, stripReadonlyFields } from './validation/rule-validator.js'; +import type { RelatedFieldBinding, RelatedRecordBinding } from './validation/rule-validator.js'; +import { declaredColumnSet } from './declared-read-columns.js'; + +/** + * One read of a `sys_migration` flag row: the verdict, plus whether the ledger + * could actually be consulted to reach it (#4769). Both negatives keep the + * gate closed; only the conclusive one is worth remembering. + */ +export interface MigrationFlagRead { + verified: boolean; + conclusive: boolean; + /** + * [#15989] Have the file-family COLUMNS on this deployment moved to the + * bare-id encoding — `hasMovedFileColumns` over the same row? + * + * Carried on the SAME read rather than fetched by a second one, because it + * is a second question about one row and two reads could answer them out of + * one another's date. It is `false` on every row that lacks the stamp, on an + * unverified row, and on every way of not having read a row at all. + */ + columnsMoved: boolean; +} + +/** + * What this boot has ADMITTED against one ADR-0104 migration's contract — + * the counterexample that forbids attesting it (#4769). `count` is a lower + * bound (every admitted value is counted, but a revocation records the total + * known at the moment it wrote); `first` is the one an operator is shown, + * because a prescription needs a place to start, not a census. + */ +export interface AdmittedValueShapeViolationTally { + count: number; + first: { object: string; field: string; type: string; detail: string }; +} + +/** + * The lifecycle events the engine actually dispatches via `triggerHooks`. This + * is the single source of truth for what a hook can subscribe to — kept in + * lockstep with the `triggerHooks(...)` call sites and with `HookEvent` in the + * spec. `beforeFind`/`afterFind` cover both `find` and `findOne`; the write + * events cover both single-id and bulk (`multi: true`) writes (#3195). A hook + * subscribing to anything outside this set would silently never fire, so + * `registerHook` refuses it where the name is one of the engine's own + * ({@link UNDISPATCHED_LIFECYCLE_HOOK_EVENTS}, #17713) and warns otherwise. + * + * ## WHEN `after*` fires, relative to the commit (#7477) + * + * `afterInsert`/`afterUpdate`/`afterDelete` are dispatched INSIDE the unit of + * work, before the enclosing transaction (if any) commits. The declared + * meaning is **"the write has been requested and will happen unless this unit + * of work is undone"** — not "the write happened". This is the ruled semantics + * (#7477, 2026-08-11), not an accident of the current call sites: an `after*` + * dispatch is deliberately NOT deferred to commit, because deferring it would + * push a handler's own `ctx.api` writes outside the transaction the write ran + * in, and an in-engine audit hook depends on landing inside it. + * + * Four ordinary paths open such a unit around the dispatch: + * - the write door itself, when the object has a metadata-bound `after*` + * hook that can abort — blocking with `onError` `abort`, or carrying a + * `condition` — so that abort rolls the row back as `HookSchema.onError` + * says (#22782, #22856, `opensAfterAbortUnit`); + * - a by-id {@link ObjectQL.delete} whose cascade is `'atomic'` — each + * dependent's own `afterDelete` fires inside the wrap the parent opened, + * and the parent's row removal can still refuse afterwards (#7413). The + * PARENT's `afterDelete` is outside that wrap by construction (inside the + * door's own unit when the first path opened one); the cascaded + * CHILDREN's are not; + * - `runAtomicBatch` in `@objectstack/metadata-protocol` — + * `batchData`/`deleteManyData` with `atomic: true` runs every member's + * `after*` inside one transaction that aborts on the first failure + * (#4620); + * - any caller that opened `transaction()` / `ctx.api.transaction()` around + * the write itself. + * + * A rollback on any of those leaves a hook that fired for a row that still + * exists. Effects routed back through this engine roll back with it and are + * therefore safe; effects that leave the engine — webhooks, notifications, + * external index updates, file deletion — are the HANDLER's responsibility to + * make rollback-tolerant (idempotent and reconcilable, or re-checked against + * the row by a worker rather than trusted from the event alone). Documented + * for authors on `HookEvent` in `@objectstack/spec/data` and in + * `content/docs/automation/hooks.mdx`. + */ +const DISPATCHABLE_HOOK_EVENTS: ReadonlySet = new Set([ + 'beforeFind', 'afterFind', + 'beforeInsert', 'afterInsert', + 'beforeUpdate', 'afterUpdate', + 'beforeDelete', 'afterDelete', +]); + +/** + * [#17713] Every verb the engine can put on an {@link OperationContext}, as + * DATA rather than as a second hand-written list. + * + * The type annotation is the weld: `Record` is exhaustive in both + * directions, so a new engine method that widens `OperationContext['operation']` + * fails to compile here until the verb is added, and a verb dropped from the + * union makes its entry an excess property. `pnpm typecheck` compiles this file + * (it is `src`, not the test layer), so the weld is enforced by the package's + * own type check and not only by a test. + * + * It exists so {@link UNDISPATCHED_LIFECYCLE_HOOK_EVENTS} below is DERIVED. A + * hand-written list of "the events we refuse" would be this card's own defect + * one layer up: it would go stale the moment a verb is added, and silently. + */ +const ENGINE_OPERATION_VERBS: Record = { + find: true, + findOne: true, + insert: true, + update: true, + delete: true, + count: true, + aggregate: true, +}; + +/** + * [#17713] The engine's own lifecycle-event NAMESPACE — `before`/`after` × every + * verb in {@link ENGINE_OPERATION_VERBS}. Fourteen names, of which + * {@link DISPATCHABLE_HOOK_EVENTS} is the eight the engine actually dispatches. + */ +const ENGINE_LIFECYCLE_HOOK_EVENTS: ReadonlySet = new Set( + Object.keys(ENGINE_OPERATION_VERBS).flatMap((verb) => { + const suffix = `${verb.charAt(0).toUpperCase()}${verb.slice(1)}`; + return [`before${suffix}`, `after${suffix}`]; + }), +); + +/** + * [#17713] The GAP: a name inside the engine's own lifecycle namespace that the + * engine never dispatches. Derived, never typed out — the difference of the two + * sets above. + * + * Today it is exactly six: `beforeFindOne` / `afterFindOne`, `beforeCount` / + * `afterCount`, `beforeAggregate` / `afterAggregate`. Registering one of them is + * REFUSED at the door ({@link assertDispatchableHookEvent}) rather than warned + * about, because each is a name an author reaches for while believing they are + * subscribing to an engine lifecycle event — and what they get back is an inert + * declaration (ADR-0078: no silently inert declaration), which on a read filter + * means a guardrail that was never armed. + * + * ⚠️ The refusal is deliberately scoped to THIS set, not to "anything outside + * `DISPATCHABLE_HOOK_EVENTS`". A name outside the namespace entirely + * (`'myPlugin:flush'`) is a plugin dispatching its own event through the public + * {@link ObjectQL.triggerHooks} — the legitimate reading that made this branch a + * warn in the first place (#3195). That reading survives untouched; what stops + * being accepted is a near-miss spelling of the engine's OWN vocabulary, where + * no such reading exists. + */ +const UNDISPATCHED_LIFECYCLE_HOOK_EVENTS: ReadonlySet = new Set( + [...ENGINE_LIFECYCLE_HOOK_EVENTS].filter((event) => !DISPATCHABLE_HOOK_EVENTS.has(event)), +); + +/** + * [#17713] Test-visible view of the three sets the refusal is computed from, so + * a pin can drive its population FROM THE ENGINE instead of restating it — the + * acceptance criterion this card wrote ("a test per event name the engine + * exposes but never dispatches") is only meaningful if the population is read, + * not typed. + * + * Module-level export, deliberately NOT added to the package barrel + * (`src/index.ts` names its exports one by one), so this widens no published + * surface. + */ +export const HOOK_EVENT_DISPATCH_VOCABULARY = { + dispatchable: DISPATCHABLE_HOOK_EVENTS, + lifecycleNamespace: ENGINE_LIFECYCLE_HOOK_EVENTS, + undispatchedLifecycle: UNDISPATCHED_LIFECYCLE_HOOK_EVENTS, +} as const; + +/** + * [#4346] The alias slots the ENGINE option bags still admit, cut from the + * spec's own table (#3795) so the engine never re-declares a mapping. + * + * The deprecated `DataEngine{Query,Update,Delete,Count,Aggregate}Options` + * contracts declare `filter` on every read AND write method, but only `find` + * folded it — `findOne`/`count`/`update`/`delete`/`aggregate` passed the bag + * through with `where === undefined`, which every driver reads as "no + * predicate": a caller filtering with `{ filter }` silently matched EVERY row + * (an over-grant on the reads, an unbounded write on `update`/`delete`). + * + * `where` is the slot every method folds; `limit` additionally applies to the + * find-shaped bags, which declare `top` (OData) as its alias. The other four + * pairs in the table are wire spellings folded at parse by + * `QueryWithTransportSchema` / the protocol normalizer — their values need shape + * lowering (`sort` records, `populate` lists) that belongs to those layers, so + * the engine deliberately does not fold them. + */ +export const ENGINE_WHERE_SLOTS: readonly QueryAliasSlot[] = + RPC_QUERY_ALIAS_SLOTS.filter((slot) => slot.canonical === 'where'); +const ENGINE_QUERY_SLOTS: readonly QueryAliasSlot[] = + RPC_QUERY_ALIAS_SLOTS.filter((slot) => slot.canonical === 'where' || slot.canonical === 'limit'); + +/** + * [#4371] The slots the engine does NOT fold — the wire-only pairs + * (`select`→`fields`, `sort`→`orderBy`, `skip`→`offset`, `populate`→`expand`), + * derived as the complement of {@link ENGINE_QUERY_SLOTS} so a seventh pair + * added to the spec table lands on exactly one side of the split. + * + * Their values need shape lowering (`sort`'s `{field: 'asc'}` record form, + * `populate`'s name list) that belongs to the RPC/protocol layers, so folding + * here would re-implement the lowering per reader — the #3795 condition. But a + * DIRECT engine call never crosses those layers: the alias key used to ride + * the AST verbatim, drivers read only the canonical name, and the parameter + * was silently dropped — three shipped "latest N in arbitrary order" bugs + * (#4370) plus the engine's own `seedAutonumber`. Declared ≠ enforced + * (AGENTS.md PD #10): the find-shaped entry points now REJECT these spellings, + * naming the canonical key and shape, so the mistake throws at the call site + * instead of degrading the result. + * + * Deliberately NOT applied to the where-only methods + * (`update`/`delete`/`count`/`aggregate`): their contracts honour no + * sort/projection/pagination at all, so "pass `orderBy` instead" would + * redirect the caller to a key those methods silently ignore too. Unknown-key + * enforcement for those bags is #4371's option (2), scoped separately. + */ +const ENGINE_WIRE_ONLY_SLOTS: readonly QueryAliasSlot[] = + RPC_QUERY_ALIAS_SLOTS.filter((slot) => !ENGINE_QUERY_SLOTS.includes(slot)); + +/** + * Canonical shape each wire-only slot's value must be rewritten into — quoted + * by the rejection so the error carries the full migration, not just the key + * rename (the value shapes differ; that is WHY the engine cannot fold them). + */ +const WIRE_ONLY_CANONICAL_SHAPES: Record = { + fields: "a string[] of field names", + orderBy: "SortNode[]: [{ field, order: 'asc' | 'desc' }]", + offset: 'a number of rows to skip', + expand: 'a record of { relationName: QueryAST }', +}; + +/** + * [#4371 option 2] The driver-option keys the engine forwards verbatim: on + * `find`/`findOne`/`update`/`delete` the option bag IS the base of the driver + * options (`buildDriverOptions(object, ctx, bag)`), which is how a caller's + * explicit `tenantId` / `bypassTenantAudit` reaches the driver (pinned in + * engine.test.ts). `count`/`aggregate` never forward the bag, so these keys + * are deliberately NOT legal there — accepting them would be the exact + * silently-ignored contract this gate exists to close. + */ +const ENGINE_DRIVER_PASSTHROUGH_KEYS = [ + 'transaction', 'tenantId', 'tenantIds', 'timezone', 'bypassTenantAudit', 'preserveAudit', +] as const; + +/** + * [#4371 option 2] Per-method legal option keys. An option bag key outside + * the method's set is REJECTED at the entry point: the engine executes none + * of them, so the call would otherwise succeed with the option silently + * ignored — the `declared ≠ enforced` shape (PD #10) one layer below the + * wire-alias rejection above. + * + * Sources, in order: the method's `Engine*OptionsSchema` declared keys (minus + * the `retiredKey` tombstones `cursor`/`distinct`/`upsert` (#8057), which get + * their tombstone quoted instead of a generic rejection — the schema keeps them + * ONLY to carry that message, and this runtime path never parses); `searchFields` (read by + * `find` and `aggregate` at the `$search` expansion, sent by the protocol layer); + * `onFieldsDropped` and `strictReadonlyWrites` (`WriteObservabilityOptions` — + * contract-declared, deliberately outside the serializable Zod schema: the + * first because a function is unrepresentable in JSON Schema, the second + * because #5126 ruled that a write-refusal switch must not be settable from a + * wire body); and the driver pass-through keys above. The alias spellings + * (`filter`/`top`) are folded and deleted BEFORE this check runs, so they + * never reach it. + * + * A drift pin in engine-unknown-option.test.ts asserts each set equals its + * schema's shape (minus tombstones, plus the documented extras) so a key + * added to the spec cannot be silently rejected here. + */ +const ENGINE_FIND_OPTION_KEYS: ReadonlySet = new Set([ + 'context', 'where', 'fields', 'orderBy', 'limit', 'offset', + 'search', 'searchFields', 'expand', + ...ENGINE_DRIVER_PASSTHROUGH_KEYS, +]); +const ENGINE_UPDATE_OPTION_KEYS: ReadonlySet = new Set([ + 'context', 'where', 'multi', 'returning', 'onFieldsDropped', 'onValidationAdvisory', 'onMentionsWithheld', 'onNotice', 'strictReadonlyWrites', + ...ENGINE_DRIVER_PASSTHROUGH_KEYS, +]); +const ENGINE_DELETE_OPTION_KEYS: ReadonlySet = new Set([ + 'context', 'where', 'multi', + ...ENGINE_DRIVER_PASSTHROUGH_KEYS, +]); +// `search` / `searchFields` are READ by `count` and `aggregate` through the +// same ADR-0061 expander `find` uses ({@link ObjectQL.expandSearchOnOptions}), +// so a count or a grouped answer under a search is over the searched rows. +// Before their schemas declared them they were refused here, and the wire paths +// left them out: the unsearched groups answered, and a paged list's `total` +// under a search was a page-local estimate (#22790). +export const ENGINE_COUNT_OPTION_KEYS: ReadonlySet = new Set(['context', 'where', 'search', 'searchFields']); +const ENGINE_AGGREGATE_OPTION_KEYS: ReadonlySet = new Set([ + 'context', 'where', 'groupBy', 'aggregations', 'having', 'timezone', + 'search', 'searchFields', +]); + +/** + * [#27732] A search-field override — `searchFields`, or the object-form + * `search.fields` — that names no field is refused `400 VALIDATION_ERROR`: + * an empty list, or a comma string with no name in it (the `GET` door's + * transport spelling, which reaches the engine as sent). Absent (or `null`) + * still searches the object's default search fields. + * + * `resolveSearchFields` reads an empty override as absent, so the search + * widened to the whole default set: the caller wrote the key to narrow and was + * not told. The door refuses this set (`assertSearchFieldsAreSearchable`, + * #27484) and `QuerySchema.searchFields` / `FullTextSearchSchema.fields` + * declare the same refusal (#27683); the words are read off that declaration + * rather than copied, and the status and code are the door's, so + * `mapDataError` answers an in-process caller and a REST caller alike. Any + * other value is left to the expansion's existing tolerance. + */ +function refuseSearchOverrideNamingNoField(object: string, requested: unknown, param: string): void { + const names = typeof requested === 'string' + ? requested.split(',').map((s) => s.trim()).filter(Boolean) + : Array.isArray(requested) ? requested : undefined; + if (names === undefined || names.length > 0) return; + const declared = FullTextSearchSchema.shape.fields.safeParse([]).error?.issues[0]?.message; + const err: any = new Error( + `ObjectQL search on '${object}' names no field in '${param}': got ${JSON.stringify(requested)}. The query was not run: ` + + "an empty search-field list was read as no narrowing, which scans the object's default search fields. " + + `${declared}.`, + ); + err.status = 400; + err.code = 'VALIDATION_ERROR'; + err.param = param; + err.object = object; + throw err; +} + +/** + * [#28120] …and an override that names no field the search can scan is refused + * `400 INVALID_FIELD`, the code the door answers the same value with + * (`assertSearchFieldsAreSearchable`, #4254): every name unknown, unsearchable, + * or withheld from the scan (`internal: true`, credential types), or no entry a + * field name at all. + * + * `resolveSearchFields` drops the names outside the allowed set and, when none + * is left, falls back to the whole set — the typo or the renamed field the + * caller wrote to narrow the search ran over the object's default fields + * instead, untold. Judged on the set the expansion resolves over (`allowed`), + * so this refuses exactly the overrides that fallback would have widened. An + * override naming at least one allowed field is left to the intersection (the + * engine's tolerance for the other names stands), and an empty `allowed` is + * left to the expansion's own refusal (#24284), which names that cause. + */ +function refuseSearchOverrideNamingNoSearchableField( + object: string, + requested: unknown, + param: string, + allowed: readonly string[], +): void { + const entries = typeof requested === 'string' + ? requested.split(',').map((s) => s.trim()).filter(Boolean) + : Array.isArray(requested) ? requested : undefined; + if (entries === undefined || entries.length === 0 || allowed.length === 0) return; + const allowSet = new Set(allowed); + if (entries.some((f) => typeof f === 'string' && allowSet.has(f))) return; + const names = entries.filter((f): f is string => typeof f === 'string'); + const err: any = new Error( + `ObjectQL search on '${object}' names no searchable field in '${param}': got ${JSON.stringify(requested)}. ` + + 'The query was not run: an override naming none of the fields the search scans was dropped, which ' + + "scans the object's default search fields. Name a searchable field, or leave the override out to " + + 'search the default fields.', + ); + err.status = 400; + err.httpStatus = 400; + err.code = 'INVALID_FIELD'; + if (names.length > 0) { + err.field = names[0]; + err.fields = names; + } + err.param = param; + err.object = object; + throw err; +} + +/** + * Rows per page for the autonumber seeding scan (#6249). This is a PAGE size, + * not a cap: the walk pages until the scope is exhausted. The number is the one + * the old single-shot `limit: 5000` used, kept so the per-read cost against a + * driver is unchanged — what changed is that reaching it no longer ends the + * scan and truncates the max. + */ +const AUTONUMBER_SEED_PAGE_SIZE = 5000; + +/** + * How many times one insert may re-seed and re-issue after a unique-constraint + * collision on an engine-issued autonumber (#6806) before it refuses. + * + * Bounded on purpose. The FIRST re-issue is the one that matters: the collision + * proves the in-memory counter sits below the store's real max, and the re-seed + * that follows reads that max back, so attempt 2 is issued from the truth. A + * further collision means another writer took the number in between — real + * concurrency, which more spinning does not resolve (each attempt costs a full + * scope scan). Two spare attempts absorb a burst; past that the write fails + * loudly rather than looping against a live competitor. + */ +const AUTONUMBER_COLLISION_ATTEMPTS = 3; + +/** + * One autonumber the ENGINE issued on a row, paired with the counter it came + * from (#6806). Only engine-issued values are listed: a value an exempt writer + * supplied is the caller's, and a collision on it is the caller's to see. + */ +interface IssuedAutonumber { + /** Field the value was written to. */ + readonly field: string; + /** `object.field.` key in {@link ObjectQL.autonumberCounters}. */ + readonly counterKey: string; +} +/** + * [#17052] The write doors' failure META, assembled at the call site because + * the level moved off `error`. + * + * ## Why the level moved + * + * `insert` / `update` / `delete` each end in a `catch` whose next statement is + * `throw e`. AGENTS.md → *Degradation log levels* calls that shape out by name: + * "a failure handed to the CALLER is not a degradation at all … Do not bolt a + * `logger.error` onto such a site". The requester IS told — the throw is the + * answer — so an `error` line is a second, louder report of a fact the caller + * already has, and nothing downstream can tell it apart from a real one. The + * measured cost was a healthy first boot of every fresh project printing + * `ERROR Insert operation failed` while `@better-auth/oauth-provider` seeded + * `sys_oauth_resource` and treated the UNIQUE collision as the no-op its own + * docblock declares it to be. + * + * ## Why the diagnosis needs this helper to survive the move + * + * The `Logger` contract (`packages/spec/src/contracts/logger.ts`) gives an + * `Error` slot to `error` and `fatal` ONLY: `warn(message, meta?)` has two + * parameters, not three. Handing the redacted driver error to `warn` as `meta` + * does not merely lose the slot — `Error.message` and `Error.stack` are + * NON-ENUMERABLE, so every logger that renders meta by spreading it would + * serialize `{}` and the entry would arrive carrying nothing at all. That is + * exactly the loss #14095 refused (the failing column, MySQL's index name, the + * driver's own frames) arriving one level down. + * + * So the two shaped-by-ruling properties are preserved by building the meta the + * way `ObjectLogger.write()` builds it from the slot — `{ …meta, error: { + * message, stack } }` — rather than by handing over the Error: + * + * - **#8682** the value redaction still runs: callers pass + * `redactBoundStatement(...)`, and it is that result's `message`/`stack` + * that are read here. + * - **#14095** the log still takes the driver's error (the envelope's + * `cause`), not the envelope: callers still choose which value to hand in. + * + * The non-Error branch mirrors `ObjectLogger.writeErrorLike()`'s own fallback + * (`{ ...errorOrMeta, ...meta }`, the later argument winning), so a thrown + * non-Error — or a `DuplicateRecordError` with no `cause` — renders as it did. + * + * @param logged The value to report — already redacted by the caller. + * @param meta The entry's own meta (`object`, and `developerMessage` on the + * delete door). + */ +export function writeFailureLogMeta( + logged: unknown, + meta: Record, +): Record { + return logged instanceof Error + ? { ...meta, error: { message: logged.message, stack: logged.stack } } + : { ...(logged as Record | undefined), ...meta }; +} + +/** + * [#24948] The refusal of a predicate (`multi`) delete of `sys_organization`: + * the standard `DELETE_RESTRICTED`, saying to delete by id. An organization is + * deleted one at a time, through plugin-auth's organization delete: its + * refusal of an organization that still owns records and its + * `beforeDeleteOrganization` clean-up of the organization's teams, team members + * and admin grants run there, once per organization. A delete matching + * organizations by a filter runs neither, and no caller needs one. + */ +function organizationPredicateDeleteRefusal(): Error { + const err: any = new Error( + 'Organizations are deleted one at a time, by id: a delete matching organizations by a filter is refused. ' + + 'Delete each organization by its id.', + ); + err.developerMessage = + `A predicate delete of ${ORGANIZATION_OBJECT} is refused: organizations are deleted one at a time, because ` + + "plugin-auth's organization delete refuses an organization that still owns records and removes its teams, " + + 'team members and admin grants per organization, and a predicate delete runs neither. Delete each organization ' + + `by id (engine.delete('${ORGANIZATION_OBJECT}', { where: { id } })).`; + err.code = 'DELETE_RESTRICTED'; + err.status = 409; + err.httpStatus = 409; + err.object = ORGANIZATION_OBJECT; + return err; +} + +/** + * Read the counter out of ONE stored autonumber value, under #6468's anchoring + * rules. Shared by the seeding scan and by the adopt-on-exempt-write resync + * (#6806) so both readings can never drift apart — a divergence here is a + * duplicate record number, which is the harm the whole family is about. + * + * The ANCHORED reading itself is NOT this package's (#6560): it is the inverse + * of `renderAutonumber`'s composition, so it lives beside it as spec's + * {@link readAutonumberCounter}, which the SQL driver's `scanMaxNumericTail` + * calls over the same two strings. This function adds only the piece the two + * sides genuinely do not share: + * + * - **Either `prefix` or `suffix` declared ⇒ ANCHORED**: spec answers, and its + * TSDoc carries the rationale (counter after the prefix, suffix stripped + * when it matches and never required to match, out-of-scope values read as + * `undefined`). + * - **Neither declared ⇒ UNANCHORED**: this engine's own legacy reading — the + * LAST digit run of the whole value — kept byte-for-byte by PR #6553. The + * SQL driver's legacy reading of the same case is deliberately DIFFERENT (it + * concatenates every digit), which is exactly why spec refuses to answer for + * an unanchored slot instead of picking one of the two. + * + * # The unanchored arm is IMPLEMENTATION DETAIL, outside the declared contract (#7287) + * + * Spec's refusal above is no longer just an absence of agreement: the #7287 + * ruling (2026-08-10, 「宣告边界」) DECLARES that a stored value carrying + * non-digit content on an unanchored format is out of contract for counter + * readback, and that `readAutonumberCounter`'s `undefined` for that slot is the + * contract rather than a gap. The boundary and its rationale live in that + * function's TSDoc in `packages/spec/src/data/autonumber-format.ts`; the pins + * are `packages/spec/src/data/autonumber-unanchored-boundary.test.ts`. + * + * So the last-digit-run reading below is THIS ENGINE'S behavior, not a promise + * the platform makes. On the inputs the contract admits — pure-digit values, + * which is what `renderAutonumber` emits for an unanchored format — it answers + * the same number the SQL driver does. On mixed-content values it may answer + * differently (`'SO-2024-0007'` → `7` here, `20240007` there), and that + * difference is out of contract on both sides, not a defect on either. + * + * Consequently the ruling moves nothing here: neither this reading nor the + * driver's was hoisted into the shared helper, precisely because doing so would + * move live behavior on the other side over record numbers already issued. + * + * The unanchored branch uses the linear `/\d+/g` — a backtracking lookahead here + * is a polynomial-ReDoS sink on stored values full of zeros (CodeQL + * js/polynomial-redos). + */ +function readStoredAutonumberCounter(value: string, prefix: string, suffix: string): number | undefined { + if (prefix !== '' || suffix !== '') return readAutonumberCounter(value, prefix, suffix); + const runs = value.match(/\d+/g); + const digits = runs ? runs[runs.length - 1] : undefined; + if (!digits) return undefined; + const n = parseInt(digits, 10); + return Number.isFinite(n) ? n : undefined; +} + +/** Tombstoned option keys: rejected with the spec's own removal notice. */ +const ENGINE_RETIRED_OPTION_MESSAGES: Record = { + cursor: QUERY_CURSOR_REMOVED, + distinct: QUERY_DISTINCT_REMOVED, + // [#8057] `update.options.upsert` — declared-but-unenforced (ADR-0049): the + // key sat on the update allowlist while nothing read it, so `{ upsert: true }` + // was accepted and silently dropped. Removed rather than implemented; the + // spec's tombstone carries the create-if-absent prescription (#7867's + // not-found gate is the semantics a caller must reconcile with). + upsert: ENGINE_UPDATE_UPSERT_REMOVED, +}; + +/** + * The per-method legal key sets, exported for the drift pin ONLY + * (engine-unknown-option.test.ts asserts each set against its schema's shape, + * so a key added to the spec cannot be silently rejected here). Not a public + * API surface — consumers pass options, they do not read this table. + */ +export const ENGINE_OPTION_KEY_SETS: Readonly>> = { + find: ENGINE_FIND_OPTION_KEYS, + findOne: ENGINE_FIND_OPTION_KEYS, + update: ENGINE_UPDATE_OPTION_KEYS, + delete: ENGINE_DELETE_OPTION_KEYS, + count: ENGINE_COUNT_OPTION_KEYS, + aggregate: ENGINE_AGGREGATE_OPTION_KEYS, +}; + +/** + * Reject option-bag keys the engine does not execute (#4371 option 2). + * + * Runs AFTER `foldEngineOptionAliases`, so alias spellings are already folded + * away (or thrown on). `null`-valued keys pass — a `null` is a withdrawal + * carrying no intent a drop could lose, same rule as the fold. Retired keys + * (`cursor`/`distinct`) quote their tombstone. Everything else gets the legal + * key set, so the error carries the fix. + */ +export function rejectUnknownEngineOptions( + object: string, + operation: string, + bag: object | undefined, + legal: ReadonlySet, +): void { + if (!bag) return; + let unknown: string[] | undefined; + for (const [key, value] of Object.entries(bag)) { + if (value == null || legal.has(key)) continue; + (unknown ??= []).push(key); + } + if (!unknown) return; + const details = unknown.map((k) => + ENGINE_RETIRED_OPTION_MESSAGES[k] ? `'${k}': ${ENGINE_RETIRED_OPTION_MESSAGES[k]}` : `'${k}'`, + ); + throw new Error( + `${operation}('${object}') does not recognise option${unknown.length > 1 ? 's' : ''} ` + + `${details.join('; ')}. The engine executes none of ${unknown.length > 1 ? 'them' : 'it'}, ` + + `so the call would succeed with the option silently ignored. ` + + `Legal keys for ${operation}: ${[...legal].sort().join(', ')}.`, + ); +} + +/** + * Fold the deprecated alias spellings of an engine option bag into their + * canonical QueryAST keys, under the #3795/#4181 rule: an alias alone moves to + * the canonical key, redundant identical spellings collapse, DIFFERENT values + * for one slot are irreconcilable and throw (picking a winner IS the silent + * drop), and an explicit `null` alias is a withdrawal. + * + * `rejectSlots` ({@link ENGINE_WIRE_ONLY_SLOTS}) names the slots whose alias + * spellings the engine can neither fold nor honour: a non-null value under one + * throws, quoting the canonical key and shape (#4371). `null` stays a + * withdrawal here too — it carries no intent a drop could lose — and rides + * through for drivers to ignore, exactly as before. + * + * Returns the SAME reference when no alias spelling is present (the common + * path allocates nothing — `withResolvedWhere` discipline); otherwise folds a + * shallow copy, because the bag belongs to the caller and may be reused (view + * metadata, flow node config). + */ +export function foldEngineOptionAliases( + object: string, + operation: string, + bag: T, + slots: readonly QueryAliasSlot[], + rejectSlots?: readonly QueryAliasSlot[], +): T { + if (!bag) return bag; + if (rejectSlots) { + const refused = rejectSlots.flatMap((slot) => + slot.aliases + .filter((alias) => (bag as Record)[alias] != null) + .map((alias) => ({ alias, canonical: slot.canonical })), + ); + if (refused.length > 0) { + throw new Error( + `${operation}('${object}') does not accept ` + + `${refused.map((r) => `'${r.alias}'`).join(', ')}: ` + + refused + .map( + (r) => + `'${r.alias}' is a wire spelling of '${r.canonical}', folded by the RPC/protocol ` + + `layer — a direct engine call bypasses that fold, so the value would be silently ` + + `dropped, not applied. Pass '${r.canonical}' ` + + `(${WIRE_ONLY_CANONICAL_SHAPES[r.canonical] ?? 'the canonical QueryAST shape'}) instead.`, + ) + .join(' '), + ); + } + } + if (!slots.some((slot) => slot.aliases.some((alias) => alias in bag))) return bag; + const folded: Record = { ...bag }; + foldQueryAliasSlots(folded, slots, (conflict) => { + throw new Error( + `Conflicting options on ${operation}('${object}'): ` + + `${conflict.spellings.map((s) => `'${s}'`).join(', ')} are spellings of the same ` + + `parameter (canonical '${conflict.canonical}') and were given different values. ` + + 'Send exactly one.', + ); + }); + return folded as T; +} + +/** + * [#20121] Is this non-array `where` a filter OBJECT — the one non-array shape + * a driver reads as a predicate? + * + * The test is the object's built-in tag (`[object Object]`), not its + * prototype, and each edge shape lands where its measured answer puts it — + * nothing that answered correctly before this gate is refused by it: + * + * - a literal `{…}` and `{}` (match-all, a REAL query — the unscoped-write + * detector reads it that way too) → accepted; + * - `Object.create(null)` and an instance of an author's class with the filter + * on its own keys → accepted: both filtered correctly on `driver-memory` and + * `SqlDriver`, because a driver reads a filter's own enumerable keys; + * - `Map`, `Set`, `Date`, a boxed primitive, a `Promise` (an un-awaited filter + * builder) → refused: each carries its tag, has no own keys, and was + * silently read as "no filter". + * + * `undefined` and `null` never reach this test — the caller skips them, and + * both keep meaning "no filter" (`findOne`'s no-predicate guard and the + * unscoped-write detector both read `null` as absent). + */ +function isWhereFilterObject(where: unknown): where is Record { + return ( + typeof where === 'object' + && where !== null + && !Array.isArray(where) + && Object.prototype.toString.call(where) === '[object Object]' + ); +} + +/** + * [#20121] How a refused non-filter `where` is named in the refusal: the + * primitive's `typeof` plus its value (bounded), or the object's built-in tag. + * The `typeof` spelling is the wire door's (`received number`); the value is + * added because a direct caller has no request to look at. + */ +function describeNonFilterWhere(where: unknown): string { + if (typeof where === 'object' && where !== null) { + return Object.prototype.toString.call(where).slice('[object '.length, -1); + } + if (typeof where === 'function') return 'function'; + const text = typeof where === 'string' ? JSON.stringify(where) : String(where); + const bounded = text.length > 80 ? `${text.slice(0, 77)}...` : text; + return `${typeof where} ${bounded}`; +} + +/** + * [#20121] What a dropped `where` would have done, per verb. The read wording is + * the wire door's verbatim; a write has no result set, and "every row in + * scope" is what an unapplied predicate on `multi: true` rewrote or removed. + */ +const UNAPPLIED_WHERE_CONSEQUENCE: Readonly> = { + find: 'returned the unfiltered result set', + findOne: 'returned the unfiltered result set', + count: 'returned the unfiltered result set', + aggregate: 'returned the unfiltered result set', + update: 'updated every row in scope', + delete: 'deleted every row in scope', +}; + +/** + * **Door 2** — lower an arriving {@link FilterArray} on `where` to the + * `FilterCondition` the AST actually declares (#5158, maintainer ruling C). + * + * `FilterArray` — `['stage', '=', 'won']`, `['and', […], […]]`, `[[…], […]]` — + * is authoring sugar, and since #5285 the spec says so in as many words: it is + * declared INPUT-ONLY (`data/filter.zod.ts`), and `QuerySchema.where` is a + * `FilterCondition` that deliberately excludes it. There were two doors into + * the runtime and only one of them read the contract that way: + * + * - **Door 1**, the protocol/HTTP face (`metadata-protocol` `protocol.ts`), + * has always run `isFilterAST` → `parseFilterAST` and answered `400 + * INVALID_FILTER` for an array it could not lower. Nothing array-shaped + * survives it. + * - **Door 2**, a direct in-process engine call, passed the array through + * verbatim, and four drivers grew a second filter compiler to meet it — + * an INFIX dialect (`[condA, 'or', condB]`) that the spec never declared, + * that `parseFilterAST` cannot even express, and that cloud's + * `RemoteTransport.buildWhereSQL` refuses outright. Same query, two + * answers, decided by whether the caller went over the wire. + * + * This is that second door, closed: every entry point lowers through the SAME + * `parseFilterAST` sink Door 1 uses, so a driver sees exactly one filter + * dialect regardless of how the query arrived. The authoring ergonomics are + * untouched — `FilterBuilder` tuples, React block `filters` props and the five + * showcase call sites all still work, because lowering is what those shapes + * were always for. + * + * Three arrivals, three answers, matching Door 1 exactly: + * + * 1. `[]` — "no filter". The key is DELETED rather than lowered, which is the + * same reading every layer already gives it (`parseFilterAST([])` is + * `undefined`; `SqlDriver.applyFilters` returns early). Note this is now + * visible to `findOne`'s #4419 guard, which is the point: `findOne({where: + * []})` used to slip past the guard as "an expression tree the driver will + * interpret" and come back with an ARBITRARY row. + * 2. A well-formed AST — lowered. `isFilterAST` gates first so the operator + * vocabulary is checked before `parseFilterAST`'s lenient `$${op}` fallback + * can turn a misspelling into a `$sounds_like` condition nothing executes. + * 3. Anything else array-shaped — REFUSED, loudly, at the call site. Today + * those reach a driver and are refused there (#3948) with driver-internal + * wording, or — for the infix dialect — silently compiled by a second + * implementation. Failing here names the caller's own value. + * + * [#20121] A `where` that is not an array has to be a filter OBJECT, and that + * is checked FIRST, before any door below: a string, a number, a boolean, a + * `Map`, a `Date` — anything that is neither absent, a filter object nor an + * array — is refused with the same `INVALID_FILTER` / 400 envelope the array + * refusal (3) now carries. See {@link isWhereFilterObject} for the accept set + * and why each edge shape lands where it does. + * + * Returns the SAME reference when `where` is not an array (the overwhelmingly + * common path allocates nothing), otherwise a shallow copy: the bag belongs to + * the caller and may be reused (view metadata, flow node config). + */ +export function lowerWhereFilterArray( + object: string, + operation: string, + bag: T, + schema?: unknown, + schemaOf?: (name: string) => unknown, +): T { + if (!bag) return bag; + const where = (bag as Record).where; + // [#20121] The SHAPE gate, first on this seam and so first on all six verbs + // that call it (`find`, `findOne`, `count`, `aggregate`, `update`, + // `delete`) — each calls this before it resolves a driver, so a refusal + // here means no driver call is made at all. + // + // Before it, a string / number / `Map` `where` fell into the object branch + // below, every door there stepped around it (they walk the keys of a filter + // object, and it has none), and the driver ignored it: the READ answered the + // unfiltered set, and a `multi: true` UPDATE or DELETE rewrote or removed + // every row of the table (measured on `driver-memory` and `SqlDriver` + // without `SecurityPlugin`, and with it under a system context; with an + // RLS-scoped caller the RLS `$and` composition put the value where the + // driver's node gate refuses it). It also stepped past the unscoped-write + // guard (`dispatchUnscopedMultiWriteHooks`), which reads only an absent or + // `null` `where` as unscoped. + // + // The words are the wire door's (`metadata-protocol`'s normalizer answers + // `?filter=5` with "must be a filter object or condition array, received + // number … It was not applied"); the consequence is spelled per verb, + // because for a write it is not a result set. + if (where !== undefined && where !== null && !Array.isArray(where) && !isWhereFilterObject(where)) { + throw invalidFilterError( + `${operation}('${object}'): 'where' must be a filter object or condition array, ` + + `received ${describeNonFilterWhere(where)}. It was not applied, and an unapplied filter ` + + `would have ${UNAPPLIED_WHERE_CONSEQUENCE[operation] ?? UNAPPLIED_WHERE_CONSEQUENCE.find}.`, + ); + } + if (!Array.isArray(where)) { + // [#5869] Door 1 lands HERE, not below: the protocol face runs its own + // `isFilterAST` → `parseFilterAST` and hands the engine an already-lowered + // `FilterCondition` object, so a gate on the array branch alone would miss + // every query that arrived over the wire. The comparand check is the same + // one either way — it reads the lowered condition, which is what both doors + // produce. + assertListComparandShapes(object, operation, where); + // [#8296] The unmaterializable-FIELD door, on the same object form. It sits + // beside the shape gate because the two answer different questions about + // the same predicate — "can this comparand run" vs "is there a column to + // run it against" — and because this seam is the one place EVERY + // caller-supplied `where` passes through, whichever verb it arrived by. + assertFilterIsMaterializable(object, operation, schema, where); + // [#15661] The DECLARED-TYPE door for the text operators, third on this + // seam and third question about the same predicate: the shape gate asks + // "can this comparand run", the materializable gate asks "is there a column + // to run it against", and this asks "can that column's declared type ever + // hold a string". It runs BEFORE the temporal gate below deliberately — a + // `$startsWith` over a `date` field is refused by that gate today, with the + // same `INVALID_FILTER` / 400 envelope but a message about the COMPARAND, + // which sends the author to fix a value that could never have made the + // filter runnable. Same wire envelope, the ruling's message. + assertTextOperatorTargetsAreStringCapable(object, operation, schema, where); + // [#8690] The TEMPORAL-comparand door, fourth on the same seam (#15661's + // declared-type door was inserted above it) and fourth question about the + // same predicate: the shape gate asks "can this comparand run", the + // materializable gate asks "is there a column to run it against", the + // declared-type gate asks "can that column ever hold a string", and this + // asks "can that column's storage rule READ this value". + // It must run BEFORE `resolveWhereTokens` (which is downstream of every + // caller of this function) because the refusal has to precede the driver — + // hence the door steps around `{placeholder}` strings rather than judging + // them; the token resolver refuses the unknown ones a moment later, loudly. + assertTemporalComparandsInterpretable(object, operation, schema, where); + // [#20351] The NUMBER-comparand declared-type door, fifth on the same seam + // and fifth question about the same predicate: is this string a number the + // declared numeric column can be compared with. It refuses a non-numeric + // string (`INVALID_FILTER` / 400) and narrows a numeric one to its number, + // copy-on-write, by `@objectstack/spec/data`'s grammar and verdict. Before + // the token resolver, like the temporal door: a `{placeholder}` resolves to + // an id or a date, never a number, so it is refused here unresolved. + // [#20546] Its walk carries a second arm, asked first at every field: a + // plain object with no `$` key where a scalar column's value belongs + // (`{ amount: { a: 1 } }`) is refused `INVALID_FILTER` / 400. Memory + // answered it with no rows (every row under `$not`), SQL with its own 400. + // [#20745] …and beneath a relation column (the nested-relation form no + // driver serves), a structured-JSON column (a whole-value match the + // drivers share no meaning for) and an undeclared `id`, in words per kind. + // [#20802] …except the nested-relation form, which this position SERVES: + // beneath a relation column it is admitted against the related object's + // declarations (`schemaOf`, the registry) and kept as written — the engine + // lowers it once it can read (`ObjectQL.lowerRelationConditions`) — or + // refused in words of its own (a key the related object does not declare, + // a second level, a related object that is not registered). + // [#21333] …and the walk's BOOLEAN arm: against a declared boolean field + // `"true"` / `"false"`, `1` / `0` and `"1"` / `"0"` narrow to their + // boolean and any other string is refused `INVALID_FILTER` / 400 — the one + // place a bare query parameter's `"true"` becomes `true`. + const numeric = narrowNumberComparands(object, operation, schema, where, 'where', { schemaOf }); + // [#7872] The comparand-type door, on the OBJECT form. `parseFilterAST` + // runs the same walk on everything it lowers or passes through, but + // NEITHER door routes an object-form filter through it — Door 1 gates on + // `isFilterAST` first and Door 2 is this very branch — so without this + // call the dominant form would bypass the door entirely (the #7956 + // divergence matrix arrived through it). Shape gate first (#5869 keeps + // its pinned wording for the list-operator shapes), type door second; + // the walk is copy-on-write, so the common path allocates nothing and a + // narrowed bigint replaces the bag rather than editing the caller's. + const normalized = normalizeFilterComparandTypes(numeric, `${operation}('${object}')`); + if (normalized !== where) { + return { ...(bag as Record), where: normalized } as T; + } + return bag; + } + + const lowered: Record = { ...bag }; + + // (1) `[]` is "no filter", not a failed filter. + if (where.length === 0) { + delete lowered.where; + return lowered as T; + } + + // (3) Not a shape `parseFilterAST` can express. + // [#20121] Through `invalidFilterError`, so this refusal carries the + // `INVALID_FILTER` / 400 envelope the wire door answers the same array with + // (`malformedFilterArrayError`) — it used to be a bare `Error`, `code` and + // `status` both `undefined`, a 400-class author error that read as a 500. + if (!isFilterAST(where)) { + throw invalidFilterError( + `${operation}('${object}') received a 'where' array that is not a filter: ` + + `${JSON.stringify(where)}. A filter array is a comparison [field, operator, value], ` + + `a logical node ["and"|"or", ...conditions], or a list of those — it is INPUT-ONLY ` + + `sugar (spec 'FilterArray'), lowered to a FilterCondition here before any driver sees ` + + `it. This value cannot be lowered, and an unapplied filter would have returned ` + + `the UNFILTERED result set. Recognised operators: ` + + `${[...VALID_AST_OPERATORS].sort().join(', ')}. Infix joins ([condA, "or", condB]) are ` + + `NOT one of the shapes — write the prefix form ["or", condA, condB].`, + ); + } + + // (2) The declared path. + // [#9228] The context prefix is passed because `parseFilterAST` now runs the + // #5869 shape gate itself, one step before this branch could: a scalar `$nin` + // is refused DURING lowering. Handing it `${operation}('${object}')` is what + // keeps this branch's pinned `find('deal'): ` prefix on that refusal — and on + // the #7872 type door's, which `parseFilterAST` has always run and which used + // to answer this branch unprefixed. + const condition = parseFilterAST(where, `${operation}('${object}')`); + if (condition === undefined) { + // Unreachable by construction — `isFilterAST` accepted the shape, so + // `parseFilterAST` has a lowering for it. Loud rather than silent because + // the failure mode of the two spec functions disagreeing is a dropped + // predicate, i.e. every row (#3948). + throw new Error( + `${operation}('${object}'): filter array ${JSON.stringify(where)} passed isFilterAST() ` + + `but parseFilterAST() lowered it to nothing. Refusing rather than running the query ` + + `unfiltered.`, + ); + } + // [#5869] Door 2's half of the same check USED to be a second + // `assertListComparandShapes(object, operation, condition)` call here. + // [#9228] moved the rule into `parseFilterAST` itself, which this branch + // calls three lines up with the same context — so the call became provably + // unreachable (nothing between the two can introduce a list operator) and is + // deleted rather than kept as a gate that reads live and never fires. The + // OBJECT branch above keeps its call: that `where` never passes + // `parseFilterAST` at all, which is the whole reason both branches exist. + // [ADR-0139 D1] The object branch's `ReadableIds` rule, on the LOWERED + // condition: `parseFilterAST` admits no leaf as a `$in` comparand (it does + // not opt in, so that spelling is its 400), but a leaf it lowers by + // reference into any other position is refused here by name, as the object + // branch refuses it — first, before the doors below read it as a value. + collectComposableReadableIds(object, operation, condition, 'where'); + // [#8296] Same door as the object branch above, on the LOWERED condition — + // the array sugar (`[['is_open','=',true]]`) names fields too, and a gate on + // one branch would answer one mistake two ways depending on the spelling. + assertFilterIsMaterializable(object, operation, schema, condition); + // [#15661] Same door as the object branch, on the LOWERED condition — the + // array sugar (`[['amount','contains','5']]`) names non-text fields too, and + // a gate on one branch would answer one mistake two ways depending on the + // spelling. + assertTextOperatorTargetsAreStringCapable(object, operation, schema, condition); + // [#8690] Same door as the object branch, on the LOWERED condition — the + // array sugar (`[['at','>=','last_30_days']]`) names temporal fields too, and + // a gate on one branch would answer one mistake two ways depending on the + // spelling. + assertTemporalComparandsInterpretable(object, operation, schema, condition); + // [#20351] Same door as the object branch, on the LOWERED condition — the + // array sugar (`[['amount','>','abc']]`) names numeric fields too. [#20546] + // …and lowers `['amount', '=', { a: 1 }]` to the no-operator object its + // second arm refuses. [#21333] …and `['active', '=', 'true']` to the + // comparand its boolean arm narrows. + lowered.where = narrowNumberComparands(object, operation, schema, condition, 'where', { schemaOf }); + return lowered as T; +} + +/** + * [#20157] Stage 2 of `where` admission: expand the filter placeholders + * (`{current_user_id}`, date macros) against the execution context. + * + * The one spelling of that step. Execution reaches it through + * `ObjectQL.resolveWhereTokens` (the read verbs) and `ObjectQL.withResolvedWhere` + * (`update` / `delete`), and the judge ({@link judgeWhereAdmission}) calls it + * directly. So "what execution resolves" and "what the judge resolves" cannot + * drift. An unknown placeholder throws `FILTER_TOKEN_UNKNOWN`; a known one the + * context carries no value for throws `FILTER_TOKEN_UNRESOLVED`. Neither ever + * resolves to `null` (see `@objectstack/core`'s `filter-tokens.ts`). + * + * [#20844] …and judges what resolved: `judgeResolved` is the position's + * {@link ResolvedTokenJudge}, handed the condition before and after, and a + * date macro that resolved outside its column's years is refused there + * (`INVALID_FILTER` / 400). Required, so no position resolves without it; the + * resolver is field-agnostic, so the column's kind comes from the position. + * + * Returns the input by reference when it holds no placeholder, and then + * judges nothing. + */ +function resolveWhereFilterTokens( + where: W, + context: Parameters[0], + judgeResolved: ResolvedTokenJudge, +): W { + const resolved = resolveFilterTokens(where, filterTokenContextFrom(context)); + if (resolved !== where) judgeResolved(where, resolved); + return resolved; +} + +/** + * [#20844] The `where` position's {@link ResolvedTokenJudge}: each key's kind + * is its declared field's in `schema`. `path` roots the refusal, as the door's + * does — `aggregations[i].filter` for a per-aggregation filter. + */ +function whereResolvedTokenJudge( + object: string, + operation: string, + schema: unknown, + path = 'where', +): ResolvedTokenJudge { + return (written, resolved) => + assertResolvedTemporalTokensInRange(object, operation, schema, written, resolved, path); +} + +/** + * [ADR-0053 D-D1, amended 2026-09-30 — #5930] Stage 2 of every filter + * position's admission, whole: resolve the placeholders + * ({@link resolveWhereFilterTokens}), THEN run the shared lowering + * (`lowerFilterCondition`, `@objectstack/spec/data`) — the `$between` split, + * the whole-day upper bound and the NULL-polarity guards. + * + * The order is the amendment's item 3: the lowering reads the comparand the + * comparison will run with, so a date macro (`{today}`, `{current_month_end}`) + * has already become the bare day the whole-day rule widens. Run beside the + * doors instead, it would meet `{today}` unresolved and leave it bare. + * + * ONE function for every position, so no verb can resolve without lowering: + * `where` on the five read / write verbs (`ObjectQL.resolveWhereTokens`, + * `ObjectQL.withResolvedWhere`), `aggregations[i].filter` and `having` on + * `aggregate`, and the judge ({@link judgeWhereAdmission}), which runs the + * same stage so its verdict is execution's. The lowering never refuses, so it + * adds no verdict of its own. + * + * `lowering` is the position's declared-type reader: this seam reads the + * object's declarations, so the whole-day rule rewrites a declared + * `datetime` column only ({@link declaredDatetimeLowering}) — the scope + * `SqlDriver` holds, so a `date`, `time` or non-temporal column reaches every + * driver byte-identical to before (the amendment's item 7). An object with no + * field map has no declarations to read, and there the rule applies + * type-blind (item 7's other half). + * + * [#20844] `judgeResolved` is the position's other declared-type reader: the + * year range of what resolved ({@link resolveWhereFilterTokens}). + * + * Returns the input by reference when nothing resolved and nothing lowered. + */ +export function resolveThenLowerWhere( + where: W, + context: Parameters[0], + lowering: FilterLoweringOptions, + judgeResolved: ResolvedTokenJudge, +): W { + return lowerFilterCondition(resolveWhereFilterTokens(where, context, judgeResolved), lowering); +} + +/** + * [ADR-0053 D-D1 item 7 — #5930] The typed reading of one object's filter + * positions: a column is `datetime` exactly when the object's declared field + * map says `type: 'datetime'` — the same test `SqlDriver` indexes its + * `datetimeFields` by, so the columns this seam widens are a subset of the + * columns every face widens today. + * + * [#20822] No field map (an object the registry does not hold, a + * registry-less host) is a seam that cannot read the declared type, and item 7 + * says what such a seam does: "applies the rewrite type-blind", as the + * type-blind emitters do (sound on `Field.date` text, where `< next-day` + * orders exactly as `<= day`). So the reader is omitted and the whole-day + * rules apply to every column. This branch used to read "no field map" as "no + * datetime column" and leave the rule to the faces; item 5 retires those + * copies, and with them gone that reading would have dropped the whole-day + * bound on every such read. + */ +export function declaredDatetimeLowering(schema: unknown): FilterLoweringOptions { + const fields = (schema as { fields?: unknown } | undefined)?.fields; + if (fields === null || typeof fields !== 'object') return {}; + return { + isDatetimeColumn: (column) => + fields !== null + && typeof fields === 'object' + && Object.prototype.hasOwnProperty.call(fields, column) + && ((fields as Record)[column]?.type === 'datetime'), + }; +} + +/** + * [#20157] Is this thrown value a door's DIAGNOSTIC (the ADR-0112 envelope: + * a string `code` and a numeric `status`), or something else? + * + * Every `where` door raises the envelope: `invalidFilterError`, the + * materializable door's `INVALID_FIELD`, spec's shape and type faces, and + * core's `FILTER_TOKEN_*` errors. A throw without it is not a verdict about the + * filter. It is a fault (the "unreachable by construction" branch of + * {@link lowerWhereFilterArray}, a bug), and the judge re-throws it rather than + * dressing it up as one. + */ +function admissionRefusalOf( + thrown: unknown, +): Extract | null { + if (!(thrown instanceof Error)) return null; + const { code, status } = thrown as Error & { code?: unknown; status?: unknown }; + if (typeof code !== 'string' || typeof status !== 'number') return null; + return { ok: false, code, status, message: thrown.message }; +} + +/** + * [#20157] Judge a `where` against an object WITHOUT executing it: the engine + * side of `IObjectQLEngine.judgeFilter` (#19995 ruling C). + * + * ## The pipeline it runs is the one every verb runs + * + * Every verb that takes a `where` (`find`, `findOne`, `count`, `aggregate`, + * `update`, `delete`) admits it in the same two stages, in the same order: + * + * 1. {@link lowerWhereFilterArray}: the shape gate, then (object form) + * `assertListComparandShapes` → `assertFilterIsMaterializable` (the dotted, + * then the virtual-field verdict) → `assertTextOperatorTargetsAreStringCapable` + * → `assertTemporalComparandsInterpretable` → `narrowNumberComparands` + * (#20351) → `normalizeFilterComparandTypes`, or (array form) `isFilterAST` + * → `parseFilterAST` → the same four field-map doors on the lowered + * condition. + * 2. {@link resolveWhereFilterTokens}: the placeholder resolver, and [#20844] + * the year range of each date macro it resolved. + * + * [#20802] Execution then lowers each nested-relation condition by READING + * the related object (`ObjectQL.lowerRelationConditions`); that read admits the + * condition through the related object's own two stages. The judge runs those + * two stages on each condition too — as a `find` on the related object — and + * stops there: which ids the read finds, the cap and the caller's permissions + * on the related object need data and a caller, and are execution's alone. + * + * What differs by verb sits BETWEEN or AROUND those stages and judges + * something other than `where`: option-key folding and refusal, the driver + * lookup (`getDriver`, before stage 1 on the writes and between the stages on + * the reads), `orderBy` / projection doors on `find`, and the per-aggregation + * and `having` doors on `aggregate`. None of those is part of this judgement. + * The judge calls the two stage functions themselves, so the doors have one + * implementation and one order. It never calls a second copy of a walk. + * + * ## What it reads + * + * The `where` value, the object's declared field map (`schema`, the + * registry's), and the execution context (for stage 2). No driver, no data, + * no hook, no middleware. Execution's own stages read nothing more, measured + * door by door on #20157. + * + * A thrown door diagnostic becomes the returned refusal, with the same `code`, + * `status` and `message` execution raises. Anything else is re-thrown + * ({@link admissionRefusalOf}). + */ +function judgeWhereAdmission( + object: string, + operation: string, + where: unknown, + schema: unknown, + context: Parameters[0], + schemaOf?: (name: string) => unknown, +): EngineFilterJudgement { + try { + const admitted = lowerWhereFilterArray(object, operation, { where }, schema, schemaOf); + const resolved = resolveWhereFilterTokens( + admitted.where, context, whereResolvedTokenJudge(object, operation, schema), + ); + // [#20802] Each nested-relation condition the door admitted: execution + // reads the related object with it (`ObjectQL.lowerRelationConditions`), + // and that read admits it through the related object's own doors and + // resolver — so the judge asks the same of it here, as a `find` on the + // related object. What the read then finds (the ids, the cap, the caller's + // permissions) needs data and a caller, and is execution's alone. The rest + // of the filter is lowered with each condition standing for the empty id + // set, which the shared lowering treats as it treats any `$in` / `$or`. + const sites = relationSitesOf(object, operation, resolved, schema, schemaOf); + for (const site of sites) { + const inner = judgeWhereAdmission(site.target, 'find', site.condition, schemaOf?.(site.target), context, schemaOf); + if (!inner.ok) return inner; + } + const related = sites.length === 0 + ? resolved + : mapRelationConditions(object, operation, schema, resolved, { + schemaOf: schemaOf ?? (() => undefined), + replace: (site) => lowerRelationSite(site, []), + }); + lowerFilterCondition(related, declaredDatetimeLowering(schema)); + return { ok: true }; + } catch (thrown) { + const refusal = admissionRefusalOf(thrown); + if (refusal) return refusal; + throw thrown; + } +} + +/** + * [#20802] The nested-relation conditions a `where` holds, in walk order: the + * collecting pass of {@link mapRelationConditions} (each admitted site is + * recorded and kept as written, so nothing is rewritten). Empty for a filter + * with none, and for a registry-less host (no field map, no relation column). + */ +export function relationSitesOf( + object: string, + operation: string, + where: unknown, + schema: unknown, + schemaOf: ((name: string) => unknown) | undefined, +): RelationFilterSite[] { + const sites: RelationFilterSite[] = []; + if (where == null) return sites; + mapRelationConditions(object, operation, schema, where, { + schemaOf: schemaOf ?? (() => undefined), + replace: (site) => { + sites.push(site); + return { kind: 'value', value: site.condition }; + }, + }); + return sites; +} + +/** + * [#20082] One operation's effective-permission resolution — see + * `ObjectQL.permissionResolution`. `get()` asks the registered resolver at most + * once for the operation and hands every caller the same answer, or the same + * rejection. + */ +export interface PermissionResolution { + get(): Promise; +} + +/** + * Is this `defaultValue` an Expression envelope (`{ dialect, source }`) — + * evaluated at insert time — rather than a literal or a runtime token? The one + * test {@link ObjectQL.applyFieldDefaults} evaluates by and + * {@link ObjectQL.resolveDefaultPermissions} plans by, so the two can never + * disagree about which defaults are expressions. + */ +export function isExpressionDefault(dv: unknown): dv is Expression { + return typeof dv === 'object' && dv !== null + && Boolean((dv as { dialect?: unknown }).dialect) + && typeof (dv as { source?: unknown }).source === 'string'; +} + +export interface FormulaPlanEntry { + name: string; + expression: Expression; + /** + * [#10280] The field's declared `scale`, when it is one this rounding can + * honour — see {@link formulaRoundingScale}. `undefined` means "return the + * evaluator's value verbatim", which is what every formula did before this. + */ + scale?: number; + /** + * [#24197] The reference fields this formula reads ONE hop through, and the + * related fields it names on each (`record.book.title` → `book → {title}`). + * Absent when the formula traverses nothing — every formula before #24197, + * and still the common case — which is what lets such a read skip the + * related read entirely. + */ + hops?: ReadonlyMap>; + /** + * [#24197] Why this formula's traversal cannot be served (a second hop; a + * reference read both through and as a value), worded for the author. Such a + * formula is not evaluated: it reads `null` and the fault sink carries this + * sentence — the lint refuses the same shape with the same words. + */ + hopConflict?: string; +} + +/** + * `toFixed`'s own domain limit: it throws `RangeError` above 100 fraction + * digits. `FieldSchema.scale` is `z.number().int().min(0)` with no upper + * bound, so an author CAN declare more — and a display-precision declaration + * must never be the reason a read fails. + */ +const MAX_FORMULA_SCALE = 100; + +/** + * [#10280] The `scale` a formula plan entry carries, or `undefined` when the + * declaration is not one this rounding can honour. + * + * Well-formedness is judged with the SAME test the write path uses + * (`validateFieldValue`'s `scale` branch: `Number.isInteger(def.scale) && + * def.scale >= 0`), so one malformed declaration cannot mean two things one + * layer apart. A malformed `scale` stays UNENFORCED rather than being + * interpreted — inventing floor/round semantics for `scale: 2.5` in a consumer + * is the guessing PD #12 forbids, and the producer already refuses it + * (`FieldSchema.scale` is `z.number().int().min(0)`). + * + * The upper bound is `toFixed`'s, not a policy: past {@link MAX_FORMULA_SCALE} + * the call throws, and asking for more decimal places than an IEEE-754 double + * carries is a no-op request either way — so the rounding is skipped and the + * value is returned whole. + */ +function formulaRoundingScale(def: { scale?: unknown } | undefined): number | undefined { + const scale = def?.scale; + if (typeof scale !== 'number' || !Number.isInteger(scale) || scale < 0) return undefined; + return scale <= MAX_FORMULA_SCALE ? scale : undefined; +} + +/** + * [#18408] Is this field's value MULTI-VALUED — the one question, asked of the + * one predicate `@objectstack/spec` publishes, and the single local seam every + * site in this file goes through. + * + * Maintainer ruling 2026-09-13 (decision batch #128 item 5, option 1′): there + * is ONE definition of "is this field multi-valued", `isMultiValueField`, and + * storage follows it. The sites below used to read `field.multiple` raw, which + * answers `true` on types the spec predicate calls single-valued (`text`, + * `master_detail`, `tree` …) and `false` on the inherently-multi option types + * (`multiselect` / `checkboxes` / `tags`) that carry no flag at all. Both + * directions put this engine on a different list from the storage it writes + * through: `driver-sql` (#17469) and `os generate migration` (#18199) both ask + * the predicate now. + * + * Takes the field rather than a resolved type because — unlike driver-sql and + * the CLI generator — no caller in this file applies a type default of its own; + * a non-string `type` is answered `false` here, the same verdict those two + * reach through their `field.type || 'string'` resolution. + * + * ⛔ The predicate is called, never re-spelled. `MULTI_CAPABLE_TYPES` / + * `MULTI_OPTION_TYPES` membership written out at a site would be a second + * answer to a question the ruling gave exactly one. + */ +export function declaredMultiValued(field: { type?: unknown; multiple?: unknown } | null | undefined): boolean { + return isMultiValueField({ + type: typeof field?.type === 'string' ? field.type : '', + multiple: field?.multiple === true, + }); +} + +/** + * [#22957] The reference types whose `deleteBehavior` states DELETE SEMANTICS: + * the relational family the referential delete walk follows. + * + * `tree` is a member because the spec says so. `FieldSchema`'s `.overwrite` + * (`packages/spec/src/data/field.zod.ts`) materializes `deleteBehavior: + * 'set_null'` on a `tree` beside `lookup`, because the type "is in the + * relational family, where the key states delete semantics". Its reference is + * self-referential by contract (`refuseForeignTreeReference`, `object.zod.ts`), + * so its `cascade` is a subtree delete. The walk used to admit only + * `master_detail` and `lookup`, so every behaviour a `tree` declared, the + * materialized default included, was inert: a parent delete left its children + * holding the deleted id. + * + * `user` is not a member of this set: see {@link USER_REFERENCE_TYPE}, which + * takes part only when it declares a delete outcome of its own. + */ +const DELETE_SEMANTICS_REFERENCE_TYPES: ReadonlySet = new Set(['master_detail', 'lookup', 'tree']); + +/** + * [#29253] A `user` field takes part in the referential delete walk when, and + * only when, it DECLARES a `deleteBehavior` (one of the enum's three values). + * + * It stores like a `lookup` (the target is fixed to `sys_user` by the type, + * {@link referenceTargetOf}), but the spec's parse materializes no + * `deleteBehavior` on it: a `user` field that authors nothing declares no + * delete outcome, so deleting the user leaves it as it was, exactly as before. + * A `user` field that authors one is a declaration the spec accepts and + * round-trips, and the walk used to read it never, so an authored `restrict` + * refused nothing and an authored `cascade` deleted nothing. It now reads like + * a `lookup`'s authored value: `cascade` deletes the rows holding the deleted + * user, `set_null` clears them (on a multi-valued field, removes the member), + * and `restrict` refuses the delete with the walk's own `DELETE_RESTRICTED`; + * a `set_null` on a REQUIRED `user` field escalates to that refusal as a + * required `lookup`'s does. + * + * Every door that deletes a user reaches this walk: better-auth deletes the + * user row (admin remove-user, self delete-user) through the plugin-auth + * adapter's `engine.delete` by id, and the generic data doors refuse a + * caller-context `sys_user` delete before it (ADR-0092 D2). + * + * ⛔ No default is invented here. "Absent" stays outside the walk, because the + * spec's parse decides what a bare `user` field declares, not this file. + */ +const USER_REFERENCE_TYPE = 'user'; +const DECLARED_DELETE_BEHAVIORS: ReadonlySet = new Set(['set_null', 'cascade', 'restrict']); + +/** + * [#22957] Does `fdef` belong to {@link DELETE_SEMANTICS_REFERENCE_TYPES}, or + * [#29253] is it a `user` field that declares a `deleteBehavior` + * ({@link USER_REFERENCE_TYPE})? This + * is the ONE family test every reader of the referential delete walk asks: + * {@link ObjectQL.cascadeRelationBehavior}, through it the collection walk and + * the dependents walk, and {@link ObjectQL.planCascadeAtomicity}. So the walk + * that runs and the verdict on whether it runs as one unit of work cannot + * disagree about which relations take part. ⛔ Never a type literal at a site. + */ +function statesDeleteSemantics(fdef: { type?: unknown; deleteBehavior?: unknown } | null | undefined): boolean { + if (typeof fdef?.type !== 'string') return false; + if (fdef.type === USER_REFERENCE_TYPE) { + return typeof fdef.deleteBehavior === 'string' && DECLARED_DELETE_BEHAVIORS.has(fdef.deleteBehavior); + } + return DELETE_SEMANTICS_REFERENCE_TYPES.has(fdef.type); +} + +/** + * [#29253] The object a delete-walk relation points at. The `reference` + * carrier is read through the one arbiter ({@link referenceCarrierOf}), so an + * unreadable carrier still throws and an absent one still answers `undefined`. + * A `user` field's target is fixed by its type, so it answers `sys_user` + * ({@link referenceTargetOf}) whether or not the carrier spells it: a raw + * `{ type: 'user' }` registration names no `reference` and still references a + * user. + */ +function deleteWalkTargetOf(fdef: { type?: unknown } | null | undefined, reader: string): string | undefined { + const carrier = referenceCarrierOf(fdef, reader); + return fdef?.type === USER_REFERENCE_TYPE ? referenceTargetOf(fdef) : carrier; +} + +/** + * Plan the formula pass of a read: which formula fields to evaluate, and — + * when the caller named a projection — the projection the DRIVER is asked for. + * + * A projection naming a formula field is widened to every stored column (plus + * `id`), because CEL's `record.` reads whatever the formula needs off + * the full row. That widening is a means of EVALUATING the formula and never + * an answer: [#22300] `widened` lists the columns it added beyond what the + * caller named, and {@link withoutFormulaWidening} cuts them back off the rows + * once the read is done, so the caller gets the projection it declared plus + * the formula's value. `widened` is absent whenever nothing was widened. + */ +/** + * ONE plan entry for a field's declared `expression`: the envelope, the + * rounding scale, and the one hop it reads through a reference field. Shared + * by the `formula` read plan ({@link planFormulaProjection}) and the stored + * computed-field write plan ({@link planStoredExpressions}), so a value + * computed on read and a value computed on write are one semantic. + */ +function formulaPlanEntry(name: string, def: any, fields: any): FormulaPlanEntry { + // Normalize string-shorthand → Expression envelope (M9 transition). + const expr: Expression = typeof def.expression === 'string' + ? { dialect: 'cel', source: def.expression } + : def.expression; + const entry: FormulaPlanEntry = { name, expression: expr, scale: formulaRoundingScale(def) }; + // [#24197] The one hop this formula reads through a reference field, if + // any — judged by the validation-rule path's own collector, so the two + // seams that serve a hop read the same hops off the same expression. + const conflicts = referenceTraversalConflicts(expr, fields, 'formula'); + if (conflicts.length > 0) { + entry.hopConflict = conflicts.map((c) => c.message).join(' '); + } else { + const hops = collectReferenceTraversals([expr], fields); + if (hops.size > 0) entry.hops = hops; + } + // Pre-compile to surface syntax errors at planning stage rather than + // per-row eval. Own-column dependency discovery is not used — CEL reads + // the row via `record.`; only the hops above are planned. + ExpressionEngine.compile(expr); + return entry; +} + +function planFormulaProjection( + schema: any, + requestedFields: string[] | undefined +): { plan: FormulaPlanEntry[]; projected?: string[]; widened?: string[] } { + if (!schema?.fields) return { plan: [] }; + const allFieldNames = Object.keys(schema.fields); + // When no explicit projection, evaluate every formula field on the schema — + // matches REST default of "return everything". Explicit projection still + // honours the caller's selection. + const targets = (Array.isArray(requestedFields) && requestedFields.length > 0) + ? requestedFields + : allFieldNames; + const plan: FormulaPlanEntry[] = []; + const projected = new Set(); + for (const f of targets) { + const def = (schema.fields as any)[f]; + if (def?.type === 'formula' && def.expression) { + plan.push(formulaPlanEntry(f, def, schema.fields)); + } else if (Array.isArray(requestedFields) && requestedFields.length > 0) { + projected.add(f); + } + } + if (plan.length === 0) return { plan: [] }; + // For formulas: project all schema fields so CEL `record.` lookups + // see complete data. Static dependency analysis on AST is M9.7 work. + if (Array.isArray(requestedFields) && requestedFields.length > 0) { + if (!projected.has('id')) projected.add('id'); + for (const fname of allFieldNames) { + // Skip formula fields themselves — they are virtual and not + // projectable by the underlying driver. Without this guard the + // SQL driver emits `SELECT response_rate ...` which fails as + // "no such column" and the driver returns [] (silently). + const fdef = (schema.fields as any)[fname]; + if (fdef?.type === 'formula') continue; + projected.add(fname); + } + // [#22300] What the widening added — `id` included when the caller did not + // name it — is what the read cuts back off once the formulas are computed. + const named = new Set(requestedFields); + const widened = Array.from(projected).filter((f) => !named.has(f)); + return { plan, projected: Array.from(projected), ...(widened.length > 0 ? { widened } : {}) }; + } + // Implicit/full projection — leave projected undefined so the driver + // returns its default columns (typically *). + return { plan }; +} + +/** + * [#22300] Cut the columns {@link planFormulaProjection} widened a projection + * by back off what a read RETURNS — the formula saw the full row, the caller + * does not. Without it, a projection naming a formula field answered with every + * stored column (tenant, owner, owning unit, audit actors and timestamps, every + * unnamed field): a flow's `get_record`, whose `config.fields` is declared as + * "only these fields are read", passed all of them on to whatever its later + * nodes sent out. + * + * ## The answer it restores + * + * The same projection without the formula, plus the formula's value: measured + * on driver-sql (a projection is exactly the named columns — no `id` unless it + * is named) and pinned against it in `engine-formula-projection-trim.test.ts`. + * `id` is cut like any other widened column when the caller did not name it. + * [#26672] The `id` {@link withFileOwnerKey} adds for a file field's owner + * verdict is cut the same way. + * + * ## Where it runs, and why there + * + * On the result `find` / `findOne` hand back, AFTER the middleware chain — + * the last internal consumer. Everything before it keeps reading the widened + * row exactly as before: the formula pass, `expand`, file-reference + * resolution, the `afterFind` hooks, the secret mask and the `__search` + * strip, and the middlewares' post-phase (field-level security's result mask, + * the audit redactions that judge a row by columns the caller did not name). + * Cutting earlier would starve those of a column they read today. + * + * It REMOVES the widened columns rather than keeping a list: a key an + * `afterFind` hook derives, or an expanded relation the caller named, is not + * the widening's and survives. A widened column a hook re-assigns is cut — the + * caller never named it. + * + * Never mutates a row: a row carrying a widened column is replaced by a copy + * without it, in the row's own key order; a row carrying none is returned as + * is. + */ +function withoutFormulaWidening(row: T, widened: ReadonlySet | undefined): T { + if (!widened || !row || typeof row !== 'object' || Array.isArray(row)) return row; + const source = row as unknown as Record; + if (!Object.keys(source).some((key) => widened.has(key))) return row; + const cut: Record = {}; + for (const key of Object.keys(source)) { + if (!widened.has(key)) cut[key] = source[key]; + } + return cut as unknown as T; +} + +/** {@link withoutFormulaWidening} over a `find` result; anything but an array passes through. */ +function rowsWithoutFormulaWidening(rows: T, widened: ReadonlySet | undefined): T { + if (!widened || !Array.isArray(rows)) return rows; + return rows.map((row) => withoutFormulaWidening(row, widened)) as unknown as T; +} + +/** + * [#26672] A projection that names a file field reads the record's `id` too, + * and the key is cut back off the answer like the formula widening is. + * + * File-reference hydration judges a reader refused `sys_file` read by the + * record that OWNS each file (#22637, `readParentDerivedFiles`), matched on + * that record's `id`. A projection that left `id` out handed it rows with no + * key, so a file the reader may read answered `{ id, metadataRefused: true }` + * to `select=` and the full value to `select=id,`. The + * verdict must not depend on whether the caller listed `id`, so the driver is + * asked for it here and {@link withoutFormulaWidening} trims it at the + * `return`: the caller still gets exactly the columns it named, and a reader + * the verdict refuses is still refused. + * + * Touches only an explicit projection that names a file field and not `id`; + * a projection the formula planner widened already carries `id`. + */ +function withFileOwnerKey( + schema: any, + requestedFields: string[] | undefined, + planned: { projected?: string[]; widened?: string[] }, +): { projected?: string[]; widened?: string[] } { + if (!schema?.fields || !Array.isArray(requestedFields) || requestedFields.length === 0) return planned; + const projected = planned.projected ?? requestedFields; + if (projected.includes('id')) return planned; + if (!requestedFields.some((f) => FILE_REFERENCE_TYPES.has((schema.fields as any)[f]?.type))) return planned; + return { projected: [...projected, 'id'], widened: [...(planned.widened ?? []), 'id'] }; +} + +/** + * [#7095] ORDER BY a field whose value is computed on read — refused HERE, on + * the engine's own public boundary, and no longer only at the REST ingress. + * + * #6994 closed this at `assertSortFieldsExist` (`400 INVALID_SORT`), which + * covers everything reaching `findData`: the REST list route, + * `POST /data/:object/query`, the export route and the RPC dispatcher. It could + * not cover a caller that reaches {@link ObjectQL.find} / {@link + * ObjectQL.findOne} DIRECTLY, and that half was not hypothetical — measured on + * this file's base (real `ObjectQL`, a driver that really sorts): + * + * ``` + * engine.find(o, { orderBy: [{ field: , order: 'asc' }] }) -> C A E B D (insertion order) + * engine.find(o, { orderBy: [{ field: , order: 'desc' }] }) -> C A E B D (byte-identical) + * ``` + * + * `asc` and `desc` coming back identical is what makes it a DROPPED sort rather + * than a coincidence, and the rows carrying the very values they were asked to + * be ordered by is what makes it invisible. `planFormulaProjection` above drops + * the virtual NAME from the projection (it must — the driver has no column and + * `SELECT sort_key` fails as "no such column"); nothing did the equivalent for + * the sort, so the ORDER BY reached the driver, found nothing, and the #3821 + * unknown-column backstop returned the rows unordered under a success. + * + * WHY A REFUSAL AND NOT AN OBSERVABLE DROP (the card offered both): ruled + * 2026-08-10 on #7095 — an ORDER BY the engine cannot apply is a 4xx with + * guidance prose, never a silent drop, the same direction as the analytics + * dataset refusal envelope and #6924's sort-hint prescription. The engine's + * documented internal-caller tolerance (`assertProjectionFieldsExist`'s + * docblock) was to survive only behind a pinned internal path and only if a + * measured internal call site relied on it; the #7095 sweep found none, so + * there is no internal path and no option to opt back into the drop. That sweep + * is recorded in the changeset — if you are adding a caller that WANTS the old + * behaviour, the answer is a stored field, not a flag. + * + * ⚠️ Post-hoc sorting after {@link applyFormulaPlan} evaluates the formulas is + * a TRAP, written down here because it looks like the generous fix: `driver.find` + * has already applied `limit`/`offset`, so re-sorting reorders an ARBITRARY + * PAGE. It would pass every small-result-set test and be wrong the moment + * pagination is involved. + * + * SCOPE — deliberately the third verdict only, on the SORT axis. `unknown` and + * `dotted` SORT names are NOT judged here: the ingress gate's precedence is + * `unknown` > `dotted` > unmaterializable (#4226 / #4256 / #6994), and widening + * this door to those two is a separate posture change on two more axes, not a + * free extension of this one — so a dotted SORT path keeps reaching the driver + * exactly as before, including one whose head is a formula field. On the + * PROJECTION axis the engine still tolerates an unknown PLAIN name by design + * (the `SELECT *` tolerance a few lines below, kept by the #7589 ruling), but + * a dotted PROJECTION entry is refused since #7589 — + * {@link assertProjectionHasNoDottedPaths}, directly below. + * + * A registry-less host (`schema` undefined) returns early, exactly as the + * ingress gate returns early when `resolveQueryFields` cannot answer: a door + * that cannot see the field map must not invent a verdict about it. + */ +function assertOrderByIsMaterializable( + object: string, + operation: 'find' | 'findOne', + schema: any, + orderBy: unknown, +): void { + if (!Array.isArray(orderBy) || orderBy.length === 0) return; + if (!schema?.fields) return; + const names = orderBy.map((node) => + typeof node === 'string' ? node : String((node as any)?.field ?? '')); + const unmaterialized = names.filter((f) => + f !== '' && !f.includes('.') && (schema.fields as any)[f]?.type === 'formula'); + if (unmaterialized.length === 0) return; + const first = unmaterialized[0]; + const type = String((schema.fields as any)[first]?.type); + const err: any = new Error( + `ObjectQL.${operation}('${object}') sorts by '${first}', a ${type} field on '${object}' — ` + + `a ${type} value is computed on read, so no driver materialises a column to order by` + + (unmaterialized.length > 1 ? ` (also: ${unmaterialized.slice(1).join(', ')})` : '') + + '. It was not applied, and an unapplied sort returns the rows in an arbitrary order — ' + + "which 'limit'/'offset' then slices into an arbitrary page." + // Deliberately the SAME remedy, in the same words, as the ingress door's + // formula and dotted refusals (#6994, #6924). One vocabulary across the + // doors: a caller refused at the REST boundary and a caller refused here + // must not be sent two different ways. + // + // [commit e5eeb499c] With the SEARCH axis the agreement is in SUBSTANCE, not in + // words, and saying otherwise here was simply false — measured from the + // running doors, #6673's correction emits "Mirror the computed value onto + // a stored text field on '' and search that instead." All three + // axes prescribe the same thing (put the value on a stored column of the + // queried object and query THAT instead); SEARCH narrows the target to a + // TEXT column, which is right for an axis that scans text-like columns and + // is a shipped, user-visible message on an axis that already landed. + // + // `query-expression-conformance.test.ts` pins that agreement as ONE thing + // rather than each wording separately, because separate wordings is + // exactly how #4256 and #6673 drifted apart in the first place. It holds + // three layers: the four SORT/FILTER doors' emitted remedies compared + // against EACH OTHER, every axis' exact sentence (SEARCH included — the + // door this claim used to skip), and the substance none of them may drop. + // So a rewording of any axis goes red. + + ` Denormalise the value onto '${object}' (a stored field, written when the` + + ' source changes) and sort by that. A formula field is virtual: with no' + + ' column behind it the ORDER BY reaches the driver, finds nothing, and is' + + ' dropped — the arbitrary order this refusal replaces.', + ); + // `INVALID_SORT`, not a new code, and 400 rather than 500: the ingress door + // reasoned that one condition — "this sort was not applied as written" — + // keeps ONE wire code however the caller reached it, and a caller reaching + // the engine directly has not stopped being that condition. A host that + // surfaces engine errors over HTTP therefore answers the same envelope on + // both doors instead of turning the direct path into an unhandled 500. + err.status = 400; + // …and `httpStatus`, the SAME number under ADR-0112 D5's spelling. `status` is + // what every HTTP door in this repo reads (`resolveThrownHttpError`), so it + // stays; `httpStatus` is what a consumer holding the THROWN error reads — + // the CLI's `--json` error envelope (`errorCodeFields`) is the measured one, + // and it saw `code` with no status at all until both spellings were stamped. + // Two keys, one value, written together at every producer in this package. + err.httpStatus = 400; + err.code = 'INVALID_SORT'; + err.field = first; + err.fields = unmaterialized; + err.object = object; + throw err; +} + +/** + * [#7589] A DOTTED projection entry — refused on the engine's own public + * boundary, the second half of #7532's ingress refusal. + * + * #7532 (PR #7588) closed this at `assertProjectionFieldsExist` + * (`400 INVALID_FIELD`), which covers everything reaching `findData`: the REST + * list route, `POST /data/:object/query`, the export route and the RPC + * dispatcher. It could not cover a caller that reaches {@link ObjectQL.find} / + * {@link ObjectQL.findOne} DIRECTLY — and that half was measured, not assumed + * (#7589): a flow-authored `get_record` node's `fields: ['name','account.name']` + * parses (`GetRecordConfigSchema` restricts nothing), travels verbatim into + * `data.find(...)`, cleared the head-only filter below on its head segment + * (`account` IS a field), and reached the driver as a projection column — where + * SQL renders `"account"."name"` against a table that was never joined, the DB + * answers `no such column`, and the #3821 recovery ladder retries `select('*')`. + * The caller asked to narrow and silently received EVERY field, byte-identical + * to no projection at all. Every hook and internal caller reaches this the + * same way (a saved report's `query.fields` did too, until the saved-report + * stack was retired in #20102). + * + * WHY A REFUSAL: ruled 2026-08-12 on #7589 (adopting the drivers seat's + * Option B) — a dotted entry the engine cannot resolve is refused loudly at + * this one site, covering every caller that reaches the engine. The head-only + * check this replaces was justified by a comment claiming the engine resolves + * relationship paths "via populate"; #7601 measured that NO populate step + * exists — the comment was the last place in the repo asserting dotted-path + * resolution does (after PR #7617) — so what is removed here is not a working + * feature but a path to widening, kept alive by a false premise. Both the typo + * (`titel.name`) and the genuine traversal intent (`account.name`) eat this + * refusal: nothing resolves either, they are not separable at this door, and + * the alternative is the over-return above (#5918's precedent, same as the + * ingress ruling). + * + * SCOPE — the dotted leg ONLY. The unknown-PLAIN-column tolerance a few lines + * below each call site is explicitly KEPT (same #7589 ruling): an unknown + * plain column is simply absent from each row, the "no records exist" failure + * that tolerance prevents is real, and it backstops registry-less hosts. A + * dotted path differs in kind — it is a projection no driver can structurally + * apply, and answering it with every column points away from both FLS and data + * minimisation. The two facts get two verdicts. + * + * A registry-less host (`schema.fields` undefined) returns early, exactly as + * the ingress gate returns early when `resolveQueryFields` cannot answer: a + * door that cannot see the field map must not invent a verdict about it. For + * that host the driver-side #3821 ladder remains the documented backstop + * (deliberately untouched — a driver-side carve-out is ruled measured-need + * only). + * + * The wording deliberately shares its core sentence and remedies with the + * ingress door's dotted refusal — one vocabulary across the doors, so a caller + * refused at the REST boundary and a caller refused here are not sent two + * different ways. Duplicated rather than imported because `metadata-protocol` + * is assembled FROM an engine, so the engine cannot import from it without + * inverting the layering; the agreement pin in + * `query-expression-conformance.test.ts` is what keeps the duplication honest + * (same mechanism as the sort axis' three-door remedy pin). + */ +function assertProjectionHasNoDottedPaths( + object: string, + operation: 'find' | 'findOne', + schema: any, + fields: unknown, +): void { + if (!Array.isArray(fields) || fields.length === 0) return; + if (!schema?.fields) return; + const dotted = fields.filter( + (f): f is string => typeof f === 'string' && f.includes('.')); + if (dotted.length === 0) return; + const first = dotted[0]; + const head = first.split('.')[0]; + const headDef: any = (schema.fields as any)[head]; + const crossesRelation = headDef != null && REFERENCE_VALUE_TYPES.has(headDef.type); + const err: any = new Error( + (crossesRelation + ? `ObjectQL.${operation}('${object}') projects '${first}', which follows the relationship ` + + `'${head}' into another object — 'fields' reaches only columns of '${object}' itself` + : `ObjectQL.${operation}('${object}') projects '${first}', a dotted path — 'fields' reaches ` + + `only whole columns of '${object}', not values inside them`) + + (dotted.length > 1 ? ` (also: ${dotted.slice(1).join(', ')})` : '') + + '. No driver resolves it: the path reaches the driver as a column name, matches no ' + + 'column, and the projection falls back to EVERY field — a narrower request answered ' + + 'with a wider response.' + + (crossesRelation + ? ` Read the related record with 'expand' (\`{ expand: { ${head}: { object: '', ` + + `fields: [''] } } }\` to choose its columns), or denormalise the value onto ` + + `'${object}' (a stored field, written when the source changes) and name that.` + : ` Name the whole column ('${head}') and read into its value in the caller.`), + ); + // `INVALID_FIELD`, not a new code, and 400 rather than 500 — the same + // reasoning `assertOrderByIsMaterializable` records for `INVALID_SORT`: one + // condition ("this projection was not applied as written") keeps ONE wire + // code however the caller reached it, so a host surfacing engine errors over + // HTTP answers the same envelope on both doors. + err.status = 400; + err.httpStatus = 400; + err.code = 'INVALID_FIELD'; + err.field = first; + err.fields = dotted; + err.object = object; + throw err; +} + +/** + * [#24284] The refusal for a non-blank `search` over an object whose RESOLVED + * search set is empty — no `searchableFields` entry naming a stored field, and + * no field of a type the auto-default scans (the junction shape: lookups plus + * a composed formula title). Without it the expansion added no predicate and + * the read answered every row. + * + * `INVALID_FIELD`, the code #4254's explicit-override gate answers on this + * same axis (and the one a search naming a formula title gets), not + * `INVALID_QUERY`: the request is well-formed, and what cannot be honoured is + * a fact about the object's FIELDS. 400 on every door. + * + * [#25672] `withheld`: the declared set is not empty, but every field in it is + * one the caller may not query, so the narrowing in `expandSearchOnAst` left + * nothing. Same code and status — the search cannot be honoured for this + * caller either way — with a message that names the cause, not a declaration + * fault the object does not have. + */ +function emptySearchSetError(object: string, schema: any, internalExcluded: boolean, withheld = false): Error { + const declared: unknown[] = Array.isArray(schema?.searchableFields) ? schema.searchableFields : []; + const err: any = withheld ? new Error( + `Object '${object}' has no field this caller may search, so 'search' cannot be honoured on it: ` + + `every field it searches is one field-level security does not let this caller query. ` + + 'Answering every row instead would read as "everything matched". Grant read access to one of ' + + `its searchable fields, or filter on a field this caller can read.`, + ) : new Error( + `Object '${object}' has no searchable field, so 'search' cannot be honoured on it` + + (declared.length > 0 + ? `: its 'searchableFields' (${declared.join(', ')}) names no stored field` + : `: it declares no 'searchableFields', and none of its fields is a type the default ` + + `search scans (${[...SEARCHABLE_TEXTUAL_TYPES, ...SEARCHABLE_ENUM_TYPES].join(' / ')})`) + + (internalExcluded ? " ('internal: true' fields are never searched)" : '') + + '. Answering every row instead would read as "everything matched". Declare ' + + `'searchableFields' on '${object}' naming a stored field, or mirror the composed title ` + + `onto a stored text field and search that.`, + ); + err.status = 400; + err.httpStatus = 400; + err.code = 'INVALID_FIELD'; + err.object = object; + err.param = 'search'; + return err; +} + +/** + * [#8682 / #8738] The DECLARED-FIELD DOOR on the WRITE paths — `insert()` and, + * since #8738, `update()`: a key the object does not declare is refused by the + * SCHEMA, before anything is produced for a request that is about to be refused + * anyway. + * + * ONE condition, ONE implementation, deliberately — the two verbs sit in the + * same file and share the shape of the problem, so #8738 gave this function its + * second caller rather than a second predicate to drift against. The verdict is + * a question about a payload and an object's field map; neither term knows + * which verb asked. + * + * ## What ran before this door existed + * + * Nothing rejected an undeclared write key. The row travelled the whole insert + * path and was refused at the very end by the DRIVER — `table crm_account has + * no column named zzz_nonexistent_field` — which `mapDataError` then translated + * into the (correct, unchanged) `400 INVALID_FIELD` the caller sees. The client + * answer was never the defect; what happened behind it was. Measured on + * `origin/main` for one mistyped key: + * + * - an id, a normalized name, owner/creator resolution and the column + * defaults were all produced; + * - the app's `beforeInsert` hooks RAN, and their derived values reached the + * statement (`period_label = 'Q3 2026'`, computed for a request nobody + * would ever store); + * - an AUTONUMBER was issued and CONSUMED — the log-independent observable, + * and the one this door is pinned on: a valid create before the bad request + * numbered `ACC-000013`, the bad request took `ACC-000014` into the void, + * and the next valid create came back `ACC-000015`. A sequence gap is + * permanent and visible to end users on a document number. + * + * A hook is not a pure function — it writes rows, calls out, stamps ledgers — + * so "it ran and we threw the result away" is not a no-op, it is a side effect + * of a request the server had already decided to refuse. + * + * ## The UPDATE path had the same hole (#8738), measured the same way + * + * #8738 filed the update half as INFERRED, not measured, and asked for it to be + * reproduced before anyone implemented against it. It was, on `origin/main` @ + * `e5eeb499c`, with a real engine and a recording driver — and it reproduces on + * BOTH branches of the verb: + * + * - by-id `driver.update` received `zzz_nonexistent_field` and refused it; + * - multi `driver.updateMany` received it and refused it likewise; + * - `beforeUpdate` RAN first on both, and its derived value reached the + * statement — the recorded payload was `{ name, zzz_nonexistent_field, + * description: 'derived-for-bad' }`, where `description` is the hook's, not + * the caller's. The card's `period_label = 'Q3 2026'` shape, one verb over. + * + * What it does NOT have is the insert path's durable damage: no autonumber is + * issued on update, so nothing user-visible is consumed by the refused request. + * The hook side effect and `declared = enforced` (PD #10) are the whole case — + * which is why the pin here is the HOOK, not a sequence gap. + * + * ## Why HERE and not one step later + * + * This runs as the first act inside the middleware body: after middleware (a + * middleware may legitimately rewrite `data`, and judging the caller's keys + * before it ran would refuse rows a sanitizing middleware had already fixed), + * and before `applyFieldDefaults` — hence before the defaults, the summary + * seeding, the hooks, the strips, validation, the autonumber and the statement. + * The same placement, and the same argument, as {@link assertWriteAllowed} and + * `enforceTransactionOrigin` one level up: a refusal this early costs nothing. + * Like those two it throws without logging — the caller is told exactly what is + * wrong, and a client typo is not a server fault to record at ERROR. + * + * `update()` places it by the same rule, which lands it one step EARLIER in + * that verb's body than a naive reading suggests: first act inside the + * middleware body, so ahead of the prior-record read as well as the hooks and + * both strips. Ahead of the read is not incidental — a refused write should not + * cost a driver round-trip either, and the read exists to serve `previous`, the + * `readonlyWhen` gate and the not-found gate, none of which a refused payload + * reaches. It is also ahead of the dispatch ladder's own `reject` verdict, so a + * call that is BOTH mis-keyed and missing its `id`/`multi` is answered on the + * payload: the ladder's message is about how to address rows and would send an + * author hunting for the wrong defect, while `INVALID_FIELD` names the typo. + * + * ## The verdict is the SCHEMA's field map, and the wire answer is unchanged + * + * `INVALID_FIELD` + 400, and the message is byte-identical to the one + * `mapDataError`'s driver-string branch produced (`Unknown field 'x' on object + * 'y'`) — so this moves WHERE the refusal is decided without moving what the + * caller reads. One condition keeps one wire shape however it was noticed, the + * same rule the read path's dotted-projection and unmaterializable-sort doors + * record. The driver-string branch stays where it is: it still backstops schema + * drift (a DECLARED field whose physical column is missing), which this door by + * construction cannot see. + * + * ## Where this door deliberately has NO opinion + * + * A door that cannot see the field map must not invent an opinion about it — + * the read-path doors' rule, and it decides two cases here: + * + * - **`schema.fields` absent** (a registry-less host), and **`schema.fields` + * EMPTY**. An empty map is indistinguishable from an unpopulated one: a real + * registered object always carries at least its primary key, and the + * registry injects the tenancy and audit columns on top, so zero declared + * fields means the host did not fill the map in — not that the object + * forbids every key. Refusing everything on that reading would be a verdict + * made from an absence, which is what this clause exists to prevent. + * - **`id` / `created_at` / `updated_at`**, which are tolerated even when the + * map does not list them. This mirrors `find()` / `findOne()`, which add + * exactly these three to their known set for exactly this reason: they are + * provisioned by the platform rather than authored, so an object may legally + * omit them from its declaration while the physical table has them. Same + * three names, same rationale, so a key accepted by a read is not refused by + * a write. + * + * In every one of those cases the driver remains the backstop it has always + * been — nothing is widened, the refusal simply moves back to where it was. + * + * @returns one entry per row: an `Error` for a row carrying undeclared keys, + * `undefined` for a row that passes. Per row rather than throw-on-first + * because the partial-success path (`insertMany`) reports per row and must + * cull the bad rows instead of failing the batch around them. + */ +export function undeclaredWriteFieldErrors( + object: string, + schema: { fields?: unknown } | undefined, + rows: readonly unknown[], +): Array { + const out: Array = new Array(rows.length); + // [#21571] The declared set and its "no opinion" cases (no map, an array + // map, an empty map) are `declaredColumnSet`'s, shared with the read verbs' + // default projection — one answer to "is this a column of the object". + const declared = declaredColumnSet(schema); + if (!declared) return out; + for (let i = 0; i < rows.length; i++) { + const row = rows[i]; + if (!row || typeof row !== 'object' || Array.isArray(row)) continue; + const undeclared = Object.keys(row as Record).filter((key) => !declared.has(key)); + if (undeclared.length === 0) continue; + const err: any = new Error(`Unknown field '${undeclared[0]}' on object '${object}'`); + err.status = 400; + err.httpStatus = 400; + err.code = 'INVALID_FIELD'; + err.field = undeclared[0]; + err.fields = undeclared; + err.object = object; + out[i] = err as Error; + } + return out; +} + +/** + * [#20805] The computed-field door: take every caller-supplied key that names a + * declared `formula` field out of a write payload, and say which keys went. + * + * ## Why a strip + * + * A `formula` field is virtual. The engine computes it on every read + * ({@link applyFormulaPlan}) and no driver has a column for it + * (`driver-sql`'s `createColumn` emits none), so a full read returns the key + * and a record written back carries it — the ordinary round trip of a form + * save, a flow's `update_record`, or `GET` then `PUT`. Before this door the key + * reached the driver and the driver decided, so the answer split by family: + * SQL refused the whole statement with its own text (`table … has no column + * named …`, no status, no field) and the schemaless family stored the value as + * a shadow column nothing reads. {@link undeclaredWriteFieldErrors} cannot see + * it: the field IS declared. + * + * The platform already has one answer for a declared field a caller cannot + * write — strip the value, complete the write, report the strip through + * `onFieldsDropped` (`DroppedFieldsEventSchema`: "stripping is legitimate + * semantics, not an error") — and a computed field takes it, under its own + * `reason`, `computed`. ⛔ Never `readonly`, which it is not: the static + * `readonly` strip exempts `isSystem`, and this one runs in EVERY context + * because the value has nowhere to land for any caller. ⛔ Never a refusal + * either: a `400` would make `formula` the one caller-read-only type that + * refuses where `readonly`, `readonlyWhen` and the primary key strip, and it + * would refuse every round trip of every object that declares one. + * + * ## Where it runs + * + * Beside the declared-field door, as the first act of each write verb's body + * and of {@link ObjectQL.validate} over the same rows: before the defaults, the + * hooks and every other strip. So a hook is never handed a value that will not + * be stored (the #16344 invariant), a `formula` field also declared + * `readonly: true` is reported once, as `computed`, and the preview strips + * exactly what the write strips. The REPORT is the verb's own report site, + * beside the other strips' (`insertDrops` on insert, `reportDroppedFields` at + * the confluence on update) — so `strictReadonlyWrites`, whose coverage is + * derived from what `onFieldsDropped` reports, refuses a computed drop too. + * + * ⛔ Keyed on `type === 'formula'` literally — the test the read path's + * projection and `driver-sql`'s `fieldHasColumn` use. `summary` is NOT here: a + * roll-up HAS a column, and a caller-supplied value is persisted (measured on + * #20805, SQLite and memory) — a separate question, not this door's. + * + * Pure: the caller's objects are never mutated. A row keeps its reference when + * nothing was taken, else it is a shallow copy without the keys. + * + * @param refused the declared-field door's per-row verdicts, index-aligned: a + * row that door refused is left exactly as it is and counts toward nothing. + * @returns the rows as the strip leaves them (index-aligned with the input); + * the union of the keys it took, in first-seen order — one report per call, + * the shape every insert-side strip already reports in; and `droppedPerRow`, + * the keys taken from EACH row (index-aligned, `[]` for a row it left alone). + * The union and the per-row lists are the same strip's one result read two + * ways: the per-row lists are what {@link droppedFieldEvents} attributes + * to a row on `validate` and on `insertMany`'s outcomes. + */ +export function stripComputedWriteFields( + schema: { fields?: unknown } | undefined, + rows: readonly unknown[], + refused?: readonly unknown[], +): { rows: unknown[]; dropped: string[]; droppedPerRow: string[][] } { + const out = rows.slice(); + const dropped: string[] = []; + const droppedPerRow: string[][] = rows.map(() => []); + const fields = schema?.fields; + if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return { rows: out, dropped, droppedPerRow }; + const computed = Object.entries(fields as Record) + .filter(([, def]) => def?.type === 'formula') + .map(([name]) => name); + if (computed.length === 0) return { rows: out, dropped, droppedPerRow }; + for (let i = 0; i < out.length; i++) { + if (refused?.[i] !== undefined) continue; + const row = out[i]; + if (!row || typeof row !== 'object' || Array.isArray(row)) continue; + // Own-property, never `in` — the same reason `stripReadonlyFields` gives. + const taken = computed.filter((name) => Object.prototype.hasOwnProperty.call(row, name)); + if (taken.length === 0) continue; + const copy: Record = { ...(row as Record) }; + for (const name of taken) { + delete copy[name]; + if (!dropped.includes(name)) dropped.push(name); + } + droppedPerRow[i] = taken; + out[i] = copy; + } + return { rows: out, dropped, droppedPerRow }; +} + +/** + * The strip report as `DroppedFieldsEvent`s: one event per reason, `computed` + * before `readonly`, then `stored_expression` (#24690) — the order the passes + * run — and an empty list when nothing was taken. + * + * ONE builder for both channels the create-side strips report through, so + * they cannot disagree on a reason or its order: the batch-level union (the + * `onFieldsDropped` events of `insert` and {@link ObjectQL.validate}, unchanged + * by the per-row channel) and the per-row lists (`results[i].droppedFields` on + * `validate`, `droppedFields` on an `ok` outcome of `insertMany`). The per-row + * lists are recorded at the strips themselves, never reconstructed from the + * union: "which rows supplied N" is not "which rows dropped N" once a hook can + * exempt a key on one row and not another. + */ +export function droppedFieldEvents( + object: string, + computed: readonly string[], + readonly: readonly string[], + storedExpression: readonly string[] = [], +): DroppedFieldsEvent[] { + const events: DroppedFieldsEvent[] = []; + if (computed.length > 0) events.push({ object, fields: [...computed], reason: 'computed' }); + if (readonly.length > 0) events.push({ object, fields: [...readonly], reason: 'readonly' }); + // [#24690] Last: the stored-expression pass runs after every strip. + if (storedExpression.length > 0) events.push({ object, fields: [...storedExpression], reason: 'stored_expression' }); + return events; +} + +/** + * [#21682] The keys the CALLER sent on each row of an insert, recorded at + * `insert`'s entry, BEFORE the middleware chain runs, and keyed by the row + * OBJECT. + * + * `insert` takes its caller snapshot (`suppliedPerRow`) inside the middleware + * chain's innermost step, so by then a write middleware may already have + * filled the payload: `@objectstack/organizations` fills an absent + * `organization_id` with the active organization, and + * `@objectstack/plugin-security` fills an absent `owner_id` with the acting + * user. Both write IN PLACE, onto the very row objects recorded here. Without + * this record the snapshot reads those fills as keys the caller sent, so the + * static-`readonly` strip took the platform's own `organization_id` and + * `droppedFields` reported it, on every walled create that named no + * organization. The console announces every non-empty `droppedFields` as a + * warning toast. This is the insert-side twin of the update path's #8093 + * (ADDRESSING IS NOT PAYLOAD): a value the platform put on the payload is + * not one the caller lost. + * + * Keyed by IDENTITY, not by index, so the answer survives a middleware that + * reorders a batch. A row a middleware REPLACED wholesale has no entry, and + * {@link callerSuppliedRow} then keeps every key it carries: the verdict from + * before this record existed. That is the over-reporting direction, never the + * under-stripping one. + * + * ⛔ No name list. Which keys are the platform's is answered by WHEN they + * appeared, so a new stamping middleware is covered without being named here. + */ +function callerKeySets(data: unknown): WeakMap> { + const sets = new WeakMap>(); + for (const row of Array.isArray(data) ? data : [data]) { + if (row !== null && typeof row === 'object' && !sets.has(row)) { + sets.set(row, new Set(Object.keys(row))); + } + } + return sets; +} + +/** + * One row of `insert`'s caller snapshot: a shallow COPY of the row as the + * middleware chain handed it on, keeping only the keys the caller sent + * (`sent`, from {@link callerKeySets}). + * + * Only WHICH keys is narrowed. A kept key keeps the value the chain handed + * on, so every key the caller did send is judged exactly as it was before + * #21682. `sent` undefined means "this row has no record" and keeps every key. + */ +function callerSuppliedRow(row: unknown, sent: ReadonlySet | undefined): Record { + const copy: Record = { ...((row ?? {}) as Record) }; + if (sent) { + for (const key of Object.keys(copy)) { + if (!sent.has(key)) delete copy[key]; + } + } + return copy; +} + +/** + * [#24822] Whether ONE write's static-`readonly` passes leave a non-system + * caller's `organization_id` on the row for the Layer 0 wall's verdict instead + * of taking it. The ONE answer both write verbs read: the insert's subject + * ({@link insertStaticReadonlySubject}) and the update's + * ({@link updateStaticReadonlySubject}). + * + * Three things must hold, and the first is the wall's own word: + * + * 1. `wallJudgesOrganization`: the seam the wall installed on THIS write + * declares that it judges the stored rows' organization + * (`PostHookWriteImageCheck.judgesOrganization`). The caller reads that + * member once, for a non-system caller only. [#25062] The write preview + * reads the same value from the wall's verdict on each row + * (`WriteWallRowVerdict`). Nothing here derives it: where no wall judged + * the column, the strip takes a caller's value exactly as before. + * 2. The object is one the INSERT strip judges, with the column in that + * subject ({@link staticReadonlyInsertSubject}). So never the reserved + * `sys_` namespace or a platform-internal `managedBy` bucket, whose own + * field-write governance a spare must not pre-empt. The UPDATE strip runs + * on those objects too (it applies neither exclusion), so on an update + * this condition is what keeps a `sys_` row's organization exactly as + * before: the update spare covers exactly the objects the insert spare + * covers. + * 3. The column is the platform's injected tenant column, the one + * `TENANT_SCOPE_FIELD_DEF` declares read-only on the platform's behalf. An + * AUTHORED `organization_id` declared `readonly: true` is the author's own + * declaration, and declaration is enforcement: it keeps the strip. Which + * one this is, is the registry's own provenance reader, never a second + * copy. + * + * Why the column is left on the row where all three hold: the wall has + * already refused, before this engine ran, every supplied value it does not + * admit, and it judges the stored rows again after the hooks: an insert's + * rows, an update's prior row merged with the final payload, and on the + * predicate path every matched row merged with it. So a value that reaches the + * strip is one the wall admitted, and taking it put another organization in + * its place (the active one stamped on an insert, the row's own kept on an + * update). That is a silent replacement of an admitted input, which ADR-0105 + * D5 and the organizations runtime's stamp rule out. + */ +function staticReadonlySparesOrganization(schema: unknown, wallJudgesOrganization: boolean): boolean { + if (!wallJudgesOrganization) return false; + const subject = staticReadonlyInsertSubject(schema as any); + if (!subject || !Object.prototype.hasOwnProperty.call(subject.fields, SystemFieldName.ORGANIZATION_ID)) return false; + return resolveInjectedColumnProvenance(schema, SystemFieldName.ORGANIZATION_ID) !== 'author'; +} + +/** + * [#24822] The fields ONE insert's static-`readonly` passes may judge: the + * {@link staticReadonlyInsertSubject} view, without `organization_id` where + * {@link staticReadonlySparesOrganization} spares it. + * + * ONE function for both passes that read the subject, the pre-hook withhold + * and the post-hook strip, so the hook view and the stored row cannot + * disagree about the column. [#25062] The write preview's strip reads it too. + */ +export function insertStaticReadonlySubject( + schema: unknown, + wallJudgesOrganization: boolean, +): ReturnType { + const subject = staticReadonlyInsertSubject(schema as any); + if (!subject || !staticReadonlySparesOrganization(schema, wallJudgesOrganization)) return subject; + const fields = { ...subject.fields }; + delete fields[SystemFieldName.ORGANIZATION_ID]; + return Object.keys(fields).length > 0 ? { name: subject.name, fields } : null; +} + +/** + * [#24822] The schema view ONE update's static-`readonly` passes judge: the + * object's whole registered schema, because the update strip applies neither + * of the insert subject's exclusions, without `organization_id` where + * {@link staticReadonlySparesOrganization} spares it. `schema` itself + * otherwise, so every update the spare does not reach is judged exactly as + * before. + * + * ONE function for every pass that asks the update's static strip: the + * pre-hook hide pass (what `beforeUpdate` is shown), the by-id and predicate + * strips (what is stored), and {@link staticReadonlyStoredView}, which the + * `readonlyWhen` strips and the master-detail settlement read. `update()` + * reads the wall's flag once and hands that one value to all of them, so the + * hook view and the stored row cannot disagree about the column. [#25062] The + * `update`-mode preview's strip reads it too. + */ +export function updateStaticReadonlySubject(schema: unknown, wallJudgesOrganization: boolean): unknown { + if (!staticReadonlySparesOrganization(schema, wallJudgesOrganization)) return schema; + const { name, fields } = schema as { name?: string; fields: Record }; + const kept = { ...fields }; + delete kept[SystemFieldName.ORGANIZATION_ID]; + return { name, fields: kept }; +} + +/** + * [#22306] The insert side of the #16344 invariant — a hook is never handed a + * value that will not be stored. Takes out of ONE caller row every value the + * create-side static-`readonly` strip will take, before the defaults and + * before `beforeInsert`, and says what it took. + * + * ## What was measured broken + * + * `insert()` dispatched `beforeInsert` on the caller's payload and ran the + * runtime-owned and static-`readonly` strips after it. A hook that stamps a + * read-only column only when it is absent — `if (!data.stage_entry_date)` — + * saw the caller's value, stood down, and the strip then took that value: the + * row stored NULL where the hook would have stamped today. Real engine over + * `driver-sql` (better-sqlite3), non-system context: caller value present ⇒ + * stored `stage_entry_date` NULL; key absent ⇒ stored the hook's stamp. The + * update path has hidden these values from its hooks since #16344; insert + * did not. + * + * ## The SAME function as the strip, so the two cannot disagree + * + * `stripReadonlyFields` over {@link staticReadonlyInsertSubject} with no + * `preserveAudit` — the function, subject and options the post-hook + * static-`readonly` strip (`staticReadonlyCreateStrip`) runs. Before any hook + * has run there is no hook write to spare, so what it would take here is what + * it WILL take after the hooks unless a hook assigns the key. A key it would + * keep — middleware-filled (not in `supplied`), or on a `sys_` or + * platform-internal object — is still shown to the hook. + * + * ⛔ Runtime-owned values (`autonumber`) are NOT withheld, though the + * runtime-owned strip takes them after the hooks too. #6339 keeps the caller's + * record number visible to `beforeInsert` on purpose, so a guard can report + * what the caller submitted — pinned as "the hook can still SEE the + * caller-submitted record number" (`engine-insert-runtime-owned-strip.test.ts`) + * and by #14259's provenance seam. Moving that is a decision of its own, not a + * rider on this one. + * + * ⚠️ One narrowing, and it is the create side's own declared channel rather + * than a second opinion: under `preserveAudit` the audit timeline + * (`AUDIT_PROVENANCE_FIELDS`) stays visible. The static strip takes those keys + * on a create whatever the flag says (the 2026-08-08 ruling made its + * `preserveAudit` exemption UPDATE-only), but the historical-import channel the + * 2026-09-06 ruling (#15964) kept working reinstates them THROUGH the audit + * binder's `preserveAudit` branch (`record.created_at ?? now`, `plugin.ts`) — + * a `beforeInsert` hook reading the caller's value and assigning it back, which + * the strip then keeps as a hook write. Those values ARE stored, so they are + * not withheld; withholding them would erase every historical `created_at` an + * import carries. A business `readonly` column under `preserveAudit` is still + * withheld: the create side strips it (and says so in the WARN). + * + * ## Why before the DEFAULTS + * + * So the hook is shown exactly the row a caller who never sent the key would + * have produced: `applyFieldDefaults` then fills a withheld key from its + * `defaultValue`, just as it fills any absent key, and that default IS what + * the create stores for it (`staticReadonlyCreateStrip` re-derives it after + * the strip). A forged `approval_status: 'approved'` is shown as `'draft'`, + * never as a hole — the hook cannot tell the two callers apart, which is the + * point. A key with no default reads absent, which is what the row will hold. + * + * Pure: neither the caller's row nor `supplied` is touched. The row keeps its + * reference when nothing was taken, else it is a shallow copy without the keys. + * + * [#24822] `wallJudgesOrganization` spares `organization_id` here exactly as + * the post-hook strip spares it ({@link insertStaticReadonlySubject}). A value + * the wall judges will be stored, so the hook is shown it. + * + * @returns the row the hook view is built from, and the withheld keys with the + * caller's values (`undefined` when nothing was withheld) — handed back after + * the hooks by {@link handBackWithheldInsertReadonly}. + */ +function withholdInsertReadonlyFromHooks( + schema: unknown, + row: Record, + supplied: Readonly>, + preserveAudit: boolean, + wallJudgesOrganization: boolean, +): { row: Record; withheld: Record | undefined } { + let subject = insertStaticReadonlySubject(schema, wallJudgesOrganization); + if (subject && preserveAudit) { + const fields = { ...subject.fields }; + for (const name of AUDIT_PROVENANCE_FIELDS) delete fields[name]; + subject = Object.keys(fields).length > 0 ? { name: subject.name, fields } : null; + } + if (!subject) return { row, withheld: undefined }; + // No logger and no `strictReadonlyWrites`, deliberately: this pass is SILENT + // by construction. The strip after the hooks owns every word said about + // these keys — the WARN, `onFieldsDropped`, the strict refusal — and says it + // over the handed-back payload exactly as it did before this pass existed. + const shown = stripReadonlyFields(subject as any, row, supplied, undefined) as Record; + if (shown === row) return { row, withheld: undefined }; + const withheld: Record = {}; + for (const key of Object.keys(row)) { + if (!(key in shown)) withheld[key] = row[key]; + } + return { row: shown, withheld }; +} + +/** + * [#22306] The other end of {@link withholdInsertReadonlyFromHooks}: put the + * caller's withheld values back on a row after its `beforeInsert` dispatch is + * SEALED, wherever the hook did not write the key — so the strips below judge, + * take, re-default, WARN, report and (under `strictReadonlyWrites`) refuse + * exactly the payload they judged before the hide existed. + * + * The update path's hand-back (#16344), carried over with one addition the + * insert view needs: a withheld key may have been SHOWN holding its + * `defaultValue` (`shown`), so "the hook left it alone" is not always "the key + * is absent". Per key: + * + * - present and `undefined` ⇒ the `data.x = data.x` self-assignment of a key + * the hook was shown absent. Undone (deleted, dropped from the record) and + * handed back, as on update: no driver can store `undefined`, and the write + * must read exactly as it would with no hook at all. + * - absent ⇒ handed back unless the hook was shown a value there, in which + * case the hook REMOVED it, and the removal stands — the row a caller who + * never sent the key would have stored. + * - present, record available ⇒ handed back unless a hook assigned the key + * (`hookWrittenKeys`): an untouched default is replaced by the caller's + * value, which the static strip then takes and re-defaults to the same + * value it showed. + * - present, no record (a hook replaced the payload object) ⇒ the #5591 value + * test: handed back only while it still holds exactly what was shown. + * + * ⛔ Never over a hook's write, and placed AFTER the seal: a hand-back inside + * the recording window would enter the record as a hook write, and the strip + * would then keep the caller's forgery as the hook's — the laundering #14088 + * exists to prevent. Every key handed back is dropped from the returned record + * for the same reason. + * + * @returns the row's `hookWrittenKeys`, narrowed by every key handed back + * (`undefined` stays `undefined`). + */ +function handBackWithheldInsertReadonly( + target: Record, + withheld: Readonly>, + shown: ReadonlyMap, + hookWrittenKeys: ReadonlySet | undefined, +): ReadonlySet | undefined { + let narrowed: Set | undefined; + for (const [key, callerValue] of Object.entries(withheld)) { + const present = Object.prototype.hasOwnProperty.call(target, key); + let handBack: boolean; + if (present && target[key] === undefined) { + delete target[key]; + handBack = true; + } else if (!present) { + handBack = !shown.has(key); + } else if (hookWrittenKeys !== undefined) { + handBack = !hookWrittenKeys.has(key); + } else { + handBack = shown.has(key) && Object.is(target[key], shown.get(key)); + } + if (!handBack) continue; + target[key] = callerValue; + if (hookWrittenKeys?.has(key)) { + narrowed ??= new Set(hookWrittenKeys); + narrowed.delete(key); + } + } + return narrowed ?? hookWrittenKeys; +} + +/** + * Evaluate formula virtual fields against the raw rows a driver handed back — + * the read path (`find` / `findOne`) and, since #5504, the write path's + * response hydration. + * + * Since #10280 this is also where a formula field's declared `scale` is + * applied: the value is rounded on its way onto the record, so every consumer + * of every one of those three call sites inherits the same rounded number. + * {@link roundFormulaValue} carries the arithmetic and the reason the rounding + * belongs at this producer rather than in the validator. + * + * The eval context is built ONCE per call and reused for every row × every + * formula field, and that is where this function's determinism comes from: one + * `now`, so a `now()`/`today()` formula cannot drift mid-operation, plus + * `os.user` / `os.org` resolved from the execution context (so a computed field + * can reference the caller, e.g. `os.user.id`). Previously this passed only + * `{ record }`, so `now()`/`today()` ran against live wall-clock and user/org + * were unreachable. + * + * [#20082] `permissions` is the acting subject's effective object-permission + * map, the one `current_user.can(object, verb)` answers from. The CALLER + * resolves it — at most once per operation, and only when a planned formula + * calls `can` ({@link ObjectQL.resolveFormulaPermissions}) — because this + * function is synchronous and owns no resolver. `undefined` is "no permission + * data", never an empty map: a `can`-calling formula then fails to evaluate and + * reads `null`, the rule every formula that does not evaluate follows here, and + * the caller has already said why in its log line. + * + * That context has the same SHAPE as `applyFieldDefaults`' — the same keys, so + * one expression vocabulary serves `formula` and `defaultValue` alike — but NOT + * the same `now` value, and the two are sourced independently on purpose + * (#5699): + * - `applyFieldDefaults` is handed the insert's `nowSnapshot`, so every + * defaulted field of every row in one write carries the same PRE-write + * instant; + * - this function reads the clock itself, once per call, because a formula is + * evaluated when a record is MATERIALIZED — at read time, and on the write + * response — not at the moment that row's defaults were resolved. + * + * So inside a single `insert` a `NOW()` default and a `now()` formula observe + * two instants one driver round-trip apart (sub-millisecond in practice; across + * a second/day boundary they can land on different calendar days). Making them + * share one instant would hand the write path a determinism guarantee the read + * path cannot have — a semantic decision, not a tidy-up, argued in #5699. Until + * it is decided this function takes NO snapshot parameter: the zero-caller + * `nowSnapshot?: Date` it carried from birth was retired there, in the same + * enforce-or-remove reflex ADR-0049 applies to spec properties, because a + * dormant parameter reads as a live one and anyone reasoning from it concludes + * the two sides already share an instant. + * + * (ADR-0053 Phase 2 will additionally thread `timezone` here once + * `ExecutionContext.timezone` exists — see #1980; this change is independent + * of timezone.) + */ +/** + * [#10280] Apply a formula field's declared `scale` to the value its + * expression just produced. + * + * ## Why the rounding lives HERE and not in the validator + * + * `validateFieldValue`'s `scale` branch is the package's only other `.scale` + * reader and the tempting seam — and it is the wrong one. That branch enforces + * `scale` by REJECTION (`max_scale`) under the maintainer's 2026-08-11 ruling, + * because a caller-supplied value has somebody to refuse and silently rounding + * it would be silently altering their data. A formula result is + * platform-computed: there is nobody to refuse, and the alternative to + * rounding is not "the author's exact number", it is fifteen digits of IEEE-754 + * tail on a record page. Two different values, two different answers, one + * declaration — so the rounding is applied at the PRODUCER, on plan entries + * that are `type === 'formula'` only, and the rejection path is left + * structurally untouched. + * + * ## The arithmetic + * + * `Number(v.toFixed(scale))`, the same expression objectui's `computeRow` + * (`GridField.tsx`) uses for a grid's client-side computed column, so the two + * halves of one platform round a computed number the same way. Per ECMA-262 + * `toFixed` extracts the sign first and then picks the larger `n` on a tie: + * round-half-**away from zero** on the magnitude. Measured, because the + * opposite was believed at dispatch time: `2.5 → 3`, `1.25 → 1.3`, + * `0.125 → 0.13`, `-1.5 → -2`, `-2.5 → -3`. ⚠️ Negatives round away from zero, + * NOT toward `+Infinity`. + * + * Three guards, each closing a measured failure rather than a hypothetical: + * + * - **`typeof value === 'number'`** — a formula legitimately returns a string + * (`"tier:" + record.tier`), a boolean, or `null`, and `'x'.toFixed` does + * not exist. Without this guard one scale-declaring string formula throws + * and takes down every read of every record of that object. + * - **`Number.isFinite`** — `NaN.toFixed(2)` yields the STRING `'NaN'`, and + * `1.0 / 0.0` really does reach here as `Infinity`. Both round-trip through + * `Number(...)` unchanged, so this guard alters no value; it is kept so the + * helper is total by construction rather than by a coincidence of two + * library behaviours. (For `|v| >= 1e21` `toFixed` returns exponential form, + * `'1e+21'`, which `Number()` round-trips losslessly — the rounding degrades + * to a no-op instead of corrupting the value, so that case needs no guard.) + * - **negative zero** — `(-0.001).toFixed(2)` is `'-0.00'` and `Number()` + * makes that `-0`. It JSON-serializes as `0`, but `Intl.NumberFormat` + * formats it as `-0.00`, which would put a brand-new display wart on the + * very record page this issue is about. Normalized to `+0` at the producer, + * where it costs one comparison, rather than in each renderer. + */ +function roundFormulaValue(value: unknown, scale: number | undefined): unknown { + if (scale === undefined) return value; + if (typeof value !== 'number' || !Number.isFinite(value)) return value; + const rounded = Number(value.toFixed(scale)); + return Object.is(rounded, -0) ? 0 : rounded; +} + +/** + * [#22019] Where {@link applyFormulaPlan} reports a formula it could not + * evaluate: the field it was computing and the evaluator's own error. The + * engine binds one per call to the object being read, so the report names the + * (object, field) pair — see {@link ObjectQL.formulaFaultSink}. + */ +type FormulaFaultSink = (field: string, error: EvalError) => void; + +/** + * [#24197] What a formula read hands {@link applyFormulaPlan} for the hops its + * plan reads: the per-row related bindings + * ({@link ObjectQL.resolveFormulaRelated}), or — when this read serves no hop + * at all — the sentence saying why, which every traversing formula of the read + * then reports. + * + * [#24410] `undeclared` is decided per FORMULA, never per reference: the + * related fields a reference's target does not declare, which the shared read + * never asks for. A formula naming one of them reads `null` with the + * `undeclared-field` reason; a sibling reading only declared fields through the + * same reference is served from the shared read exactly as it is alone. + */ +type FormulaRelated = { + readonly bindings: (row: Record) => RelatedRecordBinding | undefined; + readonly undeclared: ReadonlyMap }>; +} | string; + +/** + * [#24410] The named related fields a reference's target does not DECLARE — + * the one reading both seams that serve a hop make of it: the shared loader + * ({@link ObjectQL.resolveOneHopRelated}) and the formula read, which judges + * them per formula ({@link ObjectQL.resolveFormulaRelated}). + * + * [#8215] The PRIMARY KEY counts as declared, though no field map carries it: + * `record..id` is the repair the traversal conflict refusal prescribes. An + * unknown target (`declared` absent) declares nothing to judge against, so + * nothing is undeclared. + */ +export function undeclaredRelatedFields( + declared: Record | undefined, + named: Iterable, +): string[] { + if (!declared) return []; + return [...named].filter((n) => n !== SystemFieldName.ID && !(n in declared)); +} + +/** [#24197] Why a hop was not served where no reader supplied one. */ +const FORMULA_HOP_NOT_SERVED = + 'this evaluation reads no related record — only a read (`find` / `findOne`) or a write\'s response ' + + 'preloads the one hop a formula reads through a reference'; + +/** + * [#24197] The record ONE formula is evaluated against: a shallow copy whose + * reference fields this formula reads through carry the related row, or the + * reason it cannot be evaluated at all. + * + * The validation path's verdicts, kept ({@link RelatedFieldBinding}): a related + * record that was not found — or that this reader may not read, which a + * caller-authority read cannot tell apart and must not — a related object that + * declares no such field, and a failed read each leave the formula UNEVALUATED, + * so it reads `null` WITH a reason, never a value computed over a record that + * was not there. ⛔ Never a fallback evaluated in its place: an optional read + * would turn "not found" into the author's empty-reference default, which is + * data the reader cannot tell from a real answer. + * + * An EMPTY reference is the one case that is evaluated: the author can say + * what an empty reference means (`record.book.?title.orValue('—')`), and the + * field keeps its stored empty value, so a plain read through it faults + * exactly as before and the fault names the repair (`empty`). + * + * Per formula, onto a copy: hydrating replaces a reference's stored id with a + * row, and a sibling formula that reads the id, and the record the read + * returns, must keep seeing the id. + */ +function formulaHopScope( + rec: Record, + hops: ReadonlyMap>, + related: FormulaRelated | undefined, +): { ok: true; record: Record; empty: string[] } | { ok: false; reason: string } { + if (related === undefined || typeof related === 'string') { + return { ok: false, reason: related ?? FORMULA_HOP_NOT_SERVED }; + } + // [#24410] This formula's own undeclared reads first, before any row: they + // are a fault of the expression, whatever the record holds. + for (const [field, named] of hops) { + const missing = related.undeclared.get(field); + const own = missing ? [...named].filter((n) => missing.fields.has(n)) : []; + if (missing && own.length > 0) { + return { + ok: false, + reason: formulaHopRefusal(field, { object: missing.object, unavailable: 'undeclared-field', undeclaredFields: own }, named), + }; + } + } + const binding = related.bindings(rec); + let copy: Record | undefined; + const empty: string[] = []; + for (const [field, named] of hops) { + const b = binding?.[field]; + if (!b) return { ok: false, reason: FORMULA_HOP_NOT_SERVED }; + // ⛔ Only the row this record's own foreign key names. + if (b.row && b.row.id != null && String(b.row.id) === String(rec[field])) { + if (!copy) copy = { ...rec }; + copy[field] = b.row; + continue; + } + if (!b.row && b.unavailable === 'no-reference') { empty.push(field); continue; } + return { ok: false, reason: formulaHopRefusal(field, b, named) }; + } + return { ok: true, record: copy ?? rec, empty }; +} + +/** + * [#24197] The sentence a formula that could not read its hop reports. It names + * the RELATED object and field, as the validation path's refusal does, and it + * never says WHICH of "not found" and "not readable by this reader" held: a + * reader who may not see the related record learns nothing from the formula. + */ +function formulaHopRefusal(field: string, b: RelatedFieldBinding, named: ReadonlySet): string { + const columns = [...named].sort().map((n) => `'${n}'`).join(', '); + const on = `\`${field}\` (object '${b.object}')`; + switch (b.unavailable) { + case 'undeclared-field': { + const missing = (b.undeclaredFields ?? []).map((n) => `'${n}'`).join(', ') || columns; + return `the formula reads ${missing} through ${on}, but '${b.object}' declares no such field: fix the ` + + `expression, or declare ${missing} on '${b.object}' — not on the object carrying this formula`; + } + case 'unreadable': + return `the formula reads ${columns} through ${on}, and that read failed, so it has no value`; + case 'no-reference': + case 'unresolved': + default: + return `the formula reads ${columns} through ${on}, and the record it names was not found by this ` + + 'read (it does not exist, or this reader may not read it), so it has no value'; + } +} + +/** + * What a formula field reports when it evaluated to the invalid Date that + * `date()` / `datetime()` answer for text they cannot parse + * ({@link isUnparseableDate}). An EMPTY source is not this fault: it reads + * `null` and says nothing. The record's text is not quoted — a log line is no + * place for row data. + */ +const UNPARSEABLE_DATE_FAULT = + 'date() / datetime() was handed text it cannot parse as a date, so the formula has no value on this record; ' + + 'they take ISO text (`2026-03-04`, `2026-03-04T09:30:00Z`), a timestamp or epoch milliseconds, and an empty ' + + 'source reads null without a report.'; + +function applyFormulaPlan( + plan: FormulaPlanEntry[], + records: any[], + execCtx?: ExecutionContext, + permissions?: EvalPermissions, + onFault?: FormulaFaultSink, + related?: FormulaRelated, + /** + * [#26608] What `label(record.)` reads: the object's option labels, + * built once per call by {@link readerOptionLabels} (a read, in the reader's + * locale) or {@link declaredOptionLabels} (a stored expression). `undefined` + * when no formula of the plan calls `label()`. + */ + optionLabels?: EvalOptionLabels, +): void { + if (!plan.length) return; + const now = new Date(); + const timezone = execCtx?.timezone; + const user = execCtx?.userId ? { id: String(execCtx.userId), positions: execCtx?.positions ?? [] } : undefined; + const org = execCtx?.tenantId ? { id: String(execCtx.tenantId) } : undefined; + for (const rec of records) { + if (rec == null) continue; + for (const fp of plan) { + // [#24197] A traversal the one-hop read cannot serve is not evaluated: + // it reads null and says why, in the lint's own words. + if (fp.hopConflict) { + rec[fp.name] = null; + onFault?.(fp.name, { kind: 'runtime', message: fp.hopConflict }); + continue; + } + let scope: Record = rec; + let empty: string[] = []; + if (fp.hops) { + const hop = formulaHopScope(rec, fp.hops, related); + if (!hop.ok) { + rec[fp.name] = null; + onFault?.(fp.name, { kind: 'runtime', message: hop.reason }); + continue; + } + scope = hop.record; + empty = hop.empty; + } + const evaluated = ExpressionEngine.evaluate(fp.expression, { now, timezone, user, org, permissions, optionLabels, record: scope }); + // An invalid Date minted from text `date()` could not parse evaluates + // `ok` (a predicate relies on that) but serializes to the `null` an empty + // source gives: for a field it is a fault, said like any other. + const r = evaluated.ok && isUnparseableDate(evaluated.value) + ? { ok: false as const, error: { kind: 'runtime' as const, message: UNPARSEABLE_DATE_FAULT } } + : evaluated; + if (r.ok) { + rec[fp.name] = roundFormulaValue(r.value, fp.scale); + continue; + } + // [#22019] The answer stays `null` — a formula that cannot be computed + // has no value, and changing what a read returns is a protocol change. + // What changes is that the fault is no longer swallowed (ADR-0032: a + // call site must not silently turn a bad expression into `null`): the + // sink names the field and the evaluator's error. + rec[fp.name] = null; + onFault?.(fp.name, empty.length === 0 ? r.error : { + ...r.error, + // [#24197] The fault on a record whose reference is EMPTY names the + // repair first; the evaluator's own message follows it. + message: `${empty.map((f) => `\`${f}\``).join(', ')} holds no single reference on this record (it is ` + + 'empty, or holds several), so the formula has no related record to read through it; to give the ' + + `formula a value while it is empty, write the optional read \`record.${empty[0]}.?.orValue('…')\`.\n${r.error.message}`, + }); + } + } +} + +/** + * Hydrate `formula` virtual fields onto the records a WRITE hands back (#5504). + * + * `applyFormulaPlan` used to hang off the read path only — `find` and `findOne` + * — so `POST /data/:object` and `PATCH /data/:object/:id` answered with the + * stored document, in which a formula field is not merely `null` but ABSENT + * (formulas are virtual: no driver ever returns a column for one). The very + * next `GET` of that same row carried every one of them, so a caller that + * rendered the create response — the natural thing to do, the response calls + * itself `record` — got a blank title on every object whose `nameField` points + * at a formula, and could not tell "not configured" from "not evaluated". + * Read-your-write, restored: the write response is now the same materialization + * a read produces. + * + * Deliberately the SAME plan builder and the SAME evaluation the read path uses + * — one formula semantic, not a write-path dialect: + * - `planFormulaProjection(schema, undefined)` is exactly find's no-projection + * branch: every formula field the schema declares, and no `projected` + * rewrite (a write returns whole rows, so there is nothing to project). + * It also carries the perf threshold unchanged — an object declaring no + * formula yields an empty plan and `applyFormulaPlan` returns at its first + * line, so a write on an ordinary object pays a field-name loop and nothing + * else. + * - the execution context is threaded exactly as find threads it, so `os.user` + * / `os.org` resolve identically on both sides. Widening what that context + * carries is #1979's work and stays out of here. + * - [#20082] the permission map comes from the same place find's does: + * `permissionsFor` is the engine's {@link ObjectQL.resolveFormulaPermissions} + * bound to the WRITE's own resolution, so a write that already resolved the + * map for a default or an option gate reuses it rather than asking again. + * + * Evaluates against the record the driver returned (a full row: `create` uses + * `RETURNING *`, `update` re-reads), so no extra round-trip is needed and no + * formula sees a partial record. Mutates in place, like the read path. + * + * Non-record entries (a `null` readback when the write moved the row out of the + * caller's scope; the affected-row COUNT a predicate update resolves to) are + * skipped rather than special-cased at each call site — the primitive would + * otherwise take a property assignment, which throws under ES module strict + * mode. + */ +async function hydrateWriteFormulas( + schema: any, + results: unknown[], + execCtx: ExecutionContext | undefined, + permissionsFor: (plan: FormulaPlanEntry[], records: readonly unknown[]) => Promise, + onFault?: FormulaFaultSink, + /** [#24197] The hops the plan reads, resolved over `records` as a read resolves them. */ + relatedFor?: (plan: FormulaPlanEntry[], records: ReadonlyArray>) => Promise, + /** [#26608] The option labels `label()` reads, in the writer's locale — the reader of this response. */ + optionLabelsFor?: (plan: FormulaPlanEntry[]) => EvalOptionLabels | undefined, +): Promise { + const records = results.filter( + (r): r is Record => r != null && typeof r === 'object', + ); + if (records.length === 0) return; + const { plan } = planFormulaProjection(schema, undefined); + if (plan.length === 0) return; + applyFormulaPlan( + plan, records, execCtx, await permissionsFor(plan, records), onFault, await relatedFor?.(plan, records), + optionLabelsFor?.(plan), + ); +} + +/** + * [#24619] The stored computed fields: every field that is NOT a `formula` + * and declares an `expression`. Such a field has a column, so a summary can + * roll it up (a summary aggregates stored columns, never an on-read formula), + * and its value is the engine's to compute on every write — the declared + * `expression` over the row the write stores. A client-sent value never + * stands: it is overwritten by the computed one, the way the console grid + * already recomputes the column client-side. Planned with the `formula` read + * plan's own entry builder ({@link formulaPlanEntry}), so the scale rounding + * and the one hop are the read's. + * + * An empty plan for an object declaring none — the common case — so a write + * on an ordinary object pays a field-name loop and nothing else. + */ +function planStoredExpressions(schema: any): FormulaPlanEntry[] { + const fields = schema?.fields; + if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return []; + const plan: FormulaPlanEntry[] = []; + for (const [name, def] of Object.entries(fields as Record)) { + if (def && typeof def === 'object' && def.type !== 'formula' && def.expression) { + plan.push(formulaPlanEntry(name, def, fields)); + } + } + return plan; +} + +/** [#25073] {@link storedExpressionsReading}'s per-source member reads, bounded like the rule validator's cache. */ +const storedExpressionReadsCache = new Map | null>(); +const STORED_EXPRESSION_READS_CACHE_CAP = 512; + +/** The `record.` names one plan entry reads, or `null` when the platform's member reader cannot say. */ +function storedExpressionReads(entry: FormulaPlanEntry): ReadonlySet | null { + const expr = entry.expression as { dialect?: string; source?: unknown } | undefined; + if (!expr || (expr.dialect !== undefined && expr.dialect !== 'cel') || typeof expr.source !== 'string') return null; + const hit = storedExpressionReadsCache.get(expr.source); + if (hit !== undefined) return hit; + const analysis = analyzeRelationshipTraversals(expr.source); + const reads = analysis ? new Set([...analysis.bareFields, ...analysis.traversals.keys()]) : null; + if (storedExpressionReadsCache.size < STORED_EXPRESSION_READS_CACHE_CAP) storedExpressionReadsCache.set(expr.source, reads); + return reads; +} + +/** + * [#25073] The entries of a stored-expression plan that read one of `fields` + * (the object's `autonumber` fields), directly or through another entry that + * does, in plan order: the stored fields an insert must compute again once + * the record number is issued, because every earlier pass ran before it + * existed. + * + * Reads are the platform's one member reader's (`analyzeRelationshipTraversals`, + * `record.x`, `record.?x`, `record['x']`, inside `has()` too), so this cannot + * disagree with the lint about which spellings name a field. An entry whose + * reads it cannot name (another dialect) counts as a reader: computing it + * twice costs a pass, missing it stores a stale value. + */ +function storedExpressionsReading(plan: FormulaPlanEntry[], fields: ReadonlySet): FormulaPlanEntry[] { + if (plan.length === 0 || fields.size === 0) return []; + const moved = new Set(fields); + const readers = new Set(); + for (let grew = true; grew;) { + grew = false; + for (const entry of plan) { + if (readers.has(entry)) continue; + const reads = storedExpressionReads(entry); + if (reads !== null && ![...reads].some((name) => moved.has(name))) continue; + readers.add(entry); + moved.add(entry.name); + grew = true; + } + } + return plan.filter((entry) => readers.has(entry)); +} + +/** The `autonumber` fields an object declares. */ +function autonumberFieldNames(schema: any): Set { + const names = new Set(); + const fields = schema?.fields; + if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return names; + for (const [name, def] of Object.entries(fields as Record)) { + if (def?.type === 'autonumber') names.add(name); + } + return names; +} + +/** What {@link computeStoredExpressions} reads its permission map, fault sink and hops from. */ +interface StoredExpressionServing { + permissionsFor: (plan: FormulaPlanEntry[], records: readonly unknown[]) => Promise; + onFault?: FormulaFaultSink; + relatedFor?: (plan: FormulaPlanEntry[], records: ReadonlyArray>) => Promise; + /** + * [#26608] The option labels `label()` reads — always the DECLARED ones + * ({@link declaredOptionLabels}): a stored value is computed once and read by + * every later reader and by search, so it never depends on the writer's locale. + */ + optionLabelsFor?: (plan: FormulaPlanEntry[]) => EvalOptionLabels | undefined; +} + +/** + * [#24619] Compute a stored-expression plan over each IMAGE — the row the + * write stores: the stored row merged with the patch on an update, the + * defaulted payload on an insert — and answer each image's computed values, + * index-aligned. Evaluates against shallow COPIES, so an image is never + * mutated; the caller writes the values onto the payload it hands the driver. + * An expression that does not evaluate computes `null`, the formula read + * path's answer, with the fault reported once through `onFault`. + */ +async function computeStoredExpressions( + plan: FormulaPlanEntry[], + images: ReadonlyArray>, + execCtx: ExecutionContext | undefined, + serving: StoredExpressionServing, +): Promise>> { + if (plan.length === 0 || images.length === 0) return images.map(() => ({})); + const scratch = images.map((image) => ({ ...image })); + applyFormulaPlan( + plan, scratch, execCtx, await serving.permissionsFor(plan, scratch), serving.onFault, + await serving.relatedFor?.(plan, scratch), serving.optionLabelsFor?.(plan), + ); + return scratch.map((row) => { + const values: Record = {}; + for (const entry of plan) values[entry.name] = row[entry.name]; + return values; + }); +} + +/** + * [#25073] The stored fields an insert computes again once it holds a record + * number ({@link storedExpressionsReading}), with what computing them reads. + */ +interface AutonumberReaders { + readonly plan: FormulaPlanEntry[]; + readonly execCtx: ExecutionContext | undefined; + readonly serving: StoredExpressionServing; + /** The {@link ObjectQL.registerInsertRecomputeListener} listeners, told per row; absent when none is registered. */ + readonly recomputed?: (row: Record, fields: string[]) => Promise; +} + +/** + * [#25073] Compute the readers again over rows that now carry their issued + * number, and write the values onto the rows the driver is handed — the step + * that makes the stored value the declared expression over the stored row. + * The replaced-value report is NOT made again: it judged the caller's value + * once, before validation and before any number existed. + */ +async function recomputeAutonumberReaders( + readers: AutonumberReaders | undefined, + rows: Array>, +): Promise { + if (!readers || readers.plan.length === 0 || rows.length === 0) return; + const computed = await computeStoredExpressions(readers.plan, rows, readers.execCtx, readers.serving); + rows.forEach((row, k) => Object.assign(row, computed[k])); + if (!readers.recomputed) return; + const fields = readers.plan.map((entry) => entry.name); + for (const row of rows) await readers.recomputed(row, fields); +} + +/** + * [#24619] The one value a PREDICATE update can store for each stored + * computed field, judged over every matched row. `updateMany` writes ONE + * patch to all of them, so a field is set only when the patch moves it to the + * same value on every row; a patch that does not move it at all (it touches + * none of the expression's inputs) leaves the column alone, and a caller-sent + * value for it is taken out of the patch either way — it never stands. A patch + * that moves it to different values on different rows cannot be one + * statement, and is refused before anything is written: update those rows by + * id, where each is computed from its own stored row. + */ +function settleBulkStoredExpressions( + plan: FormulaPlanEntry[], + patch: Record, + before: ReadonlyArray>, + after: ReadonlyArray>, + object: string, +): Record { + const out: Record = { ...patch }; + const refused: FieldValidationError[] = []; + for (const entry of plan) { + delete out[entry.name]; + const moved = after.some((values, i) => !Object.is(values[entry.name], before[i]?.[entry.name])); + if (!moved) continue; + const first = after[0]?.[entry.name]; + if (after.every((values) => Object.is(values[entry.name], first))) { + out[entry.name] = first; + continue; + } + refused.push({ + field: entry.name, + code: 'invalid_value', + message: `'${entry.name}' on '${object}' is computed from its expression on every write, and this ` + + 'update changes its inputs on rows where it then computes different values, which one update of ' + + 'all the matched rows cannot store; update those rows by id, so each is computed from its own row', + }); + } + if (refused.length > 0) throw new ValidationError(refused); + return out; +} + +/** + * [#24619] Write a by-id update's computed values onto its patch: a key the + * caller (or a hook) sent is overwritten, and a key the patch does not carry + * is added only when the computed value differs from the stored one — a patch + * that moves none of the expression's inputs writes nothing extra. + */ +function applyStoredExpressionsToPatch( + patch: Record, + computed: Record, + prior: Record | null | undefined, +): void { + for (const [name, value] of Object.entries(computed)) { + if (Object.prototype.hasOwnProperty.call(patch, name) || !Object.is(prior?.[name], value)) patch[name] = value; + } +} + +/** + * [#24690] The stored computed fields whose CALLER value this write replaced — + * reported through `onFieldsDropped` under `stored_expression`, so a client + * that sent `amount: 1` is told its value did not stand, as #3407 rules for + * every other write-path strip, rather than reading success over a row that + * holds the computed value. + * + * A field is reported when all three hold: + * - the caller SENT it (`supplied`, the write's entry snapshot of the + * caller's values — never the live payload, which the pre-hook pass and the + * hooks have since written to); + * - it is still on the payload the compute pass reads (`present`): a value + * another strip took — a stored expression field also declared `readonly` + * — is that strip's to report, under its own reason, never twice; + * - the value STORED differs from the caller's on at least one row + * (`stored`: the one row of an insert or a by-id update, every matched row + * of a predicate update). A client that sends the computed value — the + * console grid's consistent save — is told nothing, because nothing it + * sent was replaced. + * + * `Object.is`, the comparison the compute passes themselves use: the caller's + * values were normalized (blank and numeric strings) before the snapshot. + */ +function replacedStoredExpressionFields( + plan: readonly FormulaPlanEntry[], + supplied: Readonly> | undefined, + present: Readonly> | null | undefined, + stored: ReadonlyArray>>, +): string[] { + if (!supplied || !present) return []; + const replaced: string[] = []; + for (const { name } of plan) { + if (!Object.prototype.hasOwnProperty.call(supplied, name)) continue; + if (!Object.prototype.hasOwnProperty.call(present, name)) continue; + if (stored.some((row) => !Object.is(row?.[name], supplied[name]))) replaced.push(name); + } + return replaced; +} + +/** + * Materialize ONE declared `formula` field against a record already in hand — + * the read path's own evaluation, narrowed to a single field, with no round + * trip (#11293). + * + * ## Why this exists rather than a second evaluator + * + * A hook body cannot reach a formula field: `ctx.previous` / `ctx.input` carry + * STORED columns, and a formula is computed on read, so a body that wants the + * record's title has to rebuild the formula inline. Measured in the exemplar + * app: five inline reimplementations of a record title inside hook bodies, four + * of them re-composing a `nameField` that is a formula. Each copy can drift + * from the declaration it copies, silently — which is the whole defect, so the + * remedy must not itself be a copy. This calls + * {@link planFormulaProjection} + {@link applyFormulaPlan}: the same plan + * builder, the same `Expression` normalization (string shorthand → CEL + * envelope), the same `scale` rounding and the same evaluation scope the read + * and write paths use. One formula semantic, not a hook-path dialect (PD #12). + * + * ## Narrowed to one field ON PURPOSE + * + * `planFormulaProjection(schema, undefined)` — the shape `find` and + * {@link hydrateWriteFormulas} use — plans EVERY formula field on the schema + * and `ExpressionEngine.compile`s each one at planning stage. Asking for one + * title would then throw on an unrelated malformed formula elsewhere on the + * object. Passing `[field]` plans exactly the requested field, so the blast + * radius of a title lookup is the title's own declaration. + * + * ## Read-path parity, including how it fails + * + * Both failure modes are the read path's, unchanged: a formula that does not + * COMPILE throws (as it does on every `find` of the object), while a formula + * that compiles and does not EVALUATE yields `null` — `applyFormulaPlan`'s own + * `r.ok ? … : null`. A caller therefore cannot mistake "this title could not be + * computed" for a computed value. + * + * ⚠️ One scope difference, named rather than hidden [#20082]: this helper is + * synchronous and passes NO permission map, so a formula that calls + * `current_user.can(…)` does not evaluate here and yields `null` by the rule + * above. [#24197] For the same reason it preloads no relationship hop, so a + * formula reading `record..` yields `null` here too. The read and write paths resolve the map first + * ({@link ObjectQL.resolveFormulaPermissions}); a helper with no resolver cannot. + * [#26608] And it holds no i18n service, so `label(record.)` reads the + * DECLARED option labels here ({@link declaredOptionLabels}), whatever + * `execCtx.locale` says. + * A caller that holds a read channel and wants the read's answer calls + * {@link ObjectQL.serveFormulaField} (through `ScopedContext.serveFormulaField`). + * + * Returns `undefined` when `field` is not a declared formula field, which is + * how a caller tells "read the stored column instead" from "the formula + * produced nothing". Evaluates against a shallow COPY: `applyFormulaPlan` + * writes the value onto the record it is handed, and the records reaching here + * are the engine's own hook payloads, observed by everything downstream. + */ +export function evaluateFormulaField( + schema: unknown, + record: Record, + field: string, + execCtx?: ExecutionContext, +): unknown { + const { plan } = planFormulaProjection(schema as any, [field]); + if (plan.length === 0) return undefined; + const scratch: Record = { ...record }; + // [#24197] Synchronous, so it preloads no hop: a formula reading through a + // reference yields `null` here, as a `can()` formula does above. + applyFormulaPlan(plan, [scratch], execCtx, undefined, undefined, undefined, declaredOptionLabels(schema, plan)); + return scratch[field]; +} + +/** + * A hook body, as registered through {@link ObjectQL.registerHook} or bound + * from metadata by `bindHooksToEngine`. + * + * ## `after*` handlers run INSIDE the unit of work (#7477) + * + * An `afterInsert` / `afterUpdate` / `afterDelete` handler is dispatched + * before the enclosing transaction commits. The guarantee it may rely on is + * **"the write has been requested and will happen unless this unit of work is + * undone"** — not "the write happened": a later refusal in the same unit rolls + * the row back after this handler has already run. See + * {@link DISPATCHABLE_HOOK_EVENTS} for the full statement and for the paths + * that open such a unit; a handler whose side effects leave the engine is the + * one that has to tolerate it. + */ +export type HookHandler = (context: HookContext) => Promise | void; + +/** + * Per-object hook entry with priority support + */ +export interface HookEntry { + handler: HookHandler; + object?: string | string[]; // undefined = global hook + /** + * [#5928] Object name(s) SUBTRACTED from whatever `object` admits — the + * negative half of a registration's scope. Absent = subtract nothing. + * + * `object` alone can only say "these objects" (plus the `'*'` full set), and + * the two are interchangeable only over a CLOSED universe. This one is open: + * `applyObjectRegistryMutation` registers objects into a running engine on a + * successful `/meta` PUT, emitting no event a plugin could subscribe to, so a + * registrant that enumerated the complement of its skip list would keep a + * frozen list and silently stop covering every object registered after boot. + * For the audit plugin that is a compliance regression, and a silent one. + * + * Read only through {@link hookMatchesObject} — never re-implemented at a + * call site (see that function's note on the one-semantic rule). + */ + excludeObjects?: string | string[]; + priority: number; + packageId?: string; + /** + * [#9719, widened to `beforeUpdate` by #9974] Opt-in: ALSO dispatch this + * handler ONCE with the whole-operation context when a predicate + * (`multi: true`) WRITE arrives with no caller predicate at all + * (`options.where` absent or `null`) — BEFORE any matched row is resolved. + * + * Why the extra dispatch exists: since #5038/#5574 the predicate path + * dispatches `beforeUpdate` / `beforeDelete` per matched row, with + * `input.id` bound — so a guard that refuses the OPERATION SHAPE (the #4757 + * unscoped-multi-delete refusal on `sys_attachment`, the #4630 one on + * `sys_comment`) never saw the shape it guards: the by-id branch of the + * handler shadowed it on every dispatch, and a zero-match unscoped write + * dispatched nothing at all. "Nothing to authorize" and "nothing was ever + * queried" are different verdicts, so the shape check cannot ride the + * per-row fan-out; it needs the one dispatch that happens regardless of what + * the predicate matches. + * + * ## Why it covers both write verbs + * + * #9719 shipped it on `beforeDelete` only, and refused it elsewhere, because + * extending it to `beforeUpdate`'s predicate path changes what the engine + * ACCEPTS on the far hotter update path — a product-behaviour decision, not + * drift. That decision was taken (#9974, maintainer ruling 2026-08-19, + * option A) on the recoverability asymmetry: a delete leaves a trace of who + * removed what, an overwrite leaves none — the old value is gone on the spot + * with nothing to restore from — and a forgotten `where` is the mistake + * generated code makes most often. So the LESS guarded verb was the one + * whose failure is LESS recoverable. One flag now covers both, rather than a + * `dispatchUnscopedMultiUpdate` sibling: the flag is already per-REGISTRATION + * and per-EVENT, so "delete only" is still said exactly — declare the flag on + * the `beforeDelete` registration and omit it from the update one, and the + * whole-operation call is then dispatched for the declared verb alone (the + * dispatcher reads the entries of ONE event, then filters those on the flag). + * + * ⚠️ That idiom is deliberately stated WITHOUT naming a live registration. + * It used to name one: `sys_attachment` declared #4757's delete refusal on + * `beforeDelete` and on no update registration — true when this paragraph was + * written (#9974), false since commit da891e0ef gave that guard's update verb a refusal + * of its own. A replacement exemplar would only be the next referent free to + * move, so none is named: the past-tense sentence cannot be falsified by the + * tree moving on, because it is a claim about what those two cards did rather + * than about today's registrations. + * + * Deliberately a REGISTRATION declaration, not an engine-wide rule: the + * engine stays neutral (no behaviour change for objects whose guards do not + * declare it — generalizing the refusal to every guarded object is a + * separate product decision), and the policy — who is refused, with what + * error — stays in the ONE handler that already declares and tests it. Valid + * on `beforeUpdate` / `beforeDelete` registrations only + * ({@link assertValidUnscopedMultiWriteFlag}); dispatched by + * {@link ObjectQL.dispatchUnscopedMultiWriteHooks}. + */ + dispatchUnscopedMultiWrite?: boolean; + /** + * Original metadata-form `Hook` definition this entry was bound from + * (when registered via `bindHooksToEngine`). Pure code-paths that call + * `engine.registerHook` directly leave this undefined. + */ + meta?: any; + /** Hook `name` from metadata; used for diagnostics & deduplication. */ + hookName?: string; +} + +/** `object` / `excludeObjects` in either accepted spelling → a name list. */ +function hookTargetList(target: string | string[] | undefined): string[] { + if (target === undefined) return []; + return Array.isArray(target) ? target : [target]; +} + +/** + * [#5928] Does `entry` cover `objectName`? **The** answer — one function, both + * consumers. + * + * ## Why this is a function and not two `if`s + * + * `triggerHooks` (dispatch) and `hasHooksFor` (the #5038 / #5284 bulk-write + * gate that decides whether the matched row set is worth reading) used to carry + * this rule as two hand-written copies of one semantic. `hasHooksFor`'s own + * comment named the hazard that arrangement carries: a gate LOOSER than the + * dispatch only wastes a query, but a gate TIGHTER than the dispatch silently + * drops hooks that were going to fire. Two copies stay in agreement only for as + * long as everyone who edits one remembers the other, and #5928 adds a second + * dimension to the rule — exactly the edit that would have desynchronised them. + * So the copies are gone: both consumers call this, and the property test in + * `hook-exclude-objects.test.ts` pins the implication (`dispatched ⇒ gate open`) + * over the full allow × exclude matrix rather than trusting the structure. + * + * ## The semantic + * + * `matches = allowMatches && !excludeMatches`, where an absent `object` admits + * everything (global) and `'*'` in `object` does the same explicitly. + * `excludeObjects` names are matched literally — `'*'` is refused at + * registration, so it can never reach this as a subtract-everything wildcard. + * + * The allow half keeps the TRUTHINESS test both copies used verbatim, rather + * than the `!== undefined` that reads more precisely. They differ on exactly one + * input, `object: ''`, which reads here as a GLOBAL hook (falsy ⇒ no filter). + * That read is deliberately UNCHANGED, and commit 708431313 says why: flipping it would turn + * a hook firing on everything into one firing on nothing, silently — the same + * class of defect pointing the other way. The shape is closed at the + * registration door instead ({@link assertValidHookObject}), so no live entry + * can carry `''` and this branch is unreachable for it in practice. It stays + * because this function is also called directly on hand-built entries, and + * because a matcher should describe one rule, not re-litigate the door's. + */ +export function hookMatchesObject( + entry: Pick, + objectName: string, +): boolean { + // Allow half: absent = global; otherwise the wildcard or a literal name. + if (entry.object) { + const targets = hookTargetList(entry.object); + if (!targets.includes('*') && !targets.includes(objectName)) return false; + } + // Subtract half: any literal hit removes the object from the admitted set. + if (entry.excludeObjects) { + if (hookTargetList(entry.excludeObjects).includes(objectName)) return false; + } + return true; +} + +/** + * [#5928] Registration-time refusal for the `excludeObjects` face, following + * the sister ruling on empty hook targets (#4281 / #4001, ADR-0078 "no silently + * inert declaration"). + * + * Two shapes are refused, both statically decidable at the call site: + * + * - **`''` / `['']`** (or any blank member) — no object is named `''`, so the + * entry subtracts nothing while reading as though it subtracts something. + * Inert, and inert in the direction that quietly keeps a hook firing on + * objects its author believed they had excluded. + * - **`'*'` anywhere in the list** — subtracting the full set from any allow + * set leaves the empty set: a hook that can never fire, registered + * "successfully". That is ADR-0078's silently-inert declaration exactly, and + * the same never-fire shape #4281 refused when `['']` produced it from the + * other side. + * + * `[]` is deliberately ACCEPTED. It is the honest spelling of "subtract + * nothing" — identical in meaning to omitting the key, and the natural value of + * a spread (`excludeObjects: [...SKIP_OBJECTS]`) whose source list is empty. + * The refusal above is not "empty is bad"; it is "a name that matches nothing" + * and "a name that matches everything". #4281 refused `[]` on `object` for a + * reason that does not exist here: there, the binder WIDENED the empty target + * into `'*'`, so blank intent silently became the maximum blast radius. Nothing + * transforms this value. + * + * A throw, not a warn: `excludeObjects` ships in this release with no callers, + * so strictness costs no migration, and unlike the unknown-event branch above + * (where a custom driver dispatching its own events is a legitimate reading) + * neither refused shape has one. + */ +function assertValidHookExcludeObjects( + excludeObjects: string | string[] | undefined, + event: string, +): void { + if (excludeObjects === undefined) return; + const names = hookTargetList(excludeObjects); + for (const name of names) { + if (typeof name !== 'string' || name.trim().length === 0) { + throw new Error( + `[ObjectQL] Hook '${event}' declares an empty \`excludeObjects\` entry. ` + + 'No object is named \'\', so it would subtract nothing while reading as if it ' + + 'subtracted something — a hook still firing on objects it appears to exclude. ' + + 'Name the object(s) to exclude — `excludeObjects: [\'sys_audit_log\']` — or omit ' + + 'the option entirely (an empty array is accepted and subtracts nothing).', + ); + } + if (name === '*') { + throw new Error( + `[ObjectQL] Hook '${event}' excludes the wildcard '*', which subtracts every ` + + 'object and leaves a hook that can never fire (ADR-0078: no silently inert ' + + 'declaration). Exclude the object names you mean — ' + + '`excludeObjects: [\'sys_audit_log\']` — or, if the hook really should not be ' + + 'registered, do not register it.', + ); + } + } +} + +/** + * [commit 708431313] Registration-time refusal for the `object` (ALLOW) face, closing + * #4281 / #4001's "an empty target is not *no* target" ruling on the path that + * ruling never reached. + * + * #4281 shut this shape at the two METADATA doors — `HookSchema.object`'s + * refine in `packages/spec`, and `normalizeObjects` in `hook-binder.ts`. The + * code door, `engine.registerHook`, goes through neither, so the same three + * spellings still walked in here, and the matching read (see + * {@link hookMatchesObject}) turns each of them into a defect: + * + * - **`''`** is FALSY, so the allow half is skipped entirely and the entry + * registers as a GLOBAL hook. #4281's headline failure mode exactly — blank + * intent becoming the broadest possible blast radius — reproduced verbatim + * on the uncovered path. + * - **`[]` / `['']`** (or any blank member) are truthy but admit no object + * name, so the entry can never fire: registered "successfully", inert + * forever. ADR-0078's silently-inert declaration. + * + * Refused at REGISTRATION rather than fixed in the matcher, and that choice is + * the whole point of the separate card. Teaching `hookMatchesObject` to read + * `''` as a real (unmatchable) name would silently convert a hook firing on + * every object into one firing on none — the same class of defect pointing the + * other way, which is why #5928 declined to do it in passing. A throw at the + * door changes no dispatch and leaves nothing to misread. + * + * Note the asymmetry with {@link assertValidHookExcludeObjects}, which + * deliberately ACCEPTS `[]`: on the subtract face an empty list is the honest + * spelling of "subtract nothing", identical to omitting the key. On the allow + * face an empty list says "admit nothing", which is a hook that can never fire. + * Same value, opposite meaning, because the faces compose in opposite + * directions — and `[]` is named in #4281's own message, so accepting it here + * would contradict the ruling this reuses. + */ +function assertValidHookObject( + object: string | string[] | undefined, + event: string, +): void { + if (object === undefined) return; + const names = hookTargetList(object); + const empty = + names.length === 0 + || names.some((name) => typeof name !== 'string' || name.trim().length === 0); + if (!empty) return; + throw new Error( + `[ObjectQL] Hook '${event}' declares an empty \`object\` target. An empty target is ` + + 'not "no target": `\'\'` is falsy, so the allow face is skipped entirely and the hook ' + + 'registers on EVERY object, while `[]` and `[\'\']` admit no object name at all, so the ' + + 'hook could never fire (ADR-0078: no silently inert declaration). Name the object(s) — ' + + "`object: 'account'` or `object: ['account', 'contact']` — or, if firing on every " + + "object really is the intent, write the wildcard explicitly: `object: '*'`.", + ); +} + +/** + * [commit 708431313] Refuse a scope whose two faces cancel each other out — + * `{ object: 'account', excludeObjects: 'account' }` and its list forms. + * + * The exclusion face subtracts from the allow face, so when the allow face is a + * FINITE enumeration and every name in it is also excluded, the admitted set is + * empty and the entry can never fire. #5928 named only three refusals (`''`, + * `['']`, and `'*'` in the excludes) and this shape falls outside their letter, + * so it was left registering silently — the same ADR-0078 inert declaration + * those three exist to prevent, reached by arithmetic instead of by a single + * bad name. + * + * Only a finite allow face can be decided here. `'*'` (and an absent `object`) + * admits an OPEN universe — `applyObjectRegistryMutation` registers objects + * into a running engine — so no finite exclusion list can empty it, and those + * scopes are left alone. `'*'` inside `excludeObjects` is the one exclusion + * that WOULD empty them, and it is already refused by + * {@link assertValidHookExcludeObjects}. + * + * Runs after both faces have been validated individually, so every name here is + * a non-blank string and the exclusion list carries no wildcard. + */ +function assertHookScopeNotSelfCancelling( + object: string | string[] | undefined, + excludeObjects: string | string[] | undefined, + event: string, +): void { + if (object === undefined || excludeObjects === undefined) return; + const allow = hookTargetList(object); + // An open allow face cannot be emptied by a finite subtraction. + if (allow.length === 0 || allow.includes('*')) return; + const deny = hookTargetList(excludeObjects); + if (deny.length === 0) return; + const denied = new Set(deny); + if (!allow.every((name) => denied.has(name))) return; + throw new Error( + `[ObjectQL] Hook '${event}' excludes every object its \`object\` target admits ` + + `(object: ${JSON.stringify(allow)}, excludeObjects: ${JSON.stringify(deny)}), leaving ` + + 'a hook that can never fire (ADR-0078: no silently inert declaration). An exclusion ' + + 'subtracts from a WIDER allow face — widen `object` (or drop it for a global hook) or ' + + 'remove the overlapping names from `excludeObjects`; if the hook really should not be ' + + 'registered, do not register it.', + ); +} + +/** + * [#9719, widened by #9974] The events whose predicate branch actually reads + * {@link HookEntry.dispatchUnscopedMultiWrite}. Declared as data beside the + * assert so the refusal message and the accept set cannot drift apart, and so + * adding a third verb is one edit rather than a rule to restate. + */ +const UNSCOPED_MULTI_WRITE_EVENTS = ['beforeUpdate', 'beforeDelete'] as const; + +/** + * [#9719, widened by #9974] Registration-time refusal for + * {@link HookEntry.dispatchUnscopedMultiWrite} on any event whose dispatch + * never reads it. + * + * The flag is consumed by exactly two dispatch sites — the predicate branches + * of `ObjectQL.update()` and `ObjectQL.delete()` — so on every other event it + * would register "successfully" and never fire: ADR-0078's silently inert + * declaration, the same shape the two scope asserts above refuse. Statically + * decidable, so refused at the door rather than warned about. + * + * ⚠️ The accept set widened by RULING, not to make the assert quieter: until + * #9974 this refused `beforeUpdate` too, and its comment named that as "a + * product-behaviour decision of its own". The decision was taken (option A, + * 2026-08-19), so `beforeUpdate` is now a declared reader — and every OTHER + * event is still refused on exactly the original grounds. + */ +function assertValidUnscopedMultiWriteFlag( + dispatchUnscopedMultiWrite: boolean | undefined, + event: string, +): void { + if (!dispatchUnscopedMultiWrite) return; + if ((UNSCOPED_MULTI_WRITE_EVENTS as readonly string[]).includes(event)) return; + throw new Error( + `[ObjectQL] Hook '${event}' declares \`dispatchUnscopedMultiWrite\`, which only the ` + + `${UNSCOPED_MULTI_WRITE_EVENTS.map((e) => `'${e}'`).join(' / ')} dispatches read — on this ` + + 'event the flag would register successfully and then never fire (ADR-0078: no silently ' + + 'inert declaration). Register the guard on one of those events, or drop the flag.', + ); +} + +/** + * [#17713] The per-seam prescription for a refused lifecycle event — what the + * author should have written instead, named concretely. + * + * The two seams are different repairs, and an author who is told only "this + * never fires" cannot tell them apart: + * + * - **`*FindOne`** — the capability EXISTS and the author already has it. + * `beforeFind`/`afterFind` fire for `findOne` too (the event attaches to + * record materialization, not to the engine method), so the fix is a + * one-word rename and nothing is lost. + * - **`*Count` / `*Aggregate`** — there is no hook seam at all, by design + * (`HookEvent` in `@objectstack/spec/data` says so: read authorization and + * row filtering are the middleware's job). `count()` and `aggregate()` run + * the middleware chain with the query AST on the operation context, so the + * repair is a DIFFERENT API, not a different event name. This is the half + * that would have saved the consumer on #17713: their read filter sat on + * `beforeCount`, so a `limit`ed list's `total` counted rows the caller could + * not see, and a `groupBy` was not narrowed at all. + */ +function undispatchedLifecycleEventPrescription(event: string): string { + if (event === 'beforeFindOne' || event === 'afterFindOne') { + const covering = event === 'beforeFindOne' ? 'beforeFind' : 'afterFind'; + return ( + `\`${covering}\` already covers \`findOne\` — the read events fire for BOTH \`find\` and ` + + `\`findOne\`, because the event attaches to record materialization, not to the engine ` + + `method. Register on '${covering}' instead; the handler sees single-record reads there.` + ); + } + const verb = event.startsWith('before') ? event.slice(6) : event.slice(5); + const method = `${verb.charAt(0).toLowerCase()}${verb.slice(1)}`; + return ( + `\`${method}()\` dispatches no hook at all, by design — it runs the MIDDLEWARE chain only, ` + + 'with the query AST on the operation context. Filter it with ' + + '`engine.registerMiddleware(fn)`, reading `ctx.operation === \'count\' | \'aggregate\'` and ' + + 'composing your predicate onto `ctx.ast.where` — the same seam RLS and sharing already use, ' + + 'so the scoping applies to the driver call itself. ⚠️ A read filter written as a HOOK does ' + + 'not narrow a `total` or a `groupBy`: those surfaces would keep answering over rows the ' + + 'caller cannot see.' + ); +} + +/** + * [#17713] Registration-time refusal for an event inside the engine's OWN + * lifecycle namespace that the engine never dispatches — see + * {@link UNDISPATCHED_LIFECYCLE_HOOK_EVENTS} for how that population is derived + * and why the refusal stops there. + * + * This was a `warn` from #3195 until #17713. The warning was loud (~40 lines a + * boot on the reporting install) and still useless: the registration SUCCEEDED, + * so a downstream consumer's read filters on `beforeFindOne` and `beforeCount` + * sat inert through every boot while the surface reported them registered. That + * is ADR-0078's prohibited fourth state — parsed, unmarked, silently inert — on + * an authorable seam, and on this seam the inert declaration is a GUARDRAIL: the + * author believes they narrowed what a caller can see, and they did not. + * + * A throw rather than a strict-mode-only failure, on the same reading as the + * four asserts above: the shape is statically decidable at the call site, it + * carries no recoverable intent that a warning could preserve, and a mode nobody + * turns on is a rule nobody gets. The `after*` names are refused alongside the + * `before*` ones even though the card only measured the `before*` half — they + * are the same gap, and refusing half a derived set would reintroduce the + * hand-written list. + * + * ⛔ NOT a general "unknown event names are refused". An event outside the + * namespace is left exactly as it was — a warn, then registered — so a plugin + * that dispatches its own events through the public {@link ObjectQL.triggerHooks} + * keeps working. That is the reading #3195 wrote down as the reason not to + * reject, and it is untouched here: this narrows the accept set by six names, + * not to eight. + */ +function assertDispatchableHookEvent(event: string): void { + if (!UNDISPATCHED_LIFECYCLE_HOOK_EVENTS.has(event)) return; + throw new Error( + `[ObjectQL] Hook '${event}' is an engine lifecycle event name the engine never dispatches, ` + + 'so the registration would succeed and the handler would never run (ADR-0078: no silently ' + + `inert declaration). ${undispatchedLifecycleEventPrescription(event)} ` + + `Dispatched events: ${[...DISPATCHABLE_HOOK_EVENTS].join(', ')}.`, + ); +} + +/** Function registry entry — see `registerFunction`. */ +export interface FunctionEntry { + handler: HookHandler; + packageId?: string; + /** + * What this function does to data, as DECLARED at registration (#4396). + * Only a `script`-node caller reads it: a flow function is contractually + * pure, and the run summary counts on that, so a function that legitimately + * writes declares `'writes'` and its step is reported as an effect the + * platform cannot count rather than as none. Absent ⇒ `'pure'`. + * + * Carried on the registry entry rather than beside it because this IS part of + * the registration — one registry, and what a caller needs to know about a + * function travels with the function. + */ + effect?: FlowFunctionEffect; +} + +/** + * Declarations that may accompany a `registerFunction` call. The bare + * `packageId` string is still accepted in that position (every existing caller + * passes one), so this widens the signature without a migration. + */ +export interface FunctionRegistrationOptions { + /** Owning package — the unit `unregisterFunctionsByPackage` removes. */ + packageId?: string; + /** Declared data effect (#4396); omit for the pure default. */ + effect?: FlowFunctionEffect; +} + +/** + * Operation Context for Middleware Chain + */ +export interface OperationContext { + object: string; + operation: 'find' | 'findOne' | 'insert' | 'update' | 'delete' | 'count' | 'aggregate'; + ast?: QueryAST; + data?: any; + options?: any; + context?: ExecutionContext; + result?: any; + /** + * [#15813 / ADR-0131 D8] What the Layer 0 tenant wall DECIDED for this + * operation — recorded by the enforcement layer (`@objectstack/plugin-security`) + * at the moment it composes the wall onto the operation's predicate, and + * read by the bulk data-event producer ({@link ObjectQL.publishBulkDataEvent}) + * to stamp `BulkDataEvent.organizationId`. + * + * The seam ruled on #15706: the wall is computed ONCE, where every input is + * visible (the posture in force, the caller's organization scope and the + * object's tenancy clauses — which, since ADR-0131 D7, include the + * deployment's #12699 platform-global declaration, recorded on the + * registered object as `systemFields.tenant: false`), and its decision + * travels here as a value. A reader + * answers from this member ALONE and re-derives nothing — a re-derivation + * is a mirror of the wall, and a mirror structurally sees only the clauses + * it was taught (the #15706 mislabel). + * + * Three states, all honest: ABSENT — no wall was composed on this + * operation (a system context, no enforcement layer mounted, a by-id write + * with no predicate); `{ kind: 'none' }` — the wall ran and contributed no + * predicate; a named verdict (`organization` / `organizations` / `deny`). + * The shape is `TenantLayer0VerdictSchema` (`@objectstack/spec/security`); + * a value that does not parse reads as absent, never as an organization. + */ + tenantLayer0Verdict?: TenantLayer0Verdict; + /** + * [commit a016f08b8] The INSERT post-image seam — where an enforcement layer gets to + * judge the row that will actually be stored. + * + * An `insert` has no pre-image, so a middleware's only image of the write is + * `opCtx.data`: the caller's payload as it arrived, before `applyFieldDefaults` + * and before the `beforeInsert` hooks. A value the app DERIVES server-side — + * an organization copied from a parent, a denormalised scoping field a caller + * is deliberately not allowed to choose — is not on that image, so a predicate + * over it judges a value that never lands and ignores the one that does. + * Measured on 17.3.0: the same identity, same object, same second, an insert + * carrying the stamped value was admitted and the identical insert leaving it + * to the hook was refused, while the stored row was identical either way — and + * an insert naming an IN-SCOPE value for a parent in ANOTHER organization was + * admitted with the parent's organization stored on it. + * + * So the enforcement layer installs its judgement here instead of running it + * against `opCtx.data`, and {@link ObjectQL.insert} calls it once the + * `beforeInsert` chain has produced the row — before the first producer with + * a side effect (the secret channel, the autonumber, the statement), so a + * refusal still costs nothing. An ARRAY insert is one operation: every live + * row is judged in the same call. + * + * [#19950] A PREDICATE `update` (`multi: true`, no row address) uses the same + * seam. The middleware cannot know which rows the write will change: they + * are the rows the COMPOSED AST selects, and that AST is complete only after + * every middleware has run. So {@link ObjectQL.update} calls it on that path, + * once the payload is final, with every matched row merged with the payload. + * + * [#19989] A by-id `update` uses it too. A middleware can read that one row, + * but only BEFORE the `beforeUpdate` chain runs, so its image is the change + * set as sent and a hook that rewrites a judged field is never judged. So + * {@link ObjectQL.update} calls it on the by-id path as well, once the + * payload is final, with the prior row merged with the payload. + * + * ABSENT is the ordinary state — no enforcement layer is mounted, or the + * write is one it does not gate. The engine never invents one. + */ + postHookWriteImageCheck?: PostHookWriteImageCheck; + + /** + * [ADR-0142 D4] The by-id write PRECONDITION: an equality predicate over the + * stored row (`{ : }`) the write is conditional on. Set by an + * enforcement layer's middleware — the ownership gate sets `{ owner_id: null }` + * on a claim, so two members claiming at once produce exactly one owner — and + * by the engine's own assignment cursor (`{ version }`). Engine-internal: + * reachable from no data door, and read only on a by-id `update`. + * + * The engine lowers the write to the driver's conditional + * `updateMany(object, { where: { id, ...expect } }, data)`; one affected row + * is re-read by id and the by-id branch continues unchanged (hooks, formulas, + * the event, the roll-up, the response); zero rows answer 409 + * `CONCURRENT_UPDATE` ("the record was taken"). A driver with no `updateMany` + * degrades to the pre-image check its setter already made, and the write + * goes through unconditionally, logged once per process at `warn`. + */ + writePrecondition?: Readonly>; +} + +/** + * [commit a016f08b8] The judgement {@link OperationContext.postHookWriteImageCheck} + * carries, and the acknowledgement its installer reads back. + * + * `evaluate` receives the images the driver is about to store, and REFUSES by + * throwing. On an `insert` those are the rows exactly as the `beforeInsert` + * chain left them, only the live ones (a row the declared-field door culled + * from a partial batch is never judged: it will not be written). On a + * predicate `update` they are the matched rows, each merged with the final + * payload; on a by-id `update` ([#19989]) the one prior row merged with the + * final payload. It is called at most once per operation. + * + * `honoured` is set by the engine immediately before `evaluate` runs. It exists + * so the installer can fail CLOSED on a seam that was never called: an + * enforcement layer that moved its gate here and finds the flag unset on the + * way out knows its check did not happen, and says so loudly rather than + * reading an unjudged write as an allowed one. + */ +export interface PostHookWriteImageCheck { + evaluate(rows: readonly Record[]): void | Promise; + honoured?: boolean; + /** + * [#24822] Declared by the installer, and only by it: `evaluate` holds every + * row it is handed to the Layer 0 tenant wall on `organization_id`. It + * admits the value or refuses the write by throwing. On an `insert` those + * rows are the inserted rows; on an `update` they are the stored images: + * the one prior row merged with the final payload on the by-id path, and + * every matched row merged with it on the predicate path. + * `@objectstack/plugin-security` sets it at step 3.7, only where it + * composes the wall's own parts into the insert's or the single-payload + * update's seam. A seam that carries a business `check` alone does not + * judge the organization, and neither does a platform operator's seam that + * crosses the wall, so neither sets it; nor is it set on an update whose + * payload is an array, where step 3.7 installs no seam of its own. + * + * {@link ObjectQL.insert} and {@link ObjectQL.update} read this member, and + * nothing else, to decide one thing: whether a non-system caller's explicit + * `organization_id` stays on the row for that judgement or is taken by the + * static-`readonly` strip (the platform's tenant column declares + * `readonly: true`). Set, the value is stored as sent where the wall admits + * it and refused where it does not: an insert's is never silently replaced + * by the active organization (ADR-0105 D5), and an update's re-point is + * never silently dropped in favour of the row's own organization. Only the + * platform's injected column is spared, and only on an object the insert's + * static-`readonly` strip judges, so never a `sys_` or platform-internal + * object. An AUTHORED `organization_id` declared `readonly: true` keeps the + * strip. An emptied value on an update is not spared into storage: the wall + * refuses it (`null` or `''`), on the payload and on the stored image. + * Absent or `false`, the strip runs exactly as before. The engine never + * sets it, and never derives it from the posture or the context: that + * would be a mirror of the wall, and a mirror is wider than the wall + * wherever the wall is not mounted. [#25062] The write preview, which + * installs no seam, reads the same value from the wall's verdict on each + * row instead ({@link WriteWallRowVerdict}, `ObjectQL.registerWriteWallVerdict`). + */ + judgesOrganization?: boolean; +} + +/** + * Trailing options for the READ methods (find / findOne / count / aggregate). + * + * Historically the read methods took their execution context INSIDE the query + * (`query.context`), while the WRITE methods (insert / update) took it in a + * trailing `options.context`. That split was a footgun: the same `{ context }` + * object is correct as the 3rd arg to `insert` but was SILENTLY DROPPED as the + * 3rd arg to `find` — a class of bugs where an intended `isSystem` bypass just + * vanished (e.g. control-plane reads coming back empty once org-scoping hooks + * were added). We now ALSO accept `context` via this trailing options arg on the + * read methods, so "execution context goes in the trailing options argument" is + * one rule across reads and writes. `query.context` remains supported; when both + * are given, `options.context` wins (it is the explicit channel). + */ +export interface EngineReadOptions { + context?: ExecutionContext; +} + +/** + * [#22445] Did the read door serve this value AS STORED — can the caller read + * the stored value in full? The `update`-mode preview keeps a stored column + * only when this holds for it (see `storedRows` in `ObjectQL.validate`), and + * [#22474] the preview keeps a master-detail header column only when this + * holds for it (see `ObjectQL.resolvePreviewParents`). + * + * Both values come off the same driver read shape (`driver.findOne` under the + * same `buildDriverOptions`; for a header, the read door's `find` under the + * caller's context and under the write's elevated one), so a column nothing + * transformed compares equal by construction; what differs is what the read + * path REWROTE for this caller (a partial mask, a masked secret, a read hook's + * rewrite). Structural, not identity: the two reads are two driver calls, so a + * `Date`, an array or a JSON object arrives as two equal objects. + * + * ⛔ Fails CLOSED: a pair it cannot prove equal — another object kind (a + * buffer, a class instance), a nesting past the depth bound — answers `false`, + * and the column is judged as empty. That direction can only make the preview + * refuse or admit on less than the write knows; the other direction would + * judge a value the caller may not read. + */ +export function servedAsStored(served: unknown, stored: unknown, depth = 0): boolean { + if (Object.is(served, stored)) return true; + if (depth >= 16) return false; + if (served instanceof Date || stored instanceof Date) { + return served instanceof Date && stored instanceof Date && served.getTime() === stored.getTime(); + } + if (Array.isArray(served) || Array.isArray(stored)) { + return Array.isArray(served) && Array.isArray(stored) && served.length === stored.length + && served.every((item, i) => servedAsStored(item, stored[i], depth + 1)); + } + const plain = (v: unknown): v is Record => { + if (v === null || typeof v !== 'object') return false; + const proto = Object.getPrototypeOf(v); + return proto === Object.prototype || proto === null; + }; + if (!plain(served) || !plain(stored)) return false; + const servedKeys = Object.keys(served); + return servedKeys.length === Object.keys(stored).length + && servedKeys.every((k) => Object.prototype.hasOwnProperty.call(stored, k) && servedAsStored(served[k], stored[k], depth + 1)); +} + +/** Merge read-path execution context from the query and the trailing options. */ +export function mergeReadContext( + fromQuery?: ExecutionContext, + fromOptions?: ExecutionContext, +): ExecutionContext | undefined { + if (fromOptions == null) return fromQuery; + if (fromQuery == null) return fromOptions; + return { ...fromQuery, ...fromOptions }; +} + +/** + * True when this write is exempt from the `state_machine` validation rule — + * both the insert `initialStates` entry check and the update `transitions` + * check are skipped for it. Either the seed-specific `seedReplay` flag (#3433) + * or the general `skipStateMachine` flag (#3479, set by the REST import runner + * for a "historical" import) turns it off. Both are server-set, never + * client-supplied. + */ +export function shouldSkipStateMachine(ctx?: ExecutionContext): boolean { + return ctx?.seedReplay === true || ctx?.skipStateMachine === true; +} + +/** + * Engine Middleware (Onion model) + */ +export type EngineMiddleware = ( + ctx: OperationContext, + next: () => Promise +) => Promise; + +/** + * "Which of these tombstoned `sys_file` rows is something still holding?" + * (commit c3c72a4bc). + * + * Takes the whole tombstoned set from ONE record read and returns the subset + * still held, as a set of stringified ids. Batched rather than per-row on + * purpose: hydration runs over many rows per read, so asking per file would be + * N queries per read. + * + * The engine declares this shape but never implements it — "still held" has one + * definition (`findFileHolder`, the union of `sys_attachment` join rows and the + * `ref_*` ownership columns) and it lives in the storage package. See + * `ObjectQL.registerHeldFileResolver`. + */ +export type HeldFileResolver = ( + rows: Array>, +) => Promise>; + +/** + * "Of these `ownerObject` records, whose field-owned files may this caller + * read?" — the download door's field-owned verdict (ADR-0104 D3; #22637, + * maintainer ruling B on #22624). + * + * Takes every owner record id one hydration needs judged and returns the + * subset whose files the caller may read, as stringified ids. Batched for the + * same reason as {@link HeldFileResolver}: one call per read, never one per + * row. + * + * The engine declares this shape but never implements it. The verdict — a + * declared `fileAccessDelegate` asked and failing closed, otherwise the + * caller's read of the owner record — has one definition, the field-owned arm + * of the storage package's download authorizer, and a copy here would be a + * second predicate for one file. See `ObjectQL.registerFieldOwnedFileReadAuthorizer`. + */ +export type FieldOwnedFileReadAuthorizer = ( + ownerObject: string, + ownerIds: string[], + context: ExecutionContext, +) => Promise>; + +/** + * [#25062] The Layer 0 tenant write wall's answer for ONE row of a write + * preview ({@link ObjectQL.validate}): admitted, saying whether the wall judges + * the row's `organization_id` (the value + * {@link PostHookWriteImageCheck.judgesOrganization} is set from on the + * write), or refused with the write's own `code` and `message`. A refused + * `field` is `organization_id` when the wall refused a value the row carries, + * and `_record` when it refused the caller's state (no active organization). + */ +export type WriteWallRowVerdict = + | { readonly admitted: true; readonly judgesOrganization: boolean } + | { + readonly admitted: false; + readonly field: 'organization_id' | '_record'; + readonly code: string; + readonly message: string; + }; + +/** + * [#25062] "What does the Layer 0 tenant write wall answer for each of these + * rows?" One verdict per row, in order, or `null` when the wall is not asked + * for this write (the engine then judges the rows as if nothing were + * registered). + * + * The engine declares this shape but never implements it. The wall has one + * definition, the enforcement layer's own (`@objectstack/plugin-security`'s + * step 3.7), and the verdict travels here as a value: a copy of the wall in + * the engine would be the mirror the `judgesOrganization` member forbids. See + * `ObjectQL.registerWriteWallVerdict`. + */ +export type WriteWallVerdict = ( + object: string, + operation: 'insert' | 'update', + context: unknown, + rows: readonly Record[], +) => Promise; + +/** + * The stack collections the engine decomposes into individual registry items — + * ONE list, read by the ONE body both registration seams run + * (`registerMetadataCollections()`, called from the manifest seam in + * `registerApp()` and the nested-plugin seam in `registerPlugin()`). + * + * ## Why this is one constant and not two lists (#7049) + * + * It used to be two: `const metadataArrayKeys = [...]` declared separately + * inside each loop. They drifted, invisibly, because nothing compared them — + * `jobs`, `emailTemplates`, `tools` and `skills` were registered from a + * manifest and NOT from a nested plugin, so a package shipping any of the four + * from a nested plugin registered nothing and stamped no ADR-0010 provenance: + * no refusal, no diagnostic. `capabilities` hit the SAME divergence and was + * patched into the nested copy by hand (#5870) without anyone diffing the rest + * of the two lists, which is how the remaining four survived it. + * + * The two loops were measured against each other before this was merged. They + * differed in four ways — which object they read (`manifest` vs `plugin`), which + * package id they stamp (both resolve to the SAME parent package: a nested + * plugin contributes under its parent's ownership), a per-key `debug` line, and + * the manifest seam's aggregated-view expansion plus its warn-on-nameless-item. + * Every one of those lived in the loop BODY. Not one of them was a reason for + * the two seams to enumerate different collections, so the enumeration was + * shared and that divergence is now unrepresentable rather than merely unnoticed. + * + * ## …and why the BODY is shared too (#7163) + * + * The last two entries on that list were not stylistic. The aggregated-view + * expansion is what makes ADR-0017's dual-read work, so the seam that lacked it + * under-registered every nested plugin's views silently — see + * `registerMetadataCollections()`, which is now the single body both seams run, + * for the measurement. Sharing the list made the seams' collection SET + * unanswerable-differently; sharing the body does the same for what they DO + * with a collection they both see. + * + * `check:stack-collection-maps` pins this list against + * `ObjectStackDefinitionSchema` in both directions (#6242); it used to pin the + * two copies as two sites and carry a waiver row recording their divergence. + * That row is gone with the divergence. + */ +const METADATA_ARRAY_KEYS = [ + // Data Protocol — shared option lists. FIRST, so a source's lists are in the + // registry before anything later in this loop reads them; objects do not + // depend on the order (they resolve a list lazily, on the next fold), so this + // is the declared loading order (`picklist` loads before `object`), not a + // precondition. Both are dispatched to their own registry verbs in + // `registerMetadataCollections`: the extensions merge into the list they + // name instead of registering as items of their own. + 'picklists', 'picklistExtensions', + // UI Protocol + 'actions', 'views', 'pages', 'dashboards', 'reports', 'datasets', 'themes', + // Automation Protocol + 'flows', 'workflows', 'approvals', 'webhooks', + 'jobs', + // Security Protocol — `capabilities` is here for the same reason as + // `permissions` (#5870, #4967 Part 2): the ONLY seam that stamps + // ADR-0010 provenance is `registerItem` → `applyProtection`, so a + // collection missing from this list reaches no registry with a + // `_packageId`. The declared-capability seeder (retired by ADR-0131 D3) + // resolved the owning package as `cap._packageId ?? cap.packageId`; while `capabilities` + // sat outside this list the first half could never be satisfied and + // `readDeclared(ql, 'capability')` returned nothing, which made the + // author-side `packageId` — documented as the FALLBACK — mandatory, + // and its omission a silent, unenforced authorization declaration. + // + // [#22203] `positions` for the same reason, and with a write-door + // consequence: ADR-0090 D3 renamed `roles` to `positions`, and the rename + // reached the artifact door's map but never this list, so a stack-declared + // position had no registry entry under its package. The metadata save door + // decides "a code package ships this" from exactly that entry + // (`isArtifactBacked` → `getArtifactItem`), so a save over a package's + // position took the runtime-create tier and was accepted, although the type + // registry declares `position` `allowEnvironmentOverlay: false`. With the entry + // present the door's type-level refusal covers it like every other + // `security`-domain type (`engine-security-catalog-package-door.test.ts`). + // `roles` stays in the list as the inert retired spelling it already was. + 'roles', 'positions', 'permissions', 'capabilities', 'profiles', 'sharingRules', 'policies', + // [ADR-0142 D1/D5] Ownership queues and the assignment rules that route + // inserted rows into them — both read from the registry on every use (the + // queue service, the assignment seam), so a declaration must reach it with + // its package stamped, as every collection on this list does. + 'queues', 'assignmentRules', + // AI Protocol + 'agents', 'tools', 'skills', 'ragPipelines', + // API Protocol + 'apis', + // Data Extensions + 'hooks', 'mappings', 'analyticsCubes', + // Integration Protocol + 'connectors', + // System Protocol — outbound mail templates. Registered here so the + // email plugin's materializer can read them back into + // `sys_email_template` (#4509); without this key an authored + // `emailTemplates:` entry never reached the registry at all, which is + // the far end of the disconnect the bridge closes. + 'emailTemplates', + // System Protocol — package documentation (ADR-0046); inert data + 'docs', + // Documentation navigation spine (ADR-0046 §6) + 'books', +] as const; + +/** + * Derive the registry key for a metadata item. + * + * Most metadata items expose a top-level `name` (or `id`). The `View` + * container defined by `@objectstack/spec/ui` is special: it aggregates + * `list / form / listViews / formViews` for a single object and is keyed by + * the OBJECT it binds to, not by its own row identity — which is what + * `/api/v1/meta/views/:object`, `getViewsByObject()` and + * `GET /meta/view?object=` all address it by. + * + * ⚠️ [commit 3c1bbd2a8] The sentence that used to stand here — "per spec, `ViewSchema` + * does NOT have a top-level `name` field" — is measurably false and was the + * premise for consulting `item.name` first. `ViewSchema` declares an optional + * `name` (`view.zod.ts`), described there as "supplied by the metadata door; + * for an object-scoped container it is the object name". "Is the object name" + * is a CONVENTION the door does not enforce, so the two never actually had to + * agree — and where they disagreed, this registrar and the other two picked + * different keys for the same document. + */ +function resolveMetadataItemName(key: string, item: any): string | undefined { + if (!item) return undefined; + // [commit 3c1bbd2a8] The aggregated `views` CONTAINER branch, taken FIRST and answered + // by the shared derivation. Everything below is unchanged. + // + // This registrar used to consult `item.name` before anything else, for every + // key including this one — so a container written as + // `{ name: 'lead_views', object: 'crm_lead', list: {…} }` registered under + // `lead_views` here while the artifact/HMR SOURCE registrar + // (`MetadataPlugin._parseAndRegisterArtifact`) and the runtime door + // (`expandRuntimeViewContainer`) both registered it under `crm_lead`. Same + // document, two source registrars, two registry keys and two sets of expanded + // item names, with `getViewsByObject()` / `GET /meta/view?object=` answering + // for the object only when the right registrar happened to load it. + // + // The 2026-08-07 meta-rule settles the direction rather than taste: one + // operation with two inconsistent implementations, the side bound by a + // DECLARATION wins. `ViewSchema.object`'s own `.describe()` names its readers + // (`getViewsByObject()` / `GET /meta/view?object=`); this loop's old order + // argued from item identity, which declares nothing about the binding. So the + // container branch adopts `deriveViewContainerObject` — by import, because a + // fourth hand-copy of a chain that already exists three times is the defect + // this repair exists to close, not the repair. + // + // The gate is `isAggregatedViewContainer`, which is what makes this the + // CONTAINER branch and nothing wider: it is false for every artifact carrying + // a `viewKind`, so the assembled `viewItems:` channel below (standalone + // ViewItems and flattened list/form overlays — every member of + // `AssembledViewArtifactSchema` requires `viewKind`) still resolves by its own + // `name` first, which is its identity and not a binding. + // + // `item.id` is untouched and stays reachable for every other key. It cannot + // fire for a spec-valid container: `ViewSchema` is a `strictObject` that + // declares `name` and `object` and no `id`, so an `id` on a container is + // refused at the authoring and metadata doors before this seam sees it. + if (key === 'views' && isAggregatedViewContainer(item)) { + return deriveViewContainerObject(item); + } + if (item.name) return item.name; + if (item.id) return item.id; + if (key === 'views') { + // A `views` entry that is NOT an aggregated container and carries neither + // `name` nor `id` — e.g. a flattened overlay whose optional `name` was + // omitted. Same derivation, and identical to the chain that used to be + // written out here: with `item.name` already known falsy, the helper's + // trailing `name` term contributes nothing. + return deriveViewContainerObject(item); + } + return undefined; +} + +/** + * ObjectQL Engine + * + * Implements the IDataEngine interface for data persistence. + * Acts as the reference implementation for: + * - CoreServiceName.data (CRUD) + * - CoreServiceName.metadata (Schema Registry) + */ +// [#6063] `SummaryDescriptor`, `summaryEmptySetValue` and the single-descriptor +// aggregate moved to `./summary-aggregate.js` — unchanged, and still the one +// place each is written down. The move exists so the one-off backfill of +// pre-#6013 `NULL` rows computes its value through the SAME code this engine +// does, instead of a second implementation that agrees only until one of them +// is edited. + +// `implements IObjectQLEngine` is the verification step of #4251 B3: every +// member the `objectql` slot's contract declares is checked against this class +// on every build, so the seven consumer-local surface declarations the contract +// replaced can never silently drift from the engine again. IObjectQLEngine +// extends IDataEngine, so the old claim rides along. +/** + * [#4889] The master id a detail row's `parent` binding resolves against: + * the write's own value for the FK when it carries one (a REPOINT must be + * judged against the master it lands on), else the prior row's. Only a scalar + * id counts — an expanded relation object or an array is not an id this read + * can bind, and guessing one would be worse than leaving `parent` unbound. + * + * [#19853] "Carries one" means the write STORES it: the UPDATE paths hand this + * the payload view `settleMasterDetailLanding` returns, which no longer holds + * an FK a later strip would have taken back out, so the prior row's FK — the + * header the row keeps — answers instead. + */ +export function masterIdOf( + fk: string, + data: Record | null | undefined, + row: Record | null | undefined, +): string | number | undefined { + const raw = data && fk in data ? data[fk] : row?.[fk]; + if (typeof raw === 'string') return raw === '' ? undefined : raw; + if (typeof raw === 'number' && Number.isFinite(raw)) return raw; + return undefined; +} + +/** + * [#19887] The UPDATE payload as the static `readonly` strip will leave it — + * `data` without every key that strip takes back out — which is the view a + * `readonlyWhen` predicate's `record` binding is built from + * (`ReadonlyWhenStripOptions.stored`). + * + * The conditional strip runs BEFORE the static one, so the payload it is + * handed still holds a value a non-system caller forged for a statically + * `readonly` field. Judged over that payload, `amount` locked by + * `record.status == 'closed'` read the forged `status: 'open'`, stayed + * writable, and the static strip then removed the `status`: the closed row + * committed with its locked amount rewritten. The strips keep their order — + * each still reports its own fields under its own reason — and only what the + * predicate reads moves. + * + * The same verdict {@link staticReadonlyStripTakes} asks for one key, asked + * for all of them by the same function with the same arguments: `data` itself + * when the strip does not run (a system caller), and a key a hook wrote, a + * value a hook overwrote or a column `preserveAudit` keeps all stay, because + * the write stores them. Silent for the same reason. + * + * [#24822] ...and over the same view of the schema: the one + * {@link updateStaticReadonlySubject} answers for `strip.wallJudgesOrganization`, + * the flag `update()` reads once off the wall's seam and hands to the strips + * themselves. So where the wall judges the update's organization, a re-point + * the strip leaves on the row is in this view too, and a `readonlyWhen` + * predicate reading `record.organization_id` reads the organization the row is + * stored in. The master-detail settlement forwards the update's descriptor + * object unchanged, so the flag reaches {@link staticReadonlyStripTakes} on that + * path too. Absent, the view is the whole schema's, exactly as before. + */ +export function staticReadonlyStoredView( + schema: unknown, + data: Record, + supplied: Readonly>, + strip: { + runs: boolean; + preserveAudit: boolean; + hookWrittenKeys: ReadonlySet | undefined; + wallJudgesOrganization?: boolean; + }, +): Record { + if (!strip.runs) return data; + return stripReadonlyFields( + updateStaticReadonlySubject(schema, strip.wallJudgesOrganization === true) as any, data, supplied, undefined, + { preserveAudit: strip.preserveAudit, hookWrittenKeys: strip.hookWrittenKeys }, + ) ?? data; +} + +/** + * What the engine knows about the transaction it opened, beyond the handle + * itself (#4619, ADR-0119 D1 follow-up). + * + * It began as a pure OBSERVABILITY record — #4619 could say a write was outside + * the transaction but deliberately did not act on it, because acting changes + * ADR-0119 D1's declared contract and that was a decision for the maintainer, + * not for the PR that found the defect. Since the 2026-08-06 ruling on #5351 + * this record is LOAD-BEARING: `enforceTransactionOrigin` refuses a + * cross-driver business write on it, and `transactionCoversDriverFor` uses it + * to keep a transaction handle from ever reaching a driver that does not own + * it. Routing itself is still untouched — a write goes exactly where `getDriver` + * sends it; what changed is whether it goes there carrying someone else's + * connection. + */ +interface TransactionScope { + /** + * The driver instance the open transaction belongs to. Compared by IDENTITY + * rather than by name: two drivers can transiently claim one name (see + * `registerDriver`'s collision branch), and identity is what actually decides + * whether a write rides this transaction's connection. + */ + readonly driver: IDataDriver; + /** The datasource name that driver is registered under — for the message. */ + readonly datasource: string; + /** + * Datasources already noted for THIS transaction. AGENTS.md's "say it once, + * at the first degradation, not once per failed write" — a 500-row audit + * batch carved out of this transaction is one note, not 500. Only the + * carve-out path consumes the budget; a refused business write throws, and a + * throw is never deduplicated. + */ + readonly reportedOutOfScope: Set; +} + +/** + * A datasource *definition* as the engine keeps it (ADR-0015) — the declarative + * facts a datasource states about itself, not a live connection (that is + * {@link ObjectQL.registerDriver}). + * + * Named rather than restated inline because three sites share it: the private + * index, {@link ObjectQL.registerDatasourceDef} which writes it, and + * {@link ObjectQL.listDatasourceDefs} which reads it back. Three copies of one + * shape is a second de-facto contract that drifts silently, and the drift this + * one produces is a credentials handle missing from a `sys_secret` sweep. + * + * The keys are a deliberate SUBSET of the spec's authored datasource surface + * (`ExternalDatasourceSettingsSchema` in `@objectstack/spec`) — the engine + * carries only what it has a use for. ⛔ Not a mirror of the spec block and not + * a place to grow one: `validation` and `queryTimeoutMs` are absent because + * nothing in this engine reads them. + */ +export interface DatasourceDef { + name: string; + /** + * [#27277] The driver id the datasource selects. Read to decide whether the + * engine floors the datasource's driver (`needsExternalDriverFloor`: a + * non-managed datasource on a driver id that is not built in). + */ + driver?: string; + schemaMode?: string; + external?: { + /** Datasource-wide write gate — ADR-0015 §5.3 Gate 3. */ + allowWrites?: boolean; + /** + * Reference into the secrets store, never an inline credential. Valid in + * EVERY `schemaMode` — it is the one `external` key a managed datasource + * may carry (#8153) — so a reader sweeping for handles must not filter by + * schema mode. + */ + credentialsRef?: string; + /** + * [#29762] The datasource's plugin-driver row cap (`external.rowCap`). + * Read by the floor on every read (`resolveExternalRowCap`), so a + * redefinition takes effect without re-wrapping the driver. + */ + rowCap?: number; + }; +} + +/** + * [#22305] The records ONE by-id cascade delete removes, collected before any + * of them is judged — see {@link ObjectQL.cascadeDeleteRelations}. + * + * Without it, every level of the depth-first walk judged its own `restrict` + * refusals with no knowledge of the cascade that called it: a record the same + * cascade was about to delete refused the delete of its sibling, and whether + * it got the chance depended on which object was registered first. The triage + * ruling on #22305 is the rule this set exists to apply: a child that is itself + * in the cascade set never restricts a sibling in the same set. + * + * Engine-internal and never part of a context: it rides + * {@link ObjectQL.cascadeDeleteSets}, an `AsyncLocalStorage`, so no request + * input can name a record into it, and a caller that passed no context still + * passes none. + */ +export interface CascadeDeleteSet { + /** + * Object name → ids the cascade deletes, the root included. Collected by a + * read-only walk across every CASCADING relation before any write, so it is + * complete before the first refusal is judged. + */ + readonly members: Map>; + /** + * Object name → ids whose own referential walk has started, in progress or + * done. The walk never re-enters one, which is what makes a cycle in the + * data terminate and what keeps a row reached by two paths from being + * deleted twice. + */ + readonly entered: Map>; + /** + * The #12166 elevation records already filed, one per deleted record per + * referenced object, so the collection walk and the deleting walk, which + * read the same relations, file each record once between them. + */ + readonly elevationsFiled: Set; + /** + * [#26718] Member key ({@link cascadeMemberKey}) → the members a + * CASCADING relation reaches from it directly, every edge the collection + * walk read, a row reached by two paths included. The reach of one member + * inside the set ({@link cascadeReachOf}) is read from it: which rows the + * delete removes only BECAUSE that member goes. + */ + readonly cascadeEdges: Map>; + /** + * [#27004] Relation key ({@link cascadeMemberKey} of the referencing object + * and its field) → the ids of the referenced object whose walk has finished + * that relation: every row outside the set has had them cleared. A walk + * over a root SET reads each relation once for many records, so a + * multi-valued row read for one of them may still list a member an earlier + * one has removed from it; the remainder written leaves those out too. + */ + readonly cleared: Map>; +} + +/** [#26718] One member of a {@link CascadeDeleteSet}, as a single key. */ +export function cascadeMemberKey(object: string, id: unknown): string { + return JSON.stringify([object, String(id)]); +} + +/** Is `id` of `object` in this index of a {@link CascadeDeleteSet}? */ +export function cascadeSetHas(index: Map>, object: string, id: unknown): boolean { + return id != null && index.get(object)?.has(String(id)) === true; +} + +/** Add `id` of `object`; `true` when it was not there yet. */ +export function cascadeSetAdd(index: Map>, object: string, id: unknown): boolean { + let ids = index.get(object); + if (!ids) { ids = new Set(); index.set(object, ids); } + const key = String(id); + if (ids.has(key)) return false; + ids.add(key); + return true; +} + +/** + * [#23221] The one child delete a cascade walk is issuing, for that delete to + * recognise itself by — see {@link ObjectQL.cascadeIssuedDeletes}. The walk + * opens one per `cascade` dependent, around its own `delete()` call, and the + * `delete()` whose object and id match claims it. + */ +interface CascadeIssuedDelete { + /** The child object the walk is deleting from, as the registry names it. */ + readonly object: string; + /** The child row's id, as a string (the walk's `String(depId)`). */ + readonly id: string; + /** + * Set by the `delete()` that recognised itself. One token, one dispatch: a + * later delete of the same row inside that call (a hook's own) finds it + * claimed and is not marked. + */ + claimed: boolean; +} + +/** + * [ADR-0010 §3.7] Stamp a manifest item with its owning package's + * `(packageId, packageVersion)` before the registry stores it: the call the + * artifact loader makes (`metadata/plugin.ts`), so both load paths serve one + * envelope. The registry's own `applyProtection` knows the id only, and + * `applyProtection` stamps a key only while it is unset, so the version + * stamped here is the one the item keeps. With no version this does nothing: + * the registry stamps the id, exactly as before. + */ +export function stampPackage(item: T, packageId: string, packageVersion: string | undefined): T { + if (packageVersion !== undefined) applyProtection(item as any, { packageId, packageVersion }); + return item; +} + +/** + * [ADR-0086 D3] The registration door's refusal of a permission set whose body + * `packageId` disagrees with the package {@link ObjectQL.registerApp} registers + * it under (`ownerId`; none for a manifest that carries no id). + * + * The judgement and its words are the one judge every door of the rule calls, + * `judgePermissionSetOwnerBinding` (`@objectstack/spec/security`, whose header + * holds the verdict table): the stack schema runs it at the build door + * (`defineStack`, `os validate`, `os build`, the artifact door's parse), so this + * seam is the door for a manifest no schema parse judged — a programmatic + * `registerApp`, a nested plugin, a `strict: false` stack, a package installed + * from a stored manifest. ⛔ Never a second comparison here. + * + * `VALIDATION_ERROR` / 400, thrown before the set is filed: the envelope of + * this seam's other body-against-key refusal, the divergent view-container + * `name` (`viewContainerNameRefusal`), which the same loop throws one statement + * earlier. `issues[]` locates the key, as the metadata write doors' refusal of + * the same rule does. No code is minted. + * + * Exported for the stack's RESIDUAL registration (`registerUnclaimedTopLevelItems` + * in `plugin.ts`), which files a top-level permission set under the stack's id + * and refuses the same set in this same envelope. ⛔ Never rebuild the envelope + * at a second door. + */ +export function permissionSetOwnerRegistrationRefusal( + item: unknown, + name: string, + ownerId: string | undefined, +): (Error & { code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }> }) | undefined { + const refusal = judgePermissionSetOwnerBinding(item, name, ownerId, 'register'); + if (!refusal) return undefined; + const err = new Error(refusal.message) as Error & { + code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }>; + }; + err.code = 'VALIDATION_ERROR'; + err.status = 400; + err.httpStatus = 400; + err.issues = [{ path: 'packageId', message: refusal.issueMessage, code: 'custom' }]; + return err; +} + +/** + * [#28534] The `_packageId` each item object was stamped with BY A + * REGISTRATION of this process — written onto the caller's own object by + * `stampPackage` / `registerItem`, recorded by {@link noteRegistrationStamp}. + * Weakly held and process-wide: the earlier registration may have been + * another engine's. Read by {@link collectionItemOwnerStampRefusal} only. + */ +const REGISTRATION_WRITTEN_STAMPS = new WeakMap(); + +/** + * [#28534] Record that a registration just wrote `item`'s `_packageId`: it was + * unset before (`stampBefore`) and is set now. A stamp the object already + * carried is never recorded, so it stays a stamp the body arrived with. + */ +function noteRegistrationStamp(item: unknown, stampBefore: unknown): void { + if (stampBefore !== undefined || !item || typeof item !== 'object') return; + const written = (item as { _packageId?: unknown })._packageId; + if (written !== undefined) REGISTRATION_WRITTEN_STAMPS.set(item, written); +} + +/** + * [#28534] The registration door's refusal of a collection item whose body + * already carries a `_packageId` other than the package it is registered under + * (`ownerId`; none for a manifest that carries no id). + * + * `_packageId` is the ADR-0010 envelope stamp: the package that registered the + * item, written by the door (`stampPackage` here, `registerItem` behind it, + * both through `applyProtection`). `applyProtection` writes it only while it is + * `undefined`, and every item schema that spreads `MetadataProtectionFields` + * (a connector entry among them) declares the key, so a body could arrive + * wearing ANOTHER package's id and keep it: an installed package naming the + * host's own id, or an id no package holds, and every reader that asks an + * item's owner (the connector materializer judging whose root a file ref + * resolves against, the disabled-package filter, the ADR-0130 artifact gates) + * then reads a stamp the body wrote about itself. + * + * Refused rather than overwritten, beside the two other body-against-key + * refusals of the same loop (the divergent view-container `name`, the + * permission set's `packageId`): resolving a disagreement between the body and + * the key it is filed under silently, in either direction, files the item + * under something nobody wrote (commit d0ee598e6). A stamp EQUAL to the owner + * is accepted — it is what an item re-registered under its own package (a + * ledger rehydrate, an artifact the metadata door already stamped in place + * with the same body id) carries. `null` and the empty string are stamps too: + * neither is overwritten by `applyProtection`, so either would survive as an + * owner no row answers. + * + * The judgement and its words are the one judge both doors of the rule call, + * `judgeCollectionItemOwnerStamp` (`@objectstack/spec/kernel`, whose header + * holds the verdict table and the collections): the stack schema runs it at + * the build door (`defineStack`, `os validate`, `os build`, the artifact door's + * parse), so this seam is the door for a body no schema parse judged — an + * installed body, a programmatic `registerApp`, a nested plugin, a + * `strict: false` stack. ⛔ Never a second comparison here. What stays here is + * the exemption below, which is this process's state, not a body rule. + * + * `VALIDATION_ERROR` / 400, `issues[]` at `_packageId`: the envelope of + * {@link permissionSetOwnerRegistrationRefusal}. Exported for the stack's + * RESIDUAL registration (`registerUnclaimedTopLevelItems` in `plugin.ts`), + * which files top-level items under the residual owner and refuses the same + * stamp in this same envelope. ⛔ Never rebuild the envelope at a second door. + * + * ## A stamp a registration wrote is not a body's claim + * + * `stampPackage` and `registerItem` stamp the CALLER'S object in place, so an + * item object registered once keeps that registration's `_packageId` for as + * long as it lives — a module constant handed to a second registration (a + * fixture another test registered under its own package, measured in + * `engine-picklist.test.ts`) arrives wearing the first registration's stamp, + * which no body wrote. Those stamps are recorded where they are written + * ({@link noteRegistrationStamp}) and are not refused here: the item + * registers as it always has. Only a stamp the object ARRIVED with is judged — + * which every installed body's is, since an install door parses its body + * fresh, so no registration has touched it. + */ +export function collectionItemOwnerStampRefusal( + item: unknown, + type: string, + name: string, + ownerId: string | undefined, +): (Error & { code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }> }) | undefined { + if (item && typeof item === 'object') { + const stamped = (item as { _packageId?: unknown })._packageId; + if (stamped !== undefined && REGISTRATION_WRITTEN_STAMPS.get(item) === stamped) return undefined; + } + const refusal = judgeCollectionItemOwnerStamp(item, type, name, ownerId, 'register'); + if (!refusal) return undefined; + const issueMessage = refusal.issueMessage; + const err = new Error(refusal.message) as Error & { + code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }>; + }; + err.code = 'VALIDATION_ERROR'; + err.status = 400; + err.httpStatus = 400; + err.issues = [{ path: '_packageId', message: issueMessage, code: 'custom' }]; + return err; +} + +/** + * [#28697] {@link collectionItemOwnerStampRefusal} for the two collections a + * package body files through verbs of their own rather than the collection + * loop: its OBJECTS (`registerObject`, the array and the name-keyed map form) + * and its APPS (`registerApp`: each `apps[]` entry the loop would register, + * and the body itself when it registers as its own app). The first refusal + * found, in that order, or `undefined`. + * + * Asked BEFORE the body registers anything, so a refused body leaves no + * package record, object or app behind. An app wearing another package's + * `_packageId` kept it (`applyProtection` stamps an unset key only), and every + * reader that asks an app's owner — the per-package list filter, the + * disabled-package filter, the packaged-base gate — then filed it under the + * package it named. An object's stamp is read by the action-parse predicate + * of `registerObject` (`schema._packageId ?? packageId`). Refused, not + * overwritten, for the reason the collection loop refuses: one judge, so the + * build door (the stack schema, which judges `objects` and `apps` too) and + * this door cannot disagree. + * + * Called by {@link ObjectQL.registerApp} and by a nested plugin's + * registration (`registerPlugin`, which registers objects and the plugin + * itself as an app, and no `apps[]`). + * + * @param body The package body or nested plugin, as it arrived. + * @param ownerId The package the door registers it under. + * @param scope Which of the app shapes this door registers. + */ +export function packageBodyObjectAndAppOwnerStampRefusal( + body: any, + ownerId: string | undefined, + scope: { apps: boolean; selfAsApp: boolean }, +): ReturnType { + const objects = body?.objects; + if (objects && typeof objects === 'object') { + const entries: Array<[unknown, unknown]> = Array.isArray(objects) + ? objects.map((def: any) => [def?.name, def]) + : Object.entries(objects); + for (const [name, def] of entries) { + const refusal = collectionItemOwnerStampRefusal(def, 'object', typeof name === 'string' && name !== '' ? name : '(unnamed)', ownerId); + if (refusal) return refusal; + } + } + if (scope.apps && Array.isArray(body?.apps)) { + for (const app of body.apps) { + // The loop registers an app with a name only; a nameless one is skipped. + const appName = app?.name || app?.id; + if (!appName) continue; + const refusal = collectionItemOwnerStampRefusal(app, 'app', String(appName), ownerId); + if (refusal) return refusal; + } + } + if (scope.selfAsApp) { + const refusal = collectionItemOwnerStampRefusal(body, 'app', String(body?.name), ownerId); + if (refusal) return refusal; + } + return undefined; +} + +/** + * [#28697] Every refusal the collection seam (`registerMetadataCollections`) + * raises that the body alone decides, asked over a package body or a nested + * plugin BEFORE anything of the package registers. The first refusal found, + * walking the collections and items in the order the seam registers them, or + * `undefined`. + * + * Asked first so a body refused by one of them leaves nothing of the package + * registered, and a refused re-register of an installed id leaves that + * install as it was. Every judge here reads the item, its derived name, the + * source label and the owning id, and nothing the registry holds, so asking + * it first gives the same answer in the same words. ⛔ No second copy of any + * judge: each is the one function its own record names. + * + * Called by {@link ObjectQL.registerApp} for the body (`'manifest'`) and for + * each nested plugin (`'nested plugin'`, under the parent's id, as + * `registerPlugin` registers it). + */ +export function packageBodyCollectionRefusal( + source: any, + ownerId: string, + sourceLabel: string, +): Error | undefined { + for (const key of METADATA_ARRAY_KEYS) { + // A `picklistExtensions` entry is not an item (the seam hands it to + // `registerPicklistExtension`, whose refusal reads the registry). + if (key === 'picklistExtensions') continue; + const items = (source as any)?.[key]; + if (!Array.isArray(items) || items.length === 0) continue; + for (const item of items) { + const itemName = resolveMetadataItemName(key, item); + // The seam skips (and warns about) an item with no derivable name. + if (!itemName) continue; + // [commit d0ee598e6] The DIVERGENT-container refusal (maintainer ruling + // 2026-09-03, direction 2). The collection seam used to reconcile a + // container's own `name` to the derived key silently: the author's + // field discarded with no diagnostic, while + // the OTHER source registrar + // (`MetadataPlugin._parseAndRegisterArtifact` -> + // `assertMetadataRegisterContract`, #7378 row 1) hard-failed the + // whole artifact load on that very same document. One document, + // two SOURCE registrars, opposite outcomes, and which one an + // author got depended on how their package was loaded. #7378 + // row 1 already rules that resolving a name/key disagreement + // silently "in either direction" files the item under a key the + // caller never wrote, so the boot loop converges onto the + // refusal; the artifact door was not moved onto the rewrite. + // + // MEASURED, and it is why only this site changed: the second + // `toRegister` ternary in the seam (the assembled + // `viewItems:` channel below) cannot carry a container at all — + // `AssembledViewArtifactSchema` is the view vocabulary MINUS the + // container branch, and every body it admits is + // `isAggregatedViewContainer === false`. Its ternary can only + // MINT a `name` onto an overlay that has none; it can never + // discard an authored one, because `resolveMetadataItemName` + // reads a non-container's own `name` FIRST. + // + // Scope is the ruling's named main risk, so the gate is three + // independent narrowings and each one is load-bearing. The first + // stays here: + // * `key === 'views'` — the seam is the GENERIC metadata loop; no + // other kind changes behaviour and they all keep the + // reconcile. + // The other two — the CONTAINER branch only, and `name` present + // AND different — and the message, the envelope and the reason + // the prose is this seam's own live in `viewContainerNameRefusal` + // (`view-container-name-refusal.ts`). + // + // [#20331] Moved there so that `os validate` runs the SAME + // judgment, in the same words, instead of passing a document this + // loop then refuses at boot. The message and the envelope moved + // unchanged. The `!itemName` skip just above is this loop's + // precondition for the check, so the function carries it too: it + // answers `undefined` for a falsy derived key, and a door calling + // it without this loop cannot refuse an entry this loop skips. + // ⛔ Do not re-inline it here, and do not write a second copy of + // it anywhere else: one judge, two doors. + if (key === 'views') { + const refusal = viewContainerNameRefusal(item, sourceLabel, ownerId); + if (refusal) return refusal; + } + // [ADR-0086 D3] A permission set whose body `packageId` disagrees + // with the package the seam registers it under is refused, in + // the same envelope as the container refusal above: + // `registerItem` stamps `ownerId` as the set's `_packageId`, the + // owner every reader reads, so the body's key used to be + // registered over in silence. See + // {@link permissionSetOwnerRegistrationRefusal}. + if (key === 'permissions') { + const refusal = permissionSetOwnerRegistrationRefusal(item, itemName, ownerId); + if (refusal) return refusal; + } + // [#28534] …and an item of ANY collection whose body already wears + // a `_packageId` other than `ownerId`: `stampPackage` and + // `registerItem` stamp the owner only while the key is unset, so + // the body's own stamp used to win. See + // {@link collectionItemOwnerStampRefusal}. + { + const refusal = collectionItemOwnerStampRefusal(item, pluralToSingular(key), itemName, ownerId); + if (refusal) return refusal; + } + // [#5320] The `views:` tighten — containers ONLY, the contract the + // stack schema has always declared (`stack.zod.ts`, + // `z.array(ViewSchema)`). The seam used to register EVERY entry + // as type `view` — the "runtime wider than schema" hole #5320 + // records: a ViewItem or flattened overlay that `defineStack` + // refuses registered there in silence. Judged by the SAME + // classifier the manifest assemblers partition with + // (`isViewContainerShaped`), so a produced manifest cannot carry + // a `views:` entry the seam refuses. Deliberately a SHAPE-CLASS + // judgement, not a full `ViewSchema.parse`: which class an entry + // belongs to is the seam's contract; whether a container's + // internals are well-formed stays the authoring/publish doors' + // job (defineStack, `os validate`, the metadata door). + // The envelope and the words: {@link viewsEntryNotContainerRefusal}, + // which the residual registration throws too. Judged on the body the + // seam registers (its `name` reconciled to the derived key). + if (key === 'views') { + const toRegister = item.name === itemName ? item : { ...item, name: itemName }; + if (!isViewContainerShaped(toRegister)) return viewsEntryNotContainerRefusal(itemName, sourceLabel, ownerId); + } + } + } + // [#5320] The `viewItems:` channel, each entry judged by its schema and + // the PARSED body's owner stamp (#28534), as the seam registers it. + const assembledItems = (source as any)?.[ASSEMBLED_VIEW_ITEMS_KEY]; + if (Array.isArray(assembledItems)) { + for (const item of assembledItems) { + const parsed = AssembledViewArtifactSchema.safeParse(item); + if (!parsed.success) return assembledViewItemRefusal(item, parsed.error, sourceLabel, ownerId); + const body = parsed.data as Record; + const itemName = resolveMetadataItemName('views', body); + if (!itemName) continue; + const refusal = collectionItemOwnerStampRefusal(body, 'view', itemName, ownerId); + if (refusal) return refusal; + } + } + return undefined; +} + +/** + * [#28697] The seed step's owner-stamp refusal ({@link collectionItemOwnerStampRefusal}, + * #28534: the one collection filed outside the collection seam), asked over a + * package body's `data` datasets before anything of the package registers. + * The first refusal found, or `undefined`. Called by {@link ObjectQL.registerApp}. + */ +export function packageBodySeedDataRefusal(manifest: any, ownerId: string): Error | undefined { + const seedData = manifest?.data; + if (!Array.isArray(seedData)) return undefined; + for (const dataset of seedData) { + // The seed step registers a dataset with a target object only. + if (!dataset?.object) continue; + const refusal = collectionItemOwnerStampRefusal(dataset, 'data', String(dataset.object), ownerId); + if (refusal) return refusal; + } + return undefined; +} + +/** + * [#5320] The refusal of a `viewItems:` entry its schema + * (`AssembledViewArtifactSchema`) does not admit: `INVALID_METADATA` / 422, + * naming the entry and its source. Asked by {@link packageBodyCollectionRefusal} + * and by the collection seam, which parses the entry again for the body it + * registers. ⛔ Never a second copy of the text. + */ +export function assembledViewItemRefusal( + item: unknown, + error: { issues: ReadonlyArray<{ message: string }> }, + sourceLabel: string, + ownerId: string | undefined, +): Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 } { + const itemName = resolveMetadataItemName('views', item) ?? '(unnamed)'; + const err = new Error( + `Invalid \`${ASSEMBLED_VIEW_ITEMS_KEY}:\` entry '${itemName}' from ${sourceLabel} '${ownerId}': ` + + 'the assembled-manifest channel carries non-container view artifacts only — a ViewItem ' + + 'record (`viewKind` + `config`) or a flattened list/form overlay ' + + '(`AssembledViewArtifactSchema`, @objectstack/spec). A view CONTAINER travels in `views:`. ' + + `First issue: ${error.issues[0]?.message ?? 'no issue detail'}`, + ) as Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 }; + err.code = 'INVALID_METADATA'; + err.status = 422; + err.httpStatus = 422; + return err; +} + +/** + * [#28697] Stamp an app object ({@link stampPackage}) and register it through + * the registry's app verb, recording the stamp the registration writes onto + * the caller's own object ({@link noteRegistrationStamp}): `stampPackage` (with + * a version) and `registerItem` (without one) stamp the object in place, so a + * module constant handed to a second registration arrives wearing the first + * one's stamp, which no body wrote. The stamp before is read BEFORE + * `stampPackage`, the order the collection and seed loops keep, so a versioned + * registration is recorded too. + */ +export function registerAppNotingStamp( + registry: SchemaRegistry, + app: any, + ownerId: string, + ownerVersion: string | undefined, +): void { + const stampBefore = (app as { _packageId?: unknown } | null | undefined)?._packageId; + registry.registerApp(stampPackage(app, ownerId, ownerVersion), ownerId); + noteRegistrationStamp(app, stampBefore); +} + +/** + * [#5320] The registration door's refusal of a `views:` entry that is not a + * view CONTAINER (`isViewContainerShaped` is false: a ViewItem's + * `viewKind`/`config`, a flattened overlay's inline view config): the stack + * `views:` collection carries containers only, the contract the stack schema + * declares (`z.array(ViewSchema)`). The caller judges the shape; this is the + * envelope and the words — `INVALID_METADATA` / 422, naming the entry by the + * name it would be filed under and the source it came from. + * + * Called by {@link ObjectQL.registerApp}'s metadata loop and by the stack's + * RESIDUAL registration (`registerUnclaimedTopLevelItems` in `plugin.ts`, with + * the `manifest` source label), so a residual entry and a body's are refused + * in one spelling. ⛔ Never a second copy of the text. + */ +export function viewsEntryNotContainerRefusal( + itemName: string | undefined, + sourceLabel: string, + ownerId: string | undefined, +): Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 } { + const err = new Error( + `Invalid \`views:\` entry '${itemName}' from ${sourceLabel} '${ownerId}': the stack ` + + '`views:` collection carries view CONTAINERS only. `viewKind`/`config`/inline view ' + + 'config belong to a single VIEW, not to the container — wrap it: ' + + '`defineView({ list: { type, data, columns, … } })`, or name it — ' + + '`defineView({ listViews: { my_view: { … } } })`. A machine-assembled manifest ' + + `(package export, environment artifact) carries non-container view artifacts under ` + + `\`${ASSEMBLED_VIEW_ITEMS_KEY}:\` instead — re-export the package with a runtime that ` + + 'writes that channel.', + ) as Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 }; + err.code = 'INVALID_METADATA'; + err.status = 422; + err.httpStatus = 422; + return err; +} + +/** + * {@link stampPackage} for an authored OBJECT definition, on a shallow copy. + * The registry stamps an object schema on its own copy (`applySystemFields` + * and friends run first), so `registerApp` leaves the caller's definition + * unstamped, and a manifest registered twice, or recorded by a second install, + * shows the same objects both times. `applyProtection` writes top-level keys + * only, so a shallow copy holds every key it touches. + */ +export function stampObjectPackage(def: ServiceObject, packageId: string, packageVersion: string | undefined): ServiceObject { + return packageVersion === undefined ? def : stampPackage({ ...def }, packageId, packageVersion); +} + +/** + * [ADR-0139 D2] What the `ReadableIds` composition asks of the registered + * `ISecurityService`: both halves of a read, the object-level one through + * `canReadObject` or, when a service predates that optional method, the + * `allowed` of its `explain`. + */ +export type ReadableIdsSecurity = Pick & Partial>; + +/** The three resolutions of the `security` service (`ObjectQL#resolveReadableIdsSecurity`). */ +type ReadableIdsSecurityResolution = + | { kind: 'usable'; svc: ReadableIdsSecurity } + | { kind: 'absent' } + | { kind: 'unusable'; why: string }; + +/** + * [ADR-0139 D2, D6 as amended] One read's `ReadableIds` composition, carried + * through the chain of composed scopes (`ObjectQL#composeReadableIdsChain`): + * the read's object and verb (every refusal names them), the security + * resolution made once for the read, the caller's context handed to every + * level as it stands, and the answers — per parent object, `{ scope }` with + * the scope already composed, or `null` for `$in: []` — so a parent is asked + * once per read at whichever level it first appears. + */ +export interface ReadableIdsChain { + readonly object: string; + readonly operation: 'find' | 'findOne' | 'count' | 'aggregate'; + readonly security: ReadableIdsSecurityResolution; + readonly context: ExecutionContext | undefined; + readonly answers: Map; +} + +/** + * [ADR-0139 D5.2 / D5.3] A `controlled_by_parent` detail whose master the + * engine composes no `ReadableIds` leaf over — + * `ObjectQL#findUnservableControlledByParentMasters`, refused at boot by + * `ObjectQLPlugin`. + */ +export type UnservableControlledByParentMaster = { + /** The `controlled_by_parent` detail. */ + readonly detail: string; + /** Its one required `master_detail` field. */ + readonly field: string; + /** The object that field names. */ + readonly master: string; +} & ( + | { readonly class: 'object-scoped-read-middleware' } + | { readonly class: 'other-datasource'; readonly objectDatasource: string | undefined; readonly parentDatasource: string | undefined } +); + +/** + * [ADR-0139 D2] The engine slot where `ObjectQLPlugin` puts the SOURCE the + * `ReadableIds` composition reads the registered `ISecurityService` from: a + * function that resolves the kernel's `security` service when a leaf is + * composed, never at init, so plugin-registration order cannot decide + * whether a leaf is composed or denied. Written by assignment, so the last + * registration wins. + * + * An empty slot is a DEFINED state, and it fails closed: so does a slot that + * holds something other than a function, a source that answers nothing or + * throws, and a service that cannot answer both halves of a read. Each + * answers every leaf `$in: []` (`ObjectQL#resolveReadableIdsSecurity`). + * + * A wiring seam between this package's plugin and its engine, and nothing + * more: ⛔ not a member of `ObjectQL`, and ⛔ not re-exported by `index.ts` or + * `core.ts`, so it adds nothing to the surface either entry publishes. A + * `Symbol.for` key (the shape `plugin.ts` already uses for its string-keyed + * `skipSchemaSync` slot), so every bundle of this package — both entries, + * ESM and CJS — names the one slot. + */ +export const SECURITY_SERVICE_SOURCE_SLOT: unique symbol = Symbol.for('objectstack.objectql.securityServiceSource'); + +/** + * [ADR-0142 D4] The engine's own channel for {@link OperationContext.writePrecondition} + * on an `update` it issues itself (the assignment cursor). A module-private + * symbol, so no options object a door builds — and nothing outside this + * module — can carry it. + */ +const WRITE_PRECONDITION_OPTION: unique symbol = Symbol('objectql.writePrecondition'); + +// [#29986 codemod] Declared above without `export`, and read by `engine.ts` or a section module. +export { + AUTONUMBER_COLLISION_ATTEMPTS, AUTONUMBER_SEED_PAGE_SIZE, AutonumberReaders, CascadeIssuedDelete, + DISPATCHABLE_HOOK_EVENTS, ENGINE_AGGREGATE_OPTION_KEYS, ENGINE_DELETE_OPTION_KEYS, ENGINE_FIND_OPTION_KEYS, + ENGINE_QUERY_SLOTS, ENGINE_UPDATE_OPTION_KEYS, ENGINE_WIRE_ONLY_SLOTS, FormulaFaultSink, FormulaRelated, + IssuedAutonumber, METADATA_ARRAY_KEYS, ReadableIdsSecurityResolution, StoredExpressionServing, TransactionScope, + WIRE_ONLY_CANONICAL_SHAPES, WRITE_PRECONDITION_OPTION, applyFormulaPlan, applyStoredExpressionsToPatch, + assertDispatchableHookEvent, assertHookScopeNotSelfCancelling, assertOrderByIsMaterializable, + assertProjectionHasNoDottedPaths, assertValidHookExcludeObjects, assertValidHookObject, + assertValidUnscopedMultiWriteFlag, autonumberFieldNames, callerKeySets, callerSuppliedRow, computeStoredExpressions, + deleteWalkTargetOf, describeNonFilterWhere, emptySearchSetError, formulaPlanEntry, handBackWithheldInsertReadonly, + hydrateWriteFormulas, isWhereFilterObject, judgeWhereAdmission, noteRegistrationStamp, + organizationPredicateDeleteRefusal, planFormulaProjection, planStoredExpressions, readStoredAutonumberCounter, + recomputeAutonumberReaders, refuseSearchOverrideNamingNoField, refuseSearchOverrideNamingNoSearchableField, + replacedStoredExpressionFields, resolveMetadataItemName, resolveWhereFilterTokens, rowsWithoutFormulaWidening, + settleBulkStoredExpressions, statesDeleteSemantics, storedExpressionsReading, whereResolvedTokenJudge, + withFileOwnerKey, withholdInsertReadonlyFromHooks, withoutFormulaWidening, +}; diff --git a/packages/objectql/src/engine.ts b/packages/objectql/src/engine.ts index a7eadd3cbbe..6a8503def95 100644 --- a/packages/objectql/src/engine.ts +++ b/packages/objectql/src/engine.ts @@ -5,7 +5,7 @@ import { QueryAST, QueryInput, HookContext, ServiceObject } from '@objectstack/s // [commit 74155c735] The defaulting node schema `fillQueryAstDefaults` runs author input // through — the declared `.default()` stays in `packages/spec`, the engine // only invokes it. -import { SortNodeSchema, FullTextSearchSchema, resolveSearchFieldResolution } from '@objectstack/spec/data'; +import { SortNodeSchema } from '@objectstack/spec/data'; import { EngineQueryOptions, DataEngineInsertOptions, @@ -13,28 +13,16 @@ import { EngineDeleteOptions, EngineAggregateOptions, EngineCountOptions, - RPC_QUERY_ALIAS_SLOTS, - foldQueryAliasSlots, - QUERY_CURSOR_REMOVED, - QUERY_DISTINCT_REMOVED, - ENGINE_UPDATE_UPSERT_REMOVED, - type QueryAliasSlot, - type DroppedFieldsEvent + type DroppedFieldsEvent, } from '@objectstack/spec/data'; import type { WriteObservabilityOptions } from '@objectstack/spec/contracts'; // The validate-only result IS the protocol's response shape (commit 18189983d): the // engine is what `metadata-protocol.validateData` returns, so letting the two // drift would put a translation layer between a verdict and its contract. import type { ValidateDataResponse } from '@objectstack/spec/api'; -import { parseAutonumberFormat, renderAutonumber, resolveAutonumberFormat, readAutonumberCounter, missingFieldValues, FILE_REFERENCE_TYPES, REFERENCE_VALUE_TYPES, referenceTargetOf, referenceCarrierOf, isCurrentUserDefaultToken, isNowDefaultToken, isMultiValueField, driverSupportsTransactions, AUDIT_PROVENANCE_FIELDS, resolveInjectedColumnProvenance } from '@objectstack/spec/data'; +import { parseAutonumberFormat, resolveAutonumberFormat, FILE_REFERENCE_TYPES, REFERENCE_VALUE_TYPES, referenceTargetOf, driverSupportsTransactions } from '@objectstack/spec/data'; // [#5158] Door 2's lowering sink — the SAME pair the protocol face (Door 1) // runs, so `FilterArray` has exactly one lowering in the product. -import { - isFilterAST, - parseFilterAST, - normalizeFilterComparandTypes, - VALID_AST_OPERATORS, -} from '@objectstack/spec/data'; // [ADR-0053 D-D1, amended 2026-09-30 — #5930] The shared `FilterCondition → // FilterCondition` lowering, run once per filter position after the doors and // after token resolution (`resolveThenLowerWhere`), so every driver and the @@ -53,36 +41,15 @@ import { type ActionActivationWriteOptions, } from './action-activation.js'; import { - assertListComparandShapes, - assertFilterIsMaterializable, - invalidFilterError, collectComposableReadableIds, - describeReadableIdsLeaf, - readableIdsRefusalError, type ComposableReadableIdsSite, } from './filter-comparand-shape.js'; -import type { ISecurityService } from '@objectstack/spec/contracts'; -import { - assertHavingTemporalComparandsInterpretable, - assertResolvedTemporalTokensInRange, - assertTemporalComparandsInterpretable, - type ResolvedTokenJudge, -} from './temporal-comparand-door.js'; -import { assertTextOperatorTargetsAreStringCapable } from './text-operator-declared-type-door.js'; -import { - mapRelationConditions, - narrowHavingNumberComparands, - narrowNumberComparands, -} from './number-comparand-declared-type-door.js'; import { - lowerRelationSite, type RELATION_FILTER_ID_CAP, - type RelationFilterSite, } from './relation-filter-lowering.js'; // Seek pagination for the walks that must read EVERY row — the autonumber seed // scan is one (#6249). Shared with `summary-backfill` rather than re-rolled: // the cursor merge is the part that is easy to get subtly wrong. -import { keysetWalk, type KeysetPageQuery } from '@objectstack/types'; import { DATA_MIGRATION_FLAG_OBJECT, FILE_REFERENCES_MIGRATION_ID, @@ -161,10 +128,9 @@ import { buildDriverOptions, driverReadScope, driverWriteTarget, engineLayer0Ver // methods of the same names below hand each body the members it reads (`MigrationFlagsHost`). import { announceOpenMigrationGates, isOrganizationOwnershipCertified, isValueShapesMigrationVerified, noteAdmittedValueShapeViolation, readFileReferencesFlagRow, readMigrationFlagVerified, supplyTenantKeyCertifiedResolver, wasDatastoreCreatedFromEmpty } from './engine-migration-flags.js'; import { ExecutionContext, ExecutionContextSchema, type InstalledPackageSource } from '@objectstack/spec/kernel'; -import { isStoredMetadataBodyObject, judgeCollectionItemOwnerStamp } from '@objectstack/spec/kernel'; +import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel'; // [#27884] The stored-row read gate, registered on the family's tables below. import { STORED_METADATA_TYPE_READ_GATE_OBJECTS, storedMetadataTypeReadGate } from './stored-metadata-type-read-gate.js'; -import type { FlowFunctionEffect } from '@objectstack/spec/automation'; // Imported from spec directly rather than through `@objectstack/core`'s // re-export block: that block is labelled backward-compatibility, and this // contract is new (#5945). @@ -181,29 +147,15 @@ import { createLogger, withTransientRetry, type RetryOptions, - filterTokenContextFrom, resolveFilterTokens, - // [#7867] The repo's ONE single-record 404 (#4435/#5138). It lives in - // `@objectstack/core` precisely so this file can reach it: ADR-0076 D2's - // boundary ratchet forbids the `/core` entry closure — engine.ts included — - // from importing `@objectstack/metadata-protocol`, where it was written. recordNotFoundError, - // The data door's object-existence 404, shared for the same reason: an - // in-process verb refuses a name the registry does not resolve with it. objectNotFoundError, - // [#24497] The ONE grants resolver, the one the request path seeds a - // context's grants, peer list and email (#24639) through — see `ScopedContext[SEED_USER_GRANTS]`. resolveUserAuthzGrants, - // [#25653] The app-automation elevation mark: set by a hook's - // `runAs: 'system'` derivation, carried across `createContext`'s parse — - // see `ScopedContext.withRunAs` — and by an app-authored hook's inherited - // elevation and `sudo()`, and `ObjectRepository.execute` — see - // `ScopedContext[APP_AUTHORED_HOOK_API]`. markAppAutomationElevation, carryAppAutomationElevation, appAutomationElevationOf, } from '@objectstack/core'; -import { WriteEpoch, isWriteEpochOperation } from './write-epoch.js'; +import { WriteEpoch } from './write-epoch.js'; import { bridgeAuthzInvalidation } from './authz-invalidation-bridge.js'; import { SummaryRecomputeError, cascadeStoppedRefusal, hookStoppedRefusal, rollbackFailedRefusal, rolledBackRefusal, type SummaryRecomputeFailure } from './summary-errors.js'; import { TransactionUnsupportedError } from './transaction-errors.js'; @@ -258,7 +210,6 @@ import { convertPackageBodyForward, registerPlugin, resolveNavObjectNames } from // answer, so it asks THEM — hand-writing a fifth dialect word-list inside the // engine is the consumer-side tolerant parsing PD #12 forbids and precedent // #5841 retired. -import { isUniqueViolationError, uniqueViolationColumn } from '@objectstack/types'; import { DuplicateRecordError } from './duplicate-record-error.js'; import { NotNullRefusalError, envelopeDriverWriteRefusal } from './not-null-refusal-error.js'; import { ConcurrentUpdateError } from './concurrent-update-error.js'; @@ -273,5113 +224,250 @@ import { redactBoundStatement, redactPropagatedDriverFault } from '@objectstack/ // write does, or is refused rather than filed under the `__global__` // pseudo-tenant. The decision is a pure function of measured facts; this file // supplies the facts and turns a `refuse` into the throw. -import { - resolveSystemWriteOrganization, - resolveTenantFieldName, - carriesOrganization, - SystemWriteOrganizationRequiredError, - ORGANIZATION_OBJECT, -} from './tenancy/system-write-organization.js'; -import { - type TenancyPosture, - type TenantLayer0Verdict, - postureUsesUnionScope, - judgePermissionSetOwnerBinding, -} from '@objectstack/spec/security'; - -/** - * Per-row outcome of {@link ObjectQL.insertMany} (framework#3172). One entry - * per input row, in input order: written rows carry the after-hook record, - * failed rows carry the per-row error (validation / autonumber / encryption). - * - * [#20922] A written row also carries `droppedFields`: the caller-supplied - * fields the engine LEGALLY stripped from THAT row, one `DroppedFieldsEvent` - * per reason — the per-row channel, recorded at the strips. Present only when - * at least one field was taken from the row. The batch-level `onFieldsDropped` - * events are unchanged and still name no row; this key is not a resolution of - * them. A failed row carries none: its write did not complete. - */ -export type InsertManyRowOutcome = - | { ok: true; record: any; droppedFields?: DroppedFieldsEvent[] } - | { ok: false; error: unknown }; -import { CoreServiceName, StorageNameMapping } from '@objectstack/spec/system'; -import { IRealtimeService } from '@objectstack/spec/contracts'; -import type { ICryptoProvider } from '@objectstack/spec/contracts'; -import { - collectInternalReadFields, - collectCredentialFields, -} from './secret-fields.js'; -// [#25522 cut 18] `count` and `aggregate()`'s credential guard -// (`rejectCredentialAggregation`); the methods of the same names below hand -// each body the members it reads (`CountAggregateHost`). -import { count, rejectCredentialAggregation } from './engine-count-aggregate.js'; -// ADR-0100: the credential-column guard's filter / sort half, beside the -// aggregate half above — one collector, one envelope. -import { rejectCredentialEvaluation } from './credential-evaluation-guard.js'; -import { assertNoInternalFieldEvaluated, rejectInternalFieldEvaluation } from './internal-field-evaluation-guard.js'; -import { assertGroupByNamesNoJsonStoredField } from './group-by-structured-json-door.js'; -import { assertProjectionNamesAField, refuseProjectionOfUnknownColumns } from './empty-projection-door.js'; -import { assertAggregationFieldTypesAccepted } from './aggregate-field-type-door.js'; -import { assertRecordCurrencyAggregatesGrouped } from './record-currency-aggregate-door.js'; -import { assertRecordCurrencyCodes, touchesRecordCurrency } from './validation/record-currency-code.js'; -import { - EXCHANGE_RATE_OBJECT, - homeAmountsTouched, - memoizeHomeAmountServing, - planHomeAmounts, - projectHomeAmounts, - settleBulkHomeAmounts, - type HomeAmountPlanEntry, - type HomeAmountServing, - type HomeLocalization, - type HomeLocalizationResolver, -} from './home-amount-projection.js'; -import { assertAggregatedColumnsAreDistinct, assertDateBucketTargetsAreTemporal } from './aggregate-query-shape-door.js'; -import { applyProtection, pluralToSingular, ExternalWriteForbiddenError } from '@objectstack/spec/shared'; -import { SchemaRegistry, type ArtifactInstallScope } from './registry.js'; -import { expandSearchToFilter, normalizeSearch, SEARCHABLE_ENUM_TYPES, SEARCHABLE_TEXTUAL_TYPES } from './search-filter.js'; -import { stripSearchCompanion } from './search-companion.js'; -import { ExpressionEngine, analyzeRelationshipTraversals, isUnparseableDate, type EvalError, type EvalOptionLabels, type EvalPermissions } from '@objectstack/formula'; -import { declaredOptionLabels, readerOptionLabels } from './formula-option-labels.js'; -import type { Expression } from '@objectstack/spec'; -import { - isAggregatedViewContainer, - expandViewContainer, - // [#5320] The assembled-manifest view channel: `views:` carries containers - // only (judged by the SAME classifier the producers partition with), and - // non-container view artifacts enter through the declared `viewItems:` key. - ASSEMBLED_VIEW_ITEMS_KEY, - AssembledViewArtifactSchema, - isViewContainerShaped, -} from '@objectstack/spec'; -// [commit 3c1bbd2a8] The ONE spelling of "which object does an aggregated `defineView` -// container bind to", imported rather than re-spelled — from the LEAF subpath, -// for the reason the `/errors` import above states (commit 3bd9b3498). See -// `resolveMetadataItemName` below for why this registrar lost its fourth copy. -import { deriveViewContainerObject } from '@objectstack/metadata/view-container'; -// [#20331] The divergent view-container `name` refusal — the ONE judge this -// registrar and `os validate` both call. -import { viewContainerNameRefusal } from './view-container-name-refusal.js'; -import { bindHooksToEngine, type BindHooksOptions } from './hook-binder.js'; -import { declaredHookCondition } from './hook-wrappers.js'; -import { validateRecord, validateRecordInScope, normalizeMultiValueFields, normalizeBlankTypedValues, normalizeNumericStringValues, coerceBooleanFields, ValidationError, type FieldValidationError, isScannableValueShapeField } from './validation/record-validator.js'; -import type { AdmittedValueShapeViolation, AdmittedValueShapeViolationSink, ValidationMessageContext } from './validation/record-validator.js'; -import type { RelatedFieldBinding, RelatedRecordBinding } from './validation/rule-validator.js'; -import { createBatchSummaryAccumulator } from './summary-batch-accumulation.js'; -import { collectReferenceTraversals, referenceTraversalConflicts, evaluateValidationRules, emitValidationAdvisories, optionVisibilityReadsPermissions, readsPermissionPredicate, needsPriorRecord, stripReadonlyWhenFields, stripReadonlyWhenFieldsMulti, hasReadonlyWhenInPayload, hasParentScopedReadonlyWhenInPayload, hasParentScopedRequiredWhen, stripReadonlyFields, stripRuntimeOwnedFields, staticReadonlyInsertSubject, preserveAuditIgnoredOnInsertWarning } from './validation/rule-validator.js'; -// [#14088] The before-phase write recorder — the provenance channel the static -// `readonly` strip needs to tell a hook's write from a caller's echo of the -// SAME value. Armed and sealed in `update()`; the module owns the argument for -// why neither end may move. -import { recordHookPayloadWrites } from './hook-write-provenance.js'; -// [commit 706ad0fcc] The hide pass's other half: when a hook faults reaching THROUGH a -// key that pass withheld, this names the key, says the platform withheld it, -// and points at `ctx.previous` — the module owns the measurement and the -// reason the explanation cannot be composed any further downstream. -import { dispatchHooksExplainingWithheldReadonly } from './hook-withheld-readonly-fault.js'; -import { UnscopedHookApi, SEED_USER_GRANTS, APP_AUTHORED_HOOK_API, type HookRunAs, type HookRunAsRef, type RunAsDerivableApi } from './hook-run-as.js'; -import type { HookWriteRecording } from './hook-write-provenance.js'; -import { resolveMasterDetailRelation } from './master-detail.js'; -// [#6457] The master-detail header a `parent`-scoped predicate reads is made -// total over the MASTER's declared fields before it leaves this engine — the -// same helper every other server seam materialises with (#1871/#4649/#4953). -import { materializeDeclaredFields } from './declared-fields.js'; -// [#21571] The declared column set — the read verbs' default projection, their -// explicit-projection filter and the write path's undeclared-key door, one list. -import { - PLATFORM_PROVISIONED_COLUMNS, - declaredColumnSet, - rowsWithDeclaredColumnsOnly, - rowsWithSelectedColumnsAnswered, - selectedStoredColumns, - storedColumnsLeftUnwritten, - withDeclaredColumnsOnly, - withSelectedColumnsAnswered, -} from './declared-read-columns.js'; -// [#21777] "Is this schema the remote's?" One predicate, shared with the boot sync. -// [#21910, #21918] And its injected-column refinement, which both cascade walks ask. -import { isFederatedObject, isFederatedUnprovisionedInjectedColumn } from './federated-object.js'; -// [#27277] The correctness floor over a plugin driver on an external datasource. -import { - createExternalDriverFloor, - externalPredicateWriteRefusal, - isExternalDriverFloor, - needsExternalDriverFloor, - unwrapExternalDriverFloor, -} from './external-driver-floor.js'; -// [#25522 cut 14] The master-detail landing, the preview's parents and the -// related reads a validation rule makes; the methods of the same names below -// hand each body the members it reads (`MasterDetailRelatedHost`). Imported after -// every module those bodies import, so the engine's module evaluation order -// is what it was. -import { resolveOneHopRelated, resolvePredicateRelated, resolvePreviewParents, settleMasterDetailLanding } from './engine-master-detail-related.js'; -// [#25522 cut 15] The `ReadableIds` chain composition, the controlled-by-parent -// boot screen, and the permission and formula resolutions an operation shares; -// the methods of the same names below hand each body the members it reads -// (`ReadableIdsPermissionFormulaHost`). Imported after every module those bodies import, so -// the engine's module evaluation order is what it was. -import { composeReadableIdsChain, findUnservableControlledByParentMasters, permissionResolution, readableIdsScopeOf, reportFormulaFault, resolveDefaultPermissions, resolveFormulaPermissions, resolveOptionPermissions } from './engine-readable-ids-permission-formula.js'; -// [#25522 cut 16] Type-only, for the `{@link}`s on `composeReadableIdsAtBoundary`'s and -// `composeReadableIdsChain`'s docblocks: the constant moved with its reader in cut 15. -import type { READABLE_IDS_MAX_CHAIN_DEPTH } from './engine-readable-ids-permission-formula.js'; -// [#25522 cut 19] The delete cascade's collection walk, the in-cascade -// `restrict` judge, the `DELETE_RESTRICTED` refusal, the reference-probe -// filter and the reference-check elevation record; the methods of the same -// names below hand each body the members it reads (`DeleteCascadeHost`). -// Imported after every module those bodies import, so the engine's module -// evaluation order is what it was. -// [#28001] …and the object-level closure of that walk the metadata read serves. -import { collectCascadeDeleteObjects, collectCascadeDeleteSet, deleteRestrictedError, recordReferenceCheckElevation, referenceProbeFilter, refuseRestrictsInsideOwnCascade } from './engine-delete-cascade.js'; -import { applyInMemoryAggregation } from './in-memory-aggregation.js'; -import { - resolveEngineDeleteDispatch, - ENGINE_DELETE_REJECT_MESSAGE, - type EngineDeleteDispatchInput, -} from './engine-delete-dispatch.js'; -import { - resolveEngineUpdateDispatch, - ENGINE_UPDATE_REJECT_MESSAGE, - // [#11142] The one reject thrower, shared with assertEngineUpdateDispatch, - // so a refusal that declares an ADR-0112 code/status (the conflicting-id - // reject) carries it identically from the real engine and from every - // pinned fake. - engineUpdateDispatchRejectError, - type EngineUpdateDispatchData, - type EngineUpdateDispatchInput, -} from './engine-update-dispatch.js'; -// [#25522 cut 16] The write preview's body; `validate` below hands it the -// members it reads (`ValidateHost`). Imported after every module that body -// imports, so the engine's module evaluation order is what it was. -import { validate } from './engine-validate.js'; -import { bindHookNotice } from './hook-notice.js'; -import { - applyHaving, - aggregatedRowColumns, - aggregatedRowColumnClasses, - aggregatedRowColumnTypes, - assertAggregationFilterIsEvaluable, - assertHavingIsEvaluable, - assertHavingIsFilterCondition, -} from './having-filter.js'; -// [#25522 cut 20] The `where`-lowering bodies (`QueryLoweringHost`), imported -// after every module they import, so module evaluation order is unchanged. -import { lowerRelationConditions, resolveSearchQueryableFields, resolveWhereTokens } from './engine-query-lowering.js'; -import { - auditDanglingReferences, - type AuditableObject, - type DanglingReferenceAuditOptions, - type DanglingReferenceReport, -} from './integrity/dangling-reference-audit.js'; - -/** - * One read of a `sys_migration` flag row: the verdict, plus whether the ledger - * could actually be consulted to reach it (#4769). Both negatives keep the - * gate closed; only the conclusive one is worth remembering. - */ -export interface MigrationFlagRead { - verified: boolean; - conclusive: boolean; - /** - * [#15989] Have the file-family COLUMNS on this deployment moved to the - * bare-id encoding — `hasMovedFileColumns` over the same row? - * - * Carried on the SAME read rather than fetched by a second one, because it - * is a second question about one row and two reads could answer them out of - * one another's date. It is `false` on every row that lacks the stamp, on an - * unverified row, and on every way of not having read a row at all. - */ - columnsMoved: boolean; -} - -/** - * What this boot has ADMITTED against one ADR-0104 migration's contract — - * the counterexample that forbids attesting it (#4769). `count` is a lower - * bound (every admitted value is counted, but a revocation records the total - * known at the moment it wrote); `first` is the one an operator is shown, - * because a prescription needs a place to start, not a census. - */ -export interface AdmittedValueShapeViolationTally { - count: number; - first: { object: string; field: string; type: string; detail: string }; -} - -/** - * The lifecycle events the engine actually dispatches via `triggerHooks`. This - * is the single source of truth for what a hook can subscribe to — kept in - * lockstep with the `triggerHooks(...)` call sites and with `HookEvent` in the - * spec. `beforeFind`/`afterFind` cover both `find` and `findOne`; the write - * events cover both single-id and bulk (`multi: true`) writes (#3195). A hook - * subscribing to anything outside this set would silently never fire, so - * `registerHook` refuses it where the name is one of the engine's own - * ({@link UNDISPATCHED_LIFECYCLE_HOOK_EVENTS}, #17713) and warns otherwise. - * - * ## WHEN `after*` fires, relative to the commit (#7477) - * - * `afterInsert`/`afterUpdate`/`afterDelete` are dispatched INSIDE the unit of - * work, before the enclosing transaction (if any) commits. The declared - * meaning is **"the write has been requested and will happen unless this unit - * of work is undone"** — not "the write happened". This is the ruled semantics - * (#7477, 2026-08-11), not an accident of the current call sites: an `after*` - * dispatch is deliberately NOT deferred to commit, because deferring it would - * push a handler's own `ctx.api` writes outside the transaction the write ran - * in, and an in-engine audit hook depends on landing inside it. - * - * Four ordinary paths open such a unit around the dispatch: - * - the write door itself, when the object has a metadata-bound `after*` - * hook that can abort — blocking with `onError` `abort`, or carrying a - * `condition` — so that abort rolls the row back as `HookSchema.onError` - * says (#22782, #22856, `opensAfterAbortUnit`); - * - a by-id {@link ObjectQL.delete} whose cascade is `'atomic'` — each - * dependent's own `afterDelete` fires inside the wrap the parent opened, - * and the parent's row removal can still refuse afterwards (#7413). The - * PARENT's `afterDelete` is outside that wrap by construction (inside the - * door's own unit when the first path opened one); the cascaded - * CHILDREN's are not; - * - `runAtomicBatch` in `@objectstack/metadata-protocol` — - * `batchData`/`deleteManyData` with `atomic: true` runs every member's - * `after*` inside one transaction that aborts on the first failure - * (#4620); - * - any caller that opened `transaction()` / `ctx.api.transaction()` around - * the write itself. - * - * A rollback on any of those leaves a hook that fired for a row that still - * exists. Effects routed back through this engine roll back with it and are - * therefore safe; effects that leave the engine — webhooks, notifications, - * external index updates, file deletion — are the HANDLER's responsibility to - * make rollback-tolerant (idempotent and reconcilable, or re-checked against - * the row by a worker rather than trusted from the event alone). Documented - * for authors on `HookEvent` in `@objectstack/spec/data` and in - * `content/docs/automation/hooks.mdx`. - */ -const DISPATCHABLE_HOOK_EVENTS: ReadonlySet = new Set([ - 'beforeFind', 'afterFind', - 'beforeInsert', 'afterInsert', - 'beforeUpdate', 'afterUpdate', - 'beforeDelete', 'afterDelete', -]); - -/** - * [#17713] Every verb the engine can put on an {@link OperationContext}, as - * DATA rather than as a second hand-written list. - * - * The type annotation is the weld: `Record` is exhaustive in both - * directions, so a new engine method that widens `OperationContext['operation']` - * fails to compile here until the verb is added, and a verb dropped from the - * union makes its entry an excess property. `pnpm typecheck` compiles this file - * (it is `src`, not the test layer), so the weld is enforced by the package's - * own type check and not only by a test. - * - * It exists so {@link UNDISPATCHED_LIFECYCLE_HOOK_EVENTS} below is DERIVED. A - * hand-written list of "the events we refuse" would be this card's own defect - * one layer up: it would go stale the moment a verb is added, and silently. - */ -const ENGINE_OPERATION_VERBS: Record = { - find: true, - findOne: true, - insert: true, - update: true, - delete: true, - count: true, - aggregate: true, -}; - -/** - * [#17713] The engine's own lifecycle-event NAMESPACE — `before`/`after` × every - * verb in {@link ENGINE_OPERATION_VERBS}. Fourteen names, of which - * {@link DISPATCHABLE_HOOK_EVENTS} is the eight the engine actually dispatches. - */ -const ENGINE_LIFECYCLE_HOOK_EVENTS: ReadonlySet = new Set( - Object.keys(ENGINE_OPERATION_VERBS).flatMap((verb) => { - const suffix = `${verb.charAt(0).toUpperCase()}${verb.slice(1)}`; - return [`before${suffix}`, `after${suffix}`]; - }), -); - -/** - * [#17713] The GAP: a name inside the engine's own lifecycle namespace that the - * engine never dispatches. Derived, never typed out — the difference of the two - * sets above. - * - * Today it is exactly six: `beforeFindOne` / `afterFindOne`, `beforeCount` / - * `afterCount`, `beforeAggregate` / `afterAggregate`. Registering one of them is - * REFUSED at the door ({@link assertDispatchableHookEvent}) rather than warned - * about, because each is a name an author reaches for while believing they are - * subscribing to an engine lifecycle event — and what they get back is an inert - * declaration (ADR-0078: no silently inert declaration), which on a read filter - * means a guardrail that was never armed. - * - * ⚠️ The refusal is deliberately scoped to THIS set, not to "anything outside - * `DISPATCHABLE_HOOK_EVENTS`". A name outside the namespace entirely - * (`'myPlugin:flush'`) is a plugin dispatching its own event through the public - * {@link ObjectQL.triggerHooks} — the legitimate reading that made this branch a - * warn in the first place (#3195). That reading survives untouched; what stops - * being accepted is a near-miss spelling of the engine's OWN vocabulary, where - * no such reading exists. - */ -const UNDISPATCHED_LIFECYCLE_HOOK_EVENTS: ReadonlySet = new Set( - [...ENGINE_LIFECYCLE_HOOK_EVENTS].filter((event) => !DISPATCHABLE_HOOK_EVENTS.has(event)), -); - -/** - * [#17713] Test-visible view of the three sets the refusal is computed from, so - * a pin can drive its population FROM THE ENGINE instead of restating it — the - * acceptance criterion this card wrote ("a test per event name the engine - * exposes but never dispatches") is only meaningful if the population is read, - * not typed. - * - * Module-level export, deliberately NOT added to the package barrel - * (`src/index.ts` names its exports one by one), so this widens no published - * surface. - */ -export const HOOK_EVENT_DISPATCH_VOCABULARY = { - dispatchable: DISPATCHABLE_HOOK_EVENTS, - lifecycleNamespace: ENGINE_LIFECYCLE_HOOK_EVENTS, - undispatchedLifecycle: UNDISPATCHED_LIFECYCLE_HOOK_EVENTS, -} as const; - -/** - * [#4346] The alias slots the ENGINE option bags still admit, cut from the - * spec's own table (#3795) so the engine never re-declares a mapping. - * - * The deprecated `DataEngine{Query,Update,Delete,Count,Aggregate}Options` - * contracts declare `filter` on every read AND write method, but only `find` - * folded it — `findOne`/`count`/`update`/`delete`/`aggregate` passed the bag - * through with `where === undefined`, which every driver reads as "no - * predicate": a caller filtering with `{ filter }` silently matched EVERY row - * (an over-grant on the reads, an unbounded write on `update`/`delete`). - * - * `where` is the slot every method folds; `limit` additionally applies to the - * find-shaped bags, which declare `top` (OData) as its alias. The other four - * pairs in the table are wire spellings folded at parse by - * `QueryWithTransportSchema` / the protocol normalizer — their values need shape - * lowering (`sort` records, `populate` lists) that belongs to those layers, so - * the engine deliberately does not fold them. - */ -export const ENGINE_WHERE_SLOTS: readonly QueryAliasSlot[] = - RPC_QUERY_ALIAS_SLOTS.filter((slot) => slot.canonical === 'where'); -const ENGINE_QUERY_SLOTS: readonly QueryAliasSlot[] = - RPC_QUERY_ALIAS_SLOTS.filter((slot) => slot.canonical === 'where' || slot.canonical === 'limit'); - -/** - * [#4371] The slots the engine does NOT fold — the wire-only pairs - * (`select`→`fields`, `sort`→`orderBy`, `skip`→`offset`, `populate`→`expand`), - * derived as the complement of {@link ENGINE_QUERY_SLOTS} so a seventh pair - * added to the spec table lands on exactly one side of the split. - * - * Their values need shape lowering (`sort`'s `{field: 'asc'}` record form, - * `populate`'s name list) that belongs to the RPC/protocol layers, so folding - * here would re-implement the lowering per reader — the #3795 condition. But a - * DIRECT engine call never crosses those layers: the alias key used to ride - * the AST verbatim, drivers read only the canonical name, and the parameter - * was silently dropped — three shipped "latest N in arbitrary order" bugs - * (#4370) plus the engine's own `seedAutonumber`. Declared ≠ enforced - * (AGENTS.md PD #10): the find-shaped entry points now REJECT these spellings, - * naming the canonical key and shape, so the mistake throws at the call site - * instead of degrading the result. - * - * Deliberately NOT applied to the where-only methods - * (`update`/`delete`/`count`/`aggregate`): their contracts honour no - * sort/projection/pagination at all, so "pass `orderBy` instead" would - * redirect the caller to a key those methods silently ignore too. Unknown-key - * enforcement for those bags is #4371's option (2), scoped separately. - */ -const ENGINE_WIRE_ONLY_SLOTS: readonly QueryAliasSlot[] = - RPC_QUERY_ALIAS_SLOTS.filter((slot) => !ENGINE_QUERY_SLOTS.includes(slot)); - -/** - * Canonical shape each wire-only slot's value must be rewritten into — quoted - * by the rejection so the error carries the full migration, not just the key - * rename (the value shapes differ; that is WHY the engine cannot fold them). - */ -const WIRE_ONLY_CANONICAL_SHAPES: Record = { - fields: "a string[] of field names", - orderBy: "SortNode[]: [{ field, order: 'asc' | 'desc' }]", - offset: 'a number of rows to skip', - expand: 'a record of { relationName: QueryAST }', -}; - -/** - * [#4371 option 2] The driver-option keys the engine forwards verbatim: on - * `find`/`findOne`/`update`/`delete` the option bag IS the base of the driver - * options (`buildDriverOptions(object, ctx, bag)`), which is how a caller's - * explicit `tenantId` / `bypassTenantAudit` reaches the driver (pinned in - * engine.test.ts). `count`/`aggregate` never forward the bag, so these keys - * are deliberately NOT legal there — accepting them would be the exact - * silently-ignored contract this gate exists to close. - */ -const ENGINE_DRIVER_PASSTHROUGH_KEYS = [ - 'transaction', 'tenantId', 'tenantIds', 'timezone', 'bypassTenantAudit', 'preserveAudit', -] as const; - -/** - * [#4371 option 2] Per-method legal option keys. An option bag key outside - * the method's set is REJECTED at the entry point: the engine executes none - * of them, so the call would otherwise succeed with the option silently - * ignored — the `declared ≠ enforced` shape (PD #10) one layer below the - * wire-alias rejection above. - * - * Sources, in order: the method's `Engine*OptionsSchema` declared keys (minus - * the `retiredKey` tombstones `cursor`/`distinct`/`upsert` (#8057), which get - * their tombstone quoted instead of a generic rejection — the schema keeps them - * ONLY to carry that message, and this runtime path never parses); `searchFields` (read by - * `find` and `aggregate` at the `$search` expansion, sent by the protocol layer); - * `onFieldsDropped` and `strictReadonlyWrites` (`WriteObservabilityOptions` — - * contract-declared, deliberately outside the serializable Zod schema: the - * first because a function is unrepresentable in JSON Schema, the second - * because #5126 ruled that a write-refusal switch must not be settable from a - * wire body); and the driver pass-through keys above. The alias spellings - * (`filter`/`top`) are folded and deleted BEFORE this check runs, so they - * never reach it. - * - * A drift pin in engine-unknown-option.test.ts asserts each set equals its - * schema's shape (minus tombstones, plus the documented extras) so a key - * added to the spec cannot be silently rejected here. - */ -const ENGINE_FIND_OPTION_KEYS: ReadonlySet = new Set([ - 'context', 'where', 'fields', 'orderBy', 'limit', 'offset', - 'search', 'searchFields', 'expand', - ...ENGINE_DRIVER_PASSTHROUGH_KEYS, -]); -const ENGINE_UPDATE_OPTION_KEYS: ReadonlySet = new Set([ - 'context', 'where', 'multi', 'returning', 'onFieldsDropped', 'onValidationAdvisory', 'onMentionsWithheld', 'onNotice', 'strictReadonlyWrites', - ...ENGINE_DRIVER_PASSTHROUGH_KEYS, -]); -const ENGINE_DELETE_OPTION_KEYS: ReadonlySet = new Set([ - 'context', 'where', 'multi', - ...ENGINE_DRIVER_PASSTHROUGH_KEYS, -]); -// `search` / `searchFields` are READ by `count` and `aggregate` through the -// same ADR-0061 expander `find` uses ({@link ObjectQL.expandSearchOnOptions}), -// so a count or a grouped answer under a search is over the searched rows. -// Before their schemas declared them they were refused here, and the wire paths -// left them out: the unsearched groups answered, and a paged list's `total` -// under a search was a page-local estimate (#22790). -export const ENGINE_COUNT_OPTION_KEYS: ReadonlySet = new Set(['context', 'where', 'search', 'searchFields']); -const ENGINE_AGGREGATE_OPTION_KEYS: ReadonlySet = new Set([ - 'context', 'where', 'groupBy', 'aggregations', 'having', 'timezone', - 'search', 'searchFields', -]); - -/** - * [#27732] A search-field override — `searchFields`, or the object-form - * `search.fields` — that names no field is refused `400 VALIDATION_ERROR`: - * an empty list, or a comma string with no name in it (the `GET` door's - * transport spelling, which reaches the engine as sent). Absent (or `null`) - * still searches the object's default search fields. - * - * `resolveSearchFields` reads an empty override as absent, so the search - * widened to the whole default set: the caller wrote the key to narrow and was - * not told. The door refuses this set (`assertSearchFieldsAreSearchable`, - * #27484) and `QuerySchema.searchFields` / `FullTextSearchSchema.fields` - * declare the same refusal (#27683); the words are read off that declaration - * rather than copied, and the status and code are the door's, so - * `mapDataError` answers an in-process caller and a REST caller alike. Any - * other value is left to the expansion's existing tolerance. - */ -function refuseSearchOverrideNamingNoField(object: string, requested: unknown, param: string): void { - const names = typeof requested === 'string' - ? requested.split(',').map((s) => s.trim()).filter(Boolean) - : Array.isArray(requested) ? requested : undefined; - if (names === undefined || names.length > 0) return; - const declared = FullTextSearchSchema.shape.fields.safeParse([]).error?.issues[0]?.message; - const err: any = new Error( - `ObjectQL search on '${object}' names no field in '${param}': got ${JSON.stringify(requested)}. The query was not run: ` - + "an empty search-field list was read as no narrowing, which scans the object's default search fields. " - + `${declared}.`, - ); - err.status = 400; - err.code = 'VALIDATION_ERROR'; - err.param = param; - err.object = object; - throw err; -} - -/** - * [#28120] …and an override that names no field the search can scan is refused - * `400 INVALID_FIELD`, the code the door answers the same value with - * (`assertSearchFieldsAreSearchable`, #4254): every name unknown, unsearchable, - * or withheld from the scan (`internal: true`, credential types), or no entry a - * field name at all. - * - * `resolveSearchFields` drops the names outside the allowed set and, when none - * is left, falls back to the whole set — the typo or the renamed field the - * caller wrote to narrow the search ran over the object's default fields - * instead, untold. Judged on the set the expansion resolves over (`allowed`), - * so this refuses exactly the overrides that fallback would have widened. An - * override naming at least one allowed field is left to the intersection (the - * engine's tolerance for the other names stands), and an empty `allowed` is - * left to the expansion's own refusal (#24284), which names that cause. - */ -function refuseSearchOverrideNamingNoSearchableField( - object: string, - requested: unknown, - param: string, - allowed: readonly string[], -): void { - const entries = typeof requested === 'string' - ? requested.split(',').map((s) => s.trim()).filter(Boolean) - : Array.isArray(requested) ? requested : undefined; - if (entries === undefined || entries.length === 0 || allowed.length === 0) return; - const allowSet = new Set(allowed); - if (entries.some((f) => typeof f === 'string' && allowSet.has(f))) return; - const names = entries.filter((f): f is string => typeof f === 'string'); - const err: any = new Error( - `ObjectQL search on '${object}' names no searchable field in '${param}': got ${JSON.stringify(requested)}. ` - + 'The query was not run: an override naming none of the fields the search scans was dropped, which ' - + "scans the object's default search fields. Name a searchable field, or leave the override out to " - + 'search the default fields.', - ); - err.status = 400; - err.httpStatus = 400; - err.code = 'INVALID_FIELD'; - if (names.length > 0) { - err.field = names[0]; - err.fields = names; - } - err.param = param; - err.object = object; - throw err; -} - -/** - * Rows per page for the autonumber seeding scan (#6249). This is a PAGE size, - * not a cap: the walk pages until the scope is exhausted. The number is the one - * the old single-shot `limit: 5000` used, kept so the per-read cost against a - * driver is unchanged — what changed is that reaching it no longer ends the - * scan and truncates the max. - */ -const AUTONUMBER_SEED_PAGE_SIZE = 5000; - -/** - * How many times one insert may re-seed and re-issue after a unique-constraint - * collision on an engine-issued autonumber (#6806) before it refuses. - * - * Bounded on purpose. The FIRST re-issue is the one that matters: the collision - * proves the in-memory counter sits below the store's real max, and the re-seed - * that follows reads that max back, so attempt 2 is issued from the truth. A - * further collision means another writer took the number in between — real - * concurrency, which more spinning does not resolve (each attempt costs a full - * scope scan). Two spare attempts absorb a burst; past that the write fails - * loudly rather than looping against a live competitor. - */ -const AUTONUMBER_COLLISION_ATTEMPTS = 3; - -/** - * One autonumber the ENGINE issued on a row, paired with the counter it came - * from (#6806). Only engine-issued values are listed: a value an exempt writer - * supplied is the caller's, and a collision on it is the caller's to see. - */ -interface IssuedAutonumber { - /** Field the value was written to. */ - readonly field: string; - /** `object.field.` key in {@link ObjectQL.autonumberCounters}. */ - readonly counterKey: string; -} -/** - * [#17052] The write doors' failure META, assembled at the call site because - * the level moved off `error`. - * - * ## Why the level moved - * - * `insert` / `update` / `delete` each end in a `catch` whose next statement is - * `throw e`. AGENTS.md → *Degradation log levels* calls that shape out by name: - * "a failure handed to the CALLER is not a degradation at all … Do not bolt a - * `logger.error` onto such a site". The requester IS told — the throw is the - * answer — so an `error` line is a second, louder report of a fact the caller - * already has, and nothing downstream can tell it apart from a real one. The - * measured cost was a healthy first boot of every fresh project printing - * `ERROR Insert operation failed` while `@better-auth/oauth-provider` seeded - * `sys_oauth_resource` and treated the UNIQUE collision as the no-op its own - * docblock declares it to be. - * - * ## Why the diagnosis needs this helper to survive the move - * - * The `Logger` contract (`packages/spec/src/contracts/logger.ts`) gives an - * `Error` slot to `error` and `fatal` ONLY: `warn(message, meta?)` has two - * parameters, not three. Handing the redacted driver error to `warn` as `meta` - * does not merely lose the slot — `Error.message` and `Error.stack` are - * NON-ENUMERABLE, so every logger that renders meta by spreading it would - * serialize `{}` and the entry would arrive carrying nothing at all. That is - * exactly the loss #14095 refused (the failing column, MySQL's index name, the - * driver's own frames) arriving one level down. - * - * So the two shaped-by-ruling properties are preserved by building the meta the - * way `ObjectLogger.write()` builds it from the slot — `{ …meta, error: { - * message, stack } }` — rather than by handing over the Error: - * - * - **#8682** the value redaction still runs: callers pass - * `redactBoundStatement(...)`, and it is that result's `message`/`stack` - * that are read here. - * - **#14095** the log still takes the driver's error (the envelope's - * `cause`), not the envelope: callers still choose which value to hand in. - * - * The non-Error branch mirrors `ObjectLogger.writeErrorLike()`'s own fallback - * (`{ ...errorOrMeta, ...meta }`, the later argument winning), so a thrown - * non-Error — or a `DuplicateRecordError` with no `cause` — renders as it did. - * - * @param logged The value to report — already redacted by the caller. - * @param meta The entry's own meta (`object`, and `developerMessage` on the - * delete door). - */ -export function writeFailureLogMeta( - logged: unknown, - meta: Record, -): Record { - return logged instanceof Error - ? { ...meta, error: { message: logged.message, stack: logged.stack } } - : { ...(logged as Record | undefined), ...meta }; -} - -/** - * [#24948] The refusal of a predicate (`multi`) delete of `sys_organization`: - * the standard `DELETE_RESTRICTED`, saying to delete by id. An organization is - * deleted one at a time, through plugin-auth's organization delete: its - * refusal of an organization that still owns records and its - * `beforeDeleteOrganization` clean-up of the organization's teams, team members - * and admin grants run there, once per organization. A delete matching - * organizations by a filter runs neither, and no caller needs one. - */ -function organizationPredicateDeleteRefusal(): Error { - const err: any = new Error( - 'Organizations are deleted one at a time, by id: a delete matching organizations by a filter is refused. ' + - 'Delete each organization by its id.', - ); - err.developerMessage = - `A predicate delete of ${ORGANIZATION_OBJECT} is refused: organizations are deleted one at a time, because ` + - "plugin-auth's organization delete refuses an organization that still owns records and removes its teams, " + - 'team members and admin grants per organization, and a predicate delete runs neither. Delete each organization ' + - `by id (engine.delete('${ORGANIZATION_OBJECT}', { where: { id } })).`; - err.code = 'DELETE_RESTRICTED'; - err.status = 409; - err.httpStatus = 409; - err.object = ORGANIZATION_OBJECT; - return err; -} - -/** - * Read the counter out of ONE stored autonumber value, under #6468's anchoring - * rules. Shared by the seeding scan and by the adopt-on-exempt-write resync - * (#6806) so both readings can never drift apart — a divergence here is a - * duplicate record number, which is the harm the whole family is about. - * - * The ANCHORED reading itself is NOT this package's (#6560): it is the inverse - * of `renderAutonumber`'s composition, so it lives beside it as spec's - * {@link readAutonumberCounter}, which the SQL driver's `scanMaxNumericTail` - * calls over the same two strings. This function adds only the piece the two - * sides genuinely do not share: - * - * - **Either `prefix` or `suffix` declared ⇒ ANCHORED**: spec answers, and its - * TSDoc carries the rationale (counter after the prefix, suffix stripped - * when it matches and never required to match, out-of-scope values read as - * `undefined`). - * - **Neither declared ⇒ UNANCHORED**: this engine's own legacy reading — the - * LAST digit run of the whole value — kept byte-for-byte by PR #6553. The - * SQL driver's legacy reading of the same case is deliberately DIFFERENT (it - * concatenates every digit), which is exactly why spec refuses to answer for - * an unanchored slot instead of picking one of the two. - * - * # The unanchored arm is IMPLEMENTATION DETAIL, outside the declared contract (#7287) - * - * Spec's refusal above is no longer just an absence of agreement: the #7287 - * ruling (2026-08-10, 「宣告边界」) DECLARES that a stored value carrying - * non-digit content on an unanchored format is out of contract for counter - * readback, and that `readAutonumberCounter`'s `undefined` for that slot is the - * contract rather than a gap. The boundary and its rationale live in that - * function's TSDoc in `packages/spec/src/data/autonumber-format.ts`; the pins - * are `packages/spec/src/data/autonumber-unanchored-boundary.test.ts`. - * - * So the last-digit-run reading below is THIS ENGINE'S behavior, not a promise - * the platform makes. On the inputs the contract admits — pure-digit values, - * which is what `renderAutonumber` emits for an unanchored format — it answers - * the same number the SQL driver does. On mixed-content values it may answer - * differently (`'SO-2024-0007'` → `7` here, `20240007` there), and that - * difference is out of contract on both sides, not a defect on either. - * - * Consequently the ruling moves nothing here: neither this reading nor the - * driver's was hoisted into the shared helper, precisely because doing so would - * move live behavior on the other side over record numbers already issued. - * - * The unanchored branch uses the linear `/\d+/g` — a backtracking lookahead here - * is a polynomial-ReDoS sink on stored values full of zeros (CodeQL - * js/polynomial-redos). - */ -function readStoredAutonumberCounter(value: string, prefix: string, suffix: string): number | undefined { - if (prefix !== '' || suffix !== '') return readAutonumberCounter(value, prefix, suffix); - const runs = value.match(/\d+/g); - const digits = runs ? runs[runs.length - 1] : undefined; - if (!digits) return undefined; - const n = parseInt(digits, 10); - return Number.isFinite(n) ? n : undefined; -} - -/** Tombstoned option keys: rejected with the spec's own removal notice. */ -const ENGINE_RETIRED_OPTION_MESSAGES: Record = { - cursor: QUERY_CURSOR_REMOVED, - distinct: QUERY_DISTINCT_REMOVED, - // [#8057] `update.options.upsert` — declared-but-unenforced (ADR-0049): the - // key sat on the update allowlist while nothing read it, so `{ upsert: true }` - // was accepted and silently dropped. Removed rather than implemented; the - // spec's tombstone carries the create-if-absent prescription (#7867's - // not-found gate is the semantics a caller must reconcile with). - upsert: ENGINE_UPDATE_UPSERT_REMOVED, -}; - -/** - * The per-method legal key sets, exported for the drift pin ONLY - * (engine-unknown-option.test.ts asserts each set against its schema's shape, - * so a key added to the spec cannot be silently rejected here). Not a public - * API surface — consumers pass options, they do not read this table. - */ -export const ENGINE_OPTION_KEY_SETS: Readonly>> = { - find: ENGINE_FIND_OPTION_KEYS, - findOne: ENGINE_FIND_OPTION_KEYS, - update: ENGINE_UPDATE_OPTION_KEYS, - delete: ENGINE_DELETE_OPTION_KEYS, - count: ENGINE_COUNT_OPTION_KEYS, - aggregate: ENGINE_AGGREGATE_OPTION_KEYS, -}; - -/** - * Reject option-bag keys the engine does not execute (#4371 option 2). - * - * Runs AFTER `foldEngineOptionAliases`, so alias spellings are already folded - * away (or thrown on). `null`-valued keys pass — a `null` is a withdrawal - * carrying no intent a drop could lose, same rule as the fold. Retired keys - * (`cursor`/`distinct`) quote their tombstone. Everything else gets the legal - * key set, so the error carries the fix. - */ -export function rejectUnknownEngineOptions( - object: string, - operation: string, - bag: object | undefined, - legal: ReadonlySet, -): void { - if (!bag) return; - let unknown: string[] | undefined; - for (const [key, value] of Object.entries(bag)) { - if (value == null || legal.has(key)) continue; - (unknown ??= []).push(key); - } - if (!unknown) return; - const details = unknown.map((k) => - ENGINE_RETIRED_OPTION_MESSAGES[k] ? `'${k}': ${ENGINE_RETIRED_OPTION_MESSAGES[k]}` : `'${k}'`, - ); - throw new Error( - `${operation}('${object}') does not recognise option${unknown.length > 1 ? 's' : ''} ` + - `${details.join('; ')}. The engine executes none of ${unknown.length > 1 ? 'them' : 'it'}, ` + - `so the call would succeed with the option silently ignored. ` + - `Legal keys for ${operation}: ${[...legal].sort().join(', ')}.`, - ); -} - -/** - * Fold the deprecated alias spellings of an engine option bag into their - * canonical QueryAST keys, under the #3795/#4181 rule: an alias alone moves to - * the canonical key, redundant identical spellings collapse, DIFFERENT values - * for one slot are irreconcilable and throw (picking a winner IS the silent - * drop), and an explicit `null` alias is a withdrawal. - * - * `rejectSlots` ({@link ENGINE_WIRE_ONLY_SLOTS}) names the slots whose alias - * spellings the engine can neither fold nor honour: a non-null value under one - * throws, quoting the canonical key and shape (#4371). `null` stays a - * withdrawal here too — it carries no intent a drop could lose — and rides - * through for drivers to ignore, exactly as before. - * - * Returns the SAME reference when no alias spelling is present (the common - * path allocates nothing — `withResolvedWhere` discipline); otherwise folds a - * shallow copy, because the bag belongs to the caller and may be reused (view - * metadata, flow node config). - */ -export function foldEngineOptionAliases( - object: string, - operation: string, - bag: T, - slots: readonly QueryAliasSlot[], - rejectSlots?: readonly QueryAliasSlot[], -): T { - if (!bag) return bag; - if (rejectSlots) { - const refused = rejectSlots.flatMap((slot) => - slot.aliases - .filter((alias) => (bag as Record)[alias] != null) - .map((alias) => ({ alias, canonical: slot.canonical })), - ); - if (refused.length > 0) { - throw new Error( - `${operation}('${object}') does not accept ` + - `${refused.map((r) => `'${r.alias}'`).join(', ')}: ` + - refused - .map( - (r) => - `'${r.alias}' is a wire spelling of '${r.canonical}', folded by the RPC/protocol ` + - `layer — a direct engine call bypasses that fold, so the value would be silently ` + - `dropped, not applied. Pass '${r.canonical}' ` + - `(${WIRE_ONLY_CANONICAL_SHAPES[r.canonical] ?? 'the canonical QueryAST shape'}) instead.`, - ) - .join(' '), - ); - } - } - if (!slots.some((slot) => slot.aliases.some((alias) => alias in bag))) return bag; - const folded: Record = { ...bag }; - foldQueryAliasSlots(folded, slots, (conflict) => { - throw new Error( - `Conflicting options on ${operation}('${object}'): ` + - `${conflict.spellings.map((s) => `'${s}'`).join(', ')} are spellings of the same ` + - `parameter (canonical '${conflict.canonical}') and were given different values. ` + - 'Send exactly one.', - ); - }); - return folded as T; -} - -/** - * [#20121] Is this non-array `where` a filter OBJECT — the one non-array shape - * a driver reads as a predicate? - * - * The test is the object's built-in tag (`[object Object]`), not its - * prototype, and each edge shape lands where its measured answer puts it — - * nothing that answered correctly before this gate is refused by it: - * - * - a literal `{…}` and `{}` (match-all, a REAL query — the unscoped-write - * detector reads it that way too) → accepted; - * - `Object.create(null)` and an instance of an author's class with the filter - * on its own keys → accepted: both filtered correctly on `driver-memory` and - * `SqlDriver`, because a driver reads a filter's own enumerable keys; - * - `Map`, `Set`, `Date`, a boxed primitive, a `Promise` (an un-awaited filter - * builder) → refused: each carries its tag, has no own keys, and was - * silently read as "no filter". - * - * `undefined` and `null` never reach this test — the caller skips them, and - * both keep meaning "no filter" (`findOne`'s no-predicate guard and the - * unscoped-write detector both read `null` as absent). - */ -function isWhereFilterObject(where: unknown): where is Record { - return ( - typeof where === 'object' - && where !== null - && !Array.isArray(where) - && Object.prototype.toString.call(where) === '[object Object]' - ); -} - -/** - * [#20121] How a refused non-filter `where` is named in the refusal: the - * primitive's `typeof` plus its value (bounded), or the object's built-in tag. - * The `typeof` spelling is the wire door's (`received number`); the value is - * added because a direct caller has no request to look at. - */ -function describeNonFilterWhere(where: unknown): string { - if (typeof where === 'object' && where !== null) { - return Object.prototype.toString.call(where).slice('[object '.length, -1); - } - if (typeof where === 'function') return 'function'; - const text = typeof where === 'string' ? JSON.stringify(where) : String(where); - const bounded = text.length > 80 ? `${text.slice(0, 77)}...` : text; - return `${typeof where} ${bounded}`; -} - -/** - * [#20121] What a dropped `where` would have done, per verb. The read wording is - * the wire door's verbatim; a write has no result set, and "every row in - * scope" is what an unapplied predicate on `multi: true` rewrote or removed. - */ -const UNAPPLIED_WHERE_CONSEQUENCE: Readonly> = { - find: 'returned the unfiltered result set', - findOne: 'returned the unfiltered result set', - count: 'returned the unfiltered result set', - aggregate: 'returned the unfiltered result set', - update: 'updated every row in scope', - delete: 'deleted every row in scope', -}; - -/** - * **Door 2** — lower an arriving {@link FilterArray} on `where` to the - * `FilterCondition` the AST actually declares (#5158, maintainer ruling C). - * - * `FilterArray` — `['stage', '=', 'won']`, `['and', […], […]]`, `[[…], […]]` — - * is authoring sugar, and since #5285 the spec says so in as many words: it is - * declared INPUT-ONLY (`data/filter.zod.ts`), and `QuerySchema.where` is a - * `FilterCondition` that deliberately excludes it. There were two doors into - * the runtime and only one of them read the contract that way: - * - * - **Door 1**, the protocol/HTTP face (`metadata-protocol` `protocol.ts`), - * has always run `isFilterAST` → `parseFilterAST` and answered `400 - * INVALID_FILTER` for an array it could not lower. Nothing array-shaped - * survives it. - * - **Door 2**, a direct in-process engine call, passed the array through - * verbatim, and four drivers grew a second filter compiler to meet it — - * an INFIX dialect (`[condA, 'or', condB]`) that the spec never declared, - * that `parseFilterAST` cannot even express, and that cloud's - * `RemoteTransport.buildWhereSQL` refuses outright. Same query, two - * answers, decided by whether the caller went over the wire. - * - * This is that second door, closed: every entry point lowers through the SAME - * `parseFilterAST` sink Door 1 uses, so a driver sees exactly one filter - * dialect regardless of how the query arrived. The authoring ergonomics are - * untouched — `FilterBuilder` tuples, React block `filters` props and the five - * showcase call sites all still work, because lowering is what those shapes - * were always for. - * - * Three arrivals, three answers, matching Door 1 exactly: - * - * 1. `[]` — "no filter". The key is DELETED rather than lowered, which is the - * same reading every layer already gives it (`parseFilterAST([])` is - * `undefined`; `SqlDriver.applyFilters` returns early). Note this is now - * visible to `findOne`'s #4419 guard, which is the point: `findOne({where: - * []})` used to slip past the guard as "an expression tree the driver will - * interpret" and come back with an ARBITRARY row. - * 2. A well-formed AST — lowered. `isFilterAST` gates first so the operator - * vocabulary is checked before `parseFilterAST`'s lenient `$${op}` fallback - * can turn a misspelling into a `$sounds_like` condition nothing executes. - * 3. Anything else array-shaped — REFUSED, loudly, at the call site. Today - * those reach a driver and are refused there (#3948) with driver-internal - * wording, or — for the infix dialect — silently compiled by a second - * implementation. Failing here names the caller's own value. - * - * [#20121] A `where` that is not an array has to be a filter OBJECT, and that - * is checked FIRST, before any door below: a string, a number, a boolean, a - * `Map`, a `Date` — anything that is neither absent, a filter object nor an - * array — is refused with the same `INVALID_FILTER` / 400 envelope the array - * refusal (3) now carries. See {@link isWhereFilterObject} for the accept set - * and why each edge shape lands where it does. - * - * Returns the SAME reference when `where` is not an array (the overwhelmingly - * common path allocates nothing), otherwise a shallow copy: the bag belongs to - * the caller and may be reused (view metadata, flow node config). - */ -export function lowerWhereFilterArray( - object: string, - operation: string, - bag: T, - schema?: unknown, - schemaOf?: (name: string) => unknown, -): T { - if (!bag) return bag; - const where = (bag as Record).where; - // [#20121] The SHAPE gate, first on this seam and so first on all six verbs - // that call it (`find`, `findOne`, `count`, `aggregate`, `update`, - // `delete`) — each calls this before it resolves a driver, so a refusal - // here means no driver call is made at all. - // - // Before it, a string / number / `Map` `where` fell into the object branch - // below, every door there stepped around it (they walk the keys of a filter - // object, and it has none), and the driver ignored it: the READ answered the - // unfiltered set, and a `multi: true` UPDATE or DELETE rewrote or removed - // every row of the table (measured on `driver-memory` and `SqlDriver` - // without `SecurityPlugin`, and with it under a system context; with an - // RLS-scoped caller the RLS `$and` composition put the value where the - // driver's node gate refuses it). It also stepped past the unscoped-write - // guard (`dispatchUnscopedMultiWriteHooks`), which reads only an absent or - // `null` `where` as unscoped. - // - // The words are the wire door's (`metadata-protocol`'s normalizer answers - // `?filter=5` with "must be a filter object or condition array, received - // number … It was not applied"); the consequence is spelled per verb, - // because for a write it is not a result set. - if (where !== undefined && where !== null && !Array.isArray(where) && !isWhereFilterObject(where)) { - throw invalidFilterError( - `${operation}('${object}'): 'where' must be a filter object or condition array, ` + - `received ${describeNonFilterWhere(where)}. It was not applied, and an unapplied filter ` + - `would have ${UNAPPLIED_WHERE_CONSEQUENCE[operation] ?? UNAPPLIED_WHERE_CONSEQUENCE.find}.`, - ); - } - if (!Array.isArray(where)) { - // [#5869] Door 1 lands HERE, not below: the protocol face runs its own - // `isFilterAST` → `parseFilterAST` and hands the engine an already-lowered - // `FilterCondition` object, so a gate on the array branch alone would miss - // every query that arrived over the wire. The comparand check is the same - // one either way — it reads the lowered condition, which is what both doors - // produce. - assertListComparandShapes(object, operation, where); - // [#8296] The unmaterializable-FIELD door, on the same object form. It sits - // beside the shape gate because the two answer different questions about - // the same predicate — "can this comparand run" vs "is there a column to - // run it against" — and because this seam is the one place EVERY - // caller-supplied `where` passes through, whichever verb it arrived by. - assertFilterIsMaterializable(object, operation, schema, where); - // [#15661] The DECLARED-TYPE door for the text operators, third on this - // seam and third question about the same predicate: the shape gate asks - // "can this comparand run", the materializable gate asks "is there a column - // to run it against", and this asks "can that column's declared type ever - // hold a string". It runs BEFORE the temporal gate below deliberately — a - // `$startsWith` over a `date` field is refused by that gate today, with the - // same `INVALID_FILTER` / 400 envelope but a message about the COMPARAND, - // which sends the author to fix a value that could never have made the - // filter runnable. Same wire envelope, the ruling's message. - assertTextOperatorTargetsAreStringCapable(object, operation, schema, where); - // [#8690] The TEMPORAL-comparand door, fourth on the same seam (#15661's - // declared-type door was inserted above it) and fourth question about the - // same predicate: the shape gate asks "can this comparand run", the - // materializable gate asks "is there a column to run it against", the - // declared-type gate asks "can that column ever hold a string", and this - // asks "can that column's storage rule READ this value". - // It must run BEFORE `resolveWhereTokens` (which is downstream of every - // caller of this function) because the refusal has to precede the driver — - // hence the door steps around `{placeholder}` strings rather than judging - // them; the token resolver refuses the unknown ones a moment later, loudly. - assertTemporalComparandsInterpretable(object, operation, schema, where); - // [#20351] The NUMBER-comparand declared-type door, fifth on the same seam - // and fifth question about the same predicate: is this string a number the - // declared numeric column can be compared with. It refuses a non-numeric - // string (`INVALID_FILTER` / 400) and narrows a numeric one to its number, - // copy-on-write, by `@objectstack/spec/data`'s grammar and verdict. Before - // the token resolver, like the temporal door: a `{placeholder}` resolves to - // an id or a date, never a number, so it is refused here unresolved. - // [#20546] Its walk carries a second arm, asked first at every field: a - // plain object with no `$` key where a scalar column's value belongs - // (`{ amount: { a: 1 } }`) is refused `INVALID_FILTER` / 400. Memory - // answered it with no rows (every row under `$not`), SQL with its own 400. - // [#20745] …and beneath a relation column (the nested-relation form no - // driver serves), a structured-JSON column (a whole-value match the - // drivers share no meaning for) and an undeclared `id`, in words per kind. - // [#20802] …except the nested-relation form, which this position SERVES: - // beneath a relation column it is admitted against the related object's - // declarations (`schemaOf`, the registry) and kept as written — the engine - // lowers it once it can read (`ObjectQL.lowerRelationConditions`) — or - // refused in words of its own (a key the related object does not declare, - // a second level, a related object that is not registered). - // [#21333] …and the walk's BOOLEAN arm: against a declared boolean field - // `"true"` / `"false"`, `1` / `0` and `"1"` / `"0"` narrow to their - // boolean and any other string is refused `INVALID_FILTER` / 400 — the one - // place a bare query parameter's `"true"` becomes `true`. - const numeric = narrowNumberComparands(object, operation, schema, where, 'where', { schemaOf }); - // [#7872] The comparand-type door, on the OBJECT form. `parseFilterAST` - // runs the same walk on everything it lowers or passes through, but - // NEITHER door routes an object-form filter through it — Door 1 gates on - // `isFilterAST` first and Door 2 is this very branch — so without this - // call the dominant form would bypass the door entirely (the #7956 - // divergence matrix arrived through it). Shape gate first (#5869 keeps - // its pinned wording for the list-operator shapes), type door second; - // the walk is copy-on-write, so the common path allocates nothing and a - // narrowed bigint replaces the bag rather than editing the caller's. - const normalized = normalizeFilterComparandTypes(numeric, `${operation}('${object}')`); - if (normalized !== where) { - return { ...(bag as Record), where: normalized } as T; - } - return bag; - } - - const lowered: Record = { ...bag }; - - // (1) `[]` is "no filter", not a failed filter. - if (where.length === 0) { - delete lowered.where; - return lowered as T; - } - - // (3) Not a shape `parseFilterAST` can express. - // [#20121] Through `invalidFilterError`, so this refusal carries the - // `INVALID_FILTER` / 400 envelope the wire door answers the same array with - // (`malformedFilterArrayError`) — it used to be a bare `Error`, `code` and - // `status` both `undefined`, a 400-class author error that read as a 500. - if (!isFilterAST(where)) { - throw invalidFilterError( - `${operation}('${object}') received a 'where' array that is not a filter: ` + - `${JSON.stringify(where)}. A filter array is a comparison [field, operator, value], ` + - `a logical node ["and"|"or", ...conditions], or a list of those — it is INPUT-ONLY ` + - `sugar (spec 'FilterArray'), lowered to a FilterCondition here before any driver sees ` + - `it. This value cannot be lowered, and an unapplied filter would have returned ` + - `the UNFILTERED result set. Recognised operators: ` + - `${[...VALID_AST_OPERATORS].sort().join(', ')}. Infix joins ([condA, "or", condB]) are ` + - `NOT one of the shapes — write the prefix form ["or", condA, condB].`, - ); - } - - // (2) The declared path. - // [#9228] The context prefix is passed because `parseFilterAST` now runs the - // #5869 shape gate itself, one step before this branch could: a scalar `$nin` - // is refused DURING lowering. Handing it `${operation}('${object}')` is what - // keeps this branch's pinned `find('deal'): ` prefix on that refusal — and on - // the #7872 type door's, which `parseFilterAST` has always run and which used - // to answer this branch unprefixed. - const condition = parseFilterAST(where, `${operation}('${object}')`); - if (condition === undefined) { - // Unreachable by construction — `isFilterAST` accepted the shape, so - // `parseFilterAST` has a lowering for it. Loud rather than silent because - // the failure mode of the two spec functions disagreeing is a dropped - // predicate, i.e. every row (#3948). - throw new Error( - `${operation}('${object}'): filter array ${JSON.stringify(where)} passed isFilterAST() ` + - `but parseFilterAST() lowered it to nothing. Refusing rather than running the query ` + - `unfiltered.`, - ); - } - // [#5869] Door 2's half of the same check USED to be a second - // `assertListComparandShapes(object, operation, condition)` call here. - // [#9228] moved the rule into `parseFilterAST` itself, which this branch - // calls three lines up with the same context — so the call became provably - // unreachable (nothing between the two can introduce a list operator) and is - // deleted rather than kept as a gate that reads live and never fires. The - // OBJECT branch above keeps its call: that `where` never passes - // `parseFilterAST` at all, which is the whole reason both branches exist. - // [ADR-0139 D1] The object branch's `ReadableIds` rule, on the LOWERED - // condition: `parseFilterAST` admits no leaf as a `$in` comparand (it does - // not opt in, so that spelling is its 400), but a leaf it lowers by - // reference into any other position is refused here by name, as the object - // branch refuses it — first, before the doors below read it as a value. - collectComposableReadableIds(object, operation, condition, 'where'); - // [#8296] Same door as the object branch above, on the LOWERED condition — - // the array sugar (`[['is_open','=',true]]`) names fields too, and a gate on - // one branch would answer one mistake two ways depending on the spelling. - assertFilterIsMaterializable(object, operation, schema, condition); - // [#15661] Same door as the object branch, on the LOWERED condition — the - // array sugar (`[['amount','contains','5']]`) names non-text fields too, and - // a gate on one branch would answer one mistake two ways depending on the - // spelling. - assertTextOperatorTargetsAreStringCapable(object, operation, schema, condition); - // [#8690] Same door as the object branch, on the LOWERED condition — the - // array sugar (`[['at','>=','last_30_days']]`) names temporal fields too, and - // a gate on one branch would answer one mistake two ways depending on the - // spelling. - assertTemporalComparandsInterpretable(object, operation, schema, condition); - // [#20351] Same door as the object branch, on the LOWERED condition — the - // array sugar (`[['amount','>','abc']]`) names numeric fields too. [#20546] - // …and lowers `['amount', '=', { a: 1 }]` to the no-operator object its - // second arm refuses. [#21333] …and `['active', '=', 'true']` to the - // comparand its boolean arm narrows. - lowered.where = narrowNumberComparands(object, operation, schema, condition, 'where', { schemaOf }); - return lowered as T; -} - -/** - * [#20157] Stage 2 of `where` admission: expand the filter placeholders - * (`{current_user_id}`, date macros) against the execution context. - * - * The one spelling of that step. Execution reaches it through - * `ObjectQL.resolveWhereTokens` (the read verbs) and `ObjectQL.withResolvedWhere` - * (`update` / `delete`), and the judge ({@link judgeWhereAdmission}) calls it - * directly. So "what execution resolves" and "what the judge resolves" cannot - * drift. An unknown placeholder throws `FILTER_TOKEN_UNKNOWN`; a known one the - * context carries no value for throws `FILTER_TOKEN_UNRESOLVED`. Neither ever - * resolves to `null` (see `@objectstack/core`'s `filter-tokens.ts`). - * - * [#20844] …and judges what resolved: `judgeResolved` is the position's - * {@link ResolvedTokenJudge}, handed the condition before and after, and a - * date macro that resolved outside its column's years is refused there - * (`INVALID_FILTER` / 400). Required, so no position resolves without it; the - * resolver is field-agnostic, so the column's kind comes from the position. - * - * Returns the input by reference when it holds no placeholder, and then - * judges nothing. - */ -function resolveWhereFilterTokens( - where: W, - context: Parameters[0], - judgeResolved: ResolvedTokenJudge, -): W { - const resolved = resolveFilterTokens(where, filterTokenContextFrom(context)); - if (resolved !== where) judgeResolved(where, resolved); - return resolved; -} - -/** - * [#20844] The `where` position's {@link ResolvedTokenJudge}: each key's kind - * is its declared field's in `schema`. `path` roots the refusal, as the door's - * does — `aggregations[i].filter` for a per-aggregation filter. - */ -function whereResolvedTokenJudge( - object: string, - operation: string, - schema: unknown, - path = 'where', -): ResolvedTokenJudge { - return (written, resolved) => - assertResolvedTemporalTokensInRange(object, operation, schema, written, resolved, path); -} - -/** - * [ADR-0053 D-D1, amended 2026-09-30 — #5930] Stage 2 of every filter - * position's admission, whole: resolve the placeholders - * ({@link resolveWhereFilterTokens}), THEN run the shared lowering - * (`lowerFilterCondition`, `@objectstack/spec/data`) — the `$between` split, - * the whole-day upper bound and the NULL-polarity guards. - * - * The order is the amendment's item 3: the lowering reads the comparand the - * comparison will run with, so a date macro (`{today}`, `{current_month_end}`) - * has already become the bare day the whole-day rule widens. Run beside the - * doors instead, it would meet `{today}` unresolved and leave it bare. - * - * ONE function for every position, so no verb can resolve without lowering: - * `where` on the five read / write verbs (`ObjectQL.resolveWhereTokens`, - * `ObjectQL.withResolvedWhere`), `aggregations[i].filter` and `having` on - * `aggregate`, and the judge ({@link judgeWhereAdmission}), which runs the - * same stage so its verdict is execution's. The lowering never refuses, so it - * adds no verdict of its own. - * - * `lowering` is the position's declared-type reader: this seam reads the - * object's declarations, so the whole-day rule rewrites a declared - * `datetime` column only ({@link declaredDatetimeLowering}) — the scope - * `SqlDriver` holds, so a `date`, `time` or non-temporal column reaches every - * driver byte-identical to before (the amendment's item 7). An object with no - * field map has no declarations to read, and there the rule applies - * type-blind (item 7's other half). - * - * [#20844] `judgeResolved` is the position's other declared-type reader: the - * year range of what resolved ({@link resolveWhereFilterTokens}). - * - * Returns the input by reference when nothing resolved and nothing lowered. - */ -export function resolveThenLowerWhere( - where: W, - context: Parameters[0], - lowering: FilterLoweringOptions, - judgeResolved: ResolvedTokenJudge, -): W { - return lowerFilterCondition(resolveWhereFilterTokens(where, context, judgeResolved), lowering); -} - -/** - * [ADR-0053 D-D1 item 7 — #5930] The typed reading of one object's filter - * positions: a column is `datetime` exactly when the object's declared field - * map says `type: 'datetime'` — the same test `SqlDriver` indexes its - * `datetimeFields` by, so the columns this seam widens are a subset of the - * columns every face widens today. - * - * [#20822] No field map (an object the registry does not hold, a - * registry-less host) is a seam that cannot read the declared type, and item 7 - * says what such a seam does: "applies the rewrite type-blind", as the - * type-blind emitters do (sound on `Field.date` text, where `< next-day` - * orders exactly as `<= day`). So the reader is omitted and the whole-day - * rules apply to every column. This branch used to read "no field map" as "no - * datetime column" and leave the rule to the faces; item 5 retires those - * copies, and with them gone that reading would have dropped the whole-day - * bound on every such read. - */ -export function declaredDatetimeLowering(schema: unknown): FilterLoweringOptions { - const fields = (schema as { fields?: unknown } | undefined)?.fields; - if (fields === null || typeof fields !== 'object') return {}; - return { - isDatetimeColumn: (column) => - fields !== null - && typeof fields === 'object' - && Object.prototype.hasOwnProperty.call(fields, column) - && ((fields as Record)[column]?.type === 'datetime'), - }; -} - -/** - * [#20157] Is this thrown value a door's DIAGNOSTIC (the ADR-0112 envelope: - * a string `code` and a numeric `status`), or something else? - * - * Every `where` door raises the envelope: `invalidFilterError`, the - * materializable door's `INVALID_FIELD`, spec's shape and type faces, and - * core's `FILTER_TOKEN_*` errors. A throw without it is not a verdict about the - * filter. It is a fault (the "unreachable by construction" branch of - * {@link lowerWhereFilterArray}, a bug), and the judge re-throws it rather than - * dressing it up as one. - */ -function admissionRefusalOf( - thrown: unknown, -): Extract | null { - if (!(thrown instanceof Error)) return null; - const { code, status } = thrown as Error & { code?: unknown; status?: unknown }; - if (typeof code !== 'string' || typeof status !== 'number') return null; - return { ok: false, code, status, message: thrown.message }; -} - -/** - * [#20157] Judge a `where` against an object WITHOUT executing it: the engine - * side of `IObjectQLEngine.judgeFilter` (#19995 ruling C). - * - * ## The pipeline it runs is the one every verb runs - * - * Every verb that takes a `where` (`find`, `findOne`, `count`, `aggregate`, - * `update`, `delete`) admits it in the same two stages, in the same order: - * - * 1. {@link lowerWhereFilterArray}: the shape gate, then (object form) - * `assertListComparandShapes` → `assertFilterIsMaterializable` (the dotted, - * then the virtual-field verdict) → `assertTextOperatorTargetsAreStringCapable` - * → `assertTemporalComparandsInterpretable` → `narrowNumberComparands` - * (#20351) → `normalizeFilterComparandTypes`, or (array form) `isFilterAST` - * → `parseFilterAST` → the same four field-map doors on the lowered - * condition. - * 2. {@link resolveWhereFilterTokens}: the placeholder resolver, and [#20844] - * the year range of each date macro it resolved. - * - * [#20802] Execution then lowers each nested-relation condition by READING - * the related object (`ObjectQL.lowerRelationConditions`); that read admits the - * condition through the related object's own two stages. The judge runs those - * two stages on each condition too — as a `find` on the related object — and - * stops there: which ids the read finds, the cap and the caller's permissions - * on the related object need data and a caller, and are execution's alone. - * - * What differs by verb sits BETWEEN or AROUND those stages and judges - * something other than `where`: option-key folding and refusal, the driver - * lookup (`getDriver`, before stage 1 on the writes and between the stages on - * the reads), `orderBy` / projection doors on `find`, and the per-aggregation - * and `having` doors on `aggregate`. None of those is part of this judgement. - * The judge calls the two stage functions themselves, so the doors have one - * implementation and one order. It never calls a second copy of a walk. - * - * ## What it reads - * - * The `where` value, the object's declared field map (`schema`, the - * registry's), and the execution context (for stage 2). No driver, no data, - * no hook, no middleware. Execution's own stages read nothing more, measured - * door by door on #20157. - * - * A thrown door diagnostic becomes the returned refusal, with the same `code`, - * `status` and `message` execution raises. Anything else is re-thrown - * ({@link admissionRefusalOf}). - */ -function judgeWhereAdmission( - object: string, - operation: string, - where: unknown, - schema: unknown, - context: Parameters[0], - schemaOf?: (name: string) => unknown, -): EngineFilterJudgement { - try { - const admitted = lowerWhereFilterArray(object, operation, { where }, schema, schemaOf); - const resolved = resolveWhereFilterTokens( - admitted.where, context, whereResolvedTokenJudge(object, operation, schema), - ); - // [#20802] Each nested-relation condition the door admitted: execution - // reads the related object with it (`ObjectQL.lowerRelationConditions`), - // and that read admits it through the related object's own doors and - // resolver — so the judge asks the same of it here, as a `find` on the - // related object. What the read then finds (the ids, the cap, the caller's - // permissions) needs data and a caller, and is execution's alone. The rest - // of the filter is lowered with each condition standing for the empty id - // set, which the shared lowering treats as it treats any `$in` / `$or`. - const sites = relationSitesOf(object, operation, resolved, schema, schemaOf); - for (const site of sites) { - const inner = judgeWhereAdmission(site.target, 'find', site.condition, schemaOf?.(site.target), context, schemaOf); - if (!inner.ok) return inner; - } - const related = sites.length === 0 - ? resolved - : mapRelationConditions(object, operation, schema, resolved, { - schemaOf: schemaOf ?? (() => undefined), - replace: (site) => lowerRelationSite(site, []), - }); - lowerFilterCondition(related, declaredDatetimeLowering(schema)); - return { ok: true }; - } catch (thrown) { - const refusal = admissionRefusalOf(thrown); - if (refusal) return refusal; - throw thrown; - } -} - -/** - * [#20802] The nested-relation conditions a `where` holds, in walk order: the - * collecting pass of {@link mapRelationConditions} (each admitted site is - * recorded and kept as written, so nothing is rewritten). Empty for a filter - * with none, and for a registry-less host (no field map, no relation column). - */ -export function relationSitesOf( - object: string, - operation: string, - where: unknown, - schema: unknown, - schemaOf: ((name: string) => unknown) | undefined, -): RelationFilterSite[] { - const sites: RelationFilterSite[] = []; - if (where == null) return sites; - mapRelationConditions(object, operation, schema, where, { - schemaOf: schemaOf ?? (() => undefined), - replace: (site) => { - sites.push(site); - return { kind: 'value', value: site.condition }; - }, - }); - return sites; -} - -/** - * [#20082] One operation's effective-permission resolution — see - * `ObjectQL.permissionResolution`. `get()` asks the registered resolver at most - * once for the operation and hands every caller the same answer, or the same - * rejection. - */ -export interface PermissionResolution { - get(): Promise; -} - -/** - * Is this `defaultValue` an Expression envelope (`{ dialect, source }`) — - * evaluated at insert time — rather than a literal or a runtime token? The one - * test {@link ObjectQL.applyFieldDefaults} evaluates by and - * {@link ObjectQL.resolveDefaultPermissions} plans by, so the two can never - * disagree about which defaults are expressions. - */ -export function isExpressionDefault(dv: unknown): dv is Expression { - return typeof dv === 'object' && dv !== null - && Boolean((dv as { dialect?: unknown }).dialect) - && typeof (dv as { source?: unknown }).source === 'string'; -} - -export interface FormulaPlanEntry { - name: string; - expression: Expression; - /** - * [#10280] The field's declared `scale`, when it is one this rounding can - * honour — see {@link formulaRoundingScale}. `undefined` means "return the - * evaluator's value verbatim", which is what every formula did before this. - */ - scale?: number; - /** - * [#24197] The reference fields this formula reads ONE hop through, and the - * related fields it names on each (`record.book.title` → `book → {title}`). - * Absent when the formula traverses nothing — every formula before #24197, - * and still the common case — which is what lets such a read skip the - * related read entirely. - */ - hops?: ReadonlyMap>; - /** - * [#24197] Why this formula's traversal cannot be served (a second hop; a - * reference read both through and as a value), worded for the author. Such a - * formula is not evaluated: it reads `null` and the fault sink carries this - * sentence — the lint refuses the same shape with the same words. - */ - hopConflict?: string; -} - -/** - * `toFixed`'s own domain limit: it throws `RangeError` above 100 fraction - * digits. `FieldSchema.scale` is `z.number().int().min(0)` with no upper - * bound, so an author CAN declare more — and a display-precision declaration - * must never be the reason a read fails. - */ -const MAX_FORMULA_SCALE = 100; - -/** - * [#10280] The `scale` a formula plan entry carries, or `undefined` when the - * declaration is not one this rounding can honour. - * - * Well-formedness is judged with the SAME test the write path uses - * (`validateFieldValue`'s `scale` branch: `Number.isInteger(def.scale) && - * def.scale >= 0`), so one malformed declaration cannot mean two things one - * layer apart. A malformed `scale` stays UNENFORCED rather than being - * interpreted — inventing floor/round semantics for `scale: 2.5` in a consumer - * is the guessing PD #12 forbids, and the producer already refuses it - * (`FieldSchema.scale` is `z.number().int().min(0)`). - * - * The upper bound is `toFixed`'s, not a policy: past {@link MAX_FORMULA_SCALE} - * the call throws, and asking for more decimal places than an IEEE-754 double - * carries is a no-op request either way — so the rounding is skipped and the - * value is returned whole. - */ -function formulaRoundingScale(def: { scale?: unknown } | undefined): number | undefined { - const scale = def?.scale; - if (typeof scale !== 'number' || !Number.isInteger(scale) || scale < 0) return undefined; - return scale <= MAX_FORMULA_SCALE ? scale : undefined; -} - -/** - * [#18408] Is this field's value MULTI-VALUED — the one question, asked of the - * one predicate `@objectstack/spec` publishes, and the single local seam every - * site in this file goes through. - * - * Maintainer ruling 2026-09-13 (decision batch #128 item 5, option 1′): there - * is ONE definition of "is this field multi-valued", `isMultiValueField`, and - * storage follows it. The sites below used to read `field.multiple` raw, which - * answers `true` on types the spec predicate calls single-valued (`text`, - * `master_detail`, `tree` …) and `false` on the inherently-multi option types - * (`multiselect` / `checkboxes` / `tags`) that carry no flag at all. Both - * directions put this engine on a different list from the storage it writes - * through: `driver-sql` (#17469) and `os generate migration` (#18199) both ask - * the predicate now. - * - * Takes the field rather than a resolved type because — unlike driver-sql and - * the CLI generator — no caller in this file applies a type default of its own; - * a non-string `type` is answered `false` here, the same verdict those two - * reach through their `field.type || 'string'` resolution. - * - * ⛔ The predicate is called, never re-spelled. `MULTI_CAPABLE_TYPES` / - * `MULTI_OPTION_TYPES` membership written out at a site would be a second - * answer to a question the ruling gave exactly one. - */ -export function declaredMultiValued(field: { type?: unknown; multiple?: unknown } | null | undefined): boolean { - return isMultiValueField({ - type: typeof field?.type === 'string' ? field.type : '', - multiple: field?.multiple === true, - }); -} - -/** - * [#22957] The reference types whose `deleteBehavior` states DELETE SEMANTICS: - * the relational family the referential delete walk follows. - * - * `tree` is a member because the spec says so. `FieldSchema`'s `.overwrite` - * (`packages/spec/src/data/field.zod.ts`) materializes `deleteBehavior: - * 'set_null'` on a `tree` beside `lookup`, because the type "is in the - * relational family, where the key states delete semantics". Its reference is - * self-referential by contract (`refuseForeignTreeReference`, `object.zod.ts`), - * so its `cascade` is a subtree delete. The walk used to admit only - * `master_detail` and `lookup`, so every behaviour a `tree` declared, the - * materialized default included, was inert: a parent delete left its children - * holding the deleted id. - * - * `user` is not a member of this set: see {@link USER_REFERENCE_TYPE}, which - * takes part only when it declares a delete outcome of its own. - */ -const DELETE_SEMANTICS_REFERENCE_TYPES: ReadonlySet = new Set(['master_detail', 'lookup', 'tree']); - -/** - * [#29253] A `user` field takes part in the referential delete walk when, and - * only when, it DECLARES a `deleteBehavior` (one of the enum's three values). - * - * It stores like a `lookup` (the target is fixed to `sys_user` by the type, - * {@link referenceTargetOf}), but the spec's parse materializes no - * `deleteBehavior` on it: a `user` field that authors nothing declares no - * delete outcome, so deleting the user leaves it as it was, exactly as before. - * A `user` field that authors one is a declaration the spec accepts and - * round-trips, and the walk used to read it never, so an authored `restrict` - * refused nothing and an authored `cascade` deleted nothing. It now reads like - * a `lookup`'s authored value: `cascade` deletes the rows holding the deleted - * user, `set_null` clears them (on a multi-valued field, removes the member), - * and `restrict` refuses the delete with the walk's own `DELETE_RESTRICTED`; - * a `set_null` on a REQUIRED `user` field escalates to that refusal as a - * required `lookup`'s does. - * - * Every door that deletes a user reaches this walk: better-auth deletes the - * user row (admin remove-user, self delete-user) through the plugin-auth - * adapter's `engine.delete` by id, and the generic data doors refuse a - * caller-context `sys_user` delete before it (ADR-0092 D2). - * - * ⛔ No default is invented here. "Absent" stays outside the walk, because the - * spec's parse decides what a bare `user` field declares, not this file. - */ -const USER_REFERENCE_TYPE = 'user'; -const DECLARED_DELETE_BEHAVIORS: ReadonlySet = new Set(['set_null', 'cascade', 'restrict']); - -/** - * [#22957] Does `fdef` belong to {@link DELETE_SEMANTICS_REFERENCE_TYPES}, or - * [#29253] is it a `user` field that declares a `deleteBehavior` - * ({@link USER_REFERENCE_TYPE})? This - * is the ONE family test every reader of the referential delete walk asks: - * {@link ObjectQL.cascadeRelationBehavior}, through it the collection walk and - * the dependents walk, and {@link ObjectQL.planCascadeAtomicity}. So the walk - * that runs and the verdict on whether it runs as one unit of work cannot - * disagree about which relations take part. ⛔ Never a type literal at a site. - */ -function statesDeleteSemantics(fdef: { type?: unknown; deleteBehavior?: unknown } | null | undefined): boolean { - if (typeof fdef?.type !== 'string') return false; - if (fdef.type === USER_REFERENCE_TYPE) { - return typeof fdef.deleteBehavior === 'string' && DECLARED_DELETE_BEHAVIORS.has(fdef.deleteBehavior); - } - return DELETE_SEMANTICS_REFERENCE_TYPES.has(fdef.type); -} - -/** - * [#29253] The object a delete-walk relation points at. The `reference` - * carrier is read through the one arbiter ({@link referenceCarrierOf}), so an - * unreadable carrier still throws and an absent one still answers `undefined`. - * A `user` field's target is fixed by its type, so it answers `sys_user` - * ({@link referenceTargetOf}) whether or not the carrier spells it: a raw - * `{ type: 'user' }` registration names no `reference` and still references a - * user. - */ -function deleteWalkTargetOf(fdef: { type?: unknown } | null | undefined, reader: string): string | undefined { - const carrier = referenceCarrierOf(fdef, reader); - return fdef?.type === USER_REFERENCE_TYPE ? referenceTargetOf(fdef) : carrier; -} - -/** - * Plan the formula pass of a read: which formula fields to evaluate, and — - * when the caller named a projection — the projection the DRIVER is asked for. - * - * A projection naming a formula field is widened to every stored column (plus - * `id`), because CEL's `record.` reads whatever the formula needs off - * the full row. That widening is a means of EVALUATING the formula and never - * an answer: [#22300] `widened` lists the columns it added beyond what the - * caller named, and {@link withoutFormulaWidening} cuts them back off the rows - * once the read is done, so the caller gets the projection it declared plus - * the formula's value. `widened` is absent whenever nothing was widened. - */ -/** - * ONE plan entry for a field's declared `expression`: the envelope, the - * rounding scale, and the one hop it reads through a reference field. Shared - * by the `formula` read plan ({@link planFormulaProjection}) and the stored - * computed-field write plan ({@link planStoredExpressions}), so a value - * computed on read and a value computed on write are one semantic. - */ -function formulaPlanEntry(name: string, def: any, fields: any): FormulaPlanEntry { - // Normalize string-shorthand → Expression envelope (M9 transition). - const expr: Expression = typeof def.expression === 'string' - ? { dialect: 'cel', source: def.expression } - : def.expression; - const entry: FormulaPlanEntry = { name, expression: expr, scale: formulaRoundingScale(def) }; - // [#24197] The one hop this formula reads through a reference field, if - // any — judged by the validation-rule path's own collector, so the two - // seams that serve a hop read the same hops off the same expression. - const conflicts = referenceTraversalConflicts(expr, fields, 'formula'); - if (conflicts.length > 0) { - entry.hopConflict = conflicts.map((c) => c.message).join(' '); - } else { - const hops = collectReferenceTraversals([expr], fields); - if (hops.size > 0) entry.hops = hops; - } - // Pre-compile to surface syntax errors at planning stage rather than - // per-row eval. Own-column dependency discovery is not used — CEL reads - // the row via `record.`; only the hops above are planned. - ExpressionEngine.compile(expr); - return entry; -} - -function planFormulaProjection( - schema: any, - requestedFields: string[] | undefined -): { plan: FormulaPlanEntry[]; projected?: string[]; widened?: string[] } { - if (!schema?.fields) return { plan: [] }; - const allFieldNames = Object.keys(schema.fields); - // When no explicit projection, evaluate every formula field on the schema — - // matches REST default of "return everything". Explicit projection still - // honours the caller's selection. - const targets = (Array.isArray(requestedFields) && requestedFields.length > 0) - ? requestedFields - : allFieldNames; - const plan: FormulaPlanEntry[] = []; - const projected = new Set(); - for (const f of targets) { - const def = (schema.fields as any)[f]; - if (def?.type === 'formula' && def.expression) { - plan.push(formulaPlanEntry(f, def, schema.fields)); - } else if (Array.isArray(requestedFields) && requestedFields.length > 0) { - projected.add(f); - } - } - if (plan.length === 0) return { plan: [] }; - // For formulas: project all schema fields so CEL `record.` lookups - // see complete data. Static dependency analysis on AST is M9.7 work. - if (Array.isArray(requestedFields) && requestedFields.length > 0) { - if (!projected.has('id')) projected.add('id'); - for (const fname of allFieldNames) { - // Skip formula fields themselves — they are virtual and not - // projectable by the underlying driver. Without this guard the - // SQL driver emits `SELECT response_rate ...` which fails as - // "no such column" and the driver returns [] (silently). - const fdef = (schema.fields as any)[fname]; - if (fdef?.type === 'formula') continue; - projected.add(fname); - } - // [#22300] What the widening added — `id` included when the caller did not - // name it — is what the read cuts back off once the formulas are computed. - const named = new Set(requestedFields); - const widened = Array.from(projected).filter((f) => !named.has(f)); - return { plan, projected: Array.from(projected), ...(widened.length > 0 ? { widened } : {}) }; - } - // Implicit/full projection — leave projected undefined so the driver - // returns its default columns (typically *). - return { plan }; -} - -/** - * [#22300] Cut the columns {@link planFormulaProjection} widened a projection - * by back off what a read RETURNS — the formula saw the full row, the caller - * does not. Without it, a projection naming a formula field answered with every - * stored column (tenant, owner, owning unit, audit actors and timestamps, every - * unnamed field): a flow's `get_record`, whose `config.fields` is declared as - * "only these fields are read", passed all of them on to whatever its later - * nodes sent out. - * - * ## The answer it restores - * - * The same projection without the formula, plus the formula's value: measured - * on driver-sql (a projection is exactly the named columns — no `id` unless it - * is named) and pinned against it in `engine-formula-projection-trim.test.ts`. - * `id` is cut like any other widened column when the caller did not name it. - * [#26672] The `id` {@link withFileOwnerKey} adds for a file field's owner - * verdict is cut the same way. - * - * ## Where it runs, and why there - * - * On the result `find` / `findOne` hand back, AFTER the middleware chain — - * the last internal consumer. Everything before it keeps reading the widened - * row exactly as before: the formula pass, `expand`, file-reference - * resolution, the `afterFind` hooks, the secret mask and the `__search` - * strip, and the middlewares' post-phase (field-level security's result mask, - * the audit redactions that judge a row by columns the caller did not name). - * Cutting earlier would starve those of a column they read today. - * - * It REMOVES the widened columns rather than keeping a list: a key an - * `afterFind` hook derives, or an expanded relation the caller named, is not - * the widening's and survives. A widened column a hook re-assigns is cut — the - * caller never named it. - * - * Never mutates a row: a row carrying a widened column is replaced by a copy - * without it, in the row's own key order; a row carrying none is returned as - * is. - */ -function withoutFormulaWidening(row: T, widened: ReadonlySet | undefined): T { - if (!widened || !row || typeof row !== 'object' || Array.isArray(row)) return row; - const source = row as unknown as Record; - if (!Object.keys(source).some((key) => widened.has(key))) return row; - const cut: Record = {}; - for (const key of Object.keys(source)) { - if (!widened.has(key)) cut[key] = source[key]; - } - return cut as unknown as T; -} - -/** {@link withoutFormulaWidening} over a `find` result; anything but an array passes through. */ -function rowsWithoutFormulaWidening(rows: T, widened: ReadonlySet | undefined): T { - if (!widened || !Array.isArray(rows)) return rows; - return rows.map((row) => withoutFormulaWidening(row, widened)) as unknown as T; -} - -/** - * [#26672] A projection that names a file field reads the record's `id` too, - * and the key is cut back off the answer like the formula widening is. - * - * File-reference hydration judges a reader refused `sys_file` read by the - * record that OWNS each file (#22637, `readParentDerivedFiles`), matched on - * that record's `id`. A projection that left `id` out handed it rows with no - * key, so a file the reader may read answered `{ id, metadataRefused: true }` - * to `select=` and the full value to `select=id,`. The - * verdict must not depend on whether the caller listed `id`, so the driver is - * asked for it here and {@link withoutFormulaWidening} trims it at the - * `return`: the caller still gets exactly the columns it named, and a reader - * the verdict refuses is still refused. - * - * Touches only an explicit projection that names a file field and not `id`; - * a projection the formula planner widened already carries `id`. - */ -function withFileOwnerKey( - schema: any, - requestedFields: string[] | undefined, - planned: { projected?: string[]; widened?: string[] }, -): { projected?: string[]; widened?: string[] } { - if (!schema?.fields || !Array.isArray(requestedFields) || requestedFields.length === 0) return planned; - const projected = planned.projected ?? requestedFields; - if (projected.includes('id')) return planned; - if (!requestedFields.some((f) => FILE_REFERENCE_TYPES.has((schema.fields as any)[f]?.type))) return planned; - return { projected: [...projected, 'id'], widened: [...(planned.widened ?? []), 'id'] }; -} - -/** - * [#7095] ORDER BY a field whose value is computed on read — refused HERE, on - * the engine's own public boundary, and no longer only at the REST ingress. - * - * #6994 closed this at `assertSortFieldsExist` (`400 INVALID_SORT`), which - * covers everything reaching `findData`: the REST list route, - * `POST /data/:object/query`, the export route and the RPC dispatcher. It could - * not cover a caller that reaches {@link ObjectQL.find} / {@link - * ObjectQL.findOne} DIRECTLY, and that half was not hypothetical — measured on - * this file's base (real `ObjectQL`, a driver that really sorts): - * - * ``` - * engine.find(o, { orderBy: [{ field: , order: 'asc' }] }) -> C A E B D (insertion order) - * engine.find(o, { orderBy: [{ field: , order: 'desc' }] }) -> C A E B D (byte-identical) - * ``` - * - * `asc` and `desc` coming back identical is what makes it a DROPPED sort rather - * than a coincidence, and the rows carrying the very values they were asked to - * be ordered by is what makes it invisible. `planFormulaProjection` above drops - * the virtual NAME from the projection (it must — the driver has no column and - * `SELECT sort_key` fails as "no such column"); nothing did the equivalent for - * the sort, so the ORDER BY reached the driver, found nothing, and the #3821 - * unknown-column backstop returned the rows unordered under a success. - * - * WHY A REFUSAL AND NOT AN OBSERVABLE DROP (the card offered both): ruled - * 2026-08-10 on #7095 — an ORDER BY the engine cannot apply is a 4xx with - * guidance prose, never a silent drop, the same direction as the analytics - * dataset refusal envelope and #6924's sort-hint prescription. The engine's - * documented internal-caller tolerance (`assertProjectionFieldsExist`'s - * docblock) was to survive only behind a pinned internal path and only if a - * measured internal call site relied on it; the #7095 sweep found none, so - * there is no internal path and no option to opt back into the drop. That sweep - * is recorded in the changeset — if you are adding a caller that WANTS the old - * behaviour, the answer is a stored field, not a flag. - * - * ⚠️ Post-hoc sorting after {@link applyFormulaPlan} evaluates the formulas is - * a TRAP, written down here because it looks like the generous fix: `driver.find` - * has already applied `limit`/`offset`, so re-sorting reorders an ARBITRARY - * PAGE. It would pass every small-result-set test and be wrong the moment - * pagination is involved. - * - * SCOPE — deliberately the third verdict only, on the SORT axis. `unknown` and - * `dotted` SORT names are NOT judged here: the ingress gate's precedence is - * `unknown` > `dotted` > unmaterializable (#4226 / #4256 / #6994), and widening - * this door to those two is a separate posture change on two more axes, not a - * free extension of this one — so a dotted SORT path keeps reaching the driver - * exactly as before, including one whose head is a formula field. On the - * PROJECTION axis the engine still tolerates an unknown PLAIN name by design - * (the `SELECT *` tolerance a few lines below, kept by the #7589 ruling), but - * a dotted PROJECTION entry is refused since #7589 — - * {@link assertProjectionHasNoDottedPaths}, directly below. - * - * A registry-less host (`schema` undefined) returns early, exactly as the - * ingress gate returns early when `resolveQueryFields` cannot answer: a door - * that cannot see the field map must not invent a verdict about it. - */ -function assertOrderByIsMaterializable( - object: string, - operation: 'find' | 'findOne', - schema: any, - orderBy: unknown, -): void { - if (!Array.isArray(orderBy) || orderBy.length === 0) return; - if (!schema?.fields) return; - const names = orderBy.map((node) => - typeof node === 'string' ? node : String((node as any)?.field ?? '')); - const unmaterialized = names.filter((f) => - f !== '' && !f.includes('.') && (schema.fields as any)[f]?.type === 'formula'); - if (unmaterialized.length === 0) return; - const first = unmaterialized[0]; - const type = String((schema.fields as any)[first]?.type); - const err: any = new Error( - `ObjectQL.${operation}('${object}') sorts by '${first}', a ${type} field on '${object}' — ` - + `a ${type} value is computed on read, so no driver materialises a column to order by` - + (unmaterialized.length > 1 ? ` (also: ${unmaterialized.slice(1).join(', ')})` : '') - + '. It was not applied, and an unapplied sort returns the rows in an arbitrary order — ' - + "which 'limit'/'offset' then slices into an arbitrary page." - // Deliberately the SAME remedy, in the same words, as the ingress door's - // formula and dotted refusals (#6994, #6924). One vocabulary across the - // doors: a caller refused at the REST boundary and a caller refused here - // must not be sent two different ways. - // - // [commit e5eeb499c] With the SEARCH axis the agreement is in SUBSTANCE, not in - // words, and saying otherwise here was simply false — measured from the - // running doors, #6673's correction emits "Mirror the computed value onto - // a stored text field on '' and search that instead." All three - // axes prescribe the same thing (put the value on a stored column of the - // queried object and query THAT instead); SEARCH narrows the target to a - // TEXT column, which is right for an axis that scans text-like columns and - // is a shipped, user-visible message on an axis that already landed. - // - // `query-expression-conformance.test.ts` pins that agreement as ONE thing - // rather than each wording separately, because separate wordings is - // exactly how #4256 and #6673 drifted apart in the first place. It holds - // three layers: the four SORT/FILTER doors' emitted remedies compared - // against EACH OTHER, every axis' exact sentence (SEARCH included — the - // door this claim used to skip), and the substance none of them may drop. - // So a rewording of any axis goes red. - + ` Denormalise the value onto '${object}' (a stored field, written when the` - + ' source changes) and sort by that. A formula field is virtual: with no' - + ' column behind it the ORDER BY reaches the driver, finds nothing, and is' - + ' dropped — the arbitrary order this refusal replaces.', - ); - // `INVALID_SORT`, not a new code, and 400 rather than 500: the ingress door - // reasoned that one condition — "this sort was not applied as written" — - // keeps ONE wire code however the caller reached it, and a caller reaching - // the engine directly has not stopped being that condition. A host that - // surfaces engine errors over HTTP therefore answers the same envelope on - // both doors instead of turning the direct path into an unhandled 500. - err.status = 400; - // …and `httpStatus`, the SAME number under ADR-0112 D5's spelling. `status` is - // what every HTTP door in this repo reads (`resolveThrownHttpError`), so it - // stays; `httpStatus` is what a consumer holding the THROWN error reads — - // the CLI's `--json` error envelope (`errorCodeFields`) is the measured one, - // and it saw `code` with no status at all until both spellings were stamped. - // Two keys, one value, written together at every producer in this package. - err.httpStatus = 400; - err.code = 'INVALID_SORT'; - err.field = first; - err.fields = unmaterialized; - err.object = object; - throw err; -} - -/** - * [#7589] A DOTTED projection entry — refused on the engine's own public - * boundary, the second half of #7532's ingress refusal. - * - * #7532 (PR #7588) closed this at `assertProjectionFieldsExist` - * (`400 INVALID_FIELD`), which covers everything reaching `findData`: the REST - * list route, `POST /data/:object/query`, the export route and the RPC - * dispatcher. It could not cover a caller that reaches {@link ObjectQL.find} / - * {@link ObjectQL.findOne} DIRECTLY — and that half was measured, not assumed - * (#7589): a flow-authored `get_record` node's `fields: ['name','account.name']` - * parses (`GetRecordConfigSchema` restricts nothing), travels verbatim into - * `data.find(...)`, cleared the head-only filter below on its head segment - * (`account` IS a field), and reached the driver as a projection column — where - * SQL renders `"account"."name"` against a table that was never joined, the DB - * answers `no such column`, and the #3821 recovery ladder retries `select('*')`. - * The caller asked to narrow and silently received EVERY field, byte-identical - * to no projection at all. Every hook and internal caller reaches this the - * same way (a saved report's `query.fields` did too, until the saved-report - * stack was retired in #20102). - * - * WHY A REFUSAL: ruled 2026-08-12 on #7589 (adopting the drivers seat's - * Option B) — a dotted entry the engine cannot resolve is refused loudly at - * this one site, covering every caller that reaches the engine. The head-only - * check this replaces was justified by a comment claiming the engine resolves - * relationship paths "via populate"; #7601 measured that NO populate step - * exists — the comment was the last place in the repo asserting dotted-path - * resolution does (after PR #7617) — so what is removed here is not a working - * feature but a path to widening, kept alive by a false premise. Both the typo - * (`titel.name`) and the genuine traversal intent (`account.name`) eat this - * refusal: nothing resolves either, they are not separable at this door, and - * the alternative is the over-return above (#5918's precedent, same as the - * ingress ruling). - * - * SCOPE — the dotted leg ONLY. The unknown-PLAIN-column tolerance a few lines - * below each call site is explicitly KEPT (same #7589 ruling): an unknown - * plain column is simply absent from each row, the "no records exist" failure - * that tolerance prevents is real, and it backstops registry-less hosts. A - * dotted path differs in kind — it is a projection no driver can structurally - * apply, and answering it with every column points away from both FLS and data - * minimisation. The two facts get two verdicts. - * - * A registry-less host (`schema.fields` undefined) returns early, exactly as - * the ingress gate returns early when `resolveQueryFields` cannot answer: a - * door that cannot see the field map must not invent a verdict about it. For - * that host the driver-side #3821 ladder remains the documented backstop - * (deliberately untouched — a driver-side carve-out is ruled measured-need - * only). - * - * The wording deliberately shares its core sentence and remedies with the - * ingress door's dotted refusal — one vocabulary across the doors, so a caller - * refused at the REST boundary and a caller refused here are not sent two - * different ways. Duplicated rather than imported because `metadata-protocol` - * is assembled FROM an engine, so the engine cannot import from it without - * inverting the layering; the agreement pin in - * `query-expression-conformance.test.ts` is what keeps the duplication honest - * (same mechanism as the sort axis' three-door remedy pin). - */ -function assertProjectionHasNoDottedPaths( - object: string, - operation: 'find' | 'findOne', - schema: any, - fields: unknown, -): void { - if (!Array.isArray(fields) || fields.length === 0) return; - if (!schema?.fields) return; - const dotted = fields.filter( - (f): f is string => typeof f === 'string' && f.includes('.')); - if (dotted.length === 0) return; - const first = dotted[0]; - const head = first.split('.')[0]; - const headDef: any = (schema.fields as any)[head]; - const crossesRelation = headDef != null && REFERENCE_VALUE_TYPES.has(headDef.type); - const err: any = new Error( - (crossesRelation - ? `ObjectQL.${operation}('${object}') projects '${first}', which follows the relationship ` - + `'${head}' into another object — 'fields' reaches only columns of '${object}' itself` - : `ObjectQL.${operation}('${object}') projects '${first}', a dotted path — 'fields' reaches ` - + `only whole columns of '${object}', not values inside them`) - + (dotted.length > 1 ? ` (also: ${dotted.slice(1).join(', ')})` : '') - + '. No driver resolves it: the path reaches the driver as a column name, matches no ' - + 'column, and the projection falls back to EVERY field — a narrower request answered ' - + 'with a wider response.' - + (crossesRelation - ? ` Read the related record with 'expand' (\`{ expand: { ${head}: { object: '', ` - + `fields: [''] } } }\` to choose its columns), or denormalise the value onto ` - + `'${object}' (a stored field, written when the source changes) and name that.` - : ` Name the whole column ('${head}') and read into its value in the caller.`), - ); - // `INVALID_FIELD`, not a new code, and 400 rather than 500 — the same - // reasoning `assertOrderByIsMaterializable` records for `INVALID_SORT`: one - // condition ("this projection was not applied as written") keeps ONE wire - // code however the caller reached it, so a host surfacing engine errors over - // HTTP answers the same envelope on both doors. - err.status = 400; - err.httpStatus = 400; - err.code = 'INVALID_FIELD'; - err.field = first; - err.fields = dotted; - err.object = object; - throw err; -} - -/** - * [#24284] The refusal for a non-blank `search` over an object whose RESOLVED - * search set is empty — no `searchableFields` entry naming a stored field, and - * no field of a type the auto-default scans (the junction shape: lookups plus - * a composed formula title). Without it the expansion added no predicate and - * the read answered every row. - * - * `INVALID_FIELD`, the code #4254's explicit-override gate answers on this - * same axis (and the one a search naming a formula title gets), not - * `INVALID_QUERY`: the request is well-formed, and what cannot be honoured is - * a fact about the object's FIELDS. 400 on every door. - * - * [#25672] `withheld`: the declared set is not empty, but every field in it is - * one the caller may not query, so the narrowing in `expandSearchOnAst` left - * nothing. Same code and status — the search cannot be honoured for this - * caller either way — with a message that names the cause, not a declaration - * fault the object does not have. - */ -function emptySearchSetError(object: string, schema: any, internalExcluded: boolean, withheld = false): Error { - const declared: unknown[] = Array.isArray(schema?.searchableFields) ? schema.searchableFields : []; - const err: any = withheld ? new Error( - `Object '${object}' has no field this caller may search, so 'search' cannot be honoured on it: ` - + `every field it searches is one field-level security does not let this caller query. ` - + 'Answering every row instead would read as "everything matched". Grant read access to one of ' - + `its searchable fields, or filter on a field this caller can read.`, - ) : new Error( - `Object '${object}' has no searchable field, so 'search' cannot be honoured on it` - + (declared.length > 0 - ? `: its 'searchableFields' (${declared.join(', ')}) names no stored field` - : `: it declares no 'searchableFields', and none of its fields is a type the default ` - + `search scans (${[...SEARCHABLE_TEXTUAL_TYPES, ...SEARCHABLE_ENUM_TYPES].join(' / ')})`) - + (internalExcluded ? " ('internal: true' fields are never searched)" : '') - + '. Answering every row instead would read as "everything matched". Declare ' - + `'searchableFields' on '${object}' naming a stored field, or mirror the composed title ` - + `onto a stored text field and search that.`, - ); - err.status = 400; - err.httpStatus = 400; - err.code = 'INVALID_FIELD'; - err.object = object; - err.param = 'search'; - return err; -} - -/** - * [#8682 / #8738] The DECLARED-FIELD DOOR on the WRITE paths — `insert()` and, - * since #8738, `update()`: a key the object does not declare is refused by the - * SCHEMA, before anything is produced for a request that is about to be refused - * anyway. - * - * ONE condition, ONE implementation, deliberately — the two verbs sit in the - * same file and share the shape of the problem, so #8738 gave this function its - * second caller rather than a second predicate to drift against. The verdict is - * a question about a payload and an object's field map; neither term knows - * which verb asked. - * - * ## What ran before this door existed - * - * Nothing rejected an undeclared write key. The row travelled the whole insert - * path and was refused at the very end by the DRIVER — `table crm_account has - * no column named zzz_nonexistent_field` — which `mapDataError` then translated - * into the (correct, unchanged) `400 INVALID_FIELD` the caller sees. The client - * answer was never the defect; what happened behind it was. Measured on - * `origin/main` for one mistyped key: - * - * - an id, a normalized name, owner/creator resolution and the column - * defaults were all produced; - * - the app's `beforeInsert` hooks RAN, and their derived values reached the - * statement (`period_label = 'Q3 2026'`, computed for a request nobody - * would ever store); - * - an AUTONUMBER was issued and CONSUMED — the log-independent observable, - * and the one this door is pinned on: a valid create before the bad request - * numbered `ACC-000013`, the bad request took `ACC-000014` into the void, - * and the next valid create came back `ACC-000015`. A sequence gap is - * permanent and visible to end users on a document number. - * - * A hook is not a pure function — it writes rows, calls out, stamps ledgers — - * so "it ran and we threw the result away" is not a no-op, it is a side effect - * of a request the server had already decided to refuse. - * - * ## The UPDATE path had the same hole (#8738), measured the same way - * - * #8738 filed the update half as INFERRED, not measured, and asked for it to be - * reproduced before anyone implemented against it. It was, on `origin/main` @ - * `e5eeb499c`, with a real engine and a recording driver — and it reproduces on - * BOTH branches of the verb: - * - * - by-id `driver.update` received `zzz_nonexistent_field` and refused it; - * - multi `driver.updateMany` received it and refused it likewise; - * - `beforeUpdate` RAN first on both, and its derived value reached the - * statement — the recorded payload was `{ name, zzz_nonexistent_field, - * description: 'derived-for-bad' }`, where `description` is the hook's, not - * the caller's. The card's `period_label = 'Q3 2026'` shape, one verb over. - * - * What it does NOT have is the insert path's durable damage: no autonumber is - * issued on update, so nothing user-visible is consumed by the refused request. - * The hook side effect and `declared = enforced` (PD #10) are the whole case — - * which is why the pin here is the HOOK, not a sequence gap. - * - * ## Why HERE and not one step later - * - * This runs as the first act inside the middleware body: after middleware (a - * middleware may legitimately rewrite `data`, and judging the caller's keys - * before it ran would refuse rows a sanitizing middleware had already fixed), - * and before `applyFieldDefaults` — hence before the defaults, the summary - * seeding, the hooks, the strips, validation, the autonumber and the statement. - * The same placement, and the same argument, as {@link assertWriteAllowed} and - * `enforceTransactionOrigin` one level up: a refusal this early costs nothing. - * Like those two it throws without logging — the caller is told exactly what is - * wrong, and a client typo is not a server fault to record at ERROR. - * - * `update()` places it by the same rule, which lands it one step EARLIER in - * that verb's body than a naive reading suggests: first act inside the - * middleware body, so ahead of the prior-record read as well as the hooks and - * both strips. Ahead of the read is not incidental — a refused write should not - * cost a driver round-trip either, and the read exists to serve `previous`, the - * `readonlyWhen` gate and the not-found gate, none of which a refused payload - * reaches. It is also ahead of the dispatch ladder's own `reject` verdict, so a - * call that is BOTH mis-keyed and missing its `id`/`multi` is answered on the - * payload: the ladder's message is about how to address rows and would send an - * author hunting for the wrong defect, while `INVALID_FIELD` names the typo. - * - * ## The verdict is the SCHEMA's field map, and the wire answer is unchanged - * - * `INVALID_FIELD` + 400, and the message is byte-identical to the one - * `mapDataError`'s driver-string branch produced (`Unknown field 'x' on object - * 'y'`) — so this moves WHERE the refusal is decided without moving what the - * caller reads. One condition keeps one wire shape however it was noticed, the - * same rule the read path's dotted-projection and unmaterializable-sort doors - * record. The driver-string branch stays where it is: it still backstops schema - * drift (a DECLARED field whose physical column is missing), which this door by - * construction cannot see. - * - * ## Where this door deliberately has NO opinion - * - * A door that cannot see the field map must not invent an opinion about it — - * the read-path doors' rule, and it decides two cases here: - * - * - **`schema.fields` absent** (a registry-less host), and **`schema.fields` - * EMPTY**. An empty map is indistinguishable from an unpopulated one: a real - * registered object always carries at least its primary key, and the - * registry injects the tenancy and audit columns on top, so zero declared - * fields means the host did not fill the map in — not that the object - * forbids every key. Refusing everything on that reading would be a verdict - * made from an absence, which is what this clause exists to prevent. - * - **`id` / `created_at` / `updated_at`**, which are tolerated even when the - * map does not list them. This mirrors `find()` / `findOne()`, which add - * exactly these three to their known set for exactly this reason: they are - * provisioned by the platform rather than authored, so an object may legally - * omit them from its declaration while the physical table has them. Same - * three names, same rationale, so a key accepted by a read is not refused by - * a write. - * - * In every one of those cases the driver remains the backstop it has always - * been — nothing is widened, the refusal simply moves back to where it was. - * - * @returns one entry per row: an `Error` for a row carrying undeclared keys, - * `undefined` for a row that passes. Per row rather than throw-on-first - * because the partial-success path (`insertMany`) reports per row and must - * cull the bad rows instead of failing the batch around them. - */ -export function undeclaredWriteFieldErrors( - object: string, - schema: { fields?: unknown } | undefined, - rows: readonly unknown[], -): Array { - const out: Array = new Array(rows.length); - // [#21571] The declared set and its "no opinion" cases (no map, an array - // map, an empty map) are `declaredColumnSet`'s, shared with the read verbs' - // default projection — one answer to "is this a column of the object". - const declared = declaredColumnSet(schema); - if (!declared) return out; - for (let i = 0; i < rows.length; i++) { - const row = rows[i]; - if (!row || typeof row !== 'object' || Array.isArray(row)) continue; - const undeclared = Object.keys(row as Record).filter((key) => !declared.has(key)); - if (undeclared.length === 0) continue; - const err: any = new Error(`Unknown field '${undeclared[0]}' on object '${object}'`); - err.status = 400; - err.httpStatus = 400; - err.code = 'INVALID_FIELD'; - err.field = undeclared[0]; - err.fields = undeclared; - err.object = object; - out[i] = err as Error; - } - return out; -} - -/** - * [#20805] The computed-field door: take every caller-supplied key that names a - * declared `formula` field out of a write payload, and say which keys went. - * - * ## Why a strip - * - * A `formula` field is virtual. The engine computes it on every read - * ({@link applyFormulaPlan}) and no driver has a column for it - * (`driver-sql`'s `createColumn` emits none), so a full read returns the key - * and a record written back carries it — the ordinary round trip of a form - * save, a flow's `update_record`, or `GET` then `PUT`. Before this door the key - * reached the driver and the driver decided, so the answer split by family: - * SQL refused the whole statement with its own text (`table … has no column - * named …`, no status, no field) and the schemaless family stored the value as - * a shadow column nothing reads. {@link undeclaredWriteFieldErrors} cannot see - * it: the field IS declared. - * - * The platform already has one answer for a declared field a caller cannot - * write — strip the value, complete the write, report the strip through - * `onFieldsDropped` (`DroppedFieldsEventSchema`: "stripping is legitimate - * semantics, not an error") — and a computed field takes it, under its own - * `reason`, `computed`. ⛔ Never `readonly`, which it is not: the static - * `readonly` strip exempts `isSystem`, and this one runs in EVERY context - * because the value has nowhere to land for any caller. ⛔ Never a refusal - * either: a `400` would make `formula` the one caller-read-only type that - * refuses where `readonly`, `readonlyWhen` and the primary key strip, and it - * would refuse every round trip of every object that declares one. - * - * ## Where it runs - * - * Beside the declared-field door, as the first act of each write verb's body - * and of {@link ObjectQL.validate} over the same rows: before the defaults, the - * hooks and every other strip. So a hook is never handed a value that will not - * be stored (the #16344 invariant), a `formula` field also declared - * `readonly: true` is reported once, as `computed`, and the preview strips - * exactly what the write strips. The REPORT is the verb's own report site, - * beside the other strips' (`insertDrops` on insert, `reportDroppedFields` at - * the confluence on update) — so `strictReadonlyWrites`, whose coverage is - * derived from what `onFieldsDropped` reports, refuses a computed drop too. - * - * ⛔ Keyed on `type === 'formula'` literally — the test the read path's - * projection and `driver-sql`'s `fieldHasColumn` use. `summary` is NOT here: a - * roll-up HAS a column, and a caller-supplied value is persisted (measured on - * #20805, SQLite and memory) — a separate question, not this door's. - * - * Pure: the caller's objects are never mutated. A row keeps its reference when - * nothing was taken, else it is a shallow copy without the keys. - * - * @param refused the declared-field door's per-row verdicts, index-aligned: a - * row that door refused is left exactly as it is and counts toward nothing. - * @returns the rows as the strip leaves them (index-aligned with the input); - * the union of the keys it took, in first-seen order — one report per call, - * the shape every insert-side strip already reports in; and `droppedPerRow`, - * the keys taken from EACH row (index-aligned, `[]` for a row it left alone). - * The union and the per-row lists are the same strip's one result read two - * ways: the per-row lists are what {@link droppedFieldEvents} attributes - * to a row on `validate` and on `insertMany`'s outcomes. - */ -export function stripComputedWriteFields( - schema: { fields?: unknown } | undefined, - rows: readonly unknown[], - refused?: readonly unknown[], -): { rows: unknown[]; dropped: string[]; droppedPerRow: string[][] } { - const out = rows.slice(); - const dropped: string[] = []; - const droppedPerRow: string[][] = rows.map(() => []); - const fields = schema?.fields; - if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return { rows: out, dropped, droppedPerRow }; - const computed = Object.entries(fields as Record) - .filter(([, def]) => def?.type === 'formula') - .map(([name]) => name); - if (computed.length === 0) return { rows: out, dropped, droppedPerRow }; - for (let i = 0; i < out.length; i++) { - if (refused?.[i] !== undefined) continue; - const row = out[i]; - if (!row || typeof row !== 'object' || Array.isArray(row)) continue; - // Own-property, never `in` — the same reason `stripReadonlyFields` gives. - const taken = computed.filter((name) => Object.prototype.hasOwnProperty.call(row, name)); - if (taken.length === 0) continue; - const copy: Record = { ...(row as Record) }; - for (const name of taken) { - delete copy[name]; - if (!dropped.includes(name)) dropped.push(name); - } - droppedPerRow[i] = taken; - out[i] = copy; - } - return { rows: out, dropped, droppedPerRow }; -} - -/** - * The strip report as `DroppedFieldsEvent`s: one event per reason, `computed` - * before `readonly`, then `stored_expression` (#24690) — the order the passes - * run — and an empty list when nothing was taken. - * - * ONE builder for both channels the create-side strips report through, so - * they cannot disagree on a reason or its order: the batch-level union (the - * `onFieldsDropped` events of `insert` and {@link ObjectQL.validate}, unchanged - * by the per-row channel) and the per-row lists (`results[i].droppedFields` on - * `validate`, `droppedFields` on an `ok` outcome of `insertMany`). The per-row - * lists are recorded at the strips themselves, never reconstructed from the - * union: "which rows supplied N" is not "which rows dropped N" once a hook can - * exempt a key on one row and not another. - */ -export function droppedFieldEvents( - object: string, - computed: readonly string[], - readonly: readonly string[], - storedExpression: readonly string[] = [], -): DroppedFieldsEvent[] { - const events: DroppedFieldsEvent[] = []; - if (computed.length > 0) events.push({ object, fields: [...computed], reason: 'computed' }); - if (readonly.length > 0) events.push({ object, fields: [...readonly], reason: 'readonly' }); - // [#24690] Last: the stored-expression pass runs after every strip. - if (storedExpression.length > 0) events.push({ object, fields: [...storedExpression], reason: 'stored_expression' }); - return events; -} - -/** - * [#21682] The keys the CALLER sent on each row of an insert, recorded at - * `insert`'s entry, BEFORE the middleware chain runs, and keyed by the row - * OBJECT. - * - * `insert` takes its caller snapshot (`suppliedPerRow`) inside the middleware - * chain's innermost step, so by then a write middleware may already have - * filled the payload: `@objectstack/organizations` fills an absent - * `organization_id` with the active organization, and - * `@objectstack/plugin-security` fills an absent `owner_id` with the acting - * user. Both write IN PLACE, onto the very row objects recorded here. Without - * this record the snapshot reads those fills as keys the caller sent, so the - * static-`readonly` strip took the platform's own `organization_id` and - * `droppedFields` reported it, on every walled create that named no - * organization. The console announces every non-empty `droppedFields` as a - * warning toast. This is the insert-side twin of the update path's #8093 - * (ADDRESSING IS NOT PAYLOAD): a value the platform put on the payload is - * not one the caller lost. - * - * Keyed by IDENTITY, not by index, so the answer survives a middleware that - * reorders a batch. A row a middleware REPLACED wholesale has no entry, and - * {@link callerSuppliedRow} then keeps every key it carries: the verdict from - * before this record existed. That is the over-reporting direction, never the - * under-stripping one. - * - * ⛔ No name list. Which keys are the platform's is answered by WHEN they - * appeared, so a new stamping middleware is covered without being named here. - */ -function callerKeySets(data: unknown): WeakMap> { - const sets = new WeakMap>(); - for (const row of Array.isArray(data) ? data : [data]) { - if (row !== null && typeof row === 'object' && !sets.has(row)) { - sets.set(row, new Set(Object.keys(row))); - } - } - return sets; -} - -/** - * One row of `insert`'s caller snapshot: a shallow COPY of the row as the - * middleware chain handed it on, keeping only the keys the caller sent - * (`sent`, from {@link callerKeySets}). - * - * Only WHICH keys is narrowed. A kept key keeps the value the chain handed - * on, so every key the caller did send is judged exactly as it was before - * #21682. `sent` undefined means "this row has no record" and keeps every key. - */ -function callerSuppliedRow(row: unknown, sent: ReadonlySet | undefined): Record { - const copy: Record = { ...((row ?? {}) as Record) }; - if (sent) { - for (const key of Object.keys(copy)) { - if (!sent.has(key)) delete copy[key]; - } - } - return copy; -} - -/** - * [#24822] Whether ONE write's static-`readonly` passes leave a non-system - * caller's `organization_id` on the row for the Layer 0 wall's verdict instead - * of taking it. The ONE answer both write verbs read: the insert's subject - * ({@link insertStaticReadonlySubject}) and the update's - * ({@link updateStaticReadonlySubject}). - * - * Three things must hold, and the first is the wall's own word: - * - * 1. `wallJudgesOrganization`: the seam the wall installed on THIS write - * declares that it judges the stored rows' organization - * (`PostHookWriteImageCheck.judgesOrganization`). The caller reads that - * member once, for a non-system caller only. [#25062] The write preview - * reads the same value from the wall's verdict on each row - * (`WriteWallRowVerdict`). Nothing here derives it: where no wall judged - * the column, the strip takes a caller's value exactly as before. - * 2. The object is one the INSERT strip judges, with the column in that - * subject ({@link staticReadonlyInsertSubject}). So never the reserved - * `sys_` namespace or a platform-internal `managedBy` bucket, whose own - * field-write governance a spare must not pre-empt. The UPDATE strip runs - * on those objects too (it applies neither exclusion), so on an update - * this condition is what keeps a `sys_` row's organization exactly as - * before: the update spare covers exactly the objects the insert spare - * covers. - * 3. The column is the platform's injected tenant column, the one - * `TENANT_SCOPE_FIELD_DEF` declares read-only on the platform's behalf. An - * AUTHORED `organization_id` declared `readonly: true` is the author's own - * declaration, and declaration is enforcement: it keeps the strip. Which - * one this is, is the registry's own provenance reader, never a second - * copy. - * - * Why the column is left on the row where all three hold: the wall has - * already refused, before this engine ran, every supplied value it does not - * admit, and it judges the stored rows again after the hooks: an insert's - * rows, an update's prior row merged with the final payload, and on the - * predicate path every matched row merged with it. So a value that reaches the - * strip is one the wall admitted, and taking it put another organization in - * its place (the active one stamped on an insert, the row's own kept on an - * update). That is a silent replacement of an admitted input, which ADR-0105 - * D5 and the organizations runtime's stamp rule out. - */ -function staticReadonlySparesOrganization(schema: unknown, wallJudgesOrganization: boolean): boolean { - if (!wallJudgesOrganization) return false; - const subject = staticReadonlyInsertSubject(schema as any); - if (!subject || !Object.prototype.hasOwnProperty.call(subject.fields, SystemFieldName.ORGANIZATION_ID)) return false; - return resolveInjectedColumnProvenance(schema, SystemFieldName.ORGANIZATION_ID) !== 'author'; -} - -/** - * [#24822] The fields ONE insert's static-`readonly` passes may judge: the - * {@link staticReadonlyInsertSubject} view, without `organization_id` where - * {@link staticReadonlySparesOrganization} spares it. - * - * ONE function for both passes that read the subject, the pre-hook withhold - * and the post-hook strip, so the hook view and the stored row cannot - * disagree about the column. [#25062] The write preview's strip reads it too. - */ -export function insertStaticReadonlySubject( - schema: unknown, - wallJudgesOrganization: boolean, -): ReturnType { - const subject = staticReadonlyInsertSubject(schema as any); - if (!subject || !staticReadonlySparesOrganization(schema, wallJudgesOrganization)) return subject; - const fields = { ...subject.fields }; - delete fields[SystemFieldName.ORGANIZATION_ID]; - return Object.keys(fields).length > 0 ? { name: subject.name, fields } : null; -} - -/** - * [#24822] The schema view ONE update's static-`readonly` passes judge: the - * object's whole registered schema, because the update strip applies neither - * of the insert subject's exclusions, without `organization_id` where - * {@link staticReadonlySparesOrganization} spares it. `schema` itself - * otherwise, so every update the spare does not reach is judged exactly as - * before. - * - * ONE function for every pass that asks the update's static strip: the - * pre-hook hide pass (what `beforeUpdate` is shown), the by-id and predicate - * strips (what is stored), and {@link staticReadonlyStoredView}, which the - * `readonlyWhen` strips and the master-detail settlement read. `update()` - * reads the wall's flag once and hands that one value to all of them, so the - * hook view and the stored row cannot disagree about the column. [#25062] The - * `update`-mode preview's strip reads it too. - */ -export function updateStaticReadonlySubject(schema: unknown, wallJudgesOrganization: boolean): unknown { - if (!staticReadonlySparesOrganization(schema, wallJudgesOrganization)) return schema; - const { name, fields } = schema as { name?: string; fields: Record }; - const kept = { ...fields }; - delete kept[SystemFieldName.ORGANIZATION_ID]; - return { name, fields: kept }; -} - -/** - * [#22306] The insert side of the #16344 invariant — a hook is never handed a - * value that will not be stored. Takes out of ONE caller row every value the - * create-side static-`readonly` strip will take, before the defaults and - * before `beforeInsert`, and says what it took. - * - * ## What was measured broken - * - * `insert()` dispatched `beforeInsert` on the caller's payload and ran the - * runtime-owned and static-`readonly` strips after it. A hook that stamps a - * read-only column only when it is absent — `if (!data.stage_entry_date)` — - * saw the caller's value, stood down, and the strip then took that value: the - * row stored NULL where the hook would have stamped today. Real engine over - * `driver-sql` (better-sqlite3), non-system context: caller value present ⇒ - * stored `stage_entry_date` NULL; key absent ⇒ stored the hook's stamp. The - * update path has hidden these values from its hooks since #16344; insert - * did not. - * - * ## The SAME function as the strip, so the two cannot disagree - * - * `stripReadonlyFields` over {@link staticReadonlyInsertSubject} with no - * `preserveAudit` — the function, subject and options the post-hook - * static-`readonly` strip (`staticReadonlyCreateStrip`) runs. Before any hook - * has run there is no hook write to spare, so what it would take here is what - * it WILL take after the hooks unless a hook assigns the key. A key it would - * keep — middleware-filled (not in `supplied`), or on a `sys_` or - * platform-internal object — is still shown to the hook. - * - * ⛔ Runtime-owned values (`autonumber`) are NOT withheld, though the - * runtime-owned strip takes them after the hooks too. #6339 keeps the caller's - * record number visible to `beforeInsert` on purpose, so a guard can report - * what the caller submitted — pinned as "the hook can still SEE the - * caller-submitted record number" (`engine-insert-runtime-owned-strip.test.ts`) - * and by #14259's provenance seam. Moving that is a decision of its own, not a - * rider on this one. - * - * ⚠️ One narrowing, and it is the create side's own declared channel rather - * than a second opinion: under `preserveAudit` the audit timeline - * (`AUDIT_PROVENANCE_FIELDS`) stays visible. The static strip takes those keys - * on a create whatever the flag says (the 2026-08-08 ruling made its - * `preserveAudit` exemption UPDATE-only), but the historical-import channel the - * 2026-09-06 ruling (#15964) kept working reinstates them THROUGH the audit - * binder's `preserveAudit` branch (`record.created_at ?? now`, `plugin.ts`) — - * a `beforeInsert` hook reading the caller's value and assigning it back, which - * the strip then keeps as a hook write. Those values ARE stored, so they are - * not withheld; withholding them would erase every historical `created_at` an - * import carries. A business `readonly` column under `preserveAudit` is still - * withheld: the create side strips it (and says so in the WARN). - * - * ## Why before the DEFAULTS - * - * So the hook is shown exactly the row a caller who never sent the key would - * have produced: `applyFieldDefaults` then fills a withheld key from its - * `defaultValue`, just as it fills any absent key, and that default IS what - * the create stores for it (`staticReadonlyCreateStrip` re-derives it after - * the strip). A forged `approval_status: 'approved'` is shown as `'draft'`, - * never as a hole — the hook cannot tell the two callers apart, which is the - * point. A key with no default reads absent, which is what the row will hold. - * - * Pure: neither the caller's row nor `supplied` is touched. The row keeps its - * reference when nothing was taken, else it is a shallow copy without the keys. - * - * [#24822] `wallJudgesOrganization` spares `organization_id` here exactly as - * the post-hook strip spares it ({@link insertStaticReadonlySubject}). A value - * the wall judges will be stored, so the hook is shown it. - * - * @returns the row the hook view is built from, and the withheld keys with the - * caller's values (`undefined` when nothing was withheld) — handed back after - * the hooks by {@link handBackWithheldInsertReadonly}. - */ -function withholdInsertReadonlyFromHooks( - schema: unknown, - row: Record, - supplied: Readonly>, - preserveAudit: boolean, - wallJudgesOrganization: boolean, -): { row: Record; withheld: Record | undefined } { - let subject = insertStaticReadonlySubject(schema, wallJudgesOrganization); - if (subject && preserveAudit) { - const fields = { ...subject.fields }; - for (const name of AUDIT_PROVENANCE_FIELDS) delete fields[name]; - subject = Object.keys(fields).length > 0 ? { name: subject.name, fields } : null; - } - if (!subject) return { row, withheld: undefined }; - // No logger and no `strictReadonlyWrites`, deliberately: this pass is SILENT - // by construction. The strip after the hooks owns every word said about - // these keys — the WARN, `onFieldsDropped`, the strict refusal — and says it - // over the handed-back payload exactly as it did before this pass existed. - const shown = stripReadonlyFields(subject as any, row, supplied, undefined) as Record; - if (shown === row) return { row, withheld: undefined }; - const withheld: Record = {}; - for (const key of Object.keys(row)) { - if (!(key in shown)) withheld[key] = row[key]; - } - return { row: shown, withheld }; -} - -/** - * [#22306] The other end of {@link withholdInsertReadonlyFromHooks}: put the - * caller's withheld values back on a row after its `beforeInsert` dispatch is - * SEALED, wherever the hook did not write the key — so the strips below judge, - * take, re-default, WARN, report and (under `strictReadonlyWrites`) refuse - * exactly the payload they judged before the hide existed. - * - * The update path's hand-back (#16344), carried over with one addition the - * insert view needs: a withheld key may have been SHOWN holding its - * `defaultValue` (`shown`), so "the hook left it alone" is not always "the key - * is absent". Per key: - * - * - present and `undefined` ⇒ the `data.x = data.x` self-assignment of a key - * the hook was shown absent. Undone (deleted, dropped from the record) and - * handed back, as on update: no driver can store `undefined`, and the write - * must read exactly as it would with no hook at all. - * - absent ⇒ handed back unless the hook was shown a value there, in which - * case the hook REMOVED it, and the removal stands — the row a caller who - * never sent the key would have stored. - * - present, record available ⇒ handed back unless a hook assigned the key - * (`hookWrittenKeys`): an untouched default is replaced by the caller's - * value, which the static strip then takes and re-defaults to the same - * value it showed. - * - present, no record (a hook replaced the payload object) ⇒ the #5591 value - * test: handed back only while it still holds exactly what was shown. - * - * ⛔ Never over a hook's write, and placed AFTER the seal: a hand-back inside - * the recording window would enter the record as a hook write, and the strip - * would then keep the caller's forgery as the hook's — the laundering #14088 - * exists to prevent. Every key handed back is dropped from the returned record - * for the same reason. - * - * @returns the row's `hookWrittenKeys`, narrowed by every key handed back - * (`undefined` stays `undefined`). - */ -function handBackWithheldInsertReadonly( - target: Record, - withheld: Readonly>, - shown: ReadonlyMap, - hookWrittenKeys: ReadonlySet | undefined, -): ReadonlySet | undefined { - let narrowed: Set | undefined; - for (const [key, callerValue] of Object.entries(withheld)) { - const present = Object.prototype.hasOwnProperty.call(target, key); - let handBack: boolean; - if (present && target[key] === undefined) { - delete target[key]; - handBack = true; - } else if (!present) { - handBack = !shown.has(key); - } else if (hookWrittenKeys !== undefined) { - handBack = !hookWrittenKeys.has(key); - } else { - handBack = shown.has(key) && Object.is(target[key], shown.get(key)); - } - if (!handBack) continue; - target[key] = callerValue; - if (hookWrittenKeys?.has(key)) { - narrowed ??= new Set(hookWrittenKeys); - narrowed.delete(key); - } - } - return narrowed ?? hookWrittenKeys; -} - -/** - * Evaluate formula virtual fields against the raw rows a driver handed back — - * the read path (`find` / `findOne`) and, since #5504, the write path's - * response hydration. - * - * Since #10280 this is also where a formula field's declared `scale` is - * applied: the value is rounded on its way onto the record, so every consumer - * of every one of those three call sites inherits the same rounded number. - * {@link roundFormulaValue} carries the arithmetic and the reason the rounding - * belongs at this producer rather than in the validator. - * - * The eval context is built ONCE per call and reused for every row × every - * formula field, and that is where this function's determinism comes from: one - * `now`, so a `now()`/`today()` formula cannot drift mid-operation, plus - * `os.user` / `os.org` resolved from the execution context (so a computed field - * can reference the caller, e.g. `os.user.id`). Previously this passed only - * `{ record }`, so `now()`/`today()` ran against live wall-clock and user/org - * were unreachable. - * - * [#20082] `permissions` is the acting subject's effective object-permission - * map, the one `current_user.can(object, verb)` answers from. The CALLER - * resolves it — at most once per operation, and only when a planned formula - * calls `can` ({@link ObjectQL.resolveFormulaPermissions}) — because this - * function is synchronous and owns no resolver. `undefined` is "no permission - * data", never an empty map: a `can`-calling formula then fails to evaluate and - * reads `null`, the rule every formula that does not evaluate follows here, and - * the caller has already said why in its log line. - * - * That context has the same SHAPE as `applyFieldDefaults`' — the same keys, so - * one expression vocabulary serves `formula` and `defaultValue` alike — but NOT - * the same `now` value, and the two are sourced independently on purpose - * (#5699): - * - `applyFieldDefaults` is handed the insert's `nowSnapshot`, so every - * defaulted field of every row in one write carries the same PRE-write - * instant; - * - this function reads the clock itself, once per call, because a formula is - * evaluated when a record is MATERIALIZED — at read time, and on the write - * response — not at the moment that row's defaults were resolved. - * - * So inside a single `insert` a `NOW()` default and a `now()` formula observe - * two instants one driver round-trip apart (sub-millisecond in practice; across - * a second/day boundary they can land on different calendar days). Making them - * share one instant would hand the write path a determinism guarantee the read - * path cannot have — a semantic decision, not a tidy-up, argued in #5699. Until - * it is decided this function takes NO snapshot parameter: the zero-caller - * `nowSnapshot?: Date` it carried from birth was retired there, in the same - * enforce-or-remove reflex ADR-0049 applies to spec properties, because a - * dormant parameter reads as a live one and anyone reasoning from it concludes - * the two sides already share an instant. - * - * (ADR-0053 Phase 2 will additionally thread `timezone` here once - * `ExecutionContext.timezone` exists — see #1980; this change is independent - * of timezone.) - */ -/** - * [#10280] Apply a formula field's declared `scale` to the value its - * expression just produced. - * - * ## Why the rounding lives HERE and not in the validator - * - * `validateFieldValue`'s `scale` branch is the package's only other `.scale` - * reader and the tempting seam — and it is the wrong one. That branch enforces - * `scale` by REJECTION (`max_scale`) under the maintainer's 2026-08-11 ruling, - * because a caller-supplied value has somebody to refuse and silently rounding - * it would be silently altering their data. A formula result is - * platform-computed: there is nobody to refuse, and the alternative to - * rounding is not "the author's exact number", it is fifteen digits of IEEE-754 - * tail on a record page. Two different values, two different answers, one - * declaration — so the rounding is applied at the PRODUCER, on plan entries - * that are `type === 'formula'` only, and the rejection path is left - * structurally untouched. - * - * ## The arithmetic - * - * `Number(v.toFixed(scale))`, the same expression objectui's `computeRow` - * (`GridField.tsx`) uses for a grid's client-side computed column, so the two - * halves of one platform round a computed number the same way. Per ECMA-262 - * `toFixed` extracts the sign first and then picks the larger `n` on a tie: - * round-half-**away from zero** on the magnitude. Measured, because the - * opposite was believed at dispatch time: `2.5 → 3`, `1.25 → 1.3`, - * `0.125 → 0.13`, `-1.5 → -2`, `-2.5 → -3`. ⚠️ Negatives round away from zero, - * NOT toward `+Infinity`. - * - * Three guards, each closing a measured failure rather than a hypothetical: - * - * - **`typeof value === 'number'`** — a formula legitimately returns a string - * (`"tier:" + record.tier`), a boolean, or `null`, and `'x'.toFixed` does - * not exist. Without this guard one scale-declaring string formula throws - * and takes down every read of every record of that object. - * - **`Number.isFinite`** — `NaN.toFixed(2)` yields the STRING `'NaN'`, and - * `1.0 / 0.0` really does reach here as `Infinity`. Both round-trip through - * `Number(...)` unchanged, so this guard alters no value; it is kept so the - * helper is total by construction rather than by a coincidence of two - * library behaviours. (For `|v| >= 1e21` `toFixed` returns exponential form, - * `'1e+21'`, which `Number()` round-trips losslessly — the rounding degrades - * to a no-op instead of corrupting the value, so that case needs no guard.) - * - **negative zero** — `(-0.001).toFixed(2)` is `'-0.00'` and `Number()` - * makes that `-0`. It JSON-serializes as `0`, but `Intl.NumberFormat` - * formats it as `-0.00`, which would put a brand-new display wart on the - * very record page this issue is about. Normalized to `+0` at the producer, - * where it costs one comparison, rather than in each renderer. - */ -function roundFormulaValue(value: unknown, scale: number | undefined): unknown { - if (scale === undefined) return value; - if (typeof value !== 'number' || !Number.isFinite(value)) return value; - const rounded = Number(value.toFixed(scale)); - return Object.is(rounded, -0) ? 0 : rounded; -} - -/** - * [#22019] Where {@link applyFormulaPlan} reports a formula it could not - * evaluate: the field it was computing and the evaluator's own error. The - * engine binds one per call to the object being read, so the report names the - * (object, field) pair — see {@link ObjectQL.formulaFaultSink}. - */ -type FormulaFaultSink = (field: string, error: EvalError) => void; - -/** - * [#24197] What a formula read hands {@link applyFormulaPlan} for the hops its - * plan reads: the per-row related bindings - * ({@link ObjectQL.resolveFormulaRelated}), or — when this read serves no hop - * at all — the sentence saying why, which every traversing formula of the read - * then reports. - * - * [#24410] `undeclared` is decided per FORMULA, never per reference: the - * related fields a reference's target does not declare, which the shared read - * never asks for. A formula naming one of them reads `null` with the - * `undeclared-field` reason; a sibling reading only declared fields through the - * same reference is served from the shared read exactly as it is alone. - */ -type FormulaRelated = { - readonly bindings: (row: Record) => RelatedRecordBinding | undefined; - readonly undeclared: ReadonlyMap }>; -} | string; - -/** - * [#24410] The named related fields a reference's target does not DECLARE — - * the one reading both seams that serve a hop make of it: the shared loader - * ({@link ObjectQL.resolveOneHopRelated}) and the formula read, which judges - * them per formula ({@link ObjectQL.resolveFormulaRelated}). - * - * [#8215] The PRIMARY KEY counts as declared, though no field map carries it: - * `record..id` is the repair the traversal conflict refusal prescribes. An - * unknown target (`declared` absent) declares nothing to judge against, so - * nothing is undeclared. - */ -export function undeclaredRelatedFields( - declared: Record | undefined, - named: Iterable, -): string[] { - if (!declared) return []; - return [...named].filter((n) => n !== SystemFieldName.ID && !(n in declared)); -} - -/** [#24197] Why a hop was not served where no reader supplied one. */ -const FORMULA_HOP_NOT_SERVED = - 'this evaluation reads no related record — only a read (`find` / `findOne`) or a write\'s response ' - + 'preloads the one hop a formula reads through a reference'; - -/** - * [#24197] The record ONE formula is evaluated against: a shallow copy whose - * reference fields this formula reads through carry the related row, or the - * reason it cannot be evaluated at all. - * - * The validation path's verdicts, kept ({@link RelatedFieldBinding}): a related - * record that was not found — or that this reader may not read, which a - * caller-authority read cannot tell apart and must not — a related object that - * declares no such field, and a failed read each leave the formula UNEVALUATED, - * so it reads `null` WITH a reason, never a value computed over a record that - * was not there. ⛔ Never a fallback evaluated in its place: an optional read - * would turn "not found" into the author's empty-reference default, which is - * data the reader cannot tell from a real answer. - * - * An EMPTY reference is the one case that is evaluated: the author can say - * what an empty reference means (`record.book.?title.orValue('—')`), and the - * field keeps its stored empty value, so a plain read through it faults - * exactly as before and the fault names the repair (`empty`). - * - * Per formula, onto a copy: hydrating replaces a reference's stored id with a - * row, and a sibling formula that reads the id, and the record the read - * returns, must keep seeing the id. - */ -function formulaHopScope( - rec: Record, - hops: ReadonlyMap>, - related: FormulaRelated | undefined, -): { ok: true; record: Record; empty: string[] } | { ok: false; reason: string } { - if (related === undefined || typeof related === 'string') { - return { ok: false, reason: related ?? FORMULA_HOP_NOT_SERVED }; - } - // [#24410] This formula's own undeclared reads first, before any row: they - // are a fault of the expression, whatever the record holds. - for (const [field, named] of hops) { - const missing = related.undeclared.get(field); - const own = missing ? [...named].filter((n) => missing.fields.has(n)) : []; - if (missing && own.length > 0) { - return { - ok: false, - reason: formulaHopRefusal(field, { object: missing.object, unavailable: 'undeclared-field', undeclaredFields: own }, named), - }; - } - } - const binding = related.bindings(rec); - let copy: Record | undefined; - const empty: string[] = []; - for (const [field, named] of hops) { - const b = binding?.[field]; - if (!b) return { ok: false, reason: FORMULA_HOP_NOT_SERVED }; - // ⛔ Only the row this record's own foreign key names. - if (b.row && b.row.id != null && String(b.row.id) === String(rec[field])) { - if (!copy) copy = { ...rec }; - copy[field] = b.row; - continue; - } - if (!b.row && b.unavailable === 'no-reference') { empty.push(field); continue; } - return { ok: false, reason: formulaHopRefusal(field, b, named) }; - } - return { ok: true, record: copy ?? rec, empty }; -} - -/** - * [#24197] The sentence a formula that could not read its hop reports. It names - * the RELATED object and field, as the validation path's refusal does, and it - * never says WHICH of "not found" and "not readable by this reader" held: a - * reader who may not see the related record learns nothing from the formula. - */ -function formulaHopRefusal(field: string, b: RelatedFieldBinding, named: ReadonlySet): string { - const columns = [...named].sort().map((n) => `'${n}'`).join(', '); - const on = `\`${field}\` (object '${b.object}')`; - switch (b.unavailable) { - case 'undeclared-field': { - const missing = (b.undeclaredFields ?? []).map((n) => `'${n}'`).join(', ') || columns; - return `the formula reads ${missing} through ${on}, but '${b.object}' declares no such field: fix the ` - + `expression, or declare ${missing} on '${b.object}' — not on the object carrying this formula`; - } - case 'unreadable': - return `the formula reads ${columns} through ${on}, and that read failed, so it has no value`; - case 'no-reference': - case 'unresolved': - default: - return `the formula reads ${columns} through ${on}, and the record it names was not found by this ` - + 'read (it does not exist, or this reader may not read it), so it has no value'; - } -} - -/** - * What a formula field reports when it evaluated to the invalid Date that - * `date()` / `datetime()` answer for text they cannot parse - * ({@link isUnparseableDate}). An EMPTY source is not this fault: it reads - * `null` and says nothing. The record's text is not quoted — a log line is no - * place for row data. - */ -const UNPARSEABLE_DATE_FAULT = - 'date() / datetime() was handed text it cannot parse as a date, so the formula has no value on this record; ' - + 'they take ISO text (`2026-03-04`, `2026-03-04T09:30:00Z`), a timestamp or epoch milliseconds, and an empty ' - + 'source reads null without a report.'; - -function applyFormulaPlan( - plan: FormulaPlanEntry[], - records: any[], - execCtx?: ExecutionContext, - permissions?: EvalPermissions, - onFault?: FormulaFaultSink, - related?: FormulaRelated, - /** - * [#26608] What `label(record.)` reads: the object's option labels, - * built once per call by {@link readerOptionLabels} (a read, in the reader's - * locale) or {@link declaredOptionLabels} (a stored expression). `undefined` - * when no formula of the plan calls `label()`. - */ - optionLabels?: EvalOptionLabels, -): void { - if (!plan.length) return; - const now = new Date(); - const timezone = execCtx?.timezone; - const user = execCtx?.userId ? { id: String(execCtx.userId), positions: execCtx?.positions ?? [] } : undefined; - const org = execCtx?.tenantId ? { id: String(execCtx.tenantId) } : undefined; - for (const rec of records) { - if (rec == null) continue; - for (const fp of plan) { - // [#24197] A traversal the one-hop read cannot serve is not evaluated: - // it reads null and says why, in the lint's own words. - if (fp.hopConflict) { - rec[fp.name] = null; - onFault?.(fp.name, { kind: 'runtime', message: fp.hopConflict }); - continue; - } - let scope: Record = rec; - let empty: string[] = []; - if (fp.hops) { - const hop = formulaHopScope(rec, fp.hops, related); - if (!hop.ok) { - rec[fp.name] = null; - onFault?.(fp.name, { kind: 'runtime', message: hop.reason }); - continue; - } - scope = hop.record; - empty = hop.empty; - } - const evaluated = ExpressionEngine.evaluate(fp.expression, { now, timezone, user, org, permissions, optionLabels, record: scope }); - // An invalid Date minted from text `date()` could not parse evaluates - // `ok` (a predicate relies on that) but serializes to the `null` an empty - // source gives: for a field it is a fault, said like any other. - const r = evaluated.ok && isUnparseableDate(evaluated.value) - ? { ok: false as const, error: { kind: 'runtime' as const, message: UNPARSEABLE_DATE_FAULT } } - : evaluated; - if (r.ok) { - rec[fp.name] = roundFormulaValue(r.value, fp.scale); - continue; - } - // [#22019] The answer stays `null` — a formula that cannot be computed - // has no value, and changing what a read returns is a protocol change. - // What changes is that the fault is no longer swallowed (ADR-0032: a - // call site must not silently turn a bad expression into `null`): the - // sink names the field and the evaluator's error. - rec[fp.name] = null; - onFault?.(fp.name, empty.length === 0 ? r.error : { - ...r.error, - // [#24197] The fault on a record whose reference is EMPTY names the - // repair first; the evaluator's own message follows it. - message: `${empty.map((f) => `\`${f}\``).join(', ')} holds no single reference on this record (it is ` - + 'empty, or holds several), so the formula has no related record to read through it; to give the ' - + `formula a value while it is empty, write the optional read \`record.${empty[0]}.?.orValue('…')\`.\n${r.error.message}`, - }); - } - } -} - -/** - * Hydrate `formula` virtual fields onto the records a WRITE hands back (#5504). - * - * `applyFormulaPlan` used to hang off the read path only — `find` and `findOne` - * — so `POST /data/:object` and `PATCH /data/:object/:id` answered with the - * stored document, in which a formula field is not merely `null` but ABSENT - * (formulas are virtual: no driver ever returns a column for one). The very - * next `GET` of that same row carried every one of them, so a caller that - * rendered the create response — the natural thing to do, the response calls - * itself `record` — got a blank title on every object whose `nameField` points - * at a formula, and could not tell "not configured" from "not evaluated". - * Read-your-write, restored: the write response is now the same materialization - * a read produces. - * - * Deliberately the SAME plan builder and the SAME evaluation the read path uses - * — one formula semantic, not a write-path dialect: - * - `planFormulaProjection(schema, undefined)` is exactly find's no-projection - * branch: every formula field the schema declares, and no `projected` - * rewrite (a write returns whole rows, so there is nothing to project). - * It also carries the perf threshold unchanged — an object declaring no - * formula yields an empty plan and `applyFormulaPlan` returns at its first - * line, so a write on an ordinary object pays a field-name loop and nothing - * else. - * - the execution context is threaded exactly as find threads it, so `os.user` - * / `os.org` resolve identically on both sides. Widening what that context - * carries is #1979's work and stays out of here. - * - [#20082] the permission map comes from the same place find's does: - * `permissionsFor` is the engine's {@link ObjectQL.resolveFormulaPermissions} - * bound to the WRITE's own resolution, so a write that already resolved the - * map for a default or an option gate reuses it rather than asking again. - * - * Evaluates against the record the driver returned (a full row: `create` uses - * `RETURNING *`, `update` re-reads), so no extra round-trip is needed and no - * formula sees a partial record. Mutates in place, like the read path. - * - * Non-record entries (a `null` readback when the write moved the row out of the - * caller's scope; the affected-row COUNT a predicate update resolves to) are - * skipped rather than special-cased at each call site — the primitive would - * otherwise take a property assignment, which throws under ES module strict - * mode. - */ -async function hydrateWriteFormulas( - schema: any, - results: unknown[], - execCtx: ExecutionContext | undefined, - permissionsFor: (plan: FormulaPlanEntry[], records: readonly unknown[]) => Promise, - onFault?: FormulaFaultSink, - /** [#24197] The hops the plan reads, resolved over `records` as a read resolves them. */ - relatedFor?: (plan: FormulaPlanEntry[], records: ReadonlyArray>) => Promise, - /** [#26608] The option labels `label()` reads, in the writer's locale — the reader of this response. */ - optionLabelsFor?: (plan: FormulaPlanEntry[]) => EvalOptionLabels | undefined, -): Promise { - const records = results.filter( - (r): r is Record => r != null && typeof r === 'object', - ); - if (records.length === 0) return; - const { plan } = planFormulaProjection(schema, undefined); - if (plan.length === 0) return; - applyFormulaPlan( - plan, records, execCtx, await permissionsFor(plan, records), onFault, await relatedFor?.(plan, records), - optionLabelsFor?.(plan), - ); -} - -/** - * [#24619] The stored computed fields: every field that is NOT a `formula` - * and declares an `expression`. Such a field has a column, so a summary can - * roll it up (a summary aggregates stored columns, never an on-read formula), - * and its value is the engine's to compute on every write — the declared - * `expression` over the row the write stores. A client-sent value never - * stands: it is overwritten by the computed one, the way the console grid - * already recomputes the column client-side. Planned with the `formula` read - * plan's own entry builder ({@link formulaPlanEntry}), so the scale rounding - * and the one hop are the read's. - * - * An empty plan for an object declaring none — the common case — so a write - * on an ordinary object pays a field-name loop and nothing else. - */ -function planStoredExpressions(schema: any): FormulaPlanEntry[] { - const fields = schema?.fields; - if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return []; - const plan: FormulaPlanEntry[] = []; - for (const [name, def] of Object.entries(fields as Record)) { - if (def && typeof def === 'object' && def.type !== 'formula' && def.expression) { - plan.push(formulaPlanEntry(name, def, fields)); - } - } - return plan; -} - -/** [#25073] {@link storedExpressionsReading}'s per-source member reads, bounded like the rule validator's cache. */ -const storedExpressionReadsCache = new Map | null>(); -const STORED_EXPRESSION_READS_CACHE_CAP = 512; - -/** The `record.` names one plan entry reads, or `null` when the platform's member reader cannot say. */ -function storedExpressionReads(entry: FormulaPlanEntry): ReadonlySet | null { - const expr = entry.expression as { dialect?: string; source?: unknown } | undefined; - if (!expr || (expr.dialect !== undefined && expr.dialect !== 'cel') || typeof expr.source !== 'string') return null; - const hit = storedExpressionReadsCache.get(expr.source); - if (hit !== undefined) return hit; - const analysis = analyzeRelationshipTraversals(expr.source); - const reads = analysis ? new Set([...analysis.bareFields, ...analysis.traversals.keys()]) : null; - if (storedExpressionReadsCache.size < STORED_EXPRESSION_READS_CACHE_CAP) storedExpressionReadsCache.set(expr.source, reads); - return reads; -} - -/** - * [#25073] The entries of a stored-expression plan that read one of `fields` - * (the object's `autonumber` fields), directly or through another entry that - * does, in plan order: the stored fields an insert must compute again once - * the record number is issued, because every earlier pass ran before it - * existed. - * - * Reads are the platform's one member reader's (`analyzeRelationshipTraversals`, - * `record.x`, `record.?x`, `record['x']`, inside `has()` too), so this cannot - * disagree with the lint about which spellings name a field. An entry whose - * reads it cannot name (another dialect) counts as a reader: computing it - * twice costs a pass, missing it stores a stale value. - */ -function storedExpressionsReading(plan: FormulaPlanEntry[], fields: ReadonlySet): FormulaPlanEntry[] { - if (plan.length === 0 || fields.size === 0) return []; - const moved = new Set(fields); - const readers = new Set(); - for (let grew = true; grew;) { - grew = false; - for (const entry of plan) { - if (readers.has(entry)) continue; - const reads = storedExpressionReads(entry); - if (reads !== null && ![...reads].some((name) => moved.has(name))) continue; - readers.add(entry); - moved.add(entry.name); - grew = true; - } - } - return plan.filter((entry) => readers.has(entry)); -} - -/** The `autonumber` fields an object declares. */ -function autonumberFieldNames(schema: any): Set { - const names = new Set(); - const fields = schema?.fields; - if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return names; - for (const [name, def] of Object.entries(fields as Record)) { - if (def?.type === 'autonumber') names.add(name); - } - return names; -} - -/** What {@link computeStoredExpressions} reads its permission map, fault sink and hops from. */ -interface StoredExpressionServing { - permissionsFor: (plan: FormulaPlanEntry[], records: readonly unknown[]) => Promise; - onFault?: FormulaFaultSink; - relatedFor?: (plan: FormulaPlanEntry[], records: ReadonlyArray>) => Promise; - /** - * [#26608] The option labels `label()` reads — always the DECLARED ones - * ({@link declaredOptionLabels}): a stored value is computed once and read by - * every later reader and by search, so it never depends on the writer's locale. - */ - optionLabelsFor?: (plan: FormulaPlanEntry[]) => EvalOptionLabels | undefined; -} - -/** - * [#24619] Compute a stored-expression plan over each IMAGE — the row the - * write stores: the stored row merged with the patch on an update, the - * defaulted payload on an insert — and answer each image's computed values, - * index-aligned. Evaluates against shallow COPIES, so an image is never - * mutated; the caller writes the values onto the payload it hands the driver. - * An expression that does not evaluate computes `null`, the formula read - * path's answer, with the fault reported once through `onFault`. - */ -async function computeStoredExpressions( - plan: FormulaPlanEntry[], - images: ReadonlyArray>, - execCtx: ExecutionContext | undefined, - serving: StoredExpressionServing, -): Promise>> { - if (plan.length === 0 || images.length === 0) return images.map(() => ({})); - const scratch = images.map((image) => ({ ...image })); - applyFormulaPlan( - plan, scratch, execCtx, await serving.permissionsFor(plan, scratch), serving.onFault, - await serving.relatedFor?.(plan, scratch), serving.optionLabelsFor?.(plan), - ); - return scratch.map((row) => { - const values: Record = {}; - for (const entry of plan) values[entry.name] = row[entry.name]; - return values; - }); -} - -/** - * [#25073] The stored fields an insert computes again once it holds a record - * number ({@link storedExpressionsReading}), with what computing them reads. - */ -interface AutonumberReaders { - readonly plan: FormulaPlanEntry[]; - readonly execCtx: ExecutionContext | undefined; - readonly serving: StoredExpressionServing; - /** The {@link ObjectQL.registerInsertRecomputeListener} listeners, told per row; absent when none is registered. */ - readonly recomputed?: (row: Record, fields: string[]) => Promise; -} - -/** - * [#25073] Compute the readers again over rows that now carry their issued - * number, and write the values onto the rows the driver is handed — the step - * that makes the stored value the declared expression over the stored row. - * The replaced-value report is NOT made again: it judged the caller's value - * once, before validation and before any number existed. - */ -async function recomputeAutonumberReaders( - readers: AutonumberReaders | undefined, - rows: Array>, -): Promise { - if (!readers || readers.plan.length === 0 || rows.length === 0) return; - const computed = await computeStoredExpressions(readers.plan, rows, readers.execCtx, readers.serving); - rows.forEach((row, k) => Object.assign(row, computed[k])); - if (!readers.recomputed) return; - const fields = readers.plan.map((entry) => entry.name); - for (const row of rows) await readers.recomputed(row, fields); -} - -/** - * [#24619] The one value a PREDICATE update can store for each stored - * computed field, judged over every matched row. `updateMany` writes ONE - * patch to all of them, so a field is set only when the patch moves it to the - * same value on every row; a patch that does not move it at all (it touches - * none of the expression's inputs) leaves the column alone, and a caller-sent - * value for it is taken out of the patch either way — it never stands. A patch - * that moves it to different values on different rows cannot be one - * statement, and is refused before anything is written: update those rows by - * id, where each is computed from its own stored row. - */ -function settleBulkStoredExpressions( - plan: FormulaPlanEntry[], - patch: Record, - before: ReadonlyArray>, - after: ReadonlyArray>, - object: string, -): Record { - const out: Record = { ...patch }; - const refused: FieldValidationError[] = []; - for (const entry of plan) { - delete out[entry.name]; - const moved = after.some((values, i) => !Object.is(values[entry.name], before[i]?.[entry.name])); - if (!moved) continue; - const first = after[0]?.[entry.name]; - if (after.every((values) => Object.is(values[entry.name], first))) { - out[entry.name] = first; - continue; - } - refused.push({ - field: entry.name, - code: 'invalid_value', - message: `'${entry.name}' on '${object}' is computed from its expression on every write, and this ` - + 'update changes its inputs on rows where it then computes different values, which one update of ' - + 'all the matched rows cannot store; update those rows by id, so each is computed from its own row', - }); - } - if (refused.length > 0) throw new ValidationError(refused); - return out; -} - -/** - * [#24619] Write a by-id update's computed values onto its patch: a key the - * caller (or a hook) sent is overwritten, and a key the patch does not carry - * is added only when the computed value differs from the stored one — a patch - * that moves none of the expression's inputs writes nothing extra. - */ -function applyStoredExpressionsToPatch( - patch: Record, - computed: Record, - prior: Record | null | undefined, -): void { - for (const [name, value] of Object.entries(computed)) { - if (Object.prototype.hasOwnProperty.call(patch, name) || !Object.is(prior?.[name], value)) patch[name] = value; - } -} - -/** - * [#24690] The stored computed fields whose CALLER value this write replaced — - * reported through `onFieldsDropped` under `stored_expression`, so a client - * that sent `amount: 1` is told its value did not stand, as #3407 rules for - * every other write-path strip, rather than reading success over a row that - * holds the computed value. - * - * A field is reported when all three hold: - * - the caller SENT it (`supplied`, the write's entry snapshot of the - * caller's values — never the live payload, which the pre-hook pass and the - * hooks have since written to); - * - it is still on the payload the compute pass reads (`present`): a value - * another strip took — a stored expression field also declared `readonly` - * — is that strip's to report, under its own reason, never twice; - * - the value STORED differs from the caller's on at least one row - * (`stored`: the one row of an insert or a by-id update, every matched row - * of a predicate update). A client that sends the computed value — the - * console grid's consistent save — is told nothing, because nothing it - * sent was replaced. - * - * `Object.is`, the comparison the compute passes themselves use: the caller's - * values were normalized (blank and numeric strings) before the snapshot. - */ -function replacedStoredExpressionFields( - plan: readonly FormulaPlanEntry[], - supplied: Readonly> | undefined, - present: Readonly> | null | undefined, - stored: ReadonlyArray>>, -): string[] { - if (!supplied || !present) return []; - const replaced: string[] = []; - for (const { name } of plan) { - if (!Object.prototype.hasOwnProperty.call(supplied, name)) continue; - if (!Object.prototype.hasOwnProperty.call(present, name)) continue; - if (stored.some((row) => !Object.is(row?.[name], supplied[name]))) replaced.push(name); - } - return replaced; -} - -/** - * Materialize ONE declared `formula` field against a record already in hand — - * the read path's own evaluation, narrowed to a single field, with no round - * trip (#11293). - * - * ## Why this exists rather than a second evaluator - * - * A hook body cannot reach a formula field: `ctx.previous` / `ctx.input` carry - * STORED columns, and a formula is computed on read, so a body that wants the - * record's title has to rebuild the formula inline. Measured in the exemplar - * app: five inline reimplementations of a record title inside hook bodies, four - * of them re-composing a `nameField` that is a formula. Each copy can drift - * from the declaration it copies, silently — which is the whole defect, so the - * remedy must not itself be a copy. This calls - * {@link planFormulaProjection} + {@link applyFormulaPlan}: the same plan - * builder, the same `Expression` normalization (string shorthand → CEL - * envelope), the same `scale` rounding and the same evaluation scope the read - * and write paths use. One formula semantic, not a hook-path dialect (PD #12). - * - * ## Narrowed to one field ON PURPOSE - * - * `planFormulaProjection(schema, undefined)` — the shape `find` and - * {@link hydrateWriteFormulas} use — plans EVERY formula field on the schema - * and `ExpressionEngine.compile`s each one at planning stage. Asking for one - * title would then throw on an unrelated malformed formula elsewhere on the - * object. Passing `[field]` plans exactly the requested field, so the blast - * radius of a title lookup is the title's own declaration. - * - * ## Read-path parity, including how it fails - * - * Both failure modes are the read path's, unchanged: a formula that does not - * COMPILE throws (as it does on every `find` of the object), while a formula - * that compiles and does not EVALUATE yields `null` — `applyFormulaPlan`'s own - * `r.ok ? … : null`. A caller therefore cannot mistake "this title could not be - * computed" for a computed value. - * - * ⚠️ One scope difference, named rather than hidden [#20082]: this helper is - * synchronous and passes NO permission map, so a formula that calls - * `current_user.can(…)` does not evaluate here and yields `null` by the rule - * above. [#24197] For the same reason it preloads no relationship hop, so a - * formula reading `record..` yields `null` here too. The read and write paths resolve the map first - * ({@link ObjectQL.resolveFormulaPermissions}); a helper with no resolver cannot. - * [#26608] And it holds no i18n service, so `label(record.)` reads the - * DECLARED option labels here ({@link declaredOptionLabels}), whatever - * `execCtx.locale` says. - * A caller that holds a read channel and wants the read's answer calls - * {@link ObjectQL.serveFormulaField} (through `ScopedContext.serveFormulaField`). - * - * Returns `undefined` when `field` is not a declared formula field, which is - * how a caller tells "read the stored column instead" from "the formula - * produced nothing". Evaluates against a shallow COPY: `applyFormulaPlan` - * writes the value onto the record it is handed, and the records reaching here - * are the engine's own hook payloads, observed by everything downstream. - */ -export function evaluateFormulaField( - schema: unknown, - record: Record, - field: string, - execCtx?: ExecutionContext, -): unknown { - const { plan } = planFormulaProjection(schema as any, [field]); - if (plan.length === 0) return undefined; - const scratch: Record = { ...record }; - // [#24197] Synchronous, so it preloads no hop: a formula reading through a - // reference yields `null` here, as a `can()` formula does above. - applyFormulaPlan(plan, [scratch], execCtx, undefined, undefined, undefined, declaredOptionLabels(schema, plan)); - return scratch[field]; -} - -/** - * A hook body, as registered through {@link ObjectQL.registerHook} or bound - * from metadata by `bindHooksToEngine`. - * - * ## `after*` handlers run INSIDE the unit of work (#7477) - * - * An `afterInsert` / `afterUpdate` / `afterDelete` handler is dispatched - * before the enclosing transaction commits. The guarantee it may rely on is - * **"the write has been requested and will happen unless this unit of work is - * undone"** — not "the write happened": a later refusal in the same unit rolls - * the row back after this handler has already run. See - * {@link DISPATCHABLE_HOOK_EVENTS} for the full statement and for the paths - * that open such a unit; a handler whose side effects leave the engine is the - * one that has to tolerate it. - */ -export type HookHandler = (context: HookContext) => Promise | void; - -/** - * Per-object hook entry with priority support - */ -export interface HookEntry { - handler: HookHandler; - object?: string | string[]; // undefined = global hook - /** - * [#5928] Object name(s) SUBTRACTED from whatever `object` admits — the - * negative half of a registration's scope. Absent = subtract nothing. - * - * `object` alone can only say "these objects" (plus the `'*'` full set), and - * the two are interchangeable only over a CLOSED universe. This one is open: - * `applyObjectRegistryMutation` registers objects into a running engine on a - * successful `/meta` PUT, emitting no event a plugin could subscribe to, so a - * registrant that enumerated the complement of its skip list would keep a - * frozen list and silently stop covering every object registered after boot. - * For the audit plugin that is a compliance regression, and a silent one. - * - * Read only through {@link hookMatchesObject} — never re-implemented at a - * call site (see that function's note on the one-semantic rule). - */ - excludeObjects?: string | string[]; - priority: number; - packageId?: string; - /** - * [#9719, widened to `beforeUpdate` by #9974] Opt-in: ALSO dispatch this - * handler ONCE with the whole-operation context when a predicate - * (`multi: true`) WRITE arrives with no caller predicate at all - * (`options.where` absent or `null`) — BEFORE any matched row is resolved. - * - * Why the extra dispatch exists: since #5038/#5574 the predicate path - * dispatches `beforeUpdate` / `beforeDelete` per matched row, with - * `input.id` bound — so a guard that refuses the OPERATION SHAPE (the #4757 - * unscoped-multi-delete refusal on `sys_attachment`, the #4630 one on - * `sys_comment`) never saw the shape it guards: the by-id branch of the - * handler shadowed it on every dispatch, and a zero-match unscoped write - * dispatched nothing at all. "Nothing to authorize" and "nothing was ever - * queried" are different verdicts, so the shape check cannot ride the - * per-row fan-out; it needs the one dispatch that happens regardless of what - * the predicate matches. - * - * ## Why it covers both write verbs - * - * #9719 shipped it on `beforeDelete` only, and refused it elsewhere, because - * extending it to `beforeUpdate`'s predicate path changes what the engine - * ACCEPTS on the far hotter update path — a product-behaviour decision, not - * drift. That decision was taken (#9974, maintainer ruling 2026-08-19, - * option A) on the recoverability asymmetry: a delete leaves a trace of who - * removed what, an overwrite leaves none — the old value is gone on the spot - * with nothing to restore from — and a forgotten `where` is the mistake - * generated code makes most often. So the LESS guarded verb was the one - * whose failure is LESS recoverable. One flag now covers both, rather than a - * `dispatchUnscopedMultiUpdate` sibling: the flag is already per-REGISTRATION - * and per-EVENT, so "delete only" is still said exactly — declare the flag on - * the `beforeDelete` registration and omit it from the update one, and the - * whole-operation call is then dispatched for the declared verb alone (the - * dispatcher reads the entries of ONE event, then filters those on the flag). - * - * ⚠️ That idiom is deliberately stated WITHOUT naming a live registration. - * It used to name one: `sys_attachment` declared #4757's delete refusal on - * `beforeDelete` and on no update registration — true when this paragraph was - * written (#9974), false since commit da891e0ef gave that guard's update verb a refusal - * of its own. A replacement exemplar would only be the next referent free to - * move, so none is named: the past-tense sentence cannot be falsified by the - * tree moving on, because it is a claim about what those two cards did rather - * than about today's registrations. - * - * Deliberately a REGISTRATION declaration, not an engine-wide rule: the - * engine stays neutral (no behaviour change for objects whose guards do not - * declare it — generalizing the refusal to every guarded object is a - * separate product decision), and the policy — who is refused, with what - * error — stays in the ONE handler that already declares and tests it. Valid - * on `beforeUpdate` / `beforeDelete` registrations only - * ({@link assertValidUnscopedMultiWriteFlag}); dispatched by - * {@link ObjectQL.dispatchUnscopedMultiWriteHooks}. - */ - dispatchUnscopedMultiWrite?: boolean; - /** - * Original metadata-form `Hook` definition this entry was bound from - * (when registered via `bindHooksToEngine`). Pure code-paths that call - * `engine.registerHook` directly leave this undefined. - */ - meta?: any; - /** Hook `name` from metadata; used for diagnostics & deduplication. */ - hookName?: string; -} - -/** `object` / `excludeObjects` in either accepted spelling → a name list. */ -function hookTargetList(target: string | string[] | undefined): string[] { - if (target === undefined) return []; - return Array.isArray(target) ? target : [target]; -} - -/** - * [#5928] Does `entry` cover `objectName`? **The** answer — one function, both - * consumers. - * - * ## Why this is a function and not two `if`s - * - * `triggerHooks` (dispatch) and `hasHooksFor` (the #5038 / #5284 bulk-write - * gate that decides whether the matched row set is worth reading) used to carry - * this rule as two hand-written copies of one semantic. `hasHooksFor`'s own - * comment named the hazard that arrangement carries: a gate LOOSER than the - * dispatch only wastes a query, but a gate TIGHTER than the dispatch silently - * drops hooks that were going to fire. Two copies stay in agreement only for as - * long as everyone who edits one remembers the other, and #5928 adds a second - * dimension to the rule — exactly the edit that would have desynchronised them. - * So the copies are gone: both consumers call this, and the property test in - * `hook-exclude-objects.test.ts` pins the implication (`dispatched ⇒ gate open`) - * over the full allow × exclude matrix rather than trusting the structure. - * - * ## The semantic - * - * `matches = allowMatches && !excludeMatches`, where an absent `object` admits - * everything (global) and `'*'` in `object` does the same explicitly. - * `excludeObjects` names are matched literally — `'*'` is refused at - * registration, so it can never reach this as a subtract-everything wildcard. - * - * The allow half keeps the TRUTHINESS test both copies used verbatim, rather - * than the `!== undefined` that reads more precisely. They differ on exactly one - * input, `object: ''`, which reads here as a GLOBAL hook (falsy ⇒ no filter). - * That read is deliberately UNCHANGED, and commit 708431313 says why: flipping it would turn - * a hook firing on everything into one firing on nothing, silently — the same - * class of defect pointing the other way. The shape is closed at the - * registration door instead ({@link assertValidHookObject}), so no live entry - * can carry `''` and this branch is unreachable for it in practice. It stays - * because this function is also called directly on hand-built entries, and - * because a matcher should describe one rule, not re-litigate the door's. - */ -export function hookMatchesObject( - entry: Pick, - objectName: string, -): boolean { - // Allow half: absent = global; otherwise the wildcard or a literal name. - if (entry.object) { - const targets = hookTargetList(entry.object); - if (!targets.includes('*') && !targets.includes(objectName)) return false; - } - // Subtract half: any literal hit removes the object from the admitted set. - if (entry.excludeObjects) { - if (hookTargetList(entry.excludeObjects).includes(objectName)) return false; - } - return true; -} - -/** - * [#5928] Registration-time refusal for the `excludeObjects` face, following - * the sister ruling on empty hook targets (#4281 / #4001, ADR-0078 "no silently - * inert declaration"). - * - * Two shapes are refused, both statically decidable at the call site: - * - * - **`''` / `['']`** (or any blank member) — no object is named `''`, so the - * entry subtracts nothing while reading as though it subtracts something. - * Inert, and inert in the direction that quietly keeps a hook firing on - * objects its author believed they had excluded. - * - **`'*'` anywhere in the list** — subtracting the full set from any allow - * set leaves the empty set: a hook that can never fire, registered - * "successfully". That is ADR-0078's silently-inert declaration exactly, and - * the same never-fire shape #4281 refused when `['']` produced it from the - * other side. - * - * `[]` is deliberately ACCEPTED. It is the honest spelling of "subtract - * nothing" — identical in meaning to omitting the key, and the natural value of - * a spread (`excludeObjects: [...SKIP_OBJECTS]`) whose source list is empty. - * The refusal above is not "empty is bad"; it is "a name that matches nothing" - * and "a name that matches everything". #4281 refused `[]` on `object` for a - * reason that does not exist here: there, the binder WIDENED the empty target - * into `'*'`, so blank intent silently became the maximum blast radius. Nothing - * transforms this value. - * - * A throw, not a warn: `excludeObjects` ships in this release with no callers, - * so strictness costs no migration, and unlike the unknown-event branch above - * (where a custom driver dispatching its own events is a legitimate reading) - * neither refused shape has one. - */ -function assertValidHookExcludeObjects( - excludeObjects: string | string[] | undefined, - event: string, -): void { - if (excludeObjects === undefined) return; - const names = hookTargetList(excludeObjects); - for (const name of names) { - if (typeof name !== 'string' || name.trim().length === 0) { - throw new Error( - `[ObjectQL] Hook '${event}' declares an empty \`excludeObjects\` entry. ` - + 'No object is named \'\', so it would subtract nothing while reading as if it ' - + 'subtracted something — a hook still firing on objects it appears to exclude. ' - + 'Name the object(s) to exclude — `excludeObjects: [\'sys_audit_log\']` — or omit ' - + 'the option entirely (an empty array is accepted and subtracts nothing).', - ); - } - if (name === '*') { - throw new Error( - `[ObjectQL] Hook '${event}' excludes the wildcard '*', which subtracts every ` - + 'object and leaves a hook that can never fire (ADR-0078: no silently inert ' - + 'declaration). Exclude the object names you mean — ' - + '`excludeObjects: [\'sys_audit_log\']` — or, if the hook really should not be ' - + 'registered, do not register it.', - ); - } - } -} - -/** - * [commit 708431313] Registration-time refusal for the `object` (ALLOW) face, closing - * #4281 / #4001's "an empty target is not *no* target" ruling on the path that - * ruling never reached. - * - * #4281 shut this shape at the two METADATA doors — `HookSchema.object`'s - * refine in `packages/spec`, and `normalizeObjects` in `hook-binder.ts`. The - * code door, `engine.registerHook`, goes through neither, so the same three - * spellings still walked in here, and the matching read (see - * {@link hookMatchesObject}) turns each of them into a defect: - * - * - **`''`** is FALSY, so the allow half is skipped entirely and the entry - * registers as a GLOBAL hook. #4281's headline failure mode exactly — blank - * intent becoming the broadest possible blast radius — reproduced verbatim - * on the uncovered path. - * - **`[]` / `['']`** (or any blank member) are truthy but admit no object - * name, so the entry can never fire: registered "successfully", inert - * forever. ADR-0078's silently-inert declaration. - * - * Refused at REGISTRATION rather than fixed in the matcher, and that choice is - * the whole point of the separate card. Teaching `hookMatchesObject` to read - * `''` as a real (unmatchable) name would silently convert a hook firing on - * every object into one firing on none — the same class of defect pointing the - * other way, which is why #5928 declined to do it in passing. A throw at the - * door changes no dispatch and leaves nothing to misread. - * - * Note the asymmetry with {@link assertValidHookExcludeObjects}, which - * deliberately ACCEPTS `[]`: on the subtract face an empty list is the honest - * spelling of "subtract nothing", identical to omitting the key. On the allow - * face an empty list says "admit nothing", which is a hook that can never fire. - * Same value, opposite meaning, because the faces compose in opposite - * directions — and `[]` is named in #4281's own message, so accepting it here - * would contradict the ruling this reuses. - */ -function assertValidHookObject( - object: string | string[] | undefined, - event: string, -): void { - if (object === undefined) return; - const names = hookTargetList(object); - const empty = - names.length === 0 - || names.some((name) => typeof name !== 'string' || name.trim().length === 0); - if (!empty) return; - throw new Error( - `[ObjectQL] Hook '${event}' declares an empty \`object\` target. An empty target is ` - + 'not "no target": `\'\'` is falsy, so the allow face is skipped entirely and the hook ' - + 'registers on EVERY object, while `[]` and `[\'\']` admit no object name at all, so the ' - + 'hook could never fire (ADR-0078: no silently inert declaration). Name the object(s) — ' - + "`object: 'account'` or `object: ['account', 'contact']` — or, if firing on every " - + "object really is the intent, write the wildcard explicitly: `object: '*'`.", - ); -} - -/** - * [commit 708431313] Refuse a scope whose two faces cancel each other out — - * `{ object: 'account', excludeObjects: 'account' }` and its list forms. - * - * The exclusion face subtracts from the allow face, so when the allow face is a - * FINITE enumeration and every name in it is also excluded, the admitted set is - * empty and the entry can never fire. #5928 named only three refusals (`''`, - * `['']`, and `'*'` in the excludes) and this shape falls outside their letter, - * so it was left registering silently — the same ADR-0078 inert declaration - * those three exist to prevent, reached by arithmetic instead of by a single - * bad name. - * - * Only a finite allow face can be decided here. `'*'` (and an absent `object`) - * admits an OPEN universe — `applyObjectRegistryMutation` registers objects - * into a running engine — so no finite exclusion list can empty it, and those - * scopes are left alone. `'*'` inside `excludeObjects` is the one exclusion - * that WOULD empty them, and it is already refused by - * {@link assertValidHookExcludeObjects}. - * - * Runs after both faces have been validated individually, so every name here is - * a non-blank string and the exclusion list carries no wildcard. - */ -function assertHookScopeNotSelfCancelling( - object: string | string[] | undefined, - excludeObjects: string | string[] | undefined, - event: string, -): void { - if (object === undefined || excludeObjects === undefined) return; - const allow = hookTargetList(object); - // An open allow face cannot be emptied by a finite subtraction. - if (allow.length === 0 || allow.includes('*')) return; - const deny = hookTargetList(excludeObjects); - if (deny.length === 0) return; - const denied = new Set(deny); - if (!allow.every((name) => denied.has(name))) return; - throw new Error( - `[ObjectQL] Hook '${event}' excludes every object its \`object\` target admits ` - + `(object: ${JSON.stringify(allow)}, excludeObjects: ${JSON.stringify(deny)}), leaving ` - + 'a hook that can never fire (ADR-0078: no silently inert declaration). An exclusion ' - + 'subtracts from a WIDER allow face — widen `object` (or drop it for a global hook) or ' - + 'remove the overlapping names from `excludeObjects`; if the hook really should not be ' - + 'registered, do not register it.', - ); -} - -/** - * [#9719, widened by #9974] The events whose predicate branch actually reads - * {@link HookEntry.dispatchUnscopedMultiWrite}. Declared as data beside the - * assert so the refusal message and the accept set cannot drift apart, and so - * adding a third verb is one edit rather than a rule to restate. - */ -const UNSCOPED_MULTI_WRITE_EVENTS = ['beforeUpdate', 'beforeDelete'] as const; - -/** - * [#9719, widened by #9974] Registration-time refusal for - * {@link HookEntry.dispatchUnscopedMultiWrite} on any event whose dispatch - * never reads it. - * - * The flag is consumed by exactly two dispatch sites — the predicate branches - * of `ObjectQL.update()` and `ObjectQL.delete()` — so on every other event it - * would register "successfully" and never fire: ADR-0078's silently inert - * declaration, the same shape the two scope asserts above refuse. Statically - * decidable, so refused at the door rather than warned about. - * - * ⚠️ The accept set widened by RULING, not to make the assert quieter: until - * #9974 this refused `beforeUpdate` too, and its comment named that as "a - * product-behaviour decision of its own". The decision was taken (option A, - * 2026-08-19), so `beforeUpdate` is now a declared reader — and every OTHER - * event is still refused on exactly the original grounds. - */ -function assertValidUnscopedMultiWriteFlag( - dispatchUnscopedMultiWrite: boolean | undefined, - event: string, -): void { - if (!dispatchUnscopedMultiWrite) return; - if ((UNSCOPED_MULTI_WRITE_EVENTS as readonly string[]).includes(event)) return; - throw new Error( - `[ObjectQL] Hook '${event}' declares \`dispatchUnscopedMultiWrite\`, which only the ` - + `${UNSCOPED_MULTI_WRITE_EVENTS.map((e) => `'${e}'`).join(' / ')} dispatches read — on this ` - + 'event the flag would register successfully and then never fire (ADR-0078: no silently ' - + 'inert declaration). Register the guard on one of those events, or drop the flag.', - ); -} - -/** - * [#17713] The per-seam prescription for a refused lifecycle event — what the - * author should have written instead, named concretely. - * - * The two seams are different repairs, and an author who is told only "this - * never fires" cannot tell them apart: - * - * - **`*FindOne`** — the capability EXISTS and the author already has it. - * `beforeFind`/`afterFind` fire for `findOne` too (the event attaches to - * record materialization, not to the engine method), so the fix is a - * one-word rename and nothing is lost. - * - **`*Count` / `*Aggregate`** — there is no hook seam at all, by design - * (`HookEvent` in `@objectstack/spec/data` says so: read authorization and - * row filtering are the middleware's job). `count()` and `aggregate()` run - * the middleware chain with the query AST on the operation context, so the - * repair is a DIFFERENT API, not a different event name. This is the half - * that would have saved the consumer on #17713: their read filter sat on - * `beforeCount`, so a `limit`ed list's `total` counted rows the caller could - * not see, and a `groupBy` was not narrowed at all. - */ -function undispatchedLifecycleEventPrescription(event: string): string { - if (event === 'beforeFindOne' || event === 'afterFindOne') { - const covering = event === 'beforeFindOne' ? 'beforeFind' : 'afterFind'; - return ( - `\`${covering}\` already covers \`findOne\` — the read events fire for BOTH \`find\` and ` - + `\`findOne\`, because the event attaches to record materialization, not to the engine ` - + `method. Register on '${covering}' instead; the handler sees single-record reads there.` - ); - } - const verb = event.startsWith('before') ? event.slice(6) : event.slice(5); - const method = `${verb.charAt(0).toLowerCase()}${verb.slice(1)}`; - return ( - `\`${method}()\` dispatches no hook at all, by design — it runs the MIDDLEWARE chain only, ` - + 'with the query AST on the operation context. Filter it with ' - + '`engine.registerMiddleware(fn)`, reading `ctx.operation === \'count\' | \'aggregate\'` and ' - + 'composing your predicate onto `ctx.ast.where` — the same seam RLS and sharing already use, ' - + 'so the scoping applies to the driver call itself. ⚠️ A read filter written as a HOOK does ' - + 'not narrow a `total` or a `groupBy`: those surfaces would keep answering over rows the ' - + 'caller cannot see.' - ); -} - -/** - * [#17713] Registration-time refusal for an event inside the engine's OWN - * lifecycle namespace that the engine never dispatches — see - * {@link UNDISPATCHED_LIFECYCLE_HOOK_EVENTS} for how that population is derived - * and why the refusal stops there. - * - * This was a `warn` from #3195 until #17713. The warning was loud (~40 lines a - * boot on the reporting install) and still useless: the registration SUCCEEDED, - * so a downstream consumer's read filters on `beforeFindOne` and `beforeCount` - * sat inert through every boot while the surface reported them registered. That - * is ADR-0078's prohibited fourth state — parsed, unmarked, silently inert — on - * an authorable seam, and on this seam the inert declaration is a GUARDRAIL: the - * author believes they narrowed what a caller can see, and they did not. - * - * A throw rather than a strict-mode-only failure, on the same reading as the - * four asserts above: the shape is statically decidable at the call site, it - * carries no recoverable intent that a warning could preserve, and a mode nobody - * turns on is a rule nobody gets. The `after*` names are refused alongside the - * `before*` ones even though the card only measured the `before*` half — they - * are the same gap, and refusing half a derived set would reintroduce the - * hand-written list. - * - * ⛔ NOT a general "unknown event names are refused". An event outside the - * namespace is left exactly as it was — a warn, then registered — so a plugin - * that dispatches its own events through the public {@link ObjectQL.triggerHooks} - * keeps working. That is the reading #3195 wrote down as the reason not to - * reject, and it is untouched here: this narrows the accept set by six names, - * not to eight. - */ -function assertDispatchableHookEvent(event: string): void { - if (!UNDISPATCHED_LIFECYCLE_HOOK_EVENTS.has(event)) return; - throw new Error( - `[ObjectQL] Hook '${event}' is an engine lifecycle event name the engine never dispatches, ` - + 'so the registration would succeed and the handler would never run (ADR-0078: no silently ' - + `inert declaration). ${undispatchedLifecycleEventPrescription(event)} ` - + `Dispatched events: ${[...DISPATCHABLE_HOOK_EVENTS].join(', ')}.`, - ); -} - -/** Function registry entry — see `registerFunction`. */ -export interface FunctionEntry { - handler: HookHandler; - packageId?: string; - /** - * What this function does to data, as DECLARED at registration (#4396). - * Only a `script`-node caller reads it: a flow function is contractually - * pure, and the run summary counts on that, so a function that legitimately - * writes declares `'writes'` and its step is reported as an effect the - * platform cannot count rather than as none. Absent ⇒ `'pure'`. - * - * Carried on the registry entry rather than beside it because this IS part of - * the registration — one registry, and what a caller needs to know about a - * function travels with the function. - */ - effect?: FlowFunctionEffect; -} - -/** - * Declarations that may accompany a `registerFunction` call. The bare - * `packageId` string is still accepted in that position (every existing caller - * passes one), so this widens the signature without a migration. - */ -export interface FunctionRegistrationOptions { - /** Owning package — the unit `unregisterFunctionsByPackage` removes. */ - packageId?: string; - /** Declared data effect (#4396); omit for the pure default. */ - effect?: FlowFunctionEffect; -} - -/** - * Operation Context for Middleware Chain - */ -export interface OperationContext { - object: string; - operation: 'find' | 'findOne' | 'insert' | 'update' | 'delete' | 'count' | 'aggregate'; - ast?: QueryAST; - data?: any; - options?: any; - context?: ExecutionContext; - result?: any; - /** - * [#15813 / ADR-0131 D8] What the Layer 0 tenant wall DECIDED for this - * operation — recorded by the enforcement layer (`@objectstack/plugin-security`) - * at the moment it composes the wall onto the operation's predicate, and - * read by the bulk data-event producer ({@link ObjectQL.publishBulkDataEvent}) - * to stamp `BulkDataEvent.organizationId`. - * - * The seam ruled on #15706: the wall is computed ONCE, where every input is - * visible (the posture in force, the caller's organization scope and the - * object's tenancy clauses — which, since ADR-0131 D7, include the - * deployment's #12699 platform-global declaration, recorded on the - * registered object as `systemFields.tenant: false`), and its decision - * travels here as a value. A reader - * answers from this member ALONE and re-derives nothing — a re-derivation - * is a mirror of the wall, and a mirror structurally sees only the clauses - * it was taught (the #15706 mislabel). - * - * Three states, all honest: ABSENT — no wall was composed on this - * operation (a system context, no enforcement layer mounted, a by-id write - * with no predicate); `{ kind: 'none' }` — the wall ran and contributed no - * predicate; a named verdict (`organization` / `organizations` / `deny`). - * The shape is `TenantLayer0VerdictSchema` (`@objectstack/spec/security`); - * a value that does not parse reads as absent, never as an organization. - */ - tenantLayer0Verdict?: TenantLayer0Verdict; - /** - * [commit a016f08b8] The INSERT post-image seam — where an enforcement layer gets to - * judge the row that will actually be stored. - * - * An `insert` has no pre-image, so a middleware's only image of the write is - * `opCtx.data`: the caller's payload as it arrived, before `applyFieldDefaults` - * and before the `beforeInsert` hooks. A value the app DERIVES server-side — - * an organization copied from a parent, a denormalised scoping field a caller - * is deliberately not allowed to choose — is not on that image, so a predicate - * over it judges a value that never lands and ignores the one that does. - * Measured on 17.3.0: the same identity, same object, same second, an insert - * carrying the stamped value was admitted and the identical insert leaving it - * to the hook was refused, while the stored row was identical either way — and - * an insert naming an IN-SCOPE value for a parent in ANOTHER organization was - * admitted with the parent's organization stored on it. - * - * So the enforcement layer installs its judgement here instead of running it - * against `opCtx.data`, and {@link ObjectQL.insert} calls it once the - * `beforeInsert` chain has produced the row — before the first producer with - * a side effect (the secret channel, the autonumber, the statement), so a - * refusal still costs nothing. An ARRAY insert is one operation: every live - * row is judged in the same call. - * - * [#19950] A PREDICATE `update` (`multi: true`, no row address) uses the same - * seam. The middleware cannot know which rows the write will change: they - * are the rows the COMPOSED AST selects, and that AST is complete only after - * every middleware has run. So {@link ObjectQL.update} calls it on that path, - * once the payload is final, with every matched row merged with the payload. - * - * [#19989] A by-id `update` uses it too. A middleware can read that one row, - * but only BEFORE the `beforeUpdate` chain runs, so its image is the change - * set as sent and a hook that rewrites a judged field is never judged. So - * {@link ObjectQL.update} calls it on the by-id path as well, once the - * payload is final, with the prior row merged with the payload. - * - * ABSENT is the ordinary state — no enforcement layer is mounted, or the - * write is one it does not gate. The engine never invents one. - */ - postHookWriteImageCheck?: PostHookWriteImageCheck; - - /** - * [ADR-0142 D4] The by-id write PRECONDITION: an equality predicate over the - * stored row (`{ : }`) the write is conditional on. Set by an - * enforcement layer's middleware — the ownership gate sets `{ owner_id: null }` - * on a claim, so two members claiming at once produce exactly one owner — and - * by the engine's own assignment cursor (`{ version }`). Engine-internal: - * reachable from no data door, and read only on a by-id `update`. - * - * The engine lowers the write to the driver's conditional - * `updateMany(object, { where: { id, ...expect } }, data)`; one affected row - * is re-read by id and the by-id branch continues unchanged (hooks, formulas, - * the event, the roll-up, the response); zero rows answer 409 - * `CONCURRENT_UPDATE` ("the record was taken"). A driver with no `updateMany` - * degrades to the pre-image check its setter already made, and the write - * goes through unconditionally, logged once per process at `warn`. - */ - writePrecondition?: Readonly>; -} - -/** - * [commit a016f08b8] The judgement {@link OperationContext.postHookWriteImageCheck} - * carries, and the acknowledgement its installer reads back. - * - * `evaluate` receives the images the driver is about to store, and REFUSES by - * throwing. On an `insert` those are the rows exactly as the `beforeInsert` - * chain left them, only the live ones (a row the declared-field door culled - * from a partial batch is never judged: it will not be written). On a - * predicate `update` they are the matched rows, each merged with the final - * payload; on a by-id `update` ([#19989]) the one prior row merged with the - * final payload. It is called at most once per operation. - * - * `honoured` is set by the engine immediately before `evaluate` runs. It exists - * so the installer can fail CLOSED on a seam that was never called: an - * enforcement layer that moved its gate here and finds the flag unset on the - * way out knows its check did not happen, and says so loudly rather than - * reading an unjudged write as an allowed one. - */ -export interface PostHookWriteImageCheck { - evaluate(rows: readonly Record[]): void | Promise; - honoured?: boolean; - /** - * [#24822] Declared by the installer, and only by it: `evaluate` holds every - * row it is handed to the Layer 0 tenant wall on `organization_id`. It - * admits the value or refuses the write by throwing. On an `insert` those - * rows are the inserted rows; on an `update` they are the stored images: - * the one prior row merged with the final payload on the by-id path, and - * every matched row merged with it on the predicate path. - * `@objectstack/plugin-security` sets it at step 3.7, only where it - * composes the wall's own parts into the insert's or the single-payload - * update's seam. A seam that carries a business `check` alone does not - * judge the organization, and neither does a platform operator's seam that - * crosses the wall, so neither sets it; nor is it set on an update whose - * payload is an array, where step 3.7 installs no seam of its own. - * - * {@link ObjectQL.insert} and {@link ObjectQL.update} read this member, and - * nothing else, to decide one thing: whether a non-system caller's explicit - * `organization_id` stays on the row for that judgement or is taken by the - * static-`readonly` strip (the platform's tenant column declares - * `readonly: true`). Set, the value is stored as sent where the wall admits - * it and refused where it does not: an insert's is never silently replaced - * by the active organization (ADR-0105 D5), and an update's re-point is - * never silently dropped in favour of the row's own organization. Only the - * platform's injected column is spared, and only on an object the insert's - * static-`readonly` strip judges, so never a `sys_` or platform-internal - * object. An AUTHORED `organization_id` declared `readonly: true` keeps the - * strip. An emptied value on an update is not spared into storage: the wall - * refuses it (`null` or `''`), on the payload and on the stored image. - * Absent or `false`, the strip runs exactly as before. The engine never - * sets it, and never derives it from the posture or the context: that - * would be a mirror of the wall, and a mirror is wider than the wall - * wherever the wall is not mounted. [#25062] The write preview, which - * installs no seam, reads the same value from the wall's verdict on each - * row instead ({@link WriteWallRowVerdict}, `ObjectQL.registerWriteWallVerdict`). - */ - judgesOrganization?: boolean; -} - -/** - * Trailing options for the READ methods (find / findOne / count / aggregate). - * - * Historically the read methods took their execution context INSIDE the query - * (`query.context`), while the WRITE methods (insert / update) took it in a - * trailing `options.context`. That split was a footgun: the same `{ context }` - * object is correct as the 3rd arg to `insert` but was SILENTLY DROPPED as the - * 3rd arg to `find` — a class of bugs where an intended `isSystem` bypass just - * vanished (e.g. control-plane reads coming back empty once org-scoping hooks - * were added). We now ALSO accept `context` via this trailing options arg on the - * read methods, so "execution context goes in the trailing options argument" is - * one rule across reads and writes. `query.context` remains supported; when both - * are given, `options.context` wins (it is the explicit channel). - */ -export interface EngineReadOptions { - context?: ExecutionContext; -} - -/** - * [#22445] Did the read door serve this value AS STORED — can the caller read - * the stored value in full? The `update`-mode preview keeps a stored column - * only when this holds for it (see `storedRows` in `ObjectQL.validate`), and - * [#22474] the preview keeps a master-detail header column only when this - * holds for it (see `ObjectQL.resolvePreviewParents`). - * - * Both values come off the same driver read shape (`driver.findOne` under the - * same `buildDriverOptions`; for a header, the read door's `find` under the - * caller's context and under the write's elevated one), so a column nothing - * transformed compares equal by construction; what differs is what the read - * path REWROTE for this caller (a partial mask, a masked secret, a read hook's - * rewrite). Structural, not identity: the two reads are two driver calls, so a - * `Date`, an array or a JSON object arrives as two equal objects. - * - * ⛔ Fails CLOSED: a pair it cannot prove equal — another object kind (a - * buffer, a class instance), a nesting past the depth bound — answers `false`, - * and the column is judged as empty. That direction can only make the preview - * refuse or admit on less than the write knows; the other direction would - * judge a value the caller may not read. - */ -export function servedAsStored(served: unknown, stored: unknown, depth = 0): boolean { - if (Object.is(served, stored)) return true; - if (depth >= 16) return false; - if (served instanceof Date || stored instanceof Date) { - return served instanceof Date && stored instanceof Date && served.getTime() === stored.getTime(); - } - if (Array.isArray(served) || Array.isArray(stored)) { - return Array.isArray(served) && Array.isArray(stored) && served.length === stored.length - && served.every((item, i) => servedAsStored(item, stored[i], depth + 1)); - } - const plain = (v: unknown): v is Record => { - if (v === null || typeof v !== 'object') return false; - const proto = Object.getPrototypeOf(v); - return proto === Object.prototype || proto === null; - }; - if (!plain(served) || !plain(stored)) return false; - const servedKeys = Object.keys(served); - return servedKeys.length === Object.keys(stored).length - && servedKeys.every((k) => Object.prototype.hasOwnProperty.call(stored, k) && servedAsStored(served[k], stored[k], depth + 1)); -} - -/** Merge read-path execution context from the query and the trailing options. */ -export function mergeReadContext( - fromQuery?: ExecutionContext, - fromOptions?: ExecutionContext, -): ExecutionContext | undefined { - if (fromOptions == null) return fromQuery; - if (fromQuery == null) return fromOptions; - return { ...fromQuery, ...fromOptions }; -} - -/** - * True when this write is exempt from the `state_machine` validation rule — - * both the insert `initialStates` entry check and the update `transitions` - * check are skipped for it. Either the seed-specific `seedReplay` flag (#3433) - * or the general `skipStateMachine` flag (#3479, set by the REST import runner - * for a "historical" import) turns it off. Both are server-set, never - * client-supplied. - */ -export function shouldSkipStateMachine(ctx?: ExecutionContext): boolean { - return ctx?.seedReplay === true || ctx?.skipStateMachine === true; -} - -/** - * Engine Middleware (Onion model) - */ -export type EngineMiddleware = ( - ctx: OperationContext, - next: () => Promise -) => Promise; - -/** - * "Which of these tombstoned `sys_file` rows is something still holding?" - * (commit c3c72a4bc). - * - * Takes the whole tombstoned set from ONE record read and returns the subset - * still held, as a set of stringified ids. Batched rather than per-row on - * purpose: hydration runs over many rows per read, so asking per file would be - * N queries per read. - * - * The engine declares this shape but never implements it — "still held" has one - * definition (`findFileHolder`, the union of `sys_attachment` join rows and the - * `ref_*` ownership columns) and it lives in the storage package. See - * `ObjectQL.registerHeldFileResolver`. - */ -export type HeldFileResolver = ( - rows: Array>, -) => Promise>; - -/** - * "Of these `ownerObject` records, whose field-owned files may this caller - * read?" — the download door's field-owned verdict (ADR-0104 D3; #22637, - * maintainer ruling B on #22624). - * - * Takes every owner record id one hydration needs judged and returns the - * subset whose files the caller may read, as stringified ids. Batched for the - * same reason as {@link HeldFileResolver}: one call per read, never one per - * row. - * - * The engine declares this shape but never implements it. The verdict — a - * declared `fileAccessDelegate` asked and failing closed, otherwise the - * caller's read of the owner record — has one definition, the field-owned arm - * of the storage package's download authorizer, and a copy here would be a - * second predicate for one file. See `ObjectQL.registerFieldOwnedFileReadAuthorizer`. - */ -export type FieldOwnedFileReadAuthorizer = ( - ownerObject: string, - ownerIds: string[], - context: ExecutionContext, -) => Promise>; - -/** - * [#25062] The Layer 0 tenant write wall's answer for ONE row of a write - * preview ({@link ObjectQL.validate}): admitted, saying whether the wall judges - * the row's `organization_id` (the value - * {@link PostHookWriteImageCheck.judgesOrganization} is set from on the - * write), or refused with the write's own `code` and `message`. A refused - * `field` is `organization_id` when the wall refused a value the row carries, - * and `_record` when it refused the caller's state (no active organization). - */ -export type WriteWallRowVerdict = - | { readonly admitted: true; readonly judgesOrganization: boolean } - | { - readonly admitted: false; - readonly field: 'organization_id' | '_record'; - readonly code: string; - readonly message: string; - }; - -/** - * [#25062] "What does the Layer 0 tenant write wall answer for each of these - * rows?" One verdict per row, in order, or `null` when the wall is not asked - * for this write (the engine then judges the rows as if nothing were - * registered). - * - * The engine declares this shape but never implements it. The wall has one - * definition, the enforcement layer's own (`@objectstack/plugin-security`'s - * step 3.7), and the verdict travels here as a value: a copy of the wall in - * the engine would be the mirror the `judgesOrganization` member forbids. See - * `ObjectQL.registerWriteWallVerdict`. - */ -export type WriteWallVerdict = ( - object: string, - operation: 'insert' | 'update', - context: unknown, - rows: readonly Record[], -) => Promise; - -/** - * The stack collections the engine decomposes into individual registry items — - * ONE list, read by the ONE body both registration seams run - * (`registerMetadataCollections()`, called from the manifest seam in - * `registerApp()` and the nested-plugin seam in `registerPlugin()`). - * - * ## Why this is one constant and not two lists (#7049) - * - * It used to be two: `const metadataArrayKeys = [...]` declared separately - * inside each loop. They drifted, invisibly, because nothing compared them — - * `jobs`, `emailTemplates`, `tools` and `skills` were registered from a - * manifest and NOT from a nested plugin, so a package shipping any of the four - * from a nested plugin registered nothing and stamped no ADR-0010 provenance: - * no refusal, no diagnostic. `capabilities` hit the SAME divergence and was - * patched into the nested copy by hand (#5870) without anyone diffing the rest - * of the two lists, which is how the remaining four survived it. - * - * The two loops were measured against each other before this was merged. They - * differed in four ways — which object they read (`manifest` vs `plugin`), which - * package id they stamp (both resolve to the SAME parent package: a nested - * plugin contributes under its parent's ownership), a per-key `debug` line, and - * the manifest seam's aggregated-view expansion plus its warn-on-nameless-item. - * Every one of those lived in the loop BODY. Not one of them was a reason for - * the two seams to enumerate different collections, so the enumeration was - * shared and that divergence is now unrepresentable rather than merely unnoticed. - * - * ## …and why the BODY is shared too (#7163) - * - * The last two entries on that list were not stylistic. The aggregated-view - * expansion is what makes ADR-0017's dual-read work, so the seam that lacked it - * under-registered every nested plugin's views silently — see - * `registerMetadataCollections()`, which is now the single body both seams run, - * for the measurement. Sharing the list made the seams' collection SET - * unanswerable-differently; sharing the body does the same for what they DO - * with a collection they both see. - * - * `check:stack-collection-maps` pins this list against - * `ObjectStackDefinitionSchema` in both directions (#6242); it used to pin the - * two copies as two sites and carry a waiver row recording their divergence. - * That row is gone with the divergence. - */ -const METADATA_ARRAY_KEYS = [ - // Data Protocol — shared option lists. FIRST, so a source's lists are in the - // registry before anything later in this loop reads them; objects do not - // depend on the order (they resolve a list lazily, on the next fold), so this - // is the declared loading order (`picklist` loads before `object`), not a - // precondition. Both are dispatched to their own registry verbs in - // `registerMetadataCollections`: the extensions merge into the list they - // name instead of registering as items of their own. - 'picklists', 'picklistExtensions', - // UI Protocol - 'actions', 'views', 'pages', 'dashboards', 'reports', 'datasets', 'themes', - // Automation Protocol - 'flows', 'workflows', 'approvals', 'webhooks', - 'jobs', - // Security Protocol — `capabilities` is here for the same reason as - // `permissions` (#5870, #4967 Part 2): the ONLY seam that stamps - // ADR-0010 provenance is `registerItem` → `applyProtection`, so a - // collection missing from this list reaches no registry with a - // `_packageId`. The declared-capability seeder (retired by ADR-0131 D3) - // resolved the owning package as `cap._packageId ?? cap.packageId`; while `capabilities` - // sat outside this list the first half could never be satisfied and - // `readDeclared(ql, 'capability')` returned nothing, which made the - // author-side `packageId` — documented as the FALLBACK — mandatory, - // and its omission a silent, unenforced authorization declaration. - // - // [#22203] `positions` for the same reason, and with a write-door - // consequence: ADR-0090 D3 renamed `roles` to `positions`, and the rename - // reached the artifact door's map but never this list, so a stack-declared - // position had no registry entry under its package. The metadata save door - // decides "a code package ships this" from exactly that entry - // (`isArtifactBacked` → `getArtifactItem`), so a save over a package's - // position took the runtime-create tier and was accepted, although the type - // registry declares `position` `allowEnvironmentOverlay: false`. With the entry - // present the door's type-level refusal covers it like every other - // `security`-domain type (`engine-security-catalog-package-door.test.ts`). - // `roles` stays in the list as the inert retired spelling it already was. - 'roles', 'positions', 'permissions', 'capabilities', 'profiles', 'sharingRules', 'policies', - // [ADR-0142 D1/D5] Ownership queues and the assignment rules that route - // inserted rows into them — both read from the registry on every use (the - // queue service, the assignment seam), so a declaration must reach it with - // its package stamped, as every collection on this list does. - 'queues', 'assignmentRules', - // AI Protocol - 'agents', 'tools', 'skills', 'ragPipelines', - // API Protocol - 'apis', - // Data Extensions - 'hooks', 'mappings', 'analyticsCubes', - // Integration Protocol - 'connectors', - // System Protocol — outbound mail templates. Registered here so the - // email plugin's materializer can read them back into - // `sys_email_template` (#4509); without this key an authored - // `emailTemplates:` entry never reached the registry at all, which is - // the far end of the disconnect the bridge closes. - 'emailTemplates', - // System Protocol — package documentation (ADR-0046); inert data - 'docs', - // Documentation navigation spine (ADR-0046 §6) - 'books', -] as const; - -/** - * Derive the registry key for a metadata item. - * - * Most metadata items expose a top-level `name` (or `id`). The `View` - * container defined by `@objectstack/spec/ui` is special: it aggregates - * `list / form / listViews / formViews` for a single object and is keyed by - * the OBJECT it binds to, not by its own row identity — which is what - * `/api/v1/meta/views/:object`, `getViewsByObject()` and - * `GET /meta/view?object=` all address it by. - * - * ⚠️ [commit 3c1bbd2a8] The sentence that used to stand here — "per spec, `ViewSchema` - * does NOT have a top-level `name` field" — is measurably false and was the - * premise for consulting `item.name` first. `ViewSchema` declares an optional - * `name` (`view.zod.ts`), described there as "supplied by the metadata door; - * for an object-scoped container it is the object name". "Is the object name" - * is a CONVENTION the door does not enforce, so the two never actually had to - * agree — and where they disagreed, this registrar and the other two picked - * different keys for the same document. - */ -function resolveMetadataItemName(key: string, item: any): string | undefined { - if (!item) return undefined; - // [commit 3c1bbd2a8] The aggregated `views` CONTAINER branch, taken FIRST and answered - // by the shared derivation. Everything below is unchanged. - // - // This registrar used to consult `item.name` before anything else, for every - // key including this one — so a container written as - // `{ name: 'lead_views', object: 'crm_lead', list: {…} }` registered under - // `lead_views` here while the artifact/HMR SOURCE registrar - // (`MetadataPlugin._parseAndRegisterArtifact`) and the runtime door - // (`expandRuntimeViewContainer`) both registered it under `crm_lead`. Same - // document, two source registrars, two registry keys and two sets of expanded - // item names, with `getViewsByObject()` / `GET /meta/view?object=` answering - // for the object only when the right registrar happened to load it. - // - // The 2026-08-07 meta-rule settles the direction rather than taste: one - // operation with two inconsistent implementations, the side bound by a - // DECLARATION wins. `ViewSchema.object`'s own `.describe()` names its readers - // (`getViewsByObject()` / `GET /meta/view?object=`); this loop's old order - // argued from item identity, which declares nothing about the binding. So the - // container branch adopts `deriveViewContainerObject` — by import, because a - // fourth hand-copy of a chain that already exists three times is the defect - // this repair exists to close, not the repair. - // - // The gate is `isAggregatedViewContainer`, which is what makes this the - // CONTAINER branch and nothing wider: it is false for every artifact carrying - // a `viewKind`, so the assembled `viewItems:` channel below (standalone - // ViewItems and flattened list/form overlays — every member of - // `AssembledViewArtifactSchema` requires `viewKind`) still resolves by its own - // `name` first, which is its identity and not a binding. - // - // `item.id` is untouched and stays reachable for every other key. It cannot - // fire for a spec-valid container: `ViewSchema` is a `strictObject` that - // declares `name` and `object` and no `id`, so an `id` on a container is - // refused at the authoring and metadata doors before this seam sees it. - if (key === 'views' && isAggregatedViewContainer(item)) { - return deriveViewContainerObject(item); - } - if (item.name) return item.name; - if (item.id) return item.id; - if (key === 'views') { - // A `views` entry that is NOT an aggregated container and carries neither - // `name` nor `id` — e.g. a flattened overlay whose optional `name` was - // omitted. Same derivation, and identical to the chain that used to be - // written out here: with `item.name` already known falsy, the helper's - // trailing `name` term contributes nothing. - return deriveViewContainerObject(item); - } - return undefined; -} - -/** - * ObjectQL Engine - * - * Implements the IDataEngine interface for data persistence. - * Acts as the reference implementation for: - * - CoreServiceName.data (CRUD) - * - CoreServiceName.metadata (Schema Registry) - */ -// [#6063] `SummaryDescriptor`, `summaryEmptySetValue` and the single-descriptor -// aggregate moved to `./summary-aggregate.js` — unchanged, and still the one -// place each is written down. The move exists so the one-off backfill of -// pre-#6013 `NULL` rows computes its value through the SAME code this engine -// does, instead of a second implementation that agrees only until one of them -// is edited. - -// `implements IObjectQLEngine` is the verification step of #4251 B3: every -// member the `objectql` slot's contract declares is checked against this class -// on every build, so the seven consumer-local surface declarations the contract -// replaced can never silently drift from the engine again. IObjectQLEngine -// extends IDataEngine, so the old claim rides along. -/** - * [#4889] The master id a detail row's `parent` binding resolves against: - * the write's own value for the FK when it carries one (a REPOINT must be - * judged against the master it lands on), else the prior row's. Only a scalar - * id counts — an expanded relation object or an array is not an id this read - * can bind, and guessing one would be worse than leaving `parent` unbound. - * - * [#19853] "Carries one" means the write STORES it: the UPDATE paths hand this - * the payload view `settleMasterDetailLanding` returns, which no longer holds - * an FK a later strip would have taken back out, so the prior row's FK — the - * header the row keeps — answers instead. - */ -export function masterIdOf( - fk: string, - data: Record | null | undefined, - row: Record | null | undefined, -): string | number | undefined { - const raw = data && fk in data ? data[fk] : row?.[fk]; - if (typeof raw === 'string') return raw === '' ? undefined : raw; - if (typeof raw === 'number' && Number.isFinite(raw)) return raw; - return undefined; -} - -/** - * [#19887] The UPDATE payload as the static `readonly` strip will leave it — - * `data` without every key that strip takes back out — which is the view a - * `readonlyWhen` predicate's `record` binding is built from - * (`ReadonlyWhenStripOptions.stored`). - * - * The conditional strip runs BEFORE the static one, so the payload it is - * handed still holds a value a non-system caller forged for a statically - * `readonly` field. Judged over that payload, `amount` locked by - * `record.status == 'closed'` read the forged `status: 'open'`, stayed - * writable, and the static strip then removed the `status`: the closed row - * committed with its locked amount rewritten. The strips keep their order — - * each still reports its own fields under its own reason — and only what the - * predicate reads moves. - * - * The same verdict {@link staticReadonlyStripTakes} asks for one key, asked - * for all of them by the same function with the same arguments: `data` itself - * when the strip does not run (a system caller), and a key a hook wrote, a - * value a hook overwrote or a column `preserveAudit` keeps all stay, because - * the write stores them. Silent for the same reason. - * - * [#24822] ...and over the same view of the schema: the one - * {@link updateStaticReadonlySubject} answers for `strip.wallJudgesOrganization`, - * the flag `update()` reads once off the wall's seam and hands to the strips - * themselves. So where the wall judges the update's organization, a re-point - * the strip leaves on the row is in this view too, and a `readonlyWhen` - * predicate reading `record.organization_id` reads the organization the row is - * stored in. The master-detail settlement forwards the update's descriptor - * object unchanged, so the flag reaches {@link staticReadonlyStripTakes} on that - * path too. Absent, the view is the whole schema's, exactly as before. - */ -export function staticReadonlyStoredView( - schema: unknown, - data: Record, - supplied: Readonly>, - strip: { - runs: boolean; - preserveAudit: boolean; - hookWrittenKeys: ReadonlySet | undefined; - wallJudgesOrganization?: boolean; - }, -): Record { - if (!strip.runs) return data; - return stripReadonlyFields( - updateStaticReadonlySubject(schema, strip.wallJudgesOrganization === true) as any, data, supplied, undefined, - { preserveAudit: strip.preserveAudit, hookWrittenKeys: strip.hookWrittenKeys }, - ) ?? data; -} - -/** - * What the engine knows about the transaction it opened, beyond the handle - * itself (#4619, ADR-0119 D1 follow-up). - * - * It began as a pure OBSERVABILITY record — #4619 could say a write was outside - * the transaction but deliberately did not act on it, because acting changes - * ADR-0119 D1's declared contract and that was a decision for the maintainer, - * not for the PR that found the defect. Since the 2026-08-06 ruling on #5351 - * this record is LOAD-BEARING: `enforceTransactionOrigin` refuses a - * cross-driver business write on it, and `transactionCoversDriverFor` uses it - * to keep a transaction handle from ever reaching a driver that does not own - * it. Routing itself is still untouched — a write goes exactly where `getDriver` - * sends it; what changed is whether it goes there carrying someone else's - * connection. - */ -interface TransactionScope { - /** - * The driver instance the open transaction belongs to. Compared by IDENTITY - * rather than by name: two drivers can transiently claim one name (see - * `registerDriver`'s collision branch), and identity is what actually decides - * whether a write rides this transaction's connection. - */ - readonly driver: IDataDriver; - /** The datasource name that driver is registered under — for the message. */ - readonly datasource: string; - /** - * Datasources already noted for THIS transaction. AGENTS.md's "say it once, - * at the first degradation, not once per failed write" — a 500-row audit - * batch carved out of this transaction is one note, not 500. Only the - * carve-out path consumes the budget; a refused business write throws, and a - * throw is never deduplicated. - */ - readonly reportedOutOfScope: Set; -} - -/** - * A datasource *definition* as the engine keeps it (ADR-0015) — the declarative - * facts a datasource states about itself, not a live connection (that is - * {@link ObjectQL.registerDriver}). - * - * Named rather than restated inline because three sites share it: the private - * index, {@link ObjectQL.registerDatasourceDef} which writes it, and - * {@link ObjectQL.listDatasourceDefs} which reads it back. Three copies of one - * shape is a second de-facto contract that drifts silently, and the drift this - * one produces is a credentials handle missing from a `sys_secret` sweep. - * - * The keys are a deliberate SUBSET of the spec's authored datasource surface - * (`ExternalDatasourceSettingsSchema` in `@objectstack/spec`) — the engine - * carries only what it has a use for. ⛔ Not a mirror of the spec block and not - * a place to grow one: `validation` and `queryTimeoutMs` are absent because - * nothing in this engine reads them. - */ -export interface DatasourceDef { - name: string; - /** - * [#27277] The driver id the datasource selects. Read to decide whether the - * engine floors the datasource's driver (`needsExternalDriverFloor`: a - * non-managed datasource on a driver id that is not built in). - */ - driver?: string; - schemaMode?: string; - external?: { - /** Datasource-wide write gate — ADR-0015 §5.3 Gate 3. */ - allowWrites?: boolean; - /** - * Reference into the secrets store, never an inline credential. Valid in - * EVERY `schemaMode` — it is the one `external` key a managed datasource - * may carry (#8153) — so a reader sweeping for handles must not filter by - * schema mode. - */ - credentialsRef?: string; - /** - * [#29762] The datasource's plugin-driver row cap (`external.rowCap`). - * Read by the floor on every read (`resolveExternalRowCap`), so a - * redefinition takes effect without re-wrapping the driver. - */ - rowCap?: number; - }; -} - -/** - * [#22305] The records ONE by-id cascade delete removes, collected before any - * of them is judged — see {@link ObjectQL.cascadeDeleteRelations}. - * - * Without it, every level of the depth-first walk judged its own `restrict` - * refusals with no knowledge of the cascade that called it: a record the same - * cascade was about to delete refused the delete of its sibling, and whether - * it got the chance depended on which object was registered first. The triage - * ruling on #22305 is the rule this set exists to apply: a child that is itself - * in the cascade set never restricts a sibling in the same set. - * - * Engine-internal and never part of a context: it rides - * {@link ObjectQL.cascadeDeleteSets}, an `AsyncLocalStorage`, so no request - * input can name a record into it, and a caller that passed no context still - * passes none. - */ -export interface CascadeDeleteSet { - /** - * Object name → ids the cascade deletes, the root included. Collected by a - * read-only walk across every CASCADING relation before any write, so it is - * complete before the first refusal is judged. - */ - readonly members: Map>; - /** - * Object name → ids whose own referential walk has started, in progress or - * done. The walk never re-enters one, which is what makes a cycle in the - * data terminate and what keeps a row reached by two paths from being - * deleted twice. - */ - readonly entered: Map>; - /** - * The #12166 elevation records already filed, one per deleted record per - * referenced object, so the collection walk and the deleting walk, which - * read the same relations, file each record once between them. - */ - readonly elevationsFiled: Set; - /** - * [#26718] Member key ({@link cascadeMemberKey}) → the members a - * CASCADING relation reaches from it directly, every edge the collection - * walk read, a row reached by two paths included. The reach of one member - * inside the set ({@link cascadeReachOf}) is read from it: which rows the - * delete removes only BECAUSE that member goes. - */ - readonly cascadeEdges: Map>; - /** - * [#27004] Relation key ({@link cascadeMemberKey} of the referencing object - * and its field) → the ids of the referenced object whose walk has finished - * that relation: every row outside the set has had them cleared. A walk - * over a root SET reads each relation once for many records, so a - * multi-valued row read for one of them may still list a member an earlier - * one has removed from it; the remainder written leaves those out too. - */ - readonly cleared: Map>; -} - -/** [#26718] One member of a {@link CascadeDeleteSet}, as a single key. */ -export function cascadeMemberKey(object: string, id: unknown): string { - return JSON.stringify([object, String(id)]); -} - -/** Is `id` of `object` in this index of a {@link CascadeDeleteSet}? */ -export function cascadeSetHas(index: Map>, object: string, id: unknown): boolean { - return id != null && index.get(object)?.has(String(id)) === true; -} - -/** Add `id` of `object`; `true` when it was not there yet. */ -export function cascadeSetAdd(index: Map>, object: string, id: unknown): boolean { - let ids = index.get(object); - if (!ids) { ids = new Set(); index.set(object, ids); } - const key = String(id); - if (ids.has(key)) return false; - ids.add(key); - return true; -} - -/** - * [#23221] The one child delete a cascade walk is issuing, for that delete to - * recognise itself by — see {@link ObjectQL.cascadeIssuedDeletes}. The walk - * opens one per `cascade` dependent, around its own `delete()` call, and the - * `delete()` whose object and id match claims it. - */ -interface CascadeIssuedDelete { - /** The child object the walk is deleting from, as the registry names it. */ - readonly object: string; - /** The child row's id, as a string (the walk's `String(depId)`). */ - readonly id: string; - /** - * Set by the `delete()` that recognised itself. One token, one dispatch: a - * later delete of the same row inside that call (a hook's own) finds it - * claimed and is not marked. - */ - claimed: boolean; -} - -/** - * [ADR-0010 §3.7] Stamp a manifest item with its owning package's - * `(packageId, packageVersion)` before the registry stores it: the call the - * artifact loader makes (`metadata/plugin.ts`), so both load paths serve one - * envelope. The registry's own `applyProtection` knows the id only, and - * `applyProtection` stamps a key only while it is unset, so the version - * stamped here is the one the item keeps. With no version this does nothing: - * the registry stamps the id, exactly as before. - */ -export function stampPackage(item: T, packageId: string, packageVersion: string | undefined): T { - if (packageVersion !== undefined) applyProtection(item as any, { packageId, packageVersion }); - return item; -} - -/** - * [ADR-0086 D3] The registration door's refusal of a permission set whose body - * `packageId` disagrees with the package {@link ObjectQL.registerApp} registers - * it under (`ownerId`; none for a manifest that carries no id). - * - * The judgement and its words are the one judge every door of the rule calls, - * `judgePermissionSetOwnerBinding` (`@objectstack/spec/security`, whose header - * holds the verdict table): the stack schema runs it at the build door - * (`defineStack`, `os validate`, `os build`, the artifact door's parse), so this - * seam is the door for a manifest no schema parse judged — a programmatic - * `registerApp`, a nested plugin, a `strict: false` stack, a package installed - * from a stored manifest. ⛔ Never a second comparison here. - * - * `VALIDATION_ERROR` / 400, thrown before the set is filed: the envelope of - * this seam's other body-against-key refusal, the divergent view-container - * `name` (`viewContainerNameRefusal`), which the same loop throws one statement - * earlier. `issues[]` locates the key, as the metadata write doors' refusal of - * the same rule does. No code is minted. - * - * Exported for the stack's RESIDUAL registration (`registerUnclaimedTopLevelItems` - * in `plugin.ts`), which files a top-level permission set under the stack's id - * and refuses the same set in this same envelope. ⛔ Never rebuild the envelope - * at a second door. - */ -export function permissionSetOwnerRegistrationRefusal( - item: unknown, - name: string, - ownerId: string | undefined, -): (Error & { code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }> }) | undefined { - const refusal = judgePermissionSetOwnerBinding(item, name, ownerId, 'register'); - if (!refusal) return undefined; - const err = new Error(refusal.message) as Error & { - code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }>; - }; - err.code = 'VALIDATION_ERROR'; - err.status = 400; - err.httpStatus = 400; - err.issues = [{ path: 'packageId', message: refusal.issueMessage, code: 'custom' }]; - return err; -} - -/** - * [#28534] The `_packageId` each item object was stamped with BY A - * REGISTRATION of this process — written onto the caller's own object by - * `stampPackage` / `registerItem`, recorded by {@link noteRegistrationStamp}. - * Weakly held and process-wide: the earlier registration may have been - * another engine's. Read by {@link collectionItemOwnerStampRefusal} only. - */ -const REGISTRATION_WRITTEN_STAMPS = new WeakMap(); - -/** - * [#28534] Record that a registration just wrote `item`'s `_packageId`: it was - * unset before (`stampBefore`) and is set now. A stamp the object already - * carried is never recorded, so it stays a stamp the body arrived with. - */ -function noteRegistrationStamp(item: unknown, stampBefore: unknown): void { - if (stampBefore !== undefined || !item || typeof item !== 'object') return; - const written = (item as { _packageId?: unknown })._packageId; - if (written !== undefined) REGISTRATION_WRITTEN_STAMPS.set(item, written); -} - -/** - * [#28534] The registration door's refusal of a collection item whose body - * already carries a `_packageId` other than the package it is registered under - * (`ownerId`; none for a manifest that carries no id). - * - * `_packageId` is the ADR-0010 envelope stamp: the package that registered the - * item, written by the door (`stampPackage` here, `registerItem` behind it, - * both through `applyProtection`). `applyProtection` writes it only while it is - * `undefined`, and every item schema that spreads `MetadataProtectionFields` - * (a connector entry among them) declares the key, so a body could arrive - * wearing ANOTHER package's id and keep it: an installed package naming the - * host's own id, or an id no package holds, and every reader that asks an - * item's owner (the connector materializer judging whose root a file ref - * resolves against, the disabled-package filter, the ADR-0130 artifact gates) - * then reads a stamp the body wrote about itself. - * - * Refused rather than overwritten, beside the two other body-against-key - * refusals of the same loop (the divergent view-container `name`, the - * permission set's `packageId`): resolving a disagreement between the body and - * the key it is filed under silently, in either direction, files the item - * under something nobody wrote (commit d0ee598e6). A stamp EQUAL to the owner - * is accepted — it is what an item re-registered under its own package (a - * ledger rehydrate, an artifact the metadata door already stamped in place - * with the same body id) carries. `null` and the empty string are stamps too: - * neither is overwritten by `applyProtection`, so either would survive as an - * owner no row answers. - * - * The judgement and its words are the one judge both doors of the rule call, - * `judgeCollectionItemOwnerStamp` (`@objectstack/spec/kernel`, whose header - * holds the verdict table and the collections): the stack schema runs it at - * the build door (`defineStack`, `os validate`, `os build`, the artifact door's - * parse), so this seam is the door for a body no schema parse judged — an - * installed body, a programmatic `registerApp`, a nested plugin, a - * `strict: false` stack. ⛔ Never a second comparison here. What stays here is - * the exemption below, which is this process's state, not a body rule. - * - * `VALIDATION_ERROR` / 400, `issues[]` at `_packageId`: the envelope of - * {@link permissionSetOwnerRegistrationRefusal}. Exported for the stack's - * RESIDUAL registration (`registerUnclaimedTopLevelItems` in `plugin.ts`), - * which files top-level items under the residual owner and refuses the same - * stamp in this same envelope. ⛔ Never rebuild the envelope at a second door. - * - * ## A stamp a registration wrote is not a body's claim - * - * `stampPackage` and `registerItem` stamp the CALLER'S object in place, so an - * item object registered once keeps that registration's `_packageId` for as - * long as it lives — a module constant handed to a second registration (a - * fixture another test registered under its own package, measured in - * `engine-picklist.test.ts`) arrives wearing the first registration's stamp, - * which no body wrote. Those stamps are recorded where they are written - * ({@link noteRegistrationStamp}) and are not refused here: the item - * registers as it always has. Only a stamp the object ARRIVED with is judged — - * which every installed body's is, since an install door parses its body - * fresh, so no registration has touched it. - */ -export function collectionItemOwnerStampRefusal( - item: unknown, - type: string, - name: string, - ownerId: string | undefined, -): (Error & { code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }> }) | undefined { - if (item && typeof item === 'object') { - const stamped = (item as { _packageId?: unknown })._packageId; - if (stamped !== undefined && REGISTRATION_WRITTEN_STAMPS.get(item) === stamped) return undefined; - } - const refusal = judgeCollectionItemOwnerStamp(item, type, name, ownerId, 'register'); - if (!refusal) return undefined; - const issueMessage = refusal.issueMessage; - const err = new Error(refusal.message) as Error & { - code: 'VALIDATION_ERROR'; status: 400; httpStatus: 400; issues: Array<{ path: string; message: string; code: string }>; - }; - err.code = 'VALIDATION_ERROR'; - err.status = 400; - err.httpStatus = 400; - err.issues = [{ path: '_packageId', message: issueMessage, code: 'custom' }]; - return err; -} - -/** - * [#28697] {@link collectionItemOwnerStampRefusal} for the two collections a - * package body files through verbs of their own rather than the collection - * loop: its OBJECTS (`registerObject`, the array and the name-keyed map form) - * and its APPS (`registerApp`: each `apps[]` entry the loop would register, - * and the body itself when it registers as its own app). The first refusal - * found, in that order, or `undefined`. - * - * Asked BEFORE the body registers anything, so a refused body leaves no - * package record, object or app behind. An app wearing another package's - * `_packageId` kept it (`applyProtection` stamps an unset key only), and every - * reader that asks an app's owner — the per-package list filter, the - * disabled-package filter, the packaged-base gate — then filed it under the - * package it named. An object's stamp is read by the action-parse predicate - * of `registerObject` (`schema._packageId ?? packageId`). Refused, not - * overwritten, for the reason the collection loop refuses: one judge, so the - * build door (the stack schema, which judges `objects` and `apps` too) and - * this door cannot disagree. - * - * Called by {@link ObjectQL.registerApp} and by a nested plugin's - * registration (`registerPlugin`, which registers objects and the plugin - * itself as an app, and no `apps[]`). - * - * @param body The package body or nested plugin, as it arrived. - * @param ownerId The package the door registers it under. - * @param scope Which of the app shapes this door registers. - */ -export function packageBodyObjectAndAppOwnerStampRefusal( - body: any, - ownerId: string | undefined, - scope: { apps: boolean; selfAsApp: boolean }, -): ReturnType { - const objects = body?.objects; - if (objects && typeof objects === 'object') { - const entries: Array<[unknown, unknown]> = Array.isArray(objects) - ? objects.map((def: any) => [def?.name, def]) - : Object.entries(objects); - for (const [name, def] of entries) { - const refusal = collectionItemOwnerStampRefusal(def, 'object', typeof name === 'string' && name !== '' ? name : '(unnamed)', ownerId); - if (refusal) return refusal; - } - } - if (scope.apps && Array.isArray(body?.apps)) { - for (const app of body.apps) { - // The loop registers an app with a name only; a nameless one is skipped. - const appName = app?.name || app?.id; - if (!appName) continue; - const refusal = collectionItemOwnerStampRefusal(app, 'app', String(appName), ownerId); - if (refusal) return refusal; - } - } - if (scope.selfAsApp) { - const refusal = collectionItemOwnerStampRefusal(body, 'app', String(body?.name), ownerId); - if (refusal) return refusal; - } - return undefined; -} - -/** - * [#28697] Every refusal the collection seam (`registerMetadataCollections`) - * raises that the body alone decides, asked over a package body or a nested - * plugin BEFORE anything of the package registers. The first refusal found, - * walking the collections and items in the order the seam registers them, or - * `undefined`. - * - * Asked first so a body refused by one of them leaves nothing of the package - * registered, and a refused re-register of an installed id leaves that - * install as it was. Every judge here reads the item, its derived name, the - * source label and the owning id, and nothing the registry holds, so asking - * it first gives the same answer in the same words. ⛔ No second copy of any - * judge: each is the one function its own record names. - * - * Called by {@link ObjectQL.registerApp} for the body (`'manifest'`) and for - * each nested plugin (`'nested plugin'`, under the parent's id, as - * `registerPlugin` registers it). - */ -export function packageBodyCollectionRefusal( - source: any, - ownerId: string, - sourceLabel: string, -): Error | undefined { - for (const key of METADATA_ARRAY_KEYS) { - // A `picklistExtensions` entry is not an item (the seam hands it to - // `registerPicklistExtension`, whose refusal reads the registry). - if (key === 'picklistExtensions') continue; - const items = (source as any)?.[key]; - if (!Array.isArray(items) || items.length === 0) continue; - for (const item of items) { - const itemName = resolveMetadataItemName(key, item); - // The seam skips (and warns about) an item with no derivable name. - if (!itemName) continue; - // [commit d0ee598e6] The DIVERGENT-container refusal (maintainer ruling - // 2026-09-03, direction 2). The collection seam used to reconcile a - // container's own `name` to the derived key silently: the author's - // field discarded with no diagnostic, while - // the OTHER source registrar - // (`MetadataPlugin._parseAndRegisterArtifact` -> - // `assertMetadataRegisterContract`, #7378 row 1) hard-failed the - // whole artifact load on that very same document. One document, - // two SOURCE registrars, opposite outcomes, and which one an - // author got depended on how their package was loaded. #7378 - // row 1 already rules that resolving a name/key disagreement - // silently "in either direction" files the item under a key the - // caller never wrote, so the boot loop converges onto the - // refusal; the artifact door was not moved onto the rewrite. - // - // MEASURED, and it is why only this site changed: the second - // `toRegister` ternary in the seam (the assembled - // `viewItems:` channel below) cannot carry a container at all — - // `AssembledViewArtifactSchema` is the view vocabulary MINUS the - // container branch, and every body it admits is - // `isAggregatedViewContainer === false`. Its ternary can only - // MINT a `name` onto an overlay that has none; it can never - // discard an authored one, because `resolveMetadataItemName` - // reads a non-container's own `name` FIRST. - // - // Scope is the ruling's named main risk, so the gate is three - // independent narrowings and each one is load-bearing. The first - // stays here: - // * `key === 'views'` — the seam is the GENERIC metadata loop; no - // other kind changes behaviour and they all keep the - // reconcile. - // The other two — the CONTAINER branch only, and `name` present - // AND different — and the message, the envelope and the reason - // the prose is this seam's own live in `viewContainerNameRefusal` - // (`view-container-name-refusal.ts`). - // - // [#20331] Moved there so that `os validate` runs the SAME - // judgment, in the same words, instead of passing a document this - // loop then refuses at boot. The message and the envelope moved - // unchanged. The `!itemName` skip just above is this loop's - // precondition for the check, so the function carries it too: it - // answers `undefined` for a falsy derived key, and a door calling - // it without this loop cannot refuse an entry this loop skips. - // ⛔ Do not re-inline it here, and do not write a second copy of - // it anywhere else: one judge, two doors. - if (key === 'views') { - const refusal = viewContainerNameRefusal(item, sourceLabel, ownerId); - if (refusal) return refusal; - } - // [ADR-0086 D3] A permission set whose body `packageId` disagrees - // with the package the seam registers it under is refused, in - // the same envelope as the container refusal above: - // `registerItem` stamps `ownerId` as the set's `_packageId`, the - // owner every reader reads, so the body's key used to be - // registered over in silence. See - // {@link permissionSetOwnerRegistrationRefusal}. - if (key === 'permissions') { - const refusal = permissionSetOwnerRegistrationRefusal(item, itemName, ownerId); - if (refusal) return refusal; - } - // [#28534] …and an item of ANY collection whose body already wears - // a `_packageId` other than `ownerId`: `stampPackage` and - // `registerItem` stamp the owner only while the key is unset, so - // the body's own stamp used to win. See - // {@link collectionItemOwnerStampRefusal}. - { - const refusal = collectionItemOwnerStampRefusal(item, pluralToSingular(key), itemName, ownerId); - if (refusal) return refusal; - } - // [#5320] The `views:` tighten — containers ONLY, the contract the - // stack schema has always declared (`stack.zod.ts`, - // `z.array(ViewSchema)`). The seam used to register EVERY entry - // as type `view` — the "runtime wider than schema" hole #5320 - // records: a ViewItem or flattened overlay that `defineStack` - // refuses registered there in silence. Judged by the SAME - // classifier the manifest assemblers partition with - // (`isViewContainerShaped`), so a produced manifest cannot carry - // a `views:` entry the seam refuses. Deliberately a SHAPE-CLASS - // judgement, not a full `ViewSchema.parse`: which class an entry - // belongs to is the seam's contract; whether a container's - // internals are well-formed stays the authoring/publish doors' - // job (defineStack, `os validate`, the metadata door). - // The envelope and the words: {@link viewsEntryNotContainerRefusal}, - // which the residual registration throws too. Judged on the body the - // seam registers (its `name` reconciled to the derived key). - if (key === 'views') { - const toRegister = item.name === itemName ? item : { ...item, name: itemName }; - if (!isViewContainerShaped(toRegister)) return viewsEntryNotContainerRefusal(itemName, sourceLabel, ownerId); - } - } - } - // [#5320] The `viewItems:` channel, each entry judged by its schema and - // the PARSED body's owner stamp (#28534), as the seam registers it. - const assembledItems = (source as any)?.[ASSEMBLED_VIEW_ITEMS_KEY]; - if (Array.isArray(assembledItems)) { - for (const item of assembledItems) { - const parsed = AssembledViewArtifactSchema.safeParse(item); - if (!parsed.success) return assembledViewItemRefusal(item, parsed.error, sourceLabel, ownerId); - const body = parsed.data as Record; - const itemName = resolveMetadataItemName('views', body); - if (!itemName) continue; - const refusal = collectionItemOwnerStampRefusal(body, 'view', itemName, ownerId); - if (refusal) return refusal; - } - } - return undefined; -} - -/** - * [#28697] The seed step's owner-stamp refusal ({@link collectionItemOwnerStampRefusal}, - * #28534: the one collection filed outside the collection seam), asked over a - * package body's `data` datasets before anything of the package registers. - * The first refusal found, or `undefined`. Called by {@link ObjectQL.registerApp}. - */ -export function packageBodySeedDataRefusal(manifest: any, ownerId: string): Error | undefined { - const seedData = manifest?.data; - if (!Array.isArray(seedData)) return undefined; - for (const dataset of seedData) { - // The seed step registers a dataset with a target object only. - if (!dataset?.object) continue; - const refusal = collectionItemOwnerStampRefusal(dataset, 'data', String(dataset.object), ownerId); - if (refusal) return refusal; - } - return undefined; -} - -/** - * [#5320] The refusal of a `viewItems:` entry its schema - * (`AssembledViewArtifactSchema`) does not admit: `INVALID_METADATA` / 422, - * naming the entry and its source. Asked by {@link packageBodyCollectionRefusal} - * and by the collection seam, which parses the entry again for the body it - * registers. ⛔ Never a second copy of the text. - */ -export function assembledViewItemRefusal( - item: unknown, - error: { issues: ReadonlyArray<{ message: string }> }, - sourceLabel: string, - ownerId: string | undefined, -): Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 } { - const itemName = resolveMetadataItemName('views', item) ?? '(unnamed)'; - const err = new Error( - `Invalid \`${ASSEMBLED_VIEW_ITEMS_KEY}:\` entry '${itemName}' from ${sourceLabel} '${ownerId}': ` - + 'the assembled-manifest channel carries non-container view artifacts only — a ViewItem ' - + 'record (`viewKind` + `config`) or a flattened list/form overlay ' - + '(`AssembledViewArtifactSchema`, @objectstack/spec). A view CONTAINER travels in `views:`. ' - + `First issue: ${error.issues[0]?.message ?? 'no issue detail'}`, - ) as Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 }; - err.code = 'INVALID_METADATA'; - err.status = 422; - err.httpStatus = 422; - return err; -} - -/** - * [#28697] Stamp an app object ({@link stampPackage}) and register it through - * the registry's app verb, recording the stamp the registration writes onto - * the caller's own object ({@link noteRegistrationStamp}): `stampPackage` (with - * a version) and `registerItem` (without one) stamp the object in place, so a - * module constant handed to a second registration arrives wearing the first - * one's stamp, which no body wrote. The stamp before is read BEFORE - * `stampPackage`, the order the collection and seed loops keep, so a versioned - * registration is recorded too. - */ -export function registerAppNotingStamp( - registry: SchemaRegistry, - app: any, - ownerId: string, - ownerVersion: string | undefined, -): void { - const stampBefore = (app as { _packageId?: unknown } | null | undefined)?._packageId; - registry.registerApp(stampPackage(app, ownerId, ownerVersion), ownerId); - noteRegistrationStamp(app, stampBefore); -} - -/** - * [#5320] The registration door's refusal of a `views:` entry that is not a - * view CONTAINER (`isViewContainerShaped` is false: a ViewItem's - * `viewKind`/`config`, a flattened overlay's inline view config): the stack - * `views:` collection carries containers only, the contract the stack schema - * declares (`z.array(ViewSchema)`). The caller judges the shape; this is the - * envelope and the words — `INVALID_METADATA` / 422, naming the entry by the - * name it would be filed under and the source it came from. - * - * Called by {@link ObjectQL.registerApp}'s metadata loop and by the stack's - * RESIDUAL registration (`registerUnclaimedTopLevelItems` in `plugin.ts`, with - * the `manifest` source label), so a residual entry and a body's are refused - * in one spelling. ⛔ Never a second copy of the text. - */ -export function viewsEntryNotContainerRefusal( - itemName: string | undefined, - sourceLabel: string, - ownerId: string | undefined, -): Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 } { - const err = new Error( - `Invalid \`views:\` entry '${itemName}' from ${sourceLabel} '${ownerId}': the stack ` - + '`views:` collection carries view CONTAINERS only. `viewKind`/`config`/inline view ' - + 'config belong to a single VIEW, not to the container — wrap it: ' - + '`defineView({ list: { type, data, columns, … } })`, or name it — ' - + '`defineView({ listViews: { my_view: { … } } })`. A machine-assembled manifest ' - + `(package export, environment artifact) carries non-container view artifacts under ` - + `\`${ASSEMBLED_VIEW_ITEMS_KEY}:\` instead — re-export the package with a runtime that ` - + 'writes that channel.', - ) as Error & { code: 'INVALID_METADATA'; status: 422; httpStatus: 422 }; - err.code = 'INVALID_METADATA'; - err.status = 422; - err.httpStatus = 422; - return err; -} - -/** - * {@link stampPackage} for an authored OBJECT definition, on a shallow copy. - * The registry stamps an object schema on its own copy (`applySystemFields` - * and friends run first), so `registerApp` leaves the caller's definition - * unstamped, and a manifest registered twice, or recorded by a second install, - * shows the same objects both times. `applyProtection` writes top-level keys - * only, so a shallow copy holds every key it touches. - */ -export function stampObjectPackage(def: ServiceObject, packageId: string, packageVersion: string | undefined): ServiceObject { - return packageVersion === undefined ? def : stampPackage({ ...def }, packageId, packageVersion); -} - -/** - * [ADR-0139 D2] What the `ReadableIds` composition asks of the registered - * `ISecurityService`: both halves of a read, the object-level one through - * `canReadObject` or, when a service predates that optional method, the - * `allowed` of its `explain`. - */ -export type ReadableIdsSecurity = Pick & Partial>; - -/** The three resolutions of the `security` service (`ObjectQL#resolveReadableIdsSecurity`). */ -type ReadableIdsSecurityResolution = - | { kind: 'usable'; svc: ReadableIdsSecurity } - | { kind: 'absent' } - | { kind: 'unusable'; why: string }; - -/** - * [ADR-0139 D2, D6 as amended] One read's `ReadableIds` composition, carried - * through the chain of composed scopes (`ObjectQL#composeReadableIdsChain`): - * the read's object and verb (every refusal names them), the security - * resolution made once for the read, the caller's context handed to every - * level as it stands, and the answers — per parent object, `{ scope }` with - * the scope already composed, or `null` for `$in: []` — so a parent is asked - * once per read at whichever level it first appears. - */ -export interface ReadableIdsChain { - readonly object: string; - readonly operation: 'find' | 'findOne' | 'count' | 'aggregate'; - readonly security: ReadableIdsSecurityResolution; - readonly context: ExecutionContext | undefined; - readonly answers: Map; -} - -/** - * [ADR-0139 D5.2 / D5.3] A `controlled_by_parent` detail whose master the - * engine composes no `ReadableIds` leaf over — - * `ObjectQL#findUnservableControlledByParentMasters`, refused at boot by - * `ObjectQLPlugin`. - */ -export type UnservableControlledByParentMaster = { - /** The `controlled_by_parent` detail. */ - readonly detail: string; - /** Its one required `master_detail` field. */ - readonly field: string; - /** The object that field names. */ - readonly master: string; -} & ( - | { readonly class: 'object-scoped-read-middleware' } - | { readonly class: 'other-datasource'; readonly objectDatasource: string | undefined; readonly parentDatasource: string | undefined } -); +import { + resolveTenantFieldName, + SystemWriteOrganizationRequiredError, + ORGANIZATION_OBJECT, +} from './tenancy/system-write-organization.js'; +import { + type TenancyPosture, + type TenantLayer0Verdict, + postureUsesUnionScope, +} from '@objectstack/spec/security'; /** - * [ADR-0139 D2] The engine slot where `ObjectQLPlugin` puts the SOURCE the - * `ReadableIds` composition reads the registered `ISecurityService` from: a - * function that resolves the kernel's `security` service when a leaf is - * composed, never at init, so plugin-registration order cannot decide - * whether a leaf is composed or denied. Written by assignment, so the last - * registration wins. - * - * An empty slot is a DEFINED state, and it fails closed: so does a slot that - * holds something other than a function, a source that answers nothing or - * throws, and a service that cannot answer both halves of a read. Each - * answers every leaf `$in: []` (`ObjectQL#resolveReadableIdsSecurity`). + * Per-row outcome of {@link ObjectQL.insertMany} (framework#3172). One entry + * per input row, in input order: written rows carry the after-hook record, + * failed rows carry the per-row error (validation / autonumber / encryption). * - * A wiring seam between this package's plugin and its engine, and nothing - * more: ⛔ not a member of `ObjectQL`, and ⛔ not re-exported by `index.ts` or - * `core.ts`, so it adds nothing to the surface either entry publishes. A - * `Symbol.for` key (the shape `plugin.ts` already uses for its string-keyed - * `skipSchemaSync` slot), so every bundle of this package — both entries, - * ESM and CJS — names the one slot. - */ -export const SECURITY_SERVICE_SOURCE_SLOT: unique symbol = Symbol.for('objectstack.objectql.securityServiceSource'); - -/** - * [ADR-0142 D4] The engine's own channel for {@link OperationContext.writePrecondition} - * on an `update` it issues itself (the assignment cursor). A module-private - * symbol, so no options object a door builds — and nothing outside this - * module — can carry it. + * [#20922] A written row also carries `droppedFields`: the caller-supplied + * fields the engine LEGALLY stripped from THAT row, one `DroppedFieldsEvent` + * per reason — the per-row channel, recorded at the strips. Present only when + * at least one field was taken from the row. The batch-level `onFieldsDropped` + * events are unchanged and still name no row; this key is not a resolution of + * them. A failed row carries none: its write did not complete. */ -const WRITE_PRECONDITION_OPTION: unique symbol = Symbol('objectql.writePrecondition'); +export type InsertManyRowOutcome = + | { ok: true; record: any; droppedFields?: DroppedFieldsEvent[] } + | { ok: false; error: unknown }; +import { CoreServiceName, StorageNameMapping } from '@objectstack/spec/system'; +import { IRealtimeService } from '@objectstack/spec/contracts'; +import type { ICryptoProvider } from '@objectstack/spec/contracts'; +import { + collectInternalReadFields, + collectCredentialFields, +} from './secret-fields.js'; +// [#25522 cut 18] `count` and `aggregate()`'s credential guard +// (`rejectCredentialAggregation`); the methods of the same names below hand +// each body the members it reads (`CountAggregateHost`). +import { count, rejectCredentialAggregation } from './engine-count-aggregate.js'; +// ADR-0100: the credential-column guard's filter / sort half, beside the +// aggregate half above — one collector, one envelope. +import { rejectCredentialEvaluation } from './credential-evaluation-guard.js'; +import { assertNoInternalFieldEvaluated, rejectInternalFieldEvaluation } from './internal-field-evaluation-guard.js'; +import { assertProjectionNamesAField, refuseProjectionOfUnknownColumns } from './empty-projection-door.js'; +import { assertRecordCurrencyCodes, touchesRecordCurrency } from './validation/record-currency-code.js'; +import { + EXCHANGE_RATE_OBJECT, + homeAmountsTouched, + memoizeHomeAmountServing, + planHomeAmounts, + projectHomeAmounts, + settleBulkHomeAmounts, + type HomeAmountPlanEntry, + type HomeAmountServing, + type HomeLocalization, + type HomeLocalizationResolver, +} from './home-amount-projection.js'; +import { SchemaRegistry, type ArtifactInstallScope } from './registry.js'; +import { stripSearchCompanion } from './search-companion.js'; +import { type EvalError, type EvalOptionLabels, type EvalPermissions } from '@objectstack/formula'; +import { declaredOptionLabels, readerOptionLabels } from './formula-option-labels.js'; +// [commit 3c1bbd2a8] The ONE spelling of "which object does an aggregated `defineView` +// container bind to", imported rather than re-spelled — from the LEAF subpath, +// for the reason the `/errors` import above states (commit 3bd9b3498). See +// `resolveMetadataItemName` below for why this registrar lost its fourth copy. +// [#20331] The divergent view-container `name` refusal — the ONE judge this +// registrar and `os validate` both call. +import { bindHooksToEngine, type BindHooksOptions } from './hook-binder.js'; +import { declaredHookCondition } from './hook-wrappers.js'; +import { validateRecord, validateRecordInScope, normalizeMultiValueFields, normalizeBlankTypedValues, normalizeNumericStringValues, coerceBooleanFields, ValidationError, isScannableValueShapeField } from './validation/record-validator.js'; +import type { AdmittedValueShapeViolation, AdmittedValueShapeViolationSink, ValidationMessageContext } from './validation/record-validator.js'; +import type { RelatedFieldBinding, RelatedRecordBinding } from './validation/rule-validator.js'; +import { createBatchSummaryAccumulator } from './summary-batch-accumulation.js'; +import { collectReferenceTraversals, evaluateValidationRules, emitValidationAdvisories, optionVisibilityReadsPermissions, readsPermissionPredicate, needsPriorRecord, stripReadonlyWhenFields, stripReadonlyWhenFieldsMulti, hasReadonlyWhenInPayload, hasParentScopedReadonlyWhenInPayload, hasParentScopedRequiredWhen, stripReadonlyFields, stripRuntimeOwnedFields, staticReadonlyInsertSubject, preserveAuditIgnoredOnInsertWarning } from './validation/rule-validator.js'; +// [#14088] The before-phase write recorder — the provenance channel the static +// `readonly` strip needs to tell a hook's write from a caller's echo of the +// SAME value. Armed and sealed in `update()`; the module owns the argument for +// why neither end may move. +import { recordHookPayloadWrites } from './hook-write-provenance.js'; +// [commit 706ad0fcc] The hide pass's other half: when a hook faults reaching THROUGH a +// key that pass withheld, this names the key, says the platform withheld it, +// and points at `ctx.previous` — the module owns the measurement and the +// reason the explanation cannot be composed any further downstream. +import { dispatchHooksExplainingWithheldReadonly } from './hook-withheld-readonly-fault.js'; +import { UnscopedHookApi, SEED_USER_GRANTS, APP_AUTHORED_HOOK_API, type HookRunAs, type HookRunAsRef, type RunAsDerivableApi } from './hook-run-as.js'; +import type { HookWriteRecording } from './hook-write-provenance.js'; +import { resolveMasterDetailRelation } from './master-detail.js'; +// [#6457] The master-detail header a `parent`-scoped predicate reads is made +// total over the MASTER's declared fields before it leaves this engine — the +// same helper every other server seam materialises with (#1871/#4649/#4953). +import { materializeDeclaredFields } from './declared-fields.js'; +// [#21571] The declared column set — the read verbs' default projection, their +// explicit-projection filter and the write path's undeclared-key door, one list. +import { + PLATFORM_PROVISIONED_COLUMNS, + declaredColumnSet, + rowsWithDeclaredColumnsOnly, + rowsWithSelectedColumnsAnswered, + selectedStoredColumns, + storedColumnsLeftUnwritten, + withDeclaredColumnsOnly, + withSelectedColumnsAnswered, +} from './declared-read-columns.js'; +// [#21777] "Is this schema the remote's?" One predicate, shared with the boot sync. +// [#21910, #21918] And its injected-column refinement, which both cascade walks ask. +import { isFederatedUnprovisionedInjectedColumn } from './federated-object.js'; +// [#27277] The correctness floor over a plugin driver on an external datasource. +import { + externalPredicateWriteRefusal, + isExternalDriverFloor, + unwrapExternalDriverFloor, +} from './external-driver-floor.js'; +// [#25522 cut 14] The master-detail landing, the preview's parents and the +// related reads a validation rule makes; the methods of the same names below +// hand each body the members it reads (`MasterDetailRelatedHost`). Imported after +// every module those bodies import, so the engine's module evaluation order +// is what it was. +import { resolveOneHopRelated, resolvePredicateRelated, resolvePreviewParents, settleMasterDetailLanding } from './engine-master-detail-related.js'; +// [#25522 cut 15] The `ReadableIds` chain composition, the controlled-by-parent +// boot screen, and the permission and formula resolutions an operation shares; +// the methods of the same names below hand each body the members it reads +// (`ReadableIdsPermissionFormulaHost`). Imported after every module those bodies import, so +// the engine's module evaluation order is what it was. +import { composeReadableIdsChain, findUnservableControlledByParentMasters, permissionResolution, readableIdsScopeOf, reportFormulaFault, resolveDefaultPermissions, resolveFormulaPermissions, resolveOptionPermissions } from './engine-readable-ids-permission-formula.js'; +// [#25522 cut 16] Type-only, for the `{@link}`s on `composeReadableIdsAtBoundary`'s and +// `composeReadableIdsChain`'s docblocks: the constant moved with its reader in cut 15. +import type { READABLE_IDS_MAX_CHAIN_DEPTH } from './engine-readable-ids-permission-formula.js'; +// [#25522 cut 19] The delete cascade's collection walk, the in-cascade +// `restrict` judge, the `DELETE_RESTRICTED` refusal, the reference-probe +// filter and the reference-check elevation record; the methods of the same +// names below hand each body the members it reads (`DeleteCascadeHost`). +// Imported after every module those bodies import, so the engine's module +// evaluation order is what it was. +// [#28001] …and the object-level closure of that walk the metadata read serves. +import { collectCascadeDeleteObjects, collectCascadeDeleteSet, deleteRestrictedError, recordReferenceCheckElevation, referenceProbeFilter, refuseRestrictsInsideOwnCascade } from './engine-delete-cascade.js'; +import { + resolveEngineDeleteDispatch, + ENGINE_DELETE_REJECT_MESSAGE, + type EngineDeleteDispatchInput, +} from './engine-delete-dispatch.js'; +import { + resolveEngineUpdateDispatch, + ENGINE_UPDATE_REJECT_MESSAGE, + // [#11142] The one reject thrower, shared with assertEngineUpdateDispatch, + // so a refusal that declares an ADR-0112 code/status (the conflicting-id + // reject) carries it identically from the real engine and from every + // pinned fake. + engineUpdateDispatchRejectError, + type EngineUpdateDispatchData, + type EngineUpdateDispatchInput, +} from './engine-update-dispatch.js'; +// [#25522 cut 16] The write preview's body; `validate` below hands it the +// members it reads (`ValidateHost`). Imported after every module that body +// imports, so the engine's module evaluation order is what it was. +import { validate } from './engine-validate.js'; +import { bindHookNotice } from './hook-notice.js'; +// [#25522 cut 20] The `where`-lowering bodies (`QueryLoweringHost`), imported +// after every module they import, so module evaluation order is unchanged. +import { lowerRelationConditions, resolveSearchQueryableFields, resolveWhereTokens } from './engine-query-lowering.js'; +import { + auditDanglingReferences, + type DanglingReferenceAuditOptions, + type DanglingReferenceReport, +} from './integrity/dangling-reference-audit.js'; +// [#29986 codemod] The top-level declarations that moved whole to `engine-top-level.ts`, as this file still reads them. +import { + AUTONUMBER_COLLISION_ATTEMPTS, DISPATCHABLE_HOOK_EVENTS, ENGINE_DELETE_OPTION_KEYS, ENGINE_FIND_OPTION_KEYS, + ENGINE_QUERY_SLOTS, ENGINE_UPDATE_OPTION_KEYS, ENGINE_WHERE_SLOTS, ENGINE_WIRE_ONLY_SLOTS, + SECURITY_SERVICE_SOURCE_SLOT, WIRE_ONLY_CANONICAL_SHAPES, WRITE_PRECONDITION_OPTION, applyFormulaPlan, + applyStoredExpressionsToPatch, assertDispatchableHookEvent, assertHookScopeNotSelfCancelling, + assertOrderByIsMaterializable, assertProjectionHasNoDottedPaths, assertValidHookExcludeObjects, + assertValidHookObject, assertValidUnscopedMultiWriteFlag, autonumberFieldNames, callerKeySets, callerSuppliedRow, + cascadeMemberKey, cascadeSetAdd, cascadeSetHas, computeStoredExpressions, declaredDatetimeLowering, + declaredMultiValued, deleteWalkTargetOf, droppedFieldEvents, foldEngineOptionAliases, formulaPlanEntry, + handBackWithheldInsertReadonly, hookMatchesObject, hydrateWriteFormulas, insertStaticReadonlySubject, + judgeWhereAdmission, lowerWhereFilterArray, masterIdOf, mergeReadContext, organizationPredicateDeleteRefusal, + packageBodyCollectionRefusal, packageBodyObjectAndAppOwnerStampRefusal, packageBodySeedDataRefusal, + planFormulaProjection, planStoredExpressions, recomputeAutonumberReaders, rejectUnknownEngineOptions, + replacedStoredExpressionFields, resolveWhereFilterTokens, rowsWithoutFormulaWidening, settleBulkStoredExpressions, + shouldSkipStateMachine, statesDeleteSemantics, staticReadonlyStoredView, storedExpressionsReading, + stripComputedWriteFields, undeclaredWriteFieldErrors, updateStaticReadonlySubject, whereResolvedTokenJudge, + withFileOwnerKey, withholdInsertReadonlyFromHooks, withoutFormulaWidening, writeFailureLogMeta, +} from './engine-top-level.js'; +import type { + AdmittedValueShapeViolationTally, AutonumberReaders, CascadeDeleteSet, CascadeIssuedDelete, DatasourceDef, + EngineMiddleware, EngineReadOptions, FieldOwnedFileReadAuthorizer, FormulaFaultSink, FormulaPlanEntry, + FormulaRelated, FunctionEntry, FunctionRegistrationOptions, HeldFileResolver, HookEntry, HookHandler, + IssuedAutonumber, MigrationFlagRead, OperationContext, PermissionResolution, ReadableIdsChain, ReadableIdsSecurity, + ReadableIdsSecurityResolution, StoredExpressionServing, TransactionScope, UnservableControlledByParentMaster, + WriteWallVerdict, +} from './engine-top-level.js'; +// [#29986 codemod] Re-exported by name, so every importer of `./engine.js` reads what it read. +export { + type AdmittedValueShapeViolationTally, type CascadeDeleteSet, type DatasourceDef, ENGINE_COUNT_OPTION_KEYS, + ENGINE_OPTION_KEY_SETS, ENGINE_WHERE_SLOTS, type EngineMiddleware, type EngineReadOptions, + type FieldOwnedFileReadAuthorizer, type FormulaPlanEntry, type FunctionEntry, type FunctionRegistrationOptions, + HOOK_EVENT_DISPATCH_VOCABULARY, type HeldFileResolver, type HookEntry, type HookHandler, type MigrationFlagRead, + type OperationContext, type PermissionResolution, type PostHookWriteImageCheck, type ReadableIdsChain, + type ReadableIdsSecurity, SECURITY_SERVICE_SOURCE_SLOT, type UnservableControlledByParentMaster, + type WriteWallRowVerdict, type WriteWallVerdict, assembledViewItemRefusal, cascadeMemberKey, cascadeSetAdd, + cascadeSetHas, collectionItemOwnerStampRefusal, declaredDatetimeLowering, declaredMultiValued, droppedFieldEvents, + evaluateFormulaField, foldEngineOptionAliases, hookMatchesObject, insertStaticReadonlySubject, isExpressionDefault, + lowerWhereFilterArray, masterIdOf, mergeReadContext, packageBodyCollectionRefusal, + packageBodyObjectAndAppOwnerStampRefusal, packageBodySeedDataRefusal, permissionSetOwnerRegistrationRefusal, + registerAppNotingStamp, rejectUnknownEngineOptions, relationSitesOf, resolveThenLowerWhere, servedAsStored, + shouldSkipStateMachine, stampObjectPackage, stampPackage, staticReadonlyStoredView, stripComputedWriteFields, + undeclaredRelatedFields, undeclaredWriteFieldErrors, updateStaticReadonlySubject, viewsEntryNotContainerRefusal, + writeFailureLogMeta, +} from './engine-top-level.js'; +// [#29986 codemod] Class head: units, hooks, functions, body runners, hook dispatch: the methods of the same names below hand the bodies what they read (`HooksAndFunctionsHost`). +import { + dispatchUnscopedMultiWriteHooks, unregisterFunctionsByPackage, unregisterHooksByPackage, +} from './engine-hooks-and-functions.js'; +// [#29986 codemod] Action system and packaged-action activation: the methods of the same names below hand the bodies what they read (`ActionsHost`). +import { listRegisteredActions, registerAction } from './engine-actions.js'; +// [#29986 codemod] Registration seams, readable-ids security, formula and default permissions: the methods of the same names below hand the bodies what they read (`SecurityResolversHost`). +import { refuseUncomposableReadableIdsClasses, resolveFormulaRelated } from './engine-security-resolvers.js'; +// [#29986 codemod] Authz invalidation, middleware, session, driver options, posture, organization probe: the methods of the same names below hand the bodies what they read (`MiddlewareAndContextHost`). +import { executeWithMiddleware, resolveSystemInsertOrganization } from './engine-middleware-and-context.js'; +// [#29986 codemod] Field defaults and autonumbers: the methods of the same names below hand the bodies what they read (`DefaultsAndAutonumbersHost`). +import { + adoptExplicitAutonumber, applyAutonumbers, applyFieldDefaults, defaultFieldEntries, isIssuedAutonumberCollision, + resolveNowDefault, resolveOptionDefault, seedAutonumber, warnAutonumberReadersUnserved, +} from './engine-defaults-and-autonumbers.js'; +// [#29986 codemod] Package, app, plugin, driver and datasource registration: the methods of the same names below hand the bodies what they read (`RegistrationHost`). +import { + applyExternalDriverFloor, assertWriteAllowed, registerDatasourceDef, registerMetadataCollections, + registerPackageBody, supplyFileColumnsMovedResolver, +} from './engine-registration.js'; +// [#29986 codemod] Realtime, i18n, home amounts, validation messages, references, master-detail: the methods of the same names below hand the bodies what they read (`ReferencesAndMasterDetailHost`). +import { inspectCascadeDelete, inspectDanglingReferences } from './engine-references-and-master-detail.js'; +// [#29986 codemod] Crypto, secrets, internal fields, schema and datasource routing, init: the methods of the same names below hand the bodies what they read (`SecretsAndRoutingHost`). +import { resolveDatasourceBinding } from './engine-secrets-and-routing.js'; +// [#29986 codemod] Summary indexes, related-record expansion, file and reference hydration: the methods of the same names below hand the bodies what they read (`SummariesAndExpansionHost`). +import { buildSummaryIndex, initializeSummaryFields } from './engine-summaries-and-expansion.js'; +// [#29986 codemod] Query lowering, search, find and findOne: the methods of the same names below hand the bodies what they read (`FindHost`). +import { expandSearchOnAst, reportFindFailure, requireFindOnePredicate } from './engine-find.js'; +// [#29986 codemod] Write preparation and insert: the methods of the same names below hand the bodies what they read (`InsertHost`). +import { staticReadonlyCreateStrip } from './engine-insert.js'; +// [#29986 codemod] Cascade and referential delete planning: the methods of the same names below hand the bodies what they read (`CascadePlanningHost`). +import { + cascadeRelationBehavior, isTenantColumnSetNull, organizationCountIsDisclosable, referenceProbeFilterOfSet, + warnCascadeNotAtomic, +} from './engine-cascade-planning.js'; +// [#29986 codemod] Delete, count, aggregate and execute: the methods of the same names below hand the bodies what they read (`DeleteAndAggregateHost`). +import { aggregate } from './engine-delete-and-aggregate.js'; export class ObjectQL implements IObjectQLEngine { /** @@ -5775,20 +863,14 @@ export class ObjectQL implements IObjectQLEngine { * idempotent, and by app uninstall flows. */ unregisterHooksByPackage(packageId: string): number { - if (!packageId) return 0; - let removed = 0; - for (const [event, entries] of this.hooks.entries()) { - const before = entries.length; - const kept = entries.filter((e) => e.packageId !== packageId); - if (kept.length !== before) { - this.hooks.set(event, kept); - removed += before - kept.length; - } - } - if (removed > 0) { - this.logger.debug('Unregistered hooks by package', { packageId, removed }); - } - return removed; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readHooks = () => this.hooks; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get hooks() { return readHooks(); }, + get logger() { return readLogger(); }, + }; + return unregisterHooksByPackage.call(host, packageId); } /** @@ -5836,18 +918,14 @@ export class ObjectQL implements IObjectQLEngine { /** Remove all functions registered under a given `packageId`. */ unregisterFunctionsByPackage(packageId: string): number { - if (!packageId) return 0; - let removed = 0; - for (const [name, entry] of this.functions.entries()) { - if (entry.packageId === packageId) { - this.functions.delete(name); - removed += 1; - } - } - if (removed > 0) { - this.logger.debug('Unregistered functions by package', { packageId, removed }); - } - return removed; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readFunctions = () => this.functions; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get functions() { return readFunctions(); }, + get logger() { return readLogger(); }, + }; + return unregisterFunctionsByPackage.call(host, packageId); } /** @@ -6317,52 +1395,17 @@ export class ObjectQL implements IObjectQLEngine { * context would retarget nothing — refused rather than ignored, the same * rule as D4 and #6752 (`HookTargetRebindError`, path `'unscoped-multi'`). */ - private async dispatchUnscopedMultiWriteHooks( + private dispatchUnscopedMultiWriteHooks( event: 'beforeUpdate' | 'beforeDelete', object: string, batchCtx: HookContext, ): Promise { - const entries = this.hooks.get(event); - if (!entries || entries.length === 0) return; - const flagged = entries.filter( - (entry) => entry.dispatchUnscopedMultiWrite && hookMatchesObject(entry, object), - ); - if (flagged.length === 0) return; - // Mirrors `triggerHooks`' opt-out rule: `skipAutomations` suppresses only - // metadata-bound entries. No metadata binding can set the flag today, so - // this is parity with the dispatch loop rather than a reachable branch — - // kept so the two rules cannot drift if that ever changes. - const skipAutomations = - (batchCtx.session as { skipAutomations?: boolean } | undefined)?.skipAutomations === true; - const wholeOpCtx = { - ...batchCtx, - dispatch: { - mode: 'record', - index: 0, - scope: (batchCtx.dispatch as { scope: Record }).scope, - }, - } as unknown as HookContext; - for (const entry of flagged) { - if (skipAutomations && entry.meta) continue; - // [#29422] As in `triggerHooks`. - try { - await entry.handler(wholeOpCtx); - } catch (e) { - throw hookStoppedRefusal(e, event, object, entry.hookName); - } - } - // `input` is identity-shared with the batch context, so the observation - // covers everything the handlers touched. - const observed = (wholeOpCtx.input as { id?: unknown }).id; - if (observed !== undefined) { - throw new HookTargetRebindError({ - object, - event, - path: 'unscoped-multi', - expectedId: undefined, - observedId: observed, - }); - } + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readHooks = () => this.hooks; + const host: ThisParameterType = { + get hooks() { return readHooks(); }, + }; + return dispatchUnscopedMultiWriteHooks.call(host, event, object, batchCtx); } /** @@ -6420,21 +1463,14 @@ export class ObjectQL implements IObjectQLEngine { * replacing a package's handler, or the reverse, is warned too. */ registerAction(objectName: string, actionName: string, handler: (ctx: any) => Promise | any, packageName?: string): void { - const key = `${objectName}:${actionName}`; - const replaced = this.actions.get(key); - if (replaced !== undefined && replaced.package !== packageName) { - const by = (name: string | undefined) => (name ? `by package '${name}'` : 'with no package'); - this.logger.warn( - `Action handler '${key}' registered ${by(packageName)} replaces the one registered ${by(replaced.package)}: ` - + `the action door now runs the handler registered ${by(packageName)}, whichever declaration object ` - + `'${objectName}' lists${packageName ? `, and withdrawing package '${packageName}' leaves the action with no handler` : ''}. ` - + `Two owners share one handler key only when both declare action '${actionName}' on object ` - + `'${objectName}' — rename one of them.`, - { objectName, actionName, package: packageName, replacedPackage: replaced.package }, - ); - } - this.actions.set(key, { handler, package: packageName }); - this.logger.debug('Registered action', { objectName, actionName, package: packageName }); + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActions = () => this.actions; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get actions() { return readActions(); }, + get logger() { return readLogger(); }, + }; + return registerAction.call(host, objectName, actionName, handler, packageName); } /** @@ -6460,17 +1496,12 @@ export class ObjectQL implements IObjectQLEngine { * makes that refusal a checklist instead of a support ticket. */ listRegisteredActions(): Array<{ objectName: string; actionName: string; package?: string }> { - const out: Array<{ objectName: string; actionName: string; package?: string }> = []; - for (const [key, entry] of this.actions.entries()) { - const sep = key.indexOf(':'); - if (sep < 0) continue; - out.push({ - objectName: key.slice(0, sep), - actionName: key.slice(sep + 1), - ...(entry.package ? { package: entry.package } : {}), - }); - } - return out; + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActions = () => this.actions; + const host: ThisParameterType = { + get actions() { return readActions(); }, + }; + return listRegisteredActions.call(host); } /** @@ -7061,32 +2092,13 @@ export class ObjectQL implements IObjectQLEngine { operation: string, sites: readonly ComposableReadableIdsSite[], ): string[] { - const firstSite = new Map(); - for (const site of sites) if (!firstSite.has(site.leaf.object)) firstSite.set(site.leaf.object, site); - const refuse = (site: ComposableReadableIdsSite, why: string): Error => readableIdsRefusalError( - object, - `${operation}('${object}'): ${describeReadableIdsLeaf(site.at, site.leaf)}, and the engine composes no ` + - `such leaf ${why}. The query was NOT run and no driver was asked. Read this class through its bounded ` + - 'probe, chosen before the leaf is written.', - ); - const composable: string[] = []; - for (const [parent, site] of firstSite) { - // A parent the registry does not hold has no record anyone may read: - // its leaf is answered `$in: []` without asking anything, and it is in - // none of D5's classes. - if (!this._registry.getObject(parent)) continue; - const uncomposable = this.uncomposableReadableIdsParent(object, parent); - if (uncomposable?.class === 'object-scoped-read-middleware') { - throw refuse(site, `over '${parent}', whose rows an object-scoped read middleware narrows: that WHERE is ` + - `built at read time and is no part of its read scope, so the composed leaf would be wider than a read of '${parent}'`); - } - if (uncomposable?.class === 'other-datasource') { - throw refuse(site, `over '${parent}', which is bound to datasource '${String(uncomposable.parentDatasource)}' while '${object}' is ` + - `bound to '${String(uncomposable.objectDatasource)}': no driver evaluates a subquery across two datasources`); - } - composable.push(parent); - } - return composable; + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + uncomposableReadableIdsParent: (...a) => this.uncomposableReadableIdsParent(...a), + }; + return refuseUncomposableReadableIdsClasses.call(host, object, operation, sites); } /** @@ -7301,52 +2313,22 @@ export class ObjectQL implements IObjectQLEngine { * one written out, this refuses one reached through a formula), and it is * what keeps two objects whose titles read each other from recursing. */ - private async resolveFormulaRelated( + private resolveFormulaRelated( schema: any, plan: readonly FormulaPlanEntry[], rows: ReadonlyArray | undefined | null>, context: unknown, query?: EngineQueryOptions, ): Promise { - const wanted = new Map>(); - for (const entry of plan) { - if (entry.hopConflict) continue; - for (const [fk, named] of entry.hops ?? []) { - let set = wanted.get(fk); - if (!set) wanted.set(fk, (set = new Set())); - for (const name of named) set.add(name); - } - } - if (wanted.size === 0) return undefined; - if (query && this.formulaHopReads.has(query)) { - return `this object was itself read one hop through a reference by another object's formula, and a ` - + 'formula resolves ONE hop: a formula read this way does not read through a reference of its own ' - + '(a second hop). Store the value on this object as a field of its own to read it from there'; - } - // [#24410] The read set per reference is the UNION of every formula's - // columns, and the loader answers a whole reference `undeclared-field` when - // any column of it is undeclared — so one formula's misspelt column nulled - // every sibling reading through the same reference. The undeclared columns - // are taken out of the shared read here and judged per formula - // ({@link formulaHopScope}); the loader reads the declared rest. - const fields = (schema?.fields ?? {}) as Record; - const undeclared = new Map }>(); - for (const [fk, named] of wanted) { - const target = referenceTargetOf(fields[fk]); - // The loader's own verdicts stand for these: no target, or a hop into - // the stored-metadata-body family, which is never read. - if (!target || isStoredMetadataBodyObject(target)) continue; - const declared = (this._registry.getObject(target) as { fields?: Record } | undefined)?.fields; - const missing = undeclaredRelatedFields(declared, named); - if (missing.length === 0) continue; - undeclared.set(fk, { object: target, fields: new Set(missing) }); - for (const name of missing) named.delete(name); - if (named.size === 0) wanted.delete(fk); - } - const lookup = wanted.size > 0 - ? await this.resolveOneHopRelated(schema, wanted, rows, context, 'caller') - : () => undefined; - return { bindings: (row) => lookup(row), undeclared }; + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_registry = () => this._registry; + const readFormulaHopReads = () => this.formulaHopReads; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get formulaHopReads() { return readFormulaHopReads(); }, + resolveOneHopRelated: (...a) => this.resolveOneHopRelated(...a), + }; + return resolveFormulaRelated.call(host, schema, plan, rows, context, query); } /** @@ -7617,53 +2599,15 @@ export class ObjectQL implements IObjectQLEngine { /** * Execute an operation through the middleware chain */ - private async executeWithMiddleware(ctx: OperationContext, executor: () => Promise): Promise { - // [#11968] Advance the write epoch FIRST — ahead of every middleware, and - // so ahead of any `isSystem` bypass one of them applies. The writes most - // likely to change what a caller may see are system ones: the platform - // seeder, a package publish, the auto-org-admin grant. A guard that only - // saw user writes would leave cached authorization standing across exactly - // the grants that matter. (`@objectstack/plugin-security` made the same - // choice inside its own middleware in #10757; hoisting it here is what - // makes the covered set identical BY CONSTRUCTION to "everything the - // middleware chain sees", instead of identical by two authors agreeing.) - // - // ⛔ Do not move this below the `applicable` filter: that filter is a - // per-object middleware selector, and an epoch that only advanced when some - // middleware happened to be registered for the written object would be a - // seam with holes in it. - if (isWriteEpochOperation(ctx.operation)) { - this.writeEpoch.bump('write'); - } - - const applicable = this.middlewares.filter(m => - !m.object || m.object === '*' || m.object === ctx.object - ); - - let index = 0; - const next = async (): Promise => { - if (index < applicable.length) { - const mw = applicable[index++]; - await mw.fn(ctx, next); - } else { - try { - ctx.result = await executor(); - } catch (e) { - // [#21274] The engine BOUNDARY for every operation this seam runs - // (find, findOne, count, aggregate, insert, update, delete): a driver - // error leaves the engine with its bound statement and the caller's - // values cut, from `message`, `stack`, the driver's own - // statement-bearing properties and `cause` — the class and the codes - // kept. Innermost, so a middleware that logs what it caught gets - // the cut too. The executors' own log lines ran before this, from - // the raw error, and are unchanged. See `redactPropagatedDriverFault`. - throw redactPropagatedDriverFault(e); - } - } + private executeWithMiddleware(ctx: OperationContext, executor: () => Promise): Promise { + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + const readMiddlewares = () => this.middlewares; + const readWriteEpoch = () => this.writeEpoch; + const host: ThisParameterType = { + get middlewares() { return readMiddlewares(); }, + get writeEpoch() { return readWriteEpoch(); }, }; - - await next(); - return ctx.result; + return executeWithMiddleware.call(host, ctx, executor); } /** @@ -8119,57 +3063,19 @@ export class ObjectQL implements IObjectQLEngine { * than by exemption. They are ordered cheapest-first, so an ordinary write * pays two property reads and a regexp. */ - private async resolveSystemInsertOrganization( + private resolveSystemInsertOrganization( object: string, execCtx: ExecutionContext | undefined, rows: readonly Record[], ): Promise { - // Already carrying an organization on the context — a session write, or a - // system write that threaded one. Nothing to resolve; this is the shape the - // ruling asks every system write to reach. - if (carriesOrganization(execCtx?.tenantId)) return undefined; - // [ADR-0131 D9, C8 stage 8b] Under every posture a platform object has no - // exemption: with a tenant column it is in scope like any other object, so - // its organization-less system insert is stamped with the derived - // organization or refused below — never left NULL. The walled half landed - // in 18.x, after the stages that gave the walled writers (identity mail, - // global audit, declared webhooks) an organization or a sibling of their - // own. ⛔ Never re-introduce a per-table or per-posture cut here: a table - // whose rows are deployment-level declares no organization column - // (`sys_secret` and `sys_flow_credential` since the steps 5+6 landing, - // #24582) and exits below, on the missing tenant field. - const objectSchema = this._registry.getObject(object) as any; - // A federated object's schema is the REMOTE's (ADR-0015); the platform's - // injected column says nothing about it, which is the same reason - // `buildDriverOptions` withholds `tenantId` there. - if (isFederatedObject(objectSchema)) return undefined; - const tenantField = resolveTenantFieldName(objectSchema); - if (!tenantField) return undefined; - // A row that names its own organization has carried one explicitly. Only a - // row still missing it can fork a counter, so a batch where every row is - // stamped resolves nothing — and a batch where some are not is decided by - // the ones that are not. - if (rows.every((row) => carriesOrganization(row?.[tenantField]))) return undefined; - - const posture = this.resolveEnginePosture(); - // [ADR-0131 D9] Zero organizations is refused under `single` only where - // the composition registers the organization object; the composition that - // registers none keeps today's branch (`no-organization-object`). The - // probe itself is unchanged and answers `[]` for both — this is the one - // fact that tells them apart, read from the same registry the probe reads. - const decision = await resolveSystemWriteOrganization({ - posture, - probeOrganizations: () => this.probeInstallOrganizations(), - organizationObjectRegistered: !!this._registry.getObject(ORGANIZATION_OBJECT), - }); - if (decision.kind === 'derived') return decision.organizationId; - if (decision.kind === 'no-organization-object') return undefined; - throw new SystemWriteOrganizationRequiredError( - object, - posture, - decision.reason, - decision.organizationCount, - ); + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + probeInstallOrganizations: (...a) => this.probeInstallOrganizations(...a), + resolveEnginePosture: (...a) => this.resolveEnginePosture(...a), + }; + return resolveSystemInsertOrganization.call(host, object, execCtx, rows); } /** @@ -8251,16 +3157,8 @@ export class ObjectQL implements IObjectQLEngine { * week out and takes the identical per-type treatment. */ private resolveNowDefault(fieldType: unknown, instant: Date): string { - const iso = instant.toISOString(); // YYYY-MM-DDTHH:MM:SS.sssZ - if (fieldType === 'date') return iso.slice(0, 10); - if (fieldType === 'time') { - const timeOfDay = iso.slice(11, 23); // HH:MM:SS.fff - // Trim a zero-millisecond `.000` so a defaulted row is byte-canonical and - // still matches an equality filter against `'HH:MM:SS'` — the same trim - // the SQL driver's time DEFAULT and `canonicalTimeOfDay` apply. - return timeOfDay.endsWith('.000') ? timeOfDay.slice(0, 8) : timeOfDay; - } - return iso; + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`. + return resolveNowDefault(fieldType, instant); } /** @@ -8399,22 +3297,8 @@ export class ObjectQL implements IObjectQLEngine { * disagree in the direction that produces a false "always-valued". */ private resolveOptionDefault(field: { type?: unknown; options?: unknown; multiple?: unknown }): unknown { - const options = field.options; - if (!Array.isArray(options)) return undefined; - const marked: unknown[] = []; - for (const o of options) { - if (!o || typeof o !== 'object') continue; - if ((o as { default?: unknown }).default !== true) continue; - const value = (o as { value?: unknown }).value; - // An option marked default but carrying no value declares nothing this - // method can apply — skipped rather than stored as `null`, which would - // be indistinguishable from "no default" downstream. - if (value === undefined || value === null) continue; - marked.push(value); - } - if (marked.length === 0) return undefined; - // [#18408] Was `field.multiple === true`. See {@link declaredMultiValued}. - return declaredMultiValued(field) ? marked : marked[0]; + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`. + return resolveOptionDefault(field); } /** @@ -8458,105 +3342,19 @@ export class ObjectQL implements IObjectQLEngine { object: string, record: Record, execCtx?: ExecutionContext, - nowSnapshot?: Date, - permissions?: EvalPermissions, - ): Record { - const fieldEntries = this.defaultFieldEntries(object); - if (!fieldEntries) return record; - const out = { ...record }; - const now = nowSnapshot ?? new Date(); - for (const f of fieldEntries) { - // Apply the default when the field is either OMITTED (`undefined`) or an - // EXPLICIT `null` — both mean "no value supplied" on insert (#2706). A - // real value (including `''`) is respected. Insert-only path, so an - // intentional "set to null" on update is never touched here. - if (out[f.name] != null) continue; - if (f.defaultValue == null) { - // No `defaultValue` — fall back to the option marked `default: true`. - // - // Reached ONLY through this branch, which is the whole point of putting - // it here (#7246): an option's `value` is a plain literal by - // construction (`SystemIdentifierSchema`), never one of the runtime - // TOKENS below, so it must not be handed to `isCurrentUserDefaultToken` - // / `isNowDefaultToken`. An option literally spelled `current_user` - // stores those twelve characters — it is a picklist value, not an - // instruction — and that is pinned by test. - // - // The `dv == null` gate above is the presence test that decides - // precedence: `defaultValue: ''` is a REAL default (`'' == null` is - // false), so it wins and this fallback never fires for it, exactly as - // `''` supplied by a caller is respected as a real value. - const fromOption = this.resolveOptionDefault(f); - if (fromOption !== undefined) out[f.name] = fromOption; - continue; - } - const dv = f.defaultValue; - if (isExpressionDefault(dv)) { - const result = ExpressionEngine.evaluate(dv, { - now, - timezone: execCtx?.timezone, - user: execCtx?.userId ? { id: String(execCtx.userId), positions: execCtx?.positions ?? [] } : undefined, - org: execCtx?.tenantId ? { id: String(execCtx.tenantId) } : undefined, - permissions, - record: out, - extra: { object }, - }); - if (result.ok) { - // Normalized to the declared type's stored shape, never assigned - // verbatim: the temporal stdlib returns a `Date` and the contract - // names an ISO-8601 string (#7373 — {@link normalizeExpressionDefault}). - out[f.name] = this.normalizeExpressionDefault(f.type, result.value); - } else { - this.logger.warn('Failed to evaluate default expression', { - object, field: f.name, error: result.error, - }); - } - } else if (isCurrentUserDefaultToken(dv)) { - // `current_user` token → the acting user's id at insert time. Declarative - // counterpart to writing a beforeInsert hook; mirrors the 'NOW()' string - // convention and is resolved app-side per request (driver-agnostic), so - // `Field.user({ defaultValue: 'current_user' })` auto-fills the actor. - // When there is no authenticated user (system/anonymous), leave it unset - // and let required-validation decide — never stamp a bogus owner. - // - // The token spelling comes from `@objectstack/spec/data` - // (`DEFAULT_VALUE_TOKENS`), the one place the family is declared, so a - // driver's DDL reads the SAME set when deciding which `defaultValue`s - // may become a physical column DEFAULT. When the two sides disagreed, - // SQL emitted `DEFAULT 'current_user'` and the DATABASE overrode the - // "leave it unset" decision below with a literal non-id (#4560). - if (execCtx?.userId != null) out[f.name] = String(execCtx.userId); - } else if (isNowDefaultToken(dv)) { - // `NOW()` token → the insert-time clock, in the storage shape the - // field's declared type calls for ({@link resolveNowDefault}). - // - // The mirror of the `current_user` crack above (#4597 / #4560): this - // token was understood by the SQL driver and NOT by the engine, so - // `out[f.name] = dv` sent the literal string `'NOW()'` to every - // driver. SQL hid it — `formatInput`'s safety net swapped in a real - // timestamp before the wire — while memory/mongodb have no such net, - // so the same declaration behaved differently per datasource. That - // split surfaced two ways: a validation-visible field was REJECTED by - // the engine's own write validator ("must be a valid datetime"), and a - // `readonly`/`system` field — which `validateRecord` then skipped, i.e. - // the ~100 `created_at`/`updated_at` platform declarations — silently - // stored the four characters `NOW()`. (Since #21663 a readonly value's - // shape is judged too, so that literal would now be refused.) - // - // Resolved from the caller's `nowSnapshot`, so every defaulted field - // in one insert (and every row of one batch) carries the SAME instant. - // - // The driver's own now-handling is unchanged and stays as defence in - // depth: `SqlDriver.formatInput`'s safety net (now unreachable from - // this path) and the native column DEFAULT, which still serves writes - // that bypass the engine entirely — the same division of labour - // `current_user` has. - out[f.name] = this.resolveNowDefault(f.type, now); - } else { - out[f.name] = dv; - } - } - return out; + nowSnapshot?: Date, + permissions?: EvalPermissions, + ): Record { + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + const readLogger = () => this.logger; + const host: ThisParameterType = { + defaultFieldEntries: (...a) => this.defaultFieldEntries(...a), + get logger() { return readLogger(); }, + normalizeExpressionDefault: (...a) => this.normalizeExpressionDefault(...a), + resolveNowDefault: (...a) => this.resolveNowDefault(...a), + resolveOptionDefault: (...a) => this.resolveOptionDefault(...a), + }; + return applyFieldDefaults.call(host, object, record, execCtx, nowSnapshot, permissions); } /** @@ -8569,12 +3367,11 @@ export class ObjectQL implements IObjectQLEngine { private defaultFieldEntries(object: string): Array<{ name: string; type?: unknown; defaultValue?: unknown; options?: unknown; multiple?: unknown; }> | undefined { - const fieldsRaw = (this.getSchema(object) as any)?.fields; - if (!fieldsRaw || typeof fieldsRaw !== 'object') return undefined; - // `fields` may be a Record (canonical) or an array (legacy). - return Array.isArray(fieldsRaw) - ? fieldsRaw - : Object.entries(fieldsRaw).map(([name, def]) => ({ name, ...(def as object) })); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + const host: ThisParameterType = { + getSchema: (...a) => this.getSchema(...a), + }; + return defaultFieldEntries.call(host, object); } /** @@ -8670,61 +3467,21 @@ export class ObjectQL implements IObjectQLEngine { * violation is one of its own numbers. An adopted (caller-supplied) value is * deliberately NOT listed: it is not the engine's to re-issue. */ - private async applyAutonumbers( + private applyAutonumbers( object: string, record: Record, execCtx?: ExecutionContext, driverOwnsAutonumber?: boolean, ): Promise { - if (driverOwnsAutonumber) return []; // driver generates persistently in create() - const fields = (this.getSchema(object) as any)?.fields; - if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return []; - const now = new Date(); - const timezone = execCtx?.timezone; - const issued: IssuedAutonumber[] = []; - for (const [name, def] of Object.entries(fields)) { - if ((def as any)?.type !== 'autonumber') continue; - // The contract answers "which format?" — canonical `autonumberFormat`, - // then the `format` shorthand (#1603), then the declared default - // `{0000}`. One resolver, shared with the SQL driver, so a format-less - // field cannot render two shapes again (#6555; see the docstring above). - // `fmt` is kept only for the diagnostic below, which now names the format - // actually rendered with rather than `undefined`. - const fmt = resolveAutonumberFormat(def as never); - const tokens = parseAutonumberFormat(fmt); - const current = record[name]; - // Respect an explicit value — reachable only for an EXEMPT writer now - // (isSystem / preserveAudit / a hook stamp): #5503's strip removed every - // other caller's value before this method was called. Respecting it is - // unchanged; what is new is that the counter LEARNS from it (#6806). - if (current != null && current !== '') { - this.adoptExplicitAutonumber(object, name, tokens, record, String(current), now, timezone); - continue; - } - // Refuse to generate when an interpolated `{field}` is empty — it would - // render to an empty prefix and merge this record into the wrong counter - // scope. Mirror the SQL driver so both paths fail identically (#1603). - const missing = missingFieldValues(tokens, record); - if (missing.length > 0) { - throw new Error( - `Cannot generate autonumber "${object}.${name}" (format "${fmt}"): ` + - `referenced field(s) [${missing.join(', ')}] are empty on the record. ` + - `Fields interpolated into an autonumber format must be set before the record is created.`, - ); - } - // The counter scope is the rendered prefix (date/field tokens before the - // sequence slot); it is independent of the counter value, so a throwaway - // render with seq 0 yields the scope and the literal prefix to seed from. - const probe = renderAutonumber({ tokens, seq: 0, record, now, timezone }); - const counterKey = `${object}.${name}.${probe.scope}`; - let next = this.autonumberCounters.get(counterKey); - if (next == null) next = await this.seedAutonumber(object, name, probe.prefix, probe.suffix, execCtx); - next += 1; - this.autonumberCounters.set(counterKey, next); - record[name] = renderAutonumber({ tokens, seq: next, record, now, timezone }).value; - issued.push({ field: name, counterKey }); - } - return issued; + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + const readAutonumberCounters = () => this.autonumberCounters; + const host: ThisParameterType = { + adoptExplicitAutonumber: (...a) => this.adoptExplicitAutonumber(...a), + get autonumberCounters() { return readAutonumberCounters(); }, + getSchema: (...a) => this.getSchema(...a), + seedAutonumber: (...a) => this.seedAutonumber(...a), + }; + return applyAutonumbers.call(host, object, record, execCtx, driverOwnsAutonumber); } /** @@ -8774,25 +3531,14 @@ export class ObjectQL implements IObjectQLEngine { now: Date, timezone?: string, ): void { - let probe: ReturnType; - try { - // An empty `{field}` would render an empty prefix and point at the wrong - // counter — the same hazard the generating branch throws on. Here it is a - // reason to learn nothing, never a reason to fail the caller's write. - if (missingFieldValues(tokens, record).length > 0) return; - probe = renderAutonumber({ tokens, seq: 0, record, now, timezone }); - } catch { - return; - } - const counterKey = `${object}.${field}.${probe.scope}`; - const seeded = this.autonumberCounters.get(counterKey); - if (seeded == null) return; // not seeded yet — the first seed scan will read this row - const supplied = readStoredAutonumberCounter(value, probe.prefix, probe.suffix); - if (supplied == null || supplied <= seeded) return; - this.autonumberCounters.set(counterKey, supplied); - this.logger.debug('Autonumber counter lifted to an externally supplied value', { - object, field, counterKey, from: seeded, to: supplied, - }); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + const readAutonumberCounters = () => this.autonumberCounters; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get autonumberCounters() { return readAutonumberCounters(); }, + get logger() { return readLogger(); }, + }; + return adoptExplicitAutonumber.call(host, object, field, tokens, record, value, now, timezone); } /** @@ -9005,11 +3751,8 @@ export class ObjectQL implements IObjectQLEngine { * attributable and a differently-named one does not. */ private isIssuedAutonumberCollision(error: unknown, issued: IssuedAutonumber[]): boolean { - if (issued.length === 0) return false; - if (!isUniqueViolationError(error)) return false; - const column = uniqueViolationColumn(error); - if (column === undefined) return true; - return issued.some((one) => one.field === column); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`. + return isIssuedAutonumberCollision(error, issued); } /** @@ -9022,19 +3765,14 @@ export class ObjectQL implements IObjectQLEngine { * said once per object per engine instance, with the remedy. */ private warnAutonumberReadersUnserved(object: string, plan: readonly FormulaPlanEntry[], driver: unknown): void { - if (this.autonumberReadersUnservedReported.has(object)) return; - this.autonumberReadersUnservedReported.add(object); - const fields = plan.map((entry) => entry.name); - const driverName = (driver as { name?: unknown } | null)?.name; - this.logger.error( - `Stored field(s) [${fields.join(', ')}] on '${object}' read a record number that driver ` + - `'${String(driverName ?? 'unknown')}' issues only inside create(), so every insert stores them computed ` + - 'WITHOUT the number while the write reports success; an update by id recomputes them later. ' + - 'Use a driver that implements IDataDriver.issueAutonumbers (driver-sql and the drivers built on it do), ' + - 'or keep the record number out of the stored expression. Reported once per object.', - undefined, - { object, fields, driver: driverName }, - ); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + const readAutonumberReadersUnservedReported = () => this.autonumberReadersUnservedReported; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get autonumberReadersUnservedReported() { return readAutonumberReadersUnservedReported(); }, + get logger() { return readLogger(); }, + }; + return warnAutonumberReadersUnserved.call(host, object, plan, driver); } /** @@ -9117,77 +3855,18 @@ export class ObjectQL implements IObjectQLEngine { * matching is LOOSER (a case-insensitive `LIKE`) must not be able to inflate * the max with another scope's rows. */ - private async seedAutonumber( + private seedAutonumber( object: string, field: string, prefix: string, suffix: string, execCtx?: ExecutionContext, ): Promise { - try { - // Canonical `fields`, not the wire spelling `select` — this call sat on - // the exact #4371 silent drop (the projection never applied; the scan - // worked only because an unprojected row still carries `field`), and the - // catch below would have swallowed the guard's rejection into "seed - // from 0", i.e. duplicate autonumbers. - const walk = keysetWalk>( - (q: KeysetPageQuery) => this.find(object, { - ...q, - fields: ['id', field], - context: execCtx, - } as any), - { - where: prefix ? { [field]: { $startsWith: prefix } } : undefined, - pageSize: AUTONUMBER_SEED_PAGE_SIZE, - }, - ); - let max = 0; - for await (const page of walk.pages()) { - for (const r of page) { - const v = r?.[field]; - if (v == null) continue; - // The reading itself lives in `readStoredAutonumberCounter` (the - // section above describes it) because #6806's adopt-on-exempt-write - // resync must read a supplied value by the SAME rules this scan reads - // a stored one — two copies of it would drift into two different - // answers for one row, which is a duplicate record number. Its - // anchored half is spec's `readAutonumberCounter`, the one the SQL - // driver's own scan calls (#6560). - const counter = readStoredAutonumberCounter(String(v), prefix, suffix); - if (counter != null) max = Math.max(max, counter); - } - } - // The walk is unbounded (no `max`), so truncation here means the scan - // could not COMPLETE: a row carried no `id` to seek past, or the reader - // never applied the seek predicate. Either way rows were left unread, and - // the max over what was read is a floor, not the max. Answering with it - // is the "seed below the real MAX" defect this method was fixed for, so - // it fails loudly instead — the same disposition #6114 gave the read - // outage: allocate nothing, write nothing. - if (walk.truncated) { - throw new Error( - `Cannot seed the autonumber counter for "${object}.${field}": the seeding scan ` + - `could not visit every stored row (it stopped after ${walk.scanned} rows without ` + - `reaching the end). Seeding from a partial scan would issue record numbers that ` + - `collide with existing ones, so no number was allocated.`, - ); - } - return max; - } catch (error) { - // [#5979] Discriminate by error TYPE. Seeding from 0 is the truth for - // exactly ONE failure reason — the table has not been provisioned, so - // there are genuinely no rows and number 1 collides with nothing. - if (isMissingTableError(error, object)) return 0; - // Every other failure (connection drop, timeout, permission denial, - // query error) means the rows may well exist and simply were not seen. - // Answering 0 there restarts the sequence at 1 against a table already - // holding N rows and issues autonumbers that COLLIDE with existing ones - // — a value written wrong, which no retry and no restart repairs. So the - // read failure propagates and the caller allocates nothing: the write - // fails loudly instead of succeeding with a forged business identifier. - // This is the hazard the #4371 comment above the read already named. - throw error; - } + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + const host: ThisParameterType = { + find: (...a) => this.find(...a), + }; + return seedAutonumber.call(host, object, field, prefix, suffix, execCtx); } /** @@ -9416,128 +4095,17 @@ export class ObjectQL implements IObjectQLEngine { source: Exclude, scope: ArtifactInstallScope | undefined, ): void { - // 1. Register the Package (manifest + lifecycle state). - // [ADR-0130 D1/D3] `scope` is the artifact's own package list, passed by - // the load path that read the artifact. It is what lets the install gate - // tell a CO-OWNER (another package from this same artifact) from a - // stranger — the question D1 corrected the gate to ask. Absent for every - // single-package caller, where the gate behaves exactly as before (D7). - this._registry.installPackage(manifest, source, undefined, scope); - this.logger.debug('Installed Package', { id: manifest.id, name: manifest.name, namespace }); - - // 2. Register owned objects - if (manifest.objects) { - if (Array.isArray(manifest.objects)) { - this.logger.debug('Registering objects from manifest (Array)', { id, objectCount: manifest.objects.length }); - for (const objDef of manifest.objects) { - const fqn = this._registry.registerObject(stampObjectPackage(objDef, id, version), id, namespace, 'own'); - this.logger.debug('Registered Object', { fqn, from: id }); - } - } else { - this.logger.debug('Registering objects from manifest (Map)', { id, objectCount: Object.keys(manifest.objects).length }); - // `manifest` is `any` (a raw authored manifest), so the map branch - // widens its values to `unknown`. State the contract once, on the - // entries, rather than casting the argument at the call: the map - // values ARE authored `ServiceObject`s — the INPUT shape, which is - // what `registerObject` takes since ADR-0122 phase 2 (commit 53068c130). - for (const [name, objDef] of Object.entries(manifest.objects) as [string, ServiceObject][]) { - // Ensure name in definition matches key - objDef.name = name; - const fqn = this._registry.registerObject(stampObjectPackage(objDef, id, version), id, namespace, 'own'); - this.logger.debug('Registered Object', { fqn, from: id }); - } - } - } - - // 2b. Register object extensions (fields added to objects owned by other packages) - if (Array.isArray(manifest.objectExtensions) && manifest.objectExtensions.length > 0) { - this.logger.debug('Registering object extensions', { id, count: manifest.objectExtensions.length }); - for (const ext of manifest.objectExtensions) { - const targetFqn = ext.extend; - const priority = ext.priority ?? 200; - // Create a partial object definition for the extension - const extDef = { - name: targetFqn, // Use the target FQN as name - fields: ext.fields, - // [ADR-0146 D10, second rule] The per-field display patches, - // the slot as `ObjectExtensionSchema.fieldPatches` declares - // it — applied by the registry's `mergeObjectDefinitions`, - // whose fold every object read shares. - fieldPatches: ext.fieldPatches, - label: ext.label, - pluralLabel: ext.pluralLabel, - description: ext.description, - validations: ext.validations, - indexes: ext.indexes, - }; - // Register as extension (namespace is undefined since we're targeting by FQN) - this._registry.registerObject(stampPackage(extDef, id, version), id, undefined, 'extend', priority); - this.logger.debug('Registered Object Extension', { target: targetFqn, priority, from: id }); - } - } - - // 3. Register apps (UI navigation definitions) as their own metadata type - // Resolve short objectName references in navigation to FQN so the - // Console UI can match them against the object registry. - if (Array.isArray(manifest.apps) && manifest.apps.length > 0) { - this.logger.debug('Registering apps from manifest', { id, count: manifest.apps.length }); - for (const app of manifest.apps) { - const appName = app.name || app.id; - if (appName) { - const resolved = namespace ? this.resolveNavObjectNames(app, namespace) : app; - registerAppNotingStamp(this._registry, resolved, id, version); - this.logger.debug('Registered App', { app: appName, from: id }); - } - } - } - - // 4. If manifest itself looks like an App (has navigation), also register as app - // This handles the case where the manifest IS the app definition (legacy/simple packages) - if (manifest.name && manifest.navigation && !manifest.apps?.length) { - const resolved = namespace ? this.resolveNavObjectNames(manifest, namespace) : manifest; - registerAppNotingStamp(this._registry, resolved, id, version); - this.logger.debug('Registered manifest-as-app', { app: manifest.name, from: id }); - } - - // (Navigation contributions, ADR-0029 D7, register in `registerApp` - // once this body has registered whole.) - - // 5. Register all other metadata types generically — the SAME seam the - // nested-plugin path uses (`registerMetadataCollections`), so the two - // entry points cannot answer differently for one collection (#7163). - this.registerMetadataCollections(manifest, id, 'manifest', version); - - // 6. Register seed data as metadata (keyed by target object name) - const seedData = (manifest as any).data; - if (Array.isArray(seedData) && seedData.length > 0) { - this.logger.debug('Registering seed data datasets', { id, count: seedData.length }); - for (const dataset of seedData) { - if (dataset.object) { - // [#28534] The collection loop's owner-stamp refusal for the - // one collection filed outside it was judged before anything - // registered ({@link packageBodySeedDataRefusal}). - const stampBefore = (dataset as { _packageId?: unknown })._packageId; - this._registry.registerItem('data', stampPackage(dataset, id, version), 'object' as any, id); - noteRegistrationStamp(dataset, stampBefore); - } - } - } - - // (Contributed kinds register in `registerApp` once this body has - // registered whole: they belong to no package, so no restore of a - // refused body could take them back.) - - // 7. Recursively register nested plugins - if (Array.isArray(manifest.plugins) && manifest.plugins.length > 0) { - this.logger.debug('Processing nested plugins', { id, count: manifest.plugins.length }); - for (const plugin of manifest.plugins) { - if (plugin && typeof plugin === 'object') { - const pluginName = plugin.name || plugin.id || 'unnamed-plugin'; - this.logger.debug('Registering nested plugin', { pluginName, parentId: id }); - this.registerPlugin(plugin, id, namespace, version); - } - } - } + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const read_registry = () => this._registry; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get logger() { return readLogger(); }, + registerMetadataCollections: (...a) => this.registerMetadataCollections(...a), + registerPlugin: (...a) => this.registerPlugin(...a), + resolveNavObjectNames: (...a) => this.resolveNavObjectNames(...a), + }; + return registerPackageBody.call(host, manifest, id, namespace, version, source, scope); } /** @@ -9617,88 +4185,14 @@ export class ObjectQL implements IObjectQLEngine { * @param sourceLabel Human-readable source name for the `debug` line. */ private registerMetadataCollections(source: any, ownerId: string, sourceLabel: string, ownerVersion?: string) { - // `ownerVersion` is the owning package's version, stamped like `ownerId`. - for (const key of METADATA_ARRAY_KEYS) { - const items = (source as any)?.[key]; - if (!Array.isArray(items) || items.length === 0) continue; - this.logger.debug(`Registering ${key} from ${sourceLabel}`, { id: ownerId, count: items.length }); - // A `picklistExtensions` entry is not an item: it has no name, only - // the list it adds to, and the registry merges it there — additive - // only, a repeated value refused loudly (`registerPicklistExtension`). - if (key === 'picklistExtensions') { - for (const extension of items) this._registry.registerPicklistExtension(extension, ownerId); - continue; - } - for (const item of items) { - const itemName = resolveMetadataItemName(key, item); - if (!itemName) { - this.logger.warn(`Skipping ${pluralToSingular(key)} without a derivable name`, { id: ownerId }); - continue; - } - // [#28697] Every refusal this seam used to raise here — the - // divergent view container, the permission set's owner, an - // item's own `_packageId`, a `views:` entry that is not a - // container — is judged from the body alone, so it is asked - // before anything of the package registers, with the reasons it - // exists: {@link packageBodyCollectionRefusal}. Both callers ask - // it first (`registerApp`, for the body and each nested plugin). - const toRegister = item.name === itemName ? item : { ...item, name: itemName }; - const stampBefore = (toRegister as { _packageId?: unknown })._packageId; - this._registry.registerItem(pluralToSingular(key), stampPackage(toRegister, ownerId, ownerVersion), 'name' as any, ownerId); - noteRegistrationStamp(toRegister, stampBefore); - // "Object has-many View" (ADR-0017): a `defineView` document - // aggregates an object's views. Register the container under the - // bare key (above, back-compat) AND expand it into - // independent ViewItems registered under ., so - // `getViewsByObject()` / `GET /meta/view?object=` surface the - // per-view `package` layer the switcher + Studio consume. - if (key === 'views' && isAggregatedViewContainer(toRegister)) { - for (const vi of expandViewContainer(itemName, toRegister)) { - for (const w of vi._diagnostics?.warnings ?? []) { - this.logger.warn(`View expansion warning for '${vi.name}': ${w.message}`, { from: ownerId }); - } - this._registry.registerItem('view', stampPackage(vi, ownerId, ownerVersion), 'name' as any, ownerId); - } - } - } - } - - // [#5320] The `viewItems:` channel — the declared entry for the - // NON-container view artifacts a runtime-ASSEMBLED manifest carries - // (tenant-authored standalone ViewItems, flattened overlays, expanded - // items a travelling container cannot re-derive). Written by the manifest - // assemblers (`partitionAssembledViewArtifacts` — package export, the - // artifact factories); refused at the authoring door (`defineStack` types - // the key `never`), so only machine-assembled manifests legitimately - // reach here carrying it. Strictly schema'd: each entry is judged by - // `AssembledViewArtifactSchema` and the PARSED body is what registers, so - // an undeclared bag neither passes nor rides through — declared = - // enforced, in both directions. - const assembledItems = (source as any)?.[ASSEMBLED_VIEW_ITEMS_KEY]; - if (Array.isArray(assembledItems) && assembledItems.length > 0) { - this.logger.debug(`Registering ${ASSEMBLED_VIEW_ITEMS_KEY} from ${sourceLabel}`, { id: ownerId, count: assembledItems.length }); - for (const item of assembledItems) { - // [#28697] The entry's parse refusal and its owner stamp were - // judged before anything registered ({@link packageBodyCollectionRefusal}); - // the parse is repeated for the body it registers. - const parsed = AssembledViewArtifactSchema.safeParse(item); - if (!parsed.success) throw assembledViewItemRefusal(item, parsed.error, sourceLabel, ownerId); - const body = parsed.data as Record; - const itemName = resolveMetadataItemName('views', body); - if (!itemName) { - this.logger.warn('Skipping viewItems entry without a derivable name', { id: ownerId }); - continue; - } - // No {@link noteRegistrationStamp} follows the `registerItem` below, on - // purpose: what it stamps is `parsed.data`, the parse's own new - // object, never the caller's entry, so this site leaves no in-place - // stamp on anything a later registration could be handed - // (`collection-item-owner-stamp.test.ts` pins the caller's entry - // unstamped). - const toRegister = body.name === itemName ? body : { ...body, name: itemName }; - this._registry.registerItem('view', stampPackage(toRegister, ownerId, ownerVersion), 'name' as any, ownerId); - } - } + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const read_registry = () => this._registry; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get logger() { return readLogger(); }, + }; + return registerMetadataCollections.call(host, source, ownerId, sourceLabel, ownerVersion); } /** @@ -9820,21 +4314,13 @@ export class ObjectQL implements IObjectQLEngine { * the engine never contradicts an explicit composition. */ private supplyFileColumnsMovedResolver(driver: IDataDriver): void { - const sink = driver as unknown as { - setFileColumnsMovedResolver?: (resolve: () => Promise) => void; + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readLogger = () => this.logger; + const host: ThisParameterType = { + haveFileColumnsMoved: (...a) => this.haveFileColumnsMoved(...a), + get logger() { return readLogger(); }, }; - if (typeof sink.setFileColumnsMovedResolver !== 'function') return; - try { - sink.setFileColumnsMovedResolver(() => this.haveFileColumnsMoved()); - } catch (e: any) { - // Installing the question must never break a driver registration. A - // driver that did not take the resolver keeps its own default, which is - // the JSON arm — the same place every other way of not knowing lands. - this.logger.debug('Driver declined the ADR-0104 media-arm resolver', { - driverName: driver.name, - error: e?.message ?? String(e), - }); - } + return supplyFileColumnsMovedResolver.call(host, driver); } /** @@ -9915,17 +4401,13 @@ export class ObjectQL implements IObjectQLEngine { * except an absent `driver` keeps the id stored before it. */ registerDatasourceDef(def: DatasourceDef): void { - if (!def?.name) return; - // [#27277] A def that names no driver keeps the id stored before it: a - // caller registering only the write-gate facts after a connect must not - // take the floor back off, so an absent key never reads as "not a plugin - // driver". It is a keep, not a source: `unregisterDriver` drops the stored - // def, so a connect that follows an eviction has nothing to keep — the - // datasource-connection service names `driver` on every registration - // ([#27666], `EngineDatasourceDef.driver`). - const driver = def.driver ?? this.datasourceDefs.get(def.name)?.driver; - this.datasourceDefs.set(def.name, { driver, schemaMode: def.schemaMode, external: def.external }); - this.applyExternalDriverFloor(def.name); + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readDatasourceDefs = () => this.datasourceDefs; + const host: ThisParameterType = { + applyExternalDriverFloor: (...a) => this.applyExternalDriverFloor(...a), + get datasourceDefs() { return readDatasourceDefs(); }, + }; + return registerDatasourceDef.call(host, def); } /** @@ -9940,25 +4422,18 @@ export class ObjectQL implements IObjectQLEngine { * identity checks) holds one instance. */ private applyExternalDriverFloor(name: string): void { - const current = this.drivers.get(name); - if (!current) return; - const floored = isExternalDriverFloor(current); - if (needsExternalDriverFloor(this.datasourceDefs.get(name))) { - if (floored) return; - this.drivers.set(name, createExternalDriverFloor(current, { - datasource: name, - fieldsOf: (object) => this._registry.getObject(object)?.fields, - // [#29762] Read from the def held NOW, on every read: a redefinition - // that keeps the floor returns above without re-wrapping the driver. - rowCap: () => this.datasourceDefs.get(name)?.external?.rowCap, - logger: this.logger, - })); - this.logger.info('Datasource served by a plugin driver: the engine applies each query in memory', { - datasource: name, - }); - } else if (floored) { - this.drivers.set(name, unwrapExternalDriverFloor(current)); - } + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const read_registry = () => this._registry; + const readDatasourceDefs = () => this.datasourceDefs; + const readDrivers = () => this.drivers; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get datasourceDefs() { return readDatasourceDefs(); }, + get drivers() { return readDrivers(); }, + get logger() { return readLogger(); }, + }; + return applyExternalDriverFloor.call(host, name); } /** @@ -10051,23 +4526,14 @@ export class ObjectQL implements IObjectQLEngine { * definition) are unaffected. */ private assertWriteAllowed(objectName: string, operation: 'insert' | 'update' | 'delete'): void { - const object = this._registry.getObject(objectName) as any; - const dsName = object?.datasource; - if (!dsName || dsName === 'default') return; - - const ds = this.datasourceDefs.get(dsName); - // No recorded definition, or an explicitly managed one ⇒ allow. - if (!ds || !ds.schemaMode || ds.schemaMode === 'managed') return; - - const dsAllows = ds.external?.allowWrites ?? false; - const objAllows = object?.external?.writable ?? false; - if (!(dsAllows && objAllows)) { - throw new ExternalWriteForbiddenError( - `Write '${operation}' blocked on object '${objectName}': datasource '${dsName}' is external ` + - `(schemaMode=${ds.schemaMode}). Requires datasource.external.allowWrites=true (got ${dsAllows}) ` + - `AND object.external.writable=true (got ${objAllows}).`, - ); - } + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const read_registry = () => this._registry; + const readDatasourceDefs = () => this.datasourceDefs; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get datasourceDefs() { return readDatasourceDefs(); }, + }; + return assertWriteAllowed.call(host, objectName, operation); } /** @@ -11227,22 +5693,20 @@ export class ObjectQL implements IObjectQLEngine { * budget never reached (#5718), `unreadableObjects` for what the datasource * refused, and `aborted` for a run called off (#4747). */ - async inspectDanglingReferences( + inspectDanglingReferences( options?: DanglingReferenceAuditOptions, ): Promise { - // Read once per run, live — the posture IN FORCE, by the engine's one reader. - const unionScope = postureUsesUnionScope(this.resolveEnginePosture()); - return auditDanglingReferences( - { - objects: () => this._registry.getAllObjects() as unknown as AuditableObject[], - find: (object, opts) => this.find(object, opts as any) as Promise>>, - probe: (target, id, organization) => this.referenceExists( - target, id, organization == null || unionScope ? undefined : ({ tenantId: organization } as ExecutionContext), - ), - warn: (msg, meta) => this.logger?.warn?.(msg, meta as any), - }, - options, - ); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). + const read_registry = () => this._registry; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + find: (...a) => this.find(...a), + get logger() { return readLogger(); }, + referenceExists: (...a) => this.referenceExists(...a), + resolveEnginePosture: (...a) => this.resolveEnginePosture(...a), + }; + return inspectDanglingReferences.call(host, options); } /** @@ -11274,23 +5738,18 @@ export class ObjectQL implements IObjectQLEngine { * dependents probe that fails for any reason but a table that was never * provisioned. ⛔ Never treat a throw as "no deeper details". */ - async inspectCascadeDelete( + inspectCascadeDelete( object: string, id: string | number, ): Promise> { - const resolved = this.resolveObjectName(object); - const set = await this.collectCascadeDeleteSet( - resolved, [id], undefined, this._registry.getAllObjects(), { fileElevations: false }, - ); - const root = { object: resolved, id: String(id) }; - const members = [root]; - for (const [memberObject, ids] of set.members) { - for (const memberId of ids) { - if (memberObject === root.object && memberId === root.id) continue; - members.push({ object: memberObject, id: memberId }); - } - } - return members; + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + collectCascadeDeleteSet: (...a) => this.collectCascadeDeleteSet(...a), + resolveObjectName: (...a) => this.resolveObjectName(...a), + }; + return inspectCascadeDelete.call(host, object, id); } /** @@ -12103,65 +6562,19 @@ export class ObjectQL implements IObjectQLEngine { /** Owning package, on `via: 'package'` only — for the debug line. */ packageId?: string; } | undefined { - const object = this._registry.getObject(objectName); - - // 1. Object's explicit datasource field (highest priority) - if (object?.datasource && object.datasource !== 'default') { - return { datasource: object.datasource, via: 'explicit' }; - } - - // 2. Check datasourceMapping rules - // - // A rule that MATCHES is a routing decision, not a hint (#4462). It used to - // fall through to steps 3-5 whenever the named datasource had no live - // driver, which put an object's rows in the DEFAULT store while every - // signal said otherwise: boot succeeded, `/ready` answered 200, the - // datasource name appeared nowhere in the log, and the write returned 201. - // An operator who routes an object to Postgres and gets the URL wrong finds - // out by going to look in Postgres and finding it empty. - // - // `default` is the one name that legitimately resolves onward: the default - // driver keeps its NATURAL name (#3826), so `drivers.has('default')` is - // false by construction and step 5 is how routing to it works. - const mappedDatasource = this.resolveDatasourceFromMapping(objectName, object); - if (mappedDatasource && mappedDatasource !== 'default') { - return { datasource: mappedDatasource, via: 'mapping' }; - } - - // 3. Lifecycle-class separation (ADR-0057 §3.6): high-frequency - // platform-generated data (telemetry / event) and the audit ledger move - // to a dedicated 'telemetry' datasource when one is registered, so their - // growth can never again pollute the business DB. `transient` stays on - // the primary deliberately: those objects are user-session data, and - // some (e.g. better-auth's sys_device_code) are also accessed outside - // the engine — splitting their storage would split their brain. - const lifecycleClass = (object as { lifecycle?: { class?: string } } | undefined)?.lifecycle?.class; - if ( - lifecycleClass !== undefined && - ObjectQL.SYSTEM_LEDGER_LIFECYCLE_CLASSES.has(lifecycleClass) && - this.drivers.has(ObjectQL.LIFECYCLE_DATASOURCE) - ) { - return { datasource: ObjectQL.LIFECYCLE_DATASOURCE, via: 'lifecycle' }; - } - - // 4. Check package's defaultDatasource - // Use the object's FQN name (from getObject) for ownership lookup - const fqn = object?.name || objectName; - const owner = this._registry.getObjectOwner(fqn); - if (owner?.packageId) { - const manifest = this.manifests.get(owner.packageId); - const packageDatasource = manifest?.defaultDatasource; - if (packageDatasource && packageDatasource !== 'default' && this.drivers.has(packageDatasource)) { - return { datasource: packageDatasource, via: 'package', packageId: owner.packageId }; - } - } - - // 5. Fallback to global default driver - if (this.defaultDriver && this.drivers.has(this.defaultDriver)) { - return { datasource: this.defaultDriver, via: 'default' }; - } - - return undefined; + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const read_registry = () => this._registry; + const readDefaultDriver = () => this.defaultDriver; + const readDrivers = () => this.drivers; + const readManifests = () => this.manifests; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get defaultDriver() { return readDefaultDriver(); }, + get drivers() { return readDrivers(); }, + get manifests() { return readManifests(); }, + resolveDatasourceFromMapping: (...a) => this.resolveDatasourceFromMapping(...a), + }; + return resolveDatasourceBinding.call(host, objectName); } /** @@ -13232,130 +7645,14 @@ export class ObjectQL implements IObjectQLEngine { byChild: Map; byParent: Map; } { - const index = new Map(); - const byParent = new Map(); - const objects: ServiceObject[] = this._registry.getAllObjects(); - for (const parent of objects as any[]) { - const fields = parent?.fields; - if (!fields || typeof fields !== 'object' || Array.isArray(fields)) continue; - for (const [summaryField, def] of Object.entries(fields)) { - const d: any = def; - if (d?.type !== 'summary' || !d.summaryOperations) continue; - const so = d.summaryOperations; - const childObject = so.object; - const fn = so.function; - if (!childObject || !fn) continue; - // Resolve the FK on the child pointing back to this parent. - let fkField: string | undefined = so.relationshipField; - // [#19082] Candidate relation fields on the child whose `reference` - // carrier no reader can read, spelled `.`. Collected - // rather than thrown on — see the two comments below. - const unreadableCarriers: string[] = []; - if (!fkField) { - const child = this._registry.getObject(childObject) as any; - const cfields = child?.fields || {}; - for (const [cfName, cdef] of Object.entries(cfields)) { - const cd: any = cdef; - if (cd?.type !== 'master_detail' && cd?.type !== 'lookup') continue; - // [#19082] The carrier is read through the ONE arbiter, the same - // one {@link ObjectQL.planCascadeAtomicity} and - // {@link ObjectQL.cascadeDeleteRelations} were routed through — - // but its refusal is CAUGHT here instead of propagated, for a - // reason those two seams do not have: this is a SCAN over every - // relation field on the child, looking for the one that points - // back. Letting the first unreadable carrier throw would hide a - // perfectly readable sibling that IS the foreign key, turning a - // roll-up that works today into a hard failure of every write. - // The three answers stay distinct: - // - // - ABSENCE (`undefined` / `null` / `''`) is unchanged and still - // silent — `referenceCarrierOf` answers `undefined` for all - // three, this `continue` skips the field, and a field that - // names no target is a legal thing to declare. - // - A READABLE carrier compares exactly as `cd.reference === - // parent.name` compared it, so every roll-up that resolved - // before resolves now. - // - UNREADABILITY is recorded, and it is what the skip below - // stops being silent about. - let carrier: string | undefined; - try { - carrier = referenceCarrierOf(cd, 'ObjectQL.buildSummaryIndex'); - } catch { - unreadableCarriers.push(`${childObject}.${cfName}`); - continue; - } - if (carrier === parent.name) { - fkField = cfName; - break; - } - } - } - if (!fkField) { - // [#19082] The skip is unchanged; its SILENCE is what ends here. - // - // An unreadable carrier compares false against every name, so - // `fkField` stayed unset and this `continue` dropped a DECLARED - // roll-up out of both indexes with no diagnostic anywhere — the - // second way this one function invents "nothing to recompute" (the - // first, the registry read, is #9154 above). It is a durability - // degradation by the repo's own test: every write keeps reporting - // success while a persisted summary value silently stops tracking - // its children, so it is an `error`, not a `warn`, and it owes both - // the consequence and the fix in the line it prints. - // - // Said once per INDEX BUILD, never once per write: - // {@link ensureSummaryIndexes} memoises the result against the - // registry's `objectRevision`, which moves only on a metadata - // mutation. And only the unreadable case speaks — a roll-up whose - // child genuinely declares no relation field at all is a different - // (and much louder at authoring time) condition, left exactly as it - // was rather than widened into here. - if (unreadableCarriers.length > 0) { - const msg = - `[summary-index] ${parent.name}.${summaryField} is NOT indexed and will NOT recompute: ` - + `its roll-up over '${childObject}' has no resolvable foreign key, because ` - + `${unreadableCarriers.length} candidate relation field(s) on '${childObject}' carry a ` - + `\`reference\` no reader can read (${unreadableCarriers.join(', ')}). ` - + `\`FieldSchema.reference\` is an optional STRING — the target object's name — and a ` - + `non-string carrier is refused by \`ObjectSchema.safeParse\`, so such a definition ` - + `reached this registry around the parse seam (a raw \`registerObject\`, or a metadata ` - + `row stored before that tightening). CONSEQUENCE: ${parent.name}.${summaryField} keeps ` - + `whatever value it holds now through EVERY insert / update / delete of '${childObject}', ` - + `while every one of those writes reports success and nothing else reports this. FIX: ` - + `spell the carrier as the target object name (reference: '${parent.name}') on the ` - + `child's master_detail/lookup field, or name the foreign key explicitly with ` - + `summaryOperations.relationshipField on ${parent.name}.${summaryField}.`; - // Sanctioned logger shape (PR #9750): reach for `error`, fall back - // to `warn` — ⛔ never an optional call like `logger.error?.()`, - // which emits nothing against a sink that has no `error`. - if (typeof this.logger.error === 'function') this.logger.error(msg); - else this.logger.warn(msg); - } - continue; // can't resolve the relationship — skip - } - // Optional per-summary predicate: only child rows matching it are - // aggregated (e.g. sum receipts where { status: 'received' }). ANDed with - // the parent-FK match at recompute time. Ignore a non-object filter. - const filter = so.filter && typeof so.filter === 'object' && !Array.isArray(so.filter) - ? so.filter as Record - : undefined; - const descriptor: SummaryDescriptor = { - parentObject: parent.name, summaryField, childObject, fkField, fn, sourceField: so.field, filter, - }; - const list = index.get(childObject) ?? []; - list.push(descriptor); - index.set(childObject, list); - // Same descriptor, parent-side view. Only descriptors that made it this - // far are indexed either way, so "seeded at insert" and "maintained by - // recompute" are the same set by construction — a roll-up whose - // relationship could not be resolved (the `continue` above) is left - // untouched on both paths rather than seeded with a 0 nothing updates. - const owned = byParent.get(parent.name) ?? []; - owned.push(descriptor); - byParent.set(parent.name, owned); - } - } - return { byChild: index, byParent }; + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts` (see `SummariesAndExpansionHost`). + const read_registry = () => this._registry; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get logger() { return readLogger(); }, + }; + return buildSummaryIndex.call(host); } /** `registry.objectRevision` the cached {@link summaryIndex} was built at. */ @@ -13482,18 +7779,11 @@ export class ObjectQL implements IObjectQLEngine { * stored with `null` stay `null` until a child write recomputes them. */ private initializeSummaryFields(object: string, record: any): any { - const descriptors = this.getOwnedSummaryDescriptors(object); - if (descriptors.length === 0) return record; - if (!record || typeof record !== 'object' || Array.isArray(record)) return record; - let out: Record = record; - for (const desc of descriptors) { - const seed = summaryEmptySetValue(desc.fn); - if (seed == null) continue; // min/max/avg — undefined on an empty set - if (out[desc.summaryField] != null) continue; // author supplied a value - if (out === record) out = { ...record }; - out[desc.summaryField] = seed; - } - return out; + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts` (see `SummariesAndExpansionHost`). + const host: ThisParameterType = { + getOwnedSummaryDescriptors: (...a) => this.getOwnedSummaryDescriptors(...a), + }; + return initializeSummaryFields.call(host, object, record); } /** @@ -14300,107 +8590,16 @@ export class ObjectQL implements IObjectQLEngine { * would hand the driver a key it does not read, which is the same silent drop * one layer down. */ - private async expandSearchOnAst( + private expandSearchOnAst( ast: QueryAST, schema: ServiceObject | undefined, context: ExecutionContext | undefined, ): Promise { - // The `$search`/`$searchFields` OData spellings are NOT read here: the - // protocol layer normalizes them to the bare keys before the engine - // (protocol.ts findData), and a direct engine call carrying one is an - // unknown option — rejected at the entry point, not silently dropped - // (#4371). - const raw = (ast as any).search; - // [#27732] An override naming no field is refused, never read as absent. - // Judged before the expansion and so before any driver read, whether or - // not a search term rode along, on both slots, as the door does. - refuseSearchOverrideNamingNoField(ast.object, (ast as any).searchFields, 'searchFields'); - if (raw !== null && typeof raw === 'object') refuseSearchOverrideNamingNoField(ast.object, raw.fields, 'search.fields'); - if (raw != null && schema?.fields) { - const requestedFields = (ast as any).searchFields - ?? (typeof raw === 'object' ? raw?.fields : undefined); - // [#22646] A `search` never scans an `internal: true` column. The flag's - // promise ("the value is never returned on a generic data exit") is a - // withhold on the read path (the row strip) and a refusal everywhere the - // value would be revealed by evaluation; `$search` is such a position — - // a cross-field `$icontains` that confirms a guessed prefix of a withheld - // credential, matching the row only when the guess is right. The field is - // dropped from the set the expansion resolves over (a WITHHOLD, like the - // auto-default's `hidden`/credential-type exclusions one layer down), so - // the default set never scans it, for every verb that reaches here - // (`find` / `findOne` / `aggregate`, and `searchAll` through them). An - // EXPLICIT `$searchFields` naming one is refused loudly at the generic - // data door instead (`assertSearchFieldsAreSearchable`, - // `@objectstack/metadata-protocol`), the #4254 posture — an unhonoured - // narrowing must be a 400, not a silent widening. A direct engine caller - // naming one beside a searchable field has it intersected away here, the - // engine's existing tolerant posture for an unknown requested search - // field; naming only such fields is refused below (#28120). - // [ADR-0100] …and never a `secret` / `password` column either, by the - // collector the credential guards read: the auto-default already skips - // those types, and an author-declared `searchableFields` naming one is - // withheld here the same way, so a search term is never evaluated over - // a value the read path masks. - const internalFields = [...new Set([...collectInternalReadFields(schema), ...collectCredentialFields(schema)])]; - const scanFields = internalFields.length === 0 - ? (schema.fields as any) - : Object.fromEntries( - Object.entries(schema.fields as Record) - .filter(([name]) => !internalFields.includes(name)), - ); - const expandOptions = { - fields: scanFields, - searchableFields: (schema as any).searchableFields, - requestedFields, - // [ADR-0079] `nameField` is the canonical primary-title pointer; - // `displayNameField` is the deprecated alias (still honored). - displayField: (schema as any).nameField ?? (schema as any).displayNameField, - }; - // [#28120] An override naming none of the fields the expansion would - // scan is refused, never widened back to the whole set. - refuseSearchOverrideNamingNoSearchableField( - ast.object, requestedFields, (ast as any).searchFields != null ? 'searchFields' : 'search.fields', - resolveSearchFieldResolution(expandOptions).allowed, - ); - // [#25672] …and never a column the CALLER may not query. The field-read - // predicate guard (plugin-security) runs after this expansion and judges - // the expanded `$or` as the caller's own predicate, so one hidden column - // in the server-resolved set refused the whole search with 403 — - // `sys_user`'s `role` / `ban_reason` / `last_login_ip` broke every user - // picker for members. The set is narrowed to the security service's - // `getQueryableFields` answer, the published complement of what that - // guard refuses (one derivation, no second field-permission rule here), - // asked with the context the middleware will see. A field the caller - // NAMED in `searchFields` is not withheld, so the guard still refuses it - // loudly. The guard itself is unchanged, and still judges every - // predicate the caller wrote. - const queryableFields = normalizeSearch(raw).query.trim() - ? await this.resolveSearchQueryableFields(ast.object, context) - : undefined; - const searchFilter = expandSearchToFilter(raw, { ...expandOptions, queryableFields }); - if (searchFilter) { - ast.where = ast.where ? { $and: [ast.where, searchFilter] } : searchFilter; - } else if (normalizeSearch(raw).query.trim()) { - // [#24284] A non-blank term that expanded to nothing means the RESOLVED - // search set is empty: `expandSearchToFilter` answers `null` for - // exactly two reasons, and the other one (a blank term) is excluded - // here. Leaving `where` untouched would answer the search with EVERY - // row, indistinguishable from "everything matched" — the silent - // widening #4254 refuses on the explicit-override axis. Judged on the - // set the expansion itself resolved (no second resolver), so it holds - // for every verb that reaches here (`find` / `findOne` / `aggregate` / - // the count) and every door (REST, dispatcher, SDK, a hook's - // `ctx.api`). An explicit override intersected to nothing is refused - // before the expansion (#28120) unless the allowed set is empty too, so - // it reaches this only then. [#25672] A set the caller's - // queryable fields withheld down to nothing gets the same answer: the - // declared set expands without the narrowing, so the refusal says so. - const withheld = queryableFields !== undefined && expandSearchToFilter(raw, expandOptions) !== null; - throw emptySearchSetError(ast.object, schema, internalFields.length > 0, withheld); - } - } - delete (ast as any).search; - delete (ast as any).searchFields; + // [#29986 codemod] The body moved to `engine-find.ts` (see `FindHost`). + const host: ThisParameterType = { + resolveSearchQueryableFields: (...a) => this.resolveSearchQueryableFields(...a), + }; + return expandSearchOnAst.call(host, ast, schema, context); } /** @@ -14557,20 +8756,8 @@ export class ObjectQL implements IObjectQLEngine { * driver, so it is a real answer and not a second silent drop. */ private requireFindOnePredicate(object: string, ast: QueryAST): void { - const where = ast.where as unknown; - const hasPredicate = - where != null && - (typeof where !== 'object' || Array.isArray(where) || Object.keys(where).length > 0); - if (hasPredicate) return; - if (Array.isArray(ast.orderBy) && ast.orderBy.length > 0) return; - throw new Error( - `findOne('${object}') selects no particular record: 'where' is absent or empty ` + - `and the query carries no 'orderBy'. findOne applies limit: 1, so this would return an ` + - `ARBITRARY row — a real, plausible-looking record unrelated to what was asked for, which ` + - `no caller's null-check can catch. Pass 'where' (or a 'search' that resolves to ` + - `one) to select the record; pass 'orderBy' if you mean "the first record in THIS order"; ` + - `or call find('${object}', { limit: 1 }) if any row will genuinely do.`, - ); + // [#29986 codemod] The body moved to `engine-find.ts`. + return requireFindOnePredicate(object, ast); } async find(object: string, query?: EngineQueryOptions, options?: EngineReadOptions): Promise { @@ -14881,17 +9068,12 @@ export class ObjectQL implements IObjectQLEngine { * `400` — a rejected request, answered to its caller. */ private reportFindFailure(object: string, error: unknown): void { - if (isMissingTableError(error, object)) { - this.logger.debug('Find operation failed', { - object, - reason: 'table-not-provisioned', - error: (error as { message?: unknown } | null | undefined)?.message, - }); - return; - } - // [#17212] `warn`, not `error`: `find`'s `catch` rethrows, so the caller - // is told — see the docblock's last section. - this.logger.warn('Find operation failed', writeFailureLogMeta(error, { object })); + // [#29986 codemod] The body moved to `engine-find.ts` (see `FindHost`). + const readLogger = () => this.logger; + const host: ThisParameterType = { + get logger() { return readLogger(); }, + }; + return reportFindFailure.call(host, object, error); } /** @@ -15120,7 +9302,7 @@ export class ObjectQL implements IObjectQLEngine { * is the row as it arrived: the caller decides whether that fails the * write or culls the row, and the taken keys are still reported. */ - private async staticReadonlyCreateStrip( + private staticReadonlyCreateStrip( object: string, readonlySubject: NonNullable>, row: Record, @@ -15134,31 +9316,12 @@ export class ObjectQL implements IObjectQLEngine { permissionResolution: PermissionResolution | undefined; }, ): Promise<{ row: Record; taken: string[]; error?: unknown }> { - const stripped = stripReadonlyFields( - readonlySubject as any, row, supplied, pass.logger, - { - strictReadonlyWrites: pass.strictReadonlyWrites, - hookWrittenKeys: pass.hookWrittenKeys, - verb: 'insert', - }, - ) as Record; - if (stripped === row) return { row, taken: [] }; - const taken = Object.keys(row).filter((k) => !(k in stripped)); - if (taken.length > 0) { - let redefaultPermissions: EvalPermissions | undefined; - try { - redefaultPermissions = (await this.resolveDefaultPermissions( - object, [stripped], context, pass.permissionResolution, taken, - ))(stripped); - } catch (err) { - return { row, taken, error: err }; - } - const redefaulted = this.applyFieldDefaults(object, stripped, context, pass.nowSnap, redefaultPermissions); - for (const k of taken) { - if (redefaulted[k] !== undefined) stripped[k] = redefaulted[k]; - } - } - return { row: stripped, taken }; + // [#29986 codemod] The body moved to `engine-insert.ts` (see `InsertHost`). + const host: ThisParameterType = { + applyFieldDefaults: (...a) => this.applyFieldDefaults(...a), + resolveDefaultPermissions: (...a) => this.resolveDefaultPermissions(...a), + }; + return staticReadonlyCreateStrip.call(host, object, readonlySubject, row, supplied, context, pass); } /** @@ -19168,18 +13331,16 @@ export class ObjectQL implements IObjectQLEngine { * mistake. */ private warnCascadeNotAtomic(object: string): void { - if (this.cascadeNotAtomicReported.has(object)) return; - this.cascadeNotAtomicReported.add(object); - this.logger.warn( - `Cascade delete of '${object}' cannot run as one unit of work: the cascade reaches an object routed ` + - `to a datasource other than the default one ('${this.defaultDriver ?? ''}'), and a transaction ` + - "covers one driver's connection only (ADR-0119 D1 — no two-phase commit). The cascade therefore runs " + - 'UNWRAPPED, as every cascade did before a single-datasource cascade was made one transaction: if a ' + - 'later dependent refuses the delete, the rows already ' + - 'removed stay removed while the call rejects. Route the cascading objects to one datasource to get the ' + - 'atomic path. Reported once per object per engine instance.', - { object, defaultDatasource: this.defaultDriver ?? undefined }, - ); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + const readCascadeNotAtomicReported = () => this.cascadeNotAtomicReported; + const readDefaultDriver = () => this.defaultDriver; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get cascadeNotAtomicReported() { return readCascadeNotAtomicReported(); }, + get defaultDriver() { return readDefaultDriver(); }, + get logger() { return readLogger(); }, + }; + return warnCascadeNotAtomic.call(host, object); } /** @@ -19681,70 +13842,11 @@ export class ObjectQL implements IObjectQLEngine { fieldName: string, fdef: any, ): string | undefined { - // [#22957] The family test is the shared predicate, so this reader, the - // two walks that read it and `planCascadeAtomicity` admit one family. - if (!statesDeleteSemantics(fdef)) return undefined; - // [#18550] Same arbiter, same absence-vs-unreadability split as - // {@link ObjectQL.planCascadeAtomicity} states above — and this is the - // seam where the silence was measurable end to end: an unreadable - // carrier made the relation invisible to the cascade, so `delete()` - // removed the parent, left a `master_detail` child behind, and - // reported success. No `restrict` refusal, no `set_null`, nothing - // logged. Absence still skips the field here exactly as before. - const ref = deleteWalkTargetOf(fdef, 'ObjectQL.cascadeDeleteRelations'); - if (!ref) return undefined; - // Match the target object by raw or resolved name. - let resolvedRef: string | undefined; - try { resolvedRef = this.resolveObjectName(ref); } catch { resolvedRef = undefined; } - if (ref !== object && resolvedRef !== object) return undefined; - - // [#21910, #21918] A lookup the registry INJECTED into a federated - // object, and the object does not provision, is not a reference to - // anything, so it is not a relation to probe. That is the tenant - // anchor `organization_id`, the ADR-0117 D1 anchor - // `owning_business_unit_id`, and the owner and audit lookups - // `owner_id` / `created_by` / `updated_by`. On a federated object each - // exists in the registered schema and nowhere else: the probe below - // was refused by the driver (`INVALID_FILTER`, no such column), its - // catch propagated the refusal as #8895 rules for a missing column, - // and deleting the organization, business unit or user it names was - // refused on a deployment with a federated object bound. - // `buildDriverOptions` and the related-record read already refuse this - // reading of the tenant column. - // {@link isFederatedUnprovisionedInjectedColumn} reads which columns - // those are from the registry's own provenance, never from a list of - // names, and {@link ObjectQL.planCascadeAtomicity} asks it too. - // - // ⛔ The probe's catch ({@link probeReferencingRows}) is deliberately NOT - // widened to pass a missing column as benign. That would invert #8895's - // discriminate or propagate for every object, not just these injected - // columns: a lookup the author declared on a federated object, including - // an author's own `organization_id` or `owner_id`, stays in the scan, and - // its probe failure still propagates. - if (isFederatedUnprovisionedInjectedColumn(child, fieldName)) return undefined; - - // A master-detail parent owns its children: cascade by default (the - // child FK is typically required, so set_null would be invalid). Only - // an explicit `restrict` deviates. A plain lookup honors its - // configured deleteBehavior (default set_null). - // - // [#22957] A `tree` reads like a `lookup`: the spec materializes the same - // `set_null` default on it, and an authored `cascade` / `restrict` is - // honoured as written. Its `cascade` is a subtree delete, and a cycle in - // the data (a corrupt row pointing at its own descendant) terminates on - // the cascade set, the same membership test any self-reference has. - // - // [#29253] A `user` field reaches here only with an authored value - // ({@link statesDeleteSemantics}), and reads it as a `lookup` reads its - // own: the `|| 'set_null'` fallback below is never what a `user` gets. - // - // [#9625] "Only an explicit `restrict` deviates" is the whole of it: - // every other value a master_detail can declare — including an - // explicit `deleteBehavior: 'set_null'` — resolves to `cascade` here, - // silently. Measured and pinned (`engine-cascade-delete.test.ts`). - return fdef.type === 'master_detail' - ? (fdef.deleteBehavior === 'restrict' ? 'restrict' : 'cascade') - : (fdef.deleteBehavior || 'set_null'); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + const host: ThisParameterType = { + resolveObjectName: (...a) => this.resolveObjectName(...a), + }; + return cascadeRelationBehavior.call(host, object, child, fieldName, fdef); } /** @@ -19903,14 +14005,11 @@ export class ObjectQL implements IObjectQLEngine { fdef: { type?: unknown; multiple?: unknown }, ids: ReadonlyArray, ): Record { - if (ids.length === 1) return this.referenceProbeFilter(fieldName, fdef, ids[0]); - if (!declaredMultiValued(fdef)) return { [fieldName]: { $in: [...ids] } }; - return { - $or: ids.flatMap((id) => { - const limb = this.referenceProbeFilter(fieldName, fdef, id); - return Array.isArray(limb.$or) ? (limb.$or as Array>) : [limb]; - }), + // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + const host: ThisParameterType = { + referenceProbeFilter: (...a) => this.referenceProbeFilter(...a), }; + return referenceProbeFilterOfSet.call(host, fieldName, fdef, ids); } /** @@ -20411,11 +14510,8 @@ export class ObjectQL implements IObjectQLEngine { fieldName: string, behavior: string | undefined, ): boolean { - return ( - object === ORGANIZATION_OBJECT && - behavior === 'set_null' && - resolveTenantFieldName(child) === fieldName - ); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`. + return isTenantColumnSetNull(object, child, fieldName, behavior); } /** @@ -20520,21 +14616,19 @@ export class ObjectQL implements IObjectQLEngine { * caller's own identity counts the same rows: the elevated count read rows * the caller may not see. (A system caller's own count is the elevated one.) */ - private async organizationCountIsDisclosable( + private organizationCountIsDisclosable( childName: string, where: Record, count: number, context: ExecutionContext | undefined, ): Promise { - try { - return (await this.count(childName, { where, context })) === count; - } catch (error) { - // The refusal stands either way; only the number is withheld. - this.logger.warn('[cascade-delete] disclosure count failed; the refusal names the table without a count', { - object: childName, error: (error as Error)?.message ?? String(error), - }); - return false; - } + // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + const readLogger = () => this.logger; + const host: ThisParameterType = { + count: (...a) => this.count(...a), + get logger() { return readLogger(); }, + }; + return organizationCountIsDisclosable.call(host, childName, where, count, context); } /** @@ -21426,495 +15520,25 @@ export class ObjectQL implements IObjectQLEngine { return rejectCredentialAggregation.call(host, object, query); } - async aggregate(object: string, query: EngineAggregateOptions, options?: EngineReadOptions): Promise { - object = this.resolveObjectName(object); - // Fold the `filter` alias into `where` (#4346) — the AST below reads - // `query.where` only, so an unfolded `{ filter }` aggregated every row. - query = foldEngineOptionAliases(object, 'aggregate', query, ENGINE_WHERE_SLOTS); - rejectUnknownEngineOptions(object, 'aggregate', query, ENGINE_AGGREGATE_OPTION_KEYS); - query = lowerWhereFilterArray(object, 'aggregate', query, this._registry.getObject(object), this.relatedSchemaOf); - // [ADR-0100] The filter half of the credential-column guard — `where` - // and every `aggregations[i].filter` — before the search term joins - // `where`; the aggregate half (`rejectCredentialAggregation`) follows. - // [internal: true] The flagged-column refusal goes first, over every - // position at once (`where`, `groupBy`, each aggregation's `field` and - // `filter`), so a non-system caller meets the data door's words here - // too rather than the aggregate guard's. - rejectInternalFieldEvaluation(object, query, mergeReadContext(query?.context, options?.context), this.relatedSchemaOf); - rejectCredentialEvaluation(object, 'aggregate', query, mergeReadContext(query?.context, options?.context), this.relatedSchemaOf); - // ADR-0061 `search` → the rows are searched BEFORE they are grouped, by - // the one expander `find` runs, at the same point in the sequence (after - // the `where` doors above, before the AST is built and tokens resolve). - query = await this.expandSearchOnOptions(object, query, options); - this.rejectCredentialAggregation(object, query); - // [#20783] …and a `groupBy` entry naming a structured-JSON field (`json`, - // `composite`, `address`, …) is refused `INVALID_FIELD` / 400 here, before - // any driver is asked: the drivers share no meaning for a JSON document as - // a group key (memory merged every row into one group, SQLite grouped each - // serialized document apart, PostgreSQL answered 500). After the - // credential refusal, which reads the same entries, so a protected field - // keeps that refusal's words. [#20808] The same door refuses a `groupBy` - // entry naming a MULTI-VALUE field (`isMultiValueField`: a list stored in - // a JSON column, split the same three ways), judged in one walk so the - // first offending position is the one named. - assertGroupByNamesNoJsonStoredField(object, this._registry.getObject(object), query.groupBy); - // [#20808] …and every aggregation's (function, declared field type) - // pair, asked of the spec table (`isAggregateCompatibleWithFieldType`) - // beside `isMultiValueField`, before any driver is asked. [#20808] took - // the `count_distinct` row (memory counted equal documents apart, SQLite - // compared serialized text, PostgreSQL answered 500); [#20914] took the - // other rows — `max` over a `json` field answered a document in memory, a - // string on SQLite and a 500 on PostgreSQL; `sum` over one answered `0`, - // `0` and a 500. Every row of the table is asked (see the door's header). - assertAggregationFieldTypesAccepted(object, this._registry.getObject(object), query.aggregations); - // [ADR-0144 D6] …and a `sum` / `avg` / `min` / `max` over a `record`-mode - // amount that does not group by its code field: the rows hold amounts in - // different currencies, which add into one number under one label. - // After the type door, so a type it refuses keeps that door's words. - assertRecordCurrencyAggregatesGrouped(object, this._registry.getObject(object), query.groupBy, query.aggregations); - // [#25818] …and the two shapes that answered a plausible wrong table: a - // `dateGranularity` over a field declared neither a date nor a datetime - // (every row in one null bucket), and two columns of the aggregated row - // under one name (the last value written won). After the two doors - // above, so a JSON field under a date bucket keeps their words. - assertDateBucketTargetsAreTemporal(object, this._registry.getObject(object), query.groupBy); - assertAggregatedColumnsAreDistinct(object, query.groupBy, query.aggregations); - // [#10576] The per-aggregation `filter` (`AggregationNodeSchema.filter`, - // the contract half of #10413) is a second filter position on this verb, - // so it walks through the same refusal doors `where` does at this seam: - // the comparand-shape gate (#5869 — a scalar `$in` etc. is refused, not - // silently matched against nothing) and the materializable-field gate - // (#8296 — a typo'd column would otherwise select ZERO rows for that one - // aggregation, silently, which is the same wrong-number shape #10413 - // measured). The path names which aggregation carries the offending key. - { - const aggregations = Array.isArray(query.aggregations) ? query.aggregations : []; - let typedAggregations: typeof aggregations | undefined; - for (const [i, agg] of aggregations.entries()) { - const aggFilter = (agg as { filter?: unknown })?.filter; - if (aggFilter == null) continue; - // [#20122] The SHAPE gate `where` takes first on its own seam - // (`lowerWhereFilterArray`, #20121) — first here too, with its - // object test (`isWhereFilterObject`) and its words, minus the - // array half (below). A filter that is not a filter object - // (a string, a number, a boolean, `''`, a `Map`, a `Date`) was - // stepped around by every door below, each of which walks a filter - // node's keys, and then dropped: the fork below reads it as "no - // filter", so the aggregation read every row of its group - // (`driver-sql`'s native aggregate answered a non-empty string - // with a 501 instead). The wire door already refuses these shapes - // through `AggregationNodeSchema`; this closes the in-process one. - // - // An ARRAY is refused too, `[]` included — unlike `where`, which - // lowers the condition-array sugar in its array branch. The slot - // is declared `FilterConditionSchema` (an object, no array form), - // the wire door refuses every array here (`VALIDATION_FAILED`), - // and in-process a condition array counted no row (the walker - // read its index positions as column names) while `[]` read as - // no filter. Same ruling as `having`'s condition check (#20099). - if (Array.isArray(aggFilter)) { - let shown: string; - try { - shown = JSON.stringify(aggFilter) ?? String(aggFilter); - } catch { - shown = String(aggFilter); - } - if (shown.length > 80) shown = `${shown.slice(0, 77)}...`; - throw invalidFilterError( - `aggregate('${object}'): 'aggregations[${i}].filter' must be a filter object, received ` + - `an array (${shown}). The condition-array form — [field, operator, value] tuples and ` + - `["and", …] groups — is input-only sugar lowered on 'where' alone; a per-aggregation ` + - `filter is declared as a filter condition object and does not take it, so an array here ` + - `was never applied as the filter it spells. Write the object form, ` + - `{ "FIELD": { "$gt": 100 } }, or omit 'filter' for no filter.`, - ); - } - if (!isWhereFilterObject(aggFilter)) { - throw invalidFilterError( - `aggregate('${object}'): 'aggregations[${i}].filter' must be a filter object, ` + - `received ${describeNonFilterWhere(aggFilter)}. It was not applied, and an ` + - `unapplied filter would have aggregated every row of each group for that aggregation.`, - ); - } - assertListComparandShapes(object, 'aggregate', aggFilter, `aggregations[${i}].filter`); - assertFilterIsMaterializable(object, 'aggregate', this._registry.getObject(object), aggFilter); - // [#15661] …and the declared-type door for the text operators: a - // `$contains` over a numeric column in ONE aggregation's filter is - // the same silent zero at a second filter position, and a door that - // spoke on `where` alone would answer one mistake two ways within a - // single verb. [#20334] Rooted at this position, as the shape door - // above is: the refusal said `at where.amount.$contains`, a `where` - // the author did not write. - assertTextOperatorTargetsAreStringCapable( - object, 'aggregate', this._registry.getObject(object), aggFilter, `aggregations[${i}].filter`, - ); - // [#20148] …and the TEMPORAL-comparand door, fourth here as it is - // fourth on `where`'s seam (#8690) — the same function, against the - // object's declared fields, since this filter reads the object's - // raw rows. A comparand a declared date / datetime / time field - // cannot interpret (`'not-a-date'`, the preset name - // `'last_30_days'`) was compared as written: no row counted, a - // `$between` with one such endpoint counted EVERY row, and the same - // bound in a `where` was refused 400 — measured on the base on - // driver-memory and driver-sql, through the engine and REST. - // Before the token resolver below, as there, so a `{placeholder}` - // is stepped around and resolved (or refused) a moment later. - // [#20334] Rooted at this position too: the refusal said - // `at where.placed_on.$gt`. - assertTemporalComparandsInterpretable( - object, 'aggregate', this._registry.getObject(object), aggFilter, `aggregations[${i}].filter`, - ); - // [#20351] …and the NUMBER-comparand door, fifth here as it is - // fifth on `where`'s seam: a non-numeric string against a declared - // numeric field counted no row (every row under `$ne`) where its - // `where` twin was a 500 on PostgreSQL, and a numeric string is - // narrowed to its number, copy-on-write, before the in-memory - // evaluator compares it. Rooted at this position. [#20546] Its - // walk's no-operator-object arm too: `{ amount: { a: 1 } }` here - // counted no row, silently, on every driver. [#20745] So did - // `{ owner: { region: 'NA' } }` beneath a lookup; a JSON object - // here is refused alike, one answer per filter at every position. - // [#21333] …and its boolean arm: `"true"` against a declared - // boolean field counted no row (every row under `$ne`) on both - // drivers; it is narrowed to `true` here, `"yes"` refused. - const numeric = narrowNumberComparands( - object, 'aggregate', this._registry.getObject(object), aggFilter, `aggregations[${i}].filter`, - ); - // [#20122] …and the two doors `having` took at its own entry - // (#20099), so a refusal here is the FILTER's, never the data's: - // 1. the comparand-TYPE door `where` takes in - // `lowerWhereFilterArray`, rooted at this position — a plain - // object, a `Map` or a function comparand was compared as it - // stood and counted no row (every row under `$ne`), and a - // `Symbol` under an ordering operator threw an uncoded - // `TypeError`, on a populated table only. An exact-range - // bigint is narrowed copy-on-write, as it is there; - // 2. the walker's own refusals, judged once - // (`assertAggregationFilterIsEvaluable`). The fallback below - // walks the filter per SOURCE row, so `{ amount: { $median: 1 } }` - // was a 400 on a populated table and a `200 []` on an empty one, - // and a `$or` whose first branch held counted every row. - // [#20148] …then, against the object's DECLARED fields, the two - // cross-field rules `where` gets from `driver-sql`'s compiler: - // a `{ $field }` names a declared field, and an `addDays` pair - // follows `FieldReferenceSchema.addDays`' class rule. Both were - // answered by the fallback — no row counted — where `where` is - // refused. Refused in `where`'s words for that comparison, which - // withhold the fields, the operator and the reason; the - // withheld half goes to this log, as the driver writes its own. - // [#21007] …and, last, `where`'s JSON-column rule: a scalar - // comparison (`$in`, `$nin`, `$eq`, an ordering, implicit - // equality, …) on a field declared JSON-stored is refused in - // `where`'s words — counted by the fallback, `$nin` counted the - // very rows it was asked to exclude. Same withholding, same log. - const typed = normalizeFilterComparandTypes(numeric, `aggregate('${object}')`, `aggregations[${i}].filter`); - assertAggregationFilterIsEvaluable(typed, i, { - object, - fields: (this._registry.getObject(object) as { fields?: unknown } | undefined)?.fields, - reportWithheld: (diagnostic) => this.logger.warn( - `aggregate('${object}'): INVALID_FILTER — refusal detail withheld from the response, as it ` - + `is for the same refusal in a where. Full diagnostic: ${diagnostic}`, - ), - }); - if (typed !== aggFilter) { - typedAggregations ??= [...aggregations]; - typedAggregations[i] = { ...(agg as object), filter: typed } as typeof agg; - } - } - if (typedAggregations) query = { ...query, aggregations: typedAggregations }; - } - // [#19974] `having` is this verb's THIRD filter position, and it walks - // through the same comparand-shape face the other two take above — - // called once, here, with the path seeded at `having`. The 2026-09-23 - // ruling on #19757 refuses an array in the equality slot at that face - // "for every driver at once", and the face refuses it for `where` on - // every driver. `having` never reaches a driver, though: the engine - // evaluates it itself (`applyHaving`, having-filter.ts), and that walker - // ANSWERED every shape the face refuses — `{ total: [5] }` by JS `==` - // coercion (`5 == [5]` is true), `{ total: { $eq: [5] } }` by `!=`, a - // scalar `$nin` or a malformed `$between` by keeping every group, a null - // `$lt` bound by keeping none. Measured on the base through this method, - // on driver-memory and driver-sqlite-wasm, on both doors below. - // - // ONE call covers BOTH `applyHaving` doors — the native - // `driver.aggregate()` path and the in-memory fallback — because it runs - // before either is chosen and before any driver is asked for a row. The - // shape of a comparand is a property of the FILTER, so it is judged once - // per query, never per aggregated row: an empty grouped set refuses the - // same `having` a populated one does. ⛔ Not a second face and not a - // walker-local check in having-filter.ts: the verdicts, the wording and - // the `INVALID_FILTER` / 400 envelope are the face's own, so a `having` - // refusal reads byte for byte as the `where` refusal of the same shape, - // path aside — and whatever arm the face gains next, `having` gains too. - // - // [#20099] …and the rest of the doors `where` takes, at the same seam, so - // the whole clause is judged once, before any row exists. In order: - // 1. `having` is a filter condition at all — `QuerySchema.having` and - // `EngineAggregateOptions.having` declare `FilterConditionSchema`, - // which refuses an array or a scalar. The walker read an array's index - // keys as columns (`[['total','>',100]]` kept no group) and a scalar - // as no condition (every group). ⛔ Not lowered: the `FilterArray` - // sugar is declared on `where` alone, and every later door steps - // silently around a non-node, so this runs first. - // 2. the comparand-shape face (above); - // 3. the comparand-TYPE door — the same `normalizeFilterComparandTypes` - // `lowerWhereFilterArray` runs on `where`, path rooted at `having`, so - // `{ total: { $eq: { v: 1 } } }` is the `where` refusal of that shape - // rather than a silent `[]`, and an exact-range bigint is narrowed - // copy-on-write exactly as it is there; - // 4. the walker's own refusals plus the `{ $field }` reference's - // position and name, judged against the aggregated row's column set - // read off THIS query (`assertHavingIsEvaluable`). The walker raised - // them per aggregated row, so an empty grouped set answered `200 []` - // for a `having` a populated one refused. [#20123] The same column - // set judges every KEY too: a key naming no column read "no value" - // in every group, so a typo for an alias kept no group (every group - // under a negation) with no error. - // A reference that passes is then RESOLVED against each aggregated row - // (having-filter.ts `compareWithReference`) on both doors below. - assertHavingIsFilterCondition(query.having); - assertListComparandShapes(object, 'aggregate', query.having, 'having'); - // [#20176] The object's declaration, read once: the per-aggregation - // `filter` reads a temporal comparand by its column's storage rule - // (`applyInMemoryAggregation` below), and `having` by its aggregated - // column's — the rule the drivers apply to the same comparand in a - // `where`. Before, both positions compared it as written, so an ISO - // instant against a `date` field counted 1 row where the `where` twin - // counted 3, over REST, on driver-memory and driver-sql alike. - const declaredFields = (this._registry.getObject(object) as { fields?: Record } | undefined)?.fields; - // [#20127] …and each aggregated column's class, read off the query and the - // object's declaration, so a `{ $field, addDays }` pair is judged by the - // rule `FieldReferenceSchema.addDays` declares (two temporal columns of - // one class) rather than answered by epoch-ms coercion. [#20176] Kept for - // both `applyHaving` doors below, which read a temporal column's - // comparands by its storage rule. - const havingColumnClasses = aggregatedRowColumnClasses(query.groupBy, query.aggregations, declaredFields); - { - const having = normalizeFilterComparandTypes(query.having, `aggregate('${object}')`, 'having'); - assertHavingIsEvaluable( - having, - aggregatedRowColumns(query.groupBy, query.aggregations), - havingColumnClasses, - // [#21255] …and each column's TYPE, which a plain `{ $field }` - // reference's class rule asks the spec's verdict of — the rule - // `where` applies to every reference, not only an `addDays` pair. - aggregatedRowColumnTypes(query.groupBy, query.aggregations, declaredFields), - ); - // [#20263] …and last, the TEMPORAL-comparand door `where` (#8690) and - // the per-aggregation `filter` (#20148) take: the same walk and the - // same `@objectstack/core` predicate, with each column's kind read - // from the class #20127 derived above (`min` / `max` of a temporal - // field keeps its kind, a `day` bucket is a date, `count` / `sum` / - // `avg` are not temporal). A comparand the column's storage rule - // cannot read (`'not-a-date'`, a `date` year outside 0..9999) was - // compared as written and kept no group or every group, on both - // `applyHaving` doors, while its `where` twin answered 400. After - // every other `having` door, so a clause one of them refuses keeps - // that refusal's words; on the caller's own clause, before the - // bigint narrowing, which is what `where`'s object form judges. - assertHavingTemporalComparandsInterpretable(object, query.having, havingColumnClasses, query); - // [#20351] …then the NUMBER-comparand door `where` and the - // per-aggregation `filter` take, over the columns #20127 classes - // `numeric`: a non-numeric string kept no group (every group under - // `$ne`) on both `applyHaving` doors, and a numeric string is narrowed - // to its number. On the bigint-narrowed clause, so the two - // narrowings compose. - // [#20546] The same walk's no-operator-object arm judges every column - // whose TYPE holds scalar values (hence the types, beside the - // classes): `{ total: { a: 1 } }` kept no group, silently, on every - // driver, where its `where` twin answered two ways. [#20745] A - // relation or JSON column's type is judged too (a lookup groupBy's - // nested-relation `having` kept no group on every driver). - // [#21333] …and, by the same types, the boolean arm: over a groupBy - // of a boolean field `"true"` kept no group (`$ne "true"` every - // group) on both drivers; it is narrowed to `true`, `"yes"` refused. - const numeric = narrowHavingNumberComparands( - object, having, havingColumnClasses, - aggregatedRowColumnTypes(query.groupBy, query.aggregations, declaredFields), - ); - if (numeric !== query.having) query = { ...query, having: numeric }; - } - const driver = this.getDriver(object); - this.logger.debug(`Aggregate on ${object} using ${driver.name}`, query); - - const opCtx: OperationContext = { - object, - operation: 'aggregate', - // On the opCtx so middleware-injected read filters (RLS/sharing) land - // in `ast.where` — same #2737 hole as count(): a locally-built AST - // aggregated over rows the caller may not see. - ast: { - object, - where: query.where, - groupBy: query.groupBy, - aggregations: query.aggregations, - // ENFORCED since #4286 (step 3). On the ast so the FLS predicate - // guard walks its references (predicate-guard.ts) and a future - // native pushdown has it in hand; the engine's post-aggregation - // applyHaving() below is authoritative either way. - having: query.having, - }, - options: query, - context: mergeReadContext(query?.context, options?.context), - }; - // [ADR-0053 D-D1, amended — #5930] Each of this verb's three filter - // positions resolves, then lowers, through `resolveThenLowerWhere`: `where` - // and `aggregations[i].filter` narrow the object's raw rows, so they read - // its declared field types; `having` narrows the aggregated row, so it - // reads each aggregated column's type (`min` / `max` of a `datetime` - // field is a `datetime`; a `count` is a number). - const rowLowering = declaredDatetimeLowering(this._registry.getObject(object)); - await this.resolveWhereTokens(opCtx.ast as QueryAST, opCtx.context, rowLowering, 'where', 'aggregate'); - // [#10576] Filter tokens (`{userId}`-style placeholders, #3810) resolve - // in per-aggregation filters exactly as they do in `where` — a filter - // position is a filter position, and an unresolved placeholder would be - // compared as a literal string, silently selecting zero rows for that one - // aggregation. Copy-on-write like `withResolvedWhere`: the entry objects - // belong to the caller (view metadata / flow config get reused), so a - // resolved filter lands on a copied entry, never written back through. - { - const astAggs = (opCtx.ast as QueryAST).aggregations; - if (Array.isArray(astAggs) && astAggs.some((a) => (a as { filter?: unknown })?.filter != null)) { - (opCtx.ast as QueryAST).aggregations = astAggs.map((a, i) => { - const f = (a as { filter?: unknown })?.filter; - if (f == null) return a; - const resolved = resolveThenLowerWhere( - f as any, opCtx.context, rowLowering, - whereResolvedTokenJudge(object, 'aggregate', this._registry.getObject(object), `aggregations[${i}].filter`), - ); - return resolved === f ? a : { ...(a as object), filter: resolved } as typeof a; - }); - } - } - // [#20334] …and in `having`, through the resolver `where` takes, so a - // relative-date token compares as the day it names rather than as its own - // text, and an unknown one is `FILTER_TOKEN_UNKNOWN` / 400 (it kept no - // group with a 200). Both `applyHaving` doors below read `ast.having`, so - // one call covers the native and the rows path, before any driver read. - // After every `having` door above, as `where`'s resolution follows its - // doors: the temporal door steps around a `{placeholder}` exactly as - // `where`'s does (so, as there, the resolved value is judged only for - // its year, by the resolution stage — #20844), and the other doors - // judged a string that resolves to a string. - { - const havingColumnTypes = aggregatedRowColumnTypes(query.groupBy, query.aggregations, declaredFields); - await this.resolveWhereTokens( - opCtx.ast as QueryAST, - opCtx.context, - { isDatetimeColumn: (column) => havingColumnTypes.get(column) === 'datetime' }, - 'having', - ); - } - - await this.executeWithMiddleware(opCtx, async () => { - // [ADR-0139 D2] The executor boundary, once for BOTH driver paths below - // (they are exclusive, and the rows path keeps this `where` by - // reference): the AST is final once the chain has run. - const ast = await this.composeReadableIdsAtBoundary(object, 'aggregate', opCtx.ast as QueryAST, opCtx.context); - - // Prefer driver.aggregate() when available — driver.find() in many - // drivers (e.g. driver-sql) does not honor `groupBy` / `aggregations` - // and would silently return ungrouped raw rows. Fall back to find() - // for drivers that handle aggregations through their query AST. - const drv = driver as any; - // Structured groupBy items ({field, dateGranularity}) require the - // driver to advertise per-granularity native bucket support via - // `supports.queryDateGranularity[g]`. If every structured item is - // supported we can push the aggregate down to the driver; otherwise - // we fall back to driver.find() + in-memory bucketing so the result - // remains correct on a face that buckets fewer (e.g. the Turso remote - // transport, which advertises none). - const groupByItems = Array.isArray(query.groupBy) ? query.groupBy : []; - const granularityCaps: Record | undefined = - drv?.supports?.queryDateGranularity; - const structuredItems = groupByItems.filter((g) => typeof g !== 'string'); - const allStructuredSupported = structuredItems.every((g) => { - if (!g?.dateGranularity) return true; // plain {field} object is fine - return granularityCaps?.[g.dateGranularity] === true; - }); - // ADR-0053 Phase 2 (D2): native driver date bucketing (driver-sql's - // `to_char` / `date_format` / `strftime`) is UTC-only — SQLite has no - // tz database and MySQL needs tz tables loaded, - // so pushing tz-aware bucketing down splits boundaries per dialect. When - // a non-UTC reference timezone is in play we therefore force the - // in-memory path: the date-range `where` still goes to the driver (only - // matching rows are fetched), but bucketing runs uniformly in JS so a - // row near a tz day-boundary lands identically on every driver. - const tz = query.timezone; - const hasDateBucket = structuredItems.some((g) => !!g?.dateGranularity); - const tzRequiresInMemory = !!tz && tz !== 'UTC' && hasDateBucket; - // [#10576] Per-aggregation filters (`aggregations[].filter`, the - // contract half of #10413) force the in-memory path: no driver compiles - // a conditional aggregate (SQL `FILTER (WHERE …)` / `CASE WHEN`) from - // this key today, and pushing it down would aggregate the UNFILTERED - // rows — the silent drop #10413 measured. Same correct-first two-tier - // shape as date bucketing and HAVING above: a driver that grows native - // support must advertise a capability flag, at which point this fork - // becomes its fallback tier. The driver faces also refuse the key - // loudly (NOT_IMPLEMENTED/501) if reached directly, so neither seam can - // drop it in silence. `{}` counts as no filter (the vacuous-filter - // convention `where` / `having` already follow). - const hasAggregationFilter = (Array.isArray(ast.aggregations) ? ast.aggregations : []) - .some((a) => { - const f = (a as { filter?: unknown })?.filter; - return f != null && typeof f === 'object' && Object.keys(f).length > 0; - }); - if (typeof drv.aggregate === 'function' && allStructuredSupported && !tzRequiresInMemory && !hasAggregationFilter) { - // HAVING is engine-owned (#4286): applied AFTER aggregation, over - // the aggregated row's own columns (aggregation aliases + groupBy - // projections), identically on both paths. No driver implements it - // natively today; one that grows native support must advertise a - // capability flag, at which point this post-filter becomes the - // fallback tier — the dateGranularity two-tier pattern. - const aggregated = await drv.aggregate(object, ast, this.buildDriverOptions(object, opCtx.context, undefined, { recordedVerdict: opCtx.tenantLayer0Verdict })); - return applyHaving(aggregated, ast.having, havingColumnClasses); - } - // In-memory fallback path: ask the driver for raw rows, then bucket + - // aggregate here. This guarantees `groupBy` (incl. structured items - // carrying `dateGranularity`) and `aggregations` always work even on - // drivers that have no native aggregation support (driver-rest, - // driver-memory, partial SQL drivers), and is the path that honours a - // non-UTC reference timezone. - // - // [#16642] The `find` call asks for ROWS, so the aggregation keys this - // path is about to evaluate ITSELF are stripped from the AST it sends - // down. `find()`'s contract says nothing about `groupBy` / - // `aggregations`, and the drivers disagree about them: `driver-sql` - // and `driver-rest` ignore both and return rows (which is the only - // reason this path has ever worked), while `driver-memory` honours - // them — its `find()` funnels straight into the same - // `performAggregation` its `aggregate(AST)` door uses, and - // `driver-mongodb` / `driver-turso` carry the same refusal on their - // own aggregation faces. Sending the keys to a driver of that kind - // made this ONE seam answer two different wrong things: - // - // * a per-aggregation `filter` (the key that ROUTED the call here) - // was refused NOT_IMPLEMENTED/501 by the driver's own #10413 - // guard — a refusal aimed at direct callers, raised against the - // engine's own lowering, so `service-analytics` answered 501 for - // a measure `filter` on the memory driver while sqlite answered - // the number (#16642); - // * a date-bucketed `groupBy` came back ALREADY grouped, on the raw - // timestamp, and `applyInMemoryAggregation` then aggregated those - // GROUP rows a second time — a count of buckets reported under - // the author's own measure name, i.e. a plausible wrong number - // rather than a refusal. - // - // Both driver refusals document themselves as "unreachable through - // `engine.aggregate`, which lowers in memory for every driver" - // (`driver-memory`'s `refusePerAggregationFilter`, `driver-sql`'s - // `unsupportedAggregationFilterError`); this is the line that makes - // that true. `having` goes with them: `applyHaving` below is the - // authority on it and no driver evaluates it in `find`. - const rowsAst: QueryAST = { ...ast }; - delete rowsAst.groupBy; - delete rowsAst.aggregations; - delete rowsAst.having; - const raw = await driver.find(object, rowsAst, this.buildDriverOptions(object, opCtx.context, undefined, { recordedVerdict: opCtx.tenantLayer0Verdict })); - return applyHaving(applyInMemoryAggregation(raw, ast, tz, declaredFields), ast.having, havingColumnClasses); - }); - - return opCtx.result as any[]; + aggregate(object: string, query: EngineAggregateOptions, options?: EngineReadOptions): Promise { + // [#29986 codemod] The body moved to `engine-delete-and-aggregate.ts` (see `DeleteAndAggregateHost`). + const read_registry = () => this._registry; + const readLogger = () => this.logger; + const readRelatedSchemaOf = () => this.relatedSchemaOf; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + buildDriverOptions: (...a) => this.buildDriverOptions(...a), + composeReadableIdsAtBoundary: (...a) => this.composeReadableIdsAtBoundary(...a), + executeWithMiddleware: (...a) => this.executeWithMiddleware(...a), + expandSearchOnOptions: (...a) => this.expandSearchOnOptions(...a), + getDriver: (...a) => this.getDriver(...a), + get logger() { return readLogger(); }, + rejectCredentialAggregation: (...a) => this.rejectCredentialAggregation(...a), + get relatedSchemaOf() { return readRelatedSchemaOf(); }, + resolveObjectName: (...a) => this.resolveObjectName(...a), + resolveWhereTokens: (...a) => this.resolveWhereTokens(...a), + }; + return aggregate.call(host, object, query, options); } /** From 1e62a085f02eae40ada01babb0dc1fb9f09ab563 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 02:40:39 +0000 Subject: [PATCH 03/13] refactor(scripts): leave the engine.ts declaration bar to the report reading objectql's bundled declarations live in a hashed shared chunk that index.d.ts re-exports, so the pilot's index-only --dts check cannot read the bar here. Ruling 6441032442 item 3 keeps the second normalisation out of scripts/, so the PR reports it on two built dists instead, as the protocol.ts codemod does. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- scripts/codemod-split-engine.mjs | 165 ++----------------------------- 1 file changed, 8 insertions(+), 157 deletions(-) diff --git a/scripts/codemod-split-engine.mjs b/scripts/codemod-split-engine.mjs index a602bc589f8..eff0bcce629 100644 --- a/scripts/codemod-split-engine.mjs +++ b/scripts/codemod-split-engine.mjs @@ -11,8 +11,6 @@ * node scripts/codemod-split-engine.mjs --write # apply the split, then verify it * node scripts/codemod-split-engine.mjs --verify --base * # re-prove a split on disk against engine.ts at - * node scripts/codemod-split-engine.mjs --dts - * # the declaration-file bar on two built dists * node scripts/codemod-split-engine.mjs --self-test * node scripts/codemod-split-engine.mjs --help # this usage, on any tree (split or not) * @@ -100,11 +98,13 @@ * (the prune above), so nothing else in it moved. The leaf's relative * VALUE imports are followed on disk: none may reach `engine.ts` or an * `engine-*` module. - * 2. The declaration-file bar of ruling `6348891160` item 1, on the built - * `dist/index.d.ts` and `dist/index.d.mts` (`--dts`), as the pilot runs it. - * A `.d.ts` order-only difference is judged by ruling `6441032442` B's - * second normalisation (ruling `6597555122` item 2), a report reading on - * the two dists, never a `scripts/` file. + * 2. The declaration-file bar of ruling `6348891160`, judged by ruling + * `6441032442` B's second normalisation (ruling `6597555122` item 2), is a + * report reading on two built dists, never a `scripts/` file (that + * ruling's item 3); the PR reports it. The pilot's `--dts` reads only + * `index.d.ts`, which here re-exports a hashed shared chunk + * (`util-.d.ts`) that holds the declarations, so this script does + * not carry it. * * ## Determinism * @@ -1526,125 +1526,6 @@ function lineCount(text) { return text.split('\n').length - 1; } -// ── Proof 2: the declaration-file bar (ruling 6348891160 item 1) ───────────── - -/** - * The ruling's one normalisation, verbatim: sort the members of each run of - * string-literal union members in place. Nothing else is normalised. - */ -const NORMALISER_ONE_LINER = - 'node -e \'const s=require("fs").readFileSync(process.argv[1],"utf8");process.stdout.write(s.replace(/"[^"\\n]*"(?:\\s*\\|\\s*"[^"\\n]*")+/g,r=>r.split(/\\s*\\|\\s*/).sort().join(" | ")))\' FILE | sha256sum'; - -function normaliseUnionOrder(text) { - return text.replace(/"[^"\n]*"(?:\s*\|\s*"[^"\n]*")+/g, (r) => r.split(/\s*\|\s*/).sort().join(' | ')); -} - -/** Export names of a bundled `.d.ts`: every `export { … }` list, plus `export Name`. */ -function dtsExportNames(text) { - const names = new Set(); - for (const m of text.matchAll(/^export\s*(?:type\s*)?\{([^}]*)\}/gm)) { - for (const part of m[1].split(',')) { - const t = part.trim().replace(/^type\s+/, ''); - if (!t) continue; - const as = /\bas\s+([\w$]+)$/.exec(t); - names.add(as ? as[1] : t); - } - } - for (const m of text.matchAll(/^export\s+(?:declare\s+)?(?:abstract\s+)?(?:class|interface|type|function|const|let|var|enum|namespace)\s+([\w$]+)/gm)) names.add(m[1]); - return [...names].sort(); -} - -/** One positive control: the first `private` member made public. */ -function injectVisibilityChange(text) { - const at = text.search(/^ {4}private /m); - return at < 0 ? null : text.slice(0, at) + ' ' + text.slice(at + ' private '.length); -} - -/** - * The number of lines deleted plus lines inserted between two line arrays — - * what `diff` prints as `<` and `>` lines (Myers' O((N+M)D) edit distance, so - * a handful of changed lines in a 9,000-line file costs nothing). - */ -function lineEditDistance(a, b) { - const n = a.length; - const m = b.length; - const max = n + m; - const off = max + 1; - const v = new Int32Array(2 * max + 3); - for (let d = 0; d <= max; d += 1) { - for (let k = -d; k <= d; k += 2) { - let x = (k === -d || (k !== d && v[off + k - 1] < v[off + k + 1])) ? v[off + k + 1] : v[off + k - 1] + 1; - let y = x - k; - while (x < n && y < m && a[x] === b[y]) { x += 1; y += 1; } - v[off + k] = x; - if (x >= n && y >= m) return d; - } - } - return max; -} - -/** - * The bar on one declaration file pair. Returns `{ failures, reading }`. - * - * `rawDiffLines` counts the lines that differ IN PLACE when both files have the - * same line count (one per swapped union, as the ruling reads it), and the - * lines `diff` would print (deleted plus inserted) when the counts differ; - * `rawDiffMethod` names which. Before, a differing line count read as 0. - */ -function dtsBarFor(file, before, after) { - const failures = []; - const nb = normaliseUnionOrder(before); - const na = normaliseUnionOrder(after); - const bl = before.split('\n'); - const al = after.split('\n'); - let rawDiffLines = 0; - const rawDiffMethod = bl.length === al.length ? 'in-place' : 'diff'; - if (bl.length !== al.length) { - rawDiffLines = lineEditDistance(bl, al); - failures.push(`${file}: ${bl.length} lines at base, ${al.length} at head — a line added or removed is a declaration change, outside the ruling`); - } else { - for (let i = 0; i < bl.length; i += 1) { - if (bl[i] === al[i]) continue; - rawDiffLines += 1; - if (normaliseUnionOrder(bl[i]) !== normaliseUnionOrder(al[i])) { - failures.push(`${file}:${i + 1} differs other than by union member order:\n base: ${bl[i].trim()}\n head: ${al[i].trim()}`); - } - } - } - if (nb !== na) failures.push(`${file}: normalised base and head differ`); - const eb = dtsExportNames(before); - const ea = dtsExportNames(after); - if (JSON.stringify(eb) !== JSON.stringify(ea)) failures.push(`${file}: the export list differs (${eb.length} at base, ${ea.length} at head)`); - const control = injectVisibilityChange(after); - const controlSha = control === null ? null : sha256(normaliseUnionOrder(control)); - if (control === null) failures.push(`${file}: no private member to make public — the positive control could not run (INCONCLUSIVE)`); - else if (controlSha === sha256(nb)) failures.push(`${file}: the positive control normalises to the base hash — the normaliser hides a visibility change (INCONCLUSIVE)`); - return { - failures, - reading: { - file, - rawBase: sha256(before), rawHead: sha256(after), rawDiffLines, rawDiffMethod, - normalisedBase: sha256(nb), normalisedHead: sha256(na), - exportsBase: eb.length, exportsHead: ea.length, - positiveControl: controlSha, - }, - }; -} - -function dtsBar(beforeDir, afterDir) { - const failures = []; - const readings = []; - for (const file of ['index.d.ts', 'index.d.mts']) { - const b = join(beforeDir, file); - const a = join(afterDir, file); - if (!existsSync(b) || !existsSync(a)) refuse(`--dts: ${file} missing under ${existsSync(b) ? afterDir : beforeDir}`); - const r = dtsBarFor(file, readFileSync(b, 'utf8'), readFileSync(a, 'utf8')); - failures.push(...r.failures); - readings.push(r.reading); - } - return { failures, readings }; -} - // ── Reports ────────────────────────────────────────────────────────────────── function planReport(planned) { @@ -1743,7 +1624,6 @@ function generatedModulesOnDisk() { const USAGE = [ 'usage: node scripts/codemod-split-engine.mjs [--write] plan (or apply) the split of an UNSPLIT engine.ts', ' node scripts/codemod-split-engine.mjs --verify --base REF re-prove a split on disk against engine.ts at REF', - ' node scripts/codemod-split-engine.mjs --dts BEFORE_DIST AFTER_DIST the declaration-file bar on two built dists', ' node scripts/codemod-split-engine.mjs --self-test', ' node scripts/codemod-split-engine.mjs --help', ].join('\n'); @@ -1766,17 +1646,6 @@ async function main(argv) { } return 0; } - if (argv.includes('--dts')) { - const i = argv.indexOf('--dts'); - const [before, after] = [argv[i + 1], argv[i + 2]]; - if (!before || !after) refuse('--dts needs '); - const { failures, readings } = dtsBar(before, after); - console.log(`normaliser (ruling 6348891160 item 2, verbatim): ${NORMALISER_ONE_LINER}`); - for (const r of readings) console.log(JSON.stringify(r)); - for (const f of failures) console.error(` ✗ ${f}`); - console.log(failures.length === 0 ? '✓ declaration-file bar: PASS' : `✗ declaration-file bar: FAIL (${failures.length})`); - return failures.length === 0 ? 0 : 1; - } const tm = await loadTsMorph(); if (argv.includes('--verify')) { const i = argv.indexOf('--base'); @@ -1843,14 +1712,13 @@ function valueSpecifiers(src) { let selfTestReachedVerdict = false; const SELF_TEST_BATTERIES = Object.freeze({ - 'declaration bar': 7, 'stay rules': 12, 'top-level moves': 11, 'byte-identity proof': 10, 'import prune': 4, 'refusals': 6, }); -const SELF_TEST_BATTERY_FLOOR = 6; +const SELF_TEST_BATTERY_FLOOR = 5; const FIXTURE_ANCHORS = Object.freeze([ ['count', 'Lifecycle', 'engine-lifecycle.ts'], @@ -1998,23 +1866,6 @@ async function selfTest() { }; const refused = (fn) => { try { fn(); return false; } catch (e) { return e instanceof CodemodRefusal; } }; - open('declaration bar'); - { - const base = 'export declare class A {\n private x;\n t: "b" | "a";\n}\nexport { A };\n'; - const swapped = base.replace('"b" | "a"', '"a" | "b"'); - check('a union-member swap passes, and is counted as one raw diff line', - (() => { const r = dtsBarFor('f', base, swapped); return r.failures.length === 0 && r.reading.rawDiffLines === 1; })()); - check('the normaliser sorts union members and nothing else', normaliseUnionOrder('x: "b" | "a"; y: B | A') === 'x: "a" | "b"; y: B | A'); - check('a retyped member fails', dtsBarFor('f', base, base.replace('"b" | "a"', '"b" | "c"')).failures.length > 0); - check('a private member made public fails (the positive control the bar runs)', - dtsBarFor('f', base, base.replace(' private x;', ' x;')).failures.length > 0); - check('an added line fails', dtsBarFor('f', base, base.replace('}\n', ' y: number;\n}\n')).failures.length > 0); - check('a line-count change is counted as diff counts it, never as 0', - (() => { const r = dtsBarFor('f', base, base.replace('}\n', ' y: number;\n}\n')).reading; return r.rawDiffLines === 1 && r.rawDiffMethod === 'diff'; })() - && lineEditDistance(['a', 'b', 'c'], ['a', 'x', 'c', 'd']) === 3); - check('a changed export list fails', dtsBarFor('f', base, base.replace('export { A };', 'export { A as B };')).failures.length > 0); - } - open('stay rules'); { const planned = fixturePlan(tm, fixtureSource()); From 26050e69cc444e268a9ef666f47cbd5dbdb6e5bb Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 03:56:43 +0000 Subject: [PATCH 04/13] fix(scripts): keep the engine.ts query-options erasure sites on the class The #4918 erasure guard grandfathers its pre-existing sites by file (eslint QUERY_OPTIONS_UNSWEPT plus a per-file count in scripts/query-options-erasure-baseline.json, which may only shrink). seedAutonumber and inspectDanglingReferences each carry one site, so moving their bodies redded `pnpm lint` and check:query-options-erasure. They become PATH_READER_PINS and stay. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- scripts/codemod-split-engine.mjs | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/scripts/codemod-split-engine.mjs b/scripts/codemod-split-engine.mjs index eff0bcce629..4f1c70f5055 100644 --- a/scripts/codemod-split-engine.mjs +++ b/scripts/codemod-split-engine.mjs @@ -191,6 +191,14 @@ const LEAF_MODULE = 'engine-top-level.ts'; * into names nobody reads. */ const PATH_READER_PINS = Object.freeze({ + // The #4918 query-options erasure guard grandfathers its pre-existing sites + // BY FILE: an `ignores` entry in eslint.config.mjs (QUERY_OPTIONS_UNSWEPT) + // plus a per-file count in scripts/query-options-erasure-baseline.json, a list + // that may only shrink. A moved body takes its `opts as any` site to a file + // the list does not name, so `pnpm lint` and `check:query-options-erasure` + // red (measured: both methods below, one site each). + seedAutonumber: 'eslint query-options/no-any-erasure grandfathering and scripts/check-query-options-erasure-ratchet.mjs (baseline keyed by path, never grows)', + inspectDanglingReferences: 'eslint query-options/no-any-erasure grandfathering and scripts/check-query-options-erasure-ratchet.mjs (baseline keyed by path, never grows)', }); /** From a8a3e88a08c2feb9bc8811a80810deb44454696c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 03:56:43 +0000 Subject: [PATCH 05/13] refactor(objectql): regenerate the engine.ts split with the pinned methods kept `node scripts/codemod-split-engine.mjs --write` on base 0b99c2b1, unedited. 37 of 278 methods and 146 of 146 top-level declarations move; engine.ts 23245 -> 16933 lines. `--verify --base 0b99c2b1` exits 0. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- .../src/engine-defaults-and-autonumbers.ts | 81 +----------- .../engine-references-and-master-detail.ts | 30 +---- packages/objectql/src/engine.ts | 118 ++++++++++++++---- 3 files changed, 97 insertions(+), 132 deletions(-) diff --git a/packages/objectql/src/engine-defaults-and-autonumbers.ts b/packages/objectql/src/engine-defaults-and-autonumbers.ts index f85bb7a9459..d1c92416c3f 100644 --- a/packages/objectql/src/engine-defaults-and-autonumbers.ts +++ b/packages/objectql/src/engine-defaults-and-autonumbers.ts @@ -5,8 +5,8 @@ * the bodies of `resolveNowDefault`, `resolveOptionDefault`, * `applyFieldDefaults`, `defaultFieldEntries`, `applyAutonumbers`, * `adoptExplicitAutonumber`, `isIssuedAutonumberCollision`, - * `warnAutonumberReadersUnserved`, `seedAutonumber` moved out of `engine.ts` - * whole by `scripts/codemod-split-engine.mjs`. + * `warnAutonumberReadersUnserved` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. * * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not @@ -26,15 +26,11 @@ */ import { isCurrentUserDefaultToken, isNowDefaultToken, missingFieldValues, parseAutonumberFormat, renderAutonumber, resolveAutonumberFormat } from '@objectstack/spec/data'; -import { isUniqueViolationError, keysetWalk, uniqueViolationColumn } from '@objectstack/types'; -import type { KeysetPageQuery } from '@objectstack/types'; +import { isUniqueViolationError, uniqueViolationColumn } from '@objectstack/types'; import type { ExecutionContext } from '@objectstack/spec/kernel'; -import { isMissingTableError } from '@objectstack/metadata/errors'; import { ExpressionEngine } from '@objectstack/formula'; import type { EvalPermissions } from '@objectstack/formula'; -import { - AUTONUMBER_SEED_PAGE_SIZE, declaredMultiValued, isExpressionDefault, readStoredAutonumberCounter, -} from './engine-top-level.js'; +import { declaredMultiValued, isExpressionDefault, readStoredAutonumberCounter } from './engine-top-level.js'; import type { FormulaPlanEntry, IssuedAutonumber } from './engine-top-level.js'; import type { ObjectQL } from './engine.js'; @@ -49,7 +45,6 @@ export interface DefaultsAndAutonumbersHost { readonly autonumberCounters: ObjectQL['autonumberCounters']; readonly autonumberReadersUnservedReported: ObjectQL['autonumberReadersUnservedReported']; defaultFieldEntries: ObjectQL['defaultFieldEntries']; - find: ObjectQL['find']; getSchema: ObjectQL['getSchema']; readonly logger: ObjectQL['logger']; normalizeExpressionDefault: ObjectQL['normalizeExpressionDefault']; @@ -305,71 +300,3 @@ export function warnAutonumberReadersUnserved(this: Pick, object: string, field: string, prefix: string, suffix: string, execCtx?: ExecutionContext): Promise { - try { - // Canonical `fields`, not the wire spelling `select` — this call sat on - // the exact #4371 silent drop (the projection never applied; the scan - // worked only because an unprojected row still carries `field`), and the - // catch below would have swallowed the guard's rejection into "seed - // from 0", i.e. duplicate autonumbers. - const walk = keysetWalk>( - (q: KeysetPageQuery) => this.find(object, { - ...q, - fields: ['id', field], - context: execCtx, - } as any), - { - where: prefix ? { [field]: { $startsWith: prefix } } : undefined, - pageSize: AUTONUMBER_SEED_PAGE_SIZE, - }, - ); - let max = 0; - for await (const page of walk.pages()) { - for (const r of page) { - const v = r?.[field]; - if (v == null) continue; - // The reading itself lives in `readStoredAutonumberCounter` (the - // section above describes it) because #6806's adopt-on-exempt-write - // resync must read a supplied value by the SAME rules this scan reads - // a stored one — two copies of it would drift into two different - // answers for one row, which is a duplicate record number. Its - // anchored half is spec's `readAutonumberCounter`, the one the SQL - // driver's own scan calls (#6560). - const counter = readStoredAutonumberCounter(String(v), prefix, suffix); - if (counter != null) max = Math.max(max, counter); - } - } - // The walk is unbounded (no `max`), so truncation here means the scan - // could not COMPLETE: a row carried no `id` to seek past, or the reader - // never applied the seek predicate. Either way rows were left unread, and - // the max over what was read is a floor, not the max. Answering with it - // is the "seed below the real MAX" defect this method was fixed for, so - // it fails loudly instead — the same disposition #6114 gave the read - // outage: allocate nothing, write nothing. - if (walk.truncated) { - throw new Error( - `Cannot seed the autonumber counter for "${object}.${field}": the seeding scan ` + - `could not visit every stored row (it stopped after ${walk.scanned} rows without ` + - `reaching the end). Seeding from a partial scan would issue record numbers that ` + - `collide with existing ones, so no number was allocated.`, - ); - } - return max; - } catch (error) { - // [#5979] Discriminate by error TYPE. Seeding from 0 is the truth for - // exactly ONE failure reason — the table has not been provisioned, so - // there are genuinely no rows and number 1 collides with nothing. - if (isMissingTableError(error, object)) return 0; - // Every other failure (connection drop, timeout, permission denial, - // query error) means the rows may well exist and simply were not seen. - // Answering 0 there restarts the sequence at 1 against a table already - // holding N rows and issues autonumbers that COLLIDE with existing ones - // — a value written wrong, which no retry and no restart repairs. So the - // read failure propagates and the caller allocates nothing: the write - // fails loudly instead of succeeding with a forged business identifier. - // This is the hazard the #4371 comment above the read already named. - throw error; - } - } diff --git a/packages/objectql/src/engine-references-and-master-detail.ts b/packages/objectql/src/engine-references-and-master-detail.ts index 4a0aba2253c..7f4c2c59193 100644 --- a/packages/objectql/src/engine-references-and-master-detail.ts +++ b/packages/objectql/src/engine-references-and-master-detail.ts @@ -3,9 +3,8 @@ /** * [#29986 codemod] The * "Realtime, i18n, home amounts, validation messages, references, master-detail" - * section of `ObjectQL`: the bodies of `inspectDanglingReferences`, - * `inspectCascadeDelete` moved out of `engine.ts` whole by - * `scripts/codemod-split-engine.mjs`. + * section of `ObjectQL`: the bodies of `inspectCascadeDelete` moved out of + * `engine.ts` whole by `scripts/codemod-split-engine.mjs`. * * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not @@ -24,10 +23,6 @@ * evaluation cycle. `index.ts` re-exports nothing from this module. */ -import type { ExecutionContext } from '@objectstack/spec/kernel'; -import { postureUsesUnionScope } from '@objectstack/spec/security'; -import { auditDanglingReferences } from './integrity/dangling-reference-audit.js'; -import type { AuditableObject, DanglingReferenceAuditOptions, DanglingReferenceReport } from './integrity/dangling-reference-audit.js'; import type { ObjectQL } from './engine.js'; /** @@ -39,30 +34,9 @@ import type { ObjectQL } from './engine.js'; export interface ReferencesAndMasterDetailHost { readonly _registry: ObjectQL['_registry']; collectCascadeDeleteSet: ObjectQL['collectCascadeDeleteSet']; - find: ObjectQL['find']; - readonly logger: ObjectQL['logger']; - referenceExists: ObjectQL['referenceExists']; - resolveEnginePosture: ObjectQL['resolveEnginePosture']; resolveObjectName: ObjectQL['resolveObjectName']; } -/** The body of `ObjectQL#inspectDanglingReferences`; the method's docblock there is its contract. */ -export async function inspectDanglingReferences(this: Pick, options?: DanglingReferenceAuditOptions): Promise { - // Read once per run, live — the posture IN FORCE, by the engine's one reader. - const unionScope = postureUsesUnionScope(this.resolveEnginePosture()); - return auditDanglingReferences( - { - objects: () => this._registry.getAllObjects() as unknown as AuditableObject[], - find: (object, opts) => this.find(object, opts as any) as Promise>>, - probe: (target, id, organization) => this.referenceExists( - target, id, organization == null || unionScope ? undefined : ({ tenantId: organization } as ExecutionContext), - ), - warn: (msg, meta) => this.logger?.warn?.(msg, meta as any), - }, - options, - ); - } - /** The body of `ObjectQL#inspectCascadeDelete`; the method's docblock there is its contract. */ export async function inspectCascadeDelete(this: Pick, object: string, id: string | number): Promise> { const resolved = this.resolveObjectName(object); diff --git a/packages/objectql/src/engine.ts b/packages/objectql/src/engine.ts index 6a8503def95..9c17d517652 100644 --- a/packages/objectql/src/engine.ts +++ b/packages/objectql/src/engine.ts @@ -50,6 +50,7 @@ import { // Seek pagination for the walks that must read EVERY row — the autonumber seed // scan is one (#6249). Shared with `summary-backfill` rather than re-rolled: // the cursor merge is the part that is easy to get subtly wrong. +import { keysetWalk, type KeysetPageQuery } from '@objectstack/types'; import { DATA_MIGRATION_FLAG_OBJECT, FILE_REFERENCES_MIGRATION_ID, @@ -383,13 +384,14 @@ import { bindHookNotice } from './hook-notice.js'; import { lowerRelationConditions, resolveSearchQueryableFields, resolveWhereTokens } from './engine-query-lowering.js'; import { auditDanglingReferences, + type AuditableObject, type DanglingReferenceAuditOptions, type DanglingReferenceReport, } from './integrity/dangling-reference-audit.js'; // [#29986 codemod] The top-level declarations that moved whole to `engine-top-level.ts`, as this file still reads them. import { - AUTONUMBER_COLLISION_ATTEMPTS, DISPATCHABLE_HOOK_EVENTS, ENGINE_DELETE_OPTION_KEYS, ENGINE_FIND_OPTION_KEYS, - ENGINE_QUERY_SLOTS, ENGINE_UPDATE_OPTION_KEYS, ENGINE_WHERE_SLOTS, ENGINE_WIRE_ONLY_SLOTS, + AUTONUMBER_COLLISION_ATTEMPTS, AUTONUMBER_SEED_PAGE_SIZE, DISPATCHABLE_HOOK_EVENTS, ENGINE_DELETE_OPTION_KEYS, + ENGINE_FIND_OPTION_KEYS, ENGINE_QUERY_SLOTS, ENGINE_UPDATE_OPTION_KEYS, ENGINE_WHERE_SLOTS, ENGINE_WIRE_ONLY_SLOTS, SECURITY_SERVICE_SOURCE_SLOT, WIRE_ONLY_CANONICAL_SHAPES, WRITE_PRECONDITION_OPTION, applyFormulaPlan, applyStoredExpressionsToPatch, assertDispatchableHookEvent, assertHookScopeNotSelfCancelling, assertOrderByIsMaterializable, assertProjectionHasNoDottedPaths, assertValidHookExcludeObjects, @@ -399,11 +401,12 @@ import { handBackWithheldInsertReadonly, hookMatchesObject, hydrateWriteFormulas, insertStaticReadonlySubject, judgeWhereAdmission, lowerWhereFilterArray, masterIdOf, mergeReadContext, organizationPredicateDeleteRefusal, packageBodyCollectionRefusal, packageBodyObjectAndAppOwnerStampRefusal, packageBodySeedDataRefusal, - planFormulaProjection, planStoredExpressions, recomputeAutonumberReaders, rejectUnknownEngineOptions, - replacedStoredExpressionFields, resolveWhereFilterTokens, rowsWithoutFormulaWidening, settleBulkStoredExpressions, - shouldSkipStateMachine, statesDeleteSemantics, staticReadonlyStoredView, storedExpressionsReading, - stripComputedWriteFields, undeclaredWriteFieldErrors, updateStaticReadonlySubject, whereResolvedTokenJudge, - withFileOwnerKey, withholdInsertReadonlyFromHooks, withoutFormulaWidening, writeFailureLogMeta, + planFormulaProjection, planStoredExpressions, readStoredAutonumberCounter, recomputeAutonumberReaders, + rejectUnknownEngineOptions, replacedStoredExpressionFields, resolveWhereFilterTokens, rowsWithoutFormulaWidening, + settleBulkStoredExpressions, shouldSkipStateMachine, statesDeleteSemantics, staticReadonlyStoredView, + storedExpressionsReading, stripComputedWriteFields, undeclaredWriteFieldErrors, updateStaticReadonlySubject, + whereResolvedTokenJudge, withFileOwnerKey, withholdInsertReadonlyFromHooks, withoutFormulaWidening, + writeFailureLogMeta, } from './engine-top-level.js'; import type { AdmittedValueShapeViolationTally, AutonumberReaders, CascadeDeleteSet, CascadeIssuedDelete, DatasourceDef, @@ -444,7 +447,7 @@ import { executeWithMiddleware, resolveSystemInsertOrganization } from './engine // [#29986 codemod] Field defaults and autonumbers: the methods of the same names below hand the bodies what they read (`DefaultsAndAutonumbersHost`). import { adoptExplicitAutonumber, applyAutonumbers, applyFieldDefaults, defaultFieldEntries, isIssuedAutonumberCollision, - resolveNowDefault, resolveOptionDefault, seedAutonumber, warnAutonumberReadersUnserved, + resolveNowDefault, resolveOptionDefault, warnAutonumberReadersUnserved, } from './engine-defaults-and-autonumbers.js'; // [#29986 codemod] Package, app, plugin, driver and datasource registration: the methods of the same names below hand the bodies what they read (`RegistrationHost`). import { @@ -452,7 +455,7 @@ import { registerPackageBody, supplyFileColumnsMovedResolver, } from './engine-registration.js'; // [#29986 codemod] Realtime, i18n, home amounts, validation messages, references, master-detail: the methods of the same names below hand the bodies what they read (`ReferencesAndMasterDetailHost`). -import { inspectCascadeDelete, inspectDanglingReferences } from './engine-references-and-master-detail.js'; +import { inspectCascadeDelete } from './engine-references-and-master-detail.js'; // [#29986 codemod] Crypto, secrets, internal fields, schema and datasource routing, init: the methods of the same names below hand the bodies what they read (`SecretsAndRoutingHost`). import { resolveDatasourceBinding } from './engine-secrets-and-routing.js'; // [#29986 codemod] Summary indexes, related-record expansion, file and reference hydration: the methods of the same names below hand the bodies what they read (`SummariesAndExpansionHost`). @@ -3855,18 +3858,77 @@ export class ObjectQL implements IObjectQLEngine { * matching is LOOSER (a case-insensitive `LIKE`) must not be able to inflate * the max with another scope's rows. */ - private seedAutonumber( + private async seedAutonumber( object: string, field: string, prefix: string, suffix: string, execCtx?: ExecutionContext, ): Promise { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). - const host: ThisParameterType = { - find: (...a) => this.find(...a), - }; - return seedAutonumber.call(host, object, field, prefix, suffix, execCtx); + try { + // Canonical `fields`, not the wire spelling `select` — this call sat on + // the exact #4371 silent drop (the projection never applied; the scan + // worked only because an unprojected row still carries `field`), and the + // catch below would have swallowed the guard's rejection into "seed + // from 0", i.e. duplicate autonumbers. + const walk = keysetWalk>( + (q: KeysetPageQuery) => this.find(object, { + ...q, + fields: ['id', field], + context: execCtx, + } as any), + { + where: prefix ? { [field]: { $startsWith: prefix } } : undefined, + pageSize: AUTONUMBER_SEED_PAGE_SIZE, + }, + ); + let max = 0; + for await (const page of walk.pages()) { + for (const r of page) { + const v = r?.[field]; + if (v == null) continue; + // The reading itself lives in `readStoredAutonumberCounter` (the + // section above describes it) because #6806's adopt-on-exempt-write + // resync must read a supplied value by the SAME rules this scan reads + // a stored one — two copies of it would drift into two different + // answers for one row, which is a duplicate record number. Its + // anchored half is spec's `readAutonumberCounter`, the one the SQL + // driver's own scan calls (#6560). + const counter = readStoredAutonumberCounter(String(v), prefix, suffix); + if (counter != null) max = Math.max(max, counter); + } + } + // The walk is unbounded (no `max`), so truncation here means the scan + // could not COMPLETE: a row carried no `id` to seek past, or the reader + // never applied the seek predicate. Either way rows were left unread, and + // the max over what was read is a floor, not the max. Answering with it + // is the "seed below the real MAX" defect this method was fixed for, so + // it fails loudly instead — the same disposition #6114 gave the read + // outage: allocate nothing, write nothing. + if (walk.truncated) { + throw new Error( + `Cannot seed the autonumber counter for "${object}.${field}": the seeding scan ` + + `could not visit every stored row (it stopped after ${walk.scanned} rows without ` + + `reaching the end). Seeding from a partial scan would issue record numbers that ` + + `collide with existing ones, so no number was allocated.`, + ); + } + return max; + } catch (error) { + // [#5979] Discriminate by error TYPE. Seeding from 0 is the truth for + // exactly ONE failure reason — the table has not been provisioned, so + // there are genuinely no rows and number 1 collides with nothing. + if (isMissingTableError(error, object)) return 0; + // Every other failure (connection drop, timeout, permission denial, + // query error) means the rows may well exist and simply were not seen. + // Answering 0 there restarts the sequence at 1 against a table already + // holding N rows and issues autonumbers that COLLIDE with existing ones + // — a value written wrong, which no retry and no restart repairs. So the + // read failure propagates and the caller allocates nothing: the write + // fails loudly instead of succeeding with a forged business identifier. + // This is the hazard the #4371 comment above the read already named. + throw error; + } } /** @@ -5693,20 +5755,22 @@ export class ObjectQL implements IObjectQLEngine { * budget never reached (#5718), `unreadableObjects` for what the datasource * refused, and `aborted` for a run called off (#4747). */ - inspectDanglingReferences( + async inspectDanglingReferences( options?: DanglingReferenceAuditOptions, ): Promise { - // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). - const read_registry = () => this._registry; - const readLogger = () => this.logger; - const host: ThisParameterType = { - get _registry() { return read_registry(); }, - find: (...a) => this.find(...a), - get logger() { return readLogger(); }, - referenceExists: (...a) => this.referenceExists(...a), - resolveEnginePosture: (...a) => this.resolveEnginePosture(...a), - }; - return inspectDanglingReferences.call(host, options); + // Read once per run, live — the posture IN FORCE, by the engine's one reader. + const unionScope = postureUsesUnionScope(this.resolveEnginePosture()); + return auditDanglingReferences( + { + objects: () => this._registry.getAllObjects() as unknown as AuditableObject[], + find: (object, opts) => this.find(object, opts as any) as Promise>>, + probe: (target, id, organization) => this.referenceExists( + target, id, organization == null || unionScope ? undefined : ({ tenantId: organization } as ExecutionContext), + ), + warn: (msg, meta) => this.logger?.warn?.(msg, meta as any), + }, + options, + ); } /** From 5c85508e94e056f0841a7364f5f701bbf6ceb9e1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 03:56:44 +0000 Subject: [PATCH 06/13] test(objectql): widen the by-path readers of engine.ts to the split modules Ruling 6597555122 A: a test or gate that reads engine.ts by path gains the new modules in its reading list. - engine-middleware-operation-vocabulary.test.ts reads engine.ts and engine-top-level.ts, where OperationContext now lives. - federated-injected-column-readers.test.ts re-keys the 13 seam rows whose sites moved, and resolves a `via` helper across engine.ts and its split modules. - check-stack-collection-maps reads METADATA_ARRAY_KEYS from the leaf. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- ...ne-middleware-operation-vocabulary.test.ts | 12 +++- .../federated-injected-column-readers.test.ts | 62 ++++++++++++++----- scripts/check-stack-collection-maps.mjs | 6 +- 3 files changed, 60 insertions(+), 20 deletions(-) diff --git a/packages/objectql/src/engine-middleware-operation-vocabulary.test.ts b/packages/objectql/src/engine-middleware-operation-vocabulary.test.ts index de44f8853d9..f6fe29fc445 100644 --- a/packages/objectql/src/engine-middleware-operation-vocabulary.test.ts +++ b/packages/objectql/src/engine-middleware-operation-vocabulary.test.ts @@ -148,13 +148,19 @@ function readDispatchedUnionFromEngineSource(): string[] { if (!testPath) { throw new Error('vitest did not report a testPath — the #7809 weld cannot locate engine.ts.'); } - const enginePath = join(dirname(testPath), 'engine.ts'); - const src = readFileSync(enginePath, 'utf8'); + // [#29986] The codemod split moved engine.ts's top-level declarations, this + // interface among them, into `engine-top-level.ts`; engine.ts re-exports it + // by name. Both files are read so the weld follows the declaration wherever + // the split put it (ruling 6597555122 A: a by-path reader gains the new + // modules in its reading list). + const src = ['engine.ts', 'engine-top-level.ts'] + .map((file) => readFileSync(join(dirname(testPath), file), 'utf8')) + .join('\n'); const iface = src.match(/export interface OperationContext\s*\{([\s\S]*?)\n\}/); if (!iface) { throw new Error( - 'Could not locate `export interface OperationContext` in engine.ts. ' + + 'Could not locate `export interface OperationContext` in engine.ts or engine-top-level.ts. ' + 'The #7809 vocabulary weld cannot verify itself — fix this parse rather than deleting it.', ); } diff --git a/packages/objectql/src/federated-injected-column-readers.test.ts b/packages/objectql/src/federated-injected-column-readers.test.ts index 919db16b916..a7597d364be 100644 --- a/packages/objectql/src/federated-injected-column-readers.test.ts +++ b/packages/objectql/src/federated-injected-column-readers.test.ts @@ -104,10 +104,33 @@ const ASKS: Partial> = { interface Row { disposition: Disposition; why: string; - /** For an ASKS disposition: the same-file function the site calls, which asks. */ + /** For an ASKS disposition: the same-file function the site calls, which asks (across {@link ENGINE_SPLIT_FILES} for the engine). */ via?: string; } +/** + * engine.ts and the modules `scripts/codemod-split-engine.mjs` (#29986) split + * out of it. A method there may keep its signature in engine.ts and its body in + * one of these, so a {@link Row.via} helper is looked up across the set. + */ +const ENGINE_SPLIT_FILES: ReadonlySet = new Set([ + 'engine.ts', + 'engine-actions.ts', + 'engine-cascade-planning.ts', + 'engine-defaults-and-autonumbers.ts', + 'engine-delete-and-aggregate.ts', + 'engine-find.ts', + 'engine-hooks-and-functions.ts', + 'engine-insert.ts', + 'engine-middleware-and-context.ts', + 'engine-references-and-master-detail.ts', + 'engine-registration.ts', + 'engine-secrets-and-routing.ts', + 'engine-security-resolvers.ts', + 'engine-summaries-and-expansion.ts', + 'engine-top-level.ts', +]); + /** The calls that are seams by name. */ const SEAM_CALLS = new Set([ 'isFederatedObject', @@ -128,7 +151,7 @@ const VALIDATION_FORMULA = "a validation rule's or a formula field's own relatio const READERS: Record = { // ── The readers this family fixed, and the one decision they ask ───────── - 'engine.ts#cascadeRelationBehavior :: isFederatedUnprovisionedInjectedColumn()': { + 'engine-cascade-planning.ts#cascadeRelationBehavior :: isFederatedUnprovisionedInjectedColumn()': { disposition: 'skips', why: 'the dependents probe never filters a federated object on a column it does not provision (both phases of the cascade read relations here)', }, @@ -139,11 +162,11 @@ const READERS: Record = { // [#29253] Both readers above find a relation's target through this one // helper; each asks the skip itself, after the reference match (the two // `isFederatedUnprovisionedInjectedColumn()` rows above). - 'engine.ts#deleteWalkTargetOf :: referenceCarrierOf()': { + 'engine-top-level.ts#deleteWalkTargetOf :: referenceCarrierOf()': { disposition: 'not-a-read', why: 'answers the object a delete-walk relation names, from the field declaration; its callers cascadeRelationBehavior and planCascadeAtomicity skip after the reference match; it reads no row', }, - 'engine.ts#deleteWalkTargetOf :: referenceTargetOf()': { + 'engine-top-level.ts#deleteWalkTargetOf :: referenceTargetOf()': { disposition: 'not-a-read', why: "answers sys_user, the target a user field's type fixes, for the same two callers; it reads no row", }, @@ -162,11 +185,11 @@ const READERS: Record = { disposition: 'not-a-read', why: 'asks whether the object has a tenant column at all, on a relation the walk already read; it reads no row', }, - 'engine.ts#isTenantColumnSetNull :: resolveTenantFieldName()': { + 'engine-cascade-planning.ts#isTenantColumnSetNull :: resolveTenantFieldName()': { disposition: 'not-a-read', why: "compares a field name with the object's tenant column, on a relation its callers already read through cascadeRelationBehavior; it reads no row", }, - 'engine.ts# :: organization_id': { + 'engine-top-level.ts# :: organization_id': { disposition: 'not-a-read', why: "names the field a write-wall verdict (WriteWallRowVerdict, #25062) refuses, in the verdict's type; the plugin that owns the wall decides it, and the engine reads no row", }, @@ -213,7 +236,7 @@ const READERS: Record = { }, // [#24410] The formula read names each reference's target to judge, per // formula, the related fields that target does not declare. - 'engine.ts#resolveFormulaRelated :: referenceTargetOf()': { + 'engine-security-resolvers.ts#resolveFormulaRelated :: referenceTargetOf()': { disposition: 'caller-predicate', why: "judges a formula's own hop against the target's declared field map; no row is read here", }, @@ -223,11 +246,11 @@ const READERS: Record = { disposition: 'caller-predicate', why: "judges a validation rule's own hop against the target's declared field map; no row is read here", }, - 'engine.ts#resolveSystemInsertOrganization :: isFederatedObject()': { + 'engine-middleware-and-context.ts#resolveSystemInsertOrganization :: isFederatedObject()': { disposition: 'exempt', why: 'a system insert into a federated object is never stamped with an organization', }, - 'engine.ts#resolveSystemInsertOrganization :: resolveTenantFieldName()': { + 'engine-middleware-and-context.ts#resolveSystemInsertOrganization :: resolveTenantFieldName()': { disposition: 'exempt', why: 'resolves the stamp column only after the federated exemption', }, @@ -311,7 +334,7 @@ const READERS: Record = { disposition: 'caller-predicate', why: VALIDATION_RULE, }, - 'engine.ts#buildSummaryIndex :: referenceCarrierOf()': { + 'engine-summaries-and-expansion.ts#buildSummaryIndex :: referenceCarrierOf()': { disposition: 'author-declared', why: "infers the foreign key of a roll-up an author declared, from the child's first relation to the " + @@ -420,24 +443,24 @@ const READERS: Record = { // [#24822] Whether a write's static-`readonly` passes spare the tenant column // where the Layer 0 wall's seam declares it judges the write's organization, // and the insert's and the update's subjects built from that one answer. - 'engine.ts#staticReadonlySparesOrganization :: resolveInjectedColumnProvenance()': { + 'engine-top-level.ts#staticReadonlySparesOrganization :: resolveInjectedColumnProvenance()': { disposition: 'not-a-read', why: "asks whether the write's organization_id is the platform's injected column or the author's; " + 'it reads a registered definition, never storage', }, - 'engine.ts#staticReadonlySparesOrganization :: organization_id': { + 'engine-top-level.ts#staticReadonlySparesOrganization :: organization_id': { disposition: 'not-a-read', why: "asks whether the insert strip's subject holds the column; it reads a schema view, never storage", }, - 'engine.ts#insertStaticReadonlySubject :: organization_id': { + 'engine-top-level.ts#insertStaticReadonlySubject :: organization_id': { disposition: 'not-a-read', why: "leaves the column out of an insert's static-readonly subject where the wall's seam judges it; " + 'it reads a schema view and the caller row in hand, never storage', }, - 'engine.ts#updateStaticReadonlySubject :: organization_id': { + 'engine-top-level.ts#updateStaticReadonlySubject :: organization_id': { disposition: 'not-a-read', why: "leaves the column out of an update's static-readonly schema view where the wall's seam judges it; " + @@ -649,7 +672,14 @@ describe('[#21918] every engine reader of an injected column has a disposition t const own = scan.calls.get(site) ?? new Set(); if (row.via) { const file = site.slice(0, site.indexOf('#')); - const helper = scan.calls.get(`${file}#${row.via}`) ?? new Set(); + // [#29986] engine.ts and the modules its codemod split wrote are one + // class: a host method's body may live in a sibling `engine-*.ts` + // module while engine.ts keeps a stub that calls it, so a via named + // from one of them resolves across all of them (ruling 6597555122 A). + const helperFiles = ENGINE_SPLIT_FILES.has(file) ? [...ENGINE_SPLIT_FILES] : [file]; + const helper = new Set( + helperFiles.flatMap((f) => [...(scan.calls.get(`${f}#${row.via}`) ?? [])]), + ); expect(own.has(row.via), `${key}: the site does not call ${row.via}`).toBe(true); expect(asks.some((p) => helper.has(p)), `${key}: ${row.via} asks none of ${asks.join(', ')}`).toBe(true); } else { @@ -665,7 +695,7 @@ describe('[#21918] every engine reader of an injected column has a disposition t .map(([key]) => siteKeyOf(key)), ); expect([...skipping].sort()).toEqual([ - 'engine.ts#cascadeRelationBehavior', + 'engine-cascade-planning.ts#cascadeRelationBehavior', 'engine.ts#organizationOwnedRowCounts', 'engine.ts#planCascadeAtomicity', 'lifecycle/lifecycle-service.ts#tenantWindowsFor', diff --git a/scripts/check-stack-collection-maps.mjs b/scripts/check-stack-collection-maps.mjs index 837dcd4b838..5fa4e7ad9c4 100644 --- a/scripts/check-stack-collection-maps.mjs +++ b/scripts/check-stack-collection-maps.mjs @@ -593,8 +593,12 @@ const SITES = [ // that remains. (Hand-adding the four names to the second copy was the // available alternative and is what #5870 did for `capabilities`; it is // what left the other four undiffed.) + // + // #29986: the codemod split of `engine.ts` (`scripts/codemod-split-engine.mjs`) + // moved its top-level declarations, this list among them, into the leaf + // `engine-top-level.ts`; both seams still read the one copy there. id: 'METADATA_ARRAY_KEYS', - file: 'packages/objectql/src/engine.ts', + file: 'packages/objectql/src/engine-top-level.ts', what: 'collections the engine registers from a manifest AND from a nested plugin (the ADR-0010 provenance seam)', extract: (src) => { const b = sliceBody(src, 'const METADATA_ARRAY_KEYS = ['); From 90b69953c0dfb7dc1cd316e16d7fc15807eb3ff9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 04:07:30 +0000 Subject: [PATCH 07/13] test(core): classify the two engine split modules that call referenceTargetOf The second-object-read exposure pin keys CALLERS by file. The engine.ts split moved resolveFormulaRelated into engine-security-resolvers.ts and deleteWalkTargetOf into engine-top-level.ts; neither serves a field of the target, so both are `not-a-served-read` rows. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- .../second-object-read-exposure.pin.test.ts | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/packages/core/src/security/second-object-read-exposure.pin.test.ts b/packages/core/src/security/second-object-read-exposure.pin.test.ts index f8d4911eb1b..fcf51dcacef 100644 --- a/packages/core/src/security/second-object-read-exposure.pin.test.ts +++ b/packages/core/src/security/second-object-read-exposure.pin.test.ts @@ -228,6 +228,19 @@ const CALLERS: Record = { '`beforeRelationTarget` hook (the data door hands it `refuseUnservedRelationTarget`). Pinned by ' + '`packages/objectql/src/internal-field-evaluation-guard.test.ts` (the refusals assert the driver was never asked)', }, + // [#29986] The codemod split of engine.ts moved two arbiter callers out of it. + 'packages/objectql/src/engine-security-resolvers.ts': { + kind: 'not-a-served-read', + reason: + "a formula field's relation hop (`resolveFormulaRelated`): it judges the formula's own path against the " + + "target's DECLARED field map and loads through `resolveOneHopRelated` (its row), serving nothing of the target here", + }, + 'packages/objectql/src/engine-top-level.ts': { + kind: 'not-a-served-read', + reason: + 'the delete walk names the object a relation points at (`deleteWalkTargetOf`: a `user` field answers `sys_user`) ' + + 'to plan a cascade or a restrict; it reads no record and serves nothing of the target', + }, // [#25522 cut 10] The write-path reference check, moved out of engine.ts. 'packages/objectql/src/engine-referential-integrity.ts': { kind: 'not-a-served-read', From 7e0e5874e0e2776b2798788f9815168799423fbc Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 05:25:20 +0000 Subject: [PATCH 08/13] docs: re-point two engine.ts symbol anchors to the split's leaf module The codemod split moved FormulaPlanEntry (whose `expression` member ADR-0072's evaluator table anchors) and planFormulaProjection into engine-top-level.ts, so check:adr-symbol-anchors and check:spec-docblock-symbol-anchors found no declaration in engine.ts. Both anchors now name engine-top-level.ts, where the declarations are. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- docs/adr/0072-reference-scope-and-resolvability.md | 2 +- packages/spec/src/kernel/functional-completeness.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/0072-reference-scope-and-resolvability.md b/docs/adr/0072-reference-scope-and-resolvability.md index 1296b8af856..dfbb72268d2 100644 --- a/docs/adr/0072-reference-scope-and-resolvability.md +++ b/docs/adr/0072-reference-scope-and-resolvability.md @@ -51,7 +51,7 @@ for (const [k, v] of Object.entries(context.record)) | Surface | Evaluator (file:line) | In-scope references | Token shape | | :-- | :-- | :-- | :-- | | Flow / edge / decision | `packages/services/service-automation/src/engine.ts#variables` | `record`,`previous`, **flattened record fields**, flow `variables`, node outputs, `$runId`/`$flowName` | bare **and** `record.x` | -| Formula field (`Field.expression`) | `packages/objectql/src/engine.ts#expression` | `{ now, timezone, user, org, record }` | `record.x` only | +| Formula field (`Field.expression`) | `packages/objectql/src/engine-top-level.ts#expression` | `{ now, timezone, user, org, record }` | `record.x` only | | Validation (`script`/`cross_field`/`when`) | `packages/objectql/src/validation/rule-validator.ts#cross_field` | `{ record: {...previous,...patch}, previous }` | `record.x` only | | Field `visibleWhen`/`requiredWhen`/`readonlyWhen` | `packages/objectql/src/validation/rule-validator.ts#readonlyWhen` | merged `record`, `previous`, (`parent` for master-detail — server-bound for `readonlyWhen` since #4889; the client grid binds it for all three) | `record.x` only | | Hook lifecycle `condition` | `packages/objectql/src/hook-wrappers.ts#condition` | `{ record }` | `record.x` only | diff --git a/packages/spec/src/kernel/functional-completeness.ts b/packages/spec/src/kernel/functional-completeness.ts index 8fe58366ad7..48b181ae501 100644 --- a/packages/spec/src/kernel/functional-completeness.ts +++ b/packages/spec/src/kernel/functional-completeness.ts @@ -32,7 +32,7 @@ * `packages/objectql/src/engine.ts#buildSummaryIndex`, verbatim * `if (d?.type !== 'summary' || !d.summaryOperations) continue;` * - `formula` w/o `expression` → - * `packages/objectql/src/engine.ts#planFormulaProjection`, verbatim + * `packages/objectql/src/engine-top-level.ts#planFormulaProjection`, verbatim * `def?.type === 'formula' && def.expression`: it builds the formula plan * only from fields WITH an expression, so a bare formula never computes. * - `lookup`/`master_detail` w/o `reference` → From 8f9ad27cc68fa3367af19d908cd6fc61c624d6f3 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 05:32:31 +0000 Subject: [PATCH 09/13] chore(spec): re-point two liveness citations to the engine split modules check:liveness's key-mention anchoring found field/summaryOperations and datasource/external.rowCap citing engine.ts, which no longer names either key: buildSummaryIndex's body moved to engine-summaries-and-expansion.ts and applyExternalDriverFloor's to engine-registration.ts. Both citations name those files now. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- packages/spec/liveness/datasource.json | 2 +- packages/spec/liveness/field.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/spec/liveness/datasource.json b/packages/spec/liveness/datasource.json index 910f6f898ed..c5d00e992e0 100644 --- a/packages/spec/liveness/datasource.json +++ b/packages/spec/liveness/datasource.json @@ -159,7 +159,7 @@ "rowCap": { "status": "live", "verifiedAt": "2026-11-03", - "evidence": "packages/objectql/src/external-driver-floor.ts#createExternalDriverFloor (`capInForce()` on every read: `const written = host.rowCap?.(); const cap = resolveExternalRowCap(written);`, then `limit: cap + 1` sent and more than `cap` rows back refused with ExternalRowCapExceededError; a written value not honoured as written is reported once per value through `host.logger.warn`); packages/objectql/src/engine.ts#applyExternalDriverFloor (`rowCap: () => this.datasourceDefs.get(name)?.external?.rowCap`, read from the def held at the time of the read)", + "evidence": "packages/objectql/src/external-driver-floor.ts#createExternalDriverFloor (`capInForce()` on every read: `const written = host.rowCap?.(); const cap = resolveExternalRowCap(written);`, then `limit: cap + 1` sent and more than `cap` rows back refused with ExternalRowCapExceededError; a written value not honoured as written is reported once per value through `host.logger.warn`); packages/objectql/src/engine-registration.ts#applyExternalDriverFloor (`rowCap: () => this.datasourceDefs.get(name)?.external?.rowCap`, read from the def held at the time of the read)", "producer": "packages/services/service-datasource/src/datasource-connection-service.ts#attemptConnect hands the stored record's whole `external` block to `registerDatasourceDef`, so an authored `rowCap` reaches the engine's def on every connect and reconnect", "note": "#29762 (maintainer ruling: a per-datasource cap bounded by a platform maximum of 10,000). The plugin-driver row cap: the most rows the engine's correctness floor reads from a plugin driver for one request, and so the most one read may match before EXTERNAL_ROW_CAP_EXCEEDED (422). Declared in packages/spec/src/data/datasource.zod.ts beside `EXTERNAL_ROW_CAP_DEFAULT` and `EXTERNAL_ROW_CAP_CEILING`; refused at parse above the ceiling, on a built-in driver, and (through the managed-block refusal) on a managed datasource. Planned in the spec slice, live since the card's second PR landed the floor's reader. The engine re-reads it on every read because a redefinition that keeps the floor does not re-wrap the driver; a def that reached the engine unparsed is held at the ceiling above it and read as the default when it is not a whole number from 1." } diff --git a/packages/spec/liveness/field.json b/packages/spec/liveness/field.json index 7caccadfe96..9ccc7f69a2f 100644 --- a/packages/spec/liveness/field.json +++ b/packages/spec/liveness/field.json @@ -120,7 +120,7 @@ }, "summaryOperations": { "status": "live", - "evidence": "packages/objectql/src/engine.ts", + "evidence": "packages/objectql/src/engine-summaries-and-expansion.ts", "note": "rollup {object,field,function,relationshipField,filter}. `filter` (a query `where` FilterCondition) is read by buildSummaryIndex and ANDed with the parent-FK match in recomputeSummaries, so one child object can feed several filtered totals." }, "requiredWhen": { From c848b0a95b430fae5eeaa0b83a15a8809d3594f6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 07:06:27 +0000 Subject: [PATCH 10/13] feat(scripts): codemod-split-engine writes ruling 6624345766's facade shape Ruling 6624345766 (on #29465) set the shape for engine.ts: docblock ownership, private-static-through-host and stateless facet classes over the host, with state staying on ObjectQL. The codemod now writes it: - Each section module holds one stateless class, `
Facet`, whose static methods are the moved bodies, unedited. Class members may take any name, so a reserved word (`delete`) or a name an import takes no longer keeps a body on the class. - A private method's docblock moves with its body; a public or protected one stays on the class, which keeps the public surface as it was. - A body that reads a private static goes through `ObjectQLStatics`, a forwarder object engine.ts declares (typed by indexed access on the class) and a static block fills. The facet module imports it AS `ObjectQL`, so the body text does not change. - A field a body assigns goes through a host setter. - Generic methods call their body instantiated with their own type parameters, defaulted identifier parameters forward their value, and a static method whose body reads no `this` moves too. - The no-yield rule is gone: the 13,000-line bar is replaced by the structural gate, which counts bodies, not lines. - Four more methods carrying a #4918 `as any` site are pinned on the class, as seedAutonumber and inspectDanglingReferences were. verify now also restores the moved docblocks, checks each placeholder, and rebuilds the forwarder declarations and static block from the base class. The self-test has 62 cases in 6 batteries. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- scripts/codemod-split-engine.mjs | 504 ++++++++++++++++++++++++------- 1 file changed, 403 insertions(+), 101 deletions(-) diff --git a/scripts/codemod-split-engine.mjs b/scripts/codemod-split-engine.mjs index 4f1c70f5055..8b7748109be 100644 --- a/scripts/codemod-split-engine.mjs +++ b/scripts/codemod-split-engine.mjs @@ -190,15 +190,21 @@ const LEAF_MODULE = 'engine-top-level.ts'; * nothing the file declares stops the script (exit 2), so the table cannot rot * into names nobody reads. */ +const QUERY_OPTIONS_GUARD = 'eslint query-options/no-any-erasure grandfathering and scripts/check-query-options-erasure-ratchet.mjs (baseline keyed by path, never grows)'; const PATH_READER_PINS = Object.freeze({ // The #4918 query-options erasure guard grandfathers its pre-existing sites // BY FILE: an `ignores` entry in eslint.config.mjs (QUERY_OPTIONS_UNSWEPT) // plus a per-file count in scripts/query-options-erasure-baseline.json, a list // that may only shrink. A moved body takes its `opts as any` site to a file // the list does not name, so `pnpm lint` and `check:query-options-erasure` - // red (measured: both methods below, one site each). - seedAutonumber: 'eslint query-options/no-any-erasure grandfathering and scripts/check-query-options-erasure-ratchet.mjs (baseline keyed by path, never grows)', - inspectDanglingReferences: 'eslint query-options/no-any-erasure grandfathering and scripts/check-query-options-erasure-ratchet.mjs (baseline keyed by path, never grows)', + // red (measured: the six methods below, one site each; the last four once + // ruling 6624345766 dropped the no-yield rule and let them move). + seedAutonumber: QUERY_OPTIONS_GUARD, + inspectDanglingReferences: QUERY_OPTIONS_GUARD, + probeReferencingRows: QUERY_OPTIONS_GUARD, + referenceExists: QUERY_OPTIONS_GUARD, + resolveMasterDetailParent: QUERY_OPTIONS_GUARD, + resolveMasterDetailParents: QUERY_OPTIONS_GUARD, }); /** @@ -228,25 +234,19 @@ const CLASS_PATH_READER_NEEDLES = Object.freeze({ */ const BOUND_HOST_METHODS = new Set(); -/** Reserved words a function declaration may not be named. */ -const RESERVED = new Set(('break case catch class const continue debugger default delete do else enum export extends ' + - 'false finally for function if import in instanceof new null return super switch this throw true try typeof var ' + - 'void while with implements interface let package private protected public static yield await').split(' ')); - /** Every reason a method stays, with the one-line why the plan prints. */ const STAY_REASONS = Object.freeze({ 'no-section': 'declared above the first section anchor', 'section-kept': 'its section maps to no module', - 'static': 'a static member (the host is an instance)', + 'static-this': 'a static method whose body reads `this` (the class; a facet static has its own)', + 'static-name-shared': 'a static method whose name an instance member also takes (one facet cannot hold both)', 'no-body': 'an overload signature or abstract member', 'generator': 'a generator', 'overloaded': 'an overloaded method', 'decorated': 'a decorated method', 'not-identifier-name': 'a computed, string or #private name', - 'reserved-name': 'its name is reserved for a function declaration', 'inferred-return-type': 'a non-private method whose return type is inferred (the .d.ts would infer it through the module)', - 'type-parameters': 'a generic method (Function#call does not carry its type parameters)', - 'parameter-shape': 'a destructured, defaulted or `this` parameter (the stub cannot forward it unchanged)', + 'parameter-shape': 'a destructured or `this` parameter (the stub cannot forward it unchanged)', 'comment-in-parameters': 'a comment inside its parameter list (one-line parameters would drop it)', 'path-reader': 'a reader takes its text out of engine.ts by path (PATH_READER_PINS / PATH_READER_NEEDLES / CLASS_PATH_READER_NEEDLES)', 'arguments': 'its body reads `arguments`', @@ -255,17 +255,18 @@ const STAY_REASONS = Object.freeze({ 'this-type': 'it names the polymorphic `this` type', 'bare-this': 'it uses `this` other than as `this.member`', 'unknown-member': 'it reads a `this.member` the class does not declare', - 'this-write': 'it assigns a `this.member` (a host getter cannot be written)', + 'this-write': 'it assigns a `this.member` that is not a field, or deletes one (the host carries a setter for a field only)', 'method-as-value': 'it reads a method as a value instead of calling it', 'field-call': 'it calls a field or accessor as a function (the host would change its receiver)', 'overloaded-host-method': 'it calls an overloaded method (the host closure cannot be typed)', - 'non-public-static': 'it reads a private or protected static of ObjectQL (unreachable from a module function)', + 'static-write': 'its static reads go through ObjectQLStatics and it assigns a static (the forwarder is read-only)', + 'static-as-value': 'its static reads go through ObjectQLStatics and it reads a static method as a value (a forwarder is another function)', + 'static-field-call': 'its static reads go through ObjectQLStatics and it calls a static field or accessor (the receiver would change)', + 'overloaded-static': 'its static reads go through ObjectQLStatics and it calls an overloaded static (one forwarder cannot carry the overloads)', + 'static-route-conflict': 'its facet module reads the statics through ObjectQLStatics, imported AS `ObjectQL`, and it names `ObjectQL` other than as `ObjectQL.`', 'module-write': 'it assigns a module-scope binding of engine.ts (an import is read-only)', 'already-moved': 'its body is already a hand-series seam stub', - 'name-taken': 'its name is already a module-scope binding in engine.ts', - 'name-collision': 'its name collides with an import its module needs', - 'parameter-shadows': 'a parameter shadows the moved function or a stub local', - 'no-yield': 'its stub would be no shorter than its body (moving it would grow engine.ts)', + 'parameter-shadows': 'a parameter shadows a stub local (`host`, `readX`, `writeX`)', }); /** Every reason a top-level declaration ahead of the class stays in engine.ts. */ @@ -491,26 +492,35 @@ function readMethod(tm, ctx, m) { if (!reasons.includes(tag)) reasons.push(tag); }; const host = new Map(); // member -> 'method' | 'getter' + const writes = new Set(); // fields the body assigns: the host carries a setter for each const imports = new Map(); // local -> { value } const moduleRefs = new Map(); // engine.ts top-level name -> { value } - - if (m.isStatic()) add('static'); + // The private-static-through-host rule: the statics the body reads as + // `ObjectQL.x`, whether one is non-public (that takes the route), what the + // route cannot carry, and whether the body names `ObjectQL` any other way. + const classStatics = new Map(); // static -> 'method' | 'getter' + const routeBlockers = []; + let readsNonPublicStatic = false; + let classOther = false; + const result = () => ({ reasons, host, writes, imports, moduleRefs, classStatics, routeBlockers, readsNonPublicStatic, classOther }); + + if (m.isStatic() && ctx.members.has(m.getName())) add('static-name-shared'); if (!m.getBody()) add('no-body'); if (m.isAbstract && m.isAbstract()) add('no-body'); if (m.isGenerator()) add('generator'); if (m.getOverloads().length > 0) add('overloaded'); if (m.getDecorators().length > 0) add('decorated'); const nameNode = m.getNameNode(); - if (!Node.isIdentifier(nameNode)) { add('not-identifier-name'); return { reasons, host, imports, moduleRefs }; } + if (!Node.isIdentifier(nameNode)) { add('not-identifier-name'); return result(); } const name = nameNode.getText(); - if (RESERVED.has(name)) add('reserved-name'); if (!m.getReturnTypeNode() && m.getScope() !== 'private') add('inferred-return-type'); - if (m.getTypeParameters().length > 0) add('type-parameters'); for (const p of m.getParameters()) { - if (!Node.isIdentifier(p.getNameNode()) || p.hasInitializer() || p.getName() === 'this') add('parameter-shape', p.getName()); + // A defaulted identifier forwards as it is: the stub keeps the default and + // hands the body the value it produced. + if (!Node.isIdentifier(p.getNameNode()) || p.getName() === 'this') add('parameter-shape', p.getName()); } const body = m.getBody(); - if (!body) return { reasons, host, imports, moduleRefs }; + if (!body) return result(); // A comment between the parentheses that no parameter's own text carries. const params = m.getParameters(); @@ -538,6 +548,7 @@ function readMethod(tm, ctx, m) { if (Node.isMetaProperty(node) && node.getKeywordToken() === SyntaxKind.NewKeyword) add('new-target'); if (kind === SyntaxKind.ThisKeyword) { if (thisContainer(tm, node) !== m) continue; + if (m.isStatic()) { add('static-this'); continue; } const parent = node.getParent(); if (!Node.isPropertyAccessExpression(parent) || parent.getExpression() !== node) { add('bare-this'); continue; } const memberNameNode = parent.getNameNode(); @@ -545,7 +556,12 @@ function readMethod(tm, ctx, m) { const member = memberNameNode.getText(); const info = ctx.members.get(member); if (!info) { add('unknown-member', member); continue; } - if (isWritten(tm, parent)) add('this-write', member); + if (isWritten(tm, parent)) { + // A field is written through a host setter; a method, an accessor or + // a `delete` cannot be. + if (info.kind === 'field' && !Node.isDeleteExpression(parent.getParent())) writes.add(member); + else add('this-write', member); + } const outer = parent.getParent(); const called = Node.isCallExpression(outer) && outer.getExpression() === parent; if (info.kind === 'method') { @@ -590,12 +606,23 @@ function readMethod(tm, ctx, m) { // for it) when it stayed. Two things a module function cannot do: // reach a non-public static of the class, and assign an import. const declName = decl.name && ts.isIdentifier(decl.name) ? decl.name.text : node.getText(); - if (ts.isClassDeclaration(decl) && declName === CLASS_NAME && !typePos) { + if (ts.isClassDeclaration(decl) && declName === CLASS_NAME) { const access = node.getParent(); - if (Node.isPropertyAccessExpression(access) && access.getExpression() === node) { + const info = !typePos && Node.isPropertyAccessExpression(access) && access.getExpression() === node + ? ctx.statics.get(access.getName()) : null; + if (info) { const member = access.getName(); - const scope = ctx.statics.get(member); - if (scope && scope !== 'public') add('non-public-static', member); + if (info.scope !== 'public') readsNonPublicStatic = true; + const outer = access.getParent(); + const called = Node.isCallExpression(outer) && outer.getExpression() === access; + const block = (tag) => { if (!routeBlockers.includes(tag)) routeBlockers.push(tag); }; + if (isWritten(tm, access) || info.kind === 'setter') block(`static-write: ${member}`); + if (info.kind === 'method' && !called) block(`static-as-value: ${member}`); + if (info.kind === 'method' && info.overloaded) block(`overloaded-static: ${member}`); + if (info.kind !== 'method' && called) block(`static-field-call: ${member}`); + classStatics.set(member, info.kind === 'method' ? 'method' : 'getter'); + } else { + classOther = true; } } if (!typePos && isWritten(tm, node)) add('module-write', declName); @@ -606,15 +633,19 @@ function readMethod(tm, ctx, m) { } const binding = ctx.scope.bindings.get(name); - if (binding) { - if (binding.kind === 'import' && SEAM_SPECIFIER.test(binding.specifier)) add('already-moved', binding.specifier); - else add('name-taken'); - } + if (binding && binding.kind === 'import' && SEAM_SPECIFIER.test(binding.specifier)) add('already-moved', binding.specifier); for (const p of params) { const pn = p.getName(); - if (pn === name || pn === 'host' || /^read[A-Z]/.test(pn)) add('parameter-shadows', pn); + if (pn === 'host' || /^(read|write)[A-Z]/.test(pn)) add('parameter-shadows', pn); + } + // A body that reads a non-public static takes the route; what the route + // cannot carry keeps it on the class. (A module another mover routes + // applies the same to every mover in it: `plan`.) + if (readsNonPublicStatic) { + for (const tag of routeBlockers) add(tag); + if (classOther) add('static-route-conflict'); } - return { reasons, host, imports, moduleRefs }; + return result(); } function isModuleScopeDecl(ts, decl) { @@ -640,6 +671,54 @@ function hostNameOf(moduleFile) { return `${slug.split('-').map((w) => w[0].toUpperCase() + w.slice(1)).join('')}Host`; } +/** `engine-transactions-section.ts` -> `TransactionsSectionFacet`: the stateless class the bodies move into. */ +function facetNameOf(moduleFile) { + return hostNameOf(moduleFile).replace(/Host$/, 'Facet'); +} + +/** The object the class fills with forwarders to the statics the facets read, and its type. */ +const STATICS_NAME = `${CLASS_NAME}Statics`; +const STATICS_HOST_NAME = `${CLASS_NAME}StaticsHost`; +/** How a routed facet module names the class as a type: `ObjectQL` there is `ObjectQLStatics`. */ +const CLASS_TYPE_ALIAS = `${CLASS_NAME}Class`; + +/** The placeholder above a moved public or protected body: its docblock stays on the class. */ +const placeholderDoc = (name) => `/** The body of \`${CLASS_NAME}#${name}\`; the method's docblock there is its contract. */`; + +/** + * The statics block `engine.ts` carries for the routed facets: the type and + * the object ahead of the class, and the static block at the end of the class + * that fills the object. `kinds` maps each routed static to `method` (a + * forwarder that calls it on the class) or `getter` (read when read). Pure: + * `verify` rebuilds both texts from the base class and compares them. + */ +function staticsTexts(kinds) { + const members = [...kinds.keys()].sort(); + const ahead = [ + `// ${MARK} The statics of \`${CLASS_NAME}\` its facet modules read, private ones included, as`, + `// the class declares them (an indexed access on the class). The static block at the end of`, + `// the class fills \`${STATICS_NAME}\` with forwarders: a method is called on the class at`, + '// call time, a field or accessor is read when the body reads it. A facet module imports', + `// \`${STATICS_NAME}\` AS \`${CLASS_NAME}\`, so each moved body reads \`${CLASS_NAME}.x\` unedited;`, + '// `index.ts` re-exports neither, so neither reaches the package.', + `export interface ${STATICS_HOST_NAME} {`, + ...members.map((n) => ` readonly ${n}: (typeof ${CLASS_NAME})['${n}'];`), + '}', + `export const ${STATICS_NAME} = {} as ${STATICS_HOST_NAME};`, + ].join('\n'); + const block = [ + ` // ${MARK} Fills \`${STATICS_NAME}\` (declared above the class) for the facet modules.`, + ' static {', + ` Object.defineProperties(${STATICS_NAME}, {`, + ...members.map((n) => (kinds.get(n) === 'method' + ? ` ${n}: { value: (...args: Parameters) => ${CLASS_NAME}.${n}(...args) },` + : ` ${n}: { get: () => ${CLASS_NAME}.${n} },`)), + ' });', + ' }', + ].join('\n'); + return { ahead, block }; +} + /** Is `node` inside a JSDoc comment? A `{@link X}` there is prose, not a use. */ function inJsDoc(tm, node) { for (let n = node.compilerNode; n; n = n.parent) { @@ -805,13 +884,25 @@ function plan(tm, text, { const cls = sf.getClass(CLASS_NAME); if (!cls) refuse(`no class ${CLASS_NAME} in ${HOST_BASENAME}`); const sectionOf = readSections(cls, anchors); + // The class's statics: scope, and what a forwarder in `ObjectQLStatics` + // would be (a method forwards as a function; a field or accessor as a getter). const statics = new Map(); for (const m of cls.getMembers()) { if (typeof m.isStatic === 'function' && m.isStatic() && typeof m.getName === 'function') { - statics.set(m.getName(), typeof m.getScope === 'function' ? m.getScope() : 'public'); + const { Node } = tm; + const prev = statics.get(m.getName()); + const kind = Node.isMethodDeclaration(m) ? 'method' : Node.isSetAccessorDeclaration(m) ? (prev ? prev.kind : 'setter') + : Node.isGetAccessorDeclaration(m) ? 'getter' : 'field'; + statics.set(m.getName(), { + scope: typeof m.getScope === 'function' ? m.getScope() : 'public', kind, + overloaded: Node.isMethodDeclaration(m) && m.getOverloads().length > 0, + }); } } const ctx = { text, sf, cls, members: instanceMembers(tm, cls), scope: moduleScope(tm, sf), statics, pins, needles, classNeedles }; + for (const n of [STATICS_NAME, STATICS_HOST_NAME, ...SECTION_ANCHORS.filter((row) => row[2]).map((row) => facetNameOf(row[2]))]) { + if (ctx.scope.bindings.has(n)) refuse(`${HOST_BASENAME} already binds ${n}, a name this script generates`); + } const methodNames = new Set(cls.getMethods().map((m) => m.getName())); const stalePins = Object.keys(pins).filter((n) => !methodNames.has(n) && !ctx.scope.bindings.has(n)); if (stalePins.length > 0) refuse(`PATH_READER_PINS names what ${HOST_BASENAME} no longer declares: ${stalePins.join(', ')}`); @@ -833,33 +924,55 @@ function plan(tm, text, { const lineCount = m.getEndLineNumber() - startLine + 1; const body = m.getBody(); const bodyLines = body ? body.getEndLineNumber() - body.getStartLineNumber() + 1 : 0; + // Docblock ownership (ruling 6624345766 item 1): a PRIVATE method's + // docblock moves with its body; a public or protected one stays on the + // facade, where the published surface prints it. + const docStart = m.getStart(true); + const doc = m.getScope() === 'private' && docStart < m.getStart() ? text.slice(docStart, m.getStart()) : null; const record = { name, node: m, section: section ? section[1] : null, module: section ? section[2] : null, - startLine, lineCount, bodyLines, ...read, + startLine, lineCount, bodyLines, doc, docLines: doc ? doc.split('\n').length - 1 : 0, ...read, }; - if (record.reasons.length === 0) { - record.stubLines = stubBody(record, record.module, ' ').split('\n').length; - if (record.stubLines >= bodyLines) record.reasons.push('no-yield'); + if (section && section[2]) { + for (const p of m.getParameters()) if (p.getName() === facetNameOf(section[2])) read.reasons.push(`parameter-shadows: ${p.getName()}`); } + if (record.reasons.length === 0) record.stubLines = stubBody(record, record.module, ' ').split('\n').length; records.push(record); } - // A function name its module's imports already use: drop it, recompute, until stable. + // The private-static-through-host rule, per facet module: once one mover + // reads the statics through `ObjectQLStatics` (imported AS `ObjectQL`), + // every mover in that module does, so what the route cannot carry keeps a + // mover on the class. A router itself never carries a blocker (readMethod + // already kept it), so the set of routed modules is stable. for (let changed = true; changed;) { changed = false; - const byModule = groupMovers(records); - for (const [moduleFile, movers] of byModule) { - const taken = new Set([CLASS_NAME, hostNameOf(moduleFile), 'ThisParameterType', 'Pick']); - for (const r of movers) { - for (const local of r.imports.keys()) taken.add(local); - for (const local of r.moduleRefs.keys()) taken.add(local); - } + for (const movers of groupMovers(records).values()) { + if (!movers.some((r) => r.readsNonPublicStatic)) continue; for (const r of movers) { - if (taken.has(r.name)) { r.reasons.push('name-collision'); changed = true; } + const extra = [...r.routeBlockers, ...(r.classOther ? ['static-route-conflict'] : [])].filter((t) => !r.reasons.includes(t)); + if (extra.length > 0) { r.reasons.push(...extra); changed = true; } } } } const modules = groupMovers(records); + const routedModules = new Set(); + const routedStatics = new Map(); + for (const [moduleFile, movers] of modules) { + if (!movers.some((r) => r.readsNonPublicStatic)) continue; + routedModules.add(moduleFile); + for (const r of movers) for (const [member, kind] of r.classStatics) routedStatics.set(member, kind); + } + for (const [moduleFile, movers] of modules) { + const facet = facetNameOf(moduleFile); + for (const r of movers) { + for (const local of [...r.imports.keys(), ...r.moduleRefs.keys()]) { + if (local === facet || local === hostNameOf(moduleFile) || local === CLASS_TYPE_ALIAS) { + refuse(`${moduleFile}: ${r.name} reads ${local}, a name this script generates in that module`); + } + } + } + } const hostUses = hostUsesOfMovers(tm, ctx, top, records); // What each side imports across the cut: the section modules read moved @@ -882,7 +995,7 @@ function plan(tm, text, { if (r.exported) continue; for (const n of r.names) if (hostUses.has(n) || sectionLeafNames.has(n)) leafExports.push(n); } - return { sf, cls, ctx, records, modules, top, hostUses, leafExports, hostExportsForSections }; + return { sf, cls, ctx, records, modules, top, hostUses, leafExports, hostExportsForSections, routedModules, routedStatics }; } /** Movers grouped by module, modules in the order their first member appears. */ @@ -899,6 +1012,7 @@ function groupMovers(records) { // ── Generation ─────────────────────────────────────────────────────────────── const readerName = (member) => `read${member[0].toUpperCase()}${member.slice(1)}`; +const writerName = (member) => `write${member[0].toUpperCase()}${member.slice(1)}`; const byName = ([a], [b]) => (a < b ? -1 : a > b ? 1 : 0); const leafSpecifier = () => `./${LEAF_MODULE.replace(/\.ts$/, '.js')}`; @@ -937,7 +1051,7 @@ function importPair(specifier, uses) { * engine.ts's import order; then the moved top-level names, from the leaf; * then what stayed in engine.ts (lever (a)), including the class. */ -function importLines(scope, users, { needsHost = false, moduleRefs = new Map(), leafNames = new Map() } = {}) { +function importLines(scope, users, { needsHost = false, moduleRefs = new Map(), leafNames = new Map(), routed = false } = {}) { const bySpecifier = new Map(); for (const r of users) { for (const [local, use] of r.imports) { @@ -972,7 +1086,13 @@ function importLines(scope, users, { needsHost = false, moduleRefs = new Map(), const fromLeaf = new Map(); const fromHost = new Map(); for (const [n, use] of moduleRefs) (leafNames.has(n) ? fromLeaf : fromHost).set(n, use); - if (needsHost && !fromHost.has(CLASS_NAME)) fromHost.set(CLASS_NAME, { value: false }); + if (routed) { + // The private-static-through-host rule: `ObjectQL.x` in a moved body reads + // the forwarder object; the host interface names the class by an alias. + fromHost.delete(CLASS_NAME); + fromHost.set(`${STATICS_NAME} as ${CLASS_NAME}`, { value: true }); + if (needsHost) fromHost.set(`${CLASS_NAME} as ${CLASS_TYPE_ALIAS}`, { value: false }); + } else if (needsHost && !fromHost.has(CLASS_NAME)) fromHost.set(CLASS_NAME, { value: false }); out.push(...importPair(leafSpecifier(), fromLeaf)); out.push(...importPair('./engine.js', fromHost)); return out; @@ -998,10 +1118,14 @@ function forwardList(m) { return m.getParameters().map((p) => (p.isRestParameter() ? `...${p.getName()}` : p.getName())); } -function moduleText(moduleFile, title, movers, scope, leafNames) { +function moduleText(moduleFile, title, movers, scope, leafNames, routed = false) { const hostName = hostNameOf(moduleFile); + const facet = facetNameOf(moduleFile); + const classType = routed ? CLASS_TYPE_ALIAS : CLASS_NAME; const hostMembers = new Map(); + const written = new Set(); for (const r of movers) for (const [member, kind] of r.host) hostMembers.set(member, kind); + for (const r of movers) for (const member of r.writes) written.add(member); const needsHost = hostMembers.size > 0; const moduleRefs = new Map(); for (const r of movers) { @@ -1019,22 +1143,31 @@ function moduleText(moduleFile, title, movers, scope, leafNames) { out.push(...wrapComment([MARK, 'The', `"${title}"`, 'section', 'of', '`ObjectQL`:', 'the', 'bodies', 'of', ...names, 'moved', 'out', 'of', '`engine.ts`', 'whole', 'by', `\`${SCRIPT}\`.`])); out.push(' *'); - out.push(' * A MOVE, not a redesign. Each function below is the body of the `ObjectQL`'); + out.push(` * A MOVE, not a redesign. {@link ${facet}} is a stateless facet of \`ObjectQL\``); + out.push(' * (ruling 6624345766): each static method of it is the body of the `ObjectQL`'); out.push(' * method of the same name exactly as it stood: not re-indented, not'); - out.push(' * formatted, not edited. The method stays on the class with its docblock and'); - out.push(' * signature, so the published `dist/*.d.ts`, an instance spy and a subclass'); - out.push(' * override are what they were; the docblock there is the contract.'); + out.push(' * formatted, not edited. The method stays on the class as a one-call stub with'); + out.push(' * its signature, so the published surface, an instance spy and a subclass'); + out.push(' * override are what they were. A public or protected method keeps its'); + out.push(' * docblock on the class, the contract; a private method\'s docblock moved here'); + out.push(' * with its body. The state stays on `ObjectQL`.'); out.push(' *'); if (needsHost) { out.push(` * The bodies reach the class through {@link ${hostName}}: exactly the`); - out.push(' * `ObjectQL` members they read, private ones included. Each method hands its'); + out.push(' * `ObjectQL` members they read, private ones included. Each stub hands its'); out.push(' * body one literal of the members that body reads, plus a `.call`: methods go'); out.push(' * as closures that call them on the engine at call time, so a subclass'); out.push(' * override or an instance spy is still the one reached; fields and accessors'); - out.push(' * go as GETTERS, read when the body reads them, as they were on the class.'); + out.push(' * go as GETTERS, read when the body reads them, as they were on the class,'); + out.push(' * and a field a body assigns goes with a SETTER that assigns it on the engine.'); } else { - out.push(' * No body reads an `ObjectQL` member, so each method calls its function'); - out.push(' * directly.'); + out.push(' * No body reads an `ObjectQL` member, so each stub calls its body directly.'); + } + if (routed) { + out.push(' *'); + out.push(` * \`ObjectQL\` here is \`${STATICS_NAME}\` (imported AS \`ObjectQL\`): forwarders to the`); + out.push(' * statics the bodies read, private ones included, which the class fills in a'); + out.push(` * static block; the class as a type is \`${CLASS_TYPE_ALIAS}\`.`); } out.push(' *'); if (readsHostValues) { @@ -1048,30 +1181,36 @@ function moduleText(moduleFile, title, movers, scope, leafNames) { } out.push(' */'); out.push(''); - out.push(...importLines(scope, movers, { needsHost, moduleRefs, leafNames })); + out.push(...importLines(scope, movers, { needsHost, moduleRefs, leafNames, routed })); if (needsHost) { out.push(''); out.push('/**'); - out.push(' * The `ObjectQL` members the functions below read — every one of them, and'); - out.push(' * no other. Each is typed AS the class member (an indexed access on the'); - out.push(' * class), so a change to a member\'s signature on the class is a change here'); - out.push(' * too. `readonly` members are the getters.'); + out.push(` * The \`ObjectQL\` members the bodies of {@link ${facet}} read — every one of`); + out.push(' * them, and no other. Each is typed AS the class member (an indexed access on'); + out.push(' * the class), so a change to a member\'s signature on the class is a change'); + out.push(' * here too. `readonly` members are the getters; the others are fields a body'); + out.push(' * assigns.'); out.push(' */'); out.push(`export interface ${hostName} {`); for (const member of [...hostMembers.keys()].sort()) { - out.push(` ${hostMembers.get(member) === 'getter' ? 'readonly ' : ''}${member}: ObjectQL['${member}'];`); + const ro = hostMembers.get(member) === 'getter' && !written.has(member); + out.push(` ${ro ? 'readonly ' : ''}${member}: ${classType}['${member}'];`); } out.push('}'); } - for (const r of movers) { + out.push(''); + out.push(`/** The bodies of the "${title}" section of \`ObjectQL\`: a stateless facet, called by the class's stubs. */`); + out.push(`export class ${facet} {`); + movers.forEach((r, i) => { const m = r.node; const picks = [...r.host.keys()].sort().map((k) => `'${k}'`).join(' | '); const params = [...(r.host.size > 0 ? [`this: Pick<${hostName}, ${picks}>`] : []), ...paramsText(m)]; const ret = m.getReturnTypeNode() ? `: ${m.getReturnTypeNode().getText()}` : ''; - out.push(''); - out.push(`/** The body of \`ObjectQL#${r.name}\`; the method's docblock there is its contract. */`); - out.push(`export ${m.isAsync() ? 'async ' : ''}function ${r.name}(${params.join(', ')})${ret} ${m.getBody().getText()}`); - } + if (i > 0) out.push(''); + const typeParams = m.getTypeParameters().length > 0 ? `<${m.getTypeParameters().map((t) => t.getText()).join(', ')}>` : ''; + out.push(` ${r.doc ?? `${placeholderDoc(r.name)}\n `}static ${m.isAsync() ? 'async ' : ''}${r.name}${typeParams}(${params.join(', ')})${ret} ${m.getBody().getText()}`); + }); + out.push('}'); out.push(''); return out.join('\n'); } @@ -1111,28 +1250,39 @@ function leafText(scope, top, leafExports) { return `${text}\n`; } +const DOC_MOVED = 'its docblock with it'; + function stubBody(r, moduleFile, closeIndent) { const ind = `${closeIndent} `; const hostName = hostNameOf(moduleFile); + // A generic method calls its body INSTANTIATED with its own type + // parameters (`Function#call` alone would erase them). + const typeArgs = r.node.getTypeParameters().map((t) => t.getName()); + const fn = typeArgs.length > 0 ? `(${facetNameOf(moduleFile)}.${r.name}<${typeArgs.join(', ')}>)` : `${facetNameOf(moduleFile)}.${r.name}`; + const fnType = `${facetNameOf(moduleFile)}.${r.name}`; const out = ['{']; const args = forwardList(r.node).join(', '); + const doc = r.doc ? `, ${DOC_MOVED}` : ''; if (r.host.size === 0) { - out.push(`${ind}// ${MARK} The body moved to \`${moduleFile}\`.`); - out.push(`${ind}return ${r.name}(${args});`); + out.push(`${ind}// ${MARK} The body moved to \`${moduleFile}\`${doc}.`); + out.push(`${ind}return ${fn}(${args});`); } else { - out.push(`${ind}// ${MARK} The body moved to \`${moduleFile}\` (see \`${hostName}\`).`); + out.push(`${ind}// ${MARK} The body moved to \`${moduleFile}\`${doc} (see \`${hostName}\`).`); const members = [...r.host.keys()].sort(); for (const member of members) { if (r.host.get(member) === 'getter') out.push(`${ind}const ${readerName(member)} = () => this.${member};`); + if (r.writes.has(member)) out.push(`${ind}const ${writerName(member)} = (v: ${CLASS_NAME}['${member}']) => { this.${member} = v; };`); } - out.push(`${ind}const host: ThisParameterType = {`); + out.push(`${ind}const host: ThisParameterType = {`); for (const member of members) { - if (r.host.get(member) === 'getter') out.push(`${ind} get ${member}() { return ${readerName(member)}(); },`); - else if (BOUND_HOST_METHODS.has(member)) out.push(`${ind} ${member}: this.${member}.bind(this),`); + if (r.host.get(member) === 'getter') { + out.push(`${ind} get ${member}() { return ${readerName(member)}(); },`); + if (r.writes.has(member)) out.push(`${ind} set ${member}(v) { ${writerName(member)}(v); },`); + } else if (BOUND_HOST_METHODS.has(member)) out.push(`${ind} ${member}: this.${member}.bind(this),`); else out.push(`${ind} ${member}: (...a) => this.${member}(...a),`); } out.push(`${ind}};`); - out.push(`${ind}return ${r.name}.call(host${args ? `, ${args}` : ''});`); + out.push(`${ind}return ${fn}.call(host${args ? `, ${args}` : ''});`); } out.push(`${closeIndent}}`); return out.join('\n'); @@ -1144,7 +1294,7 @@ function stubBody(r, moduleFile, closeIndent) { * import bindings the prune dropped. Pure. */ function generate(tm, text, planned) { - const { sf, ctx, modules, top, hostUses, leafExports, hostExportsForSections } = planned; + const { sf, ctx, modules, top, hostUses, leafExports, hostExportsForSections, routedModules, routedStatics } = planned; const edits = []; const importBlock = []; if (top.movers.length > 0) { @@ -1169,7 +1319,7 @@ function generate(tm, text, planned) { const anyHost = movers.some((r) => r.host.size > 0); importBlock.push(`\n// ${MARK} ${title}: the methods of the same names below hand the bodies` + `${anyHost ? ` what they read (\`${hostName}\`)` : ' their arguments'}.`); - importBlock.push(`\n${namedStatement('import', movers.map((r) => r.name).sort(), ` from './${moduleFile.replace(/\.ts$/, '.js')}';`)}`); + importBlock.push(`\nimport { ${facetNameOf(moduleFile)} } from './${moduleFile.replace(/\.ts$/, '.js')}';`); for (const r of movers) { const body = r.node.getBody(); const lineStart = text.lastIndexOf('\n', body.getEnd() - 1) + 1; @@ -1178,8 +1328,18 @@ function generate(tm, text, planned) { edits.push({ start: body.getStart(), end: body.getEnd(), text: stubBody(r, moduleFile, closeIndent) }); const asyncKw = r.node.getModifiers().find((x) => x.getKind() === tm.SyntaxKind.AsyncKeyword); if (asyncKw) edits.push({ start: asyncKw.getStart(), end: r.node.getNameNode().getStart(), text: '' }); + if (r.doc) edits.push({ start: r.node.getStart(true), end: r.node.getStart(), text: '' }); } } + if (routedStatics.size > 0) { + // The private-static-through-host rule: the forwarder object ahead of the + // class (ahead of its leading comments), the static block that fills it + // after the class's last member. + const { ahead, block } = staticsTexts(routedStatics); + edits.push({ start: ctx.cls.getFullStart(), end: ctx.cls.getFullStart(), text: `\n\n${ahead}` }); + const last = ctx.cls.getMembers().at(-1); + edits.push({ start: last.getEnd(), end: last.getEnd(), text: `\n\n${block}` }); + } if (hostExportsForSections.size > 0) { const spelled = [...hostExportsForSections.keys()].sort().map((n) => { const st = ctx.sf.getStatements().find((s) => (declaredNames(tm, s) ?? []).includes(n)); @@ -1206,7 +1366,7 @@ function generate(tm, text, planned) { const leafNames = top.moverNames; if (top.movers.length > 0) moduleTexts.set(LEAF_MODULE, leafText(ctx.scope, top, leafExports)); for (const [moduleFile, movers] of modules) { - moduleTexts.set(moduleFile, moduleText(moduleFile, movers[0].section, movers, ctx.scope, leafNames)); + moduleTexts.set(moduleFile, moduleText(moduleFile, movers[0].section, movers, ctx.scope, leafNames, routedModules.has(moduleFile))); } return { host: pruned.text, modules: moduleTexts, pruned: pruned.removed }; } @@ -1421,17 +1581,27 @@ function verify(tm, baseText, headText, moduleTexts, { readSibling = null } = {} } continue; } + // A facet module: imports, the host interface, and ONE class, the facet, + // whose every member is a static method that is a moved body. const fns = new Map(); + const facet = facetNameOf(file); for (const st of sf.getStatements()) { - if (Node.isFunctionDeclaration(st)) fns.set(st.getName(), st); - else if (!Node.isImportDeclaration(st) && !Node.isInterfaceDeclaration(st)) { - failures.push(`${file}: a top-level statement that is neither an import, the host interface nor a moved function: ${st.getText().slice(0, 60)}`); + if (Node.isImportDeclaration(st)) continue; + if (Node.isInterfaceDeclaration(st) && st.getName() === hostNameOf(file)) continue; + if (Node.isClassDeclaration(st) && st.getName() === facet && !fns.size) { + for (const member of st.getMembers()) { + if (Node.isMethodDeclaration(member) && member.isStatic() && member.getBody() && !fns.has(member.getName())) fns.set(member.getName(), member); + else failures.push(`${file}: ${facet} has a member that is not a moved body: ${member.getText().slice(0, 60)}`); + } + continue; } + failures.push(`${file}: a top-level statement that is neither an import, the host interface nor the facet class ${facet}: ${st.getText().slice(0, 60)}`); } - modules.set(file, { sf, fns, claimed: new Set() }); + modules.set(file, { sf, text, fns, claimed: new Set() }); } - const marker = new RegExp(`^\\{\\s*// ${MARK.replace(/[[\]]/g, '\\$&')} The body moved to \`(engine-[a-z0-9-]+\\.ts)\``); + const marker = new RegExp(`^\\{\\s*// ${MARK.replace(/[[\]]/g, '\\$&')} The body moved to \`(engine-[a-z0-9-]+\\.ts)\`(, ${DOC_MOVED})?`); + const movedDocs = []; const edits = []; for (const m of headCls.getMethods()) { const body = m.getBody(); @@ -1443,7 +1613,7 @@ function verify(tm, baseText, headText, moduleTexts, { readSibling = null } = {} const fn = mod && mod.fns.get(name); if (!fn) { failures.push(`${name}: its stub names ${hit[1]}, which has no function ${name}`); continue; } mod.claimed.add(name); - const base = baseCls.getMethod(name); + const base = baseCls.getMethods().find((x) => x.getName() === name && x.isStatic() === m.isStatic()); if (!base || !base.getBody()) { failures.push(`${name}: no method of that name with a body at base`); continue; } const fnParams = fn.getParameters().filter((p) => p.getName() !== 'this').map((p) => p.getText()); const baseParams = base.getParameters().map((p) => p.getText()); @@ -1452,16 +1622,62 @@ function verify(tm, baseText, headText, moduleTexts, { readSibling = null } = {} params: JSON.stringify(fnParams) === JSON.stringify(baseParams), returnType: (fn.getReturnTypeNode()?.getText() ?? null) === (base.getReturnTypeNode()?.getText() ?? null), async: fn.isAsync() === base.isAsync(), - typeParameters: fn.getTypeParameters().length === 0 && base.getTypeParameters().length === 0, + typeParameters: JSON.stringify(fn.getTypeParameters().map((t) => t.getText())) === JSON.stringify(base.getTypeParameters().map((t) => t.getText())), + static: m.isStatic() === base.isStatic(), }; + // Docblock ownership: the facet member's leading text is the base + // method's docblock (the stub says so, and only a private method's), else + // exactly the placeholder; the undo puts a moved docblock back. + const lead = mod.text.slice(fn.getStart(true), fn.getStart()); + const docMoved = Boolean(hit[2]); + if (docMoved) { + if (m.getStart(true) !== m.getStart()) failures.push(`${name}: its stub says the docblock moved, but the class still carries one`); + if (m.getScope() !== 'private') failures.push(`${name}: a ${m.getScope()} method's docblock moved off the class (only a private one may)`); + edits.push({ start: m.getStart(), end: m.getStart(), text: lead }); + movedDocs.push(lead); + } else if (lead !== `${placeholderDoc(name)}\n `) { + failures.push(`${name}: the facet member's leading text is neither its moved docblock nor the placeholder`); + } for (const [what, ok] of Object.entries(same)) if (!ok) failures.push(`${name}: the generated function's ${what} is not the base method's`); moved.push({ name, module: hit[1], bodySha: sha256(fn.getBody().getText()), baseBodySha: sha256(base.getBody().getText()), - lines: base.getBody().getEndLineNumber() - base.getBody().getStartLineNumber() + 1, text: fn.getText(), ...same, + lines: base.getBody().getEndLineNumber() - base.getBody().getStartLineNumber() + 1, text: fn.getText(), docMoved, ...same, }); edits.push({ start: body.getStart(), end: body.getEnd(), text: fn.getBody().getText() }); if (fn.isAsync()) edits.push({ start: m.getNameNode().getStart(), end: m.getNameNode().getStart(), text: 'async ' }); } + + // The statics route: the forwarder type and object ahead of the class and + // the static block that fills them must be exactly what `staticsTexts` + // writes for the statics the type names, each forwarded as the BASE class + // declares it (a method called, anything else read). Then the undo drops + // them. + const staticsHost = headSf.getInterface(STATICS_HOST_NAME); + const staticsVar = headSf.getVariableStatement(STATICS_NAME); + const staticBlocks = headCls.getMembers().filter((x) => Node.isClassStaticBlockDeclaration(x) && x.getFullText().includes(MARK)); + if (staticsHost || staticsVar || staticBlocks.length > 0) { + const kinds = new Map(); + for (const member of staticsHost ? staticsHost.getProperties() : []) { + const n = member.getName(); + const declared = baseCls.getMembers().filter((x) => typeof x.isStatic === 'function' && x.isStatic() && typeof x.getName === 'function' && x.getName() === n); + if (declared.length === 0) failures.push(`${STATICS_HOST_NAME}.${n}: the base class declares no static of that name`); + kinds.set(n, declared.some((x) => Node.isMethodDeclaration(x)) ? 'method' : 'getter'); + } + const { ahead, block } = staticsTexts(kinds); + const aheadAt = staticsHost ? headText.lastIndexOf(`// ${MARK} The statics of`, staticsHost.getStart()) : -1; + const aheadEnd = staticsVar ? staticsVar.getEnd() : -1; + if (!staticsHost || !staticsVar || aheadAt < 0 || headText.slice(aheadAt, aheadEnd) !== ahead) { + failures.push(`${STATICS_HOST_NAME} / ${STATICS_NAME} are not the generated forwarder declarations for the statics they name`); + } else { + edits.push({ start: headText.lastIndexOf('\n\n', aheadAt), end: aheadEnd, text: '' }); + } + const blockAt = staticBlocks.length === 1 ? headText.lastIndexOf(` // ${MARK} Fills`, staticBlocks[0].getStart()) : -1; + if (staticBlocks.length !== 1 || blockAt < 0 || headText.slice(blockAt, staticBlocks[0].getEnd()) !== block) { + failures.push(`the class's static block is not the generated filler of ${STATICS_NAME} for the statics it names`); + } else { + edits.push({ start: headText.lastIndexOf('\n\n', blockAt), end: staticBlocks[0].getEnd(), text: '' }); + } + } for (const [file, mod] of modules) { for (const name of mod.fns.keys()) if (!mod.claimed.has(name)) failures.push(`${file}: function ${name} has no stub on the class`); } @@ -1511,7 +1727,7 @@ function verify(tm, baseText, headText, moduleTexts, { readSibling = null } = {} rebuilt = assembled; } - const movedText = [...moved.map((m) => m.text), ...(leaf ? leaf.statements.map((s) => s.text) : [])].join('\n'); + const movedText = [...moved.map((m) => m.text), ...movedDocs, ...(leaf ? leaf.statements.map((s) => s.text) : [])].join('\n'); for (const n of prune.removed) { if (!new RegExp(`(^|[^\\w$])${n.replace(/\$/g, '\\$')}($|[^\\w$])`).test(movedText)) { failures.push(`the head dropped the import \`${n}\`, which no moved body or declaration reads`); @@ -1526,7 +1742,8 @@ function verify(tm, baseText, headText, moduleTexts, { readSibling = null } = {} `\n rebuilt: ${JSON.stringify(a[i] ?? '')}\n base: ${JSON.stringify(b[i] ?? '')}`); } if (moved.length === 0 && movedTop.length === 0) failures.push('no moved method or declaration found: a proof over nothing is not a proof'); - return { failures, moved, movedTop, pruned: prune.removed, linesBefore: lineCount(baseText), linesAfter: lineCount(headText) }; + return { failures, moved, movedTop, pruned: prune.removed, linesBefore: lineCount(baseText), linesAfter: lineCount(headText), + routedStatics: staticsHost ? staticsHost.getProperties().length : 0 }; } /** Lines as `wc -l` counts them: newline characters. */ @@ -1720,7 +1937,8 @@ function valueSpecifiers(src) { let selfTestReachedVerdict = false; const SELF_TEST_BATTERIES = Object.freeze({ - 'stay rules': 12, + 'stay rules': 20, + 'facets, statics and docblocks': 11, 'top-level moves': 11, 'byte-identity proof': 10, 'import prune': 4, @@ -1770,8 +1988,9 @@ export class ObjectQL { protected logger: { warn(m: string): void } = console; private readonly fn = (x: string): string => x; private static readonly SECRET = 1; + private static bump = 0; - /** Docblock that stays. */ + /** A private docblock: it moves with the body. */ private async movable(object: string, trx?: Knex.Transaction): Promise { ${pad} const a = helperImported(object) as ExportedShape | string; @@ -1779,6 +1998,7 @@ ${pad} return this.other(String(a)) + String(trx ? this.count : 0); } + /** A protected docblock: it stays on the facade. */ protected other(x: string): string { return x; } @@ -1832,6 +2052,47 @@ ${pad} return ObjectQL.SECRET; } + protected bumpsStatic(): number { +${pad} + ObjectQL.bump += 1; + return ObjectQL.SECRET; + } + + protected namesClassType(): ObjectQL | null { +${pad} + return null; + } + + protected delete(): number { +${pad} + return 4; + } + + protected deletesField(): void { +${pad} + delete this.knex; + } + + protected destructured({ a }: { a: number }): number { +${pad} + return a; + } + + protected generic(x: T): T { +${pad} + return this.other(x) as T; + } + + static staticHelper(n: number): number { +${pad} + return n + ObjectQL.SECRET; + } + + static usesThis(): number { +${pad} + return this.SECRET; + } + protected pinnedMethod(): number { ${pad} return 3; @@ -1884,12 +2145,20 @@ async function selfTest() { moves('readsLocal', 'lever (a)/(b): it imports the leaf value and the staying function'); moves('namesLocalType', 'a non-exported type it names moves to the leaf'); moves('scoped', 'a method it calls goes as a closure'); - stays('other', 'no-yield'); - stays('writes', 'this-write'); + moves('other', 'a body no longer than its stub moves too: the facade gate counts bodies, not lines'); + moves('writes', 'a field it assigns goes through a host setter'); + stays('deletesField', 'this-write'); + moves('delete', 'a reserved word is a valid facet member name'); stays('asValue', 'method-as-value'); stays('inferred', 'inferred-return-type'); - stays('defaulted', 'parameter-shape'); - stays('readsStatic', 'non-public-static'); + moves('defaulted', 'the stub keeps the default and forwards the value'); + stays('destructured', 'parameter-shape'); + moves('generic', 'the stub calls the body instantiated with its type parameters'); + moves('staticHelper', 'a static whose body reads no `this`'); + stays('usesThis', 'static-this'); + moves('readsStatic', 'a private static it reads goes through ObjectQLStatics'); + stays('bumpsStatic', 'static-write'); + stays('namesClassType', 'static-route-conflict'); check('a pinned method and a method carrying a needle stay (path-reader)', why('pinnedMethod').startsWith('path-reader') && why('carriesNeedle').startsWith('path-reader'), `${why('pinnedMethod')} | ${why('carriesNeedle')}`); @@ -1919,7 +2188,7 @@ async function selfTest() { /export \{ DERIVED, type ExportedShape \} from '\.\/engine-top-level\.js';/.test(out.host)); check('lever (a): the staying function a moved body reads is exported for its module, and the module imports it', /export \{ dependsOnPinned \};/.test(out.host) && - /import \{ dependsOnPinned \} from '\.\/engine\.js';/.test(out.modules.get('engine-lifecycle.ts') ?? '')); + /import \{ ObjectQLStatics as ObjectQL, dependsOnPinned \} from '\.\/engine\.js';/.test(out.modules.get('engine-lifecycle.ts') ?? '')); } open('byte-identity proof'); @@ -1929,7 +2198,7 @@ async function selfTest() { const out = generate(tm, input, planned); const again = generate(tm, input, fixturePlan(tm, input)); const v = verify(tm, input, out.host, out.modules); - check('the generated split proves against its base', v.failures.length === 0 && v.moved.length === 4 && v.movedTop.length === 5, + check('the generated split proves against its base', v.failures.length === 0 && v.moved.length === 12 && v.movedTop.length === 5, [...v.failures, `moved: ${v.moved.map((m) => m.name).join()}`, `top: ${v.movedTop.map((m) => m.name).join()}`].join(' | ')); check('two runs on one input write byte-identical trees', treeHash(new Map([['h', out.host], ...out.modules])) === treeHash(new Map([['h', again.host], ...again.modules]))); @@ -1954,6 +2223,39 @@ async function selfTest() { check('two moved declarations out of base order fail', swapped !== leaf && verify(tm, input, out.host, leafWith(swapped)).failures.length > 0); } + open('facets, statics and docblocks'); + { + const input = fixtureSource(); + const out = generate(tm, input, fixturePlan(tm, input)); + const mod = out.modules.get('engine-lifecycle.ts') ?? ''; + check('the bodies move into one stateless facet class of static methods', /\nexport class LifecycleFacet \{\n/.test(mod) && + /\n static async movable\(this: Pick ObjectQL.SECRET },\n });\n }\n}') && !out.host.includes('bump:')); + check('a written field gets a host setter that assigns it on the engine', + out.host.includes("const writeCount = (v: ObjectQL['count']) => { this.count = v; };") && out.host.includes('set count(v) { writeCount(v); },')); + check('a generic method calls its body instantiated with its own type parameters', + out.host.includes('return (LifecycleFacet.generic).call(host, x);') && mod.includes(' static generic(this: Pick new Map([...out.modules, ['engine-lifecycle.ts', text]]); + check('an edited moved docblock fails', verify(tm, input, out.host, + withMod(mod.replace('it moves with the body', 'it moved with the body'))).failures.length > 0); + check('an edited placeholder fails', verify(tm, input, out.host, + withMod(mod.replace(placeholderDoc('other'), placeholderDoc('other').replace('contract', 'promise')))).failures.length > 0); + check('a static forwarded as the wrong kind fails', verify(tm, input, + out.host.replace('SECRET: { get: () => ObjectQL.SECRET },', 'SECRET: { value: ObjectQL.SECRET },'), out.modules).failures.length > 0); + } + open('import prune'); { const input = fixtureSource(); From 7f26641dea8ab2abe241f2fd61372ab033c0ffe7 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 07:06:39 +0000 Subject: [PATCH 11/13] refactor(objectql): regenerate the engine.ts split in the facade shape `node scripts/codemod-split-engine.mjs --write` on main 9d1a465f, whose engine.ts is the base 0b99c2b1 one. Unedited. 161 of 278 method bodies move into 16 stateless facet classes, and the 146 top-level declarations stay in engine-top-level.ts. engine.ts goes from 23,245 to 14,140 lines. Three new modules: engine-compatibility-section.ts, engine-migration-state.ts and engine-transactions-section.ts. `--verify --base 0b99c2b1` proves 161/161 bodies and 146/146 declarations byte-identical, and the undo rebuilds the base engine.ts. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- packages/objectql/src/engine-actions.ts | 104 +- .../objectql/src/engine-cascade-planning.ts | 1184 +++- .../src/engine-compatibility-section.ts | 152 + .../src/engine-defaults-and-autonumbers.ts | 619 +- .../src/engine-delete-and-aggregate.ts | 44 +- packages/objectql/src/engine-find.ts | 341 +- .../src/engine-hooks-and-functions.ts | 247 +- packages/objectql/src/engine-insert.ts | 171 +- .../src/engine-middleware-and-context.ts | 305 +- .../objectql/src/engine-migration-state.ts | 428 ++ .../engine-references-and-master-detail.ts | 252 +- packages/objectql/src/engine-registration.ts | 212 +- .../src/engine-secrets-and-routing.ts | 183 +- .../objectql/src/engine-security-resolvers.ts | 283 +- .../src/engine-summaries-and-expansion.ts | 500 +- .../src/engine-transactions-section.ts | 197 + packages/objectql/src/engine.ts | 5055 ++++------------- 17 files changed, 6078 insertions(+), 4199 deletions(-) create mode 100644 packages/objectql/src/engine-compatibility-section.ts create mode 100644 packages/objectql/src/engine-migration-state.ts create mode 100644 packages/objectql/src/engine-transactions-section.ts diff --git a/packages/objectql/src/engine-actions.ts b/packages/objectql/src/engine-actions.ts index 9fe7841707f..a195e7c2e6f 100644 --- a/packages/objectql/src/engine-actions.ts +++ b/packages/objectql/src/engine-actions.ts @@ -2,41 +2,54 @@ /** * [#29986 codemod] The "Action system and packaged-action activation" section - * of `ObjectQL`: the bodies of `registerAction`, `listRegisteredActions` moved - * out of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * of `ObjectQL`: the bodies of `registerAction`, `executeAction`, + * `listRegisteredActions`, `removeActionsByPackage`, + * `setActionActivationStore`, `hydrateActionActivations`, `isActionEnabled`, + * `listDisabledActions`, `getDisabledActions`, `describeDisabledAction`, + * `setActionActive` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link ActionsFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link ActionsHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * * The module imports `engine.ts` for types only, so the two modules form no * evaluation cycle. `index.ts` re-exports nothing from this module. */ +import type { ActionActivationRow, ActionActivationStore, ActionActivationWriteOptions } from './action-activation.js'; import type { ObjectQL } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link ActionsFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface ActionsHost { + readonly actionActivation: ObjectQL['actionActivation']; readonly actions: ObjectQL['actions']; readonly logger: ObjectQL['logger']; } -/** The body of `ObjectQL#registerAction`; the method's docblock there is its contract. */ -export function registerAction(this: Pick, objectName: string, actionName: string, handler: (ctx: any) => Promise | any, packageName?: string): void { +/** The bodies of the "Action system and packaged-action activation" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class ActionsFacet { + /** The body of `ObjectQL#registerAction`; the method's docblock there is its contract. */ + static registerAction(this: Pick, objectName: string, actionName: string, handler: (ctx: any) => Promise | any, packageName?: string): void { const key = `${objectName}:${actionName}`; const replaced = this.actions.get(key); if (replaced !== undefined && replaced.package !== packageName) { @@ -54,8 +67,17 @@ export function registerAction(this: Pick, ob this.logger.debug('Registered action', { objectName, actionName, package: packageName }); } -/** The body of `ObjectQL#listRegisteredActions`; the method's docblock there is its contract. */ -export function listRegisteredActions(this: Pick): Array<{ objectName: string; actionName: string; package?: string }> { + /** The body of `ObjectQL#executeAction`; the method's docblock there is its contract. */ + static async executeAction(this: Pick, objectName: string, actionName: string, ctx: any): Promise { + const entry = this.actions.get(`${objectName}:${actionName}`); + if (!entry) { + throw new Error(`Action '${actionName}' on object '${objectName}' not found`); + } + return entry.handler(ctx); + } + + /** The body of `ObjectQL#listRegisteredActions`; the method's docblock there is its contract. */ + static listRegisteredActions(this: Pick): Array<{ objectName: string; actionName: string; package?: string }> { const out: Array<{ objectName: string; actionName: string; package?: string }> = []; for (const [key, entry] of this.actions.entries()) { const sep = key.indexOf(':'); @@ -68,3 +90,53 @@ export function listRegisteredActions(this: Pick): Array } return out; } + + /** The body of `ObjectQL#removeActionsByPackage`; the method's docblock there is its contract. */ + static removeActionsByPackage(this: Pick, packageName: string): void { + for (const [key, entry] of this.actions.entries()) { + if (entry.package === packageName) { + this.actions.delete(key); + } + } + } + + /** The body of `ObjectQL#setActionActivationStore`; the method's docblock there is its contract. */ + static setActionActivationStore(this: Pick, store: ActionActivationStore): void { + this.actionActivation.attach(store); + } + + /** The body of `ObjectQL#hydrateActionActivations`; the method's docblock there is its contract. */ + static async hydrateActionActivations(this: Pick): Promise { + return this.actionActivation.hydrate(); + } + + /** The body of `ObjectQL#isActionEnabled`; the method's docblock there is its contract. */ + static isActionEnabled(this: Pick, actionName: string): boolean { + return this.actionActivation.isEnabled(actionName); + } + + /** The body of `ObjectQL#listDisabledActions`; the method's docblock there is its contract. */ + static listDisabledActions(this: Pick): string[] { + return this.actionActivation.disabledNames(); + } + + /** The body of `ObjectQL#getDisabledActions`; the method's docblock there is its contract. */ + static getDisabledActions(this: Pick): string[] | undefined { + return this.actionActivation.durable ? this.actionActivation.disabledNames() : undefined; + } + + /** The body of `ObjectQL#describeDisabledAction`; the method's docblock there is its contract. */ + static describeDisabledAction(this: Pick, actionName: string): string { + this.logger.info( + `ACTION_DISABLED: packaged action '${actionName}' refused — it is switched off in the activation ledger ` + + '(sys_metadata_activation, ADR-0126 §8 item 2).', + { action: actionName, refusal: 'ACTION_DISABLED' }, + ); + return this.actionActivation.describeDisabled(actionName); + } + + /** The body of `ObjectQL#setActionActive`; the method's docblock there is its contract. */ + static async setActionActive(this: Pick, row: ActionActivationRow, options?: ActionActivationWriteOptions): Promise { + await this.actionActivation.setActive(row, options); + } +} diff --git a/packages/objectql/src/engine-cascade-planning.ts b/packages/objectql/src/engine-cascade-planning.ts index 91f259646cd..5bf930bfec0 100644 --- a/packages/objectql/src/engine-cascade-planning.ts +++ b/packages/objectql/src/engine-cascade-planning.ts @@ -2,52 +2,260 @@ /** * [#29986 codemod] The "Cascade and referential delete planning" section of - * `ObjectQL`: the bodies of `warnCascadeNotAtomic`, `cascadeRelationBehavior`, - * `referenceProbeFilterOfSet`, `isTenantColumnSetNull`, - * `organizationCountIsDisclosable` moved out of `engine.ts` whole by - * `scripts/codemod-split-engine.mjs`. + * `ObjectQL`: the bodies of `planCascadeAtomicity`, `warnCascadeNotAtomic`, + * `storedReferenceIncludes`, `remainderAfterMemberRemoval`, + * `referenceCheckContext`, `dependentCountIsDisclosable`, + * `cascadeDeleteRelations`, `cascadeRelationBehavior`, + * `fileReferenceCheckElevation`, `referenceProbeFilterOfSet`, + * `probeReferencingRowsOfSet`, `walkReferencingRelations`, + * `isTenantColumnSetNull`, `outlivesOrganization`, + * `keepsOrganizationReference`, `countReferencingRowsOutside`, + * `organizationCountContext`, `organizationCountIsDisclosable`, + * `refuseOrganizationDeleteWhileOwned`, `organizationOwnedRowCounts` moved out + * of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link CascadePlanningFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link CascadePlanningHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * - * The module imports `engine.ts` for types only, so the two modules form no - * evaluation cycle. `index.ts` re-exports nothing from this module. + * `ObjectQL` here is `ObjectQLStatics` (imported AS `ObjectQL`): forwarders to the + * statics the bodies read, private ones included, which the class fills in a + * static block; the class as a type is `ObjectQLClass`. + * + * Values that stayed in `engine.ts` are imported from it (lever (a) of + * ruling 6591753301); every use is inside a function body, so the import + * cycle is never read at evaluation time. `index.ts` re-exports nothing from + * this module. */ import type { ServiceObject } from '@objectstack/spec/data'; +import { SystemFieldName } from '@objectstack/spec/system'; import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { isMissingTableError } from '@objectstack/metadata/errors'; import { ORGANIZATION_OBJECT, resolveTenantFieldName } from './tenancy/system-write-organization.js'; import { isFederatedUnprovisionedInjectedColumn } from './federated-object.js'; -import { declaredMultiValued, deleteWalkTargetOf, statesDeleteSemantics } from './engine-top-level.js'; -import type { ObjectQL } from './engine.js'; +import { + cascadeMemberKey, cascadeSetAdd, cascadeSetHas, declaredMultiValued, deleteWalkTargetOf, statesDeleteSemantics, +} from './engine-top-level.js'; +import type { CascadeDeleteSet } from './engine-top-level.js'; +import { ObjectQLStatics as ObjectQL } from './engine.js'; +import type { ObjectQL as ObjectQLClass } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link CascadePlanningFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface CascadePlanningHost { - readonly cascadeNotAtomicReported: ObjectQL['cascadeNotAtomicReported']; - count: ObjectQL['count']; - readonly defaultDriver: ObjectQL['defaultDriver']; - readonly logger: ObjectQL['logger']; - referenceProbeFilter: ObjectQL['referenceProbeFilter']; - resolveObjectName: ObjectQL['resolveObjectName']; + readonly _registry: ObjectQLClass['_registry']; + readonly cascadeDeleteSets: ObjectQLClass['cascadeDeleteSets']; + readonly cascadeIssuedDeletes: ObjectQLClass['cascadeIssuedDeletes']; + readonly cascadeNotAtomicReported: ObjectQLClass['cascadeNotAtomicReported']; + cascadeRelationBehavior: ObjectQLClass['cascadeRelationBehavior']; + collectCascadeDeleteSet: ObjectQLClass['collectCascadeDeleteSet']; + count: ObjectQLClass['count']; + countReferencingRowsOutside: ObjectQLClass['countReferencingRowsOutside']; + readonly defaultDriver: ObjectQLClass['defaultDriver']; + delete: ObjectQLClass['delete']; + deleteRestrictedError: ObjectQLClass['deleteRestrictedError']; + readonly drivers: ObjectQLClass['drivers']; + fileReferenceCheckElevation: ObjectQLClass['fileReferenceCheckElevation']; + find: ObjectQLClass['find']; + getDriver: ObjectQLClass['getDriver']; + isTenantColumnSetNull: ObjectQLClass['isTenantColumnSetNull']; + readonly logger: ObjectQLClass['logger']; + organizationCountIsDisclosable: ObjectQLClass['organizationCountIsDisclosable']; + probeReferencingRows: ObjectQLClass['probeReferencingRows']; + probeReferencingRowsOfSet: ObjectQLClass['probeReferencingRowsOfSet']; + recordReferenceCheckElevation: ObjectQLClass['recordReferenceCheckElevation']; + referenceProbeFilter: ObjectQLClass['referenceProbeFilter']; + referenceProbeFilterOfSet: ObjectQLClass['referenceProbeFilterOfSet']; + refuseOrganizationDeleteWhileOwned: ObjectQLClass['refuseOrganizationDeleteWhileOwned']; + refuseRestrictsInsideOwnCascade: ObjectQLClass['refuseRestrictsInsideOwnCascade']; + resolveObjectName: ObjectQLClass['resolveObjectName']; + update: ObjectQLClass['update']; + walkReferencingRelations: ObjectQLClass['walkReferencingRelations']; } -/** The body of `ObjectQL#warnCascadeNotAtomic`; the method's docblock there is its contract. */ -export function warnCascadeNotAtomic(this: Pick, object: string): void { +/** The bodies of the "Cascade and referential delete planning" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class CascadePlanningFacet { + /** + * Can the by-id delete of `object` and its whole cascade run as ONE unit of + * work? [#7413] + * + * `delete()`'s by-id branch ran `cascadeDeleteRelations` and then + * `driver.delete` with no transaction around either, and the cascade + * RE-ENTERS `this.delete()` / `this.update()` per dependent row — so every + * child committed on its own. A refusal partway (the `restrict` branch below, + * a child's permission check, a later child's `beforeDelete` hook) left an + * arbitrary PREFIX of the children deleted while the caller received 409/403 + * and reasonably read it as "nothing happened". #4620 already ruled the + * principle for the batch path — *atomic honoured for real, or refused; a + * partial delete has no natural undo* — and this path never got it. + * + * Three verdicts, because "wrap it in a transaction" is only unconditionally + * right in one of them: + * + * - **`'none'`** — no registered object declares a relation of the delete + * family ({@link statesDeleteSemantics}: `master_detail` / `lookup` / + * `tree`) pointing at `object`, so `cascadeDeleteRelations` cannot write anything + * and there is no multi-write unit to make atomic. The delete runs exactly + * as it did before this card: one `driver.delete`, no transaction opened, + * no degrade warning. A plain non-cascade delete is the CONTROL for this + * change and must stay byte-identical. + * - **`'split'`** — dependents exist, but at least one participant (the + * parent or a transitively-referencing child) resolves to a driver other + * than the engine's DEFAULT one. `transaction()` opens on the default + * driver and covers that connection alone (ADR-0119 D1 — no two-phase + * commit on `IDataDriver`), so opening one here would not buy atomicity; it + * would REFUSE the cascade outright, because `enforceTransactionOrigin` + * throws {@link CrossDatasourceTransactionWriteError} for a business write + * inside a transaction that does not cover it (#5351 / #5696 point 2, the + * 2026-08-06 ruling). A cross-datasource cascade delete works today, and + * turning it into a hard refusal is a strictly worse answer than the + * non-atomic one it has always had. So it keeps that answer and says so + * once — see {@link warnCascadeNotAtomic}. + * - **`'atomic'`** — dependents exist and every participant sits on the + * default driver. This is the shape the card measured and the shape + * virtually every deployment has; the caller gets one unit of work. + * + * A pure registry walk — no I/O, no driver call. It answers from the SCHEMA + * (which objects *could* be reached), never from row counts, so the verdict + * is stable for a given schema and costs nothing on the common path. The walk + * is the transitive closure of "objects referencing X" because a cascade + * recurses into each child's own cascade; it is bounded by + * {@link ObjectQL.MAX_CASCADE_DEPTH}, the ceiling the recursion itself carries. + * + * Fails toward `'split'` on anything it cannot resolve WITHIN the walk (an + * unregistered reference, a `getDriver` throw): the degrade is today's + * behaviour, while a wrong `'atomic'` would manufacture the refusal this + * verdict exists to avoid. + * + * [#9002] That fail-toward rule is about resolving individual PARTICIPANTS. + * It never covered the registry read that produces the participant list in + * the first place, and the `catch → 'none'` that used to sit there was not an + * instance of it: `'none'` is the one verdict that asserts something positive + * about the schema — *nothing references this object* — and an unreadable + * registry cannot support that claim. The swallow's own comment argued the + * direction from `cascadeDeleteRelations` returning on the same failure, so + * that nothing was left un-cascaded to make atomic; once that seam propagates + * the premise is gone, and this one propagates with it. Note the ORDER makes + * this seam the first to speak: `delete()` calls this before it runs the + * cascade, so an unreadable registry now fails the delete before any row is + * touched. + */ + static planCascadeAtomicity(this: Pick, object: string): 'none' | 'split' | 'atomic' { + const objects: ServiceObject[] = this._registry.getAllObjects(); + + // Which objects reference `name` via a relation the cascade would follow. + // The family test ({@link statesDeleteSemantics}) + `reference` test is + // `cascadeDeleteRelations`'s own, so the two cannot disagree about who + // participates. `restrict` fields are INCLUDED deliberately: a restrict + // refusal is the card's core repro, and it must roll back the siblings + // already cascaded before it. + // + // [#22957] A `tree` takes part since it joined the family. Its reference + // names the object that declares it, so an object whose only relation is + // its own hierarchy now plans `'atomic'` rather than `'none'`: a subtree + // cascade is several deletes, and a refusal from a node deep in it must + // roll back the nodes deleted before it. + const referencing = (name: string): string[] => { + const out: string[] = []; + for (const child of objects) { + const childName = (child as any)?.name as string | undefined; + const fields = (child as any)?.fields as Record | undefined; + if (!childName || !fields) continue; + for (const [fieldName, fdef] of Object.entries(fields)) { + if (!statesDeleteSemantics(fdef)) continue; + // [#18550] The carrier is read through the ONE arbiter, so a + // `reference` no reader can read REFUSES here instead of reading as + // "this child does not reference `name`". The two answers stay + // different on purpose: + // + // - ABSENCE (`undefined` / `null` / `''`) is unchanged and still + // silent. `referenceCarrierOf` answers `undefined` for all three + // and this `continue` skips the field, which is what a field that + // names no target legitimately means (`FieldSchema.reference` is + // `.optional()`, `StrictField` declares it nullable). + // - UNREADABILITY is the loud one. An object- or array-valued + // carrier was TRUTHY here and then failed both name comparisons + // below, so the relation was dropped from the set silently — and + // this function's `'none'` is, by its own docblock above, the one + // verdict that asserts something POSITIVE about the schema + // ("nothing references this object"). An unreadable carrier can + // no more support that claim than an unreadable registry can. + const ref = deleteWalkTargetOf(fdef, 'ObjectQL.planCascadeAtomicity'); + if (!ref) continue; + let resolvedRef: string | undefined; + try { resolvedRef = this.resolveObjectName(ref); } catch { resolvedRef = undefined; } + if (ref !== name && resolvedRef !== name) continue; + // [#21910, #21918] The scan skips every column the registry injected + // into a federated object and the object does not provision, so this + // walk does too: the participant test stays the scan's own, as the + // comment above requires. A federated object this walk still reaches + // through a relation its author declared keeps the verdict + // `'split'`, because the scan probes that relation. + if (isFederatedUnprovisionedInjectedColumn(child, fieldName)) continue; + out.push(childName); + break; + } + } + return out; + }; + + const firstLevel = referencing(object); + if (firstLevel.length === 0) return 'none'; + + const defaultDriver = this.defaultDriver ? this.drivers.get(this.defaultDriver) : undefined; + if (!defaultDriver) return 'split'; + const onDefaultDriver = (name: string): boolean => { + try { return this.getDriver(name) === defaultDriver; } catch { return false; } + }; + + if (!onDefaultDriver(object)) return 'split'; + const seen = new Set([object]); + let frontier = firstLevel; + for (let depth = 0; depth < ObjectQL.MAX_CASCADE_DEPTH && frontier.length > 0; depth++) { + const next: string[] = []; + for (const name of frontier) { + if (seen.has(name)) continue; + seen.add(name); + if (!onDefaultDriver(name)) return 'split'; + next.push(...referencing(name)); + } + frontier = next; + } + return 'atomic'; + } + + /** + * The `'split'` verdict of {@link planCascadeAtomicity}, said out loud — once + * per object per engine instance (#7413). + * + * Same reasoning as {@link warnTransactionUnsupported}, which this mirrors + * deliberately: a capability is not available, so the delete keeps the + * non-atomic behaviour it has always had. `warn`, not `error` — at this + * moment nothing claimed-persisted has failed to land; what is missing is the + * ability to undo the cascade if a later child refuses. Escalating it would + * train readers to skim `error`, which AGENTS.md names as the mirror-image + * mistake. + */ + static warnCascadeNotAtomic(this: Pick, object: string): void { if (this.cascadeNotAtomicReported.has(object)) return; this.cascadeNotAtomicReported.add(object); this.logger.warn( @@ -62,8 +270,306 @@ export function warnCascadeNotAtomic(this: Pick, object: string, child: ServiceObject, fieldName: string, fdef: any): string | undefined { + /** + * [#9362] Does this STORED reference value point at `id`? The exact answer + * the `$contains` pushdown in {@link ObjectQL.referenceProbeFilter} can only + * approximate. + * + * Element-wise over the array, comparing `String(v)` — the same reading + * `dangling-reference-audit.ts` already applies to a stored reference + * (`Array.isArray(raw) ? raw : [raw]`), so the two integrity surfaces cannot + * disagree about what "this row references that record" means. An EXPANDED + * record in the slot is a read shape rather than an id write, and is skipped + * there for that reason; it is skipped here too. + * + * A non-array value is still compared rather than dismissed. A + * `multiple: true` slot holding a bare scalar is off-shape, and the two + * dispositions available are not symmetric: reading it as "references + * nothing" would drop a `restrict` refusal and let a referenced record be + * deleted — the fail-OPEN direction #8895 ruled out for this same guard — + * while reading it as a reference at most refuses a delete loudly. + */ + static storedReferenceIncludes(stored: unknown, id: string | number): boolean { + const wanted = String(id); + const values = Array.isArray(stored) ? stored : [stored]; + for (const v of values) { + if (v == null || typeof v === 'object') continue; + if (String(v) === wanted) return true; + } + return false; + } + + /** + * [#9688] What a `multiple: true` reference slot holds once the deleted + * record's membership is removed — the ONE computation behind both the + * per-row `required` judgement in {@link ObjectQL.cascadeDeleteRelations} + * and the `set_null` write that judgement gates. + * + * One function rather than two readings that happen to match, because the + * judgement decides whether the write may run at all: a predicate computing + * the remainder differently could clear a delete and then have the write + * land the empty required set the judgement exists to refuse — or refuse a + * delete whose write would have produced a perfectly legal non-empty set. + * + * `String(v) !== String(id)` is the same reading + * {@link ObjectQL.storedReferenceIncludes} applies when narrowing + * `dependents`, so the member removed is exactly the member that made the + * row a dependent. A non-array value is normalized to the array spelling + * rather than dismissed, for the same reason the narrowing compares it: an + * off-shape bare scalar in a `multiple: true` slot is still a reference. + * + * [#27004] `cleared` are the members this cascade has already removed from + * the relation ({@link CascadeDeleteSet.cleared}): a row read once for a + * whole root set may still list them, and they are not written back. + */ + static remainderAfterMemberRemoval(stored: unknown, id: string | number, cleared?: ReadonlySet): unknown[] { + const current: unknown[] = Array.isArray(stored) ? stored : [stored]; + return current.filter((v) => String(v) !== String(id) && !cleared?.has(String(v))); + } + + /** + * [#12166] The identity the pre-delete reference CHECK runs under — system, + * unconditionally (maintainer ruling 2026-08-26, option A). + * + * `sudo()`-shaped, never a bare `{ isSystem: true }`. The full reasoning sits + * at the one call site (`cascadeDeleteRelations`'s dependents probe); the + * short form is that the spread carries three things the bare spelling would + * drop — the caller's transaction handle, the TENANT scope, and `userId`, + * which is the audit ledger's triggered-by half beside `isSystem`'s + * executed-as half. + * + * A helper rather than an inline spread because the disclosure re-probe below + * has to be able to say "the caller's OWN context" by contrast, and because a + * second inline spelling is how one of the two would later gain a key the + * other lacks. + */ + static referenceCheckContext(context?: ExecutionContext): ExecutionContext { + return { ...(context ?? {}), isSystem: true } as ExecutionContext; + } + + /** + * [#12166, ruling constraint 2] May the refusal disclose HOW MANY rows + * reference this record? + * + * The elevation above is what makes this question exist. Before it, the + * dependents probe ran as the caller, so every number the `DELETE_RESTRICTED` + * envelope reported (`dependentCount`, the localized message's `count`, the + * developerMessage) was derived from rows the caller could read by + * definition. After it, the probe sees rows the caller may have no grant on + * at all — and shipping that count back would hand a caller with NO read + * permission on `childName` an exact, repeatable cardinality oracle over it: + * delete-probe each record of `object`, read the count off the 409, and + * reconstruct the hidden table's reference histogram without ever being + * allowed to read a row. + * + * The ruling closes that: the error names the referenced OBJECT and nothing + * about rows — "one notch more conservative than Salesforce". It does NOT ask + * for the count to be dropped for everyone, and dropping it would be its own + * regression (the count is what makes the refusal actionable, and it predates + * this card for callers who could always compute it). So the count is + * disclosed on exactly one condition: **the caller's own identity would have + * produced the same rows.** + * + * The comparison is on ROW IDENTITY, not on length. Two different sets can + * have the same size, and RLS narrowing is precisely the case that produces + * one — a caller who may read `childName` at the object level but whose + * row-level filter hides two of three referencing rows must not be told + * "three" either. Comparing id sets makes the object-level denial and the + * row-level narrowing the same answer, which is what they are. + * + * Costs one extra query, and only on the refusal path — the delete is being + * aborted anyway, so this is not on any successful delete's critical path. + * + * Fails CLOSED, on the DISCLOSURE axis: any throw at all (a genuine + * permission denial, an outage, a driver error) answers "not disclosable". + * That direction is deliberate and is the opposite of #8895's stance for the + * integrity probe itself — an integrity guard that cannot run must not + * silently pass, but a DISCLOSURE that cannot be justified must not silently + * happen. The two guards face opposite ways because a wrong answer costs + * opposite things: there, a permitted delete that should have been refused; + * here, a leak. Nothing about the refusal itself depends on this — an + * unanswerable probe still refuses the delete, just without the number. + */ + static async dependentCountIsDisclosable(this: Pick, childName: string, probeWhere: Record, probedIds: readonly string[], fieldName: string, multiValued: boolean, id: string | number, context?: ExecutionContext): Promise { + // The caller IS the system (seed replay, migration, an internal cascade + // recursion). Nothing was elevated PAST them, so there is nothing to + // withhold — and withholding here would strip the count from the engine's + // own internal paths for no gain. + if (context?.isSystem === true) return true; + try { + // Typed through the DECLARED options type rather than cast: `where` is + // `Record` on `EngineQueryOptionsSchema` and `context` + // rides `BaseEngineOptions`, so nothing here needs erasing + // (`check:query-options-erasure` — the grandfathered `as any` on the + // elevated probe above is a pre-existing site this card does not sweep, + // tracked in #4918). + let own: any[] = await this.find(childName, { where: probeWhere, context }); + // The SAME narrowing the elevated probe applied (#9362), or the two sets + // would be compared through different definitions of "references this + // record" and a multi-value relation would read as non-disclosable + // always. + if (multiValued && own) { + own = own.filter((row: any) => ObjectQL.storedReferenceIncludes(row?.[fieldName], id)); + } + const ownIds = new Set((own ?? []).map((r: any) => String(r?.id))); + return ownIds.size === probedIds.length && probedIds.every((rid) => ownIds.has(rid)); + } catch (error) { + // Says something rather than rethrowing, and the asymmetry is the point + // (`check:durability-read-invention` accepts either — "say something, or + // ask the error's type"). + // + // Rethrowing is WRONG here specifically. The integrity answer is already + // in hand: the elevated probe succeeded, the dependents are known, and + // this delete is being refused either way. Propagating a failure of the + // DISCLOSURE probe would convert a correct `409 DELETE_RESTRICTED` into a + // `500` — a worse answer for the caller, produced by a query that exists + // only to decide how much detail the refusal carries. + // + // Nor is the invented value a durability hazard of the class that rule + // guards: no declared set is being read, and `false` withholds. The + // failure mode of getting this wrong is a slightly less informative + // refusal, never a permitted delete and never an over-disclosure. + // + // The one thing that WOULD be wrong is doing it silently — an operator + // seeing count-free refusals has no way to tell "withheld on purpose" + // from "the disclosure probe has been failing for a week". + this.logger.warn( + `[reference-cleanup] could not determine whether the caller may be told how many '${childName}' ` + + `record(s) reference this one; withholding the count from the refusal (the refusal itself stands)`, + { object: childName, error: (error as Error)?.message ?? String(error) }, + ); + return false; + } + } + + /** + * Apply referential delete behavior for relations pointing AT this record, + * before it is removed. For every registered object with a `master_detail`, + * `lookup` or `tree` field, or a `user` field that declares a + * `deleteBehavior` ({@link statesDeleteSemantics}), referencing + * `object`, honor the field's `deleteBehavior`: + * - `cascade` → delete the dependent rows (recursively, so grandchildren + * are handled by each child's own delete), + * - `set_null` → clear the foreign key. On a MULTI-VALUED field + * ([#18408] {@link declaredMultiValued}, ⛔ not the raw + * flag) the FK is a SET, so "clear" means remove the + * deleted MEMBER and keep the rest; an emptied set on + * an optional field is stored as `null`, the same as one + * never set — the representation `FieldSchema` pins + * (`packages/spec/src/data/field.zod.ts`, the `multiple` + * doc block, ruling A on #29343, record 6591577985, + * amending #9447), + * - `restrict` → refuse the delete when dependents exist. + * `master_detail` defaults to `cascade` (the parent owns the child + * lifecycle); `lookup` and `tree` default to `set_null` — except a `set_null` default + * on a REQUIRED lookup escalates to `restrict` (you can't null a NOT NULL + * FK; restricting with a clear dependent-count message beats a misleading + * " is required" 400 from the child) — and on a `multiple: true` + * required lookup that escalation is decided per ROW after the dependents + * probe (#9688): a row that keeps another member takes the member removal, + * a row the removal would EMPTY keeps the refusal, and the refusal counts + * only those rows. A by-id delete runs it for its one record; a predicate + * delete (`multi: true`) runs it for every row it matched ([#26871], see + * `delete()`'s predicate arm). + * + * [#27004] It takes a ROOT SET: the by-id delete passes a set of one, the + * predicate arm the ids it matched. Every phase below reads a relation once + * per {@link REFERENCE_PROBE_BATCH} records of the set + * ({@link probeReferencingRowsOfSet}), not once per record, and judges and + * writes per record as before. One walk of one set: a matched row another + * matched row's cascade reaches is in the set like any other member, so it + * refuses nothing and is cleared of nothing (#22305), and the predicate + * arm's own `deleteMany` removes it. + * + * [#22305] Two phases, so the walk knows its own set before it judges + * anything: + * 1. {@link collectCascadeDeleteSet} — read-only. Every record the cascade + * deletes, transitively, across every CASCADING relation, the root + * included. + * 2. {@link walkReferencingRelations} — the walk above, unchanged in + * order, except that a `restrict` (authored, or the escalated + * required `set_null`) and a `set_null` write consider only rows + * OUTSIDE the set. A row in the set is about to be deleted by this same + * cascade: it cannot be left dangling, so it refuses nothing, and + * clearing its reference would be a write against a row about to go. + * [#26718] Between the two, {@link refuseRestrictsInsideOwnCascade}, also + * read-only: an AUTHORED `restrict` held by a member the cascade reaches + * only THROUGH the record that restrict protects (a child holding a + * master_detail and a restrict lookup to one parent; a grandchild under it) + * refuses the delete before the first write. The row is in the set, so phase + * 2 alone would let the cascade delete it. + * + * Before this, the outcome depended on registration order: with the + * contacts registered ahead of the contracts, deleting an account refused + * on a contract's required lookup to a contact, a contract the same delete + * was about to remove; the other order deleted. A refusal from a row outside + * the set is unchanged, and so is the unit of work: the set is collected + * inside the transaction that `delete()` opens for the cascade. + * + * A nested `delete()` the walk makes for a member joins the ambient set + * ({@link cascadeDeleteSets}); any other delete reaching here — a hook's, + * say — is the root of its own cascade, as before. + */ + static async cascadeDeleteRelations(this: Pick, object: string, ids: ReadonlyArray, context?: ExecutionContext, depth = 0): Promise { + const roots: Array = []; + const seen = new Set(); + for (const id of ids) { + if (id == null || seen.has(String(id))) continue; + seen.add(String(id)); + roots.push(id); + } + if (roots.length === 0 || depth >= ObjectQL.MAX_CASCADE_DEPTH) return; + // [#9002] The registry read is UNGUARDED, deliberately — and this is the + // statement that decides whether the cascade runs AT ALL. + // + // It used to sit behind a bare `catch { return; }`, which is the #8895 + // shape one layer up: #8895's `catch` invented "no dependents" for ONE + // relation whose probe could not run; this one invented "no relations" for + // EVERY relation at once, before the per-relation probe was ever reached — + // no `restrict` refusal, no `set_null`, no `cascade`, nothing logged, and + // the caller told the delete succeeded. + // + // #8895 ruled the family *discriminate or propagate*. Discrimination needs + // a benign failure class — there it was the unprovisioned child TABLE, + // which genuinely cannot hold a referencing row. Here there is none: an + // unreadable registry is never truthfully "no relations", so `propagate` + // is the whole answer and the `catch` has nothing left to do. + // + // No new error code and no new response field: whatever the registry read + // raised reaches the caller with its envelope intact, exactly as #8895's + // probe failure does. + const objects: ServiceObject[] = this._registry.getAllObjects(); + // [#22305] A member of a cascade already in progress joins that cascade's + // set. Any other delete is the root of its own cascade, and collects the + // set first: before one refusal is judged and before one row is written. + const inherited = this.cascadeDeleteSets.getStore(); + const joining = inherited ? roots.filter((id) => cascadeSetHas(inherited.members, object, id)) : []; + if (inherited && joining.length > 0) { + await this.walkReferencingRelations(object, joining, context, objects, inherited); + } + const fresh = joining.length > 0 ? roots.filter((id) => !cascadeSetHas(inherited!.members, object, id)) : roots; + if (fresh.length === 0) return; + const cascadeSet = await this.collectCascadeDeleteSet(object, fresh, context, objects); + // [#26718] Every restrict a member's own cascade would override is judged + // here, before the first write. + await this.refuseRestrictsInsideOwnCascade(cascadeSet, fresh.map((id) => ({ object, id })), context, objects); + await this.cascadeDeleteSets.run(cascadeSet, () => + this.walkReferencingRelations(object, fresh, context, objects, cascadeSet), + ); + } + + /** + * Does `fieldName` on `child` point at `object` as a relation the cascade + * walk follows, and with which behaviour? `undefined` when it does not. The + * behaviour is the RESOLVED one, before the required-`set_null` escalation, + * which also reads the field's `multiple` and, on a multi-value field, the + * rows themselves. + * + * [#22305] Lifted out of the walk so both phases of + * {@link cascadeDeleteRelations} read relations one way: the set the first + * collects is the set the second deletes. + */ + static cascadeRelationBehavior(this: Pick, object: string, child: ServiceObject, fieldName: string, fdef: any): string | undefined { // [#22957] The family test is the shared predicate, so this reader, the // two walks that read it and `planCascadeAtomicity` admit one family. if (!statesDeleteSemantics(fdef)) return undefined; @@ -130,8 +636,34 @@ export function cascadeRelationBehavior(this: Pick, fieldName: string, fdef: { type?: unknown; multiple?: unknown }, ids: ReadonlyArray): Record { + /** + * [#12166, ruling constraint 3] File the elevation record for a probe of + * `childName` made while deleting `object`/`id` — once. One record per + * referenced OBJECT, not per relation: two lookup fields on the same child + * pointing at the same parent are one "the platform read object X as + * system" fact, and filing it twice would make the ledger's row count a + * function of the child's field layout. + * + * [#22305] The dedupe is the cascade's, not one call's: the collection walk + * and the deleting walk probe the same cascading relations, and between + * them file one record per deleted record per referenced object, as the + * single walk did. + */ + static fileReferenceCheckElevation(this: Pick, set: CascadeDeleteSet, object: string, id: string | number, childName: string, fieldName: string, context: ExecutionContext | undefined): void { + const key = JSON.stringify([object, String(id), childName]); + if (set.elevationsFiled.has(key)) return; + set.elevationsFiled.add(key); + this.recordReferenceCheckElevation(object, id, childName, fieldName, context); + } + + /** + * [#27004] {@link referenceProbeFilter} for a SET of records: the rows that + * reference any of `ids`. A single-valued reference asks `$in`; a + * multi-valued one ORs each id's own limbs, the spelling the nested-relation + * lowering already sends for a multiple lookup. One id is that id's own + * filter, byte for byte. + */ + static referenceProbeFilterOfSet(this: Pick, fieldName: string, fdef: { type?: unknown; multiple?: unknown }, ids: ReadonlyArray): Record { if (ids.length === 1) return this.referenceProbeFilter(fieldName, fdef, ids[0]); if (!declaredMultiValued(fdef)) return { [fieldName]: { $in: [...ids] } }; return { @@ -142,8 +674,408 @@ export function referenceProbeFilterOfSet(this: Pick, childName: string, fieldName: string, fdef: any, ids: ReadonlyArray, context: ExecutionContext | undefined): Promise> { + const byId = new Map(); + const multiValued = declaredMultiValued(fdef); + const batchSize = ObjectQL.REFERENCE_PROBE_BATCH; + for (let i = 0; i < ids.length; i += batchSize) { + const batch = ids.slice(i, i + batchSize); + const rows = await this.probeReferencingRows( + childName, fieldName, fdef, batch.length === 1 ? batch[0] : batch, + this.referenceProbeFilterOfSet(fieldName, fdef, batch), context, + ); + if (batch.length === 1) { + byId.set(String(batch[0]), rows); + continue; + } + const found = new Map(batch.map((id) => [String(id), []])); + let unattributed = false; + for (const row of rows) { + // The members' reading is `storedReferenceIncludes`'s, element by + // element; a single value is compared as one. + const stored = row?.[fieldName]; + const values: unknown[] = multiValued && Array.isArray(stored) ? stored : [stored]; + const held = new Set(); + for (const v of values) { + if (v == null || typeof v === 'object') continue; + if (found.has(String(v))) held.add(String(v)); + } + if (held.size === 0) { unattributed = true; break; } + for (const key of held) found.get(key)!.push(row); + } + if (unattributed) { + for (const id of batch) { + found.set(String(id), await this.probeReferencingRows( + childName, fieldName, fdef, id, this.referenceProbeFilter(fieldName, fdef, id), context, + )); + } + } + for (const [key, held] of found) byId.set(key, held); + } + return byId; + } + + /** + * [#22305] Phase 2 of {@link cascadeDeleteRelations}, for one record of + * `cascadeSet`: refuse, clear or cascade each relation that points at it. + * Unchanged from the single walk it was, but for three readings of the set: + * a `restrict` and a `set_null` consider only rows OUTSIDE it, and a row the + * walk has already entered is not cascaded into again. + */ + static async walkReferencingRelations(this: Pick, object: string, ids: ReadonlyArray, context: ExecutionContext | undefined, objects: ServiceObject[], cascadeSet: CascadeDeleteSet): Promise { + // Entered before the first nested `delete()` can come back here: every + // record of the set, so one member's cascade never deletes another. + for (const id of ids) cascadeSetAdd(cascadeSet.entered, object, id); + // [#24948] An organization that still owns rows is refused here, whole, + // before the first relation below is cascaded or cleared. + if (object === ORGANIZATION_OBJECT) { + for (const id of ids) await this.refuseOrganizationDeleteWhileOwned(id, context, objects, cascadeSet); + } + for (const child of objects) { + const childName = (child as any)?.name as string | undefined; + const fields = (child as any)?.fields as Record | undefined; + if (!childName || !fields) continue; + for (const [fieldName, fdef] of Object.entries(fields)) { + const resolvedBehavior = this.cascadeRelationBehavior(object, child, fieldName, fdef); + if (resolvedBehavior === undefined) continue; + // [#24948] Judged by `refuseOrganizationDeleteWhileOwned` above, which + // found no row outside the cascade set: there is nothing to clear, and + // clearing is not on offer for this column anyway. + if (this.isTenantColumnSetNull(object, child, fieldName, resolvedBehavior)) continue; + // [#24948] A reference the organization's delete keeps as it is, + // naming the deleted organization: never cleared, never counted. + if (object === ORGANIZATION_OBJECT && resolvedBehavior === 'set_null' && + ObjectQL.keepsOrganizationReference(child, fieldName)) continue; + let relationBehavior: string = resolvedBehavior; + + // [#9689] (maintainer ruling 2026-08-19, Q3 = B): the judgement the + // #9625 comment in `cascadeRelationBehavior` deferred is now taken — `FieldSchema` REJECTS an + // authored `deleteBehavior: 'set_null'` on a `master_detail` at parse + // time, so the value is meaningful again: one that still reaches this + // site came in around the parse seam (a raw `registerObject`, or a + // stored/artifact row written before the tightening — the two + // populations parse-time rejection measurably cannot catch, since the + // engine registers raw objects and never re-parses). The coercion + // itself stays: this delete is about to CASCADE children whose + // declaration asked for them to be kept, and that divergence must be + // loud and attributable, not silent. Sanctioned logger shape per + // PR #9750: reach for `error`, fall back to `warn` — NEVER an optional + // call like `logger.error?.()`, which emits nothing against a sink + // with no `error`. Caveat, measured (#4447): a built app artifact + // materializes FieldSchema defaults, so an artifact-loaded BARE + // master_detail also carries `set_null` and logs here — that residual + // imprecision is the materialized-default defect tracked as #9784, not + // a reason to soften this log. + if (fdef.type === 'master_detail' && fdef.deleteBehavior === 'set_null') { + for (const id of ids) { + const msg = + `[cascade-delete] ${childName}.${fieldName} declares deleteBehavior: 'set_null' on a ` + + `master_detail referencing '${object}' — that value is NOT honored on master_detail: the ` + + `delete of ${object}/${String(id)} CASCADES its referencing child rows, the opposite of what ` + + `the declaration asks (children kept). FieldSchema now rejects this combination at parse time; ` + + `this row reached the engine around the parse seam (raw registration, or metadata stored ` + + `before the tightening). Re-declare the field: 'restrict' refuses the parent delete while ` + + `children exist (no data loss), 'cascade' (or omitting the key) accepts the cascade ` + + `deliberately, or make it a lookup if children must survive the parent.`; + if (typeof this.logger.error === 'function') this.logger.error(msg); + else this.logger.warn(msg); + } + } + + // A REQUIRED foreign key cannot be nulled — set_null would issue an + // UPDATE clearing the FK, which the child's required-field validator + // rejects with a misleading " is required" 400 (the field isn't + // even on the object being deleted). That's a contradiction, not the + // author's intent: a required FK means the child can't exist detached, + // so deleting the parent must be RESTRICTed (SQL's default for a + // NOT NULL FK). Authors who want the children gone set + // deleteBehavior:'cascade' explicitly. + // + // [#9625] The test reads the RESOLVED `behavior` (`cascadeRelationBehavior`), + // which no longer records how the value got there. So it escalates + // BOTH the defaulted set_null and one the author wrote out as + // `deleteBehavior: 'set_null'` — the two are indistinguishable here by + // construction, and both are measured escalating + // (`engine-cascade-delete.test.ts`). Only `cascade` and `restrict` are + // untouched. The predecessor of this comment claimed the escalation + // applied to the *defaulted* set_null alone; that was true of + // `cascade`/`restrict` and false of an explicit `set_null`, which is + // the half-truth #9625 measured. Say it the way the code behaves: the + // escalation is a property of the RESOLVED behavior plus `required`, + // not of what the author typed. + // + // [#9688] The rationale above is also what BOUNDS the escalation, and + // on a `multiple: true` field it does not reach every row. The + // set_null limb there does not clear the slot: since #9438 it removes + // the deleted MEMBER and writes the remainder. So "a cleared required + // FK trips the child's validator" is true of exactly one case — the + // row whose set the removal would EMPTY, since `[]` is what the #9447 + // ruling (maintainer, 2026-08-18) says violates `required` on a + // multi-value field, and what the record validator has rejected since + // #9476. A row that still holds another live member is written a + // NON-EMPTY set, which `required` accepts and no validator objects + // to; refusing its parent's delete was broader than the contract, and + // refused it citing a failure that could not have happened. + // + // Emptiness is a property of a ROW, not of the field — and no row has + // been read at this point. So the multi-value half of the escalation + // is DEFERRED to just after the dependents probe (search + // `requiredSetNull` below), where the rows are known and exactly + // narrowed. The single-valued half stays here and is unchanged: + // clearing a scalar FK always writes `null`, so its premise needs no + // row to hold. + // + // The deferral changes WHEN the decision is made, never who it is + // reported to: a row that does keep the refusal still refuses THIS + // relation before its own set_null write runs, so the caller is told + // `DELETE_RESTRICTED` about the record it asked to delete, instead of + // the child's own `required` 400 naming a field that is not on that + // record's object at all. + // + // [#9362] `multiValued` is declared here rather than at the probe + // because the probe's filter spelling, the set_null write below and + // — since #9688 — this escalation all turn on it. + // [#18408] Was `fdef.multiple === true`. See {@link declaredMultiValued}. + const multiValued = declaredMultiValued(fdef); + const requiredSetNull = relationBehavior === 'set_null' && fdef.required === true; + if (requiredSetNull && !multiValued) { + relationBehavior = 'restrict'; + } + + // [#12166, ruling constraint 3] File the elevation BEFORE it happens, + // and file BOTH halves — triggered-by = the deleting operator, + // executed-as = system (the Salesforce/Dataverse ledger shape). + // + // Before the probe rather than after, because the fact being recorded + // is that the platform read this object under an identity the caller + // does not hold. That is true whether the probe then finds rows, finds + // none, refuses the delete, or fails — and a record written only on the + // success path would be missing exactly the runs an auditor goes + // looking for. + for (const id of ids) this.fileReferenceCheckElevation(cascadeSet, object, id, childName, fieldName, context); + + // [#27004] One probe per batch of the set; each record then takes its + // own rows, in the set's order. + const referencing = await this.probeReferencingRowsOfSet(childName, fieldName, fdef, ids, context); + const clearedKey = cascadeMemberKey(childName, fieldName); + let cleared = cascadeSet.cleared.get(clearedKey); + if (!cleared) { cleared = new Set(); cascadeSet.cleared.set(clearedKey, cleared); } + for (const id of ids) { + let behavior: string = relationBehavior; + let dependents = referencing.get(String(id)) ?? []; + if (dependents.length === 0) continue; + + // [#12166] The one record's probe filter: the DISCLOSURE re-probe in + // `deleteRestrictedError` asks the identical question under the + // caller's own identity. [#27004] The batched probe above is built from + // this same filter, id by id ({@link referenceProbeFilterOfSet}), and + // narrowed by the same reading, so the two cannot disagree about which + // rows reference this record. + const probeWhere = this.referenceProbeFilter(fieldName, fdef, id); + + // [#12166] The elevated probe's row IDENTITY, captured here — after the + // multi-value narrowing and BEFORE the `requiredSetNull && multiValued` + // narrowing below reduces `dependents` to the emptied subset. This is + // the set the disclosure decision compares against, and the stage + // matters: the emptied subset is computed from these rows' own stored + // values, so a caller who can see exactly these rows can derive the + // emptied count too, while a caller who cannot see them can derive + // neither. Comparing at the later stage would leak the difference. + const probedIds: readonly string[] = dependents.map((r: any) => String(r?.id)); + + // [#22305] A row the cascade itself deletes refuses nothing and is + // cleared of nothing: the triage ruling on #22305 is that a child in the + // cascade set never restricts a sibling in the same set. Such a row is + // not left dangling, because this same unit of work removes it, and + // clearing its reference would be a write against a row about to go. + // (The one cascade that is not one unit of work is + // `planCascadeAtomicity`'s `'split'`: there a later refusal can strand + // such a row, the partial outcome `warnCascadeNotAtomic` already + // declares for every row that path deletes.) So `restrict` (authored, + // or either half of the required escalation) and `set_null` judge only + // the rows OUTSIDE the set, and a relation whose rows are all inside it + // is done. `cascade` keeps every row: its rows ARE the set, and the + // walk deletes them below. + // + // [#26718] …except an AUTHORED restrict held by a row the cascade reaches + // only through the record that restrict protects: that one was judged + // before the cascade wrote anything ({@link refuseRestrictsInsideOwnCascade}), + // so a row in the set reaching here refuses nothing. + // + // After `probedIds`, on purpose: the disclosure check still compares + // the caller's own view against every row the elevated probe read, so a + // count is disclosed only to a caller who can see the whole relation, + // as before; what it counts is the rows that actually refuse. + if (behavior !== 'cascade') { + dependents = dependents.filter((row) => !cascadeSetHas(cascadeSet.members, childName, row?.id)); + if (dependents.length === 0) continue; + } + + // [#9688] The deferred half of the required escalation, decided per + // ROW now that the rows are read and exactly narrowed — every row + // here genuinely holds `id`, so its remainder is its set minus that + // one member. + // + // The emptiness question is asked through + // `remainderAfterMemberRemoval`, which is the SAME computation the + // set_null write below performs. One function, two call sites, so the + // judgement cannot predict a shape the write would not produce: a + // predicate that computed the remainder even slightly differently + // could clear the delete and then let the write land the very `[]` + // this judgement exists to prevent. + // + // ANY row that would be emptied refuses the WHOLE delete — a delete + // either happens or it does not, and a partial cascade is not on + // offer. `dependents` is reduced to those rows first, because every + // number the refusal reports reads it: `dependentCount`, the + // localized message's `count`, and the developerMessage. Counting the + // rows this delete no longer refuses over is the second defect the + // card names, and it is fixed by the same statement. + if (requiredSetNull && multiValued) { + const emptied = dependents.filter( + (row) => ObjectQL.remainderAfterMemberRemoval(row?.[fieldName], id, cleared).length === 0, + ); + if (emptied.length > 0) { + behavior = 'restrict'; + dependents = emptied; + } + } + + if (behavior === 'restrict') { + throw await this.deleteRestrictedError({ + object, id, child, childName, fieldName, fdef, dependents, probeWhere, probedIds, multiValued, + objects, context, + }); + } + + for (const dep of dependents) { + const depId = dep?.id; + if (depId == null) continue; + if (behavior === 'cascade') { + // [#22305] Entered already: an ancestor of this delete, still in + // progress up the stack (a cycle in the data), or a row an earlier + // branch of this cascade has deleted (a row reached by two paths). + // Either way this cascade removes it once, and re-entering would + // loop forever on the first and answer 404 on the second. + if (cascadeSetHas(cascadeSet.entered, childName, depId)) continue; + // Recurse via the public delete so the child's own cascade, + // hooks and events fire. + // + // [#23221] Inside a token naming exactly this child, which that + // delete claims and projects onto the DECLARED + // `HookContext.referentialCascadeDelete` (see + // claimReferentialCascadeDelete): on the real request path the + // caller's context reaches the child unchanged, so without it a + // freeze guard cannot tell this delete from a hand delete of the + // same row. The context is passed as before, by reference and + // unmarked — see `cascadeIssuedDeletes` for why the marker is not + // a context key. A grandchild is issued by the child's own walk, + // inside a token of its own. + await this.cascadeIssuedDeletes.run( + { object: childName, id: String(depId), claimed: false }, + () => this.delete(childName, { where: { id: depId }, context } as any), + ); + } else { + // [#3023] Clear the FK as an engine-internal referential-integrity + // write, tagged so plugin-security's ownership-anchor guard (#3004) + // treats an `owner_id = null` cascade as integrity maintenance, not + // a user-initiated disown — otherwise deleting the referenced record + // trips the transfer guard and aborts the cascade mid-way. Marker + // rides a server-DERIVED context (set here, never from client input + // — same trust model as `__expandRead`), so it cannot be forged from + // a request to bypass the guard on an ordinary write. + // + // [commit 34ce8e7db] This same marker is what `update()`'s hook-context + // assembly projects onto the DECLARED `HookContext. + // referentialFieldClear` (see buildReferentialFieldClear), so an + // app guard can recognise the cleanup without reading an + // operation-private key — on the real request path the inherited + // caller envelope makes the cleanup otherwise indistinguishable + // from a hand-clear of the same lookup. + // + // [#20006] `__referentialFieldClearCause` rides beside it: the + // delete this cleanup serves and the reference it clears, which a + // traversing rule's refusal of the cleanup names (see + // `resolvePredicateRelated`). It authorizes nothing — it is read + // only where the marker already holds, and only to word a refusal — + // and the referencing row's id in it never reaches that text. + const referentialCtx = { + ...(context ?? {}), + __referentialFieldClear: true, + __referentialFieldClearCause: { + object, id: String(id), referencingObject: childName, referencingId: String(depId), field: fieldName, + }, + } as ExecutionContext; + if (multiValued) { + // The FK is a SET, so `set_null` clears the deleted MEMBER, not + // the slot: filter the stored array and write what remains. + // Nulling the slot instead discarded every other live reference + // (measured: `["acc_a","acc_b"]` re-read as `null` after + // `acc_a` was deleted). + // + // The residual shape when the last member goes is not this + // engine's call to make: `FieldSchema` pins it — + // `packages/spec/src/data/field.zod.ts`, the `multiple` and + // `required` doc blocks (ruling A on #29343, record + // 6591577985, amending #9447) — and this write consumes that + // contract: it hands the write door the `[]` that + // `Array.prototype.filter` yields, and the door stores an + // emptied optional set as `null` (`blankAsNullColumns`, reach + // `'emptyList'`), the same as one never set. + // + // [#9688] The remainder is computed by + // `remainderAfterMemberRemoval`, which the per-row required + // judgement above calls on the same row: this write is only + // reached for a row that judgement measured as keeping at least + // one other member, and the two must agree by construction. + // The reading it applies — `String(v) !== String(id)`, and an + // off-shape bare scalar normalized to the array spelling — is + // documented there. + const next = ObjectQL.remainderAfterMemberRemoval(dep?.[fieldName], id, cleared); + await this.update(childName, { id: depId, [fieldName]: next }, { context: referentialCtx } as any); + } else { + await this.update(childName, { id: depId, [fieldName]: null }, { context: referentialCtx } as any); + } + } + } + // [#27004] Every row outside the set has had `id` cleared from it. + cleared.add(String(id)); + } + } + } + } + + /** + * [#24948] Is `fieldName` on `child` the TENANT column of `child`, pointing at + * the organization being deleted with the `set_null` behaviour? + * + * The tenant column is the one `resolveTenantFieldName` names: the column + * the engine stamps and scopes by, whether the registry injected it + * (`TENANT_SCOPE_FIELD_DEF`) or the object declared its own, as the identity + * tables do. Every such lookup resolves to `set_null` unless its author wrote + * something else, and ADR-0131 D1 leaves NULL nothing to mean: a cleared + * tenant column is an orphan, and on a store whose column is already NOT + * NULL the clear is a constraint failure halfway through a delete. So the + * engine never clears one. {@link refuseOrganizationDeleteWhileOwned} refuses + * the delete instead. An authored `cascade` or `restrict` keeps its own + * meaning. + */ + static isTenantColumnSetNull(object: string, child: ServiceObject, fieldName: string, behavior: string | undefined): boolean { return ( object === ORGANIZATION_OBJECT && behavior === 'set_null' && @@ -151,8 +1083,102 @@ export function isTenantColumnSetNull(object: string, child: ServiceObject, fiel ); } -/** The body of `ObjectQL#organizationCountIsDisclosable`; the method's docblock there is its contract. */ -export async function organizationCountIsDisclosable(this: Pick, childName: string, where: Record, count: number, context: ExecutionContext | undefined): Promise { + /** + * [#24948] Do `child`'s rows outlive the organization they name? Read from + * the declaration the object already carries, never from a list of names: + * + * - `managedBy: 'append-only'`, a ledger (the audit bucket: `sys_activity`, + * `sys_audit_log`, `sys_email` and their siblings). Such a row is history: + * it ages out on its own retention, or is the delivery log of something + * that already happened. The organization delete itself writes + * `sys_activity` rows, so a ledger that counted would refuse every + * organization delete, the empty ones included (seat decision 6305179819). + * - `managedBy: 'engine-owned'`, a row the platform writes as a consequence + * of records or activity (approval state, automation runs, notification + * and HTTP deliveries, mirrored mail, record shares, import jobs, + * settings values), which an administrator cannot delete over the data + * API. Counting it would make any organization that has been used + * undeletable through every door (seat decision 6307135388). + * + * Neither refusal counts such a row. It is kept, and so is every reference it + * holds to the organization: it goes on naming the deleted organization, + * never NULL (ADR-0131 D1) and never deleted (ADR-0131 §8), and under tenant + * scoping the row reads as no organization's. + */ + static outlivesOrganization(child: ServiceObject): boolean { + const managedBy = (child as { managedBy?: unknown })?.managedBy; + return managedBy === 'append-only' || managedBy === 'engine-owned'; + } + + /** + * [#24948] Does an organization delete keep `child.fieldName`, a `set_null` + * lookup into `sys_organization` that is not `child`'s tenant column, as it + * is? Two kinds of reference, and only these (seat decisions 6305179819, + * 6305996369 and 6307135388): + * + * - every such lookup on an object whose rows outlive the organization + * ({@link outlivesOrganization}): `sys_audit_log.tenant_id` is the audit + * log's attribution, not its tenant column, and history outlives what it + * audits; + * - an attribution column named `organization_id` on an object whose + * tenancy is disabled (`resolveTenantFieldName` answers null): neither + * counted nor cleared. No object in this tree declares one today. + * + * Everything else keeps its `set_null`: a pointer such as a session's or an + * API key's `active_organization_id`, or a parent-organization lookup, is + * cleared exactly as before. + */ + static keepsOrganizationReference(child: ServiceObject, fieldName: string): boolean { + if (ObjectQL.outlivesOrganization(child)) return true; + return fieldName === SystemFieldName.ORGANIZATION_ID && resolveTenantFieldName(child) === null; + } + + /** + * [#24948] How many rows of `childName` name the organization `id` through + * `fieldName`, leaving out the rows the cascade itself deletes. A system + * `count`, never a row read, for every table the cascade deletes no row of, + * which is every table but the cascading children of the organization (none + * on the platform). Only for such a table are the rows probed + * ({@link probeReferencingRows}), so the cascade's own members can be left + * out exactly. A table never provisioned holds none. + */ + static async countReferencingRowsOutside(this: Pick, childName: string, fieldName: string, fdef: any, id: string | number, cascadeSet: CascadeDeleteSet, context: ExecutionContext | undefined): Promise<{ where: Record; count: number }> { + const where = this.referenceProbeFilter(fieldName, fdef, id); + if ((cascadeSet.members.get(childName)?.size ?? 0) > 0) { + const probed = await this.probeReferencingRows(childName, fieldName, fdef, id, where, context); + return { where, count: probed.filter((row) => !cascadeSetHas(cascadeSet.members, childName, row?.id)).length }; + } + try { + return { where, count: await this.count(childName, { where, context: ObjectQL.organizationCountContext(context) }) }; + } catch (error) { + if (isMissingTableError(error, childName)) return { where, count: 0 }; + throw error; + } + } + + /** + * [#24948] The context an organization's row counts run under: system + * identity, and nothing of the caller's scope. A `tenantId` carried in from + * a caller acting in another organization would scope the count to THAT + * organization and read 0 rows naming this one, so the refusal would pass on + * an undercount. Only the request's transaction and trace are kept: the + * count must see the request's own uncommitted writes. The ordinary + * reference probes keep the caller's scope ({@link referenceCheckContext}). + */ + static organizationCountContext(context?: ExecutionContext): ExecutionContext { + return { + isSystem: true, + ...(context?.transaction !== undefined ? { transaction: context.transaction } : {}), + ...(context?.traceId !== undefined ? { traceId: context.traceId } : {}), + } as ExecutionContext; + } + + /** + * [#24948] May the refusal disclose `count` to the caller? Only when the + * caller's own identity counts the same rows: the elevated count read rows + * the caller may not see. (A system caller's own count is the elevated one.) + */ + static async organizationCountIsDisclosable(this: Pick, childName: string, where: Record, count: number, context: ExecutionContext | undefined): Promise { try { return (await this.count(childName, { where, context })) === count; } catch (error) { @@ -163,3 +1189,93 @@ export async function organizationCountIsDisclosable(this: Pick, id: string | number, context: ExecutionContext | undefined, objects: ServiceObject[], cascadeSet: CascadeDeleteSet): Promise { + const owned: Array<{ object: string; count?: number }> = []; + for (const child of objects) { + const childName = (child as any)?.name as string | undefined; + const fields = (child as any)?.fields as Record | undefined; + // An append-only or engine-owned object's rows outlive the organization: not counted, kept as they are. + if (!childName || !fields || ObjectQL.outlivesOrganization(child)) continue; + for (const [fieldName, fdef] of Object.entries(fields)) { + const behavior = this.cascadeRelationBehavior(ORGANIZATION_OBJECT, child, fieldName, fdef); + if (!this.isTenantColumnSetNull(ORGANIZATION_OBJECT, child, fieldName, behavior)) continue; + this.fileReferenceCheckElevation(cascadeSet, ORGANIZATION_OBJECT, id, childName, fieldName, context); + const { where, count } = await this.countReferencingRowsOutside(childName, fieldName, fdef, id, cascadeSet, context); + if (count === 0) continue; + const disclose = await this.organizationCountIsDisclosable(childName, where, count, context); + owned.push(disclose ? { object: childName, count } : { object: childName }); + } + } + if (owned.length === 0) return; + const named = owned + .map((o) => (o.count === undefined ? o.object : `${o.object} (${o.count})`)) + .join(', '); + const err: any = new Error( + `Cannot delete this organization: it still owns records in ${named}. ` + + 'Delete or move those records first, then delete the organization.', + ); + err.developerMessage = + `Cannot delete ${ORGANIZATION_OBJECT} (${id}): rows still reference it through their tenant column ` + + `in ${named}. The engine never clears a tenant column (ADR-0131 D1), so the delete is refused ` + + 'until those rows are deleted or moved to another organization.'; + err.code = 'DELETE_RESTRICTED'; + err.status = 409; + err.httpStatus = 409; + err.object = ORGANIZATION_OBJECT; + // The first table in the `DELETE_RESTRICTED` envelope's own two keys, and + // every table in `dependents`. + err.dependentObject = owned[0].object; + if (owned[0].count !== undefined) err.dependentCount = owned[0].count; + err.dependents = owned; + throw err; + } + + /** The body of `ObjectQL#organizationOwnedRowCounts`; the method's docblock there is its contract. */ + static async organizationOwnedRowCounts(this: Pick, organizationId: string, options?: { exclude?: Iterable; context?: ExecutionContext }): Promise> { + const exclude = new Set(options?.exclude ?? []); + const counts: Array<{ object: string; count: number }> = []; + for (const child of this._registry.getAllObjects()) { + const childName = (child as any)?.name as string | undefined; + const fields = (child as any)?.fields as Record | undefined; + if (!childName || !fields || exclude.has(childName) || ObjectQL.outlivesOrganization(child)) continue; + const tenantField = resolveTenantFieldName(child); + if (!tenantField || !fields[tenantField]) continue; + const fdef = fields[tenantField]; + if (this.cascadeRelationBehavior(ORGANIZATION_OBJECT, child, tenantField, fdef) === undefined) continue; + let count: number; + try { + count = await this.count(childName, { + where: this.referenceProbeFilter(tenantField, fdef, organizationId), + context: ObjectQL.organizationCountContext(options?.context), + }); + } catch (error) { + if (isMissingTableError(error, childName)) continue; + throw error; + } + if (count > 0) counts.push({ object: childName, count }); + } + return counts; + } +} diff --git a/packages/objectql/src/engine-compatibility-section.ts b/packages/objectql/src/engine-compatibility-section.ts new file mode 100644 index 00000000000..c6cdad11d97 --- /dev/null +++ b/packages/objectql/src/engine-compatibility-section.ts @@ -0,0 +1,152 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Compatibility / convenience API" section of `ObjectQL`: + * the bodies of `unregisterObject`, `getObject`, `getDriverByName`, + * `getDriverForObject`, `dropObjectSchema`, `datasource`, `on`, `close`, + * `create` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. {@link CompatibilitySectionFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. + * + * The bodies reach the class through {@link CompatibilitySectionHost}: exactly the + * `ObjectQL` members they read, private ones included. Each stub hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. + * + * Values that stayed in `engine.ts` are imported from it (lever (a) of + * ruling 6591753301); every use is inside a function body, so the import + * cycle is never read at evaluation time. `index.ts` re-exports nothing from + * this module. + */ + +import type { HookContext, ServiceObject } from '@objectstack/spec/data'; +import { StorageNameMapping } from '@objectstack/spec/system'; +import type { IDataDriver } from '@objectstack/core'; +import { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the bodies of {@link CompatibilitySectionFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. + */ +export interface CompatibilitySectionHost { + readonly _registry: ObjectQL['_registry']; + destroy: ObjectQL['destroy']; + readonly drivers: ObjectQL['drivers']; + getDriver: ObjectQL['getDriver']; + getDriverForObject: ObjectQL['getDriverForObject']; + getSchema: ObjectQL['getSchema']; + registerHook: ObjectQL['registerHook']; +} + +/** The bodies of the "Compatibility / convenience API" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class CompatibilitySectionFacet { + /** The body of `ObjectQL#unregisterObject`; the method's docblock there is its contract. */ + static unregisterObject(this: Pick, name: string, packageId?: string): void { + if (packageId) { + this._registry.unregisterObjectsByPackage(packageId); + } else { + // Remove from generic metadata as fallback + this._registry.unregisterItem('object', name); + } + } + + /** The body of `ObjectQL#getObject`; the method's docblock there is its contract. */ + static getObject(this: Pick, name: string): ServiceObject | undefined { + return this.getSchema(name); + } + + /** The body of `ObjectQL#getDriverByName`; the method's docblock there is its contract. */ + static getDriverByName(this: Pick, name: string): IDataDriver | undefined { + return this.drivers.get(name); + } + + /** The body of `ObjectQL#getDriverForObject`; the method's docblock there is its contract. */ + static getDriverForObject(this: Pick, objectName: string): IDataDriver | undefined { + try { + return this.getDriver(objectName); + } catch { + return undefined; + } + } + + /** The body of `ObjectQL#dropObjectSchema`; the method's docblock there is its contract. */ + static async dropObjectSchema(this: Pick, objectName: string): Promise { + const obj = this._registry.getObject(objectName) as any; + const driver = this.getDriverForObject(objectName); + if (!driver || typeof (driver as any).dropTable !== 'function') return; + const tableName = StorageNameMapping.resolveTableName(obj ?? ({ name: objectName } as any)); + await (driver as any).dropTable(tableName); + } + + /** The body of `ObjectQL#datasource`; the method's docblock there is its contract. */ + static datasource(this: Pick, name: string): IDataDriver { + const driver = this.drivers.get(name); + if (!driver) { + throw new Error(`[ObjectQL] Datasource '${name}' not found`); + } + return driver; + } + + /** The body of `ObjectQL#on`; the method's docblock there is its contract. */ + static on(this: Pick, event: string, objectName: string, handler: (ctx: HookContext) => Promise | void, packageId?: string): void { + this.registerHook(event, handler, { object: objectName, packageId }); + } + + /** The body of `ObjectQL#close`; the method's docblock there is its contract. */ + static async close(this: Pick): Promise { + return this.destroy(); + } + + /** The body of `ObjectQL#create`; the method's docblock there is its contract. */ + static async create(config: { + datasources?: Record; + objects?: Record; + hooks?: Array<{ event: string; object: string; handler: (ctx: HookContext) => Promise | void }>; + }): Promise { + const ql = new ObjectQL(); + + // Register drivers + if (config.datasources) { + for (const [name, driver] of Object.entries(config.datasources)) { + // Set driver name if not already set + if (!driver.name) { + (driver as any).name = name; + } + ql.registerDriver(driver, name === 'default'); + } + } + + // Register objects + if (config.objects) { + for (const [_key, schema] of Object.entries(config.objects)) { + ql.registerObject(schema); + } + } + + // Register hooks + if (config.hooks) { + for (const hook of config.hooks) { + ql.on(hook.event, hook.object, hook.handler); + } + } + + // Initialize (connect drivers) + await ql.init(); + + return ql; + } +} diff --git a/packages/objectql/src/engine-defaults-and-autonumbers.ts b/packages/objectql/src/engine-defaults-and-autonumbers.ts index d1c92416c3f..42c2338768a 100644 --- a/packages/objectql/src/engine-defaults-and-autonumbers.ts +++ b/packages/objectql/src/engine-defaults-and-autonumbers.ts @@ -2,24 +2,28 @@ /** * [#29986 codemod] The "Field defaults and autonumbers" section of `ObjectQL`: - * the bodies of `resolveNowDefault`, `resolveOptionDefault`, - * `applyFieldDefaults`, `defaultFieldEntries`, `applyAutonumbers`, - * `adoptExplicitAutonumber`, `isIssuedAutonumberCollision`, - * `warnAutonumberReadersUnserved` moved out of `engine.ts` whole by - * `scripts/codemod-split-engine.mjs`. + * the bodies of `resolveNowDefault`, `normalizeExpressionDefault`, + * `resolveOptionDefault`, `applyFieldDefaults`, `defaultFieldEntries`, + * `applyAutonumbers`, `adoptExplicitAutonumber`, `createWithAutonumberResync`, + * `isIssuedAutonumberCollision`, `warnAutonumberReadersUnserved` moved out of + * `engine.ts` whole by `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link DefaultsAndAutonumbersFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link DefaultsAndAutonumbersHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * * The module imports `engine.ts` for types only, so the two modules form no * evaluation cycle. `index.ts` re-exports nothing from this module. @@ -28,33 +32,81 @@ import { isCurrentUserDefaultToken, isNowDefaultToken, missingFieldValues, parseAutonumberFormat, renderAutonumber, resolveAutonumberFormat } from '@objectstack/spec/data'; import { isUniqueViolationError, uniqueViolationColumn } from '@objectstack/types'; import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { envelopeDriverWriteRefusal } from './not-null-refusal-error.js'; import { ExpressionEngine } from '@objectstack/formula'; import type { EvalPermissions } from '@objectstack/formula'; -import { declaredMultiValued, isExpressionDefault, readStoredAutonumberCounter } from './engine-top-level.js'; -import type { FormulaPlanEntry, IssuedAutonumber } from './engine-top-level.js'; +import { + AUTONUMBER_COLLISION_ATTEMPTS, declaredMultiValued, isExpressionDefault, readStoredAutonumberCounter, + recomputeAutonumberReaders, +} from './engine-top-level.js'; +import type { AutonumberReaders, FormulaPlanEntry, IssuedAutonumber } from './engine-top-level.js'; import type { ObjectQL } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link DefaultsAndAutonumbersFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface DefaultsAndAutonumbersHost { adoptExplicitAutonumber: ObjectQL['adoptExplicitAutonumber']; + applyAutonumbers: ObjectQL['applyAutonumbers']; readonly autonumberCounters: ObjectQL['autonumberCounters']; readonly autonumberReadersUnservedReported: ObjectQL['autonumberReadersUnservedReported']; defaultFieldEntries: ObjectQL['defaultFieldEntries']; getSchema: ObjectQL['getSchema']; + isIssuedAutonumberCollision: ObjectQL['isIssuedAutonumberCollision']; readonly logger: ObjectQL['logger']; normalizeExpressionDefault: ObjectQL['normalizeExpressionDefault']; resolveNowDefault: ObjectQL['resolveNowDefault']; resolveOptionDefault: ObjectQL['resolveOptionDefault']; seedAutonumber: ObjectQL['seedAutonumber']; + validationMessageContext: ObjectQL['validationMessageContext']; } -/** The body of `ObjectQL#resolveNowDefault`; the method's docblock there is its contract. */ -export function resolveNowDefault(fieldType: unknown, instant: Date): string { +/** The bodies of the "Field defaults and autonumbers" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class DefaultsAndAutonumbersFacet { + /** + * Resolve the `NOW()` runtime token into the value the field's declared type + * actually stores (#4597). + * + * The engine — not the driver — owns this resolution, exactly as it owns + * `current_user`. `NOW()` is the mirror of #4560's crack: `current_user` was + * known to the engine and not to the DDL, so the DDL stored the token text; + * `NOW()` was known to the SQL driver and not to the engine, so every + * non-SQL datasource got the token text instead of a time. Resolving here + * makes one answer serve every driver. + * + * The shapes below are NOT invented: they are the canonical storage forms + * ADR-0053 already fixed and `SqlDriver.nowColumnDefault` already emits + * per type. Producing a full instant for a `Field.date` would trade the old + * cross-driver drift for a new one (SQL would keep collapsing it to a + * calendar day in `formatInput`, memory/mongodb would not), so the token + * resolves per declared type: + * + * | field type | stored form | matches | + * |---|---|---| + * | `date` | `YYYY-MM-DD` (UTC day) | `toDateOnly` / the `date` column DEFAULT | + * | `time` | `HH:MM:SS[.fff]` (UTC wall clock, `.000` trimmed) | `canonicalTimeOfDay` | + * | anything else | `YYYY-MM-DDTHH:MM:SS.sssZ` | `canonicalUtcDatetime` | + * + * "Anything else" deliberately includes non-temporal fields: a `text` field + * that opts into `NOW()` gets the instant, which is what the SQL column + * DEFAULT gives it today. + * + * `instant` is the caller's per-insert `now` snapshot when the `NOW()` token + * is what asked, so two defaulted fields on one record cannot straddle a + * millisecond boundary. + * + * It is NOT only the token's, though (#7373): this is the one place that + * knows what shape a declared type stores an instant in, so the CEL branch's + * {@link normalizeExpressionDefault} routes its own `Date` results through + * the SAME table rather than growing a second copy of it. Hence the + * parameter is an arbitrary `instant`, not "now" — `daysFromNow(7)` is a + * week out and takes the identical per-type treatment. + */ + static resolveNowDefault(fieldType: unknown, instant: Date): string { const iso = instant.toISOString(); // YYYY-MM-DDTHH:MM:SS.sssZ if (fieldType === 'date') return iso.slice(0, 10); if (fieldType === 'time') { @@ -67,8 +119,131 @@ export function resolveNowDefault(fieldType: unknown, instant: Date): string { return iso; } -/** The body of `ObjectQL#resolveOptionDefault`; the method's docblock there is its contract. */ -export function resolveOptionDefault(field: { type?: unknown; options?: unknown; multiple?: unknown }): unknown { + /** + * Put a CEL `defaultValue`'s evaluated result into the storage shape the + * field's declared type contracts for (#7373). + * + * The counterpart the expression branch was missing. `applyFieldDefaults` + * has three ways to produce a default and, before this, only two of them + * honoured the stored-value contract: the `NOW()` token routed through + * {@link resolveNowDefault}, a literal is checked against + * `valueSchemaFor(def, 'stored')` at AUTHOR time (#7127), and the expression + * envelope's result was assigned verbatim. But the temporal stdlib returns a + * JS `Date` — ADR-0053 D1 fixes `today()` / `daysFromNow(n)` / `daysAgo(n)` + * as UTC-midnight of the reference-tz calendar day, and `now()` as the raw + * instant — while `valueSchemaFor` says a stored instant is an ISO-8601 + * STRING. So `{ dialect: 'cel', source: 'daysFromNow(7)' }` on a `datetime` + * put a `Date` object in the column: a value the platform's own + * `os migrate value-shapes` scan reports as a violation. + * + * Why NORMALIZE rather than refuse the `Date` — the two shapes this could + * have taken, decided on measurement: + * + * - **The drivers already agree with this answer.** Handed a `Date`, + * `SqlDriver.formatInput` coerces it through `canonicalUtcDatetime` + * (`toISOString()`) and `toDateOnly` (`YYYY-MM-DD`); mongodb's + * `storageDatetimeValue` keeps the BSON `Date`, which is what parsing the + * ISO string produces anyway, and its `storageDateValue` collapses to the + * same `YYYY-MM-DD`. Normalizing here is therefore byte-identical on + * every SQL and mongodb-backed store and changes exactly one thing: the + * memory driver, which applies its temporal canon to filter comparands + * only (`coerceTemporalValue`) and stores writes as handed. That is the + * whole defect, and this is the smallest change that closes it. + * - **Refusal would single out one writer.** `validateRecord` accepts a + * `Date` on `date`/`datetime` from ANY caller by explicit decision + * (`if (value instanceof Date) return null`), and temporal types are not + * in ADR-0104's strict value-shape block at all. Refusing a `Date` only + * when a CEL default produced it would make the rule depend on who wrote + * the value rather than on what the value is — and would break the + * documented envelope (#7244) on precisely the SQL backends where it + * stores correctly today. + * + * This is the `NOW()` crack of #4597 / #4560 in its third form: a value the + * SQL driver silently repaired at the wire and non-SQL datasources did not, + * so one declaration stored two shapes depending on the datasource. Resolved + * engine-side, one answer serves every driver; the driver coercions stay as + * defence in depth for writes that bypass the engine. + * + * No day can shift here. ADR-0053 D1's `Date` is UTC-midnight OF the + * reference-tz calendar day, and {@link resolveNowDefault} reads it back + * with `toISOString()` — UTC getters, the same `getUTC*` reading the ADR + * names for the driver filter path. Reading those parts in LOCAL time is the + * move that would shift a day, and nothing here does it. + * + * Non-`Date` results (a string, a number, a bool, a list) pass through + * untouched: this normalizes temporal FORM, it does not police the contract, + * and a CEL default is documented as "result type is a runtime concern". + * An `Invalid Date` passes through too — the same totality the driver + * canons keep, so a value nothing can interpret is never silently rewritten + * into a wrong one. + */ + static normalizeExpressionDefault(this: Pick, fieldType: unknown, value: unknown): unknown { + if (!(value instanceof Date) || Number.isNaN(value.getTime())) return value; + return this.resolveNowDefault(fieldType, value); + } + + /** + * The initial value a field's OPTION LIST declares — the option marked + * `default: true` — or `undefined` when it declares none (#7246). + * + * `SelectOption.default` has been authorable and spec-valid since the schema + * was written, and until this method nothing on the insert path read it: a + * create that omitted the field stored `null`, not the marked option. The one + * consumer anywhere was lint's `isNullableField`, which concluded the column + * was always-valued — a **build-breaking** verdict resting on a declaration + * the engine did not honour. Enforcing it here is what makes that heuristic + * true rather than merely asserted. + * + * Deliberate decisions, each of which could reasonably have gone the other + * way: + * + * - **Precedence: `defaultValue` wins.** Not decided here — decided by the + * caller, which only reaches this method when `defaultValue == null`. The + * field-level key is the more specific declaration (it names a value for + * THIS field; the option flag describes the shared option list), and it is + * the spelling every other consumer already honours. When both are + * declared and disagree, the option flag is inert exactly as it is today. + * + * - **Type-agnostic, like the lint heuristic.** `options` is a field-level + * key on `FieldSchema`, not gated on `type: 'select'`, and + * `isNullableField` reads it without a type test. Matching that keeps the + * two sides honest BY CONSTRUCTION: there is no field the lint calls + * always-valued that this method leaves empty. + * + * - **A MULTI-VALUED field assembles an ARRAY.** That field stores an + * Array/JSON, so the shape of its default follows the field, not the + * number of marked options — one marked option on a multi-select defaults + * to a one-element array, never a bare scalar that the driver would then + * store with the wrong shape. [#18408] Multi-valued is + * {@link declaredMultiValued}, ⛔ not a raw `field.multiple`: the flag is + * inert on a type outside the two multi sets and redundant on a + * `multiselect` / `checkboxes` / `tags`, which stores an array with or + * without it — so reading the flag made this method disagree with the + * validator one frame later (`normalizeMultiValueFields` has asked the + * predicate all along) and with the column the driver builds. + * + * Refusing (throwing) was rejected: the metadata is + * spec-valid, and a runtime throw on spec-valid input is a worse answer + * than a well-defined value. Ignoring it was rejected too — it would + * preserve, for multi-selects only, precisely the inertness this change + * removes, and an author cannot see that carve-out from the schema. + * + * - **Several options marked on a SINGLE-valued field: the FIRST wins.** + * There is one slot, so declaration order decides — deterministic, and + * the same option a picker preselects when it takes the first match. + * Neither the `Field.select` builder nor the schema dedupes the flag, so + * this case is reachable; the alternative (throw) again fails a + * spec-valid declaration at runtime. Nothing in the shipped corpus marks + * more than one. + * + * - **Only the canonical `default` spelling is read.** The authoring + * aliases (`isDefault`, `selected`) are normalized by + * `SelectOptionSchema`'s alias layer before metadata reaches the engine, + * and lint reads the canonical key only. Reading the aliases here would + * make the engine honour raw shapes lint calls nullable — the two would + * disagree in the direction that produces a false "always-valued". + */ + static resolveOptionDefault(field: { type?: unknown; options?: unknown; multiple?: unknown }): unknown { const options = field.options; if (!Array.isArray(options)) return undefined; const marked: unknown[] = []; @@ -87,8 +262,44 @@ export function resolveOptionDefault(field: { type?: unknown; options?: unknown; return declaredMultiValued(field) ? marked : marked[0]; } -/** The body of `ObjectQL#applyFieldDefaults`; the method's docblock there is its contract. */ -export function applyFieldDefaults(this: Pick, object: string, record: Record, execCtx?: ExecutionContext, nowSnapshot?: Date, permissions?: EvalPermissions): Record { + /** + * Apply field defaults to an incoming insert payload. Defaults that are + * Expression envelopes (e.g. `{ dialect: 'cel', source: 'today()' }`, + * `{ dialect: 'cel', source: 'os.user.id' }`) are evaluated via + * `ExpressionEngine` against the calling user/org/now snapshot. The + * `defaultValue` runtime TOKENS (`@objectstack/spec/data`'s + * `DEFAULT_VALUE_TOKENS` — `current_user` and `NOW()`, the whole family) are + * resolved here, so one declaration behaves identically on every driver. + * Static defaults are applied verbatim. Records that already supplied a value + * for a field are left untouched. + * + * "Supplied a value" means the field is present with a non-null value. Both an + * OMITTED field (`undefined`) and an EXPLICIT `null` are treated as "not + * supplied" and get the default. This runs on the INSERT path only, where a + * `null` from a form (an unpicked control serializes to `null`, not omission) + * unambiguously means "no value"; the UPDATE path never calls this, so a + * deliberate "set to null" on update is preserved (#2706). Empty string `''` + * is a real, user-entered value and is left as-is. + * + * Implements ROADMAP §M9.9b — `defaultValue` accepts Expression so authors + * can replace "write a hook to default to today/current-user" with a + * declarative `defaultValue: cel\`today()\``. + * + * A field that declares NO `defaultValue` falls back to the option marked + * `default: true` ({@link resolveOptionDefault}, #7246) — the select idiom, + * which until then was authorable, spec-valid, and read by nothing on this + * path. + * + * [#20082] `permissions` is the acting subject's effective object-permission + * map, which an expression default calling `current_user.can(object, verb)` + * answers from. This method is synchronous, so the CALLER resolves it — once + * per write, only for a row that will be defaulted by such an expression + * ({@link resolveDefaultPermissions}, which also owns what a failed resolution + * does: it refuses the write before this runs). `undefined` is "no permission + * data": a `can` default then does not evaluate and takes this method's rule + * for every expression default that does not — left unset, with the warn below. + */ + static applyFieldDefaults(this: Pick, object: string, record: Record, execCtx?: ExecutionContext, nowSnapshot?: Date, permissions?: EvalPermissions): Record { const fieldEntries = this.defaultFieldEntries(object); if (!fieldEntries) return record; const out = { ...record }; @@ -187,8 +398,14 @@ export function applyFieldDefaults(this: Pick, object: string): Array<{ + /** + * The field entries {@link applyFieldDefaults} walks — one reading of an + * object's `fields`, shared with {@link resolveDefaultPermissions} so the + * planner and the evaluator see the same defaults. `undefined` when the object + * declares no fields, which is `applyFieldDefaults`' early return (it hands + * the SAME record reference back, and the insert path relies on knowing that). + */ + static defaultFieldEntries(this: Pick, object: string): Array<{ name: string; type?: unknown; defaultValue?: unknown; options?: unknown; multiple?: unknown; }> | undefined { const fieldsRaw = (this.getSchema(object) as any)?.fields; @@ -199,8 +416,100 @@ export function defaultFieldEntries(this: Pick ({ name, ...(def as object) })); } -/** The body of `ObjectQL#applyAutonumbers`; the method's docblock there is its contract. */ -export async function applyAutonumbers(this: Pick, object: string, record: Record, execCtx?: ExecutionContext, driverOwnsAutonumber?: boolean): Promise { + /** + * Generate values for empty `autonumber` fields on insert — ONLY for drivers + * that do not generate them natively (memory, mongodb). For SQL-backed objects + * the driver owns a persistent, atomic `_objectstack_sequences` table and + * advertises `supports.autonumber === true`; the engine then defers entirely + * and never pre-fills (so the persistent sequence is the single source of + * truth — see #1603). Required-validation exempts `autonumber` either way, so + * a `required` record number is never rejected for "missing" — the runtime + * owns the value, not the client. + * + * That ownership is now ENFORCED rather than merely asserted (#5503): the + * insert path strips a caller-supplied value before this runs, so the "respect + * explicit value" skip below is reached only by writers the strip exempts — + * `isSystem` (seed replay, migration) and an opt-in historical import + * (`preserveAudit`), plus values a `beforeInsert` hook computed server-side. + * For every other caller the slot arrives empty and the sequence issues the + * number. Do NOT "fix" a forged value here by overwriting it: the strip must + * stay upstream of this method, because a driver with + * `supports.autonumber === true` returns above without ever entering the loop. + * + * In the fallback path the next value is `max(existing) + 1`, seeded once per + * `object.field.` from the store then incremented in memory (monotonic + * within the process, resilient to deletions). The shared `autonumberFormat` + * renderer is honored end-to-end, so date tokens (`AD{YYYYMMDD}{0000}`), field + * interpolation (`{island_zone}{000}`) and per-scope reset behave identically + * to the SQL driver's persistent sequence (#1603). NOTE: this in-memory seeding + * is single-instance. + * + * # A format-LESS field renders through the contract default, not bare (#6555) + * + * "Identically to the SQL driver" was true of every DECLARED format and false + * of the one case nobody declares: with no format at all this method used to + * hand `parseAutonumberFormat` the empty string, whose empty token list + * `renderAutonumber` renders through its no-slot branch as a BARE counter — + * `1`, `2`, …. The SQL driver substituted `'{0000}'` in the same case and + * issued `0001`, `0002`, …. One metadata document, two number shapes, decided + * by which driver happened to serve it; the counter VALUE always agreed + * (#6468 pinned that), so what forked was rendering width alone. + * + * The maintainer ruling of 2026-08-08 (#6555, route 3) puts the default in the + * contract instead of in either fallback, and this method now reads it: + * {@link resolveAutonumberFormat} answers the canonical `autonumberFormat`, + * then the `format` shorthand (#1603), then `DEFAULT_AUTONUMBER_FORMAT` + * (`{0000}`). Two consequences, both deliberate: + * + * - A format-less field on this path now issues `0001` where it issued `1`. + * `{0000}` was chosen because it is the shape SQL deployments already + * store, so the flip lands on engine-fallback deployments only. + * - "Declared" is now a NON-EMPTY string (the SQL driver's long-standing + * truthiness rule) rather than the `??` this method used, so + * `autonumberFormat: ''` resolves to the default too instead of rendering + * bare — and an empty canonical key no longer masks a declared `format` + * shorthand. The bare counter is still reachable, by declaring a format + * with no `{0..0}` slot (`'PRE-'` → `PRE-1`). + * + * Seeding is untouched by all of this: `{0000}` renders prefix `''` and suffix + * `''`, so {@link readStoredAutonumberCounter} stays on its UNANCHORED legacy + * branch and reads already-stored bare values exactly as before. + * + * # Keeping the seeded counter in sync (#6806) + * + * Seeding "once per counter key" is only the truth while the engine is the + * ONLY writer of the field. It is not: the `continue` below is reached by + * every exempt writer (`isSystem` seed replay, a `preserveAudit` historical + * import, a `beforeInsert` hook stamp), and each of those persists a record + * number the counter never saw. The counter then keeps issuing from where the + * one-time seed left it — below the store's real max — and every number it + * issues up to that max is a duplicate business identifier. Two resyncs + * close that, from opposite ends: + * + * - **Adopt, here.** An exempt value passes through this very loop, so the + * counter can be lifted from it for FREE — one string parse, no query. + * {@link adoptExplicitAutonumber} does it, which makes the warm in-memory + * counter converge on what a cold re-seed of the same store would answer. + * This is the whole fix for in-process drift, and it costs nothing on the + * generating path (a caller-supplied value never reaches this method + * unless the writer is exempt). + * - **Re-seed on collision**, at the write. Adoption cannot see a writer + * outside this process (another instance, a direct driver write, a + * restore). Those surface as a unique-constraint failure on the create, + * and {@link createWithAutonumberResync} answers it by dropping the stale + * counter, re-seeding from the store and re-issuing — bounded. It costs + * nothing until a collision actually happens. + * + * Both are required and neither subsumes the other: adoption covers drift the + * engine can observe but the store cannot report, collision-resync covers + * drift the store reports but the engine could not observe. + * + * @returns the autonumbers this call ISSUED, in field order — the input + * {@link createWithAutonumberResync} needs to decide whether a unique + * violation is one of its own numbers. An adopted (caller-supplied) value is + * deliberately NOT listed: it is not the engine's to re-issue. + */ + static async applyAutonumbers(this: Pick, object: string, record: Record, execCtx?: ExecutionContext, driverOwnsAutonumber?: boolean): Promise { if (driverOwnsAutonumber) return []; // driver generates persistently in create() const fields = (this.getSchema(object) as any)?.fields; if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return []; @@ -252,8 +561,45 @@ export async function applyAutonumbers(this: Pick, object: string, field: string, tokens: ReturnType, record: Record, value: string, now: Date, timezone?: string): void { + /** + * Lift the in-memory counter to a record number an EXEMPT writer supplied + * (#6806) — the free half of the resync, and the one that closes the shape + * #5495's PROBE1 measured on a warm database. + * + * The value is parsed with {@link readStoredAutonumberCounter}, i.e. by + * exactly the anchoring rules #6468 gave the seeding scan, against the + * prefix/suffix this record's own format renders. So adopting is the same reading a cold re-seed + * would perform over the same row — which is the invariant to hold on to: a + * warm counter must answer what a restart would answer. + * + * Four deliberate refusals, each one a way this could do harm: + * + * - **Never throws.** An exempt write is not the engine's to reject, and it + * was not rejected before this method existed. A format whose `{field}` + * interpolation is empty, or a value that parses to nothing, simply + * teaches the counter nothing. + * - **Only LIFTS a counter that is already seeded.** With no seed in hand, + * writing this value in would SKIP the seeding scan and answer from one + * row — below the real max whenever the store holds a higher number, i.e. + * the duplicate-number defect itself. Doing nothing is correct: the row is + * persisted, so the first generating insert's own scan reads it. + * - **Never lowers.** A counter that has already issued numbers must not go + * back over them; the max is a floor that only rises. + * - **Only within this record's scope.** The reading returns `undefined` + * for a value that does not carry the rendered prefix, so a + * historical import into last month's date scope cannot lift THIS + * month's counter. Its own scope's counter is left untouched, which is + * harmless: a scope is derived from the write instant, so a past scope's + * counter is not one a subsequent generating insert can reach. + * + * Adoption runs BEFORE the driver write, so an exempt insert that then fails + * leaves the counter lifted over a number nobody took — a gap. Gaps are + * already the documented cost of this path (a failed attempt consumes its + * value; the counter is "resilient to deletions"), and the direction is the + * safe one: a gap is a cosmetic surprise, a duplicate is a corrupted business + * identifier. + */ + static adoptExplicitAutonumber(this: Pick, object: string, field: string, tokens: ReturnType, record: Record, value: string, now: Date, timezone?: string): void { let probe: ReturnType; try { // An empty `{field}` would render an empty prefix and point at the wrong @@ -275,8 +621,206 @@ export function adoptExplicitAutonumber(this: Pick, driver: any, object: string, row: Record, driverOptions: any, issued: IssuedAutonumber[], execCtx: ExecutionContext | undefined, driverOwnsAutonumber: boolean, driverIssued: string[] = [], readers?: AutonumberReaders): Promise { + // [#25073] Who issued this row's numbers decides who judges a collision on + // them: the driver, through the member that issued them, or this engine. + const byDriver = driverIssued.length > 0; + let attempt = 1; + for (;;) { + try { + return await driver.create(object, row, driverOptions); + } catch (error) { + // [#14095] Not OUR collision to re-issue — so this is where a driver's + // refusal leaves the single-row door, and where it stops being the + // driver's error. `envelopeUniqueViolation` returns everything that is + // not a unique violation untouched (a deadlock, a NOT NULL, an + // unreachable store), so only the recognised conflict changes shape. + // [#24768] …and a NOT NULL refusal on a declared field becomes the + // validator's own `ValidationError` (`envelopeDriverWriteRefusal`). + // [#25073] On a driver-issued number the member answers instead, and + // in answering re-seeds the counter it proved stale. + const collided: string[] = byDriver + ? ((await driver.issueAutonumbers(object, [row], driverOptions, error)) as string[][] | undefined)?.[0] ?? [] + : this.isIssuedAutonumberCollision(error, issued) ? issued.map((one) => one.field) : []; + if (collided.length === 0) { + throw envelopeDriverWriteRefusal(error, object, this.getSchema(object), 'insert', this.validationMessageContext(object, execCtx)); + } + // Whatever happens next, the stale counter must not survive this call: + // leaving it in place is what turned one collision into a storm. + for (const one of issued) this.autonumberCounters.delete(one.counterKey); + if (attempt >= AUTONUMBER_COLLISION_ATTEMPTS) { + const fields = collided.join(', '); + throw Object.assign( + new Error( + `Autonumber collision on '${object}' field(s) [${fields}]: the record number was ` + + `re-seeded from the store and re-issued ${AUTONUMBER_COLLISION_ATTEMPTS} times and the ` + + `driver rejected each one as a duplicate. No record was written.`, + ), + { code: 'ERR_AUTONUMBER_COLLISION', cause: error }, + ); + } + attempt += 1; + this.logger.warn('Autonumber collided — re-seeding the counter and re-issuing', { + object, fields: collided, attempt, + }); + let renumbered: boolean; + if (byDriver) { + // [#25073] The member emptied the slots it proved stale; issue them again. + renumbered = (((await driver.issueAutonumbers(object, [row], driverOptions)) as string[][] | undefined)?.[0] ?? []).length > 0; + } else { + // Clear the slots so `applyAutonumbers` treats them as empty again — + // leaving the burned value in place would read as an exempt writer's + // and be adopted rather than re-issued. + for (const one of issued) delete row[one.field]; + issued = await this.applyAutonumbers(object, row, execCtx, driverOwnsAutonumber); + renumbered = issued.length > 0; + } + // [#25073] A new number is a new input to every stored field that reads it. + await recomputeAutonumberReaders(readers, [row]); + // Nothing left to re-issue (the field vanished from the schema + // mid-flight) — the next failure is the caller's to see, enveloped on + // the same terms as the exit above (#14095): this call is OUTSIDE the + // loop's own `try`, so its rejection is a second driver-error exit and + // not a path the branch above covers. + if (!renumbered) { + try { + return await driver.create(object, row, driverOptions); + } catch (retryError) { + throw envelopeDriverWriteRefusal(retryError, object, this.getSchema(object), 'insert', this.validationMessageContext(object, execCtx)); + } + } + } + } + } + + /** + * Whether `error` is a unique violation attributable to one of the + * autonumbers this insert issued (#6806). See + * {@link createWithAutonumberResync} for why an unnamed column counts as + * attributable and a differently-named one does not. + */ + static isIssuedAutonumberCollision(error: unknown, issued: IssuedAutonumber[]): boolean { if (issued.length === 0) return false; if (!isUniqueViolationError(error)) return false; const column = uniqueViolationColumn(error); @@ -284,8 +828,16 @@ export function isIssuedAutonumberCollision(error: unknown, issued: IssuedAutonu return issued.some((one) => one.field === column); } -/** The body of `ObjectQL#warnAutonumberReadersUnserved`; the method's docblock there is its contract. */ -export function warnAutonumberReadersUnserved(this: Pick, object: string, plan: readonly FormulaPlanEntry[], driver: unknown): void { + /** + * [#25073] A driver that owns the record-number counter + * (`supports.autonumber`) but cannot issue a number ahead of the write + * (no `IDataDriver.issueAutonumbers`) still fills it inside `create()`, so a + * stored field whose `expression` reads it is stored as computed WITHOUT + * it, and every insert looks successful. That is a stored value that is + * not what its declaration says (AGENTS.md, degradation log levels: `error`), + * said once per object per engine instance, with the remedy. + */ + static warnAutonumberReadersUnserved(this: Pick, object: string, plan: readonly FormulaPlanEntry[], driver: unknown): void { if (this.autonumberReadersUnservedReported.has(object)) return; this.autonumberReadersUnservedReported.add(object); const fields = plan.map((entry) => entry.name); @@ -300,3 +852,4 @@ export function warnAutonumberReadersUnserved(this: Pick, object: string, query: EngineAggregateOptions, options?: EngineReadOptions): Promise { +/** The bodies of the "Delete, count, aggregate and execute" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class DeleteAndAggregateFacet { + /** The body of `ObjectQL#assertNoInternalFieldEvaluated`; the method's docblock there is its contract. */ + static assertNoInternalFieldEvaluated(this: Pick, object: string, query: InternalFieldEvaluationQuery, options?: InternalFieldEvaluationOptions): void { + assertNoInternalFieldEvaluated(object, query, this.relatedSchemaOf, options); + } + + /** The body of `ObjectQL#aggregate`; the method's docblock there is its contract. */ + static async aggregate(this: Pick, object: string, query: EngineAggregateOptions, options?: EngineReadOptions): Promise { object = this.resolveObjectName(object); // Fold the `filter` alias into `where` (#4346) — the AST below reads // `query.where` only, so an unfolded `{ filter }` aggregated every row. @@ -555,3 +568,4 @@ export async function aggregate(this: Pick, ast: QueryAST, schema: ServiceObject | undefined, context: ExecutionContext | undefined): Promise { +/** The bodies of the "Query lowering, search, find and findOne" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class FindFacet { + /** + * [#20802] Stage 2 of `where` admission on every verb, whole: resolve the + * placeholders, then LOWER EACH NESTED-RELATION CONDITION + * ({@link lowerRelationConditions}), then run the shared lowering + * ({@link resolveThenLowerWhere}'s second half, `lowerFilterCondition`). + * + * The order is ADR-0053 D-D1 item 3's, extended by one step: tokens first, + * so a condition on the related object carries the resolved values (one + * instant for the whole filter) into the related read; the relation step + * before the shared lowering, so that lowering reads the `$in` / `$contains` + * the relation step produced — including the NULL-safe `$not` over it — and + * never a nested object whose columns belong to another object. + */ + static async resolveRelateThenLowerWhere(this: Pick, object: string, operation: string, where: W, execCtx: ExecutionContext | undefined, lowering: FilterLoweringOptions): Promise { + const resolved = resolveWhereFilterTokens( + where, execCtx, whereResolvedTokenJudge(object, operation, this._registry.getObject(object)), + ); + const related = await this.lowerRelationConditions(object, operation, resolved, execCtx); + return lowerFilterCondition(related, lowering); + } + + /** + * The write-path counterpart of {@link resolveWhereTokens}: return `options` + * with `where` placeholders expanded (#3810). + * + * Returns the SAME object when nothing resolved — `resolveFilterTokens` + * returns its input by reference on a placeholder-free tree, so the common + * path allocates nothing. When something does resolve, a shallow copy is + * made rather than assigning through: `options` belongs to the caller, and + * writing back would bake one request's user id into a filter object the + * caller may reuse (view metadata and flow node config both get reused). + * + * [ADR-0053 D-D1, amended — #5930] …and lowered, in the same stage + * ({@link resolveThenLowerWhere}), with the object's declared-type reader. + * The lowering is copy-on-write too, so a `where` it rewrites lands on the + * copy, never on the caller's object. + */ + static async withResolvedWhere(this: Pick, object: string, operation: string, options: T, lowering: FilterLoweringOptions): Promise { + if (!options || options.where == null) return options; + // [#20157] Through the stage function the judge also calls. [#20802] …with + // the nested-relation step between resolution and the shared lowering. + const resolved = await this.resolveRelateThenLowerWhere(object, operation, options.where, options.context, lowering); + return resolved === options.where ? options : ({ ...options, where: resolved } as T); + } + + /** + * ADR-0061: expand `search` into a server-resolved cross-field `$or` of + * `$icontains`, AND it with any caller `where`, then strip the search keys off + * the AST. + * + * Shared by `find` and `findOne` (#4419). It lived inline in `find` and + * nowhere else, while `ENGINE_FIND_OPTION_KEYS` — the one legal-key set BOTH + * methods are checked against (see {@link ENGINE_OPTION_KEY_SETS}) — declares + * `search`/`searchFields` for both. So `findOne({ search })` passed the gate, + * rode onto the AST verbatim, and reached a driver: no driver reads + * `ast.search` (the expansion is the engine's job by ADR-0061), so the + * predicate vanished and the forced `limit: 1` turned it into the first row of + * the WHOLE object — a real, plausible-looking record unrelated to the search. + * That is #4419's reported failure exactly, under a different key than the + * `filter` #4346 closed; one expander, called from both, is what stops the + * pair drifting again. + * + * Field resolution is server-side (declared `searchableFields` → + * auto-default); the optional `searchFields` override is intersected with the + * allowed set, never widened. All drivers already execute `$or`/`$icontains`, + * so this needs no driver changes. + * + * [#29394] A resolved field whose declared type can never hold a string (the + * #15661 door's verdict) gets no clause — `expandSearchToFilter` skips it — + * because that door judges only the caller's `where`, above, and never this + * expansion. The set itself is not narrowed, so the refusals below judge it + * as before, and a set of only such fields answers no rows. + * + * The keys are deleted whether or not anything expanded — leaving them on + * would hand the driver a key it does not read, which is the same silent drop + * one layer down. + */ + static async expandSearchOnAst(this: Pick, ast: QueryAST, schema: ServiceObject | undefined, context: ExecutionContext | undefined): Promise { // The `$search`/`$searchFields` OData spellings are NOT read here: the // protocol layer normalizes them to the bare keys before the engine // (protocol.ts findData), and a direct engine call carrying one is an @@ -146,8 +236,132 @@ export async function expandSearchOnAst(this: Pick(this: Pick, object: string, query: T, options: EngineReadOptions | undefined): Promise { + if (query.search === undefined && query.searchFields === undefined) return query; + const { search, searchFields, ...rest } = query; + const carrier = { object, where: rest.where, search, searchFields } as QueryAST; + await this.expandSearchOnAst(carrier, this._registry.getObject(object), mergeReadContext(query.context, options?.context)); + if (carrier.where === undefined) return rest as T; + return { ...rest, where: carrier.where } as T; + } + + /** + * [commit 74155c735] Fill the author-state defaults the query schemas declare, so the + * AST handed to middlewares, hooks and drivers is the PARSED state + * `QueryAST` (a `z.infer` type) promises. + * + * ADR-0122 made `EngineQueryOptions` the author state (`z.input`): a key + * with a declared `.default()` is optional to write. `find`/`findOne` kept + * demanding the parsed state anyway (commit 53068c130 pinned them back) because the + * engine built its AST by bare spread and filled no default — `order: + * undefined` would have ridden straight to the driver. This is the filling. + * Each defaulting node is run through ITS OWN schema rather than + * hand-assigning values, so a default declared in `packages/spec` stays the + * single source of truth: + * + * - `orderBy[]` nodes through `SortNodeSchema` — fills `order: 'asc'`, the + * query path's one declared default. Measured before the flip: every + * driver already coalesces a missing `order` to `'asc'` (`sql-driver.ts` + * `s.order || 'asc'`, `memory-driver.ts`, `mongodb-driver.ts`, + * `remote-transport.ts` likewise), so the filled value changes no query's + * answer — it makes the AST say what the drivers were already assuming. + * Parsing also applies the node's declared strictness to type-BYPASSING + * callers: an unknown sort key, or the retired `direction` spelling, is + * now refused with the schema's own prescription instead of silently + * dropped-or-honored per driver (#4721's defect class) — the same refusal + * `normalizeSortNodes` already makes on the wire path. + * - `expand` values recurse: a nested query is the same authoring surface. + * No driver reads `ast.expand` (the engine expands post-fetch), and the + * nested read that executes re-enters `find()` — which fills again — so + * the recursion keeps the AST's type honest without a cast. + * + * `search` is deliberately NOT parsed, though `FullTextSearchSchema` carries + * three flag defaults (`fuzzy`/`operator`/`highlight`). Two measurements + * decide it. First, nothing can ever read them off the AST: no executor + * reads the flags at all (#4286 — the ADR-0061 expansion reads only `query` + * + `fields`), and {@link expandSearchOnAst} deletes `search` from the AST + * before middlewares, hooks or the driver see it, so a filled value would be + * constructed and then discarded unread. Second, parsing would REFUSE input + * the engine deliberately accepts: the wire path hands this method + * `search.fields` in the comma-STRING shape (and the `q` spelling) that + * `resolveSearchFields`/`normalizeSearch` tolerate by design — pinned in + * `query-expression-conformance.test.ts` — while the schema declares + * `fields: string[]`. The type-level gap this leaves (author-state `search` + * inside a `QueryAST`-typed value, until the key is deleted a few lines + * later) is covered by the same single cast as `expand`, below. + * + * `where`/`fields`/`limit`/`offset`/`top` carry no `.default()` or + * `.transform()` (pinned in `filter.zod.ts`'s own docs) and are not parsed — + * the cost is one small-object parse per authored sort node / search + * config, only when the key is present. + */ + static fillQueryAstDefaults>(this: Pick, query: T): T & Pick & { expand?: Record } { + const out: Record = { ...query }; + if (Array.isArray(out.orderBy)) { + out.orderBy = out.orderBy.map((node) => SortNodeSchema.parse(node)); + } + if (out.expand != null && typeof out.expand === 'object') { + const expand: Record = {}; + for (const [field, nested] of Object.entries(out.expand)) { + expand[field] = this.fillQueryAstDefaults(nested as QueryInput); + } + out.expand = expand; + } + // The one cast in the flip: `orderBy`/`expand` are rebuilt above; `search` + // is claimed-but-not-parsed, per the doc — deleted from the AST unread. + return out as T & Pick & { expand?: Record }; + } + + /** + * Refuse a `findOne` that selects nothing in particular (#4419). + * + * The AST reaching here is the CALLER's own intent: aliases folded, unknown + * keys refused, `search` expanded — but the security/sharing middlewares have + * not run yet, and that ordering is the point. An injected RLS predicate + * narrows *which* rows are visible; it does not make "whichever of them comes + * first" a thing the caller asked for. Judging the post-middleware AST would + * pass every query on a scoped object and leave the hole open where it is + * most expensive. + * + * "Selects nothing" is read the same way #3896 read an empty sharing + * criteria: absent, `null`, or `{}` — the three shapes that mean "match every + * row". This guard closes the one case that is unambiguously + * match-everything, not everything it cannot prove. + * + * [#5158] This comment used to add: "a `where` that is not a plain object (an + * expression tree) is the DRIVER'S to interpret, and counts as a predicate." + * That sentence was the engine's blessing of a second filter dialect, and it + * cost exactly what a blessing costs — `findOne({ where: [] })` counted as a + * predicate, walked past this guard, and returned an ARBITRARY row: the #4419 + * defect surviving inside #4419's own guard. `FilterArray` is now lowered by + * {@link lowerWhereFilterArray} at every entry point, so by the time this + * runs `where` is a `FilterCondition` or nothing. The `Array.isArray` arm + * below is kept as defence in depth for a subclass or a future caller that + * reaches this method without lowering — it is no longer a contract. + * + * `orderBy` is the other way to be specific, and a legitimate one — "the + * newest", "the highest priority". It is honored on this path by every + * driver, so it is a real answer and not a second silent drop. + */ + static requireFindOnePredicate(object: string, ast: QueryAST): void { const where = ast.where as unknown; const hasPredicate = where != null && @@ -164,8 +378,88 @@ export function requireFindOnePredicate(object: string, ast: QueryAST): void { ); } -/** The body of `ObjectQL#reportFindFailure`; the method's docblock there is its contract. */ -export function reportFindFailure(this: Pick, object: string, error: unknown): void { + /** + * Report a failed {@link find} at the level its CAUSE earns — then leave the + * throw exactly where it was. + * + * ## The two facts this frame used to merge + * + * "The table has not been created yet" and "the read failed" are different + * facts, and until commit 3a86a65e7 this line reported both at `error`, with a stack. + * The first one is the ordinary state of a database nobody has migrated yet, + * and every caller on that path already treats it as a normal answer and says + * so in its own code: {@link readMigrationFlagVerified} ("an unreadable table + * … → false"), `ObjectQLPlugin.readAuthoredHookRows` / + * `readAuthoredActionRows` (report `authoredRows: 0` and carry on), + * `readAuthoredTranslationLayer`, and `ObjectStoreActionActivationStore.probe` + * (whose caller already follows this frame with a `warn` stating the + * consequence in operator terms). + * + * Measured, on `os migrate plan --database-url file:`: five + * ERROR records with full stack traces, out of a command that EXITS 0 and + * prints a correct plan. Four stack traces are the first thing an operator + * sees when planning a database whose whole point is that it is not migrated + * yet — and an `error` channel that fires on a routine state is the + * over-application `sql-driver.ts` already names on a nearby line: it trains + * everyone to skim `error`. + * + * ## ⛔ What this is NOT is a broadened catch + * + * * The throw is UNCHANGED on both branches. No caller's control flow, no + * caller's `catch`, and no error envelope moves — this method decides a + * log level and nothing else. + * * Only the class that positively identifies as "relation does not exist" + * is demoted, asked through the shared {@link isMissingTableError} + * predicate (`@objectstack/metadata/errors`, #4825) — the same call + * {@link probeInstallOrganizations} and {@link resolveFileReferences} + * make, never a hand-rolled `code === '42P01'` copy. That predicate earns + * a benign verdict rather than defaulting to one: a connection drop, a + * timeout, a permission denial, a query fault — and, through its + * `excludes`, Postgres' `column "x" of relation "y" does not exist`, + * which contains a legal missing-table phrase but is a column fault on a + * table that EXISTS — all stay on the loud branch, with the stack (at + * `warn`; the last section says why not `error`). + * * The fault stays visible without this frame: the driver's own refusal + * envelope (`SqlDriver.backendStatementFault` → `logger.warn`) carries + * the table, the dialect reason and the compiled statement, and is + * untouched here. That is deliberately the surviving loud half, and it is + * the half `runtime/src/expected-read-refusal-noise.ts` already documents + * as "where this class of fault can still be picked up". + * + * `debug` rather than `warn` for exactly that reason: the driver already puts + * one warn-level line per refusal in front of a reader, so a second would + * move this noise rather than remove it. What the demotion drops is the + * duplicate and its stack; the classification survives in the meta. + * + * The `debug` demotion is READS only: a write to a table that does not exist + * is not a normal answer for any caller, and takes the write doors' `warn` + * like every other write fault does (#17052). + * + * ## The loud branch is `warn`, not `error` — the caller is told + * + * [#17212] Every cause the predicate does NOT earn a benign verdict for is + * reported at `warn`, with its message and stack, and `find`'s `catch` then + * rethrows it — `throw e` is that catch's only exit. That is AGENTS.md's + * third legal answer (*Degradation log levels*): "a failure handed to the + * CALLER is not a degradation at all … Do not bolt a `logger.error` onto + * such a site". The requester IS told — the throw is the answer — so an + * `error` line is a second, louder report of a fact the caller already has. + * The write doors moved for the same reason (#17052): their catches rethrow + * too. + * + * ⛔ Demoted, not deleted — only the level moved. The message and the stack + * survive through {@link writeFailureLogMeta}: `warn(message, meta?)` has no + * `Error` slot, and an Error handed over AS meta serializes to `{}` (its + * `message`/`stack` are non-enumerable). On the SQL read path the fault is + * also reported one frame down on the driver's own `warn` — + * `SqlDriver.backendStatementFault` for every backend fault it measured + * (connection, timeout and ACL included), `unresolvableFilterColumnRefusal` + * for an unresolvable WHERE column. The filter COMPILER's refusals + * (`uncompilableFieldReferenceError` and its siblings) have no driver line: + * they are raised before the statement runs, as refusals carrying a declared + * `400` — a rejected request, answered to its caller. + */ + static reportFindFailure(this: Pick, object: string, error: unknown): void { if (isMissingTableError(error, object)) { this.logger.debug('Find operation failed', { object, @@ -178,3 +472,4 @@ export function reportFindFailure(this: Pick, object: string // is told — see the docblock's last section. this.logger.warn('Find operation failed', writeFailureLogMeta(error, { object })); } +} diff --git a/packages/objectql/src/engine-hooks-and-functions.ts b/packages/objectql/src/engine-hooks-and-functions.ts index 8ba54862892..39941baf113 100644 --- a/packages/objectql/src/engine-hooks-and-functions.ts +++ b/packages/objectql/src/engine-hooks-and-functions.ts @@ -3,47 +3,81 @@ /** * [#29986 codemod] The * "Class head: units, hooks, functions, body runners, hook dispatch" section of - * `ObjectQL`: the bodies of `unregisterHooksByPackage`, - * `unregisterFunctionsByPackage`, `dispatchUnscopedMultiWriteHooks` moved out - * of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * `ObjectQL`: the bodies of `recordResidualOwner`, `residualOwnerSource`, + * `unregisterHooksByPackage`, `registerFunction`, `resolveFunction`, + * `resolveFunctionEntry`, `unregisterFunctionsByPackage`, + * `setDefaultBodyRunner`, `getDefaultBodyRunner`, `setDefaultActionRunner`, + * `getDefaultActionRunner`, `hasHooksFor`, `opensAfterAbortUnit`, + * `dispatchUnscopedMultiWriteHooks` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link HooksAndFunctionsFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link HooksAndFunctionsHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * * The module imports `engine.ts` for types only, so the two modules form no * evaluation cycle. `index.ts` re-exports nothing from this module. */ +import { driverSupportsTransactions } from '@objectstack/spec/data'; import type { HookContext } from '@objectstack/spec/data'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; import { hookStoppedRefusal } from './summary-errors.js'; import { HookTargetRebindError } from './hook-target-rebind-errors.js'; +import { declaredHookCondition } from './hook-wrappers.js'; import { hookMatchesObject } from './engine-top-level.js'; +import type { FunctionEntry, FunctionRegistrationOptions, HookHandler } from './engine-top-level.js'; import type { ObjectQL } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link HooksAndFunctionsFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface HooksAndFunctionsHost { + _defaultActionRunner: ObjectQL['_defaultActionRunner']; + _defaultBodyRunner: ObjectQL['_defaultBodyRunner']; + readonly defaultDriver: ObjectQL['defaultDriver']; + readonly drivers: ObjectQL['drivers']; readonly functions: ObjectQL['functions']; + getDriver: ObjectQL['getDriver']; readonly hooks: ObjectQL['hooks']; + isClosedUnitHandle: ObjectQL['isClosedUnitHandle']; readonly logger: ObjectQL['logger']; + readonly residualOwnerSources: ObjectQL['residualOwnerSources']; + readonly txStore: ObjectQL['txStore']; } -/** The body of `ObjectQL#unregisterHooksByPackage`; the method's docblock there is its contract. */ -export function unregisterHooksByPackage(this: Pick, packageId: string): number { +/** The bodies of the "Class head: units, hooks, functions, body runners, hook dispatch" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class HooksAndFunctionsFacet { + /** The body of `ObjectQL#recordResidualOwner`; the method's docblock there is its contract. */ + static recordResidualOwner(this: Pick, ownerId: string, source: 'code' | 'marketplace'): void { + if (this.residualOwnerSources.get(ownerId) === 'marketplace') return; + this.residualOwnerSources.set(ownerId, source); + } + + /** The body of `ObjectQL#residualOwnerSource`; the method's docblock there is its contract. */ + static residualOwnerSource(this: Pick, ownerId: string): 'code' | 'marketplace' | undefined { + return this.residualOwnerSources.get(ownerId); + } + + /** The body of `ObjectQL#unregisterHooksByPackage`; the method's docblock there is its contract. */ + static unregisterHooksByPackage(this: Pick, packageId: string): number { if (!packageId) return 0; let removed = 0; for (const [event, entries] of this.hooks.entries()) { @@ -60,8 +94,28 @@ export function unregisterHooksByPackage(this: Pick, packageId: string): number { + /** The body of `ObjectQL#registerFunction`; the method's docblock there is its contract. */ + static registerFunction(this: Pick, name: string, handler: HookHandler, packageIdOrOptions?: string | FunctionRegistrationOptions): void { + if (!name || typeof handler !== 'function') return; + const opts: FunctionRegistrationOptions = + typeof packageIdOrOptions === 'string' ? { packageId: packageIdOrOptions } : (packageIdOrOptions ?? {}); + const { packageId, effect } = opts; + this.functions.set(name, { handler, packageId, ...(effect ? { effect } : {}) }); + this.logger.debug('Registered function', { name, packageId, effect }); + } + + /** The body of `ObjectQL#resolveFunction`; the method's docblock there is its contract. */ + static resolveFunction(this: Pick, name: string): HookHandler | undefined { + return this.functions.get(name)?.handler; + } + + /** The body of `ObjectQL#resolveFunctionEntry`; the method's docblock there is its contract. */ + static resolveFunctionEntry(this: Pick, name: string): Readonly | undefined { + return this.functions.get(name); + } + + /** The body of `ObjectQL#unregisterFunctionsByPackage`; the method's docblock there is its contract. */ + static unregisterFunctionsByPackage(this: Pick, packageId: string): number { if (!packageId) return 0; let removed = 0; for (const [name, entry] of this.functions.entries()) { @@ -76,8 +130,162 @@ export function unregisterFunctionsByPackage(this: Pick, event: 'beforeUpdate' | 'beforeDelete', object: string, batchCtx: HookContext): Promise { + /** The body of `ObjectQL#setDefaultBodyRunner`; the method's docblock there is its contract. */ + static setDefaultBodyRunner(this: Pick, runner: any): boolean { + if (this._defaultBodyRunner) { + this.logger.debug('Default body runner already installed — keeping the first'); + return false; + } + this._defaultBodyRunner = runner; + return true; + } + + /** The body of `ObjectQL#getDefaultBodyRunner`; the method's docblock there is its contract. */ + static getDefaultBodyRunner(this: Pick): any { + return this._defaultBodyRunner; + } + + /** The body of `ObjectQL#setDefaultActionRunner`; the method's docblock there is its contract. */ + static setDefaultActionRunner(this: Pick, runner: (actionDef: any) => ((ctx: any) => Promise) | undefined): boolean { + if (this._defaultActionRunner) { + this.logger.debug('Default action runner already installed — keeping the first'); + return false; + } + this._defaultActionRunner = runner; + return true; + } + + /** The body of `ObjectQL#getDefaultActionRunner`; the method's docblock there is its contract. */ + static getDefaultActionRunner(this: Pick): ((actionDef: any) => ((ctx: any) => Promise) | undefined) | undefined { + return this._defaultActionRunner; + } + + /** + * [#5038] Would `triggerHooks(event, ctx)` reach ANY handler for `object`? + * + * Applies the SAME per-object rule the dispatch loop applies — literally the + * same function, {@link hookMatchesObject}, since #5928 — because this answer + * gates a READ of the whole matched row set on the bulk write path. Getting + * it looser than the dispatch loop only costs a wasted query; getting it + * TIGHTER would silently drop hooks that were going to fire. Until #5928 the + * two were separate hand-written copies of one semantic and this comment + * asked the reader to keep them in step; sharing the function is what + * actually keeps them in step, and the property test in + * `hook-exclude-objects.test.ts` pins the direction that matters. + * + * `session.skipAutomations` is deliberately NOT consulted: it suppresses only + * metadata-bound entries, and code-registered hooks (audit, sharing) still + * run, so the row set is still needed. Over-reading in that case is a cost, + * never a correctness loss — and it is the SAFE side of the asymmetry above, + * which is why the pin asserts an implication rather than an equality. + */ + static hasHooksFor(this: Pick, event: string, object: string): boolean { + const entries = this.hooks.get(event); + if (!entries || entries.length === 0) return false; + return entries.some((entry) => hookMatchesObject(entry, object)); + } + + /** + * [#22782] Does this write need its own unit of work? Yes when its `after*` + * dispatch can ABORT and no unit is open, so the abort rolls back what + * `HookSchema.onError` says: the row and every `ctx.api` write the chain made. "Can abort" + * is a metadata-bound hook (`meta` — the only entries with a declared + * `onError`) on `event` for `object`, automations not skipped, that either + * is blocking (not `async`) with `onError` `abort` (the default), or carries + * a `condition` (#22856): a condition that cannot be evaluated aborts the + * operation whatever `onError` says, and before an `async` hook detaches + * (#4775, `HookConditionError`), so it is read by the wrapper's own + * {@link declaredHookCondition}. Hook count and order are untouched (#7413); + * only the boundary around them moves. + * + * No unit — the pre-#22782 path — when one is open (ambient or threaded: the + * caller's unit decides; a threaded handle whose unit has closed is not open, + * #22855 `isClosedUnitHandle`), when `object` is off the default driver + * (`transaction()` covers that one alone and would refuse the write), or when + * that driver has no transactions (ADR-0119 D1; asking would only warn). + * `SummaryRecomputeError` means "the records WERE written": it is rethrown + * after the commit, never through the rollback. Only the write's OWN refusal + * arrives here: one a hook's `ctx.api` write raised was already converted + * where it left the handler (#29422, `hookStoppedRefusal`), so it rolls back. + */ + static opensAfterAbortUnit(this: Pick, event: 'afterInsert' | 'afterUpdate' | 'afterDelete', object: string, context: ExecutionContext | undefined): boolean { + const threaded = context?.transaction; + if ((threaded !== undefined && !this.isClosedUnitHandle(threaded)) || this.txStore.getStore()?.transaction !== undefined) return false; + if ((context as { skipAutomations?: unknown } | undefined)?.skipAutomations) return false; + const aborting = (this.hooks.get(event) ?? []).some((entry) => + entry.meta && hookMatchesObject(entry, object) + && ((!entry.meta.async && (entry.meta.onError ?? 'abort') === 'abort') || declaredHookCondition(entry.meta) !== undefined)); + if (!aborting) return false; + const defaultDriver = this.defaultDriver ? this.drivers.get(this.defaultDriver) : undefined; + try { + return this.getDriver(object) === defaultDriver && driverSupportsTransactions(defaultDriver); + } catch { + return false; // the write itself reports an unroutable object + } + } + + /** + * [#9719, widened to `beforeUpdate` by #9974] The whole-operation + * `beforeUpdate` / `beforeDelete` dispatch for an UNSCOPED predicate write — + * `multi: true` with no caller predicate at all (`options.where` absent or + * `null`) — delivered ONLY to registrations that declared + * {@link HookEntry.dispatchUnscopedMultiWrite} on `event`. + * + * ## Why this dispatch exists + * + * The per-row contract (#5038 / #5574) made the operation's SHAPE invisible + * to `before*` handlers on the predicate path: every per-row context arrives + * with `input.id` bound, and a zero-match predicate dispatches nothing at + * all ([D1]). A guard whose rule is about the shape — #4757's "refuse a + * predicate-less multi-delete outright, regardless of what it matches" on + * `sys_attachment`, and #4630's twin on `sys_comment` — therefore could not + * fire through the wired engine: with rows matched its by-id branch shadowed + * the check, and with none matched it never ran. Both limbs need a dispatch + * that happens BEFORE the matched-row read, keyed on the operation's shape + * alone. + * + * ## Why one helper, both verbs + * + * The update half was measured as a PARTIAL guard rather than a total + * fail-open (#9974): an unscoped `multi: true` update was refused only when + * it happened to sweep a row the caller lacked rights to — the per-row gate + * catching the shape by accident — and resolved silently when the caller was + * entitled to every row, zero-match included. A guard that fires by accident + * reads as enforcement while enforcing nothing, which is why the ruling + * closed it rather than narrowing the declaration. `event` is the ONLY thing + * that differs between the two verbs here, so it is a parameter and not a + * second copy of this method. + * + * ## What the handler receives + * + * The whole-operation context: `input.id` undefined, `input.options` the + * caller's raw options — the contractual upper-bound read `hook.zod.ts` + * blesses for `before*` handlers (`input.options.where` / `.multi`) — and + * `session` built by the engine, so the handler's own bypass rules (system + * context, no identity envelope) apply unchanged. It is a DERIVED context, + * not the batch-level one (`hook.zod.ts`: no handler is dispatched on that), + * carrying `dispatch: { mode: 'record', index: 0 }` — this call stands for + * the caller's whole write, not for one row of it — with the batch `scope` + * object kept identity-shared so a stash still reaches the after phase. + * + * ## Deliberately opt-in + * + * Undeclared registrations see NO new dispatch: generalizing an unscoped- + * multi-write refusal to every guarded object is a product decision this + * mechanism does not take (#9719 commissions the `sys_attachment` + * restoration, #9974 the `sys_comment` update half — neither commissions an + * engine-wide rule). The engine stays neutral; the policy — who is refused, + * with which error — lives in the declaring handler. + * + * ## The id slot is not a lever here either + * + * The old batch dispatch's `input.id` was present-but-`undefined`, and + * binding it rerouted the write onto the by-id branch. The ladder is now + * resolved before any handler runs, so a handler binding `input.id` on this + * context would retarget nothing — refused rather than ignored, the same + * rule as D4 and #6752 (`HookTargetRebindError`, path `'unscoped-multi'`). + */ + static async dispatchUnscopedMultiWriteHooks(this: Pick, event: 'beforeUpdate' | 'beforeDelete', object: string, batchCtx: HookContext): Promise { const entries = this.hooks.get(event); if (!entries || entries.length === 0) return; const flagged = entries.filter( @@ -120,3 +328,4 @@ export async function dispatchUnscopedMultiWriteHooks(this: Pick, object: string, readonlySubject: NonNullable>, row: Record, supplied: Readonly>, context: ExecutionContext | undefined, pass: { +/** The bodies of the "Write preparation and insert" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class InsertFacet { + /** + * [#20805] The create path's static-`readonly` strip of ONE row, together + * with the re-default of every key it takes — the door as `insert()` runs it, + * lifted out so that {@link ObjectQL.validate} runs the same door over its + * preview rows instead of a copy of it (one function per door). `insert()` + * owns everything around it: the `isSystem` gate, the per-call + * `insertDropped` union, the `preserveAudit`-ignored WARN and partial-mode + * culling; `validate()` passes no logger, because a preview writes nothing + * and a strip line would say otherwise. + * + * The field's `defaultValue` is RE-DERIVED for every key the strip took, + * which is #3043's stated contract and a guarantee in its own right: a forged + * `approval_status` becomes `draft` — the enforced initial state — never + * NULL, so a stripped forgery cannot leave a row in a state the object's own + * rules (`requiredWhen`, the state machine) were written to exclude. The + * deleted ingress copy got this for free by running BEFORE + * `applyFieldDefaults`; a strip that runs after the hooks has to ask. Asked + * over the STRIPPED row, so a `defaultValue` expression reads the payload it + * will really be stored beside, and copied back key by key: + * `applyFieldDefaults` also fills every OTHER absent field, and a hook that + * deliberately wrote `null` must keep its null (the first defaults pass, + * ahead of the hooks, is the one that owns those keys). + * + * [#20082] A re-derived `can` default takes the write's map too — from the + * same resolution, so this asks nothing new unless it is the first pass to + * need it. Scoped to the TAKEN keys: those are the only defaults this pass + * keeps, so no other field's default can make it ask, or fail. + * + * @returns the row as the strip leaves it (the SAME reference when nothing + * was taken) and the keys taken, in the row's key order. When the + * re-default's permission resolution fails, `error` carries it and `row` + * is the row as it arrived: the caller decides whether that fails the + * write or culls the row, and the taken keys are still reported. + */ + static async staticReadonlyCreateStrip(this: Pick, object: string, readonlySubject: NonNullable>, row: Record, supplied: Readonly>, context: ExecutionContext | undefined, pass: { logger: Parameters[3]; strictReadonlyWrites: boolean; hookWrittenKeys: ReadonlySet | undefined; @@ -74,3 +133,85 @@ export async function staticReadonlyCreateStrip(this: Pick, driver: IDataDriver, object: string, id: unknown, data: Record, expect: Readonly>, options: Parameters[2]): Promise { + if (typeof driver.updateMany !== 'function') { + if (!this.preconditionDegradedWarned) { + this.preconditionDegradedWarned = true; + this.logger.warn( + `[objectql] the driver for '${object}' has no conditional update (updateMany), so a by-id write ` + + 'precondition degrades to the pre-image check and the write goes through unconditionally: two ' + + 'concurrent claims of one record may both land, and the round-robin cursor may repeat a member.', + { object }, + ); + } + return driver.update(object, id as string, data, options); + } + const affected = await driver.updateMany(object, { object, where: { ...expect, id } } as QueryAST, data, options); + if (!(Number(affected) > 0)) throw new ConcurrentUpdateError(object, String(id)); + return driver.findOne(object, { object, where: { id }, limit: 1 } as QueryAST, options); + } + + /** The body of `ObjectQL#assignment`; the method's docblock there is its contract. */ + static assignment(this: Pick): AssignmentRuleEvaluator { + if (!this.assignmentEvaluator) { + this.assignmentEvaluator = new AssignmentRuleEvaluator({ + listRules: () => this._registry.listItems(ASSIGNMENT_RULE_TYPE) as unknown[], + getSchema: (object) => this._registry.getObject(object), + hasObject: (object) => Boolean(this._registry.getObject(object)), + queues: () => this.ownershipQueueService(), + find: async (object, where, context, limit) => + (await this.find(object, { where, ...(limit ? { limit } : {}) } as EngineQueryOptions, { context: context as ExecutionContext })) as Record[], + count: (object, where, context) => this.count(object, { where } as EngineCountOptions, { context: context as ExecutionContext }), + insert: (object, data, context) => this.insert(object, data, { context: context as ExecutionContext }), + updateExpecting: (object, id, data, expect, context) => + this.update(object, { ...data, id }, { context: context as ExecutionContext, [WRITE_PRECONDITION_OPTION]: expect } as any), + logger: this.logger, + }); + } + return this.assignmentEvaluator; + } + + /** The body of `ObjectQL#readUpdatePriorRow`; the method's docblock there is its contract. */ + static async readUpdatePriorRow(this: Pick, object: string, id: unknown, context: ExecutionContext | undefined, base?: unknown): Promise | null> { + const priorAst: QueryAST = { object, where: { id }, limit: 1 } as QueryAST; + const preOpts = this.buildDriverOptions(object, context, base as any); + return withDeclaredColumnsOnly( + await this.getDriver(object).findOne(object, priorAst, preOpts) as Record | null, + declaredColumnSet(this._registry.getObject(object)), + ); + } + + /** The body of `ObjectQL#insertMany`; the method's docblock there is its contract. */ + static async insertMany(this: Pick, object: string, rows: any[], options?: DataEngineInsertOptions & WriteObservabilityOptions): Promise { + if (!Array.isArray(rows)) throw new Error('insertMany expects an array of rows'); + return this.insert(object, rows, { ...(options ?? {}), __partialRowErrors: true } as any); + } +} diff --git a/packages/objectql/src/engine-middleware-and-context.ts b/packages/objectql/src/engine-middleware-and-context.ts index 8b124c71aa9..afb33640392 100644 --- a/packages/objectql/src/engine-middleware-and-context.ts +++ b/packages/objectql/src/engine-middleware-and-context.ts @@ -3,51 +3,117 @@ /** * [#29986 codemod] The * "Authz invalidation, middleware, session, driver options, posture, organization probe" - * section of `ObjectQL`: the bodies of `executeWithMiddleware`, - * `resolveSystemInsertOrganization` moved out of `engine.ts` whole by + * section of `ObjectQL`: the bodies of `attachAuthzInvalidationPubSub`, + * `detachAuthzInvalidationPubSub`, `registerMiddleware`, `hasObjectMiddleware`, + * `executeWithMiddleware`, `buildReferentialFieldClear`, + * `claimReferentialCascadeDelete`, `probeInstallOrganizations`, + * `invalidateOrganizationProbe`, `resolveSystemInsertOrganization`, + * `transactionCoversDriverFor` moved out of `engine.ts` whole by * `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link MiddlewareAndContextFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link MiddlewareAndContextHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * * The module imports `engine.ts` for types only, so the two modules form no * evaluation cycle. `index.ts` re-exports nothing from this module. */ +import type { IPubSub } from '@objectstack/spec/contracts'; import { redactPropagatedDriverFault } from '@objectstack/types'; import type { ExecutionContext } from '@objectstack/spec/kernel'; import { isWriteEpochOperation } from './write-epoch.js'; +import { bridgeAuthzInvalidation } from './authz-invalidation-bridge.js'; +import { isMissingTableError } from '@objectstack/metadata/errors'; import { ORGANIZATION_OBJECT, SystemWriteOrganizationRequiredError, carriesOrganization, resolveSystemWriteOrganization, resolveTenantFieldName } from './tenancy/system-write-organization.js'; import { isFederatedObject } from './federated-object.js'; -import type { OperationContext } from './engine-top-level.js'; +import type { EngineMiddleware, OperationContext } from './engine-top-level.js'; import type { ObjectQL } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link MiddlewareAndContextFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface MiddlewareAndContextHost { readonly _registry: ObjectQL['_registry']; + authzInvalidationBinding: ObjectQL['authzInvalidationBinding']; + authzInvalidationDetach: ObjectQL['authzInvalidationDetach']; + readonly cascadeIssuedDeletes: ObjectQL['cascadeIssuedDeletes']; + detachAuthzInvalidationPubSub: ObjectQL['detachAuthzInvalidationPubSub']; + find: ObjectQL['find']; + getDriver: ObjectQL['getDriver']; + readonly logger: ObjectQL['logger']; readonly middlewares: ObjectQL['middlewares']; + organizationProbeMemo: ObjectQL['organizationProbeMemo']; probeInstallOrganizations: ObjectQL['probeInstallOrganizations']; resolveEnginePosture: ObjectQL['resolveEnginePosture']; + readonly txStore: ObjectQL['txStore']; readonly writeEpoch: ObjectQL['writeEpoch']; } -/** The body of `ObjectQL#executeWithMiddleware`; the method's docblock there is its contract. */ -export async function executeWithMiddleware(this: Pick, ctx: OperationContext, executor: () => Promise): Promise { +/** The bodies of the "Authz invalidation, middleware, session, driver options, posture, organization probe" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class MiddlewareAndContextFacet { + /** The body of `ObjectQL#attachAuthzInvalidationPubSub`; the method's docblock there is its contract. */ + static attachAuthzInvalidationPubSub(this: Pick, pubsub: IPubSub, nodeId: string): () => void { + if ( + this.authzInvalidationBinding?.pubsub === pubsub && + this.authzInvalidationBinding?.nodeId === nodeId + ) { + return () => this.detachAuthzInvalidationPubSub(); + } + this.detachAuthzInvalidationPubSub(); + this.authzInvalidationDetach = bridgeAuthzInvalidation({ + epoch: this.writeEpoch, + pubsub, + nodeId, + logger: this.logger, + }); + this.authzInvalidationBinding = { pubsub, nodeId }; + this.logger.info('ObjectQL attached to the authz.invalidated cluster channel', { + nodeId, + }); + return () => this.detachAuthzInvalidationPubSub(); + } + + /** The body of `ObjectQL#detachAuthzInvalidationPubSub`; the method's docblock there is its contract. */ + static detachAuthzInvalidationPubSub(this: Pick): void { + const detach = this.authzInvalidationDetach; + this.authzInvalidationDetach = undefined; + this.authzInvalidationBinding = undefined; + detach?.(); + } + + /** The body of `ObjectQL#registerMiddleware`; the method's docblock there is its contract. */ + static registerMiddleware(this: Pick, fn: EngineMiddleware, options?: { object?: string }): void { + this.middlewares.push({ fn, object: options?.object }); + this.logger.debug('Registered middleware', { object: options?.object, total: this.middlewares.length }); + } + + /** The body of `ObjectQL#hasObjectMiddleware`; the method's docblock there is its contract. */ + static hasObjectMiddleware(this: Pick, objectName: string): boolean { + return this.middlewares.some((m) => !!m.object && m.object !== '*' && m.object === objectName); + } + + /** + * Execute an operation through the middleware chain + */ + static async executeWithMiddleware(this: Pick, ctx: OperationContext, executor: () => Promise): Promise { // [#11968] Advance the write epoch FIRST — ahead of every middleware, and // so ahead of any `isSystem` bypass one of them applies. The writes most // likely to change what a caller may see are system ones: the platform @@ -96,8 +162,173 @@ export async function executeWithMiddleware(this: Pick, object: string, execCtx: ExecutionContext | undefined, rows: readonly Record[]): Promise { + /** + * [commit 34ce8e7db] Build the declared `HookContext.referentialFieldClear` marker — + * the read-only hook-context projection of the operation-private + * `__referentialFieldClear` that {@link ObjectQL.cascadeDeleteRelations} + * stamps on the cleanup write's ExecutionContext (#3023). + * + * `true | undefined`, never `false`: the key is ABSENT from every dispatch + * whose envelope does not carry the marker, so a handler reads it as + * `ctx.referentialFieldClear === true` and an absent key keeps meaning "not + * a referential cleanup" — the same back-compatible direction as `dispatch`. + * + * Why a projection of the ENVELOPE rather than a fresh per-call-site signal: + * the `__` key's semantics have always been operation-scoped — the cleanup + * context is `{ ...callerContext, transaction, __referentialFieldClear: true }`, + * plugin-security's ownership-anchor exemption keys on the envelope, and a + * write a hook issues through `ctx.api` during a cleanup dispatch inherits + * it — so the declared face projects the fact faithfully instead of quietly + * re-scoping it. Consumed by `update()`'s hook-context assembly only: every + * reference-cleanup write the engine issues is an update, and the per-row / + * after-phase contexts ride the batch context's spread. + */ + static buildReferentialFieldClear(execCtx?: ExecutionContext): true | undefined { + return (execCtx as any)?.__referentialFieldClear === true ? true : undefined; + } + + /** + * [#23221] Build the declared `HookContext.referentialCascadeDelete` marker, + * the delete twin of {@link buildReferentialFieldClear}: `true` when THIS + * `delete()` of `object`/`id` is the child delete + * {@link walkReferencingRelations} issued for a `cascade` relation, and + * `undefined` for every other delete — the same "absent unless true" + * contract, so a handler reads `ctx.referentialCascadeDelete === true`. + * + * Reads {@link cascadeIssuedDeletes}, not the envelope (that field says why), + * and CLAIMS the token it matches. Called once per `delete()`, below the + * `afterDelete` unit re-entry (`opensAfterAbortUnit`): a call that re-enters + * itself returns before reaching here, so the claim is made by the inner + * call, the one that dispatches the hooks, and nothing later inside it can + * match the same token again. + */ + static claimReferentialCascadeDelete(this: Pick, object: string, id: unknown): true | undefined { + const issued = this.cascadeIssuedDeletes.getStore(); + if (!issued || issued.claimed || id == null) return undefined; + if (issued.object !== object || issued.id !== String(id)) return undefined; + issued.claimed = true; + return true; + } + + /** + * [#8844] The install's organizations, capped at two — only "none / exactly + * one / several" changes any decision. + * + * Memoised because a cron loop inserting N rows would otherwise pay N probes + * for an answer that changes only when `sys_organization` itself is written, + * and every such write passes through this engine's own write faces, which + * clear it ({@link invalidateOrganizationProbe}). The stale window is + * therefore an organization created by ANOTHER process against the same + * database — and on the `single` posture this memo serves, a second + * organization appearing is the ambiguous topology the next boot refuses + * anyway. + * + * `isSystem` on the read: resolving the install's organization must not + * depend on the caller's own reach, and this runs on writes that have no + * caller reach at all. + * + * [#9261] "The probe found no organizations" and "the probe could not run" + * are different facts (ADR-0110 D3), and this returns the FIRST one only when + * it measured it. The `catch {}` this replaces answered both with `[]`, which + * `resolveSystemWriteOrganization` then read as "no organization yet" (a + * branch that landed the row; since ADR-0131 D9 a registered-but-empty + * organization object is refused instead) — so one + * transient failure silently skipped both halves of the #8844 ruling: the + * single-organization stamp (rows land untenanted and fork exactly the + * counter the ruling exists to protect) and the multi-organization REFUSAL + * (fail-open on a guard that must be loud, the #8895 shape). The memo made + * the damage outlive the outage — the invented answer was cached until an + * organization write happened to clear it. + * + * **Exactly one cause is benign, and it was measured rather than assumed.** + * `sys_organization` ROUTES but its TABLE was never provisioned (schema sync + * not run yet): it cannot hold a row, so zero really is the count. Asked + * through the shared `isMissingTableError` predicate + * (`@objectstack/metadata/errors`, #4825) — the same call + * `resolveFileReferences` and `cascadeDeleteRelations` make — never a + * hand-rolled code test. + * + * A SECOND benign cause, now that an unresolved name is refused rather than + * handed to the driver: `sys_organization` is not registered at all (a lean + * embedding, a bare-kernel test). Before an in-process verb refused an + * unresolved name, that install reached the driver and read `[]` on a + * tolerant one or a MISSING TABLE on a strict one — the one driver cause + * above. It now resolves to the engine's own `OBJECT_NOT_FOUND` before any + * driver is asked, so this probe asks the REGISTRY first and treats an + * absent `sys_organization` as the lean-install case it always was: there is + * no organization object here, so there is no organization to derive. This + * is the probe handling absence itself, on a path a body cannot reach — it + * never relies on the resolver's old raw-table fall-through, which is now + * gone. It is not a spelling allow-list: the question is "is the object + * provisioned," asked of the one registry, not "is this one of N names." + * + * Everything else — connection loss, pool exhaustion, a timeout mid-boot, a + * datasource that never connected ({@link DatasourceUnavailableError}), a + * permission denial — means organizations may well exist and simply were not + * seen. It PROPAGATES, envelope intact, and ⛔ nothing is memoised on that + * path: the write that asked fails loudly instead of being filed under a + * guessed topology, and the next write re-probes rather than inheriting the + * guess. No new error code and no new response field — the caller receives + * the probe's own failure, the same disposition #8895 gave the + * referential-integrity probe. + */ + static async probeInstallOrganizations(this: Pick): Promise { + if (this.organizationProbeMemo) return this.organizationProbeMemo; + // The lean-install case: no `sys_organization` object is registered, so + // there is no organization to derive. Asking the registry first keeps the + // probe off the resolver's refusal for an unregistered name (a path a body + // cannot reach), exactly as the old raw-table fall-through read `[]`. + if (!this._registry.getObject(ORGANIZATION_OBJECT)) { + this.organizationProbeMemo = []; + return this.organizationProbeMemo; + } + let ids: readonly string[] = []; + try { + const rows = await this.find(ORGANIZATION_OBJECT, { + fields: ['id'], + limit: 2, + context: { isSystem: true }, + }); + ids = (Array.isArray(rows) ? rows : []) + .map((r: any) => (r?.id ?? r?._id)) + .filter((id: unknown) => id != null && String(id) !== '') + .map((id: unknown) => String(id)); + } catch (error) { + // The one benign cause only — anything else is an outage, not an + // emptiness, and must not be memoised as one. + if (!isMissingTableError(error, ORGANIZATION_OBJECT)) throw error; + ids = []; + } + this.organizationProbeMemo = ids; + return ids; + } + + /** [#8844] Drop the memo — any write that can change the organization set. */ + static invalidateOrganizationProbe(this: Pick): void { + this.organizationProbeMemo = undefined; + } + + /** + * [#8844] Resolve the organization a SYSTEM-context insert carries, per the + * 2026-08-15 ruling (Option 1) — or refuse it. + * + * Returns the organization id to thread as `DriverOptions.tenantId` (the same + * knob a session write sets, so the driver stamps the column and the counter + * scopes by it), or `undefined` when there is nothing to resolve. Throws + * {@link SystemWriteOrganizationRequiredError} wherever no owner is + * derivable (ADR-0131 D9): a wall, several organizations, or — under + * `single`, in a composition that registers the organization object — none. + * + * Called AFTER the beforeInsert hooks on purpose: a hook that stamps the + * organization itself has carried it, and must not then be refused for a + * value it just supplied. + * + * Every early return below is a population the refusal must not touch — see + * the module header for why each is outside the rule by construction rather + * than by exemption. They are ordered cheapest-first, so an ordinary write + * pays two property reads and a regexp. + */ + static async resolveSystemInsertOrganization(this: Pick, object: string, execCtx: ExecutionContext | undefined, rows: readonly Record[]): Promise { // Already carrying an organization on the context — a session write, or a // system write that threaded one. Nothing to resolve; this is the shape the // ruling asks every system write to reach. @@ -145,3 +376,43 @@ export async function resolveSystemInsertOrganization(this: Pick, object: string, tx: unknown): boolean { + const store = this.txStore.getStore(); + // The scope describes the handle in the store. An explicitly-threaded + // handle is covered by it only when it IS that handle. + const scope = store !== undefined && tx === store.transaction ? store.scope : undefined; + if (!scope) return true; + return this.getDriver(object) === scope.driver; + } +} diff --git a/packages/objectql/src/engine-migration-state.ts b/packages/objectql/src/engine-migration-state.ts new file mode 100644 index 00000000000..c3ba98442f0 --- /dev/null +++ b/packages/objectql/src/engine-migration-state.ts @@ -0,0 +1,428 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The + * "Migration flags, value-shape deviations, creation attestations, destroy" + * section of `ObjectQL`: the bodies of `objectHasMediaField`, + * `mediaValueShapeStrictFor`, `objectHasCoveredValueField`, + * `valueShapeStrictFor`, `isFileReferencesMigrationVerified`, + * `haveFileColumnsMoved`, `readMigrationFlagMemoized`, + * `valueShapeViolationsAdmitted`, `admittedViolationSink`, + * `recordObservedDeviation`, `retractCreationAttestation`, + * `invalidateDataMigrationFlags` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. {@link MigrationStateFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. + * + * The bodies reach the class through {@link MigrationStateHost}: exactly the + * `ObjectQL` members they read, private ones included. Each stub hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. + * + * `ObjectQL` here is `ObjectQLStatics` (imported AS `ObjectQL`): forwarders to the + * statics the bodies read, private ones included, which the class fills in a + * static block; the class as a type is `ObjectQLClass`. + * + * Values that stayed in `engine.ts` are imported from it (lever (a) of + * ruling 6591753301); every use is inside a function body, so the import + * cycle is never read at evaluation time. `index.ts` re-exports nothing from + * this module. + */ + +import { FILE_REFERENCE_TYPES } from '@objectstack/spec/data'; +import { DATA_MIGRATION_FLAG_OBJECT, FILE_REFERENCES_MIGRATION_ID, isDataMigrationFlagVerified } from '@objectstack/spec/system'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { isScannableValueShapeField } from './validation/record-validator.js'; +import type { AdmittedValueShapeViolationSink } from './validation/record-validator.js'; +import type { AdmittedValueShapeViolationTally } from './engine-top-level.js'; +import { ObjectQLStatics as ObjectQL } from './engine.js'; +import type { ObjectQL as ObjectQLClass } from './engine.js'; + +/** + * The `ObjectQL` members the bodies of {@link MigrationStateFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. + */ +export interface MigrationStateHost { + readonly _registry: ObjectQLClass['_registry']; + readonly admittedValueShapeViolations: ObjectQLClass['admittedValueShapeViolations']; + creationAttestationRetraction: ObjectQLClass['creationAttestationRetraction']; + deviationRecording: ObjectQLClass['deviationRecording']; + fileReferencesMigrationVerified: ObjectQLClass['fileReferencesMigrationVerified']; + findOne: ObjectQLClass['findOne']; + invalidateDataMigrationFlags: ObjectQLClass['invalidateDataMigrationFlags']; + isFileReferencesMigrationVerified: ObjectQLClass['isFileReferencesMigrationVerified']; + isValueShapesMigrationVerified: ObjectQLClass['isValueShapesMigrationVerified']; + readonly logger: ObjectQLClass['logger']; + migrationGatesAnnounced: ObjectQLClass['migrationGatesAnnounced']; + noteAdmittedValueShapeViolation: ObjectQLClass['noteAdmittedValueShapeViolation']; + objectHasCoveredValueField: ObjectQLClass['objectHasCoveredValueField']; + objectHasMediaField: ObjectQLClass['objectHasMediaField']; + readFileReferencesFlagRow: ObjectQLClass['readFileReferencesFlagRow']; + readMigrationFlagRowMemoized: ObjectQLClass['readMigrationFlagRowMemoized']; + readonly recordedDeviations: ObjectQLClass['recordedDeviations']; + readonly retractedCreationAttestations: ObjectQLClass['retractedCreationAttestations']; + update: ObjectQLClass['update']; + valueShapesMigrationVerified: ObjectQLClass['valueShapesMigrationVerified']; + wasDatastoreCreatedFromEmpty: ObjectQLClass['wasDatastoreCreatedFromEmpty']; +} + +/** The bodies of the "Migration flags, value-shape deviations, creation attestations, destroy" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class MigrationStateFacet { + /** + * Does this object declare a media field at all? Cached per schema object — + * the registry hands back the same instance per object, so this scans a + * field map once rather than on every write. + */ + static objectHasMediaField(objectSchema: any): boolean { + if (!objectSchema?.fields) return false; + const cached = ObjectQL.mediaFieldPresence.get(objectSchema); + if (cached !== undefined) return cached; + const present = Object.values(objectSchema.fields).some( + (def: any) => def && FILE_REFERENCE_TYPES.has(def.type), + ); + ObjectQL.mediaFieldPresence.set(objectSchema, present); + return present; + } + + /** + * The media value-shape verdict for a write against `objectSchema`. + * + * Dormant by design, mirroring the storage module's own rule: an object with + * no file-class field can hold no malformed media value, so it must not pay + * even one query to learn that. Only objects that actually declare media + * consult the deployment flag — and that read is itself memoized, so the + * whole mechanism costs one query per process, and zero for an app that + * stores no files. + */ + static async mediaValueShapeStrictFor(this: Pick, objectSchema: any): Promise { + if (!this.objectHasMediaField(objectSchema)) return false; + return this.isFileReferencesMigrationVerified(); + } + + /** + * Does this object declare a reference or structured-JSON field that a write + * would actually check? Same per-schema cache and same dormancy rule as + * {@link objectHasMediaField}: an object holding none can hold no violation + * of them, so it must not pay a query to learn that. + * + * The membership test is `isScannableValueShapeField` — the validator's own + * — and not raw type membership, because the registry INJECTS covered-type + * fields into every object it registers: `organization_id` and `owner_id` + * (both `system`), plus `created_by` / `updated_by` (both in `SKIP_FIELDS`), + * are all `lookup`s. A caller never writes any of them, so counting them made + * this answer `true` for literally every object — the dormancy rule above + * never fired, and this cache memoized a constant. Same predicate as the + * scanner for the same reason the scanner imports it: three readings of "a + * covered field" drifting by one clause is how a gate ends up governing + * fields nothing enforces. + * + * [#21663] The three that are `readonly` (`organization_id`, `created_by`, + * `updated_by`) DO reach the value-shape check now, on the value a system + * writer, hook or stamp stores. They still do not count here, so an object + * whose only covered fields are those stays warn-first for them: a malformed + * value is admitted, logged and reported, never stored silently. See + * `isScannableValueShapeField` for why widening this test is not the fix. + */ + static objectHasCoveredValueField(objectSchema: any): boolean { + if (!objectSchema?.fields) return false; + const cached = ObjectQL.coveredValueFieldPresence.get(objectSchema); + if (cached !== undefined) return cached; + const present = Object.entries(objectSchema.fields).some( + ([name, def]: [string, any]) => isScannableValueShapeField(name, def), + ); + ObjectQL.coveredValueFieldPresence.set(objectSchema, present); + return present; + } + + /** The reference / structured-JSON value-shape verdict for a write. */ + static async valueShapeStrictFor(this: Pick, objectSchema: any): Promise { + if (!this.objectHasCoveredValueField(objectSchema)) return false; + return this.isValueShapesMigrationVerified(); + } + + /** The body of `ObjectQL#isFileReferencesMigrationVerified`; the method's docblock there is its contract. */ + static async isFileReferencesMigrationVerified(this: Pick): Promise { + return (await this.readFileReferencesFlagRow()).verified; + } + + /** The body of `ObjectQL#haveFileColumnsMoved`; the method's docblock there is its contract. */ + static async haveFileColumnsMoved(this: Pick): Promise { + return (await this.readFileReferencesFlagRow()).columnsMoved; + } + + /** + * The memoized seam both public flag readers share — and the place where + * "read once per process" is kept from meaning "answer from a read that + * never happened" (#4769). + * + * The two negatives this read can produce are NOT the same fact: + * + * - **Conclusive.** The ledger was reachable and it does not authorise the + * gate (no row, `verified_at` null, blocking findings). Evidence was + * consulted; memoize it. A later in-process migration run announces + * itself through {@link invalidateDataMigrationFlags}. + * - **Inconclusive.** The ledger could not be ASKED — `sys_migration` is + * not registered yet, or the query threw. Both still answer `false` (an + * unaskable gate stays closed), but freezing that for the life of the + * process turns one unlucky early write into a whole boot's posture. It + * is exactly how one boot ends up lax over data the next boot rejects: + * the platform objects register during kernel init while the very first + * write can land before them, so the answer cached is about a moment + * when nothing could have answered at all. + * + * So an inconclusive read is answered but not kept. The retry costs one + * registry lookup per write until the ledger exists (it short-circuits + * before any query), and stops the moment a real read succeeds. + */ + static async readMigrationFlagMemoized(this: Pick, slot: 'fileReferencesMigrationVerified' | 'valueShapesMigrationVerified', migrationId: string, verifiedLog: string): Promise { + return (await this.readMigrationFlagRowMemoized(slot, migrationId, verifiedLog)).verified; + } + + /** The body of `ObjectQL#valueShapeViolationsAdmitted`; the method's docblock there is its contract. */ + static valueShapeViolationsAdmitted(this: Pick): Record { + const out: Record = {}; + for (const [migrationId, tally] of this.admittedValueShapeViolations) { + out[migrationId] = { count: tally.count, first: { ...tally.first } }; + } + return out; + } + + /** + * The sink `validateRecord` reports admitted violations to. Built per write + * (one closure per call, not per row) so the tally can name the object. + */ + static admittedViolationSink(this: Pick, object: string): AdmittedValueShapeViolationSink { + return (violation) => this.noteAdmittedValueShapeViolation(object, violation); + } + + /** + * Record that this deployment has ADMITTED a value its own verified contract + * rejects (#4797) — without touching `verified_at`. + * + * ## The window + * + * Once a certificate holds, the write path is strict and a non-conforming + * value cannot land, so the certificate cannot go stale on its own. The + * operator escape hatches are the exception, and they exist *precisely* to + * relax a deployment that has already verified: with + * `OS_ALLOW_LAX_MEDIA_VALUES` / `OS_ALLOW_LAX_VALUE_SHAPES` on, the value is + * admitted and persisted while `sys_migration` still reads `verified_at` + * non-null, `blocking: 0`. Turn the switch off — or let any other process or + * machine run without it — and strict returns to reject the very data this + * deployment stored. Meanwhile the `adr-0104-file-references` gate, which + * also governs reclamation of released field files, keeps deleting bytes on + * the strength of a certificate that is now false. + * + * ## Why a marker rather than a revocation + * + * {@link retractCreationAttestation} clears `verified_at`, and is right to: + * its target is a certificate issued on the inference "created empty, so + * clean", which a single counterexample fully disproves. A certificate + * earned by `os migrate … --apply` is a walk of the whole store, and one + * admitted write is not evidence of that order — overturning it would make a + * deliberately temporary switch into a one-way door, forcing a full + * re-migration on anyone who used the escape hatch once. + * + * So the authority is withdrawn in proportion to reversibility. The marker + * leaves every recoverable behaviour running (strict enforcement once the + * switch is off, tombstoning, throttling — a rejected write is retried, a + * tombstone is lifted on re-attach) and stops only what cannot be undone: + * `authorisesIrreversibleAction` is false while it stands, so the reap + * guard's byte delete refuses. A real apply-mode run clears it. + * + * ## Cost + * + * At most one ledger read+update per migration id per process, never awaited + * by the write that triggered it, and skipped entirely once the marker is + * standing. A lax write is therefore not a ledger round-trip; the FIRST lax + * write of each class is, and only while the row is still verified. + * + * Deliberately never inserts. No row means nothing was certified, so there + * is no authority to withdraw — and inserting one would fabricate a run that + * never happened. + */ + static recordObservedDeviation(this: Pick, migrationId: string): void { + if (this.recordedDeviations.has(migrationId)) return; + if (!this._registry.getObject(DATA_MIGRATION_FLAG_OBJECT)) return; + this.recordedDeviations.add(migrationId); + this.deviationRecording = this.deviationRecording + .then(async () => { + // [#20648] Same single-row route as `readMigrationFlagVerified`. + const row: any = await this.findOne(DATA_MIGRATION_FLAG_OBJECT, { + where: { id: migrationId }, + context: { isSystem: true } as ExecutionContext, + }); + if (!row || row.id !== migrationId) return; // nothing certified — no authority to withdraw + if (row.deviation_observed_at != null && String(row.deviation_observed_at) !== '') return; // already standing + // Only a row that currently authorises something can have authority + // withdrawn. An unverified row already denies both halves, so marking + // it would add a diagnostic nobody gates on — and the next apply run + // would clear it before anything read it. + const verified = isDataMigrationFlagVerified({ + id: migrationId, + last_run_at: String(row.last_run_at ?? ''), + verified_at: row.verified_at == null ? null : String(row.verified_at), + blocking: typeof row.blocking === 'number' ? row.blocking : Number(row.blocking ?? Number.NaN), + }); + if (!verified) return; + const tally = this.admittedValueShapeViolations.get(migrationId); + const now = new Date().toISOString(); + await this.update( + DATA_MIGRATION_FLAG_OBJECT, + { + id: migrationId, + deviation_observed_at: now, + deviation_detail: JSON.stringify({ + observed: 'lax-admitted-violating-value', + ...(tally?.first ?? {}), + }), + updated_at: now, + }, + { context: { isSystem: true } as ExecutionContext }, + ); + this.logger.warn( + `[value-shape] '${migrationId}': this deployment is recorded as verified, but an ` + + 'escape hatch (OS_ALLOW_LAX_MEDIA_VALUES / OS_ALLOW_LAX_VALUE_SHAPES) just admitted a ' + + `value that contract rejects (${tally?.first.object}.${tally?.first.field}: ` + + `${tally?.first.detail}). The certificate stands for everything recoverable, but ` + + 'irreversible actions are withheld — released field files are no longer collected, so ' + + 'no byte is deleted on evidence this deployment has contradicted. Fix the data and run ' + + '`os migrate ' + + (migrationId === FILE_REFERENCES_MIGRATION_ID ? 'files-to-references' : 'value-shapes') + + ' --apply` to clear it (ADR-0104).', + ); + }) + .catch((err: any) => { + // Bookkeeping must never surface as a write failure. Failing to record + // is the dangerous direction — it leaves the reclamation gate open on + // a certificate we now know is stale — so say so loudly. + this.logger.warn( + `[value-shape] could not record the observed deviation for '${migrationId}' ` + + `(${err?.message ?? err}) — the ledger still authorises irreversible collection while ` + + 'this deployment holds a value its own contract rejects; run the migration to ' + + 're-derive the gate', + ); + }); + } + + /** + * Tear up a creation attestation this boot has just contradicted (#4769). + * + * The attestation normally never gets written in the first place: it asks + * {@link valueShapeViolationsAdmitted} before recording anything, so a boot + * that seeded a violating value declines to certify. This closes the other + * order — the certificate is already in the ledger and the contradicting + * value lands afterwards, which is reachable whenever the deployment is + * still lenient at that moment (`OS_ALLOW_LAX_MEDIA_VALUES` / + * `OS_ALLOW_LAX_VALUE_SHAPES`) or whenever a writer runs after the + * attestation point at all — the `os dev` hot-reload seeder and a runtime + * marketplace install both seed on a store this boot created. Without this + * the ledger would keep asserting a fact the store contradicts, and the NEXT + * boot would enforce it against exactly the data this one wrote. + * + * The boot's own inline seed used to head that list, via the background + * continuation of a run that overran `OS_INLINE_SEED_BUDGET_MS` — the + * attestation's `kernel:ready` backstop fired mid-seed and the tail landed + * against the certificate it had just issued. #4795 closed that ordering at + * the source: the attestation now defers while the `seed-settlement` contract + * reports a source outstanding, so the inline seed can no longer contradict + * a certificate this boot issued. This stays the safety net rather than the + * first line of defence for it. + * + * Deliberately narrow: + * + * - only on a store THIS boot created from empty, so the only verified row + * that can exist is the attestation this boot issued. Evidence produced by + * a real `os migrate … --apply` run is never rewritten from here — a scan + * that walked the whole store outranks a single write's observation, and + * an operator who opted into leniency did not ask us to revoke their + * migration; + * - never inserts. No row means nothing was certified, and the attestation + * declining on the tally is what keeps it that way; + * - once per migration id per process, and never awaited by the write. The + * recorded `blocking` count is therefore a LOWER bound — which is all the + * gate reads, since any non-zero count closes it. + */ + static retractCreationAttestation(this: Pick, migrationId: string): void { + if (this.retractedCreationAttestations.has(migrationId)) return; + if (!this._registry.getObject(DATA_MIGRATION_FLAG_OBJECT)) return; + if (!this.wasDatastoreCreatedFromEmpty()) return; + this.retractedCreationAttestations.add(migrationId); + this.creationAttestationRetraction = this.creationAttestationRetraction + .then(async () => { + // [#20648] Same single-row route as `readMigrationFlagVerified`. + const row: any = await this.findOne(DATA_MIGRATION_FLAG_OBJECT, { + where: { id: migrationId }, + context: { isSystem: true } as ExecutionContext, + }); + if (!row || row.id !== migrationId) return; // nothing certified — nothing to revoke + if (row.verified_at == null) return; // gate already closed + const tally = this.admittedValueShapeViolations.get(migrationId); + const now = new Date().toISOString(); + let details: Record = {}; + if (typeof row.details === 'string' && row.details.length > 0) { + try { + const parsed = JSON.parse(row.details); + if (parsed && typeof parsed === 'object') details = parsed as Record; + } catch { + details = { previous_details: row.details }; + } + } + await this.update( + DATA_MIGRATION_FLAG_OBJECT, + { + id: migrationId, + verified_at: null, + blocking: tally?.count ?? 1, + details: JSON.stringify({ + ...details, + revoked: 'boot-admitted-violating-value', + revoked_at: now, + revoked_by: tally?.first, + }), + updated_at: now, + }, + { context: { isSystem: true } as ExecutionContext }, + ); + this.invalidateDataMigrationFlags(); + this.logger.warn( + `[value-shape] revoked '${migrationId}': this deployment was recorded as verified at ` + + 'creation, then this boot wrote a value that contradicts it ' + + `(${tally?.first.object}.${tally?.first.field}: ${tally?.first.detail}). ` + + 'The gate is closed again — fix the data, then run `os migrate ' + + (migrationId === FILE_REFERENCES_MIGRATION_ID ? 'files-to-references' : 'value-shapes') + + ' --apply` to re-earn it (ADR-0104).', + ); + }) + .catch((err: any) => { + // Bookkeeping must never surface as a write failure. Staying verified + // is the bad direction, so say so loudly rather than silently. + this.logger.warn( + `[value-shape] could not revoke the creation attestation for '${migrationId}' ` + + `(${err?.message ?? err}) — the ledger still claims this deployment is verified ` + + 'while its data contradicts that; run the migration to re-derive it', + ); + }); + } + + /** The body of `ObjectQL#invalidateDataMigrationFlags`; the method's docblock there is its contract. */ + static invalidateDataMigrationFlags(this: Pick): void { + this.fileReferencesMigrationVerified = null; + this.valueShapesMigrationVerified = null; + this.migrationGatesAnnounced = false; + this.recordedDeviations.clear(); + } +} diff --git a/packages/objectql/src/engine-references-and-master-detail.ts b/packages/objectql/src/engine-references-and-master-detail.ts index 7f4c2c59193..c4ebc24d263 100644 --- a/packages/objectql/src/engine-references-and-master-detail.ts +++ b/packages/objectql/src/engine-references-and-master-detail.ts @@ -3,42 +3,242 @@ /** * [#29986 codemod] The * "Realtime, i18n, home amounts, validation messages, references, master-detail" - * section of `ObjectQL`: the bodies of `inspectCascadeDelete` moved out of - * `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * section of `ObjectQL`: the bodies of `setRealtimeService`, `setI18nService`, + * `setHomeLocalizationResolver`, `homeAmountOrganizationOf`, + * `previewHomeAmounts`, `projectHomeAmountsOntoPatch`, + * `masterDetailHeaderExists`, `materializeParentHeader`, + * `masterDeclaredFields`, `inspectCascadeDelete`, `getCascadeDeleteObjects` + * moved out of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link ReferencesAndMasterDetailFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link ReferencesAndMasterDetailHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * - * The module imports `engine.ts` for types only, so the two modules form no - * evaluation cycle. `index.ts` re-exports nothing from this module. + * `ObjectQL` here is `ObjectQLStatics` (imported AS `ObjectQL`): forwarders to the + * statics the bodies read, private ones included, which the class fills in a + * static block; the class as a type is `ObjectQLClass`. + * + * Values that stayed in `engine.ts` are imported from it (lever (a) of + * ruling 6591753301); every use is inside a function body, so the import + * cycle is never read at evaluation time. `index.ts` re-exports nothing from + * this module. */ -import type { ObjectQL } from './engine.js'; +import type { IRealtimeService } from '@objectstack/spec/contracts'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { homeAmountsTouched, planHomeAmounts, projectHomeAmounts } from './home-amount-projection.js'; +import type { HomeAmountPlanEntry, HomeAmountServing, HomeLocalizationResolver } from './home-amount-projection.js'; +import { materializeDeclaredFields } from './declared-fields.js'; +import { collectCascadeDeleteObjects } from './engine-delete-cascade.js'; +import { ObjectQLStatics as ObjectQL } from './engine.js'; +import type { ObjectQL as ObjectQLClass } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link ReferencesAndMasterDetailFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface ReferencesAndMasterDetailHost { - readonly _registry: ObjectQL['_registry']; - collectCascadeDeleteSet: ObjectQL['collectCascadeDeleteSet']; - resolveObjectName: ObjectQL['resolveObjectName']; + readonly _registry: ObjectQLClass['_registry']; + cascadeRelationBehavior: ObjectQLClass['cascadeRelationBehavior']; + collectCascadeDeleteSet: ObjectQLClass['collectCascadeDeleteSet']; + homeAmountServing: ObjectQLClass['homeAmountServing']; + homeLocalizationResolver: ObjectQLClass['homeLocalizationResolver']; + i18nService: ObjectQLClass['i18nService']; + readonly logger: ObjectQLClass['logger']; + masterDeclaredFields: ObjectQLClass['masterDeclaredFields']; + masterDetailHeadersCallerReads: ObjectQLClass['masterDetailHeadersCallerReads']; + realtimeService: ObjectQLClass['realtimeService']; + referenceExists: ObjectQLClass['referenceExists']; + resolveObjectName: ObjectQLClass['resolveObjectName']; } -/** The body of `ObjectQL#inspectCascadeDelete`; the method's docblock there is its contract. */ -export async function inspectCascadeDelete(this: Pick, object: string, id: string | number): Promise> { +/** The bodies of the "Realtime, i18n, home amounts, validation messages, references, master-detail" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class ReferencesAndMasterDetailFacet { + /** The body of `ObjectQL#setRealtimeService`; the method's docblock there is its contract. */ + static setRealtimeService(this: Pick, service: IRealtimeService): void { + this.realtimeService = service; + this.logger.info('RealtimeService configured for data events'); + } + + /** The body of `ObjectQL#setI18nService`; the method's docblock there is its contract. */ + static setI18nService(this: Pick, service: { + t?: (key: string, locale: string, params?: Record) => string; + getLocales?: () => string[]; + getTranslations?: (locale: string) => Record; + getFallbackLocale?: () => string | undefined; + getDefaultLocale?: () => string | undefined; + }): void { + this.i18nService = service; + this.logger.info('I18nService configured for validation messages'); + } + + /** The body of `ObjectQL#setHomeLocalizationResolver`; the method's docblock there is its contract. */ + static setHomeLocalizationResolver(this: Pick, resolver: HomeLocalizationResolver | undefined): void { + this.homeLocalizationResolver = resolver; + } + + /** + * [ADR-0144 D5] The organization a written row belongs to, for its home + * projection: the row's own `organization_id`, else the writer's active + * organization, else the organization a system insert resolved — the + * assignment seam's reading, so both seams convert and route one row under + * one organization. + */ + static homeAmountOrganizationOf(row: Record | null | undefined, context: ExecutionContext | undefined, derived?: string | undefined): string | undefined { + const own = row?.organization_id; + if (own !== undefined && own !== null && own !== '') return String(own); + if (context?.tenantId) return String(context.tenantId); + return derived ?? undefined; + } + + /** + * [ADR-0144 D5] The `validate()` preview's run of the home projection: the + * write's own computation over the preview's image (the insert row, or the + * stored row with the patch over it), answering nothing and writing + * nothing — it throws exactly the refusal the write would. + */ + static async previewHomeAmounts(this: Pick, object: string, schema: unknown, mode: 'insert' | 'update', row: Record, image: Record, context: ExecutionContext | undefined, recordId: unknown): Promise { + const plan = planHomeAmounts(schema); + const entries = mode === 'insert' ? plan : homeAmountsTouched(plan, row); + if (entries.length === 0) return; + await projectHomeAmounts(entries, image, this.homeAmountServing(context), { + object, + operation: mode, + organizationId: ObjectQL.homeAmountOrganizationOf(image, context), + recordId, + }); + } + + /** + * [ADR-0144 D5] Project a by-id update's home amounts onto its FINAL patch: + * only the declarations the patch touches ({@link homeAmountsTouched}), + * computed over the stored row with the patch laid over it. Shared by the + * write and its `validate()` preview, so a dry run refuses what the write + * refuses. + */ + static async projectHomeAmountsOntoPatch(this: Pick, object: string, plan: readonly HomeAmountPlanEntry[], prior: Record | null | undefined, patch: Record, context: ExecutionContext | undefined, recordId: unknown, serving: HomeAmountServing = this.homeAmountServing(context)): Promise { + const touched = homeAmountsTouched(plan, patch); + if (touched.length === 0) return; + const image = { ...(prior ?? {}), ...patch }; + Object.assign(patch, await projectHomeAmounts(touched, image, serving, { + object, + operation: 'update', + organizationId: ObjectQL.homeAmountOrganizationOf(image, context), + recordId, + })); + } + + /** + * [#22519] The write's reference check for a `master_detail` field: does `id` + * name a header THIS CALLER can read? The write's two header reads, as + * {@link resolveMasterDetailParent} reads them: the caller's read door + * ({@link masterDetailHeadersCallerReads}), then the elevated, tenant-kept + * {@link referenceExists} for a header it returned. A header the read door + * does not return is `false` — `reference_not_found`, exactly as a header + * that exists nowhere. + * + * `null` (fail-open) only where {@link referenceExists} answers it, and for + * an unregistered master, which has no read door to ask either. A SYSTEM + * caller reaches it only as a flow run's elevated write, or a write that + * write sets off (#28881): its read door returns every header + * ({@link masterDetailHeadersCallerReads}), so the header's existence in the + * caller's tenant is what is asked. + * + * ⛔ `master_detail` only — the plain `lookup` probe is not this rule's. + */ + static async masterDetailHeaderExists(this: Pick, target: string, id: unknown, context: ExecutionContext | undefined): Promise { + const master = this.resolveObjectName(target); + if (!this._registry.getObject(master)) return null; + const readable = await this.masterDetailHeadersCallerReads(master, [id], context); + if (!readable.has(String(id))) return false; + return this.referenceExists(target, id, context); + } + + /** + * [#6457] Make a resolved master-detail header TOTAL over the MASTER + * object's declared fields, so a `parent.` predicate is evaluable + * whatever subset of columns the driver echoed back. + * + * ## The hole this closes + * + * #4953 made the `record` / `previous` roots total at every server seam + * ({@link ./declared-fields.js#materializeDeclaredFields}); the 2026-08-06 + * ruling deliberately left `parent` out, because `parent` is a row of ANOTHER + * object and its ABSENCE is #4889's fail-closed signal. What that left behind + * is the same trap one root over — visible as a THREE-row table, of which + * only the middle row moves here: + * + * | header state | `readonlyWhen: parent.status == null` | before | now | + * |---|---|---|---| + * | carries `status` | evaluates | locks per verdict | unchanged | + * | resolved, no `status` key | `No such key: status` — `parent` IS bound, so `unknownVariableOf` does not match ⇒ ordinary fail-OPEN | **declared lock let through** | evaluates (`status` reads `null`) | + * | unresolvable (`null`) | `Unknown variable: parent` | LOCKED (#4889) | LOCKED — unchanged | + * + * The middle row is the bug: whether a declared lock enforced depended on + * which columns a driver happened to return, which is not something an author + * can see or control. `requiredWhen` (#4977) shares the binding and had the + * mirror of it — fail-open there means the requirement is simply not + * enforced. One materialisation serves both consumers. + * + * ## Why HERE, and not in the strip / the evaluator + * + * `stripReadonlyWhenFields*` and `evaluateValidationRules` are pure functions + * over what they are handed; they hold the DETAIL object's field table and + * have no way to reach the MASTER's — threading a second field table through + * their signatures would put the master's schema in four call sites' hands to + * serve one binding. The engine already holds both the registry and the + * just-read header at these two seams, so the header arrives at those + * functions already total and their signatures do not move. + * + * ## The fail-CLOSED line is preserved EXACTLY (#4889) + * + * This function is only ever reached with a row IN HAND — the persisted-state + * precondition `declared-fields.ts` states. An UNRESOLVABLE header still + * returns `null` from the resolvers above, still leaves `parent` unbound, + * still faults as `Unknown variable: parent`, and is still read as LOCKED. The + * two cases stay distinguishable by construction: absence is decided before + * this function is called, materialisation only ever applies to a row that + * exists. + * + * A master the registry cannot resolve yields no field table, and the header + * passes through unchanged — sparse, i.e. exactly the pre-#6457 behaviour. + * That is the honest answer: without the declared shape we cannot know which + * absent keys are fields and which would be fabrication. + * + * COPIED before materialising, like every other caller: the row is what + * `findOne`/`find` returned and may be observed elsewhere; it must not gain + * materialised nulls behind its reader's back. + */ + static materializeParentHeader(this: Pick, master: string, row: Record): Record { + return materializeDeclaredFields({ ...row }, this.masterDeclaredFields(master)); + } + + /** The MASTER object's declared-field table, or `undefined` when the registry + * does not know it (see {@link materializeParentHeader}). */ + static masterDeclaredFields(this: Pick, master: string): Record | undefined { + const schema = this._registry.getObject(master) as { fields?: Record } | undefined; + const fields = schema?.fields; + return fields && typeof fields === 'object' ? fields : undefined; + } + + /** The body of `ObjectQL#inspectCascadeDelete`; the method's docblock there is its contract. */ + static async inspectCascadeDelete(this: Pick, object: string, id: string | number): Promise> { const resolved = this.resolveObjectName(object); const set = await this.collectCascadeDeleteSet( resolved, [id], undefined, this._registry.getAllObjects(), { fileElevations: false }, @@ -53,3 +253,15 @@ export async function inspectCascadeDelete(this: Pick, object: string): string[] | undefined { + if (!this._registry.getObject(object)) return undefined; + const host: ThisParameterType = { + cascadeRelationBehavior: (...a) => this.cascadeRelationBehavior(...a), + }; + return collectCascadeDeleteObjects.call( + host, this.resolveObjectName(object), this._registry.getAllObjects(), + ); + } +} diff --git a/packages/objectql/src/engine-registration.ts b/packages/objectql/src/engine-registration.ts index 86a0b1c68d1..e2878909f10 100644 --- a/packages/objectql/src/engine-registration.ts +++ b/packages/objectql/src/engine-registration.ts @@ -5,21 +5,28 @@ * "Package, app, plugin, driver and datasource registration" section of * `ObjectQL`: the bodies of `registerPackageBody`, * `registerMetadataCollections`, `supplyFileColumnsMovedResolver`, - * `registerDatasourceDef`, `applyExternalDriverFloor`, `assertWriteAllowed` - * moved out of `engine.ts` whole by `scripts/codemod-split-engine.mjs`. + * `unregisterDriver`, `registerDatasourceDef`, `applyExternalDriverFloor`, + * `listDatasourceDefs`, `markDatasourceUnavailable`, + * `clearDatasourceUnavailable`, `getDefaultDriverName`, + * `listUnavailableDatasources`, `assertWriteAllowed` moved out of `engine.ts` + * whole by `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link RegistrationFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link RegistrationHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * * The module imports `engine.ts` for types only, so the two modules form no * evaluation cycle. `index.ts` re-exports nothing from this module. @@ -28,6 +35,7 @@ import type { ServiceObject } from '@objectstack/spec/data'; import type { InstalledPackageSource } from '@objectstack/spec/kernel'; import type { IDataDriver } from '@objectstack/core'; +import type { DatasourceUnavailableInfo, DatasourceUnavailableKind } from './driver-connect-errors.js'; import { ExternalWriteForbiddenError, pluralToSingular } from '@objectstack/spec/shared'; import type { ArtifactInstallScope } from './registry.js'; import { ASSEMBLED_VIEW_ITEMS_KEY, AssembledViewArtifactSchema, expandViewContainer, isAggregatedViewContainer } from '@objectstack/spec'; @@ -40,25 +48,35 @@ import type { DatasourceDef } from './engine-top-level.js'; import type { ObjectQL } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link RegistrationFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface RegistrationHost { readonly _registry: ObjectQL['_registry']; applyExternalDriverFloor: ObjectQL['applyExternalDriverFloor']; readonly datasourceDefs: ObjectQL['datasourceDefs']; + defaultDriver: ObjectQL['defaultDriver']; readonly drivers: ObjectQL['drivers']; haveFileColumnsMoved: ObjectQL['haveFileColumnsMoved']; readonly logger: ObjectQL['logger']; registerMetadataCollections: ObjectQL['registerMetadataCollections']; registerPlugin: ObjectQL['registerPlugin']; resolveNavObjectNames: ObjectQL['resolveNavObjectNames']; + readonly unavailableDatasources: ObjectQL['unavailableDatasources']; } -/** The body of `ObjectQL#registerPackageBody`; the method's docblock there is its contract. */ -export function registerPackageBody(this: Pick, manifest: any, id: string, namespace: string | undefined, version: string | undefined, source: Exclude, scope: ArtifactInstallScope | undefined): void { +/** The bodies of the "Package, app, plugin, driver and datasource registration" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class RegistrationFacet { + /** + * [#28697] The part of {@link registerApp} a registry verb can refuse + * part-way: the package record, its objects, extensions, apps, collections, + * seed data and nested plugins, in that order. Every body-only refusal was + * judged before this runs; what is written after it is in `registerApp`. + */ + static registerPackageBody(this: Pick, manifest: any, id: string, namespace: string | undefined, version: string | undefined, source: Exclude, scope: ArtifactInstallScope | undefined): void { // 1. Register the Package (manifest + lifecycle state). // [ADR-0130 D1/D3] `scope` is the artifact's own package list, passed by // the load path that read the artifact. It is what lets the install gate @@ -183,8 +201,49 @@ export function registerPackageBody(this: Pick, source: any, ownerId: string, sourceLabel: string, ownerVersion?: string) { + /** + * Register one source's stack collections into the registry — the SINGLE + * body both registration seams run. + * + * ## Why this is one method and not two loops (#7163) + * + * It used to be two, and #7049 only got halfway: that card hoisted the + * ENUMERATION (`METADATA_ARRAY_KEYS`) out of the two seams after they had + * drifted four collections apart, and measured the loop bodies against each + * other on the way past — recording that they still differed in a per-key + * `debug` line, the manifest seam's aggregated-view expansion, and its + * warn-on-nameless-item. Sharing the list made "which collections does a + * seam see?" unanswerable-differently; it left "what does a seam DO with a + * collection both see?" still answered in two places. + * + * `views` is where that cost was measurable. "Object has-many View" + * (ADR-0017 §2, §3.2) makes the loader **dual-read**: an aggregated + * `defineView` container is registered under the bare `` key for + * back-compatible reads AND expanded into independent `ViewItem`s under + * `.`. Only the expanded items carry `viewKind`, and + * `getViewsByObject()` (`metadata-manager.ts`) filters on exactly that — so + * a container that is registered but never expanded is invisible to it, and + * to `GET /meta/view?object=` and the view switcher above it. The manifest + * seam expanded; the nested-plugin seam did not, so one container registered + * `['account', 'account.all_accounts', 'account.form']` through the manifest + * and `['account']` through a nested plugin. No refusal, no diagnostic: a + * package shipping its views through `manifest.plugins[]` simply had no + * views, as far as every reader of the expanded layer was concerned. + * + * Both differences had the same structure — a loop body copied, then + * improved on one side only — so the body is shared rather than reconciled, + * for the reason #7049 gave for the list: a divergence that cannot be + * written down cannot be re-introduced by the next hand patch. What legally + * varies between the seams is passed in: which object is read, which package + * id is stamped (both resolve to the SAME owning package — a nested plugin + * contributes under its parent's ownership), and the label the `debug` line + * names the source with. + * + * @param source The manifest or nested-plugin config to read collections from. + * @param ownerId The owning package id — stamped as ADR-0010 provenance. + * @param sourceLabel Human-readable source name for the `debug` line. + */ + static registerMetadataCollections(this: Pick, source: any, ownerId: string, sourceLabel: string, ownerVersion?: string) { // `ownerVersion` is the owning package's version, stamped like `ownerId`. for (const key of METADATA_ARRAY_KEYS) { const items = (source as any)?.[key]; @@ -269,8 +328,42 @@ export function registerMetadataCollections(this: Pick, driver: IDataDriver): void { + /** + * Hand a freshly-registered driver the ADR-0104 media arm — the kernel→driver + * supply seam (#15989, sequencing step 2 of ADR-0104's 2026-09-05 addendum). + * + * ## Why here, and why a closure rather than a value + * + * `registerDriver` is the one funnel every driver this engine will ever + * route to passes through, and it is early: the resolver is installed before + * `init()` connects and long before schema sync calls `initObjects`, which + * is where a SQL driver asks the closure exactly once and freezes the + * answer. A VALUE could not be supplied here at all — the fact lives in a + * `sys_migration` row this very driver has not connected to yet — so what is + * handed over is the question, not the answer. + * + * ## Duck-typed, and silent when the driver has no such seam + * + * A driver with no media arm (memory, mongodb, a third-party one) exposes no + * `setFileColumnsMovedResolver` and is left alone. That is not a degradation + * to report: a driver that never asks the question keeps whatever encoding + * it always had. + * + * ## ⛔ A host declaration WINS — this only ever fills an empty slot + * + * If the host passed `fileColumnsMoved` to the driver's own config, the + * driver refuses this resolver and says so on its own terms. The asymmetry + * is deliberate and the safety argument runs one way only: a host that + * declared `true` while the ledger says otherwise is a host that knows + * something about its own storage that the ledger does not yet record, and + * the engine silently overruling it to `false` would have the driver write + * JSON into columns that have already been retyped. The reverse — the engine + * overruling a declared `false` to `true` — writes bare ids into a JSON + * column, which is the failure direction the whole mechanism exists to + * prevent. Neither is acceptable, so the more specific authority wins and + * the engine never contradicts an explicit composition. + */ + static supplyFileColumnsMovedResolver(this: Pick, driver: IDataDriver): void { const sink = driver as unknown as { setFileColumnsMovedResolver?: (resolve: () => Promise) => void; }; @@ -288,8 +381,26 @@ export function supplyFileColumnsMovedResolver(this: Pick, def: DatasourceDef): void { + /** The body of `ObjectQL#unregisterDriver`; the method's docblock there is its contract. */ + static unregisterDriver(this: Pick, name: string): boolean { + const removed = this.drivers.delete(name); + // Cleared regardless of `removed`: a datasource that never connected has a + // def and no driver, so keying this off the driver entry would strand + // exactly the rows a failed datasource leaves behind. + this.datasourceDefs.delete(name); + if (this.defaultDriver === name) { + this.defaultDriver = null; + this.logger.info( + 'Evicted the DEFAULT driver — this engine has no default until one is registered', + { driverName: name }, + ); + } + if (removed) this.logger.info('Unregistered driver', { driverName: name }); + return removed; + } + + /** The body of `ObjectQL#registerDatasourceDef`; the method's docblock there is its contract. */ + static registerDatasourceDef(this: Pick, def: DatasourceDef): void { if (!def?.name) return; // [#27277] A def that names no driver keeps the id stored before it: a // caller registering only the write-gate facts after a connect must not @@ -303,8 +414,18 @@ export function registerDatasourceDef(this: Pick, name: string): void { + /** + * [#27277] Put the registered driver of datasource `name` behind the engine's + * correctness floor when its definition calls for one, or take it back out + * when a later definition no longer does (`external-driver-floor.ts`). + * + * Called from both doors — {@link registerDriver} and + * {@link registerDatasourceDef} — because either can come first. The floored + * driver REPLACES the entry in `drivers`, once, so every reader of the map + * (`getDriver`, `getDriverByName`, the default-driver and transaction-scope + * identity checks) holds one instance. + */ + static applyExternalDriverFloor(this: Pick, name: string): void { const current = this.drivers.get(name); if (!current) return; const floored = isExternalDriverFloor(current); @@ -326,8 +447,50 @@ export function applyExternalDriverFloor(this: Pick, objectName: string, operation: 'insert' | 'update' | 'delete'): void { + /** The body of `ObjectQL#listDatasourceDefs`; the method's docblock there is its contract. */ + static listDatasourceDefs(this: Pick): DatasourceDef[] { + return Array.from(this.datasourceDefs, ([name, def]) => ({ + name, + ...(def.driver !== undefined ? { driver: def.driver } : {}), + ...(def.schemaMode !== undefined ? { schemaMode: def.schemaMode } : {}), + ...(def.external ? { external: { ...def.external } } : {}), + })); + } + + /** The body of `ObjectQL#markDatasourceUnavailable`; the method's docblock there is its contract. */ + static markDatasourceUnavailable(this: Pick, info: { name: string; kind: DatasourceUnavailableKind; publicDetail?: string }): void { + if (!info?.name) return; + this.unavailableDatasources.set(info.name, { + kind: info.kind, + ...(info.publicDetail ? { publicDetail: info.publicDetail } : {}), + }); + } + + /** The body of `ObjectQL#clearDatasourceUnavailable`; the method's docblock there is its contract. */ + static clearDatasourceUnavailable(this: Pick, name: string): void { + this.unavailableDatasources.delete(name); + } + + /** The body of `ObjectQL#getDefaultDriverName`; the method's docblock there is its contract. */ + static getDefaultDriverName(this: Pick): string | undefined { + return this.defaultDriver ?? undefined; + } + + /** The body of `ObjectQL#listUnavailableDatasources`; the method's docblock there is its contract. */ + static listUnavailableDatasources(this: Pick): Array<{ name: string } & DatasourceUnavailableInfo> { + return Array.from(this.unavailableDatasources, ([name, info]) => ({ name, ...info })); + } + + /** + * Write gate — Gate 3 of ADR-0015 §5.3. + * + * Blocks insert/update/delete against a federated datasource + * (`schemaMode !== 'managed'`) unless BOTH the datasource opts in + * (`external.allowWrites`) AND the object opts in (`external.writable`). + * Managed datasources (the common case, including the absence of any + * definition) are unaffected. + */ + static assertWriteAllowed(this: Pick, objectName: string, operation: 'insert' | 'update' | 'delete'): void { const object = this._registry.getObject(objectName) as any; const dsName = object?.datasource; if (!dsName || dsName === 'default') return; @@ -346,3 +509,4 @@ export function assertWriteAllowed(this: Pick, objectName: string): { +/** The bodies of the "Crypto, secrets, internal fields, schema and datasource routing, init" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class SecretsAndRoutingFacet { + /** The body of `ObjectQL#setCryptoProvider`; the method's docblock there is its contract. */ + static setCryptoProvider(this: Pick, provider: ICryptoProvider): void { + this.cryptoProvider = provider; + this.logger.info('CryptoProvider configured for secret fields'); + // A listener is a re-arm, never part of this call's contract: one that + // throws must not fail the host's composition root, and must not stop the + // listeners behind it from re-arming. + for (const listener of [...this.cryptoProviderListeners]) { + try { + listener(); + } catch (err) { + this.logger.warn('CryptoProvider registration listener failed', { + error: (err as Error)?.message ?? String(err), + }); + } + } + } + + /** The body of `ObjectQL#getKeyedDigest`; the method's docblock there is its contract. */ + static getKeyedDigest(this: Pick): ((plain: string) => Promise) | undefined { + const provider = this.cryptoProvider; + return provider ? (plain: string) => provider.keyedDigest(plain) : undefined; + } + + /** The body of `ObjectQL#onCryptoProviderChange`; the method's docblock there is its contract. */ + static onCryptoProviderChange(this: Pick, listener: () => void): () => void { + this.cryptoProviderListeners.add(listener); + return () => { + this.cryptoProviderListeners.delete(listener); + }; + } + + /** The body of `ObjectQL#setTenancyPostureProvider`; the method's docblock there is its contract. */ + static setTenancyPostureProvider(this: Pick, provider: () => string | undefined): void { + this.tenancyPostureProvider = provider; + } + + /** The body of `ObjectQL#getSchema`; the method's docblock there is its contract. */ + static getSchema(this: Pick, objectName: string): ServiceObject | undefined { + return this._registry.getObject(objectName); + } + + /** + * Resolve any object identifier to the physical storage name used by drivers. + * + * Accepts the canonical short name (e.g., 'account') or, for explicit + * cross-package disambiguation, the canonical object name (e.g., 'account'). The result is + * the physical table name derived via `StorageNameMapping.resolveTableName`. + * + * One name space with the data door: the target resolves ONLY through the + * registry. A name the registry does not resolve is refused with the door's + * own `OBJECT_NOT_FOUND` 404 (`objectNotFoundError`), never handed to the + * driver as a raw table name. Before this, every in-process guard keyed by a + * registered object name could be stepped around by naming the target some + * other way, and an in-process caller (a sandboxed body, an action handler, + * a hook) read what the door refused to serve. + * + * Platform code that must address storage without a registry entry has a + * declared path a body cannot reach: the driver itself + * (`datasource(name)`, `getDriverForObject(name)`), held by host code only. + */ + static resolveObjectName(this: Pick, name: string): string { + const schema = this._registry.getObject(name); + if (schema) { + return StorageNameMapping.resolveTableName(schema); + } + throw objectNotFoundError(name); + } + + /** The body of `ObjectQL#judgeFilter`; the method's docblock there is its contract. */ + static judgeFilter(this: Pick, objectName: string, where: EngineQueryOptions['where'], options?: EngineFilterJudgementOptions): EngineFilterJudgement { + const object = this._registry.getObject(objectName) ? this.resolveObjectName(objectName) : objectName; + return judgeWhereAdmission( + object, + options?.operation ?? 'find', + where, + this._registry.getObject(object), + options?.context, + this.relatedSchemaOf, + ); + } + + /** + * Is `objectName` an append-only system ledger? — the #5351 carve-out's + * discriminator, and deliberately a property of the object's DECLARATION + * (`lifecycle.class`) rather than of the deployment's routing. + * + * Why the declaration and not "was it routed by step 3": an audit ledger + * pinned to its own datasource by an explicit `datasource:` binding, or by a + * `datasourceMapping` rule, is the same append-only compliance ledger with + * the same reason to be carved out — the routing mechanism is an operator's + * choice, the class is the author's statement about what the data IS. Judging + * by the mechanism would make the carve-out depend on which of three + * equivalent configurations a deployment happened to use. + */ + static isSystemLedgerObject(this: Pick, objectName: string): boolean { + const lifecycleClass = ( + this._registry.getObject(objectName) as { lifecycle?: { class?: string } } | undefined + )?.lifecycle?.class; + return lifecycleClass !== undefined && ObjectQL.SYSTEM_LEDGER_LIFECYCLE_CLASSES.has(lifecycleClass); + } + + /** + * WHERE does `objectName`'s data live, and WHICH step decided — the single + * implementation of the resolution order documented on {@link getDriver}. + * + * Split out for #5288 so the order exists exactly once. It had two readers + * with two different answers: `getDriver` (all five steps) and every caller + * that only needed the NAME, which read `object.datasource` — step 1 of five. + * A second, shorter copy of a routing order is the failure + * `resolveMappedDatasource` (#4462) was extracted to prevent, and it fails the + * same way: silently, in whichever direction the copy is short. + * + * Steps 1-2 answer even when the named datasource has no registered driver: + * they are BINDINGS, and `getDriver` stops there (it throws rather than + * falling through — #4462). Steps 3-5 answer only when their driver is + * registered, because that registration is what turns each of them on. + * + * `undefined` means nothing routes the object anywhere: no binding, and no + * global default driver to fall back to. + */ + static resolveDatasourceBinding(this: Pick, objectName: string): { datasource: string; via: 'explicit' | 'mapping' | 'lifecycle' | 'package' | 'default'; /** Owning package, on `via: 'package'` only — for the debug line. */ @@ -108,3 +251,15 @@ export function resolveDatasourceBinding(this: Pick, objectName: string): string | undefined { + const binding = this.resolveDatasourceBinding(objectName); + return binding && binding.via !== 'default' ? binding.datasource : undefined; + } + + /** The body of `ObjectQL#resolveMappedDatasource`; the method's docblock there is its contract. */ + static resolveMappedDatasource(this: Pick, objectName: string): string | null { + return this.resolveDatasourceFromMapping(objectName, this._registry.getObject(objectName)); + } +} diff --git a/packages/objectql/src/engine-security-resolvers.ts b/packages/objectql/src/engine-security-resolvers.ts index b4f5da6545e..460337c8c93 100644 --- a/packages/objectql/src/engine-security-resolvers.ts +++ b/packages/objectql/src/engine-security-resolvers.ts @@ -3,51 +3,219 @@ /** * [#29986 codemod] The * "Registration seams, readable-ids security, formula and default permissions" - * section of `ObjectQL`: the bodies of `refuseUncomposableReadableIdsClasses`, - * `resolveFormulaRelated` moved out of `engine.ts` whole by + * section of `ObjectQL`: the bodies of `registerHeldFileResolver`, + * `registerFieldOwnedFileReadAuthorizer`, `registerWriteGateProbe`, + * `registerWriteWallVerdict`, `registerEffectiveObjectPermissionsResolver`, + * `registerInsertRecomputeListener`, `composeReadableIdsAtBoundary`, + * `refuseUncomposableReadableIdsClasses`, `uncomposableReadableIdsParent`, + * `resolveFormulaRelated`, `serveFormulaField`, `formulaOptionLabels`, + * `formulaFaultSink` moved out of `engine.ts` whole by * `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link SecurityResolversFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link SecurityResolversHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * * The module imports `engine.ts` for types only, so the two modules form no * evaluation cycle. `index.ts` re-exports nothing from this module. */ import { referenceTargetOf } from '@objectstack/spec/data'; -import type { EngineQueryOptions } from '@objectstack/spec/data'; -import { describeReadableIdsLeaf, readableIdsRefusalError } from './filter-comparand-shape.js'; +import type { EngineQueryOptions, QueryAST } from '@objectstack/spec/data'; +import { collectComposableReadableIds, describeReadableIdsLeaf, readableIdsRefusalError } from './filter-comparand-shape.js'; import type { ComposableReadableIdsSite } from './filter-comparand-shape.js'; import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel'; -import { undeclaredRelatedFields } from './engine-top-level.js'; -import type { FormulaPlanEntry, FormulaRelated } from './engine-top-level.js'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import type { EvalOptionLabels } from '@objectstack/formula'; +import { readerOptionLabels } from './formula-option-labels.js'; +import { applyFormulaPlan, planFormulaProjection, undeclaredRelatedFields } from './engine-top-level.js'; +import type { + FieldOwnedFileReadAuthorizer, FormulaFaultSink, FormulaPlanEntry, FormulaRelated, HeldFileResolver, + ReadableIdsChain, WriteWallVerdict, +} from './engine-top-level.js'; import type { ObjectQL } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link SecurityResolversFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface SecurityResolversHost { + _effectiveObjectPermissionsResolver: ObjectQL['_effectiveObjectPermissionsResolver']; + _fieldOwnedFileReadAuthorizer: ObjectQL['_fieldOwnedFileReadAuthorizer']; + _heldFileResolver: ObjectQL['_heldFileResolver']; readonly _registry: ObjectQL['_registry']; + _writeGateProbe: ObjectQL['_writeGateProbe']; + _writeWallVerdict: ObjectQL['_writeWallVerdict']; + composeReadableIdsChain: ObjectQL['composeReadableIdsChain']; + formulaFaultSink: ObjectQL['formulaFaultSink']; readonly formulaHopReads: ObjectQL['formulaHopReads']; + formulaOptionLabels: ObjectQL['formulaOptionLabels']; + hasObjectMiddleware: ObjectQL['hasObjectMiddleware']; + readonly i18nService: ObjectQL['i18nService']; + readonly insertRecomputeListeners: ObjectQL['insertRecomputeListeners']; + readonly logger: ObjectQL['logger']; + permissionResolution: ObjectQL['permissionResolution']; + readonly readableIdsSecurityReported: ObjectQL['readableIdsSecurityReported']; + reportFormulaFault: ObjectQL['reportFormulaFault']; + resolveDatasourceBinding: ObjectQL['resolveDatasourceBinding']; + resolveFormulaPermissions: ObjectQL['resolveFormulaPermissions']; + resolveFormulaRelated: ObjectQL['resolveFormulaRelated']; resolveOneHopRelated: ObjectQL['resolveOneHopRelated']; + resolveReadableIdsSecurity: ObjectQL['resolveReadableIdsSecurity']; uncomposableReadableIdsParent: ObjectQL['uncomposableReadableIdsParent']; } -/** The body of `ObjectQL#refuseUncomposableReadableIdsClasses`; the method's docblock there is its contract. */ -export function refuseUncomposableReadableIdsClasses(this: Pick, object: string, operation: string, sites: readonly ComposableReadableIdsSite[]): string[] { +/** The bodies of the "Registration seams, readable-ids security, formula and default permissions" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class SecurityResolversFacet { + /** The body of `ObjectQL#registerHeldFileResolver`; the method's docblock there is its contract. */ + static registerHeldFileResolver(this: Pick, fn: HeldFileResolver): void { + this._heldFileResolver = fn; + this.logger.debug('Registered held-file resolver for sys_file hydration'); + } + + /** The body of `ObjectQL#registerFieldOwnedFileReadAuthorizer`; the method's docblock there is its contract. */ + static registerFieldOwnedFileReadAuthorizer(this: Pick, fn: FieldOwnedFileReadAuthorizer): void { + this._fieldOwnedFileReadAuthorizer = fn; + this.logger.debug('Registered field-owned file read authorizer for refused sys_file hydration'); + } + + /** The body of `ObjectQL#registerWriteGateProbe`; the method's docblock there is its contract. */ + static registerWriteGateProbe(this: Pick, fn: ( + object: string, + operation: 'insert' | 'update', + context: unknown, + data: unknown, + ) => Promise): void { + this._writeGateProbe = fn; + this.logger.debug('Registered write-gate probe for validate() relationship resolution'); + } + + /** The body of `ObjectQL#registerWriteWallVerdict`; the method's docblock there is its contract. */ + static registerWriteWallVerdict(this: Pick, fn: WriteWallVerdict): void { + this._writeWallVerdict = fn; + this.logger.debug('Registered write-wall verdict for validate() organization judgement'); + } + + /** The body of `ObjectQL#registerEffectiveObjectPermissionsResolver`; the method's docblock there is its contract. */ + static registerEffectiveObjectPermissionsResolver(this: Pick, fn: (context: unknown) => Promise): void { + this._effectiveObjectPermissionsResolver = fn; + this.logger.debug('Registered effective object-permission resolver for current_user.can()'); + } + + /** The body of `ObjectQL#registerInsertRecomputeListener`; the method's docblock there is its contract. */ + static registerInsertRecomputeListener(this: Pick, id: string, fn: (event: { object: string; row: Record; fields: string[] }) => void | Promise): void { + this.insertRecomputeListeners.set(id, fn); + this.logger.debug('Registered insert recompute listener', { id }); + } + + /** + * [ADR-0139 D2, D5] The executor boundary: every `ReadableIds` leaf in the + * FINAL `where` of a read is replaced by its composed form immediately + * before the driver call — after the comparand doors, the middleware chain + * and the before-hooks, so a leaf the caller, a middleware or a hook wrote + * meets the same composition. Returns `ast` by reference when it holds no + * leaf, which is every read today; otherwise a shallow copy whose `where` is + * rewritten copy-on-write (the hooks' `input.ast` is never edited). + * + * Per parent object `P`, once per call, in this order: + * + * 1. **D5's two remaining classes are refused, by name** (`INTERNAL_ERROR` + * / 500), before the security service is asked anything: a parent an + * object-scoped read middleware narrows ({@link hasObjectMiddleware}); a + * parent bound to another datasource than `object` + * ({@link resolveDatasourceBinding}). + * 2. **A parent the registry does not hold** ⇒ `$in: []`, unasked. + * 3. **The security service** — absent or unusable ⇒ `$in: []`. + * 4. **`canReadObject(P)`** (its `explain` fallback when the method is + * absent) — false or a throw ⇒ `$in: []`. + * 5. **`getReadFilter(P)`** — `undefined` ⇒ composed, no row restriction; + * the deny sentinel `RLS_DENY_FILTER`, by identity, or a throw ⇒ + * `$in: []`; any other filter ⇒ composed with it as the scope, marked + * `'policy'` and otherwise passed exactly as `P`'s own read passes its + * injected scope. The tenant wall is the driver's (D3). + * + * [ADR-0139 D5.1, as amended] A **delegated (on-behalf-of) caller** is + * composed, not refused: both halves are asked with the caller's context + * exactly as it stands — `onBehalfOf` intact — and the security service + * answers each half for the delegated principal, which under ADR-0090 D10 + * is the INTERSECTION of the principal's leg and the delegator's. The + * engine composes what it is answered and derives no second intersection + * of its own: it has one context, the delegated one, and the delegator's + * grants are the security service's to resolve. A service that cannot + * answer that context answers the deny sentinel, and step 5 turns it into + * `$in: []`, closed. + * + * [ADR-0139 D6, as amended — the nested master chain] A scope the service + * answers in step 5 may itself carry a leaf: a parent whose own read scope + * is derived from ITS master (a `controlled_by_parent` detail whose master + * is `controlled_by_parent` too) is scoped `{ master_fk: { $in: + * ReadableIds.of(G) } }`, and a leaf handed to a driver uncomposed is + * refused there (D3), never read as "no restriction". So every leaf a + * composed scope carries is composed the same way, BEFORE the leaf that + * encloses it — the same five steps, the same caller context at every level, + * recursively ({@link composeReadableIdsChain}). The driver then receives one + * composed leaf whose scope holds composed leaves, and compiles one statement + * with the subqueries nested. Each parent is asked once per read, at + * whichever level it first appears; a chain that returns to a parent already + * on its branch, or runs deeper than {@link READABLE_IDS_MAX_CHAIN_DEPTH} + * scopes, is refused by name (D5's envelope), not composed partway. + * + * Nothing is memoised across calls. `aggregate`'s `having` and its + * aggregations' own filters are judged here too, so a leaf a middleware + * wrote there is refused rather than evaluated in memory. + */ + static async composeReadableIdsAtBoundary(this: Pick, object: string, operation: 'find' | 'findOne' | 'count' | 'aggregate', ast: A, context: ExecutionContext | undefined): Promise { + if (operation === 'aggregate') { + collectComposableReadableIds(object, operation, ast.having, 'having'); + const aggregations = Array.isArray(ast.aggregations) ? ast.aggregations : []; + aggregations.forEach((agg, i) => { + collectComposableReadableIds(object, operation, (agg as { filter?: unknown } | undefined)?.filter, `aggregations[${i}].filter`); + }); + } + const sites = collectComposableReadableIds(object, operation, ast.where, 'where'); + if (sites.length === 0) return ast; + + const security = this.resolveReadableIdsSecurity(); + if (security.kind !== 'usable' && !this.readableIdsSecurityReported.has(security.kind)) { + this.readableIdsSecurityReported.add(security.kind); + this.logger.warn( + `[ObjectQL] ${operation}('${object}') composes a ReadableIds leaf with no usable "security" service ` + + `(${security.kind === 'absent' ? 'none is registered' : security.why}): every such leaf is answered ` + + '$in: [], no row of its parent, fail-closed. Register a security service for the leaf to serve rows. ' + + 'Reported once for this engine; later leaves are answered the same way.', + ); + } + const chain: ReadableIdsChain = { object, operation, security, context, answers: new Map() }; + const where = await this.composeReadableIdsChain(chain, ast.where, sites, []); + return { ...ast, where }; + } + + /** + * [ADR-0139 D5] Refuse D5's two remaining classes (D5.2 and D5.3), by name, + * and return the distinct parent objects of `sites` the security service is + * asked about, in walk order (every one but a parent the registry does not + * hold). Reads the registry and the middleware registrations only — no + * security call, and no reading of the caller's context: a delegated + * (on-behalf-of) caller is no longer a class of its own (D5.1, as amended), + * it is composed like any other through the security service's answers. + */ + static refuseUncomposableReadableIdsClasses(this: Pick, object: string, operation: string, sites: readonly ComposableReadableIdsSite[]): string[] { const firstSite = new Map(); for (const site of sites) if (!firstSite.has(site.leaf.object)) firstSite.set(site.leaf.object, site); const refuse = (site: ComposableReadableIdsSite, why: string): Error => readableIdsRefusalError( @@ -76,8 +244,49 @@ export function refuseUncomposableReadableIdsClasses(this: Pick, schema: any, plan: readonly FormulaPlanEntry[], rows: ReadonlyArray | undefined | null>, context: unknown, query?: EngineQueryOptions): Promise { + /** + * [ADR-0139 D5.2 / D5.3] Which of D5's two remaining classes a leaf over + * `parent` in a read of `object` falls in, or `undefined` when it falls in + * neither — the ONE predicate behind both the read-time refusal + * ({@link refuseUncomposableReadableIdsClasses}) and the boot-time refusal of + * a `controlled_by_parent` detail over such a master + * ({@link findUnservableControlledByParentMasters}), so the two rules cannot + * drift. D5.2 is judged first, as the read path always has. + */ + static uncomposableReadableIdsParent(this: Pick, object: string, parent: string): | { class: 'object-scoped-read-middleware' } + | { class: 'other-datasource'; objectDatasource: string | undefined; parentDatasource: string | undefined } + | undefined { + if (this.hasObjectMiddleware(parent)) return { class: 'object-scoped-read-middleware' }; + const objectDatasource = this.resolveDatasourceBinding(object)?.datasource; + const parentDatasource = this.resolveDatasourceBinding(parent)?.datasource; + if (parentDatasource !== objectDatasource) return { class: 'other-datasource', objectDatasource, parentDatasource }; + return undefined; + } + + /** + * [#24197] The related rows a formula plan reads ONE hop through a + * reference field, resolved for a whole page of `rows` at once — the + * formula half of {@link resolveOneHopRelated}, read under the CALLER's + * own authority. + * + * ## What it costs + * + * Nothing for a plan that traverses nothing (no query, no closure). For one + * that does: one batched read per reference field per call — per PAGE of a + * `find` (the rows the driver returned, after `limit` / `offset`), per row of + * a `findOne`, per write response — never one per row. A formula whose + * traversal cannot be served (`hopConflict`) asks for nothing. + * + * ## One hop, and only one + * + * A `find` that IS a hop read ({@link formulaHopReads}) serves no hop of its + * own: the related object's own formulas over its own columns are computed, + * one that reads through a reference of ITS own reads `null` and says so. + * That is the declared depth (a second hop is not loaded — the lint refuses + * one written out, this refuses one reached through a formula), and it is + * what keeps two objects whose titles read each other from recursing. + */ + static async resolveFormulaRelated(this: Pick, schema: any, plan: readonly FormulaPlanEntry[], rows: ReadonlyArray | undefined | null>, context: unknown, query?: EngineQueryOptions): Promise { const wanted = new Map>(); for (const entry of plan) { if (entry.hopConflict) continue; @@ -118,3 +327,37 @@ export async function resolveFormulaRelated(this: Pick undefined; return { bindings: (row) => lookup(row), undeclared }; } + + /** The body of `ObjectQL#serveFormulaField`; the method's docblock there is its contract. */ + static async serveFormulaField(this: Pick, objectName: string, record: Record, field: string, context?: ExecutionContext): Promise { + const schema = this._registry.getObject(objectName); + const { plan } = planFormulaProjection(schema, [field]); + if (plan.length === 0) return undefined; + const scratch: Record = { ...record }; + applyFormulaPlan(plan, [scratch], context, await this.resolveFormulaPermissions( + objectName, plan, [scratch], context, this.permissionResolution(context), + ), this.formulaFaultSink(objectName), await this.resolveFormulaRelated(schema, plan, [scratch], context), + this.formulaOptionLabels(schema, plan, context)); + return scratch[field]; + } + + /** + * [#26608] The option labels `label(record.)` reads on a read or a + * write response of `schema`'s object: in the reader's locale + * (`ExecutionContext.locale`, which the request's `Accept-Language` sets), + * translated through the bridged i18n service as `GET /meta/object/:name` + * serves the object to the same reader — see {@link readerOptionLabels}. + * `undefined` when no formula of `plan` calls `label()`. + */ + static formulaOptionLabels(this: Pick, schema: unknown, plan: FormulaPlanEntry[], context: { locale?: string } | undefined): EvalOptionLabels | undefined { + return readerOptionLabels(schema, plan, context?.locale, this.i18nService); + } + + /** + * [#22019] The sink a read or a write response of `object` hands to + * {@link applyFormulaPlan}, bound to the object so a report names the pair. + */ + static formulaFaultSink(this: Pick, object: string): FormulaFaultSink { + return (field, error) => this.reportFormulaFault(object, field, error); + } +} diff --git a/packages/objectql/src/engine-summaries-and-expansion.ts b/packages/objectql/src/engine-summaries-and-expansion.ts index 11d64562504..189f85abeea 100644 --- a/packages/objectql/src/engine-summaries-and-expansion.ts +++ b/packages/objectql/src/engine-summaries-and-expansion.ts @@ -3,47 +3,127 @@ /** * [#29986 codemod] The * "Summary indexes, related-record expansion, file and reference hydration" - * section of `ObjectQL`: the bodies of `buildSummaryIndex`, - * `initializeSummaryFields` moved out of `engine.ts` whole by - * `scripts/codemod-split-engine.mjs`. + * section of `ObjectQL`: the bodies of `invalidateSummaryIndex`, + * `buildSummaryIndex`, `ensureSummaryIndexes`, `getSummaryDescriptors`, + * `summaryForeignKeyColumns`, `getOwnedSummaryDescriptors`, + * `initializeSummaryFields`, `expandRelatedRecords` moved out of `engine.ts` + * whole by `scripts/codemod-split-engine.mjs`. * - * A MOVE, not a redesign. Each function below is the body of the `ObjectQL` + * A MOVE, not a redesign. {@link SummariesAndExpansionFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` * method of the same name exactly as it stood: not re-indented, not - * formatted, not edited. The method stays on the class with its docblock and - * signature, so the published `dist/*.d.ts`, an instance spy and a subclass - * override are what they were; the docblock there is the contract. + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. * * The bodies reach the class through {@link SummariesAndExpansionHost}: exactly the - * `ObjectQL` members they read, private ones included. Each method hands its + * `ObjectQL` members they read, private ones included. Each stub hands its * body one literal of the members that body reads, plus a `.call`: methods go * as closures that call them on the engine at call time, so a subclass * override or an instance spy is still the one reached; fields and accessors - * go as GETTERS, read when the body reads them, as they were on the class. + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. * - * The module imports `engine.ts` for types only, so the two modules form no - * evaluation cycle. `index.ts` re-exports nothing from this module. + * `ObjectQL` here is `ObjectQLStatics` (imported AS `ObjectQL`): forwarders to the + * statics the bodies read, private ones included, which the class fills in a + * static block; the class as a type is `ObjectQLClass`. + * + * Values that stayed in `engine.ts` are imported from it (lever (a) of + * ruling 6591753301); every use is inside a function body, so the import + * cycle is never read at evaluation time. `index.ts` re-exports nothing from + * this module. */ -import { referenceCarrierOf } from '@objectstack/spec/data'; -import type { ServiceObject } from '@objectstack/spec/data'; +import { REFERENCE_VALUE_TYPES, referenceCarrierOf, referenceTargetOf } from '@objectstack/spec/data'; +import type { QueryAST, ServiceObject } from '@objectstack/spec/data'; +import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; import { summaryEmptySetValue } from './summary-aggregate.js'; import type { SummaryDescriptor } from './summary-aggregate.js'; -import type { ObjectQL } from './engine.js'; +import { ENGINE_WIRE_ONLY_SLOTS, WIRE_ONLY_CANONICAL_SHAPES } from './engine-top-level.js'; +import { ObjectQLStatics as ObjectQL } from './engine.js'; +import type { ObjectQL as ObjectQLClass } from './engine.js'; /** - * The `ObjectQL` members the functions below read — every one of them, and - * no other. Each is typed AS the class member (an indexed access on the - * class), so a change to a member's signature on the class is a change here - * too. `readonly` members are the getters. + * The `ObjectQL` members the bodies of {@link SummariesAndExpansionFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. */ export interface SummariesAndExpansionHost { - readonly _registry: ObjectQL['_registry']; - getOwnedSummaryDescriptors: ObjectQL['getOwnedSummaryDescriptors']; - readonly logger: ObjectQL['logger']; + readonly _registry: ObjectQLClass['_registry']; + buildSummaryIndex: ObjectQLClass['buildSummaryIndex']; + ensureSummaryIndexes: ObjectQLClass['ensureSummaryIndexes']; + expandRelatedRecords: ObjectQLClass['expandRelatedRecords']; + find: ObjectQLClass['find']; + getOwnedSummaryDescriptors: ObjectQLClass['getOwnedSummaryDescriptors']; + getSummaryDescriptors: ObjectQLClass['getSummaryDescriptors']; + invalidateSummaryIndex: ObjectQLClass['invalidateSummaryIndex']; + readonly logger: ObjectQLClass['logger']; + serveReferenceNames: ObjectQLClass['serveReferenceNames']; + summaryIndex: ObjectQLClass['summaryIndex']; + summaryIndexByParent: ObjectQLClass['summaryIndexByParent']; + summaryIndexRevision: ObjectQLClass['summaryIndexRevision']; } -/** The body of `ObjectQL#buildSummaryIndex`; the method's docblock there is its contract. */ -export function buildSummaryIndex(this: Pick): { +/** The bodies of the "Summary indexes, related-record expansion, file and reference hydration" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class SummariesAndExpansionFacet { + /** Invalidate the cached roll-up summary index (call when metadata changes). */ + static invalidateSummaryIndex(this: Pick): void { + this.summaryIndex = null; + this.summaryIndexByParent = null; + } + + /** Scan all registered objects for `summary` fields and index them BOTH ways + * — by the child object they aggregate and by the parent object that owns + * them — resolving the child→parent FK field. One scan, two views of the + * identical descriptor objects, so the two indexes can never disagree about + * which roll-ups exist. + * + * [#9154] The registry read PROPAGATES. It used to be + * `try { (registry as any).getAllObjects?.() ?? [] } catch { [] }`, which + * answered a read that could not run with an invented *"no object declares a + * roll-up"* — the same shape #8895 ruled on (*discriminate or propagate*) + * and #9002 removed from the two delete-cascade seams, and there is no + * benign failure class to discriminate here either: an unreadable registry + * is never truthfully "no roll-ups". An empty index means + * {@link recomputeSummaries} recomputes nothing after every insert / update + * / delete, so every parent summary field keeps a stale value while every + * write reports success and nothing is logged. + * + * Both halves of the swallow are gone, not just the `catch`: the optional + * call `?.()` absorbed a registry that does not implement `getAllObjects` at + * all — the same structural omission a test double really did ship, and one + * that is invisible precisely because it never throws. A plain call fails + * loudly on it. + * + * ⚠️ This is a STRUCTURAL close, not a live defect: `SchemaRegistry`'s + * `getAllObjects()` is a walk over in-memory `Map`s calling `resolveObject()` + * — which returns `undefined` on every failure branch it models — over a + * fold that is spreads and comparisons. No I/O, no driver, no `throw` on the + * measured path. The pin lives in + * `engine-summary-index-registry-read-failure.test.ts`. + * + * [#19082] The SECOND way this one function invented *"nothing to + * recompute"* — a different input, the same outcome. The child→parent FK + * was resolved by comparing the child field's `reference` carrier to the + * parent's name, so a carrier no reader can read (a non-string where + * `FieldSchema.reference` declares an optional string) compared false + * against every name, `fkField` stayed unset, and the `continue` below + * dropped a DECLARED roll-up out of both indexes with nothing logged. + * + * ⛔ The fix is NOT a looser comparison — that would trade a silent + * stall for a MIS-MATCHED foreign key, which is more expensive. The + * carrier is read through the one arbiter and the skip now SAYS SO, at + * `error` (a persisted summary silently stops tracking its children while + * every write keeps reporting success — the durability class). Absence and + * every readable carrier behave exactly as before; the resolution rule + * itself is deliberately left where PR #18503's C2 boundary put it. Pinned + * in `engine-summary-index-unreadable-carrier.test.ts`, both directions. */ + static buildSummaryIndex(this: Pick): { byChild: Map; byParent: Map; } { @@ -173,8 +253,120 @@ export function buildSummaryIndex(this: Pick, object: string, record: any): any { + /** + * Ensure both roll-up indexes are present and current. Split out of + * {@link getSummaryDescriptors} so the parent-side view (#5749) shares the + * exact same staleness rule instead of re-deriving one. + * + * ## [#9154] A failed build leaves NO cache entry — *a poisoned cache entry + * must not survive the read that poisoned it* + * + * This is the limb that made the swallow removed from + * {@link buildSummaryIndex} worse than its two #9002 siblings. Those invented + * an answer ONCE, per delete. Here the answer is MEMOIZED and stamped with the + * registry's current `objectRevision` — so one failed read did not degrade one + * write, it installed an empty index that every subsequent write then read as + * a measured answer. And `objectRevision` moves only on a metadata MUTATION + * (`registerObject`, `unregisterObject`, `unregisterObjectsByPackage`, + * `removeObjectOverlay`, `invalidate`, `invalidateAll`, `reset`) — never on a + * data write. A steady-state deployment performs none of those, so the + * invented emptiness outlived its cause for the whole process lifetime, + * ending only at a restart or at an unrelated publish. The failure was not + * "briefly wrong"; it was wrong until an unrelated event. + * + * Two things hold the invariant, and both are load-bearing — do not reorder: + * + * 1. the build runs to completion into a LOCAL before anything is published + * to `this`, so a throw cannot leave a half-written pair of indexes, and + * the revision stamp is written LAST — an unstamped failure is retried on + * the very next call rather than being remembered as an answer; + * 2. the `catch` below clears whatever was cached and resets the stamp before + * rethrowing. Behaviourally that is a no-op today (nothing reads the + * fields without coming through here first, and the stale entry would be + * rebuilt anyway) — it is here so the guarantee is stated in code instead + * of resting on the survey in (1). ⛔ It swallows nothing: the error is + * rethrown unchanged, and `engine-summary-index-registry-read-failure.test.ts` + * pins both the propagation and the retry. + */ + static ensureSummaryIndexes(this: Pick): void { + // Rebuild whenever the REGISTRY's object set has moved since the index was + // built — not only when someone remembered to call + // `invalidateSummaryIndex`. That single site (`registerApp`) is bypassed by + // the runtime publish path, which registers straight into the registry + // (`protocol.saveMetaItem` → `registry.registerObject`). A kernel that had + // already done one write — publishing itself writes `sys_metadata` — held an + // index built before the new object existed, so every child write of a + // freshly published roll-up skipped the recompute and the parent read null + // until the process restarted. That is exactly how an AI-built app's + // "已完成任务数" shipped empty over correct metadata (cloud#970). + const revision = (this._registry as unknown as { objectRevision?: number })?.objectRevision; + const stale = typeof revision === 'number' && revision !== this.summaryIndexRevision; + if (!this.summaryIndex || !this.summaryIndexByParent || stale) { + let built: { byChild: Map; byParent: Map }; + try { + built = this.buildSummaryIndex(); + } catch (err) { + // [#9154] Nothing measured came back, so nothing is remembered — not + // even the entry that was already here, which the staleness test above + // has just judged out of date. The next call rebuilds from scratch. + this.invalidateSummaryIndex(); + this.summaryIndexRevision = -1; + throw err; + } + this.summaryIndex = built.byChild; + this.summaryIndexByParent = built.byParent; + if (typeof revision === 'number') this.summaryIndexRevision = revision; + } + } + + /** Roll-up descriptors for summaries that aggregate `childObject` — i.e. the + * ones a write to `childObject` must recompute. Semantics unchanged. */ + static getSummaryDescriptors(this: Pick, childObject: string): SummaryDescriptor[] { + this.ensureSummaryIndexes(); + return this.summaryIndex!.get(childObject) ?? []; + } + + /** [#26950] The child-side foreign-key columns the roll-ups over + * `childObject` read: what a predicate write must read off its matched rows, + * before it writes, to hand {@link recomputeSummaries} the parents to + * recompute. Empty when no roll-up aggregates `childObject`. */ + static summaryForeignKeyColumns(this: Pick, childObject: string): string[] { + return [...new Set(this.getSummaryDescriptors(childObject).map((desc) => desc.fkField))]; + } + + /** The body of `ObjectQL#getOwnedSummaryDescriptors`; the method's docblock there is its contract. */ + static getOwnedSummaryDescriptors(this: Pick, parentObject: string): SummaryDescriptor[] { + this.ensureSummaryIndexes(); + return this.summaryIndexByParent!.get(parentObject) ?? []; + } + + /** + * Seed the roll-up `summary` fields a freshly-created row owns (#5749). + * + * `recomputeSummaries` only ever visits parents named by a child write, so a + * parent that has NEVER had a child is never visited and its summary column + * keeps whatever insert put there — `null`. Delete the last child and the + * parent DOES get visited (via `previous`) and lands on 0. Same logical state, + * two different values: `filter ["task_count","=",0]` silently skipped every + * parent that never had a child, and so did sorting, GROUP BY and any formula + * reading the field (null propagation). + * + * The fix is at the producer: write the empty-collection value at create time, + * so `count`/`sum` start at 0 and only ever move to another number. + * `min`/`max`/`avg` have no empty-set value and deliberately stay `null` — + * {@link summaryEmptySetValue} is the single list both this and the recompute + * fallback read. + * + * Author-supplied values are never overwritten. The `!= null` test matches + * {@link applyFieldDefaults} exactly (#2706): on INSERT an explicit `null` is + * "no value supplied", any real value — including a deliberate 0 or a seeded + * count — is respected. Runs before the `beforeInsert` hooks for the same + * reason defaults do, so a hook still has the final say. + * + * Existing rows are untouched: this is create-time only, so parents already + * stored with `null` stay `null` until a child write recomputes them. + */ + static initializeSummaryFields(this: Pick, object: string, record: any): any { const descriptors = this.getOwnedSummaryDescriptors(object); if (descriptors.length === 0) return record; if (!record || typeof record !== 'object' || Array.isArray(record)) return record; @@ -188,3 +380,261 @@ export function initializeSummaryFields(this: Pick, objectName: string, records: any[], expand: Record, depth: number = 0, execCtx?: ExecutionContext): Promise { + if (!records || records.length === 0) return records; + if (depth >= ObjectQL.MAX_EXPAND_DEPTH) return records; + + const objectSchema = this._registry.getObject(objectName); + // If no schema registered, skip expand — return raw data + if (!objectSchema || !objectSchema.fields) return records; + + for (const [fieldName, nestedAST] of Object.entries(expand)) { + // [#4371] The nested AST is caller-authored too: a wire spelling inside + // `expand: { rel: { sort } }` used to be silently dropped exactly like + // the top-level bag (this loop reads only canonical keys). Same + // rejection, scoped to the four wire-only pairs — the nested shape is a + // QueryAST, not an option bag, so the option-key gate does not apply. + if (nestedAST && typeof nestedAST === 'object') { + for (const slot of ENGINE_WIRE_ONLY_SLOTS) { + for (const alias of slot.aliases) { + if ((nestedAST as Record)[alias] != null) { + throw new Error( + `expand['${fieldName}'] on '${objectName}' does not accept '${alias}': it is a wire ` + + `spelling of '${slot.canonical}', folded by the RPC/protocol layer — a direct engine ` + + `call bypasses that fold, so the value would be silently dropped, not applied. Pass ` + + `'${slot.canonical}' (${WIRE_ONLY_CANONICAL_SHAPES[slot.canonical] ?? 'the canonical QueryAST shape'}) instead.`, + ); + } + } + } + } + const fieldDef = objectSchema.fields[fieldName]; + + // Skip if field not found or not a relationship type. + // + // `user` is a lookup specialized to sys_user and `tree` is a hierarchical + // self-reference — both carry the same `reference` + id storage, so both + // expand through this exact path (single or multiple). + // + // [#4226] The membership test is `REFERENCE_VALUE_TYPES` — the spec's own + // list of types whose value "points at another record … the related record + // object in expanded form" — rather than a hand-copied `!==` chain. The + // chain had drifted: it omitted `tree`, so a `$expand` of a hierarchy + // field returned the raw parent id and rendered as a bare placeholder, + // while `field-value.zod.ts` and objectui's `EXPANDABLE_FIELD_TYPES` both + // declared it expandable. Reading the shared set is what stops the + // protocol's expand gate (which validates against the same set) from ever + // admitting a field this loop then silently skips. + // + // [cloud#983] The TARGET comes from `referenceTargetOf` for that same + // anti-drift reason. A raw `fieldDef.reference` read made `{ type: + // 'user' }` (no `reference`) targetless here AND at the gate — but + // `user`'s target is fixed BY THE TYPE (`sys_user`; `Field.user()` takes + // no target argument), so the field was fully specified and the request + // was refused `400 … declares no target object`. Both sides now ask the + // one function what a reference field points at. + if (!fieldDef) continue; + if (!REFERENCE_VALUE_TYPES.has(fieldDef.type)) continue; + const referenceObject = referenceTargetOf(fieldDef); + if (!referenceObject) continue; + // [#24358] ⛔ A reference into the stored-metadata-body family is never + // expanded, at any depth and for any caller. The family's projection runs + // at its doors and only for the object a request names, so this sub-read + // would nest the stored body and its content hash unprojected under the + // requested rows — material the family's own door withholds from the same + // reader. The field keeps the stored id, the answer a reference the + // caller may not expand gets (an unexposed target, #22661, or a refused + // sub-read, below), so it says nothing about the row. + if (isStoredMetadataBodyObject(referenceObject)) continue; + + // Collect all foreign key IDs from records (handle both single and multiple values) + const allIds: any[] = []; + for (const record of records) { + const val = record[fieldName]; + if (val == null) continue; + if (Array.isArray(val)) { + allIds.push(...val.filter((id: any) => id != null)); + } else if (typeof val === 'object') { + // Already expanded — skip + continue; + } else { + allIds.push(val); + } + } + + // De-duplicate IDs + const uniqueIds = [...new Set(allIds)]; + if (uniqueIds.length === 0) continue; + + // Batch-load related records using $in query + try { + // Constrain the batch to the collected FK ids. When the nested expand + // AST carries its own `where`, AND-merge it via an explicit `$and` + // group rather than spreading keys onto the id filter: a shallow spread + // would clobber the `id` constraint (if the nested filter also keys + // `id`) and is ambiguous when the nested filter is a top-level `$or` / + // `$and`. The `$and` wrapper is correct for every FilterCondition shape. + const idFilter = { id: { $in: uniqueIds } }; + const where = nestedAST.where + ? { $and: [idFilter, nestedAST.where] } + : idFilter; + + // [#2850] Resolve the referenced object through the engine's own read + // path (`this.find`) rather than the raw driver, so the security + // middleware applies the REFERENCED object's RLS + FLS to the expanded + // batch — not merely tenant isolation. A single `find` over the + // collected `id $in [...]` batch re-enters the middleware exactly once + // (no N+1), preserving the batched load. `nestedAST.limit/offset` are + // still intentionally NOT forwarded: this path batch-loads every + // parent's related records in one query, so a *per-parent* limit/offset + // can't be expressed — a global cap would silently drop records other + // parents need. Paginate by querying the related object directly. + // + // The `__expandRead` marker (set here, never from client input — + // `executionContext` is server-built) tells the security layer this is + // an expansion sub-read. It does NOT relax that layer: since #7626 the + // referenced object takes the FULL treatment — CRUD gate, + // requiredPermissions, RLS and FLS — so you may expand only rows you + // could have read directly. (#2850 shipped a waiver for "public" + // referenced objects; it keyed on an axis almost no object declares and + // never checked the caller's grants, so it disclosed private records to + // callers the gate had already refused.) A refusal is caught below and + // the bare FK id is retained, which is why the parent read still + // succeeds. `expand` is intentionally omitted from this query so `find` + // does not re-expand — nested relations recurse below under the depth + // guard. + // + // [#7537] The join key is MACHINERY, not a caller-chosen column. This + // sub-read's projection is forwarded from `nestedAST.fields`, and the + // map built below is keyed on `rec.id` — so a nested projection that did + // not name `id` returned rows carrying no `id`, built an EMPTY map, and + // fell through the `recordMap.get(...) ?? val` injection, writing the + // original foreign key back. The expansion was then indistinguishable + // from never having been requested: `200`, and the field still holds a + // valid-looking id. + // + // The failing spelling is the one the platform itself PRESCRIBES: both + // `FIELD_NODE_OBJECT_FORM_REMOVED` and `QUERY_JOINS_REMOVED` + // (`query.zod.ts`) tell authors migrating off the retired nested-select + // object form and off `query.joins` to write + // `expand: { owner: { object: 'user', fields: ['name'] } }`. Refusing the + // mismatch would turn two retirement messages' own migration target into + // an error, so the prescribed form is made to WORK instead: `id` is added + // to the sub-read unconditionally and stripped back out of the emitted + // nested record when the caller did not ask for it. + // + // `id` is the only join key there is, literally: `referenceTargetOf` + // yields the target OBJECT and a reference field carries no + // target-column metadata, so the batch filter above is `{ id: { $in } }` + // by construction. There is no configurable key to add instead. + // + // Only a driver that honours a projection exactly could ever show this — + // `SqlDriver` emits `builder.select(query.fields)` verbatim, while + // `InMemoryDriver.projectFields` force-adds `id` to every projection, + // which is why mock/in-memory suites could not reach the defect and a + // better-sqlite3 QA run (#7463) could. + const nestedFields = Array.isArray(nestedAST.fields) && nestedAST.fields.length > 0 + ? nestedAST.fields + : undefined; + const joinKeyRequested = !nestedFields || nestedFields.includes('id'); + const relatedRecords = await this.find( + referenceObject, + { + where, + // [commit 74155c735] The `as any` these two carried is gone: `find` takes the + // author state now, and the parsed nodes a `QueryAST` holds are + // valid author input (a present `order` is legal to write). + ...(nestedAST.fields + ? { fields: joinKeyRequested ? nestedAST.fields : [...nestedFields!, 'id'] } + : {}), + ...(nestedAST.orderBy ? { orderBy: nestedAST.orderBy } : {}), + context: { ...(execCtx ?? {}), __expandRead: true } as ExecutionContext, + }, + ) ?? []; + + // Build a lookup map: id → record + const recordMap = new Map(); + for (const rec of relatedRecords) { + const id = rec.id; + if (id != null) recordMap.set(String(id), rec); + } + + // Recursively expand nested relations if present + if (nestedAST.expand && Object.keys(nestedAST.expand).length > 0) { + const expandedRelated = await this.expandRelatedRecords( + referenceObject, + relatedRecords, + nestedAST.expand, + depth + 1, + execCtx, + ); + // Rebuild map with expanded records + recordMap.clear(); + for (const rec of expandedRelated) { + const id = rec.id; + if (id != null) recordMap.set(String(id), rec); + } + } + + // [#7537] Strip the join key back out when the caller did not name it. + // It was added above for THIS function's own lookup, so the emitted + // nested record carries exactly the columns the projection asked for — + // the same contract a top-level `fields` gets. Deliberately AFTER the + // recursive expand: that block rebuilds `recordMap` keyed on `rec.id` + // and still needs the key present. Each related record is stripped once + // here rather than per-parent, because one record object is injected + // into every parent that points at it. + if (!joinKeyRequested) { + for (const rec of recordMap.values()) { + if (rec && typeof rec === 'object') delete rec.id; + } + } + + // Inject expanded records back into the original result set + for (const record of records) { + const val = record[fieldName]; + if (val == null) continue; + + if (Array.isArray(val)) { + record[fieldName] = val.map((id: any) => recordMap.get(String(id)) ?? id); + } else if (typeof val !== 'object') { + record[fieldName] = recordMap.get(String(val)) ?? val; + } + // If val is already an object, leave it as-is + } + } catch (e) { + // Graceful degradation: if expand fails, keep original IDs + this.logger.warn('Failed to expand relationship field; retaining foreign key IDs', { + object: objectName, + field: fieldName, + reference: referenceObject, + error: (e as Error).message, + }); + } + + // [#29460] `referenceNameVisible` (ruling 6574398310, letter B): the ids + // the caller's own expansion above left bare are asked whether the + // caller was REFUSED them, and a refused target is served by its name + // and id only. Every other value keeps the answer the loop gave it. See + // {@link serveReferenceNames}. + if ((fieldDef as { referenceNameVisible?: unknown }).referenceNameVisible === true) { + await this.serveReferenceNames(objectName, records, fieldName, referenceObject, execCtx); + } + } + + return records; + } +} diff --git a/packages/objectql/src/engine-transactions-section.ts b/packages/objectql/src/engine-transactions-section.ts new file mode 100644 index 00000000000..16f308e9eb2 --- /dev/null +++ b/packages/objectql/src/engine-transactions-section.ts @@ -0,0 +1,197 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#29986 codemod] The "Transactions" section of `ObjectQL`: the bodies of + * `transaction`, `runOpenedUnit`, `isClosedUnitHandle`, `newTransactionScope`, + * `datasourceNameOf` moved out of `engine.ts` whole by + * `scripts/codemod-split-engine.mjs`. + * + * A MOVE, not a redesign. {@link TransactionsSectionFacet} is a stateless facet of `ObjectQL` + * (ruling 6624345766): each static method of it is the body of the `ObjectQL` + * method of the same name exactly as it stood: not re-indented, not + * formatted, not edited. The method stays on the class as a one-call stub with + * its signature, so the published surface, an instance spy and a subclass + * override are what they were. A public or protected method keeps its + * docblock on the class, the contract; a private method's docblock moved here + * with its body. The state stays on `ObjectQL`. + * + * The bodies reach the class through {@link TransactionsSectionHost}: exactly the + * `ObjectQL` members they read, private ones included. Each stub hands its + * body one literal of the members that body reads, plus a `.call`: methods go + * as closures that call them on the engine at call time, so a subclass + * override or an instance spy is still the one reached; fields and accessors + * go as GETTERS, read when the body reads them, as they were on the class, + * and a field a body assigns goes with a SETTER that assigns it on the engine. + * + * The module imports `engine.ts` for types only, so the two modules form no + * evaluation cycle. `index.ts` re-exports nothing from this module. + */ + +import { driverSupportsTransactions } from '@objectstack/spec/data'; +import { redactPropagatedDriverFault } from '@objectstack/types'; +import type { EngineTransactionInfo, EngineTransactionOptions, IDataDriver } from '@objectstack/core'; +import { rollbackFailedRefusal, rolledBackRefusal } from './summary-errors.js'; +import { TransactionUnsupportedError } from './transaction-errors.js'; +import type { TransactionScope } from './engine-top-level.js'; +import type { ObjectQL } from './engine.js'; + +/** + * The `ObjectQL` members the bodies of {@link TransactionsSectionFacet} read — every one of + * them, and no other. Each is typed AS the class member (an indexed access on + * the class), so a change to a member's signature on the class is a change + * here too. `readonly` members are the getters; the others are fields a body + * assigns. + */ +export interface TransactionsSectionHost { + datasourceNameOf: ObjectQL['datasourceNameOf']; + readonly defaultDriver: ObjectQL['defaultDriver']; + readonly drivers: ObjectQL['drivers']; + newTransactionScope: ObjectQL['newTransactionScope']; + readonly openedUnits: ObjectQL['openedUnits']; + runOpenedUnit: ObjectQL['runOpenedUnit']; + readonly txStore: ObjectQL['txStore']; + warnTransactionUnsupported: ObjectQL['warnTransactionUnsupported']; +} + +/** The bodies of the "Transactions" section of `ObjectQL`: a stateless facet, called by the class's stubs. */ +export class TransactionsSectionFacet { + /** The body of `ObjectQL#transaction`; the method's docblock there is its contract. */ + static async transaction(this: Pick, callback: (trxCtx: any, info: EngineTransactionInfo) => Promise, baseContext?: any, opts?: EngineTransactionOptions): Promise { + // ADR-0067 D2 — JOIN an already-open ambient transaction instead of + // opening a nested driver transaction. A nested begin would acquire a + // second connection (a deadlock on single-connection pools like the + // SQLite knex pool) and would NOT be covered by the outer rollback — + // exactly the half-landing this join prevents: an outer batch + // transaction (e.g. `publishPackageDrafts`) must own the one-and-only + // commit/rollback for every write made through nested helpers (the + // sys-metadata repository's `withTxn`, hook-driven writes, …). + const ambient = this.txStore.getStore(); + if (ambient?.transaction) { + // JOINED, not owned: some outer caller decides commit vs rollback (#5696). + return callback( + { ...(baseContext ?? {}), transaction: ambient.transaction }, + { owned: false }, + ); + } + const driver = this.defaultDriver ? this.drivers.get(this.defaultDriver) : undefined; + const drv = driver as any; + // [#18063] The gate is the DECLARATION, not bare method presence. A + // subclass inherits `beginTransaction` from a base whose transport has + // transactions while its own has none — it cannot opt out of a door it did + // not open, so presence alone routed it down the transactional path and the + // handle it produced covered nothing. `driverSupportsTransactions` is the + // one definition (`@objectstack/spec`), shared with the ScopedContext trio + // below so the engine's transaction entrances cannot drift apart. + if (!driverSupportsTransactions(drv)) { + const datasource = this.defaultDriver ?? drv?.name; + if (opts?.require === true) { + // Fail CLOSED (#5696 point 1): the caller declared it cannot tolerate + // running without a rollback, so refuse BEFORE the callback writes + // anything rather than degrade behind a warning it may never read. + // Generalizes `batchData`'s atomic gate (ADR-0119 D4). + throw new TransactionUnsupportedError(datasource ?? ''); + } + // Declared degrade (ADR-0119 D1) — behaviour unchanged, but no longer + // mute: the caller asked for atomicity and is not getting it (#4619). + this.warnTransactionUnsupported(datasource, drv?.supports?.transactionsUnsupported === true); + // `owned: false` — honest: there is no transaction here to own, and no + // rollback the callback may promise on the strength of it. + return callback(baseContext, { owned: false }); + } + const trx = await drv.beginTransaction(); + const trxCtx = { ...(baseContext ?? {}), transaction: trx }; + try { + // Run the callback inside the ambient transaction store so internal + // queries during writes reuse this transaction's connection (ADR-0034), + // CLOSED once the callback settles, before the commit (#22782, #22855). + const result = await this.runOpenedUnit(trx, driver!, () => callback(trxCtx, { owned: true })); + if (drv.commit) await drv.commit(trx); + else if (drv.commitTransaction) await drv.commitTransaction(trx); + return result; + } catch (err) { + let rollbackFailed = false; + try { + if (drv.rollback) await drv.rollback(trx); + else if (drv.rollbackTransaction) await drv.rollbackTransaction(trx); + } catch { + // swallow rollback failures so the original error surfaces + rollbackFailed = true; + } + // [#21274] A boundary outside the middleware seam: a commit failure, or + // a callback that reached the driver through the transaction handle. + // An error that already crossed an engine boundary inside the callback + // comes back as the same reference. A stale-roll-up refusal ("stored") + // is no longer true once this rollback has run: `rolledBackRefusal`. + // [#29421] Nor is "rolled back" once the rollback threw: + // `rollbackFailedRefusal`. + throw redactPropagatedDriverFault(rollbackFailed ? rollbackFailedRefusal(err) : rolledBackRefusal(err)); + } + } + + /** + * [#22782, #22855] Run `body`, the callback of a unit just opened on `owner` + * with handle `trx`. This is the ONE place both transaction surfaces + * ({@link transaction} and `ScopedContext.transaction`) publish a unit and + * close it again. + * + * Published: the entry goes into {@link txStore} (ADR-0034) for the body's + * lifetime, carrying its owner (#4619). Internal queries during writes reuse + * this connection, and a nested `transaction()` on either surface joins it. + * + * Closed once `body` settles, BEFORE the caller commits or rolls back. Work + * the body started and did not await (an `async: true` hook) holds the + * handle two ways: ambiently, through this entry, and explicitly, threaded + * into the callback's context and from there into the hook's `ctx.api`. That + * work used to run on the finished handle and lose its write ("Transaction + * query already complete"). Emptying the entry ends the ambient way; + * {@link openedUnits} ends the explicit one, because + * {@link buildDriverOptions} treats a closed unit's handle as absent. The + * work then runs unenrolled, on its own connection, as outside any unit: + * `opensAfterAbortUnit` reads the handle the same way, so such a write opens + * the door's own unit when its hooks call for one. + * + * A primitive handle cannot key {@link openedUnits}, so it keeps only the + * ambient half. + */ + static runOpenedUnit(this: Pick, trx: unknown, owner: IDataDriver, body: () => Promise): Promise { + const entry: { transaction: unknown; scope?: TransactionScope } = + { transaction: trx, scope: this.newTransactionScope(owner) }; + if (trx !== null && (typeof trx === 'object' || typeof trx === 'function')) this.openedUnits.set(trx, entry); + return this.txStore.run(entry, body).finally(() => { entry.transaction = undefined; entry.scope = undefined; }); + } + + /** [#22855] Is `handle` a unit {@link runOpenedUnit} published whose callback has since settled? */ + static isClosedUnitHandle(this: Pick, handle: unknown): boolean { + if (handle === null || (typeof handle !== 'object' && typeof handle !== 'function')) return false; + const entry = this.openedUnits.get(handle); + return entry !== undefined && entry.transaction === undefined; + } + + /** + * Build the observability record for a transaction this engine just opened + * (#4619). See {@link TransactionScope} — records only, routes nothing. + */ + static newTransactionScope(this: Pick, owner: IDataDriver): TransactionScope { + return { + driver: owner, + datasource: this.datasourceNameOf(owner), + reportedOutOfScope: new Set(), + }; + } + + /** + * Which datasource name is `driver` registered under? + * + * `registerDriver` keys {@link drivers} by `driver.name`, so the two normally + * agree — but the map is the routing authority (`getDriver` resolves through + * it), so the map is what is searched, and `driver.name` is only the fallback + * for a driver that was never registered. Runs at most once per transaction + * (and once per out-of-scope report), never on the hot path. + */ + static datasourceNameOf(this: Pick, driver: IDataDriver): string { + for (const [name, registered] of this.drivers) { + if (registered === driver) return name; + } + return driver?.name ?? 'unknown'; + } +} diff --git a/packages/objectql/src/engine.ts b/packages/objectql/src/engine.ts index 9c17d517652..43616eff3e6 100644 --- a/packages/objectql/src/engine.ts +++ b/packages/objectql/src/engine.ts @@ -5,7 +5,6 @@ import { QueryAST, QueryInput, HookContext, ServiceObject } from '@objectstack/s // [commit 74155c735] The defaulting node schema `fillQueryAstDefaults` runs author input // through — the declared `.default()` stays in `packages/spec`, the engine // only invokes it. -import { SortNodeSchema } from '@objectstack/spec/data'; import { EngineQueryOptions, DataEngineInsertOptions, @@ -20,14 +19,14 @@ import type { WriteObservabilityOptions } from '@objectstack/spec/contracts'; // engine is what `metadata-protocol.validateData` returns, so letting the two // drift would put a translation layer between a verdict and its contract. import type { ValidateDataResponse } from '@objectstack/spec/api'; -import { parseAutonumberFormat, resolveAutonumberFormat, FILE_REFERENCE_TYPES, REFERENCE_VALUE_TYPES, referenceTargetOf, driverSupportsTransactions } from '@objectstack/spec/data'; +import { parseAutonumberFormat, driverSupportsTransactions } from '@objectstack/spec/data'; // [#5158] Door 2's lowering sink — the SAME pair the protocol face (Door 1) // runs, so `FilterArray` has exactly one lowering in the product. // [ADR-0053 D-D1, amended 2026-09-30 — #5930] The shared `FilterCondition → // FilterCondition` lowering, run once per filter position after the doors and // after token resolution (`resolveThenLowerWhere`), so every driver and the // in-process `having` / per-aggregation evaluator receive the lowered filter. -import { lowerFilterCondition, type FilterCondition, type FilterLoweringOptions } from '@objectstack/spec/data'; +import { type FilterCondition, type FilterLoweringOptions } from '@objectstack/spec/data'; // [#5574] D6, executable. The ceiling and the refusal message live in // `packages/spec/src/data/bulk-write-hook-conformance.ts` so BOTH phases and // both verbs enforce one definition; the engine raises, the contract decides. @@ -51,12 +50,6 @@ import { // scan is one (#6249). Shared with `summary-backfill` rather than re-rolled: // the cursor merge is the part that is easy to get subtly wrong. import { keysetWalk, type KeysetPageQuery } from '@objectstack/types'; -import { - DATA_MIGRATION_FLAG_OBJECT, - FILE_REFERENCES_MIGRATION_ID, - isDataMigrationFlagVerified, - SystemFieldName, -} from '@objectstack/spec/system'; // [#25522 cut 1] The validation-message localisation helpers; the methods of the // same names below hand each body the members it reads (`ValidationMessageHost`). import { negotiatedMessageLocale, objectDisplayLabel, validationMessageContext } from './engine-validation-messages.js'; @@ -150,19 +143,16 @@ import { type RetryOptions, resolveFilterTokens, recordNotFoundError, - objectNotFoundError, resolveUserAuthzGrants, markAppAutomationElevation, carryAppAutomationElevation, appAutomationElevationOf, } from '@objectstack/core'; import { WriteEpoch } from './write-epoch.js'; -import { bridgeAuthzInvalidation } from './authz-invalidation-bridge.js'; -import { SummaryRecomputeError, cascadeStoppedRefusal, hookStoppedRefusal, rollbackFailedRefusal, rolledBackRefusal, type SummaryRecomputeFailure } from './summary-errors.js'; +import { SummaryRecomputeError, cascadeStoppedRefusal, hookStoppedRefusal, rolledBackRefusal, type SummaryRecomputeFailure } from './summary-errors.js'; import { TransactionUnsupportedError } from './transaction-errors.js'; import { aggregateSummaryValue, - summaryEmptySetValue, withoutDeletedSelfParents, type SummaryDescriptor, } from './summary-aggregate.js'; @@ -213,8 +203,7 @@ import { convertPackageBodyForward, registerPlugin, resolveNavObjectNames } from // #5841 retired. import { DuplicateRecordError } from './duplicate-record-error.js'; import { NotNullRefusalError, envelopeDriverWriteRefusal } from './not-null-refusal-error.js'; -import { ConcurrentUpdateError } from './concurrent-update-error.js'; -import { AssignmentRuleEvaluator, ASSIGNMENT_RULE_TYPE, OWNERSHIP_QUEUE_SOURCE_SLOT, type AssignmentQueueService } from './assignment-rules.js'; +import { AssignmentRuleEvaluator, OWNERSHIP_QUEUE_SOURCE_SLOT, type AssignmentQueueService } from './assignment-rules.js'; // [#8682] The write-path loggers' redaction — bound values never reach the log. // [#21274] …and its boundary face — nor the error that leaves the engine. // [#21385] It lives in `@objectstack/types` now, so `driver-sql`'s own log lines @@ -226,8 +215,6 @@ import { redactBoundStatement, redactPropagatedDriverFault } from '@objectstack/ // pseudo-tenant. The decision is a pure function of measured facts; this file // supplies the facts and turns a `refuse` into the throw. import { - resolveTenantFieldName, - SystemWriteOrganizationRequiredError, ORGANIZATION_OBJECT, } from './tenancy/system-write-organization.js'; import { @@ -251,7 +238,7 @@ import { export type InsertManyRowOutcome = | { ok: true; record: any; droppedFields?: DroppedFieldsEvent[] } | { ok: false; error: unknown }; -import { CoreServiceName, StorageNameMapping } from '@objectstack/spec/system'; +import { CoreServiceName } from '@objectstack/spec/system'; import { IRealtimeService } from '@objectstack/spec/contracts'; import type { ICryptoProvider } from '@objectstack/spec/contracts'; import { @@ -265,7 +252,7 @@ import { count, rejectCredentialAggregation } from './engine-count-aggregate.js' // ADR-0100: the credential-column guard's filter / sort half, beside the // aggregate half above — one collector, one envelope. import { rejectCredentialEvaluation } from './credential-evaluation-guard.js'; -import { assertNoInternalFieldEvaluated, rejectInternalFieldEvaluation } from './internal-field-evaluation-guard.js'; +import { rejectInternalFieldEvaluation } from './internal-field-evaluation-guard.js'; import { assertProjectionNamesAField, refuseProjectionOfUnknownColumns } from './empty-projection-door.js'; import { assertRecordCurrencyCodes, touchesRecordCurrency } from './validation/record-currency-code.js'; import { @@ -283,7 +270,7 @@ import { import { SchemaRegistry, type ArtifactInstallScope } from './registry.js'; import { stripSearchCompanion } from './search-companion.js'; import { type EvalError, type EvalOptionLabels, type EvalPermissions } from '@objectstack/formula'; -import { declaredOptionLabels, readerOptionLabels } from './formula-option-labels.js'; +import { declaredOptionLabels } from './formula-option-labels.js'; // [commit 3c1bbd2a8] The ONE spelling of "which object does an aggregated `defineView` // container bind to", imported rather than re-spelled — from the LEAF subpath, // for the reason the `/errors` import above states (commit 3bd9b3498). See @@ -291,8 +278,7 @@ import { declaredOptionLabels, readerOptionLabels } from './formula-option-label // [#20331] The divergent view-container `name` refusal — the ONE judge this // registrar and `os validate` both call. import { bindHooksToEngine, type BindHooksOptions } from './hook-binder.js'; -import { declaredHookCondition } from './hook-wrappers.js'; -import { validateRecord, validateRecordInScope, normalizeMultiValueFields, normalizeBlankTypedValues, normalizeNumericStringValues, coerceBooleanFields, ValidationError, isScannableValueShapeField } from './validation/record-validator.js'; +import { validateRecord, validateRecordInScope, normalizeMultiValueFields, normalizeBlankTypedValues, normalizeNumericStringValues, coerceBooleanFields, ValidationError } from './validation/record-validator.js'; import type { AdmittedValueShapeViolation, AdmittedValueShapeViolationSink, ValidationMessageContext } from './validation/record-validator.js'; import type { RelatedFieldBinding, RelatedRecordBinding } from './validation/rule-validator.js'; import { createBatchSummaryAccumulator } from './summary-batch-accumulation.js'; @@ -328,7 +314,6 @@ import { } from './declared-read-columns.js'; // [#21777] "Is this schema the remote's?" One predicate, shared with the boot sync. // [#21910, #21918] And its injected-column refinement, which both cascade walks ask. -import { isFederatedUnprovisionedInjectedColumn } from './federated-object.js'; // [#27277] The correctness floor over a plugin driver on an external datasource. import { externalPredicateWriteRefusal, @@ -357,7 +342,7 @@ import type { READABLE_IDS_MAX_CHAIN_DEPTH } from './engine-readable-ids-permiss // Imported after every module those bodies import, so the engine's module // evaluation order is what it was. // [#28001] …and the object-level closure of that walk the metadata read serves. -import { collectCascadeDeleteObjects, collectCascadeDeleteSet, deleteRestrictedError, recordReferenceCheckElevation, referenceProbeFilter, refuseRestrictsInsideOwnCascade } from './engine-delete-cascade.js'; +import { collectCascadeDeleteSet, deleteRestrictedError, recordReferenceCheckElevation, referenceProbeFilter, refuseRestrictsInsideOwnCascade } from './engine-delete-cascade.js'; import { resolveEngineDeleteDispatch, ENGINE_DELETE_REJECT_MESSAGE, @@ -390,23 +375,21 @@ import { } from './integrity/dangling-reference-audit.js'; // [#29986 codemod] The top-level declarations that moved whole to `engine-top-level.ts`, as this file still reads them. import { - AUTONUMBER_COLLISION_ATTEMPTS, AUTONUMBER_SEED_PAGE_SIZE, DISPATCHABLE_HOOK_EVENTS, ENGINE_DELETE_OPTION_KEYS, - ENGINE_FIND_OPTION_KEYS, ENGINE_QUERY_SLOTS, ENGINE_UPDATE_OPTION_KEYS, ENGINE_WHERE_SLOTS, ENGINE_WIRE_ONLY_SLOTS, - SECURITY_SERVICE_SOURCE_SLOT, WIRE_ONLY_CANONICAL_SHAPES, WRITE_PRECONDITION_OPTION, applyFormulaPlan, - applyStoredExpressionsToPatch, assertDispatchableHookEvent, assertHookScopeNotSelfCancelling, - assertOrderByIsMaterializable, assertProjectionHasNoDottedPaths, assertValidHookExcludeObjects, - assertValidHookObject, assertValidUnscopedMultiWriteFlag, autonumberFieldNames, callerKeySets, callerSuppliedRow, - cascadeMemberKey, cascadeSetAdd, cascadeSetHas, computeStoredExpressions, declaredDatetimeLowering, - declaredMultiValued, deleteWalkTargetOf, droppedFieldEvents, foldEngineOptionAliases, formulaPlanEntry, + AUTONUMBER_SEED_PAGE_SIZE, DISPATCHABLE_HOOK_EVENTS, ENGINE_DELETE_OPTION_KEYS, ENGINE_FIND_OPTION_KEYS, + ENGINE_QUERY_SLOTS, ENGINE_UPDATE_OPTION_KEYS, ENGINE_WHERE_SLOTS, ENGINE_WIRE_ONLY_SLOTS, + SECURITY_SERVICE_SOURCE_SLOT, WRITE_PRECONDITION_OPTION, applyFormulaPlan, applyStoredExpressionsToPatch, + assertDispatchableHookEvent, assertHookScopeNotSelfCancelling, assertOrderByIsMaterializable, + assertProjectionHasNoDottedPaths, assertValidHookExcludeObjects, assertValidHookObject, + assertValidUnscopedMultiWriteFlag, autonumberFieldNames, callerKeySets, callerSuppliedRow, computeStoredExpressions, + declaredDatetimeLowering, declaredMultiValued, droppedFieldEvents, foldEngineOptionAliases, formulaPlanEntry, handBackWithheldInsertReadonly, hookMatchesObject, hydrateWriteFormulas, insertStaticReadonlySubject, - judgeWhereAdmission, lowerWhereFilterArray, masterIdOf, mergeReadContext, organizationPredicateDeleteRefusal, + lowerWhereFilterArray, masterIdOf, mergeReadContext, organizationPredicateDeleteRefusal, packageBodyCollectionRefusal, packageBodyObjectAndAppOwnerStampRefusal, packageBodySeedDataRefusal, planFormulaProjection, planStoredExpressions, readStoredAutonumberCounter, recomputeAutonumberReaders, - rejectUnknownEngineOptions, replacedStoredExpressionFields, resolveWhereFilterTokens, rowsWithoutFormulaWidening, - settleBulkStoredExpressions, shouldSkipStateMachine, statesDeleteSemantics, staticReadonlyStoredView, - storedExpressionsReading, stripComputedWriteFields, undeclaredWriteFieldErrors, updateStaticReadonlySubject, - whereResolvedTokenJudge, withFileOwnerKey, withholdInsertReadonlyFromHooks, withoutFormulaWidening, - writeFailureLogMeta, + rejectUnknownEngineOptions, replacedStoredExpressionFields, rowsWithoutFormulaWidening, settleBulkStoredExpressions, + shouldSkipStateMachine, staticReadonlyStoredView, storedExpressionsReading, stripComputedWriteFields, + undeclaredWriteFieldErrors, updateStaticReadonlySubject, withFileOwnerKey, withholdInsertReadonlyFromHooks, + withoutFormulaWidening, writeFailureLogMeta, } from './engine-top-level.js'; import type { AdmittedValueShapeViolationTally, AutonumberReaders, CascadeDeleteSet, CascadeIssuedDelete, DatasourceDef, @@ -435,42 +418,58 @@ export { writeFailureLogMeta, } from './engine-top-level.js'; // [#29986 codemod] Class head: units, hooks, functions, body runners, hook dispatch: the methods of the same names below hand the bodies what they read (`HooksAndFunctionsHost`). -import { - dispatchUnscopedMultiWriteHooks, unregisterFunctionsByPackage, unregisterHooksByPackage, -} from './engine-hooks-and-functions.js'; +import { HooksAndFunctionsFacet } from './engine-hooks-and-functions.js'; // [#29986 codemod] Action system and packaged-action activation: the methods of the same names below hand the bodies what they read (`ActionsHost`). -import { listRegisteredActions, registerAction } from './engine-actions.js'; +import { ActionsFacet } from './engine-actions.js'; // [#29986 codemod] Registration seams, readable-ids security, formula and default permissions: the methods of the same names below hand the bodies what they read (`SecurityResolversHost`). -import { refuseUncomposableReadableIdsClasses, resolveFormulaRelated } from './engine-security-resolvers.js'; +import { SecurityResolversFacet } from './engine-security-resolvers.js'; // [#29986 codemod] Authz invalidation, middleware, session, driver options, posture, organization probe: the methods of the same names below hand the bodies what they read (`MiddlewareAndContextHost`). -import { executeWithMiddleware, resolveSystemInsertOrganization } from './engine-middleware-and-context.js'; +import { MiddlewareAndContextFacet } from './engine-middleware-and-context.js'; // [#29986 codemod] Field defaults and autonumbers: the methods of the same names below hand the bodies what they read (`DefaultsAndAutonumbersHost`). -import { - adoptExplicitAutonumber, applyAutonumbers, applyFieldDefaults, defaultFieldEntries, isIssuedAutonumberCollision, - resolveNowDefault, resolveOptionDefault, warnAutonumberReadersUnserved, -} from './engine-defaults-and-autonumbers.js'; +import { DefaultsAndAutonumbersFacet } from './engine-defaults-and-autonumbers.js'; // [#29986 codemod] Package, app, plugin, driver and datasource registration: the methods of the same names below hand the bodies what they read (`RegistrationHost`). -import { - applyExternalDriverFloor, assertWriteAllowed, registerDatasourceDef, registerMetadataCollections, - registerPackageBody, supplyFileColumnsMovedResolver, -} from './engine-registration.js'; +import { RegistrationFacet } from './engine-registration.js'; // [#29986 codemod] Realtime, i18n, home amounts, validation messages, references, master-detail: the methods of the same names below hand the bodies what they read (`ReferencesAndMasterDetailHost`). -import { inspectCascadeDelete } from './engine-references-and-master-detail.js'; +import { ReferencesAndMasterDetailFacet } from './engine-references-and-master-detail.js'; // [#29986 codemod] Crypto, secrets, internal fields, schema and datasource routing, init: the methods of the same names below hand the bodies what they read (`SecretsAndRoutingHost`). -import { resolveDatasourceBinding } from './engine-secrets-and-routing.js'; +import { SecretsAndRoutingFacet } from './engine-secrets-and-routing.js'; +// [#29986 codemod] Migration flags, value-shape deviations, creation attestations, destroy: the methods of the same names below hand the bodies what they read (`MigrationStateHost`). +import { MigrationStateFacet } from './engine-migration-state.js'; // [#29986 codemod] Summary indexes, related-record expansion, file and reference hydration: the methods of the same names below hand the bodies what they read (`SummariesAndExpansionHost`). -import { buildSummaryIndex, initializeSummaryFields } from './engine-summaries-and-expansion.js'; +import { SummariesAndExpansionFacet } from './engine-summaries-and-expansion.js'; // [#29986 codemod] Query lowering, search, find and findOne: the methods of the same names below hand the bodies what they read (`FindHost`). -import { expandSearchOnAst, reportFindFailure, requireFindOnePredicate } from './engine-find.js'; +import { FindFacet } from './engine-find.js'; // [#29986 codemod] Write preparation and insert: the methods of the same names below hand the bodies what they read (`InsertHost`). -import { staticReadonlyCreateStrip } from './engine-insert.js'; +import { InsertFacet } from './engine-insert.js'; // [#29986 codemod] Cascade and referential delete planning: the methods of the same names below hand the bodies what they read (`CascadePlanningHost`). -import { - cascadeRelationBehavior, isTenantColumnSetNull, organizationCountIsDisclosable, referenceProbeFilterOfSet, - warnCascadeNotAtomic, -} from './engine-cascade-planning.js'; +import { CascadePlanningFacet } from './engine-cascade-planning.js'; // [#29986 codemod] Delete, count, aggregate and execute: the methods of the same names below hand the bodies what they read (`DeleteAndAggregateHost`). -import { aggregate } from './engine-delete-and-aggregate.js'; +import { DeleteAndAggregateFacet } from './engine-delete-and-aggregate.js'; +// [#29986 codemod] Transactions: the methods of the same names below hand the bodies what they read (`TransactionsSectionHost`). +import { TransactionsSectionFacet } from './engine-transactions-section.js'; +// [#29986 codemod] Compatibility / convenience API: the methods of the same names below hand the bodies what they read (`CompatibilitySectionHost`). +import { CompatibilitySectionFacet } from './engine-compatibility-section.js'; + +// [#29986 codemod] The statics of `ObjectQL` its facet modules read, private ones included, as +// the class declares them (an indexed access on the class). The static block at the end of +// the class fills `ObjectQLStatics` with forwarders: a method is called on the class at +// call time, a field or accessor is read when the body reads it. A facet module imports +// `ObjectQLStatics` AS `ObjectQL`, so each moved body reads `ObjectQL.x` unedited; +// `index.ts` re-exports neither, so neither reaches the package. +export interface ObjectQLStaticsHost { + readonly MAX_CASCADE_DEPTH: (typeof ObjectQL)['MAX_CASCADE_DEPTH']; + readonly MAX_EXPAND_DEPTH: (typeof ObjectQL)['MAX_EXPAND_DEPTH']; + readonly REFERENCE_PROBE_BATCH: (typeof ObjectQL)['REFERENCE_PROBE_BATCH']; + readonly coveredValueFieldPresence: (typeof ObjectQL)['coveredValueFieldPresence']; + readonly homeAmountOrganizationOf: (typeof ObjectQL)['homeAmountOrganizationOf']; + readonly keepsOrganizationReference: (typeof ObjectQL)['keepsOrganizationReference']; + readonly mediaFieldPresence: (typeof ObjectQL)['mediaFieldPresence']; + readonly organizationCountContext: (typeof ObjectQL)['organizationCountContext']; + readonly outlivesOrganization: (typeof ObjectQL)['outlivesOrganization']; + readonly remainderAfterMemberRemoval: (typeof ObjectQL)['remainderAfterMemberRemoval']; + readonly storedReferenceIncludes: (typeof ObjectQL)['storedReferenceIncludes']; +} +export const ObjectQLStatics = {} as ObjectQLStaticsHost; export class ObjectQL implements IObjectQLEngine { /** @@ -604,8 +603,12 @@ export class ObjectQL implements IObjectQLEngine { * into the host's own by re-registering under the same id. */ recordResidualOwner(ownerId: string, source: 'code' | 'marketplace'): void { - if (this.residualOwnerSources.get(ownerId) === 'marketplace') return; - this.residualOwnerSources.set(ownerId, source); + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readResidualOwnerSources = () => this.residualOwnerSources; + const host: ThisParameterType = { + get residualOwnerSources() { return readResidualOwnerSources(); }, + }; + return HooksAndFunctionsFacet.recordResidualOwner.call(host, ownerId, source); } /** @@ -614,7 +617,12 @@ export class ObjectQL implements IObjectQLEngine { * as its residual's owner. */ residualOwnerSource(ownerId: string): 'code' | 'marketplace' | undefined { - return this.residualOwnerSources.get(ownerId); + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readResidualOwnerSources = () => this.residualOwnerSources; + const host: ThisParameterType = { + get residualOwnerSources() { return readResidualOwnerSources(); }, + }; + return HooksAndFunctionsFacet.residualOwnerSource.call(host, ownerId); } // Datasource definitions by name (ADR-0015): carries schemaMode + @@ -869,11 +877,11 @@ export class ObjectQL implements IObjectQLEngine { // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). const readHooks = () => this.hooks; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get hooks() { return readHooks(); }, get logger() { return readLogger(); }, }; - return unregisterHooksByPackage.call(host, packageId); + return HooksAndFunctionsFacet.unregisterHooksByPackage.call(host, packageId); } /** @@ -894,17 +902,24 @@ export class ObjectQL implements IObjectQLEngine { handler: HookHandler, packageIdOrOptions?: string | FunctionRegistrationOptions, ): void { - if (!name || typeof handler !== 'function') return; - const opts: FunctionRegistrationOptions = - typeof packageIdOrOptions === 'string' ? { packageId: packageIdOrOptions } : (packageIdOrOptions ?? {}); - const { packageId, effect } = opts; - this.functions.set(name, { handler, packageId, ...(effect ? { effect } : {}) }); - this.logger.debug('Registered function', { name, packageId, effect }); + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readFunctions = () => this.functions; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get functions() { return readFunctions(); }, + get logger() { return readLogger(); }, + }; + return HooksAndFunctionsFacet.registerFunction.call(host, name, handler, packageIdOrOptions); } /** Look up a registered function by name. */ resolveFunction(name: string): HookHandler | undefined { - return this.functions.get(name)?.handler; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readFunctions = () => this.functions; + const host: ThisParameterType = { + get functions() { return readFunctions(); }, + }; + return HooksAndFunctionsFacet.resolveFunction.call(host, name); } /** @@ -916,7 +931,12 @@ export class ObjectQL implements IObjectQLEngine { * drifting. */ resolveFunctionEntry(name: string): Readonly | undefined { - return this.functions.get(name); + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const readFunctions = () => this.functions; + const host: ThisParameterType = { + get functions() { return readFunctions(); }, + }; + return HooksAndFunctionsFacet.resolveFunctionEntry.call(host, name); } /** Remove all functions registered under a given `packageId`. */ @@ -924,11 +944,11 @@ export class ObjectQL implements IObjectQLEngine { // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). const readFunctions = () => this.functions; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get functions() { return readFunctions(); }, get logger() { return readLogger(); }, }; - return unregisterFunctionsByPackage.call(host, packageId); + return HooksAndFunctionsFacet.unregisterFunctionsByPackage.call(host, packageId); } /** @@ -988,17 +1008,26 @@ export class ObjectQL implements IObjectQLEngine { * already present (kept unchanged). */ setDefaultBodyRunner(runner: any): boolean { - if (this._defaultBodyRunner) { - this.logger.debug('Default body runner already installed — keeping the first'); - return false; - } - this._defaultBodyRunner = runner; - return true; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const read_defaultBodyRunner = () => this._defaultBodyRunner; + const write_defaultBodyRunner = (v: ObjectQL['_defaultBodyRunner']) => { this._defaultBodyRunner = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _defaultBodyRunner() { return read_defaultBodyRunner(); }, + set _defaultBodyRunner(v) { write_defaultBodyRunner(v); }, + get logger() { return readLogger(); }, + }; + return HooksAndFunctionsFacet.setDefaultBodyRunner.call(host, runner); } /** The installed default body-runner, if any — the public read the first-wins guard implies. */ getDefaultBodyRunner(): any { - return this._defaultBodyRunner; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const read_defaultBodyRunner = () => this._defaultBodyRunner; + const host: ThisParameterType = { + get _defaultBodyRunner() { return read_defaultBodyRunner(); }, + }; + return HooksAndFunctionsFacet.getDefaultBodyRunner.call(host); } /** @@ -1018,17 +1047,26 @@ export class ObjectQL implements IObjectQLEngine { * already present (kept unchanged). */ setDefaultActionRunner(runner: (actionDef: any) => ((ctx: any) => Promise) | undefined): boolean { - if (this._defaultActionRunner) { - this.logger.debug('Default action runner already installed — keeping the first'); - return false; - } - this._defaultActionRunner = runner; - return true; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const read_defaultActionRunner = () => this._defaultActionRunner; + const write_defaultActionRunner = (v: ObjectQL['_defaultActionRunner']) => { this._defaultActionRunner = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _defaultActionRunner() { return read_defaultActionRunner(); }, + set _defaultActionRunner(v) { write_defaultActionRunner(v); }, + get logger() { return readLogger(); }, + }; + return HooksAndFunctionsFacet.setDefaultActionRunner.call(host, runner); } /** The installed default action-runner factory, if any. */ getDefaultActionRunner(): ((actionDef: any) => ((ctx: any) => Promise) | undefined) | undefined { - return this._defaultActionRunner; + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + const read_defaultActionRunner = () => this._defaultActionRunner; + const host: ThisParameterType = { + get _defaultActionRunner() { return read_defaultActionRunner(); }, + }; + return HooksAndFunctionsFacet.getDefaultActionRunner.call(host); } /** @@ -1113,68 +1151,30 @@ export class ObjectQL implements IObjectQLEngine { } } - /** - * [#5038] Would `triggerHooks(event, ctx)` reach ANY handler for `object`? - * - * Applies the SAME per-object rule the dispatch loop applies — literally the - * same function, {@link hookMatchesObject}, since #5928 — because this answer - * gates a READ of the whole matched row set on the bulk write path. Getting - * it looser than the dispatch loop only costs a wasted query; getting it - * TIGHTER would silently drop hooks that were going to fire. Until #5928 the - * two were separate hand-written copies of one semantic and this comment - * asked the reader to keep them in step; sharing the function is what - * actually keeps them in step, and the property test in - * `hook-exclude-objects.test.ts` pins the direction that matters. - * - * `session.skipAutomations` is deliberately NOT consulted: it suppresses only - * metadata-bound entries, and code-registered hooks (audit, sharing) still - * run, so the row set is still needed. Over-reading in that case is a cost, - * never a correctness loss — and it is the SAFE side of the asymmetry above, - * which is why the pin asserts an implication rather than an equality. - */ private hasHooksFor(event: string, object: string): boolean { - const entries = this.hooks.get(event); - if (!entries || entries.length === 0) return false; - return entries.some((entry) => hookMatchesObject(entry, object)); + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts`, its docblock with it (see `HooksAndFunctionsHost`). + const readHooks = () => this.hooks; + const host: ThisParameterType = { + get hooks() { return readHooks(); }, + }; + return HooksAndFunctionsFacet.hasHooksFor.call(host, event, object); } - /** - * [#22782] Does this write need its own unit of work? Yes when its `after*` - * dispatch can ABORT and no unit is open, so the abort rolls back what - * `HookSchema.onError` says: the row and every `ctx.api` write the chain made. "Can abort" - * is a metadata-bound hook (`meta` — the only entries with a declared - * `onError`) on `event` for `object`, automations not skipped, that either - * is blocking (not `async`) with `onError` `abort` (the default), or carries - * a `condition` (#22856): a condition that cannot be evaluated aborts the - * operation whatever `onError` says, and before an `async` hook detaches - * (#4775, `HookConditionError`), so it is read by the wrapper's own - * {@link declaredHookCondition}. Hook count and order are untouched (#7413); - * only the boundary around them moves. - * - * No unit — the pre-#22782 path — when one is open (ambient or threaded: the - * caller's unit decides; a threaded handle whose unit has closed is not open, - * #22855 `isClosedUnitHandle`), when `object` is off the default driver - * (`transaction()` covers that one alone and would refuse the write), or when - * that driver has no transactions (ADR-0119 D1; asking would only warn). - * `SummaryRecomputeError` means "the records WERE written": it is rethrown - * after the commit, never through the rollback. Only the write's OWN refusal - * arrives here: one a hook's `ctx.api` write raised was already converted - * where it left the handler (#29422, `hookStoppedRefusal`), so it rolls back. - */ private opensAfterAbortUnit(event: 'afterInsert' | 'afterUpdate' | 'afterDelete', object: string, context: ExecutionContext | undefined): boolean { - const threaded = context?.transaction; - if ((threaded !== undefined && !this.isClosedUnitHandle(threaded)) || this.txStore.getStore()?.transaction !== undefined) return false; - if ((context as { skipAutomations?: unknown } | undefined)?.skipAutomations) return false; - const aborting = (this.hooks.get(event) ?? []).some((entry) => - entry.meta && hookMatchesObject(entry, object) - && ((!entry.meta.async && (entry.meta.onError ?? 'abort') === 'abort') || declaredHookCondition(entry.meta) !== undefined)); - if (!aborting) return false; - const defaultDriver = this.defaultDriver ? this.drivers.get(this.defaultDriver) : undefined; - try { - return this.getDriver(object) === defaultDriver && driverSupportsTransactions(defaultDriver); - } catch { - return false; // the write itself reports an unroutable object - } + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts`, its docblock with it (see `HooksAndFunctionsHost`). + const readDefaultDriver = () => this.defaultDriver; + const readDrivers = () => this.drivers; + const readHooks = () => this.hooks; + const readTxStore = () => this.txStore; + const host: ThisParameterType = { + get defaultDriver() { return readDefaultDriver(); }, + get drivers() { return readDrivers(); }, + getDriver: (...a) => this.getDriver(...a), + get hooks() { return readHooks(); }, + isClosedUnitHandle: (...a) => this.isClosedUnitHandle(...a), + get txStore() { return readTxStore(); }, + }; + return HooksAndFunctionsFacet.opensAfterAbortUnit.call(host, event, object, context); } /** [#22782] The unit {@link opensAfterAbortUnit} asked for; `write` re-enters the door, which joins it. */ @@ -1337,78 +1337,17 @@ export class ObjectQL implements IObjectQLEngine { return dispatchPerRowBeforeHooks.call(host, object, event, rows, batchCtx); } - /** - * [#9719, widened to `beforeUpdate` by #9974] The whole-operation - * `beforeUpdate` / `beforeDelete` dispatch for an UNSCOPED predicate write — - * `multi: true` with no caller predicate at all (`options.where` absent or - * `null`) — delivered ONLY to registrations that declared - * {@link HookEntry.dispatchUnscopedMultiWrite} on `event`. - * - * ## Why this dispatch exists - * - * The per-row contract (#5038 / #5574) made the operation's SHAPE invisible - * to `before*` handlers on the predicate path: every per-row context arrives - * with `input.id` bound, and a zero-match predicate dispatches nothing at - * all ([D1]). A guard whose rule is about the shape — #4757's "refuse a - * predicate-less multi-delete outright, regardless of what it matches" on - * `sys_attachment`, and #4630's twin on `sys_comment` — therefore could not - * fire through the wired engine: with rows matched its by-id branch shadowed - * the check, and with none matched it never ran. Both limbs need a dispatch - * that happens BEFORE the matched-row read, keyed on the operation's shape - * alone. - * - * ## Why one helper, both verbs - * - * The update half was measured as a PARTIAL guard rather than a total - * fail-open (#9974): an unscoped `multi: true` update was refused only when - * it happened to sweep a row the caller lacked rights to — the per-row gate - * catching the shape by accident — and resolved silently when the caller was - * entitled to every row, zero-match included. A guard that fires by accident - * reads as enforcement while enforcing nothing, which is why the ruling - * closed it rather than narrowing the declaration. `event` is the ONLY thing - * that differs between the two verbs here, so it is a parameter and not a - * second copy of this method. - * - * ## What the handler receives - * - * The whole-operation context: `input.id` undefined, `input.options` the - * caller's raw options — the contractual upper-bound read `hook.zod.ts` - * blesses for `before*` handlers (`input.options.where` / `.multi`) — and - * `session` built by the engine, so the handler's own bypass rules (system - * context, no identity envelope) apply unchanged. It is a DERIVED context, - * not the batch-level one (`hook.zod.ts`: no handler is dispatched on that), - * carrying `dispatch: { mode: 'record', index: 0 }` — this call stands for - * the caller's whole write, not for one row of it — with the batch `scope` - * object kept identity-shared so a stash still reaches the after phase. - * - * ## Deliberately opt-in - * - * Undeclared registrations see NO new dispatch: generalizing an unscoped- - * multi-write refusal to every guarded object is a product decision this - * mechanism does not take (#9719 commissions the `sys_attachment` - * restoration, #9974 the `sys_comment` update half — neither commissions an - * engine-wide rule). The engine stays neutral; the policy — who is refused, - * with which error — lives in the declaring handler. - * - * ## The id slot is not a lever here either - * - * The old batch dispatch's `input.id` was present-but-`undefined`, and - * binding it rerouted the write onto the by-id branch. The ladder is now - * resolved before any handler runs, so a handler binding `input.id` on this - * context would retarget nothing — refused rather than ignored, the same - * rule as D4 and #6752 (`HookTargetRebindError`, path `'unscoped-multi'`). - */ private dispatchUnscopedMultiWriteHooks( event: 'beforeUpdate' | 'beforeDelete', object: string, batchCtx: HookContext, ): Promise { - // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts` (see `HooksAndFunctionsHost`). + // [#29986 codemod] The body moved to `engine-hooks-and-functions.ts`, its docblock with it (see `HooksAndFunctionsHost`). const readHooks = () => this.hooks; - const host: ThisParameterType = { + const host: ThisParameterType = { get hooks() { return readHooks(); }, }; - return dispatchUnscopedMultiWriteHooks.call(host, event, object, batchCtx); + return HooksAndFunctionsFacet.dispatchUnscopedMultiWriteHooks.call(host, event, object, batchCtx); } /** @@ -1469,22 +1408,23 @@ export class ObjectQL implements IObjectQLEngine { // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). const readActions = () => this.actions; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get actions() { return readActions(); }, get logger() { return readLogger(); }, }; - return registerAction.call(host, objectName, actionName, handler, packageName); + return ActionsFacet.registerAction.call(host, objectName, actionName, handler, packageName); } /** * Execute a named action on an object. */ - async executeAction(objectName: string, actionName: string, ctx: any): Promise { - const entry = this.actions.get(`${objectName}:${actionName}`); - if (!entry) { - throw new Error(`Action '${actionName}' on object '${objectName}' not found`); - } - return entry.handler(ctx); + executeAction(objectName: string, actionName: string, ctx: any): Promise { + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActions = () => this.actions; + const host: ThisParameterType = { + get actions() { return readActions(); }, + }; + return ActionsFacet.executeAction.call(host, objectName, actionName, ctx); } /** @@ -1501,21 +1441,22 @@ export class ObjectQL implements IObjectQLEngine { listRegisteredActions(): Array<{ objectName: string; actionName: string; package?: string }> { // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). const readActions = () => this.actions; - const host: ThisParameterType = { + const host: ThisParameterType = { get actions() { return readActions(); }, }; - return listRegisteredActions.call(host); + return ActionsFacet.listRegisteredActions.call(host); } /** * Remove all actions registered by a specific package. */ removeActionsByPackage(packageName: string): void { - for (const [key, entry] of this.actions.entries()) { - if (entry.package === packageName) { - this.actions.delete(key); - } - } + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActions = () => this.actions; + const host: ThisParameterType = { + get actions() { return readActions(); }, + }; + return ActionsFacet.removeActionsByPackage.call(host, packageName); } // ======================================== @@ -1542,7 +1483,12 @@ export class ObjectQL implements IObjectQLEngine { * missing ledger degrades visibly instead of at the first flip. */ setActionActivationStore(store: ActionActivationStore): void { - this.actionActivation.attach(store); + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + }; + return ActionsFacet.setActionActivationStore.call(host, store); } /** @@ -1553,8 +1499,13 @@ export class ObjectQL implements IObjectQLEngine { * governance inventory runs there: the registry is final for the boot, so the * audit line describes the deployment that will actually serve requests. */ - async hydrateActionActivations(): Promise { - return this.actionActivation.hydrate(); + hydrateActionActivations(): Promise { + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + }; + return ActionsFacet.hydrateActionActivations.call(host); } /** @@ -1563,12 +1514,22 @@ export class ObjectQL implements IObjectQLEngine { * boot answers `true` for everything, exactly as it always has. */ isActionEnabled(actionName: string): boolean { - return this.actionActivation.isEnabled(actionName); + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + }; + return ActionsFacet.isActionEnabled.call(host, actionName); } /** Every action name the ledger currently switches off (operability reads). */ listDisabledActions(): string[] { - return this.actionActivation.disabledNames(); + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + }; + return ActionsFacet.listDisabledActions.call(host); } /** @@ -1580,7 +1541,12 @@ export class ObjectQL implements IObjectQLEngine { * that answer. */ getDisabledActions(): string[] | undefined { - return this.actionActivation.durable ? this.actionActivation.disabledNames() : undefined; + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + }; + return ActionsFacet.getDisabledActions.call(host); } /** @@ -1596,12 +1562,14 @@ export class ObjectQL implements IObjectQLEngine { * degraded (AGENTS.md "Degradation log levels"). */ describeDisabledAction(actionName: string): string { - this.logger.info( - `ACTION_DISABLED: packaged action '${actionName}' refused — it is switched off in the activation ledger ` + - '(sys_metadata_activation, ADR-0126 §8 item 2).', - { action: actionName, refusal: 'ACTION_DISABLED' }, - ); - return this.actionActivation.describeDisabled(actionName); + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + get logger() { return readLogger(); }, + }; + return ActionsFacet.describeDisabledAction.call(host, actionName); } /** @@ -1613,8 +1581,13 @@ export class ObjectQL implements IObjectQLEngine { * names the caller in `options.context` (elevated), so the ledger row and * its audit row record who switched the action. */ - async setActionActive(row: ActionActivationRow, options?: ActionActivationWriteOptions): Promise { - await this.actionActivation.setActive(row, options); + setActionActive(row: ActionActivationRow, options?: ActionActivationWriteOptions): Promise { + // [#29986 codemod] The body moved to `engine-actions.ts` (see `ActionsHost`). + const readActionActivation = () => this.actionActivation; + const host: ThisParameterType = { + get actionActivation() { return readActionActivation(); }, + }; + return ActionsFacet.setActionActive.call(host, row, options); } /** @@ -1640,8 +1613,16 @@ export class ObjectQL implements IObjectQLEngine { * did before the seam existed. */ registerHeldFileResolver(fn: HeldFileResolver): void { - this._heldFileResolver = fn; - this.logger.debug('Registered held-file resolver for sys_file hydration'); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_heldFileResolver = () => this._heldFileResolver; + const write_heldFileResolver = (v: ObjectQL['_heldFileResolver']) => { this._heldFileResolver = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _heldFileResolver() { return read_heldFileResolver(); }, + set _heldFileResolver(v) { write_heldFileResolver(v); }, + get logger() { return readLogger(); }, + }; + return SecurityResolversFacet.registerHeldFileResolver.call(host, fn); } /** @@ -1668,8 +1649,16 @@ export class ObjectQL implements IObjectQLEngine { * seam existed. */ registerFieldOwnedFileReadAuthorizer(fn: FieldOwnedFileReadAuthorizer): void { - this._fieldOwnedFileReadAuthorizer = fn; - this.logger.debug('Registered field-owned file read authorizer for refused sys_file hydration'); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_fieldOwnedFileReadAuthorizer = () => this._fieldOwnedFileReadAuthorizer; + const write_fieldOwnedFileReadAuthorizer = (v: ObjectQL['_fieldOwnedFileReadAuthorizer']) => { this._fieldOwnedFileReadAuthorizer = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _fieldOwnedFileReadAuthorizer() { return read_fieldOwnedFileReadAuthorizer(); }, + set _fieldOwnedFileReadAuthorizer(v) { write_fieldOwnedFileReadAuthorizer(v); }, + get logger() { return readLogger(); }, + }; + return SecurityResolversFacet.registerFieldOwnedFileReadAuthorizer.call(host, fn); } /** @@ -1803,8 +1792,16 @@ export class ObjectQL implements IObjectQLEngine { data: unknown, ) => Promise, ): void { - this._writeGateProbe = fn; - this.logger.debug('Registered write-gate probe for validate() relationship resolution'); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_writeGateProbe = () => this._writeGateProbe; + const write_writeGateProbe = (v: ObjectQL['_writeGateProbe']) => { this._writeGateProbe = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _writeGateProbe() { return read_writeGateProbe(); }, + set _writeGateProbe(v) { write_writeGateProbe(v); }, + get logger() { return readLogger(); }, + }; + return SecurityResolversFacet.registerWriteGateProbe.call(host, fn); } /** @@ -1838,8 +1835,16 @@ export class ObjectQL implements IObjectQLEngine { /** Wire the write wall's per-row verdict for `validate()` (#25062). Last registration wins. */ registerWriteWallVerdict(fn: WriteWallVerdict): void { - this._writeWallVerdict = fn; - this.logger.debug('Registered write-wall verdict for validate() organization judgement'); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_writeWallVerdict = () => this._writeWallVerdict; + const write_writeWallVerdict = (v: ObjectQL['_writeWallVerdict']) => { this._writeWallVerdict = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _writeWallVerdict() { return read_writeWallVerdict(); }, + set _writeWallVerdict(v) { write_writeWallVerdict(v); }, + get logger() { return readLogger(); }, + }; + return SecurityResolversFacet.registerWriteWallVerdict.call(host, fn); } /** @@ -1862,8 +1867,16 @@ export class ObjectQL implements IObjectQLEngine { /** Wire the effective object-permission source (#18783). Last registration wins. */ registerEffectiveObjectPermissionsResolver(fn: (context: unknown) => Promise): void { - this._effectiveObjectPermissionsResolver = fn; - this.logger.debug('Registered effective object-permission resolver for current_user.can()'); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_effectiveObjectPermissionsResolver = () => this._effectiveObjectPermissionsResolver; + const write_effectiveObjectPermissionsResolver = (v: ObjectQL['_effectiveObjectPermissionsResolver']) => { this._effectiveObjectPermissionsResolver = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _effectiveObjectPermissionsResolver() { return read_effectiveObjectPermissionsResolver(); }, + set _effectiveObjectPermissionsResolver(v) { write_effectiveObjectPermissionsResolver(v); }, + get logger() { return readLogger(); }, + }; + return SecurityResolversFacet.registerEffectiveObjectPermissionsResolver.call(host, fn); } /** [#25073] {@link registerInsertRecomputeListener}'s listeners, by registrant id. */ @@ -1895,8 +1908,14 @@ export class ObjectQL implements IObjectQLEngine { id: string, fn: (event: { object: string; row: Record; fields: string[] }) => void | Promise, ): void { - this.insertRecomputeListeners.set(id, fn); - this.logger.debug('Registered insert recompute listener', { id }); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const readInsertRecomputeListeners = () => this.insertRecomputeListeners; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get insertRecomputeListeners() { return readInsertRecomputeListeners(); }, + get logger() { return readLogger(); }, + }; + return SecurityResolversFacet.registerInsertRecomputeListener.call(host, id, fn); } /** @@ -1942,91 +1961,22 @@ export class ObjectQL implements IObjectQLEngine { return { kind: 'usable', svc: candidate as ReadableIdsSecurity }; } - /** - * [ADR-0139 D2, D5] The executor boundary: every `ReadableIds` leaf in the - * FINAL `where` of a read is replaced by its composed form immediately - * before the driver call — after the comparand doors, the middleware chain - * and the before-hooks, so a leaf the caller, a middleware or a hook wrote - * meets the same composition. Returns `ast` by reference when it holds no - * leaf, which is every read today; otherwise a shallow copy whose `where` is - * rewritten copy-on-write (the hooks' `input.ast` is never edited). - * - * Per parent object `P`, once per call, in this order: - * - * 1. **D5's two remaining classes are refused, by name** (`INTERNAL_ERROR` - * / 500), before the security service is asked anything: a parent an - * object-scoped read middleware narrows ({@link hasObjectMiddleware}); a - * parent bound to another datasource than `object` - * ({@link resolveDatasourceBinding}). - * 2. **A parent the registry does not hold** ⇒ `$in: []`, unasked. - * 3. **The security service** — absent or unusable ⇒ `$in: []`. - * 4. **`canReadObject(P)`** (its `explain` fallback when the method is - * absent) — false or a throw ⇒ `$in: []`. - * 5. **`getReadFilter(P)`** — `undefined` ⇒ composed, no row restriction; - * the deny sentinel `RLS_DENY_FILTER`, by identity, or a throw ⇒ - * `$in: []`; any other filter ⇒ composed with it as the scope, marked - * `'policy'` and otherwise passed exactly as `P`'s own read passes its - * injected scope. The tenant wall is the driver's (D3). - * - * [ADR-0139 D5.1, as amended] A **delegated (on-behalf-of) caller** is - * composed, not refused: both halves are asked with the caller's context - * exactly as it stands — `onBehalfOf` intact — and the security service - * answers each half for the delegated principal, which under ADR-0090 D10 - * is the INTERSECTION of the principal's leg and the delegator's. The - * engine composes what it is answered and derives no second intersection - * of its own: it has one context, the delegated one, and the delegator's - * grants are the security service's to resolve. A service that cannot - * answer that context answers the deny sentinel, and step 5 turns it into - * `$in: []`, closed. - * - * [ADR-0139 D6, as amended — the nested master chain] A scope the service - * answers in step 5 may itself carry a leaf: a parent whose own read scope - * is derived from ITS master (a `controlled_by_parent` detail whose master - * is `controlled_by_parent` too) is scoped `{ master_fk: { $in: - * ReadableIds.of(G) } }`, and a leaf handed to a driver uncomposed is - * refused there (D3), never read as "no restriction". So every leaf a - * composed scope carries is composed the same way, BEFORE the leaf that - * encloses it — the same five steps, the same caller context at every level, - * recursively ({@link composeReadableIdsChain}). The driver then receives one - * composed leaf whose scope holds composed leaves, and compiles one statement - * with the subqueries nested. Each parent is asked once per read, at - * whichever level it first appears; a chain that returns to a parent already - * on its branch, or runs deeper than {@link READABLE_IDS_MAX_CHAIN_DEPTH} - * scopes, is refused by name (D5's envelope), not composed partway. - * - * Nothing is memoised across calls. `aggregate`'s `having` and its - * aggregations' own filters are judged here too, so a leaf a middleware - * wrote there is refused rather than evaluated in memory. - */ - private async composeReadableIdsAtBoundary( + private composeReadableIdsAtBoundary( object: string, operation: 'find' | 'findOne' | 'count' | 'aggregate', ast: A, context: ExecutionContext | undefined, ): Promise { - if (operation === 'aggregate') { - collectComposableReadableIds(object, operation, ast.having, 'having'); - const aggregations = Array.isArray(ast.aggregations) ? ast.aggregations : []; - aggregations.forEach((agg, i) => { - collectComposableReadableIds(object, operation, (agg as { filter?: unknown } | undefined)?.filter, `aggregations[${i}].filter`); - }); - } - const sites = collectComposableReadableIds(object, operation, ast.where, 'where'); - if (sites.length === 0) return ast; - - const security = this.resolveReadableIdsSecurity(); - if (security.kind !== 'usable' && !this.readableIdsSecurityReported.has(security.kind)) { - this.readableIdsSecurityReported.add(security.kind); - this.logger.warn( - `[ObjectQL] ${operation}('${object}') composes a ReadableIds leaf with no usable "security" service ` + - `(${security.kind === 'absent' ? 'none is registered' : security.why}): every such leaf is answered ` + - '$in: [], no row of its parent, fail-closed. Register a security service for the leaf to serve rows. ' + - 'Reported once for this engine; later leaves are answered the same way.', - ); - } - const chain: ReadableIdsChain = { object, operation, security, context, answers: new Map() }; - const where = await this.composeReadableIdsChain(chain, ast.where, sites, []); - return { ...ast, where }; + // [#29986 codemod] The body moved to `engine-security-resolvers.ts`, its docblock with it (see `SecurityResolversHost`). + const readLogger = () => this.logger; + const readReadableIdsSecurityReported = () => this.readableIdsSecurityReported; + const host: ThisParameterType = { + composeReadableIdsChain: (...a) => this.composeReadableIdsChain(...a), + get logger() { return readLogger(); }, + get readableIdsSecurityReported() { return readReadableIdsSecurityReported(); }, + resolveReadableIdsSecurity: (...a) => this.resolveReadableIdsSecurity(...a), + }; + return (SecurityResolversFacet.composeReadableIdsAtBoundary).call(host, object, operation, ast, context); } /** @@ -2081,47 +2031,30 @@ export class ObjectQL implements IObjectQLEngine { return (composeReadableIdsChain).call(host, chain, node, sites, ancestors); } - /** - * [ADR-0139 D5] Refuse D5's two remaining classes (D5.2 and D5.3), by name, - * and return the distinct parent objects of `sites` the security service is - * asked about, in walk order (every one but a parent the registry does not - * hold). Reads the registry and the middleware registrations only — no - * security call, and no reading of the caller's context: a delegated - * (on-behalf-of) caller is no longer a class of its own (D5.1, as amended), - * it is composed like any other through the security service's answers. - */ private refuseUncomposableReadableIdsClasses( object: string, operation: string, sites: readonly ComposableReadableIdsSite[], ): string[] { - // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + // [#29986 codemod] The body moved to `engine-security-resolvers.ts`, its docblock with it (see `SecurityResolversHost`). const read_registry = () => this._registry; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, uncomposableReadableIdsParent: (...a) => this.uncomposableReadableIdsParent(...a), }; - return refuseUncomposableReadableIdsClasses.call(host, object, operation, sites); + return SecurityResolversFacet.refuseUncomposableReadableIdsClasses.call(host, object, operation, sites); } - /** - * [ADR-0139 D5.2 / D5.3] Which of D5's two remaining classes a leaf over - * `parent` in a read of `object` falls in, or `undefined` when it falls in - * neither — the ONE predicate behind both the read-time refusal - * ({@link refuseUncomposableReadableIdsClasses}) and the boot-time refusal of - * a `controlled_by_parent` detail over such a master - * ({@link findUnservableControlledByParentMasters}), so the two rules cannot - * drift. D5.2 is judged first, as the read path always has. - */ private uncomposableReadableIdsParent(object: string, parent: string): | { class: 'object-scoped-read-middleware' } | { class: 'other-datasource'; objectDatasource: string | undefined; parentDatasource: string | undefined } | undefined { - if (this.hasObjectMiddleware(parent)) return { class: 'object-scoped-read-middleware' }; - const objectDatasource = this.resolveDatasourceBinding(object)?.datasource; - const parentDatasource = this.resolveDatasourceBinding(parent)?.datasource; - if (parentDatasource !== objectDatasource) return { class: 'other-datasource', objectDatasource, parentDatasource }; - return undefined; + // [#29986 codemod] The body moved to `engine-security-resolvers.ts`, its docblock with it (see `SecurityResolversHost`). + const host: ThisParameterType = { + hasObjectMiddleware: (...a) => this.hasObjectMiddleware(...a), + resolveDatasourceBinding: (...a) => this.resolveDatasourceBinding(...a), + }; + return SecurityResolversFacet.uncomposableReadableIdsParent.call(host, object, parent); } /** @@ -2293,29 +2226,6 @@ export class ObjectQL implements IObjectQLEngine { */ private readonly formulaHopReads = new WeakSet(); - /** - * [#24197] The related rows a formula plan reads ONE hop through a - * reference field, resolved for a whole page of `rows` at once — the - * formula half of {@link resolveOneHopRelated}, read under the CALLER's - * own authority. - * - * ## What it costs - * - * Nothing for a plan that traverses nothing (no query, no closure). For one - * that does: one batched read per reference field per call — per PAGE of a - * `find` (the rows the driver returned, after `limit` / `offset`), per row of - * a `findOne`, per write response — never one per row. A formula whose - * traversal cannot be served (`hopConflict`) asks for nothing. - * - * ## One hop, and only one - * - * A `find` that IS a hop read ({@link formulaHopReads}) serves no hop of its - * own: the related object's own formulas over its own columns are computed, - * one that reads through a reference of ITS own reads `null` and says so. - * That is the declared depth (a second hop is not loaded — the lint refuses - * one written out, this refuses one reached through a formula), and it is - * what keeps two objects whose titles read each other from recursing. - */ private resolveFormulaRelated( schema: any, plan: readonly FormulaPlanEntry[], @@ -2323,15 +2233,15 @@ export class ObjectQL implements IObjectQLEngine { context: unknown, query?: EngineQueryOptions, ): Promise { - // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + // [#29986 codemod] The body moved to `engine-security-resolvers.ts`, its docblock with it (see `SecurityResolversHost`). const read_registry = () => this._registry; const readFormulaHopReads = () => this.formulaHopReads; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get formulaHopReads() { return readFormulaHopReads(); }, resolveOneHopRelated: (...a) => this.resolveOneHopRelated(...a), }; - return resolveFormulaRelated.call(host, schema, plan, rows, context, query); + return SecurityResolversFacet.resolveFormulaRelated.call(host, schema, plan, rows, context, query); } /** @@ -2369,37 +2279,36 @@ export class ObjectQL implements IObjectQLEngine { * `objectName`, as {@link evaluateFormulaField} does. Evaluates against a * shallow COPY of `record`. */ - async serveFormulaField( + serveFormulaField( objectName: string, record: Record, field: string, context?: ExecutionContext, ): Promise { - const schema = this._registry.getObject(objectName); - const { plan } = planFormulaProjection(schema, [field]); - if (plan.length === 0) return undefined; - const scratch: Record = { ...record }; - applyFormulaPlan(plan, [scratch], context, await this.resolveFormulaPermissions( - objectName, plan, [scratch], context, this.permissionResolution(context), - ), this.formulaFaultSink(objectName), await this.resolveFormulaRelated(schema, plan, [scratch], context), - this.formulaOptionLabels(schema, plan, context)); - return scratch[field]; + // [#29986 codemod] The body moved to `engine-security-resolvers.ts` (see `SecurityResolversHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + formulaFaultSink: (...a) => this.formulaFaultSink(...a), + formulaOptionLabels: (...a) => this.formulaOptionLabels(...a), + permissionResolution: (...a) => this.permissionResolution(...a), + resolveFormulaPermissions: (...a) => this.resolveFormulaPermissions(...a), + resolveFormulaRelated: (...a) => this.resolveFormulaRelated(...a), + }; + return SecurityResolversFacet.serveFormulaField.call(host, objectName, record, field, context); } - /** - * [#26608] The option labels `label(record.)` reads on a read or a - * write response of `schema`'s object: in the reader's locale - * (`ExecutionContext.locale`, which the request's `Accept-Language` sets), - * translated through the bridged i18n service as `GET /meta/object/:name` - * serves the object to the same reader — see {@link readerOptionLabels}. - * `undefined` when no formula of `plan` calls `label()`. - */ private formulaOptionLabels( schema: unknown, plan: FormulaPlanEntry[], context: { locale?: string } | undefined, ): EvalOptionLabels | undefined { - return readerOptionLabels(schema, plan, context?.locale, this.i18nService); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts`, its docblock with it (see `SecurityResolversHost`). + const readI18nService = () => this.i18nService; + const host: ThisParameterType = { + get i18nService() { return readI18nService(); }, + }; + return SecurityResolversFacet.formulaOptionLabels.call(host, schema, plan, context); } /** @@ -2408,12 +2317,12 @@ export class ObjectQL implements IObjectQLEngine { */ private readonly formulaFaultReported = new Set(); - /** - * [#22019] The sink a read or a write response of `object` hands to - * {@link applyFormulaPlan}, bound to the object so a report names the pair. - */ private formulaFaultSink(object: string): FormulaFaultSink { - return (field, error) => this.reportFormulaFault(object, field, error); + // [#29986 codemod] The body moved to `engine-security-resolvers.ts`, its docblock with it (see `SecurityResolversHost`). + const host: ThisParameterType = { + reportFormulaFault: (...a) => this.reportFormulaFault(...a), + }; + return SecurityResolversFacet.formulaFaultSink.call(host, object); } /** @@ -2542,32 +2451,39 @@ export class ObjectQL implements IObjectQLEngine { * @returns a disposer that detaches the bridge. */ attachAuthzInvalidationPubSub(pubsub: IPubSub, nodeId: string): () => void { - if ( - this.authzInvalidationBinding?.pubsub === pubsub && - this.authzInvalidationBinding?.nodeId === nodeId - ) { - return () => this.detachAuthzInvalidationPubSub(); - } - this.detachAuthzInvalidationPubSub(); - this.authzInvalidationDetach = bridgeAuthzInvalidation({ - epoch: this.writeEpoch, - pubsub, - nodeId, - logger: this.logger, - }); - this.authzInvalidationBinding = { pubsub, nodeId }; - this.logger.info('ObjectQL attached to the authz.invalidated cluster channel', { - nodeId, - }); - return () => this.detachAuthzInvalidationPubSub(); + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + const readAuthzInvalidationBinding = () => this.authzInvalidationBinding; + const writeAuthzInvalidationBinding = (v: ObjectQL['authzInvalidationBinding']) => { this.authzInvalidationBinding = v; }; + const readAuthzInvalidationDetach = () => this.authzInvalidationDetach; + const writeAuthzInvalidationDetach = (v: ObjectQL['authzInvalidationDetach']) => { this.authzInvalidationDetach = v; }; + const readLogger = () => this.logger; + const readWriteEpoch = () => this.writeEpoch; + const host: ThisParameterType = { + get authzInvalidationBinding() { return readAuthzInvalidationBinding(); }, + set authzInvalidationBinding(v) { writeAuthzInvalidationBinding(v); }, + get authzInvalidationDetach() { return readAuthzInvalidationDetach(); }, + set authzInvalidationDetach(v) { writeAuthzInvalidationDetach(v); }, + detachAuthzInvalidationPubSub: (...a) => this.detachAuthzInvalidationPubSub(...a), + get logger() { return readLogger(); }, + get writeEpoch() { return readWriteEpoch(); }, + }; + return MiddlewareAndContextFacet.attachAuthzInvalidationPubSub.call(host, pubsub, nodeId); } /** Tear down the `authz.invalidated` bridge. Safe to call multiple times. */ detachAuthzInvalidationPubSub(): void { - const detach = this.authzInvalidationDetach; - this.authzInvalidationDetach = undefined; - this.authzInvalidationBinding = undefined; - detach?.(); + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + const readAuthzInvalidationBinding = () => this.authzInvalidationBinding; + const writeAuthzInvalidationBinding = (v: ObjectQL['authzInvalidationBinding']) => { this.authzInvalidationBinding = v; }; + const readAuthzInvalidationDetach = () => this.authzInvalidationDetach; + const writeAuthzInvalidationDetach = (v: ObjectQL['authzInvalidationDetach']) => { this.authzInvalidationDetach = v; }; + const host: ThisParameterType = { + get authzInvalidationBinding() { return readAuthzInvalidationBinding(); }, + set authzInvalidationBinding(v) { writeAuthzInvalidationBinding(v); }, + get authzInvalidationDetach() { return readAuthzInvalidationDetach(); }, + set authzInvalidationDetach(v) { writeAuthzInvalidationDetach(v); }, + }; + return MiddlewareAndContextFacet.detachAuthzInvalidationPubSub.call(host); } /** @@ -2577,8 +2493,14 @@ export class ObjectQL implements IObjectQLEngine { * @param options Optional: target object filter */ registerMiddleware(fn: EngineMiddleware, options?: { object?: string }): void { - this.middlewares.push({ fn, object: options?.object }); - this.logger.debug('Registered middleware', { object: options?.object, total: this.middlewares.length }); + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + const readLogger = () => this.logger; + const readMiddlewares = () => this.middlewares; + const host: ThisParameterType = { + get logger() { return readLogger(); }, + get middlewares() { return readMiddlewares(); }, + }; + return MiddlewareAndContextFacet.registerMiddleware.call(host, fn, options); } /** @@ -2596,21 +2518,23 @@ export class ObjectQL implements IObjectQLEngine { * decline an object whose gates live here and let this engine serve it. */ hasObjectMiddleware(objectName: string): boolean { - return this.middlewares.some((m) => !!m.object && m.object !== '*' && m.object === objectName); + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + const readMiddlewares = () => this.middlewares; + const host: ThisParameterType = { + get middlewares() { return readMiddlewares(); }, + }; + return MiddlewareAndContextFacet.hasObjectMiddleware.call(host, objectName); } - /** - * Execute an operation through the middleware chain - */ private executeWithMiddleware(ctx: OperationContext, executor: () => Promise): Promise { - // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it (see `MiddlewareAndContextHost`). const readMiddlewares = () => this.middlewares; const readWriteEpoch = () => this.writeEpoch; - const host: ThisParameterType = { + const host: ThisParameterType = { get middlewares() { return readMiddlewares(); }, get writeEpoch() { return readWriteEpoch(); }, }; - return executeWithMiddleware.call(host, ctx, executor); + return MiddlewareAndContextFacet.executeWithMiddleware.call(host, ctx, executor); } /** @@ -2662,52 +2586,18 @@ export class ObjectQL implements IObjectQLEngine { return buildProvenance(execCtx); } - /** - * [commit 34ce8e7db] Build the declared `HookContext.referentialFieldClear` marker — - * the read-only hook-context projection of the operation-private - * `__referentialFieldClear` that {@link ObjectQL.cascadeDeleteRelations} - * stamps on the cleanup write's ExecutionContext (#3023). - * - * `true | undefined`, never `false`: the key is ABSENT from every dispatch - * whose envelope does not carry the marker, so a handler reads it as - * `ctx.referentialFieldClear === true` and an absent key keeps meaning "not - * a referential cleanup" — the same back-compatible direction as `dispatch`. - * - * Why a projection of the ENVELOPE rather than a fresh per-call-site signal: - * the `__` key's semantics have always been operation-scoped — the cleanup - * context is `{ ...callerContext, transaction, __referentialFieldClear: true }`, - * plugin-security's ownership-anchor exemption keys on the envelope, and a - * write a hook issues through `ctx.api` during a cleanup dispatch inherits - * it — so the declared face projects the fact faithfully instead of quietly - * re-scoping it. Consumed by `update()`'s hook-context assembly only: every - * reference-cleanup write the engine issues is an update, and the per-row / - * after-phase contexts ride the batch context's spread. - */ private buildReferentialFieldClear(execCtx?: ExecutionContext): true | undefined { - return (execCtx as any)?.__referentialFieldClear === true ? true : undefined; + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it. + return MiddlewareAndContextFacet.buildReferentialFieldClear(execCtx); } - /** - * [#23221] Build the declared `HookContext.referentialCascadeDelete` marker, - * the delete twin of {@link buildReferentialFieldClear}: `true` when THIS - * `delete()` of `object`/`id` is the child delete - * {@link walkReferencingRelations} issued for a `cascade` relation, and - * `undefined` for every other delete — the same "absent unless true" - * contract, so a handler reads `ctx.referentialCascadeDelete === true`. - * - * Reads {@link cascadeIssuedDeletes}, not the envelope (that field says why), - * and CLAIMS the token it matches. Called once per `delete()`, below the - * `afterDelete` unit re-entry (`opensAfterAbortUnit`): a call that re-enters - * itself returns before reaching here, so the claim is made by the inner - * call, the one that dispatches the hooks, and nothing later inside it can - * match the same token again. - */ private claimReferentialCascadeDelete(object: string, id: unknown): true | undefined { - const issued = this.cascadeIssuedDeletes.getStore(); - if (!issued || issued.claimed || id == null) return undefined; - if (issued.object !== object || issued.id !== String(id)) return undefined; - issued.claimed = true; - return true; + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it (see `MiddlewareAndContextHost`). + const readCascadeIssuedDeletes = () => this.cascadeIssuedDeletes; + const host: ThisParameterType = { + get cascadeIssuedDeletes() { return readCascadeIssuedDeletes(); }, + }; + return MiddlewareAndContextFacet.claimReferentialCascadeDelete.call(host, object, id); } /** @@ -2948,283 +2838,67 @@ export class ObjectQL implements IObjectQLEngine { return resolveEnginePosture.call(host); } - /** - * [#8844] The install's organizations, capped at two — only "none / exactly - * one / several" changes any decision. - * - * Memoised because a cron loop inserting N rows would otherwise pay N probes - * for an answer that changes only when `sys_organization` itself is written, - * and every such write passes through this engine's own write faces, which - * clear it ({@link invalidateOrganizationProbe}). The stale window is - * therefore an organization created by ANOTHER process against the same - * database — and on the `single` posture this memo serves, a second - * organization appearing is the ambiguous topology the next boot refuses - * anyway. - * - * `isSystem` on the read: resolving the install's organization must not - * depend on the caller's own reach, and this runs on writes that have no - * caller reach at all. - * - * [#9261] "The probe found no organizations" and "the probe could not run" - * are different facts (ADR-0110 D3), and this returns the FIRST one only when - * it measured it. The `catch {}` this replaces answered both with `[]`, which - * `resolveSystemWriteOrganization` then read as "no organization yet" (a - * branch that landed the row; since ADR-0131 D9 a registered-but-empty - * organization object is refused instead) — so one - * transient failure silently skipped both halves of the #8844 ruling: the - * single-organization stamp (rows land untenanted and fork exactly the - * counter the ruling exists to protect) and the multi-organization REFUSAL - * (fail-open on a guard that must be loud, the #8895 shape). The memo made - * the damage outlive the outage — the invented answer was cached until an - * organization write happened to clear it. - * - * **Exactly one cause is benign, and it was measured rather than assumed.** - * `sys_organization` ROUTES but its TABLE was never provisioned (schema sync - * not run yet): it cannot hold a row, so zero really is the count. Asked - * through the shared `isMissingTableError` predicate - * (`@objectstack/metadata/errors`, #4825) — the same call - * `resolveFileReferences` and `cascadeDeleteRelations` make — never a - * hand-rolled code test. - * - * A SECOND benign cause, now that an unresolved name is refused rather than - * handed to the driver: `sys_organization` is not registered at all (a lean - * embedding, a bare-kernel test). Before an in-process verb refused an - * unresolved name, that install reached the driver and read `[]` on a - * tolerant one or a MISSING TABLE on a strict one — the one driver cause - * above. It now resolves to the engine's own `OBJECT_NOT_FOUND` before any - * driver is asked, so this probe asks the REGISTRY first and treats an - * absent `sys_organization` as the lean-install case it always was: there is - * no organization object here, so there is no organization to derive. This - * is the probe handling absence itself, on a path a body cannot reach — it - * never relies on the resolver's old raw-table fall-through, which is now - * gone. It is not a spelling allow-list: the question is "is the object - * provisioned," asked of the one registry, not "is this one of N names." - * - * Everything else — connection loss, pool exhaustion, a timeout mid-boot, a - * datasource that never connected ({@link DatasourceUnavailableError}), a - * permission denial — means organizations may well exist and simply were not - * seen. It PROPAGATES, envelope intact, and ⛔ nothing is memoised on that - * path: the write that asked fails loudly instead of being filed under a - * guessed topology, and the next write re-probes rather than inheriting the - * guess. No new error code and no new response field — the caller receives - * the probe's own failure, the same disposition #8895 gave the - * referential-integrity probe. - */ - private async probeInstallOrganizations(): Promise { - if (this.organizationProbeMemo) return this.organizationProbeMemo; - // The lean-install case: no `sys_organization` object is registered, so - // there is no organization to derive. Asking the registry first keeps the - // probe off the resolver's refusal for an unregistered name (a path a body - // cannot reach), exactly as the old raw-table fall-through read `[]`. - if (!this._registry.getObject(ORGANIZATION_OBJECT)) { - this.organizationProbeMemo = []; - return this.organizationProbeMemo; - } - let ids: readonly string[] = []; - try { - const rows = await this.find(ORGANIZATION_OBJECT, { - fields: ['id'], - limit: 2, - context: { isSystem: true }, - }); - ids = (Array.isArray(rows) ? rows : []) - .map((r: any) => (r?.id ?? r?._id)) - .filter((id: unknown) => id != null && String(id) !== '') - .map((id: unknown) => String(id)); - } catch (error) { - // The one benign cause only — anything else is an outage, not an - // emptiness, and must not be memoised as one. - if (!isMissingTableError(error, ORGANIZATION_OBJECT)) throw error; - ids = []; - } - this.organizationProbeMemo = ids; - return ids; + private probeInstallOrganizations(): Promise { + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it (see `MiddlewareAndContextHost`). + const read_registry = () => this._registry; + const readOrganizationProbeMemo = () => this.organizationProbeMemo; + const writeOrganizationProbeMemo = (v: ObjectQL['organizationProbeMemo']) => { this.organizationProbeMemo = v; }; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + find: (...a) => this.find(...a), + get organizationProbeMemo() { return readOrganizationProbeMemo(); }, + set organizationProbeMemo(v) { writeOrganizationProbeMemo(v); }, + }; + return MiddlewareAndContextFacet.probeInstallOrganizations.call(host); } - /** [#8844] Drop the memo — any write that can change the organization set. */ private invalidateOrganizationProbe(): void { - this.organizationProbeMemo = undefined; + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it (see `MiddlewareAndContextHost`). + const readOrganizationProbeMemo = () => this.organizationProbeMemo; + const writeOrganizationProbeMemo = (v: ObjectQL['organizationProbeMemo']) => { this.organizationProbeMemo = v; }; + const host: ThisParameterType = { + get organizationProbeMemo() { return readOrganizationProbeMemo(); }, + set organizationProbeMemo(v) { writeOrganizationProbeMemo(v); }, + }; + return MiddlewareAndContextFacet.invalidateOrganizationProbe.call(host); } - /** - * [#8844] Resolve the organization a SYSTEM-context insert carries, per the - * 2026-08-15 ruling (Option 1) — or refuse it. - * - * Returns the organization id to thread as `DriverOptions.tenantId` (the same - * knob a session write sets, so the driver stamps the column and the counter - * scopes by it), or `undefined` when there is nothing to resolve. Throws - * {@link SystemWriteOrganizationRequiredError} wherever no owner is - * derivable (ADR-0131 D9): a wall, several organizations, or — under - * `single`, in a composition that registers the organization object — none. - * - * Called AFTER the beforeInsert hooks on purpose: a hook that stamps the - * organization itself has carried it, and must not then be refused for a - * value it just supplied. - * - * Every early return below is a population the refusal must not touch — see - * the module header for why each is outside the rule by construction rather - * than by exemption. They are ordered cheapest-first, so an ordinary write - * pays two property reads and a regexp. - */ private resolveSystemInsertOrganization( object: string, execCtx: ExecutionContext | undefined, rows: readonly Record[], ): Promise { - // [#29986 codemod] The body moved to `engine-middleware-and-context.ts` (see `MiddlewareAndContextHost`). + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it (see `MiddlewareAndContextHost`). const read_registry = () => this._registry; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, probeInstallOrganizations: (...a) => this.probeInstallOrganizations(...a), resolveEnginePosture: (...a) => this.resolveEnginePosture(...a), }; - return resolveSystemInsertOrganization.call(host, object, execCtx, rows); + return MiddlewareAndContextFacet.resolveSystemInsertOrganization.call(host, object, execCtx, rows); } - /** - * Does the open transaction `tx` actually cover the driver `object` resolves - * to? — the same-origin question, asked by instance IDENTITY (#5351). - * - * Answers `true` in two cases: the resolved driver IS the transaction's - * owner, or the engine cannot tell who the owner is. The second case is the - * DECLARED LIMIT of this gate, not an oversight, and it is exactly one - * shape: a handle the engine never opened and cannot attribute. - * - * `TransactionScope` (#5724) records the owner for every transaction the - * engine opens, and `transaction()` also threads that same handle down as - * `execCtx.transaction`, so the dominant explicit-threading path is covered - * by identity-matching the handle back to the store entry. What is NOT - * covered: - * - * - `ScopedContext`'s discrete `beginTransaction`/`commit`/`rollback` trio, - * which threads the handle across `setImmediate` boundaries where - * AsyncLocalStorage does not survive, and so never populates txStore; - * - any handle an outside caller obtained elsewhere and passed in as - * `execCtx.transaction`. - * - * For those the engine holds an opaque driver object with no back-reference - * to its driver, so there is no honest comparison to make. Guessing — say, - * assuming an unattributed handle belongs to the default driver — would - * refuse legitimate single-datasource work on one side and carve out writes - * that were genuinely covered on the other. So the gate declines to judge and - * the pre-#5351 behaviour stands on that path: recorded in the ADR-0067/0119 - * revision, and closable only by making handle ownership discoverable on - * `IDataDriver` (filed separately). - */ private transactionCoversDriverFor(object: string, tx: unknown): boolean { - const store = this.txStore.getStore(); - // The scope describes the handle in the store. An explicitly-threaded - // handle is covered by it only when it IS that handle. - const scope = store !== undefined && tx === store.transaction ? store.scope : undefined; - if (!scope) return true; - return this.getDriver(object) === scope.driver; + // [#29986 codemod] The body moved to `engine-middleware-and-context.ts`, its docblock with it (see `MiddlewareAndContextHost`). + const readTxStore = () => this.txStore; + const host: ThisParameterType = { + getDriver: (...a) => this.getDriver(...a), + get txStore() { return readTxStore(); }, + }; + return MiddlewareAndContextFacet.transactionCoversDriverFor.call(host, object, tx); } - /** - * Resolve the `NOW()` runtime token into the value the field's declared type - * actually stores (#4597). - * - * The engine — not the driver — owns this resolution, exactly as it owns - * `current_user`. `NOW()` is the mirror of #4560's crack: `current_user` was - * known to the engine and not to the DDL, so the DDL stored the token text; - * `NOW()` was known to the SQL driver and not to the engine, so every - * non-SQL datasource got the token text instead of a time. Resolving here - * makes one answer serve every driver. - * - * The shapes below are NOT invented: they are the canonical storage forms - * ADR-0053 already fixed and `SqlDriver.nowColumnDefault` already emits - * per type. Producing a full instant for a `Field.date` would trade the old - * cross-driver drift for a new one (SQL would keep collapsing it to a - * calendar day in `formatInput`, memory/mongodb would not), so the token - * resolves per declared type: - * - * | field type | stored form | matches | - * |---|---|---| - * | `date` | `YYYY-MM-DD` (UTC day) | `toDateOnly` / the `date` column DEFAULT | - * | `time` | `HH:MM:SS[.fff]` (UTC wall clock, `.000` trimmed) | `canonicalTimeOfDay` | - * | anything else | `YYYY-MM-DDTHH:MM:SS.sssZ` | `canonicalUtcDatetime` | - * - * "Anything else" deliberately includes non-temporal fields: a `text` field - * that opts into `NOW()` gets the instant, which is what the SQL column - * DEFAULT gives it today. - * - * `instant` is the caller's per-insert `now` snapshot when the `NOW()` token - * is what asked, so two defaulted fields on one record cannot straddle a - * millisecond boundary. - * - * It is NOT only the token's, though (#7373): this is the one place that - * knows what shape a declared type stores an instant in, so the CEL branch's - * {@link normalizeExpressionDefault} routes its own `Date` results through - * the SAME table rather than growing a second copy of it. Hence the - * parameter is an arbitrary `instant`, not "now" — `daysFromNow(7)` is a - * week out and takes the identical per-type treatment. - */ private resolveNowDefault(fieldType: unknown, instant: Date): string { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`. - return resolveNowDefault(fieldType, instant); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it. + return DefaultsAndAutonumbersFacet.resolveNowDefault(fieldType, instant); } - /** - * Put a CEL `defaultValue`'s evaluated result into the storage shape the - * field's declared type contracts for (#7373). - * - * The counterpart the expression branch was missing. `applyFieldDefaults` - * has three ways to produce a default and, before this, only two of them - * honoured the stored-value contract: the `NOW()` token routed through - * {@link resolveNowDefault}, a literal is checked against - * `valueSchemaFor(def, 'stored')` at AUTHOR time (#7127), and the expression - * envelope's result was assigned verbatim. But the temporal stdlib returns a - * JS `Date` — ADR-0053 D1 fixes `today()` / `daysFromNow(n)` / `daysAgo(n)` - * as UTC-midnight of the reference-tz calendar day, and `now()` as the raw - * instant — while `valueSchemaFor` says a stored instant is an ISO-8601 - * STRING. So `{ dialect: 'cel', source: 'daysFromNow(7)' }` on a `datetime` - * put a `Date` object in the column: a value the platform's own - * `os migrate value-shapes` scan reports as a violation. - * - * Why NORMALIZE rather than refuse the `Date` — the two shapes this could - * have taken, decided on measurement: - * - * - **The drivers already agree with this answer.** Handed a `Date`, - * `SqlDriver.formatInput` coerces it through `canonicalUtcDatetime` - * (`toISOString()`) and `toDateOnly` (`YYYY-MM-DD`); mongodb's - * `storageDatetimeValue` keeps the BSON `Date`, which is what parsing the - * ISO string produces anyway, and its `storageDateValue` collapses to the - * same `YYYY-MM-DD`. Normalizing here is therefore byte-identical on - * every SQL and mongodb-backed store and changes exactly one thing: the - * memory driver, which applies its temporal canon to filter comparands - * only (`coerceTemporalValue`) and stores writes as handed. That is the - * whole defect, and this is the smallest change that closes it. - * - **Refusal would single out one writer.** `validateRecord` accepts a - * `Date` on `date`/`datetime` from ANY caller by explicit decision - * (`if (value instanceof Date) return null`), and temporal types are not - * in ADR-0104's strict value-shape block at all. Refusing a `Date` only - * when a CEL default produced it would make the rule depend on who wrote - * the value rather than on what the value is — and would break the - * documented envelope (#7244) on precisely the SQL backends where it - * stores correctly today. - * - * This is the `NOW()` crack of #4597 / #4560 in its third form: a value the - * SQL driver silently repaired at the wire and non-SQL datasources did not, - * so one declaration stored two shapes depending on the datasource. Resolved - * engine-side, one answer serves every driver; the driver coercions stay as - * defence in depth for writes that bypass the engine. - * - * No day can shift here. ADR-0053 D1's `Date` is UTC-midnight OF the - * reference-tz calendar day, and {@link resolveNowDefault} reads it back - * with `toISOString()` — UTC getters, the same `getUTC*` reading the ADR - * names for the driver filter path. Reading those parts in LOCAL time is the - * move that would shift a day, and nothing here does it. - * - * Non-`Date` results (a string, a number, a bool, a list) pass through - * untouched: this normalizes temporal FORM, it does not police the contract, - * and a CEL default is documented as "result type is a runtime concern". - * An `Invalid Date` passes through too — the same totality the driver - * canons keep, so a value nothing can interpret is never silently rewritten - * into a wrong one. - */ private normalizeExpressionDefault(fieldType: unknown, value: unknown): unknown { - if (!(value instanceof Date) || Number.isNaN(value.getTime())) return value; - return this.resolveNowDefault(fieldType, value); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). + const host: ThisParameterType = { + resolveNowDefault: (...a) => this.resolveNowDefault(...a), + }; + return DefaultsAndAutonumbersFacet.normalizeExpressionDefault.call(host, fieldType, value); } /** @@ -3238,109 +2912,11 @@ export class ObjectQL implements IObjectQLEngine { return new ScopedContext(safeCtx, this as unknown as IDataEngine); } - /** - * The initial value a field's OPTION LIST declares — the option marked - * `default: true` — or `undefined` when it declares none (#7246). - * - * `SelectOption.default` has been authorable and spec-valid since the schema - * was written, and until this method nothing on the insert path read it: a - * create that omitted the field stored `null`, not the marked option. The one - * consumer anywhere was lint's `isNullableField`, which concluded the column - * was always-valued — a **build-breaking** verdict resting on a declaration - * the engine did not honour. Enforcing it here is what makes that heuristic - * true rather than merely asserted. - * - * Deliberate decisions, each of which could reasonably have gone the other - * way: - * - * - **Precedence: `defaultValue` wins.** Not decided here — decided by the - * caller, which only reaches this method when `defaultValue == null`. The - * field-level key is the more specific declaration (it names a value for - * THIS field; the option flag describes the shared option list), and it is - * the spelling every other consumer already honours. When both are - * declared and disagree, the option flag is inert exactly as it is today. - * - * - **Type-agnostic, like the lint heuristic.** `options` is a field-level - * key on `FieldSchema`, not gated on `type: 'select'`, and - * `isNullableField` reads it without a type test. Matching that keeps the - * two sides honest BY CONSTRUCTION: there is no field the lint calls - * always-valued that this method leaves empty. - * - * - **A MULTI-VALUED field assembles an ARRAY.** That field stores an - * Array/JSON, so the shape of its default follows the field, not the - * number of marked options — one marked option on a multi-select defaults - * to a one-element array, never a bare scalar that the driver would then - * store with the wrong shape. [#18408] Multi-valued is - * {@link declaredMultiValued}, ⛔ not a raw `field.multiple`: the flag is - * inert on a type outside the two multi sets and redundant on a - * `multiselect` / `checkboxes` / `tags`, which stores an array with or - * without it — so reading the flag made this method disagree with the - * validator one frame later (`normalizeMultiValueFields` has asked the - * predicate all along) and with the column the driver builds. - * - * Refusing (throwing) was rejected: the metadata is - * spec-valid, and a runtime throw on spec-valid input is a worse answer - * than a well-defined value. Ignoring it was rejected too — it would - * preserve, for multi-selects only, precisely the inertness this change - * removes, and an author cannot see that carve-out from the schema. - * - * - **Several options marked on a SINGLE-valued field: the FIRST wins.** - * There is one slot, so declaration order decides — deterministic, and - * the same option a picker preselects when it takes the first match. - * Neither the `Field.select` builder nor the schema dedupes the flag, so - * this case is reachable; the alternative (throw) again fails a - * spec-valid declaration at runtime. Nothing in the shipped corpus marks - * more than one. - * - * - **Only the canonical `default` spelling is read.** The authoring - * aliases (`isDefault`, `selected`) are normalized by - * `SelectOptionSchema`'s alias layer before metadata reaches the engine, - * and lint reads the canonical key only. Reading the aliases here would - * make the engine honour raw shapes lint calls nullable — the two would - * disagree in the direction that produces a false "always-valued". - */ private resolveOptionDefault(field: { type?: unknown; options?: unknown; multiple?: unknown }): unknown { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`. - return resolveOptionDefault(field); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it. + return DefaultsAndAutonumbersFacet.resolveOptionDefault(field); } - /** - * Apply field defaults to an incoming insert payload. Defaults that are - * Expression envelopes (e.g. `{ dialect: 'cel', source: 'today()' }`, - * `{ dialect: 'cel', source: 'os.user.id' }`) are evaluated via - * `ExpressionEngine` against the calling user/org/now snapshot. The - * `defaultValue` runtime TOKENS (`@objectstack/spec/data`'s - * `DEFAULT_VALUE_TOKENS` — `current_user` and `NOW()`, the whole family) are - * resolved here, so one declaration behaves identically on every driver. - * Static defaults are applied verbatim. Records that already supplied a value - * for a field are left untouched. - * - * "Supplied a value" means the field is present with a non-null value. Both an - * OMITTED field (`undefined`) and an EXPLICIT `null` are treated as "not - * supplied" and get the default. This runs on the INSERT path only, where a - * `null` from a form (an unpicked control serializes to `null`, not omission) - * unambiguously means "no value"; the UPDATE path never calls this, so a - * deliberate "set to null" on update is preserved (#2706). Empty string `''` - * is a real, user-entered value and is left as-is. - * - * Implements ROADMAP §M9.9b — `defaultValue` accepts Expression so authors - * can replace "write a hook to default to today/current-user" with a - * declarative `defaultValue: cel\`today()\``. - * - * A field that declares NO `defaultValue` falls back to the option marked - * `default: true` ({@link resolveOptionDefault}, #7246) — the select idiom, - * which until then was authorable, spec-valid, and read by nothing on this - * path. - * - * [#20082] `permissions` is the acting subject's effective object-permission - * map, which an expression default calling `current_user.can(object, verb)` - * answers from. This method is synchronous, so the CALLER resolves it — once - * per write, only for a row that will be defaulted by such an expression - * ({@link resolveDefaultPermissions}, which also owns what a failed resolution - * does: it refuses the write before this runs). `undefined` is "no permission - * data": a `can` default then does not evaluate and takes this method's rule - * for every expression default that does not — left unset, with the warn below. - */ private applyFieldDefaults( object: string, record: Record, @@ -3348,183 +2924,45 @@ export class ObjectQL implements IObjectQLEngine { nowSnapshot?: Date, permissions?: EvalPermissions, ): Record { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { defaultFieldEntries: (...a) => this.defaultFieldEntries(...a), get logger() { return readLogger(); }, normalizeExpressionDefault: (...a) => this.normalizeExpressionDefault(...a), resolveNowDefault: (...a) => this.resolveNowDefault(...a), resolveOptionDefault: (...a) => this.resolveOptionDefault(...a), }; - return applyFieldDefaults.call(host, object, record, execCtx, nowSnapshot, permissions); + return DefaultsAndAutonumbersFacet.applyFieldDefaults.call(host, object, record, execCtx, nowSnapshot, permissions); } - /** - * The field entries {@link applyFieldDefaults} walks — one reading of an - * object's `fields`, shared with {@link resolveDefaultPermissions} so the - * planner and the evaluator see the same defaults. `undefined` when the object - * declares no fields, which is `applyFieldDefaults`' early return (it hands - * the SAME record reference back, and the insert path relies on knowing that). - */ private defaultFieldEntries(object: string): Array<{ name: string; type?: unknown; defaultValue?: unknown; options?: unknown; multiple?: unknown; }> | undefined { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). + const host: ThisParameterType = { getSchema: (...a) => this.getSchema(...a), }; - return defaultFieldEntries.call(host, object); + return DefaultsAndAutonumbersFacet.defaultFieldEntries.call(host, object); } - /** - * Generate values for empty `autonumber` fields on insert — ONLY for drivers - * that do not generate them natively (memory, mongodb). For SQL-backed objects - * the driver owns a persistent, atomic `_objectstack_sequences` table and - * advertises `supports.autonumber === true`; the engine then defers entirely - * and never pre-fills (so the persistent sequence is the single source of - * truth — see #1603). Required-validation exempts `autonumber` either way, so - * a `required` record number is never rejected for "missing" — the runtime - * owns the value, not the client. - * - * That ownership is now ENFORCED rather than merely asserted (#5503): the - * insert path strips a caller-supplied value before this runs, so the "respect - * explicit value" skip below is reached only by writers the strip exempts — - * `isSystem` (seed replay, migration) and an opt-in historical import - * (`preserveAudit`), plus values a `beforeInsert` hook computed server-side. - * For every other caller the slot arrives empty and the sequence issues the - * number. Do NOT "fix" a forged value here by overwriting it: the strip must - * stay upstream of this method, because a driver with - * `supports.autonumber === true` returns above without ever entering the loop. - * - * In the fallback path the next value is `max(existing) + 1`, seeded once per - * `object.field.` from the store then incremented in memory (monotonic - * within the process, resilient to deletions). The shared `autonumberFormat` - * renderer is honored end-to-end, so date tokens (`AD{YYYYMMDD}{0000}`), field - * interpolation (`{island_zone}{000}`) and per-scope reset behave identically - * to the SQL driver's persistent sequence (#1603). NOTE: this in-memory seeding - * is single-instance. - * - * # A format-LESS field renders through the contract default, not bare (#6555) - * - * "Identically to the SQL driver" was true of every DECLARED format and false - * of the one case nobody declares: with no format at all this method used to - * hand `parseAutonumberFormat` the empty string, whose empty token list - * `renderAutonumber` renders through its no-slot branch as a BARE counter — - * `1`, `2`, …. The SQL driver substituted `'{0000}'` in the same case and - * issued `0001`, `0002`, …. One metadata document, two number shapes, decided - * by which driver happened to serve it; the counter VALUE always agreed - * (#6468 pinned that), so what forked was rendering width alone. - * - * The maintainer ruling of 2026-08-08 (#6555, route 3) puts the default in the - * contract instead of in either fallback, and this method now reads it: - * {@link resolveAutonumberFormat} answers the canonical `autonumberFormat`, - * then the `format` shorthand (#1603), then `DEFAULT_AUTONUMBER_FORMAT` - * (`{0000}`). Two consequences, both deliberate: - * - * - A format-less field on this path now issues `0001` where it issued `1`. - * `{0000}` was chosen because it is the shape SQL deployments already - * store, so the flip lands on engine-fallback deployments only. - * - "Declared" is now a NON-EMPTY string (the SQL driver's long-standing - * truthiness rule) rather than the `??` this method used, so - * `autonumberFormat: ''` resolves to the default too instead of rendering - * bare — and an empty canonical key no longer masks a declared `format` - * shorthand. The bare counter is still reachable, by declaring a format - * with no `{0..0}` slot (`'PRE-'` → `PRE-1`). - * - * Seeding is untouched by all of this: `{0000}` renders prefix `''` and suffix - * `''`, so {@link readStoredAutonumberCounter} stays on its UNANCHORED legacy - * branch and reads already-stored bare values exactly as before. - * - * # Keeping the seeded counter in sync (#6806) - * - * Seeding "once per counter key" is only the truth while the engine is the - * ONLY writer of the field. It is not: the `continue` below is reached by - * every exempt writer (`isSystem` seed replay, a `preserveAudit` historical - * import, a `beforeInsert` hook stamp), and each of those persists a record - * number the counter never saw. The counter then keeps issuing from where the - * one-time seed left it — below the store's real max — and every number it - * issues up to that max is a duplicate business identifier. Two resyncs - * close that, from opposite ends: - * - * - **Adopt, here.** An exempt value passes through this very loop, so the - * counter can be lifted from it for FREE — one string parse, no query. - * {@link adoptExplicitAutonumber} does it, which makes the warm in-memory - * counter converge on what a cold re-seed of the same store would answer. - * This is the whole fix for in-process drift, and it costs nothing on the - * generating path (a caller-supplied value never reaches this method - * unless the writer is exempt). - * - **Re-seed on collision**, at the write. Adoption cannot see a writer - * outside this process (another instance, a direct driver write, a - * restore). Those surface as a unique-constraint failure on the create, - * and {@link createWithAutonumberResync} answers it by dropping the stale - * counter, re-seeding from the store and re-issuing — bounded. It costs - * nothing until a collision actually happens. - * - * Both are required and neither subsumes the other: adoption covers drift the - * engine can observe but the store cannot report, collision-resync covers - * drift the store reports but the engine could not observe. - * - * @returns the autonumbers this call ISSUED, in field order — the input - * {@link createWithAutonumberResync} needs to decide whether a unique - * violation is one of its own numbers. An adopted (caller-supplied) value is - * deliberately NOT listed: it is not the engine's to re-issue. - */ private applyAutonumbers( object: string, record: Record, execCtx?: ExecutionContext, driverOwnsAutonumber?: boolean, ): Promise { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). const readAutonumberCounters = () => this.autonumberCounters; - const host: ThisParameterType = { + const host: ThisParameterType = { adoptExplicitAutonumber: (...a) => this.adoptExplicitAutonumber(...a), get autonumberCounters() { return readAutonumberCounters(); }, getSchema: (...a) => this.getSchema(...a), seedAutonumber: (...a) => this.seedAutonumber(...a), }; - return applyAutonumbers.call(host, object, record, execCtx, driverOwnsAutonumber); + return DefaultsAndAutonumbersFacet.applyAutonumbers.call(host, object, record, execCtx, driverOwnsAutonumber); } - /** - * Lift the in-memory counter to a record number an EXEMPT writer supplied - * (#6806) — the free half of the resync, and the one that closes the shape - * #5495's PROBE1 measured on a warm database. - * - * The value is parsed with {@link readStoredAutonumberCounter}, i.e. by - * exactly the anchoring rules #6468 gave the seeding scan, against the - * prefix/suffix this record's own format renders. So adopting is the same reading a cold re-seed - * would perform over the same row — which is the invariant to hold on to: a - * warm counter must answer what a restart would answer. - * - * Four deliberate refusals, each one a way this could do harm: - * - * - **Never throws.** An exempt write is not the engine's to reject, and it - * was not rejected before this method existed. A format whose `{field}` - * interpolation is empty, or a value that parses to nothing, simply - * teaches the counter nothing. - * - **Only LIFTS a counter that is already seeded.** With no seed in hand, - * writing this value in would SKIP the seeding scan and answer from one - * row — below the real max whenever the store holds a higher number, i.e. - * the duplicate-number defect itself. Doing nothing is correct: the row is - * persisted, so the first generating insert's own scan reads it. - * - **Never lowers.** A counter that has already issued numbers must not go - * back over them; the max is a floor that only rises. - * - **Only within this record's scope.** The reading returns `undefined` - * for a value that does not carry the rendered prefix, so a - * historical import into last month's date scope cannot lift THIS - * month's counter. Its own scope's counter is left untouched, which is - * harmless: a scope is derived from the write instant, so a past scope's - * counter is not one a subsequent generating insert can reach. - * - * Adoption runs BEFORE the driver write, so an exempt insert that then fails - * leaves the counter lifted over a number nobody took — a gap. Gaps are - * already the documented cost of this path (a failed attempt consumes its - * value; the counter is "resilient to deletions"), and the direction is the - * safe one: a gap is a cosmetic surprise, a duplicate is a corrupted business - * identifier. - */ private adoptExplicitAutonumber( object: string, field: string, @@ -3534,138 +2972,17 @@ export class ObjectQL implements IObjectQLEngine { now: Date, timezone?: string, ): void { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). const readAutonumberCounters = () => this.autonumberCounters; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get autonumberCounters() { return readAutonumberCounters(); }, get logger() { return readLogger(); }, }; - return adoptExplicitAutonumber.call(host, object, field, tokens, record, value, now, timezone); + return DefaultsAndAutonumbersFacet.adoptExplicitAutonumber.call(host, object, field, tokens, record, value, now, timezone); } - /** - * Create ONE record, re-seeding and re-issuing when the driver rejects an - * autonumber the engine itself issued as a duplicate (#6806). - * - * # What this is for - * - * A counter that sits below the store's real max — because a writer outside - * this process took numbers the engine could not observe — collides on every - * insert until it has walked past that max one number at a time. Before this, - * each of those inserts failed with the driver's raw error AND advanced the - * counter, so a warm-database storm burned a number per failed create and - * never converged on its own (#5495's PROBE3). Dropping the counter on the - * collision is what converges it: the re-seed reads the true max back, and - * the next attempt is issued from it. - * - * # Which failures qualify - * - * - The engine must have ISSUED an autonumber on this row. A row whose - * numbers all came from an exempt writer has nothing here to re-issue. - * - The error must be a unique violation, per `isUniqueViolationError` - * (#6250) — never a word-list of this method's own. - * - When the dialect names the conflicting COLUMN (`uniqueViolationColumn`, - * #6544), it must be one of the fields the engine issued. A conflict on - * some other unique field is the caller's business error and is NOT - * re-issued, exactly as #5495's disposition ruled («非本字段的冲突原样上抛»); - * since #14095 it leaves the door as the `DUPLICATE_RECORD` envelope - * (`DuplicateRecordError`) carrying that same driver error as `cause` — - * the disposition is about not RE-ISSUING, and the envelope changes - * nothing about which failures qualify. - * When the dialect names no determinable column the attribution falls back - * to "the engine issued a number on this row, and the row was refused as a - * duplicate" — deliberately, because `uniqueViolationColumn` answers - * `undefined` for every index-named dialect, which includes MongoDB's - * `E11000 ... index: doc_no_1`, i.e. the ONE fallback driver that can - * raise this at all. Requiring a named column would make the resync - * unreachable on exactly the driver that needs it. The cost of the - * fallback is a wasted re-issue when an unrelated unique field is what - * actually conflicted: the second attempt fails the same way, and the - * original error is what the caller finally sees. - * - * # ⚠ The guarantee is STORAGE-DEPENDENT — say so, do not imply otherwise - * - * This whole branch is triggered by the storage layer REJECTING the duplicate, - * so it reaches only drivers that (1) take this fallback path at all and - * (2) enforce uniqueness. Measured across all five in-repo drivers: - * - * | driver | `supports.autonumber` | fallback path? | uniqueness on the column | a collision appears as | - * |:---|:---|:---|:---|:---| - * | driver-memory | `supports = {}` | **yes** | field-level `unique`, since commit 56c093c4d (ADR-0120 D1/D3 scoping) | `UNIQUE_VIOLATION` / 409 → re-seed + re-issue, here | - * | driver-mongodb | absent (`{ batchSchemaSync: true }`) | **yes** | single-field unique index when the field declares `unique` | `E11000 duplicate key` → re-seed + re-issue, here | - * | driver-sql | `autonumber: true` | no | — | — (but see #25073 below) | - * | driver-sqlite-wasm | inherited (`extends SqlDriver`, no `supports` override) | no | — | — (inherited) | - * | driver-turso | inherited (`...super.supports`) | no | — | — (inherited) | - * - * # Numbers the DRIVER issued ahead of the write (#25073) - * - * For an object whose stored `expression` reads its record number, a driver - * that owns its counter issues the number through - * `IDataDriver.issueAutonumbers` before this method runs, and the row - * arrives here carrying it (`driverIssued`). The driver's own retry inside - * `create()` cannot answer a collision on it any more — it reserved - * nothing — and must not: a number re-issued there would leave the stored - * expression computed from the old one. So the retry is this loop's, the - * same loop the fallback drivers use. The member decides whether the error - * is a collision on its numbers (never a duplicate the caller typed, never - * inside a caller's transaction) and re-seeds the stale counter; this loop - * then issues again, recomputes the readers and writes again, under the - * same {@link AUTONUMBER_COLLISION_ATTEMPTS} bound. - * - * So the retry protects the TWO fallback backends — and it protected only one - * of them until commit 56c093c4d, because `driver-memory` enforced no uniqueness at all - * and had nothing to reject with. Which driver ISSUES the number is a - * separate question and is unmoved by that: it is the reading the repo - * already ruled and gates, in - * `scripts/driver-memory-census.ledger.json`'s disposition for - * `packages/runtime/src/autonumber-seed-cross-side-parity.integration.test.ts` - * (axis `ruled-permanent`, «#6664 A, maintainer 2026-08-08 — inherits #5704 - * Q2 = B»), which states it as: "InMemoryDriver declares `supports = {}`, so - * the ENGINE's autonumber seeding owns the counter. No SQL backend can stand - * in — SqlDriver advertises the capability and its own sequence bootstrap - * answers instead". This comment cites that ruling rather than restating it: - * a second answer to "who owns the autonumber counter" is the same - * one-contract-two-numbers defect this lane keeps closing (#6832). - * - * ⚠️ **This paragraph used to record a live defect; commit 56c093c4d closed it, and the - * ⛔ below is why the closure went where it did.** `InMemoryDriver.create` was - * a `table.push()` storing no constraints of any kind (#4065's WEAK-oracle - * docstring), so an out-of-process duplicate raised nothing for this method to - * catch and the number landed twice in the rendered field with no error - * anywhere. Since commit 56c093c4d that driver enforces field-level `unique` - * (`driver-memory`'s `memory-unique-constraint.ts`, with `driver-sql`'s - * ADR-0120 D1/D3 scoping) and refuses the collision in the ADR-0112 envelope, - * which `isUniqueViolationError` reads — so the branch below is reachable - * there and the number converges instead of duplicating. - * - * What covers driver-memory in ADDITION is the other half of this resync — - * {@link adoptExplicitAutonumber} — which needs no constraint at all because it - * never waits for a rejection. Between them: drift the engine can observe is - * fixed on every driver; drift only the store can report is fixed wherever the - * store reports it. - * - * ⛔ The remedy for the silent-duplicate row is uniqueness enforcement in the - * driver, NOT a pre-issue existence probe here: a probe costs a query on every - * insert (the cost this resync was designed to avoid) and is still racy, so it - * would trade a silent duplicate for a rarer silent duplicate at double the - * read cost. That argument is UNCHANGED by commit 56c093c4d and is not a historical - * note — it is the standing reason no probe is added here, and it is what the - * driver-side fix was chosen over. (`packages/drivers/**` was under the #5499 - * investment freeze when this comment was first written, which is why the work - * was deferred rather than misplaced; the freeze lifted on 2026-08-11 and the - * remedy landed where it always belonged, in the driver.) - * - * # And when it does not converge - * - * After {@link AUTONUMBER_COLLISION_ATTEMPTS} the write fails with a named - * engine error (`code: 'ERR_AUTONUMBER_COLLISION'`) carrying the driver's - * error as `cause`. The raw driver error is deliberately NOT the contract - * here: "your record number collided three times after re-seeding" is an - * engine-level condition a caller can act on, and the driver's prose is - * preserved rather than replaced. - */ - private async createWithAutonumberResync( + private createWithAutonumberResync( driver: any, object: string, row: Record, @@ -3676,106 +2993,34 @@ export class ObjectQL implements IObjectQLEngine { driverIssued: string[] = [], readers?: AutonumberReaders, ): Promise { - // [#25073] Who issued this row's numbers decides who judges a collision on - // them: the driver, through the member that issued them, or this engine. - const byDriver = driverIssued.length > 0; - let attempt = 1; - for (;;) { - try { - return await driver.create(object, row, driverOptions); - } catch (error) { - // [#14095] Not OUR collision to re-issue — so this is where a driver's - // refusal leaves the single-row door, and where it stops being the - // driver's error. `envelopeUniqueViolation` returns everything that is - // not a unique violation untouched (a deadlock, a NOT NULL, an - // unreachable store), so only the recognised conflict changes shape. - // [#24768] …and a NOT NULL refusal on a declared field becomes the - // validator's own `ValidationError` (`envelopeDriverWriteRefusal`). - // [#25073] On a driver-issued number the member answers instead, and - // in answering re-seeds the counter it proved stale. - const collided: string[] = byDriver - ? ((await driver.issueAutonumbers(object, [row], driverOptions, error)) as string[][] | undefined)?.[0] ?? [] - : this.isIssuedAutonumberCollision(error, issued) ? issued.map((one) => one.field) : []; - if (collided.length === 0) { - throw envelopeDriverWriteRefusal(error, object, this.getSchema(object), 'insert', this.validationMessageContext(object, execCtx)); - } - // Whatever happens next, the stale counter must not survive this call: - // leaving it in place is what turned one collision into a storm. - for (const one of issued) this.autonumberCounters.delete(one.counterKey); - if (attempt >= AUTONUMBER_COLLISION_ATTEMPTS) { - const fields = collided.join(', '); - throw Object.assign( - new Error( - `Autonumber collision on '${object}' field(s) [${fields}]: the record number was ` + - `re-seeded from the store and re-issued ${AUTONUMBER_COLLISION_ATTEMPTS} times and the ` + - `driver rejected each one as a duplicate. No record was written.`, - ), - { code: 'ERR_AUTONUMBER_COLLISION', cause: error }, - ); - } - attempt += 1; - this.logger.warn('Autonumber collided — re-seeding the counter and re-issuing', { - object, fields: collided, attempt, - }); - let renumbered: boolean; - if (byDriver) { - // [#25073] The member emptied the slots it proved stale; issue them again. - renumbered = (((await driver.issueAutonumbers(object, [row], driverOptions)) as string[][] | undefined)?.[0] ?? []).length > 0; - } else { - // Clear the slots so `applyAutonumbers` treats them as empty again — - // leaving the burned value in place would read as an exempt writer's - // and be adopted rather than re-issued. - for (const one of issued) delete row[one.field]; - issued = await this.applyAutonumbers(object, row, execCtx, driverOwnsAutonumber); - renumbered = issued.length > 0; - } - // [#25073] A new number is a new input to every stored field that reads it. - await recomputeAutonumberReaders(readers, [row]); - // Nothing left to re-issue (the field vanished from the schema - // mid-flight) — the next failure is the caller's to see, enveloped on - // the same terms as the exit above (#14095): this call is OUTSIDE the - // loop's own `try`, so its rejection is a second driver-error exit and - // not a path the branch above covers. - if (!renumbered) { - try { - return await driver.create(object, row, driverOptions); - } catch (retryError) { - throw envelopeDriverWriteRefusal(retryError, object, this.getSchema(object), 'insert', this.validationMessageContext(object, execCtx)); - } - } - } - } + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). + const readAutonumberCounters = () => this.autonumberCounters; + const readLogger = () => this.logger; + const host: ThisParameterType = { + applyAutonumbers: (...a) => this.applyAutonumbers(...a), + get autonumberCounters() { return readAutonumberCounters(); }, + getSchema: (...a) => this.getSchema(...a), + isIssuedAutonumberCollision: (...a) => this.isIssuedAutonumberCollision(...a), + get logger() { return readLogger(); }, + validationMessageContext: (...a) => this.validationMessageContext(...a), + }; + return DefaultsAndAutonumbersFacet.createWithAutonumberResync.call(host, driver, object, row, driverOptions, issued, execCtx, driverOwnsAutonumber, driverIssued, readers); } - /** - * Whether `error` is a unique violation attributable to one of the - * autonumbers this insert issued (#6806). See - * {@link createWithAutonumberResync} for why an unnamed column counts as - * attributable and a differently-named one does not. - */ private isIssuedAutonumberCollision(error: unknown, issued: IssuedAutonumber[]): boolean { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`. - return isIssuedAutonumberCollision(error, issued); + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it. + return DefaultsAndAutonumbersFacet.isIssuedAutonumberCollision(error, issued); } - /** - * [#25073] A driver that owns the record-number counter - * (`supports.autonumber`) but cannot issue a number ahead of the write - * (no `IDataDriver.issueAutonumbers`) still fills it inside `create()`, so a - * stored field whose `expression` reads it is stored as computed WITHOUT - * it, and every insert looks successful. That is a stored value that is - * not what its declaration says (AGENTS.md, degradation log levels: `error`), - * said once per object per engine instance, with the remedy. - */ private warnAutonumberReadersUnserved(object: string, plan: readonly FormulaPlanEntry[], driver: unknown): void { - // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts` (see `DefaultsAndAutonumbersHost`). + // [#29986 codemod] The body moved to `engine-defaults-and-autonumbers.ts`, its docblock with it (see `DefaultsAndAutonumbersHost`). const readAutonumberReadersUnservedReported = () => this.autonumberReadersUnservedReported; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get autonumberReadersUnservedReported() { return readAutonumberReadersUnservedReported(); }, get logger() { return readLogger(); }, }; - return warnAutonumberReadersUnserved.call(host, object, plan, driver); + return DefaultsAndAutonumbersFacet.warnAutonumberReadersUnserved.call(host, object, plan, driver); } /** @@ -4143,12 +3388,6 @@ export class ObjectQL implements IObjectQLEngine { } } - /** - * [#28697] The part of {@link registerApp} a registry verb can refuse - * part-way: the package record, its objects, extensions, apps, collections, - * seed data and nested plugins, in that order. Every body-only refusal was - * judged before this runs; what is written after it is in `registerApp`. - */ private registerPackageBody( manifest: any, id: string, @@ -4157,17 +3396,17 @@ export class ObjectQL implements IObjectQLEngine { source: Exclude, scope: ArtifactInstallScope | undefined, ): void { - // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + // [#29986 codemod] The body moved to `engine-registration.ts`, its docblock with it (see `RegistrationHost`). const read_registry = () => this._registry; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get logger() { return readLogger(); }, registerMetadataCollections: (...a) => this.registerMetadataCollections(...a), registerPlugin: (...a) => this.registerPlugin(...a), resolveNavObjectNames: (...a) => this.resolveNavObjectNames(...a), }; - return registerPackageBody.call(host, manifest, id, namespace, version, source, scope); + return RegistrationFacet.registerPackageBody.call(host, manifest, id, namespace, version, source, scope); } /** @@ -4204,57 +3443,15 @@ export class ObjectQL implements IObjectQLEngine { return registerPlugin.call(host, plugin, parentId, parentNamespace, parentVersion); } - /** - * Register one source's stack collections into the registry — the SINGLE - * body both registration seams run. - * - * ## Why this is one method and not two loops (#7163) - * - * It used to be two, and #7049 only got halfway: that card hoisted the - * ENUMERATION (`METADATA_ARRAY_KEYS`) out of the two seams after they had - * drifted four collections apart, and measured the loop bodies against each - * other on the way past — recording that they still differed in a per-key - * `debug` line, the manifest seam's aggregated-view expansion, and its - * warn-on-nameless-item. Sharing the list made "which collections does a - * seam see?" unanswerable-differently; it left "what does a seam DO with a - * collection both see?" still answered in two places. - * - * `views` is where that cost was measurable. "Object has-many View" - * (ADR-0017 §2, §3.2) makes the loader **dual-read**: an aggregated - * `defineView` container is registered under the bare `` key for - * back-compatible reads AND expanded into independent `ViewItem`s under - * `.`. Only the expanded items carry `viewKind`, and - * `getViewsByObject()` (`metadata-manager.ts`) filters on exactly that — so - * a container that is registered but never expanded is invisible to it, and - * to `GET /meta/view?object=` and the view switcher above it. The manifest - * seam expanded; the nested-plugin seam did not, so one container registered - * `['account', 'account.all_accounts', 'account.form']` through the manifest - * and `['account']` through a nested plugin. No refusal, no diagnostic: a - * package shipping its views through `manifest.plugins[]` simply had no - * views, as far as every reader of the expanded layer was concerned. - * - * Both differences had the same structure — a loop body copied, then - * improved on one side only — so the body is shared rather than reconciled, - * for the reason #7049 gave for the list: a divergence that cannot be - * written down cannot be re-introduced by the next hand patch. What legally - * varies between the seams is passed in: which object is read, which package - * id is stamped (both resolve to the SAME owning package — a nested plugin - * contributes under its parent's ownership), and the label the `debug` line - * names the source with. - * - * @param source The manifest or nested-plugin config to read collections from. - * @param ownerId The owning package id — stamped as ADR-0010 provenance. - * @param sourceLabel Human-readable source name for the `debug` line. - */ private registerMetadataCollections(source: any, ownerId: string, sourceLabel: string, ownerVersion?: string) { - // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + // [#29986 codemod] The body moved to `engine-registration.ts`, its docblock with it (see `RegistrationHost`). const read_registry = () => this._registry; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get logger() { return readLogger(); }, }; - return registerMetadataCollections.call(host, source, ownerId, sourceLabel, ownerVersion); + return RegistrationFacet.registerMetadataCollections.call(host, source, ownerId, sourceLabel, ownerVersion); } /** @@ -4340,49 +3537,14 @@ export class ObjectQL implements IObjectQLEngine { } } - /** - * Hand a freshly-registered driver the ADR-0104 media arm — the kernel→driver - * supply seam (#15989, sequencing step 2 of ADR-0104's 2026-09-05 addendum). - * - * ## Why here, and why a closure rather than a value - * - * `registerDriver` is the one funnel every driver this engine will ever - * route to passes through, and it is early: the resolver is installed before - * `init()` connects and long before schema sync calls `initObjects`, which - * is where a SQL driver asks the closure exactly once and freezes the - * answer. A VALUE could not be supplied here at all — the fact lives in a - * `sys_migration` row this very driver has not connected to yet — so what is - * handed over is the question, not the answer. - * - * ## Duck-typed, and silent when the driver has no such seam - * - * A driver with no media arm (memory, mongodb, a third-party one) exposes no - * `setFileColumnsMovedResolver` and is left alone. That is not a degradation - * to report: a driver that never asks the question keeps whatever encoding - * it always had. - * - * ## ⛔ A host declaration WINS — this only ever fills an empty slot - * - * If the host passed `fileColumnsMoved` to the driver's own config, the - * driver refuses this resolver and says so on its own terms. The asymmetry - * is deliberate and the safety argument runs one way only: a host that - * declared `true` while the ledger says otherwise is a host that knows - * something about its own storage that the ledger does not yet record, and - * the engine silently overruling it to `false` would have the driver write - * JSON into columns that have already been retyped. The reverse — the engine - * overruling a declared `false` to `true` — writes bare ids into a JSON - * column, which is the failure direction the whole mechanism exists to - * prevent. Neither is acceptable, so the more specific authority wins and - * the engine never contradicts an explicit composition. - */ private supplyFileColumnsMovedResolver(driver: IDataDriver): void { - // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + // [#29986 codemod] The body moved to `engine-registration.ts`, its docblock with it (see `RegistrationHost`). const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { haveFileColumnsMoved: (...a) => this.haveFileColumnsMoved(...a), get logger() { return readLogger(); }, }; - return supplyFileColumnsMovedResolver.call(host, driver); + return RegistrationFacet.supplyFileColumnsMovedResolver.call(host, driver); } /** @@ -4435,20 +3597,20 @@ export class ObjectQL implements IObjectQLEngine { * following a partial one) can tell a removal from a no-op. */ unregisterDriver(name: string): boolean { - const removed = this.drivers.delete(name); - // Cleared regardless of `removed`: a datasource that never connected has a - // def and no driver, so keying this off the driver entry would strand - // exactly the rows a failed datasource leaves behind. - this.datasourceDefs.delete(name); - if (this.defaultDriver === name) { - this.defaultDriver = null; - this.logger.info( - 'Evicted the DEFAULT driver — this engine has no default until one is registered', - { driverName: name }, - ); - } - if (removed) this.logger.info('Unregistered driver', { driverName: name }); - return removed; + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readDatasourceDefs = () => this.datasourceDefs; + const readDefaultDriver = () => this.defaultDriver; + const writeDefaultDriver = (v: ObjectQL['defaultDriver']) => { this.defaultDriver = v; }; + const readDrivers = () => this.drivers; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get datasourceDefs() { return readDatasourceDefs(); }, + get defaultDriver() { return readDefaultDriver(); }, + set defaultDriver(v) { writeDefaultDriver(v); }, + get drivers() { return readDrivers(); }, + get logger() { return readLogger(); }, + }; + return RegistrationFacet.unregisterDriver.call(host, name); } /** @@ -4465,37 +3627,26 @@ export class ObjectQL implements IObjectQLEngine { registerDatasourceDef(def: DatasourceDef): void { // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). const readDatasourceDefs = () => this.datasourceDefs; - const host: ThisParameterType = { + const host: ThisParameterType = { applyExternalDriverFloor: (...a) => this.applyExternalDriverFloor(...a), get datasourceDefs() { return readDatasourceDefs(); }, }; - return registerDatasourceDef.call(host, def); + return RegistrationFacet.registerDatasourceDef.call(host, def); } - /** - * [#27277] Put the registered driver of datasource `name` behind the engine's - * correctness floor when its definition calls for one, or take it back out - * when a later definition no longer does (`external-driver-floor.ts`). - * - * Called from both doors — {@link registerDriver} and - * {@link registerDatasourceDef} — because either can come first. The floored - * driver REPLACES the entry in `drivers`, once, so every reader of the map - * (`getDriver`, `getDriverByName`, the default-driver and transaction-scope - * identity checks) holds one instance. - */ private applyExternalDriverFloor(name: string): void { - // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + // [#29986 codemod] The body moved to `engine-registration.ts`, its docblock with it (see `RegistrationHost`). const read_registry = () => this._registry; const readDatasourceDefs = () => this.datasourceDefs; const readDrivers = () => this.drivers; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get datasourceDefs() { return readDatasourceDefs(); }, get drivers() { return readDrivers(); }, get logger() { return readLogger(); }, }; - return applyExternalDriverFloor.call(host, name); + return RegistrationFacet.applyExternalDriverFloor.call(host, name); } /** @@ -4525,12 +3676,12 @@ export class ObjectQL implements IObjectQLEngine { * name a plugin driver" can be written against the contract's read-back. */ listDatasourceDefs(): DatasourceDef[] { - return Array.from(this.datasourceDefs, ([name, def]) => ({ - name, - ...(def.driver !== undefined ? { driver: def.driver } : {}), - ...(def.schemaMode !== undefined ? { schemaMode: def.schemaMode } : {}), - ...(def.external ? { external: { ...def.external } } : {}), - })); + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readDatasourceDefs = () => this.datasourceDefs; + const host: ThisParameterType = { + get datasourceDefs() { return readDatasourceDefs(); }, + }; + return RegistrationFacet.listDatasourceDefs.call(host); } /** @@ -4545,16 +3696,22 @@ export class ObjectQL implements IObjectQLEngine { * cause — see {@link DatasourceUnavailableError}. */ markDatasourceUnavailable(info: { name: string; kind: DatasourceUnavailableKind; publicDetail?: string }): void { - if (!info?.name) return; - this.unavailableDatasources.set(info.name, { - kind: info.kind, - ...(info.publicDetail ? { publicDetail: info.publicDetail } : {}), - }); + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readUnavailableDatasources = () => this.unavailableDatasources; + const host: ThisParameterType = { + get unavailableDatasources() { return readUnavailableDatasources(); }, + }; + return RegistrationFacet.markDatasourceUnavailable.call(host, info); } /** Drop a {@link markDatasourceUnavailable} record (successful (re)connect / pool removal). */ clearDatasourceUnavailable(name: string): void { - this.unavailableDatasources.delete(name); + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readUnavailableDatasources = () => this.unavailableDatasources; + const host: ThisParameterType = { + get unavailableDatasources() { return readUnavailableDatasources(); }, + }; + return RegistrationFacet.clearDatasourceUnavailable.call(host, name); } /** @@ -4564,7 +3721,12 @@ export class ObjectQL implements IObjectQLEngine { * `asDefault` idempotency guard needs this rather than a name lookup. */ getDefaultDriverName(): string | undefined { - return this.defaultDriver ?? undefined; + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readDefaultDriver = () => this.defaultDriver; + const host: ThisParameterType = { + get defaultDriver() { return readDefaultDriver(); }, + }; + return RegistrationFacet.getDefaultDriverName.call(host); } /** @@ -4575,27 +3737,23 @@ export class ObjectQL implements IObjectQLEngine { * datasource that never connected was never registered (framework#3827). */ listUnavailableDatasources(): Array<{ name: string } & DatasourceUnavailableInfo> { - return Array.from(this.unavailableDatasources, ([name, info]) => ({ name, ...info })); + // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + const readUnavailableDatasources = () => this.unavailableDatasources; + const host: ThisParameterType = { + get unavailableDatasources() { return readUnavailableDatasources(); }, + }; + return RegistrationFacet.listUnavailableDatasources.call(host); } - /** - * Write gate — Gate 3 of ADR-0015 §5.3. - * - * Blocks insert/update/delete against a federated datasource - * (`schemaMode !== 'managed'`) unless BOTH the datasource opts in - * (`external.allowWrites`) AND the object opts in (`external.writable`). - * Managed datasources (the common case, including the absence of any - * definition) are unaffected. - */ private assertWriteAllowed(objectName: string, operation: 'insert' | 'update' | 'delete'): void { - // [#29986 codemod] The body moved to `engine-registration.ts` (see `RegistrationHost`). + // [#29986 codemod] The body moved to `engine-registration.ts`, its docblock with it (see `RegistrationHost`). const read_registry = () => this._registry; const readDatasourceDefs = () => this.datasourceDefs; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get datasourceDefs() { return readDatasourceDefs(); }, }; - return assertWriteAllowed.call(host, objectName, operation); + return RegistrationFacet.assertWriteAllowed.call(host, objectName, operation); } /** @@ -4605,8 +3763,16 @@ export class ObjectQL implements IObjectQLEngine { * @param service - An IRealtimeService instance for event publishing */ setRealtimeService(service: IRealtimeService): void { - this.realtimeService = service; - this.logger.info('RealtimeService configured for data events'); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). + const readLogger = () => this.logger; + const readRealtimeService = () => this.realtimeService; + const writeRealtimeService = (v: ObjectQL['realtimeService']) => { this.realtimeService = v; }; + const host: ThisParameterType = { + get logger() { return readLogger(); }, + get realtimeService() { return readRealtimeService(); }, + set realtimeService(v) { writeRealtimeService(v); }, + }; + return ReferencesAndMasterDetailFacet.setRealtimeService.call(host, service); } /** @@ -4742,8 +3908,16 @@ export class ObjectQL implements IObjectQLEngine { getFallbackLocale?: () => string | undefined; getDefaultLocale?: () => string | undefined; }): void { - this.i18nService = service; - this.logger.info('I18nService configured for validation messages'); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). + const readI18nService = () => this.i18nService; + const writeI18nService = (v: ObjectQL['i18nService']) => { this.i18nService = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get i18nService() { return readI18nService(); }, + set i18nService(v) { writeI18nService(v); }, + get logger() { return readLogger(); }, + }; + return ReferencesAndMasterDetailFacet.setI18nService.call(host, service); } /** @@ -4758,7 +3932,14 @@ export class ObjectQL implements IObjectQLEngine { * never assumes one — and an unset timezone reads UTC. */ setHomeLocalizationResolver(resolver: HomeLocalizationResolver | undefined): void { - this.homeLocalizationResolver = resolver; + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). + const readHomeLocalizationResolver = () => this.homeLocalizationResolver; + const writeHomeLocalizationResolver = (v: ObjectQL['homeLocalizationResolver']) => { this.homeLocalizationResolver = v; }; + const host: ThisParameterType = { + get homeLocalizationResolver() { return readHomeLocalizationResolver(); }, + set homeLocalizationResolver(v) { writeHomeLocalizationResolver(v); }, + }; + return ReferencesAndMasterDetailFacet.setHomeLocalizationResolver.call(host, resolver); } private homeLocalizationResolver?: HomeLocalizationResolver; @@ -4818,31 +3999,16 @@ export class ObjectQL implements IObjectQLEngine { }); } - /** - * [ADR-0144 D5] The organization a written row belongs to, for its home - * projection: the row's own `organization_id`, else the writer's active - * organization, else the organization a system insert resolved — the - * assignment seam's reading, so both seams convert and route one row under - * one organization. - */ private static homeAmountOrganizationOf( row: Record | null | undefined, context: ExecutionContext | undefined, derived?: string | undefined, ): string | undefined { - const own = row?.organization_id; - if (own !== undefined && own !== null && own !== '') return String(own); - if (context?.tenantId) return String(context.tenantId); - return derived ?? undefined; + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts`, its docblock with it. + return ReferencesAndMasterDetailFacet.homeAmountOrganizationOf(row, context, derived); } - /** - * [ADR-0144 D5] The `validate()` preview's run of the home projection: the - * write's own computation over the preview's image (the insert row, or the - * stored row with the patch over it), answering nothing and writing - * nothing — it throws exactly the refusal the write would. - */ - private async previewHomeAmounts( + private previewHomeAmounts( object: string, schema: unknown, mode: 'insert' | 'update', @@ -4851,25 +4017,14 @@ export class ObjectQL implements IObjectQLEngine { context: ExecutionContext | undefined, recordId: unknown, ): Promise { - const plan = planHomeAmounts(schema); - const entries = mode === 'insert' ? plan : homeAmountsTouched(plan, row); - if (entries.length === 0) return; - await projectHomeAmounts(entries, image, this.homeAmountServing(context), { - object, - operation: mode, - organizationId: ObjectQL.homeAmountOrganizationOf(image, context), - recordId, - }); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts`, its docblock with it (see `ReferencesAndMasterDetailHost`). + const host: ThisParameterType = { + homeAmountServing: (...a) => this.homeAmountServing(...a), + }; + return ReferencesAndMasterDetailFacet.previewHomeAmounts.call(host, object, schema, mode, row, image, context, recordId); } - /** - * [ADR-0144 D5] Project a by-id update's home amounts onto its FINAL patch: - * only the declarations the patch touches ({@link homeAmountsTouched}), - * computed over the stored row with the patch laid over it. Shared by the - * write and its `validate()` preview, so a dry run refuses what the write - * refuses. - */ - private async projectHomeAmountsOntoPatch( + private projectHomeAmountsOntoPatch( object: string, plan: readonly HomeAmountPlanEntry[], prior: Record | null | undefined, @@ -4878,15 +4033,11 @@ export class ObjectQL implements IObjectQLEngine { recordId: unknown, serving: HomeAmountServing = this.homeAmountServing(context), ): Promise { - const touched = homeAmountsTouched(plan, patch); - if (touched.length === 0) return; - const image = { ...(prior ?? {}), ...patch }; - Object.assign(patch, await projectHomeAmounts(touched, image, serving, { - object, - operation: 'update', - organizationId: ObjectQL.homeAmountOrganizationOf(image, context), - recordId, - })); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts`, its docblock with it (see `ReferencesAndMasterDetailHost`). + const host: ThisParameterType = { + homeAmountServing: (...a) => this.homeAmountServing(...a), + }; + return ReferencesAndMasterDetailFacet.projectHomeAmountsOntoPatch.call(host, object, plan, prior, patch, context, recordId, serving); } /** @@ -5129,34 +4280,20 @@ export class ObjectQL implements IObjectQLEngine { } } - /** - * [#22519] The write's reference check for a `master_detail` field: does `id` - * name a header THIS CALLER can read? The write's two header reads, as - * {@link resolveMasterDetailParent} reads them: the caller's read door - * ({@link masterDetailHeadersCallerReads}), then the elevated, tenant-kept - * {@link referenceExists} for a header it returned. A header the read door - * does not return is `false` — `reference_not_found`, exactly as a header - * that exists nowhere. - * - * `null` (fail-open) only where {@link referenceExists} answers it, and for - * an unregistered master, which has no read door to ask either. A SYSTEM - * caller reaches it only as a flow run's elevated write, or a write that - * write sets off (#28881): its read door returns every header - * ({@link masterDetailHeadersCallerReads}), so the header's existence in the - * caller's tenant is what is asked. - * - * ⛔ `master_detail` only — the plain `lookup` probe is not this rule's. - */ - private async masterDetailHeaderExists( + private masterDetailHeaderExists( target: string, id: unknown, context: ExecutionContext | undefined, ): Promise { - const master = this.resolveObjectName(target); - if (!this._registry.getObject(master)) return null; - const readable = await this.masterDetailHeadersCallerReads(master, [id], context); - if (!readable.has(String(id))) return false; - return this.referenceExists(target, id, context); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts`, its docblock with it (see `ReferencesAndMasterDetailHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + masterDetailHeadersCallerReads: (...a) => this.masterDetailHeadersCallerReads(...a), + referenceExists: (...a) => this.referenceExists(...a), + resolveObjectName: (...a) => this.resolveObjectName(...a), + }; + return ReferencesAndMasterDetailFacet.masterDetailHeaderExists.call(host, target, id, context); } /** @@ -5638,71 +4775,21 @@ export class ObjectQL implements IObjectQLEngine { return resolveOneHopRelated.call(host, schema, wanted, rows, context, authority); } - /** - * [#6457] Make a resolved master-detail header TOTAL over the MASTER - * object's declared fields, so a `parent.` predicate is evaluable - * whatever subset of columns the driver echoed back. - * - * ## The hole this closes - * - * #4953 made the `record` / `previous` roots total at every server seam - * ({@link ./declared-fields.js#materializeDeclaredFields}); the 2026-08-06 - * ruling deliberately left `parent` out, because `parent` is a row of ANOTHER - * object and its ABSENCE is #4889's fail-closed signal. What that left behind - * is the same trap one root over — visible as a THREE-row table, of which - * only the middle row moves here: - * - * | header state | `readonlyWhen: parent.status == null` | before | now | - * |---|---|---|---| - * | carries `status` | evaluates | locks per verdict | unchanged | - * | resolved, no `status` key | `No such key: status` — `parent` IS bound, so `unknownVariableOf` does not match ⇒ ordinary fail-OPEN | **declared lock let through** | evaluates (`status` reads `null`) | - * | unresolvable (`null`) | `Unknown variable: parent` | LOCKED (#4889) | LOCKED — unchanged | - * - * The middle row is the bug: whether a declared lock enforced depended on - * which columns a driver happened to return, which is not something an author - * can see or control. `requiredWhen` (#4977) shares the binding and had the - * mirror of it — fail-open there means the requirement is simply not - * enforced. One materialisation serves both consumers. - * - * ## Why HERE, and not in the strip / the evaluator - * - * `stripReadonlyWhenFields*` and `evaluateValidationRules` are pure functions - * over what they are handed; they hold the DETAIL object's field table and - * have no way to reach the MASTER's — threading a second field table through - * their signatures would put the master's schema in four call sites' hands to - * serve one binding. The engine already holds both the registry and the - * just-read header at these two seams, so the header arrives at those - * functions already total and their signatures do not move. - * - * ## The fail-CLOSED line is preserved EXACTLY (#4889) - * - * This function is only ever reached with a row IN HAND — the persisted-state - * precondition `declared-fields.ts` states. An UNRESOLVABLE header still - * returns `null` from the resolvers above, still leaves `parent` unbound, - * still faults as `Unknown variable: parent`, and is still read as LOCKED. The - * two cases stay distinguishable by construction: absence is decided before - * this function is called, materialisation only ever applies to a row that - * exists. - * - * A master the registry cannot resolve yields no field table, and the header - * passes through unchanged — sparse, i.e. exactly the pre-#6457 behaviour. - * That is the honest answer: without the declared shape we cannot know which - * absent keys are fields and which would be fabrication. - * - * COPIED before materialising, like every other caller: the row is what - * `findOne`/`find` returned and may be observed elsewhere; it must not gain - * materialised nulls behind its reader's back. - */ private materializeParentHeader(master: string, row: Record): Record { - return materializeDeclaredFields({ ...row }, this.masterDeclaredFields(master)); + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts`, its docblock with it (see `ReferencesAndMasterDetailHost`). + const host: ThisParameterType = { + masterDeclaredFields: (...a) => this.masterDeclaredFields(...a), + }; + return ReferencesAndMasterDetailFacet.materializeParentHeader.call(host, master, row); } - /** The MASTER object's declared-field table, or `undefined` when the registry - * does not know it (see {@link materializeParentHeader}). */ private masterDeclaredFields(master: string): Record | undefined { - const schema = this._registry.getObject(master) as { fields?: Record } | undefined; - const fields = schema?.fields; - return fields && typeof fields === 'object' ? fields : undefined; + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts`, its docblock with it (see `ReferencesAndMasterDetailHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + }; + return ReferencesAndMasterDetailFacet.masterDeclaredFields.call(host, master); } /** @@ -5808,12 +4895,12 @@ export class ObjectQL implements IObjectQLEngine { ): Promise> { // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). const read_registry = () => this._registry; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, collectCascadeDeleteSet: (...a) => this.collectCascadeDeleteSet(...a), resolveObjectName: (...a) => this.resolveObjectName(...a), }; - return inspectCascadeDelete.call(host, object, id); + return ReferencesAndMasterDetailFacet.inspectCascadeDelete.call(host, object, id); } /** @@ -5845,13 +4932,14 @@ export class ObjectQL implements IObjectQLEngine { * serves no key, rather than `[]`, which says nothing cascades. */ getCascadeDeleteObjects(object: string): string[] | undefined { - if (!this._registry.getObject(object)) return undefined; - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-references-and-master-detail.ts` (see `ReferencesAndMasterDetailHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, cascadeRelationBehavior: (...a) => this.cascadeRelationBehavior(...a), + resolveObjectName: (...a) => this.resolveObjectName(...a), }; - return collectCascadeDeleteObjects.call( - host, this.resolveObjectName(object), this._registry.getAllObjects(), - ); + return ReferencesAndMasterDetailFacet.getCascadeDeleteObjects.call(host, object); } /** @@ -5871,20 +4959,18 @@ export class ObjectQL implements IObjectQLEngine { * capability rather than the state that made it fail (#8022). */ setCryptoProvider(provider: ICryptoProvider): void { - this.cryptoProvider = provider; - this.logger.info('CryptoProvider configured for secret fields'); - // A listener is a re-arm, never part of this call's contract: one that - // throws must not fail the host's composition root, and must not stop the - // listeners behind it from re-arming. - for (const listener of [...this.cryptoProviderListeners]) { - try { - listener(); - } catch (err) { - this.logger.warn('CryptoProvider registration listener failed', { - error: (err as Error)?.message ?? String(err), - }); - } - } + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const readCryptoProvider = () => this.cryptoProvider; + const writeCryptoProvider = (v: ObjectQL['cryptoProvider']) => { this.cryptoProvider = v; }; + const readCryptoProviderListeners = () => this.cryptoProviderListeners; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get cryptoProvider() { return readCryptoProvider(); }, + set cryptoProvider(v) { writeCryptoProvider(v); }, + get cryptoProviderListeners() { return readCryptoProviderListeners(); }, + get logger() { return readLogger(); }, + }; + return SecretsAndRoutingFacet.setCryptoProvider.call(host, provider); } /** @@ -5903,8 +4989,12 @@ export class ObjectQL implements IObjectQLEngine { * value captured at boot would still say "none" once one is registered. */ getKeyedDigest(): ((plain: string) => Promise) | undefined { - const provider = this.cryptoProvider; - return provider ? (plain: string) => provider.keyedDigest(plain) : undefined; + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const readCryptoProvider = () => this.cryptoProvider; + const host: ThisParameterType = { + get cryptoProvider() { return readCryptoProvider(); }, + }; + return SecretsAndRoutingFacet.getKeyedDigest.call(host); } /** @@ -5926,10 +5016,12 @@ export class ObjectQL implements IObjectQLEngine { * @returns an unsubscribe function; call it when the listener's owner stops. */ onCryptoProviderChange(listener: () => void): () => void { - this.cryptoProviderListeners.add(listener); - return () => { - this.cryptoProviderListeners.delete(listener); + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const readCryptoProviderListeners = () => this.cryptoProviderListeners; + const host: ThisParameterType = { + get cryptoProviderListeners() { return readCryptoProviderListeners(); }, }; + return SecretsAndRoutingFacet.onCryptoProviderChange.call(host, listener); } /** @@ -5946,7 +5038,14 @@ export class ObjectQL implements IObjectQLEngine { * fail toward isolation, never toward exposure. */ setTenancyPostureProvider(provider: () => string | undefined): void { - this.tenancyPostureProvider = provider; + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const readTenancyPostureProvider = () => this.tenancyPostureProvider; + const writeTenancyPostureProvider = (v: ObjectQL['tenancyPostureProvider']) => { this.tenancyPostureProvider = v; }; + const host: ThisParameterType = { + get tenancyPostureProvider() { return readTenancyPostureProvider(); }, + set tenancyPostureProvider(v) { writeTenancyPostureProvider(v); }, + }; + return SecretsAndRoutingFacet.setTenancyPostureProvider.call(host, provider); } /** @@ -6455,34 +5554,21 @@ export class ObjectQL implements IObjectQLEngine { * Helper to get object definition */ getSchema(objectName: string): ServiceObject | undefined { - return this._registry.getObject(objectName); + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + }; + return SecretsAndRoutingFacet.getSchema.call(host, objectName); } - /** - * Resolve any object identifier to the physical storage name used by drivers. - * - * Accepts the canonical short name (e.g., 'account') or, for explicit - * cross-package disambiguation, the canonical object name (e.g., 'account'). The result is - * the physical table name derived via `StorageNameMapping.resolveTableName`. - * - * One name space with the data door: the target resolves ONLY through the - * registry. A name the registry does not resolve is refused with the door's - * own `OBJECT_NOT_FOUND` 404 (`objectNotFoundError`), never handed to the - * driver as a raw table name. Before this, every in-process guard keyed by a - * registered object name could be stepped around by naming the target some - * other way, and an in-process caller (a sandboxed body, an action handler, - * a hook) read what the door refused to serve. - * - * Platform code that must address storage without a registry entry has a - * declared path a body cannot reach: the driver itself - * (`datasource(name)`, `getDriverForObject(name)`), held by host code only. - */ private resolveObjectName(name: string): string { - const schema = this._registry.getObject(name); - if (schema) { - return StorageNameMapping.resolveTableName(schema); - } - throw objectNotFoundError(name); + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts`, its docblock with it (see `SecretsAndRoutingHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + }; + return SecretsAndRoutingFacet.resolveObjectName.call(host, name); } /** @@ -6509,15 +5595,15 @@ export class ObjectQL implements IObjectQLEngine { where: EngineQueryOptions['where'], options?: EngineFilterJudgementOptions, ): EngineFilterJudgement { - const object = this._registry.getObject(objectName) ? this.resolveObjectName(objectName) : objectName; - return judgeWhereAdmission( - object, - options?.operation ?? 'find', - where, - this._registry.getObject(object), - options?.context, - this.relatedSchemaOf, - ); + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const read_registry = () => this._registry; + const readRelatedSchemaOf = () => this.relatedSchemaOf; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get relatedSchemaOf() { return readRelatedSchemaOf(); }, + resolveObjectName: (...a) => this.resolveObjectName(...a), + }; + return SecretsAndRoutingFacet.judgeFilter.call(host, objectName, where, options); } /** @@ -6552,24 +5638,13 @@ export class ObjectQL implements IObjectQLEngine { 'event', ]); - /** - * Is `objectName` an append-only system ledger? — the #5351 carve-out's - * discriminator, and deliberately a property of the object's DECLARATION - * (`lifecycle.class`) rather than of the deployment's routing. - * - * Why the declaration and not "was it routed by step 3": an audit ledger - * pinned to its own datasource by an explicit `datasource:` binding, or by a - * `datasourceMapping` rule, is the same append-only compliance ledger with - * the same reason to be carved out — the routing mechanism is an operator's - * choice, the class is the author's statement about what the data IS. Judging - * by the mechanism would make the carve-out depend on which of three - * equivalent configurations a deployment happened to use. - */ private isSystemLedgerObject(objectName: string): boolean { - const lifecycleClass = ( - this._registry.getObject(objectName) as { lifecycle?: { class?: string } } | undefined - )?.lifecycle?.class; - return lifecycleClass !== undefined && ObjectQL.SYSTEM_LEDGER_LIFECYCLE_CLASSES.has(lifecycleClass); + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts`, its docblock with it (see `SecretsAndRoutingHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + }; + return SecretsAndRoutingFacet.isSystemLedgerObject.call(host, objectName); } /** @@ -6601,44 +5676,25 @@ export class ObjectQL implements IObjectQLEngine { return getDriver.call(host, objectName); } - /** - * WHERE does `objectName`'s data live, and WHICH step decided — the single - * implementation of the resolution order documented on {@link getDriver}. - * - * Split out for #5288 so the order exists exactly once. It had two readers - * with two different answers: `getDriver` (all five steps) and every caller - * that only needed the NAME, which read `object.datasource` — step 1 of five. - * A second, shorter copy of a routing order is the failure - * `resolveMappedDatasource` (#4462) was extracted to prevent, and it fails the - * same way: silently, in whichever direction the copy is short. - * - * Steps 1-2 answer even when the named datasource has no registered driver: - * they are BINDINGS, and `getDriver` stops there (it throws rather than - * falling through — #4462). Steps 3-5 answer only when their driver is - * registered, because that registration is what turns each of them on. - * - * `undefined` means nothing routes the object anywhere: no binding, and no - * global default driver to fall back to. - */ private resolveDatasourceBinding(objectName: string): { datasource: string; via: 'explicit' | 'mapping' | 'lifecycle' | 'package' | 'default'; /** Owning package, on `via: 'package'` only — for the debug line. */ packageId?: string; } | undefined { - // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts`, its docblock with it (see `SecretsAndRoutingHost`). const read_registry = () => this._registry; const readDefaultDriver = () => this.defaultDriver; const readDrivers = () => this.drivers; const readManifests = () => this.manifests; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get defaultDriver() { return readDefaultDriver(); }, get drivers() { return readDrivers(); }, get manifests() { return readManifests(); }, resolveDatasourceFromMapping: (...a) => this.resolveDatasourceFromMapping(...a), }; - return resolveDatasourceBinding.call(host, objectName); + return SecretsAndRoutingFacet.resolveDatasourceBinding.call(host, objectName); } /** @@ -6670,8 +5726,11 @@ export class ObjectQL implements IObjectQLEngine { * loudly, and a naming probe must be able to report the name that is broken. */ resolveEffectiveDatasource(objectName: string): string | undefined { - const binding = this.resolveDatasourceBinding(objectName); - return binding && binding.via !== 'default' ? binding.datasource : undefined; + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const host: ThisParameterType = { + resolveDatasourceBinding: (...a) => this.resolveDatasourceBinding(...a), + }; + return SecretsAndRoutingFacet.resolveEffectiveDatasource.call(host, objectName); } /** @@ -6691,7 +5750,13 @@ export class ObjectQL implements IObjectQLEngine { * `object.datasource` binding outranks this and is not consulted here. */ resolveMappedDatasource(objectName: string): string | null { - return this.resolveDatasourceFromMapping(objectName, this._registry.getObject(objectName)); + // [#29986 codemod] The body moved to `engine-secrets-and-routing.ts` (see `SecretsAndRoutingHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + resolveDatasourceFromMapping: (...a) => this.resolveDatasourceFromMapping(...a), + }; + return SecretsAndRoutingFacet.resolveMappedDatasource.call(host, objectName); } /** @@ -6908,78 +5973,34 @@ export class ObjectQL implements IObjectQLEngine { return resolvePrimaryDatasource.call(host); } - /** - * Does this object declare a media field at all? Cached per schema object — - * the registry hands back the same instance per object, so this scans a - * field map once rather than on every write. - */ private objectHasMediaField(objectSchema: any): boolean { - if (!objectSchema?.fields) return false; - const cached = ObjectQL.mediaFieldPresence.get(objectSchema); - if (cached !== undefined) return cached; - const present = Object.values(objectSchema.fields).some( - (def: any) => def && FILE_REFERENCE_TYPES.has(def.type), - ); - ObjectQL.mediaFieldPresence.set(objectSchema, present); - return present; + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it. + return MigrationStateFacet.objectHasMediaField(objectSchema); } private static readonly mediaFieldPresence = new WeakMap(); - /** - * The media value-shape verdict for a write against `objectSchema`. - * - * Dormant by design, mirroring the storage module's own rule: an object with - * no file-class field can hold no malformed media value, so it must not pay - * even one query to learn that. Only objects that actually declare media - * consult the deployment flag — and that read is itself memoized, so the - * whole mechanism costs one query per process, and zero for an app that - * stores no files. - */ - private async mediaValueShapeStrictFor(objectSchema: any): Promise { - if (!this.objectHasMediaField(objectSchema)) return false; - return this.isFileReferencesMigrationVerified(); + private mediaValueShapeStrictFor(objectSchema: any): Promise { + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it (see `MigrationStateHost`). + const host: ThisParameterType = { + isFileReferencesMigrationVerified: (...a) => this.isFileReferencesMigrationVerified(...a), + objectHasMediaField: (...a) => this.objectHasMediaField(...a), + }; + return MigrationStateFacet.mediaValueShapeStrictFor.call(host, objectSchema); } - /** - * Does this object declare a reference or structured-JSON field that a write - * would actually check? Same per-schema cache and same dormancy rule as - * {@link objectHasMediaField}: an object holding none can hold no violation - * of them, so it must not pay a query to learn that. - * - * The membership test is `isScannableValueShapeField` — the validator's own - * — and not raw type membership, because the registry INJECTS covered-type - * fields into every object it registers: `organization_id` and `owner_id` - * (both `system`), plus `created_by` / `updated_by` (both in `SKIP_FIELDS`), - * are all `lookup`s. A caller never writes any of them, so counting them made - * this answer `true` for literally every object — the dormancy rule above - * never fired, and this cache memoized a constant. Same predicate as the - * scanner for the same reason the scanner imports it: three readings of "a - * covered field" drifting by one clause is how a gate ends up governing - * fields nothing enforces. - * - * [#21663] The three that are `readonly` (`organization_id`, `created_by`, - * `updated_by`) DO reach the value-shape check now, on the value a system - * writer, hook or stamp stores. They still do not count here, so an object - * whose only covered fields are those stays warn-first for them: a malformed - * value is admitted, logged and reported, never stored silently. See - * `isScannableValueShapeField` for why widening this test is not the fix. - */ private objectHasCoveredValueField(objectSchema: any): boolean { - if (!objectSchema?.fields) return false; - const cached = ObjectQL.coveredValueFieldPresence.get(objectSchema); - if (cached !== undefined) return cached; - const present = Object.entries(objectSchema.fields).some( - ([name, def]: [string, any]) => isScannableValueShapeField(name, def), - ); - ObjectQL.coveredValueFieldPresence.set(objectSchema, present); - return present; + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it. + return MigrationStateFacet.objectHasCoveredValueField(objectSchema); } private static readonly coveredValueFieldPresence = new WeakMap(); - /** The reference / structured-JSON value-shape verdict for a write. */ - private async valueShapeStrictFor(objectSchema: any): Promise { - if (!this.objectHasCoveredValueField(objectSchema)) return false; - return this.isValueShapesMigrationVerified(); + private valueShapeStrictFor(objectSchema: any): Promise { + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it (see `MigrationStateHost`). + const host: ThisParameterType = { + isValueShapesMigrationVerified: (...a) => this.isValueShapesMigrationVerified(...a), + objectHasCoveredValueField: (...a) => this.objectHasCoveredValueField(...a), + }; + return MigrationStateFacet.valueShapeStrictFor.call(host, objectSchema); } /** @@ -7002,8 +6023,12 @@ export class ObjectQL implements IObjectQLEngine { * Costs nothing on a kernel without the storage objects: the registry lookup * short-circuits before any query. */ - async isFileReferencesMigrationVerified(): Promise { - return (await this.readFileReferencesFlagRow()).verified; + isFileReferencesMigrationVerified(): Promise { + // [#29986 codemod] The body moved to `engine-migration-state.ts` (see `MigrationStateHost`). + const host: ThisParameterType = { + readFileReferencesFlagRow: (...a) => this.readFileReferencesFlagRow(...a), + }; + return MigrationStateFacet.isFileReferencesMigrationVerified.call(host); } /** @@ -7087,8 +6112,12 @@ export class ObjectQL implements IObjectQLEngine { * {@link isFileReferencesMigrationVerified}: they are two questions about * one row. `invalidateDataMigrationFlags()` drops both. */ - async haveFileColumnsMoved(): Promise { - return (await this.readFileReferencesFlagRow()).columnsMoved; + haveFileColumnsMoved(): Promise { + // [#29986 codemod] The body moved to `engine-migration-state.ts` (see `MigrationStateHost`). + const host: ThisParameterType = { + readFileReferencesFlagRow: (...a) => this.readFileReferencesFlagRow(...a), + }; + return MigrationStateFacet.haveFileColumnsMoved.call(host); } /** @@ -7108,36 +6137,16 @@ export class ObjectQL implements IObjectQLEngine { return isValueShapesMigrationVerified.call(host); } - /** - * The memoized seam both public flag readers share — and the place where - * "read once per process" is kept from meaning "answer from a read that - * never happened" (#4769). - * - * The two negatives this read can produce are NOT the same fact: - * - * - **Conclusive.** The ledger was reachable and it does not authorise the - * gate (no row, `verified_at` null, blocking findings). Evidence was - * consulted; memoize it. A later in-process migration run announces - * itself through {@link invalidateDataMigrationFlags}. - * - **Inconclusive.** The ledger could not be ASKED — `sys_migration` is - * not registered yet, or the query threw. Both still answer `false` (an - * unaskable gate stays closed), but freezing that for the life of the - * process turns one unlucky early write into a whole boot's posture. It - * is exactly how one boot ends up lax over data the next boot rejects: - * the platform objects register during kernel init while the very first - * write can land before them, so the answer cached is about a moment - * when nothing could have answered at all. - * - * So an inconclusive read is answered but not kept. The retry costs one - * registry lookup per write until the ledger exists (it short-circuits - * before any query), and stops the moment a real read succeeds. - */ - private async readMigrationFlagMemoized( + private readMigrationFlagMemoized( slot: 'fileReferencesMigrationVerified' | 'valueShapesMigrationVerified', migrationId: string, verifiedLog: string, ): Promise { - return (await this.readMigrationFlagRowMemoized(slot, migrationId, verifiedLog)).verified; + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it (see `MigrationStateHost`). + const host: ThisParameterType = { + readMigrationFlagRowMemoized: (...a) => this.readMigrationFlagRowMemoized(...a), + }; + return MigrationStateFacet.readMigrationFlagMemoized.call(host, slot, migrationId, verifiedLog); } /** @@ -7232,11 +6241,12 @@ export class ObjectQL implements IObjectQLEngine { * answer about `adr-0104-file-references`. */ valueShapeViolationsAdmitted(): Record { - const out: Record = {}; - for (const [migrationId, tally] of this.admittedValueShapeViolations) { - out[migrationId] = { count: tally.count, first: { ...tally.first } }; - } - return out; + // [#29986 codemod] The body moved to `engine-migration-state.ts` (see `MigrationStateHost`). + const readAdmittedValueShapeViolations = () => this.admittedValueShapeViolations; + const host: ThisParameterType = { + get admittedValueShapeViolations() { return readAdmittedValueShapeViolations(); }, + }; + return MigrationStateFacet.valueShapeViolationsAdmitted.call(host); } /** Per-migration tally of admitted violations; see the accessor above. */ @@ -7256,12 +6266,12 @@ export class ObjectQL implements IObjectQLEngine { /** Serializes deviation writes so concurrent violations issue one update. */ private deviationRecording: Promise = Promise.resolve(); - /** - * The sink `validateRecord` reports admitted violations to. Built per write - * (one closure per call, not per row) so the tally can name the object. - */ private admittedViolationSink(object: string): AdmittedValueShapeViolationSink { - return (violation) => this.noteAdmittedValueShapeViolation(object, violation); + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it (see `MigrationStateHost`). + const host: ThisParameterType = { + noteAdmittedValueShapeViolation: (...a) => this.noteAdmittedValueShapeViolation(...a), + }; + return MigrationStateFacet.admittedViolationSink.call(host, object); } private noteAdmittedValueShapeViolation(object: string, violation: AdmittedValueShapeViolation): void { @@ -7275,214 +6285,48 @@ export class ObjectQL implements IObjectQLEngine { return noteAdmittedValueShapeViolation.call(host, object, violation); } - /** - * Record that this deployment has ADMITTED a value its own verified contract - * rejects (#4797) — without touching `verified_at`. - * - * ## The window - * - * Once a certificate holds, the write path is strict and a non-conforming - * value cannot land, so the certificate cannot go stale on its own. The - * operator escape hatches are the exception, and they exist *precisely* to - * relax a deployment that has already verified: with - * `OS_ALLOW_LAX_MEDIA_VALUES` / `OS_ALLOW_LAX_VALUE_SHAPES` on, the value is - * admitted and persisted while `sys_migration` still reads `verified_at` - * non-null, `blocking: 0`. Turn the switch off — or let any other process or - * machine run without it — and strict returns to reject the very data this - * deployment stored. Meanwhile the `adr-0104-file-references` gate, which - * also governs reclamation of released field files, keeps deleting bytes on - * the strength of a certificate that is now false. - * - * ## Why a marker rather than a revocation - * - * {@link retractCreationAttestation} clears `verified_at`, and is right to: - * its target is a certificate issued on the inference "created empty, so - * clean", which a single counterexample fully disproves. A certificate - * earned by `os migrate … --apply` is a walk of the whole store, and one - * admitted write is not evidence of that order — overturning it would make a - * deliberately temporary switch into a one-way door, forcing a full - * re-migration on anyone who used the escape hatch once. - * - * So the authority is withdrawn in proportion to reversibility. The marker - * leaves every recoverable behaviour running (strict enforcement once the - * switch is off, tombstoning, throttling — a rejected write is retried, a - * tombstone is lifted on re-attach) and stops only what cannot be undone: - * `authorisesIrreversibleAction` is false while it stands, so the reap - * guard's byte delete refuses. A real apply-mode run clears it. - * - * ## Cost - * - * At most one ledger read+update per migration id per process, never awaited - * by the write that triggered it, and skipped entirely once the marker is - * standing. A lax write is therefore not a ledger round-trip; the FIRST lax - * write of each class is, and only while the row is still verified. - * - * Deliberately never inserts. No row means nothing was certified, so there - * is no authority to withdraw — and inserting one would fabricate a run that - * never happened. - */ private recordObservedDeviation(migrationId: string): void { - if (this.recordedDeviations.has(migrationId)) return; - if (!this._registry.getObject(DATA_MIGRATION_FLAG_OBJECT)) return; - this.recordedDeviations.add(migrationId); - this.deviationRecording = this.deviationRecording - .then(async () => { - // [#20648] Same single-row route as `readMigrationFlagVerified`. - const row: any = await this.findOne(DATA_MIGRATION_FLAG_OBJECT, { - where: { id: migrationId }, - context: { isSystem: true } as ExecutionContext, - }); - if (!row || row.id !== migrationId) return; // nothing certified — no authority to withdraw - if (row.deviation_observed_at != null && String(row.deviation_observed_at) !== '') return; // already standing - // Only a row that currently authorises something can have authority - // withdrawn. An unverified row already denies both halves, so marking - // it would add a diagnostic nobody gates on — and the next apply run - // would clear it before anything read it. - const verified = isDataMigrationFlagVerified({ - id: migrationId, - last_run_at: String(row.last_run_at ?? ''), - verified_at: row.verified_at == null ? null : String(row.verified_at), - blocking: typeof row.blocking === 'number' ? row.blocking : Number(row.blocking ?? Number.NaN), - }); - if (!verified) return; - const tally = this.admittedValueShapeViolations.get(migrationId); - const now = new Date().toISOString(); - await this.update( - DATA_MIGRATION_FLAG_OBJECT, - { - id: migrationId, - deviation_observed_at: now, - deviation_detail: JSON.stringify({ - observed: 'lax-admitted-violating-value', - ...(tally?.first ?? {}), - }), - updated_at: now, - }, - { context: { isSystem: true } as ExecutionContext }, - ); - this.logger.warn( - `[value-shape] '${migrationId}': this deployment is recorded as verified, but an ` + - 'escape hatch (OS_ALLOW_LAX_MEDIA_VALUES / OS_ALLOW_LAX_VALUE_SHAPES) just admitted a ' + - `value that contract rejects (${tally?.first.object}.${tally?.first.field}: ` + - `${tally?.first.detail}). The certificate stands for everything recoverable, but ` + - 'irreversible actions are withheld — released field files are no longer collected, so ' + - 'no byte is deleted on evidence this deployment has contradicted. Fix the data and run ' + - '`os migrate ' + - (migrationId === FILE_REFERENCES_MIGRATION_ID ? 'files-to-references' : 'value-shapes') + - ' --apply` to clear it (ADR-0104).', - ); - }) - .catch((err: any) => { - // Bookkeeping must never surface as a write failure. Failing to record - // is the dangerous direction — it leaves the reclamation gate open on - // a certificate we now know is stale — so say so loudly. - this.logger.warn( - `[value-shape] could not record the observed deviation for '${migrationId}' ` + - `(${err?.message ?? err}) — the ledger still authorises irreversible collection while ` + - 'this deployment holds a value its own contract rejects; run the migration to ' + - 're-derive the gate', - ); - }); + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it (see `MigrationStateHost`). + const read_registry = () => this._registry; + const readAdmittedValueShapeViolations = () => this.admittedValueShapeViolations; + const readDeviationRecording = () => this.deviationRecording; + const writeDeviationRecording = (v: ObjectQL['deviationRecording']) => { this.deviationRecording = v; }; + const readLogger = () => this.logger; + const readRecordedDeviations = () => this.recordedDeviations; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get admittedValueShapeViolations() { return readAdmittedValueShapeViolations(); }, + get deviationRecording() { return readDeviationRecording(); }, + set deviationRecording(v) { writeDeviationRecording(v); }, + findOne: (...a) => this.findOne(...a), + get logger() { return readLogger(); }, + get recordedDeviations() { return readRecordedDeviations(); }, + update: (...a) => this.update(...a), + }; + return MigrationStateFacet.recordObservedDeviation.call(host, migrationId); } - /** - * Tear up a creation attestation this boot has just contradicted (#4769). - * - * The attestation normally never gets written in the first place: it asks - * {@link valueShapeViolationsAdmitted} before recording anything, so a boot - * that seeded a violating value declines to certify. This closes the other - * order — the certificate is already in the ledger and the contradicting - * value lands afterwards, which is reachable whenever the deployment is - * still lenient at that moment (`OS_ALLOW_LAX_MEDIA_VALUES` / - * `OS_ALLOW_LAX_VALUE_SHAPES`) or whenever a writer runs after the - * attestation point at all — the `os dev` hot-reload seeder and a runtime - * marketplace install both seed on a store this boot created. Without this - * the ledger would keep asserting a fact the store contradicts, and the NEXT - * boot would enforce it against exactly the data this one wrote. - * - * The boot's own inline seed used to head that list, via the background - * continuation of a run that overran `OS_INLINE_SEED_BUDGET_MS` — the - * attestation's `kernel:ready` backstop fired mid-seed and the tail landed - * against the certificate it had just issued. #4795 closed that ordering at - * the source: the attestation now defers while the `seed-settlement` contract - * reports a source outstanding, so the inline seed can no longer contradict - * a certificate this boot issued. This stays the safety net rather than the - * first line of defence for it. - * - * Deliberately narrow: - * - * - only on a store THIS boot created from empty, so the only verified row - * that can exist is the attestation this boot issued. Evidence produced by - * a real `os migrate … --apply` run is never rewritten from here — a scan - * that walked the whole store outranks a single write's observation, and - * an operator who opted into leniency did not ask us to revoke their - * migration; - * - never inserts. No row means nothing was certified, and the attestation - * declining on the tally is what keeps it that way; - * - once per migration id per process, and never awaited by the write. The - * recorded `blocking` count is therefore a LOWER bound — which is all the - * gate reads, since any non-zero count closes it. - */ private retractCreationAttestation(migrationId: string): void { - if (this.retractedCreationAttestations.has(migrationId)) return; - if (!this._registry.getObject(DATA_MIGRATION_FLAG_OBJECT)) return; - if (!this.wasDatastoreCreatedFromEmpty()) return; - this.retractedCreationAttestations.add(migrationId); - this.creationAttestationRetraction = this.creationAttestationRetraction - .then(async () => { - // [#20648] Same single-row route as `readMigrationFlagVerified`. - const row: any = await this.findOne(DATA_MIGRATION_FLAG_OBJECT, { - where: { id: migrationId }, - context: { isSystem: true } as ExecutionContext, - }); - if (!row || row.id !== migrationId) return; // nothing certified — nothing to revoke - if (row.verified_at == null) return; // gate already closed - const tally = this.admittedValueShapeViolations.get(migrationId); - const now = new Date().toISOString(); - let details: Record = {}; - if (typeof row.details === 'string' && row.details.length > 0) { - try { - const parsed = JSON.parse(row.details); - if (parsed && typeof parsed === 'object') details = parsed as Record; - } catch { - details = { previous_details: row.details }; - } - } - await this.update( - DATA_MIGRATION_FLAG_OBJECT, - { - id: migrationId, - verified_at: null, - blocking: tally?.count ?? 1, - details: JSON.stringify({ - ...details, - revoked: 'boot-admitted-violating-value', - revoked_at: now, - revoked_by: tally?.first, - }), - updated_at: now, - }, - { context: { isSystem: true } as ExecutionContext }, - ); - this.invalidateDataMigrationFlags(); - this.logger.warn( - `[value-shape] revoked '${migrationId}': this deployment was recorded as verified at ` + - 'creation, then this boot wrote a value that contradicts it ' + - `(${tally?.first.object}.${tally?.first.field}: ${tally?.first.detail}). ` + - 'The gate is closed again — fix the data, then run `os migrate ' + - (migrationId === FILE_REFERENCES_MIGRATION_ID ? 'files-to-references' : 'value-shapes') + - ' --apply` to re-earn it (ADR-0104).', - ); - }) - .catch((err: any) => { - // Bookkeeping must never surface as a write failure. Staying verified - // is the bad direction, so say so loudly rather than silently. - this.logger.warn( - `[value-shape] could not revoke the creation attestation for '${migrationId}' ` + - `(${err?.message ?? err}) — the ledger still claims this deployment is verified ` + - 'while its data contradicts that; run the migration to re-derive it', - ); - }); + // [#29986 codemod] The body moved to `engine-migration-state.ts`, its docblock with it (see `MigrationStateHost`). + const read_registry = () => this._registry; + const readAdmittedValueShapeViolations = () => this.admittedValueShapeViolations; + const readCreationAttestationRetraction = () => this.creationAttestationRetraction; + const writeCreationAttestationRetraction = (v: ObjectQL['creationAttestationRetraction']) => { this.creationAttestationRetraction = v; }; + const readLogger = () => this.logger; + const readRetractedCreationAttestations = () => this.retractedCreationAttestations; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get admittedValueShapeViolations() { return readAdmittedValueShapeViolations(); }, + get creationAttestationRetraction() { return readCreationAttestationRetraction(); }, + set creationAttestationRetraction(v) { writeCreationAttestationRetraction(v); }, + findOne: (...a) => this.findOne(...a), + invalidateDataMigrationFlags: (...a) => this.invalidateDataMigrationFlags(...a), + get logger() { return readLogger(); }, + get retractedCreationAttestations() { return readRetractedCreationAttestations(); }, + update: (...a) => this.update(...a), + wasDatastoreCreatedFromEmpty: (...a) => this.wasDatastoreCreatedFromEmpty(...a), + }; + return MigrationStateFacet.retractCreationAttestation.call(host, migrationId); } /** @@ -7498,10 +6342,24 @@ export class ObjectQL implements IObjectQLEngine { * write per migration id per re-run. */ invalidateDataMigrationFlags(): void { - this.fileReferencesMigrationVerified = null; - this.valueShapesMigrationVerified = null; - this.migrationGatesAnnounced = false; - this.recordedDeviations.clear(); + // [#29986 codemod] The body moved to `engine-migration-state.ts` (see `MigrationStateHost`). + const readFileReferencesMigrationVerified = () => this.fileReferencesMigrationVerified; + const writeFileReferencesMigrationVerified = (v: ObjectQL['fileReferencesMigrationVerified']) => { this.fileReferencesMigrationVerified = v; }; + const readMigrationGatesAnnounced = () => this.migrationGatesAnnounced; + const writeMigrationGatesAnnounced = (v: ObjectQL['migrationGatesAnnounced']) => { this.migrationGatesAnnounced = v; }; + const readRecordedDeviations = () => this.recordedDeviations; + const readValueShapesMigrationVerified = () => this.valueShapesMigrationVerified; + const writeValueShapesMigrationVerified = (v: ObjectQL['valueShapesMigrationVerified']) => { this.valueShapesMigrationVerified = v; }; + const host: ThisParameterType = { + get fileReferencesMigrationVerified() { return readFileReferencesMigrationVerified(); }, + set fileReferencesMigrationVerified(v) { writeFileReferencesMigrationVerified(v); }, + get migrationGatesAnnounced() { return readMigrationGatesAnnounced(); }, + set migrationGatesAnnounced(v) { writeMigrationGatesAnnounced(v); }, + get recordedDeviations() { return readRecordedDeviations(); }, + get valueShapesMigrationVerified() { return readValueShapesMigrationVerified(); }, + set valueShapesMigrationVerified(v) { writeValueShapesMigrationVerified(v); }, + }; + return MigrationStateFacet.invalidateDataMigrationFlags.call(host); } /** @@ -7653,154 +6511,77 @@ export class ObjectQL implements IObjectQLEngine { */ summaryRetryOptions: RetryOptions = {}; - /** Invalidate the cached roll-up summary index (call when metadata changes). */ private invalidateSummaryIndex(): void { - this.summaryIndex = null; - this.summaryIndexByParent = null; - } - - /** Scan all registered objects for `summary` fields and index them BOTH ways - * — by the child object they aggregate and by the parent object that owns - * them — resolving the child→parent FK field. One scan, two views of the - * identical descriptor objects, so the two indexes can never disagree about - * which roll-ups exist. - * - * [#9154] The registry read PROPAGATES. It used to be - * `try { (registry as any).getAllObjects?.() ?? [] } catch { [] }`, which - * answered a read that could not run with an invented *"no object declares a - * roll-up"* — the same shape #8895 ruled on (*discriminate or propagate*) - * and #9002 removed from the two delete-cascade seams, and there is no - * benign failure class to discriminate here either: an unreadable registry - * is never truthfully "no roll-ups". An empty index means - * {@link recomputeSummaries} recomputes nothing after every insert / update - * / delete, so every parent summary field keeps a stale value while every - * write reports success and nothing is logged. - * - * Both halves of the swallow are gone, not just the `catch`: the optional - * call `?.()` absorbed a registry that does not implement `getAllObjects` at - * all — the same structural omission a test double really did ship, and one - * that is invisible precisely because it never throws. A plain call fails - * loudly on it. - * - * ⚠️ This is a STRUCTURAL close, not a live defect: `SchemaRegistry`'s - * `getAllObjects()` is a walk over in-memory `Map`s calling `resolveObject()` - * — which returns `undefined` on every failure branch it models — over a - * fold that is spreads and comparisons. No I/O, no driver, no `throw` on the - * measured path. The pin lives in - * `engine-summary-index-registry-read-failure.test.ts`. - * - * [#19082] The SECOND way this one function invented *"nothing to - * recompute"* — a different input, the same outcome. The child→parent FK - * was resolved by comparing the child field's `reference` carrier to the - * parent's name, so a carrier no reader can read (a non-string where - * `FieldSchema.reference` declares an optional string) compared false - * against every name, `fkField` stayed unset, and the `continue` below - * dropped a DECLARED roll-up out of both indexes with nothing logged. - * - * ⛔ The fix is NOT a looser comparison — that would trade a silent - * stall for a MIS-MATCHED foreign key, which is more expensive. The - * carrier is read through the one arbiter and the skip now SAYS SO, at - * `error` (a persisted summary silently stops tracking its children while - * every write keeps reporting success — the durability class). Absence and - * every readable carrier behave exactly as before; the resolution rule - * itself is deliberately left where PR #18503's C2 boundary put it. Pinned - * in `engine-summary-index-unreadable-carrier.test.ts`, both directions. */ + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). + const readSummaryIndex = () => this.summaryIndex; + const writeSummaryIndex = (v: ObjectQL['summaryIndex']) => { this.summaryIndex = v; }; + const readSummaryIndexByParent = () => this.summaryIndexByParent; + const writeSummaryIndexByParent = (v: ObjectQL['summaryIndexByParent']) => { this.summaryIndexByParent = v; }; + const host: ThisParameterType = { + get summaryIndex() { return readSummaryIndex(); }, + set summaryIndex(v) { writeSummaryIndex(v); }, + get summaryIndexByParent() { return readSummaryIndexByParent(); }, + set summaryIndexByParent(v) { writeSummaryIndexByParent(v); }, + }; + return SummariesAndExpansionFacet.invalidateSummaryIndex.call(host); + } + private buildSummaryIndex(): { byChild: Map; byParent: Map; } { - // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts` (see `SummariesAndExpansionHost`). + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). const read_registry = () => this._registry; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, get logger() { return readLogger(); }, }; - return buildSummaryIndex.call(host); + return SummariesAndExpansionFacet.buildSummaryIndex.call(host); } /** `registry.objectRevision` the cached {@link summaryIndex} was built at. */ private summaryIndexRevision = -1; - /** - * Ensure both roll-up indexes are present and current. Split out of - * {@link getSummaryDescriptors} so the parent-side view (#5749) shares the - * exact same staleness rule instead of re-deriving one. - * - * ## [#9154] A failed build leaves NO cache entry — *a poisoned cache entry - * must not survive the read that poisoned it* - * - * This is the limb that made the swallow removed from - * {@link buildSummaryIndex} worse than its two #9002 siblings. Those invented - * an answer ONCE, per delete. Here the answer is MEMOIZED and stamped with the - * registry's current `objectRevision` — so one failed read did not degrade one - * write, it installed an empty index that every subsequent write then read as - * a measured answer. And `objectRevision` moves only on a metadata MUTATION - * (`registerObject`, `unregisterObject`, `unregisterObjectsByPackage`, - * `removeObjectOverlay`, `invalidate`, `invalidateAll`, `reset`) — never on a - * data write. A steady-state deployment performs none of those, so the - * invented emptiness outlived its cause for the whole process lifetime, - * ending only at a restart or at an unrelated publish. The failure was not - * "briefly wrong"; it was wrong until an unrelated event. - * - * Two things hold the invariant, and both are load-bearing — do not reorder: - * - * 1. the build runs to completion into a LOCAL before anything is published - * to `this`, so a throw cannot leave a half-written pair of indexes, and - * the revision stamp is written LAST — an unstamped failure is retried on - * the very next call rather than being remembered as an answer; - * 2. the `catch` below clears whatever was cached and resets the stamp before - * rethrowing. Behaviourally that is a no-op today (nothing reads the - * fields without coming through here first, and the stale entry would be - * rebuilt anyway) — it is here so the guarantee is stated in code instead - * of resting on the survey in (1). ⛔ It swallows nothing: the error is - * rethrown unchanged, and `engine-summary-index-registry-read-failure.test.ts` - * pins both the propagation and the retry. - */ private ensureSummaryIndexes(): void { - // Rebuild whenever the REGISTRY's object set has moved since the index was - // built — not only when someone remembered to call - // `invalidateSummaryIndex`. That single site (`registerApp`) is bypassed by - // the runtime publish path, which registers straight into the registry - // (`protocol.saveMetaItem` → `registry.registerObject`). A kernel that had - // already done one write — publishing itself writes `sys_metadata` — held an - // index built before the new object existed, so every child write of a - // freshly published roll-up skipped the recompute and the parent read null - // until the process restarted. That is exactly how an AI-built app's - // "已完成任务数" shipped empty over correct metadata (cloud#970). - const revision = (this._registry as unknown as { objectRevision?: number })?.objectRevision; - const stale = typeof revision === 'number' && revision !== this.summaryIndexRevision; - if (!this.summaryIndex || !this.summaryIndexByParent || stale) { - let built: { byChild: Map; byParent: Map }; - try { - built = this.buildSummaryIndex(); - } catch (err) { - // [#9154] Nothing measured came back, so nothing is remembered — not - // even the entry that was already here, which the staleness test above - // has just judged out of date. The next call rebuilds from scratch. - this.invalidateSummaryIndex(); - this.summaryIndexRevision = -1; - throw err; - } - this.summaryIndex = built.byChild; - this.summaryIndexByParent = built.byParent; - if (typeof revision === 'number') this.summaryIndexRevision = revision; - } + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). + const read_registry = () => this._registry; + const readSummaryIndex = () => this.summaryIndex; + const writeSummaryIndex = (v: ObjectQL['summaryIndex']) => { this.summaryIndex = v; }; + const readSummaryIndexByParent = () => this.summaryIndexByParent; + const writeSummaryIndexByParent = (v: ObjectQL['summaryIndexByParent']) => { this.summaryIndexByParent = v; }; + const readSummaryIndexRevision = () => this.summaryIndexRevision; + const writeSummaryIndexRevision = (v: ObjectQL['summaryIndexRevision']) => { this.summaryIndexRevision = v; }; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + buildSummaryIndex: (...a) => this.buildSummaryIndex(...a), + invalidateSummaryIndex: (...a) => this.invalidateSummaryIndex(...a), + get summaryIndex() { return readSummaryIndex(); }, + set summaryIndex(v) { writeSummaryIndex(v); }, + get summaryIndexByParent() { return readSummaryIndexByParent(); }, + set summaryIndexByParent(v) { writeSummaryIndexByParent(v); }, + get summaryIndexRevision() { return readSummaryIndexRevision(); }, + set summaryIndexRevision(v) { writeSummaryIndexRevision(v); }, + }; + return SummariesAndExpansionFacet.ensureSummaryIndexes.call(host); } - /** Roll-up descriptors for summaries that aggregate `childObject` — i.e. the - * ones a write to `childObject` must recompute. Semantics unchanged. */ private getSummaryDescriptors(childObject: string): SummaryDescriptor[] { - this.ensureSummaryIndexes(); - return this.summaryIndex!.get(childObject) ?? []; + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). + const readSummaryIndex = () => this.summaryIndex; + const host: ThisParameterType = { + ensureSummaryIndexes: (...a) => this.ensureSummaryIndexes(...a), + get summaryIndex() { return readSummaryIndex(); }, + }; + return SummariesAndExpansionFacet.getSummaryDescriptors.call(host, childObject); } - /** [#26950] The child-side foreign-key columns the roll-ups over - * `childObject` read: what a predicate write must read off its matched rows, - * before it writes, to hand {@link recomputeSummaries} the parents to - * recompute. Empty when no roll-up aggregates `childObject`. */ private summaryForeignKeyColumns(childObject: string): string[] { - return [...new Set(this.getSummaryDescriptors(childObject).map((desc) => desc.fkField))]; + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). + const host: ThisParameterType = { + getSummaryDescriptors: (...a) => this.getSummaryDescriptors(...a), + }; + return SummariesAndExpansionFacet.summaryForeignKeyColumns.call(host, childObject); } /** Roll-up descriptors for the summary fields `parentObject` OWNS (#5749) — @@ -7812,42 +6593,21 @@ export class ObjectQL implements IObjectQLEngine { * same FK resolution, same filter, same staleness rule. Re-deriving them * would be a second index that disagrees the first time either moves. */ getOwnedSummaryDescriptors(parentObject: string): SummaryDescriptor[] { - this.ensureSummaryIndexes(); - return this.summaryIndexByParent!.get(parentObject) ?? []; + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts` (see `SummariesAndExpansionHost`). + const readSummaryIndexByParent = () => this.summaryIndexByParent; + const host: ThisParameterType = { + ensureSummaryIndexes: (...a) => this.ensureSummaryIndexes(...a), + get summaryIndexByParent() { return readSummaryIndexByParent(); }, + }; + return SummariesAndExpansionFacet.getOwnedSummaryDescriptors.call(host, parentObject); } - /** - * Seed the roll-up `summary` fields a freshly-created row owns (#5749). - * - * `recomputeSummaries` only ever visits parents named by a child write, so a - * parent that has NEVER had a child is never visited and its summary column - * keeps whatever insert put there — `null`. Delete the last child and the - * parent DOES get visited (via `previous`) and lands on 0. Same logical state, - * two different values: `filter ["task_count","=",0]` silently skipped every - * parent that never had a child, and so did sorting, GROUP BY and any formula - * reading the field (null propagation). - * - * The fix is at the producer: write the empty-collection value at create time, - * so `count`/`sum` start at 0 and only ever move to another number. - * `min`/`max`/`avg` have no empty-set value and deliberately stay `null` — - * {@link summaryEmptySetValue} is the single list both this and the recompute - * fallback read. - * - * Author-supplied values are never overwritten. The `!= null` test matches - * {@link applyFieldDefaults} exactly (#2706): on INSERT an explicit `null` is - * "no value supplied", any real value — including a deliberate 0 or a seeded - * count — is respected. Runs before the `beforeInsert` hooks for the same - * reason defaults do, so a hook still has the final say. - * - * Existing rows are untouched: this is create-time only, so parents already - * stored with `null` stay `null` until a child write recomputes them. - */ private initializeSummaryFields(object: string, record: any): any { - // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts` (see `SummariesAndExpansionHost`). - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). + const host: ThisParameterType = { getOwnedSummaryDescriptors: (...a) => this.getOwnedSummaryDescriptors(...a), }; - return initializeSummaryFields.call(host, object, record); + return SummariesAndExpansionFacet.initializeSummaryFields.call(host, object, record); } /** @@ -7950,267 +6710,24 @@ export class ObjectQL implements IObjectQLEngine { return failures; } - /** - * Post-process expand: resolve lookup/master_detail fields by batch-loading related records. - * - * This is a driver-agnostic implementation that uses secondary queries ($in batches) - * to load related records, then injects them into the result set. - * - * @param objectName - The source object name - * @param records - The records returned by the driver - * @param expand - The expand map from QueryAST (field name → nested QueryAST) - * @param depth - Current recursion depth (0-based) - * @returns Records with expanded lookup fields (IDs replaced by full objects) - */ - private async expandRelatedRecords( + private expandRelatedRecords( objectName: string, records: any[], expand: Record, depth: number = 0, execCtx?: ExecutionContext, ): Promise { - if (!records || records.length === 0) return records; - if (depth >= ObjectQL.MAX_EXPAND_DEPTH) return records; - - const objectSchema = this._registry.getObject(objectName); - // If no schema registered, skip expand — return raw data - if (!objectSchema || !objectSchema.fields) return records; - - for (const [fieldName, nestedAST] of Object.entries(expand)) { - // [#4371] The nested AST is caller-authored too: a wire spelling inside - // `expand: { rel: { sort } }` used to be silently dropped exactly like - // the top-level bag (this loop reads only canonical keys). Same - // rejection, scoped to the four wire-only pairs — the nested shape is a - // QueryAST, not an option bag, so the option-key gate does not apply. - if (nestedAST && typeof nestedAST === 'object') { - for (const slot of ENGINE_WIRE_ONLY_SLOTS) { - for (const alias of slot.aliases) { - if ((nestedAST as Record)[alias] != null) { - throw new Error( - `expand['${fieldName}'] on '${objectName}' does not accept '${alias}': it is a wire ` + - `spelling of '${slot.canonical}', folded by the RPC/protocol layer — a direct engine ` + - `call bypasses that fold, so the value would be silently dropped, not applied. Pass ` + - `'${slot.canonical}' (${WIRE_ONLY_CANONICAL_SHAPES[slot.canonical] ?? 'the canonical QueryAST shape'}) instead.`, - ); - } - } - } - } - const fieldDef = objectSchema.fields[fieldName]; - - // Skip if field not found or not a relationship type. - // - // `user` is a lookup specialized to sys_user and `tree` is a hierarchical - // self-reference — both carry the same `reference` + id storage, so both - // expand through this exact path (single or multiple). - // - // [#4226] The membership test is `REFERENCE_VALUE_TYPES` — the spec's own - // list of types whose value "points at another record … the related record - // object in expanded form" — rather than a hand-copied `!==` chain. The - // chain had drifted: it omitted `tree`, so a `$expand` of a hierarchy - // field returned the raw parent id and rendered as a bare placeholder, - // while `field-value.zod.ts` and objectui's `EXPANDABLE_FIELD_TYPES` both - // declared it expandable. Reading the shared set is what stops the - // protocol's expand gate (which validates against the same set) from ever - // admitting a field this loop then silently skips. - // - // [cloud#983] The TARGET comes from `referenceTargetOf` for that same - // anti-drift reason. A raw `fieldDef.reference` read made `{ type: - // 'user' }` (no `reference`) targetless here AND at the gate — but - // `user`'s target is fixed BY THE TYPE (`sys_user`; `Field.user()` takes - // no target argument), so the field was fully specified and the request - // was refused `400 … declares no target object`. Both sides now ask the - // one function what a reference field points at. - if (!fieldDef) continue; - if (!REFERENCE_VALUE_TYPES.has(fieldDef.type)) continue; - const referenceObject = referenceTargetOf(fieldDef); - if (!referenceObject) continue; - // [#24358] ⛔ A reference into the stored-metadata-body family is never - // expanded, at any depth and for any caller. The family's projection runs - // at its doors and only for the object a request names, so this sub-read - // would nest the stored body and its content hash unprojected under the - // requested rows — material the family's own door withholds from the same - // reader. The field keeps the stored id, the answer a reference the - // caller may not expand gets (an unexposed target, #22661, or a refused - // sub-read, below), so it says nothing about the row. - if (isStoredMetadataBodyObject(referenceObject)) continue; - - // Collect all foreign key IDs from records (handle both single and multiple values) - const allIds: any[] = []; - for (const record of records) { - const val = record[fieldName]; - if (val == null) continue; - if (Array.isArray(val)) { - allIds.push(...val.filter((id: any) => id != null)); - } else if (typeof val === 'object') { - // Already expanded — skip - continue; - } else { - allIds.push(val); - } - } - - // De-duplicate IDs - const uniqueIds = [...new Set(allIds)]; - if (uniqueIds.length === 0) continue; - - // Batch-load related records using $in query - try { - // Constrain the batch to the collected FK ids. When the nested expand - // AST carries its own `where`, AND-merge it via an explicit `$and` - // group rather than spreading keys onto the id filter: a shallow spread - // would clobber the `id` constraint (if the nested filter also keys - // `id`) and is ambiguous when the nested filter is a top-level `$or` / - // `$and`. The `$and` wrapper is correct for every FilterCondition shape. - const idFilter = { id: { $in: uniqueIds } }; - const where = nestedAST.where - ? { $and: [idFilter, nestedAST.where] } - : idFilter; - - // [#2850] Resolve the referenced object through the engine's own read - // path (`this.find`) rather than the raw driver, so the security - // middleware applies the REFERENCED object's RLS + FLS to the expanded - // batch — not merely tenant isolation. A single `find` over the - // collected `id $in [...]` batch re-enters the middleware exactly once - // (no N+1), preserving the batched load. `nestedAST.limit/offset` are - // still intentionally NOT forwarded: this path batch-loads every - // parent's related records in one query, so a *per-parent* limit/offset - // can't be expressed — a global cap would silently drop records other - // parents need. Paginate by querying the related object directly. - // - // The `__expandRead` marker (set here, never from client input — - // `executionContext` is server-built) tells the security layer this is - // an expansion sub-read. It does NOT relax that layer: since #7626 the - // referenced object takes the FULL treatment — CRUD gate, - // requiredPermissions, RLS and FLS — so you may expand only rows you - // could have read directly. (#2850 shipped a waiver for "public" - // referenced objects; it keyed on an axis almost no object declares and - // never checked the caller's grants, so it disclosed private records to - // callers the gate had already refused.) A refusal is caught below and - // the bare FK id is retained, which is why the parent read still - // succeeds. `expand` is intentionally omitted from this query so `find` - // does not re-expand — nested relations recurse below under the depth - // guard. - // - // [#7537] The join key is MACHINERY, not a caller-chosen column. This - // sub-read's projection is forwarded from `nestedAST.fields`, and the - // map built below is keyed on `rec.id` — so a nested projection that did - // not name `id` returned rows carrying no `id`, built an EMPTY map, and - // fell through the `recordMap.get(...) ?? val` injection, writing the - // original foreign key back. The expansion was then indistinguishable - // from never having been requested: `200`, and the field still holds a - // valid-looking id. - // - // The failing spelling is the one the platform itself PRESCRIBES: both - // `FIELD_NODE_OBJECT_FORM_REMOVED` and `QUERY_JOINS_REMOVED` - // (`query.zod.ts`) tell authors migrating off the retired nested-select - // object form and off `query.joins` to write - // `expand: { owner: { object: 'user', fields: ['name'] } }`. Refusing the - // mismatch would turn two retirement messages' own migration target into - // an error, so the prescribed form is made to WORK instead: `id` is added - // to the sub-read unconditionally and stripped back out of the emitted - // nested record when the caller did not ask for it. - // - // `id` is the only join key there is, literally: `referenceTargetOf` - // yields the target OBJECT and a reference field carries no - // target-column metadata, so the batch filter above is `{ id: { $in } }` - // by construction. There is no configurable key to add instead. - // - // Only a driver that honours a projection exactly could ever show this — - // `SqlDriver` emits `builder.select(query.fields)` verbatim, while - // `InMemoryDriver.projectFields` force-adds `id` to every projection, - // which is why mock/in-memory suites could not reach the defect and a - // better-sqlite3 QA run (#7463) could. - const nestedFields = Array.isArray(nestedAST.fields) && nestedAST.fields.length > 0 - ? nestedAST.fields - : undefined; - const joinKeyRequested = !nestedFields || nestedFields.includes('id'); - const relatedRecords = await this.find( - referenceObject, - { - where, - // [commit 74155c735] The `as any` these two carried is gone: `find` takes the - // author state now, and the parsed nodes a `QueryAST` holds are - // valid author input (a present `order` is legal to write). - ...(nestedAST.fields - ? { fields: joinKeyRequested ? nestedAST.fields : [...nestedFields!, 'id'] } - : {}), - ...(nestedAST.orderBy ? { orderBy: nestedAST.orderBy } : {}), - context: { ...(execCtx ?? {}), __expandRead: true } as ExecutionContext, - }, - ) ?? []; - - // Build a lookup map: id → record - const recordMap = new Map(); - for (const rec of relatedRecords) { - const id = rec.id; - if (id != null) recordMap.set(String(id), rec); - } - - // Recursively expand nested relations if present - if (nestedAST.expand && Object.keys(nestedAST.expand).length > 0) { - const expandedRelated = await this.expandRelatedRecords( - referenceObject, - relatedRecords, - nestedAST.expand, - depth + 1, - execCtx, - ); - // Rebuild map with expanded records - recordMap.clear(); - for (const rec of expandedRelated) { - const id = rec.id; - if (id != null) recordMap.set(String(id), rec); - } - } - - // [#7537] Strip the join key back out when the caller did not name it. - // It was added above for THIS function's own lookup, so the emitted - // nested record carries exactly the columns the projection asked for — - // the same contract a top-level `fields` gets. Deliberately AFTER the - // recursive expand: that block rebuilds `recordMap` keyed on `rec.id` - // and still needs the key present. Each related record is stripped once - // here rather than per-parent, because one record object is injected - // into every parent that points at it. - if (!joinKeyRequested) { - for (const rec of recordMap.values()) { - if (rec && typeof rec === 'object') delete rec.id; - } - } - - // Inject expanded records back into the original result set - for (const record of records) { - const val = record[fieldName]; - if (val == null) continue; - - if (Array.isArray(val)) { - record[fieldName] = val.map((id: any) => recordMap.get(String(id)) ?? id); - } else if (typeof val !== 'object') { - record[fieldName] = recordMap.get(String(val)) ?? val; - } - // If val is already an object, leave it as-is - } - } catch (e) { - // Graceful degradation: if expand fails, keep original IDs - this.logger.warn('Failed to expand relationship field; retaining foreign key IDs', { - object: objectName, - field: fieldName, - reference: referenceObject, - error: (e as Error).message, - }); - } - - // [#29460] `referenceNameVisible` (ruling 6574398310, letter B): the ids - // the caller's own expansion above left bare are asked whether the - // caller was REFUSED them, and a refused target is served by its name - // and id only. Every other value keeps the answer the loop gave it. See - // {@link serveReferenceNames}. - if ((fieldDef as { referenceNameVisible?: unknown }).referenceNameVisible === true) { - await this.serveReferenceNames(objectName, records, fieldName, referenceObject, execCtx); - } - } - - return records; + // [#29986 codemod] The body moved to `engine-summaries-and-expansion.ts`, its docblock with it (see `SummariesAndExpansionHost`). + const read_registry = () => this._registry; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + expandRelatedRecords: (...a) => this.expandRelatedRecords(...a), + find: (...a) => this.find(...a), + get logger() { return readLogger(); }, + serveReferenceNames: (...a) => this.serveReferenceNames(...a), + }; + return SummariesAndExpansionFacet.expandRelatedRecords.call(host, objectName, records, expand, depth, execCtx); } /** @@ -8520,31 +7037,20 @@ export class ObjectQL implements IObjectQLEngine { return resolveWhereTokens.call(host, ast, execCtx, lowering, position, operation); } - /** - * [#20802] Stage 2 of `where` admission on every verb, whole: resolve the - * placeholders, then LOWER EACH NESTED-RELATION CONDITION - * ({@link lowerRelationConditions}), then run the shared lowering - * ({@link resolveThenLowerWhere}'s second half, `lowerFilterCondition`). - * - * The order is ADR-0053 D-D1 item 3's, extended by one step: tokens first, - * so a condition on the related object carries the resolved values (one - * instant for the whole filter) into the related read; the relation step - * before the shared lowering, so that lowering reads the `$in` / `$contains` - * the relation step produced — including the NULL-safe `$not` over it — and - * never a nested object whose columns belong to another object. - */ - private async resolveRelateThenLowerWhere( + private resolveRelateThenLowerWhere( object: string, operation: string, where: W, execCtx: ExecutionContext | undefined, lowering: FilterLoweringOptions, ): Promise { - const resolved = resolveWhereFilterTokens( - where, execCtx, whereResolvedTokenJudge(object, operation, this._registry.getObject(object)), - ); - const related = await this.lowerRelationConditions(object, operation, resolved, execCtx); - return lowerFilterCondition(related, lowering); + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it (see `FindHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + lowerRelationConditions: (...a) => this.lowerRelationConditions(...a), + }; + return (FindFacet.resolveRelateThenLowerWhere).call(host, object, operation, where, execCtx, lowering); } /** @@ -8593,77 +7099,29 @@ export class ObjectQL implements IObjectQLEngine { return (lowerRelationConditions).call(host, object, operation, where, execCtx); } - /** - * The write-path counterpart of {@link resolveWhereTokens}: return `options` - * with `where` placeholders expanded (#3810). - * - * Returns the SAME object when nothing resolved — `resolveFilterTokens` - * returns its input by reference on a placeholder-free tree, so the common - * path allocates nothing. When something does resolve, a shallow copy is - * made rather than assigning through: `options` belongs to the caller, and - * writing back would bake one request's user id into a filter object the - * caller may reuse (view metadata and flow node config both get reused). - * - * [ADR-0053 D-D1, amended — #5930] …and lowered, in the same stage - * ({@link resolveThenLowerWhere}), with the object's declared-type reader. - * The lowering is copy-on-write too, so a `where` it rewrites lands on the - * copy, never on the caller's object. - */ - private async withResolvedWhere( + private withResolvedWhere( object: string, operation: string, options: T, lowering: FilterLoweringOptions, ): Promise { - if (!options || options.where == null) return options; - // [#20157] Through the stage function the judge also calls. [#20802] …with - // the nested-relation step between resolution and the shared lowering. - const resolved = await this.resolveRelateThenLowerWhere(object, operation, options.where, options.context, lowering); - return resolved === options.where ? options : ({ ...options, where: resolved } as T); + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it (see `FindHost`). + const host: ThisParameterType = { + resolveRelateThenLowerWhere: (...a) => this.resolveRelateThenLowerWhere(...a), + }; + return (FindFacet.withResolvedWhere).call(host, object, operation, options, lowering); } - /** - * ADR-0061: expand `search` into a server-resolved cross-field `$or` of - * `$icontains`, AND it with any caller `where`, then strip the search keys off - * the AST. - * - * Shared by `find` and `findOne` (#4419). It lived inline in `find` and - * nowhere else, while `ENGINE_FIND_OPTION_KEYS` — the one legal-key set BOTH - * methods are checked against (see {@link ENGINE_OPTION_KEY_SETS}) — declares - * `search`/`searchFields` for both. So `findOne({ search })` passed the gate, - * rode onto the AST verbatim, and reached a driver: no driver reads - * `ast.search` (the expansion is the engine's job by ADR-0061), so the - * predicate vanished and the forced `limit: 1` turned it into the first row of - * the WHOLE object — a real, plausible-looking record unrelated to the search. - * That is #4419's reported failure exactly, under a different key than the - * `filter` #4346 closed; one expander, called from both, is what stops the - * pair drifting again. - * - * Field resolution is server-side (declared `searchableFields` → - * auto-default); the optional `searchFields` override is intersected with the - * allowed set, never widened. All drivers already execute `$or`/`$icontains`, - * so this needs no driver changes. - * - * [#29394] A resolved field whose declared type can never hold a string (the - * #15661 door's verdict) gets no clause — `expandSearchToFilter` skips it — - * because that door judges only the caller's `where`, above, and never this - * expansion. The set itself is not narrowed, so the refusals below judge it - * as before, and a set of only such fields answers no rows. - * - * The keys are deleted whether or not anything expanded — leaving them on - * would hand the driver a key it does not read, which is the same silent drop - * one layer down. - */ private expandSearchOnAst( ast: QueryAST, schema: ServiceObject | undefined, context: ExecutionContext | undefined, ): Promise { - // [#29986 codemod] The body moved to `engine-find.ts` (see `FindHost`). - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it (see `FindHost`). + const host: ThisParameterType = { resolveSearchQueryableFields: (...a) => this.resolveSearchQueryableFields(...a), }; - return expandSearchOnAst.call(host, ast, schema, context); + return FindFacet.expandSearchOnAst.call(host, ast, schema, context); } /** @@ -8688,140 +7146,33 @@ export class ObjectQL implements IObjectQLEngine { return resolveSearchQueryableFields.call(host, object, context); } - /** - * The aggregate and count verbs' door into {@link expandSearchOnAst} — ⛔ not - * a second expander. `QuerySchema.search` (ADR-0061 D1) is declared beside - * `groupBy` / `aggregations` with no carve-out, so a grouped read under a - * search groups the SEARCHED rows, and a count under one counts them (#22790: - * `findData`'s paged `total`): the same cross-field `$or`, resolved from the - * same server-side searchable set, `internal: true` exclusion and - * `searchFields` narrowing, ANDed with `where` exactly as on `find` — before - * the security middlewares run, so an injected RLS predicate composes with it - * the same way on every verb. - * - * The expansion runs on a carrier AST holding only the three keys it reads, - * and the result is a COPY of `query` with `where` replaced and the two - * search keys gone: the bag belongs to the caller (view metadata and flow - * node config are reused), and a search key left on it would ride to - * `opCtx.options` unexpanded. The common path — no search key — returns the - * same reference and allocates nothing. - */ - private async expandSearchOnOptions( + private expandSearchOnOptions( object: string, query: T, options: EngineReadOptions | undefined, ): Promise { - if (query.search === undefined && query.searchFields === undefined) return query; - const { search, searchFields, ...rest } = query; - const carrier = { object, where: rest.where, search, searchFields } as QueryAST; - await this.expandSearchOnAst(carrier, this._registry.getObject(object), mergeReadContext(query.context, options?.context)); - if (carrier.where === undefined) return rest as T; - return { ...rest, where: carrier.where } as T; + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it (see `FindHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + expandSearchOnAst: (...a) => this.expandSearchOnAst(...a), + }; + return (FindFacet.expandSearchOnOptions).call(host, object, query, options); } - /** - * [commit 74155c735] Fill the author-state defaults the query schemas declare, so the - * AST handed to middlewares, hooks and drivers is the PARSED state - * `QueryAST` (a `z.infer` type) promises. - * - * ADR-0122 made `EngineQueryOptions` the author state (`z.input`): a key - * with a declared `.default()` is optional to write. `find`/`findOne` kept - * demanding the parsed state anyway (commit 53068c130 pinned them back) because the - * engine built its AST by bare spread and filled no default — `order: - * undefined` would have ridden straight to the driver. This is the filling. - * Each defaulting node is run through ITS OWN schema rather than - * hand-assigning values, so a default declared in `packages/spec` stays the - * single source of truth: - * - * - `orderBy[]` nodes through `SortNodeSchema` — fills `order: 'asc'`, the - * query path's one declared default. Measured before the flip: every - * driver already coalesces a missing `order` to `'asc'` (`sql-driver.ts` - * `s.order || 'asc'`, `memory-driver.ts`, `mongodb-driver.ts`, - * `remote-transport.ts` likewise), so the filled value changes no query's - * answer — it makes the AST say what the drivers were already assuming. - * Parsing also applies the node's declared strictness to type-BYPASSING - * callers: an unknown sort key, or the retired `direction` spelling, is - * now refused with the schema's own prescription instead of silently - * dropped-or-honored per driver (#4721's defect class) — the same refusal - * `normalizeSortNodes` already makes on the wire path. - * - `expand` values recurse: a nested query is the same authoring surface. - * No driver reads `ast.expand` (the engine expands post-fetch), and the - * nested read that executes re-enters `find()` — which fills again — so - * the recursion keeps the AST's type honest without a cast. - * - * `search` is deliberately NOT parsed, though `FullTextSearchSchema` carries - * three flag defaults (`fuzzy`/`operator`/`highlight`). Two measurements - * decide it. First, nothing can ever read them off the AST: no executor - * reads the flags at all (#4286 — the ADR-0061 expansion reads only `query` - * + `fields`), and {@link expandSearchOnAst} deletes `search` from the AST - * before middlewares, hooks or the driver see it, so a filled value would be - * constructed and then discarded unread. Second, parsing would REFUSE input - * the engine deliberately accepts: the wire path hands this method - * `search.fields` in the comma-STRING shape (and the `q` spelling) that - * `resolveSearchFields`/`normalizeSearch` tolerate by design — pinned in - * `query-expression-conformance.test.ts` — while the schema declares - * `fields: string[]`. The type-level gap this leaves (author-state `search` - * inside a `QueryAST`-typed value, until the key is deleted a few lines - * later) is covered by the same single cast as `expand`, below. - * - * `where`/`fields`/`limit`/`offset`/`top` carry no `.default()` or - * `.transform()` (pinned in `filter.zod.ts`'s own docs) and are not parsed — - * the cost is one small-object parse per authored sort node / search - * config, only when the key is present. - */ private fillQueryAstDefaults>( query: T, ): T & Pick & { expand?: Record } { - const out: Record = { ...query }; - if (Array.isArray(out.orderBy)) { - out.orderBy = out.orderBy.map((node) => SortNodeSchema.parse(node)); - } - if (out.expand != null && typeof out.expand === 'object') { - const expand: Record = {}; - for (const [field, nested] of Object.entries(out.expand)) { - expand[field] = this.fillQueryAstDefaults(nested as QueryInput); - } - out.expand = expand; - } - // The one cast in the flip: `orderBy`/`expand` are rebuilt above; `search` - // is claimed-but-not-parsed, per the doc — deleted from the AST unread. - return out as T & Pick & { expand?: Record }; + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it (see `FindHost`). + const host: ThisParameterType = { + fillQueryAstDefaults: (...a) => this.fillQueryAstDefaults(...a), + }; + return (FindFacet.fillQueryAstDefaults).call(host, query); } - /** - * Refuse a `findOne` that selects nothing in particular (#4419). - * - * The AST reaching here is the CALLER's own intent: aliases folded, unknown - * keys refused, `search` expanded — but the security/sharing middlewares have - * not run yet, and that ordering is the point. An injected RLS predicate - * narrows *which* rows are visible; it does not make "whichever of them comes - * first" a thing the caller asked for. Judging the post-middleware AST would - * pass every query on a scoped object and leave the hole open where it is - * most expensive. - * - * "Selects nothing" is read the same way #3896 read an empty sharing - * criteria: absent, `null`, or `{}` — the three shapes that mean "match every - * row". This guard closes the one case that is unambiguously - * match-everything, not everything it cannot prove. - * - * [#5158] This comment used to add: "a `where` that is not a plain object (an - * expression tree) is the DRIVER'S to interpret, and counts as a predicate." - * That sentence was the engine's blessing of a second filter dialect, and it - * cost exactly what a blessing costs — `findOne({ where: [] })` counted as a - * predicate, walked past this guard, and returned an ARBITRARY row: the #4419 - * defect surviving inside #4419's own guard. `FilterArray` is now lowered by - * {@link lowerWhereFilterArray} at every entry point, so by the time this - * runs `where` is a `FilterCondition` or nothing. The `Array.isArray` arm - * below is kept as defence in depth for a subclass or a future caller that - * reaches this method without lowering — it is no longer a contract. - * - * `orderBy` is the other way to be specific, and a legitimate one — "the - * newest", "the highest priority". It is honored on this path by every - * driver, so it is a real answer and not a second silent drop. - */ private requireFindOnePredicate(object: string, ast: QueryAST): void { - // [#29986 codemod] The body moved to `engine-find.ts`. - return requireFindOnePredicate(object, ast); + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it. + return FindFacet.requireFindOnePredicate(object, ast); } async find(object: string, query?: EngineQueryOptions, options?: EngineReadOptions): Promise { @@ -9050,94 +7401,13 @@ export class ObjectQL implements IObjectQLEngine { return rowsWithoutFormulaWidening(opCtx.result as any[], _findWidened); } - /** - * Report a failed {@link find} at the level its CAUSE earns — then leave the - * throw exactly where it was. - * - * ## The two facts this frame used to merge - * - * "The table has not been created yet" and "the read failed" are different - * facts, and until commit 3a86a65e7 this line reported both at `error`, with a stack. - * The first one is the ordinary state of a database nobody has migrated yet, - * and every caller on that path already treats it as a normal answer and says - * so in its own code: {@link readMigrationFlagVerified} ("an unreadable table - * … → false"), `ObjectQLPlugin.readAuthoredHookRows` / - * `readAuthoredActionRows` (report `authoredRows: 0` and carry on), - * `readAuthoredTranslationLayer`, and `ObjectStoreActionActivationStore.probe` - * (whose caller already follows this frame with a `warn` stating the - * consequence in operator terms). - * - * Measured, on `os migrate plan --database-url file:`: five - * ERROR records with full stack traces, out of a command that EXITS 0 and - * prints a correct plan. Four stack traces are the first thing an operator - * sees when planning a database whose whole point is that it is not migrated - * yet — and an `error` channel that fires on a routine state is the - * over-application `sql-driver.ts` already names on a nearby line: it trains - * everyone to skim `error`. - * - * ## ⛔ What this is NOT is a broadened catch - * - * * The throw is UNCHANGED on both branches. No caller's control flow, no - * caller's `catch`, and no error envelope moves — this method decides a - * log level and nothing else. - * * Only the class that positively identifies as "relation does not exist" - * is demoted, asked through the shared {@link isMissingTableError} - * predicate (`@objectstack/metadata/errors`, #4825) — the same call - * {@link probeInstallOrganizations} and {@link resolveFileReferences} - * make, never a hand-rolled `code === '42P01'` copy. That predicate earns - * a benign verdict rather than defaulting to one: a connection drop, a - * timeout, a permission denial, a query fault — and, through its - * `excludes`, Postgres' `column "x" of relation "y" does not exist`, - * which contains a legal missing-table phrase but is a column fault on a - * table that EXISTS — all stay on the loud branch, with the stack (at - * `warn`; the last section says why not `error`). - * * The fault stays visible without this frame: the driver's own refusal - * envelope (`SqlDriver.backendStatementFault` → `logger.warn`) carries - * the table, the dialect reason and the compiled statement, and is - * untouched here. That is deliberately the surviving loud half, and it is - * the half `runtime/src/expected-read-refusal-noise.ts` already documents - * as "where this class of fault can still be picked up". - * - * `debug` rather than `warn` for exactly that reason: the driver already puts - * one warn-level line per refusal in front of a reader, so a second would - * move this noise rather than remove it. What the demotion drops is the - * duplicate and its stack; the classification survives in the meta. - * - * The `debug` demotion is READS only: a write to a table that does not exist - * is not a normal answer for any caller, and takes the write doors' `warn` - * like every other write fault does (#17052). - * - * ## The loud branch is `warn`, not `error` — the caller is told - * - * [#17212] Every cause the predicate does NOT earn a benign verdict for is - * reported at `warn`, with its message and stack, and `find`'s `catch` then - * rethrows it — `throw e` is that catch's only exit. That is AGENTS.md's - * third legal answer (*Degradation log levels*): "a failure handed to the - * CALLER is not a degradation at all … Do not bolt a `logger.error` onto - * such a site". The requester IS told — the throw is the answer — so an - * `error` line is a second, louder report of a fact the caller already has. - * The write doors moved for the same reason (#17052): their catches rethrow - * too. - * - * ⛔ Demoted, not deleted — only the level moved. The message and the stack - * survive through {@link writeFailureLogMeta}: `warn(message, meta?)` has no - * `Error` slot, and an Error handed over AS meta serializes to `{}` (its - * `message`/`stack` are non-enumerable). On the SQL read path the fault is - * also reported one frame down on the driver's own `warn` — - * `SqlDriver.backendStatementFault` for every backend fault it measured - * (connection, timeout and ACL included), `unresolvableFilterColumnRefusal` - * for an unresolvable WHERE column. The filter COMPILER's refusals - * (`uncompilableFieldReferenceError` and its siblings) have no driver line: - * they are raised before the statement runs, as refusals carrying a declared - * `400` — a rejected request, answered to its caller. - */ private reportFindFailure(object: string, error: unknown): void { - // [#29986 codemod] The body moved to `engine-find.ts` (see `FindHost`). + // [#29986 codemod] The body moved to `engine-find.ts`, its docblock with it (see `FindHost`). const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get logger() { return readLogger(); }, }; - return reportFindFailure.call(host, object, error); + return FindFacet.reportFindFailure.call(host, object, error); } /** @@ -9332,40 +7602,6 @@ export class ObjectQL implements IObjectQLEngine { return withoutFormulaWidening(opCtx.result, _findOneWidened); } - /** - * [#20805] The create path's static-`readonly` strip of ONE row, together - * with the re-default of every key it takes — the door as `insert()` runs it, - * lifted out so that {@link ObjectQL.validate} runs the same door over its - * preview rows instead of a copy of it (one function per door). `insert()` - * owns everything around it: the `isSystem` gate, the per-call - * `insertDropped` union, the `preserveAudit`-ignored WARN and partial-mode - * culling; `validate()` passes no logger, because a preview writes nothing - * and a strip line would say otherwise. - * - * The field's `defaultValue` is RE-DERIVED for every key the strip took, - * which is #3043's stated contract and a guarantee in its own right: a forged - * `approval_status` becomes `draft` — the enforced initial state — never - * NULL, so a stripped forgery cannot leave a row in a state the object's own - * rules (`requiredWhen`, the state machine) were written to exclude. The - * deleted ingress copy got this for free by running BEFORE - * `applyFieldDefaults`; a strip that runs after the hooks has to ask. Asked - * over the STRIPPED row, so a `defaultValue` expression reads the payload it - * will really be stored beside, and copied back key by key: - * `applyFieldDefaults` also fills every OTHER absent field, and a hook that - * deliberately wrote `null` must keep its null (the first defaults pass, - * ahead of the hooks, is the one that owns those keys). - * - * [#20082] A re-derived `can` default takes the write's map too — from the - * same resolution, so this asks nothing new unless it is the first pass to - * need it. Scoped to the TAKEN keys: those are the only defaults this pass - * keeps, so no other field's default can make it ask, or fail. - * - * @returns the row as the strip leaves it (the SAME reference when nothing - * was taken) and the keys taken, in the row's key order. When the - * re-default's permission resolution fails, `error` carries it and `row` - * is the row as it arrived: the caller decides whether that fails the - * write or culls the row, and the taken keys are still reported. - */ private staticReadonlyCreateStrip( object: string, readonlySubject: NonNullable>, @@ -9380,12 +7616,12 @@ export class ObjectQL implements IObjectQLEngine { permissionResolution: PermissionResolution | undefined; }, ): Promise<{ row: Record; taken: string[]; error?: unknown }> { - // [#29986 codemod] The body moved to `engine-insert.ts` (see `InsertHost`). - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-insert.ts`, its docblock with it (see `InsertHost`). + const host: ThisParameterType = { applyFieldDefaults: (...a) => this.applyFieldDefaults(...a), resolveDefaultPermissions: (...a) => this.resolveDefaultPermissions(...a), }; - return staticReadonlyCreateStrip.call(host, object, readonlySubject, row, supplied, context, pass); + return InsertFacet.staticReadonlyCreateStrip.call(host, object, readonlySubject, row, supplied, context, pass); } /** @@ -9609,34 +7845,7 @@ export class ObjectQL implements IObjectQLEngine { return validate.call(host, object, data, options); } - /** - * [#22445] The by-id update's prior-row read: the STORED row an update of - * `id` judges, or `null` when the store holds none. One spelling, read by - * `update()`'s by-id branch and by the `update`-mode preview - * ({@link ObjectQL.validate}), so the preview judges the image the write - * judges rather than a second reading of it. - * - * A raw driver read, not the read door: the rules judge the row as STORED, - * so no formula is evaluated, no file reference resolved, no read hook run. - * ⛔ It answers no access question — `update()` reaches it only after the - * write gates in its middleware chain, and the preview asks the read door - * first (see `storedRows` in `validate()`). - * - * `buildDriverOptions` carries the open transaction and the tenant scope onto - * the raw read, as on every engine read. [#21613] The row is shaped like - * every other row this engine reads off a driver: the declared field set, so - * the update's `previous` (bound for its hooks, and the audit ledger's side - * of every update diff) and its returned row are ONE view. Shaping only the - * returned row would make every update of a row carrying a retired column - * record that column as changed, its stored value included. - */ - /** - * [ADR-0142 D4] The by-id write under {@link OperationContext.writePrecondition}: - * the driver's conditional `updateMany` over `{ id, ...expect }`, then a - * readback by id. Zero rows matched is {@link ConcurrentUpdateError}. A driver - * with no `updateMany` writes unconditionally, logged once per process. - */ - private async updateByIdExpecting( + private updateByIdExpecting( driver: IDataDriver, object: string, id: unknown, @@ -9644,21 +7853,16 @@ export class ObjectQL implements IObjectQLEngine { expect: Readonly>, options: Parameters[2], ): Promise { - if (typeof driver.updateMany !== 'function') { - if (!this.preconditionDegradedWarned) { - this.preconditionDegradedWarned = true; - this.logger.warn( - `[objectql] the driver for '${object}' has no conditional update (updateMany), so a by-id write ` + - 'precondition degrades to the pre-image check and the write goes through unconditionally: two ' + - 'concurrent claims of one record may both land, and the round-robin cursor may repeat a member.', - { object }, - ); - } - return driver.update(object, id as string, data, options); - } - const affected = await driver.updateMany(object, { object, where: { ...expect, id } } as QueryAST, data, options); - if (!(Number(affected) > 0)) throw new ConcurrentUpdateError(object, String(id)); - return driver.findOne(object, { object, where: { id }, limit: 1 } as QueryAST, options); + // [#29986 codemod] The body moved to `engine-insert.ts`, its docblock with it (see `InsertHost`). + const readLogger = () => this.logger; + const readPreconditionDegradedWarned = () => this.preconditionDegradedWarned; + const writePreconditionDegradedWarned = (v: ObjectQL['preconditionDegradedWarned']) => { this.preconditionDegradedWarned = v; }; + const host: ThisParameterType = { + get logger() { return readLogger(); }, + get preconditionDegradedWarned() { return readPreconditionDegradedWarned(); }, + set preconditionDegradedWarned(v) { writePreconditionDegradedWarned(v); }, + }; + return InsertFacet.updateByIdExpecting.call(host, driver, object, id, data, expect, options); } /** [ADR-0142 D4] Set once the precondition's no-`updateMany` degradation has been logged. */ @@ -9683,36 +7887,39 @@ export class ObjectQL implements IObjectQLEngine { } private assignment(): AssignmentRuleEvaluator { - if (!this.assignmentEvaluator) { - this.assignmentEvaluator = new AssignmentRuleEvaluator({ - listRules: () => this._registry.listItems(ASSIGNMENT_RULE_TYPE) as unknown[], - getSchema: (object) => this._registry.getObject(object), - hasObject: (object) => Boolean(this._registry.getObject(object)), - queues: () => this.ownershipQueueService(), - find: async (object, where, context, limit) => - (await this.find(object, { where, ...(limit ? { limit } : {}) } as EngineQueryOptions, { context: context as ExecutionContext })) as Record[], - count: (object, where, context) => this.count(object, { where } as EngineCountOptions, { context: context as ExecutionContext }), - insert: (object, data, context) => this.insert(object, data, { context: context as ExecutionContext }), - updateExpecting: (object, id, data, expect, context) => - this.update(object, { ...data, id }, { context: context as ExecutionContext, [WRITE_PRECONDITION_OPTION]: expect } as any), - logger: this.logger, - }); - } - return this.assignmentEvaluator; + // [#29986 codemod] The body moved to `engine-insert.ts` (see `InsertHost`). + const read_registry = () => this._registry; + const readAssignmentEvaluator = () => this.assignmentEvaluator; + const writeAssignmentEvaluator = (v: ObjectQL['assignmentEvaluator']) => { this.assignmentEvaluator = v; }; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get assignmentEvaluator() { return readAssignmentEvaluator(); }, + set assignmentEvaluator(v) { writeAssignmentEvaluator(v); }, + count: (...a) => this.count(...a), + find: (...a) => this.find(...a), + insert: (...a) => this.insert(...a), + get logger() { return readLogger(); }, + ownershipQueueService: (...a) => this.ownershipQueueService(...a), + update: (...a) => this.update(...a), + }; + return InsertFacet.assignment.call(host); } - private async readUpdatePriorRow( + private readUpdatePriorRow( object: string, id: unknown, context: ExecutionContext | undefined, base?: unknown, ): Promise | null> { - const priorAst: QueryAST = { object, where: { id }, limit: 1 } as QueryAST; - const preOpts = this.buildDriverOptions(object, context, base as any); - return withDeclaredColumnsOnly( - await this.getDriver(object).findOne(object, priorAst, preOpts) as Record | null, - declaredColumnSet(this._registry.getObject(object)), - ); + // [#29986 codemod] The body moved to `engine-insert.ts` (see `InsertHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + buildDriverOptions: (...a) => this.buildDriverOptions(...a), + getDriver: (...a) => this.getDriver(...a), + }; + return InsertFacet.readUpdatePriorRow.call(host, object, id, context, base); } /** @@ -11128,9 +9335,12 @@ export class ObjectQL implements IObjectQLEngine { * reported name as "at least one row dropped this field"; `insertManyData` * surfaces it at batch level for exactly this reason. */ - async insertMany(object: string, rows: any[], options?: DataEngineInsertOptions & WriteObservabilityOptions): Promise { - if (!Array.isArray(rows)) throw new Error('insertMany expects an array of rows'); - return this.insert(object, rows, { ...(options ?? {}), __partialRowErrors: true } as any); + insertMany(object: string, rows: any[], options?: DataEngineInsertOptions & WriteObservabilityOptions): Promise { + // [#29986 codemod] The body moved to `engine-insert.ts` (see `InsertHost`). + const host: ThisParameterType = { + insert: (...a) => this.insert(...a), + }; + return InsertFacet.insertMany.call(host, object, rows, options); } /** @@ -13233,178 +11443,32 @@ export class ObjectQL implements IObjectQLEngine { return opCtx.result; } - /** - * Can the by-id delete of `object` and its whole cascade run as ONE unit of - * work? [#7413] - * - * `delete()`'s by-id branch ran `cascadeDeleteRelations` and then - * `driver.delete` with no transaction around either, and the cascade - * RE-ENTERS `this.delete()` / `this.update()` per dependent row — so every - * child committed on its own. A refusal partway (the `restrict` branch below, - * a child's permission check, a later child's `beforeDelete` hook) left an - * arbitrary PREFIX of the children deleted while the caller received 409/403 - * and reasonably read it as "nothing happened". #4620 already ruled the - * principle for the batch path — *atomic honoured for real, or refused; a - * partial delete has no natural undo* — and this path never got it. - * - * Three verdicts, because "wrap it in a transaction" is only unconditionally - * right in one of them: - * - * - **`'none'`** — no registered object declares a relation of the delete - * family ({@link statesDeleteSemantics}: `master_detail` / `lookup` / - * `tree`) pointing at `object`, so `cascadeDeleteRelations` cannot write anything - * and there is no multi-write unit to make atomic. The delete runs exactly - * as it did before this card: one `driver.delete`, no transaction opened, - * no degrade warning. A plain non-cascade delete is the CONTROL for this - * change and must stay byte-identical. - * - **`'split'`** — dependents exist, but at least one participant (the - * parent or a transitively-referencing child) resolves to a driver other - * than the engine's DEFAULT one. `transaction()` opens on the default - * driver and covers that connection alone (ADR-0119 D1 — no two-phase - * commit on `IDataDriver`), so opening one here would not buy atomicity; it - * would REFUSE the cascade outright, because `enforceTransactionOrigin` - * throws {@link CrossDatasourceTransactionWriteError} for a business write - * inside a transaction that does not cover it (#5351 / #5696 point 2, the - * 2026-08-06 ruling). A cross-datasource cascade delete works today, and - * turning it into a hard refusal is a strictly worse answer than the - * non-atomic one it has always had. So it keeps that answer and says so - * once — see {@link warnCascadeNotAtomic}. - * - **`'atomic'`** — dependents exist and every participant sits on the - * default driver. This is the shape the card measured and the shape - * virtually every deployment has; the caller gets one unit of work. - * - * A pure registry walk — no I/O, no driver call. It answers from the SCHEMA - * (which objects *could* be reached), never from row counts, so the verdict - * is stable for a given schema and costs nothing on the common path. The walk - * is the transitive closure of "objects referencing X" because a cascade - * recurses into each child's own cascade; it is bounded by - * {@link ObjectQL.MAX_CASCADE_DEPTH}, the ceiling the recursion itself carries. - * - * Fails toward `'split'` on anything it cannot resolve WITHIN the walk (an - * unregistered reference, a `getDriver` throw): the degrade is today's - * behaviour, while a wrong `'atomic'` would manufacture the refusal this - * verdict exists to avoid. - * - * [#9002] That fail-toward rule is about resolving individual PARTICIPANTS. - * It never covered the registry read that produces the participant list in - * the first place, and the `catch → 'none'` that used to sit there was not an - * instance of it: `'none'` is the one verdict that asserts something positive - * about the schema — *nothing references this object* — and an unreadable - * registry cannot support that claim. The swallow's own comment argued the - * direction from `cascadeDeleteRelations` returning on the same failure, so - * that nothing was left un-cascaded to make atomic; once that seam propagates - * the premise is gone, and this one propagates with it. Note the ORDER makes - * this seam the first to speak: `delete()` calls this before it runs the - * cascade, so an unreadable registry now fails the delete before any row is - * touched. - */ private planCascadeAtomicity(object: string): 'none' | 'split' | 'atomic' { - const objects: ServiceObject[] = this._registry.getAllObjects(); - - // Which objects reference `name` via a relation the cascade would follow. - // The family test ({@link statesDeleteSemantics}) + `reference` test is - // `cascadeDeleteRelations`'s own, so the two cannot disagree about who - // participates. `restrict` fields are INCLUDED deliberately: a restrict - // refusal is the card's core repro, and it must roll back the siblings - // already cascaded before it. - // - // [#22957] A `tree` takes part since it joined the family. Its reference - // names the object that declares it, so an object whose only relation is - // its own hierarchy now plans `'atomic'` rather than `'none'`: a subtree - // cascade is several deletes, and a refusal from a node deep in it must - // roll back the nodes deleted before it. - const referencing = (name: string): string[] => { - const out: string[] = []; - for (const child of objects) { - const childName = (child as any)?.name as string | undefined; - const fields = (child as any)?.fields as Record | undefined; - if (!childName || !fields) continue; - for (const [fieldName, fdef] of Object.entries(fields)) { - if (!statesDeleteSemantics(fdef)) continue; - // [#18550] The carrier is read through the ONE arbiter, so a - // `reference` no reader can read REFUSES here instead of reading as - // "this child does not reference `name`". The two answers stay - // different on purpose: - // - // - ABSENCE (`undefined` / `null` / `''`) is unchanged and still - // silent. `referenceCarrierOf` answers `undefined` for all three - // and this `continue` skips the field, which is what a field that - // names no target legitimately means (`FieldSchema.reference` is - // `.optional()`, `StrictField` declares it nullable). - // - UNREADABILITY is the loud one. An object- or array-valued - // carrier was TRUTHY here and then failed both name comparisons - // below, so the relation was dropped from the set silently — and - // this function's `'none'` is, by its own docblock above, the one - // verdict that asserts something POSITIVE about the schema - // ("nothing references this object"). An unreadable carrier can - // no more support that claim than an unreadable registry can. - const ref = deleteWalkTargetOf(fdef, 'ObjectQL.planCascadeAtomicity'); - if (!ref) continue; - let resolvedRef: string | undefined; - try { resolvedRef = this.resolveObjectName(ref); } catch { resolvedRef = undefined; } - if (ref !== name && resolvedRef !== name) continue; - // [#21910, #21918] The scan skips every column the registry injected - // into a federated object and the object does not provision, so this - // walk does too: the participant test stays the scan's own, as the - // comment above requires. A federated object this walk still reaches - // through a relation its author declared keeps the verdict - // `'split'`, because the scan probes that relation. - if (isFederatedUnprovisionedInjectedColumn(child, fieldName)) continue; - out.push(childName); - break; - } - } - return out; - }; - - const firstLevel = referencing(object); - if (firstLevel.length === 0) return 'none'; - - const defaultDriver = this.defaultDriver ? this.drivers.get(this.defaultDriver) : undefined; - if (!defaultDriver) return 'split'; - const onDefaultDriver = (name: string): boolean => { - try { return this.getDriver(name) === defaultDriver; } catch { return false; } + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const read_registry = () => this._registry; + const readDefaultDriver = () => this.defaultDriver; + const readDrivers = () => this.drivers; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get defaultDriver() { return readDefaultDriver(); }, + get drivers() { return readDrivers(); }, + getDriver: (...a) => this.getDriver(...a), + resolveObjectName: (...a) => this.resolveObjectName(...a), }; - - if (!onDefaultDriver(object)) return 'split'; - const seen = new Set([object]); - let frontier = firstLevel; - for (let depth = 0; depth < ObjectQL.MAX_CASCADE_DEPTH && frontier.length > 0; depth++) { - const next: string[] = []; - for (const name of frontier) { - if (seen.has(name)) continue; - seen.add(name); - if (!onDefaultDriver(name)) return 'split'; - next.push(...referencing(name)); - } - frontier = next; - } - return 'atomic'; + return CascadePlanningFacet.planCascadeAtomicity.call(host, object); } - /** - * The `'split'` verdict of {@link planCascadeAtomicity}, said out loud — once - * per object per engine instance (#7413). - * - * Same reasoning as {@link warnTransactionUnsupported}, which this mirrors - * deliberately: a capability is not available, so the delete keeps the - * non-atomic behaviour it has always had. `warn`, not `error` — at this - * moment nothing claimed-persisted has failed to land; what is missing is the - * ability to undo the cascade if a later child refuses. Escalating it would - * train readers to skim `error`, which AGENTS.md names as the mirror-image - * mistake. - */ private warnCascadeNotAtomic(object: string): void { - // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). const readCascadeNotAtomicReported = () => this.cascadeNotAtomicReported; const readDefaultDriver = () => this.defaultDriver; const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { get cascadeNotAtomicReported() { return readCascadeNotAtomicReported(); }, get defaultDriver() { return readDefaultDriver(); }, get logger() { return readLogger(); }, }; - return warnCascadeNotAtomic.call(host, object); + return CascadePlanningFacet.warnCascadeNotAtomic.call(host, object); } /** @@ -13475,85 +11539,23 @@ export class ObjectQL implements IObjectQLEngine { return referenceProbeFilter(fieldName, fdef, id); } - /** - * [#9362] Does this STORED reference value point at `id`? The exact answer - * the `$contains` pushdown in {@link ObjectQL.referenceProbeFilter} can only - * approximate. - * - * Element-wise over the array, comparing `String(v)` — the same reading - * `dangling-reference-audit.ts` already applies to a stored reference - * (`Array.isArray(raw) ? raw : [raw]`), so the two integrity surfaces cannot - * disagree about what "this row references that record" means. An EXPANDED - * record in the slot is a read shape rather than an id write, and is skipped - * there for that reason; it is skipped here too. - * - * A non-array value is still compared rather than dismissed. A - * `multiple: true` slot holding a bare scalar is off-shape, and the two - * dispositions available are not symmetric: reading it as "references - * nothing" would drop a `restrict` refusal and let a referenced record be - * deleted — the fail-OPEN direction #8895 ruled out for this same guard — - * while reading it as a reference at most refuses a delete loudly. - */ private static storedReferenceIncludes(stored: unknown, id: string | number): boolean { - const wanted = String(id); - const values = Array.isArray(stored) ? stored : [stored]; - for (const v of values) { - if (v == null || typeof v === 'object') continue; - if (String(v) === wanted) return true; - } - return false; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.storedReferenceIncludes(stored, id); } - /** - * [#9688] What a `multiple: true` reference slot holds once the deleted - * record's membership is removed — the ONE computation behind both the - * per-row `required` judgement in {@link ObjectQL.cascadeDeleteRelations} - * and the `set_null` write that judgement gates. - * - * One function rather than two readings that happen to match, because the - * judgement decides whether the write may run at all: a predicate computing - * the remainder differently could clear a delete and then have the write - * land the empty required set the judgement exists to refuse — or refuse a - * delete whose write would have produced a perfectly legal non-empty set. - * - * `String(v) !== String(id)` is the same reading - * {@link ObjectQL.storedReferenceIncludes} applies when narrowing - * `dependents`, so the member removed is exactly the member that made the - * row a dependent. A non-array value is normalized to the array spelling - * rather than dismissed, for the same reason the narrowing compares it: an - * off-shape bare scalar in a `multiple: true` slot is still a reference. - * - * [#27004] `cleared` are the members this cascade has already removed from - * the relation ({@link CascadeDeleteSet.cleared}): a row read once for a - * whole root set may still list them, and they are not written back. - */ private static remainderAfterMemberRemoval( stored: unknown, id: string | number, cleared?: ReadonlySet, ): unknown[] { - const current: unknown[] = Array.isArray(stored) ? stored : [stored]; - return current.filter((v) => String(v) !== String(id) && !cleared?.has(String(v))); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.remainderAfterMemberRemoval(stored, id, cleared); } - /** - * [#12166] The identity the pre-delete reference CHECK runs under — system, - * unconditionally (maintainer ruling 2026-08-26, option A). - * - * `sudo()`-shaped, never a bare `{ isSystem: true }`. The full reasoning sits - * at the one call site (`cascadeDeleteRelations`'s dependents probe); the - * short form is that the spread carries three things the bare spelling would - * drop — the caller's transaction handle, the TENANT scope, and `userId`, - * which is the audit ledger's triggered-by half beside `isSystem`'s - * executed-as half. - * - * A helper rather than an inline spread because the disclosure re-probe below - * has to be able to say "the caller's OWN context" by contrast, and because a - * second inline spelling is how one of the two would later gain a key the - * other lacks. - */ private static referenceCheckContext(context?: ExecutionContext): ExecutionContext { - return { ...(context ?? {}), isSystem: true } as ExecutionContext; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.referenceCheckContext(context); } /** @@ -13613,50 +11615,7 @@ export class ObjectQL implements IObjectQLEngine { return recordReferenceCheckElevation.call(host, object, id, childName, fieldName, context); } - /** - * [#12166, ruling constraint 2] May the refusal disclose HOW MANY rows - * reference this record? - * - * The elevation above is what makes this question exist. Before it, the - * dependents probe ran as the caller, so every number the `DELETE_RESTRICTED` - * envelope reported (`dependentCount`, the localized message's `count`, the - * developerMessage) was derived from rows the caller could read by - * definition. After it, the probe sees rows the caller may have no grant on - * at all — and shipping that count back would hand a caller with NO read - * permission on `childName` an exact, repeatable cardinality oracle over it: - * delete-probe each record of `object`, read the count off the 409, and - * reconstruct the hidden table's reference histogram without ever being - * allowed to read a row. - * - * The ruling closes that: the error names the referenced OBJECT and nothing - * about rows — "one notch more conservative than Salesforce". It does NOT ask - * for the count to be dropped for everyone, and dropping it would be its own - * regression (the count is what makes the refusal actionable, and it predates - * this card for callers who could always compute it). So the count is - * disclosed on exactly one condition: **the caller's own identity would have - * produced the same rows.** - * - * The comparison is on ROW IDENTITY, not on length. Two different sets can - * have the same size, and RLS narrowing is precisely the case that produces - * one — a caller who may read `childName` at the object level but whose - * row-level filter hides two of three referencing rows must not be told - * "three" either. Comparing id sets makes the object-level denial and the - * row-level narrowing the same answer, which is what they are. - * - * Costs one extra query, and only on the refusal path — the delete is being - * aborted anyway, so this is not on any successful delete's critical path. - * - * Fails CLOSED, on the DISCLOSURE axis: any throw at all (a genuine - * permission denial, an outage, a driver error) answers "not disclosable". - * That direction is deliberate and is the opposite of #8895's stance for the - * integrity probe itself — an integrity guard that cannot run must not - * silently pass, but a DISCLOSURE that cannot be justified must not silently - * happen. The two guards face opposite ways because a wrong answer costs - * opposite things: there, a permitted delete that should have been refused; - * here, a leak. Nothing about the refusal itself depends on this — an - * unanswerable probe still refuses the delete, just without the number. - */ - private async dependentCountIsDisclosable( + private dependentCountIsDisclosable( childName: string, probeWhere: Record, probedIds: readonly string[], @@ -13665,176 +11624,32 @@ export class ObjectQL implements IObjectQLEngine { id: string | number, context?: ExecutionContext, ): Promise { - // The caller IS the system (seed replay, migration, an internal cascade - // recursion). Nothing was elevated PAST them, so there is nothing to - // withhold — and withholding here would strip the count from the engine's - // own internal paths for no gain. - if (context?.isSystem === true) return true; - try { - // Typed through the DECLARED options type rather than cast: `where` is - // `Record` on `EngineQueryOptionsSchema` and `context` - // rides `BaseEngineOptions`, so nothing here needs erasing - // (`check:query-options-erasure` — the grandfathered `as any` on the - // elevated probe above is a pre-existing site this card does not sweep, - // tracked in #4918). - let own: any[] = await this.find(childName, { where: probeWhere, context }); - // The SAME narrowing the elevated probe applied (#9362), or the two sets - // would be compared through different definitions of "references this - // record" and a multi-value relation would read as non-disclosable - // always. - if (multiValued && own) { - own = own.filter((row: any) => ObjectQL.storedReferenceIncludes(row?.[fieldName], id)); - } - const ownIds = new Set((own ?? []).map((r: any) => String(r?.id))); - return ownIds.size === probedIds.length && probedIds.every((rid) => ownIds.has(rid)); - } catch (error) { - // Says something rather than rethrowing, and the asymmetry is the point - // (`check:durability-read-invention` accepts either — "say something, or - // ask the error's type"). - // - // Rethrowing is WRONG here specifically. The integrity answer is already - // in hand: the elevated probe succeeded, the dependents are known, and - // this delete is being refused either way. Propagating a failure of the - // DISCLOSURE probe would convert a correct `409 DELETE_RESTRICTED` into a - // `500` — a worse answer for the caller, produced by a query that exists - // only to decide how much detail the refusal carries. - // - // Nor is the invented value a durability hazard of the class that rule - // guards: no declared set is being read, and `false` withholds. The - // failure mode of getting this wrong is a slightly less informative - // refusal, never a permitted delete and never an over-disclosure. - // - // The one thing that WOULD be wrong is doing it silently — an operator - // seeing count-free refusals has no way to tell "withheld on purpose" - // from "the disclosure probe has been failing for a week". - this.logger.warn( - `[reference-cleanup] could not determine whether the caller may be told how many '${childName}' ` + - `record(s) reference this one; withholding the count from the refusal (the refusal itself stands)`, - { object: childName, error: (error as Error)?.message ?? String(error) }, - ); - return false; - } + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const readLogger = () => this.logger; + const host: ThisParameterType = { + find: (...a) => this.find(...a), + get logger() { return readLogger(); }, + }; + return CascadePlanningFacet.dependentCountIsDisclosable.call(host, childName, probeWhere, probedIds, fieldName, multiValued, id, context); } - /** - * Apply referential delete behavior for relations pointing AT this record, - * before it is removed. For every registered object with a `master_detail`, - * `lookup` or `tree` field, or a `user` field that declares a - * `deleteBehavior` ({@link statesDeleteSemantics}), referencing - * `object`, honor the field's `deleteBehavior`: - * - `cascade` → delete the dependent rows (recursively, so grandchildren - * are handled by each child's own delete), - * - `set_null` → clear the foreign key. On a MULTI-VALUED field - * ([#18408] {@link declaredMultiValued}, ⛔ not the raw - * flag) the FK is a SET, so "clear" means remove the - * deleted MEMBER and keep the rest; an emptied set on - * an optional field is stored as `null`, the same as one - * never set — the representation `FieldSchema` pins - * (`packages/spec/src/data/field.zod.ts`, the `multiple` - * doc block, ruling A on #29343, record 6591577985, - * amending #9447), - * - `restrict` → refuse the delete when dependents exist. - * `master_detail` defaults to `cascade` (the parent owns the child - * lifecycle); `lookup` and `tree` default to `set_null` — except a `set_null` default - * on a REQUIRED lookup escalates to `restrict` (you can't null a NOT NULL - * FK; restricting with a clear dependent-count message beats a misleading - * " is required" 400 from the child) — and on a `multiple: true` - * required lookup that escalation is decided per ROW after the dependents - * probe (#9688): a row that keeps another member takes the member removal, - * a row the removal would EMPTY keeps the refusal, and the refusal counts - * only those rows. A by-id delete runs it for its one record; a predicate - * delete (`multi: true`) runs it for every row it matched ([#26871], see - * `delete()`'s predicate arm). - * - * [#27004] It takes a ROOT SET: the by-id delete passes a set of one, the - * predicate arm the ids it matched. Every phase below reads a relation once - * per {@link REFERENCE_PROBE_BATCH} records of the set - * ({@link probeReferencingRowsOfSet}), not once per record, and judges and - * writes per record as before. One walk of one set: a matched row another - * matched row's cascade reaches is in the set like any other member, so it - * refuses nothing and is cleared of nothing (#22305), and the predicate - * arm's own `deleteMany` removes it. - * - * [#22305] Two phases, so the walk knows its own set before it judges - * anything: - * 1. {@link collectCascadeDeleteSet} — read-only. Every record the cascade - * deletes, transitively, across every CASCADING relation, the root - * included. - * 2. {@link walkReferencingRelations} — the walk above, unchanged in - * order, except that a `restrict` (authored, or the escalated - * required `set_null`) and a `set_null` write consider only rows - * OUTSIDE the set. A row in the set is about to be deleted by this same - * cascade: it cannot be left dangling, so it refuses nothing, and - * clearing its reference would be a write against a row about to go. - * [#26718] Between the two, {@link refuseRestrictsInsideOwnCascade}, also - * read-only: an AUTHORED `restrict` held by a member the cascade reaches - * only THROUGH the record that restrict protects (a child holding a - * master_detail and a restrict lookup to one parent; a grandchild under it) - * refuses the delete before the first write. The row is in the set, so phase - * 2 alone would let the cascade delete it. - * - * Before this, the outcome depended on registration order: with the - * contacts registered ahead of the contracts, deleting an account refused - * on a contract's required lookup to a contact, a contract the same delete - * was about to remove; the other order deleted. A refusal from a row outside - * the set is unchanged, and so is the unit of work: the set is collected - * inside the transaction that `delete()` opens for the cascade. - * - * A nested `delete()` the walk makes for a member joins the ambient set - * ({@link cascadeDeleteSets}); any other delete reaching here — a hook's, - * say — is the root of its own cascade, as before. - */ - private async cascadeDeleteRelations( + private cascadeDeleteRelations( object: string, ids: ReadonlyArray, context?: ExecutionContext, depth = 0, ): Promise { - const roots: Array = []; - const seen = new Set(); - for (const id of ids) { - if (id == null || seen.has(String(id))) continue; - seen.add(String(id)); - roots.push(id); - } - if (roots.length === 0 || depth >= ObjectQL.MAX_CASCADE_DEPTH) return; - // [#9002] The registry read is UNGUARDED, deliberately — and this is the - // statement that decides whether the cascade runs AT ALL. - // - // It used to sit behind a bare `catch { return; }`, which is the #8895 - // shape one layer up: #8895's `catch` invented "no dependents" for ONE - // relation whose probe could not run; this one invented "no relations" for - // EVERY relation at once, before the per-relation probe was ever reached — - // no `restrict` refusal, no `set_null`, no `cascade`, nothing logged, and - // the caller told the delete succeeded. - // - // #8895 ruled the family *discriminate or propagate*. Discrimination needs - // a benign failure class — there it was the unprovisioned child TABLE, - // which genuinely cannot hold a referencing row. Here there is none: an - // unreadable registry is never truthfully "no relations", so `propagate` - // is the whole answer and the `catch` has nothing left to do. - // - // No new error code and no new response field: whatever the registry read - // raised reaches the caller with its envelope intact, exactly as #8895's - // probe failure does. - const objects: ServiceObject[] = this._registry.getAllObjects(); - // [#22305] A member of a cascade already in progress joins that cascade's - // set. Any other delete is the root of its own cascade, and collects the - // set first: before one refusal is judged and before one row is written. - const inherited = this.cascadeDeleteSets.getStore(); - const joining = inherited ? roots.filter((id) => cascadeSetHas(inherited.members, object, id)) : []; - if (inherited && joining.length > 0) { - await this.walkReferencingRelations(object, joining, context, objects, inherited); - } - const fresh = joining.length > 0 ? roots.filter((id) => !cascadeSetHas(inherited!.members, object, id)) : roots; - if (fresh.length === 0) return; - const cascadeSet = await this.collectCascadeDeleteSet(object, fresh, context, objects); - // [#26718] Every restrict a member's own cascade would override is judged - // here, before the first write. - await this.refuseRestrictsInsideOwnCascade(cascadeSet, fresh.map((id) => ({ object, id })), context, objects); - await this.cascadeDeleteSets.run(cascadeSet, () => - this.walkReferencingRelations(object, fresh, context, objects, cascadeSet), - ); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const read_registry = () => this._registry; + const readCascadeDeleteSets = () => this.cascadeDeleteSets; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + get cascadeDeleteSets() { return readCascadeDeleteSets(); }, + collectCascadeDeleteSet: (...a) => this.collectCascadeDeleteSet(...a), + refuseRestrictsInsideOwnCascade: (...a) => this.refuseRestrictsInsideOwnCascade(...a), + walkReferencingRelations: (...a) => this.walkReferencingRelations(...a), + }; + return CascadePlanningFacet.cascadeDeleteRelations.call(host, object, ids, context, depth); } /** @@ -13889,43 +11704,19 @@ export class ObjectQL implements IObjectQLEngine { return collectCascadeDeleteSet.call(host, object, ids, context, objects, opts); } - /** - * Does `fieldName` on `child` point at `object` as a relation the cascade - * walk follows, and with which behaviour? `undefined` when it does not. The - * behaviour is the RESOLVED one, before the required-`set_null` escalation, - * which also reads the field's `multiple` and, on a multi-value field, the - * rows themselves. - * - * [#22305] Lifted out of the walk so both phases of - * {@link cascadeDeleteRelations} read relations one way: the set the first - * collects is the set the second deletes. - */ private cascadeRelationBehavior( object: string, child: ServiceObject, fieldName: string, fdef: any, ): string | undefined { - // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const host: ThisParameterType = { resolveObjectName: (...a) => this.resolveObjectName(...a), }; - return cascadeRelationBehavior.call(host, object, child, fieldName, fdef); + return CascadePlanningFacet.cascadeRelationBehavior.call(host, object, child, fieldName, fdef); } - /** - * [#12166, ruling constraint 3] File the elevation record for a probe of - * `childName` made while deleting `object`/`id` — once. One record per - * referenced OBJECT, not per relation: two lookup fields on the same child - * pointing at the same parent are one "the platform read object X as - * system" fact, and filing it twice would make the ledger's row count a - * function of the child's field layout. - * - * [#22305] The dedupe is the cascade's, not one call's: the collection walk - * and the deleting walk probe the same cascading relations, and between - * them file one record per deleted record per referenced object, as the - * single walk did. - */ private fileReferenceCheckElevation( set: CascadeDeleteSet, object: string, @@ -13934,10 +11725,11 @@ export class ObjectQL implements IObjectQLEngine { fieldName: string, context: ExecutionContext | undefined, ): void { - const key = JSON.stringify([object, String(id), childName]); - if (set.elevationsFiled.has(key)) return; - set.elevationsFiled.add(key); - this.recordReferenceCheckElevation(object, id, childName, fieldName, context); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const host: ThisParameterType = { + recordReferenceCheckElevation: (...a) => this.recordReferenceCheckElevation(...a), + }; + return CascadePlanningFacet.fileReferenceCheckElevation.call(host, set, object, id, childName, fieldName, context); } /** @@ -14057,421 +11849,58 @@ export class ObjectQL implements IObjectQLEngine { return dependents ?? []; } - /** - * [#27004] {@link referenceProbeFilter} for a SET of records: the rows that - * reference any of `ids`. A single-valued reference asks `$in`; a - * multi-valued one ORs each id's own limbs, the spelling the nested-relation - * lowering already sends for a multiple lookup. One id is that id's own - * filter, byte for byte. - */ private referenceProbeFilterOfSet( fieldName: string, fdef: { type?: unknown; multiple?: unknown }, ids: ReadonlyArray, ): Record { - // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). - const host: ThisParameterType = { + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const host: ThisParameterType = { referenceProbeFilter: (...a) => this.referenceProbeFilter(...a), }; - return referenceProbeFilterOfSet.call(host, fieldName, fdef, ids); + return CascadePlanningFacet.referenceProbeFilterOfSet.call(host, fieldName, fdef, ids); } - /** - * [#27004] The dependents probe ({@link probeReferencingRows}) for a SET of - * records of one object: the rows of `childName` referencing each of `ids`, - * keyed by `String(id)`. One read per {@link REFERENCE_PROBE_BATCH} ids, so a - * walk over a predicate delete's matched rows costs one probe per relation - * per batch rather than one per row; a batch of one is the by-id probe, - * filter and all. - * - * Each row is then given to the ids it references: its stored value on a - * single-valued field, its members on a multi-valued one (the #9362 exact - * reading, {@link storedReferenceIncludes}). A row of a single-valued field - * whose value names none of the batch (a driver comparing ids in a way - * `String()` does not) is never dropped: that batch is asked again id by - * id, which is the by-id answer. - */ - private async probeReferencingRowsOfSet( + private probeReferencingRowsOfSet( childName: string, fieldName: string, fdef: any, ids: ReadonlyArray, context: ExecutionContext | undefined, ): Promise> { - const byId = new Map(); - const multiValued = declaredMultiValued(fdef); - const batchSize = ObjectQL.REFERENCE_PROBE_BATCH; - for (let i = 0; i < ids.length; i += batchSize) { - const batch = ids.slice(i, i + batchSize); - const rows = await this.probeReferencingRows( - childName, fieldName, fdef, batch.length === 1 ? batch[0] : batch, - this.referenceProbeFilterOfSet(fieldName, fdef, batch), context, - ); - if (batch.length === 1) { - byId.set(String(batch[0]), rows); - continue; - } - const found = new Map(batch.map((id) => [String(id), []])); - let unattributed = false; - for (const row of rows) { - // The members' reading is `storedReferenceIncludes`'s, element by - // element; a single value is compared as one. - const stored = row?.[fieldName]; - const values: unknown[] = multiValued && Array.isArray(stored) ? stored : [stored]; - const held = new Set(); - for (const v of values) { - if (v == null || typeof v === 'object') continue; - if (found.has(String(v))) held.add(String(v)); - } - if (held.size === 0) { unattributed = true; break; } - for (const key of held) found.get(key)!.push(row); - } - if (unattributed) { - for (const id of batch) { - found.set(String(id), await this.probeReferencingRows( - childName, fieldName, fdef, id, this.referenceProbeFilter(fieldName, fdef, id), context, - )); - } - } - for (const [key, held] of found) byId.set(key, held); - } - return byId; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const host: ThisParameterType = { + probeReferencingRows: (...a) => this.probeReferencingRows(...a), + referenceProbeFilter: (...a) => this.referenceProbeFilter(...a), + referenceProbeFilterOfSet: (...a) => this.referenceProbeFilterOfSet(...a), + }; + return CascadePlanningFacet.probeReferencingRowsOfSet.call(host, childName, fieldName, fdef, ids, context); } - /** - * [#22305] Phase 2 of {@link cascadeDeleteRelations}, for one record of - * `cascadeSet`: refuse, clear or cascade each relation that points at it. - * Unchanged from the single walk it was, but for three readings of the set: - * a `restrict` and a `set_null` consider only rows OUTSIDE it, and a row the - * walk has already entered is not cascaded into again. - */ - private async walkReferencingRelations( + private walkReferencingRelations( object: string, ids: ReadonlyArray, context: ExecutionContext | undefined, objects: ServiceObject[], cascadeSet: CascadeDeleteSet, ): Promise { - // Entered before the first nested `delete()` can come back here: every - // record of the set, so one member's cascade never deletes another. - for (const id of ids) cascadeSetAdd(cascadeSet.entered, object, id); - // [#24948] An organization that still owns rows is refused here, whole, - // before the first relation below is cascaded or cleared. - if (object === ORGANIZATION_OBJECT) { - for (const id of ids) await this.refuseOrganizationDeleteWhileOwned(id, context, objects, cascadeSet); - } - for (const child of objects) { - const childName = (child as any)?.name as string | undefined; - const fields = (child as any)?.fields as Record | undefined; - if (!childName || !fields) continue; - for (const [fieldName, fdef] of Object.entries(fields)) { - const resolvedBehavior = this.cascadeRelationBehavior(object, child, fieldName, fdef); - if (resolvedBehavior === undefined) continue; - // [#24948] Judged by `refuseOrganizationDeleteWhileOwned` above, which - // found no row outside the cascade set: there is nothing to clear, and - // clearing is not on offer for this column anyway. - if (this.isTenantColumnSetNull(object, child, fieldName, resolvedBehavior)) continue; - // [#24948] A reference the organization's delete keeps as it is, - // naming the deleted organization: never cleared, never counted. - if (object === ORGANIZATION_OBJECT && resolvedBehavior === 'set_null' && - ObjectQL.keepsOrganizationReference(child, fieldName)) continue; - let relationBehavior: string = resolvedBehavior; - - // [#9689] (maintainer ruling 2026-08-19, Q3 = B): the judgement the - // #9625 comment in `cascadeRelationBehavior` deferred is now taken — `FieldSchema` REJECTS an - // authored `deleteBehavior: 'set_null'` on a `master_detail` at parse - // time, so the value is meaningful again: one that still reaches this - // site came in around the parse seam (a raw `registerObject`, or a - // stored/artifact row written before the tightening — the two - // populations parse-time rejection measurably cannot catch, since the - // engine registers raw objects and never re-parses). The coercion - // itself stays: this delete is about to CASCADE children whose - // declaration asked for them to be kept, and that divergence must be - // loud and attributable, not silent. Sanctioned logger shape per - // PR #9750: reach for `error`, fall back to `warn` — NEVER an optional - // call like `logger.error?.()`, which emits nothing against a sink - // with no `error`. Caveat, measured (#4447): a built app artifact - // materializes FieldSchema defaults, so an artifact-loaded BARE - // master_detail also carries `set_null` and logs here — that residual - // imprecision is the materialized-default defect tracked as #9784, not - // a reason to soften this log. - if (fdef.type === 'master_detail' && fdef.deleteBehavior === 'set_null') { - for (const id of ids) { - const msg = - `[cascade-delete] ${childName}.${fieldName} declares deleteBehavior: 'set_null' on a ` + - `master_detail referencing '${object}' — that value is NOT honored on master_detail: the ` + - `delete of ${object}/${String(id)} CASCADES its referencing child rows, the opposite of what ` + - `the declaration asks (children kept). FieldSchema now rejects this combination at parse time; ` + - `this row reached the engine around the parse seam (raw registration, or metadata stored ` + - `before the tightening). Re-declare the field: 'restrict' refuses the parent delete while ` + - `children exist (no data loss), 'cascade' (or omitting the key) accepts the cascade ` + - `deliberately, or make it a lookup if children must survive the parent.`; - if (typeof this.logger.error === 'function') this.logger.error(msg); - else this.logger.warn(msg); - } - } - - // A REQUIRED foreign key cannot be nulled — set_null would issue an - // UPDATE clearing the FK, which the child's required-field validator - // rejects with a misleading " is required" 400 (the field isn't - // even on the object being deleted). That's a contradiction, not the - // author's intent: a required FK means the child can't exist detached, - // so deleting the parent must be RESTRICTed (SQL's default for a - // NOT NULL FK). Authors who want the children gone set - // deleteBehavior:'cascade' explicitly. - // - // [#9625] The test reads the RESOLVED `behavior` (`cascadeRelationBehavior`), - // which no longer records how the value got there. So it escalates - // BOTH the defaulted set_null and one the author wrote out as - // `deleteBehavior: 'set_null'` — the two are indistinguishable here by - // construction, and both are measured escalating - // (`engine-cascade-delete.test.ts`). Only `cascade` and `restrict` are - // untouched. The predecessor of this comment claimed the escalation - // applied to the *defaulted* set_null alone; that was true of - // `cascade`/`restrict` and false of an explicit `set_null`, which is - // the half-truth #9625 measured. Say it the way the code behaves: the - // escalation is a property of the RESOLVED behavior plus `required`, - // not of what the author typed. - // - // [#9688] The rationale above is also what BOUNDS the escalation, and - // on a `multiple: true` field it does not reach every row. The - // set_null limb there does not clear the slot: since #9438 it removes - // the deleted MEMBER and writes the remainder. So "a cleared required - // FK trips the child's validator" is true of exactly one case — the - // row whose set the removal would EMPTY, since `[]` is what the #9447 - // ruling (maintainer, 2026-08-18) says violates `required` on a - // multi-value field, and what the record validator has rejected since - // #9476. A row that still holds another live member is written a - // NON-EMPTY set, which `required` accepts and no validator objects - // to; refusing its parent's delete was broader than the contract, and - // refused it citing a failure that could not have happened. - // - // Emptiness is a property of a ROW, not of the field — and no row has - // been read at this point. So the multi-value half of the escalation - // is DEFERRED to just after the dependents probe (search - // `requiredSetNull` below), where the rows are known and exactly - // narrowed. The single-valued half stays here and is unchanged: - // clearing a scalar FK always writes `null`, so its premise needs no - // row to hold. - // - // The deferral changes WHEN the decision is made, never who it is - // reported to: a row that does keep the refusal still refuses THIS - // relation before its own set_null write runs, so the caller is told - // `DELETE_RESTRICTED` about the record it asked to delete, instead of - // the child's own `required` 400 naming a field that is not on that - // record's object at all. - // - // [#9362] `multiValued` is declared here rather than at the probe - // because the probe's filter spelling, the set_null write below and - // — since #9688 — this escalation all turn on it. - // [#18408] Was `fdef.multiple === true`. See {@link declaredMultiValued}. - const multiValued = declaredMultiValued(fdef); - const requiredSetNull = relationBehavior === 'set_null' && fdef.required === true; - if (requiredSetNull && !multiValued) { - relationBehavior = 'restrict'; - } - - // [#12166, ruling constraint 3] File the elevation BEFORE it happens, - // and file BOTH halves — triggered-by = the deleting operator, - // executed-as = system (the Salesforce/Dataverse ledger shape). - // - // Before the probe rather than after, because the fact being recorded - // is that the platform read this object under an identity the caller - // does not hold. That is true whether the probe then finds rows, finds - // none, refuses the delete, or fails — and a record written only on the - // success path would be missing exactly the runs an auditor goes - // looking for. - for (const id of ids) this.fileReferenceCheckElevation(cascadeSet, object, id, childName, fieldName, context); - - // [#27004] One probe per batch of the set; each record then takes its - // own rows, in the set's order. - const referencing = await this.probeReferencingRowsOfSet(childName, fieldName, fdef, ids, context); - const clearedKey = cascadeMemberKey(childName, fieldName); - let cleared = cascadeSet.cleared.get(clearedKey); - if (!cleared) { cleared = new Set(); cascadeSet.cleared.set(clearedKey, cleared); } - for (const id of ids) { - let behavior: string = relationBehavior; - let dependents = referencing.get(String(id)) ?? []; - if (dependents.length === 0) continue; - - // [#12166] The one record's probe filter: the DISCLOSURE re-probe in - // `deleteRestrictedError` asks the identical question under the - // caller's own identity. [#27004] The batched probe above is built from - // this same filter, id by id ({@link referenceProbeFilterOfSet}), and - // narrowed by the same reading, so the two cannot disagree about which - // rows reference this record. - const probeWhere = this.referenceProbeFilter(fieldName, fdef, id); - - // [#12166] The elevated probe's row IDENTITY, captured here — after the - // multi-value narrowing and BEFORE the `requiredSetNull && multiValued` - // narrowing below reduces `dependents` to the emptied subset. This is - // the set the disclosure decision compares against, and the stage - // matters: the emptied subset is computed from these rows' own stored - // values, so a caller who can see exactly these rows can derive the - // emptied count too, while a caller who cannot see them can derive - // neither. Comparing at the later stage would leak the difference. - const probedIds: readonly string[] = dependents.map((r: any) => String(r?.id)); - - // [#22305] A row the cascade itself deletes refuses nothing and is - // cleared of nothing: the triage ruling on #22305 is that a child in the - // cascade set never restricts a sibling in the same set. Such a row is - // not left dangling, because this same unit of work removes it, and - // clearing its reference would be a write against a row about to go. - // (The one cascade that is not one unit of work is - // `planCascadeAtomicity`'s `'split'`: there a later refusal can strand - // such a row, the partial outcome `warnCascadeNotAtomic` already - // declares for every row that path deletes.) So `restrict` (authored, - // or either half of the required escalation) and `set_null` judge only - // the rows OUTSIDE the set, and a relation whose rows are all inside it - // is done. `cascade` keeps every row: its rows ARE the set, and the - // walk deletes them below. - // - // [#26718] …except an AUTHORED restrict held by a row the cascade reaches - // only through the record that restrict protects: that one was judged - // before the cascade wrote anything ({@link refuseRestrictsInsideOwnCascade}), - // so a row in the set reaching here refuses nothing. - // - // After `probedIds`, on purpose: the disclosure check still compares - // the caller's own view against every row the elevated probe read, so a - // count is disclosed only to a caller who can see the whole relation, - // as before; what it counts is the rows that actually refuse. - if (behavior !== 'cascade') { - dependents = dependents.filter((row) => !cascadeSetHas(cascadeSet.members, childName, row?.id)); - if (dependents.length === 0) continue; - } - - // [#9688] The deferred half of the required escalation, decided per - // ROW now that the rows are read and exactly narrowed — every row - // here genuinely holds `id`, so its remainder is its set minus that - // one member. - // - // The emptiness question is asked through - // `remainderAfterMemberRemoval`, which is the SAME computation the - // set_null write below performs. One function, two call sites, so the - // judgement cannot predict a shape the write would not produce: a - // predicate that computed the remainder even slightly differently - // could clear the delete and then let the write land the very `[]` - // this judgement exists to prevent. - // - // ANY row that would be emptied refuses the WHOLE delete — a delete - // either happens or it does not, and a partial cascade is not on - // offer. `dependents` is reduced to those rows first, because every - // number the refusal reports reads it: `dependentCount`, the - // localized message's `count`, and the developerMessage. Counting the - // rows this delete no longer refuses over is the second defect the - // card names, and it is fixed by the same statement. - if (requiredSetNull && multiValued) { - const emptied = dependents.filter( - (row) => ObjectQL.remainderAfterMemberRemoval(row?.[fieldName], id, cleared).length === 0, - ); - if (emptied.length > 0) { - behavior = 'restrict'; - dependents = emptied; - } - } - - if (behavior === 'restrict') { - throw await this.deleteRestrictedError({ - object, id, child, childName, fieldName, fdef, dependents, probeWhere, probedIds, multiValued, - objects, context, - }); - } - - for (const dep of dependents) { - const depId = dep?.id; - if (depId == null) continue; - if (behavior === 'cascade') { - // [#22305] Entered already: an ancestor of this delete, still in - // progress up the stack (a cycle in the data), or a row an earlier - // branch of this cascade has deleted (a row reached by two paths). - // Either way this cascade removes it once, and re-entering would - // loop forever on the first and answer 404 on the second. - if (cascadeSetHas(cascadeSet.entered, childName, depId)) continue; - // Recurse via the public delete so the child's own cascade, - // hooks and events fire. - // - // [#23221] Inside a token naming exactly this child, which that - // delete claims and projects onto the DECLARED - // `HookContext.referentialCascadeDelete` (see - // claimReferentialCascadeDelete): on the real request path the - // caller's context reaches the child unchanged, so without it a - // freeze guard cannot tell this delete from a hand delete of the - // same row. The context is passed as before, by reference and - // unmarked — see `cascadeIssuedDeletes` for why the marker is not - // a context key. A grandchild is issued by the child's own walk, - // inside a token of its own. - await this.cascadeIssuedDeletes.run( - { object: childName, id: String(depId), claimed: false }, - () => this.delete(childName, { where: { id: depId }, context } as any), - ); - } else { - // [#3023] Clear the FK as an engine-internal referential-integrity - // write, tagged so plugin-security's ownership-anchor guard (#3004) - // treats an `owner_id = null` cascade as integrity maintenance, not - // a user-initiated disown — otherwise deleting the referenced record - // trips the transfer guard and aborts the cascade mid-way. Marker - // rides a server-DERIVED context (set here, never from client input - // — same trust model as `__expandRead`), so it cannot be forged from - // a request to bypass the guard on an ordinary write. - // - // [commit 34ce8e7db] This same marker is what `update()`'s hook-context - // assembly projects onto the DECLARED `HookContext. - // referentialFieldClear` (see buildReferentialFieldClear), so an - // app guard can recognise the cleanup without reading an - // operation-private key — on the real request path the inherited - // caller envelope makes the cleanup otherwise indistinguishable - // from a hand-clear of the same lookup. - // - // [#20006] `__referentialFieldClearCause` rides beside it: the - // delete this cleanup serves and the reference it clears, which a - // traversing rule's refusal of the cleanup names (see - // `resolvePredicateRelated`). It authorizes nothing — it is read - // only where the marker already holds, and only to word a refusal — - // and the referencing row's id in it never reaches that text. - const referentialCtx = { - ...(context ?? {}), - __referentialFieldClear: true, - __referentialFieldClearCause: { - object, id: String(id), referencingObject: childName, referencingId: String(depId), field: fieldName, - }, - } as ExecutionContext; - if (multiValued) { - // The FK is a SET, so `set_null` clears the deleted MEMBER, not - // the slot: filter the stored array and write what remains. - // Nulling the slot instead discarded every other live reference - // (measured: `["acc_a","acc_b"]` re-read as `null` after - // `acc_a` was deleted). - // - // The residual shape when the last member goes is not this - // engine's call to make: `FieldSchema` pins it — - // `packages/spec/src/data/field.zod.ts`, the `multiple` and - // `required` doc blocks (ruling A on #29343, record - // 6591577985, amending #9447) — and this write consumes that - // contract: it hands the write door the `[]` that - // `Array.prototype.filter` yields, and the door stores an - // emptied optional set as `null` (`blankAsNullColumns`, reach - // `'emptyList'`), the same as one never set. - // - // [#9688] The remainder is computed by - // `remainderAfterMemberRemoval`, which the per-row required - // judgement above calls on the same row: this write is only - // reached for a row that judgement measured as keeping at least - // one other member, and the two must agree by construction. - // The reading it applies — `String(v) !== String(id)`, and an - // off-shape bare scalar normalized to the array spelling — is - // documented there. - const next = ObjectQL.remainderAfterMemberRemoval(dep?.[fieldName], id, cleared); - await this.update(childName, { id: depId, [fieldName]: next }, { context: referentialCtx } as any); - } else { - await this.update(childName, { id: depId, [fieldName]: null }, { context: referentialCtx } as any); - } - } - } - // [#27004] Every row outside the set has had `id` cleared from it. - cleared.add(String(id)); - } - } - } + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const readCascadeIssuedDeletes = () => this.cascadeIssuedDeletes; + const readLogger = () => this.logger; + const host: ThisParameterType = { + get cascadeIssuedDeletes() { return readCascadeIssuedDeletes(); }, + cascadeRelationBehavior: (...a) => this.cascadeRelationBehavior(...a), + delete: (...a) => this.delete(...a), + deleteRestrictedError: (...a) => this.deleteRestrictedError(...a), + fileReferenceCheckElevation: (...a) => this.fileReferenceCheckElevation(...a), + isTenantColumnSetNull: (...a) => this.isTenantColumnSetNull(...a), + get logger() { return readLogger(); }, + probeReferencingRowsOfSet: (...a) => this.probeReferencingRowsOfSet(...a), + referenceProbeFilter: (...a) => this.referenceProbeFilter(...a), + refuseOrganizationDeleteWhileOwned: (...a) => this.refuseOrganizationDeleteWhileOwned(...a), + update: (...a) => this.update(...a), + }; + return CascadePlanningFacet.walkReferencingRelations.call(host, object, ids, context, objects, cascadeSet); } /** @@ -14553,91 +11982,27 @@ export class ObjectQL implements IObjectQLEngine { return deleteRestrictedError.call(host, args); } - /** - * [#24948] Is `fieldName` on `child` the TENANT column of `child`, pointing at - * the organization being deleted with the `set_null` behaviour? - * - * The tenant column is the one `resolveTenantFieldName` names: the column - * the engine stamps and scopes by, whether the registry injected it - * (`TENANT_SCOPE_FIELD_DEF`) or the object declared its own, as the identity - * tables do. Every such lookup resolves to `set_null` unless its author wrote - * something else, and ADR-0131 D1 leaves NULL nothing to mean: a cleared - * tenant column is an orphan, and on a store whose column is already NOT - * NULL the clear is a constraint failure halfway through a delete. So the - * engine never clears one. {@link refuseOrganizationDeleteWhileOwned} refuses - * the delete instead. An authored `cascade` or `restrict` keeps its own - * meaning. - */ private isTenantColumnSetNull( object: string, child: ServiceObject, fieldName: string, behavior: string | undefined, ): boolean { - // [#29986 codemod] The body moved to `engine-cascade-planning.ts`. - return isTenantColumnSetNull(object, child, fieldName, behavior); + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.isTenantColumnSetNull(object, child, fieldName, behavior); } - /** - * [#24948] Do `child`'s rows outlive the organization they name? Read from - * the declaration the object already carries, never from a list of names: - * - * - `managedBy: 'append-only'`, a ledger (the audit bucket: `sys_activity`, - * `sys_audit_log`, `sys_email` and their siblings). Such a row is history: - * it ages out on its own retention, or is the delivery log of something - * that already happened. The organization delete itself writes - * `sys_activity` rows, so a ledger that counted would refuse every - * organization delete, the empty ones included (seat decision 6305179819). - * - `managedBy: 'engine-owned'`, a row the platform writes as a consequence - * of records or activity (approval state, automation runs, notification - * and HTTP deliveries, mirrored mail, record shares, import jobs, - * settings values), which an administrator cannot delete over the data - * API. Counting it would make any organization that has been used - * undeletable through every door (seat decision 6307135388). - * - * Neither refusal counts such a row. It is kept, and so is every reference it - * holds to the organization: it goes on naming the deleted organization, - * never NULL (ADR-0131 D1) and never deleted (ADR-0131 §8), and under tenant - * scoping the row reads as no organization's. - */ private static outlivesOrganization(child: ServiceObject): boolean { - const managedBy = (child as { managedBy?: unknown })?.managedBy; - return managedBy === 'append-only' || managedBy === 'engine-owned'; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.outlivesOrganization(child); } - /** - * [#24948] Does an organization delete keep `child.fieldName`, a `set_null` - * lookup into `sys_organization` that is not `child`'s tenant column, as it - * is? Two kinds of reference, and only these (seat decisions 6305179819, - * 6305996369 and 6307135388): - * - * - every such lookup on an object whose rows outlive the organization - * ({@link outlivesOrganization}): `sys_audit_log.tenant_id` is the audit - * log's attribution, not its tenant column, and history outlives what it - * audits; - * - an attribution column named `organization_id` on an object whose - * tenancy is disabled (`resolveTenantFieldName` answers null): neither - * counted nor cleared. No object in this tree declares one today. - * - * Everything else keeps its `set_null`: a pointer such as a session's or an - * API key's `active_organization_id`, or a parent-organization lookup, is - * cleared exactly as before. - */ private static keepsOrganizationReference(child: ServiceObject, fieldName: string): boolean { - if (ObjectQL.outlivesOrganization(child)) return true; - return fieldName === SystemFieldName.ORGANIZATION_ID && resolveTenantFieldName(child) === null; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.keepsOrganizationReference(child, fieldName); } - /** - * [#24948] How many rows of `childName` name the organization `id` through - * `fieldName`, leaving out the rows the cascade itself deletes. A system - * `count`, never a row read, for every table the cascade deletes no row of, - * which is every table but the cascading children of the organization (none - * on the platform). Only for such a table are the rows probed - * ({@link probeReferencingRows}), so the cascade's own members can be left - * out exactly. A table never provisioned holds none. - */ - private async countReferencingRowsOutside( + private countReferencingRowsOutside( childName: string, fieldName: string, fdef: any, @@ -14645,121 +12010,50 @@ export class ObjectQL implements IObjectQLEngine { cascadeSet: CascadeDeleteSet, context: ExecutionContext | undefined, ): Promise<{ where: Record; count: number }> { - const where = this.referenceProbeFilter(fieldName, fdef, id); - if ((cascadeSet.members.get(childName)?.size ?? 0) > 0) { - const probed = await this.probeReferencingRows(childName, fieldName, fdef, id, where, context); - return { where, count: probed.filter((row) => !cascadeSetHas(cascadeSet.members, childName, row?.id)).length }; - } - try { - return { where, count: await this.count(childName, { where, context: ObjectQL.organizationCountContext(context) }) }; - } catch (error) { - if (isMissingTableError(error, childName)) return { where, count: 0 }; - throw error; - } + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const host: ThisParameterType = { + count: (...a) => this.count(...a), + probeReferencingRows: (...a) => this.probeReferencingRows(...a), + referenceProbeFilter: (...a) => this.referenceProbeFilter(...a), + }; + return CascadePlanningFacet.countReferencingRowsOutside.call(host, childName, fieldName, fdef, id, cascadeSet, context); } - /** - * [#24948] The context an organization's row counts run under: system - * identity, and nothing of the caller's scope. A `tenantId` carried in from - * a caller acting in another organization would scope the count to THAT - * organization and read 0 rows naming this one, so the refusal would pass on - * an undercount. Only the request's transaction and trace are kept: the - * count must see the request's own uncommitted writes. The ordinary - * reference probes keep the caller's scope ({@link referenceCheckContext}). - */ private static organizationCountContext(context?: ExecutionContext): ExecutionContext { - return { - isSystem: true, - ...(context?.transaction !== undefined ? { transaction: context.transaction } : {}), - ...(context?.traceId !== undefined ? { traceId: context.traceId } : {}), - } as ExecutionContext; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it. + return CascadePlanningFacet.organizationCountContext(context); } - /** - * [#24948] May the refusal disclose `count` to the caller? Only when the - * caller's own identity counts the same rows: the elevated count read rows - * the caller may not see. (A system caller's own count is the elevated one.) - */ private organizationCountIsDisclosable( childName: string, where: Record, count: number, context: ExecutionContext | undefined, ): Promise { - // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). const readLogger = () => this.logger; - const host: ThisParameterType = { + const host: ThisParameterType = { count: (...a) => this.count(...a), get logger() { return readLogger(); }, }; - return organizationCountIsDisclosable.call(host, childName, where, count, context); + return CascadePlanningFacet.organizationCountIsDisclosable.call(host, childName, where, count, context); } - /** - * [#24948] The engine half of organization delete: refuse while any row - * outside the cascade set still names the organization in its tenant column, - * a row that outlives the organization excepted ({@link outlivesOrganization}). - * - * Runs before the walk touches the first relation, and reads every tenant - * relation before it throws, so the one refusal names every table that holds - * rows, each with its count. It is the safety net under plugin-auth's own - * refusal, which answers the same question before better-auth deletes the - * organization's members: whatever path reaches a `sys_organization` delete, - * no row is orphaned to NULL, on a store born with a nullable tenant column - * or anywhere else. - * - * Each relation is COUNTED ({@link countReferencingRowsOutside}: system - * identity, its elevation filed first, a missing table read as no rows), - * never read row by row, so a large organization costs one `count` per - * table. A count reaches the refusal only when the caller's own identity - * counts the same ({@link organizationCountIsDisclosable}); otherwise the - * table is named without one. The wire code is the standard catalog's `DELETE_RESTRICTED`, - * because that is the condition: a delete refused while dependents exist. - */ - private async refuseOrganizationDeleteWhileOwned( + private refuseOrganizationDeleteWhileOwned( id: string | number, context: ExecutionContext | undefined, objects: ServiceObject[], cascadeSet: CascadeDeleteSet, ): Promise { - const owned: Array<{ object: string; count?: number }> = []; - for (const child of objects) { - const childName = (child as any)?.name as string | undefined; - const fields = (child as any)?.fields as Record | undefined; - // An append-only or engine-owned object's rows outlive the organization: not counted, kept as they are. - if (!childName || !fields || ObjectQL.outlivesOrganization(child)) continue; - for (const [fieldName, fdef] of Object.entries(fields)) { - const behavior = this.cascadeRelationBehavior(ORGANIZATION_OBJECT, child, fieldName, fdef); - if (!this.isTenantColumnSetNull(ORGANIZATION_OBJECT, child, fieldName, behavior)) continue; - this.fileReferenceCheckElevation(cascadeSet, ORGANIZATION_OBJECT, id, childName, fieldName, context); - const { where, count } = await this.countReferencingRowsOutside(childName, fieldName, fdef, id, cascadeSet, context); - if (count === 0) continue; - const disclose = await this.organizationCountIsDisclosable(childName, where, count, context); - owned.push(disclose ? { object: childName, count } : { object: childName }); - } - } - if (owned.length === 0) return; - const named = owned - .map((o) => (o.count === undefined ? o.object : `${o.object} (${o.count})`)) - .join(', '); - const err: any = new Error( - `Cannot delete this organization: it still owns records in ${named}. ` + - 'Delete or move those records first, then delete the organization.', - ); - err.developerMessage = - `Cannot delete ${ORGANIZATION_OBJECT} (${id}): rows still reference it through their tenant column ` + - `in ${named}. The engine never clears a tenant column (ADR-0131 D1), so the delete is refused ` + - 'until those rows are deleted or moved to another organization.'; - err.code = 'DELETE_RESTRICTED'; - err.status = 409; - err.httpStatus = 409; - err.object = ORGANIZATION_OBJECT; - // The first table in the `DELETE_RESTRICTED` envelope's own two keys, and - // every table in `dependents`. - err.dependentObject = owned[0].object; - if (owned[0].count !== undefined) err.dependentCount = owned[0].count; - err.dependents = owned; - throw err; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts`, its docblock with it (see `CascadePlanningHost`). + const host: ThisParameterType = { + cascadeRelationBehavior: (...a) => this.cascadeRelationBehavior(...a), + countReferencingRowsOutside: (...a) => this.countReferencingRowsOutside(...a), + fileReferenceCheckElevation: (...a) => this.fileReferenceCheckElevation(...a), + isTenantColumnSetNull: (...a) => this.isTenantColumnSetNull(...a), + organizationCountIsDisclosable: (...a) => this.organizationCountIsDisclosable(...a), + }; + return CascadePlanningFacet.refuseOrganizationDeleteWhileOwned.call(host, id, context, objects, cascadeSet); } /** @@ -14780,33 +12074,19 @@ export class ObjectQL implements IObjectQLEngine { * itself. An append-only or engine-owned object is never counted ({@link * outlivesOrganization}), the same set the engine's own refusal leaves out. */ - async organizationOwnedRowCounts( + organizationOwnedRowCounts( organizationId: string, options?: { exclude?: Iterable; context?: ExecutionContext }, ): Promise> { - const exclude = new Set(options?.exclude ?? []); - const counts: Array<{ object: string; count: number }> = []; - for (const child of this._registry.getAllObjects()) { - const childName = (child as any)?.name as string | undefined; - const fields = (child as any)?.fields as Record | undefined; - if (!childName || !fields || exclude.has(childName) || ObjectQL.outlivesOrganization(child)) continue; - const tenantField = resolveTenantFieldName(child); - if (!tenantField || !fields[tenantField]) continue; - const fdef = fields[tenantField]; - if (this.cascadeRelationBehavior(ORGANIZATION_OBJECT, child, tenantField, fdef) === undefined) continue; - let count: number; - try { - count = await this.count(childName, { - where: this.referenceProbeFilter(tenantField, fdef, organizationId), - context: ObjectQL.organizationCountContext(options?.context), - }); - } catch (error) { - if (isMissingTableError(error, childName)) continue; - throw error; - } - if (count > 0) counts.push({ object: childName, count }); - } - return counts; + // [#29986 codemod] The body moved to `engine-cascade-planning.ts` (see `CascadePlanningHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + cascadeRelationBehavior: (...a) => this.cascadeRelationBehavior(...a), + count: (...a) => this.count(...a), + referenceProbeFilter: (...a) => this.referenceProbeFilter(...a), + }; + return CascadePlanningFacet.organizationOwnedRowCounts.call(host, organizationId, options); } async delete(object: string, options?: EngineDeleteOptions): Promise { @@ -15531,7 +12811,12 @@ export class ObjectQL implements IObjectQLEngine { query: InternalFieldEvaluationQuery, options?: InternalFieldEvaluationOptions, ): void { - assertNoInternalFieldEvaluated(object, query, this.relatedSchemaOf, options); + // [#29986 codemod] The body moved to `engine-delete-and-aggregate.ts` (see `DeleteAndAggregateHost`). + const readRelatedSchemaOf = () => this.relatedSchemaOf; + const host: ThisParameterType = { + get relatedSchemaOf() { return readRelatedSchemaOf(); }, + }; + return DeleteAndAggregateFacet.assertNoInternalFieldEvaluated.call(host, object, query, options); } /** @@ -15589,7 +12874,7 @@ export class ObjectQL implements IObjectQLEngine { const read_registry = () => this._registry; const readLogger = () => this.logger; const readRelatedSchemaOf = () => this.relatedSchemaOf; - const host: ThisParameterType = { + const host: ThisParameterType = { get _registry() { return read_registry(); }, buildDriverOptions: (...a) => this.buildDriverOptions(...a), composeReadableIdsAtBoundary: (...a) => this.composeReadableIdsAtBoundary(...a), @@ -15602,7 +12887,7 @@ export class ObjectQL implements IObjectQLEngine { resolveObjectName: (...a) => this.resolveObjectName(...a), resolveWhereTokens: (...a) => this.resolveWhereTokens(...a), }; - return aggregate.call(host, object, query, options); + return DeleteAndAggregateFacet.aggregate.call(host, object, query, options); } /** @@ -15675,147 +12960,61 @@ export class ObjectQL implements IObjectQLEngine { * `convertLead`, which creates an account + contact + opportunity + flips * the lead in a single unit of work). */ - async transaction( + transaction( callback: (trxCtx: any, info: EngineTransactionInfo) => Promise, baseContext?: any, opts?: EngineTransactionOptions, ): Promise { - // ADR-0067 D2 — JOIN an already-open ambient transaction instead of - // opening a nested driver transaction. A nested begin would acquire a - // second connection (a deadlock on single-connection pools like the - // SQLite knex pool) and would NOT be covered by the outer rollback — - // exactly the half-landing this join prevents: an outer batch - // transaction (e.g. `publishPackageDrafts`) must own the one-and-only - // commit/rollback for every write made through nested helpers (the - // sys-metadata repository's `withTxn`, hook-driven writes, …). - const ambient = this.txStore.getStore(); - if (ambient?.transaction) { - // JOINED, not owned: some outer caller decides commit vs rollback (#5696). - return callback( - { ...(baseContext ?? {}), transaction: ambient.transaction }, - { owned: false }, - ); - } - const driver = this.defaultDriver ? this.drivers.get(this.defaultDriver) : undefined; - const drv = driver as any; - // [#18063] The gate is the DECLARATION, not bare method presence. A - // subclass inherits `beginTransaction` from a base whose transport has - // transactions while its own has none — it cannot opt out of a door it did - // not open, so presence alone routed it down the transactional path and the - // handle it produced covered nothing. `driverSupportsTransactions` is the - // one definition (`@objectstack/spec`), shared with the ScopedContext trio - // below so the engine's transaction entrances cannot drift apart. - if (!driverSupportsTransactions(drv)) { - const datasource = this.defaultDriver ?? drv?.name; - if (opts?.require === true) { - // Fail CLOSED (#5696 point 1): the caller declared it cannot tolerate - // running without a rollback, so refuse BEFORE the callback writes - // anything rather than degrade behind a warning it may never read. - // Generalizes `batchData`'s atomic gate (ADR-0119 D4). - throw new TransactionUnsupportedError(datasource ?? ''); - } - // Declared degrade (ADR-0119 D1) — behaviour unchanged, but no longer - // mute: the caller asked for atomicity and is not getting it (#4619). - this.warnTransactionUnsupported(datasource, drv?.supports?.transactionsUnsupported === true); - // `owned: false` — honest: there is no transaction here to own, and no - // rollback the callback may promise on the strength of it. - return callback(baseContext, { owned: false }); - } - const trx = await drv.beginTransaction(); - const trxCtx = { ...(baseContext ?? {}), transaction: trx }; - try { - // Run the callback inside the ambient transaction store so internal - // queries during writes reuse this transaction's connection (ADR-0034), - // CLOSED once the callback settles, before the commit (#22782, #22855). - const result = await this.runOpenedUnit(trx, driver!, () => callback(trxCtx, { owned: true })); - if (drv.commit) await drv.commit(trx); - else if (drv.commitTransaction) await drv.commitTransaction(trx); - return result; - } catch (err) { - let rollbackFailed = false; - try { - if (drv.rollback) await drv.rollback(trx); - else if (drv.rollbackTransaction) await drv.rollbackTransaction(trx); - } catch { - // swallow rollback failures so the original error surfaces - rollbackFailed = true; - } - // [#21274] A boundary outside the middleware seam: a commit failure, or - // a callback that reached the driver through the transaction handle. - // An error that already crossed an engine boundary inside the callback - // comes back as the same reference. A stale-roll-up refusal ("stored") - // is no longer true once this rollback has run: `rolledBackRefusal`. - // [#29421] Nor is "rolled back" once the rollback threw: - // `rollbackFailedRefusal`. - throw redactPropagatedDriverFault(rollbackFailed ? rollbackFailedRefusal(err) : rolledBackRefusal(err)); - } + // [#29986 codemod] The body moved to `engine-transactions-section.ts` (see `TransactionsSectionHost`). + const readDefaultDriver = () => this.defaultDriver; + const readDrivers = () => this.drivers; + const readTxStore = () => this.txStore; + const host: ThisParameterType = { + get defaultDriver() { return readDefaultDriver(); }, + get drivers() { return readDrivers(); }, + runOpenedUnit: (...a) => this.runOpenedUnit(...a), + get txStore() { return readTxStore(); }, + warnTransactionUnsupported: (...a) => this.warnTransactionUnsupported(...a), + }; + return (TransactionsSectionFacet.transaction).call(host, callback, baseContext, opts); } - /** - * [#22782, #22855] Run `body`, the callback of a unit just opened on `owner` - * with handle `trx`. This is the ONE place both transaction surfaces - * ({@link transaction} and `ScopedContext.transaction`) publish a unit and - * close it again. - * - * Published: the entry goes into {@link txStore} (ADR-0034) for the body's - * lifetime, carrying its owner (#4619). Internal queries during writes reuse - * this connection, and a nested `transaction()` on either surface joins it. - * - * Closed once `body` settles, BEFORE the caller commits or rolls back. Work - * the body started and did not await (an `async: true` hook) holds the - * handle two ways: ambiently, through this entry, and explicitly, threaded - * into the callback's context and from there into the hook's `ctx.api`. That - * work used to run on the finished handle and lose its write ("Transaction - * query already complete"). Emptying the entry ends the ambient way; - * {@link openedUnits} ends the explicit one, because - * {@link buildDriverOptions} treats a closed unit's handle as absent. The - * work then runs unenrolled, on its own connection, as outside any unit: - * `opensAfterAbortUnit` reads the handle the same way, so such a write opens - * the door's own unit when its hooks call for one. - * - * A primitive handle cannot key {@link openedUnits}, so it keeps only the - * ambient half. - */ private runOpenedUnit(trx: unknown, owner: IDataDriver, body: () => Promise): Promise { - const entry: { transaction: unknown; scope?: TransactionScope } = - { transaction: trx, scope: this.newTransactionScope(owner) }; - if (trx !== null && (typeof trx === 'object' || typeof trx === 'function')) this.openedUnits.set(trx, entry); - return this.txStore.run(entry, body).finally(() => { entry.transaction = undefined; entry.scope = undefined; }); + // [#29986 codemod] The body moved to `engine-transactions-section.ts`, its docblock with it (see `TransactionsSectionHost`). + const readOpenedUnits = () => this.openedUnits; + const readTxStore = () => this.txStore; + const host: ThisParameterType = { + newTransactionScope: (...a) => this.newTransactionScope(...a), + get openedUnits() { return readOpenedUnits(); }, + get txStore() { return readTxStore(); }, + }; + return (TransactionsSectionFacet.runOpenedUnit).call(host, trx, owner, body); } - /** [#22855] Is `handle` a unit {@link runOpenedUnit} published whose callback has since settled? */ private isClosedUnitHandle(handle: unknown): boolean { - if (handle === null || (typeof handle !== 'object' && typeof handle !== 'function')) return false; - const entry = this.openedUnits.get(handle); - return entry !== undefined && entry.transaction === undefined; + // [#29986 codemod] The body moved to `engine-transactions-section.ts`, its docblock with it (see `TransactionsSectionHost`). + const readOpenedUnits = () => this.openedUnits; + const host: ThisParameterType = { + get openedUnits() { return readOpenedUnits(); }, + }; + return TransactionsSectionFacet.isClosedUnitHandle.call(host, handle); } - /** - * Build the observability record for a transaction this engine just opened - * (#4619). See {@link TransactionScope} — records only, routes nothing. - */ private newTransactionScope(owner: IDataDriver): TransactionScope { - return { - driver: owner, - datasource: this.datasourceNameOf(owner), - reportedOutOfScope: new Set(), + // [#29986 codemod] The body moved to `engine-transactions-section.ts`, its docblock with it (see `TransactionsSectionHost`). + const host: ThisParameterType = { + datasourceNameOf: (...a) => this.datasourceNameOf(...a), }; + return TransactionsSectionFacet.newTransactionScope.call(host, owner); } - /** - * Which datasource name is `driver` registered under? - * - * `registerDriver` keys {@link drivers} by `driver.name`, so the two normally - * agree — but the map is the routing authority (`getDriver` resolves through - * it), so the map is what is searched, and `driver.name` is only the fallback - * for a driver that was never registered. Runs at most once per transaction - * (and once per out-of-scope report), never on the hot path. - */ private datasourceNameOf(driver: IDataDriver): string { - for (const [name, registered] of this.drivers) { - if (registered === driver) return name; - } - return driver?.name ?? 'unknown'; + // [#29986 codemod] The body moved to `engine-transactions-section.ts`, its docblock with it (see `TransactionsSectionHost`). + const readDrivers = () => this.drivers; + const host: ThisParameterType = { + get drivers() { return readDrivers(); }, + }; + return TransactionsSectionFacet.datasourceNameOf.call(host, driver); } /** @@ -15948,12 +13147,12 @@ export class ObjectQL implements IObjectQLEngine { * Unregister a single object by name. */ unregisterObject(name: string, packageId?: string): void { - if (packageId) { - this._registry.unregisterObjectsByPackage(packageId); - } else { - // Remove from generic metadata as fallback - this._registry.unregisterItem('object', name); - } + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + }; + return CompatibilitySectionFacet.unregisterObject.call(host, name, packageId); } /** @@ -15961,7 +13160,11 @@ export class ObjectQL implements IObjectQLEngine { * Alias for getSchema() — matches @objectql/core API. */ getObject(name: string): ServiceObject | undefined { - return this.getSchema(name); + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const host: ThisParameterType = { + getSchema: (...a) => this.getSchema(...a), + }; + return CompatibilitySectionFacet.getObject.call(host, name); } /** @@ -15984,7 +13187,12 @@ export class ObjectQL implements IObjectQLEngine { * this method directly looks up a driver by its registered name. */ getDriverByName(name: string): IDataDriver | undefined { - return this.drivers.get(name); + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const readDrivers = () => this.drivers; + const host: ThisParameterType = { + get drivers() { return readDrivers(); }, + }; + return CompatibilitySectionFacet.getDriverByName.call(host, name); } /** @@ -16026,11 +13234,11 @@ export class ObjectQL implements IObjectQLEngine { * @returns The resolved IDataDriver, or undefined if no driver is available. */ getDriverForObject(objectName: string): IDataDriver | undefined { - try { - return this.getDriver(objectName); - } catch { - return undefined; - } + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const host: ThisParameterType = { + getDriver: (...a) => this.getDriver(...a), + }; + return CompatibilitySectionFacet.getDriverForObject.call(host, objectName); } /** @@ -16083,12 +13291,14 @@ export class ObjectQL implements IObjectQLEngine { * Resolves the physical table name from the registered definition, falling * back to the bare name if the def was already removed. */ - async dropObjectSchema(objectName: string): Promise { - const obj = this._registry.getObject(objectName) as any; - const driver = this.getDriverForObject(objectName); - if (!driver || typeof (driver as any).dropTable !== 'function') return; - const tableName = StorageNameMapping.resolveTableName(obj ?? ({ name: objectName } as any)); - await (driver as any).dropTable(tableName); + dropObjectSchema(objectName: string): Promise { + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const read_registry = () => this._registry; + const host: ThisParameterType = { + get _registry() { return read_registry(); }, + getDriverForObject: (...a) => this.getDriverForObject(...a), + }; + return CompatibilitySectionFacet.dropObjectSchema.call(host, objectName); } /** @@ -16098,11 +13308,12 @@ export class ObjectQL implements IObjectQLEngine { * @throws Error if the datasource is not found */ datasource(name: string): IDataDriver { - const driver = this.drivers.get(name); - if (!driver) { - throw new Error(`[ObjectQL] Datasource '${name}' not found`); - } - return driver; + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const readDrivers = () => this.drivers; + const host: ThisParameterType = { + get drivers() { return readDrivers(); }, + }; + return CompatibilitySectionFacet.datasource.call(host, name); } /** @@ -16118,7 +13329,11 @@ export class ObjectQL implements IObjectQLEngine { handler: (ctx: HookContext) => Promise | void, packageId?: string ): void { - this.registerHook(event, handler, { object: objectName, packageId }); + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const host: ThisParameterType = { + registerHook: (...a) => this.registerHook(...a), + }; + return CompatibilitySectionFacet.on.call(host, event, objectName, handler, packageId); } /** @@ -16140,8 +13355,12 @@ export class ObjectQL implements IObjectQLEngine { * Gracefully shut down the engine, disconnecting all drivers. * Alias for destroy() — matches @objectql/core close() API. */ - async close(): Promise { - return this.destroy(); + close(): Promise { + // [#29986 codemod] The body moved to `engine-compatibility-section.ts` (see `CompatibilitySectionHost`). + const host: ThisParameterType = { + destroy: (...a) => this.destroy(...a), + }; + return CompatibilitySectionFacet.close.call(host); } /** @@ -16175,42 +13394,30 @@ export class ObjectQL implements IObjectQLEngine { * objects: { user: { name: 'user', fields: { ... } } } * }); */ - static async create(config: { + static create(config: { datasources?: Record; objects?: Record; hooks?: Array<{ event: string; object: string; handler: (ctx: HookContext) => Promise | void }>; }): Promise { - const ql = new ObjectQL(); - - // Register drivers - if (config.datasources) { - for (const [name, driver] of Object.entries(config.datasources)) { - // Set driver name if not already set - if (!driver.name) { - (driver as any).name = name; - } - ql.registerDriver(driver, name === 'default'); - } - } - - // Register objects - if (config.objects) { - for (const [_key, schema] of Object.entries(config.objects)) { - ql.registerObject(schema); - } - } - - // Register hooks - if (config.hooks) { - for (const hook of config.hooks) { - ql.on(hook.event, hook.object, hook.handler); - } - } - - // Initialize (connect drivers) - await ql.init(); - - return ql; + // [#29986 codemod] The body moved to `engine-compatibility-section.ts`. + return CompatibilitySectionFacet.create(config); + } + + // [#29986 codemod] Fills `ObjectQLStatics` (declared above the class) for the facet modules. + static { + Object.defineProperties(ObjectQLStatics, { + MAX_CASCADE_DEPTH: { get: () => ObjectQL.MAX_CASCADE_DEPTH }, + MAX_EXPAND_DEPTH: { get: () => ObjectQL.MAX_EXPAND_DEPTH }, + REFERENCE_PROBE_BATCH: { get: () => ObjectQL.REFERENCE_PROBE_BATCH }, + coveredValueFieldPresence: { get: () => ObjectQL.coveredValueFieldPresence }, + homeAmountOrganizationOf: { value: (...args: Parameters) => ObjectQL.homeAmountOrganizationOf(...args) }, + keepsOrganizationReference: { value: (...args: Parameters) => ObjectQL.keepsOrganizationReference(...args) }, + mediaFieldPresence: { get: () => ObjectQL.mediaFieldPresence }, + organizationCountContext: { value: (...args: Parameters) => ObjectQL.organizationCountContext(...args) }, + outlivesOrganization: { value: (...args: Parameters) => ObjectQL.outlivesOrganization(...args) }, + remainderAfterMemberRemoval: { value: (...args: Parameters) => ObjectQL.remainderAfterMemberRemoval(...args) }, + storedReferenceIncludes: { value: (...args: Parameters) => ObjectQL.storedReferenceIncludes(...args) }, + }); } } From d949aea4c40131735a228c344b07b26b4f0498c4 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 07:06:50 +0000 Subject: [PATCH 12/13] test: re-key the engine.ts readers for the facade-shape split The regenerated split moved more bodies out of engine.ts, so two by-path readers follow them (ruling 6597555122 A): - federated-injected-column-readers.test.ts: 8 rows re-keyed to the facet modules that now hold their sites (planCascadeAtomicity, organizationOwnedRowCounts, keepsOrganizationReference, expandRelatedRecords, recordObservedDeviation, retractCreationAttestation), and ENGINE_SPLIT_FILES names the three new modules. - core second-object-read-exposure.pin.test.ts: the `executes` row for `ObjectQL.expandRelatedRecords` names engine-summaries-and-expansion.ts, where its referenceTargetOf call now is. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- .../second-object-read-exposure.pin.test.ts | 3 ++- .../federated-injected-column-readers.test.ts | 21 +++++++++++-------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/packages/core/src/security/second-object-read-exposure.pin.test.ts b/packages/core/src/security/second-object-read-exposure.pin.test.ts index fcf51dcacef..4c18daa0ccb 100644 --- a/packages/core/src/security/second-object-read-exposure.pin.test.ts +++ b/packages/core/src/security/second-object-read-exposure.pin.test.ts @@ -133,7 +133,8 @@ const CALLERS: Record = { wiring: { file: 'packages/metadata-protocol/src/protocol.ts', call: 'this.servedExpand(' }, pin: 'packages/metadata-protocol/src/protocol.expand-target-exposure.test.ts', }, - 'packages/objectql/src/engine.ts': { + // [#29986] The codemod split moved the body of `ObjectQL.expandRelatedRecords` out of engine.ts. + 'packages/objectql/src/engine-summaries-and-expansion.ts': { kind: 'executes', read: "the expansion sub-read (`ObjectQL.expandRelatedRecords`), for the data door's `$expand`", decidedIn: 'packages/metadata-protocol/src/protocol-query-validation.ts', diff --git a/packages/objectql/src/federated-injected-column-readers.test.ts b/packages/objectql/src/federated-injected-column-readers.test.ts index a7597d364be..ea7ee72fba8 100644 --- a/packages/objectql/src/federated-injected-column-readers.test.ts +++ b/packages/objectql/src/federated-injected-column-readers.test.ts @@ -117,18 +117,21 @@ const ENGINE_SPLIT_FILES: ReadonlySet = new Set([ 'engine.ts', 'engine-actions.ts', 'engine-cascade-planning.ts', + 'engine-compatibility-section.ts', 'engine-defaults-and-autonumbers.ts', 'engine-delete-and-aggregate.ts', 'engine-find.ts', 'engine-hooks-and-functions.ts', 'engine-insert.ts', 'engine-middleware-and-context.ts', + 'engine-migration-state.ts', 'engine-references-and-master-detail.ts', 'engine-registration.ts', 'engine-secrets-and-routing.ts', 'engine-security-resolvers.ts', 'engine-summaries-and-expansion.ts', 'engine-top-level.ts', + 'engine-transactions-section.ts', ]); /** The calls that are seams by name. */ @@ -155,7 +158,7 @@ const READERS: Record = { disposition: 'skips', why: 'the dependents probe never filters a federated object on a column it does not provision (both phases of the cascade read relations here)', }, - 'engine.ts#planCascadeAtomicity :: isFederatedUnprovisionedInjectedColumn()': { + 'engine-cascade-planning.ts#planCascadeAtomicity :: isFederatedUnprovisionedInjectedColumn()': { disposition: 'skips', why: "the participant test is the scan's own, so the plan and the scan agree", }, @@ -172,16 +175,16 @@ const READERS: Record = { }, // [#24948] The organization-delete refusal reads the tenant column of every // relation the cascade follows into `sys_organization`. - 'engine.ts#organizationOwnedRowCounts :: resolveTenantFieldName()': { + 'engine-cascade-planning.ts#organizationOwnedRowCounts :: resolveTenantFieldName()': { disposition: 'skips', via: 'cascadeRelationBehavior', why: "counts a tenant column only once the cascade's own relation reading admits it, so a federated object's unprovisioned injected column is never counted", }, - 'engine.ts#keepsOrganizationReference :: organization_id': { + 'engine-cascade-planning.ts#keepsOrganizationReference :: organization_id': { disposition: 'not-a-read', why: "names the attribution column a tenancy-disabled object keeps through an organization delete; the walk reads its relation through cascadeRelationBehavior; it reads no row", }, - 'engine.ts#keepsOrganizationReference :: resolveTenantFieldName()': { + 'engine-cascade-planning.ts#keepsOrganizationReference :: resolveTenantFieldName()': { disposition: 'not-a-read', why: 'asks whether the object has a tenant column at all, on a relation the walk already read; it reads no row', }, @@ -286,7 +289,7 @@ const READERS: Record = { disposition: 'target-by-id', why: "the id a caller wrote into a reference field is looked up on the TARGET object", }, - 'engine.ts#expandRelatedRecords :: referenceTargetOf()': { + 'engine-summaries-and-expansion.ts#expandRelatedRecords :: referenceTargetOf()': { disposition: 'target-by-id', why: 'the ids read off the rows in hand load the TARGET rows; a federated row carries none', }, @@ -362,8 +365,8 @@ const READERS: Record = { 'engine-compatibility-api.ts#syncObjectSchema :: isFederatedObject()': { disposition: 'not-a-read', why: SYNC_ROUTE }, 'engine-compatibility-api.ts#syncSchemas :: isFederatedObject()': { disposition: 'not-a-read', why: SYNC_ROUTE }, 'engine-secret-internal-fields.ts#encryptSecretFields :: created_at': { disposition: 'not-a-read', why: PLATFORM_ROW }, - 'engine.ts#recordObservedDeviation :: updated_at': { disposition: 'not-a-read', why: PLATFORM_ROW }, - 'engine.ts#retractCreationAttestation :: updated_at': { disposition: 'not-a-read', why: PLATFORM_ROW }, + 'engine-migration-state.ts#recordObservedDeviation :: updated_at': { disposition: 'not-a-read', why: PLATFORM_ROW }, + 'engine-migration-state.ts#retractCreationAttestation :: updated_at': { disposition: 'not-a-read', why: PLATFORM_ROW }, 'plugin.ts#reconcileFederatedBindings :: isFederatedObject()': { disposition: 'not-a-read', why: SYNC_ROUTE }, 'plugin.ts#registerSchemasWithoutDdl :: isFederatedObject()': { disposition: 'not-a-read', why: SYNC_ROUTE }, 'plugin.ts#syncRegisteredSchemas :: isFederatedObject()': { disposition: 'not-a-read', why: SYNC_ROUTE }, @@ -696,8 +699,8 @@ describe('[#21918] every engine reader of an injected column has a disposition t ); expect([...skipping].sort()).toEqual([ 'engine-cascade-planning.ts#cascadeRelationBehavior', - 'engine.ts#organizationOwnedRowCounts', - 'engine.ts#planCascadeAtomicity', + 'engine-cascade-planning.ts#organizationOwnedRowCounts', + 'engine-cascade-planning.ts#planCascadeAtomicity', 'lifecycle/lifecycle-service.ts#tenantWindowsFor', ]); }); From f48078bb671eee96feaac7f33c281416dd25f0dc Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 08:33:06 +0000 Subject: [PATCH 13/13] chore: re-point the liveness citation and census register after the facade split Two more by-path readers follow the regenerated split: - check:liveness: field/autonumberFormat cited engine.ts#applyAutonumbers, whose body is now in engine-defaults-and-autonumbers.ts. The evidence names that file. - check-system-context-census: `--write` regenerated the site register, which moves the dependentCountIsDisclosable read from engine.ts to engine-cascade-planning.ts. No row changed. Claude-Session: https://claude.ai/code/session_01WGufzhfKpcY1cmybduYBcp Co-authored-by: Claude --- content/docs/permissions/system-context.mdx | 2 +- packages/spec/liveness/field.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/permissions/system-context.mdx b/content/docs/permissions/system-context.mdx index 17460dc1d10..5c3f0736cf3 100644 --- a/content/docs/permissions/system-context.mdx +++ b/content/docs/permissions/system-context.mdx @@ -532,6 +532,7 @@ and CI refuses one that differs from what the tree produces. | `packages/metadata-core/src/meta-write-capability.ts#walledCatalogWriteVerdict` | 1 | 55b | | `packages/metadata-core/src/object-schema-fls.ts#isObjectSchemaMaskExempt` | 1 | 8 | | `packages/objectql/src/credential-evaluation-guard.ts#rejectCredentialEvaluation` | 1 | 18c | +| `packages/objectql/src/engine-cascade-planning.ts#dependentCountIsDisclosable` | 1 | 28 | | `packages/objectql/src/engine-delete-cascade.ts#recordReferenceCheckElevation` | 1 | 29 | | `packages/objectql/src/engine-driver-options.ts#buildDriverOptions` | 1 | 24 | | `packages/objectql/src/engine-execution-plumbing.ts#buildSession` | 1 | 63 | @@ -541,7 +542,6 @@ and CI refuses one that differs from what the tree produces. | `packages/objectql/src/engine-secret-internal-fields.ts#encryptSecretFields` | 1 | 23e | | `packages/objectql/src/engine-secret-internal-fields.ts#stripSearchCompanionFromRead` | 1 | 27 | | `packages/objectql/src/engine-validate.ts#validate` | 1 | 21b | -| `packages/objectql/src/engine.ts#dependentCountIsDisclosable` | 1 | 28 | | `packages/objectql/src/engine.ts#inheritsElevation` | 1 | 1b | | `packages/objectql/src/engine.ts#insert` | 3 | 21, 22 | | `packages/objectql/src/engine.ts#isSystem` | 1 | 64 | diff --git a/packages/spec/liveness/field.json b/packages/spec/liveness/field.json index 9ccc7f69a2f..0e65b0f726e 100644 --- a/packages/spec/liveness/field.json +++ b/packages/spec/liveness/field.json @@ -324,7 +324,7 @@ "autonumberFormat": { "status": "live", "verifiedAt": "2026-08-28", - "evidence": "packages/objectql/src/engine.ts#applyAutonumbers (`const fmt = resolveAutonumberFormat(def as never)` — canonical `autonumberFormat`, then the `format` shorthand (#1603), then the declared default `{0000}`; one resolver shared with the SQL driver so a format-less field cannot render two shapes, #6555)", + "evidence": "packages/objectql/src/engine-defaults-and-autonumbers.ts#applyAutonumbers (`const fmt = resolveAutonumberFormat(def as never)` — canonical `autonumberFormat`, then the `format` shorthand (#1603), then the declared default `{0000}`; one resolver shared with the SQL driver so a format-less field cannot render two shapes, #6555)", "note": "CAVEAT — runtime sequence formatting live; UI AutoNumberField ignores it. 2026-08-28: RE-ANCHORED (commit 8f10a79f7) and REPOINTED — `:765` had rotted onto a `// [#8690]` comment about temporal-filter materializability, ~3,495 lines above the autonumber path. Re-closed by hand against 8cb96ec41." }, "maxLength": {