From 018b6805a1eef989405c3cccc3cc21475b00bb1b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 02:32:13 +0000 Subject: [PATCH 1/3] feat(runtime,client,cli): serve the whole-deployment export, GET /packages/export and os package export The runtime dispatcher answers GET /api/v1/packages/export with every installed package whose registry row is not `code`, disabled ones included, each exactly as GET /packages/:id/export answers it, in dependency order. Same read gate as the per-package export, closed empty query set, whole or absent. client.packages.exportAll() calls it, and os package export [--out DIR] writes one numbered directory per package, each installable with os package install. The spec description drops its PLANNED marker. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01JQwxbWrCtyQtTGt4iYTTvX --- .../28914-package-export-whole-deployment.md | 18 ++ content/docs/concepts/metadata-lifecycle.mdx | 4 +- content/docs/deployment/cli.mdx | 49 ++++ packages/cli/src/commands/package/export.ts | 156 ++++++++++++ packages/cli/src/index.ts | 1 + packages/cli/test/package-export.test.ts | 189 ++++++++++++++ packages/client/src/index.ts | 18 ++ .../client/src/packages-lifecycle.test.ts | 8 + .../client/src/unwrap-misfire.pin.test.ts | 4 + ...e-export-bundle-round-trip.dogfood.test.ts | 230 ++++++++++++++++++ packages/runtime/src/dispatcher-plugin.ts | 4 + .../domains/packages-export-bundle.test.ts | 183 ++++++++++++++ packages/runtime/src/domains/packages.ts | 78 ++++++ packages/runtime/src/route-ledger.ts | 4 + .../spec/src/api/package-lifecycle.test.ts | 15 +- .../spec/src/api/package-lifecycle.zod.ts | 52 ++-- 16 files changed, 973 insertions(+), 40 deletions(-) create mode 100644 .changeset/28914-package-export-whole-deployment.md create mode 100644 packages/cli/src/commands/package/export.ts create mode 100644 packages/cli/test/package-export.test.ts create mode 100644 packages/qa/dogfood/test/package-export-bundle-round-trip.dogfood.test.ts create mode 100644 packages/runtime/src/domains/packages-export-bundle.test.ts diff --git a/.changeset/28914-package-export-whole-deployment.md b/.changeset/28914-package-export-whole-deployment.md new file mode 100644 index 00000000000..b28e50e72e3 --- /dev/null +++ b/.changeset/28914-package-export-whole-deployment.md @@ -0,0 +1,18 @@ +--- +'@objectstack/runtime': patch +'@objectstack/client': patch +'@objectstack/cli': patch +'@objectstack/spec': patch +--- + +feat(runtime,client,cli): export a deployment's whole ontology as installable packages in one read (`GET /packages/export`, `os package export`) + +Clause-②: yes (widening) +Metadata-protocol: none + +A tenant could take its ontology out of a deployment only one package at a time (`GET /packages/:id/export`). The whole-deployment export declared in `@objectstack/spec` (`PackageExportBundleSchema`) is now served. + +- **`@objectstack/runtime`**: `GET /api/v1/packages/export` answers `{ packages }`: every installed package whose registry row's `source` is not `code` (the marketplace and authored packages, each base's `.custom` pair among them), disabled ones included, each exactly as `GET /packages/:id/export` answers it, dependencies first, so a pair follows its base. It reads with the per-package export's gate (`studio.access` or `setup.access`), refuses any query parameter `400 VALIDATION_ERROR` naming it, and is whole or absent: a package whose export cannot be assembled fails the read, naming that package. No credential value is carried: credential positions are withheld as on every served metadata read, and a connector's `credentialRef` stays a reference. +- **`@objectstack/client`**: `client.packages.exportAll()` calls it. +- **`@objectstack/cli`**: `os package export [--out DIR]` writes one directory per package, numbered in install order (`DIR/01-ID/objectstack.json`), each installable with `os package install`. A refusal is printed with its status and code and nothing is written; an `--out` directory that already holds files is refused. +- **`@objectstack/spec`**: the `[PLANNED — not yet served]` marker comes off `PackageExportBundleSchema`'s description, which now names the route and the command. diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index ebe9309c4e4..d6e2f253a0c 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -158,7 +158,9 @@ document **whole** for every read that serves the base's name layered read in `overlay` with `effective` = the base's document. - **Export carries the declaration.** `GET /packages/hotcrm.custom/export` carries the overriding item with its `overrides` block; `GET /packages/hotcrm/export` carries the - base unchanged. + base unchanged. `GET /packages/export` (`os package export`) answers the whole + deployment in one read: every installed package its own code does not register, the + pair as its own entry after its base, each entry exactly the per-package export. ## Overlay whitelist (shared-DB tenancy invariant) diff --git a/content/docs/deployment/cli.mdx b/content/docs/deployment/cli.mdx index 252727b2da1..acb873ec3a0 100644 --- a/content/docs/deployment/cli.mdx +++ b/content/docs/deployment/cli.mdx @@ -2940,6 +2940,7 @@ runtime**. |---------|----------|-------------| | `os package publish [artifact]` | ObjectOS Cloud | Upload a compiled artifact as a versioned package in your organization | | `os package install ` | A running runtime | Install a package into a live kernel, from that runtime's catalog or from a local artifact | +| `os package export` | A running runtime | Export the deployment's whole ontology as installable packages, one directory per package, in install order | For which one to reach for and the preview patterns around them, see [Publish & preview](/docs/deployment/publish-and-preview). This section is the @@ -3134,6 +3135,54 @@ composed without it will not accept installs. In catalog mode, a runtime connected to an ObjectOS Cloud account presents its connection credential to the cloud catalog — see [Cloud Connection](/docs/deployment/cloud-connection). +#### `os package export` + +Takes a running deployment's whole ontology out as packages you can install on +any runtime, the open Apache-2.0 one included. One read, +`GET /api/v1/packages/export`, answers every installed package the deployment's +own code does not register: the marketplace and authored packages, each base's +`.custom` customization package among them. Each comes exactly as +`GET /api/v1/packages/:id/export` answers it, with the namespace and +dependencies it was installed with, and dependencies come first, so a `.custom` +package follows its base. A package registered by the deployment's own code +(`defineStack()`, a plugin, the platform) is not exported: it travels with that +code. + +```bash +os package export --out ./ontology --url https://acme.example.com +# ./ontology/01-com.acme.crm/objectstack.json +# ./ontology/02-com.acme.crm.custom/objectstack.json +# ./ontology/03-com.acme.billing/objectstack.json + +# on the target runtime, in the numbered order: +os package install ./ontology/01-com.acme.crm/objectstack.json +os package install ./ontology/02-com.acme.crm.custom/objectstack.json +os package install ./ontology/03-com.acme.billing/objectstack.json +``` + +Each file is one package's export, installable as it stands with +[`os package install`](#os-package-install). Install them in the numbered order; +the command prints that order too, and never re-sorts what the read answered. + +- **No secret leaves.** Credential positions are withheld, as on every served + metadata read, and a connector's `auth.credentialRef` stays the reference it + is: fill the secret it names on the target runtime. +- **No enabled state.** Every package installs enabled, including one that was + switched off where it was exported. +- **Whole or nothing.** A package whose export cannot be assembled fails the + read, and the command prints the runtime's refusal, which names the package, + and writes nothing. It refuses an `--out` directory that already holds files. +- **Who can export.** The same callers who can read a single package's export: + `studio.access` or `setup.access`. + +**Options:** + +| Flag | Env equivalent | Purpose | +|---|---|---| +| `-o, --out ` | — | Directory to write into (default `./objectstack-export`); created, and refused when it already holds files | +| `-u, --url ` | `OS_CLOUD_URL` | Server URL of the deployment to export (default: the stored `os login` session's, else `http://localhost:3000`) | +| `-t, --token ` | `OS_TOKEN` | Authentication token (default: the stored `os login` session's) | + ## Configuration The CLI looks for `objectstack.config.ts` (or `.js`, `.mjs`) in the current directory: diff --git a/packages/cli/src/commands/package/export.ts b/packages/cli/src/commands/package/export.ts new file mode 100644 index 00000000000..859985ff070 --- /dev/null +++ b/packages/cli/src/commands/package/export.ts @@ -0,0 +1,156 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `objectstack package export` — take a running deployment's whole ontology + * out as installable packages (#28914). + * + * os package export [--out DIR] + * + * One read, `GET /api/v1/packages/export` (`client.packages.exportAll()`, + * `PackageExportBundleSchema`): every installed package the deployment's own + * code does not register — the marketplace and authored packages, each base's + * `.custom` pair among them — each exactly as `GET /packages/:id/export` + * answers it, dependencies first. This command writes them as they came, one + * directory per package, numbered in that order: + * + * DIR/01-com.acme.crm/objectstack.json + * DIR/02-com.acme.crm.custom/objectstack.json + * + * Each file is the package's export, and each is installable as it stands with + * `os package install DIR/NN-ID/objectstack.json` on an open runtime. Install + * them in the numbered order: the order is the read's, and this command never + * re-sorts it. + * + * Loud, never partial: the read is whole or absent (a package whose export + * cannot be assembled fails it, and the runtime's refusal names that package), + * and this command prints that refusal with its status and code and writes + * nothing. It refuses an `--out` directory that already holds files, so an + * export is never mixed into an older one. + * + * No credential value travels: the runtime withholds credential positions on + * this read as on every served metadata read, and a connector's + * `credentialRef` stays the reference it is. + */ + +import { mkdir, readdir, writeFile } from 'node:fs/promises'; +import { join, resolve as resolvePath } from 'node:path'; +import { Command, Flags } from '@oclif/core'; +import { printHeader, printKV, printSuccess, printWarning, printError, printStep, isExitSignal } from '../../utils/format.js'; +import { createApiClient, requireAuth } from '../../utils/api-client.js'; + +/** The file each package directory holds: the name `os package install` is shown first in its examples. */ +export const EXPORTED_PACKAGE_FILE = 'objectstack.json'; + +/** `NN-ID`: the package's place in the read's order, zero-padded so a listing sorts in install order. */ +export function exportedPackageDirectory(index: number, total: number, id: string): string { + const width = Math.max(2, String(total).length); + return `${String(index + 1).padStart(width, '0')}-${id}`; +} + +export default class PackageExport extends Command { + static override description = + "Export a running deployment's whole ontology as installable packages (one directory per package, in install order)"; + + static override examples = [ + '$ os package export', + '$ os package export --out ./ontology --url https://acme.objectos.app', + '$ os package install ./ontology/01-com.acme.crm/objectstack.json # then each directory, in order', + ]; + + static override flags = { + out: Flags.string({ + char: 'o', + description: 'Directory to write the packages into (created; refused when it already holds files)', + default: './objectstack-export', + }), + url: Flags.string({ + char: 'u', + description: 'Server URL of the deployment to export', + env: 'OS_CLOUD_URL', + }), + token: Flags.string({ + char: 't', + description: 'Authentication token', + env: 'OS_TOKEN', + }), + }; + + async run(): Promise { + const { flags } = await this.parse(PackageExport); + + printHeader('Export Packages'); + const outDir = resolvePath(process.cwd(), flags.out); + + try { + const existing = await readdir(outDir).catch((err: NodeJS.ErrnoException) => { + if (err?.code === 'ENOENT') return [] as string[]; + throw err; + }); + if (existing.length > 0) { + printError(`${outDir} already holds files. Export into a new or empty directory (--out DIR).`); + this.exit(1); + return; + } + + const { client, token, baseUrl } = await createApiClient({ url: flags.url, token: flags.token }); + requireAuth(token); + + printStep(`Reading the whole-deployment export from ${baseUrl}...`); + let bundle: unknown; + try { + bundle = await client.packages.exportAll(); + } catch (err: any) { + const status = typeof err?.httpStatus === 'number' ? err.httpStatus : undefined; + const code = typeof err?.code === 'string' ? err.code : undefined; + const tag = [status, code].filter((v) => v !== undefined).join(' '); + printError(`Export refused${tag ? ` (${tag})` : ''}: ${err?.message ?? String(err)}. Nothing was written.`); + this.exit(1); + return; + } + + const packages = (bundle as { packages?: unknown } | null | undefined)?.packages; + if (!Array.isArray(packages)) { + printError('The runtime answered no `packages` list: it does not serve the whole-deployment export. Nothing was written.'); + this.exit(1); + return; + } + const nameless = packages.findIndex( + (entry) => typeof (entry as { id?: unknown } | null)?.id !== 'string' || (entry as { id: string }).id === '', + ); + if (nameless !== -1) { + printError(`Entry ${nameless + 1} of the export names no package id, so it cannot be installed. Nothing was written.`); + this.exit(1); + return; + } + if (packages.length === 0) { + printWarning( + 'Nothing to export: every package of this deployment is registered by its own code, which carries it. Nothing was written.', + ); + return; + } + + await mkdir(outDir, { recursive: true }); + const written: string[] = []; + for (const [index, entry] of packages.entries()) { + const id = (entry as { id: string }).id; + const dir = join(outDir, exportedPackageDirectory(index, packages.length, id)); + await mkdir(dir, { recursive: true }); + const file = join(dir, EXPORTED_PACKAGE_FILE); + await writeFile(file, `${JSON.stringify(entry, null, 2)}\n`, 'utf-8'); + written.push(file); + } + + console.log(''); + printSuccess(`Exported ${packages.length} package${packages.length === 1 ? '' : 's'}`); + printKV(' Directory', outDir); + console.log(''); + console.log(' Install them on an open runtime in this order:'); + for (const file of written) console.log(` os package install ${file}`); + console.log(''); + } catch (error) { + if (isExitSignal(error)) throw error; + printError((error as Error).message); + this.exit(1); + } + } +} diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 60c5fe93256..0681e061d5d 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -55,6 +55,7 @@ export { default as CloudWhoamiCommand } from './commands/cloud/whoami.js'; // ─── Package topic subcommands ────────────────────────────────────── export { default as PackagePublishCommand } from './commands/package/publish.js'; export { default as PackageInstallCommand } from './commands/package/install.js'; +export { default as PackageExportCommand } from './commands/package/export.js'; // ─── Entry-point guard (#10111) ───────────────────────────────────── // This file is the package `main`. It is a LIBRARY entry — a barrel of diff --git a/packages/cli/test/package-export.test.ts b/packages/cli/test/package-export.test.ts new file mode 100644 index 00000000000..f6208246964 --- /dev/null +++ b/packages/cli/test/package-export.test.ts @@ -0,0 +1,189 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#28914] `os package export` — a running deployment's whole ontology, + * written as installable packages. + * + * Drives the real command and the real `@objectstack/client` + * (`packages.exportAll()`, its `fetch` wrapper and unwrap), stubbing only the + * credential boundary: `createApiClient` hands back a real `ObjectStackClient` + * whose `fetch` answers one canned response, as + * `src/commands/meta/delete-json-error-code.test.ts` does. + * + * Pinned: + * - one directory per package, numbered in the read's order, each holding the + * entry exactly as served, and the install commands printed in that order; + * - each written file is what `os package install FILE` sends as its inline + * `manifest` (the real install command, its POST captured), so the export + * round-trips through the command that consumes it; + * - a refusal is printed with its status and code, exits 1, and writes nothing; + * - an `--out` directory that already holds files is refused before the read; + * - an empty bundle writes nothing and says why. + */ + +import { mkdtempSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync, existsSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { describe, it, expect, vi, afterEach, beforeEach } from 'vitest'; +import { ObjectStackClient } from '@objectstack/client'; +import PackageExport, { EXPORTED_PACKAGE_FILE, exportedPackageDirectory } from '../src/commands/package/export.js'; +import PackageInstall from '../src/commands/package/install.js'; + +const stub = vi.hoisted(() => ({ client: undefined as any, calls: [] as string[] })); + +vi.mock('../src/utils/api-client.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + createApiClient: async () => ({ client: stub.client, token: 'test-token', baseUrl: 'https://door.test' }), + }; +}); + +const BASE = { + id: 'com.acme.crm', name: 'crm', version: '1.2.0', namespace: 'crm', + objects: [{ name: 'crm_account', label: 'Account', fields: { name: { type: 'text' } } }], + connectors: [{ name: 'crm_erp', label: 'ERP', type: 'api', provider: 'rest', auth: { type: 'bearer', credentialRef: 'CRM_ERP_TOKEN' } }], +}; +const PAIR = { + id: 'com.acme.crm.custom', name: 'crm-custom', version: '0.1.0', namespace: 'crm_custom', + dependencies: { 'com.acme.crm': '^1' }, + dashboards: [{ name: 'crm_pulse', label: 'Pulse (customized)', overrides: { package: 'com.acme.crm', basedOn: '1.2.0' } }], +}; +const EXT = { + id: 'com.acme.billing', name: 'billing', version: '2.0.0', namespace: 'billing', + dependencies: { 'com.acme.crm': '^1' }, + objectExtensions: [{ extend: 'crm_account', fields: { billing_code: { type: 'text' } } }], +}; + +/** A client whose `fetch` answers one canned response, recording each URL it was asked for. */ +function clientAnswering(status: number, body: unknown): ObjectStackClient { + return new ObjectStackClient({ + baseUrl: 'https://door.test', + token: 'test-token', + fetch: async (url: any) => { + stub.calls.push(String(url)); + return new Response(JSON.stringify(body), { status, headers: { 'content-type': 'application/json' } }); + }, + } as any); +} + +async function run(command: typeof PackageExport | typeof PackageInstall, argv: string[]): Promise<{ out: string; exit: number | undefined }> { + const lines: string[] = []; + const capture = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + vi.spyOn(console, 'log').mockImplementation(capture); + vi.spyOn(console, 'error').mockImplementation(capture); + vi.spyOn(console, 'warn').mockImplementation(capture); + let exit: number | undefined; + try { + await command.run(argv); + } catch (err: any) { + exit = err?.oclif?.exit ?? 1; + } + return { out: lines.join('\n').replace(/\x1b\[[0-9;]*m/g, ''), exit }; +} + +describe('os package export', () => { + let root: string; + + beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'os-package-export-')); + stub.calls = []; + }); + + afterEach(() => { + rmSync(root, { recursive: true, force: true }); + vi.unstubAllGlobals(); + vi.restoreAllMocks(); + }); + + it('writes one directory per package, numbered in the order served, each holding its entry as served', async () => { + stub.client = clientAnswering(200, { success: true, data: { packages: [BASE, PAIR, EXT] } }); + const out = join(root, 'ontology'); + + const r = await run(PackageExport, ['--out', out]); + + expect(r.exit, r.out).toBeUndefined(); + expect(stub.calls).toEqual(['https://door.test/api/v1/packages/export']); + expect(readdirSync(out)).toEqual(['01-com.acme.crm', '02-com.acme.crm.custom', '03-com.acme.billing']); + for (const [i, entry] of [BASE, PAIR, EXT].entries()) { + const file = join(out, exportedPackageDirectory(i, 3, entry.id), EXPORTED_PACKAGE_FILE); + expect(JSON.parse(readFileSync(file, 'utf-8'))).toEqual(entry); + } + // The credential stays the reference it is: written as served, nothing resolved. + expect(readFileSync(join(out, '01-com.acme.crm', EXPORTED_PACKAGE_FILE), 'utf-8')).toContain('"credentialRef": "CRM_ERP_TOKEN"'); + // The install commands, in the order served. + const installs = r.out.split('\n').filter((l) => l.trim().startsWith('os package install ')); + expect(installs.map((l) => l.trim())).toEqual([BASE, PAIR, EXT].map((e, i) => + `os package install ${join(out, exportedPackageDirectory(i, 3, e.id), EXPORTED_PACKAGE_FILE)}`)); + }, 60_000); + + it('each written file is the manifest `os package install FILE` sends, unchanged', async () => { + stub.client = clientAnswering(200, { success: true, data: { packages: [BASE, PAIR, EXT] } }); + const out = join(root, 'ontology'); + expect((await run(PackageExport, ['--out', out])).exit).toBeUndefined(); + vi.restoreAllMocks(); + + for (const [i, entry] of [BASE, PAIR, EXT].entries()) { + const posted: any[] = []; + vi.stubGlobal('fetch', vi.fn(async (_url: string, init: any) => { + posted.push(JSON.parse(init.body)); + return new Response(JSON.stringify({ success: true, data: { manifestId: entry.id, version: entry.version, hotLoaded: true } }), { status: 200 }); + })); + const file = join(out, exportedPackageDirectory(i, 3, entry.id), EXPORTED_PACKAGE_FILE); + const r = await run(PackageInstall, [file, '--runtime', 'http://target.test']); + expect(r.exit, r.out).toBeUndefined(); + expect(posted).toHaveLength(1); + expect(posted[0].manifest).toEqual(entry); + vi.unstubAllGlobals(); + vi.restoreAllMocks(); + } + }, 60_000); + + it('prints a refusal with its status and code, exits 1, and writes nothing', async () => { + const message = "The export of package 'com.acme.crm.custom' could not be assembled, so no bundle is answered: metadata store unavailable"; + stub.client = clientAnswering(503, { success: false, error: { code: 'SERVICE_UNAVAILABLE', message } }); + const out = join(root, 'ontology'); + + const r = await run(PackageExport, ['--out', out]); + + expect(r.exit).toBe(1); + expect(r.out).toContain(`Export refused (503 SERVICE_UNAVAILABLE): ${message}`); + expect(existsSync(out)).toBe(false); + }, 60_000); + + it('refuses an --out directory that already holds files, before reading anything', async () => { + stub.client = clientAnswering(200, { success: true, data: { packages: [BASE] } }); + const out = join(root, 'ontology'); + mkdirSync(out); + writeFileSync(join(out, 'older.json'), '{}'); + + const r = await run(PackageExport, ['--out', out]); + + expect(r.exit).toBe(1); + expect(r.out).toContain('already holds files'); + expect(stub.calls).toEqual([]); + expect(readdirSync(out)).toEqual(['older.json']); + }, 60_000); + + it('writes nothing for an empty bundle, and says the deployment\'s own code carries every package', async () => { + stub.client = clientAnswering(200, { success: true, data: { packages: [] } }); + const out = join(root, 'ontology'); + + const r = await run(PackageExport, ['--out', out]); + + expect(r.exit).toBeUndefined(); + expect(r.out).toContain('Nothing to export'); + expect(existsSync(out)).toBe(false); + }, 60_000); + + it('refuses an entry that names no package id, writing nothing', async () => { + stub.client = clientAnswering(200, { success: true, data: { packages: [BASE, { name: 'no-id', version: '1.0.0' }] } }); + const out = join(root, 'ontology'); + + const r = await run(PackageExport, ['--out', out]); + + expect(r.exit).toBe(1); + expect(r.out).toContain('Entry 2 of the export names no package id'); + expect(existsSync(out)).toBe(false); + }, 60_000); +}); diff --git a/packages/client/src/index.ts b/packages/client/src/index.ts index 76fa83d3fb0..8c8b6c8395d 100644 --- a/packages/client/src/index.ts +++ b/packages/client/src/index.ts @@ -128,6 +128,7 @@ import { RevertPackageCommitResponse, RollbackToPackageCommitResponse, PackageExportManifest, + PackageExportBundle, ReassignOrphanedMetadataResponse, DuplicatePackageResponse, EnsureCustomizationPackageResponse, @@ -3113,6 +3114,23 @@ export class ObjectStackClient { return this.unwrapResponse(res); }, + /** + * [#28914] The deployment's whole ontology as installable packages, in one + * read (`GET /packages/export`, `PackageExportBundleSchema`): every + * installed package the deployment's own code does not register (registry + * `source` other than `code`), enabled or not, each exactly as + * {@link export} answers it, dependencies first, so a `.custom` pair + * follows its base. Install the entries in the order listed; never + * re-sort them. No credential value travels: a `credentialRef` stays the + * reference it is. A package whose export cannot be assembled fails the + * whole read, naming it. + */ + exportAll: async (): Promise => { + const route = this.getRoute('packages'); + const res = await this.fetch(`${this.baseUrl}${route}/export`); + return this.unwrapResponse(res); + }, + /** * [#28411, ADR-0146 D8 item 4] What this deployment changed on an installed * base: its `.custom` pair's served overrides, the members the pair adds to diff --git a/packages/client/src/packages-lifecycle.test.ts b/packages/client/src/packages-lifecycle.test.ts index e98593275ab..6ee6f0db4ff 100644 --- a/packages/client/src/packages-lifecycle.test.ts +++ b/packages/client/src/packages-lifecycle.test.ts @@ -81,6 +81,14 @@ describe('client.packages lifecycle (#3563 PR-4)', () => { expect(manifest.id).toBe('p1'); }); + it('exportAll GETs the whole-deployment export, with no query, and returns the bundle in the order served', async () => { + const packages = [{ id: 'com.acme.crm', name: 'crm', version: '1.0.0' }, { id: 'com.acme.crm.custom', name: 'crm-custom', version: '0.1.0' }]; + const { client, fetchMock } = createMockClient({ success: true, data: { packages } }); + const bundle = await client.packages.exportAll(); + expect(String(fetchMock.mock.calls[0][0])).toBe(`${BASE}/export`); + expect(bundle.packages.map((p) => p.id)).toEqual(['com.acme.crm', 'com.acme.crm.custom']); + }); + it('duplicate requires targetPackageId and forwards the renaming options', async () => { const { client, fetchMock } = createMockClient({ success: true, data: { ok: true } }); await client.packages.duplicate('p1', 'p2', { targetName: 'Copy', targetNamespace: 'copy' }); diff --git a/packages/client/src/unwrap-misfire.pin.test.ts b/packages/client/src/unwrap-misfire.pin.test.ts index a703698debb..2cd59f213b2 100644 --- a/packages/client/src/unwrap-misfire.pin.test.ts +++ b/packages/client/src/unwrap-misfire.pin.test.ts @@ -48,6 +48,7 @@ import { RevertPackageCommitResponseSchema, RollbackToPackageCommitResponseSchema, PackageExportManifestSchema, + PackageExportBundleSchema, ReassignOrphanedMetadataResponseSchema, DuplicatePackageResponseSchema, } from '@objectstack/spec/api'; @@ -71,6 +72,9 @@ const BOUND_PAYLOAD_SCHEMAS: ReadonlyArray = [ ['RevertPackageCommitResponseSchema', RevertPackageCommitResponseSchema], ['RollbackToPackageCommitResponseSchema', RollbackToPackageCommitResponseSchema], ['PackageExportManifestSchema', PackageExportManifestSchema], + // [#28914] `packages.exportAll` — `{ packages }`, strict; each entry is the + // export manifest above, so its open half is covered by the plural pin below. + ['PackageExportBundleSchema', PackageExportBundleSchema], ['ReassignOrphanedMetadataResponseSchema', ReassignOrphanedMetadataResponseSchema], ['DuplicatePackageResponseSchema', DuplicatePackageResponseSchema], ]; diff --git a/packages/qa/dogfood/test/package-export-bundle-round-trip.dogfood.test.ts b/packages/qa/dogfood/test/package-export-bundle-round-trip.dogfood.test.ts new file mode 100644 index 00000000000..c2111c8f07f --- /dev/null +++ b/packages/qa/dogfood/test/package-export-bundle-round-trip.dogfood.test.ts @@ -0,0 +1,230 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// [#28914] The whole-deployment export, `GET /packages/export`, round-trips +// onto a fresh runtime: a deployment holding a base package, the base's +// `.custom` pair with a published override, and a third package that extends +// the base exports three installable packages, and installing them, in the +// order served, through the door `os package install FILE` posts to +// (`POST /api/v1/marketplace/install-local`, `{ manifest }`) yields the same +// metadata names and counts, base and pair alike. +// +// And nothing secret leaves: the base ships a connector whose `auth` names its +// secret with `credentialRef`. The bundle carries the reference, and the value +// the reference resolves to on the source runtime appears nowhere in it. +// +// ## What this file pins, over two showcase boots +// +// 1. source: the bundle answers exactly the three non-`code` packages, +// dependencies first (the base ahead of its pair and of the package that +// extends it), the pair carrying its namespace, its base edge and its +// override; the showcase's own `code` package is not in it; +// 2. source: the connector is exported with its `credentialRef` and without +// the secret it resolves to; +// 3. target: each entry installs 200, in the order served; +// 4. target: the target's own bundle answers the same packages, in the same +// order, with the same namespace, dependencies, and the same item names +// and counts per collection. +// +// Boots fixture stacks of its own (the install-local plugin mounted on both), +// so it stays out of `SHARED_SHOWCASE`. Every credential below is an +// obviously-fake placeholder, and every assertion on one is its absence. + +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { type VerifyStack } from '@objectstack/verify'; +import { MarketplaceInstallLocalPlugin } from '@objectstack/cloud-connection'; +import { PackageExportBundleSchema } from '@objectstack/spec/api'; +import { bootShowcase } from './showcase-boot.js'; +import { declaredOverride, ensurePair, inPair, pairOf } from './customization-pair.js'; + +const ROUTE = '/marketplace/install-local'; +const HOST = 'com.example.showcase'; +const BASE = 'com.dogfood.exportbase'; +const PAIR = pairOf(BASE); +const EXT = 'com.dogfood.exportext'; +const OBJECT = 'exportbase_account'; +const DASHBOARD = 'exportbase_pulse'; +const EDITED = 'Pulse (customized)'; +const CONNECTOR = 'exportbase_erp'; +/** The reference the connector names, and the fake secret it resolves to on the source runtime. */ +const CREDENTIAL_REF = 'EXPORTBASE_ERP_TOKEN'; +const SECRET = 'TEST_PLACEHOLDER_EXPORT_BUNDLE_SECRET'; + +/** The install body `os package install` sends for an inline manifest. */ +const BASE_INSTALL = { + manifest: { + id: BASE, + namespace: 'exportbase', + name: 'Export bundle base', + version: '1.0.0', + type: 'app', + objects: [{ name: OBJECT, label: 'Account', fields: { title: { type: 'text', label: 'Title' } } }], + apps: [{ name: 'exportbase_app', label: 'Export base', navigation: [{ id: 'nav_account', type: 'object', label: 'Accounts', objectName: OBJECT }] }], + dashboards: [{ name: DASHBOARD, label: 'Pulse', columns: 12, widgets: [] }], + flows: [{ + name: 'exportbase_probe', + label: 'Export base probe', + type: 'autolaunched', + status: 'active', + nodes: [{ id: 'start', type: 'start', label: 'Start' }, { id: 'end', type: 'end', label: 'End' }], + edges: [{ id: 'e1', source: 'start', target: 'end' }], + }], + permissions: [{ name: 'exportbase_reader', label: 'Export base reader', objects: { [OBJECT]: { allowRead: true } } }], + connectors: [{ + name: CONNECTOR, + label: 'ERP', + type: 'api', + provider: 'rest', + providerConfig: { baseUrl: 'http://127.0.0.1:9/' }, + auth: { type: 'bearer', credentialRef: CREDENTIAL_REF }, + }], + }, +}; + +/** A third package that depends on the base and extends its object. */ +const EXT_INSTALL = { + manifest: { + id: EXT, + namespace: 'exportext', + name: 'Export bundle extension', + version: '1.0.0', + type: 'app', + dependencies: { [BASE]: '^1' }, + objectExtensions: [{ extend: OBJECT, fields: { exportext_code: { type: 'text', label: 'Billing code' } } }], + flows: [{ + name: 'exportext_probe', + label: 'Export extension probe', + type: 'autolaunched', + status: 'active', + nodes: [{ id: 'start', type: 'start', label: 'Start' }, { id: 'end', type: 'end', label: 'End' }], + edges: [{ id: 'e1', source: 'start', target: 'end' }], + }], + }, +}; + +type Answer = { status: number; body: any }; + +async function call(stack: VerifyStack, token: string, method: string, path: string, body?: unknown): Promise { + const res = await stack.apiAs(token, method, path, body); + return { status: res.status, body: await res.json().catch(() => null) }; +} + +/** One entry's identity and, per collection, its item names (sorted): what "the same names and counts" compares. */ +function inventory(entry: Record): Record { + const out: Record = { id: entry.id, namespace: entry.namespace, dependencies: entry.dependencies ?? null }; + for (const [key, value] of Object.entries(entry)) { + if (!Array.isArray(value)) continue; + out[key] = value.map((item: any) => String(item?.name ?? item?.extend ?? '?')).sort(); + } + return out; +} + +const ids = (bundle: any): string[] => (bundle?.packages ?? []).map((p: any) => p.id); + +describe('dogfood: the whole-deployment export round-trips onto a fresh runtime, and carries no secret', () => { + let previousRef: string | undefined; + const dirs: string[] = []; + const answers: Record = {}; + let sourceBundle: any; + let sourceText = ''; + const installs: Array<{ id: string; answer: Answer }> = []; + let targetBundle: any; + + beforeAll(async () => { + previousRef = process.env[CREDENTIAL_REF]; + process.env[CREDENTIAL_REF] = SECRET; + + const sourceDir = mkdtempSync(join(tmpdir(), 'dogfood-export-bundle-source-')); + dirs.push(sourceDir); + const source = await bootShowcase({ + extraPlugins: [new MarketplaceInstallLocalPlugin({ controlPlaneUrl: 'off', storageDir: sourceDir })], + }); + try { + const token = await source.signIn(); + answers.base = await call(source, token, 'POST', ROUTE, BASE_INSTALL); + answers.ext = await call(source, token, 'POST', ROUTE, EXT_INSTALL); + await ensurePair(source, token, BASE); + answers.saved = await call(source, token, 'PUT', inPair(`/meta/dashboard/${DASHBOARD}?mode=draft`, BASE), + declaredOverride({ name: DASHBOARD, label: EDITED, columns: 12, widgets: [] }, BASE, '1.0.0')); + answers.published = await call(source, token, 'POST', `/packages/${PAIR}/publish-drafts`, {}); + const res = await source.apiAs(token, 'GET', '/packages/export'); + sourceText = await res.text(); + answers.sourceExport = { status: res.status, body: JSON.parse(sourceText) }; + sourceBundle = answers.sourceExport.body?.data; + } finally { + await source.stop(); + } + + const targetDir = mkdtempSync(join(tmpdir(), 'dogfood-export-bundle-target-')); + dirs.push(targetDir); + const target = await bootShowcase({ + extraPlugins: [new MarketplaceInstallLocalPlugin({ controlPlaneUrl: 'off', storageDir: targetDir })], + }); + try { + const token = await target.signIn(); + for (const entry of sourceBundle?.packages ?? []) { + installs.push({ id: entry.id, answer: await call(target, token, 'POST', ROUTE, { manifest: entry }) }); + } + answers.targetExport = await call(target, token, 'GET', '/packages/export'); + targetBundle = answers.targetExport.body?.data; + } finally { + await target.stop(); + } + }, 900_000); + + afterAll(() => { + if (previousRef === undefined) delete process.env[CREDENTIAL_REF]; + else process.env[CREDENTIAL_REF] = previousRef; + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + }); + + it('source: the three packages install and the pair\'s override publishes (preconditions)', () => { + for (const key of ['base', 'ext', 'saved', 'published'] as const) { + expect(answers[key]?.status, `${key}: ${JSON.stringify(answers[key]?.body)}`).toBe(200); + } + }); + + it('source: the bundle is exactly the three non-`code` packages, dependencies first, and parses as declared', () => { + expect(answers.sourceExport.status, sourceText.slice(0, 400)).toBe(200); + expect(PackageExportBundleSchema.safeParse(sourceBundle).success).toBe(true); + const order = ids(sourceBundle); + expect([...order].sort()).toEqual([BASE, PAIR, EXT].sort()); + expect(order).not.toContain(HOST); + expect(order[0]).toBe(BASE); + const pair = sourceBundle.packages.find((p: any) => p.id === PAIR); + expect(pair?.namespace).toBe('exportbase_custom'); + expect(Object.keys(pair?.dependencies ?? {})).toEqual([BASE]); + expect((pair?.dashboards ?? []).map((d: any) => [d.name, d.label, d.overrides?.package])).toEqual([[DASHBOARD, EDITED, BASE]]); + const base = sourceBundle.packages.find((p: any) => p.id === BASE); + expect((base?.dashboards ?? []).map((d: any) => [d.name, d.label])).toEqual([[DASHBOARD, 'Pulse']]); + }); + + it('source: the connector travels with its `credentialRef`, and the secret it resolves to appears nowhere', () => { + const base = sourceBundle.packages.find((p: any) => p.id === BASE); + const connector = (base?.connectors ?? []).find((c: any) => c?.name === CONNECTOR); + expect(connector, JSON.stringify(base?.connectors)).toBeDefined(); + expect(connector.auth?.credentialRef).toBe(CREDENTIAL_REF); + // CONTROL: the reference resolves to the secret on the source runtime. + expect(process.env[CREDENTIAL_REF]).toBe(SECRET); + expect(sourceText.includes(SECRET)).toBe(false); + }); + + it('target: every entry installs, in the order served', () => { + expect(installs.map((i) => i.id)).toEqual(ids(sourceBundle)); + for (const { id, answer } of installs) { + expect(answer.status, `${id}: ${JSON.stringify(answer.body)}`).toBe(200); + expect(answer.body?.data?.manifestId).toBe(id); + } + }); + + it('target: the same packages come back, in the same order, with the same names and counts', () => { + expect(answers.targetExport.status, JSON.stringify(answers.targetExport.body)).toBe(200); + expect(ids(targetBundle)).toEqual(ids(sourceBundle)); + expect(targetBundle.packages.map(inventory)).toEqual(sourceBundle.packages.map(inventory)); + const pair = targetBundle.packages.find((p: any) => p.id === PAIR); + expect((pair?.dashboards ?? []).map((d: any) => [d.name, d.label, d.overrides?.package])).toEqual([[DASHBOARD, EDITED, BASE]]); + }); +}); diff --git a/packages/runtime/src/dispatcher-plugin.ts b/packages/runtime/src/dispatcher-plugin.ts index 352715cbdc9..66bb830856f 100644 --- a/packages/runtime/src/dispatcher-plugin.ts +++ b/packages/runtime/src/dispatcher-plugin.ts @@ -1586,6 +1586,10 @@ export function createDispatcherPlugin(config: DispatcherPluginConfig = {}): Plu mountPackagesRoute('get', '', () => ''); mountPackagesRoute('post', '', () => ''); + // [#28914] The whole-deployment export, registered ahead of + // `/:id` so the literal segment wins; `export` is never a + // package id (two dot-separated segments at least). + mountPackagesRoute('get', '/export', () => '/export'); mountPackagesRoute('get', '/:id/export', (req) => `/${req.params.id}/export`); // ADR-0146 D8 item 4 — what this deployment changed on an // installed base (overrides, extensions, switched-off items). diff --git a/packages/runtime/src/domains/packages-export-bundle.test.ts b/packages/runtime/src/domains/packages-export-bundle.test.ts new file mode 100644 index 00000000000..1575860e6f3 --- /dev/null +++ b/packages/runtime/src/domains/packages-export-bundle.test.ts @@ -0,0 +1,183 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#28914] `GET /packages/export` — the deployment's whole ontology as + * installable packages, in one read (`PackageExportBundleSchema`). + * + * The door is the real dispatcher over a registry and protocol double, so what + * is pinned is the door's own rules, each the declaration's: + * + * - the per-package export's read gate, asked before anything is read; + * - a closed, empty query set (a parameter is refused 400 `VALIDATION_ERROR`, + * naming it); + * - which packages: every registry row whose `source` is not `code`, a + * disabled one included, each once; + * - the order: dependencies first (ADR-0141 D5), whatever order the registry + * lists the rows in, so a `.custom` pair follows its base; + * - one producer: each entry is exactly what `GET /packages/:id/export` + * answers for that package; + * - whole or absent: a package whose export cannot be assembled fails the + * read, naming it, and no bundle is answered. + * + * The round trip onto a fresh runtime, and the connector credential that stays + * a reference, run on real showcase boots in + * `packages/qa/dogfood/test/package-export-bundle-round-trip.dogfood.test.ts`. + */ +import { describe, expect, it, vi } from 'vitest'; +import { PackageExportBundleSchema } from '@objectstack/spec/api'; +import { HttpDispatcher } from '../http-dispatcher.js'; +import { ROUTE_LEDGER } from '../route-ledger.js'; + +const BASE = 'com.example.crm'; +const PAIR = 'com.example.crm.custom'; +const EXT = 'com.example.billing'; +const OFF = 'com.example.archive'; +const CODE = 'com.example.host'; + +const ctx = (executionContext: any): any => ({ request: {}, environmentId: 'platform', executionContext }); +const system = () => ctx({ isSystem: true }); +const holding = (caps: string[]) => ctx({ userId: 'u_1', isSystem: false, systemPermissions: caps }); + +/** Registry rows, listed dependents FIRST, so only a real sort puts the base ahead. */ +const ROWS = [ + { id: EXT, source: 'marketplace', enabled: true, manifest: { id: EXT, name: 'billing', version: '2.0.0', namespace: 'billing', dependencies: { [BASE]: '^1' } } }, + { id: PAIR, source: 'authored', enabled: true, manifest: { id: PAIR, name: 'crm-custom', version: '0.1.0', namespace: 'crm_custom', dependencies: { [BASE]: '^1' } } }, + { id: CODE, source: 'code', enabled: true, manifest: { id: CODE, name: 'host', version: '1.0.0' } }, + { id: BASE, source: 'marketplace', enabled: true, manifest: { id: BASE, name: 'crm', version: '1.2.0', namespace: 'crm' } }, + { id: OFF, source: 'authored', enabled: false, status: 'disabled', manifest: { id: OFF, name: 'archive', version: '1.0.0' } }, +]; + +/** Each package's own items, as the per-package layer answers them. */ +const ITEMS: Record> = { + [BASE]: { object: [{ name: 'crm_account', label: 'Account', fields: {}, _packageId: BASE }], dashboard: [{ name: 'crm_pulse', label: 'Pulse', _packageId: BASE }] }, + [PAIR]: { dashboard: [{ name: 'crm_pulse', label: 'Pulse (customized)', overrides: { package: BASE, basedOn: '1.2.0' }, _packageId: PAIR }] }, + [EXT]: { flow: [{ name: 'billing_close', label: 'Close', _packageId: EXT }] }, + [OFF]: { report: [{ name: 'archive_old', label: 'Old', _packageId: OFF }] }, + [CODE]: { object: [{ name: 'host_thing', label: 'Thing', _packageId: CODE }] }, +}; + +function make(over: { protocol?: any; rows?: any[] } = {}) { + const rows = over.rows ?? ROWS; + const registry = { + getAllPackages: vi.fn(() => rows), + getPackage: vi.fn((id: string) => rows.find((r) => r.id === id)), + }; + const protocol = over.protocol === undefined + ? { + getMetaItemsPerPackage: vi.fn(async ({ type, packageId }: { type: string; packageId: string }) => ({ items: ITEMS[packageId]?.[type] ?? [] })), + getMetaItems: vi.fn(async () => ({ items: [] })), + } + : over.protocol; + const kernel: any = { + context: { getService: (name: string) => (name === 'objectql' ? { registry } : name === 'protocol' ? protocol : null) }, + }; + return { dispatcher: new HttpDispatcher(kernel), registry, protocol }; +} + +const bundleOf = (r: any): any => r.response?.body?.data; + +describe('[#28914] GET /packages/export answers the whole deployment as installable packages', () => { + it('answers every package whose registry row is not `code`, a disabled one included, each once, and parses as the declared bundle', async () => { + const { dispatcher } = make(); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, {}, system()); + expect(r.response?.status, JSON.stringify(r.response?.body)).toBe(200); + const bundle = bundleOf(r); + expect(PackageExportBundleSchema.safeParse(bundle).success).toBe(true); + const ids = bundle.packages.map((p: any) => p.id); + expect([...ids].sort()).toEqual([BASE, PAIR, EXT, OFF].sort()); + expect(ids).not.toContain(CODE); + expect(new Set(ids).size).toBe(ids.length); + }); + + it('lists dependencies first: the base before its `.custom` pair and before the package that depends on it', async () => { + const { dispatcher } = make(); + const ids = bundleOf(await dispatcher.handlePackages('/export', 'GET', undefined, {}, system())).packages.map((p: any) => p.id); + // CONTROL: the registry lists both dependents ahead of the base. + expect(ROWS.map((r) => r.id).indexOf(BASE)).toBeGreaterThan(ROWS.map((r) => r.id).indexOf(PAIR)); + expect(ids.indexOf(BASE)).toBeLessThan(ids.indexOf(PAIR)); + expect(ids.indexOf(BASE)).toBeLessThan(ids.indexOf(EXT)); + }); + + it('each entry is exactly what `GET /packages/:id/export` answers for that package (one producer)', async () => { + const { dispatcher } = make(); + const bundle = bundleOf(await dispatcher.handlePackages('/export', 'GET', undefined, {}, system())); + for (const entry of bundle.packages) { + const single = await dispatcher.handlePackages(`/${encodeURIComponent(entry.id)}/export`, 'GET', undefined, {}, system()); + expect(single.response?.status).toBe(200); + expect(entry).toEqual(single.response?.body?.data); + } + const pair = bundle.packages.find((p: any) => p.id === PAIR); + expect(pair).toMatchObject({ namespace: 'crm_custom', dependencies: { [BASE]: '^1' } }); + expect(pair.dashboards).toEqual([{ name: 'crm_pulse', label: 'Pulse (customized)', overrides: { package: BASE, basedOn: '1.2.0' } }]); + }); + + it('answers an empty list when every package is registered by the deployment\'s own code', async () => { + const { dispatcher } = make({ rows: ROWS.filter((r) => r.source === 'code') }); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, {}, system()); + expect(r.response?.status).toBe(200); + expect(bundleOf(r)).toEqual({ packages: [] }); + }); + + it('reads with the per-package export\'s gate: `studio.access` or `setup.access`, else 403 before anything is read', async () => { + for (const caps of [['studio.access'], ['setup.access']]) { + const { dispatcher } = make(); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, {}, holding(caps)); + expect(r.response?.status, caps.join()).toBe(200); + } + const { dispatcher, registry, protocol } = make(); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, {}, holding(['manage_metadata'])); + expect(r.response?.status).toBe(403); + expect(registry.getAllPackages).not.toHaveBeenCalled(); + expect(protocol.getMetaItemsPerPackage).not.toHaveBeenCalled(); + }); + + it('refuses any query parameter 400 VALIDATION_ERROR, naming it, before anything is read', async () => { + const { dispatcher, registry } = make(); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, { format: 'zip' }, system()); + expect(r.response?.status).toBe(400); + expect(r.response?.body?.error?.code).toBe('VALIDATION_ERROR'); + expect(r.response?.body?.error?.message).toContain('format'); + expect(registry.getAllPackages).not.toHaveBeenCalled(); + }); + + it('is whole or absent: a package whose export the protocol cannot assemble fails the read, naming it', async () => { + const { dispatcher } = make({ protocol: null }); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, {}, system()); + expect(r.response?.status).toBe(500); + expect(r.response?.body?.data).toBeUndefined(); + expect(r.response?.body?.error?.message).toContain(`'${BASE}'`); + }); + + it('is whole or absent: a package whose export throws fails the read with its status, naming that package', async () => { + let reads = 0; + const protocol = { + getMetaItemsPerPackage: async ({ type, packageId }: { type: string; packageId: string }) => ({ items: ITEMS[packageId]?.[type] ?? [] }), + // Read once per assembled package: the second package's assembly throws. + get getMetaItems() { + reads += 1; + if (reads === 2) throw Object.assign(new Error('metadata store unavailable'), { status: 503 }); + return async () => ({ items: [] }); + }, + }; + const { dispatcher } = make({ protocol }); + const r = await dispatcher.handlePackages('/export', 'GET', undefined, {}, system()); + expect(r.response?.status).toBe(503); + expect(r.response?.body?.data).toBeUndefined(); + const message: string = r.response?.body?.error?.message ?? ''; + expect(message).toMatch(/^The export of package '[^']+' could not be assembled/); + expect(message).toContain('metadata store unavailable'); + expect(message).not.toContain(`'${BASE}'`); + }); + + it('is not read as a package id: `export` never reaches `GET /packages/:id`', async () => { + const { dispatcher, registry } = make(); + await dispatcher.handlePackages('/export', 'GET', undefined, {}, system()); + expect(registry.getPackage).not.toHaveBeenCalledWith('export'); + }); + + it('has its route-ledger row: the SDK method and the declared response schema', () => { + const row = ROUTE_LEDGER.find((e) => e.route === 'GET /packages/export'); + expect(row?.client).toBe('packages.exportAll'); + expect(row?.responseSchema).toBe('PackageExportBundleSchema'); + }); +}); diff --git a/packages/runtime/src/domains/packages.ts b/packages/runtime/src/domains/packages.ts index 1a91dc116ce..8a46b0cd25a 100644 --- a/packages/runtime/src/domains/packages.ts +++ b/packages/runtime/src/domains/packages.ts @@ -70,6 +70,10 @@ import { customizationPairManifestRefusal } from '@objectstack/metadata-protocol // [#28117] The one reader of a manifest's declared dependency ids (ADR-0141), // shared with the registry's co-owner gate. import { declaredDependencyIds } from '@objectstack/core'; +// [#28914, ADR-0141 D5] The one sorter of installed packages by their declared +// dependencies: the order a boot hydrates them in is the order the +// whole-deployment export lists them in. +import { orderHydrationBatch } from '@objectstack/core'; // [#9960] The uninstall seam's DECLARED shapes, from the same producer and for // the same reason as the predicate above: this door reached `deletePackage` // through `protocol` and routinely sent two keys — `organizationId` @@ -2570,6 +2574,23 @@ export async function handlePackagesRequest(deps: DomainHandlerDeps, path: strin return { handled: true, response: deps.error('Metadata service not available', 503) }; } + // GET /packages/export → [#28914] the deployment's whole ontology as + // installable packages, in one read (`PackageExportBundleSchema`). + // Matched here, ahead of `GET /packages/:id`: `export` is never a + // package id (`MANIFEST_ID_PATTERN` needs two dot-separated segments), + // so the path shadows no package. See {@link assemblePackageExportBundle}. + if (parts.length === 1 && parts[0] === 'export' && m === 'GET') { + // The per-package export's read gate, asked before anything is read. + const denied = requireReadCapability(deps, _context); if (denied) return denied; + // Closed query set: the door reads none, so a parameter is a + // request it would otherwise answer as if it had been honoured. + const unknownQuery = unknownQueryParamRefusalMessage(query, []); + if (unknownQuery !== undefined) { + return { handled: true, response: deps.error(unknownQuery, 400, { code: 'VALIDATION_ERROR' }) }; + } + return { handled: true, response: await assemblePackageExportBundle(deps, registry, _context) }; + } + // GET /packages/:id/export → assemble a portable manifest from // sys_metadata overlay rows bound to this package (offline export). if (parts.length === 2 && parts[1] === 'export' && m === 'GET') { @@ -3532,6 +3553,63 @@ context: HttpProtocolContext, return manifest; } +/** + * [#28914] The answer of `GET /packages/export`: every package whose registry + * row's `source` is not `code`, enabled or not, each once, dependencies first, + * each entry exactly what `GET /packages/:id/export` answers for it + * ({@link assemblePackageManifest}, the one producer: ⛔ no item is assembled, + * cleaned or redacted a second way here). `PackageExportBundleSchema` is the + * contract; its docblock argues each rule. + * + * - **Which packages.** A `code` row is registered by the deployment's own + * code and travels with it, never with the bundle; the row's `source` is the + * one key that says so (⛔ not `manifest.scope`, not `writable`). + * - **Order.** ADR-0141 D5's, through the boot's own sorter + * ({@link orderHydrationBatch}) over each row's declared dependency ids + * ({@link declaredDependencyIds}): a `.custom` pair comes after its base, + * and a cycle keeps the registry's listing order among its members. + * - **Whole or absent.** A package whose export cannot be assembled fails the + * read, naming that package; ⛔ never a bundle that silently lacks one. + */ +async function assemblePackageExportBundle( +deps: DomainHandlerDeps, +registry: any, +context: HttpProtocolContext, +): Promise> { + const rows: any[] = Array.isArray(registry.getAllPackages?.()) ? registry.getAllPackages() : []; + const seen = new Set(); + const nodes: Array<{ item: string; label: string; packageIds: string[]; declaredIds: string[] }> = []; + for (const row of rows) { + if (row?.source === 'code') continue; + const id = row?.manifest?.id ?? row?.id; + if (typeof id !== 'string' || id.length === 0 || seen.has(id)) continue; + seen.add(id); + nodes.push({ item: id, label: id, packageIds: [id], declaredIds: declaredDependencyIds(row?.manifest) }); + } + const packages: Record[] = []; + for (const id of orderHydrationBatch(nodes).ordered) { + let manifest: Record | null; + try { + manifest = await assemblePackageManifest(deps, id, registry, context); + } catch (e: any) { + const answer: any = deps.errorFromThrown(e, 500); + const error = answer?.body?.error; + if (error && typeof error.message === 'string') { + error.message = `The export of package '${id}' could not be assembled, so no bundle is answered: ${error.message}`; + } + return answer; + } + if (!manifest) { + return deps.error( + `The export of package '${id}' could not be assembled (the metadata protocol answered nothing for it), so no bundle is answered.`, + 500, + ); + } + packages.push(manifest); + } + return deps.success({ packages }); +} + /** * Apply just-published `seed` metadata: load each seed's rows into its * target object so publishing a seed draft makes the data live (the runtime diff --git a/packages/runtime/src/route-ledger.ts b/packages/runtime/src/route-ledger.ts index 60b255866d0..0fd627e3395 100644 --- a/packages/runtime/src/route-ledger.ts +++ b/packages/runtime/src/route-ledger.ts @@ -543,6 +543,10 @@ export const ROUTE_LEDGER: readonly RouteLedgerEntry[] = [ responseSchema: 'RollbackToPackageCommitResponseSchema', note: 'Enveloped — the named schema is the `data`: `rollbackToPackageCommit`\'s declared return, the ADR-0067 COMMIT rollback. Fillable only after the retirement of `PackageRollbackResponseSchema` + `PackageApiContracts.rollbackPackage`, the VERSION-rollback declaration the spec had falsely bound to this exact path. Conformance: spec `api/package-lifecycle.test.ts`' }, { route: 'POST /packages/:id/revert', domain: '/packages', disposition: 'sdk', client: 'packages.revert' }, + // [#28914] The whole-deployment export: every installed package as installable packages, in one read. + { route: 'GET /packages/export', domain: '/packages', disposition: 'sdk', client: 'packages.exportAll', + responseSchema: 'PackageExportBundleSchema', + note: 'Enveloped — the named schema is the `data`: `{ packages }`, each entry exactly what `GET /packages/:id/export` answers (`assemblePackageManifest`, the one producer), for every registry row whose `source` is not `code`, disabled ones included, in ADR-0141 D5 dependency order (`orderHydrationBatch`, so a `.custom` pair follows its base). Same read gate as the per-package export; closed query set (none: a parameter is refused 400 `VALIDATION_ERROR` naming it); whole or absent (a package whose export cannot be assembled fails the read, naming it). Matched ahead of `GET /packages/:id`; `export` is never a package id. Pinned in `domains/packages-export-bundle.test.ts`, which drives THIS handler and parses the payload through the response schema; the round trip onto a fresh runtime is the dogfood `package-export-bundle-round-trip.dogfood.test.ts`' }, { route: 'GET /packages/:id/export', domain: '/packages', disposition: 'sdk', client: 'packages.export', responseSchema: 'PackageExportManifestSchema', note: 'Enveloped — the named schema is the `data`: the ADR-0070 portable manifest. Four fixed keys (`id`, `name`, `version`, `label?`) plus an OPEN catch-all — the remaining keys are registry-derived per metadata type present, deliberately not enumerated (freezes nothing). Conformance: spec `api/package-lifecycle.test.ts`' }, diff --git a/packages/spec/src/api/package-lifecycle.test.ts b/packages/spec/src/api/package-lifecycle.test.ts index bcf00bb46ae..8a0b6862626 100644 --- a/packages/spec/src/api/package-lifecycle.test.ts +++ b/packages/spec/src/api/package-lifecycle.test.ts @@ -370,14 +370,15 @@ describe('PackageExportBundleSchema declares the whole-deployment export as comp expect((result.error.issues[0] as { keys?: string[] }).keys).toEqual(['cycles']); }); - // Declared before it is served (ADR-0049): no runtime mounts - // `GET /packages/export` until the server slice of #28914 lands. That - // change drops the marker, adds the route's ledger row and flips this test. - it('is PLANNED until a runtime serves the route: the describe says so, naming no tracker id', () => { + // Served (#28914's server slice mounted `GET /packages/export` on the runtime + // dispatcher, with its ledger row): the ADR-0049 marker is gone, and the + // describe names the route and the command that calls it. + it('is served: the describe carries no PLANNED marker, names the route and the command, and no tracker id', () => { const description = PackageExportBundleSchema.description ?? ''; - expect(description).toMatch(/^\[PLANNED — not yet served\] /); - expect(description).toContain('ADR-0049'); - expect(description).toContain('no runtime serves `GET /packages/export` yet'); + expect(description).not.toMatch(/PLANNED/); + expect(description).not.toMatch(/not yet served/); + expect(description).toContain('`GET /packages/export`'); + expect(description).toContain('`os package export`'); // Customer-facing prose carries no tracker id (check:doc-authoring); the slice is named in the comments. expect(description).not.toMatch(/#\d{3,}/); }); diff --git a/packages/spec/src/api/package-lifecycle.zod.ts b/packages/spec/src/api/package-lifecycle.zod.ts index aa958041959..c17674d1d85 100644 --- a/packages/spec/src/api/package-lifecycle.zod.ts +++ b/packages/spec/src/api/package-lifecycle.zod.ts @@ -264,34 +264,27 @@ export const PackageExportManifestSchema = lazySchema(() => z.object({ * packages, in one read (#28914): the exit right that * {@link PackageExportManifestSchema}'s route grants one package at a time. * - * ⚠️ DECLARED AHEAD OF ITS PRODUCER (ADR-0049). No runtime serves - * `GET /packages/export` yet: no handler mounts it, so a call today is not - * answered with this shape. This declaration is the contract the server slice - * of #28914 must meet (Clause-②, widening): it mounts the route on the runtime - * dispatcher, enveloped like its lifecycle siblings (`{ success, data }`), and - * answers this schema as the `data`. Until it does, the describe carries - * `[PLANNED — not yet served]`. The house has no other planned channel for an - * API response: the liveness ledgers are per metadata type, and the runtime - * route ledger records only what the dispatcher serves. So the change that - * mounts the route drops the marker, adds the route's ledger row and flips the - * test that pins the marker. - * - * Everything below is what that door MUST do. None of it is served today. + * Served by the runtime dispatcher (`packages/runtime/src/domains/packages.ts`, + * `assemblePackageExportBundle`), enveloped like its lifecycle siblings + * (`{ success, data }`), with this schema as the `data`; the route's ledger row + * names it. It was declared ahead of its producer (ADR-0049) and carried + * `[PLANNED — not yet served]` until the change that mounted the route, which + * dropped the marker and flipped the test that pinned it. * * ## One producer * - * Each entry must be exactly what `GET /packages/:id/export` (served today) - * answers for that package. The bundle composes the per-package export and - * must never assemble, clean or redact an item a second way, so the two reads - * cannot disagree about a package. Two consequences follow on purpose: + * Each entry is exactly what `GET /packages/:id/export` answers for that + * package. The bundle composes the per-package export and never assembles, + * cleans or redacts an item a second way, so the two reads cannot disagree + * about a package. Two consequences follow on purpose: * * - **A `.custom` pair is its own entry**, carrying its overriding items with * their `overrides` blocks (ADR-0146 D8 item 3), and its base's entry carries * the base unchanged. Environment-scope customizations stay where the * per-package export already places them: under the package that owns them. * - **No credential value leaves.** The per-package export already withholds - * credential positions, as every served metadata read does. The bundle must - * add no rule of its own: a `credentialRef` stays the reference it is. + * credential positions, as every served metadata read does. The bundle adds + * no rule of its own: a `credentialRef` stays the reference it is. * * ## Which packages, in which order * @@ -313,7 +306,7 @@ export const PackageExportManifestSchema = lazySchema(() => z.object({ * switched off comes back on, and the consumer disables it again if it wants it * off. * - * The order must be ADR-0141 D5's, the order a boot hydrates installed + * The order is ADR-0141 D5's, the order a boot hydrates installed * packages in: dependencies first. An entry comes after every entry its * package's `manifest.dependencies` names, so a pair comes after its base. * Installing the entries in the order listed is the whole consumption rule: a @@ -322,15 +315,15 @@ export const PackageExportManifestSchema = lazySchema(() => z.object({ * * ## The door * - * The read gate must be the per-package export's: `studio.access` or + * The read gate is the per-package export's: `studio.access` or * `setup.access`, or a system context, asked before anything is read. The - * query set must be CLOSED and empty: the door refuses a query parameter + * query set is CLOSED and empty: the door refuses a query parameter * `400 VALIDATION_ERROR`, naming it. No request schema is declared, because * the door reads no path id, no body and no parameter. `export` can never be a * package id (one has at least two dot-separated segments, * `MANIFEST_ID_PATTERN`), so the path never shadows `GET /packages/:id`. * - * The answer must be whole or absent: a package whose export cannot be + * The answer is whole or absent: a package whose export cannot be * assembled fails the read, naming that package. ⛔ Never a bundle that * silently lacks a package. Strict at the top level: a member the door starts * serving is a change to this declaration first. Each entry stays open, as @@ -345,16 +338,11 @@ export const PackageExportBundleSchema = lazySchema(() => z.object({ + 'state, so every entry installs enabled. Empty when every package of the deployment is registered by its ' + 'own code.', ), - // Declared before it is served (ADR-0049): the marker comes off when the - // server slice of #28914 mounts `GET /packages/export`, in the same change - // as the route's ledger row. }).strict().describe( - '[PLANNED — not yet served] Whole-deployment export: every installed package the deployment\'s own code does ' - + 'not register (registry `source` other than `code`), as installable packages in dependency order. Carries no ' - + 'credential value and no enabled state. Declared ahead of its producer ' - + '(ADR-0049): no runtime serves `GET /packages/export` yet. The server half of the whole-deployment package ' - + 'export mounts it, together with a future CLI command that calls it; until then this is the contract that ' - + 'door must answer, not a read a client can make.', + 'Whole-deployment export, the answer of `GET /packages/export` (`os package export`): every installed package ' + + 'the deployment\'s own code does not register (registry `source` other than `code`), as installable packages ' + + 'in dependency order. Carries no credential value and no enabled state. A package whose export cannot be ' + + 'assembled fails the whole read, naming it.', )); /** From 699399a4284818ab1c14298499759d976c5ea2cc Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 02:35:00 +0000 Subject: [PATCH 2/3] chore(spec): regenerate the package-lifecycle reference for the served export bundle Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01JQwxbWrCtyQtTGt4iYTTvX --- content/docs/references/api/package-lifecycle.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/references/api/package-lifecycle.mdx b/content/docs/references/api/package-lifecycle.mdx index 686288df26d..06abe6eee0b 100644 --- a/content/docs/references/api/package-lifecycle.mdx +++ b/content/docs/references/api/package-lifecycle.mdx @@ -317,7 +317,7 @@ A field or flow another package contributes to one of the base's objects. ## PackageExportBundle -[PLANNED — not yet served] Whole-deployment export: every installed package the deployment's own code does not register (registry `source` other than `code`), as installable packages in dependency order. Carries no credential value and no enabled state. Declared ahead of its producer (ADR-0049): no runtime serves `GET /packages/export` yet. The server half of the whole-deployment package export mounts it, together with a future CLI command that calls it; until then this is the contract that door must answer, not a read a client can make. +Whole-deployment export, the answer of `GET /packages/export` (`os package export`): every installed package the deployment's own code does not register (registry `source` other than `code`), as installable packages in dependency order. Carries no credential value and no enabled state. A package whose export cannot be assembled fails the whole read, naming it. ### Properties From 43a7c5502594cd085aa926a84fd8ff779aa38b40 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 03:08:23 +0000 Subject: [PATCH 3/3] chore(runtime): route-ledger census sentence counts the export bundle row Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01JQwxbWrCtyQtTGt4iYTTvX --- packages/runtime/src/route-ledger.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/runtime/src/route-ledger.ts b/packages/runtime/src/route-ledger.ts index 0fd627e3395..ab7303f2c61 100644 --- a/packages/runtime/src/route-ledger.ts +++ b/packages/runtime/src/route-ledger.ts @@ -301,7 +301,7 @@ export const NON_DISPATCH_MOUNT_PREFIXES = [ /** * The ledger. * - * CENSUS (generated): this list holds 116 rows. + * CENSUS (generated): this list holds 117 rows. * * ⛔ THAT NUMBER IS WRITTEN BY A TOOL — never by hand. * `pnpm check:route-ledger-census` counts the rows below and fails when the two