Skip to content

XERK-2554: Move an existing hub's fleet and data into its owner's tenant - #1376

Merged
xerhab merged 3 commits into
mainfrom
XERK-2554
Nov 1, 2026
Merged

xerhab merged 3 commits into
mainfrom
XERK-2554

Conversation

@xerhab

@xerhab xerhab commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary: An existing single-tenant hub can now be upgraded in place: on boot it moves all of its pre-tenancy hosts and data into one named owner's tenant. Hosts keep their tokens and nobody re-enrolls.

Fixes XERK-2554

Why

Multi-tenancy (XERK-2529) left today's fleet "untenanted": only break-glass and admin operators can reach it, and binding a host never carried its history. The production hub needs a safe, one-time path to hand its existing fleet, settings, usage and archive to the person who owns them.

What changed

  • TURMA_MIGRATE_OWNER (local:<user>, oidc:<sub>, or a bare existing user) names the owner. A prefixed owner is created if missing; a bare unknown name is refused.
  • A resumable migration runs on the leader with one idempotent step per store (turma/tenant-migration.js). Each step is flushed to disk before the marker records it. TURMA_MIGRATE_DRY_RUN=1 logs counts per store and changes nothing.
  • Hosts are bound like a roll-token?bind=1; a host still on a name or master token is handed its identity token. Org stores, the ticket queue, push devices, the routing policy, the three ledgers and the archive rows are re-keyed into the tenant. A live host's rows move at its own bind beat.
  • Until the migration completes, tenant users get 503; operators are unaffected. Afterwards, the shared Tenir and trigger tokens act for the owner.
  • Design choice: a 14-day leftover sweep covers only the hosts and orgs recorded when the run started, so hosts and settings operators add later stay theirs.

Risk

Medium. The migration is one-way and touches every store, but it is gated by an env var that is unset by default, and with it unset nothing changes. Under HA the Postgres re-key paths are unit-tested only. Before migrating, size the owner's archive quota: the moved history counts toward it (the run warns when it would be exceeded). Rollback: restore the pre-migration /data snapshot (plus a Postgres dump under HA) and unset the variable. Steps are in docs/multi-tenant-adr.md.

Testing

QA result: PASS on the 7th pass (passes 1–4 and 6 failed, all fixed) — real node server.js boots with name-token, master-token and identity-token hosts; kill -9 right after completion; dry run; bad owner; operator host reusing a migrated name; revoke; ledger, outcome-row and archive re-key; trigger and meetings tokens.
Not tested: HA (Valkey and Postgres), real OIDC owner creation, and a real hub-agent adopting setToken (simulated by heartbeats) — QA has no HA stack or IdP here.

Test evidence
  • cd turma && node --test tests/*.test.js (after merging main): 3586 tests, 0 fail.
  • New suites: tenant-migration.test.js (12, own process: every partitioned store, the gate, the bind beat, rerun no-op, leftovers, reused names, revoke, lost setToken), tenant-migration-runner.test.js (6), tenant-migration-dryrun.test.js (1); XERK-2554: cases in the usage, permission and outcome ledger suites.
  • QA pass history:
    1. FAIL: crash after "done" left unmoved data; leftover sweep took operators' later org settings; dry run created the owner.
    2. FAIL: one-host bind scope split outcome rows; refused-path log flood.
    3. FAIL: an operator host reusing a recorded name had its rows and settings moved.
    4. FAIL: a re-sent setToken left a ghost record.
    5. PASS. The fix-mutations QA re-applied in every pass were caught by the tests.
    6. (after merging main) FAIL: the new archive-quota warning counted only rows movable at once, so it never fired when hosts were in the registry. Migration, admin-panel purge of the owner, and the documented quota fix all confirmed working.
    7. PASS: the warning fires in dry and real runs with hosts in the registry; silent under quota or with no quota.
  • /verify: real boot with the owner set → migration done → name-token host handed setToken → bind beat 200 → operator view no longer lists it.
  • Found while fixing pass 4: uidHostRefusal checked the usage ledger before the tenant-bind exception, so a migrated host whose rows were already moved got 403 on its bind beat. Fixed here.

Follow-ups

  • Main's XERK-2618 charges a pre-partition archive row to its row's tenant, so the moved history counts toward the owner's ARCHIVE_TENANT_MAX_BYTES: fixed here with a warning (dry run too) and an upgrade note; the quota itself is the operator's to size.
  • A typo'd owner keeps the tenant routes closed until fixed (logged hourly): accepted as the safe default, documented.
  • Once an operator's post-move host in a migrated org is removed, that org's plain settings move to the owner on the next sweep: accepted, documented.
  • A host pruned before the run starts that is on a name token returns untenanted (rows moved): accepted residual, documented (roll with ?bind=1).
  • A tenant-bind beat now skips the usage-ledger name check, so a bound host can inherit same-tenant history left by a removed host of its name: accepted (within one tenant, the bind was asked for).

Review focus

The hostMovable / siteMovable / migratingRecord rules in turma/server.js (who counts as "the migrated host"), and the moved tenant-bind line in uidHostRefusal.

A one-shot, resumable boot migration (TURMA_MIGRATE_OWNER) binds every
untenanted host to the owner's tenant (hosts keep their tokens) and
re-keys every partitioned store: org stores, queue, devices, routing
policy, the three ledgers and the archive rows. Tenant users get 503
until it completes; TURMA_MIGRATE_DRY_RUN logs counts per store.
Leftovers follow at each host's bind beat and in a 14-day sweep.
Rollback notes in docs/multi-tenant-adr.md.
Resolves the external-stores test list (keeps both tenantMigration and
adminAudit). Main's XERK-2618 charges pre-partition rows to their row's
tenant quota, so the migrated archive now counts toward the owner's
ARCHIVE_TENANT_MAX_BYTES: the run and the dry run warn when it would
exceed it, and the ADR's upgrade notes say to size it first.
The warning summed only rows movable at once, which skips every host still
in the registry (their rows move at the bind beat) — so a real upgrade never
warned and the owner's pushes were then refused as full (QA pass 6).
@xerhab
xerhab merged commit 518f7df into main Nov 1, 2026
6 checks passed
@xerhab
xerhab deleted the XERK-2554 branch November 1, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant