Skip to content

fix(deps): update Platform to 63cf57f with database compatibility - #981

Merged
lklimek merged 3 commits into
v1.0-devfrom
update-platform-pin
Sep 11, 2026
Merged

lklimek merged 3 commits into
v1.0-devfrom
update-platform-pin

Conversation

@lklimek

@lklimek lklimek commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

TL;DR: Update Platform while keeping existing wallets, identities and preferences readable, with retained backups for recovery.

User story

As a wallet user, I want to upgrade without losing my saved data or having to recreate my profile.

Scenario

Base flow

Open the updated application with a profile created by the previously pinned development version.

Actual behavior

A direct dependency update cannot open the old databases. An identity ownership change can also delete saved identity metadata.

Expected behavior

The application upgrades recognized old databases automatically, verifies their contents before committing, and preserves live identity metadata.

Detailed discussion

What was done

  • Pin all four Platform dependencies to 63cf57f40d0000bf3b2b26026c8fa1c71162852d (4.2.0-dev.8), including the pending contact-crypto/identity-removal serialization fix in fix(platform-wallet): serialize pending crypto writes with identity removal platform#4649.
  • Bridge the old feat(platform-wallet-storage): embeddable SQLite persistence backend with seedless rehydration platform#3968 database lineage for both app and network stores: exact schema/history recognition, named retained backups, staged strict loading, row comparison and atomic reconstruction. Keep the encrypted seed vault unchanged.
  • Preserve identity metadata during ownership promotion, adapt query/error APIs, remove swept transactions from displayed history, and enable the passing single-UTXO Max-send regression.
  • The new pin has the same migration history/schema as e3cd7cf; the existing destination schema guard remains valid.
  • Stabilize network-switch cancellation coverage with controlled startup and a separate fast-completion regression; synchronize UI removal assertions and password-prompt fixture setup with asynchronous completion.

Testing

  • cargo fmt --all
  • Library suite: 2521 passed; UI suite: 329 passed; E2E suites: 11 passed, 69 live-network tests ignored; doctests: 7 passed, 12 ignored.
  • cargo clippy --locked --all-features --all-targets -- -D warnings: passed.
  • Migration coverage includes populated old wallets, balances/identities, app preferences, backups, process interruption, rollback, WAL snapshots, writer exclusion and repeated open. Synthetic data only.

Remaining limitations

The contact-account scan-generation fix from dashpay/platform#4587 remains absent at this pin. The old FFI proof-size gate from dashpay/platform#4585 is also absent, though DET uses the Rust API. Hosts with memory pages larger than 16 KiB are unsupported by the new secret storage. Live-network and live-desktop GUI checks were not run; headless UI integration tests passed.

The dependency audit found the same five pre-existing advisories before and after the upgrade. See the compatibility review for details.

Attribution

🤖 Co-authored by Claudius the Magnificent AI Agent

Pin Platform v4.2-dev at e3cd7cf (4.2.0-dev.8) and adapt wallet APIs.
Automatically bridge the previous PR's database lineage for both app and
network stores with retained backups, strict staged loading and atomic
reconstruction. Preserve live identity metadata during ownership promotion.

Handle swept transaction events and enable the passing Max-send regression.
Document retained functionality, remaining upstream gaps and audit results.

Validation: cargo fmt --all; 2520 library tests passed; exact CI Clippy flags
--all-features --all-targets -- -D warnings passed.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Include the pending contact-crypto persistence and identity-removal
serialization fix. Migration history and schema remain unchanged;
retain the verified database compatibility bridge and other lockfile
resolutions.

Validation: formatter, 2520 library tests, and CI Clippy flags passed
with the locked dependency graph.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 42 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 40ec5601-8331-4622-9116-1c18c5dbaa3e

📥 Commits

Reviewing files that changed from the base of the PR and between e25438b and 4f5fc04.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (35)
  • CHANGELOG.md
  • Cargo.toml
  • docs/ai-design/2026-09-10-platform-pin/upgrade-notes.md
  • src/backend_task/contract.rs
  • src/backend_task/dashpay/contact_requests.rs
  • src/backend_task/document.rs
  • src/backend_task/error.rs
  • src/backend_task/identity/discover_identities.rs
  • src/backend_task/identity/load_identity.rs
  • src/backend_task/identity/load_identity_by_dpns_name.rs
  • src/backend_task/identity/load_identity_from_wallet.rs
  • src/backend_task/identity/refresh_loaded_identities_dpns_names.rs
  • src/backend_task/identity/register_dpns_name.rs
  • src/backend_task/mod.rs
  • src/backend_task/platform_info.rs
  • src/context/identity_db.rs
  • src/context/mod.rs
  • src/context/wallet_lifecycle/bootstrap.rs
  • src/context/wallet_lifecycle/tests.rs
  • src/ui/tokens/view_token_claims_screen.rs
  • src/wallet_backend/dashpay.rs
  • src/wallet_backend/event_bridge.rs
  • src/wallet_backend/identity_ops.rs
  • src/wallet_backend/mod.rs
  • src/wallet_backend/payments.rs
  • src/wallet_backend/platform_compatibility/engine.rs
  • src/wallet_backend/platform_compatibility/engine/tests.rs
  • src/wallet_backend/platform_compatibility/fixtures/67d4ef3.sql
  • src/wallet_backend/platform_compatibility/fixtures/README.md
  • src/wallet_backend/platform_compatibility/fixtures/e3cd7cf.sql
  • src/wallet_backend/platform_compatibility/fixtures/public-rows.sql
  • src/wallet_backend/platform_compatibility/mod.rs
  • src/wallet_backend/snapshot.rs
  • tests/kittest/masternode_tab.rs
  • tests/kittest/secret_prompt.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lklimek
lklimek marked this pull request as ready for review September 10, 2026 17:21
@thepastaclaw

thepastaclaw commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

🕓 Queued for automated review — 12th in line, estimated start in ~2.2 h (commit 4f5fc04)
Estimated review time once started: ~25 min (two-phase automated review; median of recent runs).

  • Request priority review — tick this box and the review moves to the front of the queue.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 2 only (queue backlog)

The compatibility migration is carefully scoped and includes durable backups, staged validation, rollback coverage, and identity metadata preservation. One in-scope architecture issue remains: validation failures erase their concrete error types behind a blanket trait object, weakening the typed error chain used by the application.

🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 2: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); final verifier: gpt-6-astra (agent: astra-verifier, role: final-verifier)

  • Triage: critical by gpt-6-astra (effort low) — The large, intricate diff directly changes storage migration and compatibility logic in src/wallet_backend/platform_compatibility/engine.rs and related context/database paths, a critical storage-migration surface that can affect preservation and recovery of wallet, identity, and preference data.
  • Phase 1 reviewers: not run (skipped for throughput: 12 PRs queued, above the 10 limit)
  • Fresh verifier: gpt-6-astra — final-verifier; agent astra-verifier
  • Phase 2 reviewers: gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `src/wallet_backend/platform_compatibility/engine.rs`:
- [SUGGESTION] src/wallet_backend/platform_compatibility/engine.rs:39-42: Preserve concrete validation error types instead of boxing dyn Error
  `TypedValidation` stores validation failures as `Box<dyn std::error::Error + Send + Sync>`. The caller in `platform_compatibility/mod.rs` wraps failures from staging the database, loading wallet data, and loading unowned identities into this variant, so downstream code can no longer structurally distinguish the originating storage or persistence error. This conflicts with the repository's typed-error convention and makes diagnostics and future error handling depend on downcasting a blanket trait object. Introduce a dedicated validation error enum with concrete source variants for each validation operation, then use that enum as the source of `UpgradeError::TypedValidation` while retaining the existing user-facing message.

Comment on lines +39 to +42
"The updated application could not read your wallet data. Keep your data folder and reopen the previous application version."
)]
TypedValidation(#[source] Box<dyn std::error::Error + Send + Sync>),
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Preserve concrete validation error types instead of boxing dyn Error

TypedValidation stores validation failures as Box<dyn std::error::Error + Send + Sync>. The caller in platform_compatibility/mod.rs wraps failures from staging the database, loading wallet data, and loading unowned identities into this variant, so downstream code can no longer structurally distinguish the originating storage or persistence error. This conflicts with the repository's typed-error convention and makes diagnostics and future error handling depend on downcasting a blanket trait object. Introduce a dedicated validation error enum with concrete source variants for each validation operation, then use that enum as the source of UpgradeError::TypedValidation while retaining the existing user-facing message.

source: gpt-6-astra (phase2-reviewer: rust-quality)

Control the SPV startup future in the cancellation test and separately
verify registration remains queued after a fast network switch. Preserve
the production network-switch behavior.

Wait for the masternode removal result before checking rendered cards,
and register the password-prompt fixture after storage preparation.

Validation: 2521 library tests, 329 UI tests, 11 E2E tests and 7 doctests
passed; formatter and Clippy passed. Network-dependent tests remain ignored.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
@lklimek
lklimek merged commit 768b920 into v1.0-dev Sep 11, 2026
5 checks passed
@lklimek
lklimek deleted the update-platform-pin branch September 11, 2026 08:52
lklimek added a commit that referenced this pull request Sep 21, 2026
* fix(deps): update platform while preserving existing profiles

Pin Platform v4.2-dev at e3cd7cf (4.2.0-dev.8) and adapt wallet APIs.
Automatically bridge the previous PR's database lineage for both app and
network stores with retained backups, strict staged loading and atomic
reconstruction. Preserve live identity metadata during ownership promotion.

Handle swept transaction events and enable the passing Max-send regression.
Document retained functionality, remaining upstream gaps and audit results.

Validation: cargo fmt --all; 2520 library tests passed; exact CI Clippy flags
--all-features --all-targets -- -D warnings passed.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix(deps): pin platform to 63cf57f

Include the pending contact-crypto persistence and identity-removal
serialization fix. Migration history and schema remain unchanged;
retain the verified database compatibility bridge and other lockfile
resolutions.

Validation: formatter, 2520 library tests, and CI Clippy flags passed
with the locked dependency graph.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* docs(migration): add fixture contract, capture recipe and matrix design

Groundwork for the cross-version migration matrix. No code, no fixture
payload: this is the contract every later workstream builds against.

- tests/migration-fixtures/README.md: what a fixture is, the threat model
  for a public-by-design testnet fixture wallet (dedicated, dust-only,
  never the backend-E2E wallet; recovery phrases stay in environment
  variables, the protected wallet's password is a fixed literal matching
  the existing PROTECTED_PASSWORD precedent), packing rules, the GitHub
  Actions artifact storage model and its retention consequences, and a
  field reference for the manifest.
- tests/migration-fixtures/manifest.json: schema v1, seeded with the one
  required baseline (v0.9.3, wallet-identity-dpns, GUI capture). Artifact
  fields are null until the capture workstream packs and uploads it.
- tests/migration-fixtures/profiles/: acceptance checklists for the
  wallet-only and wallet-identity-dpns profiles, with era-specific
  verification. The v0.9.3 DPNS name lives inside the opaque identity.data
  bincode blob, so it is verifiable only in the UI, not by SQL.
- docs/gui-testing/scenarios/migration-fixture-capture.md: the v0.9.3
  capture recipe. That tag has no DASH_EVO_DATA_DIR, so isolation is via
  XDG_CONFIG_HOME, and its own .env.example must be used. It also has no
  SPV stack, so it sees funds only through a local Dash Core node over
  RPC/ZMQ and cannot build a registration's asset lock unaided; the
  identity is registered beforehand with the current build and only loaded
  by ID during capture.
- docs/ai-design/2026-09-10-migration-matrix/design.md: condensed design —
  the three migration systems, PR #981's compatibility bridge and the
  synthetic-fixture gap it names, why v0.9.3 is structurally immune to
  that divergence, the conditional assertion table, phases and sequencing.
- docs/user-stories.md: WAL-033 [Gap] for surviving an app upgrade.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* feat(mcp): add read-only identity_list and app_storage_status tools

The migration-verification harness needs to inspect a profile after an
upgrade without a GUI and without moving the profile forward. Two
read-only tools cover that:

- `identity_list` reports the identities persisted for the active
  network with their DPNS names, balances and wallet bindings. Like
  `core_wallets_list` it hydrates storage but skips the SPV gate: every
  field comes from persisted state, so it reports what is on disk rather
  than what the chain currently says.
- `app_storage_status` reports the stored `data.db` schema version
  against the version this build's ladder targets, the newest upstream
  wallet-storage migration applied to `det-app.sqlite` (version, name and
  checksum — the fingerprint that identifies which schema line a profile
  came from), the live migration state machine, and each network's four
  upgrade completion markers with the timestamp and build version that
  wrote them. It deliberately hydrates nothing and starts no sync, so it
  observes an upgrade instead of triggering one.

The markers are read through the key functions `finish_unwire` already
exports, so there is one definition of each key. The schema-history read
is best-effort: a profile the upstream store has never opened has no
history table, which is an answer rather than a failure.

`DEFAULT_DB_VERSION` was re-exported for tests only; widen it to the MCP
feature set so the tool can name the version it compares against.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* test(migration): add cross-version migration matrix harness

Boots profiles captured by older DET releases against the binary built
from this tree and asserts the upgrade lands them intact.

The harness spawns the real `det-cli` as a subprocess with
`DASH_EVO_DATA_DIR` pointed at a staged copy of the fixture. An
in-process `AppState` boot is not an option: under the `testing`
feature `boot_inputs` substitutes an in-memory database, which would
leave every migration assertion testing nothing.

Assertions are conditional on where the fixture starts, so an
already-current profile is not asserted to have migrated:

- no panic and no `WalletDataIncompatible` on either stream;
- `data.db` reaches `DEFAULT_DB_VERSION` only when it started below it;
  otherwise its schema must be unchanged (strict byte identity is
  opt-in per fixture, since a normal boot legitimately writes rows);
- the active network, wallet aliases and identity ids survive;
- the `finish_unwire` completion sentinel is recorded;
- a second boot adds no backup files and rewrites no sentinel.

Fixtures are multi-megabyte captures kept as CI build artifacts, so the
matrix runs only when a workflow sets `MIGRATION_FIXTURES_DIR`. Unset,
it reports the skip and passes, keeping
`cargo test --all-features --workspace` green; the harness's own unit
tests still run in that path.

`DEFAULT_DB_VERSION` is re-exported from `database` so the assertion
reads the live constant instead of a copy that would go stale.

Left for the parallel workstreams: identity state is read straight out
of the upstream `meta_identity` table until the read-only
`identity_list` / `app_storage_status` MCP tools land.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* docs(migration): record real v0.9.3 fixture capture and its proof-format limit

Wire the manifest entry to the fixture that was actually captured:
- profile is wallet-only (not wallet-identity-dpns) with two real
  wallets (plain + password-protected), sha256 and size recorded,
  archive held at a durable local path pending CI artifact wiring
- known_limitations documents the confirmed reason: v0.9.3's SDK only
  decodes GroveDB proof variant 0, current testnet nodes return
  variant 1, so v0.9.3 cannot load any identity regardless of how or
  when it was registered (a real identity + DPNS name were registered
  on testnet with the current build specifically to test this, and
  the load attempt failed with UnexpectedVariant as documented)
- note the dead TESTNET_dapi_addresses list in v0.9.3's .env.example

Add the capture scenario to the GUI-testing scenario index, flagged as
not an A/B comparison per its own deviation note.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* ci(migration): add migration-matrix workflow, fixture scripts and pin guard

- scripts/migration-fixtures/check-pin-ancestry.sh: verifies the pinned
  platform-wallet(-storage) rev is an ancestor of v4.2-dev AND fingerprints
  the rendered migration SQL (ancestry alone would miss a same-numbered
  migration content change, e.g. the 762c66cf -> 67d4ef3f V001 drift).
  Verified for real: current pin passes against today's v4.2-dev tip.
- pack.sh: packs a data dir into a fixture archive, refuses to pack a
  live MCP_API_KEY, reproducible tar flags.
- capture-headless.sh: drives det-cli through wallet import, SPV wait,
  address creation, discovery, verified against docs/CLI.md's tool set.
- download-fixtures.sh: resolves fixture artifacts via gh run download
  (run-id exact) or a name-pattern fallback, no third-party Action.
- check-coverage.sh: fails the schedule-only run if a published weekly
  newer than the newest manifest fixture has no entry; ENFORCE_AFTER
  defaults to v1.0.0-weekly.20260908 per the plan's narrowed scope.
- .github/workflows/migration-matrix.yml: pin-ancestry -> build -> matrix
  jobs, path-filtered plus weekly schedule and workflow_dispatch. The
  fixture step tolerates "no fixtures found" for now (TODO: make it a
  hard failure once the manifest has real artifact-backed entries).
- .github/workflows/weekly-build.yml: add migration-matrix.yml to the
  existing CI-green gate loop before a weekly can be cut.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* fix(migration): resolve fixture archive by documented manifest key

The harness only recognised `archive`/`file`/`path` for the fixture archive
name, while the manifest and its README schema use `archive_filename`. With
every field defaulted the pointer deserialized to None, so staging fell back
to `<id>.tar.zst` and would miss the archive CI actually downloads.

- Accept `archive_filename` as an alias for `Artifact::archive`.
- Add a regression test that parses the committed manifest via include_str!,
  so reader and manifest key names cannot drift apart unnoticed again.
- Point the v0.9.3 entry's staging_note and the fixtures README at the
  manual migration-fixture-bootstrap.yml upload path (staged separately:
  .github/workflows/** is not agent-writable in this repo).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci(migration): add fixture bootstrap upload and make the matrix fail on missing fixtures

- migration-fixture-bootstrap.yml (new): workflow_dispatch uploads a
  base64-encoded fixture archive as a real GitHub Actions build artifact.
  Name, filename, sha256 and size are pulled from the committed manifest
  entry named by fixture_id, so a dispatch can only publish bytes a
  reviewed manifest already describes. A pull_request trigger on the
  file itself registers the workflow before it reaches the default
  branch, since GitHub only dispatches workflows that already exist on
  a branch or have run at least once.
- migration-matrix.yml: download-fixtures.sh now runs with --strict and
  the `|| true` tolerance is gone; an empty resolved fixture set is a
  hard ::error::+exit 1 instead of a warning. Also fixes two env
  mismatches that would otherwise make --strict fail even with a
  correctly wired fixture: pass MIGRATION_FIXTURES_MANIFEST (the
  harness never received a manifest path) and rename MIGRATION_DET_CLI
  to DET_CLI_BIN (the name the harness actually reads).
- download-fixtures.sh: resolve manifest entries via the documented key
  names (artifact_name/workflow_run_id/workflow_name/sha256) with the
  old ad hoc names kept as a fallback, and verify the downloaded file's
  name against archive_filename.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* test(migration): record v0.9.3 fixture artifact run

Records the real GitHub Actions build artifact produced by dispatching
migration-fixture-bootstrap.yml (run 34579191654): workflow_name,
workflow_run_id, retention_days and expires_at. Drops staging_note,
now that the fixture is resolvable via download-fixtures.sh --strict
instead of only existing as a durable local file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* fix(migration-matrix): stop a test-env leak from clobbering DET_CLI_BIN

`a_bogus_binary_override_is_reported` set DET_CLI_BIN then unconditionally
removed it. cargo test runs every #[test] in this integration-test binary
in one process on a shared thread pool, and `migration_matrix` (main.rs)
also reads DET_CLI_BIN via locate_binary(). The trivial unit test finishes
almost instantly while migration_matrix reaches its own locate_binary()
call much later (after loading the manifest and staging the fixture), so
the CI-provided DET_CLI_BIN was reliably gone by then, not just
occasionally flaky — the migration-matrix workflow's real first run against
the v0.9.3 artifact failed on exactly this ("det-cli was not built
alongside this test"), even though the fixture itself resolved and
verified correctly.

Save the original value and restore it instead of deleting it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* docs(ai-design): add platform_compatibility bridge pattern reference

Standing pattern card for the next platform-wallet-storage pin
divergence: when check-pin-ancestry.sh fires, why checksum/numbering
reconciliation alone doesn't fix a genuine schema divergence, the
bridge's recognize/backup/convert/rebuild mechanism, how to build a
fixture for a new bridge instance (synthetic + real captured profile),
and the per-incident (not general-translator) scope of the pattern.

Distinct from docs/ai-design/2026-09-10-platform-pin/upgrade-notes.md,
which is the point-in-time PR #981 review record.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* fix(mcp): restore the legacy network before the det-cli boot picks one

Standalone det-cli (`init_app_context`) read the active network only from
the AppSettings k/v blob, which does not exist yet on the first launch after
upgrading from a pre-rewrite release, so it fell back to mainnet. The GUI
runs `legacy_settings::import_legacy_settings` before choosing the network
(`AppState::new_inner`); det-cli never did. On a real v0.9.3 testnet data
dir this booted mainnet, ran the one-shot legacy wallet drain against
mainnet (imported=0), and listed no wallets.

Run the same import before the network is read. On import failure the GUI
forces a network chooser; det-cli has no one to ask, so it refuses to boot
instead of guessing, and the unwritten sentinel makes the next launch retry.
A TODO marks the call for the planned "doctor" migration module.

The regression test boots the real `init_app_context` over a v0.9.3-shaped
data dir (serialized by DASH_EVO_DATA_DIR_LOCK) and asserts testnet; it
fails with `Mainnet != Testnet` without the import.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(migration-matrix): assert the read-only data.db contract and umask-proof staging

Since the platform-wallet rewrite (#860) every boot path opens an existing
data.db with SQLITE_OPEN_READ_ONLY; the schema ladder only builds a fresh
file. The harness still demanded that a v0.9.3 data.db climb from version 11
to DEFAULT_DB_VERSION, contradicting the product contract that
v093_upgrade.rs asserts (byte-unchanged).

- check_schema_outcome: an existing data.db must keep its version and
  schema; an absent one must be created at DEFAULT_DB_VERSION.
- Byte identity of an existing data.db is now unconditional; the
  data_db_byte_identical opt-in and its "must be migrated" branch go.
- check_bytes_unchanged names a same-size rewrite correctly.
- The stager restricts every directory between the sandbox root and the
  data dir to 0700. Under a 002 umask `unpacked/` was 0775 and the wallet
  store refused to open (insecure_parent_dir), so the harness passed on CI
  runners (umask 022) but not locally.
- design.md and the fixtures README describe the read-only contract.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(migration-matrix): expect StorageUpdateNeedsDesktop for password-protected wallets

The v0.9.3 baseline holds a password-protected wallet. By design
(docs/ai-design/2026-07-14-migration-password-prompt/design.md) headless
det-cli cannot prompt for its password, so the storage update stops at
StorageUpdateNeedsDesktop { source: InteractivePromptUnavailable } and every
wallet tool stays gated until the desktop app finishes it. The matrix
treated that documented outcome as a failed boot.

Make the outcome explicit and per wallet instead of tolerated:

- manifest.json: contents.wallets[].expected_outcome is `migrated`
  (default) or `needs_desktop`; unknown values fail the parse. Documented
  in tests/migration-fixtures/README.md.
- A fixture with a `needs_desktop` wallet must fail both wallet-gated boots
  with exactly that typed error. Completing, or failing any other way, is a
  regression.
- Per-wallet storage check: the legacy data.db and the upstream store share
  no wallet id, but both keep the BIP44 account-0 xpub (78-byte
  serialization vs base58check text in account_registrations). `migrated`
  wallets must be registered, `needs_desktop` ones must not.
- The wallet-drain completion sentinel must stay unwritten while the update
  is unfinished; address derivation is skipped in that mode.

The full harness now passes against the real v0.9.3 artifact with the
CI-equivalent configuration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(logging): cap rmcp logging so raw MCP tool arguments never reach logs

rmcp 1.7.0 logs every inbound request with its raw tool arguments at DEBUG
(`service.rs`: "received request") and every streamable-HTTP response message
at TRACE. Tool arguments carry secrets -- the core_wallet_import recovery
phrase, masternode private keys -- so any RUST_LOG at debug or finer wrote
them to det.log or stderr.

`logging::sensitive_target_cap()` is a `Targets` global filter layered over
the EnvFilter in every subscriber DET installs (`initialize_logger`, used by
the GUI, the stdio server and headless mode, plus det-cli's own subscriber).
As a separate layer it is ANDed with RUST_LOG, so no directive -- not even
an explicit `rmcp::service=trace` -- can lift the INFO ceiling on `rmcp`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(storage-update): finish a protected-wallet storage update non-interactively

A standalone or headless upgrade of an installation holding a
password-protected wallet could only stop at StorageUpdateNeedsDesktop.
Operators running DET without the desktop app had no way to finish it.

- MCP tool `app_storage_update { password: SecretString, network? }`.
  The password deserializes straight into guarded memory, Debug is redacted,
  the schema is a plain string with no default, and it is borrowed down to
  the drain for one call -- no shared password state
  (`prepare_storage_with_wallet_password` -> `run_gated` ->
  `register_migrated_wallets`).
- The drain tries it on every locked wallet through `handle_wallet_unlocked`
  with `UntilStorageUpdateComplete` -- the boundary the desktop prompt
  submits to -- before the interactive-capability check. A wallet it does not
  open fails the update with the typed `WalletPasswordRejected` /
  `TaskError::StorageUpdatePasswordRejected`: nothing is skipped and the
  sentinel is withheld. Without a password nothing changes:
  `InteractivePromptUnavailable`, at once.
- A desktop session refuses the tool (`McpToolError::DesktopOwnsPasswordPrompt`,
  -32006): no remote channel into the desktop prompt.
- det-cli `--password-stdin` / `--password-file <path>` read the password
  client-side and pass it as the tool's `password` parameter. Refused: a TTY
  on stdin, a file with any group/other permission bit (checked on the opened
  fd), anything but one non-empty line within MAX_PASSPHRASE_LEN, an inline
  `password=...`, and --password-file on non-Unix (TODO: ACL check). A tool
  call is now resolved -- password included -- before connecting.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(migration): boot the v0.9.3 fixture again with its password supplied

The matrix keeps the no-password scenario (the protected wallet must stop
at StorageUpdateNeedsDesktop) and adds the manifest's `password_runs`: the
same archive, freshly staged, finished through
`app-storage-update --password-file <0600 file>`, where both wallets must
migrate, the second boot needs no password, data.db stays byte-identical,
and every listed wallet derives an address.

The password boot logs DET, det-cli and rmcp at trace, and any command
whose output contains the password fails the run. Scenarios are derived
and validated from the manifest: one shared password, known aliases, and
no `needs_desktop` in a password run. The fixture password never appears
in a harness Debug or error either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(storage-update): record the non-interactive password decision

- migration-password-prompt design: the original rationale stays; a new
  "Decision 2026-09-11" section records who decided and why, why stdin or
  an owner-only file and never an env var or argument, why fail-fast still
  holds, the one-password/no-skip and desktop-refusal rules, the rmcp log
  cap, and the residuals.
- jit-secret-access Q-HEADLESS: notes the storage-update exception;
  signing keeps the ruling.
- MCP.md / CLI.md: the `app_storage_update` tool and the det-cli flags;
  WAL-032 gains the operator path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(det-cli): keep wallet passwords off plain HTTP and out of stdin's buffer

Security review follow-up for the non-interactive storage-update path.

- SEC-004: --password-stdin / --password-file are refused in HTTP mode
  unless the address is https or plain http to loopback (127.0.0.0/8,
  ::1, localhost). The check runs before the password is read and
  before any connection is made; 0.0.0.0 and unparsable addresses are
  refused too.
- SEC-002: --password-stdin reads a duplicated raw stdin descriptor as
  a File, bypassing std's process-wide 8 KiB stdin buffer that is never
  wiped. The zeroizing read buffer is again the only user-space copy.
- SEC-001 (desktop check is in-process only) and SEC-003 (no password
  attempt throttling) stay open as TODOs in app_storage_update and are
  recorded as known, accepted limitations in the design doc.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(migration-matrix): skip SPV-gated address derivation for legacy-drained v0.9.3 wallets

CI run 34592733217 timed out in the password scenario's address derivation:
the legacy drain registers both v0.9.3 wallets as WalletOrigin::Imported
(birth height 0) with no committed sync height, and dash-spv starts the
compact-filter download at max(birth_height, committed + 1, header_start).
The first SPV-gated call therefore waits for a scan from genesis, ~1.55M
testnet filters, which a GitHub runner does not finish inside the product's
10-minute SPV gate (~590k filters in that run). Not transient: fixtures ship
no spv/ state, so every run starts from zero.

The v0.9.3 entry opts out with expect.derive_address = false, pinned by the
manifest test. Key integrity stays covered offline: each registration is
matched by its account xpub. Bounding or resuming that scan belongs to the
doctor module, hence TODO(doctor-module) in the harness rather than a longer
wait.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(migration): real-data platform_compatibility fixture from v1.0.0-weekly.20260908

The bridge was proven only on synthetic rows. Add a fixture a released build
wrote: v1.0.0-weekly.20260908 (Platform pin 67d4ef3f), a fresh testnet-only
wallet funded by one 1 tDASH faucet payment, captured via the weekly's own
det-cli plus one GUI launch to persist the testnet selection
(capture_method gui+det-cli). The archive keeps the uncheckpointed WAL the
app's clean shutdown leaves, as a real profile has it.

- real_fixture_tests.rs: env-gated on MIGRATION_FIXTURES_DIR, checksum-bound
  to the manifest. Asserts the plain open fails with a typed migration error,
  the bridge open and typed loads succeed, the wallet keeps its captured
  balance, no row is lost and added tables start empty, exactly one backup
  holds the original rows, and a second open is a no-op. Reads go through
  copies so no check checkpoints the file under test.
- manifest.json: the weekly entry, asserted by alias (this era's data.db has
  no wallet table); artifact pointer pending the draft-release relay upload.
- manifest.rs: pins the entry's shape, including derivation staying on.
- README: gui+det-cli capture method, the release_tag relay, the expect
  table, platform_rev/staging_note, MIGRATION_FIXTURE_WEEKLY_MNEMONIC.
- det-cli: note that process::exit skips the WAL checkpoint.

The recovery phrase stays in the operator's store; the archive is never
committed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci(migration): relay large fixture archives through a draft release and run the real-data bridge test

Bootstrap's workflow_dispatch input is capped at ~48 KB (base64), too
small for a current-era fixture that carries a raw SQLite WAL. Add a
release_tag input: a temporary draft release carries the archive, and
a single contents:write job (fetch-draft-release) downloads it and
hands it to the read-only upload job as a same-run artifact. The
verifying job still checks bytes and sha256 against the committed
manifest entry either way.

migration-matrix.yml gains a step that runs the platform_compatibility
real-data test against the downloaded weekly-20260908 fixture and
fails if it did not actually execute (env-gated tests can otherwise
silently no-op).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* chore(migration-fixtures): record the weekly-20260908 fixture's uploaded artifact

Fills in workflow_run_id/retention_days/expires_at from the successful
migration-fixture-bootstrap.yml dispatch (run 34597906288) that
relayed the archive through a temporary draft release, and drops the
staging_note now that the artifact exists.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017EmWboC3cubxQec5W4dAch

* ci(migration): run the matrix on every PR 983 branch push

Reuse the migration workflow without path or draft filtering for this branch, and pass the fixture password through an Actions secret.

Co-Authored-By: Codex <noreply@openai.com>

* test: harden migration matrix outcomes and password handling

* fix(migration): enforce fixture integrity and credential isolation

* fix(mcp): inspect storage without initialization and report wallet ownership

Co-Authored-By: Codex <noreply@openai.com>

* ci(migration): validate fixture scripts before building

Record the migration test and diagnostic additions and document runtime-only fixture password configuration.

Co-Authored-By: Codex <noreply@openai.com>

* test(migration): use runner-provided grep for script assertions

Co-Authored-By: Codex <noreply@openai.com>

* test(cli): check request read in redirect regression

Co-Authored-By: Codex GPT-6 <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix(migration): address retry and matrix review findings

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix(migration): preserve captured data and secure CLI transport

Persist CLI network selections across processes and verify capture restarts.
Check legacy wallet declarations, pre-existing migration markers, and committed
WAL rows; correct the historical schema inspection instructions.
Disable HTTP proxies for CLI tool requests and cover transport with a real
loopback proxy regression.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix(cli): preserve active context when network settings fail

Persist the selected network before stopping the outgoing backend and swapping
contexts. Shut down the candidate backend if saving fails. Cover the failure
with an injected storage error and an active-context identity assertion.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix(ci): run migration matrix for PR 983

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix(test): update migration fixture RNG import for rand 0.10

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* fix: keep passphrase-limit import in single-key tests

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

* test(migration): cover historical public identities and DPNS

---------

Co-authored-by: Lukasz Klimek <842586+lklimek@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants