Skip to content

feat(platform-wallet-storage): persist DashPay backfill coverage and outbound-account markers - #5479

Open
HashEngineering wants to merge 3 commits into
v5.1-devfrom
feat/sqlite-dashpay-backfill-persistence
Open

HashEngineering wants to merge 3 commits into
v5.1-devfrom
feat/sqlite-dashpay-backfill-persistence

Conversation

@HashEngineering

@HashEngineering HashEngineering commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Basic explanation

What this does: Teaches the SQLite wallet store to save two facts that #5026 added to the wallet's persistence contract: how far the DashPay "backfill" rescan has already got (so a restart does not start it again), and which account version each DashPay contact's outbound payment account was built for (so a restart does not rebuild it).

Value: Wallets on the SQLite backend stop re-walking the same block range on every launch once a DashPay backfill has run — the same fix #5026 gives the Kotlin and Swift hosts. On the reference mainnet wallet that re-walk was ~377,000 filters per launch. Contacts' outbound accounts also stop being torn down and rebuilt on every cold start.

Risks: Low. No consensus impact — this is client-side wallet storage only. The migration only adds nullable columns, so an upgraded database behaves exactly as before until the first backfill is recorded (one more re-run on first launch). The one behaviour change to existing data is that a stored sync cursor can now go down when Rust marks it as a deliberate rewind; ordinary advances still keep the highest value.

Issue being fixed or feature implemented

Follows #5026 (fixes #4302 for the FFI hosts). That PR made the DIP-15 §12.6 coreHeight backfill durable through a new DashPayBackfillRecord that hosts store and hand back on load, plus a synced_height_is_rewind flag the host must honour. Kotlin implements it; the SQLite backend did not:

  • It loaded every wallet with an empty record and kept a monotonic-max synced_height, so on this backend the backfill re-ran on every launch — the Persist DashPay coreHeight backfill completion (rescan_triggered is in-memory only) #4302 behaviour.
  • In FlushMode::Immediate it already qualified for outbound-account markers (ACCOUNT_ADDRESS_POOLS is an alias of the ASSET_LOCK_FUNDING_INDICES it declares), so markers were handed to it — and silently dropped, since no column held them.

What was done?

Migration V022__dashpay_backfill.rs (purely additive, nullable columns only — no new table, no blob)

Backfill record — src/sqlite/schema/dashpay_backfill.rs

  • apply replaces the wallet's record inside the store's transaction, after the round's account registrations: it clears every dashpay_covered_from, sets it on each covered account's row (addressed through the same typed-column helpers the account writer binds), then sets or clears the extent. An empty record only clears (the upstream invalidation).
  • A covered entry whose receiving account has no row is dropped, and if none remain the extent is cleared too. A contact missing from the record is only backfilled again, which is the safe direction.
  • The account upsert only rewrites account_xpub_bytes, so coverage survives an xpub rotation.
  • load_for_wallet joins the columns back through DashPayBackfillRecord::from_entries (canonical order). Coverage without an extent reads as "no record". Wired into load_one_wallet → ClientWalletStartState::dashpay_backfill.
  • SqlitePersister::stores_dashpay_backfill() returns true. Durable coverage still requires FlushMode::Immediate through the existing store_commits_inline gate in persists_backfill_coverage; Manual mode writes and clears records but keeps coverage session-only.
  • New Domain::DashPayBackfill in schema/versions.rs (replaces the explicit dashpay_backfill: _ ignore). Its version label is "dashpay_backfill", like "core" a domain rather than a table name.

Rewind — schema/core_state.rs::upsert_sync_state

  • Restore (rehydrate.rs::apply_persisted_core_state) now assigns the stored synced_height as written instead of maxing it against the birth_height - 1 seed — a backfill rewind can sit below the wallet's birth, and the FFI restore path already assigns it directly.
  • A synced_height marked synced_height_is_rewind overwrites the stored cursor; advances, last_processed_height and chainlock_height keep the monotonic max. A rewind can only lower the tombstone finality boundary (min(chainlock, synced)), so it never collects more.

Outbound-account marker — schema/contacts.rs

  • Written on established rows, restored on load. WalletPersister::store already strips it on backends that cannot keep it, and the manager's load clears restored markers it cannot trust. A trusted marker relies on apply_registrations overwriting the account xpub on rotation, plus the load-side check that distrusts a marker when an external pool holds addresses its current xpub does not derive.

Identity API — ExternalAccountRegistration / RegisterExternalError are re-exported from platform_wallet::wallet::identity, so callers can match register_external_contact_account's outcome by type.

Docs — SCHEMA.md (the new columns on core_sync_state, account_registrations and contacts, the V022 log entry, the rewind exception on core_sync_state); a stale comment in rs-platform-wallet/src/changeset/changeset.rs that said SQLite ignores the rewind flag.

How Has This Been Tested?

cargo test -p platform-wallet-storage --all-features (572 integration + 431 unit, all pass) and cargo clippy -p platform-wallet-storage -p platform-wallet --all-features --all-targets -- -D warnings (clean).

  • tests/integration/sqlite_dashpay_backfill_relaunch.rs (new, end-to-end): the real PlatformWalletManager over a real SqlitePersister, with the file closed and reopened between sessions:

    • Persist DashPay coreHeight backfill completion (rescan_triggered is in-memory only) #4302 regression: the backfill rewinds once (cursor 1000 → 100, the record on disk), the scan climbs to 400, then the relaunch restores 400 and does not rewind again. Control: with the record's columns cleared, the relaunch rewinds again. Mutation-checked against the rewind arm.
    • The outbound-account marker comes back with its DashpayExternalAccount present and the same xpub, so registering it again reports AlreadyExisted.
    • Adds test_support::encrypt_contact_xpub_for_test (test-utils only) so the storage tests can build a contact's encrypted xpub offline.
  • tests/integration/sqlite_dashpay_backfill.rs (new):

    • Immediate vs Manual capability split.
    • The record round-trips across a reopen, is empty when never written, is replaced wholesale, and is cleared by an empty write.
    • Coverage without an extent reads as no record.
    • A covered entry whose receiving account has no row is dropped; dropping all of them clears the record.
    • Coverage survives the receiving account being re-registered with a new xpub.
    • Rewind vs advance on the stored cursor, and the rewound cursor restored beside its record.
    • Manual-mode last-write-wins.
    • Rewind ordering through the Manual buffer, across a failed-flush restore, and across identity re-add segments.
    • V019→V022 upgrade (no marker, no record) and delete-wallet cleanup.
    • Mutation-checked: dropping the rewind arm fails all five cursor tests.
    • A rewound cursor below the wallet's birth restores as written (mutation-checked).
  • sqlite_contacts_keys_rehydration.rs: the outbound-account marker round-trips.

  • sqlite_version_bump.rs, sqlite_schema_pinning.rs, the new domain and the migration fingerprints.

Breaking Changes

None. The schema change is an additive migration; an older binary cannot open a V022 database, which is the existing forward-version gate for every migration.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed
  • If I added or changed GroveDB structure, I described it in the area's structure.rs, regenerated grovedb-structure.json, and checked the structure viewer link posted on this pull request

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Generated with Claude Code

PR Hygiene · 30bcd54

  • Bots — coderabbitai skipped after the window · thepastaclaw requested changes — dismiss the review or push a fix, 1 thread unresolved — resolve it
  • Self-review — address lklimek requested changes, then post /self-reviewed
  • Build green
  • Approvals
    • wallet-storage (packages/rs-platform-wallet-storage/SCHEMA.md, packages/rs-platform-wallet-storage/migrations/V022__dashpay_backfill.rs, packages/rs-platform-wallet-storage/src/sqlite/persister.rs and 12 more) — lklimek
    • rs-platform-wallet (packages/rs-platform-wallet/src/changeset/changeset.rs, packages/rs-platform-wallet/src/test_support.rs, packages/rs-platform-wallet/src/wallet/identity/mod.rs and 1 more) — ZocoLini or llbartekll or romchornyi

When every merge requirement is met, the PR Hygiene check passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: dashpay/platform/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4943ebad-c136-4791-9e16-cb584a2d07c7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: dashpay/platform/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9d9d0762-b488-4607-825d-244fc7c89178
📥 Commits

Reviewing files that changed from the base of the PR and between f282f29 and 88283da.

📒 Files selected for processing: 19
  • packages/rs-platform-wallet-storage/SCHEMA.md
  • packages/rs-platform-wallet-storage/migrations/V019__dashpay_backfill.rs
  • packages/rs-platform-wallet-storage/src/sqlite/persister.rs
  • packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs
  • packages/rs-platform-wallet-storage/src/sqlite/schema/contacts.rs
  • packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs
  • packages/rs-platform-wallet-storage/src/sqlite/schema/dashpay_backfill.rs
  • packages/rs-platform-wallet-storage/src/sqlite/schema/mod.rs
  • packages/rs-platform-wallet-storage/src/sqlite/schema/versions.rs
  • packages/rs-platform-wallet-storage/tests/integration/main.rs
  • packages/rs-platform-wallet-storage/tests/integration/sqlite_contacts_keys_rehydration.rs
  • packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill.rs
  • packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs
  • packages/rs-platform-wallet-storage/tests/integration/sqlite_schema_pinning.rs
  • packages/rs-platform-wallet-storage/tests/integration/sqlite_version_bump.rs
  • packages/rs-platform-wallet/src/changeset/changeset.rs
  • packages/rs-platform-wallet/src/test_support.rs
  • packages/rs-platform-wallet/src/wallet/identity/mod.rs
  • packages/rs-platform-wallet/src/wallet/identity/network/mod.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

SQLite storage now persists DashPay backfill coverage with rewind cursors and restores both after reopening. The V019 migration adds a backfill table and an external account reference column for established contacts. SQLite contact loading and saving now preserve that reference.

Changes: DashPay SQLite persistence

Layer / File(s) Summary
V019 schema and persistence domain
packages/rs-platform-wallet-storage/migrations/V019__dashpay_backfill.rs, packages/rs-platform-wallet-storage/src/sqlite/schema/dashpay_backfill.rs, packages/rs-platform-wallet-storage/src/sqlite/schema/versions.rs, packages/rs-platform-wallet-storage/src/sqlite/schema/mod.rs, packages/rs-platform-wallet-storage/SCHEMA.md, packages/rs-platform-wallet-storage/tests/integration/sqlite_schema_pinning.rs, packages/rs-platform-wallet-storage/tests/integration/sqlite_version_bump.rs
V019 adds the wallet-keyed backfill table and nullable contact marker column. The SQLite persistence domain now tracks backfill changes, including empty records that clear stored data. Schema documentation and migration tests include the additions.
Persist and restore backfill rewinds
packages/rs-platform-wallet-storage/src/sqlite/persister.rs, packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs, packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs, packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill.rs, packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs, packages/rs-platform-wallet-storage/tests/integration/main.rs, packages/rs-platform-wallet/src/changeset/changeset.rs
The persister writes backfill records in the same transaction as changeset data and restores them when loading a wallet. Rewind updates can lower the stored cursor; rehydration restores that cursor directly. Integration tests cover persistence, buffered updates, and relaunch behavior.
Persist contact account references
packages/rs-platform-wallet-storage/src/sqlite/schema/contacts.rs, packages/rs-platform-wallet-storage/tests/integration/sqlite_contacts_keys_rehydration.rs, packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs, packages/rs-platform-wallet/src/wallet/identity/mod.rs, packages/rs-platform-wallet/src/wallet/identity/network/mod.rs, packages/rs-platform-wallet/src/test_support.rs
Established-contact storage now saves and restores the optional external account reference. Reopen tests cover the marker and associated account state. The registration types are publicly re-exported, and test support adds a contact-xpub encryption helper.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant WalletManager
  participant SqlitePersister
  participant SQLiteDatabase
  participant RelaunchedWalletManager
  WalletManager->>SqlitePersister: Submit changeset with rewind and backfill record
  SqlitePersister->>SQLiteDatabase: Commit cursor and backfill record in one transaction
  RelaunchedWalletManager->>SqlitePersister: Load wallet start state
  SqlitePersister->>SQLiteDatabase: Read cursor and backfill record
  SQLiteDatabase-->>SqlitePersister: Return persisted state
  SqlitePersister-->>RelaunchedWalletManager: Restore cursor and backfill record
Loading

Suggested reviewers: thepastaclaw, quantumexplorer

Merge Risk: ⚪ Minimal · up to 88283

No identified issue remains that should delay merging after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 70.97% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 62 functions across 18 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Issues #5026 and #4302 are closed. They provide historical context only. No active directly linked issue supplies coding requirements. The reported SQLite implementation matches the current PR intent …
Out of Scope Changes check Passed The changes stay within the stated SQLite persistence scope. Migration, schema code, restore logic, tests, schema documentation, and the test-only encryption helper support DashPay backfill or outboun…
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the primary changes: persisting DashPay backfill coverage and outbound-account markers in platform wallet storage.
Full details: Linked Issues check

Explanation

Issues #5026 and #4302 are closed. They provide historical context only. No active directly linked issue supplies coding requirements. The reported SQLite implementation matches the current PR intent for backfill persistence, cursor rewinds, and outbound-account markers.

Full details: Out of Scope Changes check

Explanation

The changes stay within the stated SQLite persistence scope. Migration, schema code, restore logic, tests, schema documentation, and the test-only encryption helper support DashPay backfill or outbound-account marker persistence. The public type re-exports support the tested registration behavior. No unrelated change is demonstrated.

Full details: Docstring Coverage

Explanation

Docstring coverage is 70.97% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 62 functions across 18 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw

thepastaclaw commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

⛔ Final review complete — 1 blocking finding(s) (commit 30bcd54) · triage: critical

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

Verified the supplied findings against head b3a06b1. Two blocking persistence defects remain: restoration can advance an explicitly rewound cursor, and the backfill writer accepts records that its reader rejects; the new integration tests also expose a smaller public-API usability issue. This was a static review: current CI checks pass, including Kotlin build/tests, but do not independently confirm the reported Rust storage-suite results.

🔴 2 blocking | 🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 5: gemini-3.8-flash-high (agent: phase1-reviewer, role: general); reviewer 6: gemini-3.8-flash-high (agent: phase1-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: critical by gpt-6.1-sol (effort low) — The change intricately couples durable backfill coverage, cursor rewinds, buffered persistence and restart recovery while changing storage schema in migrations/V019__dashpay_backfill.rs and cursor semantics in core_state.rs::upsert_sync_state.
  • Phase 1 reviewers: gemini-3.8-flash-high — general (completed, effort high); agent phase1-reviewer, gemini-3.8-flash-high — rust-quality (completed, effort high); agent phase1-reviewer
  • Phase 1 model: gemini-3.8-flash-high — antigravity quota: weekly 71% left, 5h 100% left
  • Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs:293-299: Preserve an explicit rewind below the wallet-birth seed
  The claimed clamp does not exist in the pinned dependency: WalletInfoInterface::update_synced_height directly assigns its argument, and reconcile_dashpay_rescan passes the contact checkpoint without clamping it to wallet birth. For example, a wallet with birth_height = 501 can persist a backfill cursor of 100, but reopening seeds synced_height at 500 and this max advances the restored cursor to 500. Because this PR now restores the coverage record too, reconciliation treats the contact as covered and does not rewind again. When the shared scanner has history below this wallet's birth, such as alongside an older wallet, the unscanned range between the saved cursor and 500 is skipped for this wallet. Restore the explicit stored synced_height instead of treating the initialization seed as observed progress, and add a relaunch regression with a checkpoint below a nonzero birth height.

In `packages/rs-platform-wallet-storage/src/sqlite/schema/dashpay_backfill.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/schema/dashpay_backfill.rs:37-42: Reject coverage blobs that the load path cannot read
  The writer inserts record.covered_bytes() without enforcing the 16 MiB limit checked by load_for_wallet at line 75. DashPayBackfillRecord::from_entries does not cap the entry count, V019 only requires a nonempty multiple of 72 bytes, and the connection's SQLite limit is 32 MiB. A record containing 233,017 distinct account keys therefore produces a 16,777,224-byte blob that store can successfully commit, but the next default load fails with BlobTooLarge and aborts restoration; Recovery mode omits the affected wallet. Although this requires a large coverage set, it lets a successful persistence operation make the wallet unrestorable through the normal load path. Check the encoded length before allocating and writing the blob, and add a boundary regression proving rejection preserves both the previous record and its cursor.

In `packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs`:
- [SUGGESTION] packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs:469: Expose registration outcomes instead of asserting Debug strings
  The new test compares the Debug strings "Built" and "AlreadyExisted" because ExternalAccountRegistration is public inside the private network::contacts module but is not re-exported. The public DashPayView method therefore returns an outcome whose variants external callers cannot name, and the integration test works around that API gap by depending on presentation text rather than typed outcomes. Re-export ExternalAccountRegistration and RegisterExternalError from the wallet's public identity API, then use typed assertions here and at the relaunch assertion. This is a non-blocking API-usability improvement, not a layer-ownership violation.
Out-of-scope follow-up suggestions (1)

These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.

  • Track SQLite outbound-account rotation conflicts separately — Both the base and reviewed head reject an outbound-account rebuild when a genuinely rotated xpub supplies different typed pool keys at existing indices: core_pool::apply_pools returns TypedPoolKeyConflict. The rotation teardown removes only in-memory accounts, leaving the conflicting SQLite pool rows in place for subsequent retries. This predates the PR, and the registration transaction rolls back rather than persisting a falsely trusted marker.
    • Follow-up: Track a separate SQLite rotation operation that atomically replaces the old outbound pool with its account registration, preserves immutable-key protections for other account types, and covers rotation plus reopen through the real manager.

Comment thread packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs Outdated
Base automatically changed from fix/dashpay-backfill-persistence-4302 to v5.0-dev October 17, 2026 03:29
@github-actions github-actions Bot added this to the v5.0.0 milestone Oct 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Waiting for bot review — coderabbitai not yet · thepastaclaw requested changes — dismiss the review or push a fix, 2 threads unresolved — resolve them. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added the waiting-bots Waiting for the review bots to report on this head label Oct 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Your move: thepastaclaw requested changes on this head; dismiss the review or push a fix; thepastaclaw left review threads unresolved; resolve them.
Full checklist in the description.

@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed bot-review-skipped A required review bot did not report; it was skipped by the window or by a person. and removed waiting-bots Waiting for the review bots to report on this head labels Oct 17, 2026
@HashEngineering
HashEngineering force-pushed the feat/sqlite-dashpay-backfill-persistence branch from b3a06b1 to 88283da Compare October 17, 2026 17:21
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Waiting for bot review — coderabbitai ✓ · thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added waiting-bots Waiting for the review bots to report on this head and removed waiting-self-review Waiting for the author to post /self-reviewed bot-review-skipped A required review bot did not report; it was skipped by the window or by a person. labels Oct 17, 2026

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

Reviewed the complete diff at 88283da and independently confirmed that all three prior findings are fixed, with no remaining in-scope defects identified. SQLite preserves atomic record/cursor writes, restores deliberate rewinds faithfully, and persists outbound-account markers while leaving wallet policy in rs-platform-wallet. Validation was static locally; current exact-head CI passes scoped Rust wallet tests and clippy, Swift schema/build/tests, and Kotlin build/tests, while the full Rust workspace suite was skipped.

🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 5: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 6: gpt-6-astra (agent: phase2-reviewer, role: architecture-layering); reviewer 7: gpt-6-astra (agent: phase2-reviewer, role: platform-versioning); reviewer 8: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 9: gemini-3.8-flash-high (agent: phase1-reviewer, role: general); reviewer 10: gemini-3.8-flash-high (agent: phase1-reviewer, role: rust-quality); reviewer 11: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 12: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 13: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 14: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: critical by gpt-6.1-sol (effort low) — The change combines a storage migration in migrations/V019__dashpay_backfill.rs with intricate persistence and recovery semantics spanning atomic backfill records, deliberate cursor rewinds, and restart rehydration.
  • Phase 1 reviewers: gemini-3.8-flash-high — general (completed, effort high); agent phase1-reviewer, gemini-3.8-flash-high — rust-quality (completed, effort high); agent phase1-reviewer
  • Phase 1 model: gemini-3.8-flash-high — antigravity quota: weekly 71% left, 5h 100% left
  • Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
  • Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — architecture-layering (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — platform-versioning (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer
  • Model comparison: every Phase-2 reviewer also ran on gpt-6-astra; the verifier weighed both sets without knowing which model wrote which
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.

No unresolved findings remain from the prior review on this head.
Out-of-scope follow-up suggestions (1)

These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.

  • Coordinate explicit below-birth backfills with the SPV scan floor — Both the pinned and current default rust-dashcore revisions derive earliest_required_height from the minimum loaded-wallet birth height, and dash-spv applies that floor during scan startup and restart. With only a wallet born at 501 loaded, correctly restoring its cursor to 100 therefore does not make the scanner traverse heights 101–500, although reconcile_dashpay_rescan can record coverage from that earlier checkpoint. This is a concrete pre-existing wallet/scanner contract limitation, not a defect introduced by SQLite persistence; restoring the stored cursor remains the correct adapter behavior.
    • Follow-up: Track a separate coordinated platform-wallet/rust-dashcore fix that communicates explicit backfill requirements to the scanner and ensures recorded coverage reflects the reachable scan range. Include a real-scanner regression with a contact checkpoint below the only loaded wallet's birth height.

@github-actions

Copy link
Copy Markdown
Contributor

Bots are done — your move: post /self-reviewed.
Full checklist in the description.

@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed and removed waiting-bots Waiting for the review bots to report on this head labels Oct 17, 2026
@HashEngineering

Copy link
Copy Markdown
Collaborator Author

/self-reviewed

@HashEngineering HashEngineering self-assigned this Oct 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Ready for review — wallet-storage: lklimek · rs-platform-wallet: ZocoLini or llbartekll or romchornyi.
Full checklist in the description.

@github-actions github-actions Bot added ready-for-human Bots have reported, the author has self-reviewed, and the build is green: this needs a human. and removed waiting-self-review Waiting for the author to post /self-reviewed labels Oct 17, 2026
Comment thread packages/rs-platform-wallet-storage/migrations/V019__dashpay_backfill.rs Outdated
Comment thread packages/rs-platform-wallet-storage/migrations/V019__dashpay_backfill.rs Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Bots are done — your move: address lklimek requested changes, then post /self-reviewed.
Full checklist in the description.

@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed and removed ready-for-human Bots have reported, the author has self-reviewed, and the build is green: this needs a human. labels Oct 19, 2026
Comment thread packages/rs-platform-wallet-storage/migrations/V019__dashpay_backfill.rs Outdated

@lklimek lklimek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed rs-platform-wallets-storage, some suggestions to the db schema.

HashEngineering and others added 3 commits October 19, 2026 13:01
…outbound-account markers

The SQLite backend loaded every wallet with an empty DashPay backfill
record and kept a monotonic-max cursor, so on this backend the coreHeight
backfill re-ran on every launch (#4302). Outbound-account
markers were handed to it in Immediate mode (ACCOUNT_ADDRESS_POOLS is an
alias of the ASSET_LOCK_FUNDING_INDICES it already declares) but silently
dropped, since no column held them, so each cold start rebuilt every
outbound account.

- V019 adds `dashpay_backfill` (one row per wallet: floor, rewound_from,
  and the covered receival accounts as one `covered_bytes` blob — the
  layout the FFI hosts store) and a nullable
  `contacts.external_account_reference`.
- The record is written in the store's transaction beside the cursor it
  vouches for; every write replaces it whole and an empty record clears it.
  Load hands it back through `ClientWalletStartState::dashpay_backfill`; a
  torn cover set reads as no record, the codec's safe direction.
- `stores_dashpay_backfill()` is true; durable coverage still needs
  `FlushMode::Immediate` via `store_commits_inline`.
- A cursor marked `synced_height_is_rewind` overwrites the stored
  `synced_height`; advances keep the monotonic max. Tests pin the order
  through the Manual-mode buffer, a failed-flush restore, and identity
  re-add segments.
- New `DashPayBackfill` version domain; migration fingerprints bumped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…irth and refuse unreadable cover sets

- Restore the stored `synced_height` as written instead of maxing it
  against the `birth_height - 1` seed. A DashPay backfill rewind can lower
  the cursor below the wallet's birth (`reconcile_dashpay_rescan` passes
  the contact checkpoint to `update_synced_height` unclamped), and the
  restored record vouches for that cursor, so the seed skipped the range
  between them. Matches the FFI restore path; replaces a comment that
  claimed a clamp the pinned key-wallet does not have.
- Refuse, before writing, a backfill record whose encoded cover set
  exceeds the blob limit the loader reads under, so a successful store can
  never leave the wallet unloadable. The round fails whole; the record and
  cursor already on file stand.
- Re-export `ExternalAccountRegistration` and `RegisterExternalError` from
  the identity API so callers (and the relaunch test) can match the
  outcome of `register_external_contact_account` by type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as plain columns

Review asked for values rather than a blob, on the tables that already
hold the facts. The record's extent moves to
core_sync_state.dashpay_backfill_floor / dashpay_backfill_rewound_from,
beside the cursor it lowered, and each covered entry becomes
account_registrations.dashpay_covered_from on the receiving account it
names. The dashpay_backfill table and its 72-byte cover-set blob are
gone, and with them the blob size limit the writer had to enforce.

A covered entry whose receiving account has no row is dropped (re-armed,
the safe direction); when none remain the extent is cleared too. The
account upsert only rewrites the xpub, so coverage survives a rotation.

The migration is renumbered V022: V019 is taken on v5.1-dev and V020 /
V021 are claimed by open PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HashEngineering
HashEngineering force-pushed the feat/sqlite-dashpay-backfill-persistence branch from 88283da to 30bcd54 Compare October 20, 2026 18:00
@HashEngineering
HashEngineering changed the base branch from v5.0-dev to v5.1-dev October 20, 2026 18:00
@github-actions github-actions Bot modified the milestones: v5.0.0, v5.1.0 Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Waiting for bot review — coderabbitai not yet · thepastaclaw requested changes — dismiss the review or push a fix, 1 thread unresolved — resolve it. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added waiting-bots Waiting for the review bots to report on this head and removed waiting-self-review Waiting for the author to post /self-reviewed labels Oct 20, 2026

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

Verified both reviewer sets against head 30bcd54: one blocking consistency issue remains because wallet restoration can combine a pre-rewind cursor with newly committed backfill coverage. All three prior findings are fixed; the identifier-fallback and release-note findings do not establish actionable defects. This was a static review only; current CI still has Rust workspace tests and PR Hygiene pending, and the supplied base-drift evidence reports no conflicting files.

🔴 1 blocking

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 6: muse-spark-1.3-contributor (agent: phase1-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: critical by gpt-6.1-sol (effort low) — The change combines a storage migration in migrations/V022__dashpay_backfill.rs with intricate transactional persistence, rehydration, and deliberate cursor-rewind semantics in schema/dashpay_backfill.rs and related code.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer, muse-spark-1.3-contributor — rust-quality (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (lane failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-platform-wallet-storage/src/sqlite/persister.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/persister.rs:1864-1865: Restore the backfill record and cursor from one SQLite snapshot
  The newly restored coverage suppresses historical rescanning, but `load()` does not open a read transaction. `core_state::load_state` finishes its cursor query before this later coverage read, so another connection can commit between them. The connection mutex and open-path registry exclude only in-process access; normal opens do not enable lifetime EXCLUSIVE locking, and the default journal mode is WAL. For example, with the receiving account and contact already persisted, a reader can capture cursor 1000, then another process can atomically commit rewind 100 plus coverage and exit, after which the reader captures that coverage. A fresh manager retains it, and `reconcile_dashpay_rescan` skips the rewind because the contact is covered, leaving the historical range untested against its addresses and potentially hiding payments. Hold one SQLite read snapshot across reconstruction, including accounts, cursor, contacts, and coverage. A transaction only inside `dashpay_backfill::load_for_wallet` starts too late. Add a two-connection regression that commits the rewind between the cursor and coverage reads.

Comment on lines +1864 to +1865
let dashpay_backfill = schema::dashpay_backfill::load_for_wallet(conn, &wallet_id)
.map_err(PersistenceError::from)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Blocking: Restore the backfill record and cursor from one SQLite snapshot

The newly restored coverage suppresses historical rescanning, but load() does not open a read transaction. core_state::load_state finishes its cursor query before this later coverage read, so another connection can commit between them. The connection mutex and open-path registry exclude only in-process access; normal opens do not enable lifetime EXCLUSIVE locking, and the default journal mode is WAL. For example, with the receiving account and contact already persisted, a reader can capture cursor 1000, then another process can atomically commit rewind 100 plus coverage and exit, after which the reader captures that coverage. A fresh manager retains it, and reconcile_dashpay_rescan skips the rewind because the contact is covered, leaving the historical range untested against its addresses and potentially hiding payments. Hold one SQLite read snapshot across reconstruction, including accounts, cursor, contacts, and coverage. A transaction only inside dashpay_backfill::load_for_wallet starts too late. Add a two-connection regression that commits the rewind between the cursor and coverage reads.

source: gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality)

@github-actions

Copy link
Copy Markdown
Contributor

Your move: thepastaclaw requested changes on this head; dismiss the review or push a fix; thepastaclaw left review threads unresolved; resolve them.
Full checklist in the description.

@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed bot-review-skipped A required review bot did not report; it was skipped by the window or by a person. and removed waiting-bots Waiting for the review bots to report on this head labels Oct 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot-review-skipped A required review bot did not report; it was skipped by the window or by a person. waiting-self-review Waiting for the author to post /self-reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persist DashPay coreHeight backfill completion (rescan_triggered is in-memory only)

3 participants