fix(drive-abci)!: claw a storage refund back from the epochs it was priced for (PV14)

[QuantumExplorer]

Issue being fixed or feature implemented

When data is removed in epoch E, calculate_storage_fee_refund_amount_and_leftovers prices the owner's refund in epoch E: the refund is the data's share of every epoch from E+1 on. The refund is recorded as pending and taken out of the epoch storage pools at the next epoch change. There, add_distribute_storage_fee_to_epochs_operations v0 calls subtract_refunds_from_epoch_credits_collection with the then current epoch, which is E+1 or later. It restores the storage fee as if the refund covered E+2 onward, subtracts per-epoch shares from E+2 on and takes only the leftovers from the current epoch.

The total taken out equals the refund, so the credit sum balances, but the shares land on the wrong pools. The current epoch keeps most of its refunded share, and every later epoch gives back more than its own. When epochs were skipped, the offset grows with the number of skipped epochs.

This is an accounting correction between epoch pools. Measured on a chain that stores a document in epoch 1 and removes it in epoch 3, with the refund taken out at the change into epoch 4 (482 bytes removed, refund 12,964,399 credits, 40 epochs per era):

Epoch pool	Share the refund returned	Before (PV13) takes	After (PV14) takes
4 (current)	16,267	1,023	17,623
5	16,267	16,286	16,266
41	15,616	15,635	15,615
81	14,966	14,983	14,965
1000	5,856	5,863	5,855
2000	406	407	406
5 to 2000 together	12,948,132	12,963,376	12,946,776
All pools	12,964,399	12,964,399	12,964,399

Before, 15,244 credits of the refund came from pools other than the ones it was priced for. After, the only difference is rounding: 1,356 credits (explained below) come from the current epoch instead of a credit each from 1,356 later epochs. With epochs 4 and 5 skipped by a halt (removed in epoch 3, taken out at the change into epoch 6), the misplaced amount was 47,840 credits before and 1,354 after.

What was done?

• add_distribute_storage_fee_to_epochs_operations v1 takes each pending refund back from the epochs it was priced for. Every refund pending at an epoch change was priced in the epoch of the block before it (epoch_info.previous_epoch_index()): a non-change block adds its refunds to the pending ones (fetch_and_add_pending_epoch_refunds_to_collection), while the first block of an epoch keeps only its own (add_delete_pending_epoch_refunds_except_specified_operations, then an insert that replaces the stored values). v1 restores the storage fee from the pricing epoch + 1 and subtracts per epoch from there.
• The shares of epochs between the pricing epoch and the current one, which had no block, are taken from the current epoch, as are the leftovers.
• A refund priced at the end of the storage window keeps the guard from #3799 (start_era >= PERPETUAL_STORAGE_ERAS in refund_storage_fee_to_epochs_map) and is taken whole from the current epoch.
• New dpp helper subtract_refunds_priced_in_epoch_from_epoch_credits_collection (checked arithmetic). A refund whose storage epoch is after its pricing epoch, which removal cannot produce, is taken whole from the current epoch instead of restored. subtract_refunds_from_epoch_credits_collection and its v0 call are unchanged.
• The dispatcher takes previous_epoch_index: Option<EpochIndex>. v1 returns CorruptedCodeExecution if refunds are pending at an epoch change without a previous epoch (only the genesis change has none, and it returns before distributing).
• DRIVE_ABCI_METHOD_VERSIONS_V10 (PV14 only) selects v1, amended in place. PV14 note 47 describes the change.

Rounding: the refund is a sum of floored per-epoch shares, and the fee restored from it through the refunded fraction (restore_original_removed_credits_amount) comes out slightly below the fee it was priced from. So a later epoch gives back its share or one credit less, and the current epoch the rest, under the existing leftover rule. When every share is a whole number of credits and the restored fee is exact, each pool gives back exactly its share (unit tests with a 10,000,000,000 credit fee).

In-place changes to shipped generations

• add_process_epoch_change_operations v0 (selected by every protocol version) now passes epoch_info.previous_epoch_index() to add_distribute_storage_fee_to_epochs_operations. Only v1 reads it, and v1 is selected only by PV14. Up to PV13 the dispatcher selects v0, whose signature and body are unchanged and which never sees the argument, so the operations it produces are byte-identical. The PV13 end-to-end test below pins the old per-epoch amounts.

How Has This Been Tested?

End-to-end tests in block_processing_end_events/tests.rs drive a chain block by block through process_raw_state_transitions and process_block_fees_and_validate_sum_trees (sum trees verified every block). They store documents in epoch S, remove one in epoch E and cross the epoch change, then compare each pool's change with the shares the refund was priced from:

• PV14: (S, E) = (1, 1), (1, 3), (1, 3) with the removal in the epoch's first block, and (2, 45): every pool gives back its share (to within the rounding above), and the total equals the refund.
• PV14 with epochs 4 and 5 skipped: epoch 6 gives back the shares of 4, 5 and 6, and the skipped pools are untouched.
• PV14 at the end of the storage window (removed in epoch 1999, taken out at 2000 and at 2001): the whole refund comes from the current epoch.
• PV13: pins the old distribution (epoch 4 gives back 1,023 of 16,267; 15,244 and 47,840 credits misplaced).

Also:

• v1 unit tests through the dispatcher.
• dpp unit tests for the helper: exact shares, closed epochs, rounding bound, agreement with v0 when priced in the current epoch, end-of-window guard, zero refund, overflow.
• The v0 unit test pinned to PV13.

Ran locally:

• cargo test -p dpp --lib (4,820 passed)
• cargo test -p platform-version
• cargo test -p drive-abci --lib (3,562 passed, before rebasing onto #5011; the fee distribution and block end tests again after)
• strategy tests identity_and_document_tests, core_height_increase and comprehensive_tests (31 tests, including those that pin app hashes and those that delete documents across epoch changes)
• cargo clippy -p dpp -p drive-abci --all-features --all-targets -- -D warnings
• cargo fmt --all -- --check

Breaking Changes

Consensus change at protocol version 14 (unreleased): from PV14 on, an epoch change distributes pending storage refunds over the epoch pools differently, so nodes must agree on the version. Chains at PV13 and earlier are unchanged.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
• I have made corresponding changes to the documentation if needed
• If I added or changed GroveDB structure, I described it in the area's structure.rs, regenerated grovedb-structure.json, and checked the structure viewer link posted on this pull request

For repository code-owners and collaborators only

• I have assigned this pull request to a milestone

🤖 Generated with Claude Code

PR Hygiene · fc1854d

• Bots — coderabbitai not yet · thepastaclaw not yet — /skip-bots proceeds without the ones not yet reported
• Self-review — post /self-reviewed once the bots are done
• Within your 5 open PRs
• Build running
• Approvals — you own every area touched; none needed

When every box is checked the PR Hygiene check passes and this can merge.

[coderabbitai]

Warning

Review limit reached

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: dashpay/platform/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 9ae963b3-34ed-476b-9068-fc891bb878a4

📥 Commits

Reviewing files that changed from the base of the PR and between d23f444 and fc1854d.

📒 Files selected for processing (8)

• packages/rs-dpp/src/fee/epoch/distribution.rs
• packages/rs-drive-abci/src/execution/platform_events/block_processing_end_events/add_process_epoch_change_operations/v0/mod.rs
• packages/rs-drive-abci/src/execution/platform_events/block_processing_end_events/tests.rs
• packages/rs-drive-abci/src/execution/platform_events/fee_pool_inwards_distribution/add_distribute_storage_fee_to_epochs_operations/mod.rs
• packages/rs-drive-abci/src/execution/platform_events/fee_pool_inwards_distribution/add_distribute_storage_fee_to_epochs_operations/v0/mod.rs
• packages/rs-drive-abci/src/execution/platform_events/fee_pool_inwards_distribution/add_distribute_storage_fee_to_epochs_operations/v1/mod.rs
• packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v10.rs
• packages/rs-platform-version/src/version/v14.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thepastaclaw]

🕓 Queued for automated review — 1st in line, estimated start in ~5 min (commit fc1854d)
Estimated review time once started: ~40 min (two-phase automated review; median of recent runs).

• Request priority review — click to move this review to the front of the queue.

[QuantumExplorer]

Reviewed