diff --git a/book/src/contributing/coding-conventions.md b/book/src/contributing/coding-conventions.md index d564bab8f3b..ab5943fe5fd 100644 --- a/book/src/contributing/coding-conventions.md +++ b/book/src/contributing/coding-conventions.md @@ -154,8 +154,8 @@ behaviour-preserving. Add the next `SYSTEM_LIMITS_V{n+1}` for the unreleased protocol version with the new value. Keep a real method version only where the logic differs: in `packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/`, `v0` computes a tiered percentage of total credits and `v2` reads -`daily_withdrawal_limit_percent` and `max_daily_withdrawal_amount` from -`SystemLimits`; that is a logic change and earns its own version. Raising the +`daily_withdrawal_limit_percent` from `SystemLimits`; that is a logic change +and earns its own version. Raising the percentage later would be a table edit, not a `v3`. Why: reviewers look for limits in the tables. A constant hidden in a method diff --git a/book/src/versioning/feature-versions.md b/book/src/versioning/feature-versions.md index f2395089285..7f1d65ac42b 100644 --- a/book/src/versioning/feature-versions.md +++ b/book/src/versioning/feature-versions.md @@ -332,7 +332,10 @@ pub struct SystemLimits { pub max_withdrawal_amount: u64, /// `None` for the protocol versions that predate the relative rule. pub daily_withdrawal_limit_percent: Option, - pub max_daily_withdrawal_amount: Option, + pub core_credit_pool_unlock_limit_percent: Option, + pub core_credit_pool_unlock_limit_floor: Option, + pub core_credit_pool_window_blocks: Option, + pub regtest_core_credit_pool_window_blocks: Option, pub min_withdrawal_amount: u64, pub max_contract_group_size: u16, pub max_token_redemption_cycles: u32, @@ -355,6 +358,7 @@ pub struct DriveAbciWithdrawalConstants { pub core_expiration_blocks: u32, pub cleanup_expired_locks_of_withdrawal_amounts_limit: u16, pub total_credits_history_prune_limit: u16, + pub core_blocks_scanned_per_block_limit: u16, } // drive_abci_versions/drive_abci_validation_versions/mod.rs @@ -405,9 +409,8 @@ the situation the tables exist to prevent, and it leaves a dead module behind every time the number moves. A new method version is warranted only when the *logic* changes. `daily_withdrawal_limit` in `rs-dpp` is the reference case: `v0` derives the limit from the current total credits, `v2` reads -`daily_withdrawal_limit_percent` and `max_daily_withdrawal_amount` from -`SystemLimits`. Raising the percentage later is a `SYSTEM_LIMITS_V5`, not a -`v3`. +`daily_withdrawal_limit_percent` from `SystemLimits`. Raising the percentage +later is a `SYSTEM_LIMITS_V5`, not a `v3`. ## How Subsystem Version Constants Compose diff --git a/book/src/versioning/versioned-dispatch.md b/book/src/versioning/versioned-dispatch.md index 7a59a994be1..3031863e99e 100644 --- a/book/src/versioning/versioned-dispatch.md +++ b/book/src/versioning/versioned-dispatch.md @@ -705,8 +705,11 @@ The Drive helpers that build the initial state structure follow the same rule for the same reason: they run once, at chain creation, under the chain's initial protocol version, and a chain that already exists gets the same trees from its upgrade rung. `add_initial_withdrawal_state_structure_operations` -adds the withdrawal sum trees behind `>= 4` and the credit history trees behind -`>= 14`; replaying mainnet's genesis at protocol version 1 takes neither branch. +adds the withdrawal sum trees behind `>= 4`, which replaying mainnet's genesis at +protocol version 1 does not take. The withdrawal limit trees of protocol version +14 are not in that batch: genesis and `transition_to_version_14` both add them +one insert at a time through `Drive::insert_withdrawal_limit_trees`, so both +build the withdrawals Merk in the same shape. ## Rules diff --git a/packages/dashmate/configs/defaults/getBaseConfigFactory.js b/packages/dashmate/configs/defaults/getBaseConfigFactory.js index 54ac9eade1e..3722703b6fc 100644 --- a/packages/dashmate/configs/defaults/getBaseConfigFactory.js +++ b/packages/dashmate/configs/defaults/getBaseConfigFactory.js @@ -94,6 +94,7 @@ export default function getBaseConfigFactory() { 'getbestchainlock', 'getblockchaininfo', 'getrawtransaction', 'submitchainlock', 'verifychainlock', 'protxlistdiff', 'quorumlistextended', 'quoruminfo', 'getassetunlockstatuses', 'sendrawtransaction', 'mnsyncstatus', 'getblockheader', 'getblockhash', + 'getspecialtxes', ], lowPriority: false, }, diff --git a/packages/dashmate/configs/getConfigFileMigrationsFactory.js b/packages/dashmate/configs/getConfigFileMigrationsFactory.js index e1d07a74f27..bba15f5f96c 100644 --- a/packages/dashmate/configs/getConfigFileMigrationsFactory.js +++ b/packages/dashmate/configs/getConfigFileMigrationsFactory.js @@ -1777,6 +1777,20 @@ export default function getConfigFileMigrationsFactory(homeDir, defaultConfigs) return configFile; }, + '5.0.0-beta.2': (configFile) => { + Object.entries(configFile.configs) + .forEach(([, options]) => { + // Drive's withdrawal limit (protocol version 14) reads Core's credit pool + // balance from each block's coinbase (getspecialtxes). Core refuses it to the + // consensus user until its whitelist names it, and loads the whitelist only + // when Core itself restarts. + if (options.core?.rpc?.users?.drive_consensus) { + options.core.rpc.users.drive_consensus.whitelist = base.getStored('core.rpc.users.drive_consensus.whitelist'); + } + }); + + return configFile; + }, }; } diff --git a/packages/dashmate/test/unit/config/configFile/tenderdashImageMigration.spec.js b/packages/dashmate/test/unit/config/configFile/tenderdashImageMigration.spec.js index cce57d9a980..c93aa55fa9e 100644 --- a/packages/dashmate/test/unit/config/configFile/tenderdashImageMigration.spec.js +++ b/packages/dashmate/test/unit/config/configFile/tenderdashImageMigration.spec.js @@ -2,6 +2,7 @@ import fs from 'fs'; import path from 'path'; import getBaseConfigFactory from '../../../../configs/defaults/getBaseConfigFactory.js'; import getConfigFileMigrationsFactory from '../../../../configs/getConfigFileMigrationsFactory.js'; +import getConfigFormatVersion from '../../../../src/config/configFile/getConfigFormatVersion.js'; import migrateConfigFileFactory from '../../../../src/config/configFile/migrateConfigFileFactory.js'; import { PACKAGE_ROOT_DIR } from '../../../../src/constants.js'; @@ -38,7 +39,10 @@ describe('Tenderdash image migration', () => { expect(migrated.configs.withoutDocker).to.deep.equal({ platform: { drive: { tenderdash: {} } }, }); - expect(migrated.configFormatVersion).to.equal(version); + // Stamped with the format this build produces: the newest migration while it is ahead + // of the package version, the package version once a release has passed it. + expect(migrated.configFormatVersion) + .to.equal(getConfigFormatVersion(getMigrations(), version)); expect(migrateConfigFile(migrated, migrated.configFormatVersion, version)).to.equal(migrated); }); } diff --git a/packages/rs-dpp/src/withdrawal/core_credit_pool_unlock_limit/mod.rs b/packages/rs-dpp/src/withdrawal/core_credit_pool_unlock_limit/mod.rs new file mode 100644 index 00000000000..e5b40f96c55 --- /dev/null +++ b/packages/rs-dpp/src/withdrawal/core_credit_pool_unlock_limit/mod.rs @@ -0,0 +1,119 @@ +use crate::fee::Credits; +use crate::ProtocolError; +use dashcore::Network; +use platform_version::version::PlatformVersion; + +mod v0; + +/// Core's credit pool window on `network`: how many Core blocks before an asset unlock's block +/// lies the balance Core v24 measures the unlock limit from (`CreditPoolPeriodBlocks` in Dash +/// Core's chain parameters), as the protocol version's system limits pin it +/// (`core_credit_pool_window_blocks`, `regtest_core_credit_pool_window_blocks` on regtest). +/// +/// # Errors +/// +/// `ProtocolError::CorruptedCodeExecution` when the protocol version predates the +/// Core-anchored withdrawal limit and sets no window. +pub fn core_credit_pool_window_blocks( + network: Network, + platform_version: &PlatformVersion, +) -> Result { + let system_limits = &platform_version.system_limits; + let window_blocks = match network { + Network::Mainnet | Network::Testnet | Network::Devnet => { + system_limits.core_credit_pool_window_blocks + } + Network::Regtest => system_limits.regtest_core_credit_pool_window_blocks, + }; + window_blocks.ok_or_else(|| { + ProtocolError::CorruptedCodeExecution( + "the protocol version sets no Core credit pool window".to_string(), + ) + }) +} + +/// Returns how much Core's credit pool may still give up to asset unlocks, given its balance +/// now and its balance at the start of the window the limit is measured over, both in credits. +/// +/// This is the Core-anchored half of the withdrawal limit: a stricter copy of Core v24's own +/// asset unlock rule, so Platform never pools a withdrawal Core will refuse to mine. The pool +/// may end no lower than its window start balance minus an allowed drop +/// (`core_credit_pool_unlock_limit_percent` of that balance, at least +/// `core_credit_pool_unlock_limit_floor`); what it gained since the window start (asset locks, +/// the per-block Platform reward) is withdrawable on top, and the pool can never go negative. +/// +/// # Parameters +/// +/// * `balance`: Core's credit pool balance now, in credits. +/// * `window_start_balance`: Core's credit pool balance at the window start, in credits; `0` +/// when that block has no credit pool. +/// * `platform_version`: The platform version. +/// +/// # Returns +/// +/// * `Ok(Credits)`: The credits that may still be unlocked, between `0` and `balance`. +/// * `Err(ProtocolError)`: When the method version is unknown or not active, or the system +/// limits it reads are not configured. +pub fn core_credit_pool_unlock_limit( + balance: Credits, + window_start_balance: Credits, + platform_version: &PlatformVersion, +) -> Result { + match platform_version.dpp.methods.core_credit_pool_unlock_limit { + Some(0) => { + v0::core_credit_pool_unlock_limit_v0(balance, window_start_balance, platform_version) + } + Some(version) => Err(ProtocolError::UnknownVersionMismatch { + method: "core_credit_pool_unlock_limit".to_string(), + known_versions: vec![0], + received: version, + }), + None => Err(ProtocolError::UnknownVersionError( + "core_credit_pool_unlock_limit is not active in this protocol version".to_string(), + )), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::dash_to_credits; + + #[test] + fn should_use_cores_window_of_each_network() { + let v14 = PlatformVersion::get(14).expect("expected protocol version 14"); + for (network, window_blocks) in [ + (Network::Mainnet, 576), + (Network::Testnet, 576), + (Network::Devnet, 576), + (Network::Regtest, 100), + ] { + assert_eq!( + core_credit_pool_window_blocks(network, v14).expect("expected the window"), + window_blocks + ); + } + + let v13 = PlatformVersion::get(13).expect("expected protocol version 13"); + assert!(core_credit_pool_window_blocks(Network::Mainnet, v13).is_err()); + } + + #[test] + fn should_only_exist_from_protocol_version_14() { + let v13 = PlatformVersion::get(13).expect("expected protocol version 13"); + assert!(core_credit_pool_unlock_limit( + dash_to_credits!(10000), + dash_to_credits!(10000), + v13 + ) + .is_err()); + + let v14 = PlatformVersion::get(14).expect("expected protocol version 14"); + // 15% of a 20,000 Dash pool that has not moved. + assert_eq!( + core_credit_pool_unlock_limit(dash_to_credits!(20000), dash_to_credits!(20000), v14) + .expect("expected the limit"), + dash_to_credits!(3000) + ); + } +} diff --git a/packages/rs-dpp/src/withdrawal/core_credit_pool_unlock_limit/v0/mod.rs b/packages/rs-dpp/src/withdrawal/core_credit_pool_unlock_limit/v0/mod.rs new file mode 100644 index 00000000000..c6d9a953469 --- /dev/null +++ b/packages/rs-dpp/src/withdrawal/core_credit_pool_unlock_limit/v0/mod.rs @@ -0,0 +1,167 @@ +use crate::fee::Credits; +use crate::ProtocolError; +use platform_version::version::PlatformVersion; + +/// The pool may not end below `window_start_balance - allowed_drop`, where +/// `allowed_drop = max(window_start_balance * percent / 100, floor)`; everything above that +/// line is withdrawable, but never more than the pool holds: +/// +/// `limit = min(max(0, allowed_drop - (window_start_balance - balance)), balance)` +/// +/// Core v24 applies the same shape with 20% and a 2000 Dash floor over its window (576 blocks, +/// 100 on regtest); the system limits of protocol version 14 set a lower percent and floor, and +/// the caller picks the highest balance among the window starts Core may use, so the result +/// never exceeds what Core admits. Integer arithmetic in u128 throughout; truncation only ever makes +/// the limit stricter. +pub(super) fn core_credit_pool_unlock_limit_v0( + balance: Credits, + window_start_balance: Credits, + platform_version: &PlatformVersion, +) -> Result { + let percent = platform_version + .system_limits + .core_credit_pool_unlock_limit_percent + .ok_or_else(|| { + ProtocolError::CorruptedCodeExecution( + "core_credit_pool_unlock_limit v0 requires system_limits.core_credit_pool_unlock_limit_percent" + .to_string(), + ) + })?; + + let floor = platform_version + .system_limits + .core_credit_pool_unlock_limit_floor + .ok_or_else(|| { + ProtocolError::CorruptedCodeExecution( + "core_credit_pool_unlock_limit v0 requires system_limits.core_credit_pool_unlock_limit_floor" + .to_string(), + ) + })?; + + let allowed_drop = + ((window_start_balance as u128) * (percent as u128) / 100).max(floor as u128); + + // What may leave: the allowed drop plus whatever the pool gained since the window start, + // or minus whatever it already lost. u128 holds the sum of any two u64 values. + let withdrawable = + (allowed_drop + balance as u128).saturating_sub(window_start_balance as u128); + + let limit = withdrawable.min(balance as u128); + + Credits::try_from(limit).map_err(|_| ProtocolError::Overflow("core credit pool unlock limit")) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::dash_to_credits; + + fn limit(balance: Credits, window_start_balance: Credits) -> Credits { + core_credit_pool_unlock_limit_v0(balance, window_start_balance, PlatformVersion::latest()) + .expect("expected the limit") + } + + #[test] + fn should_allow_the_percent_of_an_unchanged_pool() { + // 15% of 37,000 Dash, the mainnet pool #7712 measured. + assert_eq!( + limit(dash_to_credits!(37000), dash_to_credits!(37000)), + dash_to_credits!(5550) + ); + } + + #[test] + fn should_apply_the_floor_to_a_small_pool() { + // 15% of 5,000 Dash is 750 Dash, below the 1,500 Dash floor. + assert_eq!( + limit(dash_to_credits!(5000), dash_to_credits!(5000)), + dash_to_credits!(1500) + ); + } + + #[test] + fn should_add_what_the_pool_gained_inside_the_window() { + // A 4,000 Dash deposit inside the window is withdrawable on top of the allowed drop. + assert_eq!( + limit(dash_to_credits!(41000), dash_to_credits!(37000)), + dash_to_credits!(9550) + ); + } + + #[test] + fn should_subtract_what_the_pool_already_lost_inside_the_window() { + // 2,000 Dash already unlocked inside the window leaves 3,550 of the 5,550 allowed. + assert_eq!( + limit(dash_to_credits!(35000), dash_to_credits!(37000)), + dash_to_credits!(3550) + ); + // Once the drop reaches the allowance nothing is left, and it never goes negative. + assert_eq!(limit(dash_to_credits!(31450), dash_to_credits!(37000)), 0); + assert_eq!(limit(dash_to_credits!(20000), dash_to_credits!(37000)), 0); + } + + #[test] + fn should_never_exceed_the_pool() { + // A pool that grew from nothing inside the window: everything in it is withdrawable, + // but no more than it holds. + assert_eq!(limit(dash_to_credits!(1000), 0), dash_to_credits!(1000)); + assert_eq!(limit(0, 0), 0); + } + + #[test] + fn should_stay_below_cores_own_v24_limit() { + // Core v24: max(20% of the window start, 2000 Dash) - (window start - balance), at most + // the balance, over the same window. + fn core_v24(balance: Credits, window_start_balance: Credits) -> Credits { + let allowed_drop = (window_start_balance / 5).max(dash_to_credits!(2000)); + (allowed_drop + balance) + .saturating_sub(window_start_balance) + .min(balance) + } + + for (balance, window_start_balance) in [ + (dash_to_credits!(37000), dash_to_credits!(37000)), + (dash_to_credits!(41000), dash_to_credits!(37000)), + (dash_to_credits!(35000), dash_to_credits!(37000)), + (dash_to_credits!(5000), dash_to_credits!(5000)), + (dash_to_credits!(12000), dash_to_credits!(11000)), + (dash_to_credits!(1000), 0), + ] { + assert!( + limit(balance, window_start_balance) <= core_v24(balance, window_start_balance), + "balance {balance}, window start {window_start_balance}" + ); + } + } + + #[test] + fn should_not_overflow_at_the_largest_balances() { + assert_eq!( + limit(Credits::MAX, Credits::MAX), + ((Credits::MAX as u128) * 15 / 100) as Credits + ); + assert_eq!(limit(Credits::MAX, 0), Credits::MAX); + assert_eq!(limit(0, Credits::MAX), 0); + } + + #[test] + fn should_fail_when_the_limits_are_not_configured() { + let mut platform_version = PlatformVersion::latest().clone(); + platform_version + .system_limits + .core_credit_pool_unlock_limit_percent = None; + assert!(matches!( + core_credit_pool_unlock_limit_v0(1, 1, &platform_version), + Err(ProtocolError::CorruptedCodeExecution(_)) + )); + + let mut platform_version = PlatformVersion::latest().clone(); + platform_version + .system_limits + .core_credit_pool_unlock_limit_floor = None; + assert!(matches!( + core_credit_pool_unlock_limit_v0(1, 1, &platform_version), + Err(ProtocolError::CorruptedCodeExecution(_)) + )); + } +} diff --git a/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/mod.rs b/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/mod.rs index a4a777cd499..aea6979bf92 100644 --- a/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/mod.rs +++ b/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/mod.rs @@ -14,8 +14,8 @@ mod v2; /// total credits in Platform for version 0 (10% of it, bounded; required), ignored /// by version 1 (a flat 2000 Dash), and the total credits Platform held a day ago /// for version 2 (`daily_withdrawal_limit_percent` of it, never below one maximal -/// withdrawal nor above `max_daily_withdrawal_amount`; the flat limit of version 1 -/// while that day-old total is not known yet). +/// withdrawal; the flat limit of version 1 while that day-old total is not known +/// yet). pub fn daily_withdrawal_limit( reference_total_credits: Option, platform_version: &PlatformVersion, @@ -52,9 +52,9 @@ mod tests { (dash_to_credits!(50), dash_to_credits!(500)), (dash_to_credits!(2000), dash_to_credits!(500)), (dash_to_credits!(20000), dash_to_credits!(3000)), - // Above Core's unlock capacity per day (4000 Dash) the limit is capped there. - (dash_to_credits!(30000), dash_to_credits!(4000)), - (dash_to_credits!(1000000), dash_to_credits!(4000)), + // No fixed cap: what Core will mine is bounded by the Core-anchored limit instead. + (dash_to_credits!(30000), dash_to_credits!(4500)), + (dash_to_credits!(1000000), dash_to_credits!(150000)), ] { // v13 keeps the flat 2000 Dash whatever the total is. assert_eq!( diff --git a/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/v2/mod.rs b/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/v2/mod.rs index 1ae243bd992..ba842a67d92 100644 --- a/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/v2/mod.rs +++ b/packages/rs-dpp/src/withdrawal/daily_withdrawal_limit/v2/mod.rs @@ -8,17 +8,18 @@ use platform_version::version::PlatformVersion; /// Using a day-old base means a sudden jump in the total credits does not raise /// the limit for a day. /// -/// Three guards keep it usable: +/// Two guards keep it usable: /// * it is never below `max_withdrawal_amount`, so every withdrawal Platform /// accepts eventually fits the daily maximum and cannot block the pooling /// queue behind it; -/// * it is never above `max_daily_withdrawal_amount`, Core's credit-pool unlock -/// capacity per day: pooling more than Core will mine only cycles those -/// unlocks through expiry and re-signing; /// * while the total credits a day ago are not known (`None`: the history is /// younger than a day, i.e. right after this rule activates), the flat limit /// of version 1 applies, so the lag cannot be skipped by inflating the total /// before or at activation. +/// +/// There is no fixed upper bound: pooling more than Core will mine is prevented +/// by the Core-anchored limit pooling also applies, which follows Core's own +/// credit pool rather than a fixed figure. pub fn daily_withdrawal_limit_v2( total_credits_in_platform_a_day_ago: Option, platform_version: &PlatformVersion, @@ -37,33 +38,12 @@ pub fn daily_withdrawal_limit_v2( ) })?; - let max_daily_withdrawal_amount = platform_version - .system_limits - .max_daily_withdrawal_amount - .ok_or_else(|| { - ProtocolError::CorruptedCodeExecution( - "daily_withdrawal_limit v2 requires system_limits.max_daily_withdrawal_amount" - .to_string(), - ) - })?; - - let max_withdrawal_amount = platform_version.system_limits.max_withdrawal_amount; - if max_daily_withdrawal_amount < max_withdrawal_amount { - // A cap below one maximal withdrawal would let an accepted withdrawal never fit the - // daily maximum; that is a contradictory configuration, not a limit to apply. - return Err(ProtocolError::CorruptedCodeExecution(format!( - "daily_withdrawal_limit v2 requires system_limits.max_daily_withdrawal_amount ({max_daily_withdrawal_amount}) to be at least max_withdrawal_amount ({max_withdrawal_amount})" - ))); - } - // u128 keeps `total * percent` from overflowing for any u64 total. let relative_limit = (total_credits_a_day_ago as u128) * (percent as u128) / 100; let relative_limit = Credits::try_from(relative_limit) .map_err(|_| ProtocolError::Overflow("daily withdrawal limit overflow"))?; - Ok(relative_limit - .max(max_withdrawal_amount) - .min(max_daily_withdrawal_amount)) + Ok(relative_limit.max(platform_version.system_limits.max_withdrawal_amount)) } #[cfg(test)] @@ -77,7 +57,6 @@ mod tests { .system_limits .daily_withdrawal_limit_percent = percent; platform_version.system_limits.max_withdrawal_amount = dash_to_credits!(500); - platform_version.system_limits.max_daily_withdrawal_amount = Some(dash_to_credits!(4000)); platform_version } @@ -121,25 +100,18 @@ mod tests { } #[test] - fn should_never_exceed_cores_unlock_capacity_per_day() { + fn should_not_cap_a_large_lagged_total() { let platform_version = platform_version_with(Some(15)); - // 15% of 30000 Dash is 4500 Dash, above what Core mines per day. assert_eq!( daily_withdrawal_limit_v2(Some(dash_to_credits!(30000)), &platform_version) .expect("expected limit"), - dash_to_credits!(4000) + dash_to_credits!(4500) ); assert_eq!( daily_withdrawal_limit_v2(Some(Credits::MAX), &platform_version) .expect("expected limit"), - dash_to_credits!(4000) - ); - // Just under the boundary the percent still applies. - assert_eq!( - daily_withdrawal_limit_v2(Some(dash_to_credits!(26666)), &platform_version) - .expect("expected limit"), - dash_to_credits!(3999.9) + ((Credits::MAX as u128) * 15 / 100) as Credits ); } @@ -154,41 +126,11 @@ mod tests { } #[test] - fn should_fail_when_the_percent_or_the_cap_is_not_configured() { + fn should_fail_when_the_percent_is_not_configured() { let platform_version = platform_version_with(None); assert!(matches!( daily_withdrawal_limit_v2(Some(dash_to_credits!(100)), &platform_version), Err(ProtocolError::CorruptedCodeExecution(_)) )); - - let mut platform_version = platform_version_with(Some(15)); - platform_version.system_limits.max_daily_withdrawal_amount = None; - assert!(matches!( - daily_withdrawal_limit_v2(Some(dash_to_credits!(100)), &platform_version), - Err(ProtocolError::CorruptedCodeExecution(_)) - )); - } - - #[test] - fn should_fail_when_the_cap_is_below_one_maximal_withdrawal() { - let mut platform_version = platform_version_with(Some(15)); - platform_version.system_limits.max_daily_withdrawal_amount = - Some(dash_to_credits!(500) - 1); - - // Whatever the total, a cap below the floor is a contradictory configuration. - for total in [0, dash_to_credits!(100), dash_to_credits!(30000)] { - assert!(matches!( - daily_withdrawal_limit_v2(Some(total), &platform_version), - Err(ProtocolError::CorruptedCodeExecution(_)) - )); - } - - // Exactly the floor is allowed and the limit is that floor. - platform_version.system_limits.max_daily_withdrawal_amount = Some(dash_to_credits!(500)); - assert_eq!( - daily_withdrawal_limit_v2(Some(dash_to_credits!(30000)), &platform_version) - .expect("expected limit"), - dash_to_credits!(500) - ); } } diff --git a/packages/rs-dpp/src/withdrawal/mod.rs b/packages/rs-dpp/src/withdrawal/mod.rs index 7eb217edbe7..b33fc05f110 100644 --- a/packages/rs-dpp/src/withdrawal/mod.rs +++ b/packages/rs-dpp/src/withdrawal/mod.rs @@ -1,3 +1,4 @@ +pub mod core_credit_pool_unlock_limit; mod core_dust_threshold; pub mod daily_withdrawal_limit; #[cfg(all(feature = "withdrawals-contract", feature = "system_contracts"))] diff --git a/packages/rs-drive-abci/src/core/wait_for_core_to_sync/v0/mod.rs b/packages/rs-drive-abci/src/core/wait_for_core_to_sync/v0/mod.rs index d1f79b1e55d..d9b75631965 100644 --- a/packages/rs-drive-abci/src/core/wait_for_core_to_sync/v0/mod.rs +++ b/packages/rs-drive-abci/src/core/wait_for_core_to_sync/v0/mod.rs @@ -7,7 +7,8 @@ use std::time::Duration; const CORE_SYNC_STATUS_CHECK_TIMEOUT: Duration = Duration::from_secs(5); -/// Blocks execution until Core is synced +/// Blocks execution until Core is synced, then checks that Core answers the credit pool balance +/// read block execution needs from protocol version 14. /// This isn't in consensus, however we still version it just in case we will upgrade it on a /// version pub fn wait_for_core_to_sync_v0( @@ -42,5 +43,23 @@ pub fn wait_for_core_to_sync_v0( } } + if cancel.is_cancelled() { + return Ok(()); + } + + // From protocol version 14 every block reads Core's credit pool balance from a chain locked + // block's coinbase (`getspecialtxes`). Core loads the consensus user's `rpcwhitelist` only + // when Core itself starts, so a Core not restarted since its whitelist gained the method + // answers every other call until that version activates and refuses every block after. + // Ask once here, so such a node fails to start instead. + let chain_lock = core_rpc.get_best_chain_lock()?; + if let Err(error) = core_rpc.get_credit_pool_balance(chain_lock.block_height) { + tracing::error!( + ?error, + "core cannot read the credit pool balance (getspecialtxes); restart core so it loads the rpc whitelist of drive's consensus user" + ); + return Err(error.into()); + } + Ok(()) } diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs index a6661af0cfc..04c67e62192 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs @@ -19,8 +19,7 @@ use drive::drive::identity::key::fetch::{ IdentityKeysRequest, KeyIDIdentityPublicKeyPairBTreeMap, KeyRequestType, }; use drive::drive::identity::withdrawals::paths::{ - get_withdrawal_root_path, WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, - WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, WITHDRAWAL_TRANSACTIONS_BROADCASTED_KEY, + get_withdrawal_root_path, WITHDRAWAL_TRANSACTIONS_BROADCASTED_KEY, WITHDRAWAL_TRANSACTIONS_SUM_AMOUNT_TREE_KEY, }; use drive::drive::prefunded_specialized_balances::prefunded_specialized_balances_for_voting_path_vec; @@ -776,29 +775,6 @@ impl Platform { platform_version, )?; - // Total credits history under the withdrawals tree: the daily withdrawal limit becomes - // a share of the total credits Platform held a day ago, recorded here every block. - self.drive.grove_insert_if_not_exists( - get_withdrawal_root_path().as_slice().into(), - &WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, - Element::empty_tree(), - Some(transaction), - None, - &platform_version.drive, - )?; - - // Credit inflows sum tree: every credit mint is recorded here so the daily withdrawal - // limit counts net outflow instead of gross — credits that entered Platform within the - // window may leave again without consuming the withdrawal budget of other users. - self.drive.grove_insert_if_not_exists( - get_withdrawal_root_path().as_slice().into(), - &WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, - Element::empty_sum_tree(), - Some(transaction), - None, - &platform_version.drive, - )?; - // Contract version items: from this version the storage writer stores every // contract's version as a four-byte item beside it, and // `getDataContractsLatestVersions` reads and proves that item instead of the @@ -859,10 +835,21 @@ impl Platform { // indexes every document of a type declaring a `ttl` (a keyword protocol version 14 // introduces) by when it expires, and the lifetime storage fee pools sum tree under // `Pools`, which holds their storage fees until an epoch change spreads them. Fresh - // chains call the same helper last in `create_initial_state_structure` v4. + // chains call the same helper in `create_initial_state_structure` v4, in the same + // position: just before the withdrawal limit trees. self.drive .insert_document_ttl_trees(Some(transaction), platform_version)?; + // Withdrawal limit trees under the withdrawals tree: the total credits history (the + // daily withdrawal limit becomes a share of the total credits Platform held a day ago, + // recorded every block), the credit inflows sum tree (every credit mint, so the daily + // limit counts net outflow instead of gross) and Core's credit pool balance per Core + // block read (the Core-anchored withdrawal limit). Through the same helper as genesis, + // which also calls it last, so both build the withdrawals Merk by the same sequence of + // inserts. + self.drive + .insert_withdrawal_limit_trees(Some(transaction), platform_version)?; + Ok(()) } } @@ -878,11 +865,17 @@ mod tests { use drive::drive::credit_pools::epochs::epochs_root_tree_key_constants::KEY_LIFETIME_STORAGE_FEE_POOLS; use drive::drive::credit_pools::pools_path; use drive::drive::document::expiration::paths::DOCUMENTS_EXPIRATIONS_KEY; + use drive::drive::identity::withdrawals::paths::{ + WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, + WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, + }; use drive::drive::shielded::paths::{ shielded_credit_pool_path, MAIN_SHIELDED_CREDIT_POOL_KEY_U8, SHIELDED_ANCHORS_IN_POOL_KEY, SHIELDED_NOTES_KEY, SHIELDED_NULLIFIERS_KEY, }; use drive::drive::tokens::paths::TOKEN_SHIELDED_POOLS_KEY; + use drive::grovedb::operations::proof::{GroveDBProof, ProofBytes}; + use drive::grovedb::{PathQuery, Query}; use drive::util::grove_operations::DirectQueryType; /// Recursively compares the GroveDB subtree rooted at `root_path` between @@ -2446,6 +2439,122 @@ mod tests { } } + /// Merk's shape depends on insertion order, and genesis builds the withdrawals tree's first + /// keys in its batch while the upgrade adds the trees of version 14 to an existing one: + /// both insert those trees one at a time through `insert_withdrawal_limit_trees`, so the + /// withdrawals tree element (its root key), every element below it and the shape of its + /// Merk are the same on a chain born at 14 and one upgraded to it. Batching them into + /// genesis again would root the Merk at another key. + #[test] + fn should_build_the_withdrawal_trees_as_a_chain_born_at_14_does() { + let platform_version = PlatformVersion::latest(); + let born_at_14 = TestPlatformBuilder::new() + .with_initial_protocol_version(14) + .build_with_mock_rpc() + .set_genesis_state(); + let upgraded = TestPlatformBuilder::new() + .with_initial_protocol_version(13) + .build_with_mock_rpc() + .set_genesis_state(); + + let transaction = upgraded.drive.grove.start_transaction(); + let block_info = BlockInfo { + time_ms: 1_000_000, + height: 100, + core_height: 100, + epoch: Epoch::new(1).expect("expected epoch"), + }; + upgraded + .transition_to_version_14(&block_info, &transaction, platform_version) + .expect("expected version 14 transition to succeed"); + + let withdrawals_element = + |platform: &Platform, transaction: Option<&Transaction>| { + platform + .drive + .grove_get_raw( + (&[] as &[&[u8]; 0]).into(), + &[RootTree::WithdrawalTransactions as u8], + DirectQueryType::StatefulDirectQuery, + transaction, + &mut vec![], + &platform_version.drive, + ) + .expect("expected to read the withdrawals tree") + .expect("expected the withdrawals tree to exist") + }; + assert_eq!( + withdrawals_element(&born_at_14, None), + withdrawals_element(&upgraded, Some(&transaction)), + "the withdrawals tree differs between a chain born at version 14 and one upgraded to it" + ); + + let diffs = collect_subtree_diffs( + &born_at_14, + &upgraded, + &transaction, + vec![vec![RootTree::WithdrawalTransactions as u8]], + ); + assert!( + diffs.is_empty(), + "the withdrawal trees differ between a chain born at version 14 and one upgraded \ + to it:\n{}", + diffs.join("\n"), + ); + + // Equal elements and an equal root key can still sit in differently shaped Merks (the + // same keys inserted in another order). A proof of the whole withdrawals tree encodes + // its Merk node by node, so it differs whenever the shape does. + upgraded + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .expect("expected to commit the upgrade"); + let withdrawals_merk_proof = |platform: &Platform| { + let withdrawals_key = vec![RootTree::WithdrawalTransactions as u8]; + let mut query = Query::new(); + query.insert_all(); + let proof = platform + .drive + .grove + .prove_query_non_serialized( + &PathQuery::new_unsized(vec![withdrawals_key.clone()], query), + None, + &platform_version.drive.grove_version, + ) + .unwrap() + .expect("expected to prove the withdrawals tree"); + match proof { + GroveDBProof::V0(proof) => proof + .root_layer + .lower_layers + .get(&withdrawals_key) + .expect("expected the withdrawals layer") + .merk_proof + .clone(), + GroveDBProof::V1(proof) => { + match &proof + .root_layer + .lower_layers + .get(&withdrawals_key) + .expect("expected the withdrawals layer") + .merk_proof + { + ProofBytes::Merk(merk_proof) => merk_proof.clone(), + _ => panic!("expected a Merk proof of the withdrawals tree"), + } + } + } + }; + assert_eq!( + withdrawals_merk_proof(&born_at_14), + withdrawals_merk_proof(&upgraded), + "the withdrawals Merk is shaped differently on a chain born at version 14 and one \ + upgraded to it" + ); + } + #[test] fn test_transition_to_version_14_creates_total_credits_history_tree() { let platform_version = PlatformVersion::latest(); @@ -2462,6 +2571,7 @@ mod tests { for key in [ &WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, &WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, + &WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, ] { assert!(platform .drive @@ -2512,6 +2622,20 @@ mod tests { .expect("credit inflows sum tree should exist after the v14 transition"); assert!(element.is_sum_tree()); + let element = platform + .drive + .grove + .get( + SubtreePath::from(&get_withdrawal_root_path()), + &WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, + Some(&transaction), + &platform_version.drive.grove_version, + ) + .value + .expect("the Core credit pool balances tree should exist after the v14 transition"); + assert!(element.is_any_tree()); + assert!(!element.is_sum_tree()); + // Running it again is harmless and the tree stays usable platform .transition_to_version_14(&block_info, &transaction, platform_version) diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/calculate_core_anchored_withdrawal_limit/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/calculate_core_anchored_withdrawal_limit/mod.rs new file mode 100644 index 00000000000..2c4077c1d1a --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/calculate_core_anchored_withdrawal_limit/mod.rs @@ -0,0 +1,67 @@ +mod v0; + +use crate::error::execution::ExecutionError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::rpc::core::CoreRPCLike; +use dpp::block::block_info::BlockInfo; +use dpp::fee::Credits; +use dpp::version::PlatformVersion; +use drive::grovedb::TransactionArg; + +impl Platform +where + C: CoreRPCLike, +{ + /// How many more credits withdrawals pooled now may take out of Core's credit pool: a + /// stricter copy of Core's own asset unlock limit (`core_credit_pool_unlock_limit`), less + /// what is queued or broadcast and not completed yet. It reads Core's credit pool balance + /// at the block's chain locked height, and the highest balance among the window starts + /// Core may measure an unlock pooled now from: Core's credit pool window (576 blocks, 100 + /// on regtest) back from the chain locked height, up to Core's asset unlock validity + /// (`withdrawal_constants.core_expiration_blocks`, 48) later, as Core mines an unlock + /// until that many blocks past the height it is signed at while its window moves on and + /// older deposits leave it. A balance the scan has not recorded yet is read from Core, + /// which every node answers alike for a chain locked height. + /// + /// # Parameters + /// + /// * `block_info`: The block being executed; its Core chain locked height is the newest + /// Core block the limit reads. + /// * `transaction`: The GroveDB transaction. + /// * `platform_version`: The platform version. + /// + /// # Returns + /// + /// * `Ok(Credits)`: The credits still available on the Core side. + /// * `Err(Error)` when the method version is unknown or not active, a system limit it + /// reads is not configured, Core cannot be asked, or a read fails. + pub(in crate::execution) fn calculate_core_anchored_withdrawal_limit( + &self, + block_info: &BlockInfo, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + match platform_version + .drive_abci + .methods + .withdrawals + .calculate_core_anchored_withdrawal_limit + { + Some(0) => self.calculate_core_anchored_withdrawal_limit_v0( + block_info, + transaction, + platform_version, + ), + Some(version) => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { + method: "calculate_core_anchored_withdrawal_limit".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Execution(ExecutionError::VersionNotActive { + method: "calculate_core_anchored_withdrawal_limit".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/calculate_core_anchored_withdrawal_limit/v0/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/calculate_core_anchored_withdrawal_limit/v0/mod.rs new file mode 100644 index 00000000000..44e410b56af --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/calculate_core_anchored_withdrawal_limit/v0/mod.rs @@ -0,0 +1,509 @@ +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::rpc::core::CoreRPCLike; +use dpp::block::block_info::BlockInfo; +use dpp::fee::Credits; +use dpp::identity::convert_duffs_to_credits; +use dpp::version::PlatformVersion; +use dpp::withdrawal::core_credit_pool_unlock_limit::{ + core_credit_pool_unlock_limit, core_credit_pool_window_blocks, +}; +use drive::grovedb::TransactionArg; +use std::ops::RangeInclusive; + +impl Platform +where + C: CoreRPCLike, +{ + pub(super) fn calculate_core_anchored_withdrawal_limit_v0( + &self, + block_info: &BlockInfo, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + // Core's asset unlock validity, which `update_broadcasted_withdrawal_statuses` also + // expires withdrawals by. + let unlock_validity_blocks = platform_version + .drive_abci + .withdrawal_constants + .core_expiration_blocks; + let window_blocks = core_credit_pool_window_blocks(self.config.network, platform_version)?; + + let chain_locked_height = block_info.core_height; + + // An unlock signed at request height r is mined in a block M with r < M <= r + 48 (Core + // checks the previous block's height against r + 48) and measured from the balance + // after M - 1 - window: a window start from r - window to r + 47 - window. Pooling at + // chain locked height h signs at h, or at h + 1 when the chain locked height moves before + // the later Platform block that signs it, so the window starts read run from h - window + // to h + 48 - window. A window start before the chain's start has no credit pool, which + // Core reads as a balance of 0: it never raises the highest. + let window_start_balance = match chain_locked_height + .saturating_add(unlock_validity_blocks) + .checked_sub(window_blocks) + { + None => 0, + Some(nearest_window_start) => self.highest_core_credit_pool_balance( + chain_locked_height.saturating_sub(window_blocks)..=nearest_window_start, + transaction, + platform_version, + )?, + }; + + let balance = self.highest_core_credit_pool_balance( + chain_locked_height..=chain_locked_height, + transaction, + platform_version, + )?; + + let limit = core_credit_pool_unlock_limit(balance, window_start_balance, platform_version)?; + + // Core's own limit only reflects unlocks already mined: subtract every queued and + // broadcast one. `update_broadcasted_withdrawal_statuses` removes the ones Core mined by + // the chain locked height from the broadcast tree in the first block at that height, + // up to its batch of withdrawal documents, and one signed since cannot be mined by it, + // so only a broadcast backlog beyond that batch is subtracted again after Core mined + // it: the limit is then lower than Core's, never higher, until the statuses catch up. + let in_flight = self + .drive + .fetch_in_flight_withdrawal_amount(transaction, platform_version)?; + + Ok(limit.saturating_sub(in_flight)) + } + + /// The highest of Core's credit pool balances after the chain locked Core blocks at + /// `core_heights`, in credits: recorded by the scan where it has, read from Core otherwise. + fn highest_core_credit_pool_balance( + &self, + core_heights: RangeInclusive, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let recorded = self.drive.fetch_core_credit_pool_balances( + core_heights.clone(), + transaction, + platform_version, + )?; + let mut highest: Credits = 0; + for core_height in core_heights { + let balance = match recorded.get(&core_height) { + Some(balance) => *balance, + None => self.core_credit_pool_balance_from_core(core_height)?, + }; + highest = highest.max(balance); + } + Ok(highest) + } + + /// Core's credit pool balance after the chain locked Core block at `core_height`, in + /// credits, as Core answers it. + pub(in crate::execution::platform_events::withdrawals) fn core_credit_pool_balance_from_core( + &self, + core_height: u32, + ) -> Result { + Ok(convert_duffs_to_credits( + self.core_rpc.get_credit_pool_balance(core_height)?, + )?) + } +} + +#[cfg(test)] +mod tests { + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; + use dpp::block::block_info::BlockInfo; + use dpp::dash_to_credits; + use dpp::dashcore::consensus::Encodable; + use dpp::dashcore::transaction::special_transaction::asset_unlock::unqualified_asset_unlock::{ + AssetUnlockBasePayload, AssetUnlockBaseTransactionInfo, + }; + use dpp::dashcore::{ScriptBuf, TxOut}; + use dpp::fee::Credits; + use dpp::version::PlatformVersion; + use drive::grovedb::Transaction; + use drive::util::batch::DriveOperation; + use std::sync::{Arc, Mutex}; + + const DUFFS_PER_DASH: u64 = 100_000_000; + + /// A Core whose credit pool balance at each height is `balance_at(height)` Dash. + fn core_with_balances(balance_at: fn(u32) -> u64) -> MockCoreRPCLike { + let mut core_rpc = MockCoreRPCLike::new(); + core_rpc + .expect_get_credit_pool_balance() + .returning(move |core_height| Ok(balance_at(core_height) * DUFFS_PER_DASH)); + core_rpc + } + + fn block(core_height: u32) -> BlockInfo { + BlockInfo { + core_height, + ..Default::default() + } + } + + #[test] + fn should_allow_the_percent_of_an_unchanged_pool() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + platform.core_rpc = core_with_balances(|_| 37_000); + let transaction = platform.drive.grove.start_transaction(); + + assert_eq!( + platform + .calculate_core_anchored_withdrawal_limit( + &block(10_000), + Some(&transaction), + PlatformVersion::latest() + ) + .expect("expected the limit"), + dash_to_credits!(5550) + ); + } + + /// The edge case the whole limit exists for: a large deposit mined a day ago is inside + /// the window start balance, so it adds only the percent of itself, whenever Platform + /// learns of it. + #[test] + fn should_add_only_the_percent_of_a_deposit_mined_before_the_band() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + // 37,000 Dash, plus a 5,000 Dash asset lock mined at Core height 9,000. + platform.core_rpc = core_with_balances( + |core_height| { + if core_height >= 9_000 { + 42_000 + } else { + 37_000 + } + }, + ); + let transaction = platform.drive.grove.start_transaction(); + + // 15% of 42,000: the deposit adds 750, not 5,000. + assert_eq!( + platform + .calculate_core_anchored_withdrawal_limit( + &block(10_000), + Some(&transaction), + PlatformVersion::latest() + ) + .expect("expected the limit"), + dash_to_credits!(6300) + ); + } + + /// A deposit Core still counts in full is withdrawable on top; one that leaves Core's + /// window before an unlock pooled now may be mined already counts as if it had. + #[test] + fn should_count_a_deposit_in_full_only_while_it_is_younger_than_the_band() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + // 37,000 Dash, plus a 5,000 Dash asset lock mined at Core height 9,500. + platform.core_rpc = core_with_balances( + |core_height| { + if core_height >= 9_500 { + 42_000 + } else { + 37_000 + } + }, + ); + let transaction = platform.drive.grove.start_transaction(); + let limit = |core_height: u32| { + platform + .calculate_core_anchored_withdrawal_limit( + &block(core_height), + Some(&transaction), + PlatformVersion::latest(), + ) + .expect("expected the limit") + }; + + // At 10,000 the window starts are 9,424..=9,472: the deposit counts in full. + assert_eq!(limit(10_000), dash_to_credits!(10550)); + assert_eq!(limit(10_027), dash_to_credits!(10550)); + // At 10,028 the nearest window start reaches 9,500: an unlock pooled now and signed one + // Core block later may be mined at 10,077, measured from the balance after 9,500, which + // already holds the deposit. + assert_eq!(limit(10_028), dash_to_credits!(6300)); + } + + /// An untied withdrawal transaction paying out 1,000 Dash with a 1,000 duff fee. + fn untied_transaction(index: u64) -> Vec { + let untied = AssetUnlockBaseTransactionInfo { + version: 1, + lock_time: 0, + output: vec![TxOut { + value: 1_000 * DUFFS_PER_DASH, + script_pubkey: ScriptBuf::new(), + }], + base_payload: AssetUnlockBasePayload { + version: 1, + index, + fee: 1_000, + }, + }; + let mut bytes = vec![]; + untied + .consensus_encode(&mut bytes) + .expect("expected to encode"); + bytes + } + + /// Pools the given 1,000 Dash withdrawals, then moves the first `broadcast` of them to the + /// broadcast tree as signing does. + fn pool_withdrawals( + platform: &TempPlatform, + indices: &[u64], + broadcast: u16, + transaction: &Transaction, + platform_version: &PlatformVersion, + ) { + let mut drive_operations: Vec = vec![]; + platform + .drive + .add_enqueue_untied_withdrawal_transaction_operations( + indices + .iter() + .map(|index| (*index, untied_transaction(*index))) + .collect(), + dash_to_credits!(1000) * indices.len() as u64, + &mut drive_operations, + platform_version, + ) + .expect("expected to enqueue"); + if broadcast > 0 { + platform + .drive + .apply_drive_operations( + drive_operations, + true, + &BlockInfo::default(), + Some(transaction), + platform_version, + None, + ) + .expect("expected to apply"); + drive_operations = vec![]; + platform + .drive + .dequeue_untied_withdrawal_transactions( + broadcast, + Some(transaction), + &mut drive_operations, + platform_version, + ) + .expect("expected to dequeue"); + } + platform + .drive + .apply_drive_operations( + drive_operations, + true, + &BlockInfo::default(), + Some(transaction), + platform_version, + None, + ) + .expect("expected to apply"); + } + + #[test] + fn should_subtract_what_is_pooled_and_not_mined_yet() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + platform.core_rpc = core_with_balances(|_| 37_000); + let platform_version = PlatformVersion::latest(); + let transaction = platform.drive.grove.start_transaction(); + + pool_withdrawals(&platform, &[0], 0, &transaction, platform_version); + + assert_eq!( + platform + .calculate_core_anchored_withdrawal_limit( + &block(10_000), + Some(&transaction), + platform_version + ) + .expect("expected the limit"), + dash_to_credits!(4550) - 1_000_000 + ); + } + + /// Queued and broadcast unlocks are both subtracted, from state alone: Core is not asked + /// whether it mined a broadcast one (the mock has no answer for that and would panic). + #[test] + fn should_subtract_queued_and_broadcast_unlocks_without_asking_core() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + platform.core_rpc = core_with_balances(|_| 37_000); + let platform_version = PlatformVersion::latest(); + let transaction = platform.drive.grove.start_transaction(); + + // Indices 0 and 1 broadcast, index 2 still queued. + pool_withdrawals(&platform, &[0, 1, 2], 2, &transaction, platform_version); + + assert_eq!( + platform + .calculate_core_anchored_withdrawal_limit( + &block(10_000), + Some(&transaction), + platform_version + ) + .expect("expected the limit"), + dash_to_credits!(2550) - 3_000_000 + ); + } + + /// The scan, the limit and the cleanup over a chain locked height that advances one Core + /// block per Platform block, as in run_block_proposal: once the scan has caught up, the + /// only balance asked of Core is the new chain locked height's, the limit follows a deposit + /// out of the band, and the cleanup keeps exactly the heights the band can still read. + #[test] + fn should_read_only_the_new_core_block_once_the_scan_has_caught_up() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_version = PlatformVersion::latest(); + // 37,000 Dash, plus a 5,000 Dash asset lock mined at Core height 10,000. + let reads = Arc::new(Mutex::new(vec![])); + let read = reads.clone(); + let mut core_rpc = MockCoreRPCLike::new(); + core_rpc + .expect_get_credit_pool_balance() + .returning(move |core_height| { + read.lock().expect("lock").push(core_height); + let dash = if core_height >= 10_000 { + 42_000 + } else { + 37_000 + }; + Ok(dash * DUFFS_PER_DASH) + }); + platform.core_rpc = core_rpc; + let transaction = platform.drive.grove.start_transaction(); + + let run_block = |core_height: u32| -> Credits { + let block_info = BlockInfo { + time_ms: 1_000_000, + core_height, + ..Default::default() + }; + platform + .scan_core_blocks_for_withdrawals(&block_info, Some(&transaction), platform_version) + .expect("expected to scan"); + let limit = platform + .calculate_core_anchored_withdrawal_limit( + &block_info, + Some(&transaction), + platform_version, + ) + .expect("expected the limit"); + platform + .clean_up_expired_locks_of_withdrawal_amounts( + &block_info, + &transaction, + platform_version, + ) + .expect("expected the cleanup"); + limit + }; + + // The scan reads 32 Core blocks per Platform block: 577 from 9,424 to 10,000 take 19. + for _ in 0..19 { + run_block(10_000); + } + + for core_height in 10_001..=10_600 { + reads.lock().expect("lock").clear(); + let limit = run_block(core_height); + assert_eq!(*reads.lock().expect("lock"), vec![core_height]); + + // The deposit counts in full until the nearest window start reaches it. + let expected = if core_height < 10_528 { + dash_to_credits!(10550) + } else { + dash_to_credits!(6300) + }; + assert_eq!(limit, expected, "at Core height {core_height}"); + } + + let recorded: Vec = platform + .drive + .fetch_core_credit_pool_balances(0..=u32::MAX, Some(&transaction), platform_version) + .expect("expected the balances") + .into_keys() + .collect(); + assert_eq!(recorded, (10_600 - 576..=10_600).collect::>()); + } + + /// Recorded balances are read from state; Core is asked only for what is missing. + #[test] + fn should_prefer_recorded_balances_to_asking_core() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_version = PlatformVersion::latest(); + // Core says 37,000 everywhere. + platform.core_rpc = core_with_balances(|_| 37_000); + let transaction = platform.drive.grove.start_transaction(); + + // The scan recorded 40,000 Dash at one window start of the band. + platform + .drive + .record_core_credit_pool_blocks( + &[(9_430, dash_to_credits!(40000))], + Some(&transaction), + platform_version, + ) + .expect("expected to record the block"); + + // The highest window start is the recorded 40,000: 15% of it, minus the 3,000 drop. + assert_eq!( + platform + .calculate_core_anchored_withdrawal_limit( + &block(10_000), + Some(&transaction), + platform_version + ) + .expect("expected the limit"), + dash_to_credits!(3000) + ); + } + + #[test] + fn should_treat_window_starts_before_the_chain_as_an_empty_pool() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + platform.core_rpc = core_with_balances(|_| 1_000); + let transaction = platform.drive.grove.start_transaction(); + + // Height 100 has no window start in the chain: the whole pool entered inside the + // window and is withdrawable. + assert_eq!( + platform + .calculate_core_anchored_withdrawal_limit( + &block(100), + Some(&transaction), + PlatformVersion::latest() + ) + .expect("expected the limit"), + dash_to_credits!(1000) + ); + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/cleanup_expired_locks_of_withdrawal_amounts/v1/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/cleanup_expired_locks_of_withdrawal_amounts/v1/mod.rs index 8ff497632f6..995d3e5d7b5 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/cleanup_expired_locks_of_withdrawal_amounts/v1/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/cleanup_expired_locks_of_withdrawal_amounts/v1/mod.rs @@ -5,7 +5,9 @@ use crate::rpc::core::CoreRPCLike; use dpp::block::block_info::BlockInfo; use dpp::version::PlatformVersion; +use dpp::withdrawal::core_credit_pool_unlock_limit::core_credit_pool_window_blocks; use drive::drive::identity::withdrawals::paths::{ + get_withdrawal_core_credit_pool_balances_path_vec, get_withdrawal_credit_inflows_sum_tree_path_vec, get_withdrawal_transactions_sum_tree_path_vec, }; use drive::grovedb::{MaybeTree, PathQuery, QueryItem, Transaction}; @@ -15,10 +17,13 @@ impl Platform where C: CoreRPCLike, { - /// Version 1 differs from version 0 in also pruning the expired entries of the credit - /// inflows sum tree, which exists from protocol version 14: both trees are keyed by the - /// block time their entries stop counting toward the daily withdrawal limit, on the same - /// 25 hour schedule, and both are pruned with the same per-block limit. + /// Version 1 differs from version 0 in also pruning the trees of the withdrawal limit that + /// exist from protocol version 14, each with the same per-block limit: + /// + /// * the expired entries of the credit inflows sum tree, keyed like the reservations by + /// the block time they stop counting toward the daily withdrawal limit; + /// * the recorded Core credit pool balances older than the farthest window start the + /// Core-anchored limit reads (Core's credit pool window back). pub(super) fn cleanup_expired_locks_of_withdrawal_amounts_v1( &self, block_info: &BlockInfo, @@ -35,16 +40,40 @@ where return Ok(()); } + // The Core-anchored limit never reads a balance older than its farthest window start. + let oldest_read_core_height = + block_info + .core_height + .checked_sub(core_credit_pool_window_blocks( + self.config.network, + platform_version, + )?); + + // Each tree with the key its expired entries sort below: the reservations and the + // credit inflows by the block time they stop counting, the recorded Core credit pool + // balances by Core height. + let expired_below = [ + Some(( + get_withdrawal_transactions_sum_tree_path_vec(), + block_info.time_ms.to_be_bytes().to_vec(), + )), + Some(( + get_withdrawal_credit_inflows_sum_tree_path_vec(), + block_info.time_ms.to_be_bytes().to_vec(), + )), + oldest_read_core_height.map(|core_height| { + ( + get_withdrawal_core_credit_pool_balances_path_vec(), + core_height.to_be_bytes().to_vec(), + ) + }), + ]; + let mut batch_operations = vec![]; - for path in [ - get_withdrawal_transactions_sum_tree_path_vec(), - get_withdrawal_credit_inflows_sum_tree_path_vec(), - ] { - let mut path_query = PathQuery::new_single_query_item( - path, - QueryItem::RangeTo(..block_info.time_ms.to_be_bytes().to_vec()), - ); + for (path, before_key) in expired_below.into_iter().flatten() { + let mut path_query = + PathQuery::new_single_query_item(path, QueryItem::RangeTo(..before_key)); path_query.query.limit = Some(limit); @@ -173,4 +202,49 @@ mod tests { assert_eq!(keys, vec![now_ms.to_be_bytes().to_vec()]); } } + + /// Recorded Core credit pool balances older than the farthest window start the + /// Core-anchored limit reads are pruned by Core height. + #[test] + fn should_prune_core_credit_pool_balances_older_than_the_window() { + let platform_version = PlatformVersion::latest(); + let platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + let transaction = platform.drive.grove.start_transaction(); + + platform + .drive + .record_core_credit_pool_blocks( + &[(423, 1), (424, 1), (425, 1)], + Some(&transaction), + platform_version, + ) + .expect("expected to record the blocks"); + + platform + .cleanup_expired_locks_of_withdrawal_amounts_v1( + &BlockInfo { + time_ms: 1_000_000, + height: 100, + core_height: 1000, + epoch: Epoch::default(), + }, + &transaction, + platform_version, + ) + .expect("expected the cleanup to succeed"); + + // Core's mainnet window at 1000 starts at 424: 423 goes. + assert_eq!( + platform + .drive + .fetch_core_credit_pool_balances(0..=1000, Some(&transaction), platform_version) + .expect("expected the balances") + .into_keys() + .collect::>(), + vec![424, 425] + ); + } } diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/mod.rs index 5b167ad7818..3d7b93ab02f 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/mod.rs @@ -1,5 +1,6 @@ pub(in crate::execution) mod append_signatures_and_broadcast_withdrawal_transactions; pub(in crate::execution) mod build_untied_withdrawal_transactions_from_documents; +pub(in crate::execution) mod calculate_core_anchored_withdrawal_limit; pub(in crate::execution) mod cleanup_expired_locks_of_withdrawal_amounts; pub(in crate::execution) mod dequeue_and_build_unsigned_withdrawal_transactions; pub(in crate::execution) mod fetch_transactions_block_inclusion_status; @@ -8,4 +9,5 @@ pub(in crate::execution) mod pool_withdrawals_into_transactions_queue; pub(in crate::execution) mod rebroadcast_expired_withdrawal_documents; pub(in crate::execution) mod record_credit_inflows_for_withdrawals; pub(in crate::execution) mod record_total_credits_history_for_withdrawals; +pub(in crate::execution) mod scan_core_blocks_for_withdrawals; pub(in crate::execution) mod update_broadcasted_withdrawal_statuses; diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/mod.rs index 66be57e5e0e..7db143cce67 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/mod.rs @@ -10,6 +10,7 @@ use drive::grovedb::TransactionArg; mod v0; mod v1; +mod v2; impl Platform where @@ -52,9 +53,14 @@ where transaction, platform_version, ), + 2 => self.pool_withdrawals_into_transactions_queue_v2( + block_info, + transaction, + platform_version, + ), version => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "pool_withdrawals_into_transactions_queue".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, })), } diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v0/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v0/mod.rs index a3c1f187ac5..2bdb96557cb 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v0/mod.rs @@ -229,8 +229,10 @@ mod tests { /// attempting to build any transactions. #[test] fn v1_returns_ok_when_no_queued_documents() { - let platform_version = PlatformVersion::latest(); + // Version 1 is frozen: the last protocol version that selects it. + let platform_version = PlatformVersion::get(13).expect("expected protocol version 13"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(13) .build_with_mock_rpc() .set_initial_state_structure(); diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v1/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v1/mod.rs index d0bb8c3c74a..2c079903994 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v1/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v1/mod.rs @@ -3,6 +3,7 @@ use metrics::gauge; use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::document::DocumentV0Getters; +use dpp::fee::Credits; use dpp::platform_value::btreemap_extensions::BTreeValueMapHelper; use dpp::version::PlatformVersion; use drive::grovedb::TransactionArg; @@ -31,6 +32,37 @@ where block_info: &BlockInfo, transaction: TransactionArg, platform_version: &PlatformVersion, + ) -> Result<(), Error> { + self.pool_withdrawals_up_to_limit_v1( + block_info, + |available, daily_maximum| { + let current_withdrawal_limit = available; + + // Store prometheus metrics + gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_AVAILABLE) + .set(current_withdrawal_limit as f64); + gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_TOTAL).set(daily_maximum as f64); + + Ok(current_withdrawal_limit) + }, + transaction, + platform_version, + ) + } + + /// Version 1's pooling, given the amount to pool up to once the daily withdrawal limit is + /// known (`current_withdrawal_limit`, called with its available amount and its daily + /// maximum, only when withdrawals are queued). Extracted in place so version 2 can reuse + /// it, inert for protocol versions 1 to 13 (those selecting version 1, and through version + /// 0, which delegates to it, those selecting version 0): the closure of version 1 returns + /// the available daily limit and sets the gauges exactly where they were set before, so + /// every such block pools the same documents and writes the same state. + pub(super) fn pool_withdrawals_up_to_limit_v1( + &self, + block_info: &BlockInfo, + current_withdrawal_limit: impl FnOnce(Credits, Credits) -> Result, + transaction: TransactionArg, + platform_version: &PlatformVersion, ) -> Result<(), Error> { let documents = self.drive.fetch_oldest_withdrawal_documents_by_status( withdrawals_contract::WithdrawalStatus::QUEUED.into(), @@ -65,11 +97,8 @@ where "Calculated withdrawal limit info" ); - let current_withdrawal_limit = withdrawals_info.available(); - - // Store prometheus metrics - gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_AVAILABLE).set(current_withdrawal_limit as f64); - gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_TOTAL).set(withdrawals_info.daily_maximum as f64); + let current_withdrawal_limit = + current_withdrawal_limit(withdrawals_info.available(), withdrawals_info.daily_maximum)?; // Only process documents up to the current withdrawal limit. let mut total_withdrawal_amount = 0u64; @@ -210,8 +239,10 @@ mod tests { #[test] fn test_pooling() { - let platform_version = PlatformVersion::latest(); + // Version 1 is frozen: the last protocol version that selects it. + let platform_version = PlatformVersion::get(13).expect("expected protocol version 13"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(13) .build_with_mock_rpc() .set_initial_state_structure(); diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v2/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v2/mod.rs new file mode 100644 index 00000000000..8bd5a66bad4 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/pool_withdrawals_into_transactions_queue/v2/mod.rs @@ -0,0 +1,227 @@ +use dpp::block::block_info::BlockInfo; +use metrics::gauge; + +use dpp::version::PlatformVersion; +use drive::grovedb::TransactionArg; + +use crate::metrics::{ + GAUGE_CREDIT_WITHDRAWAL_LIMIT_AVAILABLE, GAUGE_CREDIT_WITHDRAWAL_LIMIT_CORE_AVAILABLE, + GAUGE_CREDIT_WITHDRAWAL_LIMIT_TOTAL, +}; +use crate::{error::Error, platform_types::platform::Platform, rpc::core::CoreRPCLike}; + +impl Platform +where + C: CoreRPCLike, +{ + /// Pool withdrawal documents into transactions. + /// + /// Version 2 differs from version 1 only in the amount it pools up to: the smaller of the + /// daily withdrawal limit and the Core-anchored limit, a stricter copy of Core's own asset + /// unlock rule read from Core's credit pool balances. Platform's own accounting can grant + /// more than Core will mine (an asset lock published to Platform after Core mined it, the + /// epoch Core rewards minted in one block); an unlock over Core's limit waits unmined, + /// expires and is re-signed, and while Core's mempool holds more than the limit, Core + /// InstantSend-locks no withdrawal at all. What the Core side holds back stays queued on + /// Platform instead. It first reads the Core blocks the chain locked height passed + /// (`scan_core_blocks_for_withdrawals`). + pub(super) fn pool_withdrawals_into_transactions_queue_v2( + &self, + block_info: &BlockInfo, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + // Bring the Core blocks up to date first, every block whether or not anything is + // queued: their credit pool balances feed the Core-anchored limit below, and a long + // jump of the chain locked height is read over several blocks. + self.scan_core_blocks_for_withdrawals(block_info, transaction, platform_version)?; + + self.pool_withdrawals_up_to_limit_v1( + block_info, + |available, daily_maximum| { + // Computed whenever withdrawals are queued, so its gauge stays current. Once the + // scan has recorded the band and the chain locked height, it reads state only. + let core_anchored_withdrawal_limit = self + .calculate_core_anchored_withdrawal_limit( + block_info, + transaction, + platform_version, + )?; + + tracing::trace!( + core_anchored_withdrawal_limit, + "Calculated Core-anchored withdrawal limit" + ); + + let current_withdrawal_limit = available.min(core_anchored_withdrawal_limit); + + // Store prometheus metrics: what may be pooled, as in version 1, and the Core + // side on its own. + gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_AVAILABLE) + .set(current_withdrawal_limit as f64); + gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_TOTAL).set(daily_maximum as f64); + gauge!(GAUGE_CREDIT_WITHDRAWAL_LIMIT_CORE_AVAILABLE) + .set(core_anchored_withdrawal_limit as f64); + + Ok(current_withdrawal_limit) + }, + transaction, + platform_version, + ) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::TestPlatformBuilder; + use dpp::block::epoch::Epoch; + use dpp::dash_to_credits; + use dpp::data_contract::accessors::v0::DataContractV0Getters; + use dpp::data_contracts::SystemDataContract; + use dpp::identifier::Identifier; + use dpp::identity::core_script::CoreScript; + use dpp::platform_value::platform_value; + use dpp::system_data_contracts::load_system_data_contract; + use dpp::system_data_contracts::withdrawals_contract; + use dpp::system_data_contracts::withdrawals_contract::v1::document_types::withdrawal; + use dpp::tests::fixtures::get_withdrawal_document_fixture; + use dpp::withdrawal::Pooling; + use drive::config::DEFAULT_QUERY_LIMIT; + use drive::util::test_helpers::setup::{setup_document, setup_system_data_contract}; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::sync::Arc; + + /// Pools two queued withdrawals of `amount` credits each through the dispatcher, at + /// `platform_version`, against a Core whose credit pool holds `pool_duffs` at every height. + /// Returns how many were pooled and how many credit pool balances were asked of Core. + fn pool(platform_version: &PlatformVersion, amount: u64, pool_duffs: u64) -> (usize, usize) { + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(platform_version.protocol_version) + .build_with_mock_rpc() + .set_initial_state_structure(); + + let balance_reads = Arc::new(AtomicUsize::new(0)); + let mut core_rpc = MockCoreRPCLike::new(); + let reads = balance_reads.clone(); + core_rpc + .expect_get_credit_pool_balance() + .returning(move |_| { + reads.fetch_add(1, Ordering::SeqCst); + Ok(pool_duffs) + }); + platform.core_rpc = core_rpc; + + let transaction = platform.drive.grove.start_transaction(); + + let block_info = BlockInfo { + time_ms: 1, + height: 1, + core_height: 10_000, + epoch: Epoch::default(), + }; + + let data_contract = + load_system_data_contract(SystemDataContract::Withdrawals, platform_version) + .expect("to load system data contract"); + setup_system_data_contract(&platform.drive, &data_contract, Some(&transaction)); + let document_type = data_contract + .document_type_for_name(withdrawal::NAME) + .expect("expected to get document type"); + + for transaction_index in [1u64, 2] { + let document = get_withdrawal_document_fixture( + &data_contract, + Identifier::new([1u8; 32]), + platform_value!({ + "amount": amount, + "coreFeePerByte": 1u32, + "pooling": Pooling::Never as u8, + "outputScript": CoreScript::from_bytes((0..23).collect::>()), + "status": withdrawals_contract::WithdrawalStatus::QUEUED as u8, + "transactionIndex": transaction_index, + }), + None, + platform_version.protocol_version, + ) + .expect("expected withdrawal document"); + setup_document( + &platform.drive, + &document, + &data_contract, + document_type, + Some(&transaction), + ); + } + + let platform_state = platform.state.load(); + platform + .pool_withdrawals_into_transactions_queue( + &block_info, + &platform_state, + Some(&transaction), + platform_version, + ) + .expect("to pool withdrawal documents into transactions"); + + let pooled = platform + .drive + .fetch_oldest_withdrawal_documents_by_status( + withdrawals_contract::WithdrawalStatus::POOLED.into(), + DEFAULT_QUERY_LIMIT, + Some(&transaction), + platform_version, + ) + .expect("to fetch withdrawal documents") + .len(); + (pooled, balance_reads.load(Ordering::SeqCst)) + } + + #[test] + fn should_pool_what_fits_both_the_daily_limit_and_cores_credit_pool() { + // Plenty in Core's pool: both withdrawals pool. + assert_eq!( + pool(PlatformVersion::latest(), 1000, 1_000_000_000_000).0, + 2 + ); + } + + #[test] + fn should_leave_queued_what_cores_credit_pool_could_not_give_up() { + // A pool of 1 duff, 1,000 credits: it fits one 1,000 credit withdrawal, not two, + // although the daily limit (2,000 Dash before any history) would allow both. + assert_eq!(pool(PlatformVersion::latest(), 1000, 1).0, 1); + assert_eq!(pool(PlatformVersion::latest(), 1000, 0).0, 0); + } + + /// Through the dispatcher on both sides of the gate: version 1 (protocol version 13) pools + /// on the daily limit alone, version 2 (14) also on Core's credit pool. + #[test] + fn should_hold_back_on_cores_credit_pool_only_from_protocol_version_14() { + let version_13 = PlatformVersion::get(13).expect("expected protocol version 13"); + assert_eq!(pool(version_13, 1000, 1), (2, 0)); + assert_eq!(pool(PlatformVersion::latest(), 1000, 1).0, 1); + } + + /// While the oldest queued withdrawal does not fit Platform's own limit nothing pools, + /// however much Core's pool admits, and the Core side is still read so its gauge stays + /// current: the scan asks Core for 32 Core blocks from 10,000 - 576, the limit for the 17 + /// window starts and the chain locked height the scan has not recorded yet. + #[test] + fn should_pool_nothing_while_the_oldest_does_not_fit_the_daily_limit() { + // 3,000 Dash each, over the flat 2,000 Dash before any history. + assert_eq!( + pool( + PlatformVersion::latest(), + dash_to_credits!(3000), + 1_000_000_000_000 + ), + (0, 50) + ); + assert_eq!( + pool(PlatformVersion::latest(), 1000, 1_000_000_000_000), + (2, 50) + ); + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/scan_core_blocks_for_withdrawals/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/scan_core_blocks_for_withdrawals/mod.rs new file mode 100644 index 00000000000..6c9e90aa01e --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/scan_core_blocks_for_withdrawals/mod.rs @@ -0,0 +1,63 @@ +mod v0; + +use crate::error::execution::ExecutionError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::rpc::core::CoreRPCLike; +use dpp::block::block_info::BlockInfo; +use dpp::version::PlatformVersion; +use drive::grovedb::TransactionArg; + +impl Platform +where + C: CoreRPCLike, +{ + /// Reads the Core blocks the chain locked height has passed since the last one read and + /// records Core's credit pool balance after each, which the Core-anchored withdrawal limit + /// reads. Reads at most `core_blocks_scanned_per_block_limit` Core blocks per block, oldest + /// first, and never one older than the band the limit reads (Core's credit pool window + /// back); the rest follow in the next blocks. + /// + /// Only chain locked Core blocks are read, so every node reads the same. Pooling calls it + /// every block before it reads the withdrawal limits, so they see the newest Core blocks; + /// a long jump of the chain locked height is read over several blocks. + /// + /// # Parameters + /// + /// * `block_info`: The block being executed; its Core chain locked height is the newest + /// Core block read. + /// * `transaction`: The GroveDB transaction. + /// * `platform_version`: The platform version. + /// + /// # Returns + /// + /// * `Ok(())` once the Core blocks are recorded. + /// * `Err(Error)` when the method version (or a Drive method it calls) is unknown or not + /// active, Core cannot be asked, or a write fails. + pub(in crate::execution) fn scan_core_blocks_for_withdrawals( + &self, + block_info: &BlockInfo, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + match platform_version + .drive_abci + .methods + .withdrawals + .scan_core_blocks_for_withdrawals + { + Some(0) => { + self.scan_core_blocks_for_withdrawals_v0(block_info, transaction, platform_version) + } + Some(version) => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { + method: "scan_core_blocks_for_withdrawals".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Execution(ExecutionError::VersionNotActive { + method: "scan_core_blocks_for_withdrawals".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/withdrawals/scan_core_blocks_for_withdrawals/v0/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/scan_core_blocks_for_withdrawals/v0/mod.rs new file mode 100644 index 00000000000..dc0698c11b0 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/withdrawals/scan_core_blocks_for_withdrawals/v0/mod.rs @@ -0,0 +1,201 @@ +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::rpc::core::CoreRPCLike; +use dpp::block::block_info::BlockInfo; +use dpp::version::PlatformVersion; +use dpp::withdrawal::core_credit_pool_unlock_limit::core_credit_pool_window_blocks; +use drive::grovedb::TransactionArg; + +impl Platform +where + C: CoreRPCLike, +{ + pub(super) fn scan_core_blocks_for_withdrawals_v0( + &self, + block_info: &BlockInfo, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + let limit = platform_version + .drive_abci + .withdrawal_constants + .core_blocks_scanned_per_block_limit; + if limit == 0 { + return Ok(()); + } + + let chain_locked_height = block_info.core_height; + + // Nothing older than the band the limit reads is worth reading: its balance is never + // read again. + let oldest_useful_height = chain_locked_height.saturating_sub( + core_credit_pool_window_blocks(self.config.network, platform_version)?, + ); + + let first_height = match self + .drive + .fetch_last_recorded_core_credit_pool_height(transaction, platform_version)? + { + Some(last_recorded_height) => match last_recorded_height.checked_add(1) { + Some(next_height) => next_height.max(oldest_useful_height), + None => return Ok(()), + }, + None => oldest_useful_height, + }; + + if first_height > chain_locked_height { + return Ok(()); + } + + let last_height = + chain_locked_height.min(first_height.saturating_add(u32::from(limit) - 1)); + + let balances = (first_height..=last_height) + .map(|core_height| { + Ok(( + core_height, + self.core_credit_pool_balance_from_core(core_height)?, + )) + }) + .collect::, Error>>()?; + + self.drive + .record_core_credit_pool_blocks(&balances, transaction, platform_version)?; + + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use crate::error::execution::ExecutionError; + use crate::error::Error; + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::TestPlatformBuilder; + use dpp::block::block_info::BlockInfo; + use dpp::version::PlatformVersion; + use std::collections::BTreeMap; + use std::sync::{Arc, Mutex}; + + /// The Core heights read, in order, by a mock that answers every height with a balance of + /// `height * 1000` duffs. + fn recording_core(read: Arc>>) -> MockCoreRPCLike { + let mut core_rpc = MockCoreRPCLike::new(); + core_rpc + .expect_get_credit_pool_balance() + .returning(move |core_height| { + read.lock().expect("lock").push(core_height); + Ok(u64::from(core_height) * 1000) + }); + core_rpc + } + + #[test] + fn should_read_each_core_block_once_oldest_first_and_bounded_per_block() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_version = PlatformVersion::latest(); + let read = Arc::new(Mutex::new(vec![])); + platform.core_rpc = recording_core(read.clone()); + let transaction = platform.drive.grove.start_transaction(); + let block = |core_height: u32| BlockInfo { + core_height, + ..Default::default() + }; + + // First run: starts at the far edge of the band (1000 - 576, Core's mainnet window) + // and reads 32 blocks. + platform + .scan_core_blocks_for_withdrawals(&block(1000), Some(&transaction), platform_version) + .expect("expected to scan"); + assert_eq!(*read.lock().expect("lock"), (424..=455).collect::>()); + + // It goes on from the next unread block. + read.lock().expect("lock").clear(); + platform + .scan_core_blocks_for_withdrawals(&block(1000), Some(&transaction), platform_version) + .expect("expected to scan"); + assert_eq!(*read.lock().expect("lock"), (456..=487).collect::>()); + + // A jump past the band skips what is too old to matter. + read.lock().expect("lock").clear(); + platform + .scan_core_blocks_for_withdrawals(&block(2000), Some(&transaction), platform_version) + .expect("expected to scan"); + assert_eq!( + *read.lock().expect("lock"), + (1424..=1455).collect::>() + ); + + // Balances are recorded in credits. + assert_eq!( + platform + .drive + .fetch_core_credit_pool_balances(1424..=1425, Some(&transaction), platform_version) + .expect("expected the balances"), + BTreeMap::from([(1424, 1_424_000_000), (1425, 1_425_000_000)]) + ); + } + + #[test] + fn should_read_nothing_once_caught_up_with_the_chain_locked_height() { + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_version = PlatformVersion::latest(); + let read = Arc::new(Mutex::new(vec![])); + platform.core_rpc = recording_core(read.clone()); + let transaction = platform.drive.grove.start_transaction(); + let block = |core_height: u32| BlockInfo { + core_height, + ..Default::default() + }; + + // A young chain: everything from height 0 is in the band. + platform + .scan_core_blocks_for_withdrawals(&block(5), Some(&transaction), platform_version) + .expect("expected to scan"); + assert_eq!(*read.lock().expect("lock"), (0..=5).collect::>()); + + read.lock().expect("lock").clear(); + platform + .scan_core_blocks_for_withdrawals(&block(5), Some(&transaction), platform_version) + .expect("expected to scan"); + assert!(read.lock().expect("lock").is_empty()); + + platform + .scan_core_blocks_for_withdrawals(&block(7), Some(&transaction), platform_version) + .expect("expected to scan"); + assert_eq!(*read.lock().expect("lock"), vec![6, 7]); + } + + #[test] + fn should_not_exist_before_protocol_version_14() { + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(13) + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_version = + PlatformVersion::get(13).expect("expected to get platform version 13"); + let read = Arc::new(Mutex::new(vec![])); + platform.core_rpc = recording_core(read.clone()); + let transaction = platform.drive.grove.start_transaction(); + + let result = platform.scan_core_blocks_for_withdrawals( + &BlockInfo { + core_height: 1000, + ..Default::default() + }, + Some(&transaction), + platform_version, + ); + assert!(matches!( + result, + Err(Error::Execution(ExecutionError::VersionNotActive { .. })) + )); + assert!(read.lock().expect("lock").is_empty()); + } +} diff --git a/packages/rs-drive-abci/src/main.rs b/packages/rs-drive-abci/src/main.rs index 347a21aeca1..b5d964684d5 100644 --- a/packages/rs-drive-abci/src/main.rs +++ b/packages/rs-drive-abci/src/main.rs @@ -451,6 +451,9 @@ mod snapshot_bake_main { fn send_raw_transaction(&self, _: &[u8]) -> Result { unreachable!() } + fn get_credit_pool_balance(&self, _: u32) -> Result { + unreachable!() + } } /// Produce a shielded-pool snapshot at `out_path` from a fresh temporary diff --git a/packages/rs-drive-abci/src/metrics.rs b/packages/rs-drive-abci/src/metrics.rs index 72b862a7381..fb0b79a8801 100644 --- a/packages/rs-drive-abci/src/metrics.rs +++ b/packages/rs-drive-abci/src/metrics.rs @@ -35,6 +35,9 @@ pub const LABEL_CHECK_TX_MODE: &str = "check_tx_mode"; pub const GAUGE_CREDIT_WITHDRAWAL_LIMIT_AVAILABLE: &str = "credit_withdrawal_limit_available"; /// Total withdrawal daily limit in credits pub const GAUGE_CREDIT_WITHDRAWAL_LIMIT_TOTAL: &str = "credit_withdrawal_limit_total"; +/// Credits still available to withdrawals on the Core-anchored side of the withdrawal limit +pub const GAUGE_CREDIT_WITHDRAWAL_LIMIT_CORE_AVAILABLE: &str = + "credit_withdrawal_limit_core_available"; /// Error returned by metrics subsystem #[derive(thiserror::Error, Debug)] @@ -237,6 +240,11 @@ impl Prometheus { GAUGE_CREDIT_WITHDRAWAL_LIMIT_TOTAL, "Total withdrawal limit for last 24 hours in credits" ); + + describe_gauge!( + GAUGE_CREDIT_WITHDRAWAL_LIMIT_CORE_AVAILABLE, + "Credits withdrawals may still take from Core's credit pool, by the stricter copy of Core's unlock limit" + ); }); } } diff --git a/packages/rs-drive-abci/src/platform_types/platform/mock.rs b/packages/rs-drive-abci/src/platform_types/platform/mock.rs index 9157e2a8ff5..8b5fb08caaf 100644 --- a/packages/rs-drive-abci/src/platform_types/platform/mock.rs +++ b/packages/rs-drive-abci/src/platform_types/platform/mock.rs @@ -34,6 +34,16 @@ impl Platform { "tx": [], })) }); + + // A credit pool of 10 million Dash at every height, so the Core-anchored withdrawal limit + // never binds. Registered first: mockall uses the first matching expectation, so an + // answer a test adds later on this platform is never reached. To bind the Core side, + // call `core_rpc.checkpoint()` (which also drops the answers above) and set every + // answer the test needs, or replace `core_rpc` with a new `MockCoreRPCLike`. + core_rpc_mock + .expect_get_credit_pool_balance() + .returning(|_| Ok(1_000_000_000_000_000)); + Self::open_with_client(path, config, core_rpc_mock, initial_protocol_version) } diff --git a/packages/rs-drive-abci/src/rpc/core.rs b/packages/rs-drive-abci/src/rpc/core.rs index cf9a387616f..749936920c7 100644 --- a/packages/rs-drive-abci/src/rpc/core.rs +++ b/packages/rs-drive-abci/src/rpc/core.rs @@ -1,5 +1,7 @@ use crate::rpc::prefetch::CorePrefetcher; +use dpp::dashcore::consensus::encode::deserialize_partial; use dpp::dashcore::ephemerealdata::chain_lock::ChainLock; +use dpp::dashcore::transaction::special_transaction::TransactionPayload; use dpp::dashcore::{Block, BlockHash, QuorumHash, Transaction, Txid}; use dpp::dashcore::{Header, InstantLock}; use dpp::dashcore_rpc::dashcore_rpc_json::{ @@ -16,6 +18,35 @@ use std::time::Duration; /// Information returned by QuorumListExtended pub type QuorumListExtendedInfo = HashMap; +/// The special transaction type of a coinbase (`TRANSACTION_COINBASE` in Dash Core). +const COINBASE_TRANSACTION_TYPE: u16 = 5; + +/// Reads Core's credit pool balance after a block, in duffs, from the block's serialized +/// coinbase transaction; `0` when it carries no payload or its payload predates version 3, +/// before the credit pool existed, which is how Core's own unlock limit reads such a block. +/// +/// Decoded with `deserialize_partial`: the pinned payload decoder reads the fields of version 3 +/// for every later version and stops after the balance, while the version 4 payload Core v24 +/// requires appends `merkleRootAssetUnlocks` after it. A strict `deserialize`, and so a whole +/// `Block` decode, refuses those unread bytes. Core has only ever appended fields to the +/// payload, and its consensus rules (`CheckCbTx`) refuse versions it does not know, so the +/// balance stays where version 3 put it unless a Core release moves it, which Platform would +/// have to follow anyway. +pub(crate) fn credit_pool_balance_from_coinbase(coinbase: &[u8]) -> Result { + let (transaction, _) = deserialize_partial::(coinbase) + .map_err(|e| format!("coinbase cannot be decoded: {e}"))?; + match transaction.special_transaction_payload { + None => Ok(0), + Some(TransactionPayload::CoinbasePayloadType(payload)) => { + Ok(payload.asset_locked_amount.unwrap_or_default()) + } + Some(payload) => Err(format!( + "coinbase carries a {:?} payload", + payload.get_type() + )), + } +} + /// Core height must be of type u32 (Platform heights are u64) pub type CoreHeight = u32; /// Core RPC interface @@ -126,6 +157,11 @@ pub trait CoreRPCLike { /// Sends raw transaction to the network fn send_raw_transaction(&self, transaction: &[u8]) -> Result; + + /// Get Core's credit pool balance after the block at `height`, in duffs, read from the + /// block's coinbase (only the coinbase is transferred). Only ask for a chain locked height: + /// the answer is then the same on every node. + fn get_credit_pool_balance(&self, height: CoreHeight) -> Result; } #[derive(Debug)] @@ -387,4 +423,124 @@ impl CoreRPCLike for DefaultCoreRPC { .inner .get_asset_unlock_statuses(indices, Some(core_chain_locked_height))) } + + fn get_credit_pool_balance(&self, height: CoreHeight) -> Result { + let block_hash = self.get_block_hash(height)?; + // Only the coinbase, as raw hex: the special transactions of type 5, the first one, + // verbosity 1. An empty answer is a coinbase without a payload, before DIP3. + let args = [ + Value::String(block_hash.to_string()), + Value::from(COINBASE_TRANSACTION_TYPE), + Value::from(1), + Value::from(0), + Value::from(1), + ]; + let coinbases = retry!(self.inner.call::>("getspecialtxes", &args))?; + let Some(coinbase) = coinbases.first() else { + return Ok(0); + }; + let coinbase = hex::decode(coinbase).map_err(|e| { + Error::UnexpectedStructure(format!("getspecialtxes answered invalid hex: {e}")) + })?; + credit_pool_balance_from_coinbase(&coinbase).map_err(Error::UnexpectedStructure) + } +} + +#[cfg(test)] +mod tests { + use super::credit_pool_balance_from_coinbase; + use dpp::dashcore::bls_sig_utils::BLSSignature; + use dpp::dashcore::consensus::serialize; + use dpp::dashcore::hash_types::{MerkleRootMasternodeList, MerkleRootQuorums}; + use dpp::dashcore::hashes::Hash; + use dpp::dashcore::transaction::special_transaction::coinbase::CoinbasePayload; + use dpp::dashcore::transaction::special_transaction::TransactionPayload; + use dpp::dashcore::{OutPoint, ScriptBuf, Transaction, TxIn, TxOut}; + + const BALANCE_DUFFS: u64 = 3_700_000_000_000; + + /// A coinbase as the pinned rust-dashcore encodes it. + fn coinbase(payload: Option) -> Vec { + serialize(&Transaction { + version: 3, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint::null(), + script_sig: ScriptBuf::from(vec![0x51, 0x51]), + sequence: u32::MAX, + witness: Default::default(), + }], + output: vec![TxOut { + value: 5_000_000, + script_pubkey: ScriptBuf::from(vec![0x76; 25]), + }], + special_transaction_payload: payload, + }) + } + + fn version_3_payload() -> TransactionPayload { + TransactionPayload::CoinbasePayloadType(CoinbasePayload { + version: 3, + height: 1_000, + merkle_root_masternode_list: MerkleRootMasternodeList::from_byte_array([1; 32]), + merkle_root_quorums: MerkleRootQuorums::from_byte_array([2; 32]), + best_cl_height: Some(30), + best_cl_signature: Some(BLSSignature::from([3; 96])), + asset_locked_amount: Some(BALANCE_DUFFS), + }) + } + + #[test] + fn should_read_the_balance_of_a_version_3_coinbase() { + assert_eq!( + credit_pool_balance_from_coinbase(&coinbase(Some(version_3_payload()))), + Ok(BALANCE_DUFFS) + ); + } + + /// Core v24 blocks carry a version 4 payload, which appends `merkleRootAssetUnlocks` + /// after the balance; the pinned transaction decoder cannot read it. + #[test] + fn should_read_the_balance_of_a_version_4_coinbase() { + let version_3 = coinbase(Some(version_3_payload())); + // The payload is last: its 1-byte length (175), then the payload itself. + let payload_start = version_3.len() - 175; + assert_eq!(version_3[payload_start - 1], 175); + let mut version_4 = version_3[..payload_start - 1].to_vec(); + version_4.push(175 + 32); + version_4.extend_from_slice(&4u16.to_le_bytes()); + version_4.extend_from_slice(&version_3[payload_start + 2..]); + version_4.extend_from_slice(&[0xaa; 32]); + + assert_eq!( + credit_pool_balance_from_coinbase(&version_4), + Ok(BALANCE_DUFFS) + ); + } + + #[test] + fn should_read_no_balance_from_a_coinbase_before_the_credit_pool() { + let version_2 = TransactionPayload::CoinbasePayloadType(CoinbasePayload { + version: 2, + height: 1_000, + merkle_root_masternode_list: MerkleRootMasternodeList::from_byte_array([1; 32]), + merkle_root_quorums: MerkleRootQuorums::from_byte_array([2; 32]), + best_cl_height: None, + best_cl_signature: None, + asset_locked_amount: None, + }); + assert_eq!( + credit_pool_balance_from_coinbase(&coinbase(Some(version_2))), + Ok(0) + ); + assert_eq!(credit_pool_balance_from_coinbase(&coinbase(None)), Ok(0)); + } + + #[test] + fn should_fail_on_a_truncated_coinbase() { + let full = coinbase(Some(version_3_payload())); + // Cut inside the payload, before the balance. + assert!(credit_pool_balance_from_coinbase(&full[..full.len() - 60]).is_err()); + assert!(credit_pool_balance_from_coinbase(&full[..40]).is_err()); + } } diff --git a/packages/rs-drive-abci/tests/strategy_tests/test_cases/withdrawal_tests.rs b/packages/rs-drive-abci/tests/strategy_tests/test_cases/withdrawal_tests.rs index 835b60e7e4a..85c8007e9dc 100644 --- a/packages/rs-drive-abci/tests/strategy_tests/test_cases/withdrawal_tests.rs +++ b/packages/rs-drive-abci/tests/strategy_tests/test_cases/withdrawal_tests.rs @@ -3050,6 +3050,251 @@ mod tests { assert_eq!(withdrawal_documents_broadcasted.len(), 80); } + #[tokio::test] + async fn should_hold_back_withdrawals_over_the_core_anchored_limit() { + // Latest protocol version, and a Core whose credit pool holds 120 Dash at every + // height. The chain is so young that every window start Core may measure an unlock + // from lies before it, an empty pool, so Core admits unlocks up to the pool itself. + // Platform's own daily limit (the flat 2,000 Dash while no recorded total is a day old, + // plus the 10,000 Dash of funding inflows) is far above that, so the Core-anchored + // side alone decides what is pooled. + let platform_version = PlatformVersion::latest(); + let start_strategy = NetworkStrategy { + strategy: Strategy { + start_contracts: vec![], + operations: vec![], + start_identities: StartIdentities::default(), + start_addresses: StartAddresses::default(), + identity_inserts: IdentityInsertInfo { + frequency: Frequency { + times_per_block_range: 10..11, + chance_per_block: None, + }, + start_keys: 3, + extra_keys: [( + Purpose::TRANSFER, + [(SecurityLevel::CRITICAL, vec![KeyType::ECDSA_SECP256K1])].into(), + )] + .into(), + start_balance_range: dash_to_duffs!(500)..=dash_to_duffs!(500), + }, + identity_contract_nonce_gaps: None, + signer: None, + }, + total_hpmns: 100, + extra_normal_mns: 0, + validator_quorum_count: 24, + chain_lock_quorum_count: 24, + upgrading_info: None, + + proposer_strategy: Default::default(), + rotate_quorums: false, + failure_testing: None, + query_testing: None, + verify_state_transition_results: true, + ..Default::default() + }; + + let minute_in_ms = 1000 * 60; + let config = PlatformConfig { + validator_set: ValidatorSetConfig::default_100_67(), + chain_lock: ChainLockConfig::default_100_67(), + instant_lock: InstantLockConfig::default_100_67(), + execution: ExecutionConfig { + verify_sum_trees: true, + ..Default::default() + }, + block_spacing_ms: minute_in_ms, + testing_configs: PlatformTestConfig::default_minimal_verifications(), + ..Default::default() + }; + + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .with_config(config.clone()) + .build_with_mock_rpc(); + + // Replace the default Core answers (a credit pool of 10 million Dash) with the small + // pool; Core has mined none of the unlocks. + platform.core_rpc.checkpoint(); + platform + .core_rpc + .expect_get_block_hash() + .returning(|_| Ok(BlockHash::all_zeros())); + platform + .core_rpc + .expect_get_block_json() + .returning(|_| Ok(serde_json::json!({ "tx": [] }))); + platform + .core_rpc + .expect_get_credit_pool_balance() + .returning(|_| Ok(dash_to_duffs!(120))); + platform + .core_rpc + .expect_send_raw_transaction() + .returning(move |_| Ok(Txid::all_zeros())); + platform + .core_rpc + .expect_get_asset_unlock_statuses() + .returning(|indices, _| { + Ok(indices + .iter() + .map(|index| AssetUnlockStatusResult { + index: *index, + status: AssetUnlockStatus::Unknown, + }) + .collect()) + }); + platform + .core_rpc + .expect_get_best_chain_lock() + .returning(|| { + Ok(ChainLock { + block_height: 1, + block_hash: BlockHash::from_byte_array([1; 32]), + signature: BLSSignature::from([2; 96]), + }) + }); + + // Blocks 1 and 2 create 20 identities of 500 Dash. + let ChainExecutionOutcome { + abci_app, + proposers, + validator_quorums: quorums, + current_validator_quorum_hash: current_quorum_hash, + current_proposer_versions, + end_time_ms, + identity_nonce_counter, + identity_contract_nonce_counter, + instant_lock_quorums, + identities, + addresses_with_balance, + signer, + .. + } = run_chain_for_strategy( + &mut platform, + 2, + start_strategy, + config.clone(), + 1, + &mut None, + &mut None, + ) + .await; + + let continue_strategy_only_withdrawal = NetworkStrategy { + strategy: Strategy { + start_contracts: vec![], + operations: vec![Operation { + op_type: OperationType::IdentityWithdrawal( + dash_to_credits!(50)..=dash_to_credits!(50), + ), + frequency: Frequency { + times_per_block_range: 4..5, // 50 Dash x 4 Withdrawals = 200 Dash + chance_per_block: None, + }, + }], + start_identities: StartIdentities::default(), + start_addresses: StartAddresses::default(), + identity_inserts: IdentityInsertInfo::default(), + identity_contract_nonce_gaps: None, + signer: Some(signer), + }, + total_hpmns: 100, + extra_normal_mns: 0, + validator_quorum_count: 24, + chain_lock_quorum_count: 24, + upgrading_info: None, + + proposer_strategy: Default::default(), + rotate_quorums: false, + failure_testing: None, + query_testing: None, + verify_state_transition_results: true, + ..Default::default() + }; + + // Blocks 3 to 5 withdraw 200 Dash each. + let outcome = continue_chain_for_strategy( + abci_app, + ChainExecutionParameters { + block_start: 3, + core_height_start: 1, + block_count: 3, + proposers, + validator_quorums: quorums, + current_validator_quorum_hash: current_quorum_hash, + current_proposer_versions: Some(current_proposer_versions), + current_identity_nonce_counter: identity_nonce_counter, + current_identity_contract_nonce_counter: identity_contract_nonce_counter, + current_votes: BTreeMap::default(), + start_time_ms: GENESIS_TIME_MS, + current_time_ms: end_time_ms, + instant_lock_quorums, + current_identities: identities, + current_addresses_with_balance: addresses_with_balance, + }, + continue_strategy_only_withdrawal, + config, + StrategyRandomness::SeedEntropy(2), + ) + .await; + + for tx_results_per_block in outcome.state_transition_results_per_block.values() { + assert_eq!(tx_results_per_block.len(), 4); + for (state_transition, result) in tx_results_per_block { + assert_eq!( + result.code, 0, + "state transition got code {} : {:?}", + result.code, state_transition + ); + } + } + + let documents_with_status = |status: withdrawals_contract::WithdrawalStatus| { + outcome + .abci_app + .platform + .drive + .fetch_oldest_withdrawal_documents_by_status( + status.into(), + DEFAULT_QUERY_LIMIT, + None, + platform_version, + ) + .expect("expected to fetch withdrawal documents") + .len() + }; + + // The first withdrawal block pools two (100 Dash) of its four into the 120 Dash Core + // admits; after that what is pooled and not mined leaves under 50 Dash, so the other + // ten wait in the queue although Platform's own limit has thousands left. + assert_eq!( + documents_with_status(withdrawals_contract::WithdrawalStatus::POOLED) + + documents_with_status(withdrawals_contract::WithdrawalStatus::BROADCASTED), + 2 + ); + assert_eq!( + documents_with_status(withdrawals_contract::WithdrawalStatus::QUEUED), + 10 + ); + + let locked_amount = outcome + .abci_app + .platform + .drive + .grove_get_sum_tree_total_value( + (&get_withdrawal_root_path()).into(), + &WITHDRAWAL_TRANSACTIONS_SUM_AMOUNT_TREE_KEY, + DirectQueryType::StatefulDirectQuery, + None, + &mut vec![], + &platform_version.drive, + ) + .expect("expected to get locked amount"); + assert_eq!(locked_amount, dash_to_credits!(100) as i64); + } + #[tokio::test] async fn run_chain_withdraw_from_identities_many_small_withdrawals() { // TEST_PLATFORM_V3 is like v4, but without the single quorum can sign withdrawals restriction diff --git a/packages/rs-drive/grovedb-structure.json b/packages/rs-drive/grovedb-structure.json index 19ee70108a9..98826dbc904 100644 --- a/packages/rs-drive/grovedb-structure.json +++ b/packages/rs-drive/grovedb-structure.json @@ -4034,6 +4034,46 @@ "children": [] } ] + }, + { + "id": "withdrawals.core_credit_pool_balances", + "key": { + "type": "fixed", + "hex": "06", + "label": "CoreCreditPoolBalances", + "constant": "WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY" + }, + "kinds": [ + "Tree" + ], + "since": 14, + "presence": "always", + "source": "packages/rs-drive/src/drive/identity/withdrawals/paths.rs", + "description": "Core's credit pool balance after each Core block read, for the Core-anchored withdrawal limit.", + "children": [ + { + "id": "withdrawals.core_credit_pool_balances.balance", + "key": { + "type": "dynamic", + "name": "core_height", + "matcher": { + "type": "len", + "len": 4 + }, + "encoding": "u32_be", + "description": "The Core block height" + }, + "kinds": [ + "Item" + ], + "value": "credits, u64 big endian", + "since": 14, + "presence": "always", + "source": "packages/rs-drive/src/drive/identity/withdrawals/paths.rs", + "description": "The credit pool balance after that Core block.", + "children": [] + } + ] } ] }, @@ -6229,20 +6269,23 @@ "withdrawals": { "origin": "genesis@14", "tree": { - "hex": "03", + "hex": "02", "left": { "hex": "01", "left": { "hex": "00" - }, - "right": { - "hex": "02" } }, "right": { - "hex": "05", + "hex": "04", "left": { - "hex": "04" + "hex": "03" + }, + "right": { + "hex": "05", + "right": { + "hex": "06" + } } } } diff --git a/packages/rs-drive/src/drive/identity/withdrawals/calculate_current_withdrawal_limit/v1/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/calculate_current_withdrawal_limit/v1/mod.rs index 523cbe6d075..963ba63f15e 100644 --- a/packages/rs-drive/src/drive/identity/withdrawals/calculate_current_withdrawal_limit/v1/mod.rs +++ b/packages/rs-drive/src/drive/identity/withdrawals/calculate_current_withdrawal_limit/v1/mod.rs @@ -49,12 +49,8 @@ impl Drive { /// other users. Only inflows younger than the snapshot count — an older one is already /// inside the base, and adding it again would allow the pool level to drop below the /// guaranteed share of the day-old total — and only unexpired ones, so an entry the - /// bounded cleanup has not deleted yet cannot outlive its 25 hours here. The base - /// stays capped by `max_daily_withdrawal_amount` (Core's unlock capacity per day) but - /// the inflows ride above the cap: capping the sum would hand the whole capped budget - /// back to a deposit-withdraw cycle whenever the base reaches the cap, and outflow - /// funded by same-window deposits mirrors the net credit pool rule Core adopts - /// alongside this; + /// bounded cleanup has not deleted yet cannot outlive its 25 hours here. Outflow + /// funded by same-window deposits mirrors Core v24's net credit pool rule; /// * the withdrawal reservations are bounded the same way: one pooled at or before the /// snapshot describes an outflow the base already reflects (the history is recorded /// after state transitions executed), so subtracting it again would deny budget the @@ -106,12 +102,9 @@ impl Drive { let total_credits_a_day_ago = recorded_a_day_ago.map(|recorded| recorded.total_credits); - // The base is capped at Core's unlock capacity inside `daily_withdrawal_limit`; the - // inflows ride on top of the capped base, not under the cap. Capping the sum would - // discard the inflows exactly when the base reaches the cap — at that point a - // deposit-withdraw cycle would consume the whole capped budget again (#4471 under - // mainnet totals). Outflow funded by same-window deposits mirrors the net credit - // pool rule Core adopts alongside this (see #4471), so it may exceed the cap. + // The base has no fixed cap; what Core will mine bounds pooling separately, through + // the Core-anchored limit. Outflow funded by same-window deposits mirrors Core v24's + // net credit pool rule (see #4471). let daily_maximum = daily_withdrawal_limit(total_credits_a_day_ago, platform_version)? .saturating_add(credit_inflows_since_the_snapshot); @@ -364,102 +357,6 @@ mod tests { assert_eq!(info.available(), dash_to_credits!(1500)); } - /// Inflows extend the daily maximum past the capped base: the cap bounds what Core mines - /// out of the standing pool per day, and capping the sum instead would hand the whole - /// capped budget back to a deposit-withdraw cycle whenever the base reaches the cap — - /// #4471 under mainnet totals. At a 30,000 Dash total the base is capped at 4,000; a - /// 4,000 Dash deposit-withdraw cycle must leave the full 4,000 available to others. - #[test] - fn a_cycle_at_the_capped_base_should_not_consume_the_budget_of_others() { - let drive = setup_drive_with_initial_state_structure(None); - let mut platform_version = PlatformVersion::latest().clone(); - platform_version - .system_limits - .daily_withdrawal_limit_percent = Some(15); - let transaction = drive.grove.start_transaction(); - - let t0 = 10 * DAY_IN_MS; - let block = |time_ms: u64| BlockInfo { - time_ms, - ..Default::default() - }; - let limit = |time_ms: u64| { - drive - .calculate_current_withdrawal_limit( - &block(time_ms), - Some(&transaction), - &platform_version, - ) - .expect("expected the limit") - }; - - // Platform holds 30,000 Dash: 15% would be 4,500, so the base sits at the 4,000 Dash - // cap — the network conditions of #4471. - drive - .add_to_system_credits( - dash_to_credits!(30000), - Some(&transaction), - &platform_version, - ) - .expect("expected to add credits"); - drive - .record_total_credits_history(&block(t0), 64, Some(&transaction), &platform_version) - .expect("expected to record"); - - let day_one = t0 + DAY_IN_MS; - assert_eq!(limit(day_one).daily_maximum, dash_to_credits!(4000)); - - // The attacker deposits the whole capped budget and withdraws it again. - drive - .add_to_system_credits( - dash_to_credits!(4000), - Some(&transaction), - &platform_version, - ) - .expect("expected to add the deposit"); - drive - .record_credit_inflow( - dash_to_credits!(4000), - &block(day_one), - Some(&transaction), - &platform_version, - ) - .expect("expected to record the inflow"); - let mut drive_operations = vec![]; - drive - .add_enqueue_untied_withdrawal_transaction_operations( - vec![(1, vec![0u8; 32])], - dash_to_credits!(4000), - &mut drive_operations, - &platform_version, - ) - .expect("expected to enqueue the withdrawal"); - drive - .apply_drive_operations( - drive_operations, - true, - &block(day_one), - Some(&transaction), - &platform_version, - None, - ) - .expect("expected to apply the pooling operations"); - drive - .remove_from_system_credits( - dash_to_credits!(4000), - Some(&transaction), - &platform_version, - ) - .expect("expected to remove the withdrawn credits"); - - // The inflow rides above the cap, so the cycle nets out: everyone else still has the - // full capped budget available. - let info = limit(day_one); - assert_eq!(info.daily_maximum, dash_to_credits!(8000)); - assert_eq!(info.withdrawals_amount, dash_to_credits!(4000)); - assert_eq!(info.available(), dash_to_credits!(4000)); - } - /// An inflow that is already part of the day-old base must not extend the daily maximum a /// second time: while the deposit block is younger than a day the inflow counts against /// the older snapshot, and the moment the deposit block itself becomes the snapshot the diff --git a/packages/rs-drive/src/drive/identity/withdrawals/fetch_core_credit_pool_balances/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/fetch_core_credit_pool_balances/mod.rs new file mode 100644 index 00000000000..9f974d749b2 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/fetch_core_credit_pool_balances/mod.rs @@ -0,0 +1,55 @@ +mod v0; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use dpp::fee::Credits; +use grovedb::TransactionArg; +use platform_version::version::PlatformVersion; +use std::collections::BTreeMap; +use std::ops::RangeInclusive; + +impl Drive { + /// Fetches the recorded Core credit pool balances (in credits) of the Core heights in + /// `core_heights`. A height that was never recorded, or was pruned, is absent from the + /// result. + /// + /// # Parameters + /// + /// * `core_heights`: The Core heights to read. + /// * `transaction`: The GroveDB transaction. + /// * `platform_version`: The platform version. + /// + /// # Returns + /// + /// * `Ok(BTreeMap)`: The balance of every recorded height in the range. + /// * `Err(Error)` when the method version is unknown or not active, an entry is corrupted, + /// or the read fails. + pub fn fetch_core_credit_pool_balances( + &self, + core_heights: RangeInclusive, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + match platform_version + .drive + .methods + .identity + .withdrawals + .fetch_core_credit_pool_balances + { + Some(0) => { + self.fetch_core_credit_pool_balances_v0(core_heights, transaction, platform_version) + } + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "fetch_core_credit_pool_balances".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "fetch_core_credit_pool_balances".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/fetch_core_credit_pool_balances/v0/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/fetch_core_credit_pool_balances/v0/mod.rs new file mode 100644 index 00000000000..b675374c0e9 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/fetch_core_credit_pool_balances/v0/mod.rs @@ -0,0 +1,110 @@ +use crate::drive::identity::withdrawals::paths::get_withdrawal_core_credit_pool_balances_path_vec; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use dpp::fee::Credits; +use grovedb::query_result_type::QueryResultType; +use grovedb::{Element, PathQuery, Query, QueryItem, TransactionArg}; +use platform_version::version::PlatformVersion; +use std::collections::BTreeMap; +use std::ops::RangeInclusive; + +impl Drive { + pub(super) fn fetch_core_credit_pool_balances_v0( + &self, + core_heights: RangeInclusive, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if core_heights.is_empty() { + return Ok(BTreeMap::new()); + } + + let path_query = PathQuery::new_unsized( + get_withdrawal_core_credit_pool_balances_path_vec(), + Query::new_single_query_item(QueryItem::RangeInclusive( + core_heights.start().to_be_bytes().to_vec() + ..=core_heights.end().to_be_bytes().to_vec(), + )), + ); + + let (results, _) = self.grove_get_raw_path_query( + &path_query, + transaction, + QueryResultType::QueryKeyElementPairResultType, + &mut vec![], + &platform_version.drive, + )?; + + results + .to_key_elements() + .into_iter() + .map(|(key, element)| { + let core_height = u32::from_be_bytes(key.try_into().map_err(|_| { + Error::Drive(DriveError::CorruptedSerialization( + "core credit pool balance key is not 4 bytes".to_string(), + )) + })?); + let Element::Item(value, _) = element else { + return Err(Error::Drive(DriveError::CorruptedElementType( + "core credit pool balance is not an item", + ))); + }; + let balance = u64::from_be_bytes(value.try_into().map_err(|_| { + Error::Drive(DriveError::CorruptedSerialization( + "core credit pool balance is not 8 bytes".to_string(), + )) + })?); + Ok((core_height, balance)) + }) + .collect() + } +} + +#[cfg(test)] +mod tests { + use crate::util::test_helpers::setup::setup_drive_with_initial_state_structure; + use dpp::version::PlatformVersion; + use std::collections::BTreeMap; + + #[test] + fn should_return_the_recorded_balances_of_a_range_and_the_last_recorded_height() { + let drive = setup_drive_with_initial_state_structure(None); + let platform_version = PlatformVersion::latest(); + let transaction = drive.grove.start_transaction(); + + assert_eq!( + drive + .fetch_last_recorded_core_credit_pool_height(Some(&transaction), platform_version) + .expect("expected the last height"), + None + ); + + drive + .record_core_credit_pool_blocks( + &[(10, 1_000), (11, 1_100), (12, 1_200)], + Some(&transaction), + platform_version, + ) + .expect("expected to record the blocks"); + + assert_eq!( + drive + .fetch_core_credit_pool_balances(11..=20, Some(&transaction), platform_version) + .expect("expected the balances"), + BTreeMap::from([(11, 1_100), (12, 1_200)]) + ); + assert_eq!( + drive + .fetch_core_credit_pool_balances(0..=9, Some(&transaction), platform_version) + .expect("expected the balances"), + BTreeMap::new() + ); + assert_eq!( + drive + .fetch_last_recorded_core_credit_pool_height(Some(&transaction), platform_version) + .expect("expected the last height"), + Some(12) + ); + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/fetch_in_flight_withdrawal_amount/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/fetch_in_flight_withdrawal_amount/mod.rs new file mode 100644 index 00000000000..cefc9b09381 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/fetch_in_flight_withdrawal_amount/mod.rs @@ -0,0 +1,51 @@ +mod v0; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use dpp::fee::Credits; +use grovedb::TransactionArg; +use platform_version::version::PlatformVersion; + +impl Drive { + /// Reads what the pooled withdrawal transactions not completed yet (the queue and the + /// broadcast tree) will take out of Core's credit pool once mined, in credits: each + /// transaction's outputs plus its fee, as Core counts an asset unlock. Core's own unlock + /// limit only reflects unlocks already mined, so the Core-anchored withdrawal limit + /// subtracts this sum. + /// + /// # Parameters + /// + /// * `transaction`: The GroveDB transaction. + /// * `platform_version`: The platform version. + /// + /// # Returns + /// + /// * `Ok(Credits)`: The sum over the queued and broadcast transactions, in credits. + /// * `Err(Error)` when the method version is unknown or not active, a stored transaction + /// cannot be decoded, or the sum overflows. + pub fn fetch_in_flight_withdrawal_amount( + &self, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + match platform_version + .drive + .methods + .identity + .withdrawals + .fetch_in_flight_withdrawal_amount + { + Some(0) => self.fetch_in_flight_withdrawal_amount_v0(transaction, platform_version), + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "fetch_in_flight_withdrawal_amount".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "fetch_in_flight_withdrawal_amount".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/fetch_in_flight_withdrawal_amount/v0/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/fetch_in_flight_withdrawal_amount/v0/mod.rs new file mode 100644 index 00000000000..9cabc3fa0d6 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/fetch_in_flight_withdrawal_amount/v0/mod.rs @@ -0,0 +1,193 @@ +use crate::drive::identity::withdrawals::paths::{ + get_withdrawal_transactions_broadcasted_path_vec, get_withdrawal_transactions_queue_path_vec, +}; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use dpp::dashcore::consensus::Decodable; +use dpp::dashcore::transaction::special_transaction::asset_unlock::unqualified_asset_unlock::AssetUnlockBaseTransactionInfo; +use dpp::fee::Credits; +use dpp::identity::convert_duffs_to_credits; +use grovedb::query_result_type::QueryResultType; +use grovedb::{Element, PathQuery, Query, TransactionArg}; +use platform_version::version::PlatformVersion; + +impl Drive { + pub(super) fn fetch_in_flight_withdrawal_amount_v0( + &self, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let mut in_flight: Credits = 0; + for path in [ + get_withdrawal_transactions_queue_path_vec(), + get_withdrawal_transactions_broadcasted_path_vec(), + ] { + for amount in + self.fetch_withdrawal_transaction_amounts(path, transaction, platform_version)? + { + in_flight = in_flight.checked_add(amount).ok_or(Error::Drive( + DriveError::CriticalCorruptedState("in-flight withdrawal amount overflow"), + ))?; + } + } + + Ok(in_flight) + } + + /// What each untied withdrawal transaction under `path` takes out of Core's credit pool, in + /// credits: its outputs plus its fee. + fn fetch_withdrawal_transaction_amounts( + &self, + path: Vec>, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let mut query = Query::new(); + query.insert_all(); + let path_query = PathQuery::new_unsized(path, query); + + let (results, _) = self.grove_get_raw_path_query( + &path_query, + transaction, + QueryResultType::QueryElementResultType, + &mut vec![], + &platform_version.drive, + )?; + + let overflow = || { + Error::Drive(DriveError::CriticalCorruptedState( + "in-flight withdrawal amount overflow", + )) + }; + + results + .to_elements() + .into_iter() + .map(|element| { + let Element::Item(bytes, _) = element else { + return Err(Error::Drive(DriveError::CorruptedElementType( + "withdrawal transaction is not an item", + ))); + }; + // Both trees hold the untied transaction as it was pooled; the request height + // and quorum signature added when signing do not change what it unlocks. + let untied = + AssetUnlockBaseTransactionInfo::consensus_decode(&mut bytes.as_slice()) + .map_err(|_| { + Error::Drive(DriveError::CorruptedSerialization( + "withdrawal transaction cannot be decoded".to_string(), + )) + })?; + + // What Core counts against its limit: the outputs plus the fee. + let mut duffs = untied.base_payload.fee as u64; + for output in &untied.output { + duffs = duffs.checked_add(output.value).ok_or_else(overflow)?; + } + + convert_duffs_to_credits(duffs).map_err(|_| overflow()) + }) + .collect() + } +} + +#[cfg(test)] +mod tests { + use crate::util::batch::DriveOperation; + use crate::util::test_helpers::setup::setup_drive_with_initial_state_structure; + use dpp::block::block_info::BlockInfo; + use dpp::dashcore::consensus::Encodable; + use dpp::dashcore::transaction::special_transaction::asset_unlock::unqualified_asset_unlock::{ + AssetUnlockBasePayload, AssetUnlockBaseTransactionInfo, + }; + use dpp::dashcore::{ScriptBuf, TxOut}; + use dpp::version::PlatformVersion; + + fn untied_transaction(index: u64, payout_duffs: u64, fee_duffs: u32) -> Vec { + let transaction = AssetUnlockBaseTransactionInfo { + version: 1, + lock_time: 0, + output: vec![TxOut { + value: payout_duffs, + script_pubkey: ScriptBuf::new(), + }], + base_payload: AssetUnlockBasePayload { + version: 1, + index, + fee: fee_duffs, + }, + }; + let mut bytes = vec![]; + transaction + .consensus_encode(&mut bytes) + .expect("expected to encode"); + bytes + } + + #[test] + fn should_read_the_outputs_and_fees_of_queued_and_broadcast_transactions_in_credits() { + let drive = setup_drive_with_initial_state_structure(None); + let platform_version = PlatformVersion::latest(); + let transaction = drive.grove.start_transaction(); + + assert_eq!( + drive + .fetch_in_flight_withdrawal_amount(Some(&transaction), platform_version) + .expect("expected the amount"), + 0 + ); + + let mut drive_operations: Vec = vec![]; + drive + .add_enqueue_untied_withdrawal_transaction_operations( + vec![ + (0, untied_transaction(0, 100_000, 2_000)), + (1, untied_transaction(1, 50_000, 1_000)), + ], + 153_000_000, + &mut drive_operations, + platform_version, + ) + .expect("expected to enqueue"); + drive + .apply_drive_operations( + drive_operations, + true, + &BlockInfo::default(), + Some(&transaction), + platform_version, + None, + ) + .expect("expected to apply"); + + // Move one to the broadcast tree, as signing does. + let mut drive_operations: Vec = vec![]; + drive + .dequeue_untied_withdrawal_transactions( + 1, + Some(&transaction), + &mut drive_operations, + platform_version, + ) + .expect("expected to dequeue"); + drive + .apply_drive_operations( + drive_operations, + true, + &BlockInfo::default(), + Some(&transaction), + platform_version, + None, + ) + .expect("expected to apply"); + + // Queued: 50,000 + 1,000 duffs; broadcast: 100,000 + 2,000 duffs; in credits. + assert_eq!( + drive + .fetch_in_flight_withdrawal_amount(Some(&transaction), platform_version) + .expect("expected the amount"), + 153_000_000 + ); + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/fetch_last_recorded_core_credit_pool_height/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/fetch_last_recorded_core_credit_pool_height/mod.rs new file mode 100644 index 00000000000..65966b38a74 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/fetch_last_recorded_core_credit_pool_height/mod.rs @@ -0,0 +1,51 @@ +mod v0; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use grovedb::TransactionArg; +use platform_version::version::PlatformVersion; + +impl Drive { + /// Fetches the highest Core height whose credit pool balance was recorded: the last Core + /// block the scan of `scan_core_blocks_for_withdrawals` read. Shares the + /// `fetch_core_credit_pool_balances` method version. + /// + /// # Parameters + /// + /// * `transaction`: The GroveDB transaction. + /// * `platform_version`: The platform version. + /// + /// # Returns + /// + /// * `Ok(Some(u32))`: The highest recorded Core height. + /// * `Ok(None)`: When no balance was recorded yet. + /// * `Err(Error)` when the method version is unknown or not active, the entry is corrupted, + /// or the read fails. + pub fn fetch_last_recorded_core_credit_pool_height( + &self, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + match platform_version + .drive + .methods + .identity + .withdrawals + .fetch_core_credit_pool_balances + { + Some(0) => { + self.fetch_last_recorded_core_credit_pool_height_v0(transaction, platform_version) + } + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "fetch_last_recorded_core_credit_pool_height".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "fetch_last_recorded_core_credit_pool_height".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/fetch_last_recorded_core_credit_pool_height/v0/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/fetch_last_recorded_core_credit_pool_height/v0/mod.rs new file mode 100644 index 00000000000..ca7654a0257 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/fetch_last_recorded_core_credit_pool_height/v0/mod.rs @@ -0,0 +1,45 @@ +use crate::drive::identity::withdrawals::paths::get_withdrawal_core_credit_pool_balances_path_vec; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use grovedb::query_result_type::QueryResultType; +use grovedb::{PathQuery, Query, SizedQuery, TransactionArg}; +use platform_version::version::PlatformVersion; + +impl Drive { + pub(super) fn fetch_last_recorded_core_credit_pool_height_v0( + &self, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let mut query = Query::new(); + query.insert_all(); + query.left_to_right = false; + + let path_query = PathQuery::new( + get_withdrawal_core_credit_pool_balances_path_vec(), + SizedQuery::new(query, Some(1), None), + ); + + let (results, _) = self.grove_get_raw_path_query( + &path_query, + transaction, + QueryResultType::QueryKeyElementPairResultType, + &mut vec![], + &platform_version.drive, + )?; + + results + .to_key_elements() + .into_iter() + .next() + .map(|(key, _)| { + key.try_into().map(u32::from_be_bytes).map_err(|_| { + Error::Drive(DriveError::CorruptedSerialization( + "core credit pool balance key is not 4 bytes".to_string(), + )) + }) + }) + .transpose() + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/mod.rs index 22db7a870c1..480e0980a5b 100644 --- a/packages/rs-drive/src/drive/identity/withdrawals/mod.rs +++ b/packages/rs-drive/src/drive/identity/withdrawals/mod.rs @@ -10,10 +10,18 @@ mod calculate_current_withdrawal_limit; /// same schedule so a deposit and the withdrawal it funds cancel exactly over the whole window. // Best to use a constant here and not a versioned item as this most likely will not change pub const DAY_AND_A_HOUR_IN_MS: dpp::prelude::TimestampMillis = 90_000_000; //25 hours +/// Functions related to the Core credit pool balances the Core-anchored withdrawal limit reads +pub mod fetch_core_credit_pool_balances; +/// Functions related to what pooled withdrawals take out of Core's credit pool once mined +pub mod fetch_in_flight_withdrawal_amount; +/// Functions related to the Core credit pool balances the Core-anchored withdrawal limit reads +pub mod fetch_last_recorded_core_credit_pool_height; /// Functions related to the per-block record of total credits the daily withdrawal limit reads pub mod fetch_total_credits_in_platform_a_day_ago; /// Functions and constants related to GroveDB paths pub mod paths; +/// Functions related to the Core blocks the Core-anchored withdrawal limit reads +pub mod record_core_credit_pool_blocks; /// Functions related to the per-block record of credit inflows the daily withdrawal limit adds pub mod record_credit_inflow; /// Functions related to the per-block record of total credits the daily withdrawal limit reads diff --git a/packages/rs-drive/src/drive/identity/withdrawals/paths.rs b/packages/rs-drive/src/drive/identity/withdrawals/paths.rs index d0b6d9d0843..967efd2ce29 100644 --- a/packages/rs-drive/src/drive/identity/withdrawals/paths.rs +++ b/packages/rs-drive/src/drive/identity/withdrawals/paths.rs @@ -1,7 +1,8 @@ use crate::drive::{Drive, RootTree}; +use crate::error::Error; use crate::util::batch::grovedb_op_batch::GroveDbOpBatchV0Methods; use crate::util::batch::GroveDbOpBatch; -use grovedb::Element; +use grovedb::{Element, TransactionArg}; use platform_version::version::PlatformVersion; /// constant key for transaction counter @@ -22,6 +23,11 @@ pub const WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY: [u8; 1] = [4]; /// that entered Platform within the window may leave again without consuming the budget of /// other users. Exists from protocol version 14. pub const WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY: [u8; 1] = [5]; +/// constant id for the subtree recording Core's credit pool balance after each Core block +/// Platform read (key: Core block height, big-endian; value: the balance in credits, +/// big-endian). The Core-anchored withdrawal limit reads the balance at the chain locked height +/// and at the start of Core's unlock window. Exists from protocol version 14. +pub const WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY: [u8; 1] = [6]; impl Drive { /// Add operations for creating initial withdrawal state structure @@ -50,17 +56,40 @@ impl Drive { WITHDRAWAL_TRANSACTIONS_BROADCASTED_KEY.to_vec(), ); } + } - if platform_version.protocol_version >= 14 { - batch.add_insert_empty_tree( - vec![vec![RootTree::WithdrawalTransactions as u8]], - WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY.to_vec(), - ); - batch.add_insert_empty_sum_tree( - vec![vec![RootTree::WithdrawalTransactions as u8]], - WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY.to_vec(), - ); + /// Inserts the withdrawal limit trees of protocol version 14 under the withdrawals tree, + /// one after the other: the total credits history, the credit inflows sum tree and the + /// Core credit pool balances. Genesis (`create_initial_state_structure` 4, after its batch) + /// and the upgrade (`Platform::transition_to_version_14`) both call it, so the withdrawals + /// Merk is built by the same sequence of inserts on both node populations: adding the + /// trees to the genesis batch would root it at another key than the upgrade does. + pub fn insert_withdrawal_limit_trees( + &self, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + for (key, tree) in [ + (WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, Element::empty_tree()), + ( + WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, + Element::empty_sum_tree(), + ), + ( + WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, + Element::empty_tree(), + ), + ] { + self.grove_insert_if_not_exists( + (&get_withdrawal_root_path()).into(), + &key, + tree, + transaction, + None, + &platform_version.drive, + )?; } + Ok(()) } } @@ -153,3 +182,19 @@ pub fn get_withdrawal_credit_inflows_sum_tree_path() -> [&'static [u8]; 2] { &WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, ] } + +/// Helper function to get the Core credit pool balances path as Vec +pub fn get_withdrawal_core_credit_pool_balances_path_vec() -> Vec> { + vec![ + vec![RootTree::WithdrawalTransactions as u8], + WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY.to_vec(), + ] +} + +/// Helper function to get the Core credit pool balances path as [u8] +pub fn get_withdrawal_core_credit_pool_balances_path() -> [&'static [u8]; 2] { + [ + Into::<&[u8; 1]>::into(RootTree::WithdrawalTransactions), + &WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, + ] +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/record_core_credit_pool_blocks/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/record_core_credit_pool_blocks/mod.rs new file mode 100644 index 00000000000..a16dcef747f --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/record_core_credit_pool_blocks/mod.rs @@ -0,0 +1,52 @@ +mod v0; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use dpp::fee::Credits; +use grovedb::TransactionArg; +use platform_version::version::PlatformVersion; + +impl Drive { + /// Records Core's credit pool balance after each of the given Core blocks, under the block's + /// height, for the Core-anchored withdrawal limit, in one batch. + /// + /// # Parameters + /// + /// * `balances`: Each Core block's height with Core's credit pool balance after it, in + /// credits. + /// * `transaction`: The GroveDB transaction. + /// * `platform_version`: The platform version. + /// + /// # Returns + /// + /// * `Ok(())` once the balances are stored, or at once when there are none. + /// * `Err(Error)` when the method version is unknown or not active, or the write fails. + pub fn record_core_credit_pool_blocks( + &self, + balances: &[(u32, Credits)], + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + match platform_version + .drive + .methods + .identity + .withdrawals + .record_core_credit_pool_blocks + { + Some(0) => { + self.record_core_credit_pool_blocks_v0(balances, transaction, platform_version) + } + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "record_core_credit_pool_blocks".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "record_core_credit_pool_blocks".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/record_core_credit_pool_blocks/v0/mod.rs b/packages/rs-drive/src/drive/identity/withdrawals/record_core_credit_pool_blocks/v0/mod.rs new file mode 100644 index 00000000000..9804cd91c2d --- /dev/null +++ b/packages/rs-drive/src/drive/identity/withdrawals/record_core_credit_pool_blocks/v0/mod.rs @@ -0,0 +1,44 @@ +use crate::drive::identity::withdrawals::paths::get_withdrawal_core_credit_pool_balances_path_vec; +use crate::drive::Drive; +use crate::error::Error; +use crate::util::object_size_info::PathKeyElementInfo; +use dpp::fee::Credits; +use grovedb::{Element, TransactionArg}; +use platform_version::version::PlatformVersion; + +impl Drive { + pub(super) fn record_core_credit_pool_blocks_v0( + &self, + balances: &[(u32, Credits)], + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + if balances.is_empty() { + return Ok(()); + } + + let mut drive_operations = vec![]; + + for (core_height, credit_pool_balance) in balances { + self.batch_insert( + PathKeyElementInfo::PathKeyElement::<0>(( + get_withdrawal_core_credit_pool_balances_path_vec(), + core_height.to_be_bytes().to_vec(), + Element::new_item(credit_pool_balance.to_be_bytes().to_vec()), + )), + &mut drive_operations, + &platform_version.drive, + )?; + } + + self.apply_batch_low_level_drive_operations( + None, + transaction, + drive_operations, + &mut vec![], + &platform_version.drive, + )?; + + Ok(()) + } +} diff --git a/packages/rs-drive/src/drive/identity/withdrawals/structure.rs b/packages/rs-drive/src/drive/identity/withdrawals/structure.rs index 1334b4daa83..6e74d294c70 100644 --- a/packages/rs-drive/src/drive/identity/withdrawals/structure.rs +++ b/packages/rs-drive/src/drive/identity/withdrawals/structure.rs @@ -1,7 +1,8 @@ use crate::drive::identity::withdrawals::paths::{ - WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, - WITHDRAWAL_TRANSACTIONS_BROADCASTED_KEY, WITHDRAWAL_TRANSACTIONS_NEXT_INDEX_KEY, - WITHDRAWAL_TRANSACTIONS_QUEUE_KEY, WITHDRAWAL_TRANSACTIONS_SUM_AMOUNT_TREE_KEY, + WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, WITHDRAWAL_CREDIT_INFLOWS_SUM_TREE_KEY, + WITHDRAWAL_TOTAL_CREDITS_HISTORY_KEY, WITHDRAWAL_TRANSACTIONS_BROADCASTED_KEY, + WITHDRAWAL_TRANSACTIONS_NEXT_INDEX_KEY, WITHDRAWAL_TRANSACTIONS_QUEUE_KEY, + WITHDRAWAL_TRANSACTIONS_SUM_AMOUNT_TREE_KEY, }; use crate::drive::RootTree; use crate::structure::{ElementKind, KeyEncoding, KeyMatcher, StructureNode}; @@ -158,5 +159,30 @@ pub(crate) fn structure() -> StructureNode { .value("credits") .describe("Credits that entered Platform at that time."), ), + StructureNode::fixed( + "core_credit_pool_balances", + &WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY, + "CoreCreditPoolBalances", + "WITHDRAWAL_CORE_CREDIT_POOL_BALANCES_KEY", + ) + .kind(ElementKind::Tree) + .since(14) + .source("packages/rs-drive/src/drive/identity/withdrawals/paths.rs") + .describe( + "Core's credit pool balance after each Core block \ + read, for the Core-anchored withdrawal limit.", + ) + .child( + StructureNode::dynamic( + "balance", + "core_height", + KeyMatcher::Len(4), + KeyEncoding::U32Be, + "The Core block height", + ) + .kind(ElementKind::Item) + .value("credits, u64 big endian") + .describe("The credit pool balance after that Core block."), + ), ]) } diff --git a/packages/rs-drive/src/drive/initialization/v4/mod.rs b/packages/rs-drive/src/drive/initialization/v4/mod.rs index 78dc92a1b23..fb4cf1b4f39 100644 --- a/packages/rs-drive/src/drive/initialization/v4/mod.rs +++ b/packages/rs-drive/src/drive/initialization/v4/mod.rs @@ -115,9 +115,17 @@ impl Drive { // it expires, and the lifetime storage fee pools sum tree under `Pools`, which holds // their storage fees until an epoch change spreads them. After the batch apply, which // creates `Misc` and the fee pools under `Pools`, and through the same helper as the - // upgrade path (`Platform::transition_to_version_14`), in the same position: last. + // upgrade path (`Platform::transition_to_version_14`), in the same position: just + // before the withdrawal limit trees. self.insert_document_ttl_trees(transaction, platform_version)?; + // Withdrawal limit trees (protocol version 14): the total credits history, the credit + // inflows and the Core credit pool balances under the withdrawals tree, which the batch + // apply creates. Inserted one after the other through the same helper as the upgrade + // path (`Platform::transition_to_version_14`), in the same position: last, so the + // withdrawals Merk is built by the same sequence of inserts on both node populations. + self.insert_withdrawal_limit_trees(transaction, platform_version)?; + Ok(()) } } diff --git a/packages/rs-drive/src/structure/tests.rs b/packages/rs-drive/src/structure/tests.rs index 328e32306bb..2b1f1900625 100644 --- a/packages/rs-drive/src/structure/tests.rs +++ b/packages/rs-drive/src/structure/tests.rs @@ -1223,6 +1223,16 @@ mod fixtures { conformance_of(&drive, "address_balances", run); } + /// The Core-anchored withdrawal accounting: a recorded Core credit pool balance. + fn core_anchored_withdrawal_accounting(run: &mut FixtureRun) { + let platform_version = PlatformVersion::latest(); + let drive = setup_drive_with_initial_state_structure(Some(platform_version)); + drive + .record_core_credit_pool_blocks(&[(100, 5_000_000)], None, platform_version) + .expect("expected to record a Core block"); + conformance_of(&drive, "core_anchored_withdrawal_accounting", run); + } + /// An epoch while it runs, then after it was paid out: payout deletes the /// proposers and both fee items and keeps the epoch tree. The finished /// epoch info is written at payout, so no epoch ever holds all nine keys. @@ -1840,6 +1850,7 @@ mod fixtures { contract_with_team_actions(&mut run); tokens_and_group_actions(&mut run); address_balances(&mut run); + core_anchored_withdrawal_accounting(&mut run); current_then_paid_epoch(&mut run); contested_documents(&mut run); token_distributions(&mut run); diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/mod.rs index 0e39efc722d..2bccfe9f418 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/mod.rs @@ -11,6 +11,10 @@ pub struct DPPMethodVersions { pub deduct_fee_from_outputs_or_remaining_balance_of_inputs: FeatureVersion, pub compute_minimum_shielded_fee: FeatureVersion, pub shielded_extra_sighash_data: FeatureVersion, + /// The Core-anchored withdrawal limit: how much Core's credit pool may still drop given its + /// balance now and at the window start (`SystemLimits::core_credit_pool_unlock_limit_percent` + /// and `core_credit_pool_unlock_limit_floor`). Exists from protocol version 14. + pub core_credit_pool_unlock_limit: OptionalFeatureVersion, /// The preimage the outputs-only bundles of the credit pool (`Shield`, `ShieldFromIdentity`, /// `ShieldFromAssetLock`) bind into their Orchard sighash. `None` on versions that predate the /// binding: they bind nothing, which is what every shipped verifier expects. `Some(0)`: they diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v1.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v1.rs index f2ddc819b9c..74421138b6e 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v1.rs @@ -5,5 +5,6 @@ pub const DPP_METHOD_VERSIONS_V1: DPPMethodVersions = DPPMethodVersions { deduct_fee_from_outputs_or_remaining_balance_of_inputs: 0, compute_minimum_shielded_fee: 0, shielded_extra_sighash_data: 0, + core_credit_pool_unlock_limit: None, credit_pool_bundle_binding: None, }; diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v2.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v2.rs index ba78347f6f9..1977f95c9eb 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v2.rs @@ -5,5 +5,6 @@ pub const DPP_METHOD_VERSIONS_V2: DPPMethodVersions = DPPMethodVersions { deduct_fee_from_outputs_or_remaining_balance_of_inputs: 0, compute_minimum_shielded_fee: 0, shielded_extra_sighash_data: 0, + core_credit_pool_unlock_limit: None, credit_pool_bundle_binding: None, }; diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v3.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v3.rs index 8d1e13f480e..39a14dc0ba6 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v3.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_method_versions/v3.rs @@ -2,7 +2,8 @@ use crate::version::dpp_versions::dpp_method_versions::DPPMethodVersions; /// DPP method versions 3. Introduced in protocol v14: `daily_withdrawal_limit` 1 → 2 replaces the /// flat daily withdrawal limit with a percentage of the total credits Platform held a day ago -/// (`SystemLimits::daily_withdrawal_limit_percent`), and `credit_pool_bundle_binding` +/// (`SystemLimits::daily_withdrawal_limit_percent`), `core_credit_pool_unlock_limit` `None -> +/// Some(0)` adds the Core-anchored withdrawal limit, and `credit_pool_bundle_binding` /// `None` → `Some(0)` binds a kind tag and an owner into the credit pool's outputs-only bundles. /// Everything else matches V2. pub const DPP_METHOD_VERSIONS_V3: DPPMethodVersions = DPPMethodVersions { @@ -11,5 +12,6 @@ pub const DPP_METHOD_VERSIONS_V3: DPPMethodVersions = DPPMethodVersions { deduct_fee_from_outputs_or_remaining_balance_of_inputs: 0, compute_minimum_shielded_fee: 0, shielded_extra_sighash_data: 0, + core_credit_pool_unlock_limit: Some(0), credit_pool_bundle_binding: Some(0), }; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/mod.rs index b24ad0e43c0..111ec0c7211 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/mod.rs @@ -174,6 +174,14 @@ pub struct DriveAbciIdentityCreditWithdrawalMethodVersions { /// limit's daily maximum; exists from protocol version 14. pub record_credit_inflows_for_withdrawals: OptionalFeatureVersion, pub record_total_credits_history_for_withdrawals: OptionalFeatureVersion, + /// Reads the Core blocks the chain lock height advanced over and records each one's credit + /// pool balance. Feeds the Core-anchored withdrawal limit; called by + /// `pool_withdrawals_into_transactions_queue` 2; exists from protocol version 14. + pub scan_core_blocks_for_withdrawals: OptionalFeatureVersion, + /// How much more Core's credit pool may give up to withdrawals pooled now: a stricter copy + /// of Core's own unlock limit, less what is queued or broadcast and not completed yet. + /// Exists from protocol version 14. + pub calculate_core_anchored_withdrawal_limit: OptionalFeatureVersion, /// Whether the next block has withdrawal work waiting (queued transactions to sign or expired /// documents to re-queue); drives the `propose_next_block_immediately` hint to Tenderdash. /// Not consensus: it never touches the state or the app hash. diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v1.rs index da0c6e7a88c..e0c2edd2657 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v1.rs @@ -91,6 +91,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V1: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v10.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v10.rs index 29c9e4a39ef..76f2e022cc1 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v10.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v10.rs @@ -16,9 +16,14 @@ use crate::version::drive_abci_versions::drive_abci_method_versions::{ /// `record_total_credits_history_for_withdrawals` (`Some(0)`), the per-block record of the total /// credits in Platform that the day-lagged daily withdrawal limit reads, and bumps /// `cleanup_expired_locks_of_withdrawal_amounts` to 1 so the per-block cleanup also prunes the -/// expired entries of the credit inflows sum tree the net daily withdrawal limit reads, and +/// expired entries of the credit inflows sum tree the net daily withdrawal limit reads and the +/// Core credit pool balances older than the Core-anchored limit's window, and /// bumps `rebroadcast_expired_withdrawal_documents` to 2 so an expired withdrawal whose /// payout is below Core's dust threshold is marked FAILED instead of re-signed forever. +/// `pool_withdrawals_into_transactions_queue` 2 pools only what also fits the Core-anchored +/// withdrawal limit (`calculate_core_anchored_withdrawal_limit`, `Some(0)`), fed by +/// `scan_core_blocks_for_withdrawals` (`Some(0)`), which records Core's credit pool balance per +/// Core block. /// `decode_raw_state_transitions` 1 refuses bytes left over after a raw state transition. /// `add_distribute_storage_fee_to_epochs_operations` 1 claws each pending storage refund back /// from the epochs it was priced for. @@ -102,14 +107,16 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V10: DriveAbciMethodVersions = DriveAbciMet build_untied_withdrawal_transactions_from_documents: 0, dequeue_and_build_unsigned_withdrawal_transactions: 0, fetch_transactions_block_inclusion_status: 0, - pool_withdrawals_into_transactions_queue: 1, + pool_withdrawals_into_transactions_queue: 2, // changed in v14: pools only what also fits the Core-anchored limit update_broadcasted_withdrawal_statuses: 0, rebroadcast_expired_withdrawal_documents: 2, // changed in v14: an expired withdrawal whose payout is Core dust is marked FAILED instead of re-signed append_signatures_and_broadcast_withdrawal_transactions: 0, has_pending_withdrawal_work: 0, - cleanup_expired_locks_of_withdrawal_amounts: 1, // changed in v14: also prunes expired entries of the credit inflows sum tree + cleanup_expired_locks_of_withdrawal_amounts: 1, // changed in v14: also prunes expired entries of the credit inflows sum tree and old Core credit pool balances record_credit_inflows_for_withdrawals: Some(0), // new in v14: the block's credit mints recorded as an inflow for the net daily withdrawal limit record_total_credits_history_for_withdrawals: Some(0), // changed in v14: per-block total credits history for the day-lagged daily withdrawal limit + scan_core_blocks_for_withdrawals: Some(0), // new in v14: Core credit pool balances for the Core-anchored withdrawal limit + calculate_core_anchored_withdrawal_limit: Some(0), // new in v14: withdrawals also fit a stricter copy of Core's unlock limit }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v2.rs index bb31f850ab3..f0c138b912a 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v2.rs @@ -92,6 +92,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V2: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v3.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v3.rs index a9bee34837c..b1232c723af 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v3.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v3.rs @@ -91,6 +91,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V3: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v4.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v4.rs index 4a3de319a20..16389b8230d 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v4.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v4.rs @@ -91,6 +91,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V4: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v5.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v5.rs index f252dbf851d..8af13d666c2 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v5.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v5.rs @@ -95,6 +95,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V5: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v6.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v6.rs index 5351e5dad31..61b49f620e7 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v6.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v6.rs @@ -93,6 +93,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V6: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v7.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v7.rs index ab7b39e7b9c..6706865c4e3 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v7.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v7.rs @@ -100,6 +100,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V7: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v8.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v8.rs index ad6918635ca..6773f557a58 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v8.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v8.rs @@ -102,6 +102,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V8: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v9.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v9.rs index bd58eca313d..d34b017e96e 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v9.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v9.rs @@ -111,6 +111,8 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V9: DriveAbciMethodVersions = DriveAbciMeth cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/mod.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/mod.rs index 9f5f046d8b6..56208d9b2f9 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/mod.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/mod.rs @@ -9,4 +9,8 @@ pub struct DriveAbciWithdrawalConstants { /// Maximum number of entries `record_total_credits_history_for_withdrawals` prunes from /// the total credits history per block (`0` disables pruning). pub total_credits_history_prune_limit: u16, + /// Maximum number of Core blocks `scan_core_blocks_for_withdrawals` reads per Platform + /// block. When the chain lock height jumps further, the rest is read in the blocks after + /// (`0` disables the scan; protocol versions before 14 have no scan). + pub core_blocks_scanned_per_block_limit: u16, } diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v1.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v1.rs index 72e24c9305c..1d047eb7c4e 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v1.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v1.rs @@ -4,5 +4,6 @@ pub const DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V1: DriveAbciWithdrawalConstants = DriveAbciWithdrawalConstants { core_expiration_blocks: 48, total_credits_history_prune_limit: 0, + core_blocks_scanned_per_block_limit: 0, cleanup_expired_locks_of_withdrawal_amounts_limit: 0, }; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v2.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v2.rs index 66abe011053..5f970d64a62 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v2.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v2.rs @@ -4,5 +4,6 @@ pub const DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V2: DriveAbciWithdrawalConstants = DriveAbciWithdrawalConstants { core_expiration_blocks: 48, total_credits_history_prune_limit: 0, + core_blocks_scanned_per_block_limit: 0, cleanup_expired_locks_of_withdrawal_amounts_limit: 64, }; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v3.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v3.rs index 350f3b04280..68546db7cfc 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v3.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_withdrawal_constants/v3.rs @@ -3,10 +3,12 @@ use crate::version::drive_abci_versions::drive_abci_withdrawal_constants::DriveA /// Withdrawal constants for protocol version 14 and above: identical to /// [`super::v2::DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V2`] plus `total_credits_history_prune_limit`, /// bounding how many stale entries the per-block total credits history (the base of the -/// day-lagged daily withdrawal limit) drops per block. +/// day-lagged daily withdrawal limit) drops per block, and `core_blocks_scanned_per_block_limit`, +/// bounding how many Core blocks the Core-anchored withdrawal limit's scan reads per block. pub const DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V3: DriveAbciWithdrawalConstants = DriveAbciWithdrawalConstants { core_expiration_blocks: 48, cleanup_expired_locks_of_withdrawal_amounts_limit: 64, total_credits_history_prune_limit: 64, + core_blocks_scanned_per_block_limit: 32, }; diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs index 7b99529799e..ae534ebf94f 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs @@ -27,6 +27,14 @@ pub struct DriveIdentityWithdrawalMethodVersions { /// tree the net daily withdrawal limit reads back. The subtree exists from protocol /// version 14. pub record_credit_inflows: OptionalFeatureVersion, + /// Record Core blocks' credit pool balances for the Core-anchored withdrawal limit. The + /// subtree exists from protocol version 14. + pub record_core_credit_pool_blocks: OptionalFeatureVersion, + /// Read the recorded Core credit pool balances. Exists from protocol version 14. + pub fetch_core_credit_pool_balances: OptionalFeatureVersion, + /// Sum what the queued and broadcast withdrawal transactions take out of Core's credit + /// pool once mined. Exists from protocol version 14. + pub fetch_in_flight_withdrawal_amount: OptionalFeatureVersion, } #[derive(Clone, Debug, Default)] diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs index c2e4ba4b933..f2db37e35d9 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs @@ -171,5 +171,8 @@ pub const DRIVE_IDENTITY_METHOD_VERSIONS_V1: DriveIdentityMethodVersions = record_total_credits_history: None, fetch_total_credits_in_platform_a_day_ago: None, record_credit_inflows: None, + record_core_credit_pool_blocks: None, + fetch_core_credit_pool_balances: None, + fetch_in_flight_withdrawal_amount: None, }, }; diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs index 6b2cde9a209..d189ab87895 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs @@ -51,9 +51,8 @@ use crate::version::drive_versions::drive_identity_method_versions::{ /// * `withdrawals.calculate_current_withdrawal_limit` 0 -> 1: the daily /// maximum derives from the total credits Platform held a day ago (the /// relative daily withdrawal limit) instead of the current total. The -/// `max_daily_withdrawal_amount` cap applies to that day-old base; credit -/// inflows from the active window are added after the cap so matching -/// deposit-withdraw cycles do not consume the capped budget. +/// credit inflows from the active window are added on top so matching +/// deposit-withdraw cycles do not consume the budget of others. /// * `withdrawals.record_total_credits_history` and /// `withdrawals.fetch_total_credits_in_platform_a_day_ago` `None -> Some(0)`: /// the per-block total credits history under the withdrawals tree that the @@ -64,6 +63,13 @@ use crate::version::drive_versions::drive_identity_method_versions::{ /// tree so the daily withdrawal limit counts net outflow instead of gross — /// a deposit -> withdraw cycle no longer consumes the budget of other users. /// The subtree does not exist before v14, so V1 keeps the slot `None`. +/// * `withdrawals.record_core_credit_pool_blocks`, +/// `withdrawals.fetch_core_credit_pool_balances` and +/// `withdrawals.fetch_in_flight_withdrawal_amount` `None -> Some(0)`: the +/// Core-anchored withdrawal limit, which reads Core's credit pool balance as +/// recorded per Core block and what pooled withdrawals take out of the pool +/// once mined. The subtree does not exist before v14, so V1 keeps the slots +/// `None`. pub const DRIVE_IDENTITY_METHOD_VERSIONS_V2: DriveIdentityMethodVersions = DriveIdentityMethodVersions { fetch: DriveIdentityFetchMethodVersions { @@ -222,5 +228,8 @@ pub const DRIVE_IDENTITY_METHOD_VERSIONS_V2: DriveIdentityMethodVersions = record_total_credits_history: Some(0), // new in v14: total credits history for the day-lagged daily withdrawal limit fetch_total_credits_in_platform_a_day_ago: Some(0), // new in v14 record_credit_inflows: Some(0), // new in v14: credit inflows sum tree for the net daily withdrawal limit + record_core_credit_pool_blocks: Some(0), // new in v14: Core credit pool balances for the Core-anchored withdrawal limit + fetch_core_credit_pool_balances: Some(0), // new in v14 + fetch_in_flight_withdrawal_amount: Some(0), // new in v14 }, }; diff --git a/packages/rs-platform-version/src/version/mocks/v2_test.rs b/packages/rs-platform-version/src/version/mocks/v2_test.rs index 3595a1c3ff0..c55d774b659 100644 --- a/packages/rs-platform-version/src/version/mocks/v2_test.rs +++ b/packages/rs-platform-version/src/version/mocks/v2_test.rs @@ -598,7 +598,10 @@ pub const TEST_PLATFORM_V2: PlatformVersion = PlatformVersion { retry_signing_expired_withdrawal_documents_per_block_limit: 1, max_withdrawal_amount: 50_000_000_000_000, daily_withdrawal_limit_percent: None, - max_daily_withdrawal_amount: None, + core_credit_pool_unlock_limit_percent: None, + core_credit_pool_unlock_limit_floor: None, + core_credit_pool_window_blocks: None, + regtest_core_credit_pool_window_blocks: None, min_withdrawal_amount: 190_000, core_dust_relay_fee_per_kb: None, max_core_fee_per_byte: None, diff --git a/packages/rs-platform-version/src/version/mocks/v3_test.rs b/packages/rs-platform-version/src/version/mocks/v3_test.rs index 978528f03ca..6062953e1b4 100644 --- a/packages/rs-platform-version/src/version/mocks/v3_test.rs +++ b/packages/rs-platform-version/src/version/mocks/v3_test.rs @@ -126,6 +126,8 @@ pub const TEST_PLATFORM_V3: PlatformVersion = PlatformVersion { cleanup_expired_locks_of_withdrawal_amounts: 0, record_credit_inflows_for_withdrawals: None, record_total_credits_history_for_withdrawals: None, + scan_core_blocks_for_withdrawals: None, + calculate_core_anchored_withdrawal_limit: None, }, voting: DriveAbciVotingMethodVersions { keep_record_of_finished_contested_resource_vote_poll: 0, diff --git a/packages/rs-platform-version/src/version/system_limits/mod.rs b/packages/rs-platform-version/src/version/system_limits/mod.rs index d5db9fdef43..d9897b9769d 100644 --- a/packages/rs-platform-version/src/version/system_limits/mod.rs +++ b/packages/rs-platform-version/src/version/system_limits/mod.rs @@ -167,13 +167,33 @@ pub struct SystemLimits { /// version 1 applied a flat 2000 Dash. Versioned: see `daily_withdrawal_limit_percent` in /// each `SYSTEM_LIMITS_V*`. pub daily_withdrawal_limit_percent: Option, - /// Upper bound (in credits) of the relative daily withdrawal limit from protocol version 14: - /// Core's credit-pool unlock capacity per day, `LimitAmountV24` = 4000 Dash per 576-block - /// window (Core v24). Platform cannot usefully pool more than Core will mine — the excess - /// only cycles through expiry and re-signing — so the limit never exceeds this whatever the - /// total credits are; raise it together with Core. Must be at least `max_withdrawal_amount`. - /// `None` for the protocol versions that predate the relative rule. - pub max_daily_withdrawal_amount: Option, + /// Allowed drop of Core's credit pool per window, as a percentage of its balance at the + /// window start, in the Core-anchored withdrawal limit of protocol version 14, read by + /// `core_credit_pool_unlock_limit` method version 0. Platform pools a withdrawal only while + /// it also fits this limit, a stricter copy of Core v24's own unlock rule (20%, at least + /// 2000 Dash), so it does not pool more than Core will mine. `None` for the protocol + /// versions that predate the Core-anchored limit. + pub core_credit_pool_unlock_limit_percent: Option, + /// Smallest allowed drop (in credits) of Core's credit pool per window in the Core-anchored + /// withdrawal limit, applied when `core_credit_pool_unlock_limit_percent` of the window start + /// balance is less; read by `core_credit_pool_unlock_limit` method version 0. Below Core + /// v24's own 2000 Dash floor, so small pools keep a margin too, and at least + /// `max_withdrawal_amount` so a queued withdrawal always fits eventually. `None` for the + /// protocol versions that predate the Core-anchored limit. + pub core_credit_pool_unlock_limit_floor: Option, + /// Core's credit pool window on mainnet, testnet and devnets (`CreditPoolPeriodBlocks` in + /// Dash Core's chain parameters): how many Core blocks before an asset unlock's block lies + /// the balance Core v24 measures the unlock limit from. The Core-anchored withdrawal limit + /// reads its window starts this far back, up to Core's asset unlock validity + /// (`withdrawal_constants.core_expiration_blocks`) later, so the window must be at least + /// that long, and recorded balances older than it are pruned. Read through + /// `core_credit_pool_window_blocks` in dpp. `None` for the protocol versions that predate + /// the Core-anchored limit. + pub core_credit_pool_window_blocks: Option, + /// Core's credit pool window on regtest, which Dash Core shortens; see + /// `core_credit_pool_window_blocks`. `None` for the protocol versions that predate the + /// Core-anchored limit. + pub regtest_core_credit_pool_window_blocks: Option, /// Minimum net amount (in credits) a withdrawal may send to Core, shared by the /// transparent (identity + address) and shielded withdrawal paths. The dust floor that /// keeps Core from rejecting the resulting `TxOut`. Versioned: see `min_withdrawal_amount` @@ -465,6 +485,76 @@ mod tests { ); } + /// The Core-anchored withdrawal limit never drops below its floor, and pooling stops at the + /// first queued withdrawal that does not fit: a floor below one maximal withdrawal would let + /// a maximal withdrawal wait forever on a small credit pool, with everything queued behind + /// it. + #[test] + fn should_keep_the_core_credit_pool_floor_at_least_one_maximal_withdrawal() { + let with_a_floor: Vec<_> = PLATFORM_VERSIONS + .iter() + .filter_map(|platform_version| { + platform_version + .system_limits + .core_credit_pool_unlock_limit_floor + .map(|floor| (platform_version, floor)) + }) + .collect(); + assert!( + !with_a_floor.is_empty(), + "no protocol version sets a Core credit pool floor; this test would assert nothing" + ); + for (platform_version, floor) in with_a_floor { + assert!( + floor >= platform_version.system_limits.max_withdrawal_amount, + "protocol version {} sets a Core credit pool floor of {} credits, below one \ + maximal withdrawal ({} credits)", + platform_version.protocol_version, + floor, + platform_version.system_limits.max_withdrawal_amount + ); + } + } + + /// The Core-anchored withdrawal limit reads window starts from the chain locked height back + /// by Core's credit pool window up to Core's asset unlock validity later. A window shorter + /// than that validity would put the nearest window start above the chain locked height, + /// whose balance is not final and so not the same on every node. + #[test] + fn should_keep_every_core_credit_pool_window_at_least_the_unlock_validity() { + let with_a_window: Vec<_> = PLATFORM_VERSIONS + .iter() + .flat_map(|platform_version| { + let system_limits = &platform_version.system_limits; + [ + system_limits.core_credit_pool_window_blocks, + system_limits.regtest_core_credit_pool_window_blocks, + ] + .into_iter() + .flatten() + .map(move |window_blocks| (platform_version, window_blocks)) + }) + .collect(); + assert!( + !with_a_window.is_empty(), + "no protocol version sets a Core credit pool window; this test would assert nothing" + ); + for (platform_version, window_blocks) in with_a_window { + let unlock_validity_blocks = platform_version + .drive_abci + .withdrawal_constants + .core_expiration_blocks; + assert!( + window_blocks >= unlock_validity_blocks, + "protocol version {} sets a Core credit pool window of {} blocks, shorter than \ + Core's asset unlock validity ({} blocks)", + platform_version.protocol_version, + window_blocks, + unlock_validity_blocks + ); + } + } + /// The withdrawal structure generations selected from protocol version 14 read the cap /// through `dpp::withdrawal::validate_core_fee_per_byte_cap`, which treats `None` as "no /// cap" per the field's contract. A table that selected one of those generations without a diff --git a/packages/rs-platform-version/src/version/system_limits/v1.rs b/packages/rs-platform-version/src/version/system_limits/v1.rs index 8067c8a3353..1d224dff3a5 100644 --- a/packages/rs-platform-version/src/version/system_limits/v1.rs +++ b/packages/rs-platform-version/src/version/system_limits/v1.rs @@ -42,7 +42,10 @@ pub const SYSTEM_LIMITS_V1: SystemLimits = SystemLimits { retry_signing_expired_withdrawal_documents_per_block_limit: 1, max_withdrawal_amount: 50_000_000_000_000, //500 Dash daily_withdrawal_limit_percent: None, // relative daily withdrawal limit arrives in v14 - max_daily_withdrawal_amount: None, + core_credit_pool_unlock_limit_percent: None, + core_credit_pool_unlock_limit_floor: None, + core_credit_pool_window_blocks: None, + regtest_core_credit_pool_window_blocks: None, // = dpp MIN_WITHDRAWAL_AMOUNT: ASSET_UNLOCK_TX_SIZE(190) * MIN_CORE_FEE_PER_BYTE(1) // * CREDITS_PER_DUFF(1000) = 190_000 credits = 190 duffs. min_withdrawal_amount: 190_000, diff --git a/packages/rs-platform-version/src/version/system_limits/v2.rs b/packages/rs-platform-version/src/version/system_limits/v2.rs index 42379729ab8..69d871f128b 100644 --- a/packages/rs-platform-version/src/version/system_limits/v2.rs +++ b/packages/rs-platform-version/src/version/system_limits/v2.rs @@ -25,7 +25,10 @@ pub const SYSTEM_LIMITS_V2: SystemLimits = SystemLimits { retry_signing_expired_withdrawal_documents_per_block_limit: 1, max_withdrawal_amount: 50_000_000_000_000, //500 Dash daily_withdrawal_limit_percent: None, // relative daily withdrawal limit arrives in v14 - max_daily_withdrawal_amount: None, + core_credit_pool_unlock_limit_percent: None, + core_credit_pool_unlock_limit_floor: None, + core_credit_pool_window_blocks: None, + regtest_core_credit_pool_window_blocks: None, min_withdrawal_amount: 1_000_000, //1000 duffs (raised from 190 in v12) core_dust_relay_fee_per_kb: None, // expired dust withdrawals fail from v14 max_core_fee_per_byte: None, diff --git a/packages/rs-platform-version/src/version/system_limits/v3.rs b/packages/rs-platform-version/src/version/system_limits/v3.rs index 4f3482d4d0b..a50a426ff32 100644 --- a/packages/rs-platform-version/src/version/system_limits/v3.rs +++ b/packages/rs-platform-version/src/version/system_limits/v3.rs @@ -27,7 +27,10 @@ pub const SYSTEM_LIMITS_V3: SystemLimits = SystemLimits { retry_signing_expired_withdrawal_documents_per_block_limit: 1, max_withdrawal_amount: 50_000_000_000_000, //500 Dash daily_withdrawal_limit_percent: None, // relative daily withdrawal limit arrives in v14 - max_daily_withdrawal_amount: None, + core_credit_pool_unlock_limit_percent: None, + core_credit_pool_unlock_limit_floor: None, + core_credit_pool_window_blocks: None, + regtest_core_credit_pool_window_blocks: None, min_withdrawal_amount: 1_000_000, //1000 duffs (raised from 190 in v12) core_dust_relay_fee_per_kb: None, // expired dust withdrawals fail from v14 max_core_fee_per_byte: None, diff --git a/packages/rs-platform-version/src/version/system_limits/v4.rs b/packages/rs-platform-version/src/version/system_limits/v4.rs index b4718b13a39..71468c61bb4 100644 --- a/packages/rs-platform-version/src/version/system_limits/v4.rs +++ b/packages/rs-platform-version/src/version/system_limits/v4.rs @@ -21,10 +21,18 @@ use crate::version::system_limits::SystemLimits; /// /// * The daily withdrawal limit becomes relative: `daily_withdrawal_limit_percent` is set to 15, /// so Platform pools at most 15% of the total credits it held a day ago into asset unlock -/// transactions per 24 hours — never below one maximal withdrawal and never above -/// `max_daily_withdrawal_amount`, Core's 4000 Dash unlock capacity per day — instead of the -/// flat 2000 Dash that applied from v8 (matching Core v22's `LimitAmountV22`). v13 is already -/// live on networks with the flat limit, so the change gates here. +/// transactions per 24 hours, never below one maximal withdrawal, instead of the flat 2000 +/// Dash that applied from v8 (matching Core v22's `LimitAmountV22`). v13 is already live on +/// networks with the flat limit, so the change gates here. The limit has no fixed cap. +/// * Withdrawals also fit a Core-anchored limit, a stricter copy of Core v24's relative net +/// unlock rule read from Core's own credit pool balances: the pool may drop by at most +/// `core_credit_pool_unlock_limit_percent` (15, Core allows 20) of its highest balance at a +/// window start Core may use for the unlock (Core's window, `core_credit_pool_window_blocks` +/// 576 or `regtest_core_credit_pool_window_blocks` 100, back from the chain locked height, +/// up to Core's asset unlock validity, `withdrawal_constants.core_expiration_blocks` 48, +/// later), at least +/// `core_credit_pool_unlock_limit_floor` (1500 Dash, Core's floor is 2000), less what is +/// queued or broadcast and not completed yet. /// * `max_time_range_overlap_factor` is set: a `timeRange` index transform may declare at most /// 24 overlapping windows per timestamp (a day-long window sliding hourly). The rule cannot /// exist before v14 because the `timeRange` keyword itself is only admitted by the v14 @@ -119,8 +127,11 @@ pub const SYSTEM_LIMITS_V4: SystemLimits = SystemLimits { retry_signing_expired_withdrawal_documents_per_block_limit: 1, max_withdrawal_amount: 50_000_000_000_000, //500 Dash daily_withdrawal_limit_percent: Some(15), // 15% of the total credits a day ago (replaces the flat 2000 Dash in v14) - max_daily_withdrawal_amount: Some(400_000_000_000_000), // 4000 Dash: Core's unlock capacity per day (LimitAmountV24) - min_withdrawal_amount: 1_000_000, //1000 duffs (raised from 190 in v12) + core_credit_pool_unlock_limit_percent: Some(15), // Core v24 allows 20% of the pool a window ago + core_credit_pool_unlock_limit_floor: Some(150_000_000_000_000), // 1500 Dash; Core v24's floor is 2000 Dash + core_credit_pool_window_blocks: Some(576), // Core's credit pool window (CreditPoolPeriodBlocks), mainnet, testnet and devnets + regtest_core_credit_pool_window_blocks: Some(100), // Core's credit pool window on regtest + min_withdrawal_amount: 1_000_000, //1000 duffs (raised from 190 in v12) core_dust_relay_fee_per_kb: Some(3000), // Core's default dust relay fee: 546-duff P2PKH threshold; expired withdrawals below it fail instead of re-signing max_core_fee_per_byte: Some(6_765), max_group_member_count: 256, diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index cbea2079199..00fb82acb16 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -77,17 +77,14 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// (`SYSTEM_LIMITS_V4.daily_withdrawal_limit_percent`, read by /// `daily_withdrawal_limit` v2 through `DPP_METHOD_VERSIONS_V3`), never below /// one maximal withdrawal (`max_withdrawal_amount`) so every accepted -/// withdrawal eventually fits and cannot block the pooling queue. The base is -/// capped at `max_daily_withdrawal_amount` (4000 Dash, Core's unlock capacity -/// per day under V24 as written); the credit inflows of the active window — -/// every credit mint, recorded per block by -/// `record_credit_inflows_for_withdrawals` in the credit inflows sum tree — -/// are added after the cap, so the limit counts net outflow and a matching -/// deposit -> withdraw cycle does not consume the capped budget of other -/// users (#4471). Outflow funded by same-window deposits may therefore -/// exceed the cap; this mirrors the net credit-pool rule Core adopts for V24 -/// alongside this change (tracked in #4471), which must land before V24 -/// activates. Both the inflows and the pooled reservations count over the +/// withdrawal eventually fits and cannot block the pooling queue. The base has +/// no fixed cap: what Core will mine bounds pooling through the Core-anchored +/// limit of note 74 instead. The credit inflows of the active window — every +/// credit mint, recorded per block by `record_credit_inflows_for_withdrawals` +/// in the credit inflows sum tree — are added to the base, so the limit +/// counts net outflow and a matching deposit -> withdraw cycle does not +/// consume the budget of other users (#4471), mirroring Core v24's net +/// credit-pool rule. Both the inflows and the pooled reservations count over the /// interval after the base snapshot only — an entry the snapshot already /// reflects is neither added nor subtracted again. The base is /// the total credits recorded at the latest block at least 24 hours before @@ -105,9 +102,8 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// already pooled in the last 24 hours keep counting against the maximum /// exactly as before. Pre-V24 Core caps unlocks at `LimitAmountV22` (2000 /// Dash) per *block*, with the amount checked only at block level, so any -/// daily total is still minable across blocks; V24's 4000 Dash per 576-block -/// window matches the capped base and is raised to the same net rule before -/// activation (see above). +/// daily total is still minable across blocks; V24 limits the net drop of +/// its credit pool per 576-block window, which note 74 follows. /// 5. **Time-range indexes**: an index can declare a `timeRange` transform /// that buckets a required system timestamp (`$createdAt` / /// `$updatedAt` / `$transferredAt`) into fixed-length, regularly-spaced, @@ -1927,7 +1923,6 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// the key of an index a delete clears that skips nothing, so no two /// documents in state share one of its entries. Inert for every contract /// without the keyword, which every earlier grammar refuses. -/// /// 70. **A contested type sums only small values**: parser generation 3, in /// place, refuses under full validation a document type with a contested /// index and a summed property (`summable`, `averageable`, @@ -1938,7 +1933,6 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// into the type's sums with no transition to refuse, so the values must be /// small enough that the sums stay in `i64`, which they do short of 2^36 /// documents. A stored contract still parses. -/// /// 71. **Documents deleted only when consumed (`canBeDeleted: /// "onlyWhenConsumed"`)**: a third `canBeDeleted` value of meta-schema v3 /// and parser generation 3, in place @@ -2035,6 +2029,34 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// proofs, and the unproven total fails, as released; the prover is /// unchanged. /// +/// 74. **Withdrawals also fit a Core-anchored limit**: pooling +/// (`pool_withdrawals_into_transactions_queue` 2, which reuses version 1's +/// pooling through a shared helper) admits withdrawals up to the smaller of +/// the daily withdrawal limit (note 4) and +/// `calculate_core_anchored_withdrawal_limit`, a stricter copy of Core v24's +/// relative net unlock rule (dash#7712) read from Core's own credit pool +/// balances at chain locked heights: the pool may drop by at most +/// `core_credit_pool_unlock_limit_percent` (15; Core allows 20) of its +/// highest balance at a window start Core may use for the unlock (Core's +/// window, `core_credit_pool_window_blocks` 576 or +/// `regtest_core_credit_pool_window_blocks` 100, back from the chain locked +/// height, up to Core's asset unlock validity, `core_expiration_blocks` 48, +/// later), at least +/// `core_credit_pool_unlock_limit_floor` (1500 Dash; Core's floor is 2000), +/// less what is queued or broadcast and not completed yet. The formula is +/// `core_credit_pool_unlock_limit` 0 in `DPP_METHOD_VERSIONS_V3`. Before +/// pooling, `scan_core_blocks_for_withdrawals` reads the Core blocks the +/// chain locked height passed (at most `core_blocks_scanned_per_block_limit`, +/// 32, per block) and records each one's credit pool balance, read from the +/// block's coinbase alone (`getspecialtxes`), under the withdrawals tree. The +/// Platform-side accounting can grant more than Core will mine (an asset lock published to +/// Platform after Core mined it, a whole epoch of Core rewards minted in one +/// block); over Core's limit an unlock waits unmined and is re-signed, and +/// while Core's mempool holds more than the limit Core InstantSend-locks no +/// withdrawal at all. The balance tree is created at genesis and by +/// `transition_to_version_14`, and `cleanup_expired_locks_of_withdrawal_amounts` +/// 1 prunes it by Core height. +/// /// 75. **No reference by id to an indexOnly document type**: the contract /// reference validation 0 (`validate_data_contract_references`), in place, /// refuses a `permanentDocument`, `deletableDocument` or