ChannelDbConnectionPool transaction support

[mdaigle]

Rebased onto main now that #4429 has merged. Unit tests: 834 passed / 0 failed.

Summary

Implements transaction support in ChannelDbConnectionPool, using WaitHandleDbConnectionPool as the reference for correct behavior. Before this change the channel pool constructed a TransactedConnectionPool but never used it, and three IDbConnectionPool members threw NotImplementedException.

Changes

• PutObjectFromTransactedPool (was NotImplementedException) — returns a connection to general circulation once its transaction has ended, or destroys it if the pool is no longer running or the connection can't be pooled.
• TransactionEnded (was NotImplementedException) — delegates to TransactedConnectionPool.TransactionEnded, which calls back into PutObjectFromTransactedPool.
• ReturnInternalConnection — rewritten to mirror WaitHandleDbConnectionPool.DeactivateObject. It now deactivates first (deactivation is what detaches a completed transaction, so reading EnlistedTransaction beforehand could park a connection under an already-ended transaction), then decides under the connection lock between the transacted pool, stasis, the idle channel, and destruction. The idle-channel path moved into a new PutConnectionInIdleChannel helper.
• GetFromTransactedPool (new) — vends a connection already enlisted in the ambient transaction. Transacted connections are exempt from idle-timeout and clear-generation eviction, since closing them would abort a possibly-distributed transaction, so only liveness is checked. A dead transaction root rethrows rather than silently retrying, because its delegated transaction cannot be recovered on another connection.
• GetInternalConnection / PrepareConnection — consult the transacted pool when the pool group has transaction affinity, and pass the ambient transaction through to ActivateConnection.
• Async acquisition path — takes the ambient transaction from taskCompletionSource.Task.AsyncState and threads it explicitly through the open, rather than assigning Transaction.Current on the thread pool thread. See the section below.
• RemoveConnection — no longer disposes a transaction root that is still waiting for its delegated transaction to end (parity with DestroyObject). It comes back through PutObjectFromTransactedPool when the transaction completes.
• ReplaceConnection — no functional change; the two TODO: Full transaction enlistment support (Story 2) markers from ChannelDbConnectionPool replace connection #4429 are removed now that enlistment is wired through.

How connections move between the idle channel and the transacted store

The transacted store (TransactedConnectionPool, keyed by Transaction) reserves a connection for one specific transaction, so reusing it avoids promoting that transaction to a distributed one. The only edge into it is a return while still enlisted; the only edges out are a pop by a caller in the same transaction, or the transaction ending.

Return path (ReturnInternalConnection)

flowchart TD
    R["ReturnInternalConnection"] --> V["ValidateOwnershipAndSetPoolingState"]
    V --> D["DeactivateConnection"]
    D --> Doomed{"IsConnectionDoomed?"}
    Doomed -- yes --> Destroy["RemoveConnection (Destroy)"]
    Doomed -- no --> Poolable{"State is Running and CanBePooled?"}

    Poolable -- no --> Root{"IsTransactionRoot?"}
    Root -- yes --> Stasis["SetInStasis (HeldByTransaction)"]
    Root -- no --> Destroy

    Poolable -- yes --> Enl{"EnlistedTransaction is not null?"}
    Enl -- yes --> Park["PutTransactedObject (HeldByTransaction)"]
    Enl -- no --> Reuse["PutConnectionInIdleChannel (Reuse)"]

Loading

DeactivateConnection runs before EnlistedTransaction is read, because deactivation is what detaches an already-completed transaction. Reading first would park the connection under a transaction that has already ended, and it would never be released.

Transaction end: back to general circulation

flowchart TD
    Sig["System.Transactions signals completion"] --> Where{"where is the connection?"}
    Where -- "parked in the transacted store" --> TE["pool.TransactionEnded"]
    TE --> TCP["TransactedConnectionPool.TransactionEnded removes it from the list"]
    TCP --> Put["PutObjectFromTransactedPool"]
    Where -- "in stasis" --> DTE["DelegatedTransactionEnded then TerminateStasis(true)"]
    DTE --> Put
    Where -- "never parked, still checked out" --> NoOp["no-op: stays with its owner"]

    Put --> Ok{"State is Running and CanBePooled?"}
    Ok -- yes --> Reset["ResetConnection then PutConnectionInIdleChannel"]
    Ok -- no --> Rm["RemoveConnection"]

Loading

A connection in stasis reaches PutObjectFromTransactedPool too, but by definition it got there because the pool was stopping or it was unpoolable, so it always takes the RemoveConnection branch.

A parked connection keeps its _connectionSlots reservation, so it still counts toward Count and MaxPoolSize, but not IdleCount. Only RemoveConnection releases the slot.

Tests

• New ChannelDbConnectionPoolTransactionTest (18 tests): return routing (enlisted, not enlisted, completed transaction, Enlist=false, shut-down pool), vending from the transacted store under the same and different transactions, commit/rollback, completion after shutdown, TransactionEnded for a connection that was never parked, enlistment carry-over through ReplaceConnection, and the ambient-transaction flow cases below. Every test asserts full pool state (Count, IdleCount, transacted count) after each step, and the pool is built with a frozen TimeProvider.
• Removed the stale tests asserting NotImplementedException from the three now-implemented members.

Validation

Unit tests: 834 passed / 0 failed on net9.0.

Manual/integration tests were run against a local SQL Server with Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2 enabled, across TransactionEnlistmentTest, TransactionPoolTest, SQL.TransactionTest, ParallelTransactionsTest, ConnectionPoolTest, PoolBlockPeriodTest and DistributedTransactionTest (48 tests):

Run	Passed	Failed
V1 (WaitHandle) baseline	39	6
V2 without this change	36	9
V2 with this change	37	8

This change fixes three previously failing tests: TestAutoEnlistment_TxScopeNonComplete, TestManualEnlistment_Enlist and TestManualEnlistment_Enlist_TxScopeComplete.

The 6 failures shared with the V1 baseline are all PlatformNotSupportedException: This platform does not support distributed transactions — MSDTC isn't available on the test machine. The 2 remaining failures (ConnectionPoolTest.ReclaimEmancipatedOnOpenTest) are a pre-existing V2 gap: ReclaimEmancipatedObjects has never been implemented in ChannelDbConnectionPool. Both were confirmed by reverting this change and reproducing.

A broader V2 sweep (SqlCommand, AsyncTest, MARSTest, DataReaderTest, ConnectivityTests, WeakRefTest, AdapterTest, ExceptionTest, RetryLogic) gave 213 passed / 9 failed, where all 9 are named-pipe tests that fail identically on V1.

Ambient transaction flow on the async path

Transaction.Current does not flow into a Task.Run unless the TransactionScope was created with TransactionScopeAsyncFlowOption.Enabled (the default is Suppress, which keeps the ambient transaction in thread-static storage). The async open path therefore cannot simply read Transaction.Current — it has to take the transaction from the TaskCompletionSource's AsyncState, which is where SqlConnection.InternalOpenAsync captures it. That is the only site in the repo that constructs a TaskCompletionSource<DbConnectionInternal>, and it always passes the ambient transaction, so the mechanism is reliable (including across OpenAsyncRetry.Retry, which reuses the same TCS).

The original approach was to restore it by assigning ADP.SetCurrentTransaction(...) inside the Task.Run. That is unsafe here: assigning Transaction.Current writes to thread-static storage that ExecutionContext does not unwind, so the transaction outlives the open and is observable by unrelated work later scheduled onto the same thread pool thread — most notably the login-time auto-enlistment that non-pooled connections perform against Transaction.Current. A try/finally restore doesn't fix it either, because the async continuation may resume on a different thread than the one that was polluted. (WaitHandleDbConnectionPool does the same assignment safely because WaitForPendingOpen asserts it is not on a thread pool thread.)

The fix is to never mutate Transaction.Current in the pool: the transaction is captured on the caller's thread and threaded explicitly through GetInternalConnection into GetFromTransactedPool and PrepareConnection. The sync path passes ADP.GetCurrentTransaction() directly, since it runs on the caller's thread.

Four regression tests cover this:

Test	What it pins
GetConnection_Sync_UsesAmbientTransactionFromCallersThread	the sync path reads Transaction.Current, which is correct because it runs on the caller's thread
GetConnectionAsync_UsesAmbientTransactionCapturedOnCallersThread	a scope with AsyncFlowOption.Enabled still enlists
GetConnectionAsync_WithAsyncFlowDisabled_StillEnlistsInAmbientTransaction	a default (Suppress) scope still enlists, even though the transaction provably does not flow off the caller's thread
GetConnectionAsync_EnteredFromThreadWithoutAmbientTransaction_EnlistsFromAsyncState	the transaction comes from AsyncState, not from whatever is ambient on the entering thread — this is the OpenAsyncRetry.Retry case

The last one is load-bearing and not redundant: TryGetConnection reads AsyncState while still on the caller's thread, so in the scope-based tests Transaction.Current happens to agree with AsyncState. Only entering the pool from a thread with no ambient transaction distinguishes them. Verified by mutation — replacing the AsyncState read with null fails 5 tests, and replacing it with ADP.GetCurrentTransaction() fails only the retry-path and no-leak tests.

Checklist

• Tests added or updated
• Public API changes documented — n/a, no public API change
• Verified against customer repro (if applicable) — n/a
• Ensure no breaking changes introduced — behavior is behind the existing UseConnectionPoolV2 switch, which defaults to off

Notes

• ReclaimEmancipatedObjects remains unimplemented in the channel pool; it is orthogonal to transactions and left for a follow-up.

[Copilot]

Pull request overview

Adds full transaction enlistment/routing support to the V2 ChannelDbConnectionPool, aligning behavior with the legacy WaitHandleDbConnectionPool so pooled connections correctly participate in ambient System.Transactions flows (including async acquisition).

Changes:

• Implemented transaction lifecycle plumbing in ChannelDbConnectionPool (PutObjectFromTransactedPool, TransactionEnded, transacted acquisition path, and updated return/deactivation logic).
• Enabled async acquisition to restore the captured ambient transaction on the worker thread (ADP.SetCurrentTransaction(...)).
• Added a comprehensive unit test suite for channel-pool transaction behavior and removed the now-stale NotImplementedException assertions.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTransactionTest.cs	New transaction-focused unit tests for the channel-based pool, mirroring WaitHandle pool coverage and adding channel-specific scenarios.
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTest.cs	Removes tests that asserted transaction methods were unimplemented (now implemented).
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs	Implements transaction support: transacted pool vending/parking, correct return paths for enlisted connections, and async ambient transaction propagation.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTransactionTest.cs:649

• task2Done is declared but never used, which adds noise and makes the synchronization intent harder to follow. Remove it (or use it if it was meant to assert task2 completion).

        using var task2Done = new ManualResetEventSlim(false);

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:1206

• GetInternalConnection creates a CancellationTokenSource even when a connection was successfully retrieved from the transacted pool, which adds avoidable allocations on that hot path. Consider returning early after GetFromTransactedPool succeeds so the CTS/loop is skipped entirely.

            // Derive a CancellationTokenSource from the TimeoutTimer so pool-internal wait operations
            // (channel reads, semaphore waits) are cancelled when the overall budget expires.
            using CancellationTokenSource cancellationTokenSource = timeout.CreateCancellationTokenSource();
            CancellationToken cancellationToken = cancellationTokenSource.Token;

[mdaigle]

Overall, the tests need a lot of cleanup. Verify pool count, idle, general stats and metrics at each step. Remove tests that are a strict subset of other tests. Add comments, think deeply about which tests are really required and provide good coverage. Use code coverage metrics to guide your decisions.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:260

• The new XML doc for HasTransactionAffinity says connections may be "vended from (and parked in)" the TransactedConnectionPool when enabled. The code still parks connections based on connection.EnlistedTransaction in DecideReturnDisposition even when transaction affinity is disabled (e.g., manually enlisted connections), so the doc is misleading about the parking behavior. Consider rewording to clarify that this flag controls automatic transaction affinity (consulting the transacted pool / auto-enlisting on activation), not whether parking can occur at all.

        /// <summary>
        /// Indicates whether connections may be vended from (and parked in) the
        /// <see cref="TransactedConnectionPool"/>. This mirrors automatic transaction enlistment:
        /// when enlistment is disabled a connection is never bound to an ambient transaction, so
        /// the transacted store must not be consulted.
        /// </summary>
        private bool HasTransactionAffinity => PoolGroupOptions.HasTransactionAffinity;

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:260

• The HasTransactionAffinity doc comment currently states the transacted store “must not be consulted” when enlistment is disabled, but the return path still parks any manually-enlisted connection (connection.EnlistedTransaction != null) in the transacted store (matching WaitHandle behavior). The summary should be narrowed to describe ambient-transaction consultation/activation only, to avoid misleading future maintainers.

        /// Indicates whether connections may be vended from (and parked in) the
        /// <see cref="TransactedConnectionPool"/>. This mirrors automatic transaction enlistment:
        /// when enlistment is disabled a connection is never bound to an ambient transaction, so
        /// the transacted store must not be consulted.
        /// </summary>

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:258

• The HasTransactionAffinity summary is misleading: the pool can still park explicitly-enlisted connections in the TransactedConnectionPool even when transaction affinity (ambient auto-enlist) is disabled. The property is only used to decide whether to consult the transacted store for the ambient transaction and pass it to activation.

        /// <summary>
        /// Indicates whether connections may be vended from (and parked in) the
        /// <see cref="TransactedConnectionPool"/>. This mirrors automatic transaction enlistment:
        /// when enlistment is disabled a connection is never bound to an ambient transaction, so
        /// the transacted store must not be consulted.

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:1366

• SqlClientDiagnostics.Metrics free-connection accounting looks inconsistent in ChannelDbConnectionPool: this path decrements free connections when vending from the transacted pool, and TransactedConnectionPool.TransactionEnded also decrements before calling PutObjectFromTransactedPool, but the channel pool never increments/decrements free-connection metrics when writing to/reading from the idle channel (unlike WaitHandleDbConnectionPool.PutNewObject/GetFromGeneralPool). This can leave free-connection telemetry incorrect after transaction completion (and generally makes metrics hard to interpret for the V2 pool).

                connection.ObjectID);

            SqlClientDiagnostics.Metrics.ExitFreeConnection();

            // Transacting connections are exempt from idle-timeout and clear-generation eviction

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[codecov]

Codecov Report

❌ Patch coverage is 88.23529% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.74%. Comparing base (deabcc2) to head (540c51b).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...qlClient/ConnectionPool/ChannelDbConnectionPool.cs	88.23%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4487      +/-   ##
==========================================
- Coverage   64.71%   62.74%   -1.98%     
==========================================
  Files         288      283       -5     
  Lines       44088    67136   +23048     
==========================================
+ Hits        28532    42122   +13590     
- Misses      15556    25014    +9458     

Flag	Coverage Δ
CI-SqlClient	?
PR-SqlClient-Project	62.74% <88.23%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTransactionTest.cs:897

• EnlistTransaction(null) is commonly used to unenlist/detach. The mock currently ignores null, which can leave EnlistedTransaction set and make the tests diverge from real provider behavior (and potentially mask bugs around detachment). Consider setting EnlistedTransaction = transaction; unconditionally so null clears enlistment.

        public override void EnlistTransaction(Transaction? transaction)
        {
            if (transaction != null)
            {
                EnlistedTransaction = transaction;
            }
        }

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTransactionTest.cs:902

• The new transaction-affinity tests don’t exercise the transaction-root pathways added/relied on by the implementation (stasis via IsTransactionRoot / IsTxRootWaitingForTxEnd, and the dead-root rethrow behavior in GetFromTransactedPool). In this file, MockDbConnectionInternal never overrides IsTransactionRoot (so it stays false) and never simulates liveness failures, so those branches in ChannelDbConnectionPool aren’t covered by unit tests.

    internal class MockDbConnectionInternal : DbConnectionInternal
    {
        private static int s_nextId = 1;
        public int MockId { get; } = Interlocked.Increment(ref s_nextId);

        public override string ServerVersion => "Mock";

        public override ConnectionCapabilities Capabilities => new();

        public override DbTransaction BeginTransaction(System.Data.IsolationLevel il)
        {
            throw new NotImplementedException();
        }

        public override void EnlistTransaction(Transaction? transaction)
        {
            if (transaction != null)
            {
                EnlistedTransaction = transaction;
            }
        }

        protected override void Activate(Transaction? transaction)
        {
            EnlistedTransaction = transaction;
        }

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[cheenamalhotra]

LGTM, I'd like to see Metrics coming live for the v2 connection pool, abd that's the next big thing I'm looking forward to testing!

[apoorvdeshmukh]

Hey @mdaigle , can you take a look at this failure?
It seems related to the changes.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.