Fix the intermittent Interop unit test failures

[damyanpetev]

Closes #329

Four intermittent failures in IgniteUI.Blazor.Tests, four different causes: three about interop messages crossing threads they should not, one about the flush never getting a thread at all. Each cause is fixed in its own commit, plus a fifth the review turned up. One of them is a genuine product bug that reproduces from ordinary application code, with no test involvement at all.

The failures

Failure	Symptom
SnackbarTests.Methods_FollowContract (run)	InvalidOperationException: The LinkedList is empty out of BaseRendererControl.Update()
ScriptPropTests.ScriptProps_TransmitScriptRefs(IgbTabs) (run, run)	Expected: "handleChangeScript" / Actual: "Change"
CheckboxTests.Binds_FollowContract (#329)	binding transmitted no "Change" event registration
InputTests and RadioTests Events_FollowContract (run)	no event-handler registration transmission was observed

All intermittent, all green on re-run.

How component messaging works

Every BaseRendererControl owns a LinkedList<RendererMessage> _messageQueue and reaches it through two different paths.

The deferred path. A property setter marks state dirty, which appends a message and asks for a flush:

setter → MarkPropDirty → MarkContentDirty → SendDescriptionMessage → _messageQueue.AddLast + QueueUpdate
OnRefChanged                              → SendMessage            → _messageQueue.AddLast + QueueUpdate

QueueUpdate deliberately does not flush inline:

if (!_updateQueued && _ready)
{
    _updateQueued = true;
    Task.Delay(0).ContinueWith((t) => InvokeAsync(Update));
}

The deferral is load-bearing: Update serialises the component, and doing that inline from a setter would serialise a half-applied component and emit one description per dirty property instead of one per batch. So the flush hops onto the thread pool, then posts back to the renderer's dispatcher, where it drains FIFO.

The immediate path. An API call cannot be deferred — the caller is awaiting a return value — so it flushes synchronously, on the calling thread, to stay ordered behind the queue:

private async Task<object> SendMessageImmediate(RendererMessage m)
{
    Update();                       // ← drains the queue on the caller's thread
    return await SendJsonImmediate(m);
}

Both paths are correct alone. Together, the queue is drained from two places and Update is reachable from any thread a component is driven from. On the dispatcher that is serialised; off it two flushes run at once on an unsynchronised LinkedList:

• Count > 0 then RemoveFirst() on an emptied list → InvalidOperationException: The LinkedList is empty.
• AddLast interleaved with RemoveFirst → torn nodes: NullReferenceException, or a message silently dropped.
• Both drains dequeue, then race to igSendMessage → the client receives two messages in the wrong order. Ref updates apply last-write-wins, so an inverted pair silently keeps the older value.

The four causes

1. The queue is not thread-safe (product). Reachable from application code — see below. This killed SnackbarTests: the contract test called ShowAsync() from the xUnit thread, so the inline drain ran concurrently with the renderer's.

2. The harness read a partially flushed queue (test). IgbTabs.ChangeScript and the Change event registration ride the same ref name (<containerId>/Change), because IgbTabs' constructor installs its own handler via EnsureChangeHandled() before the parameter is applied, so FindPropertyUpdate relies on "newest wins". But a flush hands its messages to JS one at a time, and the recorded traffic showed the wire order was always correct — the scan landed between the two sends, saw only the event registration, and returned it without retrying, because it had found something.

3. bUnit's invocation recorder is not thread-safe (test). This explains CheckboxTests, where the message was never observed even after ~2s of retrying. Two threads calling InvokeAsync on bUnit's JSInterop — the number a single off-dispatcher API call creates — fail 4 runs out of 4 with An item with the same key has already been added and Operations that change non-concurrent collections must have exclusive access, an unsynchronised List<T>.Add. So a raced send is not merely late; it can be lost permanently, which no retry budget recovers.

4. The flush needs a thread-pool thread, and the tests waiting for it hold those threads (test). This explains the Events_FollowContract pair — two failures in one run, in two different per-framework processes, both after exactly the two seconds a read spends before concluding absence. QueueUpdate hands the flush to the pool, and a test waiting for that traffic blocks a pool thread while it waits, because xUnit runs test cases on pool threads. The pool's floor is one thread per core and xUnit's parallelism is also one collection per core, so on a four-core runner four waiting reads can hold every thread the pool gives out without throttling, leaving the flush on its injection of new ones — a thread or two per second. The floor is per process, so the three per-framework processes widen the window by competing for cores rather than for each other's threads. Nothing races here: the messages are queued and never sent inside the budget.

The product bug reproduces without tests

The library has exactly one off-dispatcher hop, and it immediately re-enters the dispatcher. But a component reaches its queue from wherever it is driven from, and JsonDataSource.OnCollectionChanged runs synchronously on whichever thread mutated the bound collection, straight into SendMessage → AddLast, with no dispatcher hop.

So this is enough — bind a collection, fill it from a background load:

var data = new ObservableCollection<Row>();
Render<IgbCombo<Row>>(ps => ps.Add(c => c.Data, data));
await Task.Run(() => { for (var i = 0; i < 20000; i++) data.Add(new Row($"row-{i}")); });

A single writer, so the ObservableCollection is never used concurrently; no component API called off the dispatcher; no test-only threading. The only thing seeing two threads is the component's own queue: this task's change notifications, and the renderer's flush.

Without the fix this crashes 6 times out of 6 (NullReferenceException inside LinkedList.AddLast); with it, clean. Kept as the regression test, asserting the transmitted insertions rather than the collection it just filled — tearing the queue drops notifications as readily as it throws, and the producer's own count shows neither.

The fixes

1. fix: guard the renderer message queue against concurrent access

One reentrant lock over _messageQueue and the _updateQueued flag that decides when it is flushed, covering the whole drain rather than just the dequeue, and the start of an immediate send — so nothing the component sends can interleave with a flush. _invokeId also goes Interlocked: a plain ++ can hand two calls the same id, which collides in _methodTasks.

2. test: drive components through the renderer's dispatcher

Adds InteropHarness.OnDispatcher, makes the dispatcher required, and routes everything that makes a component transmit through it: contract invocations, because application code calls component APIs from event handlers, and the harness's JS-to-.NET entries (RaiseEvent, MakeReady, CompleteDeferred), because that is where Blazor delivers the real ones. The generic overload hands back a still-running task instead of awaiting on the dispatcher, which would hold it until the task completes and deadlock a deferred return needing it to get there.

3. test: read interop traffic only once it has settled

FindPropertyUpdate no longer reads a queue that is still flushing. Nothing observable says a flush has finished: a gap in the traffic looks the same whether the renderer was descheduled mid-flush or has nothing left to send, and the thread-pool hop that posts a flush cannot be seen from outside. So rather than trying to detect completion, the harness waits for what it expects to arrive — it drains the dispatcher before reading, and DataItemInsertions(containerId, expected) waits for the count the caller asked for, then until that count stops moving, so both a shortfall and an overshoot come back to be asserted on. Bounded throughout, so a component that never stops transmitting cannot hang a test.

4. test: keep the thread pool from starving interop flushes

Raises the pool's minimum thread count for the test assembly, so a blocking wait cannot hold the thread that the flush it waits for needs. At identical pressure — 128 pool threads blocked — a registration the default floor never delivers within two seconds arrives in 47ms. Absence is also ambiguous on its own, a message never queued reading exactly like one queued and never flushed, so the failures reporting one now say what the instance did transmit; "the instance sent nothing at all" is the giveaway for this cause.

5. fix: guard the invocation bookkeeping against concurrent access

Raised in review rather than by a failure, but the same exposure: _methodTasks and _methodReturns pair an invocation with its return, and off the dispatcher concurrent calls raced both. Atomic collections would not have been enough: a return landing between the caller registering its task and checking for an already-stored result leaves the call awaiting forever. Both now go through the _semLock that was already declared for them.

6. fix: close the message queue at teardown

Also from review: the disposed check sat outside the lock, so a producer or an API call could pass it and still send after teardown. It now sits inside, at the enqueue and both immediate sends. Its test is skipped — it orders traffic against the cleanup message, which disposal never transmits today.

Verification

• Repeated clean full-suite runs, plus runs under DOTNET_PROCESSOR_COUNT=1 with 24 CPU hogs to approximate CI's three concurrent per-TFM processes. Only net10.0 ran locally, so that concurrency is approximated rather than reproduced.
• Each commit builds and passes on its own, so the history bisects. Suite runtime better than halved.

Notes for review

• The lock spans ProcessMessage and the start of an immediate send, so Serialize() and a JS interop call. Should be safe — per-component, reentrant, and nothing inside waits on another thread (SendJson fires InvokeAsync without awaiting; the sync path blocks on its own thread) — but it is the most reviewable claim here. The narrower alternative locks only the enqueue/dequeue, which stops the crash and leaves send ordering unguarded.
• IgbTabs.ChangeScript sharing a ref name with the internal Change registration looks like a real product wrinkle, not a test artifact: whether a user's ChangeScript wins depends on ordering against the handler the constructor installs. Out of scope, but worth its own look.
• One residual in FindPropertyUpdate: if a flush has not been posted yet the barrier finds nothing to drain, quiet reads as settled, and an update already in the window is returned. Not reproducible here and much narrowed by the pool floor; closing it means waiting on an expected value rather than on quiet.

[Copilot]

Pull request overview

Improves interop reliability by synchronizing renderer messages and aligning tests with Blazor dispatcher behavior.

Changes:

• Protects message queues and invocation IDs from concurrency.
• Routes test interop through the renderer dispatcher.
• Adds settled-traffic handling and a threading regression test.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/componentsBase/BaseRendererControl.cs	Synchronizes queue access and invocation IDs.
tests/IgniteUI.Blazor.Tests/BlazorComponentTestBase.cs	Configures harness dispatchers.
tests/IgniteUI.Blazor.Tests/ComponentWithContractTestBase.cs	Dispatches contract invocations.
tests/IgniteUI.Blazor.Tests/Interop/InteropHarness.cs	Adds dispatcher helpers.
tests/IgniteUI.Blazor.Tests/Interop/RendererMessageInteropHarness.cs	Dispatches callbacks and settles traffic reads.
tests/IgniteUI.Blazor.Tests/InteropReadinessTests.cs	Dispatches readiness API calls.
tests/IgniteUI.Blazor.Tests/InteropThreadingTests.cs	Adds background collection regression coverage.
tests/IgniteUI.Blazor.Tests/MethodInteropTests.cs	Dispatches deferred method invocation.
tests/IgniteUI.Blazor.Tests/ScriptPropTests.cs	Dispatches script-property clearing.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

[Copilot]

🟡 Changes recommended

Immediate sends can still reorder, and invocation result dictionaries remain unsafe under concurrent calls.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The early deferred-return race lacks deterministic regression coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Disposal can still race with producers and transmit messages after cleanup, while the central two-drain behavior lacks deterministic coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

tests/IgniteUI.Blazor.Tests/InteropThreadingTests.cs:41

• This regression explicitly does not exercise the two-drain race, even though preventing two flushes from interleaving is the reason Update now holds the lock across ProcessMessage. The contract calls were also moved onto the dispatcher, removing the previous incidental reproduction. Add a deterministic test that overlaps an off-dispatcher immediate API call with a queued dispatcher flush and asserts wire ordering/no loss; otherwise narrowing the lock to dequeue-only would still pass the new tests.

        // count shows neither. Only the renderer drains here, so this says nothing about two
        // drains interleaving - that is what holding the lock across the whole drain is for.
        Assert.Equal(Enumerable.Range(0, Rows), Interop.DataItemInsertions(Interop.ContainerIdOf(cut), Rows));

• Files reviewed: 12/12 changed files
• Comments generated: 1
• Review effort level: Balanced

[damyanpetev]

This is almost going a bit out of scope, but will address.
This actually brought up an issue the agent found locally - DisposeAsync doesn't actually send, currently doing this:

disposedValue = true;
_shouldReevaluateRuntime = true;
await TrySendCleanupAsync();     // → SendMessageImmediate → if (disposedValue) return null;

That didn't show up on the diff for #335 and I completely missed it too, but it's quite correct. Doesn't help that all the tests are also of the "doesn't throw" variety, which of course it doesn't do when not sending anything as well :D
@MayaKirova We might need to address this in a separate fix before releasing it, cuz I think we kinda killed the cleanup.. so following guidance, but just a bit too soon😆
I'll see if I can leave a test in for this comment for after the fix, even if it can't run atm due to that.

[Copilot]

🔵 Needs a closer look

Immediate sends retain disposal races, and property observation can still return stale traffic.

Review details

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

tests/IgniteUI.Blazor.Tests/Interop/RendererMessageInteropHarness.cs:315

• These two comment lines are duplicated verbatim, which obscures the actual settling logic. Keep a single copy.

src/componentsBase/BaseRendererControl.cs:1616

• The disposed check is still outside the lock. An API call can observe false, lose the lock to DisposeAsync (which publishes disposal and tears down the object reference), then acquire it and send after teardown. Move the check into this critical section so disposal atomically closes the immediate-send path.

            lock (_messageQueueLock)
            {
                Update();
                sent = SendJsonImmediate(m);

src/componentsBase/BaseRendererControl.cs:1630

• The synchronous path has the same check-then-lock race: disposal can begin after the outer check but before this lock is acquired, allowing a synchronous invocation to send after teardown. Check disposedValue while holding _messageQueueLock.

            lock (_messageQueueLock)
            {
                UpdateSync();
                return SendJsonImmediateSync(m);

tests/IgniteUI.Blazor.Tests/Interop/RendererMessageInteropHarness.cs:265

• This can still return the stale same-name event registration. If QueueUpdate's thread-pool continuation has not posted yet, the dispatcher barrier overtakes it; one quiet millisecond then looks settled, and FindPropertyUpdate immediately returns the existing registration without using its retry budget. Wait for the expected value/predicate (or isolate observations before the mutation) rather than treating any prior matching update as complete.

            // One snapshot+parse per attempt, newest-first, taken once the instance stops
            // transmitting: mid-flush the newest update recorded is not yet the newest one sent.
            var messages = SettledMessagesFor(containerId);

• Files reviewed: 13/13 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The settled-message logic retains a stale-read race, and key concurrency guarantees lack regression coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

tests/IgniteUI.Blazor.Tests/Interop/RendererMessageInteropHarness.cs:326

• This still has the original stale-read race: the dispatcher barrier only waits for flush work that has already been posted. If QueueUpdate's thread-pool continuation has not posted yet, both counts are stable, this method returns the existing same-name event registration, and the flush carrying the script ref can start immediately afterward; FindPropertyUpdate then returns the stale value without retrying. Raising the pool floor narrows but does not establish ordering. Make the read wait for an expected value/predicate (or expose an explicit flush-completion signal) rather than treating a quiet snapshot as final.

            OnDispatcher(() => { });
            var sends = SendCountFor(containerId);
            Thread.Sleep(1);
            if (SendCountFor(containerId) == sends)

• Files reviewed: 13/13 changed files
• Comments generated: 2
• Review effort level: Balanced