perf(desktop): coalesce streamed text chunks before transport emit - #2164
perf(desktop): coalesce streamed text chunks before transport emit#2164YodonTan wants to merge 2 commits into
Conversation
wsp1911
left a comment
There was a problem hiding this comment.
[P1] 避免队列排空循环饿死 50ms flush timer
src/apps/desktop/src/lib.rs:2244 的 tokio::select! 只会在进入队列分支前 poll 一次 flush timer。一旦选中队列分支,2246-2265 的内层循环就会持续调用 dequeue_configured_batch(),直到取到空 batch,期间不会回到外层 select!。
这会产生两个相关问题:
- 如果生产者持续让队列保持非空,timer 就不会被再次 poll,已经到期的 deadline 也无法触发 flush。
- 更关键的是,首次 deadline 直到排空循环结束后,才在
2272-2277设置。因此首个 chunk 进入 coalescer 时,实际上并没有立即开始它的 50ms 窗口。
在持续流式输出下,如果入队速度不低于 event_router.route() 加 deliver_event_to_webview() 的处理速度,队列可能长期保持非空。此时已缓冲的文本可能等待远超 50ms,直到队列出现空隙,或 non-chunk 事件通过 coalescer.push(non_chunk) 强制 flush。这违背了 PR 所描述的 throttle 语义,也无法满足“文本交付最多增加约 50ms 延迟”的保证。
建议限制每次队列分支的处理量,例如处理一个 batch 或有限数量事件后就回到 select!,并在首个 chunk 被缓冲时立即设置 deadline。另一种方式是在 drain 循环中检查 flush deadline,到期后先执行 flush,再继续处理队列。最好再补充一个“队列持续非空”的回归测试,覆盖真实 event loop 的调度行为。
…ine under sustained load
|
Thanks for the review. The P1 issue is fixed in
The same commit also adds the rate-adaptive window (30-100ms, calibrated at the measured median stream rate, 50ms at average speed) so fast streams merge more chunks per message while slow streams keep a small window, plus a fix to count buffered content in Unicode characters instead of UTF-8 bytes (Chinese text would otherwise overestimate the stream rate 3x). |
Why
During streaming AI responses, the agent runtime emits one
TextChunk/ThinkingChunkevent per provider chunk. The desktop event loop currently forwards every chunk to the WebView as a separate Tauri IPC message (JSON serialization, WebView2 boundary crossing, JS parse + dispatch) and, when peer devices are attached, as a separate end-to-end encrypted relay message.This change coalesces streamed text/thinking chunks in the desktop event loop before the transport emit:
EventBatcherbecomes arrival-driven for fast streams (~20Hz instead of ~31Hz store/React updates), reducing re-renders and layout churn while streaming.How
(session, turn, round, attempt, contentType)within a 50ms time window; text is concatenated and the thinkingisEndflag is OR-ed.event_router.route) still receive the raw per-chunk events; only WebView / peer delivery is coalesced.EventBatchermerges by the same key), ordering is preserved, and ACP/TUI/CLI consumers plus event contracts are untouched. Text delivery adds at most ~50ms, absorbed by the frontend typewriter.Fixes #2161