Under heavy load, a reply typed right after a prompt-less read starts can be queued instead of answering it

Observed

In the v0.21.0 release audit, the local canonical suite on dev 433af8fd (macOS arm64, Node 26.10, 8 GB, load average 10–16 during the run) failed one test:

tests/inputOwnershipLive.test.ts → live zsh: a waiting read gets the answer typed at once or after a pause; nothing is queued, run or drawn — timed out waiting for ANSWER=hello; the status row showed Probably waiting for input · zsh · … · ≡ 1 queued: the answer had been queued instead of reaching read -r QA_REPLY.

Reproduction (load-dependent, pre-existing)

Four copies of that test file at once (each through the repository harness with NMSH_TEST_TMPDIR=/Volumes/T7Tmp/ai-dev), load average rising from 8 to about 27:

Code	Copies failing	Failures (all `… nothing was queued`)
`dev` `433af8fd` (v0.21 candidate)	2 of 4	zsh hidden line after 300 ms; bash single key after 0 ms
`master` `6af5cbab` (v0.20.0 line)	3 of 4	bash single key after 0 ms; bash line after 300 ms; bash hidden line after 300 ms

The same file alone passed 3 of 3 runs (6/6 each) at load 5–8, and CI is green. v0.21 does not change input ownership, the session layer or InputWatch; this is not a v0.21 regression.

Cause (from the code)

A key typed while a command runs, before NMSh has evidence that the command reads input, is held for at most OWNERSHIP_LOOK_MS (800 ms, src/app/TerminalApp.ts) while the session looks (InputWatch.lookNow). On macOS a prompt-less read/read -n 1 is recognized only from the shell holding its own terminal and being idle across two process-table samples (src/session/InputWatch.ts, evidence 4). When those samples and the session round trip take longer than 800 ms, the held keys are routed without evidence, so Enter queues the line.

Impact

Only on a heavily loaded machine, and only for a reply typed within about a second of starting a command that reads without a prompt. The read keeps waiting, the status row says it probably is, and the queue shows the typed reply as an entry. Commands that print a prompt, and replies typed later, are detected from other evidence.

Acceptance

Decide the intended behavior when the look misses its deadline (for example, keep holding an Enter while a look is still in flight, bounded), without delaying type-ahead for an ordinary next command.
A deterministic regression for a look slower than the deadline (fake probe), not a timing-dependent one.
The live test passes 4× in parallel on the same 8 GB Mac.

Correction after a closer look (diagnostic run with the queue entries recorded):

The failures seen in CI (PR #425, Ubuntu) and in most of the parallel local runs were a test race, not this product behavior. The test typed its follow-up echo BACK-… as soon as the answer's output appeared and checked the queue at once. On a busy host the shell's prompt for the answered command arrives later, so that Enter queued the follow-up, as designed. The recorded queued entry was {"c":"echo BACK-0-21","q":true}, and the answer had reached the program. Fixed in #427 (test only).

This issue is the other failure: the answer itself queued, so the read never got it. The run times out waiting for the program's output while the status row shows Probably waiting for input · … · ≡ 1 queued. Seen on dev 433af8fd:

once in the full local suite (zsh, line without a prompt, load 10–16);
after the Input ownership test: type the follow-up after the answered command completes #427 fix, in 2 of 4 parallel copies at load 20–33 (zsh, ANSWER=hello never arrived).

I have not reproduced it at ordinary load or in CI. The cause and acceptance criteria above stand. Pre-existing on master (v0.20 line); v0.21 does not change this code.

v0.22: fixed in draft #433 (held until #430 merges; part of #431). Physical QA still required, so this stays open.

Decided behavior when the look is slow: keys typed before NMSh knows who reads them now wait for the look (bounded at 5 s) instead of an 800 ms deadline. Nothing typed is drawn, sent or queued until then; after 800 ms the composer says Checking whether <program> is waiting for input · your typing is kept. A look that answers quickly adds no delay, and Ctrl+C still releases at once.

Why not "route text at 800 ms, hold only the Enter": it fixes the queueing but still draws a hidden reply whenever the look is slow, for a prompt-less read -s and for a prompted sudo/ssh password alike. The unfixed baseline failed the live test's the hidden answer is never drawn assertion exactly that way under load.

Second cause, also fixed: one process-table sample that caught the shell still runnable (normal on a loaded CPU right after its exec marker) was read as busy, so no follow-up sample was taken. A runnable first sample is now inconclusive when the shell holds its terminal; the follow-up decides.

Evidence

Deterministic regressions (fake look on mocked timers): 7 fail on a8b383c2, pass on the fix. On the old code a password typed at once for a prompt-less read -s is queued as a shell command.
inputOwnershipLive 4× in parallel on this Mac: the unfixed baseline reproduced this issue (timed out waiting for ANSWER=hello, Probably waiting for input · zsh · ≡ 1 queued) and, in another round, a drawn hidden reply; the fix was clean in 15 of 16 copies. The one failure was the Ctrl+C typing test, which then passed 18/18 targeted runs on both builds (details in Input ownership: keys wait for the look, however long a loaded machine makes it (#426) #433).
CPU load alone does not reproduce it (a probe stays ~10 ms at load 64); the slow looks come with memory pressure.
Full npm run verify on the branch: 3,152 tests, 3,150 pass, 2 skipped, 0 failed.
Checked and ruled out: macOS ps -o wchan prints - for every process, so macOS cannot get a one-sample "reading the terminal" signal like Linux's.

Input ownership: never route typing on a guess; echo-off input is nev…

Input ownership: a possible password never runs as a command by itself

typingAfterRead: accept a command NMSh holds after a hidden answer, r…

Release security review for v0.22.0 (#444). On the green candidate, two paths still let a hidden reply run as a shell command. Both are now fixed in #437 (6486d9c5); details are there.

A second question with no prompt that gives up before NMSh sees it. The reply typed right after the first answer could run as the next command. It now waits at the prompt, hidden, for Tab or Esc.
A reply entered before a question with no prompt existed. It was queued, and ran after a successful exit. Once the command asks for hidden input, such entries now wait for approval.

The second path was reproduced live on 567cf533 with real zsh and Bash; the fixed build passes the same test.

Limits that remain, documented in the changelog:

A reply typed before such a question exists is still drawn and queued, since nothing has asked for it yet.
If that question gives up before NMSh sees it and the command succeeds, the queued entry runs.
An older session service routes typing as before.

This issue stays open until physical QA (pass 9 of docs/testing/v022-physical-qa.md).

A residual of this issue, seen in release CI (run 39224469156, first attempt, macOS shard 1, inputOwnershipLive).

What happened: zsh: read -r QA_REPLY with no prompt, and hello typed at once. The answer was queued (≡ 1 queued) while the status row said Probably waiting for input.

Cause. On macOS, a read with no prompt is recognised only when the shell is seen holding its terminal and idle in two process-table samples (InputWatch.decide).

If the second sample still finds the shell running (CPU contention on the runner), the look can only say that the terminal echoes.
The held keys are then routed as type-ahead, and their Enter queues them.
The watch recognises the read moments later.

Scope

This is Input ownership: keys wait for the look, however long a loaded machine makes it (#426) #433's code, unchanged since the green candidate 567cf533. The same code passed this test on 18144fb9 (run 39223339352). None of the release review's changes run in this case: there is no hidden answer and no hidden question.
Hidden input is not affected. With echo off, a look reports hidden input, and the typing is held or sent to the program, never drawn or queued.
The queued reply is visible in the queue and can be removed. If left, it runs after the command succeeds.

Not changed for v0.22.0. Holding typing whenever the shell itself holds the terminal would also delay ordinary type-ahead during shell builtins (loops, source), and sometimes discard it when the command ends. That is too risky for the release freeze. It will be listed as a known limitation in the final release commit. The failed job was re-run to measure the failure rate, not to replace this result.

Release v0.22.0 candidate: security gate review, limitation wording, …

Input ownership: a function definition or plain shell syntax typed ah…

Release v0.22.0 candidate: post-QA stabilization (F01–F04, #426, #443)…

v0.22 post-QA security review (candidate fa7ec665)

Physical QA A9 (Ghostty + zsh) passed on 68c1cbb7; its CPU-load step was not run. That review re-examined the documented residual cases with synthetic secrets and bounded tests.

Residual reproduced and fixed (#448). A reply typed and entered before a question with no prompt exists is queued, since nothing has asked yet. If that read gives up before any look sees it (a timed read between looks seconds apart, or any program's read on macOS, which cannot be reported) and the command succeeds, the entry ran as a shell command. Reproduced live in zsh, Bash and Fish (sleep 2; perl -e 'alarm 1; <STDIN>'; true): the "reply" entered NMSh's journal, and so the shell's history and its command not found output. Now a line entered with a plain Enter while a command runs, whose first command word isn't one the shell knows, waits for approval; the note never repeats its text. Real commands, function definitions and assignments typed ahead still run. Live test: fails first on all three shells, 7/7 after.

Also: a window attached to an older session service (v0.20.0's: same protocol, so new windows attach) now warns that typing protections aren't active there, and when the service restarts (#456). In Always mode the intent model can no longer re-read a refused Ask request (#447).

Still open:

A reply that is itself a command name (yes) typed before an unseen timed read runs after the command succeeds.
The underlying race (a reply to a read NMSh hasn't seen yet can be queued under heavy load) is unchanged: the entry is shown, and when the question is seen it waits for approval.

Leaving this open for those; the limitation wording in the changelog states each case.

Unchanged and re-verified on the integrated candidate: Tab/Esc/Ctrl+C/Ctrl+Q on held typing, hidden input never drawn or logged (NMSH_DEBUG_KEYS skips it), consecutive prompt-less hidden reads, delayed sudo-style prompts, entries queued while a question is seen. Deterministic input/queue suites and the live secrets, ownership, queue and typing-after-read suites pass (45/45 live).

Input ownership: a line begun after the shell said done is type-ahead…

Input ownership: a program that survives Ctrl+C gets the line typed f…

Tests: live commands typed for the prompt wait for it; Dock Top and p…

Input ownership: a late report of the question just answered never ta…

Docs: v0.22 hardening ledger, exit-time typing limitation (#426, #443)

Tests: decide from the journal and queue, never from redrawn screen t…

Merge #457: v0.22 hardening: input ownership by lifecycle (#426), sta…

Tests: GNU screen's resize probe waits for each probe's completion be…

Final hardening (#457, merged into release/v0.22.0 = e8dd44e4): ownership, not vocabulary. #448's rule (hold a line only if its first word isn't a command) is replaced. Execution now depends on the input's lifecycle: a line runs as a shell command only when it is submitted at a verified prompt, queued with Ctrl+Q, or approved.

Both remaining routes are closed:

Withheld lines. A plain Enter while a command runs, with no question NMSh saw, holds the line for approval.
- The session's owner enforces this, not only the window; the line is shown masked, and Up never recalls it.
- Reproduced live on fa7ec665: the reply ls ran in zsh, Bash and Fish. Now it is held.
Delivered bytes. Input sent to a program that the program never read used to reach the shell.
- On fa7ec665, in all three shells: echo QZ-RAN ran, then QZSECRETecho NEXT ran (a partial secret glued onto the next command).
- Now the shell's first prompt hook takes that input back, and the owner holds each line.

Also:

Ctrl+C type-ahead waits for the verified prompt. A line begun after the shell's done marker runs at the prompt.
A stale report of an answered question is ignored.
An older service fails closed.

Remaining limits (changelog Known limitations):

A sub-millisecond window at command end.
An unseen question asked after Ctrl+C that then gives up.
v0.20 services, which have no shell-side drain.

Stays open for physical QA (R4, R5).

Navigation Menu

Repository navigation

Under heavy load, a reply typed right after a prompt-less read starts can be queued instead of answering it #426

Observed

Reproduction (load-dependent, pre-existing)

Cause (from the code)

Impact

Acceptance

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Under heavy load, a reply typed right after a prompt-less read starts can be queued instead of answering it #426

Description

Observed

Reproduction (load-dependent, pre-existing)

Cause (from the code)

Impact

Acceptance

Activity

raiseCatError commented on Oct 18, 2026

raiseCatError commented on Oct 18, 2026

raiseCatError commented on Oct 19, 2026

raiseCatError commented on Oct 19, 2026

raiseCatError commented on Oct 19, 2026

raiseCatError commented on Oct 19, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions