Upstream names in the idle-gateway warning are sanitized and capped like every other status-file label (#680) - #681
Upstream names in the idle-gateway warning are sanitized and capped like every other status-file label (#680)#681philcunliffe wants to merge 3 commits into
Conversation
…rust (#680) `gateway_idle_no_upstreams` lifts the dropped upstreams' names out of `status.json` and joins them into the warning it prints, filtering only for "non-empty string". `recentEntrypointsFromSources` reads the same file, in the same module, on the way to the same terminal, and runs every label through `sanitizeLabel` and a count cap first, because `status.json` is a file and core cannot assume the daemon that wrote it was this version, this build, or well behaved (LLP 0164). Give the upstream names the same treatment: each one cleaned through `sanitizeLabel`, the printed list capped at 8, and the names held back counted in the line (`, +N more`) so a sampled list never reads as a complete one. The count keeps coming off the raw list, before either filter, since it is what an older status file's missing `upstreams_configured` falls back to and it is the whole signal that separates a dropped upstream from a legitimately upstream-less gateway: cleaning bounds what is printed, never what was configured. Co-Authored-By: Claude <noreply@anthropic.com>
…g, and the count guard only half pinned the count Two findings from reviewing this PR's own claim. 1. `listen_fallback_from` is the same defect, twenty lines up. `gatewayIdleWithConfiguredUpstreams` was not the only reader in this module joining an unsanitized `status.json` string into a diagnostic: `gatewaySourceDetails` filters `details.listen_fallback_from` for "non-empty string" and nothing else, and `gateway_port_fallback` interpolates it into both its message and its repair line. It has the same provenance as an upstream `name` (config-authored, arriving through a file this build did not necessarily write) and the same destination (a terminal), so an ESC erase-line sequence, a newline, or 5000 characters reach the operator's screen from it. The PR's own stated principle - two paths reading one file should not disagree about whether it is trusted - reaches three paths, so the third is cleaned through `sanitizeLabel` too, and a value that sanitizes away entirely falls back to the generic antecedent exactly as an absent one already did. `host`, read two lines above it, is deliberately left alone: it is not display-only (it composes the endpoint `attach` writes into client settings files), so clamping it could turn a long-but-legal host into a truncated wrong one. That is a separate change with a wider blast radius, not this one. 2. The count guard pinned the cap and not the sanitizer. `an older status file counts every name it holds, capped or not` held 20 well-formed names, so it caught a `total` taken after the cap but not one taken after `sanitizeLabel`: deriving the count from printable names only left all 14 tests green. Two of the 20 names now sanitize away, so the test fails for either mutation, which is what it claims to pin. npm test: 3787 pass, 0 fail. npm run typecheck: clean. Co-Authored-By: Claude <noreply@anthropic.com>
Verdict: findings (2 actionable, both fixed on the branch), 2 left with a reasonThe change is right and the reasoning behind it holds. Verified independently
Cap semantics check out at the boundaries. The Finding 1 (medium, fixed): the same module prints a second raw status-file string
Fixed: Finding 2 (low, fixed): the count guard pinned the cap but not the sanitizer
Left, with a reason
On the
|
…g a JSON parse error that quotes the file back The sanitize-on-read this PR applies to the idle warning's upstream names, and round 1 applied to `listen_fallback_from`, is a property of *reading status.json* rather than of any one detail. Enumerating every route from that file to the terminal turns up four more that were still raw, all of them in the text renderer rather than the collector: - `daemon.state` and `daemon.mode`, taken straight off the snapshot (`collectHypAwareStatus` reads `mode` from it whenever the pid file did not already supply one) and printed as `state=` / `mode=`. - `sources[]` and `sinks[]`, which with no runtime attached come off the file verbatim and are printed as `- name (plugin) [state]` and `- instance (plugin, kind)`. Round 1 left `sources[]` on the grounds that `name` / `plugin` are identity keys and part of the `--json` contract. That reason is real, but it argues against cleaning them *in the collector*, not against cleaning them on the way into a terminal - and it does not reach `state`, `kind`, or `sinks[]` at all, which are display and nothing else. - `daemon.error`, which is the sharpest of the four: a `status.json` that is not valid JSON reaches the line as `JSON.parse`'s own message, and V8 quotes an excerpt of its input back verbatim. A file whose first bytes are an erase-line sequence and a newline puts both on the terminal. All four are cleaned at the interpolation in `renderStatusText` / `describeDaemon` rather than in the collector, so `--json` stays byte for byte what it was (a consumer escapes for itself, and `sources[].name` / `sinks[].instance` stay usable as keys), and the raw values the provenance lookups match on are untouched. `error=` gets a wider bound than a label's 120: it carries a real message naming a full path, and truncating that is the one way this cleaning could cost a reader an answer. Four tests in `test/core/status-text-status-file-labels.test.js`, each failing against the pre-fix renderer. They assert over the whole C0/DEL/C1 range rather than the one sequence each case drives, and pin the forged newline directly: one snapshot entry must render as one line. Co-Authored-By: Claude <noreply@anthropic.com>
Verdict: findings (1 actionable, fixed on the branch,
|
| # | Field | Route to terminal | Before this round | Now |
|---|---|---|---|---|
| 1 | sources[].details.recent_entrypoints[].entrypoint / .client_name |
recentEntrypointsFromSources to recent clients |
sanitized + capped 32 (pre-existing, LLP 0164) | unchanged |
| 2 | sources[].details.upstreams[] |
gateway_idle_no_upstreams message |
fixed by this PR (sanitize + cap 8 + +N more) |
unchanged |
| 3 | sources[].details.listen_fallback_from |
gateway_port_fallback message and repair line |
fixed in round 1 | unchanged |
| 4 | state |
daemon: line, state= |
RAW | printable() |
| 5 | mode |
daemon: line, mode= (when the pid file supplied none) |
RAW | printable() |
| 6 | (the file's own bytes) | daemon: line, error=, via JSON.parse's message |
RAW | printable(_, 400) |
| 7 | sources[].name / .plugin / .state |
sources: line, with no runtime attached |
RAW (round 1 left it) | printable() |
| 8 | sinks[].instance / .plugin / .kind |
sinks: line, with no runtime and no configured sinks |
RAW (round 1 never enumerated it) | printable() |
| 9 | sources[].details.host |
endpointFromListen, not a display string |
left (see below) | unchanged |
| 10 | sources[].details.port, .listening, .listen_fallback, .upstreams_configured |
numeric/boolean, type-checked at the read | n/a | n/a |
| 11 | runId |
--json run_id only, never text |
n/a | n/a |
| 12 | sinks[].lastTickAt / .lastSuccessAt |
--json only, never text |
n/a | n/a |
| 13 | pid (pid file, not status.json) |
daemon: line, pid= |
readPidFile rejects a non-number (pid.js:68) |
n/a |
Nothing else in DaemonStatus reaches renderStatusText. Rows 4-8 are the
finding.
Finding 1 (medium, fixed): four more status-file strings printed raw
src/core/commands/status.js:327 (sources), :336 (sinks), :574 / :576 /
:577 (describeDaemon), all pre-fix line numbers.
Reproduced, not reasoned about. Writing a status.json whose sources[0].name
is gw + an ESC erase-line sequence + a newline + hyp: all good, then
rendering the report through renderStatusText, puts four output lines
carrying a raw ESC-[2K and an embedded newline on stdout - the exact
forged-status-line attack this PR fixes for upstream names, from the same file,
in the same command.
Row 6 is the sharpest and was invisible to everyone so far, because it is not a
field at all. A status.json that is not valid JSON never becomes a
DaemonStatus; it surfaces as daemon.error, which is JSON.parse's message
- and V8 quotes an excerpt of its input back verbatim. With a status file
whose bytes arex+ ESC-[2K+ newline +hyp: all good:
daemon.error: "Unexpected token 'x', \"x<ESC>[2K\nhyp: all good\" is not valid JSON"
daemon: not installed, running, pid=..., mode=foreground, error=Unexpected token 'x', "x<ESC>[2K
hyp: all good" is not valid JSON
So a status file needs no valid structure whatsoever to reach the terminal. It
just has to be malformed in the right place.
On round 1's stated reason for leaving sources[]. It said name /
plugin are identity keys and part of the --json contract, so sanitizing
them is its own decision. That is a real constraint and I kept it - but it
argues against cleaning them in the collector, not against cleaning them on
the way into a terminal, and the two are separable. It also does not reach
state, kind, or sinks[] at all, which are display strings and nothing
else. So the reasoning was sound as far as it went and stopped one step short
of the fix it implied.
The fix cleans at the interpolation in renderStatusText /
describeDaemon, not in the collector:
--jsonis byte for byte what it was. A consumer escapes for itself, and
sources[].name/sinks[].instancestay usable as keys. A test asserts
the raw value still comes out ofrenderStatusJson.- The raw values
isCentralPlugin/isCentralSinkmatch on are untouched,
so provenance tags cannot break. error=gets a 400-char bound rather than a label's 120. It carries a real
message naming a full path, and truncating that is the one way this cleaning
could cost a reader an answer - the same argument round 1 used to leave
hostalone, applied where it actually bites.
Four tests in test/core/status-text-status-file-labels.test.js. All four
fail against the pre-fix renderer and pass after (verified by reverting only
src/core/commands/status.js: # pass 0 # fail 4, then restoring:
# pass 4 # fail 0). They assert over the whole C0/DEL/C1 range rather than
the one sequence each case drives, and they pin the forged newline directly -
one snapshot entry must render as one line - rather than only asserting the
absence of a byte.
Focus items checked and found sound
Round 1's generic antecedent is not misleading. listen_fallback_from
only reaches the ?? 'its default listen address' branch when it sanitizes
away entirely, i.e. the value was nothing but control/invisible characters.
Any address with one printable character survives (an address followed by an
ESC sequence renders as the address plus the sequence's printable residue), so
there is no case where a real address exists and the message claims a default.
The repair line says "free its default listen address", which is generic but
not false.
Informational, pre-existing, not fixed: the generic branch produces
"the gateway's default listen its default listen address was taken at
boot", which is garbled. The ?? is on master unchanged (status.js:655
there), so this PR did not introduce it - but it did widen when the branch
fires, and the new test at status-gateway-fallback.test.js:159 now pins the
garbled phrasing. Rewording is LLP 0114's message and belongs in its own change.
Round 1's three fallback tests do not have the blind spot F2 fixed. They
drive genuinely hostile input (an erase-line sequence plus newline, a
5000-char value, a pure zero-width run) and assert over the whole control
range, not the one sequence. They are pinning what they claim.
The cap arithmetic holds at every boundary. Probed directly rather than
read:
| raw names | printed | rendered |
|---|---|---|
| exactly 8 | 8 | no +0 more |
| exactly 9 | 8 | +1 more |
| 12, of which 5 zero-width, interleaved | 7 | +5 more |
| 3, all zero-width | 0 | 3 upstreams are configured, no parenthetical |
0 names, upstreams_configured: 2 |
0 | 2 upstreams are configured |
hidden = raw.length - names.length adds up in every case, and no path prints
a sampled list that reads as complete.
The all-sanitize-away row deserves a note, since the code comment at
status.js:731 says the withheld names "are counted rather than dropped
silently" and here the +3 more does not appear (named is gated on
names.length > 0). This is benign, not a bug: with no list printed there
is no sampled list to misread, and the three names are already counted - they
are the 3 that leads the sentence, because total comes off the raw list.
Nothing is under-reported. Left as is.
Left, with a reason
details.host(src/core/daemon/status.js:119). Round 1's reasoning is
correct and I kept it:hostfeedsendpointFromListen, and the endpoint
attach writes into client settings files, sosanitizeLabel's clamp could
turn a long-but-legal host into a truncated wrong one. It is not
display-only. Note the row-6 fix above adopts the same reasoning for
error=.- Everything reached from files other than
status.json.activePlugins
andlayered.drops(config),remoteConfigetags and rollback reasons
(config-control/),clientActionsreasons and request keys (the marker
store) all print raw too. Same class of exposure, different files, and on a
joined host some of it is remote-authored - which makes it a wider decision
than "what core reads back out ofstatus.json", the principle this PR and
LLP 0164 are about. Deliberately out of scope; worth its own issue.
On the @ref and the LLP
The new printable() cites LLP 0164#status-reads-it-from-the-status-file
[constrained-by], the same anchor (verified present as an explicit
<a id>) for the same reason as rounds 0 and 1: this applies a settled
principle to further read paths rather than revising it. No LLP edit needed,
and none made - LLP 0164 is Accepted.
Checks
npm test: 3791 pass, 0 fail, 6 skipped. npm run typecheck: clean. All 9 CI
checks green on 33c3934. No semicolons, no em dashes, no @typedef, no
inline import('...') types; the new test file uses a root-anchored @import.
Merge base
Unchanged from round 1: behind origin/master by #679 (4fb95dc), which
touches status.js only at the folderAsk read. Clean merge. The
merge-base rung's business, not a review finding.
This was round 2; the ladder caps here.
Triage: held (
|
gateway_idle_no_upstreamsreads the dropped upstreams' names out ofstatus.jsonand joins them straight into the warning it prints, filteringonly for "non-empty string" (
src/core/daemon/status.js, thenamedparenthetical).
recentEntrypointsFromSourcesreads the same file, in the samemodule, on the way to the same terminal, and passes every label through
sanitizeLabeland a count cap first - deliberately duplicating what thegateway already did on the way in, because
status.jsonis a file and corecannot assume the daemon that wrote it was this version, this build, or well
behaved (LLP 0164#status-reads-it-from-the-status-file). Two paths reading one
file should not disagree about whether it is trusted.
What this changes
printableUpstreamNamescleans each name throughsanitizeLabel(stripscontrol, line-breaking, bidi and zero-width characters; clamps to 120 chars
with a truncation marker) and caps the printed list at 8.
, +N more), so a sampledlist never reads as a complete one.
older status file's missing
upstreams_configuredfalls back to, and it isthe whole signal separating a dropped upstream from a legitimately
upstream-less (hermes-only) gateway, so cleaning bounds what is printed and
never what was configured.
Evidence
Three new tests in
test/core/status-gateway-idle.test.jsfail onmasterandpass with the fix, one per way a name can be hostile:
a hostile upstream name cannot drive the terminal from the warning- beforethe fix, a raw
ESC [ 2 Kerase-line sequence and an embedded newline reachdiag.message, which is enough to forge a plausible extra status line.an unbounded upstream name is clamped in the warning- before the fix, all5000 characters are printed.
an unbounded number of upstream names is capped, and the rest counted-before the fix, all 50 names are spelled out.
A fourth (
an older status file counts every name it holds, capped or not) is aguard, green either way: it pins that neither the cap nor the sanitizer may
revise the count.
npm test(3784 pass, 0 fail) andnpm run typecheckare clean.Deliberately not fixed here
This is based on
master, not on the #678 / #658 stack, so it can land on itsown. The other two items in #680 are not master-fixable and are left:
it concerns,
droppedUpstreamConsequence, exists only onfix/issue-676;masterhas no such branch in the message. Closing it needs the gatewaysource to distinguish a nameless drop from a deduped one - a new
upstreams_dropped_unnameddetail - which is a source-side feature, not awording fix. It belongs after A partially dropped gateway upstream is visible in status, and LLP 0193 records why the gateway idles (#653) #658 and The dropped-upstream warning hedges about a table it already has (#676) #678 merge.
registered_presetssnapshot skew). Onmasternothing insrc/reads
registered_presetsat all; the only reader(
attributeDroppedUpstreams) is The dropped-upstream warning hedges about a table it already has (#676) #678's code. There is no user-visible defecton
masterto fix, and on the branch the skew is unreachable with bundledplugins, which all register presets in
activate()before any source starts.Once the stack merges,
gatewayDroppedUpstreams(which replaces the readertouched here) should pick up
printableUpstreamNamesin place of its barestringList(details.upstreams); the sanitize-on-read then covers thedroppedUpstreamConsequencemessage too.Fixes #680