Repository navigation
fix(csv-parse): refresh trim state after BOM detection - #537
nk-batman21 wants to merge 1 commit into
Conversation
|
Independent validation on Node.js 24.22.1, comparing base
import { parse } from './packages/csv-parse/lib/sync.js';
parse(Buffer.from('\ufeff"a"\u2003,"b"', 'utf16le'), {
bom: true, trim: true,
});
// CSV_NON_TRIMABLE_CHAR_AFTER_CLOSING_QUOTE
// Expected: [['a', 'b']]
parse(Buffer.from('"a"\u2003,"b"', 'utf8'), { trim: true });
// Same error, without BOM detection.This quoted-field behavior is not introduced by this PR: both the base and head also fail when parsing the same BOM-free fixture with explicit The remaining boundary appears to be in the field-append branch: after a closing quote, The independent matrix supports the encoding-state refresh for unquoted fields and identifies this separate after-quote boundary for follow-up; it should not hold up the existing BOM fix. |
Fixes #536.
When a UTF-16LE BOM changes the encoding, trimming still uses buffers prepared for the initial encoding. For example, parsing
Buffer.from("\ufeff a , b ", "utf16le")with{ bom: true, trim: true }returns[["愀", "戀"]]instead of[["a", "b"]].Refresh the encoding-dependent trim buffers and lookahead sizes after BOM detection, preserving the existing parser state and byte offsets. Add sync and byte-sized streaming regressions for spaces and nonbreaking spaces with
trimandltrim.Validation on Node.js 24.18.0:
group_columns_by_namevalidation assertion.git diff --checkpass.The full test command still fails that existing assertion: it expects
"an boolean"while the source now returns"a boolean"after #525. The same failure was verified with pristine production code.