Repository navigation
fix: match Lua 5.1 number rendering and string.format %s/%c/%q semantics - #557
madhavschahar wants to merge 1 commit into
Conversation
GopherLua renders numbers with Go's default formatting and implements
string.format by delegating to fmt.Sprintf, so several results differ
from the reference Lua 5.1 interpreter:
- tostring/concat/%s rendered numbers the Go way (1000000000000000,
0.30000000000000004, +Inf, NaN) instead of the reference's
lua_number2str "%.14g" (1e+15, 0.3, inf, nan), and negative zero lost
its sign because the allocator mapped -0 onto the preloaded +0.
- math.huge was math.MaxFloat64 instead of HUGE_VAL (positive
infinity), so math.huge == 1/0 was false.
- string.format accepted any value for %s/%q, measured %s width and
precision in runes, kept NUL bytes, and %c emitted a UTF-8 rune. The
reference formats through C sprintf/strlen, so it accepts only
strings and numbers, counts bytes, stops %s at the first NUL byte
(unless the string is 100+ bytes long with no precision, in which
case it is kept whole), writes %c as a single byte where a zero byte
adds nothing, and escapes %q output as backslash+newline, \r and
\000.
Replace LNumber.String with the reference "%.14g" rendering (plus the
inf/-inf/nan/-nan spellings), keep -0 out of the non-negative preload
table, set math.huge to +Inf, and give strFormat a small scanner that
implements %s, %c and %q with Lua 5.1 semantics while still delegating
every other verb to fmt (LNumber and LString implement fmt.Formatter,
so %d, %x, %f, ... behave as before).
Assertions covering each behaviour are added to
_glua-tests/{base,math,strings}.lua, and the previously diverging
string.format assertions in _lua5.1-tests/strings.lua (lines 102-105 of
the official suite) are re-enabled and now pass.
Related: yuin#468
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kimjooyoon
left a comment
There was a problem hiding this comment.
Checked head 6d5c36cc856e0825fcfde8fd00e1332d90696846 and base 75f497656b1c6864139dd2a7d88cf96d09550814 against unmodified C Lua 5.1.5 on macOS arm64. The unpadded NUL %c, wrapped-byte %c, embedded-NUL %s and %q round-trip cases in this sample now match the reference. Two remaining format-parity gaps are detailed inline; both were already different on the base as well.
Reproduction: identical 22-case program, captured byte outputs and pinned sources. No newly introduced reference mismatch was observed in this small sample; it is not a full number-rendering or Lua-conformance review.
| item := "" | ||
| if b := byte(n); b != 0 { | ||
| item = string([]byte{b}) | ||
| } | ||
| strFormatPad(&out, item, width, leftAlign) |
There was a problem hiding this comment.
[P2] Apply %c width before truncating its NUL byte
With zero and a field width, removing the byte before padding changes the result: on this head string.format("L%3c|R", 0) returns "L |R", while Lua 5.1.5 returns "L |R". For "L%-3c|R", the reference returns "L|R", but this branch again inserts three spaces. The same applies to 256 after the byte conversion. Lua pads the one-byte character with sprintf first, then takes strlen of that formatted item. Please pad the one-byte item before appending its prefix up to the first NUL, and cover both alignments. The base also differed from Lua here; this is a remaining gap in the NUL handling rather than a new regression.
| width := -1 | ||
| for i < len(str) && str[i] >= '0' && str[i] <= '9' { | ||
| if width < 0 { | ||
| width = 0 | ||
| } | ||
| width = width*10 + int(str[i]-'0') | ||
| i++ |
There was a problem hiding this comment.
[P2] Preserve Lua 5.1's two-digit format limits
The scanner accepts a third width digit here and a third precision digit in the loop below. Consequently pcall(string.format, "%100s", "xy") and pcall(string.format, "%.100s", "xy") both succeed on the head; C Lua 5.1.5 rejects them with invalid format (width or precision too long). The %99s and %.99s controls succeed on both. Please mirror scanformat's two-digit limits for width and precision and add these boundary cases. This is also an existing compatibility gap on the base; the new scanner has not closed it.
GopherLua renders values to text through Go's formatting machinery (
fmt.SprintinLNumber.String,fmt.Sprintfinstring.format), which makes a number of results differ from the reference Lua 5.1 interpreter. This PR makes the rendering core match the reference. Every "reference 5.1" output below was checked against a lua-5.1.5 binary built from the lua.org sources (linux/amd64, glibc).1. Number → string rendering (
tostring, concatenation,table.concat,%s)Reference 5.1 converts numbers with
LUAI_NUMFMT="%.14g"(luaconf.h), and spells infinities and NaNs the way Cprintfdoes.tostring(1e15)1e+1510000000000000001e+15tostring(2^53)9.007199254741e+1590071992547409929.007199254741e+15tostring(0.1 + 0.2)0.30.300000000000000040.3tostring(1/0)/tostring(-1/0)inf/-inf+Inf/-Infinf/-inftostring(0/0)-nan(glibc/amd64)NaN-nanlocal a = 0.0; tostring(-a)-00-0The same rendering is used by concatenation,
table.concat,io.write,printandstring.format("%s", n), so all of those change together (1e15 .. ""is now"1e+15", etc.).The
-0row needed a second fix besides the renderer:allocator.LNumber2Imaps any value that compares equal to a small non-negative integer onto a preloaded constant, so-0(which compares equal to0) was replaced by the shared+0and silently lost its sign bit before it could be rendered.2.
math.hugeReference 5.1 defines
math.hugeasHUGE_VAL, i.e. positive infinity; gopher-lua usedmath.MaxFloat64.true true inffalse false 1.7976931348623157e+308true true inf3.
string.format:%s,%cand%qReference 5.1 implements these conversions through C
sprintfandstrlen(str_format/addquotedin lstrlib.c), which gives them byte-oriented semantics thatfmt.Sprintfdoes not reproduce:string.format("%s", nil)string expected, got nil"nil"string.format("%s", {})string expected, got table"table: 0x..."#string.format("%s", "a\0b")1(stops at NUL)31#string.format("%s", s)where#s == 121incl. a NUL, no precision121(kept whole: a string of 100+ bytes with no precision bypasses the C buffer)121121#string.format("%.1s", "é")(two-byte UTF-8)1(precision counts bytes)2(counted runes)1string.format("%c", 72)HHH#string.format("%c", 0)0(strlen sees nothing)10#string.format("%c", 256)0((char)256is byte 0)2(emitted U+0100 as UTF-8)0string.format("%q", "a\nb")"a\+ real newline +b""a\nb"(literal\n)"a\+ real newline +b"string.format("%q", "x\0y")"x\000y""x\x00y""x\000y"string.format("%q", "a\rb")"a\rb"(escaped)"a\rb"%qoutput round-trips throughloadstringfor every byte value 0–255 (raw CR/LF inside a quoted literal would otherwise break the lexer); a test asserts this.strFormatnow scans the format string itself and implements only%s,%cand%q; every other verb (%d,%x,%f,%g,%e,%o, ...) is still delegated tofmt.Sprintfper-directive exactly as before (LNumber/LStringimplementfmt.Formatter), so their behaviour is unchanged.Tests
_glua-tests/base.lua,_glua-tests/math.luaand_glua-tests/strings.lua. They fail on master and pass with this change.string.formatassertions in_lua5.1-tests/strings.lua(lines 102–105 of the official 5.1 suite, which diverged before) are re-enabled and now pass.go test ./...passes;gofmtclean;go vetoutput unchanged from master.Compatibility notes
string.format("%s", v)with a boolean, table or nil now raisesbad argumentas the reference does; callers relying on the old lenient behaviour needtostring(v)(which still works for every type).1e+15instead of1000000000000000), matching%.14g.nan/-nanspelling follows the sign bit, which matches Cprintfon glibc; the new tests accept either spelling and only assert that negation flips it.Related: #468 (
LNumber.String()going throughfmt.Sprint).🤖 Generated with Claude Code