Repository navigation
Conversation
284km
left a comment
There was a problem hiding this comment.
Reviewed ad15079 against e45622a. No actionable findings introduced by this change.
The dedicated titlecase/swapcase exceptions and identity-preserving fold entries address the mapping differences directly. I also checked the case-method dispatch, bang-method behavior, legacy-encoding path, and every updated call to the double-byte validator.
Validation:
- Fresh O2 build with the CI-pinned Mere
991f299bcompiler and mgz revision. - Regenerated all five tables with Ruby 4.0.9: every table matches exactly, including 86 identity fold exceptions and 77 identity titlecase exceptions.
- The new corpus regression matched Ruby with indexing ON/OFF, normally and with
MERE_RUBY_GC_STRESS=100. - Exhaustive single-byte probes across the 18 affected encodings and all 256 byte values matched all 78,336 reference rows. These cover ordinary/:ascii/:turkic/:lithuanian options, folding, result bytes/encoding/validity, and bang results.
- Mixed-string probes over the 401 distinct fold/title/swap exception codepoints matched all 20,050 mapping rows across UTF-8, UTF-16LE/BE, and UTF-32LE/BE, including ASCII/Turkic modes and folding.
- The repository runner completed the focused upstream case-mapping/options files: 18/21 tests matched. The three residual failures also reproduce on the earlier baseline; the mapping file improved from 13/18 to 17/18. These residual compatibility failures are not attributed to this patch.
- CI for this head reports 523/523 corpus programs matching Ruby and a successful Linux build. The comprehensive Unicode file is assigned to shard 5, which remains pending along with other jobs.
git diff --checkpassed.
I did not repeat the full comprehensive Unicode file or Linux execution locally. The reported full-suite result and its narrow margin under the 600-second limit still need the pending CI measurement to confirm the available headroom.
| when :swap then ch.downcase == ch ? ch.upcase : ch.downcase | ||
| else ch | ||
| end | ||
| next if out == fallback |
There was a problem hiding this comment.
Positive: Comparing against each operation's fallback preserves identity exceptions that an out == ch filter would lose. Regeneration with Ruby 4.0.9 matches all five embedded tables exactly, including the Cherokee identity folds and titlecase exceptions.
| let body = | ||
| (if enc_ascii_compat e && enc_multibyte e && (if enc_utf8_family e then false else true) then es_case_ascii (sget sh) e kind | ||
| else if sbcs_cased e && (if str_is_ascii sh then false else true) then sbcs_case (sget sh) e kind mode | ||
| else if sbcs_cased e && ((if str_is_ascii sh then false else true) || str_eq mode "t") then sbcs_case (sget sh) e kind mode |
There was a problem hiding this comment.
Positive: An ASCII-only receiver still needs the single-byte encoder path when Turkic mapping can produce a non-ASCII byte. This condition preserves that encoding-specific behavior instead of copying UTF-8 output into a legacy-encoded result. The exhaustive byte/option probes matched Ruby across all 18 encodings handled by sbcs_cased, including bang methods.
Unicode case conversion used uppercase-plus-lowercase as a substitute for titlecase, applied final-sigma context rules that Ruby does not use, and omitted identity exceptions from case folding. Correct the mappings for
capitalize,swapcaseanddowncase(:fold), including Cherokee, titlecase digraphs and Turkic cases. Preserve the receiver encoding when ASCII input maps to non-ASCII bytes. Accept the valid single byte 0x80 in GBK/CP936 and CP949 when native operations validate their input.Generate explicit titlecase and swapcase exceptions from the pinned reference Ruby, retain identity folding exceptions, and add a regression covering UTF-8/16/32, single-byte encodings, bang methods and multibyte validation.
Validation so far:
ruby/enc/test_case_comprehensive.rbfile improved from 64 failures and 4 errors to 422 tests, 2,901,771 assertions, zero failures/errors/skips, within the unchanged 600-second limit (591.5 seconds including startup; 1,727,120 KiB peak process-family RSS).