Recognize the Japanese maiden marker 旧姓 (#309)

docs/release_log.rst

@@ -33,6 +33,7 @@ Release Log
     - Fix an ASCII period after a CJK honorific stopping it being recognized: ``씨.``, ``様.``, ``氏.``, ``님.``, ``군.``, ``양.`` and ``殿.`` now route to ``suffix`` like their periodless spellings, where the trailing period had left them inside the name — the family name in ``"김민준 씨."``, the given name in ``"김민준, 씨."``. The cause was v1's initial regex, ``^(\w\.|[A-Z])?$`` (``REGEXES["initial"]``, still public v1 API), whose ``\w`` is Unicode-aware and so matched a hangul syllable or a Han ideograph as readily as a letter; the strict suffix test applies that as a veto (``V.`` in ``"John V. Smith"`` is a middle initial, not roman five), and a veto written for Latin was being asked of scripts it was never about. The cost ran past the honorific itself: because the vetoed token read as name text, the glued-honorific peel's scan back for its site stopped at it instead of stepping over it, took it as the site, found no honorific at the end of it and gave up — so ``"田中さん 様."`` kept ``さん`` inside the name, reading given ``田中さん`` and family ``様.``, while ``"田中さん 様"`` peeled it. The comma form ``"田中さん, 様."`` reached the same scan when this was written; since the #319 entry above it no longer does — its post-comma run is now declined as suffix-shaped before the scan begins, so ``様.`` gets to ``suffix`` through classification rather than by being stepped over. Measured against 1.4.0, ``"김민준, 씨."`` and ``"田中さん, 様."`` were returning exactly what 1.x returns, so the honorific work earlier in this release had a hole in it wherever the honorific was written with a period. An initial is a single LETTER standing in for a name, and Han ideographs, hangul syllables and kana are morphemes and syllables rather than letters, so the veto now asks its question only of the scripts where it means something. Alphabets keep their initials untouched — ``"А. С. Пушкин"``, ``"م. الفارسي"`` and ``"Ա. Խաչատրյան"`` are unaffected, and so is the Ukrainian conjunction entry below, where a punctuated ``Й.`` still outranks the conjunction ``й``. The public ``initial`` tag follows the same line: ``씨.`` no longer carries it. Read ``period`` strictly here: the fix is scoped to the ASCII full stop U+002E, because that is the only period ``_normalize`` strips. The fullwidth U+FF0E and the ideographic U+3002 (with its halfwidth twin U+FF61) — the stops a CJK writer is likelier to type — leave the honorific unmatchable by the vocabulary lookup, which runs before the veto is ever consulted, so ``"김민준 씨．"`` still reads the honorific as the family name. That is a separate, still-open gap in ``_normalize`` rather than in the veto: those spellings parse identically before and after this change, and widening the strip is follow-up work. **Default-on**, and it reaches ``HumanName`` too (#320)
     - Fix NFD-decomposed input missing the East Asian defaults entirely: script classification now normalizes to NFC before deciding, so a Korean or Japanese name typed on macOS — where decomposed text is routine — gets the same order rule as its composed twin, which it silently did not before. Segmentation MATCHING deliberately stays raw, so an unspaced NFD hangul name is ordered correctly but not split, rather than being split in the wrong place. One gotcha worth stating plainly: parse output preserves the encoding it was given, so for NFD input ``name.family == "김"`` is ``False`` even though it is the same name — compare NFC-normalized text when comparing across encodings (#272)
     - Fix the Ukrainian conjunction ``й`` not joining the pieces around it: it is the euphonic alternate of ``і``, the two chosen by the surrounding vowel and consonant rather than by meaning (``"Олесь і Олена"`` but ``"Марія й Петро"``), so real Ukrainian data carries both spellings and shipping only ``і`` recognized just one of them. ``"Олесь й Олена Коваленки"`` now gives given ``"Олесь й Олена"`` where the ``й`` previously landed in ``middle``. Same treatment as the ``и``/``і`` entries added in 2.0.0, single-letter carve-out included: the conjunction joins only once the name has enough pieces, and a punctuated initial still wins, so ``"Й. Сліпий"`` is unaffected. Raised in a comment on #267
+    - Add the Japanese maiden-name marker ``旧姓`` to the default vocabulary (#309): ``"山田花子 旧姓 佐藤"`` now gives family ``山田花子`` and maiden ``佐藤``, where 1.4.0 left the marker in the name (first ``山田花子``, middle ``旧姓``, last ``佐藤``). It sits beside the Cyrillic ``урожд.`` and German ``geb.`` entries rather than in ``locales.JA``, on the rule that admitted those: a native-script marker cannot collide with a Latin-script name and matching is whole-token, so it is safe as a default and needs no my-data-is-Japanese declaration -- it can only ever match Han text. **Scope worth knowing before you rely on it:** this reaches the SPACED form only. Japanese more often brackets the marker, and ``"山田（旧姓：佐藤）"`` under ``Policy(maiden_delimiters=...)`` still returns maiden ``"旧姓：佐藤"`` with the marker and its colon attached, because delimited content is claimed whole before the marker inside it is classified. That is not a Japanese limitation -- ``"Jane Smith (née Jones)"`` keeps its marker the same way -- and closing it is #329. **Default-on**, and it reaches ``HumanName`` too (#309)
 
     **Documentation**
 

nameparser/config/maiden_markers.py

@@ -17,6 +17,7 @@
     'урожденная',
     'урождённый',
     'урожденный',
+    '旧姓',
 }
 """
 Marker words that introduce a birth surname, e.g. "Jane Smith née Jones"
@@ -29,6 +30,24 @@
 awaits the same vetting. Entries are stored normalized: lowercase, no
 periods.
 
+Japanese 旧姓 is here rather than in locales.JA, on the rule that
+admitted the Cyrillic entries: a native-script marker cannot collide
+with a Latin-script name, and matching is whole-token, so it is safe
+as a default. Neither character appears in any shipped surname, title,
+suffix, conjunction, particle or bound-given vocabulary. locales.JA
+is for what needs the my-data-is-Japanese declaration -- segmentation,
+where a pure-Han string cannot say which language wrote it -- and this
+needs none, since it can only ever match Han text.
+
+It reaches the SPACED form only ("山田花子 旧姓 佐藤" gives maiden
+佐藤). Japanese more often writes the marker inside brackets, and
+"山田（旧姓：佐藤）" under Policy(maiden_delimiters=...) still yields
+maiden "旧姓：佐藤" with the marker and its colon attached: extract
+claims delimited content whole, before classify has tagged anything
+inside it, so group's marker-consuming rule never sees it. That
+asymmetry is not Japanese -- "Jane Smith (née Jones)" keeps its marker
+the same way -- and closing it is #329.
+
 Consumed by the 2.0 parser's default lexicon. The 1.x parser does not
 read this module.
 

tests/v2/cases.py

@@ -428,6 +428,31 @@ def __post_init__(self) -> None:
          {"given": "Jane", "family": "Smith", "maiden": "Jones"},
          classification="fix(#274)",
          notes="v1 mangles to middle='Smith née'"),
+    Case("maiden_marker_kyusei", "山田花子 旧姓 佐藤",
+         {"family": "山田花子", "maiden": "佐藤"},
+         classification="fix(#309)",
+         notes="旧姓 is default vocabulary, not pack data: a "
+               "native-script marker cannot collide with a Latin-script "
+               "name and matching is whole-token, the same rule that "
+               "admitted урожд. Reaches the SPACED form only -- Japanese "
+               "more often brackets the marker, and '山田（旧姓：佐藤）' "
+               "under maiden_delimiters still gives maiden '旧姓：佐藤', "
+               "marker and colon attached, because extract claims "
+               "delimited content before classify tags anything inside "
+               "it. Not a Japanese problem: 'Jane Smith (née Jones)' "
+               "keeps its marker the same way (#329). 1.4.0 read this "
+               "first 山田花子 / middle 旧姓 / last 佐藤 -- the marker "
+               "sat in the name"),
+    Case("maiden_marker_kyusei_segmented", "山田 花子 旧姓 佐藤",
+         {"given": "花子", "family": "山田", "maiden": "佐藤"},
+         classification="fix(#309)",
+         notes="the row above with the family name spaced, so the "
+               "marker is consumed from a name that already has a "
+               "given side -- pinned because #274's consuming rule "
+               "takes the marker plus the piece after it, and the "
+               "pieces before it are what could have gone wrong. "
+               "1.4.0 read this first 山田 / middle '花子 旧姓' / "
+               "last 佐藤"),
     Case("east_slavic", "Сидоров Иван Петрович",
          {"given": "Иван", "middle": "Петрович", "family": "Сидоров"},
          policy=_ES),

tools/differential/corpus_cjk.jsonl

@@ -22,8 +22,10 @@
 "威廉・莎士比亚"
 "山田 エミ"
 "山田 太郎 (マイケル・ジャクソン)"
+"山田 花子 旧姓 佐藤"
 "山田「タロ」太郎"
 "山田太郎様"
+"山田花子 旧姓 佐藤"
 "张伟"
 "毛 泽东"
 "毛 김"

tools/differential/expected_changes.toml

@@ -66,6 +66,19 @@ issue = "fix(#274) maiden markers consumed"
 name_regex = "(?i)\\b(n[ée]e|born|geb\\.?|roz\\.?)\\b"
 fields = ["maiden", "middle", "last"]
 
+[[change]]
+issue = "fix(cjk-maiden-marker) maiden marker consumed, compounding with the CJK order flip"
+# Its own rule rather than a widening of fix(#274) above: that rule's
+# fields stop at maiden/middle/last because a Latin marker moves only
+# those, while a Han one also flips `first` -- the name left after the
+# marker is consumed is wholly Han, so it reads family-first (#271).
+# Adding `first` to #274's list would make that rule broader than its
+# prose and let it absorb diffs that have nothing to do with markers
+# (#328). The regex is the marker itself, so this rule can claim
+# nothing else.
+name_regex = "旧姓"
+fields = ["first", "middle", "last", "maiden"]
+
 [[change]]
 issue = "fix(comma-family) lone post-comma piece routes to suffix/title, not first"
 # 'Smith, Dr.' / 'Andrews, M.D.': v1 put the lone strict-suffix-or-title