diff --git a/.changeset/26463-locale-language-name.md b/.changeset/26463-locale-language-name.md new file mode 100644 index 00000000000..8ff5e2806b4 --- /dev/null +++ b/.changeset/26463-locale-language-name.md @@ -0,0 +1,27 @@ +--- +'@objectstack/spec': minor +'@objectstack/platform-objects': minor +'@objectstack/service-messaging': minor +--- + +feat(spec)!: a single-word language name (`Chinese`, `English`, `Japanese`) is refused as a locale + +The 18.x locale shape refused `zh_CN`, a padded value and a multi-word name such as `Chinese (Simplified)`, but its 2–8 letter language subtag still admitted any single word of that length. An author who wrote `locale: 'Chinese'` got a green save, and the strings were stored under a locale no session ever asks for, so they never rendered. The refusal text already said a language name is refused. + +The language subtag is now 2–3 letters, the ISO 639 codes every tag in use starts with (BCP-47 reserves 4 letters and registers no 5–8 letter language). Everything else in the shape is unchanged: hyphen-separated 1–8 letter-or-digit subtags (`en`, `zh-CN`, `zh-Hans-CN`, `es-419`, `yue-HK`), and case is not judged. The one shape moves in all three places that hold it: + +- `LocaleSchema` (`@objectstack/spec/system`): a `translation` item's `locale`, translation bundle keys, `i18n.defaultLocale` / `fallbackLocale` / `supportedLocales`, a seed dataset's `locale` scope and the seed loader's `locale`. +- `SYS_USER_LOCALE_TAG_PATTERN` (`@objectstack/platform-objects/identity`): the `locale_bcp47_shape` rule on a user's own `sys_user.locale` column. +- `LOCALE_TAG_SHAPE` (`@objectstack/service-messaging`): the read side that picks a recipient's notification locale, byte-identical to the column rule. + +Clause-②: no (narrowing) — the locale shape refuses values it used to accept. Nothing is widened. + +Metadata-protocol: narrowed (a locale whose language subtag is 4–8 letters, in practice a language name, is refused on translation items, bundle keys, the i18n configuration and seed datasets) + + + +**BREAKING** for an author or user who wrote a language name as a locale. + +- **Before.** `defineTranslation({ name: 'qa_x', locale: 'Chinese', … })` parsed and its strings never rendered; `PATCH` of a user's `locale` to `Japanese` was stored, and that user's notifications fell to the delivery ladder's floor. +- **Now.** The value is refused at `locale` (or at the bundle key) with a message naming the tag shape, and the user's write is refused by the column's `locale_bcp47_shape` rule. +- **Fix.** Write the tag, not the name: `Chinese` → `zh-CN`, `English` → `en`, `Japanese` → `ja-JP`. A language name has no single tag it can have meant, so `os migrate meta --stored` lists a stored translation row as a TODO (D3 entry `translation-locale-non-tag-refused`) and you rewrite it by hand. A user whose stored `locale` is already a name such as `Japanese` keeps loading: their notifications fall to the deployment default, and saving that value again is refused until it is replaced with a tag. diff --git a/packages/platform-objects/src/identity/sys-user.object.ts b/packages/platform-objects/src/identity/sys-user.object.ts index 616eb62d078..3c402b0a022 100644 --- a/packages/platform-objects/src/identity/sys-user.object.ts +++ b/packages/platform-objects/src/identity/sys-user.object.ts @@ -4,11 +4,17 @@ import { ObjectSchema, Field } from '@objectstack/spec/data'; import { INVITATION_ROLE_OPTIONS } from '@objectstack/spec/identity'; /** - * The BCP-47 shape `sys_user.locale` accepts — a 2–8 letter language subtag + * The BCP-47 shape `sys_user.locale` accepts — a 2–3 letter language subtag * followed by any number of 1–8 alphanumeric subtags (`zh`, `zh-CN`, * `zh-Hans-CN`, `es-419`). Shape only: membership in a shipped template bundle * is the bundle's business, and the delivery-time ladders handle a - * shipped-nowhere tag by falling to their floor. + * shipped-nowhere tag by falling to their floor. The primary subtag is held to + * the 2–3 letters of the ISO 639 codes real tags start with, so a language + * name (`Japanese`) or a stringified nothing (`null`) is refused as + * malformed: BCP-47 reserves 4 letters and registers no 5–8 letter language. + * The same pattern as `LOCALE_TAG_SHAPE` behind `LocaleSchema` in + * `@objectstack/spec/system`, so a translation is stored only under a tag a + * user can hold. * * A JS regex SOURCE string, because that is what a `format` validation rule * carries (`new RegExp(rule.regex)` in objectql's rule validator) — anchored at @@ -30,7 +36,7 @@ import { INVITATION_ROLE_OPTIONS } from '@objectstack/spec/identity'; * `recipient-locale-shape-parity.test.ts` in service-messaging asserts the two * spellings are byte-identical and names what breaks if they are not. */ -export const SYS_USER_LOCALE_TAG_PATTERN = '^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$'; +export const SYS_USER_LOCALE_TAG_PATTERN = '^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{1,8})*$'; /** * sys_user — System User Object diff --git a/packages/plugins/plugin-auth/src/sys-user-locale-write-contract.test.ts b/packages/plugins/plugin-auth/src/sys-user-locale-write-contract.test.ts index 0afc8d0c323..d530c7a8ed7 100644 --- a/packages/plugins/plugin-auth/src/sys-user-locale-write-contract.test.ts +++ b/packages/plugins/plugin-auth/src/sys-user-locale-write-contract.test.ts @@ -202,16 +202,18 @@ describe('sys_user.locale — the guard and the shape check compose', () => { expect(refusal(data, 'update')).toMatchObject({ code: 'VALIDATION_FAILED' }); }); - it('the check is of SHAPE, not of membership — an 8-letter word is a legal tag', () => { - // Measured while writing this file, and worth a pin rather than a comment: - // `Japanese` is eight letters, so it satisfies the primary-subtag rule and - // the column accepts it. That is correct and deliberate — membership in a - // shipped template bundle is the bundle's business, and a tag nothing - // ships falls to the delivery ladder's floor (`en-US` in `sendTemplate`, - // `DEFAULT_LOCALE` in the store) rather than dead-lettering. The ruling's - // safety property is about malformed values, NOT unknown ones, and a - // reader who expects this rule to reject `Japanese` would "fix" it into a - // closed vocabulary the platform does not have. - expect(refusal({ locale: 'Japanese' }, 'update')).toBeNull(); + it('the check is of SHAPE, not of membership — a language name is malformed, an unshipped tag is not', () => { + // The ruling asks for a BCP-47 SHAPE check and fixes no subtag length. + // Since #26463 the primary subtag is the 2–3 letters of the ISO 639 codes + // real tags start with (BCP-47 reserves 4 letters and registers no 5–8 + // letter language), so `Japanese` is refused as malformed: it is a name, + // and no session ever asks for it. That is still shape, not membership — + // a well-formed tag nothing ships (`tlh-QQ`) is accepted and falls to the + // delivery ladder's floor (`en-US` in `sendTemplate`, `DEFAULT_LOCALE` in + // the store) rather than dead-lettering. A reader who wants this rule to + // reject an unshipped tag would "fix" it into a closed vocabulary the + // platform does not have. + expect(refusal({ locale: 'Japanese' }, 'update')).toMatchObject({ code: 'VALIDATION_FAILED' }); + expect(refusal({ locale: 'tlh-QQ' }, 'update')).toBeNull(); }); }); diff --git a/packages/services/service-messaging/src/recipient-locale-shape-parity.test.ts b/packages/services/service-messaging/src/recipient-locale-shape-parity.test.ts index 563bba40226..bc5140ce5e5 100644 --- a/packages/services/service-messaging/src/recipient-locale-shape-parity.test.ts +++ b/packages/services/service-messaging/src/recipient-locale-shape-parity.test.ts @@ -57,7 +57,7 @@ describe('recipient locale shape ↔ sys_user.locale write rule', () => { it.each([ 'zh', 'zh-CN', 'zh-Hans-CN', 'es-419', 'en-US', 'yue-HK', - 'Chinese (Simplified)', 'zh_CN', '-CN', 'zh-Hansumlaut', '1zh', 'zh CN', + 'Chinese (Simplified)', 'Chinese', 'Japanese', 'null', 'zh_CN', '-CN', 'zh-Hansumlaut', '1zh', 'zh CN', '../../etc/passwd', '', ])('agrees with the compiled write-side pattern on %j', (value) => { // The behavioural half. Redundant while the sources are equal — and that @@ -69,17 +69,15 @@ describe('recipient locale shape ↔ sys_user.locale write rule', () => { expect(LOCALE_TAG_SHAPE.test(value)).toBe(writeSide); }); - it('the read side is deliberately STRICTER on the stringified-nothing literals', () => { - // `"null"` is four letters, so it is shape-legal on both sides — the write - // rule accepts it and only `NOTHING_LITERALS` in this module stops it - // reaching a template lookup. Pinned as intended asymmetry so a future - // reader does not "fix" the shared regex to close it: a lossy producer's - // stringified nothing arrives below the data API, where the write rule - // never runs, so tightening the shape would cost real tags (`null` is not - // a language, but neither is it reachable by a user picking one from a - // list) without closing the path it arrived by. - expect(new RegExp(SYS_USER_LOCALE_TAG_PATTERN).test('null')).toBe(true); - expect(LOCALE_TAG_SHAPE.test('null')).toBe(true); + it('the stringified-nothing literals fail the shape on both sides, and the read side refuses them anyway', () => { + // Since #26463 the primary subtag is 2–3 letters, so `"null"` (four + // letters) is shape-refused on both sides, like any language name. + // `NOTHING_LITERALS` in this module still refuses it on its own: a lossy + // producer's stringified nothing arrives below the data API, where the + // write rule never runs, so the read side keeps that floor rather than + // depending on the shared shape staying this narrow. + expect(new RegExp(SYS_USER_LOCALE_TAG_PATTERN).test('null')).toBe(false); + expect(LOCALE_TAG_SHAPE.test('null')).toBe(false); expect(normalizeRecipientLocale('null')).toBeUndefined(); // `"undefined"` is nine letters and fails the shape on both sides — it // never depended on the literal list. diff --git a/packages/services/service-messaging/src/recipient-locale.ts b/packages/services/service-messaging/src/recipient-locale.ts index f6806023730..a45f856d310 100644 --- a/packages/services/service-messaging/src/recipient-locale.ts +++ b/packages/services/service-messaging/src/recipient-locale.ts @@ -78,7 +78,7 @@ export const RECIPIENT_LOCALE_FIELD = 'locale'; export const USER_OBJECT = 'sys_user'; /** - * BCP-47-shaped: a 2–8 letter language subtag, then any number of 1–8 + * BCP-47-shaped: a 2–3 letter language subtag, then any number of 1–8 * alphanumeric subtags (`zh`, `zh-CN`, `zh-Hans-CN`, `es-419`). Shape only — * membership is the template bundle's business, and the ladders below handle * a shipped-nowhere tag by falling to their floor. @@ -99,20 +99,20 @@ export const USER_OBJECT = 'sys_user'; * This module keeps its independent job either way, and the two are NOT * redundant. The write-side rule guards values ARRIVING through the data API; * this guards values already at rest — rows written before the rule existed - * and rows written below the data API. It is also strictly the stricter of the - * two: `"null"` is four letters and therefore SHAPE-LEGAL, so the write rule - * would accept it and only {@link NOTHING_LITERALS} below stops it reaching a - * template lookup. That asymmetry is deliberate rather than a gap in the - * regex — the shape is shared with the write side byte-for-byte, and a - * lossy producer's stringified nothing is a data-at-rest problem, not a - * language a user can be typing. + * and rows written below the data API. It is also never the looser of the + * two: the shape is shared with the write side byte-for-byte, and on top of it + * {@link NOTHING_LITERALS} below refuses a lossy producer's stringified + * nothing. Since the primary subtag is 2–3 letters both of those literals + * also fail the shape, so the list no longer changes an answer; it stays as + * the read side's own floor, so a future widening of the shared shape cannot + * hand `"null"` to a template lookup again. * * Exported for that pin and for nothing else — the normalizer below is the * supported entry point, and a caller that reaches for the raw regex is * re-implementing {@link normalizeRecipientLocale} minus the parts that * matter. */ -export const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$/; +export const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{1,8})*$/; /** * The stringified-nothing literals a lossy producer can leave in a column. diff --git a/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts b/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts index f3b01aa71ef..00aa160c310 100644 --- a/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts +++ b/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts @@ -15,7 +15,7 @@ export const entry: SemanticMigration = { + 'bundle locale key, i18n.defaultLocale / fallbackLocale / supportedLocales, a seed dataset ' + 'locale scope, the seed loader locale', replacement: - 'the BCP-47 tag the value was written for: a 2-8 letter language subtag, then hyphen-separated ' + 'the BCP-47 tag the value was written for: a 2-3 letter language subtag, then hyphen-separated ' + '1-8 letter-or-digit subtags (`en`, `zh-CN`, `zh-Hans-CN`, `es-419`). Write `zh-CN` for ' + '`zh_CN` and drop surrounding whitespace; the conversion does both for stored rows and built ' + 'artifacts. A language name or any other text has no single tag it can have meant, so rewrite ' diff --git a/packages/spec/src/system/translation.test.ts b/packages/spec/src/system/translation.test.ts index 7de3fcebe38..81afea834b0 100644 --- a/packages/spec/src/system/translation.test.ts +++ b/packages/spec/src/system/translation.test.ts @@ -47,13 +47,18 @@ describe('LocaleSchema', () => { }); }); - it.each(['zh-Hans-CN', 'es-419', 'yue-HK', 'zh-cn'])('accepts the tag %j (case is not judged)', (locale) => { + it.each(['en', 'zh-CN', 'zh-Hans-CN', 'es-419', 'yue-HK', 'zh-cn'])('accepts the tag %j (case is not judged)', (locale) => { expect(LocaleSchema.safeParse(locale).success).toBe(true); }); // #25822: each of these saved green and was stored under a locale no session // ever asks for, so its strings never rendered. - it.each(['zh_CN', 'zh-CN ', ' en', 'zh CN', 'Chinese (Simplified)', 'not a locale!!', '', '-CN', '1zh', 'zh-Hansumlaut'])( + // #26463: a single-word language name passed the first 2–8 letter primary + // subtag; the subtag is now the 2–3 letters real tags start with. + it.each([ + 'zh_CN', 'zh-CN ', ' en', 'zh CN', 'Chinese (Simplified)', 'not a locale!!', '', '-CN', '1zh', 'zh-Hansumlaut', + 'Chinese', 'English', 'Japanese', 'chinese-CN', 'null', + ])( 'refuses %j as a regex-format issue that names the release', (locale) => { const result = LocaleSchema.safeParse(locale); @@ -71,6 +76,10 @@ describe('LocaleSchema', () => { expect(item.success).toBe(false); expect(item.error!.issues.map((i) => i.path.join('.'))).toEqual(['locale']); expect(() => defineTranslation({ name: 'zh_cn', locale: 'zh_CN', messages: { x: 'y' } })).toThrow(/BCP-47 language tag/); + // The card's reproduction: a single-word language name at the item door. + const named = TranslationItemSchema.safeParse({ name: 'qa_bad_locale', locale: 'Chinese', objects: { t: { label: 'x' } } }); + expect(named.success).toBe(false); + expect(named.error!.issues.map((i) => [i.path.join('.'), i.code])).toEqual([['locale', 'invalid_format']]); // The bundle door: the map key itself (`defineTranslationBundle`, `stack.translations`). const bundle = TranslationBundleSchema.safeParse({ zh_CN: { messages: { x: 'y' } } }); @@ -79,6 +88,7 @@ describe('LocaleSchema', () => { expect(bundle.error!.issues[0]!.path).toEqual(['zh_CN']); expect(PlatformTranslationBundleSchema.safeParse({ 'en ': { messages: { x: 'y' } } }).success).toBe(false); expect(() => defineTranslationBundle({ zh_CN: { messages: { x: 'y' } } })).toThrow(); + expect(TranslationBundleSchema.safeParse({ Chinese: { messages: { x: 'y' } } }).success).toBe(false); // The stack's i18n configuration. const config = TranslationConfigSchema.safeParse({ defaultLocale: 'en', supportedLocales: ['en', 'zh_CN'] }); diff --git a/packages/spec/src/system/translation.zod.ts b/packages/spec/src/system/translation.zod.ts index d8bd010ad95..7337fedb9d9 100644 --- a/packages/spec/src/system/translation.zod.ts +++ b/packages/spec/src/system/translation.zod.ts @@ -14,27 +14,32 @@ import { SnakeCaseIdentifierSchema } from '../shared/identifiers.zod'; import { textSlotTemplateRefusal } from '../automation/flow-text-slot-template'; /** - * The shape a locale tag must have: a 2–8 letter language subtag, then any + * The shape a locale tag must have: a 2–3 letter language subtag, then any * number of hyphen-separated 1–8 letter-or-digit subtags (`en`, `zh-CN`, - * `zh-Hans-CN`, `es-419`). The same pattern as `SYS_USER_LOCALE_TAG_PATTERN` - * (`@objectstack/platform-objects/identity`), the shape a user's own `locale` - * column accepts, so a translation is stored only under tags a session can - * ask for. + * `zh-Hans-CN`, `es-419`, `yue-HK`). The same pattern as + * `SYS_USER_LOCALE_TAG_PATTERN` (`@objectstack/platform-objects/identity`), + * the shape a user's own `locale` column accepts, so a translation is stored + * only under tags a session can ask for. * * Shape only, by decision: membership has no registry to check against * (`Intl.getCanonicalLocales('xx-YY')` succeeds, which is why * `shared/value-domain.zod.ts` dropped a `bcp47_locale` domain), and case is * not judged because the resolvers fold it (`zh-cn` resolves to `zh-CN`). - * What it refuses is what no resolver ever matches: an underscore separator - * (`zh_CN`), whitespace, a language name. + * The primary subtag is held to the 2–3 letters of the ISO 639 codes every + * tag in use starts with: BCP-47 reserves 4 letters and registers no 5–8 + * letter language, so the only values that wider grammar ever admitted were + * words. What it refuses is what no resolver ever matches: an underscore + * separator (`zh_CN`), whitespace, a language name (`Chinese`, + * `Chinese (Simplified)`). */ -const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$/; +const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{1,8})*$/; const LOCALE_TAG_REFUSAL = - 'must be a BCP-47 language tag such as `en`, `zh-CN` or `zh-Hans-CN`: a 2–8 letter language ' + 'must be a BCP-47 language tag such as `en`, `zh-CN` or `zh-Hans-CN`: a 2–3 letter language ' + 'subtag, then hyphen-separated subtags. Refused since @objectstack/spec 18, because a translation ' - + 'stored under any other spelling (`zh_CN`, a trailing space, a language name) is under a locale no ' - + 'session ever asks for, so its strings never render. Separate subtags with `-`, not `_`.'; + + 'stored under any other spelling (`zh_CN`, a trailing space, a language name such as `Chinese`) ' + + 'is under a locale no session ever asks for, so its strings never render. Separate subtags with ' + + '`-`, not `_`, and write the tag, not the name (`zh-CN`, not `Chinese`).'; export const LocaleSchema = lazySchema(() => z .string()