Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .changeset/26463-locale-language-name.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
'@objectstack/spec': minor
'@objectstack/platform-objects': minor
'@objectstack/service-messaging': minor
---

feat(spec)!: a single-word language name (`Chinese`, `English`, `Japanese`) is refused as a locale

The 18.x locale shape refused `zh_CN`, a padded value and a multi-word name such as `Chinese (Simplified)`, but its 2–8 letter language subtag still admitted any single word of that length. An author who wrote `locale: 'Chinese'` got a green save, and the strings were stored under a locale no session ever asks for, so they never rendered. The refusal text already said a language name is refused.

The language subtag is now 2–3 letters, the ISO 639 codes every tag in use starts with (BCP-47 reserves 4 letters and registers no 5–8 letter language). Everything else in the shape is unchanged: hyphen-separated 1–8 letter-or-digit subtags (`en`, `zh-CN`, `zh-Hans-CN`, `es-419`, `yue-HK`), and case is not judged. The one shape moves in all three places that hold it:

- `LocaleSchema` (`@objectstack/spec/system`): a `translation` item's `locale`, translation bundle keys, `i18n.defaultLocale` / `fallbackLocale` / `supportedLocales`, a seed dataset's `locale` scope and the seed loader's `locale`.
- `SYS_USER_LOCALE_TAG_PATTERN` (`@objectstack/platform-objects/identity`): the `locale_bcp47_shape` rule on a user's own `sys_user.locale` column.
- `LOCALE_TAG_SHAPE` (`@objectstack/service-messaging`): the read side that picks a recipient's notification locale, byte-identical to the column rule.

Clause-②: no (narrowing) — the locale shape refuses values it used to accept. Nothing is widened.

Metadata-protocol: narrowed (a locale whose language subtag is 4–8 letters, in practice a language name, is refused on translation items, bundle keys, the i18n configuration and seed datasets)

<!-- adr-0087: not-required (already-registered translation-locale-non-tag-refused) the 18.x D3 entry already prescribes rewriting a language name by hand, and its conversion already lists such a stored row as a TODO through LocaleSchema; this diff narrows the same shape and corrects that entry's subtag length -->

**BREAKING** for an author or user who wrote a language name as a locale.

- **Before.** `defineTranslation({ name: 'qa_x', locale: 'Chinese', … })` parsed and its strings never rendered; `PATCH` of a user's `locale` to `Japanese` was stored, and that user's notifications fell to the delivery ladder's floor.
- **Now.** The value is refused at `locale` (or at the bundle key) with a message naming the tag shape, and the user's write is refused by the column's `locale_bcp47_shape` rule.
- **Fix.** Write the tag, not the name: `Chinese` → `zh-CN`, `English` → `en`, `Japanese` → `ja-JP`. A language name has no single tag it can have meant, so `os migrate meta --stored` lists a stored translation row as a TODO (D3 entry `translation-locale-non-tag-refused`) and you rewrite it by hand. A user whose stored `locale` is already a name such as `Japanese` keeps loading: their notifications fall to the deployment default, and saving that value again is refused until it is replaced with a tag.
12 changes: 9 additions & 3 deletions packages/platform-objects/src/identity/sys-user.object.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,17 @@ import { ObjectSchema, Field } from '@objectstack/spec/data';
import { INVITATION_ROLE_OPTIONS } from '@objectstack/spec/identity';

/**
* The BCP-47 shape `sys_user.locale` accepts — a 2–8 letter language subtag
* The BCP-47 shape `sys_user.locale` accepts — a 2–3 letter language subtag
* followed by any number of 1–8 alphanumeric subtags (`zh`, `zh-CN`,
* `zh-Hans-CN`, `es-419`). Shape only: membership in a shipped template bundle
* is the bundle's business, and the delivery-time ladders handle a
* shipped-nowhere tag by falling to their floor.
* shipped-nowhere tag by falling to their floor. The primary subtag is held to
* the 2–3 letters of the ISO 639 codes real tags start with, so a language
* name (`Japanese`) or a stringified nothing (`null`) is refused as
* malformed: BCP-47 reserves 4 letters and registers no 5–8 letter language.
* The same pattern as `LOCALE_TAG_SHAPE` behind `LocaleSchema` in
* `@objectstack/spec/system`, so a translation is stored only under a tag a
* user can hold.
*
* A JS regex SOURCE string, because that is what a `format` validation rule
* carries (`new RegExp(rule.regex)` in objectql's rule validator) — anchored at
Expand All @@ -30,7 +36,7 @@ import { INVITATION_ROLE_OPTIONS } from '@objectstack/spec/identity';
* `recipient-locale-shape-parity.test.ts` in service-messaging asserts the two
* spellings are byte-identical and names what breaks if they are not.
*/
export const SYS_USER_LOCALE_TAG_PATTERN = '^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$';
export const SYS_USER_LOCALE_TAG_PATTERN = '^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{1,8})*$';

/**
* sys_user — System User Object
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,16 +202,18 @@ describe('sys_user.locale — the guard and the shape check compose', () => {
expect(refusal(data, 'update')).toMatchObject({ code: 'VALIDATION_FAILED' });
});

it('the check is of SHAPE, not of membership — an 8-letter word is a legal tag', () => {
// Measured while writing this file, and worth a pin rather than a comment:
// `Japanese` is eight letters, so it satisfies the primary-subtag rule and
// the column accepts it. That is correct and deliberate — membership in a
// shipped template bundle is the bundle's business, and a tag nothing
// ships falls to the delivery ladder's floor (`en-US` in `sendTemplate`,
// `DEFAULT_LOCALE` in the store) rather than dead-lettering. The ruling's
// safety property is about malformed values, NOT unknown ones, and a
// reader who expects this rule to reject `Japanese` would "fix" it into a
// closed vocabulary the platform does not have.
expect(refusal({ locale: 'Japanese' }, 'update')).toBeNull();
it('the check is of SHAPE, not of membership — a language name is malformed, an unshipped tag is not', () => {
// The ruling asks for a BCP-47 SHAPE check and fixes no subtag length.
// Since #26463 the primary subtag is the 2–3 letters of the ISO 639 codes
// real tags start with (BCP-47 reserves 4 letters and registers no 5–8
// letter language), so `Japanese` is refused as malformed: it is a name,
// and no session ever asks for it. That is still shape, not membership —
// a well-formed tag nothing ships (`tlh-QQ`) is accepted and falls to the
// delivery ladder's floor (`en-US` in `sendTemplate`, `DEFAULT_LOCALE` in
// the store) rather than dead-lettering. A reader who wants this rule to
// reject an unshipped tag would "fix" it into a closed vocabulary the
// platform does not have.
expect(refusal({ locale: 'Japanese' }, 'update')).toMatchObject({ code: 'VALIDATION_FAILED' });
expect(refusal({ locale: 'tlh-QQ' }, 'update')).toBeNull();
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ describe('recipient locale shape ↔ sys_user.locale write rule', () => {

it.each([
'zh', 'zh-CN', 'zh-Hans-CN', 'es-419', 'en-US', 'yue-HK',
'Chinese (Simplified)', 'zh_CN', '-CN', 'zh-Hansumlaut', '1zh', 'zh CN',
'Chinese (Simplified)', 'Chinese', 'Japanese', 'null', 'zh_CN', '-CN', 'zh-Hansumlaut', '1zh', 'zh CN',
'../../etc/passwd', '',
])('agrees with the compiled write-side pattern on %j', (value) => {
// The behavioural half. Redundant while the sources are equal — and that
Expand All @@ -69,17 +69,15 @@ describe('recipient locale shape ↔ sys_user.locale write rule', () => {
expect(LOCALE_TAG_SHAPE.test(value)).toBe(writeSide);
});

it('the read side is deliberately STRICTER on the stringified-nothing literals', () => {
// `"null"` is four letters, so it is shape-legal on both sides — the write
// rule accepts it and only `NOTHING_LITERALS` in this module stops it
// reaching a template lookup. Pinned as intended asymmetry so a future
// reader does not "fix" the shared regex to close it: a lossy producer's
// stringified nothing arrives below the data API, where the write rule
// never runs, so tightening the shape would cost real tags (`null` is not
// a language, but neither is it reachable by a user picking one from a
// list) without closing the path it arrived by.
expect(new RegExp(SYS_USER_LOCALE_TAG_PATTERN).test('null')).toBe(true);
expect(LOCALE_TAG_SHAPE.test('null')).toBe(true);
it('the stringified-nothing literals fail the shape on both sides, and the read side refuses them anyway', () => {
// Since #26463 the primary subtag is 2–3 letters, so `"null"` (four
// letters) is shape-refused on both sides, like any language name.
// `NOTHING_LITERALS` in this module still refuses it on its own: a lossy
// producer's stringified nothing arrives below the data API, where the
// write rule never runs, so the read side keeps that floor rather than
// depending on the shared shape staying this narrow.
expect(new RegExp(SYS_USER_LOCALE_TAG_PATTERN).test('null')).toBe(false);
expect(LOCALE_TAG_SHAPE.test('null')).toBe(false);
expect(normalizeRecipientLocale('null')).toBeUndefined();
// `"undefined"` is nine letters and fails the shape on both sides — it
// never depended on the literal list.
Expand Down
18 changes: 9 additions & 9 deletions packages/services/service-messaging/src/recipient-locale.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ export const RECIPIENT_LOCALE_FIELD = 'locale';
export const USER_OBJECT = 'sys_user';

/**
* BCP-47-shaped: a 2–8 letter language subtag, then any number of 1–8
* BCP-47-shaped: a 2–3 letter language subtag, then any number of 1–8
* alphanumeric subtags (`zh`, `zh-CN`, `zh-Hans-CN`, `es-419`). Shape only —
* membership is the template bundle's business, and the ladders below handle
* a shipped-nowhere tag by falling to their floor.
Expand All @@ -99,20 +99,20 @@ export const USER_OBJECT = 'sys_user';
* This module keeps its independent job either way, and the two are NOT
* redundant. The write-side rule guards values ARRIVING through the data API;
* this guards values already at rest — rows written before the rule existed
* and rows written below the data API. It is also strictly the stricter of the
* two: `"null"` is four letters and therefore SHAPE-LEGAL, so the write rule
* would accept it and only {@link NOTHING_LITERALS} below stops it reaching a
* template lookup. That asymmetry is deliberate rather than a gap in the
* regex — the shape is shared with the write side byte-for-byte, and a
* lossy producer's stringified nothing is a data-at-rest problem, not a
* language a user can be typing.
* and rows written below the data API. It is also never the looser of the
* two: the shape is shared with the write side byte-for-byte, and on top of it
* {@link NOTHING_LITERALS} below refuses a lossy producer's stringified
* nothing. Since the primary subtag is 2–3 letters both of those literals
* also fail the shape, so the list no longer changes an answer; it stays as
* the read side's own floor, so a future widening of the shared shape cannot
* hand `"null"` to a template lookup again.
*
* Exported for that pin and for nothing else — the normalizer below is the
* supported entry point, and a caller that reaches for the raw regex is
* re-implementing {@link normalizeRecipientLocale} minus the parts that
* matter.
*/
export const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$/;
export const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{1,8})*$/;

/**
* The stringified-nothing literals a lossy producer can leave in a column.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ export const entry: SemanticMigration = {
+ 'bundle locale key, i18n.defaultLocale / fallbackLocale / supportedLocales, a seed dataset '
+ 'locale scope, the seed loader locale',
replacement:
'the BCP-47 tag the value was written for: a 2-8 letter language subtag, then hyphen-separated '
'the BCP-47 tag the value was written for: a 2-3 letter language subtag, then hyphen-separated '
+ '1-8 letter-or-digit subtags (`en`, `zh-CN`, `zh-Hans-CN`, `es-419`). Write `zh-CN` for '
+ '`zh_CN` and drop surrounding whitespace; the conversion does both for stored rows and built '
+ 'artifacts. A language name or any other text has no single tag it can have meant, so rewrite '
Expand Down
14 changes: 12 additions & 2 deletions packages/spec/src/system/translation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,18 @@ describe('LocaleSchema', () => {
});
});

it.each(['zh-Hans-CN', 'es-419', 'yue-HK', 'zh-cn'])('accepts the tag %j (case is not judged)', (locale) => {
it.each(['en', 'zh-CN', 'zh-Hans-CN', 'es-419', 'yue-HK', 'zh-cn'])('accepts the tag %j (case is not judged)', (locale) => {
expect(LocaleSchema.safeParse(locale).success).toBe(true);
});

// #25822: each of these saved green and was stored under a locale no session
// ever asks for, so its strings never rendered.
it.each(['zh_CN', 'zh-CN ', ' en', 'zh CN', 'Chinese (Simplified)', 'not a locale!!', '', '-CN', '1zh', 'zh-Hansumlaut'])(
// #26463: a single-word language name passed the first 2–8 letter primary
// subtag; the subtag is now the 2–3 letters real tags start with.
it.each([
'zh_CN', 'zh-CN ', ' en', 'zh CN', 'Chinese (Simplified)', 'not a locale!!', '', '-CN', '1zh', 'zh-Hansumlaut',
'Chinese', 'English', 'Japanese', 'chinese-CN', 'null',
])(
'refuses %j as a regex-format issue that names the release',
(locale) => {
const result = LocaleSchema.safeParse(locale);
Expand All @@ -71,6 +76,10 @@ describe('LocaleSchema', () => {
expect(item.success).toBe(false);
expect(item.error!.issues.map((i) => i.path.join('.'))).toEqual(['locale']);
expect(() => defineTranslation({ name: 'zh_cn', locale: 'zh_CN', messages: { x: 'y' } })).toThrow(/BCP-47 language tag/);
// The card's reproduction: a single-word language name at the item door.
const named = TranslationItemSchema.safeParse({ name: 'qa_bad_locale', locale: 'Chinese', objects: { t: { label: 'x' } } });
expect(named.success).toBe(false);
expect(named.error!.issues.map((i) => [i.path.join('.'), i.code])).toEqual([['locale', 'invalid_format']]);

// The bundle door: the map key itself (`defineTranslationBundle`, `stack.translations`).
const bundle = TranslationBundleSchema.safeParse({ zh_CN: { messages: { x: 'y' } } });
Expand All @@ -79,6 +88,7 @@ describe('LocaleSchema', () => {
expect(bundle.error!.issues[0]!.path).toEqual(['zh_CN']);
expect(PlatformTranslationBundleSchema.safeParse({ 'en ': { messages: { x: 'y' } } }).success).toBe(false);
expect(() => defineTranslationBundle({ zh_CN: { messages: { x: 'y' } } })).toThrow();
expect(TranslationBundleSchema.safeParse({ Chinese: { messages: { x: 'y' } } }).success).toBe(false);

// The stack's i18n configuration.
const config = TranslationConfigSchema.safeParse({ defaultLocale: 'en', supportedLocales: ['en', 'zh_CN'] });
Expand Down
27 changes: 16 additions & 11 deletions packages/spec/src/system/translation.zod.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,27 +14,32 @@ import { SnakeCaseIdentifierSchema } from '../shared/identifiers.zod';
import { textSlotTemplateRefusal } from '../automation/flow-text-slot-template';

/**
* The shape a locale tag must have: a 2–8 letter language subtag, then any
* The shape a locale tag must have: a 2–3 letter language subtag, then any
* number of hyphen-separated 1–8 letter-or-digit subtags (`en`, `zh-CN`,
* `zh-Hans-CN`, `es-419`). The same pattern as `SYS_USER_LOCALE_TAG_PATTERN`
* (`@objectstack/platform-objects/identity`), the shape a user's own `locale`
* column accepts, so a translation is stored only under tags a session can
* ask for.
* `zh-Hans-CN`, `es-419`, `yue-HK`). The same pattern as
* `SYS_USER_LOCALE_TAG_PATTERN` (`@objectstack/platform-objects/identity`),
* the shape a user's own `locale` column accepts, so a translation is stored
* only under tags a session can ask for.
*
* Shape only, by decision: membership has no registry to check against
* (`Intl.getCanonicalLocales('xx-YY')` succeeds, which is why
* `shared/value-domain.zod.ts` dropped a `bcp47_locale` domain), and case is
* not judged because the resolvers fold it (`zh-cn` resolves to `zh-CN`).
* What it refuses is what no resolver ever matches: an underscore separator
* (`zh_CN`), whitespace, a language name.
* The primary subtag is held to the 2–3 letters of the ISO 639 codes every
* tag in use starts with: BCP-47 reserves 4 letters and registers no 5–8
* letter language, so the only values that wider grammar ever admitted were
* words. What it refuses is what no resolver ever matches: an underscore
* separator (`zh_CN`), whitespace, a language name (`Chinese`,
* `Chinese (Simplified)`).
*/
const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$/;
const LOCALE_TAG_SHAPE = /^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{1,8})*$/;

const LOCALE_TAG_REFUSAL =
'must be a BCP-47 language tag such as `en`, `zh-CN` or `zh-Hans-CN`: a 2–8 letter language '
'must be a BCP-47 language tag such as `en`, `zh-CN` or `zh-Hans-CN`: a 2–3 letter language '
+ 'subtag, then hyphen-separated subtags. Refused since @objectstack/spec 18, because a translation '
+ 'stored under any other spelling (`zh_CN`, a trailing space, a language name) is under a locale no '
+ 'session ever asks for, so its strings never render. Separate subtags with `-`, not `_`.';
+ 'stored under any other spelling (`zh_CN`, a trailing space, a language name such as `Chinese`) '
+ 'is under a locale no session ever asks for, so its strings never render. Separate subtags with '
+ '`-`, not `_`, and write the tag, not the name (`zh-CN`, not `Chinese`).';

export const LocaleSchema = lazySchema(() => z
.string()
Expand Down
Loading