choiyounggi · choiyounggi · Sep 14, 2026
@@ -104,3 +104,4 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
 ## [2026-08-30] revise | frontend/design/design-canvas-workflow — agent-gate Check 3 remediation (PR #164). The gate's fact finding was half right: its "no skill named `design` exists" conclusion is refuted by ground truth (the skill is listed in the authoring session's available-skills roster as `design: Create a design canvas…` and its bundled payload exists on the authoring machine's disk; the reviewer's CI environment lacks the preview, and it conflated the skill with the unrelated `/design-sync` component-upload tool) — but the verifiability objection stands: the skill is an early research preview absent from public docs, so a single un-fetchable in-session source cannot carry `confidence: verified`. Fixed per AGENTS.md: confidence downgraded to field-tested with the experience context stated in the body; two live-fetched public sources added (anthropic.com/news/claude-design-anthropic-labs — official Claude Design product announcement, research preview, Claude Code handoff; explainx.ai 2026 post — /design ships in Claude Code as a research-preview command producing editable artboards via the Artifacts runtime); the mandatory-routing directive and the frontend/index.md load-when line conditioned on the skill appearing in the session's available-skills list, with the existing no-skill edge case as the explicit fallback; the unverifiable "design-review skill" name-drop in the audit edge case generalized to session-provided audit tooling.
 ## [2026-09-03] ingest | databases/selection — new category: datastore selection by workload (4 pages: choosing-a-datastore-by-workload, relational-jsonb-vs-document-store, vector-search-engine-selection, graph-workloads-relational-vs-graph-db); brave-search research, sources cited per page
 ## [2026-09-04] ingest | infrastructure/agent-orchestration/code-graph-as-orientation-layer (field-tested — 1 new page). A locally built code knowledge graph (graphify) is a freshness-gated, lead-not-evidence orientation layer for planning and parallel task decomposition: gate on graph mtime vs git log, symbol-anchored explain/path only, CLI delivery with bounded output, graph-derived assumptions named in reports. Sources: Developers Digest, AQ Score, Autonoma, Tiare Balbi, graphify README + measurements on graphifyy 0.4.23.
+## [2026-09-14] ingest | knowledge-flush of 3 claimed candidates — 1 new page, 1 amended page, 1 dropped. New: backend/common/integrations/checksum-restored-identifiers (verified — a spreadsheet numeric cell drops an identifier's leading zero and the importer pads it back under a check-digit gate: warn on every successful repair, not only failures, carry a restored flag, measure the check's false-accept rate (200,000 random inputs through a weighted mod-11 MST validator → 9.04%), and require independent confirmation before a repaired value acts as a deciding field; Microsoft Excel leading-zero docs live-fetched and quoted). Amended: backend/python/language/bytecode-cache-staleness +1 edge row and +1 field reproduction (a mutate-and-revert done by a delegated auditor subagent sharing the worktree leaves the parent's next suite run on stale bytecode with no signal — purge `__pycache__` after any delegated verification step returns). Backlink added on databases/schema-design/column-data-types. Dropped as superseded: "evaluate a per-row rejection rule against the whole batch's set of source domains" — the originating codebase later moved batch extraction to per-page results and pinned a regression test showing batch-wide domain widening wrongly flags a row whose email matches a sibling page's domain.
@@ -0,0 +1,78 @@
+---
+id: backend-common-integrations-checksum-restored-identifiers
+domain: backend
+category: integrations
+applies_to: [general]
+confidence: verified
+sources:
+  - https://support.microsoft.com/en-us/office/keeping-leading-zeros-and-large-numbers-1bf7b935-36e1-4985-842f-5dfa51f85fe7
+  - https://support.microsoft.com/en-us/excel/set-automatic-data-conversions
+  - https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html
+last_verified: 2026-09-14
+related: [databases-schema-design-column-data-types, backend-common-integrations-consumer-required-fields, security-input-validation-at-trust-boundaries]
+---
+
+# Identifiers Repaired by a Check Digit During Import
+
+## When this applies
+
+An import reads identifiers (tax codes, business registration numbers, account
+numbers) from a spreadsheet or a CSV exported from one, the column was a numeric
+cell so the leading zero is gone (`0101096393` arrives as `101096393` or
+`101096393.0`), and the importer pads the zero back and accepts the result when
+the identifier's check digit passes. Also any other repair (re-inserting a
+dropped digit, fixing a transposition) whose only confirmation is a check digit.
+
+## Do this
+
+1. **Emit a row-level warning for every successful repair, naming the raw and the
+   repaired value**, in addition to the warning for a failed repair. A check
+   digit confirms plausibility, not identity: a single mod-11 check digit accepts
+   roughly 1 in 11 arbitrary inputs, so a wrongly repaired code is accepted
+   silently at that rate unless the success path also reports.
+2. **Carry the repair as a flag on the stored record** (`tax_code_restored`, or a
+   warnings list keyed by row), not only in a transient log line — downstream
+   stages need to know which values the user never typed.
+3. **Measure the check's false-accept rate before citing it as confirmation.**
+   Feed a few hundred thousand random inputs of the repaired shape through your
+   own validator and report the pass fraction. The expected figure for a weighted
+   mod-11 digit that rejects remainder 10 is 1/11 ≈ 9.1%.
+4. **Decide how a repaired value may gate downstream work:**
+
+| The identifier is used as | Do |
+|---------------------------|----|
+| A display/search field only | Keep the repaired value with its flag and warning |
+| A deciding field — a mismatch rejects pages, merges, or records | Require independent confirmation (registry lookup, user acknowledgement) before it gates; until then, route mismatches to review instead of rejecting |
+| A join key into another system | Confirm against that system before joining; a false repair joins to a real, unrelated entity |
+
+5. **Stop the loss at the source when you control the reader.** For CSV, read the
+   column as text (pandas `dtype=str`, or Power Query's "Text" column type per
+   Microsoft); the zero is still in the file. For an `.xlsx` numeric cell the zero
+   is not stored at all, so only the repair path (with steps 1–4) is available —
+   ask for a text-formatted re-export when the value is a deciding field.
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| The numeric cell comes through as a float string (`101096393.0`) | Strip the `.0` before the length test; otherwise the repair branch never runs and the row is rejected as malformed |
+| A branch suffix is present (9 + 3 digits became 12 digits) | Pad and re-split (`0101096393-001`) before validating; validate the base part only if the scheme checks only the base |
+| More than one repair passes the check (zero padded at different positions, two transpositions) | Reject as ambiguous and warn; picking the first passing candidate turns a 1/11 false-accept into a coin toss among valid-looking codes |
+| The identifier has 16+ digits | Excel keeps 15 significant digits and zeroes the rest; no check digit can recover that — reject and request a text re-export |
+| The user fixes the source and re-imports | Clear the flag only when the new raw value is full-length and passes as entered |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Warn only when the repaired value fails the check | Warn on every repair, pass or fail | The passing-but-wrong case is the one nobody else will catch — measured 9.04% of random 9-digit bodies pass |
+| Store the repaired value exactly as if the user had entered it | Store it with a restored flag | A false repair used as a deciding field makes every true record for that entity look mismatched, and the data vanishes without an error |
+| Apply a text/custom number format to the column after the zeros are gone | Re-import the column as text, or repair with a flag | Microsoft: formatting "will not restore leading zeros that were removed prior to formatting" |
+
+## Sources
+
+- https://support.microsoft.com/en-us/office/keeping-leading-zeros-and-large-numbers-1bf7b935-36e1-4985-842f-5dfa51f85fe7 — "Excel automatically removes leading zeros, and converts large numbers to scientific notation"; custom formatting "will not restore leading zeros that were removed prior to formatting"; Get & Transform (Power Query) can set a column to Text at import; 15 significant digits of precision
+- https://support.microsoft.com/en-us/excel/set-automatic-data-conversions — the "Remove leading zeros and convert to number" automatic conversion option (Excel for Microsoft 365 / 2024, Windows and Mac), which is on by default
+- https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html — `dtype` parameter for reading columns as strings
+- Measurement 2026-09-14 (Vietnamese MST validator, weights 31/29/23/19/17/13/7/5/3, check digit = 10 − (sum mod 11), remainder 0 rejected): 200,000 random `0`+9-digit inputs → 18,089 passed (9.04%); 200,000 random 10-digit inputs → 18,117 (9.06%). An independent reviewer in the originating session measured 18,242/200,000 on the same validator
+- Field context 2026-09-14 (company-import for a Vietnamese contact crawler): the importer restored leading zeros under a checksum gate and warned only on failure; the tax code is a deciding field for page identity, so a false restoration would mark the company's real pages as mismatched. The fix added a success-path warning ("restored … leading 0 lost in a numeric cell, verify")
@@ -111,6 +111,7 @@ Match your situation to a "load when" line; load only matching pages.
 | [externally-owned-defaults](common/integrations/externally-owned-defaults.md) | A code/config default names a resource the repo does not own (model alias, endpoint, bucket, queue, index) — reviewing or merging a PR that claims that default works, adding a startup check that the name still resolves, or diagnosing a default path that broke with no code change |
 | [consumer-required-fields](common/integrations/consumer-required-fields.md) | Writing an adapter that maps one module's records into the payload a second module (scoring engine, plugin, external client) consumes, with the target shape taken from a docstring, README example, or sample payload; such an adapter runs end to end with no error and the downstream numbers come out low; deciding which mapped fields need their own assertion |
 | [robots-txt-and-source-selection](common/integrations/robots-txt-and-source-selection.md) | Choosing which site to fetch a published dataset from and reading its robots.txt to decide whether your client may crawl it; the file contains a `Disallow: /` and you are deciding whose group it belongs to; setting the crawler's User-Agent and checking that token against the file; robots.txt returned a non-200 status; the origin restricts your token and you are looking for a portal that republishes the same records |
+| [checksum-restored-identifiers](common/integrations/checksum-restored-identifiers.md) | Importing identifiers (tax codes, registration or account numbers) from a spreadsheet or spreadsheet-exported CSV whose numeric cells dropped the leading zero, and the importer pads it back when the check digit passes; deciding whether a checksum-confirmed repair needs a warning or flag; a repaired identifier is about to act as a deciding field (identity match, join key) downstream |
 | [estimate-derived-thresholds](common/integrations/estimate-derived-thresholds.md) | Submitting an action to an external system whose actual outcome can differ from the decision-time estimate (market order fill vs quote) while persisting absolute trigger values derived from that estimate (SL/TP prices, alert thresholds); derived triggers fire immediately or at the wrong level right after the action confirms; choosing where to recompute them from the actual outcome |
 
 ### storage

@@ -60,6 +60,7 @@ no effect at all.
 | The harness backs the file up and restores it with `shutil.copy2` | `copy2` "also attempts to preserve file metadata" via `copystat`, so the restore stamps the *original* mtime back — the same certainty as the row above, reached through the idiomatic backup/restore call. Apply that row's remedy (hash-based `.pyc`, or clear `__pycache__` unconditionally), and switch the restore to `shutil.copyfile` ("no metadata") or `shutil.copy` ("the file's creation and modification times, is not preserved") so the write at least stops re-stamping the old timestamp. A bare `os.utime(path, None)` after the restore is not sufficient on its own: it sets mtime to *now*, which collides again whenever the cached compile happened in the same second |
 | The tree is read-only, or the harness runs under `-B` / `PYTHONDONTWRITEBYTECODE` | Both settings govern writing only — Python "won't try to write `.pyc` files on the import of source modules" — so a `.pyc` left on disk by an earlier run is still validated and reused, and this failure still occurs. Purge `__pycache__` once before the run and keep `-B` set for the rest of it: with nothing cached and nothing written, every iteration recompiles from source |
 | The harness re-imports the mutated file through a fresh `importlib.util.spec_from_file_location` + `module_from_spec` + `exec_module` each iteration | A fresh spec and module object bypass `sys.modules`, not the on-disk cache: the source loader still validates `__pycache__` and reuses it. Purge the cache (or compile with hash-based invalidation) between iterations, or read the file yourself and `exec(compile(text, path, "exec"), ns)`, which consults no cache |
+| The mutate-and-revert was done by a different actor sharing the worktree (a test-quality auditor subagent, a reviewer agent, a teammate's script) and your session only runs the suite afterwards | You never saw the mutation, so nothing in your own loop prompts a purge. Clear `__pycache__` under the tree before the first full-suite run after any delegated verification step returns, and treat a failure that `grep` on the source contradicts as this row before debugging the code |
 | The stale module was already imported in a long-lived process | Clearing `__pycache__` does not help; the module object is in `sys.modules` and only a fresh process (or an explicit reload) picks the change up |
 | An installed package ships `.pyc` files without sources | The unchecked-hash variant is assumed valid whenever it exists; edits to a co-located source are never consulted |
 
@@ -81,4 +82,5 @@ no effect at all.
 - https://docs.python.org/3/library/shutil.html — `copyfile` copies "the contents (no metadata)"; `copy` copies data and permission mode and "Other metadata, like the file's creation and modification times, is not preserved"; `copy2` is "Identical to `copy()` except that `copy2()` also attempts to preserve file metadata" and "uses `copystat()` to copy the file metadata" — so `copy2` is the mtime-restoring member of the family
 - Field reproduction 2026-08-11 (batch mutation harness, one process, byte-length-preserving mutation of a numeric literal, restore via `shutil.copy2`): three consecutive mutants scored GREEN in the batch and the third scored RED when run alone; printing the mutated constant from a fresh subprocess showed all three runs loading the *first* mutant's value. Purging `__pycache__` and calling `os.utime(path, None)` between iterations flipped the third to RED while a no-op control mutation stayed GREEN
 - Field reproduction 2026-08-11 (Python 3.14.6, macOS, fresh-spec import path): `mod.py` holding `VERSION = "3.1.0"` was pinned to a fixed mtime and loaded via `spec_from_file_location` + `module_from_spec` + `exec_module`, which wrote `__pycache__/mod.cpython-314.pyc`. Rewriting the file to the same-length `"3.1.1"` and re-pinning the same mtime, the identical loader printed `3.1.0`; re-running under `python3 -B` with that `.pyc` still present also printed `3.1.0`. A fresh spec and `-B` each leave stale bytecode in play
+- Field reproduction 2026-09-13 (orchestrated worktree, delegated audit): a test-quality-auditor subagent mutated a same-length string literal (`"n/a"` → `"N/A"`) in `src/pipeline.py` to prove a test was load-bearing, then reverted it. The parent session's next full-suite run kept printing `"N/A"` while `grep` on the file showed `"n/a"`; deleting `__pycache__`/`*.pyc` and rerunning fixed it, repeatably across several runs — the parent had no signal that a mutation had happened
 - Field reproduction 2026-08-04 (Python 3.14.6, macOS): with `mod.py` pinned to a fixed mtime via `touch -t` and every revision exactly 18 bytes, compiling `VERSION = "3.1.1"` and then reverting the file to `VERSION = "3.1.0"` left `import mod` reporting `3.1.1` — reverted source, mutant bytecode. Deleting `__pycache__` returned `3.1.0`; a bare `touch mod.py` (mtime bumped, cache left in place) also returned `3.1.0`. The `.pyc` header decoded to `flags=0` (timestamp invalidation) with the source's exact mtime and `size=18`
@@ -9,7 +9,7 @@ sources:
   - https://wiki.postgresql.org/wiki/Don%27t_Do_This
   - https://dev.mysql.com/doc/refman/8.0/en/data-types.html
 last_verified: 2026-07-10
-related: [databases-schema-design-primary-key-choice, databases-schema-design-nullability-and-defaults, databases-schema-design-requirements-to-tables, databases-schema-design-online-schema-changes, databases-selection-relational-jsonb-vs-document-store]
+related: [databases-schema-design-primary-key-choice, databases-schema-design-nullability-and-defaults, databases-schema-design-requirements-to-tables, databases-schema-design-online-schema-changes, databases-selection-relational-jsonb-vs-document-store, backend-common-integrations-checksum-restored-identifiers]
 ---
 
 # Choosing Column Data Types