Add fleet semantic PaCMAP and verified specialty graph maps

[realmarcin]

Adds a common semantic PaCMAP for all 2,877 active ingredient records and regenerates both specialty graph views from verified KG-Microbe vectors. The text map uses the fleet's pinned BGE-large model and links every point to its exact browser record; navigation is enabled only after the complete current input and selected artifact bundle validate.

The common-map validator now rejects symbolic links for the active pointer, generation directory and manifest, and requires the selected generation name to match the canonical manifest identity (CultureBotAI/culturebotai-claw#440). The installed validation/staging runtime is SHA256 9ef7708303150938c8f2c2a81885d9184ed0a7af0a78d9acc88c01224339b5fe, from reviewed canonical candidate 16ce0e452af975cdfe4acc181efb6bad4688f5b7. The existing numerical artifact was built with runtime SHA256 470a3ee46db65b1ad041ead394588f3b75b53b0fb97bb62e1852827a2e7e2dc6; that build provenance remains unchanged. A fresh full export of all 2,877 prepared records passed inspect, check and generation-bound staging with the new runtime; all three staged files exactly match the installed and previously published bundle. 38 existing focused adapter/publication/recipe tests passed after the update. The runtime commit changes only scripts/embedding_pipeline.py, preserves the deployed pin and all generated artifacts, and performs no new model inference, projection or site regeneration.

The graph views now project 2,212 records and explicitly report the 665 without matching vectors. They remove implicit synthetic centroids, preserve matching-method coverage, and bind the parsed source bytes, current corpus, ordered matrix, actual PaCMAP/sfdp reducer and published outputs in checksum receipts. The browser verifies those receipts and presents accurate coverage and matching explanations. Both specialty views remain available alongside the common map. A lightweight publication preflight independently binds both graph receipts to every current mapped/unmapped YAML file, including ignored and rejected records. Pages runs it before any site writes or upload; local build-docs and check-visualizations run it first. A notes-only MediaDive source correction now refuses stale graph publication even when the common text input is unchanged. Ordinary CI never regenerates vectors or layouts.

Validation:

• Full repository suite after Gate graph publication on current source YAML independently of common text freshness #684: 2,483 passed, 6 skipped; package coverage gate and advisory scripts report passed. Flat-export publication coverage passed during the earlier complete integration review.
• Final formatting, Ruff and mypy gates passed; 37 focused common-map adapter/site/recipe tests passed.
• Full common-map input/cache/bundle checks and all 2,877 browser destinations passed. Both complete specialty maps passed source/corpus/output receipt checks, finite-coordinate checks and browser provenance tests; adversarial checksum, cache, publication-binding and quoted-argument controls passed independent review.
• The Gate graph publication on current source YAML independently of common text freshness #684 gate passed against both complete 2,212-point graphs and all 2,951 current YAML files using only the standard library. Its 43 focused tests passed; fresh-process corpus, local-order and workflow-trigger mutation controls each passed green/red/green.
• The complete semantic JSONL remained byte-identical after the typing fixes. Main's data tree remained unchanged at the final release preflight.
• All new canonical files passed governance plan/apply/check against reviewed CLAW commit dbac7ddc8f1351493b90c6dc8850319769fdf620; the unchanged deployed pin still validates all 16 currently governed files.

This is the additive fleet bootstrap in the reviewed release plan. It deliberately retains main's exact c8b8f89ecf29574d7e24f6c65cfd925aefbdd007 pin. After every consumer has the new files and CultureBotAI/culturebotai-claw#432 lands, a separate follow-up will pin its actual main-reachable merge commit, with artifact bytes verified against the reviewed source. This draft does not request a merge before that coordinated review.

Fixes #678. Fixes #679. Fixes #680. Fixes #681. Fixes #682. Fixes #684.
Related: CultureBotAI/culturebotai-claw#426, CultureBotAI/culturebotai-claw#429, CultureBotAI/culturebotai-claw#430, CultureBotAI/culturebotai-claw#434.

[realmarcin]

Adversarial review of head 67032070. Read-only; nothing was pushed to this branch.

Filed

#685 — after this lands, a merged synonym or label edit fails the whole Pages deploy on main. Both new steps in generate-pages.yaml run before the docs/ upload with no continue-on-error, and the workflow triggers on data/ingredients/** but not on PRs. Canary in a clean 3.13 venv with only PyYAML: the committed tree passes both gates; after adding one synonym to Glucose.yaml, check_graph_receipts.py and stage_text_map.py both exit 1. A curation_history-only edit leaves the text map passing, because the adapter reads semantic fields only.

main has had 56 record-touching commits in 30 days. The graph half is survivable, since curators already regenerate that JSON in-PR (23 of 26 recent regenerations). The text-map refresh needs the torch runtime, and no recipe reaches it. The refusal itself is right; the problem is that it happens post-merge and blocks the entire site. Options are in the issue.

Checked and found clean

• Preserve quoted arguments in the proposed text-map-inputs recipe #680 quoting: the recipes pass "$@", which only works with set positional-arguments. That setting is present at justfile:10, so quoted paths survive.
• CI-environment staging: with only PyYAML installed, check_graph_receipts.py passes and stage_text_map.py stages docs/text-map/ byte-identical to the committed files. The heavy imports in embedding_pipeline.py are lazy (lines 325, 399, 424).
• Duplicate points: 2,212 points with 2,212 unique ids in both graph JSONs. The 11 points that share a record identifier fall on 8 identifiers, all already dispositioned in duplicate_identifier_baseline.tsv.
• Browser destinations: all 2,877 text-map #record= links resolve to exactly one source_file in docs/data/ingredients.json.
• Exclude implicit synthetic vectors from graph projections and disclose embedding lookup methods #678: the mean-plus-noise synthetic path is removed, and both receipts report synthetic_records: 0.
• Bind graph cache reuse to source bytes before issuing provenance #679: the basename/size/mtime pickle cache is gone; GraphSource hashes the bytes it actually parses and refuses a source that changes mid-read.
• Update graph interpretation text after removing synthetic centroid embeddings #681: the "synthetic embeddings near cluster center" text is removed from both graph pages, and the explanation is now driven by the receipt.
• Browser receipt checks use textContent throughout, with no HTML injection from receipt fields.
• Gates: lint, format-check, typecheck, check-instruction-refs and check-graph-receipts all pass locally; all 14 CI checks pass.

Merge

The description says this draft should not merge before culturebotai-claw#432. That PR is still an open draft, and #434 (pin after queue squash) is also open, so I have not merged or marked this ready.