Skip to content

Bind graph cache reuse to source bytes before issuing provenance #905

Description

@realmarcin

An adversarial review of the embedding rollout reproduced stale graph-vector reuse when compressed source bytes change while basename, byte length and integer mtime remain unchanged. These are currently the graph pickle cache identity fields; they do not establish source-content identity.

The reproduction uses a tiny valid TSV.gz with a header and CHEBI:1. First load: vector [1,0]. Replace the source row with [0,1], preserving byte length and mtime. The next ordinary load returns cached [1,0]. The same behavior was reproduced in CultureMech, MediaIngredientMech and CommunityMech in fresh temporary fixtures; no production source or cache was modified.

This matters for the new provenance work: hashing today's source alongside a stale cache would falsely attribute old vectors to new source bytes. Verified generation must parse and bind the actual selected source, or reuse a cache whose source digest, parser/filter policy and vector bytes are verified together. Existing unverified pickle caches must not be promoted merely by attaching a new receipt.

Acceptance: a regression changing source bytes while preserving size/mtime returns the new vectors (or refuses the old cache); a failed source parse/cache write preserves the previous valid state; freshly generated map receipts bind the actual parsed source and vector bytes. This is a concrete cache-identity subissue of the ongoing graph-provenance rollout coordinated in CultureBotAI/culturebotai-claw#426.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions