Skip to content

test(verify): reproduce malformed TRACE Claim shape escapes - #596

Draft
altrudev wants to merge 3 commits into
agentrust-io:mainfrom
altrudev:fix/verify-malformed-claims-592
Draft

test(verify): reproduce malformed TRACE Claim shape escapes#596
altrudev wants to merge 3 commits into
agentrust-io:mainfrom
altrudev:fix/verify-malformed-claims-592

Conversation

@altrudev

Copy link
Copy Markdown

What

Executable reproducer for #592.

This draft adds a focused malformed-shape matrix for verify_trace_claim() and intentionally asserts only the behavior already uncontested in the issue:

malformed externally supplied claim structure returns a VerificationResult and does not leak a host-language exception.

The vectors cover malformed trace, cnf, jwk, jwk.x, gateway, audit_chain, and attestation_evidence intermediates.

Why test-first

There are two effects in the current path:

  1. some malformed nested shapes escape as TypeError / AttributeError after schema validation has already failed; and
  2. non-crashing malformed records can have the earlier CLAIM_MALFORMED result overwritten by signature/key-binding failures.

The first invariant is clear. The second is a failure-precedence decision that I do not want to encode implicitly before maintainer review. This draft therefore makes the exception-safety defect executable while leaving the precise precedence rule open.

Baseline

Pinned to 2c0a601805bfb31fddc49db32ae122bae8e2251e.

Expected current result

The matrix is expected to fail on the malformed shapes that currently escape. It is a reproducer, not a claim that the implementation is already fixed.

Once the precedence rule is confirmed, the implementation should remain narrow: either make schema validity a prerequisite for structure-dependent checks, or make those checks independently object-safe without allowing later results to launder the established schema failure.

No schema or wire-format change is proposed.

AI-assistance disclosure: ChatGPT assisted with source triage, adversarial matrix design, and drafting. altrudev reviewed the bounded claim and remains responsible for the contribution.

Signed-off-by: altrudev <266135212+altrudev@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

🟡 Contributor Check: MEDIUM

Check Result
Profile MEDIUM
Credential LOW
Overall MEDIUM

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:MEDIUM Contributor check flagged MEDIUM risk label Aug 30, 2026
@imran-siddique

imran-siddique commented Aug 31, 2026

Copy link
Copy Markdown
Member

Batch response for this cluster is here: agentrust-io/agent-manifest#357 (comment)

Short version: the finding class is real and welcome. Your CI had never run, held under first-time-contributor gating, until I released 36 runs across your PRs an hour ago, and five of your eight are now red. Please fix those, sequence trace-spec#258 against #252 which touch the same two files, and tell me the order you want them reviewed in.

Copy link
Copy Markdown
Author

I reconciled the released CI failure. The red step was ruff import formatting in the new test file, before the reproducer could execute; that is fixed at 4e2cce1.

Because this PR is explicitly a draft test-first reproducer, I also changed the current-head behavior at 99ba8a3 so only the known host-language escape (AttributeError / TypeError) is reported with pytest.xfail. If verify_trace_claim() returns normally, the original VerificationResult and schema assertions run unchanged. This keeps a known-current defect from making the whole draft CI red without weakening the desired invariant or silently choosing the still-open failure-precedence rule.

I am keeping #596 out of the review queue until the implementation/precedence decision is ready. Fresh CI is currently waiting on workflow approval; no green claim yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:MEDIUM Contributor check flagged MEDIUM risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants