Skip to content

Add additive ProtocolEvidence JSON export for traceability-matrix-dhf - #1

Merged
LSaiko merged 1 commit into
masterfrom
evidence-export
Sep 23, 2026
Merged

LSaiko merged 1 commit into
masterfrom
evidence-export

Conversation

@LSaiko

@LSaiko LSaiko commented Sep 23, 2026

Copy link
Copy Markdown
Owner

What changed

  • samd_toolkit/evidence.py adds export_evidence(session, rmf=None, requirement_ids=None, path=None). It turns an executed IQ/OQ/PQ ValidationSession into a Pydantic v2 ProtocolEvidence model. It can also include an ISO 14971 RiskManagementFile. If path is given, it writes the result as JSON.
  • samd_toolkit/protocol-evidence.schema.json is the committed contract (JSON Schema 2020-12, schema_version: "1.0", source: "samd-val-kit"). A test fails if it drifts from the model.
  • setup.py: pydantic is an optional [evidence] extra, so the core toolkit still has no dependencies. pydantic and jsonschema are added to [dev]. The schema file is included via package_data.
  • CI: the coverage floor goes from 70% to 80% (current total is 85%, and evidence.py is at 100%).
  • README: adds a coverage badge (≥80%, the enforced floor) and an "Evidence Export" section.

Why

This gives traceability-matrix-dhf a machine-readable record of executed IQ/OQ/PQ protocols, supporting 21 CFR 820.30(f)/(g) and IEC 62304 §5.7.

Reviewer notes

  • This is a sibling schema, not a ValidationEvidence payload. The shared contract in traceability-matrix-dhf and ml-samd-validator requires baseline, snapshot, drift and fairness. This toolkit doesn't produce those, and filling them in would create false DHF records. The export reuses the shared envelope (generated_at, evidence_id, requirement_ids) and the iec_62304 / iso_14971 block shapes. The IQ/OQ/PQ record goes in a new protocol block.
  • traceability-matrix-dhf can't ingest this yet. ValidationEvidenceRecord.from_validation_evidence only accepts ml-samd-validator payloads, so it needs a samd-val-kit source adapter. The README says so.
  • Existing behaviour is unchanged. No existing module was modified. I compared the HTML report, IQ/OQ/PQ items, RMF summary and risks, IEC 62304 gap analysis, cyber controls and SPDX SBOM for Class I, II and III devices. With IDs and timestamps pinned, the output was byte-identical to master. test_export_does_not_alter_existing_output guards against mutation.
  • There's no --emit-evidence CLI flag. The CLI only runs demos and has no session to export.
  • Python 3.9–3.11 have only run in CI; I checked locally on 3.12 and 3.14.

Known pre-existing issues (not addressed here)

  • The README architecture tree and usage examples reference files and APIs that don't exist (checklist.py, pdf_reporter.py, export_pdf, generate --device-class, rmf.generate_report).
  • HTMLReporter labels its timestamp "UTC" but uses naive local datetime.now(). I left it alone to keep existing output byte-identical.

🤖 Generated with Claude Code

- samd_toolkit/evidence.py: export_evidence() serialises an executed IQ/OQ/PQ
  session (+ optional ISO 14971 RMF) to a Pydantic v2 ProtocolEvidence model.
  Shares envelope and iec_62304/iso_14971 block shapes with ml-samd-validator's
  ValidationEvidence; carries no drift/fairness data (not produced here).
- Committed JSON Schema contract, kept in sync by test.
- pydantic is an optional [evidence] extra; core stays dependency-free.
- CI coverage floor 70% -> 80% (current 85%); README coverage badge + section.

Existing IQ/OQ/PQ, HTML, RMF, IEC 62304, cyber and SPDX output verified
byte-identical against the previous commit for Class I/II/III devices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LSaiko
LSaiko merged commit f1fb7f4 into master Sep 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant