[chris-colinsky]

Wires ten of the twelve observability per-fixture keys left in the unread ledger. The OTel drivers for 004, 005, 009 and 011 asserted their claims with hardcoded values instead of reading the fixture, so an edited fixture value went unnoticed. Each now reads its expected keys and is registered in _DRIVER_EXPECTED_KEYS, so a key it does not read fails the run.

• 004 matches the declared span_tree from every root its exporter saw, so a span outside the invocation's tree fails too. It reads no_edge_spans as a closed world over span names.
• 005 matches span_tree, span_tree_private and span_tree_global, and reads no_llm_provider_span and no_openarmature_spans_on_global. A leak to the global provider is found by instrumentation scope rather than span name.
• 009 runs every case through one driver. It matches span_tree, counts traces against invocation_count and checks each correlation_id invariant by name. A <uuid> placeholder now matches any canonical UUIDv4 (conformance-adapter section 5.10).
• 011 compares the runs and aspects determinism_check names, applies the ignore_attributes patterns and checks its two invariants.
• The Langfuse runner reads 037's resume.from_first_run and builds 039 case 3 from the fixture's subgraph_call and fan_out declarations.
• The expected-key guard runs after the hollow-case check, as in the Langfuse runner, so an invariants-only case is reported as hollow. The harness fidelity test also fails when a registered driver is still listed as unguarded.

The second commit reads a declared children list as the complete set at every level. children: [] asserts a leaf, a span the tree does not declare fails with its parent named, and a node without children leaves them unchecked. The Langfuse observation matcher already did this for a non-empty list and now enforces children: [] too. The observability harness note says nothing about children, so this reading is pending a spec ruling. Every fixture that runs today passes under it.

008's parent_trace and traces and 058's traces stay in the ledger for a follow-up that matches spans per trace.

Testing:

• Nine source mutants in the observer and engine fail the rewired drivers. A control that varies an openarmature.timing.* attribute between runs stays green, since 011 ignores those attributes.
• 30 edited-fixture and harness-list probes fail, plus 3 for 039 case 3.
• For 009, four source mutants fail: a non-UUID id, node spans with a different id, node spans with their own valid UUID, and an id reused across runs. Each also fails with span_tree removed, so the invariant checks catch it on their own.
• An extra child span under every node span fails 21 fixtures, against 5 before. An extra child under each embedding observation fails Langfuse 083 and 140, against none before.
• A stray span for 004's unreachable node fails whether it is emitted as a child or as its own root.

Review follow-ups:

• _is_uuidv4 uses fullmatch(), so a trailing newline no longer passes <uuid> or 009's UUID invariants. The UUID pattern is now lowercase-only (RFC 9562), as the checkpoint runner's own check already is, and the 031 span test goes through the same helper. Newline and uppercase mutants on the generated correlation id pass 009 and the 031 span test before these changes and fail after them.
• Every OTel span-tree check now matches from all the roots the exporter saw, not only the invocation root, so a stray root span fails. Under a mutant that emits a stray root carrying the invocation's attributes, 15 more fixtures fail: 038, 110, 124, 126-129, 131-133, 146, 147, 149, 152 and 153.

[Copilot]

🟡 Changes recommended

UUID validation and root-span matching still allow invalid output to pass conformance checks.

2 open issues

• UUID matcher accepts trailing newlines · New
• Tree matcher ignores undeclared exporter roots · New

What changed in this PR

Strengthens observability conformance checks by replacing hardcoded assertions with fixture-driven validation.

Changes:

• Wires OTel and Langfuse fixture declarations into their drivers.
• Enforces complete declared span and observation child lists.
• Updates expected-key guards and the unread-key ledger.

File	Description
tests/​conformance/​test_observability.py	Adds fixture-driven checks and stricter tree matching.
tests/​conformance/​test_observability_langfuse.py	Reads resume and nested-subgraph declarations.
tests/​conformance/​test_harness_fidelity.py	Rejects guarded drivers listed as unguarded.
tests/​conformance/​harness/​vocabulary.py	Removes newly consumed keys from the unread ledger.

_{Review effort: Balanced}