governance: make stale-source handling operational #22

Activity

1. zackees

   added a parent issue on Aug 25, 2026

   • audit: adversarially review the paralegal research corpus #19
2. zackees commented on Aug 25, 2026

   

   zackees

   on Aug 25, 2026

   MemberAuthor

   More actions

   Sol adversarial review - 2026-08-25

   Verdict

   Leave open; the core deterministic-governance gap is confirmed, but the issue overstates the absence of safeguards and does not establish P1 urgency. This is a repository workflow/schema defect, not a new substantive legal conclusion. I did not need external merits research: the claim is testable against origin/main at 40ee7108f4b46b6b8fb1976f5c8b3cd996bdfb42, the current issue/timeline, and the repository's own contract.

   Reproducible repository evidence

   Existing safeguards that narrow the claim:

   • ANSWERING-GUIDE.md:13-14 already requires rechecking pending cases, current statutes, official-license status, and stale dates before relying; authorities/INDEX.md:124-125 repeats that current-answer gate.
   • PROPOSITION-MAP.md:18-38 makes a recheck column part of every route. Its AGPL network-only route expressly requires current dockets, decisions, state law, and license text.
   • All 102/102 authority cards contain a prose Recheck trigger:. AUTH-0005 (sfc-v-vizio.md:7,62-63) is structured as authority_status: pending and says to verify status before every use.
   • All 20/20 matter memos have a History section. AGENTS.md:51-52 forbids updating last_verified without checking all time-sensitive authorities, and its quality gate at lines 135-140 requires recent checks of current cases and statutes.
   • Research status and legal-review status are already separate in the matter template, README.md, and INDEX.md; that acceptance criterion is preservation of an existing invariant.

   The deterministic gap nevertheless reproduces:

   matter files: 20
   authority files: 102
   last_verified present: 122/122
   last_verified values: 2026-08-25 = 122
   freshness_class: 0/122
   review_due: 0/122
   pending_docket: 0/122
   recheck_required: 0/122
   research freshness validator/test files: 0
   

   templates/MATTER.md:19 and templates/AUTHORITY.md:17 provide only last_verified; the authority template's later-history/recheck entries are prose. INDEX.md and authorities/INDEX.md expose verification dates but no computed current/overdue/unknown state. The only executable validator is tools/license_headers.py, and every test in tests/test_license_headers.py concerns source headers, artifacts, or the separate LEGAL-REVIEW.md release gate. .github/workflows/test.yml runs those tests/lint/type checks but has no research-freshness check. Thus no executable path can decide when a date becomes stale or translate stale/pending/unknown into a retrieval disposition.

   GitHub/history checks found no duplicate or resolution: issue #22 has no linked/cross-referenced resolving PR, its timeline contains only the #19 parent relation, relevant searches find only #22 and its parent, and origin/main has no commit after the issue was filed. The corpus landed in PR #18 with the unresolved cadence question still present at COVERAGE.md:96-105.

   Overbroad or underspecified parts

   • The shared 2026-08-25 date is not itself evidence of bad cadence; all records were imported/verified in the same corpus pass that day. It proves the schema cannot express differential future treatment, not that a record is presently stale.
   • The repository does have pending-docket behavior for AUTH-0005 and ubiquitous event-driven triggers. The defect is that these are inconsistently machine-readable and non-computable, not that no safeguards exist.
   • A standalone boolean pending_docket may duplicate and later contradict authority_status, procedural_posture, and prose later history. Define a normalized lifecycle/freshness model instead of requiring that exact field.
   • Calendar cadence is a governance choice, not a repository fact or legal holding. Event-driven invalidation may be more appropriate for some immutable final decisions, while a pending docket may require every-use checking regardless of review_due.
   • P1 urgency is unsupported on this record: the corpus is same-day fresh, explicitly preliminary, and already requires pre-reliance checks. Priority should be assigned by the corpus owner based on deployment plans and consequences.
   • “Surfaces them in canonical indexes” does not specify whether indexes are generated, checked for committed annotations, or accompanied by a deterministic report. “Force a qualified answer” does not name a retrieval program/API or an observable output.
   • Requiring a search-log update for every refresh is disproportionate for a single official-text check. A history entry with source, result, and checker may suffice; systematic docket/search sweeps should update a reproducible log.

   Additional information needed

   1. Name the cadence owner/approver, the target retrieval consumer, and whether this repository will contain executable retrieval logic or only a validator plus a machine-readable result.
   2. Define as_of semantics (date/time zone), allowed clock skew, grace/waiver behavior, and whether CI evaluates wall-clock age or only schema consistency.
   3. Decide whether matter freshness is explicit or derived from the least-current material authority/search result, and how mixed-volatility records are aggregated.
   4. Separate immutable source-content verification from mutable external status (later history, steward/OSI listing, statutory currency, translation quality). One authority can contain both.
   5. Decide whether offline/unavailable primary sources yield unknown, blocked, or qualified-background-only, and who may override that state.

   Improved acceptance criteria

   Repository defects (testable):

   • Add a versioned freshness schema/config with controlled classes, required fields, allowed overrides, and deterministic precedence among overdue, unknown, event-invalidated, and every-use states. Unknown/missing data must not evaluate as current.
   • Migrate all 122 existing records, or explicitly mark them unknown/recheck-required; preserve research_status, legal_review, and authority posture as distinct dimensions.
   • Add a read-only validator with an injected --as-of YYYY-MM-DD (and stable machine-readable output) that validates dates/classes, detects overdue/unknown/pending-every-use records, and rejects contradictory metadata.
   • Add focused tests for boundary dates, invalid/missing fields, pending-every-use behavior, event invalidation, matter aggregation, and the invariant that legal review does not make research fresh. Test fixtures must prove stale/unknown/pending cannot produce an unqualified-current disposition.
   • Define the retrieval contract's observable result, for example current, recheck-required, qualified-background-only, or blocked; wire the answering guide to that result. Do not claim an AI is “forced” unless an executable consumer actually enforces it.
   • Make index exposure deterministic: either generate freshness columns/reports from records or validate committed columns against computed output. CI must always fail malformed/inconsistent metadata; the owner must separately decide whether passage of time alone fails CI or instead blocks current-answer retrieval.
   • A refresh must record the checked sources, later-history/current-status result, date, and responsible reviewer in record history. Require a search-log update only when the refresh was a systematic search or changes a bounded negative result.

   Policy recommendations requiring owner/counsel decision:

   • Approve class intervals, every-use categories, grace/override rules, and risk-triggered live checking; these numbers should not be invented by an implementation PR.
   • Decide whether high-risk advice always requires an answer-time primary-source/docket check even when metadata is nominally current.
   • Decide whether counsel approves only the policy design or each refresh result. Preserve the existing attorney-review gate and never infer legal approval from freshness.

   Attorney/governance questions

   • Which propositions are too consequential to rely on by age alone, and who can authorize an exception when the primary source is unavailable?
   • Should pending litigation, official-registry status, and bounded negative searches be rechecked every use, or may an approved maximum interval apply?
   • What audit evidence is adequate for a refresh, and must an attorney review changed conclusions versus unchanged source checks?
   • Who owns emergency invalidation when a new decision, amendment, steward change, or corrected translation arrives between scheduled reviews?