jevcheck

Turn recurring code-review judgements into testable, repeatable semantic lint rules.

Traditional linters are great at enforcing what can be proved from syntax and types. General-purpose AI review can notice more, but its findings are often difficult to reproduce, calibrate, or enforce consistently. jevcheck sits between them: deterministic code selects a narrow candidate; Jev answers one bounded yes/no question; deterministic policy decides what happens next.

• Start with advisory (shadow) findings; never block a build on an unqualified rule.
• Inspect exactly what would be sent to a model without an API key.
• Test rules against labelled examples, track drift and recall, and promote them to blocking (owned) checks only when the evidence is sound.
• Run the same rule over source files, staged changes, branch diffs, or CI.
• Gate rules on declared Node, .NET, or Go dependencies using manifest-aware projectWhen.hasDependency (no package-manager execution).

jevcheck uses @mhingston5/jev-cli for provider selection. Node.js 20+ is required.

See why it exists: a queue acknowledgement bug

A worker API returns 202 Accepted to a caller once a job has been published to its queue. A refactor catches a publish failure to log it, but still returns success. A reviewer catches the mistake once; the team wants to prevent similar regressions without asking an open-ended AI reviewer to reinterpret its standards on every PR.

The queue acknowledgement walkthrough includes the actual source, a deliberately broken handler, a corrected fixture, a scoped rule, and commands you can run locally. It is a small, production-inspired illustration, not a claim of a real incident or validated model accuracy.

Try the rule without calling a model

From a checkout of this repository:

npm ci
npm run build
node dist/cli.js inspect examples/queue-ack/src/handler.ts \
  --config examples/queue-ack/jevcheck.config.json \
  --rule reliability/no-success-after-publish-failure \
  --verbose

inspect requires no provider credentials. By default it summarises whether a rule is eligible for semantic checking, deterministically covered, or inapplicable (with a reason), plus candidate counts and a provider-free preflight (upper bound on uncached semantic judge calls and total eligible request characters, not tokens/cost). Add --verbose to show full bounded requests and semantic keys, or use --format json for structured data.inspections[]. Use --preview-skipped to see hypothetical candidates for policy-skipped rules. Unscoped full-repository inspection also reports advisory per-rule scope health (data.scopeHealth[]): stale globs or excluded directories become no-files-in-scope warnings (exit 0), while missing projectWhen dependencies, retired ADRs, verified coverage and legitimate zero candidate matches retain distinct states. Diff-only or explicitly path-limited inspections never call a rule dormant solely because it had no selected files. Inspection does not assert a violation. See inspect policy semantics. To ask a model for a judgement, configure a provider and run the same rule in shadow mode; see the walkthrough.

To install it in your own repository:

npm install --save-dev @mhingston5/jevcheck

Create a repository-local jevcheck.config.json based on a concern your team has actually reviewed:

{
  "include": ["src/**/*.ts"],
  "rules": [
    {
      "id": "reliability/queue-ack",
      "ast": "function_declaration",
      "contains": "queue.publish(",
      "question": "Can this handler return success when queue publication fails without recovery?"
    }
  ]
}

ast as a string means an AST node kind, and contains matches literal text without regex escaping. Rules default to advisory shadow mode. Run npx jevcheck inspect --rule reliability/queue-ack src/handler.ts --verbose to preview candidates and the exact model request without an API key. For AST patterns, regex filtering and evidence controls see the configuration reference.

Rule-authoring guidance explains how to decide if that concern belongs here.

When should I use it?

Situation	Better tool	Why
Forbid an import, detect a syntactic call, enforce formatting or types	ESLint, ast-grep, compiler, tests	Exact checks are cheaper and more reliable
Enforce a recurring bounded behavioural or architectural judgement for a known code candidate	jevcheck	Semantics matter and the candidate/evidence can be scoped
Ask what is wrong with an unfamiliar subsystem or suggest broad improvements	Human/AI code review	The desired judgement is exploratory, not a stable binary invariant
Need interprocedural runtime behaviour that the selected context cannot establish	Tests, tracing, specialised analysis	A bounded model question cannot invent missing evidence

Three candidate use cases worth exploring:

• Behavioural regression — a handler acknowledges accepted work after publication fails (the runnable example).
• Architecture decision — a scoped operation bypasses a service boundary described by an ADR, provided the selected context can prove it. Architecture/ADR guidance.
• Recurring review judgement — a defined failure response exposes internal implementation details, with explicit safe/unsafe examples.

A rule is not justified just because a model can answer it. First check whether deterministic code or a test can prove the invariant. If a narrow focus cannot show the relevant facts, do not build a semantic rule around it.

How adoption works

real review concern -> write invariant -> select bounded code -> inspect (offline)
                   -> shadow findings -> labelled fixtures -> measure
                   -> audit evidence -> owned/CI (only if dependable)

You can stop at shadow and keep a useful advisory check. owned error rules block only when the documented evidence gates are met, and fail closed when that evidence goes stale. They also fail closed when applicable code cannot be evaluated (e.g. an unsupported AST language or chunkChars bound); genuine zero-matches remain successful. Incomplete evaluation is warning-only for shadow and owned warning rules. Run jevcheck rules audit before promoting a rule.

Goal	Command
Inspect a rule without calling Jev	npx jevcheck inspect --rule <id> src/file.ts
Run repository rules	npx jevcheck
Check staged changes	npx jevcheck --staged
Review branch changes	npx jevcheck --changed --base origin/main
Preview ownership/evidence readiness	npx jevcheck rules audit
Produce code-scanning results	npx jevcheck --changed --base origin/main --format sarif

Coverage fingerprints are not CI passes: coverage record only records sources. After human review, use coverage record --attest-reviewed. Shadow rules may then skip advisory judgement, but owned rules also require each coveredBy check to pass via coverage verify --check <id> -- <command> in the same GitHub Actions job/revision. Missing/failed verification cannot authorize an owned skip. See the coverage trust model.

Config-only PRs now get a full-policy scan: --changed, --staged and --changed --base detect changes to the active config file (including custom relative --config paths) and expand selection to every source under the selected revision's current include/exclude policy. No change to config means diff-only performance. Run npx jevcheck inspect --changed --base origin/main --format json first: data.selection.mode reports full-policy-change and selectedFiles reports the scope; this can increase semantic provider requests substantially. Empty scopes are explicit, and deleted/unreadable configs fail with an error. See Git snapshot semantics.

Revision consistency: --staged evaluates the Git index, including staged rules, manifests, ADRs, fixtures and evidence; unstaged edits cannot change the judgement. --changed --base <ref> selects paths changed by <ref>...HEAD and evaluates their committed HEAD versions, not local edits. Both modes require policy artifacts to exist in that revision (ignored/untracked evidence does not count). See Git snapshot semantics. Provider configuration: export TYPESAFE_API_KEY (default provider), or set OPENROUTER_API_KEY plus JEV_PROVIDER=openrouter. Never put credentials in rules. Model-visible code is sent to your selected provider: exclude secrets, private keys and sensitive source. See provider details.

Documentation

• CI adoption recipes — copyable GitHub Actions and Azure Pipelines workflows, offline fixture, provider opt-in, SARIF and coverage trust boundaries.

• Runnable queue acknowledgement walkthrough — code, config, inspect, safe/unsafe fixtures, optional provider run.

• What makes a good semantic rule? — applicability, architecture decisions, deterministic coverage, context sufficiency.

• Rule lifecycle and evidence — fixtures, calibration, drift, recall, robustness, confirmation, baselines and replay.

• CLI and configuration reference — provider options, commands, configuration, exit codes, API and development.

C#, Python, Go, Java, and Rust support language-specific AST selection using ast.language or automatic .cs, .py, .go, .java and .rs detection. See the supported-language reference, multilanguage AST examples, and the C# walkthrough. The C# walkthrough includes safe/unsafe fixtures, native parser installation requirements, and an offline inspect example.

Source discovery is not AST support. With no include override, jevcheck discovers JS/TS, Python, Go, Rust, Java, C#, HTML (.html/.htm), CSS, Ruby (.rb), PHP (.php), Vue (.vue) and Svelte (.svelte) files. All discovered UTF-8 files can be examined with text/chunk semantic selectors. Ruby, PHP, Vue and Svelte have no built-in AST inference or grammar: a rule using ast on them emits incomplete-evaluation, blocking an applicable owned/error rule rather than silently passing. Scope mixed-language AST rules with files/exclude or explicitly configure ast.language only for compatible syntax. See the discovery/AST matrix.

The bundled agent skill guides automated rule authoring without changing the deterministic/model boundary.

MIT licensed.

CLI output for humans and agents

Default stylish output begins with a summary. check groups findings by file/rule, rules audit shows readiness and actionable blockers, and inspect summarises runtime policy, eligible semantic workload, preview-only exclusions and selector diagnostics without printing requests. Use inspect --verbose for exact requests/keys or rules audit --verbose for every evidence check. Reviewed coverage and CI-verified coverage are displayed as distinct states.

Breaking JSON output change: --format json now returns one consistent object with schemaVersion, command, outcome, summary, diagnostics, and data fields for every supported command, including structured errors. Previous raw objects/arrays are now under data; there is no separate JSON version flag. For preflight, read jevcheck inspect --format json → data.preflight; do not infer billed calls from summary.candidates (it includes previews). For example, run jevcheck rules audit --format json to get actionable evidence for agents. Audit outcome: "block" indicates readiness blockers even when the audit command exits 0; an owned-rule failure returns outcome: "error", gate diagnostics and data.evidenceReports[]. See the CLI output contract.

SpecForge integration (experimental, opt-in)

A checked SpecForge graph can be projected into advisory jevcheck rules. Keep the specification authoritative; do not use a model to silently translate every invariant into an enforceable rule. The mapping contains the reviewed guarantee, a bounded question and candidate selection; the projector rejects guarantee drift, missing/unlinked invariants, unsafe provenance (including URL/drive prefixes and whitespace-padded traversal) and attempts to bypass jevcheck's ownership gates.

This repository dogfoods the approach in:

• spec/projection.spec — SpecForge requirements for the projector itself
• spec/jevcheck-map.json — an explicit approved mapping for one invariant
• examples/specforge/jevcheck.config.json — generated config for CI drift detection
• tests/specforge.test.ts — deterministic regressions and offline inspection

With SpecForge installed, from the repository root:

specforge check --strict
specforge export --format graph > /tmp/jevcheck-specforge-graph.json
npm run build
node dist/cli.js specforge project \
  --graph /tmp/jevcheck-specforge-graph.json \
  --mapping spec/jevcheck-map.json \
  --output examples/specforge/jevcheck.config.json --check
node dist/cli.js inspect src/cli.ts \
  --config examples/specforge/jevcheck.config.json \
  --rule reliability/specforge-no-success-after-failure --verbose

Remove --check to regenerate the advisory config after reviewing a changed specification and adjusting the mapping. --check never writes. The projector consumes a full unscoped JSON Graph Protocol export, not brief/context, budgeted or schema-ref scoped exports, and does not invoke SpecForge itself. Always run specforge check --strict first. Do not run commands supplied by the graph or mapping.

The projected rule remains shadow. Offline inspection tests only candidate selection, not model accuracy. Calibrate against safe/unsafe fixtures and measure drift and recall before considering manual promotion with rules audit. Promotion is deliberately outside the projector.