Add a leakage risk assessment model, replacing the AUC risk bands

[TheColdIce]

Closes #443. Jira: LDG-30.

What this adds

leakpro/risk/, which turns MIA audit results plus a declared use-case profile into a risk assessment, and replaces the webapp's AUC-threshold risk bands with it.

The problem it solves: LeakPro measured attack success but had no defensible way to state risk. Summary.tsx scored it from hardcoded AUC bands (>= 0.75 HIGH, >= 0.60 MEDIUM) with no provenance, and AUC is an average-case measure that hides the worst-case per-record leakage that actually determines exposure — a model can sit at AUC 0.58 and still have training records that are perfectly identifiable. The core library had no risk concept at all, so CLI and notebook users got nothing.

Design: a decomposition, not a score

Risk is reported in three blocks that are never collapsed:

• Measured (reproducible from the audit): TPR at a chosen operating point α, advantage, lift, and the count of members flagged at that point.
• Declared (by the data controller, echoed verbatim): attacker prior π and the four harm factors.
• Combined (every factor traceable to one of the above): precision, expected exposed subjects, Risk = LEF × LM, and an advisory band.

Structure follows Sion et al. (IWPE 2019): LM = DTS × NR × DST × NDS. That paper supplies the factor structure but deliberately no numeric values — §III-G states every input is an analyst estimate. So the four factors are user-supplied numbers defaulting to a neutral 1.0, and there is no combined risk band at all, only an advisory vulnerability_band over the measured lift, explicitly marked heuristic. The suggested sensitivity scale is cited to CNIL PIA-3, which §VI-A points at.

Precision follows Jayaraman et al. (PoPETs 2021, Thm 4.2): PPV = TPR / (TPR + γ·α), γ = (1−π)/π. This is not decoration. Our audits run on a balanced split, so every TPR and AUC LeakPro reports implicitly assumes π = 0.5. At TPR(1% FPR) = 0.10 that is 91% precision, but about 1% at π = 0.001. Both are always reported so the flattering figure cannot be quoted by accident.

Two assumptions the published model forces into the open, emitted in assumptions on every assessment: LEF = V only because retention period × threat event frequency is assumed to be 1 (a single attempt on a retained model), and the LM product treats its factors as independent, which Sion §VI-B itself flags as a simplification.

API

assess_risk is a separate call, not part of run_audit, so one expensive audit can be re-assessed cheaply under different α and π — exactly the parameters users vary. run_audit keeps its signature and return type.

results = leakpro.run_audit()
assessment = assess_risk(results, profile, num_train=leakpro.handler.target_model_metadata.num_train)
assessment.save(f"{leakpro.report_dir}/risk_assessment.json")

A profile can be declared in audit.yaml under an optional top-level use_case: block. Declaring it triggers nothing; absent it, behaviour is unchanged.

Four guards, each a way a naive version reports a confidently wrong number

1. Unresolvable operating point. MIAResult._get_result_fixed_fpr uses max(..., default=0.0), so TPR at an FPR the audit set cannot express comes back as 0.0. A 200-non-member audit asked for α = 0.001 would read as zero risk. The layer refuses below 1 / n_non_members, or (non-strict, used by the webapp) withholds derived figures with a warning.
2. Inverted ROC. attack_p currently reports an inverted ROC for every target (measured AUC 0.214 where correctly oriented is 0.786). Results with AUC < 0.5 are rejected by name rather than averaged in. The underlying bug is out of scope here.
3. Attack selection. MIAResult.get_strongest maximises ROC AUC; risk needs the attack that wins at the user's operating point. These disagree routinely.
4. Units. Two TPR conventions coexist: the live MIAResult tabulates fractions (TPR@1%FPR), the legacy metrics/attack_result.py helper returns percentages (TPR@1.0%FPR). The layer accepts fractions and rejects a percentage-valued table rather than letting a 100× error reach a risk figure.

Webapp

All scoring moved to Python behind POST /jobs/{job_id}/risk; the frontend posts a profile and renders the response, so no thresholds live in TypeScript. The results view leads with numbers and puts explanations behind info buttons, with a three-box schematic of the model. Warnings deliberately stay visible as a caveat chip — a caveat like "this operating point was not measurable" must not require a click to discover.

Two bugs found and fixed while building it: risk state was lost when switching result tabs (the tab bar unmounts the tab components), and in cross-job compare mode only the first job's models were ever assessed, with model-name keys that collide across jobs.

Scope

MIA only, behind a family-agnostic VulnerabilityMeasurement schema so MINV, GIA and synthetic data can implement it later without reworking the risk layer. assessment.py imports nothing from leakpro.attacks.

Not doing the FAIR Bayesian network: it needs published priors we do not have, and it buys a defensible combine for inputs that are the controller's own judgement either way. Sion's PERT + Monte Carlo treatment of uncertainty is the natural v2 and would let us report a distribution rather than a point.

Testing

85 tests in leakpro/tests/risk/, all passing, and the full suite is 362 passed / 0 failed.

• 79 unit tests: PPV against hand-computed values at three priors, the unresolvable-α guard, inverted-result rejection, TPR-over-AUC selection, percent-vs-fraction rejection, no monetary figure without a declared cost, byte-identical output for identical input.
• 6 end-to-end tests that run a real LiRA audit through run_audit and then assess it, asserting the JSON is self-contained and the PDF gains a Risk section.
• ruff clean over leakpro/; frontend tsc and vite build pass.

License compliance

• No new dependencies, Python or npm (pyproject.toml and package.json unchanged).
• Apache 2.0 boilerplate header on all six new source files.
• No third-party code copied, so no NOTICE update needed.

Review notes

• The UI has not been reviewed visually. Geometry of the SVG schematic is verified by a coordinate check (no overlaps, nothing clipped or out of frame), but whether it reads well on screen needs a human look.
• examples/mia/cifar/ gains a commented use_case: block and two notebook cells as documentation; no example behaviour changes.
• The advisory band thresholds are the one unsourced judgement left in the layer. They are versioned, overridable, and printed alongside any band they produce, but if you would rather the UI showed no band at all, that is a one-line change.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB