Manipulation-evidence reports for vocal tracks — scored per cue, banded, and auditable by a human. It never answers "is this AI?". That field exists in the schema and is permanently null, because at any realistic operating point a verdict is a defamatory-shaped claim about a named artist that can be wrong about one. What ships is a report: which markers fired, what a reviewer should check, and how certain the pipeline is not.
Elevated evidence: 2 of 14 conversion-consistent markers present.
- Accompaniment fingerprint matches track #4471 (bit error rate 0.02); that track's
singer is #12, this vocal identifies as #38.
- Vocal stem cuts off at 16000 Hz against 21000 Hz for the accompaniment (5000 Hz gap).
- no C2PA manifest; no synthesis-watermark detector registered, so watermarking was not
tested — not evidence of authenticity
Not assessed:
- cm_score: no countermeasure checkpoint (VOCALFORENSICS_CM_CHECKPOINT unset)
Severity: medium.
Flag precision at this operating point: not yet measured.
Not a determination. Evidence only, for human review.
pip install -e . # core: schema, cues, scoring, renderer, CLI
pip install -e ".[base]" # + demucs, chromaprint, Praat, speaker embeddings
pip install -e ".[cm]" # + Tier-2 training and inference (torch)
pip install -e ".[dev]" # + pytestRun these from the repository root (where pyproject.toml is), not from inside the
vocalforensics/ package directory.
Core needs only numpy and scipy. WAV decoding, contour analysis, fingerprint comparison, scoring and reporting all work without the extras.
PCM WAV — the pipeline's own stem format — is decoded by the standard library; FLAC and
OGG go through soundfile, which bundles libsndfile. ffmpeg is optional: when present it
handles AAC, M4A, Opus and WMA (formats libsndfile cannot decode) and decodes at the
file's native sample rate so it cannot shift the spectral ceiling that Tier-1 cues read.
One external binary is needed for real fingerprint generation (FR-8.7): Chromaprint's
fpcalc, from apt install libchromaprint-tools, brew install chromaprint, or the
Chromaprint releases on Windows. Without it,
fingerprint comparison and everything in the demo still work — only ingest needs it.
vocalforensics ingest track.flac --stem-dir stems # probe, separate, fingerprint, measure
vocalforensics analyze 42 # evidence report for one track
vocalforensics analyze --json # whole catalog, §4.8.3 contract
vocalforensics enroll 1 --singer "Adele" # declare a track bona fide for a singer
vocalforensics identify # embed tracks that have no embedding yet
vocalforensics reattribute # re-score identity from stored vectors
vocalforensics review 42 false_positive --reviewer ops:alex --note "heavily produced"
vocalforensics precision # §9.3 flag precision over reviews
vocalforensics measure 42 # re-run metrics on existing stems
vocalforensics fingerprint 42 # backfill a missing fingerprint
vocalforensics calibrate --percentile 1 # measure Tier-1 thresholds
vocalforensics rescore --run r2 # appends version-tagged rows
vocalforensics triage # suggest a review order (optional)python scripts/demo_pipeline.pyGenerates a small catalog of real WAV files — a singer with six bona-fide recordings, an
original, and one track built like an AI cover (a different voice, band-limited to 8 kHz,
over the original's instrumental) — then runs probe, measure, fingerprint, embed and
score, and prints the reviewer's report, the review workflow, calibration and the
deletion path. The bona-fide tracks come back no_signal; the suspect comes back
strong on six markers. Separation and the speaker embedder are the only stand-ins, and
both are named in the output.
Optional features are configured by environment variable — copy .env.example to .env.
Every module also self-checks in isolation:
python -m vocalforensics.db # schema invariants
python -m vocalforensics.contour # vibrato, F0-snap, formant tracking
python -m vocalforensics.cm.evaluate # EER vs precision@1%FPR
pytest # 114 tests, including every demo abovedecode → validate → fingerprint → [cache]
→ authenticate:tier0 (container, watermark, C2PA — pre-separation)
→ separate (htdemucs_ft, version-pinned) → [cache stems]
→ metrics (CPU) ─┐
→ embed (GPU) ───┤
→ authenticate:tier1-3 ─┴→ persist → report
| Tier | Cues | Cost | Notes |
|---|---|---|---|
| 0 | watermark / C2PA, container provenance | free | Pre-separation. Near-certain when present, recall low and falling. |
| 3 | accompaniment fingerprint match, inter-stem bandwidth, segment variance, margin profile, attribution split | cheap | Needs the catalog. The differentiated part — build and trust this first. |
| 1 | jitter/shimmer floor, vibrato regularity, F0 quantization, breaths, formant smoothness, HF flatness | cheap | Derived from existing metrics. Every cue has a production confound. |
| 2 | learned countermeasure | expensive | Off until a checkpoint exists. One voter, never the decision. |
Cue weights combine noisy-or, so agreement compounds and no lone cue can carry a track
past weak. Bands are absolute-anchored and catalog-relative, taking whichever is more
severe: a uniformly synthetic catalog cannot normalise itself into "fine", and an outlier
in a clean catalog still escalates.
Demucs output has its own signature — smeared transients, phase incoherence, high-frequency loss — and those are the same feature families a synthesis detector keys on. A detector trained on clean bona-fide audio flags every stem this pipeline produces. So the separator build is pinned, recorded on every row, and Tier 2 refuses any checkpoint calibrated against a different one. Bumping demucs invalidates calibration; the pipeline says so rather than letting you discover it after a dependency upgrade.
Emit a determination. Let a model phrase an authenticity claim. Report a single cue above
weak/low (except a Tier-0 watermark, which is a declaration, not an inference). Read
a missing watermark — or a detector that never ran — as authenticity. Score a cue whose
parent metric is invalid. Block ingest. Score voice samples for cloning suitability. Clone
enrolled singers to manufacture training positives.
Those are enforced in code, not documented and hoped for: determination carries a
CHECK (determination IS NULL) constraint, NotAssessed is a distinct type from a cue
result so it cannot decay into a zero, cue-to-template coverage is a startup error, and
override records are insert-only at the database level.
vocalforensics triage asks Gemini which pending flags deserve a reviewer's next hour.
It is a work-queue tool and nothing else. The model never hears audio, never sees the band
or the evidence score, and its output cannot reach scoring, the report, or
determination. The request carries cue keys and numbers under opaque per-request labels
with paths, track ids and singer ids stripped, because §11 treats authenticity records as
sensitive records about people. Set GEMINI_API_KEY to enable it; GEMINI_MODEL
overrides the default. Without a key, without a network, or on an unusable reply, the
queue keeps its default order.
An authenticity record characterises a named singer's recorded output, and in the
conversion case encodes that someone's voice identity was used without them.
db.purge_singer() is the §11 deletion path: it removes the singer, their enrolment,
embeddings, authenticity records, overrides and triage rows, and cascades to the tracks,
stems, fingerprints and metrics derived from their voice — metrics.voice_json is jitter,
shimmer and formant data, which is biometric information about that person. It is
all-or-nothing, and it returns the on-disk audio paths rather than deleting files, so an
irreversible filesystem operation stays an explicit decision by whoever received the
request.
Tiers 0, 1 and 3, the full output contract, threshold calibration, and the review
workflow are implemented and tested. Tier 2 is a complete harness with no checkpoint — the
cue reports not_assessed until one exists.
Honest gaps, largest first:
- No real-audio validation. Tests run on synthetic fixtures and real WAV files generated in-test. Nothing has been run against a licensed catalog, so no precision number exists.
- Nothing is calibrated.
calibrateworks but needs >=30 enrolled real tracks per metric. Until then Tier-1 thresholds are literature guesses, and each report says which of the two fired it. Cue weights are unmeasured regardless. - FR-8.11 has no prosody representation. Column, cue, template and test all exist; nothing computes a prosody-dominant embedding.
- No watermark detector is registered. C2PA manifest detection is real (presence
only). Synthesis-watermark schemes need their own licensed detectors —
watermark.register_detector(scheme, fn)is the seam. - Stage C has no data. SingFake / CtrSVDD / ASVspoof-line licensing is unresolved.
Who reviews flags. What a confirmed detection triggers — delist, annotate, notify, nothing. Whether the accompaniment fingerprint index reaches beyond your own catalog. Who owns quarterly re-scoring. A detector nobody re-runs is worse than none, because it carries stale authority.
MIT — see LICENSE.
The licence covers the code. It does not settle what you may lawfully do with the output: authenticity records describe identifiable people, several jurisdictions have emerging law on voice likeness, and §11 of the spec treats publishing a band without its measured error rate as prohibited. Those obligations travel with the deployment, not with the licence.