Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,16 @@ All notable changes will be documented here. The project intends to follow seman

### Changed

- JPEG parser records internal fill-aware rewrite ranges while retaining existing public source offsets.
- Parse-limit validation is shared by inspection and cleaning.
- Normalized cleaning defaults and deprecated ICC alias precedence through one immutable semantic policy across JPEG, WebP, and PNG.
- Verification now omits not-applicable format concepts and fails closed when metadata-entry limits truncate reporting.
- Aligned metadata-entry, diagnostic, status, ICC-classification, and source-order invariants across supported formats.

### Fixed

- Enforced `maxMetadataEntries` within a single TIFF IFD and across normalized public metadata reports.
- Enforced `maxDiagnostics` while container/TIFF diagnostics are emitted and in typed incomplete-cleaner errors.
- Kept WebP structural failure state independent from capped diagnostic storage.
- Removed unreachable foundation-era `NotImplementedError` and unused diagnostic codes.

### Foundation

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ import {

GPS rational components remain exact numerator/denominator pairs; decimal coordinates are not derived. Unknown TIFF tags and MakerNote are represented structurally without dumping or recursively parsing their payloads.

`cleanMetadata` supports JPEG, WebP, and PNG. JPEG removes EXIF, XMP, Photoshop/IPTC, and comments. WebP removes EXIF and XMP while repairing RIFF size and applicable VP8X flags. PNG removes `eXIf`, XMP `iTXt`, ordinary `tEXt`/`zTXt`/`iTXt`, and `tIME`; it preserves `iCCP`, rendering/color chunks, image and APNG chunks, unknown chunks, critical chunks, and trailing bytes. All formats preserve ICC by default.
`DEFAULT_CLEANING_POLICY` is the authoritative semantic default: remove recognized EXIF, XMP, IPTC, comments, ordinary text, and standalone timestamps; preserve ICC, unknown, rendering, and image data. Each format maps only applicable concepts to physical containers. The deprecated `preserveColorProfiles` alias is used only when explicit `preserveIcc` is absent.

`verifyMetadata` supports `absent`, `present`, or `ignore` expectations. PNG defaults check EXIF, XMP, ordinary text, and timestamps, while ICC is ignored unless explicitly requested. Single-file verification observes supported container presence or absence and cannot prove provenance or pixel privacy.
`verifyMetadata` supports `absent`, `present`, or `ignore` expectations. Concepts not implemented for a format produce no check rather than implying an exhaustive search. Verification fails closed if metadata reporting reaches its configured entry limit. Single-file verification observes supported container presence or absence and cannot prove provenance or pixel privacy.

## Security philosophy

Expand Down
49 changes: 21 additions & 28 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,33 @@
# Architecture

`secure-metadata` is a side-effect-free binary library with format-specific containers and a shared metadata decoder.
`secure-metadata` is a side-effect-free binary library with shared semantic policy and format-specific container logic.

```text
JPEG APP1 Exif\0\0 ─┐
PNG eXIf ┴──→ bounded TIFF/EXIF core → normalized entries
WebP EXIF → normalized container entry only
input
→ format detection
→ JPEG / WebP / PNG bounded container parser
→ optional shared TIFF decoder (JPEG EXIF, PNG eXIf)
→ normalized metadata report
→ normalized semantic cleaning policy
→ format-specific conservative reconstruction
→ one output re-inspection
→ observational verification
```

JPEG passes the TIFF decoder the view after its six-byte EXIF identifier. PNG passes the exact `eXIf` data view directly. In both cases TIFF offset zero is the beginning of that bounded view; integrations relocate source offsets and diagnostics only after parsing.
Shared semantic policy does not imply a shared binary writer. JPEG copies retained marker/scan ranges. WebP copies retained chunks and repairs RIFF size plus applicable VP8X metadata bits. PNG copies complete retained chunks and original CRC bytes. Each cleaner performs one input container parse, one final output allocation, and one output inspection.

## TIFF core
## Shared TIFF core

The decoder validates byte order, magic 42, complete IFD tables, field sizes, offset values, and linked traversal. A FIFO queue plus visited-offset set provides deterministic IFD0, ExifIFD, GPSIFD, and next-IFD traversal. `maxIfdEntries`, `maxIfdDepth`, `maxMetadataEntries`, and `maxStringBytes` bound work. Known values retain exact rationals; unknown tags remain structural, and MakerNote stays opaque.
JPEG passes the view after `Exif\0\0`; PNG passes exact `eXIf` data. TIFF byte zero, tag definitions, rational representation, diagnostics, and internal paths such as `IFD0/ExifIFD/DateTimeOriginal` are shared. Outer source containers and relocated absolute offsets remain format-specific. WebP EXIF remains container-only.

## Inspection status

- `format-only`: unknown input where only format detection applies.
- `container-inspected`: complete JPEG, WebP, or PNG traversal without TIFF decoding.
- `container-partial`: traversal stopped on structural invalidity or a limit.
- `metadata-partial`: complete JPEG or PNG traversal where common TIFF/EXIF decoding was attempted while broader metadata remains intentionally opaque.
- `metadata-inspected`: reserved for future broader decoders.

## Cleaning flows
Traversal validates byte order, magic, complete IFD tables, field sizes, offset values, cycles, and progress. `maxIfdEntries`, `maxIfdDepth`, `maxMetadataEntries`, `maxStringBytes`, and `maxDiagnostics` bound work and reporting.

JPEG and WebP use their format-specific parsers and reconstruction rules. JPEG copies retained marker/scan ranges into one output. WebP copies retained chunks, repairs RIFF size, and aligns retained VP8X metadata bits.
## Inspection status

```text
PNG bytes
→ bounded PNG chunk parser and metadata classification
→ shared TIFF decoder for eXIf inspection
→ direct keep/remove policy
→ checked retained physical ranges
→ one output allocation and ordered byte copies
→ inspectMetadata(output)
→ structured verification checks
```
- `format-only`: only format detection is available; currently unknown input.
- `container-inspected`: the supported JPEG, WebP, or PNG container structure was fully traversed without shared TIFF decoding.
- `container-partial`: container traversal stopped because structure was unsafe or a structural limit was reached.
- `metadata-partial`: JPEG or PNG container traversal completed and the supported TIFF/EXIF subset was attempted, while broader metadata semantics remain intentionally incomplete.
- `metadata-inspected`: reserved for future exhaustive metadata decoders.

The PNG cleaner parses the source once for boundaries, never routes decisions through semantic entries, and does not decode TIFF before removing a bounded `eXIf`. It copies the signature, retained complete chunks, and bytes after IEND. Retained length/type/data/CRC bytes and relative order are unchanged. IDAT, APNG, compressed text, and ICC payloads stay opaque.
A report includes `metadataTruncated: true` when its entry budget is reached; a diagnostic is also emitted when the diagnostic budget permits. Verification fails closed rather than deriving absence from a truncated report.
72 changes: 41 additions & 31 deletions docs/cleaning-policy.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,53 @@
# Cleaning Policy

Privacy Clean removes complete recognized metadata containers and never decodes or re-encodes image payloads.
Privacy Clean applies one normalized semantic policy, then maps it directly to each format's physical containers. Binary reconstruction remains format-specific.

## JPEG
## Semantic defaults

| JPEG structure | Default action |
| ------------------------------------------------- | -------------- |
| EXIF APP1; standard/extended XMP APP1 | Remove |
| Photoshop/IPTC APP13; COM | Remove |
| ICC APP2; JFIF/JFXX; Adobe APP14 | Preserve |
| Unknown APP; structural/scan data; data after EOI | Preserve |
| Policy field | Default | Meaning |
| -------------------- | ------- | ----------------------------------------------- |
| `removeExif` | `true` | Remove recognized EXIF containers |
| `removeXmp` | `true` | Remove recognized XMP containers |
| `removeIptc` | `true` | Remove recognized IPTC containers |
| `removeComments` | `true` | Remove recognized comment containers |
| `removeTextMetadata` | `true` | Remove recognized ordinary text metadata |
| `removeTimestamps` | `true` | Remove recognized standalone timestamp metadata |
| `preserveIcc` | `true` | Preserve recognized ICC containers |

## WebP
Unknown, rendering, and image data are preserved. `preserveColorProfiles` remains a deprecated alias for `preserveIcc`: explicit `preserveIcc` wins, otherwise the alias is used, otherwise the default applies. The exported `DEFAULT_CLEANING_POLICY` is authoritative; legacy format-named defaults reference the same frozen object.

| WebP chunk or data | Default action |
| ---------------------------------------- | ---------------------------------- |
| EXIF; XMP | Remove |
| ICCP; VP8/VP8L; ALPH; ANIM/ANMF; unknown | Preserve |
| VP8X | Preserve; align ICC/EXIF/XMP flags |
| Data after declared RIFF boundary | Preserve |
## JPEG mapping

WebP cleaning removes targeted physical chunks including padding, repairs RIFF size, and patches only the three VP8X metadata bits. No VP8X is synthesized.
| Semantic field | Physical mapping |
| ---------------- | ------------------------------ |
| `removeExif` | EXIF APP1 |
| `removeXmp` | Standard and extended XMP APP1 |
| `removeIptc` | Photoshop/IPTC APP13 |
| `removeComments` | COM |
| `preserveIcc` | ICC APP2 |

## PNG
JFIF/JFXX, Adobe APP14, unknown APP segments, structural markers, scan data, and data after EOI are preserved.

| PNG chunk or data | Default action |
| -------------------------------------- | -------------- |
| `eXIf` | Remove |
| XMP `iTXt` | Remove |
| Ordinary `tEXt`, `zTXt`, and `iTXt` | Remove |
| `tIME` | Remove |
| `iCCP` | Preserve |
| `gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs` | Preserve |
| `IDAT`; APNG structure | Preserve |
| Unknown ancillary; critical chunks | Preserve |
| Data after `IEND` | Preserve |
## WebP mapping

Compressed text and ICC payloads are removed or preserved as whole chunks without decompression. Retained physical chunks—including their original CRC bytes—and trailing data remain byte-identical and ordered.
| Semantic field | Physical mapping |
| -------------- | ---------------- |
| `removeExif` | EXIF chunk |
| `removeXmp` | XMP chunk |
| `preserveIcc` | ICCP chunk |

The shared fields `removeExif`, `removeXmp`, and `preserveIcc` apply across supported formats. PNG also uses `removeTextMetadata` and `removeTimestamps`; JPEG-only `removeIptc` and `removeComments` have no PNG effect. `preserveColorProfiles` remains a deprecated alias for `preserveIcc`. Unknown removal is intentionally unavailable.
Other semantic fields are not applicable. VP8/VP8L, ALPH, ANIM/ANMF, unknown chunks, and trailing data are preserved. RIFF size and only necessary VP8X ICC/EXIF/XMP bits are repaired.

`cleanMetadata` always returns a new `Uint8Array`, change evidence, diagnostics, and a re-inspection report. Unsafe container boundaries reject cleaning before output. Unknown formats return a typed unsupported-format error.
## PNG mapping

| Semantic field | Physical mapping |
| -------------------- | ----------------------------------- |
| `removeExif` | `eXIf` |
| `removeXmp` | Exact XMP `iTXt` |
| `removeTextMetadata` | Ordinary `tEXt`, `zTXt`, and `iTXt` |
| `removeTimestamps` | `tIME` |
| `preserveIcc` | `iCCP` |

Rendering/color chunks, IDAT, APNG structure, unknown ancillary and critical chunks, retained CRCs, and data after IEND are preserved. Compressed text and ICC payloads are never decompressed.

One removed physical container produces one source-ordered change record. `cleanMetadata` always returns a distinct output view, change evidence, bounded diagnostics, and one re-inspection report. Valid outer boundaries permit whole-container removal even when inner metadata is malformed; unsafe container boundaries produce a typed incomplete-format error without output.
2 changes: 1 addition & 1 deletion docs/format-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ The shared TIFF subset covers common IFD0, ExifIFD, GPSIFD, and next-IFD entries

JPEG removes EXIF, XMP, Photoshop/IPTC, and comments. WebP removes EXIF and XMP, repairs RIFF size, and aligns retained VP8X flags. PNG removes `eXIf`, XMP and ordinary text chunks, and `tIME`; it preserves ICC, rendering/color, IDAT, APNG, unknown, critical, CRC, and trailing bytes by default.

Verification reports observable supported metadata-container presence or absence. It does not decode XMP/IPTC/ICC or compressed PNG text, prove byte provenance, or prove complete removal of personal information.
Verification reports observable supported metadata-container presence or absence. Expectations for concepts not implemented by a format are omitted as not applicable; they do not generate synthetic passing checks. If `maxMetadataEntries` truncates reporting, the report carries `metadataTruncated: true` and verification returns `valid: false` with no checks. It does not decode XMP/IPTC/ICC or compressed PNG text, prove byte provenance, or prove complete removal of personal information.
44 changes: 19 additions & 25 deletions docs/security-model.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,28 @@
# Security Model

Binary metadata parsing processes attacker-controlled structures, sizes, offsets, encodings, and nesting. Malformed files, parser crashes, excessive allocation or traversal, and incorrect offset arithmetic are security concerns.
All binary input is attacker-controlled. The cross-format guarantees are:

## Invariants
1. bounded reads and checked range arithmetic;
2. advancing or terminating parser loops;
3. hard segment, chunk, IFD, entry, depth, string, diagnostic, and input limits;
4. no recursion over untrusted container structures;
5. no pixel decoding or image re-encoding;
6. no compressed metadata inflation;
7. safe whole-container removal when outer boundaries are trustworthy;
8. unknown metadata preservation by default;
9. ICC and rendering/color preservation by default;
10. deterministic, source-ordered output;
11. exactly one output re-inspection per cleaner;
12. observational verification without provenance or privacy claims.

1. All input is untrusted and all reads use bounded primitives.
2. Parsers use checked range arithmetic, finite iteration limits, and no unbounded recursion.
3. Core functions make no network requests and access no filesystem or DOM APIs.
4. Image pixels and compressed image/metadata payloads are never decoded.
5. Unknown structures are not assigned speculative meaning or removed by default.
6. ICC, rendering/color, and image structures are preserved by default.
7. Cleaner output is re-inspected before return.
8. Metadata absence never proves an image has no private pixels, unsupported metadata, steganography, malware, or provenance concerns.
`maxMetadataEntries` bounds processed TIFF work and normalized public entries. `maxDiagnostics` is enforced while JPEG, WebP, PNG, and TIFF diagnostics are emitted and again when reports are combined. `maxStringBytes` bounds TIFF values and PNG keyword extraction. `maxDecompressedBytes` is reserved and currently unused because no decompression exists.

## Bounded binary and TIFF properties
## Format-specific reconstruction

Offsets and lengths must be non-negative safe integers. Ranges use subtraction-based capacity checks before access. TIFF decoders receive bounded TIFF-only views: after the JPEG EXIF identifier or at PNG `eXIf` data byte zero. IFD table size, `count × typeSize`, inline/offset value locations, linked depth, entry count, metadata count, string length, and cycles are checked. Unsupported values produce diagnostics; MakerNote, thumbnails, and pixels remain opaque.
JPEG requires trustworthy traversal through EOI and preserves retained marker, fill, scan, restart, and trailing bytes. WebP requires a complete RIFF/chunk boundary, preserves padding and trailing data, repairs RIFF size, and changes only applicable VP8X metadata flags. PNG requires a complete IEND boundary, preserves trailing data, and copies every retained length/type/data/CRC byte unchanged. Each reconstruction uses one final output allocation and honors the caller's exact `Uint8Array` view.

## JPEG and WebP properties
Malformed inner EXIF/TIFF or textual payloads do not block removal of their bounded JPEG segment, WebP chunk, or PNG chunk. Unsafe outer boundaries produce `IncompleteJpegError`, `IncompleteWebPError`, or `IncompletePngError` without partial output.

JPEG traversal validates marker and scan progression through EOI before cleaning; malformed structure produces `IncompleteJpegError`. Retained scan, marker, and trailing bytes are copied in one allocation. WebP validates the RIFF boundary, complete chunk headers/payload/padding, VP8X constraints, and chunk limits; malformed structure produces `IncompleteWebPError`. Its cleaner copies retained chunks, repairs RIFF size, and updates only applicable VP8X metadata bits.
Verification reports only supported `present` or `absent` observations. Not-applicable format concepts produce no check. Truncated metadata reporting is recorded independently of diagnostic output, produces no checks, and fails verification. The library does not establish authenticity, provenance, absence of proprietary metadata, visible-person privacy, steganography safety, malware safety, or complete metadata absence.

## PNG parsing and cleaning properties

The PNG parser requires the complete signature and validates every big-endian length, four-letter type, data range, and CRC field before advancing. `maxChunks` bounds traversal. IEND stops logical parsing; trailing bytes are warned about and preserved rather than interpreted. Missing IEND, truncated fields, impossible ranges, and limit failures produce `IncompletePngError` before output.

CRC-32 is checked over each chunk type and data. A mismatch produces a warning but does not obscure otherwise valid whole-chunk boundaries; the cleaner may remove targeted chunks but never repairs or mutates retained CRCs.

IDAT and APNG payloads are never decoded or rewritten. `zTXt`, compressed `iTXt`, and `iCCP` are never inflated, avoiding metadata decompression-bomb exposure in this sprint. A malformed but bounded text or `eXIf` payload can still be removed as a whole chunk. Unknown ancillary, critical, ICC, and rendering/color chunks are preserved by default. Reconstruction parses once, calculates checked retained ranges, allocates one output, and copies complete chunks and trailing bytes in order. Exact caller subviews are honored and inputs are never mutated.

## Environment and dependencies

Core code is local-only and side-effect-free, with zero runtime dependencies. Verification observes supported container presence only; it cannot prove provenance, absence of unknown metadata, or complete removal of personal information.
Core production code has zero runtime dependencies and no network, analytics, telemetry, filesystem, DOM, Node `Buffer`, or required platform-global behavior.
Loading
Loading