Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ All notable changes will be documented here. The project intends to follow seman

### Added

- Bounded PNG chunk parsing with chunk-count, IEND, trailing-data, type, range, CRC-field, and compact CRC-32 validation.
- PNG text, exact XMP `iTXt`, `eXIf`, ICC, timestamp, rendering/color, APNG, and unknown ancillary classification.
- Shared TIFF/EXIF field decoding for exact bounded PNG `eXIf` data views.
- Deterministic PNG Privacy Clean and verification with single-allocation reconstruction and byte-identical retained chunks, CRCs, IDAT/APNG data, and trailing bytes.

- Bounded WebP RIFF/chunk parsing with declared-size, padding, VP8X, and chunk-count validation.
- WebP EXIF, XMP, and ICCP container inspection with image/alpha/animation distinction.
- Deterministic WebP Privacy Clean with RIFF-size and VP8X metadata-flag repair.
Expand Down
31 changes: 13 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,18 @@
Current implementation:

- bounded binary input and endian-aware read core;
- JPEG, PNG, and WebP signature detection;
- bounded JPEG marker, segment, and entropy-scan traversal;
- JPEG EXIF, XMP, ICC, Photoshop/IPTC, and comment container detection;
- shared little- and big-endian TIFF/EXIF decoder;
- iterative IFD0, ExifIFD, GPSIFD, and next-IFD traversal with cycle and depth protection;
- common TIFF, EXIF, and GPS field decoding with exact rational values;
- deterministic whole-segment JPEG Privacy Clean with byte-preserving reconstruction;
- structured JPEG verification for observable container presence or absence;
- bounded WebP RIFF/chunk inspection with EXIF, XMP, and ICCP container detection;
- deterministic WebP Privacy Clean with RIFF-size and VP8X metadata-flag repair;
- structured WebP verification for EXIF, XMP, and ICC presence.

Not implemented: MakerNote or thumbnail decoding, XMP/IPTC/ICC payload parsing, WebP EXIF field decoding, and PNG container parsing, cleaning, or verification.
- JPEG inspection, common TIFF/EXIF decoding, whole-segment Privacy Clean, and verification;
- WebP RIFF/chunk inspection, EXIF/XMP/ICC container detection, Privacy Clean, and verification;
- PNG chunk inspection with text, XMP, EXIF, ICC, timestamp, rendering, and APNG classification;
- shared TIFF/EXIF decoding for JPEG EXIF and PNG `eXIf` payloads;
- deterministic PNG Privacy Clean and verification with retained chunks and CRC bytes preserved exactly;
- iterative IFD0, ExifIFD, GPSIFD, and next-IFD traversal with cycle and depth protection.

Not implemented: MakerNote or thumbnail decoding, XMP/IPTC/ICC payload parsing, WebP EXIF field decoding, compressed PNG text or ICC decompression, and image/pixel decoding.

## Format status

JPEG reports can be `container-inspected`, `container-partial`, or `metadata-partial`. `metadata-partial` means supported TIFF/EXIF fields were attempted while the wider metadata space remains intentionally incomplete. WebP reports are `container-inspected` or `container-partial` and expose metadata containers only. PNG remains `format-only`. See [format support](docs/format-support.md).
JPEG supports bounded inspection, common TIFF/EXIF field decoding, cleaning, and verification. WebP supports bounded RIFF/chunk inspection and container-level cleaning and verification. PNG supports bounded chunk inspection, direct shared-TIFF decoding of `eXIf`, container-level cleaning, and verification. Compressed `zTXt`, compressed `iTXt`, and `iCCP` payloads remain opaque. See [format support](docs/format-support.md).

## Installation

Expand All @@ -39,17 +34,17 @@ import {
} from "secure-metadata";
```

`inspectMetadata` accepts `Uint8Array | ArrayBuffer`, enforces relevant parser limits, and returns deterministic normalized entries. JPEG EXIF reports retain the EXIF container entry and add decoded child entries with exact TIFF tag, type, count, source offset, and path information.
`inspectMetadata` accepts `Uint8Array | ArrayBuffer`, enforces relevant parser limits, and returns deterministic normalized entries. JPEG and PNG EXIF reports retain the EXIF container entry and add decoded child entries with exact TIFF tag, type, count, source offset, and path information.

GPS rational components remain exact numerator/denominator pairs; decimal coordinates are not derived. Unknown TIFF tags and MakerNote are represented structurally without dumping or recursively parsing their payloads.

`cleanMetadata` supports JPEG and WebP. JPEG Privacy Clean removes EXIF, standard/extended XMP, Photoshop/IPTC, and COM segments while preserving ICC, JFIF/JFXX, Adobe APP14, unknown APP segments, structural data, scan bytes, and trailing bytes. WebP Privacy Clean removes EXIF and XMP chunks while preserving ICCP, VP8/VP8L, VP8X, ALPH, ANIM/ANMF, unknown chunks, and trailing bytes; it repairs RIFF size and VP8X metadata flags.
`cleanMetadata` supports JPEG, WebP, and PNG. JPEG removes EXIF, XMP, Photoshop/IPTC, and comments. WebP removes EXIF and XMP while repairing RIFF size and applicable VP8X flags. PNG removes `eXIf`, XMP `iTXt`, ordinary `tEXt`/`zTXt`/`iTXt`, and `tIME`; it preserves `iCCP`, rendering/color chunks, image and APNG chunks, unknown chunks, critical chunks, and trailing bytes. All formats preserve ICC by default.

`verifyMetadata` supports `absent`, `present`, or `ignore` expectations. JPEG defaults check EXIF, XMP, IPTC, and comments; WebP defaults check EXIF and XMP. Single-file verification observes presence or absence and cannot prove that bytes came from an original file.
`verifyMetadata` supports `absent`, `present`, or `ignore` expectations. PNG defaults check EXIF, XMP, ordinary text, and timestamps, while ICC is ignored unless explicitly requested. Single-file verification observes supported container presence or absence and cannot prove provenance or pixel privacy.

## Security philosophy

Every byte is untrusted. All offsets are interpreted within bounded views, traversal is iterative and limited, repeated IFD offsets are rejected, and malformed entries recover without unchecked access. Unknown JPEG APP segments and WebP chunks remain unknown and are preserved by cleaning. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md).
Every byte is untrusted. All offsets are interpreted within bounded views, traversal is iterative and limited, and malformed structures fail without unchecked access. PNG image data and compressed metadata are never inflated. Unknown JPEG APP segments, WebP chunks, and PNG ancillary chunks are preserved by default. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md).

## Non-goals

Expand Down
57 changes: 17 additions & 40 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,63 +1,40 @@
# Architecture

`secure-metadata` is a side-effect-free binary library with format-specific containers and shared metadata decoders.
`secure-metadata` is a side-effect-free binary library with format-specific containers and a shared metadata decoder.

```text
JPEG APP1 Exif\0\0 ─┐
PNG eXIf (future) ┴──→ bounded TIFF/EXIF core → normalized entries
WebP EXIF → normalized container entry only
PNG eXIf ┴──→ bounded TIFF/EXIF core → normalized entries
WebP EXIF → normalized container entry only
```

JPEG integration passes the TIFF decoder only the byte view after the six-byte EXIF identifier. The decoder has no JPEG marker or absolute file-offset knowledge. Every TIFF offset is relative to byte zero of that view. Integration relocates decoded source offsets and diagnostics only after parsing.
JPEG passes the TIFF decoder the view after its six-byte EXIF identifier. PNG passes the exact `eXIf` data view directly. In both cases TIFF offset zero is the beginning of that bounded view; integrations relocate source offsets and diagnostics only after parsing.

## TIFF core

The decoder explicitly validates `II` or `MM`, magic value 42, and the first IFD offset. `TiffReader` centralizes endian-aware unsigned 16-/32-bit and signed 32-bit access over the bounded binary core.

Each IFD table is validated as a complete `2 + count × 12 + 4` byte range before entries are visited. Field sizes support BYTE, ASCII, SHORT, LONG, RATIONAL, UNDEFINED, SLONG, and SRATIONAL. Values of four bytes or fewer use the entry's inline bytes in TIFF byte order; larger values use a bounded TIFF-relative offset.

Traversal uses a FIFO work queue. Root IFD0 has depth 1; ExifIFD, GPSIFD, and next-IFD work is queued deterministically in that order. A visited-offset set rejects cycles and repeated references. `maxIfdEntries` bounds each table, `maxIfdDepth` bounds linked depth, and `maxMetadataEntries` caps total processed entries and queued IFD work.

## Value and entry behavior

Supported known values are decoded without converting exact rational pairs to floating point. ASCII stops at the first NUL within its declared count and maps non-ASCII bytes conservatively. Zero rational denominators remain represented and produce diagnostics.

Unknown tags retain namespace, tag number, TIFF type, count, entry offset, and source path without exposing arbitrary payload bytes. Duplicate tags remain separate ordered entries. MakerNote is recognized but opaque and is never interpreted as nested standard TIFF.
The decoder validates byte order, magic 42, complete IFD tables, field sizes, offset values, and linked traversal. A FIFO queue plus visited-offset set provides deterministic IFD0, ExifIFD, GPSIFD, and next-IFD traversal. `maxIfdEntries`, `maxIfdDepth`, `maxMetadataEntries`, and `maxStringBytes` bound work. Known values retain exact rationals; unknown tags remain structural, and MakerNote stays opaque.

## Inspection status

- `format-only`: signature detection only; currently PNG and unknown input.
- `container-inspected`: JPEG or WebP container traversal completed without deep metadata decoding.
- `container-partial`: JPEG or WebP traversal stopped on corruption, truncation, structural invalidity, or a limit.
- `metadata-partial`: JPEG container traversal completed and common TIFF/EXIF decoding was attempted; XMP/IPTC/ICC and unknown fields remain incomplete.
- `format-only`: unknown input where only format detection applies.
- `container-inspected`: complete JPEG, WebP, or PNG traversal without TIFF decoding.
- `container-partial`: traversal stopped on structural invalidity or a limit.
- `metadata-partial`: complete JPEG or PNG traversal where common TIFF/EXIF decoding was attempted while broader metadata remains intentionally opaque.
- `metadata-inspected`: reserved for future broader decoders.

## JPEG clean and verify flow
## Cleaning flows

JPEG and WebP use their format-specific parsers and reconstruction rules. JPEG copies retained marker/scan ranges into one output. WebP copies retained chunks, repairs RIFF size, and aligns retained VP8X metadata bits.

```text
input JPEG
→ bounded JPEG parser and existing APP classification
PNG bytes
→ bounded PNG chunk parser and metadata classification
→ shared TIFF decoder for eXIf inspection
→ direct keep/remove policy
→ checked retained ranges
→ checked retained physical ranges
→ one output allocation and ordered byte copies
→ inspectMetadata(output)
→ structured verification checks
```

The parser remains the structural source of truth. Internal rewrite ranges include marker fill bytes associated with a removed marker while public source offsets retain their existing meaning. Cleaning does not invoke TIFF decoding on the source: a structurally bounded EXIF APP1 can be removed even if its TIFF body is malformed. The post-write inspection and verifier use the normal inspection layer.

## WebP clean and verify flow

```text
WebP bytes
→ bounded RIFF/WebP parser and FourCC classification
→ direct EXIF/XMP/ICC policy
→ retained chunks
→ minimal VP8X metadata-bit patch
→ RIFF size patch
→ one output allocation and ordered chunk copies
→ inspectMetadata(output)
→ structured verification checks
```

Chunk payloads remain bounded views and image, alpha, and animation bytes are opaque. The cleaner does not synthesize VP8X; a valid retained VP8X has only its ICC, EXIF, and XMP bits aligned with actual retained chunks. Bytes outside the declared RIFF container are copied as uninterpreted trailing data.
The PNG cleaner parses the source once for boundaries, never routes decisions through semantic entries, and does not decode TIFF before removing a bounded `eXIf`. It copies the signature, retained complete chunks, and bytes after IEND. Retained length/type/data/CRC bytes and relative order are unchanged. IDAT, APNG, compressed text, and ICC payloads stay opaque.
59 changes: 31 additions & 28 deletions docs/cleaning-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,40 @@ Privacy Clean removes complete recognized metadata containers and never decodes

## JPEG

| JPEG structure | Default action |
| -------------------------------------- | -------------- |
| EXIF APP1 | Remove |
| Standard/extended XMP APP1 | Remove |
| Photoshop/IPTC APP13 | Remove |
| COM | Remove |
| ICC APP2 | Preserve |
| JFIF/JFXX APP0 | Preserve |
| Adobe APP14 | Preserve |
| Unknown APP | Preserve |
| Structural markers and image/scan data | Preserve |
| Data after EOI | Preserve |

JPEG v0.1 removes the entire EXIF APP1, including malformed TIFF bodies whose JPEG segment boundary is valid. Selective GPS/tag rewriting and TIFF reserialization are deferred.
| JPEG structure | Default action |
| ------------------------------------------------- | -------------- |
| EXIF APP1; standard/extended XMP APP1 | Remove |
| Photoshop/IPTC APP13; COM | Remove |
| ICC APP2; JFIF/JFXX; Adobe APP14 | Preserve |
| Unknown APP; structural/scan data; data after EOI | Preserve |

## WebP

| WebP chunk or data | Default action |
| --------------------------------- | ---------------------------------- |
| EXIF | Remove |
| XMP | Remove |
| ICCP | Preserve |
| VP8 / VP8L | Preserve |
| VP8X | Preserve; align ICC/EXIF/XMP flags |
| ALPH | Preserve |
| ANIM / ANMF | Preserve |
| Unknown chunks | Preserve |
| Data after declared RIFF boundary | Preserve |
| WebP chunk or data | Default action |
| ---------------------------------------- | ---------------------------------- |
| EXIF; XMP | Remove |
| ICCP; VP8/VP8L; ALPH; ANIM/ANMF; unknown | Preserve |
| VP8X | Preserve; align ICC/EXIF/XMP flags |
| Data after declared RIFF boundary | Preserve |

WebP cleaning removes targeted physical chunks including padding, repairs RIFF size, and patches only the three VP8X metadata bits. No VP8X is synthesized.

## PNG

| PNG chunk or data | Default action |
| -------------------------------------- | -------------- |
| `eXIf` | Remove |
| XMP `iTXt` | Remove |
| Ordinary `tEXt`, `zTXt`, and `iTXt` | Remove |
| `tIME` | Remove |
| `iCCP` | Preserve |
| `gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs` | Preserve |
| `IDAT`; APNG structure | Preserve |
| Unknown ancillary; critical chunks | Preserve |
| Data after `IEND` | Preserve |

WebP cleaning removes every targeted physical chunk including odd-byte padding, repairs the RIFF size, and patches only the three VP8X metadata feature bits. Alpha, animation, reserved, and other VP8X bits remain unchanged. No VP8X is synthesized. Structurally bounded malformed metadata payloads remain removable.
Compressed text and ICC payloads are removed or preserved as whole chunks without decompression. Retained physical chunks—including their original CRC bytes—and trailing data remain byte-identical and ordered.

The shared policy fields `removeExif`, `removeXmp`, and `preserveIcc` apply to both formats. JPEG-only `removeIptc` and `removeComments` have no WebP effect. `preserveColorProfiles` remains a deprecated alias for `preserveIcc`. Unknown removal is intentionally unavailable.
The shared fields `removeExif`, `removeXmp`, and `preserveIcc` apply across supported formats. PNG also uses `removeTextMetadata` and `removeTimestamps`; JPEG-only `removeIptc` and `removeComments` have no PNG effect. `preserveColorProfiles` remains a deprecated alias for `preserveIcc`. Unknown removal is intentionally unavailable.

`cleanMetadata` returns a new `Uint8Array`, container-level change evidence, diagnostics, and an inspection report of its output. Structurally incomplete input is rejected before output allocation. PNG and unknown formats return a typed unsupported-format error.
`cleanMetadata` always returns a new `Uint8Array`, change evidence, diagnostics, and a re-inspection report. Unsafe container boundaries reject cleaning before output. Unknown formats return a typed unsupported-format error.
Loading
Loading