Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,16 @@ All notable changes will be documented here. The project intends to follow seman

### Added

- Bounded JPEG marker and length-prefixed segment traversal.
- Standalone, fill-byte, restart-marker, EOI, and multi-scan handling.
- JPEG APP and COM container classification.
- EXIF, standard/extended XMP, ICC, Photoshop/IPTC, JFIF/JFXX, and Adobe presence detection.
- Structured malformed, truncation, trailing-data, and segment-limit diagnostics.
- Normalized JPEG metadata-container entries and complete/partial container status.
- Bounded binary reader, no-copy input normalization, and JPEG/PNG/WebP format detection.
- Binary boundary, sliced-view, format, malformed-input, and JPEG container tests.
- Shared bounded little- and big-endian TIFF/EXIF decoder.
- Iterative IFD0, ExifIFD, GPSIFD, and next-IFD traversal.
- IFD entry/depth limits and repeated-offset cycle protection.
- Inline and TIFF-relative offset value handling for common field types.
- Exact RATIONAL/SRATIONAL, conservative ASCII, and full-range LONG/SLONG decoding.
- Common IFD0, ExifIFD, and GPS tag normalization with deterministic source paths.
- JPEG EXIF child-field inspection through bounded TIFF-only subviews.
- Structured malformed header, table, pointer, value, type, rational, limit, and cycle diagnostics.
- Bounded JPEG marker traversal and EXIF/XMP/ICC/IPTC container detection.
- Binary boundary, JPEG container, TIFF endian, malformed, cycle, and integration tests.

### Foundation

Expand Down
26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@ Current implementation:

- bounded binary input and endian-aware read core;
- JPEG, PNG, and WebP signature detection;
- bounded JPEG marker and segment traversal;
- JPEG entropy-scan skipping without image decoding;
- JPEG EXIF, XMP, extended XMP, ICC, Photoshop/IPTC, and comment container-presence detection.
- bounded JPEG marker, segment, and entropy-scan traversal;
- JPEG EXIF, XMP, ICC, Photoshop/IPTC, and comment container detection;
- shared little- and big-endian TIFF/EXIF decoder;
- iterative IFD0, ExifIFD, GPSIFD, and next-IFD traversal with cycle and depth protection;
- common TIFF, EXIF, and GPS field decoding with exact rational values.

Not implemented: TIFF/EXIF or GPS field decoding, XML/IPTC/ICC payload decoding, PNG/WebP container parsing, metadata cleaning, and verification.
Not implemented: MakerNote or thumbnail decoding, XMP/IPTC/ICC payload parsing, PNG/WebP container parsing, metadata cleaning, and verification.

## Format status

JPEG reports can be `container-inspected` or `container-partial`. PNG and WebP remain `format-only`. See [format support](docs/format-support.md) for the precise matrix.

A detected or traversed container is not necessarily a decodable image. The JPEG parser validates marker and segment boundaries, not quantization, Huffman, frame, scan-header, or entropy semantics.
JPEG reports can be `container-inspected`, `container-partial`, or `metadata-partial`. `metadata-partial` means supported TIFF/EXIF fields were attempted while the wider metadata space remains intentionally incomplete. PNG and WebP remain `format-only`. See [format support](docs/format-support.md).

## Installation

Expand All @@ -34,21 +34,23 @@ import {
} from "secure-metadata";
```

`inspectMetadata` accepts `Uint8Array | ArrayBuffer`, enforces relevant parse limits, and returns a deterministic report. For JPEG it inventories the container and emits one normalized entry per recognized privacy/color metadata container. Entries identify container presence only; payload values are not decoded.
`inspectMetadata` accepts `Uint8Array | ArrayBuffer`, enforces relevant parser limits, and returns deterministic normalized entries. JPEG EXIF reports retain the EXIF container entry and add decoded child entries with exact TIFF tag, type, count, source offset, and path information.

GPS rational components remain exact numerator/denominator pairs; decimal coordinates are not derived. Unknown TIFF tags and MakerNote are represented structurally without dumping or recursively parsing their payloads.

`cleanMetadata` and `verifyMetadata` still throw a typed `NotImplementedError`. Node.js `Buffer` values work structurally as `Uint8Array` but are not part of the public contract.
`cleanMetadata` and `verifyMetadata` still throw a typed `NotImplementedError`.

## Security philosophy

Every byte is untrusted. Binary reads use centrally checked ranges, parser input views retain their original boundaries, traversal is hard bounded, and malformed or tiny inputs are ordinary data. Unknown APP segments remain unknown and should be preserved by future cleaning. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md).
Every byte is untrusted. All offsets are interpreted within bounded views, traversal is iterative and limited, repeated IFD offsets are rejected, and malformed entries recover without unchecked access. Unknown structures remain unknown and should be preserved by future cleaning. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md).

## Non-goals

The library does not perform image decoding or encoding, visual redaction, pixel-content privacy analysis, steganography detection, or malware scanning. Absence of recognized metadata containers is never proof that an image contains no private information.
The library does not perform image decoding or encoding, visual redaction, pixel-content privacy analysis, steganography detection, or malware scanning. Absence of recognized or decoded metadata is never proof that an image contains no private information.

## Secure Tools ecosystem

This is an independent open-source library in the broader Secure Tools ecosystem. It has its own package, lifecycle, and repository; future Secure Tools integration will use a pinned browser artifact rather than coupling application code to this repository.
This is an independent open-source library in the broader Secure Tools ecosystem. Future integration will use a pinned browser artifact rather than coupling application code to this repository.

## License

Expand Down
50 changes: 20 additions & 30 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,34 @@
# Architecture

`secure-metadata` is organized as a side-effect-free binary library. Its current and planned flow is:
`secure-metadata` is a side-effect-free binary library with format-specific containers and shared metadata decoders.

```text
Input bytes implemented
Safe binary view / bounded reads implemented
Format detection implemented
JPEG container parser implemented for JPEG
Metadata container classification implemented for JPEG
Metadata payload decoder planned
Normalization / field classification planned
Policy engine and cleaner planned
Output re-inspection / verification planned
JPEG APP1 Exif\0\0 ─┐
PNG eXIf (future) ├──→ bounded TIFF/EXIF core
WebP EXIF (future) ─┘ ↓
normalized entries
```

## Binary core
The TIFF decoder receives only the TIFF byte view after the six-byte EXIF identifier. It has no JPEG marker or absolute file-offset knowledge. Every TIFF offset is relative to byte zero of that view. Integration relocates decoded source offsets and diagnostics only after parsing.

Input normalization returns the caller's exact `Uint8Array` view or creates a no-copy view over an `ArrayBuffer`. `ByteReader` validates offsets and lengths as non-negative safe integers and checks remaining capacity with subtraction before every read. It provides bounded unsigned 8-, 16-, and 32-bit reads, subarray views, and allocation-free signature matching.
## TIFF core

## JPEG container layer
The decoder explicitly validates `II` or `MM`, magic value 42, and the first IFD offset. `TiffReader` centralizes endian-aware unsigned 16-/32-bit and signed 32-bit access over the bounded binary core.

The iterative JPEG parser validates SOI and walks markers using the binary core. A central marker model distinguishes SOI, EOI, TEM, RST0–RST7, APP0–APP15, COM, SOS, common image-structure markers, and length-prefixed unknown markers. Repeated `FF` fill bytes are collapsed to one marker; declared lengths include their two-byte length field and must fit fully before offsets advance.
Each IFD table is validated as a complete `2 + count × 12 + 4` byte range before entries are visited. Field sizes support BYTE, ASCII, SHORT, LONG, RATIONAL, UNDEFINED, SLONG, and SRATIONAL. Values of four bytes or fewer use the entry's inline bytes in TIFF byte order; larger values use a bounded TIFF-relative offset.

After SOS, the parser scans rather than decodes entropy data. `FF 00` remains stuffed data, restart markers are recorded without terminating the scan, and the next real marker resumes normal traversal. This supports multiple scans. Every marker, including SOI, EOI, SOS, and restarts, counts toward `maxSegments`.
Traversal uses a FIFO work queue. Root IFD0 has depth 1; ExifIFD, GPSIFD, and next-IFD work is queued deterministically in that order. A visited-offset set rejects cycles and repeated references. `maxIfdEntries` bounds each table, `maxIfdDepth` bounds linked depth, and `maxMetadataEntries` caps total processed entries and queued IFD work.

APP signatures are checked within segment payload boundaries without retaining payload copies. EXIF, standard/extended XMP, ICC, Photoshop/IPTC, JFIF/JFXX, Adobe, and unknown classifications remain container-level observations.
## Value and entry behavior

## Inspection status
Supported known values are decoded without converting exact rational pairs to floating point. ASCII stops at the first NUL within its declared count and maps non-ASCII bytes conservatively. Zero rational denominators remain represented and produce diagnostics.

Unknown tags retain namespace, tag number, TIFF type, count, entry offset, and source path without exposing arbitrary payload bytes. Duplicate tags remain separate ordered entries. MakerNote is recognized but opaque and is never interpreted as nested standard TIFF.

- `format-only`: a signature was detected; no container parser ran. Currently PNG, WebP, unknown, and short arbitrary inputs.
- `container-inspected`: JPEG traversal reached EOI safely.
- `container-partial`: JPEG identity is known, but traversal stopped on a structural error, truncation, or limit.
- `metadata-inspected`: reserved for future payload decoders.
## Inspection status

An empty entry list means no supported metadata container was recognized during the completed portion of traversal. It does not prove metadata or private information is absent.
- `format-only`: signature detection only; currently PNG, WebP, and unknown input.
- `container-inspected`: JPEG reached EOI and no EXIF decode was attempted.
- `container-partial`: JPEG traversal stopped on corruption, truncation, or a limit.
- `metadata-partial`: JPEG container traversal completed and common TIFF/EXIF decoding was attempted; XMP/IPTC/ICC and unknown fields remain incomplete.
- `metadata-inspected`: reserved for future broader decoders.
45 changes: 21 additions & 24 deletions docs/format-support.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,30 @@
# Format Support

| Capability | JPEG | PNG | WebP |
| --------------------------- | --------------------------------- | --------- | --------- |
| Signature detection | Supported | Supported | Supported |
| Bounded container traversal | Supported | Not yet | Not yet |
| APP/COM classification | Supported | N/A | N/A |
| EXIF container detection | Supported | Not yet | Not yet |
| XMP container detection | Supported, including extended XMP | Not yet | Not yet |
| ICC container detection | Supported | Not yet | Not yet |
| IPTC/Photoshop detection | Supported | Not yet | Not yet |
| Metadata field decoding | Not yet | Not yet | Not yet |
| Cleaning | Not yet | Not yet | Not yet |
| Capability | JPEG | PNG | WebP |
| --------------------------- | ---------------------- | -------------- | -------------- |
| Signature detection | Supported | Supported | Supported |
| Bounded container traversal | Supported | Not yet | Not yet |
| EXIF container detection | Supported | Not yet | Not yet |
| TIFF header and IFD0 | Supported through JPEG | Not integrated | Not integrated |
| ExifIFD and GPSIFD | Supported through JPEG | Not integrated | Not integrated |
| Common EXIF/GPS fields | Supported subset | Not integrated | Not integrated |
| MakerNote decoding | Not supported | Not supported | Not supported |
| XMP payload decoding | Not yet | Not yet | Not yet |
| IPTC/ICC payload decoding | Not yet | Not yet | Not yet |
| Cleaning and verification | Not yet | Not yet | Not yet |

## JPEG
## TIFF/EXIF subset

JPEG detection requires `FF D8`. Container inspection validates marker boundaries and two-byte big-endian declared lengths, recognizes standalone markers and fill bytes, stops at EOI, and reports trailing bytes. SOS headers are traversed, while entropy-coded bytes are skipped without decoding; `FF 00`, RST0–RST7, and multiple scans are handled structurally.
Both `II` and `MM` byte orders are supported. Traversal covers IFD0, ExifIFDPointer, GPSInfoIFDPointer, and next-IFD links with table, entry, depth, offset, and cycle checks.

Payload signatures identify:
Decoded IFD0 tags: ImageDescription, Make, Model, Orientation, Software, DateTime, Artist, and Copyright.

- APP0 `JFIF\0` and `JFXX\0` as technical container data;
- APP1 `Exif\0\0` as EXIF;
- APP1 standard and extended Adobe XMP identifiers as XMP;
- APP2 `ICC_PROFILE\0` as ICC;
- APP13 `Photoshop 3.0\0` as Photoshop/IPTC;
- APP14 `Adobe` as rendering/container data;
- COM as comment metadata.
Decoded ExifIFD tags: ExposureTime, FNumber, PhotographicSensitivity, ExifVersion, DateTimeOriginal, DateTimeDigitized, FocalLength, PixelXDimension, PixelYDimension, and FocalLengthIn35mmFilm. MakerNote is named and retained as opaque structure.

Unknown APP payloads remain unknown. No TIFF, EXIF, XMP XML, ICC, IPTC, thumbnail, frame, Huffman, quantization, or entropy payload is decoded.
Decoded GPS tags: GPSVersionID, GPSLatitudeRef, GPSLatitude, GPSLongitudeRef, GPSLongitude, GPSAltitudeRef, GPSAltitude, GPSTimeStamp, and GPSDateStamp. Coordinates remain exact raw rational components plus reference fields; decimal coordinates are not derived.

## PNG and WebP
Unknown tags remain structurally represented without speculative meaning or large binary values.

PNG requires its complete eight-byte signature. WebP requires `RIFF` at offset 0 and `WEBP` at offset 8. Their chunk structures, sizes, CRCs, metadata, and image payloads are not yet parsed.
## Remaining container support

JPEG marker and scan traversal remains supported. XMP, ICC, and Photoshop/IPTC signatures are container-detected only. PNG requires its complete signature and WebP requires `RIFF....WEBP`; their chunks and metadata are not parsed yet.
Loading
Loading