Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,17 @@ All notable changes will be documented here. The project intends to follow seman

## Unreleased

- Repository foundation.
- TypeScript library scaffold.
### Added

- Bounded unsigned byte and endian-aware integer reads.
- Checked binary range validation with typed library errors.
- No-copy normalization for `Uint8Array` and `ArrayBuffer` inputs.
- Signature-based JPEG, PNG, and WebP format detection.
- Explicit format-only inspection reports with input-size enforcement.
- Binary boundary, sliced-view, format, and malformed-input tests.

### Foundation

- Repository and TypeScript library scaffold.
- Public API skeleton.
- Security and architecture documentation.
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,21 @@

`secure-metadata` is a pre-release TypeScript library for deterministic, security-conscious inspection, cleaning, and verification of metadata in binary image formats. It is being built for privacy-first, entirely local use with no analytics, telemetry, network access, runtime CDN, or pixel decoding.

> **Development status:** Sprint 0 establishes the repository and API foundation. JPEG, PNG, WebP, EXIF, TIFF, XMP, IPTC, and ICC parsing and all cleaning behavior are not implemented yet.
## Development status

The current implementation provides a bounded binary input core and signature-based JPEG, PNG, and WebP format detection. `inspectMetadata` returns an explicit `format-only` report; metadata decoding, cleaning, and verification are not implemented.

## Format status

JPEG, WebP, and PNG are planned, in that order. See [format support](docs/format-support.md) for the intended progression.
JPEG is detected from `FF D8`, PNG from its complete eight-byte signature, and WebP from `RIFF` plus `WEBP` identifiers. Detection identifies a likely container only. It does not yet validate JPEG segments, PNG chunks, WebP RIFF sizes or chunks, or any metadata. See [format support](docs/format-support.md).

## Installation

The package is not published. Installation instructions will be added for the first pre-release.

## Public API

The future top-level API is deliberately small:
The top-level API is deliberately small:

```ts
import {
Expand All @@ -24,15 +26,17 @@ import {
} from "secure-metadata";
```

All three functions currently throw a typed `NotImplementedError`. Public binary inputs are `Uint8Array | ArrayBuffer`; Node.js `Buffer` values work structurally as `Uint8Array` but are not part of the public contract.
`inspectMetadata` currently normalizes `Uint8Array | ArrayBuffer` input without copying it, enforces `maxInputBytes`, detects the container signature, and returns an empty-entry report marked `inspectionStatus: "format-only"`. That status means metadata has not been decoded; it does not claim metadata is absent.

`cleanMetadata` and `verifyMetadata` still throw a typed `NotImplementedError`. Node.js `Buffer` values work structurally as `Uint8Array` but are not part of the public contract.

## Security philosophy

Every byte is untrusted. Future binary reads will use bounded primitives, traversal will have hard limits, and malformed input must not crash a parser. Cleaning will preserve unknown structures and ICC/color information by default, and cleaner output will be independently inspectable. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md).
Every byte is untrusted. Binary reads use centrally checked ranges, parser input views retain their original boundaries, and malformed or tiny inputs are ordinary data. Cleaning will preserve unknown structures and ICC/color information by default, and cleaner output will be independently inspectable. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md).

## Non-goals

The library does not perform image decoding or encoding, visual redaction, pixel-content privacy analysis, steganography detection, or malware scanning. Absence of metadata is never proof that an image contains no private information.
The library does not perform image decoding or encoding, visual redaction, pixel-content privacy analysis, steganography detection, or malware scanning. Absence of decoded metadata is never proof that an image contains no private information.

## Secure Tools ecosystem

Expand Down
35 changes: 21 additions & 14 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,41 @@
`secure-metadata` is organized as a side-effect-free binary library. Its planned data flow is:

```text
Input bytes
Input bytes implemented
Format detection
Safe binary view / bounded reads implemented
Container parser
Format detection implemented
Metadata decoder
Container parser planned
Metadata normalization/classification
Metadata decoder planned
Inspector / policy engine
Metadata normalization/classification planned
Cleaner
Inspector / policy engine planned
Output bytes
Cleaner planned
Re-inspection / verification
Output bytes planned
Re-inspection / verification planned
```

## Binary core

All future format parsers build on `src/core/binary`. Input normalization returns the caller's exact `Uint8Array` view or creates a no-copy view over an `ArrayBuffer`. `ByteReader` validates offsets and lengths as non-negative safe integers and checks remaining capacity with subtraction before every read. It provides bounded unsigned 8-, 16-, and 32-bit reads in both endian orders, subarray views, and allocation-free signature matching.

`ByteReader` is an internal implementation primitive, not part of the stable package exports. Its `DataView` is constrained to the input view's `byteOffset` and `byteLength`, and accesses occur only after project-owned bounds validation.

## Layer boundaries

- **Container parsing** identifies and bounds JPEG segments, PNG chunks, or WebP RIFF chunks without decoding pixels.
- **Metadata decoding** interprets known metadata payloads. EXIF/TIFF will be one shared decoder reused by JPEG, PNG, and WebP.
- **Format detection** identifies PNG, JPEG, and WebP signatures in that explicit order. It does not imply structural validity.
- **Container parsing** will identify and bound JPEG segments, PNG chunks, or WebP RIFF chunks without decoding pixels.
- **Metadata decoding** will interpret known metadata payloads. EXIF/TIFF will be one shared decoder reused by JPEG, PNG, and WebP.
- **Normalization and classification** maps format-specific fields to stable namespaces and semantic categories.
- **Privacy relevance** is an independent description of whether an entry can concern privacy. It is not a contextual risk score.
- **Cleaning policy** decides which proven structures to remove while preserving required, color, rendering, image-payload, and unknown data by default.
- **Verification** independently re-inspects cleaner output and compares it with an explicit expectation.

Format packages will depend on bounded primitives in `src/core/binary`. Decoders and policy code must not perform ad hoc binary reads. Public APIs accept bytes and return values without filesystem, network, browser-global, or other environmental side effects.

Sprint 0 establishes interfaces and boundaries only. Format detection, binary primitives, parsers, decoders, cleaning, and verification logic are intentionally not implemented.
`inspectMetadata` currently stops after detection and returns `inspectionStatus: "format-only"`. Empty entries therefore mean “not decoded,” not “confirmed absent.” Public APIs remain free of filesystem, network, browser-global, and other environmental side effects.
18 changes: 13 additions & 5 deletions docs/format-support.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,29 @@
# Format Support

No image format is parsed in Sprint 0. Planned implementation priority is:
Sprint 1 implements deterministic container signature detection only. Planned parser implementation priority remains:

1. JPEG
2. WebP
3. PNG

Detection order is explicitly PNG, JPEG, WebP, then unknown. The supported signatures are distinct, so the ordering does not create heuristic ambiguity.

## JPEG

JPEG is first because its segment model provides the initial foundation for marker parsing and bounded traversal. Future work covers APP segments, EXIF, shared TIFF IFD and GPS decoding, XMP, IPTC, comments, and the distinction between privacy metadata and ICC profiles.
**Implemented:** detection when the first two bytes are `FF D8`.

This does not require an EOI marker and does not validate or traverse markers or segments. APP segments, EXIF, shared TIFF IFD and GPS decoding, XMP, IPTC, comments, and ICC distinctions remain future work. Malformed trailing bytes do not change a matching Sprint 1 signature classification.

## WebP

WebP support will add bounded RIFF chunk parsing, EXIF, XMP, ICCP, image and animation payload distinctions, and consistent handling of VP8X feature flags when metadata chunks change.
**Implemented:** detection of `RIFF` at offset 0 and `WEBP` at offset 8, requiring at least 12 bytes.

The four RIFF size bytes are deliberately ignored. RIFF size validation, chunk traversal, EXIF, XMP, ICCP, image and animation payload distinctions, and VP8X consistency handling remain future work.

## PNG

PNG support will add chunk parsing, textual metadata, eXIf, XMP, ICC and color chunks, privacy-relevant ancillary chunks, and detection of compressed metadata. Compressed metadata decompression is not part of Sprint 0.
**Implemented:** detection of the complete eight-byte PNG signature `89 50 4E 47 0D 0A 1A 0A`.

A truncated or corrupted signature is unknown. IHDR and chunk structure are not inspected. Textual metadata, eXIf, XMP, ICC and color chunks, privacy-relevant ancillary chunks, CRC checking, and compressed metadata remain future work.

Format claims will track implemented and tested behavior; planned items are not advertised as supported.
`inspectionStatus: "format-only"` records this boundary in API results. A detected signature is not a claim that a file is structurally valid or that its metadata has been inspected.
12 changes: 9 additions & 3 deletions docs/security-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Binary metadata parsing processes attacker-controlled structure, sizes, offsets,
## Invariants

1. All binary input is untrusted.
2. All binary reads must eventually go through bounded primitives.
2. All binary reads go through bounded primitives.
3. No parser may perform unchecked offset arithmetic.
4. Parser traversal must be hard bounded.
5. TIFF/IFD traversal must eventually include cycle detection.
Expand All @@ -19,12 +19,18 @@ Binary metadata parsing processes attacker-controlled structure, sizes, offsets,
13. The library must never claim that an image contains no private information merely because metadata is absent.
14. Steganography detection, malware scanning, visual redaction, and pixel-content privacy analysis are outside project scope.

## Bounded binary reads

Offsets and lengths must be non-negative safe integers. Ranges are checked with `length <= inputLength - offset`, avoiding overflow-prone addition during validation. Invalid offsets, invalid lengths, and out-of-bounds ranges throw typed library errors before `DataView` access. Signature mismatches and insufficient signature bytes return `false` rather than throwing.

Normalization does not copy whole inputs. A supplied `Uint8Array` retains its exact offset and length, so bytes elsewhere in its backing buffer are inaccessible to the reader. An `ArrayBuffer` receives a no-copy byte view. The inspector never writes through either representation.

## Hard limits

Default limits bound input size, container counts, metadata entries, TIFF depth and entry counts, strings, future decompressed data, and diagnostics. The defaults are exported as `DEFAULT_PARSE_LIMITS`. They are conservative operational safeguards, not permanent API guarantees, and may evolve during `0.x` development.
`inspectMetadata` enforces the effective `maxInputBytes` before detection and allocation-intensive parsing. Other default limits remain reserved for the parsers that will use them. The defaults are exported as `DEFAULT_PARSE_LIMITS`; they are conservative safeguards, not permanent API guarantees, and may evolve during `0.x` development.

Limits complement bounds checks; they do not replace them. Future parsers must fail safely or produce bounded diagnostics rather than crash on malformed input.

## Environment and dependencies

Core code is local-only and side-effect-free. It has no network, analytics, telemetry, filesystem, DOM, or pixel-codec behavior. The package starts with zero runtime dependencies. Development tools are not part of the shipped runtime.
Core code is local-only and side-effect-free. It has no network, analytics, telemetry, filesystem, DOM, or pixel-codec behavior. The package has zero runtime dependencies. Development tools are not part of the shipped runtime.
34 changes: 34 additions & 0 deletions src/core/binary/bounds.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { BinaryBoundsError } from "../errors.js";

export function hasValidRange(
inputLength: number,
offset: number,
length: number,
): boolean {
return (
Number.isSafeInteger(offset) &&
Number.isSafeInteger(length) &&
offset >= 0 &&
length >= 0 &&
offset <= inputLength &&
length <= inputLength - offset
);
}

export function assertValidRange(
inputLength: number,
offset: number,
length: number,
): void {
if (!Number.isSafeInteger(offset) || offset < 0) {
throw new BinaryBoundsError("INVALID_OFFSET", inputLength, offset, length);
}

if (!Number.isSafeInteger(length) || length < 0) {
throw new BinaryBoundsError("INVALID_LENGTH", inputLength, offset, length);
}

if (offset > inputLength || length > inputLength - offset) {
throw new BinaryBoundsError("OUT_OF_BOUNDS", inputLength, offset, length);
}
}
64 changes: 64 additions & 0 deletions src/core/binary/byte-reader.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import { assertValidRange, hasValidRange } from "./bounds.js";

/** Read-only access to a caller-supplied byte view through checked ranges. */
export class ByteReader {
readonly length: number;

readonly #bytes: Uint8Array;
readonly #view: DataView;

constructor(bytes: Uint8Array) {
this.#bytes = bytes;
this.#view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
this.length = bytes.byteLength;
}

has(offset: number, length = 1): boolean {
return hasValidRange(this.length, offset, length);
}

u8(offset: number): number {
assertValidRange(this.length, offset, 1);
return this.#view.getUint8(offset);
}

u16LE(offset: number): number {
assertValidRange(this.length, offset, 2);
return this.#view.getUint16(offset, true);
}

u16BE(offset: number): number {
assertValidRange(this.length, offset, 2);
return this.#view.getUint16(offset, false);
}

u32LE(offset: number): number {
assertValidRange(this.length, offset, 4);
return this.#view.getUint32(offset, true);
}

u32BE(offset: number): number {
assertValidRange(this.length, offset, 4);
return this.#view.getUint32(offset, false);
}

/** Returns a bounded view, not a copy, after validating the complete range. */
slice(offset: number, length: number): Uint8Array {
assertValidRange(this.length, offset, length);
return this.#bytes.subarray(offset, offset + length);
}

matches(offset: number, signature: readonly number[]): boolean {
if (!this.has(offset, signature.length)) {
return false;
}

for (let index = 0; index < signature.length; index += 1) {
if (this.#bytes[offset + index] !== signature[index]) {
return false;
}
}

return true;
}
}
3 changes: 3 additions & 0 deletions src/core/binary/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export { ByteReader } from "./byte-reader.js";
export { assertValidRange, hasValidRange } from "./bounds.js";
export { toUint8Array } from "./input.js";
6 changes: 6 additions & 0 deletions src/core/binary/input.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import type { BinaryInput } from "../types.js";

/** Normalizes supported input without copying or widening a Uint8Array view. */
export function toUint8Array(input: BinaryInput): Uint8Array {
return input instanceof Uint8Array ? input : new Uint8Array(input);
}
24 changes: 24 additions & 0 deletions src/core/detect-format.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { type ByteReader } from "./binary/index.js";
import type { ImageFormat } from "./types.js";

const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];
const JPEG_SIGNATURE = [0xff, 0xd8];
const RIFF_SIGNATURE = [0x52, 0x49, 0x46, 0x46];
const WEBP_SIGNATURE = [0x57, 0x45, 0x42, 0x50];

/** Identifies a container signature without validating its internal structure. */
export function detectFormat(reader: ByteReader): ImageFormat {
if (reader.matches(0, PNG_SIGNATURE)) {
return "png";
}

if (reader.matches(0, JPEG_SIGNATURE)) {
return "jpeg";
}

if (reader.matches(0, RIFF_SIGNATURE) && reader.matches(8, WEBP_SIGNATURE)) {
return "webp";
}

return "unknown";
}
55 changes: 54 additions & 1 deletion src/core/errors.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
export type SecureMetadataErrorCode = "NOT_IMPLEMENTED";
export type SecureMetadataErrorCode =
| "NOT_IMPLEMENTED"
| "INVALID_OFFSET"
| "INVALID_LENGTH"
| "OUT_OF_BOUNDS"
| "INVALID_LIMIT"
| "INPUT_LIMIT_EXCEEDED";

export class SecureMetadataError extends Error {
override readonly name: string = "SecureMetadataError";
Expand All @@ -22,3 +28,50 @@ export class NotImplementedError extends SecureMetadataError {
);
}
}

export type BinaryBoundsErrorCode =
"INVALID_OFFSET" | "INVALID_LENGTH" | "OUT_OF_BOUNDS";

export class BinaryBoundsError extends SecureMetadataError {
override readonly name: string = "BinaryBoundsError";

constructor(
code: BinaryBoundsErrorCode,
readonly inputLength: number,
readonly offset: number,
readonly requestedLength: number,
) {
super(
`Invalid binary range: offset ${String(offset)}, length ${String(requestedLength)}, input length ${String(inputLength)}.`,
code,
);
}
}

export class InvalidParseLimitError extends SecureMetadataError {
override readonly name: string = "InvalidParseLimitError";

constructor(
readonly limitName: string,
readonly value: number,
) {
super(
`Parse limit ${limitName} must be a non-negative safe integer; received ${String(value)}.`,
"INVALID_LIMIT",
);
}
}

export class InputLimitExceededError extends SecureMetadataError {
override readonly name: string = "InputLimitExceededError";

constructor(
readonly inputLength: number,
readonly maximumLength: number,
) {
super(
`Input length ${String(inputLength)} exceeds maxInputBytes ${String(maximumLength)}.`,
"INPUT_LIMIT_EXCEEDED",
);
}
}
4 changes: 4 additions & 0 deletions src/core/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,12 @@ export interface InspectOptions {
readonly limits?: Partial<ParseLimits>;
}

export type InspectionStatus = "format-only" | "metadata-inspected";

export interface MetadataReport {
readonly format: ImageFormat;
readonly size: number;
readonly inspectionStatus: InspectionStatus;
readonly entries: readonly MetadataEntry[];
readonly diagnostics: readonly Diagnostic[];
}
Expand Down
Loading
Loading