diff --git a/CHANGELOG.md b/CHANGELOG.md index a079d43..ab62d76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,12 +6,14 @@ All notable changes will be documented here. The project intends to follow seman ### Added -- Bounded unsigned byte and endian-aware integer reads. -- Checked binary range validation with typed library errors. -- No-copy normalization for `Uint8Array` and `ArrayBuffer` inputs. -- Signature-based JPEG, PNG, and WebP format detection. -- Explicit format-only inspection reports with input-size enforcement. -- Binary boundary, sliced-view, format, and malformed-input tests. +- Bounded JPEG marker and length-prefixed segment traversal. +- Standalone, fill-byte, restart-marker, EOI, and multi-scan handling. +- JPEG APP and COM container classification. +- EXIF, standard/extended XMP, ICC, Photoshop/IPTC, JFIF/JFXX, and Adobe presence detection. +- Structured malformed, truncation, trailing-data, and segment-limit diagnostics. +- Normalized JPEG metadata-container entries and complete/partial container status. +- Bounded binary reader, no-copy input normalization, and JPEG/PNG/WebP format detection. +- Binary boundary, sliced-view, format, malformed-input, and JPEG container tests. ### Foundation diff --git a/README.md b/README.md index 4f30eb8..e607588 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,24 @@ # secure-metadata -`secure-metadata` is a pre-release TypeScript library for deterministic, security-conscious inspection, cleaning, and verification of metadata in binary image formats. It is being built for privacy-first, entirely local use with no analytics, telemetry, network access, runtime CDN, or pixel decoding. +`secure-metadata` is a pre-release TypeScript library for deterministic, security-conscious inspection, cleaning, and verification of metadata in binary image formats. It is built for privacy-first, entirely local use with no analytics, telemetry, network access, runtime CDN, or pixel decoding. ## Development status -The current implementation provides a bounded binary input core and signature-based JPEG, PNG, and WebP format detection. `inspectMetadata` returns an explicit `format-only` report; metadata decoding, cleaning, and verification are not implemented. +Current implementation: + +- bounded binary input and endian-aware read core; +- JPEG, PNG, and WebP signature detection; +- bounded JPEG marker and segment traversal; +- JPEG entropy-scan skipping without image decoding; +- JPEG EXIF, XMP, extended XMP, ICC, Photoshop/IPTC, and comment container-presence detection. + +Not implemented: TIFF/EXIF or GPS field decoding, XML/IPTC/ICC payload decoding, PNG/WebP container parsing, metadata cleaning, and verification. ## Format status -JPEG is detected from `FF D8`, PNG from its complete eight-byte signature, and WebP from `RIFF` plus `WEBP` identifiers. Detection identifies a likely container only. It does not yet validate JPEG segments, PNG chunks, WebP RIFF sizes or chunks, or any metadata. See [format support](docs/format-support.md). +JPEG reports can be `container-inspected` or `container-partial`. PNG and WebP remain `format-only`. See [format support](docs/format-support.md) for the precise matrix. + +A detected or traversed container is not necessarily a decodable image. The JPEG parser validates marker and segment boundaries, not quantization, Huffman, frame, scan-header, or entropy semantics. ## Installation @@ -16,8 +26,6 @@ The package is not published. Installation instructions will be added for the fi ## Public API -The top-level API is deliberately small: - ```ts import { cleanMetadata, @@ -26,17 +34,17 @@ import { } from "secure-metadata"; ``` -`inspectMetadata` currently normalizes `Uint8Array | ArrayBuffer` input without copying it, enforces `maxInputBytes`, detects the container signature, and returns an empty-entry report marked `inspectionStatus: "format-only"`. That status means metadata has not been decoded; it does not claim metadata is absent. +`inspectMetadata` accepts `Uint8Array | ArrayBuffer`, enforces relevant parse limits, and returns a deterministic report. For JPEG it inventories the container and emits one normalized entry per recognized privacy/color metadata container. Entries identify container presence only; payload values are not decoded. `cleanMetadata` and `verifyMetadata` still throw a typed `NotImplementedError`. Node.js `Buffer` values work structurally as `Uint8Array` but are not part of the public contract. ## Security philosophy -Every byte is untrusted. Binary reads use centrally checked ranges, parser input views retain their original boundaries, and malformed or tiny inputs are ordinary data. Cleaning will preserve unknown structures and ICC/color information by default, and cleaner output will be independently inspectable. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md). +Every byte is untrusted. Binary reads use centrally checked ranges, parser input views retain their original boundaries, traversal is hard bounded, and malformed or tiny inputs are ordinary data. Unknown APP segments remain unknown and should be preserved by future cleaning. See the [security model](docs/security-model.md), [architecture](docs/architecture.md), and [cleaning policy](docs/cleaning-policy.md). ## Non-goals -The library does not perform image decoding or encoding, visual redaction, pixel-content privacy analysis, steganography detection, or malware scanning. Absence of decoded metadata is never proof that an image contains no private information. +The library does not perform image decoding or encoding, visual redaction, pixel-content privacy analysis, steganography detection, or malware scanning. Absence of recognized metadata containers is never proof that an image contains no private information. ## Secure Tools ecosystem diff --git a/docs/architecture.md b/docs/architecture.md index de7140c..51209dc 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,6 @@ # Architecture -`secure-metadata` is organized as a side-effect-free binary library. Its planned data flow is: +`secure-metadata` is organized as a side-effect-free binary library. Its current and planned flow is: ```text Input bytes implemented @@ -9,35 +9,36 @@ Safe binary view / bounded reads implemented ↓ Format detection implemented ↓ -Container parser planned +JPEG container parser implemented for JPEG ↓ -Metadata decoder planned +Metadata container classification implemented for JPEG ↓ -Metadata normalization/classification planned +Metadata payload decoder planned ↓ -Inspector / policy engine planned +Normalization / field classification planned ↓ -Cleaner planned +Policy engine and cleaner planned ↓ -Output bytes planned - ↓ -Re-inspection / verification planned +Output re-inspection / verification planned ``` ## Binary core -All future format parsers build on `src/core/binary`. Input normalization returns the caller's exact `Uint8Array` view or creates a no-copy view over an `ArrayBuffer`. `ByteReader` validates offsets and lengths as non-negative safe integers and checks remaining capacity with subtraction before every read. It provides bounded unsigned 8-, 16-, and 32-bit reads in both endian orders, subarray views, and allocation-free signature matching. +Input normalization returns the caller's exact `Uint8Array` view or creates a no-copy view over an `ArrayBuffer`. `ByteReader` validates offsets and lengths as non-negative safe integers and checks remaining capacity with subtraction before every read. It provides bounded unsigned 8-, 16-, and 32-bit reads, subarray views, and allocation-free signature matching. + +## JPEG container layer + +The iterative JPEG parser validates SOI and walks markers using the binary core. A central marker model distinguishes SOI, EOI, TEM, RST0–RST7, APP0–APP15, COM, SOS, common image-structure markers, and length-prefixed unknown markers. Repeated `FF` fill bytes are collapsed to one marker; declared lengths include their two-byte length field and must fit fully before offsets advance. + +After SOS, the parser scans rather than decodes entropy data. `FF 00` remains stuffed data, restart markers are recorded without terminating the scan, and the next real marker resumes normal traversal. This supports multiple scans. Every marker, including SOI, EOI, SOS, and restarts, counts toward `maxSegments`. -`ByteReader` is an internal implementation primitive, not part of the stable package exports. Its `DataView` is constrained to the input view's `byteOffset` and `byteLength`, and accesses occur only after project-owned bounds validation. +APP signatures are checked within segment payload boundaries without retaining payload copies. EXIF, standard/extended XMP, ICC, Photoshop/IPTC, JFIF/JFXX, Adobe, and unknown classifications remain container-level observations. -## Layer boundaries +## Inspection status -- **Format detection** identifies PNG, JPEG, and WebP signatures in that explicit order. It does not imply structural validity. -- **Container parsing** will identify and bound JPEG segments, PNG chunks, or WebP RIFF chunks without decoding pixels. -- **Metadata decoding** will interpret known metadata payloads. EXIF/TIFF will be one shared decoder reused by JPEG, PNG, and WebP. -- **Normalization and classification** maps format-specific fields to stable namespaces and semantic categories. -- **Privacy relevance** is an independent description of whether an entry can concern privacy. It is not a contextual risk score. -- **Cleaning policy** decides which proven structures to remove while preserving required, color, rendering, image-payload, and unknown data by default. -- **Verification** independently re-inspects cleaner output and compares it with an explicit expectation. +- `format-only`: a signature was detected; no container parser ran. Currently PNG, WebP, unknown, and short arbitrary inputs. +- `container-inspected`: JPEG traversal reached EOI safely. +- `container-partial`: JPEG identity is known, but traversal stopped on a structural error, truncation, or limit. +- `metadata-inspected`: reserved for future payload decoders. -`inspectMetadata` currently stops after detection and returns `inspectionStatus: "format-only"`. Empty entries therefore mean “not decoded,” not “confirmed absent.” Public APIs remain free of filesystem, network, browser-global, and other environmental side effects. +An empty entry list means no supported metadata container was recognized during the completed portion of traversal. It does not prove metadata or private information is absent. diff --git a/docs/format-support.md b/docs/format-support.md index 31a0f7d..e2da878 100644 --- a/docs/format-support.md +++ b/docs/format-support.md @@ -1,29 +1,33 @@ # Format Support -Sprint 1 implements deterministic container signature detection only. Planned parser implementation priority remains: - -1. JPEG -2. WebP -3. PNG - -Detection order is explicitly PNG, JPEG, WebP, then unknown. The supported signatures are distinct, so the ordering does not create heuristic ambiguity. +| Capability | JPEG | PNG | WebP | +| --------------------------- | --------------------------------- | --------- | --------- | +| Signature detection | Supported | Supported | Supported | +| Bounded container traversal | Supported | Not yet | Not yet | +| APP/COM classification | Supported | N/A | N/A | +| EXIF container detection | Supported | Not yet | Not yet | +| XMP container detection | Supported, including extended XMP | Not yet | Not yet | +| ICC container detection | Supported | Not yet | Not yet | +| IPTC/Photoshop detection | Supported | Not yet | Not yet | +| Metadata field decoding | Not yet | Not yet | Not yet | +| Cleaning | Not yet | Not yet | Not yet | ## JPEG -**Implemented:** detection when the first two bytes are `FF D8`. - -This does not require an EOI marker and does not validate or traverse markers or segments. APP segments, EXIF, shared TIFF IFD and GPS decoding, XMP, IPTC, comments, and ICC distinctions remain future work. Malformed trailing bytes do not change a matching Sprint 1 signature classification. - -## WebP - -**Implemented:** detection of `RIFF` at offset 0 and `WEBP` at offset 8, requiring at least 12 bytes. +JPEG detection requires `FF D8`. Container inspection validates marker boundaries and two-byte big-endian declared lengths, recognizes standalone markers and fill bytes, stops at EOI, and reports trailing bytes. SOS headers are traversed, while entropy-coded bytes are skipped without decoding; `FF 00`, RST0–RST7, and multiple scans are handled structurally. -The four RIFF size bytes are deliberately ignored. RIFF size validation, chunk traversal, EXIF, XMP, ICCP, image and animation payload distinctions, and VP8X consistency handling remain future work. +Payload signatures identify: -## PNG +- APP0 `JFIF\0` and `JFXX\0` as technical container data; +- APP1 `Exif\0\0` as EXIF; +- APP1 standard and extended Adobe XMP identifiers as XMP; +- APP2 `ICC_PROFILE\0` as ICC; +- APP13 `Photoshop 3.0\0` as Photoshop/IPTC; +- APP14 `Adobe` as rendering/container data; +- COM as comment metadata. -**Implemented:** detection of the complete eight-byte PNG signature `89 50 4E 47 0D 0A 1A 0A`. +Unknown APP payloads remain unknown. No TIFF, EXIF, XMP XML, ICC, IPTC, thumbnail, frame, Huffman, quantization, or entropy payload is decoded. -A truncated or corrupted signature is unknown. IHDR and chunk structure are not inspected. Textual metadata, eXIf, XMP, ICC and color chunks, privacy-relevant ancillary chunks, CRC checking, and compressed metadata remain future work. +## PNG and WebP -`inspectionStatus: "format-only"` records this boundary in API results. A detected signature is not a claim that a file is structurally valid or that its metadata has been inspected. +PNG requires its complete eight-byte signature. WebP requires `RIFF` at offset 0 and `WEBP` at offset 8. Their chunk structures, sizes, CRCs, metadata, and image payloads are not yet parsed. diff --git a/docs/security-model.md b/docs/security-model.md index d54bc78..3a7558f 100644 --- a/docs/security-model.md +++ b/docs/security-model.md @@ -1,6 +1,6 @@ # Security Model -Binary metadata parsing processes attacker-controlled structure, sizes, offsets, encodings, and nesting. The project therefore treats malformed files, parser panics, excessive allocation or traversal, and incorrect offset arithmetic as security concerns. +Binary metadata parsing processes attacker-controlled structure, sizes, offsets, encodings, and nesting. Malformed files, parser crashes, excessive allocation or traversal, and incorrect offset arithmetic are security concerns. ## Invariants @@ -21,15 +21,25 @@ Binary metadata parsing processes attacker-controlled structure, sizes, offsets, ## Bounded binary reads -Offsets and lengths must be non-negative safe integers. Ranges are checked with `length <= inputLength - offset`, avoiding overflow-prone addition during validation. Invalid offsets, invalid lengths, and out-of-bounds ranges throw typed library errors before `DataView` access. Signature mismatches and insufficient signature bytes return `false` rather than throwing. +Offsets and lengths must be non-negative safe integers. Ranges use `length <= inputLength - offset`, avoiding overflow-prone addition during validation. Invalid offsets, lengths, and ranges throw typed library errors before `DataView` access. A supplied `Uint8Array` retains its exact offset and length; `ArrayBuffer` normalization creates a no-copy byte view. Inspection never writes through either representation. -Normalization does not copy whole inputs. A supplied `Uint8Array` retains its exact offset and length, so bytes elsewhere in its backing buffer are inaccessible to the reader. An `ArrayBuffer` receives a no-copy byte view. The inspector never writes through either representation. +## JPEG-specific properties -## Hard limits +- Marker reads and fill-byte scans remain within the supplied input view. +- Every recorded marker, including restart markers inside scans, counts toward `maxSegments`. +- A declared segment length must be at least two and fit completely before subtraction or offset advancement. +- APP signatures must fit within their segment payload and cannot match across segment boundaries. +- Entropy-coded scan data is traversed but never decoded or copied. +- `FF 00` stuffing remains data; RST0–RST7 do not terminate a scan. +- Normal parsing resumes at non-stuffed, non-restart markers, allowing multiple SOS scans. +- EOI stops traversal; trailing bytes produce a warning rather than being parsed as JPEG. +- Malformed and truncated JPEGs return bounded structured diagnostics instead of uncontrolled native bounds exceptions. + +All parser loops are iterative. Each successful branch advances its cursor or returns, which prevents non-progress cycles on hostile fill, scan, or marker data. -`inspectMetadata` enforces the effective `maxInputBytes` before detection and allocation-intensive parsing. Other default limits remain reserved for the parsers that will use them. The defaults are exported as `DEFAULT_PARSE_LIMITS`; they are conservative safeguards, not permanent API guarantees, and may evolve during `0.x` development. +## Hard limits -Limits complement bounds checks; they do not replace them. Future parsers must fail safely or produce bounded diagnostics rather than crash on malformed input. +`inspectMetadata` enforces `maxInputBytes` before parsing. JPEG traversal enforces `maxSegments`; unused limits remain reserved for their future parsers. Defaults are conservative safeguards rather than permanent `0.x` API guarantees. ## Environment and dependencies diff --git a/src/core/diagnostics.ts b/src/core/diagnostics.ts index 5264b51..c690773 100644 --- a/src/core/diagnostics.ts +++ b/src/core/diagnostics.ts @@ -1,7 +1,19 @@ export type DiagnosticSeverity = "warning" | "error"; export type DiagnosticCode = - "NOT_IMPLEMENTED" | "INPUT_LIMIT_EXCEEDED" | "MALFORMED_INPUT"; + | "NOT_IMPLEMENTED" + | "INPUT_LIMIT_EXCEEDED" + | "MALFORMED_INPUT" + | "JPEG_INVALID_SOI" + | "JPEG_INVALID_MARKER" + | "JPEG_TRUNCATED_MARKER" + | "JPEG_TRUNCATED_SEGMENT_LENGTH" + | "JPEG_INVALID_SEGMENT_LENGTH" + | "JPEG_TRUNCATED_SEGMENT" + | "JPEG_TRUNCATED_SCAN" + | "JPEG_MISSING_EOI" + | "JPEG_SEGMENT_LIMIT_EXCEEDED" + | "JPEG_TRAILING_DATA"; export interface Diagnostic { readonly severity: DiagnosticSeverity; diff --git a/src/core/types.ts b/src/core/types.ts index b1b7858..5d88f19 100644 --- a/src/core/types.ts +++ b/src/core/types.ts @@ -62,7 +62,11 @@ export interface InspectOptions { readonly limits?: Partial; } -export type InspectionStatus = "format-only" | "metadata-inspected"; +export type InspectionStatus = + | "format-only" + | "container-inspected" + | "container-partial" + | "metadata-inspected"; export interface MetadataReport { readonly format: ImageFormat; diff --git a/src/inspect.ts b/src/inspect.ts index b5d5c34..71f45f2 100644 --- a/src/inspect.ts +++ b/src/inspect.ts @@ -10,25 +10,55 @@ import type { InspectOptions, MetadataReport, } from "./core/types.js"; +import { jpegMetadataEntries } from "./jpeg/metadata.js"; +import { parseJpeg } from "./jpeg/parser.js"; + +function effectiveLimit( + name: "maxInputBytes" | "maxSegments", + configured: number | undefined, +): number { + const value = configured ?? DEFAULT_PARSE_LIMITS[name]; + if (!Number.isSafeInteger(value) || value < 0) { + throw new InvalidParseLimitError(name, value); + } + return value; +} export function inspectMetadata( input: BinaryInput, options?: InspectOptions, ): MetadataReport { const bytes = toUint8Array(input); - const maxInputBytes = - options?.limits?.maxInputBytes ?? DEFAULT_PARSE_LIMITS.maxInputBytes; - - if (!Number.isSafeInteger(maxInputBytes) || maxInputBytes < 0) { - throw new InvalidParseLimitError("maxInputBytes", maxInputBytes); - } + const maxInputBytes = effectiveLimit( + "maxInputBytes", + options?.limits?.maxInputBytes, + ); if (bytes.byteLength > maxInputBytes) { throw new InputLimitExceededError(bytes.byteLength, maxInputBytes); } + const reader = new ByteReader(bytes); + const format = detectFormat(reader); + if (format === "jpeg") { + const maxSegments = effectiveLimit( + "maxSegments", + options?.limits?.maxSegments, + ); + const jpeg = parseJpeg(reader, maxSegments); + return { + format, + size: bytes.byteLength, + inspectionStatus: jpeg.complete + ? "container-inspected" + : "container-partial", + entries: jpegMetadataEntries(jpeg), + diagnostics: jpeg.diagnostics, + }; + } + return { - format: detectFormat(new ByteReader(bytes)), + format, size: bytes.byteLength, inspectionStatus: "format-only", entries: [], diff --git a/src/jpeg/classify.ts b/src/jpeg/classify.ts new file mode 100644 index 0000000..a42d892 --- /dev/null +++ b/src/jpeg/classify.ts @@ -0,0 +1,128 @@ +import { type ByteReader } from "../core/binary/index.js"; +import { JPEG_MARKER } from "./markers.js"; +import type { + JpegMetadataKind, + JpegMetadataSubtype, + JpegSegmentKind, +} from "./types.js"; + +const JFIF_SIGNATURE = [0x4a, 0x46, 0x49, 0x46, 0x00]; +const JFXX_SIGNATURE = [0x4a, 0x46, 0x58, 0x58, 0x00]; +const EXIF_SIGNATURE = [0x45, 0x78, 0x69, 0x66, 0x00, 0x00]; +const XMP_SIGNATURE = [ + 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x6e, 0x73, 0x2e, 0x61, 0x64, 0x6f, + 0x62, 0x65, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x78, 0x61, 0x70, 0x2f, 0x31, 0x2e, + 0x30, 0x2f, 0x00, +]; +const EXTENDED_XMP_SIGNATURE = [ + 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x6e, 0x73, 0x2e, 0x61, 0x64, 0x6f, + 0x62, 0x65, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x78, 0x6d, 0x70, 0x2f, 0x65, 0x78, + 0x74, 0x65, 0x6e, 0x73, 0x69, 0x6f, 0x6e, 0x2f, 0x00, +]; +const ICC_SIGNATURE = [ + 0x49, 0x43, 0x43, 0x5f, 0x50, 0x52, 0x4f, 0x46, 0x49, 0x4c, 0x45, 0x00, +]; +const PHOTOSHOP_SIGNATURE = [ + 0x50, 0x68, 0x6f, 0x74, 0x6f, 0x73, 0x68, 0x6f, 0x70, 0x20, 0x33, 0x2e, 0x30, + 0x00, +]; +const ADOBE_SIGNATURE = [0x41, 0x64, 0x6f, 0x62, 0x65]; + +export interface JpegApplicationClassification { + readonly metadataKind: JpegMetadataKind; + readonly metadataSubtype?: JpegMetadataSubtype; +} + +function matchesPayload( + reader: ByteReader, + payloadOffset: number, + payloadLength: number, + signature: readonly number[], +): boolean { + return ( + signature.length <= payloadLength && + reader.matches(payloadOffset, signature) + ); +} + +export function classifySegmentKind(marker: number): JpegSegmentKind { + if (marker >= 0xe0 && marker <= 0xef) { + return "application"; + } + if (marker === JPEG_MARKER.COM) { + return "comment"; + } + if (marker === JPEG_MARKER.SOS) { + return "scan"; + } + if ( + marker === JPEG_MARKER.TEM || + marker === JPEG_MARKER.SOI || + marker === JPEG_MARKER.EOI || + (marker >= 0xd0 && marker <= 0xd7) + ) { + return "standalone"; + } + if (marker >= 0xc0 && marker <= 0xdf) { + return "image-structure"; + } + return "unknown"; +} + +export function classifyApplicationSegment( + reader: ByteReader, + marker: number, + payloadOffset: number, + payloadLength: number, +): JpegApplicationClassification { + if (marker === 0xe0) { + if (matchesPayload(reader, payloadOffset, payloadLength, JFIF_SIGNATURE)) { + return { metadataKind: "jfif", metadataSubtype: "jfif" }; + } + if (matchesPayload(reader, payloadOffset, payloadLength, JFXX_SIGNATURE)) { + return { metadataKind: "jfif", metadataSubtype: "jfxx" }; + } + } + + if (marker === 0xe1) { + if (matchesPayload(reader, payloadOffset, payloadLength, EXIF_SIGNATURE)) { + return { metadataKind: "exif" }; + } + if (matchesPayload(reader, payloadOffset, payloadLength, XMP_SIGNATURE)) { + return { metadataKind: "xmp", metadataSubtype: "standard-xmp" }; + } + if ( + matchesPayload( + reader, + payloadOffset, + payloadLength, + EXTENDED_XMP_SIGNATURE, + ) + ) { + return { metadataKind: "xmp", metadataSubtype: "extended-xmp" }; + } + } + + if ( + marker === 0xe2 && + matchesPayload(reader, payloadOffset, payloadLength, ICC_SIGNATURE) + ) { + return { metadataKind: "icc" }; + } + + if ( + marker === 0xed && + matchesPayload(reader, payloadOffset, payloadLength, PHOTOSHOP_SIGNATURE) + ) { + return { metadataKind: "iptc", metadataSubtype: "photoshop" }; + } + + if ( + marker === 0xee && + matchesPayload(reader, payloadOffset, payloadLength, ADOBE_SIGNATURE) + ) { + return { metadataKind: "adobe" }; + } + + return { metadataKind: "unknown" }; +} diff --git a/src/jpeg/index.ts b/src/jpeg/index.ts new file mode 100644 index 0000000..e1ff49b --- /dev/null +++ b/src/jpeg/index.ts @@ -0,0 +1,2 @@ +export { parseJpeg } from "./parser.js"; +export type { JpegParseResult, JpegSegment } from "./types.js"; diff --git a/src/jpeg/markers.ts b/src/jpeg/markers.ts new file mode 100644 index 0000000..eeefe2e --- /dev/null +++ b/src/jpeg/markers.ts @@ -0,0 +1,60 @@ +export const JPEG_MARKER = { + TEM: 0x01, + SOF0: 0xc0, + SOF1: 0xc1, + SOF2: 0xc2, + DHT: 0xc4, + SOI: 0xd8, + EOI: 0xd9, + SOS: 0xda, + DQT: 0xdb, + DRI: 0xdd, + COM: 0xfe, +} as const; + +const MARKER_NAMES: Readonly> = { + [JPEG_MARKER.TEM]: "TEM", + [JPEG_MARKER.SOF0]: "SOF0", + [JPEG_MARKER.SOF1]: "SOF1", + [JPEG_MARKER.SOF2]: "SOF2", + [JPEG_MARKER.DHT]: "DHT", + [JPEG_MARKER.SOI]: "SOI", + [JPEG_MARKER.EOI]: "EOI", + [JPEG_MARKER.SOS]: "SOS", + [JPEG_MARKER.DQT]: "DQT", + [JPEG_MARKER.DRI]: "DRI", + [JPEG_MARKER.COM]: "COM", +}; + +export function isApplicationMarker(marker: number): boolean { + return marker >= 0xe0 && marker <= 0xef; +} + +export function isRestartMarker(marker: number): boolean { + return marker >= 0xd0 && marker <= 0xd7; +} + +export function isStandaloneMarker(marker: number): boolean { + return ( + marker === JPEG_MARKER.TEM || + marker === JPEG_MARKER.SOI || + marker === JPEG_MARKER.EOI || + isRestartMarker(marker) + ); +} + +export function isValidMarkerCode(marker: number): boolean { + return marker === JPEG_MARKER.TEM || (marker >= 0xc0 && marker <= 0xfe); +} + +export function markerName(marker: number): string { + if (isApplicationMarker(marker)) { + return `APP${String(marker - 0xe0)}`; + } + + if (isRestartMarker(marker)) { + return `RST${String(marker - 0xd0)}`; + } + + return MARKER_NAMES[marker] ?? `UNKNOWN_${marker.toString(16).toUpperCase()}`; +} diff --git a/src/jpeg/metadata.ts b/src/jpeg/metadata.ts new file mode 100644 index 0000000..e432842 --- /dev/null +++ b/src/jpeg/metadata.ts @@ -0,0 +1,83 @@ +import type { MetadataEntry } from "../core/types.js"; +import { JPEG_MARKER } from "./markers.js"; +import type { JpegParseResult, JpegSegment } from "./types.js"; + +function source(segment: JpegSegment): MetadataEntry["source"] { + return { + format: "jpeg", + container: "jpeg-segment", + offset: segment.offset, + length: segment.length, + jpegMarker: segment.marker, + }; +} + +export function jpegMetadataEntries( + result: JpegParseResult, +): readonly MetadataEntry[] { + const entries: MetadataEntry[] = []; + + for (const segment of result.segments) { + if (segment.marker === JPEG_MARKER.COM) { + entries.push({ + id: `jpeg-comment-${String(segment.offset)}`, + namespace: "jpeg-comment", + name: "JPEG comment", + category: "description", + privacy: "potentially-sensitive", + source: source(segment), + }); + continue; + } + + switch (segment.metadataKind) { + case "exif": + entries.push({ + id: `jpeg-exif-${String(segment.offset)}`, + namespace: "exif", + name: "EXIF container", + category: "unknown", + privacy: "potentially-sensitive", + source: source(segment), + }); + break; + case "xmp": + entries.push({ + id: `jpeg-xmp-${String(segment.offset)}`, + namespace: "xmp", + name: + segment.metadataSubtype === "extended-xmp" + ? "Extended XMP container" + : "XMP container", + category: "unknown", + privacy: "potentially-sensitive", + source: source(segment), + }); + break; + case "icc": + entries.push({ + id: `jpeg-icc-${String(segment.offset)}`, + namespace: "icc", + name: "ICC profile container", + category: "color", + privacy: "non-sensitive", + source: source(segment), + }); + break; + case "iptc": + entries.push({ + id: `jpeg-iptc-${String(segment.offset)}`, + namespace: "iptc", + name: "Photoshop/IPTC container", + category: "unknown", + privacy: "potentially-sensitive", + source: source(segment), + }); + break; + default: + break; + } + } + + return entries; +} diff --git a/src/jpeg/parser.ts b/src/jpeg/parser.ts new file mode 100644 index 0000000..aeb17a1 --- /dev/null +++ b/src/jpeg/parser.ts @@ -0,0 +1,364 @@ +import { type ByteReader } from "../core/binary/index.js"; +import type { Diagnostic, DiagnosticCode } from "../core/diagnostics.js"; +import { classifyApplicationSegment, classifySegmentKind } from "./classify.js"; +import { + isApplicationMarker, + isRestartMarker, + isStandaloneMarker, + isValidMarkerCode, + JPEG_MARKER, + markerName, +} from "./markers.js"; +import type { JpegParseResult, JpegSegment } from "./types.js"; + +interface MarkerPosition { + readonly marker: number; + readonly markerOffset: number; + readonly afterMarker: number; +} + +interface ParserState { + readonly segments: JpegSegment[]; + readonly diagnostics: Diagnostic[]; +} + +function diagnostic( + severity: Diagnostic["severity"], + code: DiagnosticCode, + message: string, + offset?: number, +): Diagnostic { + return offset === undefined + ? { severity, code, message } + : { severity, code, message, offset }; +} + +function readMarker( + reader: ByteReader, + offset: number, +): MarkerPosition | Diagnostic { + if (reader.u8(offset) !== 0xff) { + return diagnostic( + "error", + "JPEG_INVALID_MARKER", + "Expected a JPEG marker prefix.", + offset, + ); + } + + let cursor = offset; + while (reader.has(cursor) && reader.u8(cursor) === 0xff) { + cursor += 1; + } + + if (!reader.has(cursor)) { + return diagnostic( + "error", + "JPEG_TRUNCATED_MARKER", + "JPEG input ends within marker fill bytes.", + offset, + ); + } + + const marker = reader.u8(cursor); + if (marker === 0x00 || !isValidMarkerCode(marker)) { + return diagnostic( + "error", + "JPEG_INVALID_MARKER", + `Invalid JPEG marker code 0x${marker.toString(16).padStart(2, "0")}.`, + cursor, + ); + } + + return { + marker, + markerOffset: cursor - 1, + afterMarker: cursor + 1, + }; +} + +function addSegment( + state: ParserState, + segment: JpegSegment, + maxSegments: number, +): boolean { + if (state.segments.length >= maxSegments) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_SEGMENT_LIMIT_EXCEEDED", + `JPEG marker count exceeds maxSegments ${String(maxSegments)}.`, + segment.offset, + ), + ); + return false; + } + + state.segments.push(segment); + return true; +} + +function incompleteResult( + state: ParserState, + sawSoi: boolean, +): JpegParseResult { + return { + segments: state.segments, + complete: false, + sawSoi, + sawEoi: false, + diagnostics: state.diagnostics, + }; +} + +function skipScanData( + reader: ByteReader, + scanOffset: number, + state: ParserState, + maxSegments: number, +): number | undefined { + let cursor = scanOffset; + + while (reader.has(cursor)) { + if (reader.u8(cursor) !== 0xff) { + cursor += 1; + continue; + } + + const fillStart = cursor; + cursor += 1; + while (reader.has(cursor) && reader.u8(cursor) === 0xff) { + cursor += 1; + } + + if (!reader.has(cursor)) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_TRUNCATED_SCAN", + "JPEG entropy-coded scan ends within marker fill bytes.", + fillStart, + ), + ); + return undefined; + } + + const marker = reader.u8(cursor); + if (marker === 0x00) { + cursor += 1; + continue; + } + + if (isRestartMarker(marker)) { + const markerOffset = cursor - 1; + if ( + !addSegment( + state, + { + marker, + markerName: markerName(marker), + offset: markerOffset, + length: 2, + kind: "standalone", + }, + maxSegments, + ) + ) { + return undefined; + } + cursor += 1; + continue; + } + + return cursor - 1; + } + + state.diagnostics.push( + diagnostic( + "error", + "JPEG_TRUNCATED_SCAN", + "JPEG entropy-coded scan reaches EOF before a terminating marker.", + scanOffset, + ), + ); + return undefined; +} + +export function parseJpeg( + reader: ByteReader, + maxSegments: number, +): JpegParseResult { + const state: ParserState = { segments: [], diagnostics: [] }; + + if (!reader.matches(0, [0xff, JPEG_MARKER.SOI])) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_INVALID_SOI", + "JPEG input does not begin with the SOI marker.", + 0, + ), + ); + return incompleteResult(state, false); + } + + if ( + !addSegment( + state, + { + marker: JPEG_MARKER.SOI, + markerName: "SOI", + offset: 0, + length: 2, + kind: "standalone", + }, + maxSegments, + ) + ) { + return incompleteResult(state, true); + } + + let offset = 2; + while (reader.has(offset)) { + const markerResult = readMarker(reader, offset); + if ("severity" in markerResult) { + state.diagnostics.push(markerResult); + return incompleteResult(state, true); + } + + const { marker, markerOffset, afterMarker } = markerResult; + if (marker === JPEG_MARKER.SOI) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_INVALID_MARKER", + "Unexpected SOI marker inside JPEG container.", + markerOffset, + ), + ); + return incompleteResult(state, true); + } + + if (isStandaloneMarker(marker)) { + if ( + !addSegment( + state, + { + marker, + markerName: markerName(marker), + offset: markerOffset, + length: 2, + kind: "standalone", + }, + maxSegments, + ) + ) { + return incompleteResult(state, true); + } + + offset = afterMarker; + if (marker === JPEG_MARKER.EOI) { + if (offset < reader.length) { + state.diagnostics.push( + diagnostic( + "warning", + "JPEG_TRAILING_DATA", + `JPEG contains ${String(reader.length - offset)} trailing byte(s) after EOI.`, + offset, + ), + ); + } + return { + segments: state.segments, + complete: true, + sawSoi: true, + sawEoi: true, + diagnostics: state.diagnostics, + }; + } + continue; + } + + if (!reader.has(afterMarker, 2)) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_TRUNCATED_SEGMENT_LENGTH", + `${markerName(marker)} is missing its two-byte segment length.`, + afterMarker, + ), + ); + return incompleteResult(state, true); + } + + const declaredLength = reader.u16BE(afterMarker); + if (declaredLength < 2) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_INVALID_SEGMENT_LENGTH", + `${markerName(marker)} declares invalid length ${String(declaredLength)}.`, + afterMarker, + ), + ); + return incompleteResult(state, true); + } + + if (!reader.has(afterMarker, declaredLength)) { + state.diagnostics.push( + diagnostic( + "error", + "JPEG_TRUNCATED_SEGMENT", + `${markerName(marker)} extends beyond the JPEG input.`, + markerOffset, + ), + ); + return incompleteResult(state, true); + } + + const payloadOffset = afterMarker + 2; + const payloadLength = declaredLength - 2; + const segmentEnd = afterMarker + declaredLength; + const classification = isApplicationMarker(marker) + ? classifyApplicationSegment(reader, marker, payloadOffset, payloadLength) + : undefined; + const segment: JpegSegment = { + marker, + markerName: markerName(marker), + offset: markerOffset, + length: declaredLength + 2, + payloadOffset, + payloadLength, + kind: classifySegmentKind(marker), + ...(classification ?? {}), + }; + + if (!addSegment(state, segment, maxSegments)) { + return incompleteResult(state, true); + } + + offset = segmentEnd; + if (marker === JPEG_MARKER.SOS) { + const nextMarkerOffset = skipScanData( + reader, + segmentEnd, + state, + maxSegments, + ); + if (nextMarkerOffset === undefined) { + return incompleteResult(state, true); + } + offset = nextMarkerOffset; + } + } + + state.diagnostics.push( + diagnostic( + "error", + "JPEG_MISSING_EOI", + "JPEG input ends before an EOI marker.", + reader.length, + ), + ); + return incompleteResult(state, true); +} diff --git a/src/jpeg/types.ts b/src/jpeg/types.ts new file mode 100644 index 0000000..6e93882 --- /dev/null +++ b/src/jpeg/types.ts @@ -0,0 +1,35 @@ +import type { Diagnostic } from "../core/diagnostics.js"; + +export type JpegSegmentKind = + | "standalone" + | "application" + | "comment" + | "image-structure" + | "scan" + | "unknown"; + +export type JpegMetadataKind = + "exif" | "xmp" | "icc" | "iptc" | "jfif" | "adobe" | "unknown"; + +export type JpegMetadataSubtype = + "standard-xmp" | "extended-xmp" | "jfif" | "jfxx" | "photoshop"; + +export interface JpegSegment { + readonly marker: number; + readonly markerName: string; + readonly offset: number; + readonly length: number; + readonly payloadOffset?: number; + readonly payloadLength?: number; + readonly kind: JpegSegmentKind; + readonly metadataKind?: JpegMetadataKind; + readonly metadataSubtype?: JpegMetadataSubtype; +} + +export interface JpegParseResult { + readonly segments: readonly JpegSegment[]; + readonly complete: boolean; + readonly sawSoi: boolean; + readonly sawEoi: boolean; + readonly diagnostics: readonly Diagnostic[]; +} diff --git a/tests/helpers/jpeg-builder.ts b/tests/helpers/jpeg-builder.ts new file mode 100644 index 0000000..0045a93 --- /dev/null +++ b/tests/helpers/jpeg-builder.ts @@ -0,0 +1,64 @@ +export const MARKER = { + SOI: 0xd8, + EOI: 0xd9, + SOS: 0xda, + DHT: 0xc4, + DQT: 0xdb, + COM: 0xfe, + APP0: 0xe0, + APP1: 0xe1, + APP2: 0xe2, + APP13: 0xed, + APP14: 0xee, +} as const; + +export function ascii(value: string): Uint8Array { + return Uint8Array.from(value, (character) => character.charCodeAt(0)); +} + +export function concat(...parts: readonly Uint8Array[]): Uint8Array { + const length = parts.reduce((total, part) => total + part.byteLength, 0); + const result = new Uint8Array(length); + let offset = 0; + for (const part of parts) { + result.set(part, offset); + offset += part.byteLength; + } + return result; +} + +export function marker(code: number, fillBytes = 1): Uint8Array { + return Uint8Array.from([...new Uint8Array(fillBytes).fill(0xff), code]); +} + +export function segment( + code: number, + payload: Uint8Array = new Uint8Array(), + fillBytes = 1, +): Uint8Array { + const declaredLength = payload.byteLength + 2; + return concat( + marker(code, fillBytes), + Uint8Array.of(Math.floor(declaredLength / 0x100), declaredLength % 0x100), + payload, + ); +} + +export function jpeg(...parts: readonly Uint8Array[]): Uint8Array { + return concat(marker(MARKER.SOI), ...parts, marker(MARKER.EOI)); +} + +export const EXIF = concat(ascii("Exif"), Uint8Array.of(0, 0)); +export const XMP = concat( + ascii("http://ns.adobe.com/xap/1.0/"), + Uint8Array.of(0), +); +export const EXTENDED_XMP = concat( + ascii("http://ns.adobe.com/xmp/extension/"), + Uint8Array.of(0), +); +export const ICC = concat(ascii("ICC_PROFILE"), Uint8Array.of(0)); +export const PHOTOSHOP = concat(ascii("Photoshop 3.0"), Uint8Array.of(0)); +export const JFIF = concat(ascii("JFIF"), Uint8Array.of(0)); +export const JFXX = concat(ascii("JFXX"), Uint8Array.of(0)); +export const ADOBE = ascii("Adobe"); diff --git a/tests/malformed/jpeg-malformed.test.ts b/tests/malformed/jpeg-malformed.test.ts new file mode 100644 index 0000000..707b680 --- /dev/null +++ b/tests/malformed/jpeg-malformed.test.ts @@ -0,0 +1,142 @@ +import { describe, expect, it } from "vitest"; + +import { ByteReader } from "../../src/core/binary/byte-reader.js"; +import { parseJpeg } from "../../src/jpeg/parser.js"; +import { + concat, + EXIF, + jpeg, + marker, + MARKER, + segment, +} from "../helpers/jpeg-builder.js"; + +function parse(input: Uint8Array, maxSegments = 100) { + return parseJpeg(new ByteReader(input), maxSegments); +} + +function declaredSegment( + code: number, + declaredLength: number, + payload: Uint8Array = new Uint8Array(), +): Uint8Array { + return concat( + marker(code), + Uint8Array.of(Math.floor(declaredLength / 0x100), declaredLength % 0x100), + payload, + ); +} + +describe("malformed JPEG lengths and markers", () => { + it.each([ + [0, "JPEG_INVALID_SEGMENT_LENGTH"], + [1, "JPEG_INVALID_SEGMENT_LENGTH"], + ] as const)("rejects declared segment length %i", (length, code) => { + const result = parse( + concat(marker(MARKER.SOI), declaredSegment(MARKER.APP1, length)), + ); + + expect(result.complete).toBe(false); + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code }), + ); + }); + + it("reports a truncated two-byte segment length", () => { + const result = parse(concat(marker(MARKER.SOI), marker(MARKER.APP1))); + + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_TRUNCATED_SEGMENT_LENGTH" }), + ); + }); + + it("reports a declared segment that extends past EOF", () => { + const result = parse( + concat( + marker(MARKER.SOI), + declaredSegment(MARKER.APP1, 8, Uint8Array.of(0x45)), + ), + ); + + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_TRUNCATED_SEGMENT" }), + ); + }); + + it("reports a marker truncated after FF", () => { + const result = parse(Uint8Array.of(0xff, 0xd8, 0xff)); + + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_TRUNCATED_MARKER" }), + ); + }); + + it.each([ + Uint8Array.of(0xff, 0xd8, 0xff, 0x00), + Uint8Array.of(0xff, 0xd8, 0x12, 0x34), + Uint8Array.of(0xff, 0xd8, 0xff, 0x7f), + ])("reports invalid marker syntax without a native exception", (input) => { + expect(() => parse(input)).not.toThrow(); + expect(parse(input).diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_INVALID_MARKER" }), + ); + }); + + it("reports invalid SOI when called directly", () => { + const result = parse(Uint8Array.of(0, 1, 2)); + + expect(result).toMatchObject({ complete: false, sawSoi: false }); + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_INVALID_SOI" }), + ); + }); + + it("reports a missing EOI after otherwise valid segments", () => { + const result = parse( + concat(marker(MARKER.SOI), segment(MARKER.APP1, EXIF)), + ); + + expect(result.complete).toBe(false); + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_MISSING_EOI" }), + ); + }); + + it("reports a scan truncated before a terminating marker", () => { + const result = parse( + concat( + marker(MARKER.SOI), + segment(MARKER.SOS), + Uint8Array.of(1, 2, 3, 0xff, 0x00), + ), + ); + + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_TRUNCATED_SCAN" }), + ); + }); + + it("reports scan fill bytes truncated at EOF", () => { + const result = parse( + concat(marker(MARKER.SOI), segment(MARKER.SOS), Uint8Array.of(1, 0xff)), + ); + + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_TRUNCATED_SCAN" }), + ); + }); + + it("enforces maxSegments across structural and restart markers", () => { + const input = jpeg( + segment(MARKER.SOS), + Uint8Array.of(1, 0xff, 0xd0, 2, 0xff, 0xd1, 3), + ); + const result = parse(input, 3); + + expect(result.complete).toBe(false); + expect(result.segments).toHaveLength(3); + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_SEGMENT_LIMIT_EXCEEDED" }), + ); + }); +}); diff --git a/tests/unit/inspection.test.ts b/tests/unit/inspection.test.ts index 5a0fb51..ee50410 100644 --- a/tests/unit/inspection.test.ts +++ b/tests/unit/inspection.test.ts @@ -70,12 +70,12 @@ describe("format-only inspection", () => { }); it("accepts ArrayBuffer input without changing the contract", () => { - const input = Uint8Array.of(0xff, 0xd8).buffer; + const input = Uint8Array.of(0xff, 0xd8, 0xff, 0xd9).buffer; expect(inspectMetadata(input)).toMatchObject({ format: "jpeg", - size: 2, - inspectionStatus: "format-only", + size: 4, + inspectionStatus: "container-inspected", }); }); }); diff --git a/tests/unit/jpeg-inspection.test.ts b/tests/unit/jpeg-inspection.test.ts new file mode 100644 index 0000000..7dd4e01 --- /dev/null +++ b/tests/unit/jpeg-inspection.test.ts @@ -0,0 +1,200 @@ +import { describe, expect, it } from "vitest"; + +import { inspectMetadata } from "../../src/index.js"; +import { + ADOBE, + concat, + EXIF, + EXTENDED_XMP, + ICC, + JFIF, + jpeg, + marker, + MARKER, + PHOTOSHOP, + segment, + XMP, +} from "../helpers/jpeg-builder.js"; + +describe("JPEG metadata-container inspection", () => { + it("returns a complete container report for minimal JPEG", () => { + expect(inspectMetadata(jpeg())).toEqual({ + format: "jpeg", + size: 4, + inspectionStatus: "container-inspected", + entries: [], + diagnostics: [], + }); + }); + + it("creates an EXIF container entry without decoding TIFF", () => { + const input = jpeg( + segment(MARKER.APP1, concat(EXIF, Uint8Array.of(0x49, 0x49, 0x2a, 0))), + ); + const report = inspectMetadata(input); + + expect(report.entries).toEqual([ + expect.objectContaining({ + namespace: "exif", + name: "EXIF container", + category: "unknown", + privacy: "potentially-sensitive", + source: expect.objectContaining({ + format: "jpeg", + container: "jpeg-segment", + offset: 2, + jpegMarker: MARKER.APP1, + }), + }), + ]); + expect(report.entries[0]).not.toHaveProperty("value"); + }); + + it.each([ + [XMP, "XMP container"], + [EXTENDED_XMP, "Extended XMP container"], + ] as const)( + "classifies standard and extended XMP signatures", + (payload, name) => { + expect( + inspectMetadata(jpeg(segment(MARKER.APP1, payload))).entries, + ).toEqual([expect.objectContaining({ namespace: "xmp", name })]); + }, + ); + + it("does not infer EXIF or XMP from unknown APP1", () => { + expect( + inspectMetadata(jpeg(segment(MARKER.APP1, Uint8Array.of(1, 2, 3)))) + .entries, + ).toEqual([]); + }); + + it("classifies ICC as non-sensitive color metadata", () => { + expect(inspectMetadata(jpeg(segment(MARKER.APP2, ICC))).entries).toEqual([ + expect.objectContaining({ + namespace: "icc", + category: "color", + privacy: "non-sensitive", + }), + ]); + }); + + it("classifies signed Photoshop APP13 as potentially-sensitive IPTC", () => { + expect( + inspectMetadata(jpeg(segment(MARKER.APP13, PHOTOSHOP))).entries, + ).toEqual([ + expect.objectContaining({ + namespace: "iptc", + privacy: "potentially-sensitive", + }), + ]); + }); + + it("does not infer IPTC from unknown APP13", () => { + expect( + inspectMetadata(jpeg(segment(MARKER.APP13, Uint8Array.of(1, 2, 3)))) + .entries, + ).toEqual([]); + }); + + it.each([new Uint8Array(), Uint8Array.of(0xff, 0x00, 0x80)])( + "records COM presence without text decoding", + (payload) => { + const entries = inspectMetadata( + jpeg(segment(MARKER.COM, payload)), + ).entries; + + expect(entries).toEqual([ + expect.objectContaining({ + namespace: "jpeg-comment", + category: "description", + privacy: "potentially-sensitive", + }), + ]); + expect(entries[0]).not.toHaveProperty("value"); + }, + ); + + it("detects JFIF and Adobe internally without adding privacy entries", () => { + const report = inspectMetadata( + jpeg(segment(MARKER.APP0, JFIF), segment(MARKER.APP14, ADOBE)), + ); + + expect(report.entries).toEqual([]); + expect(report.inspectionStatus).toBe("container-inspected"); + }); +}); + +describe("JPEG inspection safety and status", () => { + it("returns partial status and diagnostics for missing EOI", () => { + const input = concat(marker(MARKER.SOI), segment(MARKER.APP1, EXIF)); + const report = inspectMetadata(input); + + expect(report).toMatchObject({ + format: "jpeg", + inspectionStatus: "container-partial", + }); + expect(report.entries).toHaveLength(1); + expect(report.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_MISSING_EOI" }), + ); + }); + + it("enforces a custom JPEG segment limit", () => { + const report = inspectMetadata( + jpeg(segment(MARKER.APP0), segment(MARKER.APP1), segment(MARKER.APP2)), + { limits: { maxSegments: 2 } }, + ); + + expect(report.inspectionStatus).toBe("container-partial"); + expect(report.diagnostics).toContainEqual( + expect.objectContaining({ code: "JPEG_SEGMENT_LIMIT_EXCEEDED" }), + ); + }); + + it("calculates offsets relative to an exact Uint8Array subview", () => { + const embedded = jpeg(segment(MARKER.APP1, EXIF)); + const backing = concat( + Uint8Array.of(0xaa, 0xbb), + embedded, + Uint8Array.of(0xff, 0xd8), + ); + const view = new Uint8Array( + backing.buffer, + backing.byteOffset + 2, + embedded.byteLength, + ); + const report = inspectMetadata(view); + + expect(report).toMatchObject({ + size: embedded.byteLength, + inspectionStatus: "container-inspected", + diagnostics: [], + }); + expect(report.entries[0]?.source.offset).toBe(2); + }); + + it.each([ + jpeg(), + jpeg(segment(MARKER.APP1, EXIF)), + jpeg(segment(MARKER.SOS), Uint8Array.of(1, 0xff, 0x00, 2)), + ])("is deterministic and preserves input", (input) => { + const before = Uint8Array.from(input); + + expect(inspectMetadata(input)).toEqual(inspectMetadata(input)); + expect(input).toEqual(before); + }); + + it.each([ + [Uint8Array.of(0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a), "png"], + [ + Uint8Array.of(0x52, 0x49, 0x46, 0x46, 0, 0, 0, 0, 0x57, 0x45, 0x42, 0x50), + "webp", + ], + ] as const)("leaves %s inspection at format-only", (input, format) => { + expect(inspectMetadata(input)).toMatchObject({ + format, + inspectionStatus: "format-only", + }); + }); +}); diff --git a/tests/unit/jpeg-parser.test.ts b/tests/unit/jpeg-parser.test.ts new file mode 100644 index 0000000..3779bfd --- /dev/null +++ b/tests/unit/jpeg-parser.test.ts @@ -0,0 +1,196 @@ +import { describe, expect, it } from "vitest"; + +import { ByteReader } from "../../src/core/binary/byte-reader.js"; +import { parseJpeg } from "../../src/jpeg/parser.js"; +import { + ADOBE, + concat, + EXIF, + JFIF, + JFXX, + jpeg, + marker, + MARKER, + segment, +} from "../helpers/jpeg-builder.js"; + +function parse(input: Uint8Array, maxSegments = 100) { + return parseJpeg(new ByteReader(input), maxSegments); +} + +describe("JPEG marker parser", () => { + it("parses a minimal SOI/EOI container", () => { + const result = parse(jpeg()); + + expect(result).toMatchObject({ + complete: true, + sawSoi: true, + sawEoi: true, + }); + expect(result.segments.map(({ markerName }) => markerName)).toEqual([ + "SOI", + "EOI", + ]); + expect(result.diagnostics).toEqual([]); + }); + + it("records structural, standalone, and unknown length-prefixed markers", () => { + const input = jpeg( + marker(0x01), + marker(0xd0), + segment(MARKER.DQT), + segment(0xf0), + ); + const result = parse(input); + + expect( + result.segments.map(({ markerName, kind }) => [markerName, kind]), + ).toEqual([ + ["SOI", "standalone"], + ["TEM", "standalone"], + ["RST0", "standalone"], + ["DQT", "image-structure"], + ["UNKNOWN_F0", "unknown"], + ["EOI", "standalone"], + ]); + }); + + it("uses declared lengths that include the two-byte length field", () => { + const input = jpeg( + segment(0xe3), + segment(0xe4, Uint8Array.of(0xaa)), + segment(0xe5, Uint8Array.of(1, 2, 3)), + ); + const result = parse(input); + const applications = result.segments.filter( + ({ kind }) => kind === "application", + ); + + expect( + applications.map(({ length, payloadLength }) => [length, payloadLength]), + ).toEqual([ + [4, 0], + [5, 1], + [7, 3], + ]); + }); + + it("handles repeated marker fill bytes", () => { + const input = concat( + marker(MARKER.SOI), + segment(MARKER.APP1, EXIF, 3), + marker(MARKER.EOI, 4), + ); + const result = parse(input); + + expect(result.complete).toBe(true); + expect(result.segments[1]).toMatchObject({ + markerName: "APP1", + metadataKind: "exif", + offset: 4, + }); + expect(result.segments.at(-1)?.markerName).toBe("EOI"); + }); + + it("classifies JFIF, JFXX, Adobe, and unknown APP payloads conservatively", () => { + const result = parse( + jpeg( + segment(MARKER.APP0, JFIF), + segment(MARKER.APP0, JFXX), + segment(MARKER.APP14, ADOBE), + segment(MARKER.APP1, Uint8Array.of(1, 2, 3)), + ), + ); + const applications = result.segments.filter( + ({ kind }) => kind === "application", + ); + + expect( + applications.map(({ metadataKind, metadataSubtype }) => [ + metadataKind, + metadataSubtype, + ]), + ).toEqual([ + ["jfif", "jfif"], + ["jfif", "jfxx"], + ["adobe", undefined], + ["unknown", undefined], + ]); + }); + + it("does not match a payload signature across the segment boundary", () => { + const partialExif = EXIF.slice(0, 4); + const result = parse( + jpeg(segment(MARKER.APP1, partialExif), segment(0xe3, EXIF.slice(4))), + ); + + expect(result.segments[1]?.metadataKind).toBe("unknown"); + }); + + it("stops at EOI and reports trailing data", () => { + const result = parse(concat(jpeg(), Uint8Array.of(1, 2, 3))); + + expect(result.complete).toBe(true); + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ + severity: "warning", + code: "JPEG_TRAILING_DATA", + offset: 4, + }), + ); + }); +}); + +describe("JPEG scan traversal", () => { + it("skips scan bytes and FF 00 stuffing without decoding", () => { + const input = jpeg( + segment(MARKER.SOS), + Uint8Array.of(0x12, 0x34, 0xff, 0x00, 0x56, 0x78), + ); + const result = parse(input); + + expect(result.complete).toBe(true); + expect(result.segments.map(({ markerName }) => markerName)).toEqual([ + "SOI", + "SOS", + "EOI", + ]); + }); + + it("records restart markers without terminating a scan", () => { + const input = jpeg( + segment(MARKER.SOS), + Uint8Array.of(0x11, 0xff, 0xd0, 0x22, 0xff, 0xd1, 0x33), + ); + const result = parse(input); + + expect(result.complete).toBe(true); + expect(result.segments.map(({ markerName }) => markerName)).toEqual([ + "SOI", + "SOS", + "RST0", + "RST1", + "EOI", + ]); + }); + + it("resumes normal traversal and supports multiple scans", () => { + const input = jpeg( + segment(MARKER.SOS), + Uint8Array.of(0x11, 0x22), + segment(MARKER.DHT), + segment(MARKER.SOS), + Uint8Array.of(0x33, 0xff, 0x00, 0x44), + ); + const result = parse(input); + + expect(result.complete).toBe(true); + expect(result.segments.map(({ markerName }) => markerName)).toEqual([ + "SOI", + "SOS", + "DHT", + "SOS", + "EOI", + ]); + }); +});