Skip to content

feat(envelope): qualify hostile and production resource envelopes (#19) - #107

Open
mberrys wants to merge 12 commits into
devfrom
cc/issue-19-resource-envelopes
Open

mberrys wants to merge 12 commits into
devfrom
cc/issue-19-resource-envelopes

Conversation

@mberrys

@mberrys mberrys commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Implements #19 (L01-05, "Qualify hostile and production resource envelopes"). Session 11 (#542) left the envelope incomplete, with 0 of 6 fixtures measured. There were two causes: PdfTool benchmark never measured preflight or recovery, so every record was hardcoded incomplete, and the large fixtures only existed in an external DIV2K corpus that CI can't reach. This PR makes the strict matrix runnable and passable on hosted Linux and Windows runners. Close #19 with gh issue close once the qualification run's evidence is recorded.

PdfTool (no LoopLibCore changes, so no protected paths are touched):

  • benchmark --profile <profile.json> runs a measured preflight phase through the existing pdf::inspectPreflightFile. An envelope is now complete only when preflight ran, every page rendered, and there was no cancellation or budget exhaustion. Every other case keeps a named incomplete_reason.
  • Rendering runs in slices of 4 × the rasterizer count and checks for cancellation between slices. An interrupt now stops the run within one slice instead of after all 10,000 pages.
  • Windows now handles SIGBREAK. CTRL_BREAK_EVENT is the only console interrupt that reaches a child started in its own process group, and PdfTool previously died on it without writing an envelope.
  • The new narrow option flag BenchmarkPreflightProfile exposes only --profile on benchmark, not the full preflight option set.

Qualification tooling:

  • scripts/resource_envelope/synthetic_workload.py deterministically generates the office (≈2.0 MB), image-heavy (≈501 MB) and 10,000-page (≈62 MB) fixtures, using SHAKE-256 noise images stored with FlateDecode. The pathological-vector and transparency-spots fixtures come from the existing generator, and their digests match the Session 11 records. The 10,000-page fixture uses a new synthetic-image-heavy workload whose caps equal the DIV2K ones.
  • run_matrix.py:
    • Every run gets the preflight profile, and the workload comes from the manifest.
    • Crashes (exit codes outside PdfTool's defined codes, or InternalError) and timeouts fail the record outright. A non-zero exit or missing envelope only flags it.
    • A separate cancellation probe interrupts one run. A recovery probe then times a fresh process reopening the fixture and rendering its first page (recovery_ms).
    • A hostile lane runs every UnitTests/testdata/budget_exhaustion/ PDF. Each must end in a contained exit code within its timeout and under the resident ceiling.
    • --strict now requires the probe and the hostile lane.
  • build_resource_envelope_evidence.py builds schema-2 evidence carrying the CI run id and URL. validate_resource_envelope_evidence.py accepts passed only when both platforms measured everything on one candidate SHA. Schema 1 (Session 11) is unchanged.
  • .github/workflows/resource-envelope-qualification.yml covers Linux and Windows builds of PdfTool, fixture generation, the strict matrix (3 repetitions), and an evidence job. It runs on workflow_dispatch and on PRs touching envelope paths, so normal CI stays fast.

Decisions made with the requester: synthetic fixtures generated in CI instead of the DIV2K bundle; recovery_ms defined as reopen-after-cancel; qualification in a separate workflow.

Release changelog

Topic fragment: changes/cc-issue-19-resource-envelopes.md (Category: added). Refs #19.

Proof

  • check-change.py --base origin/dev: the source checks (changelog, source integrity, catalogs, policy adapters, architecture contracts, qt runtime) pass locally. The native build:* and focused_tests checks fail locally only because this worktree has no configured build directory; configuring one needs approval under AGENTS.md. The C++ is compiled and tested only by CI on this PR.
  • One changes/<sanitized-head-branch>.md fragment, plus changes/cc-issue-19-resource-envelopes.evidence.yaml
  • Changed behaviour has tests that fail without the change:
    • Python: 50 resource-envelope tests pass locally (probes, crash and timeout handling, the hostile lane, generator determinism and well-formedness, evidence build and validation).
    • C++: UnitTestsPdfToolContract gains benchmarkWithoutPreflightProfileIsIncomplete and benchmarkWithPreflightProfileIsComplete. These run in CI only.
  • Protected-path or contract change: none. docs/RESOURCE_ENVELOPE_BUDGETS.json gains a synthetic-image-heavy workload with the same caps as DIV2K, and no cap is loosened. benchmark gains the --profile option. The matrix JSON schema version goes 2 → 3 (it adds probe and hostile sections), and the evidence schema adds version 2.
  • Acceptance evidence is still to come. The resource-envelope-qualification run on this PR produces the measurements. They get committed under docs/evidence/issue-19-resource-envelope/, with the run id, as a follow-up commit on this PR. If the run fails a budget (the 500 MB fixture against the 768 MiB resident ceiling is the most likely), the disposition says so rather than being tuned away.

Local fixture check: the full bundle builds in 18 s and every fixture passes qpdf --check.

Internal logic (touched behavior-bearing code)

  • Guard clauses: a bad --profile fails with InvalidInvocation before any work, and a tampered hostile fixture or wrong manifest digest fails before running
  • Untrusted input is parsed once at the boundary: manifests are validated in _load_fixture_manifest, and profiles through importPreflightProfile
  • Invalid state stops before publication: an unmeasured phase never becomes complete, and -1 is never promoted to a measurement
  • Names carry the domain intent, and comments explain rationale

Quality pass

  • Redundant or explanatory comments that do not match the file's style removed
  • Abnormal defensive checks and broad try/catch blocks removed where a trusted upstream boundary already guarantees the invariant, with real boundary and safety checks kept
  • No any or equivalent cast added only to suppress a type error
  • Python imports stay at file scope unless a local import is required
  • Generated boilerplate, needless wrappers, and local-style drift removed
  • Validation, security, cancellation, provenance, and failure handling preserved

Quality summary (1-3 sentences):

An early draft added a cancellation hook to PDFRasterizerPool in LoopLibCore/sources (a protected path). I replaced it with PdfTool-side sliced rendering so no protected interface changes. The old in-fixture cancellation path in run_matrix.py is replaced by the separate probe, and the duplicated post-hoc identity checks in run_matrix are folded into one helper.

Security and rollback

  • Untrusted input validated at the trust boundary; hostile PDFs run only through PdfTool's existing budgets, and the lane only observes the disposition
  • Rollback: revert the PR. The new workflow is path-filtered and manual, the benchmark's default behaviour without --profile is unchanged, and Windows SIGBREAK handling is the only behaviour change outside benchmark

Docs

  • Updated docs/RESOURCE_ENVELOPE.md and docs/RESOURCE_ENVELOPE_QUALIFICATION.md

Self-review (BSP-002 §4.3)

  • Reviewed in the diff view, not the editor, at least 30 minutes after the final commit; overnight if the change touches security-sensitive code, data handling, or public API surface

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

mberrys and others added 5 commits September 27, 2026 18:26
PdfTool benchmark gains a measured preflight phase (--profile), renders in cancellable slices, and handles SIGBREAK on Windows. A synthetic fixture generator, cancellation/recovery probes, a hostile corpus lane, schema-2 evidence, and a hosted Linux/Windows qualification workflow make the strict matrix runnable in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QFile::atEnd() is true immediately for procfs files, so
currentRssHighWaterBytes() always returned -1 on Linux. That left
preflight_high_water_bytes at -1, failing
benchmarkWithPreflightProfileIsComplete and flagging every Linux matrix
record as unmeasured.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NT3L5ao8UyQuTPPKySRDcP
The job log only carried summary counts, so a failing hosted run could
not be diagnosed without downloading the matrix artifact.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NT3L5ao8UyQuTPPKySRDcP

mberrys commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

Status on head e3c2626: agent-fast / build is now green (Linux /proc/self/status RSS reader fixed). The linux and windows resource-envelope qualification jobs still fail: the strict matrix reports 0 measured fixtures, a failed cancellation/recovery probe and a 6/7 hostile lane, and the job log only prints summary counts. d7eeebb makes run_matrix.py print each failing fixture, probe and hostile case with its reason to the job log. I'll fix the specific causes from the next run's output. The evidence job just follows those two.


Generated by Claude Code

Exit code 5 (PartialOutput) alone does not say which page failed or why.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NT3L5ao8UyQuTPPKySRDcP

mberrys commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

Linux qualification results for d7eeebb (first run with per-record reasons):

  • office-2mb, pathological-vector, transparency-spots: every run exits 5 (PartialOutput), so at least one page reports a render error. The log doesn't say which. c9af5df now logs the render-error table and stderr tail.
  • image-heavy-500mb: no envelope on any run (benchmark-envelope-missing). c9af5df logs its stderr too.
  • ten-thousand-page: all 3 runs hit the 900 s timeout, against a 120 s wall_time_ms budget. The cancellation probe on the same fixture also never stopped after the interrupt. It isn't yet known whether the time goes to the preflight phase or to rendering.
  • Hostile raster-probe-pixel-budget: exit 5 with RSS 15.5 GB against the 768 MiB ceiling. This only shows up now that RSS is measured on Linux (e3c2626). It looks like a real finding rather than tooling, and per the PR description it shouldn't be tuned away.

Next: read the render-error and stderr output from the run on c9af5df, then decide which of these are fixable in this PR and which are qualification findings to record.


Generated by Claude Code

… DPI

At 300 DPI a Letter page image is 33.7 MB and each rasterizer holds one,
so 8 rasterizers exceed the 128 MiB raster-tile-cache pool: the extra
pages are rejected as budget-exceeded and every run exits PartialOutput.
Three fit. Also stop repeating a fixture after its first timeout.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NT3L5ao8UyQuTPPKySRDcP

mberrys commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

Linux results for c9af5df, and what 648d160 changes:

  • Exit 5 on every measured fixture: benchmark renders at the default 300 DPI (33.7 MB per Letter page) with 8 rasterizers, but the raster-tile-cache pool is 128 MiB. Only three pages fit, and the rest are rejected as budget-exceeded and left unrendered. Session 11 recorded the same thing (23 of 677 pages). 648d160 pins 3 rasterizers (101 MB), keeping 300 DPI. It also stops repeating a fixture after its first timeout.
  • ten-thousand-page timeout and the cancellation probe that never stopped: cause unknown. It may be the preflight phase over 10,000 pages rather than rendering. The next run should show more.
  • Hostile raster-probe-pixel-budget (15.5 GB RSS vs 768 MiB): left failing as a qualification finding. It looks like the default thin-parts raster probe running without an effective pixel cap. The fix would be in the preflight engine or profile, so it needs a separate decision.
  • The Windows run on c9af5df was cancelled by this push, so Windows reasons will come from the new run.

Generated by Claude Code

mberrys and others added 5 commits September 28, 2026 19:28
The rasterizer was released before the image was processed, so with more
worker threads than rasterizers, more images than rasterizers held
raster-tile reservations at once. The hosted linux qualification run shed
7 reservations at 3 rasterizers (101 MB of the 134 MB pool used) and every
render exited PartialOutput with budget-exceeded. A budgeted run now
releases the reservation and then the rasterizer after processImage;
unbudgeted callers keep the early release.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…llation

PDFEvidenceCollectSettings and PDFColorInventorySettings gain an optional
operationControl. The colour inventory polls it per page and reports
cancelled; the evidence collector polls it per page and returns an
incomplete graph with incompleteReason "cancelled". PreflightEngine passes
its operation control through and reports errorCode "cancelled" instead of
evidence-incomplete, so an interrupt during the benchmark's preflight phase
stops within one page instead of after the whole document.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Preflight renders and walks every page it covers, about 0.5-0.7 s per page
on a hosted runner, so a full pass over the 10,000-page fixture cannot fit
any practical timeout. benchmark gains --preflight-page-last <n>, which
limits the preflight phase to pages 1..n while rendering still covers every
page. PDFEvidenceCollectSettings and PDFColorInventorySettings gain
pageIndices so the render and content walk skip pages the profile scope
already discards. run_matrix.py passes 256 for fixtures above 1,000 pages
and records it as profile.preflight_page_last.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e render-only

The hostile raster-probe-pixel-budget case reached 15 GB RSS on both hosted
platforms: the colour inventory probes each page at 150 DPI with several float
bitmaps per pixel and no size limit. PDFColorInventorySettings gains
maxProbePixels (2.5 million); larger pages are probed at a proportionally
lower DPI.

The cancellation probe ran with the preflight profile, whose document-wide
setup does not poll for cancellation, so latency was 11-16 s against a 5 s
policy. The probe now interrupts a render-only run, like the recovery probe.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants