[JaredP94]

Description

llama.cpp PR 3 of 4: a Go-owned model catalog, verified downloads with progress and asynchronous cancel, installed inventory, and owned-model delete. Desktop and TUI hub screens remain PR 4.

• New engine:model-catalog {query?: string} RPC: curated ggml-org, bartowski and unsloth (50 repos each); public search at most 50 results, 100 query characters, 20 cached queries, six-hour TTL and 20-second aggregate request deadline (broker 25 seconds; future clients must allow 30 seconds). Only public, ungated text-generation GGUF repositories with one unsplit Q4_K_M candidate; IDs owner/repo:Q4_K_M. Strict bounded ASCII identifiers/filenames and bounded metadata bodies.
• Existing engine:action gains llama.cpp pull_model, list_downloaded, and delete_model; existing pull/cancel/remote RPCs and progress streams are reused. Installed inventory combines the ownership manifest with a bounded no-follow regular-GGUF scan; manually supplied files outside the reserved generated-filename namespace are selectable but never PAIR-deletable. Catalog IDs also resolve for --model. Installed scans honor a five-second context between bounded per-file metadata reads (broker ten seconds; future clients fifteen seconds).
• Resolve the exact repository commit, LFS SHA-256 and size before transfer. Enforce 64 GiB per file, a 256 MiB free-space reserve, dial/TLS/header/idle-read bounds, bounded headers and HTTPS-only credential-free redirects; record final host. Verify size, checksum and single-file GGUF metadata, fsync, then rename without overwriting existing destinations.
• Commit identity: unpredictable exclusive openat staging records dev/ino immediately. After no-replace rename, fstatat(AT_SYMLINK_NOFOLLOW) must match the verified descriptor's dev/ino and expected size before directory fsync and manifest commit. A mismatch reports an error, writes no ownership record, remains unselectable, and is left untouched. The manifest records dev/ino, size and SHA-256. Selection and the actual inherited launch descriptor must match the recorded dev/ino/size; post-commit replacements are refused without re-hashing up to 64 GiB. Generated names and record matches are case-insensitive to prevent filename-case aliases on default macOS filesystems.
• Cleanup identity: immediately before unlinkat, no-follow fstatat must match the operation's recorded dev/ino (plus recorded size for completed models). Substituted temp/completed entries are left untouched. Skipped cleanup produces a clear error through Show model download progress and allow confirmed cancellation #50 progress/state, never successful cancellation. Manifest staging and rollback use the same checked cleanup. The temp descriptor stays open through cleanup.
• Streaming space: recheck fstatfs on the open transfer fd every 256 MiB or five seconds, whichever occurs first at a streaming boundary. Space must cover the remaining file bytes plus the 256 MiB margin; a drop aborts with “not enough disk space” and identity-checked temp cleanup.
• Cancel accepts immediately; the terminal event confirms release and cleanup. Restart interruption and operation-ID fencing reuse Show model download progress and allow confirmed cancellation #50. Concurrent cache mutations/duplicates reject rather than queue; no state lock is held over network/file waits. Delete requires a stopped engine and no lifecycle operation. Cache mutations use pinned openat/no-follow directory descriptors.

Intentional difference from NVIDIA/Personal-AI-Router@6d2d67f: PAIR downloads in Go into {install_dir}-models, not through llama-server's router download API. No blocked download launch option is enabled. Only the operation's identity-matched temp is eligible for cleanup; transfer URLs and bodies are never logged.

Cancellation-cause follow-up (current head)

Signed-off commit 1613ca01081d6273fb94b54e18bf5e632f35c14b replaces order-based failure fencing with the shared pull tracker's cause-based rule: only an operation's requested user cancel plus an error wrapping context.Canceled reports cancelled without a pull-failed engine error. Unrelated checksum/size, HTTP, disk, identity and manifest failures remain errors regardless of cancellation during teardown. Metadata/download request and body-read cancellation causes are preserved without exposing URLs; plain read errors after cancellation teardown use the cancelled operation context. Filesystem helper causes remain wrapped. Successful committed downloads still disable further cancellation.

New tables: TestLlamaMetadataPreservesCancellationCause, TestLlamaDownloadClassifiesByCause, TestLlamaDownloadCancellationCausePhases, and TestLlamaDownloadChecksumCancelDuringTeardown. Metadata cause loss and real failures incorrectly relabelled as cancellation were reproduced against the previous code before the fix. The existing cancellation-phase fixtures now wrap their simulated cancellation causes rather than return unrelated plain errors. Existing llama.cpp regression tests pass.

Local validation: engine-manager built with go test -race -c -o /tmp/pair-cause-final.test .; the full binary passed twice with -test.timeout=300s. Darwin/Linux/Windows vet, gofmt, diff checks and SPDX (1548 checked, zero missing) pass. No fake-engine/llama-server survivors; the implementation worktree is removed. No additional version bump or CHANGELOG change.

Final-head CI 42412304611, head 1613ca01081d6273fb94b54e18bf5e632f35c14b: all six jobs pass, without a rerun. Downloaded the Linux cross-process artifact with gh run download into new empty /var/folders/sb/5bf7gsds34v599d8y8x4pfb00000gn/T/pair-ci62-final-96otanlq; python3 -I /tmp/pair-review/scripts/noskip.py <dir>/cross-process.jsonl reports 127 passing tests, zero skips/failures. These source/CI results do not recertify the earlier packaged live run below.

Scope

Services, generated desktop contracts and a bridge parameter type/overload in the original PR. Review fixes touch only engine-manager code/tests and LLAMACPP.md. No desktop/TUI screens, real engine installation, real model download, or CHANGELOG edits.

Version decision: retain the existing unreleased engine-manager 1.5.0 and broker 2.6.0 additive bumps. No additional version bump; product/installer and other binaries are unchanged.

Validation

Latest review fixes: macOS arm64, isolated /tmp/pair-work/llamacpp-review, based on 06d5565; signed-off commits pushed normally:

• c15d066a: commit and launch identity.
• d8806cff: identity-checked cleanup/deletion.
• 8babe5f3: streaming disk-space checks.
• 2c98dbf0: small follow-up for a filename-case selection bypass found during verification of the first fix.
• 590690c5: preserve close-on-exec atomically when duplicating the pinned directory for launch identity checks.

Evidence on the final source:

• Full engine-manager and ui-broker race test binaries pass, built with go test -race -c -o <tmpfile> . and run directly with -test.timeout=300s; broker nodemetrics, relay and workloadstore race binaries also pass.
• Native Darwin, Linux and Windows go vet ./... pass for both modules; gofmt and git diff --check are clean. SPDX: 1546 checked, zero missing.
• TestLlamaDownloadTempSwapAtCommit, TestLlamaRecordedReplacementRefusedAtLaunch, TestLlamaDownloadCleanupLeavesSubstitutedTemp (failure/cancel), and TestLlamaDownloadFreeSpaceDropsDuringStreaming (byte/time interval) all fail against 06d5565 and pass after the fixes. Commit mismatch checks no ownership record, no selection/launch, and preservation of the mismatching file. TestLlamaDeleteLeavesReplacedCompletedModel and TestLlamaRecordedIdentityCannotBeBypassedByFilenameCase pass; the case regression also failed before its guard was added.
• Tests use generated fixtures and local HTTP(S) servers only. No real model/engine changes. No fake-engine/llama-server survivors after local suites or final verification. Both implementation and baseline worktrees have been removed.
• Final-head CI run 42404010329, attempt 1, head 590690c5b295e33385fc9b6a67f4a098873e25d2: all six jobs pass, including both Go-module/race jobs, both cross-process jobs, desktop and SPDX. Downloaded the Linux artifact with gh run download into new empty /tmp/pair-llamacpp-review-ci.1zI46g; python3 -I /tmp/pair-review/scripts/noskip.py /tmp/pair-llamacpp-review-ci.1zI46g/cross-process.jsonl reports 127 passing tests, zero skips/failures, and the package terminal event is pass. No rerun was needed.

Earlier PR validation (before these review fixes): desktop typecheck, zero-warning lint, 945 unit tests, dead-code and service-contract checks passed. Prior CI run 42399660708 passed all six jobs on attempt 2, with 127 Linux cross-process tests and zero skips; attempt 1 had only the unchanged TestModelsHTTPEnrichment discovery flake. Those earlier results are not claimed as final-head certification.

Risk

The cache lives in the user's own app data: entry swaps require another process running as the same user. macOS has no rename-by-descriptor or unlink-by-descriptor, so perfect atomicity is unavailable. Post-rename identity checks fail closed, and cleanup checks occur immediately before unlinkat; a same-user replacement after the last check leaves the minimum OS-allowed check-to-use/unlink window. This is documented in code and LLAMACPP.md, not presented as an atomicity guarantee. Launch validates the descriptor inherited by the child, so later pathname replacement cannot redirect its opens. Identity checks detect replacement, not writes in place by an already-authorized same-user process.

No live model/inference or packaged-asset certification. The engine remains macOS arm64-only. A crash between model rename and manifest commit may leave an unrecorded reserved-name file, which PAIR will neither select nor delete; abandoned crash temps are not automatically reclaimed. Inventory checks metadata and recorded identity/size, not a fresh full-file checksum. Streaming checks happen at read/write boundaries; a blocked read writes no additional bytes. Filesystem syscall latency itself is not context-preemptible. Desktop/TUI consumers must use a 30-second catalog deadline in PR 4.

Checklist

• I have read the Contributing Guidelines.
• Every commit is signed off (git commit -s).
• New or existing tests cover the change.
• Relevant documentation is updated.
• I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
• I recorded validation commands and results above.
• I applied the component version policy and described user-visible changes above.

Live check

2026-11-07 rerun, head 1613ca01081d6273fb94b54e18bf5e632f35c14b, packaged macOS arm64 build on the owner's macOS 27.2 arm64 machine (Python 3.9.6). a–e PASS; f FAIL only on the strict empty-cache requirement: after successful model deletion and engine uninstall, the surviving cache contained the empty ownership manifest. Tests stopped at that finding; the owner's published install was restored and verified.

The previous live run at 590690c found cancellation during Hugging Face metadata resolution incorrectly ending as error (model metadata request failed). That cancellation-classification bug is fixed in 1613ca0 and passed this packaged rerun: immediate cancel was accepted and terminal progress/result were cancelled, with no pull-failed engine error.

Built in detached /tmp/pair-work/live62b using npm ci --no-audit --no-fund && npm run -s build:mac:arm64; no node_modules symlink and no build in the main checkout. ZIP size 200,591,852 bytes; local and remote shasum -a 256 matched 1f64f066683c3cc091eb4f9a7225691d4caaa0ed9c26cf59c4699e3f60ca1eea. Reused the previous remote driver/control/phase/tree-verification scripts in fresh ~/pair-live62b, without modifying previous runs or backups. Fresh app/app-data backups made with ditto, checked with diff -rq (Singleton*/logs excluded for app data) and bytewise regular-file, symlink, mode and structure verification. The packaged broker ran standalone with the flags the new desktop passes.

Step	Result	Live evidence
a	PASS	13 workers running; proxy listeners 11434/1234/1236/1237 verified. Saved engine settings unchanged: Ollama r8 11435/11434, LM Studio r1 1235/1234, oMLX r1 1235/1236; llama.cpp defaults r1 8082/1237.
b	PASS	PAIR installed and started llama.cpp healthy; /props HTTP 200 with role router. Catalog returned 90 entries: ggml-org 7, bartowski 50, unsloth 33; selected model present.
c	PASS	Automatic cancel at first pulling progress event; {accepted:true} acknowledged 0.665 ms after that event. Terminal stage cancelled, result {"status":"cancelled"}; engine errors empty; cache listing empty before and after, no temp file.
d	PASS	Same model completed on the second and final pull: 103 progress events, terminal done. Cached file 101,016,128 bytes, remote SHA-256 matched e3131339bf4e8065265593d4fd8f7bb7ff2d3abff1edb5618aa1197b89cad9f5; read-only ownership manifest and installed-models inventory agreed.
e	PASS	Applied --model ggml-org/SmolLM2-135M-GGUF:Q4_K_M; settings succeeded and engine restarted healthy. Proxy 1237 /v1/models HTTP 200 listed ID /dev/fd/3. Non-streaming and streaming chat completions and /v1/messages with anthropic-version: 2023-06-01 all HTTP 200. Stream emitted 14 events and [DONE]; recorded only 12 content characters per response.
f	FAIL (empty-cache requirement)	Original llama.cpp settings restored exactly (launchText:""), healthy router restart confirmed. PAIR requires stopped-engine deletion, so stopped it before delete_model. Delete returned {"ok":true}, inventory became [], GGUF disappeared; stop/uninstall succeeded and install directory disappeared. Surviving llamacpp-models was not empty: only .pair-models.json (13 bytes), content {"models":[]}.

Exact failed check: Expected empty surviving model cache; found [{"name": ".pair-models.json", "size": 13, "content": "{\"models\":[]}"}]. This is a checker assertion about observed PAIR residue, not a JSON-RPC error: PAIR returned success for delete and uninstall. No GGUF or download temp remained. No further product tests or remediation were attempted after the finding.

The only model used was the prior run's selection, ggml-org/SmolLM2-135M-GGUF:Q4_K_M (SmolLM2-135M-Q4_K_M.gguf), 101,016,128 bytes, expected SHA-256 above; exactly two pull attempts (one cancelled during metadata, one completed). The committed cache filename was 12926887e1d5c71725345eed940cb84b214e23fad6dc6cbe8e2063d4edbdf18e.gguf; manifest recorded matching size/hash and file identity. Only PAIR executed llama.cpp, installed/uninstalled only llama.cpp, and managed the model lifecycle. No owner engine directories or OS security settings were changed.

Restoration: clean broker shutdown returned null and left no nvpair/PAIR service or llama-server processes. Moved current app data aside, restored /Applications/PAIR.app and app data from ~/pair-live62b/backup using ditto, verified diffs, then relaunched with open -a /Applications/PAIR.app. Final state: published 0.9.0, broker + 12 workers alive as broker children, oMLX proxy 1236 /v1/models HTTP 200 (7 models), original saved settings and revisions restored, no .gguf files or llama.cpp paths under app data, no llama.cpp survivors. Removed ~/pair-live62b/incoming and ~/pair-live62b/replaced-appdata; kept the fresh backup and all older backups. Local /tmp/pair-work/live62b removed.

Evidence: /tmp/pair-review/live-llamacpp/evidence-62b.jsonl (incremental checks plus archived driver requests/responses/progress). Checker-only corrections: the initial listener probe wrongly required an inventory HTTP 200 before engine availability; replaced it with TCP/listener verification. Cancellation verification initially referenced state instead of the wire field stage; corrected verification without repeating that pull. Neither script error is a PAIR failure or an additional download attempt. This run verifies only the tested local macOS arm64 package and flows, not other platforms or untested scenarios.

[JaredP94]

Merge note: the live re-run's step f was recorded as a failure only because the check expected the surviving model cache to be completely empty. Uninstall deliberately keeps the model cache (PR 1's rule), and what remained was PAIR's own empty ownership record (.pair-models.json = {"models":[]}) after the model was deleted. That's expected. Steps a–e passed, including the cancel fix, the verified download (size and SHA-256), and inference through port 1237.

Found during the run and fixed separately: llama-server reports a model launched from the verified descriptor as /dev/fd/3, so clients would have to request that name. A follow-up PR passes PAIR's own model ID to llama-server as its alias.