Skip to content

Add llama.cpp model management to desktop and terminal clients - #64

Merged
JaredP94 merged 18 commits into
mainfrom
feat/llamacpp-desktop
Nov 8, 2026
Merged

JaredP94 merged 18 commits into
mainfrom
feat/llamacpp-desktop

Conversation

@JaredP94

@JaredP94 JaredP94 commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Description

Expose the Go llama.cpp model hub in desktop and TUI, following #60, #61, #62 and #63. Resolve the requested verified-peer installed-list/delete parity by extending the existing peer inventory and authenticated remote operation, not by introducing a parallel peer API.

Scope

  • Cache-error isolation follow-up: a llama.cpp cache read/validation failure returns HTTP 200 with healthy served/loaded inventory intact and the fixed, bounded downloadErrorsByEngine.llamacpp: "cache_unavailable" code (never filesystem details or absolute paths). Manual peers, scanner refreshes, broker discovery and both clients preserve this separate diagnostic. Only llama.cpp's installed-model pane says “Couldn't read llama.cpp's downloaded models”; recovery clears it. Desktop does not label an unreadable cache empty. Genuine whole-inventory failures keep their prior behavior.
  • Reuse engine-manager GET /v1/models → pinned cluster mTLS → scanner/verified manual-peer enrichment → broker discovery snapshots → desktop bridge and TUI. Add bounded downloadedByEngine.llamacpp rows containing catalog ID/file-name, cache-relative file, size, digest and verified ownership, including while stopped. Keep served-model routing inventory separate. Previews and withdrawn proofs receive no cache inventory.
  • Desktop projects installed peer rows and owned deletion. Running-engine Delete is disabled with a stop-engine reason; service refusal/error reasons remain visible. The downloaded-model bridge reads verified discovery for peers, never the local cache. Remote deletion is awaited through the existing engine:remote-delete-model → pinned-mTLS POST /v1/models/delete → engine-manager identity-checked delete_model path.
  • TUI m lists installed peer cache files and d confirms owned deletion through the same remote method. Running/unowned guards remain; stale target/catalog responses cannot overwrite the current pane. Companion commits fix pending target state and cover desktop startup-picker edge cases.
  • Retain publisher-grouped Go catalog, bounded public search, existing progress/cancel/retry tracking and revisioned startup-model/router-mode preview/apply. Catalog IDs remain names; descriptors are not names. No new polling loop.
  • Startup-model pickers remain local, as with other engines. Desktop peer Settings arguments still accept a known installed catalog ID. This is not a claim of a peer startup picker.
  • Component READMEs, getting-started, engine-lifecycle, terminal-interface, known-issues and release-readiness guidance updated. Runtime downloads are not bundled dependencies. CHANGELOG and root release claims are untouched.
  • Version decision relative to v0.9.0: retain already sufficient unreleased minor bumps: engine-manager 1.5.0 (from 1.4.0), scanner/manual-nodes 1.4.0 (from 1.3.0), broker 2.6.0 (from 2.5.0), TUI 1.4.0 (from 1.3.0). All transitive shared/noderec binary importers were also checked against v0.9.0 and already have sufficient unreleased bumps; llamacpp-proxy is a new 1.0.0 component. No extra service/product/desktop release version bump.

Validation

The existing /tmp/pair-work/fix64-cache worktree on feat/llamacpp-desktop was used. Its own existing node_modules is a real directory, not a symlink. No cross-process suite was run locally.

  • Desktop: typecheck, zero-warning lint, 966 unit tests / 98 files, dead-code, generated service-contract write/check, and licenses check pass (296 dependencies / 54 Go modules). Regeneration leaves the generated method listing unchanged.
  • Go: fresh go test -race -c -o <tmpfile> . binaries for every test-bearing package in engine-manager, manual-nodes, scanner, broker, TUI and shared; run directly with -test.timeout=300s -test.count=1 under external QUIT timeouts. All pass. Darwin/Linux/Windows vet pass for all six modules; gofmt/diff checks are clean. SPDX: 1567 checked, zero missing.
  • Observed red→green with only production fixes temporarily removed/restored: TestLlamaCacheFailurePreservesOtherEngineInventory returned HTTP 503 over pinned mTLS without the fix; TestPinnedManualPeerCacheErrorPreservesServedInventory kept the peer verified but lost both models and both engine-up flags. With the fix, the real HTTP producer keeps both served engines and returns the separate cache diagnostic.
  • TestPinnedScannerCacheErrorPreservesServedInventory exercises that same real producer over pinned mTLS, scanner refresh publication and recovery. Manual-peer and scanner tests retain their distinct existing whole-inventory-outage behavior.
  • Desktop projection/rendering and TestLlamaPeerCacheErrorIsEngineScoped were observed red without their production changes and green restored, preserving Ollama/LM Studio models and showing the message on llama.cpp only. A further red→green desktop regression prevents the false “No model/weight files downloaded” claim during a cache failure.
  • Broker tests cover scanner projection, manual peer discovery + relay, error-only recovery publication and proof retirement. The retained shared/engineinventorytest fixture builds the real inventory producer against isolated fake engines; only test files import it, and it has SPDX headers.
  • Retained the WIP's required inventory types, relay/cache/withdrawal hops, component docs and regressions; no unrelated runtime changes were found. Replaced its WIP history with logical signed-off service/client commits.
  • Final head 6f2e04dd46b2b254e146d7af1ad4129fefcefbdf, CI run 42457822517, attempt 1: all six jobs passed. Downloaded cross-process-ubuntu-latest with gh run download into fresh empty /tmp/fix64-ci-linux.C8lJuX. python3 -I ~/.pair-review/scripts/noskip.py /tmp/fix64-ci-linux.C8lJuX/cross-process.jsonl reports 127 passes, zero failures/skips. An independent all-event scan of the same run's Linux artifact found zero skip/failure events and a successful package terminal event. No cross-process suite was run locally.

Risk

No real engine installation/model download, packaged GUI, live-peer inference or signing/notarization certification was performed. Installed inventory still has the existing discovery-refresh delay after peer mutations. Ownership is rechecked at deletion on the owning node, not trusted from UI metadata. Cache failures now leave healthy served inventory available with an engine-scoped diagnostic; genuine whole-inventory failures preserve the previous manual-peer/scanner behavior. Startup setting changes may restart an engine after confirmation. Packaged install/update/rollback, download/cancel/shutdown, inference/stop/restart and deletion smoke tests remain maintainer release gates. No independent review approval is claimed.

Checklist

  • I have read the Contributing Guidelines.
  • Every commit is signed off (git commit -s).
  • New/existing service, desktop and TUI tests cover the change.
  • Relevant component and release documentation is updated.
  • I checked changed files/messages for private data and generated artifacts.
  • Existing appropriate unreleased component versions are retained and explained above.
  • Peer installed-cache list/delete gap resolved; local-only startup pickers documented.
  • All six final-head CI jobs and Linux zero-skip artifact verified.

@JaredP94
JaredP94 marked this pull request as draft November 7, 2026 22:44
Reuse the model hub, download rows and engine settings editor for the Go-owned llama.cpp cache. Show publisher groups, search errors, confirmed cancellation, safe stopped-engine deletion and cache-relative startup selection. Label the unavailable peer installed-cache listing rather than presenting served inventory as installed models.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Expose catalog search, installed cache inventory and model downloads through the existing table and operation tracker. Preview startup model changes with node-owned settings and confirm before Apply. Require a stopped engine and owned cache entry for deletion, and state the peer installed-list contract limitation. Retain TUI 1.4.0, already minor-bumped relative to v0.9.0.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Bypass the generic Hugging Face name prettifier for llama.cpp rows so publisher and quantization remain part of the displayed catalog ID. Cover installed and hub rows with red-green regressions, and never render an inherited descriptor as a model name.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Document catalog search, download cancellation, startup selection and stopped-engine deletion in getting started, lifecycle and terminal guides. Keep missing peer cache inventory and packaged smoke testing as explicit open gates. Record runtime-downloaded engines and models outside the bundled dependency inventory, and regenerate the canonical notices without adding unbundled components.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Let Go own catalog cache expiry instead of caching llama.cpp results for the renderer lifetime. Clear TUI action rows while switching catalog and installed inventories, fence late results, and keep served-inventory pushes from invalidating the independent catalog. Regression tests failed before these fixes and pass with the full desktop and race-enabled TUI suites.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Await the existing local delete operation before refreshing the installed cache instead of acknowledging it before Go finishes. Clear the optimistic model action when that acknowledgement settles so a stopped engine does not leave a stale deleting row until the timeout. Existing engines retain their fire-and-forget command paths; bridge and renderer regressions cover the completion boundary.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Abandoned llama.cpp requests could leave peer inventory loading indefinitely or replace a reopened pane with an old catalog. Clear target-local loading state and invalidate requests when toggling the models pane. Cover delayed cache and catalog responses across target and pane transitions. Keep the existing unreleased TUI 1.4.0 version.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Exercise selecting No model through preview and apply, preserving unrelated arguments while removing the saved model. Cover a deleted startup model's preview validation error and delayed inventory arriving after leaving and returning to a device.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Extend the existing pinned peer inventory and discovery relay with bounded downloaded-file metadata, separate from served-model routing. Preserve cache ownership through scanner and verified manual discovery, and strip it from previews and retired proofs. Remote deletion continues through the authenticated engine action and its identity and running-engine guards.

Retain the existing unreleased minor versions relative to v0.9.0: engine-manager 1.5.0, scanner and manual-nodes 1.4.0, and broker 2.6.0.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Project the existing discovery cache metadata into peer model rows instead of the served-only inventory. Keep ownership-based delete gating, explain the stopped-engine requirement, and await remote deletion through the existing authenticated operation. The installed-cache bridge reads verified peer discovery without substituting local data. Add projection, preview rejection, row-action and completion regressions.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Render installed peer cache rows from discovery instead of served-only names, preserving ownership gates and stopped-engine refusal. Reuse remote-delete-model for confirmed peer deletion and keep the startup picker local. Cover peer list/delete, running and unrecorded-file refusals, previews, and the existing pending-target regressions with the new inventory shape. Retain the unreleased TUI 1.4.0 minor version.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Drive the existing ec mux and outbound remote client with mutually pinned test leaves. Verify successful owned deletion, running refusal, substituted-file refusal, and unpinned caller rejection without touching real engines.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Remove the resolved served-only peer inventory limitation and document installed-list and owned-delete parity over the existing authenticated path. Keep the local-only startup-picker and live-peer release verification limitations explicit.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Update the desktop and engine-manager component references to the verified discovery cache path and existing remote deletion operation. Preserve the local-only startup-picker limitation.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Keep assembling the best-effort served-model result before checking downloaded-file metadata. An invalid llama.cpp ownership manifest must make the authenticated inventory unavailable without erasing another engine from the local best-effort result. Add a regression observed failing before the ordering fix.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Fresh supported installations may not have an engine-bin directory yet. Preserve the no-follow directory walk error cause and recognize a missing ancestor as an empty downloaded cache, instead of making the existing model inventory return unavailable. Symlink, permission and identity failures still fail closed. Add a failing-before regression reproducing the macOS CI finding.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
A failed llama.cpp cache scan made the inventory HTTP handler return 503.
Verified manual peers then lost healthy engines' served models and liveness
while their authenticated identity remained valid.

Return served and loaded inventory with a fixed engine-scoped
cache_unavailable diagnostic instead. Carry downloadErrorsByEngine through
manual peers, scanner refreshes and broker discovery, clearing it on recovery
or proof withdrawal. The TUI displays a calm message only in llama.cpp's
installed-model pane. Genuine whole-inventory failures keep their existing
behavior.

Exercise the real HTTP producer over mutually pinned transport in manual-peer
and scanner regressions. Keep the isolated shared fixture test-only and cover
broker publication, recovery and the TUI's per-engine rendering. Existing
unreleased component bumps, including noderec importers, are sufficient
relative to v0.9.0.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Project the fixed cache_unavailable inventory code into llama.cpp's model
state without changing other engines' models or status. Show a calm installed
inventory message instead of filesystem details, and clear it on recovery.

An unreadable cache is unknown rather than empty, so suppress the empty-cache
placeholder while the diagnostic is present. Keep local read errors separate
from peer state and cover projection, per-engine rendering and recovery.

Regenerate and verify the service contract documentation; the generated
channel listing is unchanged because no methods were added.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
@JaredP94
JaredP94 force-pushed the feat/llamacpp-desktop branch from 6f2e04d to c8a7c4d Compare November 8, 2026 07:37
@JaredP94
JaredP94 marked this pull request as ready for review November 8, 2026 07:52
@JaredP94
JaredP94 merged commit 515e38a into main Nov 8, 2026
6 checks passed
@JaredP94
JaredP94 deleted the feat/llamacpp-desktop branch November 8, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant