Skip to content

Let paired machines reconnect after address changes - #55

Merged
JaredP94 merged 2 commits into
mainfrom
fix/peer-address-roaming
Nov 6, 2026
Merged

JaredP94 merged 2 commits into
mainfrom
fix/peer-address-roaming

Conversation

@JaredP94

@JaredP94 JaredP94 commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Description

Let a paired machine recover when its peer moves away from the LAN address recorded at pairing. Learn a bounded, durable set of address hints from pinned cluster mTLS roster traffic, without treating an address or name as identity. Also give the scanner's first local node-info read a bounded readiness grace so normal startup does not report an outage.

Scope

  • 9213215c: cluster-manager alone owns learned hints in cluster/members.json (atomic writes, 0600), keyed to the member/admission. Sources are the authenticated inbound connection's source host, the pinned sender's own additive selfAddresses HTTP roster field, and successful pinned outbound roster responses. Discovery, public previews, and third-party relayed addresses cannot teach this set.
  • Eight distinct endpoints maximum, most recent authenticated success first; permanent pairing-time fallback; learned hints expire after 30 days without success. Repeated self-claims do not renew expiry. Numeric unicast IPs and valid ports only; reject unspecified/multicast/broadcast and self-reported loopback; remap IPv6 link-local zones to the local connection's scope.
  • Existing reachability chooser, whole-walk deadline, and exact certificate pin checks remain. Only one unproven endpoint per member per 30 seconds enters a dial walk, including concurrent requests and restarts. A failed retry-budget write withholds unproven dials. Redirects are refused. Removal and admission rechecks prevent in-flight learning from restoring a removed member.
  • Cluster-derived manual-node entries follow the reached address in the existing detail projection. User-saved entries retain their address/name/kind/precedence. No desktop source or JSON-RPC schema change; HTTP producer and consumer are both cluster-manager. Service contracts regenerated/checked with no generated diff.
  • bfb09d9b (separate commit): first local node-info enrichment retries using existing 2/4/8-second telemetry backoff under one 15-second budget. A sustained outage still warns; subsequent outages are reported immediately.
  • All affected binaries were released in 0.9.0: cluster-manager 2.2.0 → 2.3.0 (additive HTTP roster field), manual-nodes 1.3.0 → 1.3.1 and scanner 1.3.0 → 1.3.1 (fixes). No other runtime module changed.

Validation

Isolated worktree /tmp/pair-work/peer-roaming, branch fix/peer-address-roaming from origin/main at 9d3c71e2 (0.9.0). Implementation written directly, not delegated. Rebased before push; no changes to engine-manager or shared/jsonrpc.

  • go test -race -count=1 ./... passed in cluster-manager, manual-nodes and scanner. Full cluster-manager non-race suite passed too. Native/Linux/Windows go vet ./... passed in all three modules and the cross-process test module. gofmt -l clean; git diff --check passed.
  • node scripts/spdx-headers.mjs: 1423 checked, zero missing.
  • Own worktree npm ci --no-audit --no-fund; desktop typecheck, lint with --max-warnings=0, 936 tests across 94 files, dead-code check, service-contracts:write and service-contracts:check passed. No desktop source changes.
  • Actual origin/main red runs in a temporary detached worktree, with only new tests copied in: TestRoamingSelfReportsBelongOnlyToPinnedSender, TestRoamingCandidatesBoundedOrderedPersistedAndExpired, TestRoamingIgnoresDiscoveryAndRelayedAddresses, TestRoamingInboundSourceSurvivesRestart, TestRoamingPinnedPeerCannotRedirectCandidateDials, TestRoamingSuccessfulPairingFallbackBecomesMostRecent, the cluster_seed subtest of TestClusterSeedFollowsRoamingAddressWithoutOverwritingSavedEntry, TestLocalNodeInfoStartupWaitsBeforeWarning, and TestLocalNodeInfoRealStartupOutageEventuallyWarns all failed on missing behavior (not compile errors). Restored implementation passes. The hostile-certificate guard already passed on origin/main and is retained, not claimed as a newly failing regression.
  • Additional unit coverage: TestRoamingCandidateValidationAndIPv6Zones, TestRoamingNewCandidateDialBudgetSurvivesClaimChurnAndReload, TestRoamingInFlightRequestCannotRestoreRemovedMember, TestRoamingLearnsSelfReportFromPinnedResponseOnly, TestRoamingSuccessfulPairingFallbackBecomesMostRecent, and TestRoamingNewCandidateNeedsDurableDialBudget. The durable-write regression failed before withholding the new dial, then passed with the fix.
  • A full race run encountered the old fixed-port failover fixture not answering (its fire-and-forget listener errors were discarded); the focused rerun passed. Port contention is suspected, not established. Replaced that modified fixture with an ephemeral real mTLS listener, then the complete race and non-race suites passed.
  • Added TestPeerRoamingLearnsInboundAndDialsAfterRestart: real isolated workers, stale recorded address, pinned inbound repair, persisted pairing fallback, restart, then successful authenticated removal delivery at the learned endpoint. No cross-process tests were run locally; CI supplies that evidence.

Latest-head CI: run 42202311657, head bfb09d9. Four of six jobs passed: SPDX, desktop checks (including contracts/licenses), Linux Go modules, and Linux cross-process. Both macos-15 jobs never acquired runners (runner_name empty, zero steps) and were cancelled after about 15 minutes. In particular, Go modules (macos-15) never acquired a runner. The run concludes failure; all-six-job success is not confirmed, and macOS cross-process remains untested in CI.

Downloaded cross-process-ubuntu-latest using gh run download into the new empty directory /tmp/pair-review/peer-roaming-linux-42202311657-EOGsoW. Ran exactly python3 -I /tmp/pair-review/scripts/noskip.py /tmp/pair-review/peer-roaming-linux-42202311657-EOGsoW/cross-process.jsonl: 123 passing cases, zero failures, ZERO skips. TestPeerRoamingLearnsInboundAndDialsAfterRestart passed in 5.14 s. Artifact contents were parsed, not executed. No cross-process suite was run locally.

Risk

Address hints are not identity authority: wrong/hostile endpoints fail the exact pin check. At least one authenticated rendezvous path must remain; if both machines lose every known address, explicit reachable configuration is still needed. Old peers may omit selfAddresses; inbound source learning and the pairing fallback still work. Certificate endorsements continue to establish transitive membership, but a relayed third-party address is no longer an address-learning source. The scanner can take up to 15 seconds to finish its first local inventory enrichment during a real outage; its control reader remains independent.

Unit/CI evidence is not live two-Mac, Windows runtime, packaged-asset, signing or notarization certification.

Checklist

  • I have read the Contributing Guidelines.
  • Every commit is signed off (git commit -s), certifying the Developer Certificate of Origin.
  • New or existing tests cover the change.
  • Relevant documentation is updated.
  • I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
  • I recorded the validation commands and results above.
  • I bumped any affected component in services/versions.json, and described user-visible changes above so they reach the release notes.

…ming

A paired machine can change networks while its member record retains only the old LAN endpoint. Learn a bounded, expiring set of endpoints from pinned roster connections and each authenticated peer’s self-report, retain the pairing-time fallback, and budget unproven dials across concurrent reconciles and restarts. Every application connection still verifies the exact pin and refuses redirects.

Keep address hints local to members.json rather than extending the JSON-RPC projection. Serialize learning with admission removal, ignore discovery and third-party address claims, and update cluster-derived manual-node entries without changing saved entries. Add unit regressions and an isolated real-worker roaming/restart/removal test for CI. Bump cluster-manager for the additive roster field and manual-nodes for the endpoint-refresh fix.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Scanner starts before node-info has necessarily opened its listener. Give only the first local enrichment read a bounded fifteen-second readiness budget using the existing telemetry backoff. Normal startup stays quiet; exhaustion still warns and subsequent outages retain immediate transition reporting. Cover delayed readiness and a sustained outage, and bump the scanner patch version.

Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
@JaredP94

JaredP94 commented Nov 6, 2026

Copy link
Copy Markdown
Owner Author

CI note before merge: on run 42202311657, five of six jobs passed, including both cross-process jobs (Linux: 123 passed, 0 skipped; macOS passed on re-run). The "Go modules (macos-15)" job was cancelled twice without acquiring a runner (GitHub macOS arm64 capacity). It never started.

That job's exact steps were run instead on macOS arm64 at bfb09d9 with the CI toolchain (Go 1.26.7): gofmt -l services was clean, and go vet ./... plus go test -count=1 -timeout=5m ./... passed in all 16 modules (services/tests excluded, as in CI).

@JaredP94
JaredP94 merged commit a73c542 into main Nov 6, 2026
13 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant