feat(computing-unit): surface failed/unhealthy computing units instead of an endless "Connecting"

[eugenegujing]

What changes were proposed in this PR?

When a computing unit becomes unhealthy(the pod is OOM-killed into a crash loop, evicted for disk pressure, stuck on an image pull, or its node becomes unreachable), the UI used to show "(Connecting)" with a "starting up" tooltip forever, because ComputingUnitState only had Running and Pending and status resolution only looked at pod.status.phase.

This PR follows the decisions settled in #7670: mirror Kubernetes for the status vocabulary; owner-only, actionable failure reasons; no auto-recovery — users delete and recreate (the existing terminate flow already handles dead pods).

Screenshots

Two representative states as examples, and the complete set of states and reasons is the mapping table in the next section, and every display combination (status × reason × owner) was verified manually.

An Unknown unit (node unreachable): red badge, disabled "Unit Unavailable" run button, and the owner-facing reason on the row tooltip:

A recovered OOM-killed unit: stays green and runnable, but the tooltip carries the out-of-memory warning with the restart count:

Status vocabulary and reasons

ComputingUnitState gains Failed, Unknown, Terminating. The full mapping from observed pod state to status and owner-facing statusReason:

Observed pod state	Status	Owner-facing statusReason
deletionTimestamp set	Terminating	—
phase Failed + reason Evicted, message mentions ephemeral/disk	Failed	"The computing unit was evicted because it ran out of local disk storage. Consider storing less data on the unit's local file system, or recreate it with more storage."
phase Failed + reason Evicted, other	Failed	"The computing unit was evicted by the cluster (<first sentence of the cluster message, capped at 120 chars>). Consider recreating it."
container waiting ImagePullBackOff / ErrImagePull / InvalidImageName	Failed	"The computing unit's image could not be pulled. Please recreate the unit or contact an administrator."
container waiting CrashLoopBackOff, last termination OOMKilled	Failed	"The computing unit keeps crashing because it runs out of memory. Please terminate it and recreate it with a higher memory limit."
container waiting CrashLoopBackOff, other	Failed	"The computing unit is repeatedly crashing (restarted N times). Please terminate and recreate it, or contact an administrator."
phase Failed, not evicted	Failed	"The computing unit stopped unexpectedly. Please terminate and recreate it, or contact an administrator."
phase Unknown	Unknown	"The state of the computing unit cannot be determined (its node may be unreachable)."
phase Pending + PodScheduled=False/Unschedulable condition	Pending	"The computing unit is waiting for cluster resources to become available."
phase Running + a container's last termination OOMKilled	Running	"The last run was terminated because the computing unit ran out of memory (restarted N times). Consider recreating the unit with a higher memory limit before running the same workload."
phase Running, healthy	Running	—
anything else / pod absent / local unit	Pending / Running (unchanged)	—

Precedence is top-to-bottom; in particular a CrashLoopBackOff failure wins over the recovered-OOM warning, and Terminating wins over everything.

statusReason is owner-only: the backend sends null to shared users at every DTO construction site, and the frontend then falls back to a generic text. Frontend fallbacks when statusReason is null: Running → "Ready to use", Pending → "Computing unit is starting up", Failed/Unknown → "This computing unit is unavailable.", Terminating → "Computing unit is shutting down", other states → the status word itself.

Backend

• The existing single namespace-level pod listing now yields a PodStatusSnapshot per pod (phase, deletionTimestamp, pod reason/message, Unschedulable condition, per-container waiting reason / last termination reason / restart count) via a pure, unit-testable transform — no additional Kubernetes round trips.
• The restartPolicy subtlety that shaped the design: with restartPolicy: Always, an OOM-killed container restarts in place and the pod phase never leaves Running, so OOM kills are only visible through containerStatuses[].lastState.terminated.reason. A unit that recovered stays Running with a warning; a unit that cannot come back up lands in CrashLoopBackOff and is reported Failed.
• Vanished-pod reconciliation (refactor(computing-unit-managing-service): share CU listing helpers and batch k8s calls #6853/feat(computing-unit-managing-service): add admin endpoint to list all computing units #6854) is untouched: it stays keyed purely on pod presence, and a present-but-Failed pod is not treated as vanished (covered by tests).
• local units are unchanged (always Running) — local liveness is out of scope here, per the discussion.

Frontend

• status union widened to the five states; statusReason? added.
• Failed/Unknown: red badge in the dropdown (the previously dead red branch, now reachable).
• No text suffixes next to the unit name for any state: a "(Unavailable)" label truncated at real dropdown widths, and the pre-existing Pending "(Connecting)" suffix was equally redundant (gold badge + tooltip + the run button's own "Connecting" spinner) — status is uniformly conveyed by the badge color and explained by the row tooltip.
• Each dropdown row has exactly one tooltip surface — the row itself: hovering the row body shows the status/reason, plus "Cannot select." when the unit is not selectable. The nested badge and name tooltips were removed because they stacked a second bubble on top of the row's (the name's tooltip was the raw pod URI, which stays available in the details modal; the action icons keep their own tooltips, as on main). This also makes the recovered-OOM warning on a Running unit reachable by hovering the row rather than a few-pixel dot.
• The run button shows a disabled "Unit Unavailable" state instead of the endless "Connecting" spinner when the selected unit is Failed/Unknown (the spinner branch never consulted unit status).
• The ". Cannot select." concatenation now trims a trailing period off the status tooltip before appending, since a statusReason is a full sentence — previously this produced a doubled dot.

Any related issues, documentation, discussions?

Closes #7669. The design questions (status vocabulary, how much detail reaches which users, auto-recovery) were discussed and settled in #7670.

How was this PR tested?

• TDD: one backend test per decision-table row using PodBuilder-built pods through the pure snapshot + mapping functions, including both eviction wordings, all three image-pull reasons, crash-loop with and without OOM history, the crash-loop-beats-recovered-OOM precedence, multi-container pods, owner gating, and the absent-pod path used by creation polling. fabric8 null-guard paths (status, conditions, containerStatuses, state, lastState, terminated) have dedicated tests.
• Mutation-checked: precedence swap, Evicted→Pending, wording-branch removal, restart-count hardcoding, owner-gate removal, absent-pod→Unknown, and run-button branch removal each make at least one test fail.
• Edge-case sweep on top of the row-by-row tests: Terminating precedence over eviction (and over a status-less pod), eviction-message truncation/whitespace/case-insensitivity, mixed multi-container pods (the crash-looping container's own history decides the wording), stale/malformed Unschedulable conditions, image-pull-beats-crash-loop precedence, empty-string statusReason not shadowing the fallback, and a pinning test documenting that phase Succeeded maps to Pending.
• ComputingUnitManagingService util specs 90/90, resource specs 60/60, scalafmt clean; frontend tsc --noEmit clean, prettier clean, all changed spec files pass under ng test.
• Manually verified every frontend display state (all status × reason × owner combinations) against a live dev stack.

Notes for more information: Option[String] → null serialization relies on DefaultScalaModule, registered on the service's Dropwizard ObjectMapper; there is no end-to-end JSON test for the new field. A bare pod in phase Succeeded still maps to Pending (pinned by a test) — with restartPolicy: Always this phase is practically unreachable, so it is left as a known limitation. Follow-ups deliberately out of scope (per #7670): frontend timeout for the "pod Running but engine unreachable" zombie case, UX for the silent vanish-reconcile, and local CU liveness.

Was this PR authored or co-authored using generative AI tooling?

Co-authored by: Claude Code (Claude Fable 5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Neilk1021, @kunwp1, @aglinxinyuan
  You can notify them by mentioning @Neilk1021, @kunwp1, @aglinxinyuan in a comment.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 91.30435% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.13%. Comparing base (875aa72) to head (d8707f2).
⚠️ Report is 47 commits behind head on main.

Files with missing lines	Patch %	Lines
...che/texera/service/util/ComputingUnitHelpers.scala	89.18%	0 Missing and 8 partials ⚠️
...apache/texera/service/util/PodStatusSnapshot.scala	92.30%	1 Missing and 2 partials ⚠️
...rvice/resource/ComputingUnitManagingResource.scala	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7944      +/-   ##
============================================
+ Coverage     91.51%   92.13%   +0.61%     
- Complexity     4476     4530      +54     
============================================
  Files          1171     1180       +9     
  Lines         47199    47531     +332     
  Branches       5303     5329      +26     
============================================
+ Hits          43196    43794     +598     
+ Misses         2344     2120     -224     
+ Partials       1659     1617      -42     

Flag	Coverage Δ		*Carryforward flag
access-control-service	81.00% <ø> (ø)
agent-service	98.62% <ø> (ø)		Carriedforward from 875aa72
amber	88.08% <ø> (ø)		Carriedforward from 875aa72
computing-unit-managing-service	76.41% <90.24%> (+2.74%)	⬆️
config-service	86.73% <ø> (ø)
file-service	84.48% <ø> (+15.57%)	⬆️
frontend	94.40% <100.00%> (+1.15%)	⬆️
notebook-migration-service	79.13% <ø> (ø)
pyamber	97.57% <ø> (ø)		Carriedforward from 875aa72
workflow-compiling-service	77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[eugenegujing]

/request-review @kunwp1