Repository navigation
Conversation
…se 1)
Implements the task-executor-refactor design doc's core: devx.yaml is now
a flat map of task names to task definitions (run/deps/ready/detached/...)
instead of profiles/services/deps rendered to Docker Compose or k8s
manifests. run: executes directly via the system shell — nothing is
rendered to an intermediate format.
New packages:
- internal/config: Manifest/Task/ReadyCheck schema, includes: resolution
with namespacing and per-file ScopeVars, Validate/ValidateTools/ValidateSetup.
- internal/graph: generalized topological sort + Closure() over
map[string]config.Task (was *config.Profile-specific).
- internal/task: the new execution engine — ${VAR} substitution and
run:-text inference (resolve.go), process/container drivers (driver.go),
os:/sources:/generates: inference helpers, a .devx/tasks.json state
registry so status/logs/exec/down can reconstruct running tasks across
separate CLI invocations, and a levelized parallel scheduler (Up/Run)
with --dry-run/--keep-going/--group support.
CLI: up/down/run/status/logs/exec/validate/doctor/tasks/new/init rewritten
for the Task model; devx <name> is shorthand for devx run <name>; render/
lock/providers/export are stubbed with a clear "not supported" error,
since there's no compose/k8s artifact to render anymore.
devx down now actually stops plain-process detached tasks: the state
registry records each task's process-group PID, and down sends
SIGTERM/SIGKILL to it directly (previously a no-op for anything that
wasn't a container).
Deleted: internal/compose, internal/k8s, internal/providers, internal/lock,
internal/ai/export.go (GenerateDeployment depended on the deleted Profile
type and its only caller was already a stub).
Docs: docs/manifest.md and schemas/devx.schema.json rewritten for the Task
schema; examples/basic/devx.yaml rewritten (db/cache/migrate/api/worker
tasks) and verified end-to-end against real Docker (up -d, status, down).
Known gaps for a later phase (not silently dropped — flagging explicitly):
- internal/process.Supervisor, which the design doc assumed already
existed, did not exist anywhere in the codebase; internal/task's
process/container drivers were built from scratch instead.
- The old container: auto|true|false convenience (auto-dispatch a run to
the devcontainer) has no Task-model equivalent yet.
- --watch is 1s mtime polling, not fsnotify; run: uses the system shell,
not an embedded shell interpreter — both deferred per the design doc.
- Cobra CLI migration and the workspace/agents feature are out of scope
for this change.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In-depth review (code-review agent) of the task-executor-core PR found
four real bugs/gaps, fixed here:
- internal/task/resolve.go: port inference (`-p HOST:CONTAINER`) captured
the container-internal port instead of the host-reachable one whenever
they differ (e.g. `-p 8080:80` inferred port 80, not 8080) — only the
common host==container case worked by accident. Rewrote portPattern to
capture the whole mapping value and added hostPortFromMapping to pick
the correct field out of `[ip:]hostPort:containerPort[/proto]`.
- internal/config/manifest.go: an included file's own deps: entries
(referencing another task defined in that same included file) were
never namespaced during Load's includes: merge, so two tasks in a
reusable infra/devx.yaml that depend on each other broke with "unknown
dependency" after merging. Added namespaceDeps to rewrite intra-include
deps to their merged name, leaving already-qualified ("ns:task") or
root-level dep references untouched.
- cmd/devx/cmd_logs.go: streamPrefixed's follow-mode loop had no
sleep/backoff when a scanner hit EOF with no new data, busy-spinning a
full CPU core for as long as `devx logs --follow` or attached `devx up`
ran. Added the same 300ms poll interval internal/task's followReader
already uses.
- cmd/devx/cmd_down.go: stopTimeout: was documented/validated but never
read by devx down — a single hardcoded 10s applied to every task
regardless of a task's own stopTimeout: override. Added stopTimeoutFor
to resolve each task's own value (skipped under --force, which always
means immediate SIGKILL).
Also flagged (not fixed — correctly scoped as a later-phase feature,
matching how agents: is already handled): restart: never|on-failure|
always is validated and persisted but devx has no supervisor loop that
watches a detached task for exit and restarts it. Documented as reserved
in docs/manifest.md and schemas/devx.schema.json rather than silently
implying it works.
Added regression tests: TestInfer_HostPortWhenDifferentFromContainerPort,
TestLoad_NamespacesIntraIncludeDeps. Re-verified the full examples/basic
lifecycle against real Docker (validate/up -d/status/curl/down) — the
api task's -p ${API_PORT}:80 mapping is exactly the port-inference bug
scenario and now resolves correctly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In-depth reviewRan a thorough code review (concurrency, correctness, dependency-resolution logic, ${VAR} substitution precedence, YAML unmarshalling edge cases, schema/docs drift) over the full diff. Found and fixed 4 real issues:
Also flagged, not fixed (correctly out of scope for this phase, now documented as such rather than silently implied as working): All fixes verified: See commit da314a2 for details. |
- tests/integration/cli_test.go, devx_test.go: rewrite every manifest fixture from the old Profile/Compose schema (profiles:/services:/ defaultProfile:/--no-telemetry/--volumes) to the flat Task model, and replace the now-permanently-erroring 'devx render compose' test with coverage asserting render/lock/providers/export all return the documented removal error. - devx_test.go: add TestDevxDown_StopsBackgroundProcess, a real subprocess-level regression test for the PID-based process-group stop fix from the prior review; TestDevxTasksJSON and TestDevxRunShorthand for the Task-model-specific 'devx tasks --json' and 'devx <name>' shorthand behaviors that had no integration coverage; and make TestDevxDoctor tolerant of a missing podman runtime (optional, environment-dependent) instead of treating it as a hard failure. - cmd/devx: add helpers_test.go, cmd_validate_test.go, cmd_run_test.go, cmd_down_test.go, main_test.go — the package previously had zero unit tests (the old main_test.go/cmd_validate_test.go were deleted during the schema migration and never replaced). Covers parseCLIVars, loadManifest/loadValidManifest, ensureDevxDir/ensureGitignore, runValidate + extendedWarnings (built-in name collisions, port collisions), runTasks (human/--json), runRun (unknown task, --dry-run, --watch validation, one-shot execution), and stopProcessGroup/ stopTimeoutFor (devx down's PID-signal logic, using a real grouped subprocess). - Use a hand-rolled chdirTemp(t) test helper instead of testing.T.Chdir, since golangci-lint's stdversion check flags T.Chdir (Go 1.24+) against this module's go 1.21 go.mod directive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes: - internal/task/driver.go: processHandle.Wait()/Status() used a single-value "done" channel that only the first reader could ever observe, so calling Wait() permanently starved Status() into reporting "running" forever. Replaced with a close-once "exited" channel + stored exitErr so both methods are safe to call any number of times in any order. New tests: - internal/task/driver_test.go: processDriver Start/Wait/Status/Stop/ Exec, followReader busy-spin regression, tailFile, CheckReady/ WaitReady (cmd/HTTP/TCP paths, timeouts, nil ready-check). - internal/task/state_test.go: StateStore LoadState/Save/Load/Remove/ Lookup/All, including malformed-JSON and snapshot-copy semantics. - internal/task/inference_helpers_test.go: osMatches coverage (empty filter, GOOS match, macos/darwin alias, multi-entry). - internal/config/manifest_test.go: Validate() gaps (version != 1, no tasks, ai: block provider/model requirements, restart: value and detached-only validation, scope: value validation). - internal/config/tools_test.go: ValidateSetup missing name/run. - internal/ui/ui_test.go: PrintTable column alignment. - internal/util/maps_test.go: SortedKeys ordering/empty-map. All packages verified with go build/vet/test, golangci-lint run (0 issues), and the full tagged integration suite against Docker. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
internal/doctor: coverage 12.6% -> 47.4% - checkPortConflicts, checkTaskRequires, checkTools (no-tools case), checkDevcontainerCLI, checkAI (not-configured/malformed/configured paths). Run/detectAllRuntimes/detectCompose remain untested as they depend on live docker/podman and are better suited to integration tests. cmd/devx: coverage 11.4% -> 19.4% - cmd_down.go: reverseDependencyOrder (dependents-before-dependencies, cycle detection) - cmd_env.go: sourceLabel, isLikelySecret, maskSecret, loadDotEnvToMap, dotEnvPath - cmd_clone.go: expandRepoArg, repoBaseName - cmd_update.go: binaryAssetName, normaliseVersion - cmd_auth.go: statusIcon - cmd_dev.go: shortFeatureName, reCapture, mountSummary, commandString, truncate, stripJSONComments, lifecycleCommands, vscodeExtensions - cmd_ci.go: stackFromDevcontainerFeatures, globExists, readJSONC, detectStack, generateCIWorkflow Remaining 0%-coverage code is almost entirely thin wrappers around external tools (docker/podman/devcontainer CLIs, AI provider APIs, self-update network calls) better exercised via integration tests than unit mocks. Verified: go build/vet, go test ./... -race (all green), golangci-lint run ./... (0 issues), and the full Docker-backed integration suite (go test ./tests/integration/... -tags integration, DEVX_BIN set). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bugs fixed: - cmd_new.go: `devx new <name> --run ... --desc ...` (the command's own documented usage) silently failed with a usage error, because Go's flag package stops parsing at the first non-flag argument, so a task name given before its flags left them unparsed. Added extractPositional() to pull the bare task-name positional out of args regardless of its position relative to --run/--desc, so the name can appear before, after, or between flags. - cmd_dev.go/cmd_ci.go: stripJSONComments (and the separate, now-removed duplicate regex in readJSONC) used `//[^\n]*` to strip JSONC comments, which also ate anything after "//" inside a JSON string value — including the "https://..." URL devx's own `devx init` writes into initializeCommand. This broke `devx dev status`/`doctor` JSON parsing for any devcontainer.json containing a URL. Rewrote as a string-aware stripper that tracks whether it's inside a JSON string literal before treating "//" as a comment start. New test coverage (cmd/devx: 19.4% -> 27.4%): - cmd_new_test.go: round-trips a new task into valid YAML, the bare-shorthand path, missing-manifest and bad-arg-count errors, and regression coverage for the flag-ordering bug above. - cmd_init_test.go: writeInitManifest produces parseable YAML for every stack (+ai block), writeDevcontainer produces valid JSONC and never overwrites an existing file. - cmd_status_test.go: runStatus --json/human output with no running tasks, missing-manifest error, devcontainerState/aiStatusLine not-configured paths. - watch_test.go: snapshotMTimes glob matching, changed() added/updated/ unchanged snapshot comparison. - helpers_test.go: lookPathQuiet, selectContainerRuntime. Verified: go build/vet, go test ./... -race (all green), golangci-lint run ./... (0 issues), and the full Docker-backed integration suite (DEVX_BIN set). Manually reproduced and confirmed the fix for the `devx new` flag-ordering bug against the built binary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ompts, down, dev files, logs, exec Covers previously-untested pure/file-based logic: runRender/runLock/ runProviders/runExport error messages, completion script generation, checkEnvKey masking, setupStatusIcon, the AI wizard's promptChoice/ promptString/promptYN/printYAMLSnippet/printAIUsage helpers, stopEntry/ stopContainerByName dispatch paths, requireDevcontainerCLI/devcontainerID/ printDevUsage, loadDevcontainerJSON/findBaseImages, runExec argument validation, containerStatus, and cmd_logs.go's streamPrefixed/tailOne. cmd/devx coverage: 27.4% -> 37.6%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New tests cover runInit's interactive wizard (stdin-fed, full round-trip
to devx.yaml + devcontainer.json), runEnv's variable listing/masking,
runAuth's check list, runClone/runDown/runUp/runSetup/runLogs argument
and missing-manifest validation, runCI (stdout vs --write), and
runDevStatus's devcontainer.json summary output.
Found and fixed a real bug: tailAll (devx logs with no task name and no
--follow) spawned one goroutine per running task to stream its log but
never waited for them before returning, so main() could exit before any
output was flushed — devx logs would often print nothing. Added a
sync.WaitGroup so non-follow mode waits for all goroutines to finish;
follow mode is unaffected (it already blocks forever on select{}).
Reproduced the drop deterministically over 5 repeated test runs before
the fix, and confirmed 10/10 passes after.
cmd/devx coverage: 37.6% -> 50.2%.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…runAIReset, runCILocal Covers runDev's subcommand dispatch table (help/unknown/status default), runDevBuild/runDevUp/runDevExec's devcontainer-CLI-missing error paths, runDevOpen's no-devcontainer error path, runDevPS's docker ps output, runAIStatus/runAIReset's not-configured and confirm/decline-reset flows (stdin-fed), and runCILocal's act-not-installed instructions. cmd/devx coverage: 50.2% -> 54.2%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… chat, runAI dispatch Covers wizardLocal/wizardRemote/wizardCloud/wizardPickModel/wizardTool (stdin-fed, declining any real backend setup/tool launch), runAIModel's list/pull/remove subcommands and usage/validation paths, runAIChat's no-config error, runExternalCommand, and runAI's full dispatch including the end-to-end first-time wizard flow via runAI(nil). Note: this sandbox has a real Ollama server reachable via host.docker.internal, so runAIModelList's test tolerates either a detected-backend or no-backend outcome rather than asserting one. cmd/devx coverage: 54.2% -> 63.2%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…and signal handling - devcontainerBuild/runDevcontainerCLI/runDevLogs/runDevShell: error paths when the devcontainer CLI or a running container isn't present. - runAIModelPull/runAIModelRemove: MLX/remote-hint branches plus the ollama-not-installed error path. - watchAndRerun: verifies the ctx.Done() case returns promptly instead of blocking on the 1s ticker, using an already-cancelled context. - attachAndWaitForInterrupt: sends SIGINT to the test process itself to verify it unblocks and runs runDown instead of hanging forever. All new tests stable across repeated -race runs (5x). cmd/devx coverage: 63.2% -> 66.8%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ad EF Core detection These three packages had zero test coverage before this change: - internal/ai (92.8%): Client.New/complete across openai/anthropic/ azure-openai/ollama providers (via httptest servers), DetectFrameworks' per-language/framework marker detection, Detect()'s JSON-fence-stripping and error paths. - internal/localai (77.4%): model suggestion lists, backend probing (checkURL/fetchMLXModel/probeRemote via httptest, ResolveBackend/ IsAppleSilicon), SavedAIConfig load/save/delete round-trips, ToolsForProvider/buildToolCmd/aiderArgs/setenv, LaunchTool's backend-unreachable and unknown-tool error paths, DetectCloudProviders, buildContinueConfig/WriteContinueConfig for every provider branch, and Setup()'s unknown-backend/non-Apple-Silicon-MLX error paths (without ever invoking the real network-installing code paths). - internal/plugins (100%): Discover()'s PATH-scanning, prefix matching, cross-directory deduplication, and unreadable-PATH-entry handling. Bug fix: internal/ai/hints.go's EF Core detection passed the literal string .csproj to hasContentMatch's exact-filename check instead of an actual matched file name, so 'EntityFramework' content detection could never fire for any real *.csproj file. Fixed to check content of the actual matched .csproj/.fsproj files. Also removed the now-unused anyMatch helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use fake shell-script binaries (via the Runtime.Binary field) to deterministically exercise Up/Down/Exec/Logs/Status/ResolveImageDigest and the underlying exec.Command plumbing, without requiring real docker/podman compose behavior. Brings both packages from 0% to ~93%/~92.6% coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…Env, and doctor.Run - internal/task/container_driver_test.go: covers containerDriver.Start, containerHandle Status/Stop/Logs/Exec/Wait/PID using fake docker/podman binaries injected via PATH. - internal/task/scheduler_resolve_test.go: covers Scheduler.ResolveOne, ResolveAll, probeExisting (container + process paths), processAlive, sanitize, taskTimeout. - internal/task/resolve_test.go: adds RedactedEnv coverage. - internal/doctor/doctor_test.go: adds Run, detectAllRuntimes, detectCompose coverage. internal/task: 65.8% -> 78.9%, internal/doctor: 47.4% -> 74.1%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change releasesAPI/releaseDL from consts to vars so tests can point fetchLatestVersion at a local httptest.Server instead of the real GitHub API. Covers fetchLatestVersion's success/malformed-JSON/ unreachable-server paths, and runUpdate's dev-build-skip, already-on-latest, --check, and fetch-error branches. selfUpdate and main() remain untested: selfUpdate replaces the currently running binary via os.Executable(), which would corrupt the test binary itself if exercised directly. cmd/devx: 66.8% -> 68.3%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
Replaces the Profile/Service/Dep + Docker Compose/Kubernetes rendering architecture with the flat Task-based manifest and scheduler described in
docs/design/task-executor-refactor.md(Phase 1 of 3).Motivation
The old model's strong coupling to Compose/k8s schema versions made adding capability feel like "a new feature every time," and the design doc (reviewed and refined across several prior review cycles — see
design/task-executor-refactor-docbranch, PR #8) proposes a simpler, Make-like flat task graph withrun:/deps:/ready:instead. This PR builds that core.Changes
internal/config: newManifest/Task/ReadyCheckschema — task names are top-leveldevx.yamlkeys directly (noprofiles:/services:/deps:wrapper).includes:resolves and namespaces another file's tasks (infra:db), with per-filevars:scoping. NewValidate/ValidateTools/ValidateSetup.internal/graph: generalized tomap[string]config.Task; addedClosure()for partialup/run <name>invocations.internal/task(new package — the architectural core):${VAR}substitution +run:-text inference (ports/names/detached-ness), process and container drivers,os:/sources:/generates:inference helpers, a.devx/tasks.jsonstate registry sostatus/logs/exec/downcan reconstruct what's running across separate CLI invocations, and a levelized parallel scheduler (Up/Run) with--dry-run/--keep-going/--group.internal/doctor: dropped k8s/registry checks, addedrequires:binary checks, rewired port-collision checking to the new Task model.up/down/run/status/logs/exec/validate/doctor/tasks/new/initrewritten for the Task model.devx <name>is shorthand fordevx run <name>.render/lock/providers/exportnow print a clear "not supported" error (no compose/k8s artifact to render anymore).devx downcorrectness fix: the state registry now records each detached process task's process-group PID, anddownsignals it directly (SIGTERM, then SIGKILL afterstopTimeout) — previously a no-op for anything that wasn't a container.internal/compose,internal/k8s,internal/providers,internal/lock, andinternal/ai/export.go(GenerateDeploymentdepended on the deletedProfiletype; its only caller was already a stub).docs/manifest.mdandschemas/devx.schema.jsonrewritten for the Task schema;examples/basic/devx.yamlrewritten (db/cache/migrate/api/worker tasks).Known gaps, flagged explicitly (not silently dropped)
internal/process.Supervisoralready existed — it didn't, anywhere in the codebase.internal/task's process/container drivers were built from scratch.container: auto|true|falseconvenience (auto-dispatch a run into the devcontainer) has no Task-model equivalent yet.--watchis 1s mtime polling, notfsnotify;run:executes via the system shell, not an embedded shell interpreter — both intentionally deferred per the design doc's phasing.Testing
internal/config,internal/graph,internal/task(new:resolve_test.go,scheduler_test.go),internal/doctorall pass.tests/integration/still targets the old schema and will need a follow-up rewrite.go build ./...,go vet ./...,go test ./...,golangci-lint run ./...all pass clean.examples/basic/devx.yaml:devx validate(clean),devx up -d(db/cache/migrate/api/worker all start and become ready),devx status(shows all 4 containers running),curl localhost:8080/(API responds),devx down(all 4 containers stop and are removed).Checklist
go build ./...passesgo test ./...passesdocs/manifest.md,schemas/devx.schema.json,examples/basic/devx.yaml)CHANGELOG.mdupdated under[Unreleased]