Repository navigation
Stop safe mode containers on cancel and cap concurrent runs - #31
Conversation
Jabe03
left a comment
There was a problem hiding this comment.
Review of the safe mode run lifecycle. I'll address each comment with its own commit and reply here.
Not raised here: a cancel that arrives before Docker creates the container (pending issue 3 in the safe mode notes), since how to handle it hasn't been decided yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When Kind 2's output ends, the docker client is usually still waiting for the container's exit status, so checking isAlive() right away made almost every finished run call docker kill. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replacing the semaphore left callers that were already waiting on the old one, so a lower limit was not enforced and a higher one did not wake them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
acquireSafeModeRun now returns a closeable slot (a no-op one outside safe mode) and null only when cancelled, so callers can't forget the release. It also checks the monitor once more after acquiring, so a run cancelled just as a slot frees up does not start a container. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The interpreter has no monitor, so a call made while the run limit was reached blocked the caller's thread until a long check finished. It now throws a Kind2Exception instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkAvailable, checks and the interpreter now share getKind2Command, so checkAvailable runs the same command that execute does. The generated command lines are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
stderr was never read, so docker errors in safe mode were lost, and enough stderr output blocked Kind 2 and hung the run. It is now read on its own thread, and if no trace comes back, interpret throws a Kind2Exception with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was encoded with the platform's default charset while the output is read as UTF-8, so non-ASCII programs broke on Windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
daniel-larraz
left a comment
There was a problem hiding this comment.
Review of the safe mode run management and interpreter error reporting. I checked each finding against the code; gson behavior was tested against 2.8.6.
The two worth fixing first are on the new interpreter error-reporting path: harmless stderr now replaces the empty result (L407), and parse failures escape as raw gson exceptions (L398). The rest are lower severity: cleanup/race edge cases, and a few smaller consistency and robustness points.
🤖 Generated with Claude Code
Only JsonIOException (empty stdout) was caught, and it was rethrown as is. Non-JSON stdout threw an uncaught JsonSyntaxException that skipped the stderr report. Both are JsonParseExceptions, which are now caught and wrapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Any stderr text made a run without a trace throw, so a harmless docker warning (such as the missing swap limit support) replaced a Kind 2 error reported as JSON on stdout. stderr is now used only when stdout is empty or cannot be parsed; otherwise a run without a trace returns "" as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onitor execute(String) and execute(Program) use a monitor that never cancels, so once every run slot was taken they waited forever. Like interpret, they now throw a Kind2Exception after SAFE_MODE_SLOT_WAIT_SECONDS. Calls that pass their own monitor still wait until it is cancelled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the flag set, the waits in exitsOnItsOwn and killContainer threw at once, so the run slot was released before docker kill had stopped the container, and a hung docker kill was never destroyed. A shared stopRun now clears the flag while it stops the run and restores it afterwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docker kill misses a container that docker is still creating. If the client then dies before starting it, the container stays in the Created state, which --rm never removes; if the client starts it first, it runs with nobody tracking it. After a kill, stopRun now waits for the client to exit and runs docker rm -f on the container. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In safe mode, checkAvailable started a container with no name, no run slot and no timeout, so a hung docker blocked it forever with nothing to kill. It now names its container, takes a run slot, gives up after 30 s, and stops the container like any other run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
newContainerName is now the only place a run reads safeMode. The run slot, the command and the Kind 2 options all follow from whether the run has a container name, so a concurrent setSafeMode can no longer produce a docker run without a name, or a native run that holds a slot. getOptions() keeps its behavior and delegates to getOptions(boolean). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
acquireSafeModeRun now returns RunSlot.CANCELLED instead of null when the monitor is cancelled while waiting, so callKind2 acquires its slot in the try-with-resources header like the other callers and checks isCancelled() inside it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
readAll turned each byte into a char, which decodes it as Latin-1, so non-ASCII text in docker and Kind 2 messages (such as a host path quoted in a docker error) came out garbled. It now decodes the whole stream as UTF-8, like the rest of the process I/O. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stop safe mode containers on cancel and cap concurrent runs
Follow-up to #30. Needed by kind2-mc/kind2-language-server#22, which cancels out-of-date parses: without this, cancelling a safe mode run stops the
dockerclient but can leave the container running.Changes
docker runcommand:getDockerRunCommand()builds the resource limits and the Kind 2/solver mounts once, for checks, the interpreter andcheckAvailable().--init: the container runs an init process as PID 1, so signals reach Kind 2 and orphaned solver processes are reaped.--name(kind2-api-<uuid>). If the run is cancelled or the process is still alive when it ends, the API runsdocker killon that container before destroying the client process. Destroying the client alone doesn't reliably stop the container (for example on Windows).Kind2Apiinstance limits how many safe mode runs happen at once (default 2, set with the new staticsetSafeModeMaxRuns(int)). Waiting runs still respond to their monitor's cancellation.checkAvailable()in safe mode now runs Kind 2 inside the container, and waits for the process instead of readingexitValue()before it has exited.--no_includeis only passed when safe mode is on (it used to be passed as--no_include falsewhen safe mode was explicitly off).interpretoverloads now share onerunInterpretermethod.docker killtimeout now applies; finished runs get a short grace period instead of being killed;setSafeModeMaxRunsresizes the limit in place, so it also applies to waiting runs; slots are held with try-with-resources; the interpreter stops waiting for a slot after 60 s; one helper builds the Kind 2 command prefix; the interpreter reads stderr and reports it when no trace comes back; programs are sent as UTF-8.Kind2Exception;checkAvailabletakes a run slot, names its container and times out after 30 s; after a kill, the container is also removed withdocker rm -fonce the client has exited (covers a cancel that lands while Docker is still creating it);execute(String)/execute(Program)stop waiting for a slot after 60 s; cleanup clears the interrupt flag while it waits;safeModeis read once per run; one slot-holding pattern;ApiUtil.readAlldecodes UTF-8.Outside safe mode the command line is unchanged.
Testing
./gradlew clean buildpasses.kind2(plus a docker wrapper that delayskillor splitsrunintocreate+start), reproducing the problem on the previous head where possible: Created container left after an early cancel (2/2 before, 0 after), slot released 2 s beforedocker killended on an interrupted thread, harmless stderr hiding the result, garbled non-ASCII paths,checkAvailablehanging on a stuckdocker run.kind2(sleeps, then prints a trace): finished checks and interpreter runs call nodocker kill, a cancelled check is killed with no container left, raising and lowering the limit while runs wait takes effect right away, and the interpreter gives up after 60 s when no slot is free.Kind2Api(setSafeMode(true)) left no running containers and no leftoverz3processes in 4 of 4 tries.🤖 Generated with Claude Code