control: replace the /dev/shm registry with abstract Unix sockets - #207
Open
congwang-mk wants to merge 14 commits into
Open
control: replace the /dev/shm registry with abstract Unix sockets#207congwang-mk wants to merge 14 commits into
congwang-mk wants to merge 14 commits into
Conversation
Whether a sandbox shows up in sandlock ps is the operator's concern, not a policy setting, and no tool exposes the knob anyway: it had no CLI flag, no SDK field, and serde skipped it, so only the Rust builder could reach it and nothing in-tree ever did. Worse, setting it false skipped setup_runtime_dir, which is also where the UID-wide name collision check lives, so the knob quietly disabled name uniqueness. The one real reason to run without a socket, a nested sandbox whose outer policy blocks /dev/shm, is already handled by the best-effort fallback that warns and continues. Signed-off-by: Cong Wang <cwang@multikernel.io>
The /dev/shm runtime directory needed a writable path from any outer policy, left stale directories for ps to prune, and made name uniqueness a side effect of a directory existing. An abstract Unix socket named sandlock/<uid>/<name> has none of those: bind fails on a taken name, the name dies with the process, and /proc/net/unix lists it. Names must now avoid whitespace and control characters because that listing is whitespace-delimited. Signed-off-by: Cong Wang <cwang@multikernel.io>
Every sandbox now binds its abstract control socket at the same point in do_spawn, whether or not a seccomp-notify supervisor exists, so ps sees --no-supervisor and nested sandboxes without a pid file. The new info verb carries what the pid file and mode marker held. The peer uid check turns from a warning into a refusal because an abstract name has no directory mode to rely on. Signed-off-by: Cong Wang <cwang@multikernel.io>
ps lists the caller's abstract control sockets and asks each for info; a socket that does not answer is still shown, as unresponsive, because its existence proves the process is alive. kill gets both pids from the same verb and signals them as before, with nothing left on disk to clean up afterwards. Signed-off-by: Cong Wang <cwang@multikernel.io>
The prune and pid-file tests have no subject left. In their place: a SIGKILLed supervisor's name disappears and is reusable at once, a stopped supervisor is still listed as unresponsive, a --no-supervisor sandbox answers info, config, and an empty ports map, and another uid gets no response (root only). Signed-off-by: Cong Wang <cwang@multikernel.io>
A failed assertion must not leak a live sandbox, so the supervisor is killed and reaped before the assert instead of after it. Signed-off-by: Cong Wang <cwang@multikernel.io>
wait() only aborted the control task, and the listener that holds the abstract name was dropped whenever the runtime got to cancelling it, so a caller that ran the same name twice in a row could hit the collision check. Awaiting the aborted task makes the release synchronous, which is what the runtime directory removal used to be. Signed-off-by: Cong Wang <cwang@multikernel.io>
A doc comment still described the /dev/shm runtime dir, and two lines this branch introduced tripped clippy (int_plus_one in a unit test, print_literal in the ps header). Signed-off-by: Cong Wang <cwang@multikernel.io>
Abstract names have no owner, so another uid could bind sandlock/<uid>/<name> and hand ps and kill pids of its choosing; the client now checks SO_PEERCRED on the listener as the server already does on the peer. A request is bounded to five seconds so one stalled connection cannot wedge introspection, which kill now depends on. The uid each side expects is a parameter so the refusal paths are tested without a second uid, replacing a root-only test that had never run. Signed-off-by: Cong Wang <cwang@multikernel.io>
The Landlock abstract-socket scope is now what keeps the confined child off its supervisor's control socket, which is worth saying where the mask is built. The README ps and kill samples predate the STATUS and PORTS columns and the two-pid kill message. Signed-off-by: Cong Wang <cwang@multikernel.io>
/proc/net/unix renders abstract names as raw bytes, so any local process could bind a non-UTF-8 name and make read_to_string fail for every user's sandlock ps. Ours are ASCII by validation, so a lossy decode only affects names that never match the prefix. Signed-off-by: Cong Wang <cwang@multikernel.io>
An abstract name stays bound while any fd refers to it, and a forked child keeps every inherited fd until it execs, or forever in the case of a COW clone. A sandbox created but not yet started therefore pinned the control socket name of every sibling in the process, which showed up as a spurious "already running" when a sibling's name was reused right after wait(). The child now scans /proc/self/fd and closes any socket whose abstract name is a sandlock control socket before it does anything else. That also means a confined child never holds a handle to another supervisor's control socket, even briefly. Signed-off-by: Cong Wang <cwang@multikernel.io>
Contributor
Author
|
@sachin2605 Please take a look. Thanks! |
kill and ps used the info verb to find the child and supervisor pids, so a supervisor that never answered, stopped or deadlocked, could only be found by grepping ss for its abstract socket. listen() stamps the caller's pid into a socket and SO_PEERCRED hands that stamp to whoever connects, so the pids need no cooperation if the right process calls listen(). The supervisor now binds a second abstract socket, sandlock/<uid>/<name>/pgrp, before it forks; the child inherits it, calls listen() right after setpgid(), and closes it, while the supervisor keeps the fd and drains the never-spoken-on connections. kill connects to both sockets, reads the peer pids, and does killpg on the child and kill on the supervisor without sending a request. Both pids are live by construction: the sockets die with the supervisor, and a dead child stays a zombie holding its pid until the supervisor reaps it, at which point wait() closes the sockets. The pids leave the wire protocol, info carries only the mode, and binding before the fork means a name collision fails with no child to reap. Signed-off-by: Cong Wang <cwang@multikernel.io>
Several comments still explained unique sandbox names by the per-UID runtime directory they used to claim, and the mode field's doc said the marker was written there at spawn time. The registry is gone; names are claimed by binding an abstract socket and the mode is served over it, so the comments now say that. Signed-off-by: Cong Wang <cwang@multikernel.io>
Contributor
|
Hi @congwang-mk . This is a much more clean implemention for supervisor free mode. GTG from me. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the
/dev/shm/sandlock-$UID/<name>/runtime directory (pid file, mode marker, named socket) with two abstract Unix sockets per sandbox,\0sandlock/<uid>/<name>and\0sandlock/<uid>/<name>/pgrp, both bound by the supervisor before it forks. The first commit drops thecontrol_socketopt-out knob: whether a sandbox is visible tosandlock psis the operator's concern, not a policy field, and the knob silently skipped the name uniqueness check.Why:
sandlock psstill reaches an inner supervisor.psno longer prunes directories, probes pids, or guards against setup races.bindon a taken name fails with EADDRINUSE; that is the UID-wide name mutex now, instead of a side effect of a directory existing.listen()stamps the caller's pid into a socket and SO_PEERCRED hands it to whoever connects. The supervisor listens on the control socket; the child listens on the pgrp socket right aftersetpgid()and closes its copy. Sosandlock killlearns the supervisor's pid and the child's process group from twoconnect()calls and never needs the supervisor to answer, which makes it work on a supervisor that is stopped or wedged.Design
--no-supervisor, and nested. Handlers that need supervisor state takeOption<Arc<SupervisorCtx>>; without one,configserves the static policy andportsan empty map.listen()on it aftersetpgid(), then closes it; the supervisor keeps the fd for the life of the sandbox and drains the (never-spoken-on) connections it receives so the backlog stays empty.infoverb returns{mode}, replacing the mode marker. Wire protocol otherwise unchanged, still version 1.killpids of its choosing. Each request is bounded to five seconds so a stalled client cannot wedge introspection.psreads/proc/net/unix, keeps listening sockets under@sandlock/<uid>/(skipping the/pgrpentries), takes the child pid from the pgrp socket, and asks the control socket for the mode and ports. A supervisor that does not answer within the timeout is still listed, asunresponsivewith its real pid, since its socket proves it is alive.killdoeskillpg(child)andkill(supervisor)with the two peer pids and sends no request.wait()awaits the aborted control task so the names are released before it returns; running the same name twice in a row works. A forked child closes any inherited sandlock control socket other than its own pgrp socket as its first act, so a created-but-not-started sandbox (or a COW clone, which never execs) cannot pin a sibling's name, and a confined child never holds a handle to another supervisor's socket./proc/net/unixis whitespace-delimited.Removed: the runtime dir helpers,
setup_runtime_dir,cleanup_runtime_dir,list_live_sandboxes,SandboxRuntime.control_dir, and all prune logic. Pre-1.0 hard break.Tests
info,config, and an emptyports, with the supervisor pid checked against the spawned CLI process and the child pid checked to lead its own group.unresponsive;sandlock killterminates a SIGSTOPped supervisor and its child and frees the name; the same name runs twice in a row within one process./proc/net/unixparser against captured lines, including a/pgrpentry; bind is the name mutex and the listing follows the listener; the longest name plus the/pgrpsuffix fitssun_path; the client reads both pids from the kernel with nothing serving the sockets, reportsstill startingbefore the child has listened, and refuses a listener owned by another uid; the server answers its own uid and closes on another without a byte; listing survives a foreign non-UTF-8 abstract name. The uid each side expects is a parameter, so the refusal paths run unprivileged on every CI job.🤖 Generated with Claude Code