Skip to content

control: replace the /dev/shm registry with abstract Unix sockets - #207

Open
congwang-mk wants to merge 14 commits into
mainfrom
drop-control-socket-knob
Open

control: replace the /dev/shm registry with abstract Unix sockets#207
congwang-mk wants to merge 14 commits into
mainfrom
drop-control-socket-knob

Conversation

@congwang-mk

@congwang-mk congwang-mk commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the /dev/shm/sandlock-$UID/<name>/ runtime directory (pid file, mode marker, named socket) with two abstract Unix sockets per sandbox, \0sandlock/<uid>/<name> and \0sandlock/<uid>/<name>/pgrp, both bound by the supervisor before it forks. The first commit drops the control_socket opt-out knob: whether a sandbox is visible to sandlock ps is the operator's concern, not a policy field, and the knob silently skipped the name uniqueness check.

Why:

  • Nesting needs no outer grant. A nested sandlock only needs permission to create an AF_UNIX socket, not a writable /dev/shm. The Landlock abstract-socket scope restricts connecting out of a domain; a host-side sandlock ps still reaches an inner supervisor.
  • Nothing is ever stale. The names vanish with the supervisor, so ps no longer prunes directories, probes pids, or guards against setup races.
  • Uniqueness is enforced by the kernel. bind on a taken name fails with EADDRINUSE; that is the UID-wide name mutex now, instead of a side effect of a directory existing.
  • Pids come from the kernel, not from the supervisor. listen() stamps the caller's pid into a socket and SO_PEERCRED hands it to whoever connects. The supervisor listens on the control socket; the child listens on the pgrp socket right after setpgid() and closes its copy. So sandlock kill learns the supervisor's pid and the child's process group from two connect() calls and never needs the supervisor to answer, which makes it work on a supervisor that is stopped or wedged.

Design

  • Every mode binds: supervised, --no-supervisor, and nested. Handlers that need supervisor state take Option<Arc<SupervisorCtx>>; without one, config serves the static policy and ports an empty map.
  • Both sockets are bound before the fork, so a name collision fails with no child to reap. The child inherits the pgrp socket, calls listen() on it after setpgid(), then closes it; the supervisor keeps the fd for the life of the sandbox and drains the (never-spoken-on) connections it receives so the backlog stays empty.
  • New info verb returns {mode}, replacing the mode marker. Wire protocol otherwise unchanged, still version 1.
  • Abstract names have no permission bits and no owner, so authentication is mutual: the supervisor checks SO_PEERCRED on every accept and closes any connection from another uid without a response, and the client checks the listener's SO_PEERCRED before trusting either socket, so another uid cannot squat a name and feed kill pids of its choosing. Each request is bounded to five seconds so a stalled client cannot wedge introspection.
  • ps reads /proc/net/unix, keeps listening sockets under @sandlock/<uid>/ (skipping the /pgrp entries), takes the child pid from the pgrp socket, and asks the control socket for the mode and ports. A supervisor that does not answer within the timeout is still listed, as unresponsive with its real pid, since its socket proves it is alive.
  • kill does killpg(child) and kill(supervisor) with the two peer pids and sends no request.
  • wait() awaits the aborted control task so the names are released before it returns; running the same name twice in a row works. A forked child closes any inherited sandlock control socket other than its own pgrp socket as its first act, so a created-but-not-started sandbox (or a COW clone, which never execs) cannot pin a sibling's name, and a confined child never holds a handle to another supervisor's socket.
  • Sandbox names additionally reject whitespace and control characters, because /proc/net/unix is whitespace-delimited.

Removed: the runtime dir helpers, setup_runtime_dir, cleanup_runtime_dir, list_live_sandboxes, SandboxRuntime.control_dir, and all prune logic. Pre-1.0 hard break.

Tests

  • Control suite rewritten around the new mechanism: list, inspect, ports, ps ports column, name collision in both modes, invalid names (now including whitespace), no_supervisor answering info, config, and an empty ports, with the supervisor pid checked against the spawned CLI process and the child pid checked to lead its own group.
  • New: a SIGKILLed supervisor's name disappears and is reusable at once; a SIGSTOPped supervisor is listed as unresponsive; sandlock kill terminates a SIGSTOPped supervisor and its child and frees the name; the same name runs twice in a row within one process.
  • Unit: /proc/net/unix parser against captured lines, including a /pgrp entry; bind is the name mutex and the listing follows the listener; the longest name plus the /pgrp suffix fits sun_path; the client reads both pids from the kernel with nothing serving the sockets, reports still starting before the child has listened, and refuses a listener owned by another uid; the server answers its own uid and closes on another without a byte; listing survives a foreign non-UTF-8 abstract name. The uid each side expects is a parameter, so the refusal paths run unprivileged on every CI job.

🤖 Generated with Claude Code

Whether a sandbox shows up in sandlock ps is the operator's concern,
not a policy setting, and no tool exposes the knob anyway: it had no
CLI flag, no SDK field, and serde skipped it, so only the Rust builder
could reach it and nothing in-tree ever did. Worse, setting it false
skipped setup_runtime_dir, which is also where the UID-wide name
collision check lives, so the knob quietly disabled name uniqueness.
The one real reason to run without a socket, a nested sandbox whose
outer policy blocks /dev/shm, is already handled by the best-effort
fallback that warns and continues.

Signed-off-by: Cong Wang <cwang@multikernel.io>
The /dev/shm runtime directory needed a writable path from any outer
policy, left stale directories for ps to prune, and made name
uniqueness a side effect of a directory existing. An abstract Unix
socket named sandlock/<uid>/<name> has none of those: bind fails on a
taken name, the name dies with the process, and /proc/net/unix lists
it. Names must now avoid whitespace and control characters because
that listing is whitespace-delimited.

Signed-off-by: Cong Wang <cwang@multikernel.io>
Every sandbox now binds its abstract control socket at the same point
in do_spawn, whether or not a seccomp-notify supervisor exists, so ps
sees --no-supervisor and nested sandboxes without a pid file. The new
info verb carries what the pid file and mode marker held. The peer
uid check turns from a warning into a refusal because an abstract
name has no directory mode to rely on.

Signed-off-by: Cong Wang <cwang@multikernel.io>
ps lists the caller's abstract control sockets and asks each for
info; a socket that does not answer is still shown, as unresponsive,
because its existence proves the process is alive. kill gets both
pids from the same verb and signals them as before, with nothing
left on disk to clean up afterwards.

Signed-off-by: Cong Wang <cwang@multikernel.io>
The prune and pid-file tests have no subject left. In their place:
a SIGKILLed supervisor's name disappears and is reusable at once, a
stopped supervisor is still listed as unresponsive, a --no-supervisor
sandbox answers info, config, and an empty ports map, and another
uid gets no response (root only).

Signed-off-by: Cong Wang <cwang@multikernel.io>
A failed assertion must not leak a live sandbox, so the supervisor is
killed and reaped before the assert instead of after it.

Signed-off-by: Cong Wang <cwang@multikernel.io>
wait() only aborted the control task, and the listener that holds the
abstract name was dropped whenever the runtime got to cancelling it, so
a caller that ran the same name twice in a row could hit the collision
check. Awaiting the aborted task makes the release synchronous, which is
what the runtime directory removal used to be.

Signed-off-by: Cong Wang <cwang@multikernel.io>
A doc comment still described the /dev/shm runtime dir, and two lines
this branch introduced tripped clippy (int_plus_one in a unit test,
print_literal in the ps header).

Signed-off-by: Cong Wang <cwang@multikernel.io>
Abstract names have no owner, so another uid could bind
sandlock/<uid>/<name> and hand ps and kill pids of its choosing; the
client now checks SO_PEERCRED on the listener as the server already does
on the peer. A request is bounded to five seconds so one stalled
connection cannot wedge introspection, which kill now depends on. The
uid each side expects is a parameter so the refusal paths are tested
without a second uid, replacing a root-only test that had never run.

Signed-off-by: Cong Wang <cwang@multikernel.io>
The Landlock abstract-socket scope is now what keeps the confined child
off its supervisor's control socket, which is worth saying where the
mask is built. The README ps and kill samples predate the STATUS and
PORTS columns and the two-pid kill message.

Signed-off-by: Cong Wang <cwang@multikernel.io>
/proc/net/unix renders abstract names as raw bytes, so any local process
could bind a non-UTF-8 name and make read_to_string fail for every
user's sandlock ps. Ours are ASCII by validation, so a lossy decode only
affects names that never match the prefix.

Signed-off-by: Cong Wang <cwang@multikernel.io>
An abstract name stays bound while any fd refers to it, and a forked
child keeps every inherited fd until it execs, or forever in the case of
a COW clone. A sandbox created but not yet started therefore pinned the
control socket name of every sibling in the process, which showed up as
a spurious "already running" when a sibling's name was reused right
after wait(). The child now scans /proc/self/fd and closes any socket
whose abstract name is a sandlock control socket before it does anything
else. That also means a confined child never holds a handle to another
supervisor's control socket, even briefly.

Signed-off-by: Cong Wang <cwang@multikernel.io>
@congwang-mk congwang-mk changed the title sandbox: drop the control_socket opt-out knob control: replace the /dev/shm registry with abstract Unix sockets Sep 3, 2026
@congwang-mk

Copy link
Copy Markdown
Contributor Author

@sachin2605 Please take a look. Thanks!

kill and ps used the info verb to find the child and supervisor pids,
so a supervisor that never answered, stopped or deadlocked, could only
be found by grepping ss for its abstract socket. listen() stamps the
caller's pid into a socket and SO_PEERCRED hands that stamp to whoever
connects, so the pids need no cooperation if the right process calls
listen(). The supervisor now binds a second abstract socket,
sandlock/<uid>/<name>/pgrp, before it forks; the child inherits it,
calls listen() right after setpgid(), and closes it, while the
supervisor keeps the fd and drains the never-spoken-on connections.
kill connects to both sockets, reads the peer pids, and does killpg on
the child and kill on the supervisor without sending a request. Both
pids are live by construction: the sockets die with the supervisor,
and a dead child stays a zombie holding its pid until the supervisor
reaps it, at which point wait() closes the sockets. The pids leave the
wire protocol, info carries only the mode, and binding before the fork
means a name collision fails with no child to reap.

Signed-off-by: Cong Wang <cwang@multikernel.io>
Several comments still explained unique sandbox names by the per-UID
runtime directory they used to claim, and the mode field's doc said the
marker was written there at spawn time. The registry is gone; names are
claimed by binding an abstract socket and the mode is served over it,
so the comments now say that.

Signed-off-by: Cong Wang <cwang@multikernel.io>
@sachin2605

Copy link
Copy Markdown
Contributor

Hi @congwang-mk . This is a much more clean implemention for supervisor free mode. GTG from me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants