diff --git a/README.md b/README.md index bb75431..dea4972 100644 --- a/README.md +++ b/README.md @@ -1349,11 +1349,13 @@ nothing changed, which is the point. #### Accounts and groups -New accounts land in `DEFAULT_GROUPS` (`sudo,admin` unless configured) when -`--groups` is not passed. An unattended run — `--refresh`, or anything piped -into bash — never reaches the group prompt, so that default is what *every* -account it creates gets; set the key in `server.conf` on a box where new people -should not be root-equivalent. +New accounts land in `DEFAULT_GROUPS` when `--groups` is not passed. That +default is `users` while confinement is on and `sudo,admin` when it is off — an +unattended run (`--refresh`, or anything piped into bash) never reaches the +group prompt, so it is what *every* account it creates gets, and handing a new +tenant sudo on a confined box would be handing them the way straight back out +of it. Setting the key yourself, in the environment or in `server.conf`, always +wins over both. A full run is the wrong tool for a one-line change — putting somebody in `docker` should not drag an apt upgrade and a possible reboot behind it — and @@ -1409,6 +1411,53 @@ ACL support; `ls -l` then shows a trailing `+` and `getfacl` shows the rest. mode that says private while a leftover entry still hands the directory to a daemon is worse than no lock at all. +#### Confinement + +These are multi-tenant boxes — several people on one dev server, and the root +VPSes we sell hand a customer an *account*, never the root password. So every +account that is not an admin is confined, by default, with nothing to remember: + +| | before | confined | +|---|---|---| +| someone else's home | `0751`, walk in and read what is world-readable | `0700` | +| `ps -ef` | every command line on the box, tokens and all | your own processes | +| memory / tasks | unbounded; one runaway build is everybody's outage | `MemoryMax=50%`, `TasksMax=4096` | +| fork bomb | takes the box | stopped at `nproc` | +| `ssh -R` | relay out under our IP | refused (`ssh -L` still works) | +| kernel | `kptr`/`dmesg`/`perf` readable | raised to a floor | + +```sh +./root-ubuntu.sh confine # what is confined here, and what is not +./root-ubuntu.sh confine apply # re-apply without a full run +CONFINE=0 ./root-ubuntu.sh --refresh # turn the whole thing off +``` + +**root is never confined, and neither is an admin.** Every cap is lifted again +for uid 0 and for everyone in `CONFINE_EXEMPT_GROUPS`, explicitly, by uid — +because `user-.slice.d` is a prefix drop-in that otherwise applies to root's own +slice too, and the account you fix a wedged box with must not be subject to the +cap that is wedging it. Membership is recomputed on every run, so promoting +somebody releases them and demoting them confines them again without anyone +having to remember which flag was passed when the account was made. + +**It will not take sudo away from an account that already has it.** Demoting a +live sudoer unattended is how you lose a box: it might be a colleague, the only +other admin, or the account your own automation logs in as. `confine` names them +and stops there; `groups rm sudo` is the rest, with a human present. + +Two details worth knowing. nginx has to walk through a home to reach +`~/public_html`, and the old answer — `chmod o+x` — opens that path to every +account on the box; confinement grants `www-data` an **execute-only ACL** +instead, so it can traverse without listing, and only where something is +actually published. And the kernel settings are **floors, not values**: Ubuntu +ships `perf_event_paranoid=4` and `unprivileged_bpf_disabled=2` already stricter +than what is asked for, so writing them unconditionally would loosen the box in +the name of hardening it. Anything already stricter is left alone. + +Setting a home to `0700` closes the path to what is inside it; it does not +change the mode of files that are already there. `CONFINE_UMASK` stops new ones +being created world-readable behind it. + #### Configuring it Read from the environment first, then `$SERVER_CONFIG`, then diff --git a/root-ubuntu.sh b/root-ubuntu.sh index 2c44a53..24836d2 100755 --- a/root-ubuntu.sh +++ b/root-ubuntu.sh @@ -39,6 +39,9 @@ # 9. a per-user ssh-agent as a systemd user service # 10. motd from $MOTD_URL # 11. nginx per-user pages, per-user dev apps, TLS +# 12. confinement: every non-admin account confined (home, /proc, memory, +# tasks, ssh forwarding). Runs after the accounts exist, since which +# side of the line someone is on is decided by their groups. # # Usage, as root: # ./root-ubuntu.sh # first run, or a refresh @@ -54,6 +57,18 @@ # ./root-ubuntu.sh share /mnt/volume --group www-data -R # # ...to a second group as well (acl) # +# Confinement (see "confinement" below, or `confine --help`): +# ./root-ubuntu.sh confine # what is confined on this box, and what is not +# ./root-ubuntu.sh confine apply # re-apply it without a full run +# +# Every account that is not an admin is confined, by default, with nothing to +# remember: its home is 0700, it sees only its own processes, its memory and +# task count are capped, and it cannot relay out over ssh. root and anyone in +# sudo/admin are exempt -- explicitly, by uid, because the account you fix a +# wedged box with must not be subject to the cap that is wedging it. It never +# takes sudo away from an account that already has it; `confine` names those +# and leaves the decision to you. CONFINE=0 turns the whole thing off. +# # Sandboxed tenants -- an account that gets its own machine instead of a shell # on this one (see "sandboxes" below, or `sandbox --help`): # ./root-ubuntu.sh alice --sandbox # alice gets a container, no host shell @@ -118,9 +133,22 @@ # EARLYOOM_MEM=10,5 / EARLYOOM_SWAP=10,5 SIGTERM% , SIGKILL% # EARLYOOM_AVOID=... / EARLYOOM_PREFER= unquoted regexes (see the note) # ASSUME_YES=1 don't prompt (defaults: $DEFAULT_GROUPS; no privkey copy) -# DEFAULT_GROUPS=... groups an account lands in when --groups is not passed -# (default sudo,admin). An unattended run never prompts, so -# this is what every account it creates gets. +# DEFAULT_GROUPS=... groups an account lands in when --groups is not passed. +# An unattended run never prompts, so this is what every +# account it creates gets. Default: users when confinement is +# on, sudo,admin when it is not. +# CONFINE=0 do not confine accounts at all (default: 1) +# CONFINE_EXEMPT_GROUPS=sudo,admin who counts as an admin, and is exempt +# CONFINE_HOME_MODE=0700 mode every human home is set to +# CONFINE_UMASK=027 umask for accounts created from here on +# CONFINE_HIDEPID=0 leave /proc world-readable (default: hidepid) +# CONFINE_MEMORY_MAX=50% / CONFINE_MEMORY_HIGH=35% / CONFINE_TASKS_MAX=4096 +# per-account caps on the systemd user slice +# CONFINE_CPU_QUOTA=200% a hard CPU ceiling; by default CPU is a weight, +# so a build gets the whole box when the box is idle +# CONFINE_NPROC=4096 / CONFINE_NOFILE=16384 / CONFINE_MAXLOGINS=20 +# CONFINE_SSH_FORWARDING=local|no|yes ssh -L yes, ssh -R no (default local) +# CONFINE_PROC_UNITS=polkit.service units that must still read /proc # NO_REBOOT=1 skip the reboot at the end # MOTD_URL=... override the motd endpoint # TS_AUTHKEY=... tailscale auth key, to join the tailnet unattended @@ -484,8 +512,98 @@ GROUP_CHOICES=(sudo admin docker adm www-data users) # unattended run takes it verbatim for every account it creates. Assigning it # unconditionally (as this line used to) meant a box that had configured, say, # www-data,users,docker still got its new accounts put in sudo,admin. +# Was it said out loud, or is it just the default? Confinement turns the +# default from sudo,admin into users, and the difference between "nobody chose +# this" and "the config file chose this" is the whole basis for being allowed +# to change it underneath them. +DEFAULT_GROUPS_EXPLICIT="${DEFAULT_GROUPS+1}" DEFAULT_GROUPS="${DEFAULT_GROUPS:-sudo,admin}" +# ------------------------------------------------------- confinement --- +# +# These boxes are multi-tenant. Several people share one dev server, and the +# root VPSes we sell hand a customer an ACCOUNT, never the root password. So +# the default has to be that an account is confined and an admin is let out, +# rather than the other way round -- a model where you have to remember to +# lock someone down is a model that eventually forgets. +# +# Three tiers, and the boundary is group membership: +# +# root untouched, always. Never capped, never blinded, never denied. +# Every limit below is explicitly lifted for uid 0, because the +# account you use to fix a wedged box must not be the account the +# wedge applies to. +# admin anyone in $CONFINE_EXEMPT_GROUPS. Us. Exempt from the resource +# caps, and kept able to see every process on the box. +# confined everybody else, by default, with nothing to remember. Confined +# home, own processes only, capped memory and tasks, no relay out +# over ssh. +# +# What each mechanism is actually worth is written up at configure_confinement +# below. CONFINE=0 turns all of it off and gives back the older behaviour. +CONFINE="${CONFINE:-1}" + +# The group that carries the confinement. Membership is recomputed on every run +# from "is this account in an exempt group", so it converges rather than +# drifting: promote someone and the next run takes them back out of it. +CONFINE_GROUP="${CONFINE_GROUP:-confined}" +CONFINE_EXEMPT_GROUPS="${CONFINE_EXEMPT_GROUPS:-sudo,admin}" + +# Members still see every process once /proc is mounted hidepid. Admins go in +# here; a monitoring agent that reads /proc should too. +CONFINE_PROC_GROUP="${CONFINE_PROC_GROUP:-proc}" +CONFINE_HIDEPID="${CONFINE_HIDEPID:-1}" + +# Units that must keep seeing other people's processes. polkit runs as polkitd +# rather than as root and reads /proc/ of whoever is asking it for +# authorisation, so it goes blind under hidepid without this. Anything else you +# run that reads /proc as a non-root user belongs on this list. +CONFINE_PROC_UNITS="${CONFINE_PROC_UNITS:-polkit.service}" + +# 0700, so a home is the account's own business. This is the single biggest +# item in here: useradd on Ubuntu makes a home 0750 and this script then +# chmod o+x'd it, which is enough for anyone with a shell on the box to walk +# into someone else's ~/.config and read whatever is world-readable in there. +CONFINE_HOME_MODE="${CONFINE_HOME_MODE:-0700}" + +# ...and 027, so what gets CREATED in there from now on is not world-readable +# either. Fixing the mode of the home does not fix the mode of the files +# already inside it, but it does close the path to them. +CONFINE_UMASK="${CONFINE_UMASK:-027}" + +CONFINE_SYSCTL="${CONFINE_SYSCTL:-1}" + +# Resource caps on the per-user systemd slice. Memory and tasks are HARD caps, +# because their failure mode is the whole box going down with one account. CPU +# is a weight rather than a quota, because ITS failure mode is only slowness, +# and throttling a build while the box is otherwise idle is a bad trade. Set +# CONFINE_CPU_QUOTA (e.g. 200%) when you would rather have the ceiling. +CONFINE_MEMORY_HIGH="${CONFINE_MEMORY_HIGH:-35%}" +CONFINE_MEMORY_MAX="${CONFINE_MEMORY_MAX:-50%}" +CONFINE_TASKS_MAX="${CONFINE_TASKS_MAX:-4096}" +CONFINE_CPU_WEIGHT="${CONFINE_CPU_WEIGHT:-100}" +CONFINE_CPU_QUOTA="${CONFINE_CPU_QUOTA:-}" + +# PAM limits, which bite at login rather than in the cgroup: a fork bomb is +# stopped by nproc long before MemoryMax notices it happening. +CONFINE_NPROC="${CONFINE_NPROC:-4096}" +CONFINE_NOFILE="${CONFINE_NOFILE:-16384}" +CONFINE_MAXLOGINS="${CONFINE_MAXLOGINS:-20}" + +# local -- ssh -L to their own app still works, ssh -R relays do not +# no -- no forwarding at all, for a box whose tenants only need a shell +# yes -- off; this box is a jump host on purpose +CONFINE_SSH_FORWARDING="${CONFINE_SSH_FORWARDING:-local}" +CONFINE_SSH="${CONFINE_SSH:-1}" + +# An unattended run creates accounts with $DEFAULT_GROUPS, and that default was +# sudo,admin -- which on a confined box would hand every new tenant the way +# out of confinement on the first --refresh. Only the default moves: a +# DEFAULT_GROUPS in the environment or in server.conf is an answer, and stands. +if [[ "$CONFINE" == 1 && -z "$DEFAULT_GROUPS_EXPLICIT" ]]; then + DEFAULT_GROUPS="users" +fi + USERS=() # alice@example -- new this run, get the full treatment USER_GROUPS=() # sudo,admin -- index-matched to USERS KNOWN_USERS=() # logins provisioned by an earlier run, refreshed not created @@ -1762,6 +1880,697 @@ cmd_groups() { esac } +# --------------------------------------------------------- user confine --- +# +# Confine every account that is not an admin. See the tier model up at the +# CONFINE knobs; this is what each tier actually costs the tenant. +# +# The threat is not a stranger -- it is the account we just created. Someone +# with a shell on a shared box, or the customer of a VPS we sold, who should be +# able to do their own work and nothing else. Five things they could do before: +# +# 1. read other people's homes. A home is 0750 and this script chmod o+x'd +# it, so ~/.config was walkable and anything world-readable in there -- +# session tokens, api keys people leave in dotfiles -- was readable by +# every account on the box. This is the one that has actually leaked. +# 2. watch other people work. /proc is world-readable, so `ps -ef` hands over +# every command line on the machine, and command lines carry tokens. +# 3. take the box down. No per-account memory or task cap, so one runaway +# build or fork bomb is everybody's outage. earlyoom (above) picks up the +# pieces; this stops the pieces. +# 4. relay through us. ssh -R turns the box into an open proxy under our IP, +# which is how a dev box ends up on a blocklist. +# 5. read the kernel back. kptr/dmesg/perf leak the addresses and traces that +# turn a local bug into a local root. +# +# What it deliberately does NOT do is take sudo away from an account that +# already has it. Demoting a live sudoer unattended is how you lose a box -- +# maybe it is a colleague, maybe it is the only other admin, maybe it is the +# account your own automation logs in as. `confine status` names them and +# `groups rm sudo` demotes them, both with a human present. + +_confine_on() { [[ "$CONFINE" == 1 ]]; } + +# Is this account exempt -- root, or an admin? The safe direction on every +# unknown is "no": an account we cannot classify is confined, not let out. +_confine_exempt() { + local login="$1" g uid + uid="$(id -u "$login" 2>/dev/null)" || return 1 + [[ "$uid" == 0 ]] && return 0 + for g in ${CONFINE_EXEMPT_GROUPS//,/ }; do + [[ -n "${g// }" ]] || continue + _groups_in "$login" "$g" && return 0 + done + return 1 +} + +# Every human account on the box. Not $KNOWN_USERS: an account this script +# never created still shares the machine, and confinement with a hole in it the +# shape of the cloud image's default user is not confinement. +_confine_humans() { + getent passwd | awk -F: -v floor="${GROUPS_SYSTEM_FLOOR:-1000}" \ + '$3 >= floor && $3 < 65534 && $7 !~ /(nologin|\/false|sync)$/ { print $1 }' +} + +_confine_group_ensure() { + local g="$1" + getent group "$g" >/dev/null && return 0 + valid_group "$g" || { warn "confine: '$g' is not a valid group name"; return 1; } + groupadd "$g" >/dev/null 2>&1 || { warn "confine: could not create group $g"; return 1; } + note "created group $g" + return 0 +} + +# Recompute who is in which tier. Convergent in both directions: a promotion +# takes someone out of the confined group on the next run, a demotion puts them +# back, and neither needs the flag that was passed when the account was made. +sync_confine_membership() { + local login rc=0 + _confine_group_ensure "$CONFINE_GROUP" || return 1 + if [[ "$CONFINE_HIDEPID" == 1 ]]; then + _confine_group_ensure "$CONFINE_PROC_GROUP" || rc=1 + fi + + while read -r login; do + [[ -n "$login" ]] || continue + if _confine_exempt "$login"; then + if _groups_in "$login" "$CONFINE_GROUP"; then + gpasswd -d "$login" "$CONFINE_GROUP" >/dev/null 2>&1 \ + && note "confine: $login is an admin -- released from $CONFINE_GROUP" + fi + if [[ "$CONFINE_HIDEPID" == 1 ]] && getent group "$CONFINE_PROC_GROUP" >/dev/null \ + && ! _groups_in "$login" "$CONFINE_PROC_GROUP"; then + usermod -aG "$CONFINE_PROC_GROUP" "$login" \ + && note "confine: $login -> $CONFINE_PROC_GROUP (keeps seeing every process)" + fi + else + if ! _groups_in "$login" "$CONFINE_GROUP"; then + usermod -aG "$CONFINE_GROUP" "$login" \ + && note "confine: confined $login" \ + || { warn "confine: could not confine $login"; rc=1; } + fi + fi + done < <(_confine_humans) + return $rc +} + +# Emit "key = value", but only when the running kernel is BELOW it. +# +# Every key here is "higher is stricter", and Ubuntu already ships two of them +# stricter than the floor we ask for -- kernel.perf_event_paranoid=4 against +# our 3, kernel.unprivileged_bpf_disabled=2 against our 1. Writing the file +# unconditionally would therefore LOOSEN the box in the name of hardening it, +# quietly, on exactly the distribution we run everywhere. So each line is a +# floor rather than a setting, and a key the kernel does not have at all +# (yama on a kernel built without it) is skipped rather than guessed at. +_confine_sysctl_floor() { + # "floor", not "want": `want` is an array elsewhere in this file, and the + # linter does not scope locals, so that name costs two false SC2178/SC2128 + # warnings on a string that is only ever a number. (A comment cannot open + # with the linter's own name either -- that is read as a directive.) + local key="$1" floor="$2" have + have="$(sysctl -n "$key" 2>/dev/null)" || return 1 + [[ "$have" =~ ^-?[0-9]+$ ]] || return 1 + (( have >= floor )) && return 1 + printf '%s = %s\n' "$key" "$floor" + return 0 +} + +configure_confine_sysctl() { + local conf=/etc/sysctl.d/62-profullstack-confine.conf body="" dumpable + + if [[ "$CONFINE_SYSCTL" != 1 ]]; then + info "confine: sysctl hardening off (CONFINE_SYSCTL=0)" + return 0 + fi + + # kptr/dmesg/perf: stop handing out the kernel addresses and traces that + # turn a local bug into local root. ptrace_scope 1 keeps a debugger working + # on your own children while stopping it from attaching to anything else. + # protected_*: the /tmp symlink and hardlink games, which only matter on a + # box where someone else is also writing to /tmp. suid_dumpable 0: a core + # from a setuid binary is a memory image nobody should be handed. + body+="$(_confine_sysctl_floor kernel.kptr_restrict 2)"$'\n' + body+="$(_confine_sysctl_floor kernel.dmesg_restrict 1)"$'\n' + body+="$(_confine_sysctl_floor kernel.perf_event_paranoid 3)"$'\n' + body+="$(_confine_sysctl_floor kernel.yama.ptrace_scope 1)"$'\n' + body+="$(_confine_sysctl_floor kernel.unprivileged_bpf_disabled 1)"$'\n' + body+="$(_confine_sysctl_floor net.core.bpf_jit_harden 2)"$'\n' + body+="$(_confine_sysctl_floor fs.protected_symlinks 1)"$'\n' + body+="$(_confine_sysctl_floor fs.protected_hardlinks 1)"$'\n' + body+="$(_confine_sysctl_floor fs.protected_fifos 2)"$'\n' + body+="$(_confine_sysctl_floor fs.protected_regular 2)"$'\n' + + # fs.suid_dumpable is the one key here that does NOT run in one direction, + # so it cannot be a floor: 0 (never dump) is safest, 2 (dump, readable only + # by root) is safe, and 1 -- dump like any other process, into a file the + # account can read back -- is the dangerous one in the middle. Ordering + # them numerically and taking the larger, which is what a floor does, would + # have left a box sitting on 2 alone and called it hardened. It is an exact + # value instead, and 0 can never be a loosening of anything. + dumpable="$(sysctl -n fs.suid_dumpable 2>/dev/null)" + [[ "$dumpable" =~ ^[0-9]+$ && "$dumpable" != 0 ]] && body+='fs.suid_dumpable = 0'$'\n' + + body="$(printf '%s' "$body" | grep -v '^$')" + + if [[ -z "$body" ]]; then + # Nothing to raise. Leave no file behind claiming otherwise. + if [[ -f "$conf" ]]; then + rm -f "$conf" && note "confine: kernel already at or above every floor -- removed $conf" + else + info "confine: kernel already at or above every hardening floor" + fi + return 0 + fi + + if write_if_changed "$conf" 0644 </dev/null || true + note "confine: kernel hardening ($(printf '%s' "$body" | wc -l) floor(s) raised)" + fi + return 0 +} + +# /proc mounted hidepid, so an account sees its own processes and nothing else. +# +# Order matters and is the whole difficulty. The live remount is attempted +# FIRST and /etc/fstab is only written once it has actually worked: a kernel +# that rejects the option would otherwise fail the mount at boot, and finding +# that out at boot is finding it out in the worst place. hidepid=invisible is +# the modern spelling; older kernels want the numeric 2, so both are tried. +# +# root is not affected by any of this -- it sees everything regardless, which +# is why earlyoom (running as root) can still find the hog to kill. +configure_confine_proc() { + local gid opts="" want spelling ok=0 + + [[ "$CONFINE_HIDEPID" == 1 ]] || { info "confine: hidepid off (CONFINE_HIDEPID=0)"; return 0; } + + # A container is handed its /proc by the runtime and cannot remount it. + if [[ "$(systemd-detect-virt --container 2>/dev/null)" != "none" ]]; then + info "confine: inside a container -- /proc is the runtime's to mount, skipping hidepid" + return 0 + fi + + getent group "$CONFINE_PROC_GROUP" >/dev/null || _confine_group_ensure "$CONFINE_PROC_GROUP" || return 1 + gid="$(getent group "$CONFINE_PROC_GROUP" | cut -d: -f3)" + [[ -n "$gid" ]] || { warn "confine: no gid for group $CONFINE_PROC_GROUP"; return 1; } + + # The units that read other people's /proc as a non-root user, before the + # mount that would blind them rather than after. + _confine_proc_units "$gid" + + for spelling in invisible 2; do + want="hidepid=$spelling,gid=$gid" + if mount -o "remount,$want" /proc 2>/dev/null \ + && grep -q ' /proc ' /proc/mounts && grep ' /proc ' /proc/mounts | grep -q hidepid; then + opts="rw,nosuid,nodev,noexec,relatime,$want" + ok=1 + break + fi + done + + if [[ "$ok" != 1 ]]; then + warn "confine: this kernel would not remount /proc with hidepid -- processes stay visible" + return 1 + fi + + if _confine_fstab_proc "$opts" "$gid"; then + note "confine: /proc hidepid -- an account sees only its own processes" + else + info "confine: /proc already hidepid ($opts)" + fi + return 0 +} + +# SupplementaryGroups for the units that would otherwise go blind. Written per +# unit, and only for units this box actually has. +_confine_proc_units() { + local u dir + for u in ${CONFINE_PROC_UNITS//,/ }; do + [[ -n "${u// }" ]] || continue + systemctl list-unit-files "$u" >/dev/null 2>&1 || continue + [[ -n "$(systemctl list-unit-files --no-legend "$u" 2>/dev/null)" ]] || continue + dir="/etc/systemd/system/$u.d" + install -d -m 0755 "$dir" + if write_if_changed "$dir/50-profullstack-confine.conf" 0644 < of processes belonging to other people, so it needs the group +# that is still allowed to see them. +[Service] +SupplementaryGroups=$CONFINE_PROC_GROUP +EOF + then + systemctl daemon-reload 2>/dev/null || true + systemctl try-restart "$u" 2>/dev/null || true + note "confine: $u keeps /proc visibility" + fi + done + return 0 +} + +# Rewrite the /proc line in /etc/fstab. Our own comment lines carry a marker so +# that they are stripped and re-added rather than accumulating a fresh pair on +# every single run. +_confine_fstab_proc() { + local opts="$1" gid="$2" tmp + tmp="$(mktemp)" || return 1 + grep -vE '^#root-ubuntu:proc|^[^#]*[[:space:]]/proc[[:space:]]' /etc/fstab >"$tmp" 2>/dev/null + { + printf '#root-ubuntu:proc hidepid -- an account sees only its own processes.\n' + printf '#root-ubuntu:proc gid %s is the "%s" group; its members still see them all.\n' \ + "$gid" "$CONFINE_PROC_GROUP" + printf 'proc /proc proc %s 0 0\n' "$opts" + } >>"$tmp" + if cmp -s "$tmp" /etc/fstab; then + rm -f "$tmp" + return 1 + fi + install -m 0644 "$tmp" /etc/fstab + rm -f "$tmp" + return 0 +} + +# Resource caps, in two places because they catch two different things. +# +# PAM limits bite at login: nproc stops a fork bomb in the fork, before there +# is anything for a cgroup to account. The systemd slice bites afterwards, and +# is the only one of the two that can cap memory. +# +# Then both are LIFTED for root and for every admin, explicitly, by name. That +# is the point of the exercise: the account you fix the box with must not be +# subject to the cap that is wedging it. +configure_confine_limits() { + local dir=/etc/systemd/system/user-.slice.d changed=0 quota="" + + if write_if_changed /etc/security/limits.d/60-profullstack-confine.conf 0644 </dev/null + return 0 +} + +# Lift the caps back off root and every admin, and take the lifting away again +# from anyone who is no longer one. Written per uid because that is the only +# way to override a prefix drop-in for one instance of it. +_confine_slice_exemptions() { + local login uid want=() u f changed=0 + + want=(0) + while read -r login; do + [[ -n "$login" ]] || continue + _confine_exempt "$login" || continue + uid="$(id -u "$login" 2>/dev/null)" || continue + want+=("$uid") + done < <(_confine_humans) + + for uid in "${want[@]}"; do + install -d -m 0755 "/etc/systemd/system/user-$uid.slice.d" + if write_if_changed "/etc/systemd/system/user-$uid.slice.d/60-profullstack-exempt.conf" 0644 </dev/null || true + changed=1 + note "confine: uid $u is no longer an admin -- resource caps now apply" + done + + [[ "$changed" == 1 ]] +} + +# UMASK and HOME_MODE in login.defs, so that what is created from here on is +# born private. Neither fixes a file that already exists -- that is what +# fix_home_permissions does to the home itself -- but together they stop the +# box from generating the problem again with every new account. +configure_confine_login() { + local changed=0 + _set_login_def UMASK "$CONFINE_UMASK" && { changed=1; note "confine: login.defs UMASK $CONFINE_UMASK"; } + _set_login_def HOME_MODE "$CONFINE_HOME_MODE" && { changed=1; note "confine: login.defs HOME_MODE $CONFINE_HOME_MODE"; } + return 0 +} + +# Set one key in /etc/login.defs. Not a file we own, so the existing line is +# edited in place -- commented or not -- and only appended when there is none. +_set_login_def() { + local key="$1" val="$2" file=/etc/login.defs tmp + [[ -w "$file" ]] || return 1 + grep -qE "^[[:space:]]*${key}[[:space:]]+${val}[[:space:]]*$" "$file" && return 1 + tmp="$(mktemp)" || return 1 + if grep -qE "^[[:space:]]*#?[[:space:]]*${key}[[:space:]]" "$file"; then + sed -E "0,/^[[:space:]]*#?[[:space:]]*${key}[[:space:]].*$/s##${key}\t${val}#" "$file" >"$tmp" + else + { cat "$file"; printf '\n# root-ubuntu.sh: born-private defaults for new accounts\n%s\t%s\n' "$key" "$val"; } >"$tmp" + fi + if cmp -s "$tmp" "$file"; then rm -f "$tmp"; return 1; fi + install -m 0644 "$tmp" "$file" + rm -f "$tmp" + return 0 +} + +# What a confined account may do over ssh. +# +# Forwarding is the interesting one, and the interesting direction is OUT. A +# tenant with -L can reach a service on localhost -- but so can any shell on +# the box, so -L costs nothing they did not already have. -R is different: it +# turns the machine into an open relay under our IP, and that is how a dev box +# ends up on a blocklist. Hence "local" by default: keep ssh -L for previewing +# your own app, drop the relay. +configure_confine_ssh() { + local conf=/etc/ssh/sshd_config.d/60-profullstack-confine.conf backup fwd + + [[ "$CONFINE_SSH" == 1 ]] || { info "confine: sshd policy off (CONFINE_SSH=0)"; return 0; } + [[ -d /etc/ssh/sshd_config.d ]] || { info "confine: no sshd_config.d on this box -- skipping"; return 0; } + + # A drop-in in a directory nothing includes is a policy that does not exist. + if ! grep -qE '^[[:space:]]*Include[[:space:]]+/etc/ssh/sshd_config\.d/' /etc/ssh/sshd_config 2>/dev/null; then + warn "confine: /etc/ssh/sshd_config has no Include for sshd_config.d -- ssh policy NOT applied" + return 1 + fi + + case "$CONFINE_SSH_FORWARDING" in + local|no|yes) fwd="$CONFINE_SSH_FORWARDING" ;; + *) warn "confine: CONFINE_SSH_FORWARDING='$CONFINE_SSH_FORWARDING' is not local|no|yes -- using local"; fwd=local ;; + esac + + backup="$(mktemp)" + [[ -f "$conf" ]] && cp -f "$conf" "$backup" + + write_if_changed "$conf" 0644 </dev/null; then + if [[ -s "$backup" ]]; then + install -m 0644 "$backup" "$conf" + else + rm -f "$conf" + fi + rm -f "$backup" + warn "confine: sshd rejected the policy drop-in -- rolled back, sshd untouched" + return 1 + fi + rm -f "$backup" + + systemctl reload ssh 2>/dev/null || systemctl reload sshd 2>/dev/null || true + note "confine: sshd policy for @$CONFINE_GROUP (forwarding=$fwd)" + return 0 +} + +# A home is the account's own business: $CONFINE_HOME_MODE, and an ACL for +# nginx where nginx actually has something to serve. +# +# The mode alone would break the per-user web pages, because www-data has to +# walk through the home to reach ~/public_html. The old answer was chmod o+x, +# which opens that path to every account on the box as well. An ACL says the +# same thing to one user instead of to everyone -- and it is EXECUTE only, so +# www-data can traverse the home without being able to list it. +# +# Granted only where there is something published, so a home with no web +# content grants nothing at all. +_confine_home_mode() { + local home="$1" login="$2" + + chmod "$CONFINE_HOME_MODE" "$home" 2>/dev/null || { + warn "confine: could not chmod $CONFINE_HOME_MODE $home" + return 1 + } + + if [[ -d "$home/public_html" || -d "$home/apps" ]] && getent group "$WEB_GROUP" >/dev/null; then + if command -v setfacl >/dev/null 2>&1; then + setfacl -m "u:$WEB_GROUP:--x" "$home" 2>/dev/null \ + || warn "confine: could not grant $WEB_GROUP traverse on $home" + else + # No acl package yet (it arrives in the apt stage). Fall back to the + # old behaviour rather than silently breaking the user's web page -- + # less private, but a working box, and the next run fixes it. + chmod o+x "$home" + warn "confine: setfacl missing -- $home left traversable; re-run after apt installs acl" + fi + fi + return 0 +} + +# Every human home, not only the accounts this script provisioned. The cloud +# image's own `ubuntu` was never created by us and so never had its permissions +# touched -- and it is on every box. +configure_confine_homes() { + local login home before + while read -r login; do + [[ -n "$login" ]] || continue + home="$(user_home "$login")" + [[ -n "$home" && -d "$home" ]] || continue + before="$(stat -c '%a' "$home" 2>/dev/null)" + _confine_home_mode "$home" "$login" || continue + [[ "$(stat -c '%a' "$home" 2>/dev/null)" != "$before" ]] \ + && note "confine: $home $before -> $(stat -c '%a' "$home" 2>/dev/null)" + done < <(_confine_humans) + return 0 +} + +configure_confinement() { + local rc=0 + sync_confine_membership || rc=1 + configure_confine_homes || rc=1 + configure_confine_sysctl || rc=1 + configure_confine_login || rc=1 + configure_confine_limits || rc=1 + configure_confine_proc || rc=1 + configure_confine_ssh || rc=1 + _confine_report_sudoers + return $rc +} + +# The one thing this will not do by itself. Naming them is the whole point: +# an account with sudo is outside confinement no matter what else is set. +_confine_report_sudoers() { + local login extra=() + while read -r login; do + [[ -n "$login" ]] || continue + _confine_exempt "$login" || continue + extra+=("$login") + done < <(_confine_humans) + [[ ${#extra[@]} -gt 0 ]] || return 0 + info "confine: NOT confined (admins): ${extra[*]}" + info "confine: each of those can leave confinement at will -- \`groups rm sudo\` to demote" + return 0 +} + +confine_usage() { + cat < sudo +Putting them back: $0 groups rm sudo +EOF + exit "${1:-0}" +} + +# A read-only report on the posture. The point is to be able to answer "is this +# box actually tight?" without reading five config files -- and to be honest +# about the parts that are not, rather than printing a row of ticks. +_confine_status() { + local login home mode tier admins=() confined=() leaky=() key want have + + printf '\n\033[1mconfine\033[0m: %s\n' \ + "$([[ "$CONFINE" == 1 ]] && echo "on" || echo "OFF (CONFINE=0)")" + printf ' admin groups : %s\n' "$CONFINE_EXEMPT_GROUPS" + printf ' confine group: %s\n' "$CONFINE_GROUP" + + printf '\n\033[1maccounts\033[0m\n' + while read -r login; do + [[ -n "$login" ]] || continue + home="$(user_home "$login")" + mode="$(stat -c '%a' "$home" 2>/dev/null || echo '?')" + if _confine_exempt "$login"; then + tier="admin"; admins+=("$login") + else + tier="confined"; confined+=("$login") + fi + # 'other' with any bit set means every account on the box can at least + # walk in. That is the leak this whole thing exists to close. + if [[ "$mode" =~ ^[0-7]?[0-7][0-7][1-7]$ ]]; then + leaky+=("$login") + printf ' %-8s %-16s %s \033[1;33m<- other can traverse/read\033[0m\n' "$tier" "$login" "$mode" + else + printf ' %-8s %-16s %s\n' "$tier" "$login" "$mode" + fi + done < <(_confine_humans) + + printf '\n\033[1mmechanisms\033[0m\n' + if grep ' /proc ' /proc/mounts 2>/dev/null | grep -q hidepid; then + printf ' hidepid yes (an account sees only its own processes)\n' + else + printf ' hidepid \033[1;33mno\033[0m (ps -ef shows every command line on the box)\n' + fi + if [[ -f /etc/systemd/system/user-.slice.d/50-profullstack-confine.conf ]]; then + printf ' slice caps yes MemoryMax=%s TasksMax=%s\n' "$CONFINE_MEMORY_MAX" "$CONFINE_TASKS_MAX" + printf ' exempt uids %s\n' \ + "$(ls -d /etc/systemd/system/user-*.slice.d 2>/dev/null \ + | sed 's#.*/user-##; s#\.slice\.d##' | tr '\n' ' ')" + else + printf ' slice caps \033[1;33mno\033[0m (one account can take the box down)\n' + fi + if [[ -f /etc/security/limits.d/60-profullstack-confine.conf ]]; then + printf ' pam limits yes nproc=%s nofile=%s\n' "$CONFINE_NPROC" "$CONFINE_NOFILE" + else + printf ' pam limits \033[1;33mno\033[0m\n' + fi + if [[ -f /etc/ssh/sshd_config.d/60-profullstack-confine.conf ]]; then + printf ' ssh policy yes forwarding=%s\n' "$CONFINE_SSH_FORWARDING" + else + printf ' ssh policy \033[1;33mno\033[0m (ssh -R can relay through this box)\n' + fi + + printf '\n\033[1mkernel floors\033[0m\n' + for key in kernel.kptr_restrict:2 kernel.dmesg_restrict:1 kernel.perf_event_paranoid:3 \ + kernel.yama.ptrace_scope:1 fs.protected_fifos:2 fs.protected_regular:2; do + want="${key#*:}"; key="${key%%:*}" + have="$(sysctl -n "$key" 2>/dev/null)" || { printf ' %-32s absent on this kernel\n' "$key"; continue; } + if [[ "$have" =~ ^-?[0-9]+$ ]] && (( have >= want )); then + printf ' %-32s %s\n' "$key" "$have" + else + printf ' %-32s \033[1;33m%s\033[0m (floor %s)\n' "$key" "$have" "$want" + fi + done + # Reported separately for the same reason it is set separately: 0 is safest + # and 1 is the dangerous one, so "bigger is better" does not hold here. + have="$(sysctl -n fs.suid_dumpable 2>/dev/null)" + if [[ "$have" == 0 ]]; then + printf ' %-32s %s\n' fs.suid_dumpable "$have" + else + printf ' %-32s \033[1;33m%s\033[0m (wanted 0)\n' fs.suid_dumpable "${have:-?}" + fi + + printf '\n' + if [[ ${#admins[@]} -gt 0 ]]; then + printf '%d account(s) are NOT confined, because they are admins: %s\n' \ + "${#admins[@]}" "${admins[*]}" + printf 'An admin can leave confinement at will. Demote with: %s groups rm sudo\n' "$0" + fi + [[ ${#confined[@]} -gt 0 ]] && printf '%d account(s) confined: %s\n' "${#confined[@]}" "${confined[*]}" + if [[ ${#leaky[@]} -gt 0 ]]; then + printf '\n\033[1;33m%d home(s) are still reachable by other accounts: %s\033[0m\n' \ + "${#leaky[@]}" "${leaky[*]}" + printf 'Fix with: sudo %s confine apply\n' "$0" + fi + return 0 +} + +cmd_confine() { + local action="${1:-status}" + case "$action" in + -h|--help|help) confine_usage 0 ;; + status|"") _confine_status ;; + apply) + [[ $EUID -eq 0 ]] || die "confine apply: must run as root (try: sudo $0 confine apply)" + [[ "$CONFINE" == 1 ]] || die "confine apply: CONFINE=0 -- nothing to apply" + configure_confinement + printf '\n' + _confine_status + ;; + *) die "confine: unknown action '$action' (try: status, apply)" ;; + esac +} # --------------------------------------------------------------- sandboxes --- # @@ -2668,7 +3477,7 @@ usage() { SUBCMD="" SUBARGS=() case "${1:-}" in - mount|umount|mounts|share|groups|sandbox) + mount|umount|mounts|share|groups|sandbox|confine) SUBCMD="$1"; shift; SUBARGS=("$@"); set -- ;; esac @@ -2727,6 +3536,7 @@ case "$SUBCMD" in share) cmd_share ${SUBARGS[@]+"${SUBARGS[@]}"}; exit $? ;; groups) cmd_groups ${SUBARGS[@]+"${SUBARGS[@]}"}; exit $? ;; sandbox) cmd_sandbox ${SUBARGS[@]+"${SUBARGS[@]}"}; exit $? ;; + confine) cmd_confine ${SUBARGS[@]+"${SUBARGS[@]}"}; exit $? ;; esac # Root, not sudo-capable: this writes to /etc, creates accounts and drives @@ -3965,8 +4775,12 @@ fix_home_permissions() { [[ -d "$home" ]] || { warn "no home dir $home"; return 1; } chown "$login:$login" "$home" - chmod g-w,o-w "$home" - chmod o+x "$home" + if _confine_on; then + _confine_home_mode "$home" "$login" + else + chmod g-w,o-w "$home" + chmod o+x "$home" + fi if [[ -d "$home/.ssh" ]]; then chown -R "$login:$login" "$home/.ssh" @@ -5862,11 +6676,9 @@ refresh_user() { return 0 } -# Before the accounts, so that the unit is already in place by the time -# refresh_user turns on lingering for each of them. -# Before the accounts too: creating a tenant needs a pool, a bridge and a -# profile to exist, and reconciling one needs the daemon up. A box with no -# tenants and SANDBOX=0 does none of this and never installs incus. +# Before the accounts: creating a tenant needs a pool, a bridge and a profile +# to exist, and reconciling one needs the daemon up. A box with no tenants and +# SANDBOX=0 does none of this and never installs incus. if [[ "$SKIP_SANDBOX" == 1 ]]; then log "skipping sandboxes (--skip-sandbox)" elif _sandbox_configured; then @@ -5876,6 +6688,8 @@ elif _sandbox_configured; then fi fi +# Before the accounts, so that the unit is already in place by the time +# refresh_user turns on lingering for each of them. log "installing the ssh-agent user service" try "ssh-agent" install_ssh_agent @@ -5895,6 +6709,16 @@ if [[ ${#KNOWN_USERS[@]} -gt 0 ]]; then unset _l fi +# After every account exists and has its groups, because who is confined is +# decided by group membership -- and before the tools, so that anything the +# rest of the run installs into a home lands under the new umask. +if [[ "$CONFINE" == 1 ]]; then + log "confining accounts (confine)" + try "confine" configure_confinement +else + log "confine is OFF (CONFINE=0) -- accounts are not confined" +fi + log "installing dotfiles for root" try "root dotfiles" install_dotfiles /root root # our .zshrc has a dedicated root prompt, so root runs zsh too diff --git a/server.conf.example b/server.conf.example index 383d4d8..7e633f7 100644 --- a/server.conf.example +++ b/server.conf.example @@ -88,6 +88,70 @@ # ...or, for a zone at Cloudflare instead: #CLOUDFLARE_API_TOKEN= +# ── confinement ────────────────────────────────────────────────────────────── + +# These boxes are multi-tenant, and the root VPSes we sell hand a customer an +# account rather than the root password. So every account that is not an admin +# is confined by default, with nothing to remember: +# +# home 0700 nobody walks into anybody else's ~/.config +# /proc hidepid `ps -ef` shows you your own processes and no one else's +# slice caps one runaway build is not everybody's outage +# pam limits a fork bomb is stopped in the fork +# ssh forwarding ssh -L to your own app yes, ssh -R relay no +# +# root and anyone in CONFINE_EXEMPT_GROUPS are exempt -- explicitly, by uid, +# because the account you fix a wedged box with must not be subject to the cap +# that is wedging it. `root-ubuntu.sh confine` prints what is confined here and +# what is not. +# +# It never takes sudo away from an account that already has it: demoting a live +# sudoer unattended is how you lose a box. `confine` names them and leaves that +# to you (`root-ubuntu.sh groups rm sudo`). +#CONFINE=1 + +# Who counts as an admin, and so is not confined. +#CONFINE_EXEMPT_GROUPS=sudo,admin + +# The group that carries the confinement. Recomputed every run from the line +# above, so a promotion or demotion converges without being told twice. +#CONFINE_GROUP=confined + +# The mode every human home is set to, and the umask new accounts get. Setting +# the home closes the path; the umask stops new files being born world-readable +# behind it. Neither fixes the mode of a file that is already there. +#CONFINE_HOME_MODE=0700 +#CONFINE_UMASK=027 + +# hidepid needs a group whose members still see every process -- admins go in +# it automatically. Add a monitoring agent's unit to CONFINE_PROC_UNITS if it +# reads /proc as a non-root user; polkit already does and is handled. +#CONFINE_HIDEPID=1 +#CONFINE_PROC_GROUP=proc +#CONFINE_PROC_UNITS=polkit.service + +# Per-account resource caps on the systemd user slice. Memory and tasks are +# hard ceilings because their failure mode is the whole box. CPU is only a +# weight -- it bites under contention and gets out of the way when the box is +# idle -- so set CONFINE_CPU_QUOTA (e.g. 200%) only if you want a real ceiling +# and are willing to throttle a build on an otherwise empty machine. +#CONFINE_MEMORY_HIGH=35% +#CONFINE_MEMORY_MAX=50% +#CONFINE_TASKS_MAX=4096 +#CONFINE_CPU_WEIGHT=100 +#CONFINE_CPU_QUOTA= + +# PAM limits, which bite at login rather than in the cgroup. +#CONFINE_NPROC=4096 +#CONFINE_NOFILE=16384 +#CONFINE_MAXLOGINS=20 + +# local -- ssh -L to their own app works, ssh -R relays do not. The relay is +# the one that matters: it turns the box into an open proxy under our IP. +# no -- no forwarding at all. yes -- off, this box is a jump host on purpose. +#CONFINE_SSH_FORWARDING=local +#CONFINE_SSH=1 + # ── tailscale ──────────────────────────────────────────────────────────────── # Joins the tailnet unattended. Without it the script prints the command to run diff --git a/test/root-ubuntu.test.ts b/test/root-ubuntu.test.ts index 1b6b8d8..9ea2b61 100644 --- a/test/root-ubuntu.test.ts +++ b/test/root-ubuntu.test.ts @@ -503,7 +503,9 @@ describe('the groups subcommand', () => { }); it('is peeled off before the root check, like the share subcommands', () => { - expect(SOURCE).toContain('mount|umount|mounts|share|groups)'); + // The alternation grows as subcommands are added, so match the verb + // inside it rather than pinning the whole list. + expect(SOURCE).toMatch(/^\tmount\|[a-z|]*\bgroups\b[a-z|]*\)/m); expect(SOURCE).toMatch(/groups\)\s+cmd_groups/); }); @@ -882,3 +884,456 @@ describe('configure_sensors', () => { expect(SOURCE).toContain('try "sensors" configure_sensors'); }); }); + +describe('the user confine', () => { + /** + * The model is "root and admins are never confined, everybody else is + * confined by default", so most of what is worth testing is the boundary: + * who ends up on which side of it, and whether any of the mechanisms can + * quietly do the opposite of what it claims. + */ + + const CONFINE_DECLS = [ + ...SOURCE.matchAll(/^(?:CONFINE|CONFINE_[A-Z_]+)=.*$/gm), + ] + .map((m) => m[0]) + .join('\n'); + + describe('_confine_exempt', () => { + const FNS = ['_confine_exempt']; + const stubs = ` + CONFINE_EXEMPT_GROUPS=sudo,admin + id() { [[ "\${2:-}" == root ]] && { echo 0; return 0; } + [[ "\${2:-}" == ghost ]] && return 1 + echo 1000; } + _groups_in() { [[ " \$FAKE_GROUPS " == *" \$2 "* ]]; } + `; + const exempt = (login: string, groups = '') => + status(FNS, `${stubs}\nFAKE_GROUPS="${groups}" _confine_exempt ${login}`); + + it('always exempts root, whatever groups say', () => { + expect(exempt('root')).toBe(0); + }); + + it('exempts an admin', () => { + expect(exempt('alice', 'sudo users')).toBe(0); + expect(exempt('alice', 'admin')).toBe(0); + }); + + it('confines an ordinary account', () => { + expect(exempt('alice', 'users')).toBe(1); + }); + + it('confines an account it cannot classify, rather than letting it out', () => { + // The safe direction on an unknown. An account that does not resolve is + // not evidence of an admin, and treating it as one is how a confine + // grows a hole shaped like whatever the lookup failed on. + expect(exempt('ghost', 'sudo')).toBe(1); + }); + }); + + describe('_confine_humans', () => { + const FNS = ['_confine_humans']; + const passwd = [ + 'root:x:0:0:root:/root:/bin/bash', + 'daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin', + 'www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin', + 'sync:x:4:65534:sync:/bin:/bin/sync', + 'alice:x:1000:1000::/home/alice:/bin/zsh', + 'bob:x:1001:1001::/home/bob:/bin/bash', + 'locked:x:1002:1002::/home/locked:/bin/false', + 'nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin', + ].join('\n'); + // %b, not %s: JSON.stringify turns the newlines into two-character \n + // escapes, and printf %s hands awk one very long single record. + const humans = () => + shell(FNS, `getent() { printf '%b\\n' ${JSON.stringify(passwd)}; }\n_confine_humans`); + + it('is every human account, not only the ones this script created', () => { + // The cloud image's own `ubuntu` was never created by us and still + // shares the box. A confine with a hole shaped like the default user + // is not a confine. + expect(humans().split('\n')).toEqual(['alice', 'bob']); + }); + + it('leaves system accounts and locked shells out of it', () => { + const out = humans(); + expect(out).not.toContain('www-data'); + expect(out).not.toContain('nobody'); + expect(out).not.toContain('locked'); + expect(out).not.toContain('sync'); + }); + }); + + describe('_confine_sysctl_floor', () => { + const FNS = ['_confine_sysctl_floor']; + const floor = (have: string, want: number) => + shell( + FNS, + `sysctl() { [[ "\${2:-}" == missing ]] && return 1; printf '%s\\n' ${JSON.stringify( + have, + )}; }\n_confine_sysctl_floor a.key ${want} || true`, + ); + + it('raises a key the kernel sets lower', () => { + expect(floor('1', 2)).toBe('a.key = 2'); + }); + + it('NEVER writes a key the kernel already sets stricter', () => { + // This is the one that matters. Ubuntu ships perf_event_paranoid=4 and + // unprivileged_bpf_disabled=2, both stricter than the floor asked for, + // so a file written unconditionally would loosen the box in the name of + // hardening it — silently, on the distribution we run everywhere. + expect(floor('4', 3)).toBe(''); + expect(floor('2', 1)).toBe(''); + }); + + it('leaves a key alone when it is already exactly at the floor', () => { + expect(floor('2', 2)).toBe(''); + }); + + it('skips a key this kernel does not have rather than guessing', () => { + expect( + shell(FNS, `sysctl() { return 1; }\n_confine_sysctl_floor a.key 2 || true`), + ).toBe(''); + }); + + it('skips a value it cannot read as a number', () => { + expect(floor('unknown', 2)).toBe(''); + }); + }); + + describe('fs.suid_dumpable', () => { + /** + * The one key in the set that does not run in one direction: 0 (never + * dump) is safest, 2 (dump, root-readable only) is safe, and 1 — dump + * like anything else, into a file the account can read back — is the + * dangerous one, in the middle. Treating it as a floor and taking the + * larger number leaves a box on 2 and calls it hardened, and leaves a + * box on 1 alone entirely. + */ + const FNS = [ + '_confine_sysctl_floor', + 'configure_confine_sysctl', + 'write_if_changed', + 'file_sha', + ]; + + function generated(dumpable: string): string { + const dir = mkdtempSync(join(tmpdir(), 'root-ubuntu-suid-')); + shell( + FNS, + `CONFINE_SYSCTL=1 + note() { :; }; info() { :; }; warn() { :; } + sysctl() { + case "\${2:-}" in + fs.suid_dumpable) printf '%s\\n' ${JSON.stringify(dumpable)} ;; + -p|*) return 0 ;; + esac + } + eval "$(declare -f configure_confine_sysctl | sed 's#/etc/sysctl.d/62-profullstack-confine.conf#${dir}/out.conf#g')" + configure_confine_sysctl`, + ); + try { + return readFileSync(join(dir, 'out.conf'), 'utf8'); + } catch { + return ''; + } + } + + it('sets it to 0 from Ubuntu’s default of 2', () => { + expect(generated('2')).toContain('fs.suid_dumpable = 0'); + }); + + it('sets it to 0 from the dangerous 1', () => { + expect(generated('1')).toContain('fs.suid_dumpable = 0'); + }); + + it('leaves a box that is already at 0 alone', () => { + expect(generated('0')).not.toContain('fs.suid_dumpable'); + }); + }); + + describe('_confine_fstab_proc', () => { + const FNS = ['_confine_fstab_proc']; + + function box(fstab: string): string { + const dir = mkdtempSync(join(tmpdir(), 'root-ubuntu-confine-')); + writeFileSync(join(dir, 'fstab'), fstab); + return dir; + } + + const run = (dir: string, opts = 'rw,hidepid=invisible,gid=1001') => + shell( + FNS, + `CONFINE_PROC_GROUP=proc + eval "$(declare -f _confine_fstab_proc | sed 's#/etc/fstab#${dir}/fstab#g')" + _confine_fstab_proc '${opts}' 1001 && echo CHANGED || echo SAME`, + ); + + const fstabOf = (dir: string) => readFileSync(join(dir, 'fstab'), 'utf8'); + + it('adds the hidepid mount so it survives a reboot', () => { + const dir = box('/dev/sda1 / ext4 defaults 0 1\n'); + expect(run(dir)).toBe('CHANGED'); + expect(fstabOf(dir)).toMatch(/^proc \/proc proc rw,hidepid=invisible,gid=1001 0 0$/m); + expect(fstabOf(dir)).toContain('/dev/sda1 / ext4'); + }); + + it('is safe to run twice: no second line, and no second pair of comments', () => { + // The comments carry a marker precisely so they can be stripped and + // rewritten. Without it every run appends another two lines and the + // file grows forever while cmp keeps reporting a change. + const dir = box('/dev/sda1 / ext4 defaults 0 1\n'); + run(dir); + const first = fstabOf(dir); + expect(run(dir)).toBe('SAME'); + expect(fstabOf(dir)).toBe(first); + expect(first.match(/hidepid --/g) ?? []).toHaveLength(1); + }); + + it('replaces an existing /proc line instead of adding a rival to it', () => { + const dir = box('proc /proc proc defaults 0 0\n/dev/sda1 / ext4 defaults 0 1\n'); + expect(run(dir)).toBe('CHANGED'); + expect(fstabOf(dir).match(/^proc /gm) ?? []).toHaveLength(1); + expect(fstabOf(dir)).not.toContain('proc /proc proc defaults'); + }); + + it('keeps unrelated comments', () => { + const dir = box('# my own note\n/dev/sda1 / ext4 defaults 0 1\n'); + run(dir); + expect(fstabOf(dir)).toContain('# my own note'); + }); + }); + + describe('_set_login_def', () => { + const FNS = ['_set_login_def']; + + function box(contents: string): string { + const dir = mkdtempSync(join(tmpdir(), 'root-ubuntu-logindefs-')); + writeFileSync(join(dir, 'login.defs'), contents); + return dir; + } + + const run = (dir: string, key: string, val: string) => + shell( + FNS, + `eval "$(declare -f _set_login_def | sed 's#/etc/login.defs#${dir}/login.defs#g')" + _set_login_def ${key} ${val} && echo CHANGED || echo SAME`, + ); + + const defsOf = (dir: string) => readFileSync(join(dir, 'login.defs'), 'utf8'); + + it('sets a key that is not there at all', () => { + const dir = box('UID_MIN 1000\n'); + expect(run(dir, 'UMASK', '027')).toBe('CHANGED'); + expect(defsOf(dir)).toMatch(/^UMASK\t027$/m); + expect(defsOf(dir)).toContain('UID_MIN 1000'); + }); + + it('edits the line in place rather than appending a second one', () => { + const dir = box('HOME_MODE\t0750\nUID_MIN 1000\n'); + expect(run(dir, 'HOME_MODE', '0700')).toBe('CHANGED'); + expect(defsOf(dir).match(/^HOME_MODE/gm) ?? []).toHaveLength(1); + expect(defsOf(dir)).toMatch(/^HOME_MODE\t0700$/m); + }); + + it('uncomments a commented-out key, which is how Ubuntu ships UMASK', () => { + const dir = box('#UMASK\t022\n'); + expect(run(dir, 'UMASK', '027')).toBe('CHANGED'); + expect(defsOf(dir)).toMatch(/^UMASK\t027$/m); + expect(defsOf(dir)).not.toMatch(/^#UMASK/m); + }); + + it('reports no change when the value is already right', () => { + const dir = box('UMASK\t027\n'); + expect(run(dir, 'UMASK', '027')).toBe('SAME'); + }); + }); + + describe('what the confine will not do on its own', () => { + it('never removes sudo from an account that already has it', () => { + // Demoting a live sudoer unattended is how you lose a box: it might be + // a colleague, the only other admin, or the account your automation + // logs in as. The script names them and stops. + expect(SOURCE).toContain('_confine_report_sudoers'); + const fn = SOURCE.slice(SOURCE.indexOf('sync_confine_membership() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body).not.toMatch(/gpasswd -d[^\n]*sudo/); + expect(body).not.toMatch(/deluser[^\n]*sudo/); + }); + + it('lifts every cap back off root, by uid, explicitly', () => { + // user-.slice.d is a prefix drop-in: it applies to user-0.slice too. + // Without the exemption the account you fix a wedged box with is + // subject to the cap that is wedging it. + expect(SOURCE).toMatch(/want=\(0\)/); + expect(SOURCE).toContain('MemoryMax=infinity'); + expect(SOURCE).toContain('TasksMax=infinity'); + }); + + it('takes the exemption away again from someone who is no longer an admin', () => { + expect(SOURCE).toContain('is no longer an admin -- resource caps now apply'); + }); + }); + + describe('the sshd policy', () => { + it('closes its own Match block', () => { + // Not because it currently has to: OpenSSH scopes a Match to the file it + // appears in, and 10.2p1 gives a byte-identical effective config with + // and without the closing line. It is here because this directory is + // included from the first line of sshd_config, so everything in the main + // file comes after this block — and leaving a Match open as the last + // thing in an included file is a footgun waiting for the version where + // that scoping changes. + const fn = SOURCE.slice(SOURCE.indexOf('configure_confine_ssh() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + // Anchored to the start of a line: both phrases also appear in the + // comment above the block explaining why this matters, and matching + // those instead would pass while the directive itself was missing. + const opened = body.match(/^Match Group /m); + const closed = body.match(/^Match all$/m); + expect(opened?.index).toBeDefined(); + expect(closed?.index).toBeDefined(); + expect(closed!.index!).toBeGreaterThan(opened!.index!); + }); + + it('validates with sshd -t and rolls back rather than locking the box', () => { + const fn = SOURCE.slice(SOURCE.indexOf('configure_confine_ssh() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body).toContain('sshd -t'); + expect(body).toContain('rolled back'); + // the reload only happens after the check + expect(body.indexOf('sshd -t')).toBeLessThan(body.indexOf('systemctl reload')); + }); + + it('refuses to write a drop-in that nothing includes', () => { + // A policy file in a directory no config includes is a policy that does + // not exist, and looks applied from every angle except the running sshd. + expect(SOURCE).toContain('has no Include for sshd_config.d'); + }); + }); + + describe('hidepid', () => { + it('proves the option works on the running kernel before trusting fstab to it', () => { + // An fstab entry the kernel rejects fails the mount at boot, which is + // the worst possible place to discover it. So: remount first, write the + // file only once that has actually worked. + const fn = SOURCE.slice(SOURCE.indexOf('configure_confine_proc() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body.indexOf('mount -o "remount,$want" /proc')).toBeLessThan( + body.indexOf('_confine_fstab_proc'), + ); + }); + + it('gives the units that read other people’s /proc the group first', () => { + const fn = SOURCE.slice(SOURCE.indexOf('configure_confine_proc() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body.indexOf('_confine_proc_units')).toBeLessThan( + body.indexOf('mount -o "remount,$want" /proc'), + ); + expect(SOURCE).toContain('SupplementaryGroups=$CONFINE_PROC_GROUP'); + }); + + it('does not try to remount /proc inside a container', () => { + expect(SOURCE).toContain("systemd-detect-virt --container"); + }); + }); + + describe('the default groups a new account lands in', () => { + const decls = CONFINE_DECLS; + const explicit = SOURCE.slice( + SOURCE.indexOf('DEFAULT_GROUPS_EXPLICIT='), + SOURCE.indexOf('USERS=()'), + ); + + const groupsFor = (env: string) => + execFileSync('bash', ['-c', `set -uo pipefail\n${env}\n${decls}\n${explicit}\nprintf '%s' "$DEFAULT_GROUPS"`], { + encoding: 'utf8', + }); + + it('is users, not sudo, once the box is confined', () => { + // Otherwise the first unattended --refresh hands every new tenant the + // way straight out of the confine. + expect(groupsFor('')).toBe('users'); + }); + + it('is still sudo,admin when the confine is off', () => { + expect(groupsFor('CONFINE=0')).toBe('sudo,admin'); + }); + + it('never overrides a DEFAULT_GROUPS somebody actually set', () => { + // The environment and server.conf both win over a default, everywhere + // else in this script. This is no different. + expect(groupsFor('DEFAULT_GROUPS=sudo,docker')).toBe('sudo,docker'); + expect(groupsFor('CONFINE=1 DEFAULT_GROUPS=admin')).toBe('admin'); + }); + }); + + describe('the confine subcommand', () => { + it('reports without root, since reading a posture is not a privileged act', () => { + const out = execFileSync('bash', [SCRIPT, 'confine', 'status'], { encoding: 'utf8' }); + expect(out).toContain('confine'); + expect(out).toMatch(/admin groups/); + }); + + it('refuses to apply without root, and names what to run instead', () => { + let out = ''; + try { + execFileSync('bash', [SCRIPT, 'confine', 'apply'], { encoding: 'utf8', stdio: 'pipe' }); + } catch (error) { + out = String((error as { stderr?: Buffer }).stderr ?? ''); + } + expect(out).toContain('must run as root'); + }); + + it('is peeled off before the root check, like the other subcommands', () => { + expect(SOURCE).toMatch(/^\tmount\|[a-z|]*\bconfine\b[a-z|]*\)/m); + }); + + it('is offered by the top-level help', () => { + const out = execFileSync('bash', [SCRIPT, '--help'], { encoding: 'utf8' }); + expect(out).toContain('confine'); + }); + }); + + describe('homes', () => { + it('grants nginx traverse with an ACL rather than opening the home to everyone', () => { + // chmod o+x is what this script used to do, and it hands the path to + // every account on the box. u:www-data:--x says the same thing to one + // user — and execute only, so it can walk through without listing. + expect(SOURCE).toContain('setfacl -m "u:$WEB_GROUP:--x"'); + }); + + it('only grants it where there is something published', () => { + const fn = SOURCE.slice(SOURCE.indexOf('_confine_home_mode() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body).toMatch(/-d "\$home\/public_html" \|\| -d "\$home\/apps"/); + }); + + it('falls back to the old behaviour when acl is not installed yet', () => { + // setfacl arrives with the apt stage. Failing closed here would break + // the user's web page on a first run instead of on no run at all. + const fn = SOURCE.slice(SOURCE.indexOf('_confine_home_mode() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body).toContain('command -v setfacl'); + expect(body).toContain('chmod o+x'); + }); + + it('covers accounts this script never created', () => { + const fn = SOURCE.slice(SOURCE.indexOf('configure_confine_homes() {')); + const body = fn.slice(0, fn.indexOf('\n}\n')); + expect(body).toContain('_confine_humans'); + expect(body).not.toContain('KNOWN_USERS'); + }); + }); + + it('is wired into an ordinary run, not only into the subcommand', () => { + expect(SOURCE).toContain('try "confine" configure_confinement'); + }); + + it('can be turned off in one place, and says so when it is', () => { + expect(SOURCE).toContain('CONFINE="${CONFINE:-1}"'); + expect(SOURCE).toContain('confine is OFF (CONFINE=0)'); + }); +});