CFE-90: reconcile storage mount options when they drift (opt-in remount)

[nickanderson]

Storage/mount promises only applied mount_options to the initial mount and (if edit_fstab is enabled) the fstab entry. A filesystem already mounted with different options was never corrected, because mount -a skips already-mounted filesystems. This PR adds opt-in reconciliation of a live mount when its options drift from the promise, and along the way fixes several long-standing storage-promise bugs (CFE-1539, CFE-1863, CFE-2350, CFE-3366).

Opt-in live remount reconciliation (CFE-90, CFE-1864)

New mount body attributes:

• remount (default false) — reconcile a live mount's options when they differ. When false, a mounted filesystem with the correct source is kept regardless of option drift (backwards compatible; options still drive the initial mount and, with edit_fstab, the fstab entry).
• remount_methods (default { remount }) — mechanisms tried in order, re-reading the live mount after each (the kernel returns success from a remount even when it silently ignores NFS-negotiated options). Defaults to the non-disruptive in-place remount only; the disruptive unmount_mount (which tears the filesystem down and back up) is opt-in, and is required to change options a live remount cannot — e.g. NFS-negotiated vers=/rsize=, or the server.
• remount_timeout — protect against a hung or unreachable server.

Option comparison (live mount). Only the options the promise names are enforced; any option it does not name is left unpoliced — its provenance is unknown (kernel-negotiated like vers=/rsize=/proto=/sec=, or added by a prior manual mount -o, indistinguishable). The promise is first resolved with util-linux "last wins" semantics — exactly as mount -o applies a list, a later option overrides an earlier conflicting one, so defaults,ro is a read-only mount and ro,rw is rw. Each surviving option must then hold on the live mount: its inverse absent, and it either present or a default-on flag. Inverse pairs (noatime/relatime, hard/soft, ro/rw, sync/async, generic no<opt>/<opt>) and tcp/udp↔proto= aliases are recognized; an option the promise specifies (including a negotiated one such as rsize=8192) must be present. A correctly-mounted filesystem converges instead of being reported changed every run, and an override (ro shadowing an earlier rw) is logged at verbose.

The defaults pseudo-option. defaults (= rw,suid,dev,exec,auto,nouser,async, per mount(8)) is never echoed by the kernel, so it's expanded to its checkable components rw,suid,dev,exec,async and subjected to the same last-wins resolution. It holds unless one of the violating negatives is present — ro (vs rw), nosuid/nodev/noexec (vs suid/dev/exec), or sync (vs async) — and a later explicit option overrides the matching component (so defaults,ro requires read-only, defaults,nosuid allows nosuid). auto/nouser are fstab / mount-permission concepts, not runtime state, so they aren't enforced. When reconciling a drifted defaults mount in place, the remount command likewise uses rw,suid,dev,exec,async (util-linux doesn't apply the options implied by a bare defaults on a remount), and mount's own last-wins applies any trailing override — so a mount that drifted to ro/nosuid/etc. is restored non-disruptively rather than only via unmount_mount.

fstab maintenance for already-mounted filesystems (CFE-1539)

Previously a filesystem already mounted with the correct source was reported "mounted as promised" and fstab was never consulted, so a missing fstab entry was not restored and an options change was not written until the mount happened to be redone. VerifyInFstab now runs on the mounted-correctly path too (when edit_fstab => "true"), deliberately independent of the opt-in live remount: keeping fstab correct is the documented behavior of mount_options. fstab option comparison uses strcmp because option order matters.

Surgical single-filesystem mount, not mount -a (CFE-1863)

A storage promise for a not-yet-mounted filesystem used to arm mount -a (mount -va on Linux), which mounts every unmounted fstab entry — unrelated devices and foreign filesystem types included — as a side effect of a single promise. The not-mounted path now mounts just the promised filesystem surgically (VerifyMount), then persists it to fstab. The mount -a mechanism (MountAll) is retained only for the explicit mountfilesystems agent-control attribute, which still means "mount everything in fstab".

Target a specific mount on unmount (CFE-2350)

An unmount promise that named mount_source/mount_server was logged as "probably an error", and the server was never used to pick which mount to act on — so you couldn't unmount one specific mount (e.g. from a server being migrated away) without affecting others. The bogus warning is removed, and the server (host) is now part of the "mounted correctly" identity check, gated on remount or unmount so it only engages when the promise opts into disruptive mount management. An unmount promise that finds a different filesystem at the mount point leaves it — and its fstab entry — untouched, and LiveMountConverged treats the server as identity so a remount-in-place that can't change it escalates to unmount_mount.

Correct dry-run / warn reporting (CFE-3366)

The mount, unmount and remount outcomes are based on the promise action (MakingInternalChanges) rather than a bare !DONTDO. A dry-run (-n) or warn promise now reports WARN without defining promise_repaired, so dependent promises no longer fire on a no-op run.

Supporting mount-info fixes

• GetFstabEntryOptions returned the fstab type field instead of the options field (spurious rewrite every run).
• ReplaceFstabEntry leaked the previous entry string.
• Foreign-filesystem detection is restored by keeping the fstype separate from the kernel-resolved options (options vs raw_opts).
• The mount point directory is again created unconditionally before mounting.
• "device busy" interruptions are logged at LOG_LEVEL_ERR (the outcome is INTERRUPTED), and leaked options strings on the error paths are freed.

Testing

Unit coverage in tests/unit/nfs_test.c (option subset matching, inverse/alias pairs, the defaults negative-violation check, contradiction detection, and the defaults→rw,suid,dev,exec,async remount expansion), runnable unprivileged via make -C tests/unit check. The behavioral NFS reconcile/escalation, fstab maintenance, and surgical single-filesystem mount — which need root and a real NFS server — are covered by the system-testing PR and were exercised against a loopback NFS export during development.

Commits

1. Added opt-in remount reconciliation for storage mount options (CFE-90, CFE-1864, CFE-3366)
2. Added unit tests for mount option matching and contradiction detection
3. Maintained fstab entries for already-mounted filesystems (CFE-1539)
4. Mounted a single filesystem surgically instead of running mount -a (CFE-1863)
5. Allowed mount promises to target a specific server (CFE-2350)

References

• mount -a skipping already-mounted filesystems, and defaults = rw,suid,dev,exec,auto,nouser,async — mount(8)
• NFS options negotiated by client/server (reported in /proc/mounts) and NFS-specific options not modifiable on remount — nfs(5)

---

Resolves CFE-90 (and its duplicate CFE-1864), CFE-1539 (fstab maintenance), CFE-1863 (mount -a scope), CFE-2350 (target a specific mount on unmount), and CFE-3366 (dry-run/warn defining promise_repaired) for storage mount promises.

Ticket: https://northerntech.atlassian.net/browse/CFE-90

Together with: https://github.com/cfengine/system-testing/pull/693