DRAFT (phase 3, after T063): T072, 13.1 — the bundled PostgreSQL server, the local install's own child (plan 034)

[brettheap]

Lane: openxfactory-4 (openXfactory-4-openDox_extraction)

Arc: neutral-product-standalone-operability

Plan 034 (specs/034-opendox-standalone-operation/tasks.md, read at openxFactory main 91e4685f), phase-3 slice P3-I, install mode and the bundle:

• T072 (#1144's 13.1): the bundled PostgreSQL server, as T007 batch H's 13.1 addendum reads (openxFactory#1206 → f99a2097).
• Falsifier: F13.1's TCP-listener block, which reads the kernel's socket table at run time, and its runtime status block.
• After: T070 (openDox-code#67, not landed). Stacked on DRAFT (phase 3, after T063): T070, 13.4-13.6 — OPENDOX_INSTALL_MODE and generate-and-open --local (plan 034) #67's branch, so this diff is T072 alone.

Ruled:

• R1Q22 (a), 5817152735;
• R1Q16 (i)–(iv), 5850003126;
• the phase-3 draft-ahead widening, 5901112350.

Claimed on openxFactory#656 in 5901575394. DRAFT, authored ahead: it does not go READY before T063 lands and the holder says so.

What it does, by R1Q16's four parts

• (i) The document server starts it as its own child, and reports it.
  • opendox generate-and-open --local (or OPENDOX_INSTALL_MODE=local) starts postgres as a direct child of the process serving the document surface. It uses subprocess.Popen and never pg_ctl, which would re-parent it.
  • It prints the socket, the pid and what it migrated.
  • runtime status reports database_bundle (data_dir, socket_dir, pid). It reads the pid from the server's own postmaster.pid and believes it only while a postgres runs there.
• (ii) Started and migrated, and nothing more.
  • initdb runs once per data directory.
  • An idempotent bootstrap makes the database and the SERVED role. Its grants are the compose stack's (init-runtime-role.sh): CONNECT, USAGE on public, and DML on what the owner creates, by default privileges.
  • Then migrations.MigrationRunner runs as the owner, with the served role and database declared. The run narrows the ledger to SELECT for the served role and verifies its access, exactly as a hosted runtime migrate does.
  • runtime migrate under local migrates the bundle too.
• (iii) It ships as the opendox[local] extra, which is opendox[runtime] plus pixeltable-pgserver>=0.6.0 (RULED, openxFactory#656 5916000030 item 2). The test extra joins it, so F9.1's .[test] install still runs every case.
• (iv) It stops with the entry point.
  • SIGTERM is read as the Ctrl-C the serve loop already stops on, followed by a PostgreSQL fast shutdown (then an immediate one, then SIGKILL, each bounded).
  • The backstop is PR_SET_PDEATHSIG on Linux, so a SIGKILLed entry point still takes its server with it.
  • The server runs in its own session, so a terminal's Ctrl-C reaches the entry point, and the stop happens in order.

13.1's fixed identity

• The data and socket directories live under OPENDOX_STATE_DIR, at <state>/postgres/data and <state>/postgres/run.
  • The setting is new. It defaults to $XDG_STATE_HOME/opendox, else ~/.local/state/opendox, and must be absolute, because the server's process and a runtime status run from elsewhere must derive the same socket.
  • A state directory too long for the kernel's sun_path is refused, naming the setting.
• No TCP listener: listen_addresses is empty, and the socket directory is narrowed to 0700.
• Peer authentication (RULED, openxFactory#656 5916000030 item 3, "Peer auth + accept (Recommended)"):
  • initdb runs with --auth-local=peer --auth-host=reject.
  • Before every launch the bundle rewrites pg_hba.conf and pg_ident.conf (atomically, mode 0600). pg_hba.conf holds one rule, local all all peer map=opendox, plus host … reject for IPv4 and IPv6. pg_ident.conf maps the running OS user, and nobody else, to opendox and opendox_runtime.
  • The kernel reports the connecting uid, so the DSNs carry no password because there is none. A cluster an older build left as trust is put back to peer on its next start.
• Both DSNs are supplied: two users (owner opendox for migrations, opendox_runtime for serving) over the one socket, with port spelled so a stray PGPORT cannot redirect libpq. They pass T071's three checks for the reason those exist: one dialect, one database, and never one credential in both settings.
  • An operator DSN given beside local is refused by name. It is added to T070's HOSTED_ONLY_SETTINGS, a holder reading on openxFactory#656 that Brett may overrule.
• A second entry point on the same state directory is refused, naming the running pid. One install's database belongs to one entry point at a time.

The migrations gap (assigned to T072 by the holder)

• The migrations were not package data. migrations/ sits at the repository root, and only the image copies it (WORKDIR /app), so pip install "opendox[local]" run outside a checkout had nothing to apply.
• Now pyproject.toml's [tool.setuptools.data-files] maps migrations/*.sql into the wheel's data directory (share/opendox/migrations). The root migrations/ does not move.
• config.migrations_dir resolves an unset OPENDOX_MIGRATIONS_DIR in this order:
  1. migrations wherever the working directory has one (a checkout, or the image's /app), which is today's default, unchanged;
  2. otherwise the copy the installed distribution's RECORD lists (packaged_migrations_dir);
  3. for an editable install, which installs no data files, the source tree's own migrations/.
• The canonical digest gate is what proves any copy found is the pinned one.
• test_a_wheel_install_migrates_its_bundled_server_outside_a_checkout:
  • builds this package's wheel offline (--no-build-isolation, --no-index);
  • installs it under a --prefix outside the checkout;
  • runs from a directory with no migrations/, asserting that opendox is the wheel's copy and that the migrations dir is under the prefix's share/opendox;
  • migrates the bundled server there (applied == ["0001", "0002"]).

The server package (RULED, openxFactory#656 5916000030 item 2: "pixeltable-pgserver (Recommended)")

• pixeltable-pgserver 0.6.0, the maintained fork of pgserver, uploaded 2026-07-14. Apache-2.0, as its dist-info LICENSE and OSI classifier say. It carries PostgreSQL 16.14 under the PostgreSQL License (initdb --version and postgres --version from pixeltable_pgserver/pginstall/bin). It also carries an 18.4 under pginstall18/, which this package does not use.
• Only its binaries are used, found with importlib.util.find_spec("pixeltable_pgserver") without importing it. Its own manager is not used, because:
  • it daemonizes through pg_ctl, against (i);
  • it stops from atexit, which SIGTERM never runs, against (iv);
  • it may put the socket under the user's runtime directory, opened to 0777, against 13.1.
• Linkage, re-verified on the installed wheel (readelf -d, ldd):
  • postgres needs libz, libpthread, librt, libdl, libm, libc;
  • initdb needs those less libz and libdl, plus the wheel's own vendored libpq. That libpq resolves through RPATH $ORIGIN/../../../pixeltable_pgserver.libs and itself needs only libc, libm and libpthread;
  • the server's loadable modules need libc, and one needs the vendored libpq.
    So the system libraries are the C library and libz only: no system PostgreSQL, no ICU.
• Its floor and its size:
  • Wheels exist for cp310 to cp314, on Linux x86_64 and aarch64, macOS and Windows.
  • The Linux wheels are tagged manylinux_2_27 and manylinux_2_28, so they need glibc 2.27 or later. That is the tag's floor. The highest GLIBC symbol any binary or module needs is 2.25, by objdump -T.
  • Each wheel is about 24.7 MB (the cp312 x86_64 wheel is 24,704,230 bytes), because it carries two server majors.
• It pulls in fasteners, platformdirs, psutil and typing-extensions, which nothing here imports. All four were already pinned.
• Superseded: pgserver 0.1.4 carried PostgreSQL 16.2 and had no wheel after cp312 (Copilot r4139811507 and r4139811528, both now resolved with this ruling cited). Also rejected: postgresql-binaries, which links the system's ICU and untars at first use, and pgembed, which is PostgreSQL 17.

Outside src/ and tests/

• pyproject.toml: the local extra, the test extra joining it, setuptools>=70.1 in test (the wheel test's offline build; 70.1 is the first release that builds a wheel with no wheel package), and the data-files map. It is a single-writer file (T057 → T072). T057, openDox's own validator and its input set (7.1, 7.1a, 7.1b, 7.2) (plan 034) #58 (T057) adds package data there, and the merge-from-main round takes it.
• constraints-cpython312-linux.txt, in its own commit as its header asks. It was extended under its own pins in a clean 3.12.3 venv (psutil==7.2.2, platformdirs==4.12.2, fasteners==0.20 and setuptools==84.0.0 new). At 84a6c041 it was re-resolved in a clean environment under the pins less pgserver. The one line that moved is pgserver==0.1.4 → pixeltable-pgserver==0.6.0.
• deploy/ — one line, and the task requires it. deploy/compose/.env.example gains OPENDOX_STATE_DIR=, because test_every_runtime_setting_is_documented_in_env_example requires every SETTINGS entry there. The compose stack is hosted and never reads it. docs/: untouched.
• Not touched: serve.py (T073 adds the install block) and validate.yml. It already installs .[runtime,test], and test now carries local.
• Existing tests changed:
  • tests/test_doxbench_entrypoint.py stands the bundle in, with a tripwire. Its cases test the model port, which reads nothing from the store (R1Q16 (ii)).
  • T070's test_install_mode.py and test_install_mode_entrypoint.py stop passing DSNs beside local.
  • test_runtime_surface.py declares opendox.runtime.bundle stdlib-only at import, because opendox.cli imports it and opendox --help runs with no extra installed.

The falsifier

F13.1's runtime status block and its TCP-listener block, verbatim in their assertions (f13-1-local.sh), against a server generate-and-open --local started in the background, with no broker and no operator database, under set -euo pipefail. Three deviations are forced, and none weakens a check:

• Generation is stood in. The start is tests_runtime/local_entrypoint_driver.py generate-and-open --local …, which is the real cli.main with only the three openXdox generation names and serve's two binding stand-ins replaced, exactly as tests/test_doxbench_entrypoint.py replaces them. This stack's base predates T055/T056's standalone generate (T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59 and its successor).
• The pid comes from runtime status. The TCP-listener block reads the server's pid from runtime status's database_bundle, not from caps.json. /capabilities' install block is T073's, so F13.1's caps.json block is T073's to run.
• The corpus is a one-file stand-in, because T050's fixture is not on this base.

BEFORE is #67's head b50e3b1; AFTER is this branch:

=== BEFORE (b50e3b1)
ready=1
runtime status rc=1
AssertionError: no bundled database answered: None OPENDOX_DATABASE_URL is required and is not set: …
FAIL status-block
AssertionError: the bundle reports no server pid: None
FAIL tcp-listener-block
=== AFTER (this branch, set -euo pipefail, exit 0)
ready=1
runtime status rc=0
PASS status-block
  server pid 638810, sockets ['3692911'], TCP LISTEN rows: none
PASS tcp-listener-block
PASS stops-with-entry-point (pid 638810 gone)
--- server stdout:
  database /tmp/tmp.v0wydKm5Zm/postgres/run (bundled, pid 638810, migrations applied now: ['0001', '0002'])

T070's F13.1 refusal probes and F13.1's load_settings block still pass on this branch (F13.1 REFUSALS + T070 PAIR: ALL PASSED).

In the suite, tests_runtime/test_bundled_postgres.py runs the same two blocks on the same background launch, and adds three checks: the server's PPid is the entry point's pid (i), the socket directory is 0700, and SIGTERM ends the entry point with exit 0 and the server gone (iv). Beside that it has a SIGKILL case (the parent-death backstop), the second-server refusal, runtime migrate under local, the wheel case above, and the layout and refusal cases. Under CI it fails rather than skips if the server is missing, because validate.yml pins EXPECT_SKIPPED=11 exactly.

A mutant of each new refusal and guarantee, killed

Each mutant was applied alone, and test_bundled_postgres.py plus test_install_mode.py were run with -x, with a 240 s bound so a hang could not pass for a kill:

mutant	killed by
M1 an operator DSN accepted beside local	test_migrate_under_the_local_mode_uses_the_bundle_and_refuses_a_dsn
M2 a socket path too long accepted	test_a_state_dir_too_long_for_a_unix_socket_is_refused_naming_it
M3 a relative state dir accepted	test_a_relative_state_dir_is_refused_naming_it
M4 the server opens a TCP listener (listen_addresses=127.0.0.1)	test_the_entry_point_owns_a_migrated_server_with_no_tcp_listener
M5 the server is not the entry point's child (setsid -f)	same
M6 stop() a no-op	test_a_second_entry_point_on_the_same_state_dir_is_refused
M7 no parent-death signal	test_the_server_stops_even_when_the_entry_point_is_killed_outright
M8 SIGTERM not read as an interrupt	test_the_entry_point_owns_a_migrated_server_with_no_tcp_listener
M9 started but not migrated	same
M10 the served DSN is the owner's (a collapse)	test_the_two_dsns_are_two_users_over_the_one_socket
M11 no packaged migrations found	test_a_wheel_install_migrates_its_bundled_server_outside_a_checkout
M12 the socket directory left 0755	test_the_entry_point_owns_a_migrated_server_with_no_tcp_listener
M13 a second server on one state dir not refused	test_a_second_entry_point_on_the_same_state_dir_is_refused

A defect this PR's own test found in itself. The first cut of the wheel case ran pip install --prefix without --ignore-installed. pip then read the suite's own editable opendox as the installed copy of the same project and uninstalled it, emptying the environment the suite runs in (measured: pip list lost opendox and both console scripts). The flag is now there, with a comment, and the case asserts afterwards that the suite's own opendox still resolves.

The repo's own suite

Full python -m pytest -q, LANG=C.UTF-8, CI=true, against a postgres:16 like validate.yml's:

	selected	passed	skipped	failed	errors
#60's head f097fd8	2485	2474	11	0	0
#67 (T070) b50e3b1	2531	2520	11	0	0
#67 (T070) 525f61c, its fix round 2	2538	2527	11	0	0
this branch before the merge, c3a70a2	2545	2534	11	0	0
32db5d8 (merges 525f61c)	2552	2541	11	0	0
ac61596 (merges 02dadc5)	2554	2543	11	0	0
28bdccd (fix round 4, and merges 859b37b6)	2580	2569	11	0	0
96b2699f (fix round 5)	2584	2573	11	0	0
a0fb7c8d (merges 026f00ea; fix round 6)	2584	2573	11	0	0
0f77d5c1 (fix round 7)	2595	2584	11	0	0
379fbb14 (fix round 8)	2601	2590	11	0	0
84a6c041 (the carrier and peer authentication, as ruled)	2613	2602	11	0	0
this branch, f8e6e9e9 (fix round 10)	2619	2608	11	0	0

EXPECT_SKIPPED=11 holds exactly, and the floors allow the rise unchanged. The new module adds about 32 s to the run (ten cases, each server start about 1.5 s).

Merging #67's fix rounds. 95fe16f merges 32683e8, and 32db5d8 merges 525f61c: runtime migrate and reset refuse what a local install cannot be.

• 525f61c and this PR both rewrite load_migration_settings. The conflict resolves to this PR's structure: under local, the refusal is asked first, and only then is the bundle's migration DSN read. T070's reason is carried into the comment.
• The two merged cases set the local shape as this PR defines it, with the mode and the state dir and no operator DSN. Beside local a DSN is itself refused here, so a case that set one would have tested the DSN refusal instead of the broker or bind refusal it names.
• A mutant that drops the refusal from the migration loader fails all 7 merged cases.
• ac61596 merges 02dadc5, cleanly. With the runtime extra absent, status's early return now reports a local install's broker as not configured. The merged case uses this PR's local shape and also asserts that database_bundle is reported on that return: present for local, null for hosted.

Fix rounds 4 and 5: Copilot's twelve threads (5e52872, 96b2699f)

Copilot reviewed 95fe16f, 32db5d8 and ac61596a and opened twelve threads. Ten are fixed, answered and resolved. The new cases are in tests_runtime/test_local_lifecycle.py (new, hermetic), plus two in test_bundled_postgres.py.

• Migrations (r4139811473, r4139880241).
  • An explicit OPENDOX_MIGRATIONS_DIR is used as given.
  • Unset, a local install uses only its own installation's copy, never the working directory's. That copy is the source tree __file__ came from first, then the RECORD of the distribution that holds the running module. Where there is none, it is refused.
  • A hosted install's default is unchanged (13.6).
• The pid (r4139811555, r4139938402).
  • A pid is believed only when /proc proves it is an executable named postgres running in this data directory. A proven-stale lock is removed before the launch.
  • Where there is no /proc, nothing is believed, and PostgreSQL's own interlocks stand. The price is a status with no pid on macOS and the BSDs, recorded in the thread.
• initdb (r4139880213): it runs into an attempt directory that is renamed into place only on success. Abandoned attempts are removed. A non-cluster data/ is refused and left alone.
• start() (r4139880279): every phase is one guarded operation, and every failure is the one named refusal, with its phase and class.
• Interrupts (r4139880267): SIGTERM or Ctrl-C anywhere in the local lifecycle is a clean stop, exiting 128 + the signal number. A served run still exits 0.
• Refusal wording (r4139880298): broker settings and DSNs are two classes, each with its own reason.
• State dir (r4139938444): an unknown ~user, or no home, refuses naming OPENDOX_STATE_DIR.
• Test helper (r4139811584): bounded by a selector. With a silent 8 s child and a 1 s deadline, it returned after 1.0 s where the old loop took 8.0 s.

Evidence:

• Against ac61596's source, 17 of round 4's first 18 cases fail; the one that passes is the unchanged hosted default. Round 5's cases fail against 28bdccd.
• Mutants: 23 of round 4 and 5 of round 5, all killed (runs/mutants-t072-r4.txt, -r5.txt in the writer's workdir).

Round 6 (a0fb7c8d): Copilot's review at 28bdccd9 opened two more threads.

• The unknown-_serves point was already fixed at 96b2699f; it is answered and resolved.
• The pyproject note named a function that no longer exists. It now names installation_migrations_dir, and the packaging case checks every opendox.runtime.config.<name> pyproject names.
• 4aed6278 merges T070's 026f00ea, a docstring change.

Round 7 (0f77d5c1): Copilot's review at a0fb7c8d opened three threads, all fixed and resolved. SonarCloud raised one reliability finding.

• libpq's environment. Every PG* variable is lifted out of os.environ for the duration and put back afterwards, around generate-and-open --local's lifecycle and the runtime verbs under local. PGHOSTADDR, PGSERVICE and PGOPTIONS can no longer redirect the bundle's connections. The real entry point and status are proven with all three set.
• The socket's path. The resolved state tree must be this user's own and writable by no one else. Every ancestor must be owned by the user or root, and sticky where others can write it; a group-writable ancestor of the user's own group is allowed. Symlinks inside the tree are refused before any chmod.
• A relative HOME is refused for the default state directory.
• SonarCloud S6466 (reliability): server_binaries no longer indexes a list.
• Evidence: 8 new cases fail against a0fb7c8d, and 11 mutants are killed.

Round 8 (379fbb14): Copilot's review at 0f77d5c1 opened two threads, both fixed and resolved.

• A group-writable ancestor is refused whatever its group, because a primary group can have other members.
• The configured path is checked as configured, as well as resolved: both chains' ancestors, every link's owner, and no .. anywhere.
• Evidence: 5 new cases fail against 0f77d5c1, and 5 mutants are killed.

Round 9 (84a6c041) applies Brett's two rulings, openxFactory#656 5916000030 items 2 and 3: the carrier and peer authentication, as described above.

• The server confirms it, in its own views:
  • pg_hba_file_rules has exactly the one peer rule (with map=opendox) and the two rejects;
  • pg_ident_file_mappings has exactly the two mappings, for this OS user;
  • system_user is peer:<os user> for both roles.
• The map decides. The same OS user asking for a role outside the map is refused (peer authentication failed). A non-root suite cannot connect as a second OS user, so for that case the map, read back from the server, stands: it names no other user.
• Evidence: 13 new cases fail against 379fbb14. 9 mutants are killed:
  • initdb trust;
  • no map;
  • a trust rule;
  • a wildcard system user;
  • a third role;
  • no re-assertion;
  • any name admitted;
  • files 0644;
  • the old carrier.
    The auth mutants are also killed by the real-server cases alone.

Round 10 (f8e6e9e9): Copilot's review at 84a6c041 raised three points, all fixed.

• An existing postgres/data joins the tree check, a broken link included (r4147680113, resolved).
• Missing parent directories are created exactly 0700 whatever the umask. mkdir(parents=True) under umask 0002 made them group-writable.
• Readiness requires the data directory's lock file to name the launched child, so a racing loser cannot adopt the winner's socket.
• Evidence: 6 new cases fail against 84a6c041, and 4 mutants are killed.

SonarCloud S2115, ACCEPTED, as ruled.

• Issue: AaDvyyOCiqwq-gAw53M3, python:S2115, "Add password protection to this database", on src/opendox/runtime/config.py DatabaseBundle.dsn.
• New status: accept (SonarCloud now reports it RESOLVED). Set with the SonarQube tool on openxFactory#656 5916000030 item 3's authority.
• Rationale: the DSN has no password because the server authenticates Unix-socket connections by PEER. The kernel verifies the connecting uid (SO_PEERCRED), and pg_ident.conf maps only this install's OS user to the two roles. The socket directory is 0700, the server has no TCP listener (listen_addresses is empty), and every host connection is rejected. The same rationale is in the DSN's docstring.
• Gate: after the change, SonarCloud reports the PR's quality gate OK on every condition.

Downstream, for the holder

• Out of scope, and not T072's: serve.py cannot bind ::1. This is pre-existing, measured at DRAFT (phase 3, after T063): T071, 13.2 and 13.3 — load_settings refuses a non-PostgreSQL DSN and a collapsed DSN pair (plan 034) #60's head f097fd8: gaierror [Errno -9] Address family for hostname not supported. serve.LOOPBACK_HOSTS lists ::1, but ThreadingHTTPServer is IPv4. Recorded on DRAFT (phase 3, after T063): T070, 13.4-13.6 — OPENDOX_INSTALL_MODE and generate-and-open --local (plan 034) #67 (T070) for a serve.py follow-up in that file's single-writer order. This PR does not touch serve.py.
• At the merge-from-main round after T055 and T056 land, local_entrypoint_driver.py's stand-ins go, and its test runs the real opendox generate-and-open --local on tests/fixtures/plain-documents.
• T073 reads database_bundle from the server object the entry point keeps (args.database_bundle) for /capabilities' install block.
• T074 runs F13.1 whole, caps.json block included.
• Overlap (gh pr diff -R opensoft/openDox-code): pyproject.toml (T057, openDox's own validator and its input set (7.1, 7.1a, 7.1b, 7.2) (plan 034) #58) and cli.py (T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59, and the T058 writer after it). The plan orders them before this stack, which takes its merge-from-main round after they land. serve.py is not touched here.

🤖 Generated with Claude Code

Summary by Sourcery

Add a self-contained local installation mode that owns, migrates, reports, and cleans up its bundled PostgreSQL server.

New Features:

• Add a standalone local-install mode with a bundled PostgreSQL server managed by the document entry point.
• Expose bundled database location and process information through runtime status, including separate migration and serving credentials over a Unix socket.
• Package migrations and the PostgreSQL carrier in local-install wheels so installations outside a checkout can initialize and migrate their database.

Bug Fixes:

• Prevent local installs from using operator-supplied database or migration DSNs and from accidentally applying migrations from another working directory.
• Ensure stale or conflicting bundled-server state is detected safely and refuse multiple entry points sharing one state directory.

Enhancements:

• Enforce a state-directory layout, absolute-path and Unix-socket length constraints, local-only socket access, and no TCP listener.
• Manage bundled PostgreSQL lifecycle with child-process ownership, graceful shutdown, parent-death cleanup, and bounded startup failure handling.
• Resolve local and hosted migration sources independently while preserving the hosted migration default.

Build:

• Add the local and test dependency extras, pin bundled-server dependencies, and include migrations in built wheels.

Deployment:

• Document OPENDOX_STATE_DIR in the compose environment example.

Tests:

• Add end-to-end coverage for bundled PostgreSQL startup, migration, process ownership, lifecycle cleanup, socket security, refusal behavior, and wheel installation outside a checkout.
• Extend install-mode and runtime-surface tests for local settings, bundled database reporting, packaging, and lifecycle interrupt handling.

[sourcery-ai]

Reviewer's Guide

This PR makes local installation mode self-contained by packaging PostgreSQL binaries and migrations, deriving a private Unix-socket database under OPENDOX_STATE_DIR, managing the server as the document process's child through startup, migration, status reporting, and shutdown, and adding integration and wheel-install tests that falsify the process, security, packaging, and lifecycle guarantees.

Sequence diagram for the local bundled PostgreSQL lifecycle

sequenceDiagram
    participant User
    participant CLI as opendox CLI
    participant Bundle as BundledServer
    participant Postgres as postgres child
    participant Runner as MigrationRunner

    User->>CLI: generate-and-open --local
    CLI->>Bundle: start()
    Bundle->>Bundle: server_binaries()
    Bundle->>Bundle: _initdb()
    Bundle->>Postgres: subprocess.Popen()
    Bundle->>Postgres: wait for readiness
    Bundle->>Bundle: _bootstrap()
    Bundle->>Runner: apply()
    Runner-->>Bundle: applied migrations
    Bundle-->>CLI: report()
    CLI-->>User: serve document surface
    User->>CLI: SIGTERM or Ctrl-C
    CLI->>Bundle: stop()
    Bundle->>Postgres: SIGINT fast shutdown
    Bundle->>Postgres: SIGQUIT if needed
    Bundle->>Postgres: SIGKILL if needed

Loading

Sequence diagram for local runtime status reporting

sequenceDiagram
    participant Operator
    participant RuntimeCLI as runtime status
    participant Config as runtime.config
    participant PID as postmaster.pid

    Operator->>RuntimeCLI: runtime status
    RuntimeCLI->>Config: load_settings()
    Config-->>RuntimeCLI: state_dir and local settings
    RuntimeCLI->>Config: database_bundle(state_dir)
    RuntimeCLI->>PID: read pid
    PID-->>RuntimeCLI: postgres pid or none
    RuntimeCLI-->>Operator: database_bundle data_dir, socket_dir, pid

Loading

File-Level Changes

Change	Details	Files
Add a bundled PostgreSQL lifecycle for local installs, owned directly by the document-serving process.	Resolve a per-install state directory and fixed Unix-socket layout. Start PostgreSQL with subprocess.Popen, no TCP listener, socket permissions, parent-death handling, and bounded shutdown escalation. Bootstrap roles/database, run migrations, expose bundle metadata, and reject concurrent owners.	src/opendox/runtime/bundle.py src/opendox/cli.py src/opendox/runtime/cli.py src/opendox/runtime/config.py
Make local mode self-contained in configuration and packaging.	Derive separate owner and served DSNs while rejecting operator DSNs beside local mode. Add the local extra with pinned PostgreSQL packaging dependencies and include it through the test extra. Package migrations as wheel data and resolve them from the checkout, installed distribution, or editable source tree.	pyproject.toml constraints-cpython312-linux.txt deploy/compose/.env.example src/opendox/runtime/config.py
Expand tests and runtime falsification around local bundled-server guarantees.	Add background entry-point tests for migration, process ancestry, shutdown, socket permissions, no TCP listeners, SIGKILL cleanup, and duplicate-server refusal. Add packaging/wheel installation coverage outside a checkout and update local-mode/configuration tests and import-surface contracts. Use a narrowly scoped generation/serve stand-in until the stacked upstream changes land.	tests_runtime/test_bundled_postgres.py tests_runtime/local_entrypoint_driver.py tests/test_doxbench_entrypoint.py tests/test_install_mode.py tests/test_install_mode_entrypoint.py tests_runtime/test_runtime_surface.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[Copilot]

Copilot review overview

🟡 Changes recommended

Migration-source trust, startup races, lifecycle guarantees, compatibility, and bundled database security require correction.

Review effort: Balanced
Findings: 1 · 4

Open (5)

• Unset migrations path trusts CWD and executes untrusted SQL · New
• Bundled PostgreSQL 16.2 is missing critical security updates · New
• Local extra fails on supported Python 3.13+ interpreters · New
• Stale PID validation accepts unrelated PostgreSQL processes · New
• Blocking readline defeats the 90-second startup timeout · New

What changed in this PR

Adds lifecycle-managed PostgreSQL for local openDox installations, including packaged migrations and runtime reporting.

Changes:

• Starts, migrates, reports, and stops a bundled PostgreSQL child process.
• Adds local state-directory and packaged-migration resolution.
• Adds dependencies, constraints, configuration, and integration coverage.

File	Description
src/​opendox/​runtime/​bundle.py	Implements bundled PostgreSQL lifecycle.
src/​opendox/​runtime/​config.py	Adds bundle paths, DSNs, and migrations lookup.
src/​opendox/​runtime/​cli.py	Reports bundle state.
src/​opendox/​cli.py	Integrates bundle with local entrypoint.
pyproject.toml	Adds local extra and migration data.
constraints-cpython312-linux.txt	Pins new dependencies.
deploy/​compose/​.env.example	Documents the state directory.
tests_runtime/​test_bundled_postgres.py	Adds end-to-end bundle tests.
tests_runtime/​local_entrypoint_driver.py	Provides an integration-test driver.
tests_runtime/​test_install_mode.py	Updates local-mode expectations.
tests_runtime/​test_runtime_surface.py	Verifies lightweight imports.
tests/​test_install_mode_entrypoint.py	Updates entrypoint shape tests.
tests/​test_doxbench_entrypoint.py	Adds a bundle stand-in.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Startup recovery, signal handling, PID validation, migration discovery, and a blocking test timeout require correction.

Review effort: Balanced
Findings: 3 · 6 · 1

Open (10)

• Atomically install initialized data directories · New
• Prefer migration files from the executing source tree · New
• Unset migrations path trusts CWD and executes untrusted SQL
• Handle SIGTERM throughout the complete local lifecycle · New
• Normalize errors across the entire startup operation · New
• Blocking readline defeats the 90-second startup timeout
• Stale PID validation accepts unrelated PostgreSQL processes
• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates
• Correctly classify and explain refused DSN settings · New

[Copilot]

Copilot review overview

🟡 Changes recommended

Startup failures, path expansion errors, and stale PID handling can currently produce tracebacks or incorrect bundle status.

Review effort: Balanced
Findings: 3 · 8 · 1

Open (12)

• Prefer migration files from the executing source tree
• Atomically install initialized data directories
• Unset migrations path trusts CWD and executes untrusted SQL
• PID fallback accepts dead or unrelated processes · New
• Unknown home users escape configuration error handling · New
• Normalize errors across the entire startup operation
• Handle SIGTERM throughout the complete local lifecycle
• Blocking readline defeats the 90-second startup timeout
• Stale PID validation accepts unrelated PostgreSQL processes
• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates
• Correctly classify and explain refused DSN settings

[Copilot]

Copilot review overview

🟡 Changes recommended

Stale PID handling can block or misidentify the bundled server, and packaging compatibility concerns remain unresolved.

Review effort: Balanced
Findings: 5 · 1

Open (6)

• Unknown process identity is treated as a verified match · New
• Unknown home users escape configuration error handling
• PID fallback accepts dead or unrelated processes
• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates
• Comment references nonexistent packaged_migrations_dir function · New

Resolved since last review (8)

• Prefer migration files from the executing source tree
• Atomically install initialized data directories
• Unset migrations path trusts CWD and executes untrusted SQL
• Normalize errors across the entire startup operation
• Handle SIGTERM throughout the complete local lifecycle
• Blocking readline defeats the 90-second startup timeout
• Stale PID validation accepts unrelated PostgreSQL processes
• Correctly classify and explain refused DSN settings

[Copilot]

Copilot review overview

🟡 Changes recommended

Relative HOME values can violate the absolute state-directory invariant, while existing carrier compatibility and security concerns remain unresolved.

Review effort: Balanced
Findings: 3

Open (3)

• Reject relative HOME fallback to preserve absolute state paths · New
• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates

Resolved since last review (12)

• Prefer migration files from the executing source tree
• Atomically install initialized data directories
• Unset migrations path trusts CWD and executes untrusted SQL
• Unknown process identity is treated as a verified match
• Unknown home users escape configuration error handling
• PID fallback accepts dead or unrelated processes
• Normalize errors across the entire startup operation
• Handle SIGTERM throughout the complete local lifecycle
• Blocking readline defeats the 90-second startup timeout
• Stale PID validation accepts unrelated PostgreSQL processes
• Comment references nonexistent packaged_migrations_dir function
• Correctly classify and explain refused DSN settings

[Copilot]

Copilot review overview

🟡 Changes recommended

Inherited libpq settings can redirect local connections, and unsafe state-directory parents undermine the socket trust boundary.

Review effort: Balanced
Findings: 2 · 3

Open (5)

• Unsafe state directory ancestors undermine private socket security · New
• Inherited PG* variables can redirect local database connections · New
• Reject relative HOME fallback to preserve absolute state paths
• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates

[Copilot]

Copilot review overview

🟡 Changes recommended

Filesystem trust checks and interrupted initialization can allow socket redirection or leave child processes running.

Review effort: Balanced
Findings: 2 · 2

Open (4)

• Reject group-writable ancestors even when the primary GID matches · New
• Prevent symlink retargeting of the configured state path · New
• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates

Resolved since last review (3)

• Inherited PG* variables can redirect local database connections
• Unsafe state directory ancestors undermine private socket security
• Reject relative HOME fallback to preserve absolute state paths

Previously missed (2)

In code that hasn't changed since last review

Remove stale locks when an inaccessible PID is not the postmaster

src/​opendox/​runtime/​bundle.py:251

A PermissionError here means the PID is live but belongs to a process this user cannot signal, so by this module's same-user ownership invariant it cannot be this bundle's postmaster. Leaving that stale lock delegates to PostgreSQL, which treats a live PID as an active postmaster and can refuse every restart until the unrelated process exits. Distinguish this case from an unverifiable platform and remove the lock only after _serves() confirms a mismatch.

Terminate and reap initdb on signal-driven interruption

src/​opendox/​runtime/​bundle.py:553

subprocess.run() does not terminate initdb when the SIGTERM handler raises KeyboardInterrupt; its Popen cleanup waits briefly and re-raises, while self.process is still None. The entry point can therefore return 143 with initdb still writing to an attempt named with the now-dead parent PID, and a restart may delete that live attempt. Manage initdb with an explicit child handle and terminate/kill/wait it on BaseException before removing the attempt.

[Copilot]

Copilot review overview

🟡 Changes recommended

The unresolved PostgreSQL 16.2 security age and unsupported Python 3.13+ carrier compatibility remain release blockers.

Review effort: Balanced
Findings: 2

Open (2)

• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates

Resolved since last review (2)

• Prevent symlink retargeting of the configured state path
• Reject group-writable ancestors even when the primary GID matches

[Copilot]

Copilot review overview

🟡 Changes recommended

Startup can fail under common umasks, omit data-directory symlink validation, and accept another concurrently launched server as its own.

Review effort: Balanced
Findings: 1

Open (1)

• Validate data symlinks in ownership and safety checks · New

Resolved since last review (2)

• Local extra fails on supported Python 3.13+ interpreters
• Bundled PostgreSQL 16.2 is missing critical security updates

Previously missed (2)

In code that hasn't changed since last review

Create parent directories explicitly with mode 0700

src/​opendox/​runtime/​bundle.py:534

parents=True does not apply mode=0o700 to missing parents: pathlib creates those with its default 0777 masked by the process umask. With a common umask 0002, a fresh default such as ~/.local/state/opendox can therefore create .local/state as 0775, after which _refuse_an_unsafe_tree() rejects the directories this method just created. Create each missing component explicitly with mode 0700 so first startup works independently of umask.

Tie readiness checks to the launched PostgreSQL process

src/​opendox/​runtime/​bundle.py:710

Readiness is not tied to the process just launched. If two entry points race from an idle state (or process identity cannot be verified without /proc), the losing postgres can still be alive briefly while this connection succeeds against the winner's already-listening socket; this instance then completes bootstrap/migration and serves while owning no database child. Before and after connecting, require postmaster.pid to name self.process.pid; otherwise keep waiting for this child to exit/refuse.

[sonarqubecloud]

Quality Gate passed

Issues
30 New issues
1 Accepted issue

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Copilot review overview

🔵 Needs a closer look

Authentication overrides, unsafe pre-validation filesystem mutations, stale locks, and restrictive-umask failures remain unresolved.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Validate data symlinks in ownership and safety checks

Previously missed (1)

In code that hasn't changed since last review

Handle EPERM stale locks with process identity checks

src/​opendox/​runtime/​bundle.py:265

PermissionError means this PID is not signalable, but returning here skips the existing identity check. If a stale lock's PID has been recycled by another user, PostgreSQL treats kill(pid, 0) == EPERM as a live process and the local install remains blocked until that unrelated process exits. Continue to _serves; remove the lock only when the available process inspection says it is not this data directory's server.

This issue also appears in the following locations of the same file:

• line 438
• line 558
• line 659
• line 699
• line 703

[brettheap]

On the overview's "previously missed" note at f8e6e9e9, "Handle EPERM stale locks with process identity checks" (bundle.py _remove_a_proven_stale_lock): the early return on PermissionError is deliberate. The note's premise about PostgreSQL does not hold for 16.

The note says PostgreSQL treats kill(pid, 0) == EPERM as a live process. PostgreSQL 16 does the opposite. src/backend/utils/init/miscinit.c (REL_16_STABLE), CreateLockFile, reads:

if (kill(other_pid, 0) == 0 ||
    (errno != ESRCH && errno != EPERM))
{
    /* lockfile belongs to a live process */
    ereport(FATAL, ... "lock file \"%s\" already exists" ...

EPERM is read as stale, and the comment above that code says why: a postmaster cannot attach to a data directory owned by another uid. So a lock whose pid was recycled to another user's process is removed by PostgreSQL itself at the next start, and nothing here needs to prove anything.

That is why _remove_a_proven_stale_lock returns on both ProcessLookupError and PermissionError ("PostgreSQL's own rule covers both"). It removes a lock itself only in the one case PostgreSQL refuses: a same-user live process that /proc proves is not this data directory's postmaster.

running_pid also returns None for EPERM, so status never reports another user's process as this server. That behaviour is pinned by test_another_users_process_is_not_this_server.

No code change.

Lane: openxfactory-4 (openXfactory-4-openDox_extraction)