Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# AGENTS.md

## First action in any coding-agent session

Run `uv run manage.py dev agent-info` from the checkout. It describes the
supported local lifecycle commands, ownership, automatic port allocation,
worktree isolation, cleanup and restart rules. Run `uv run manage.py dev status`
before launching services. `uv run manage.py dev` is idempotent: if the current
worktree already has a live owner, it reports the URL and exits.

**Never use `pkill`, `kill -9`, `lsof -ti ... | xargs kill`, or the old `devctl.sh`
port-clearing logic.** Ordinary Django Python edits use runserver autoreload;
template edits require only a browser refresh. Hatchet worker Python code does
not automatically reload in the current architecture; coordinate a full
`dev restart` only when no audit must be interrupted. Do not point another
worktree's Python at the main checkout server and claim that branch's code
was tested. `reset`/`uninstall` destroy data and require explicit permission.

SimpleAudit Studio — an audit platform for LLM systems. It wraps the external
SimpleAudit engine (Target → Auditor → Judge runs over versioned scenario sets)
with a Django web UI, a durable worker, and an optional Open WebUI chat module.
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,15 @@ uv run manage.py setup_local
uv run manage.py dev
```

For an existing instance, `dev` reuses it rather than restarting the entire
stack. Run `uv run manage.py dev status` to see the actual port. Run
`uv run manage.py dev agent-info` for detailed agent-safe instructions.
`dev down` stops without data loss, `dev reset` recreates clean local data,
and `dev uninstall` removes owned state and app-scoped uv caches. The same
lifecycle actions are available as `uvx --from simpleaudit-studio spin ACTION`.
See [development lifecycle](docs/development-lifecycle.md) for isolation and
safety details.

The app opens at <http://localhost:8000> with live reload. Use `--no-browser` to skip opening a browser.

## What it does
Expand Down
115 changes: 13 additions & 102 deletions devctl.sh
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -12,107 +12,18 @@
# Ports: Studio :8001, OWUI :8080, Caddy :8000
set -euo pipefail

PORT_STUDIO=8001
PORT_OWUI=8080
PORT_CADDY=8000
PIDFILE="$HOME/.simpleaudit-studio/dev.pid"
LOGFILE="$HOME/.simpleaudit-studio/dev.log"
DB="dev.sqlite3"

REPO="$(cd "$(dirname "$0")" && pwd)"

_kill_port() {
local port=$1
local pids
pids=$(lsof -ti :"$port" 2>/dev/null || true)
if [[ -n "$pids" ]]; then
# shellcheck disable=SC2086
kill -9 $pids 2>/dev/null || true
fi
}

stop() {
echo "Stopping stack (ports $PORT_STUDIO $PORT_OWUI $PORT_CADDY)..."
for p in "$PORT_STUDIO" "$PORT_OWUI" "$PORT_CADDY"; do _kill_port "$p"; done
# Also kill any lingering manage.py dev processes
local mpids
mpids=$(pgrep -f "manage.py dev" 2>/dev/null || true)
if [[ -n "$mpids" ]]; then
# shellcheck disable=SC2086
kill -9 $mpids 2>/dev/null || true
fi
rm -f "$PIDFILE"
sleep 2
# Verify
local remaining
remaining=$(lsof -ti :$PORT_STUDIO,$PORT_OWUI,$PORT_CADDY 2>/dev/null || true)
if [[ -n "$remaining" ]]; then
echo "WARNING: ports still occupied: $remaining"
else
echo "Stopped."
fi
}

start() {
if [[ -f "$PIDFILE" ]] && kill -0 "$(cat "$PIDFILE")" 2>/dev/null; then
echo "Already running (PID $(cat "$PIDFILE")). Use 'restart' to restart."
return 0
fi
stop # ensure clean slate

# Clear stale bytecode so code changes are always picked up
find "$REPO" -path "*__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true

echo "Starting stack (Studio :$PORT_STUDIO, OWUI :$PORT_OWUI, Caddy :$PORT_CADDY)..."
mkdir -p "$(dirname "$LOGFILE")"

cd "$REPO"
export SIMULACHAT_API_KEY="${SIMULACHAT_API_KEY:-${CUSTOM_SIMULACHAT_KEY:-}}"
SIMPLEAUDIT_LOCAL_SQLITE=1 \
SIMPLEAUDIT_SQLITE_PATH="$DB" \
SIMPLEAUDIT_BROWSER_PORT="$PORT_CADDY" \
nohup uv run manage.py dev --port "$PORT_STUDIO" > "$LOGFILE" 2>&1 &
local pid=$!
echo "$pid" > "$PIDFILE"
echo "Started (PID $pid). Log: $LOGFILE"

# Wait for health
local i
for i in $(seq 1 40); do
if curl -s -o /dev/null -w "%{http_code}" "http://127.0.0.1:$PORT_STUDIO/healthz" 2>/dev/null | grep -q 200; then
echo "Studio healthy."
local owui
owui=$(curl -s "http://127.0.0.1:$PORT_OWUI/chat/health" 2>/dev/null || echo "")
local caddy
caddy=$(curl -s -o /dev/null -w "%{http_code}" "http://127.0.0.1:$PORT_CADDY/" 2>/dev/null || echo "000")
echo "OWUI: ${owui:-down} Caddy(8000): HTTP $caddy"
return 0
fi
sleep 2
done
echo "ERROR: Studio did not become healthy in 80s. Check $LOGFILE"
return 1
}

status() {
local h
h=$(curl -s -o /dev/null -w "%{http_code}" "http://127.0.0.1:$PORT_STUDIO/healthz" 2>/dev/null || echo "000")
local o
o=$(curl -s "http://127.0.0.1:$PORT_OWUI/chat/health" 2>/dev/null || echo "down")
local c
c=$(curl -s -o /dev/null -w "%{http_code}" "http://127.0.0.1:$PORT_CADDY/" 2>/dev/null || echo "000")
echo "Studio(:$PORT_STUDIO): HTTP $h"
echo "OWUI(:$PORT_OWUI): $o"
echo "Caddy(:$PORT_CADDY): HTTP $c"
if [[ -f "$PIDFILE" ]]; then
echo "PID file: $(cat "$PIDFILE")"
fi
}

# Compatibility wrapper delegating to the ownership-safe lifecycle manager.
# The legacy kill-by-port cleanup was removed: no path here may terminate a
# process by port, name, or broad pattern (another agent's worktree included).
_REPO="$(cd "$(dirname "$0")" && pwd)"
case "${1:-}" in
start) start ;;
stop) stop ;;
status) status ;;
restart) stop; start ;;
*) echo "Usage: $0 {start|stop|status|restart}"; exit 1 ;;
start)
mkdir -p "$HOME/.local/state/simpleaudit-studio"
(cd "$_REPO" && nohup uv run manage.py dev --no-browser \
>> "$HOME/.local/state/simpleaudit-studio/devctl.log" 2>&1 </dev/null &)
exit 0 ;;
stop) cd "$_REPO"; exec uv run manage.py dev down ;;
status) cd "$_REPO"; exec uv run manage.py dev status ;;
restart) cd "$_REPO"; exec uv run manage.py dev restart --no-browser ;;
*) echo "Usage: $0 {start|stop|status|restart}"; exit 2 ;;
esac
1 change: 1 addition & 0 deletions infra/management/commands/dev.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,5 @@ def handle(self, *args, **options):
embedded=True,
no_reload=options["no_reload"],
no_browser=options["no_browser"],
no_force_kill=True, # never evict another agent's stack
)
38 changes: 34 additions & 4 deletions infra/management/commands/dev_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ def handle(self, *args, **options):
chat_enabled=_chat_config.ENABLED,
chat_internal_port=_chat_config.INTERNAL_PORT,
chat_upstream_url=_chat_config.UPSTREAM,
force_kill=not options["no_force_kill"],
force_kill=False, # no implicit process termination in development
yes=options["yes"],
command_hint="manage.py dev --port <free port>",
)
Expand Down Expand Up @@ -447,9 +447,39 @@ def report():
return process

def _stop_web(self, web_proc):
if web_proc is not None:
web_proc.terminate()
web_proc.wait(timeout=5)
"""Stop the auto-reload web server (guard → reloader parent → server).

Terminating only the guard can leave an orphaned runserver holding the
internal port, because the reloader parent does not always notice the
guard's death in time. We therefore SIGTERM the runserver processes by
PID (never by process group — the reloader tree shares the owner's
process group with Caddy/Open WebUI, which must keep running) and then
wait for the guard to exit.
"""
if web_proc is None:
return
self._terminate_runserver_tree(web_proc.pid)
import subprocess

try:
web_proc.wait(timeout=8)
except subprocess.TimeoutExpired:
logger.warning("web server did not stop in time; left as-is")

@staticmethod
def _terminate_runserver_tree(guard_pid: int) -> None:
"""SIGTERM the runserver processes under ``guard_pid`` (by PID, not pgid)."""
import psutil

try:
guard = psutil.Process(guard_pid)
except psutil.NoSuchProcess:
return
for proc in guard.children(recursive=True):
try:
proc.terminate()
except psutil.NoSuchProcess:
pass

def _stop_chat(self):
try:
Expand Down
27 changes: 26 additions & 1 deletion manage.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,32 @@ def main() -> None:

if command == "dev":
os.environ.setdefault("SIMPLEAUDIT_MODE", "dev")
single_process = runtime.is_single_process_run(command, argv)
# Handle lifecycle requests before django.setup(), migrations, or any
# imports that lock in chat ports/settings.
from pathlib import Path

from simpleaudit_studio.lifecycle import ACTIONS, Instance, act, start

instance = Instance("dev", Path(__file__).resolve().parent)
rest = sys.argv[2:]
action = rest[0] if rest and rest[0] in ACTIONS else None
if act(instance, action, yes="--yes" in rest):
return
if action in {"reset", "restart"}:
sys.argv = [sys.argv[0], "dev"] + [x for x in rest[1:] if x != "--yes"]
rest = sys.argv[2:]
requested = int(os.environ.get("PORT", "8000"))
explicit = False
for index, arg in enumerate(rest):
if arg == "--port" and index + 1 < len(rest):
requested = int(rest[index + 1])
explicit = True
elif arg.startswith("--port="):
requested = int(arg.split("=", 1)[1])
explicit = True
if not start(instance, requested_port=requested, explicit_port=explicit):
return
single_process = runtime.is_single_process_run(command, sys.argv)
# Zero-Docker modes imply SQLite for the domain DB. Decide before Django
# loads settings: a Postgres `.env` would otherwise make settings import
# the Postgres driver first.
Expand Down

This file was deleted.

Loading
Loading