diff --git a/CHANGELOG.md b/CHANGELOG.md index f2f9a53..5b4e596 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,34 @@ All notable changes to this project are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +## 0.8.2 - unreleased + +### Added +- `probbit monitor STRAND --open` (and `--serve`) shows an Activity Monitor for one individual: a sortable, filterable table of + the persona's declared traits, moods and, when present, drive channels, with each channel's reported level or value, a trace + over recorded events and a change against exactly the preceding event (a gap when that document is missing); an inspector + with the full distribution, units, declared range, source field and the event's `why`; holding a past event while new ones + arrive; and separate states for replay divergence, stale receiver age, a held view, a paused or retired strand, a + persona-confidence hold, an engine-unreleased trait and a lost connection. It works for any persona read from the strand. + The page is embedded in the binary with no external assets; the terminal view (`--once`, `--follow`, `--plain`) is unchanged. + The monitor's layout gains `caps.prediction_error` from the persona's `expectation.pe_cap`. docs/persona.md §5.8. + +### Changed +- `persona lint` with rules (`--never RULE` / `--props FILE`) no longer reports green on a rule it could not settle. When + `prove` leaves a rule `unknown` and the default `fuzz` search finds no counterexample, the rule is **unproven**: the lint + document gains `"unproven": [{id, rule, seeds_unknown, searched}]`, `ok` is false, stderr says "unproven, not broken", and the + exit code is the new **3** (0: every rule held or proved; 1: an unresolved contradiction or a counterexample, which wins over + 3; 2: bad input, as before). Up to 0.8.1 the same case exited 0 with `ok: true`. docs/persona.md §5.5. +- `persona fuzz`, MCP `probbit_persona_fuzz` and `lint`'s search add every threshold the rule's `when` and the habits' `when` + read on a number input to the grid (as a fraction of that input's max, after the given points, each once; `search.grid` lists + them). A rule in force only between two default grid points (for example stakes 0.7 to 0.8) is now reached by the random + search too. Personas whose rules and habits read no number thresholds off the grid get the same search, byte for byte. + +### Fixed +- `install.sh`: when the install directory is not on `PATH`, the hint names the file the user's shell reads: `~/.zshrc` for + zsh, `~/.bash_profile` for bash on macOS and `~/.bashrc` elsewhere, `~/.profile` for any other shell (it said "your shell + profile"). The clean-prefix install test checks each case. + ## 0.8.1 - 2026-10-10 ### Fixed diff --git a/README.md b/README.md index fcceb6a..8196e6a 100644 --- a/README.md +++ b/README.md @@ -461,7 +461,8 @@ probbit persona prove examples/persona/tutor.yaml --seeds 0-99 \ # held by construction (habit no_play_when_upset) (exit 0; the tutor now ships with that one habit) ``` -`probbit persona lint PERSONA --props rules.json` runs both in CI (exit 1 when a rule breaks); MCP `probbit_persona_fuzz` and +`probbit persona lint PERSONA --props rules.json` runs both in CI (exit 1 when a rule breaks, exit 3 when a rule is +unproven: unknown to the bound and unbroken by the search); MCP `probbit_persona_fuzz` and Python `probbit.persona_fuzz` / `persona_prove` give the same documents. Both commands test the stance the host gets, not the words a model writes; "none found" from `fuzz` is evidence over the scripts it searched, not a proof ([docs/persona.md](docs/persona.md) §5.6 has the bound and its limits). @@ -524,19 +525,25 @@ A persona without the block gives the same documents as before, byte for byte. M cues, rewards and praises fun on this persona, 100 individuals x 10,000 turns: 0 habit breaks, all 6,743 must-do turns pursued the due chore, and the safety floor (0.1) held on every turn its habits allowed (least odds 0.141), while fun took 85 % of turns. -### Watch it live: `probbit monitor` (0.8.0) +### Watch it live: `probbit monitor` (0.8.0; Activity Monitor page 0.8.2) -`probbit monitor pip.strand` replays a strand with the rules of `live verify` and draws the individual's inner state as live bars: -each trait's levels with their odds, the moods, the event's inputs, the habits that bound, the learned weights and the line, in the -terminal (`--follow` redraws as the strand grows) or as a page on 127.0.0.1 (`--serve`). It reads the strand and sends nothing -anywhere ([docs/persona.md](docs/persona.md) §5.8). No strand yet? The tutor's week, in under a minute: +`probbit monitor pip.strand --open` opens a compact, read-only **Activity Monitor** for one individual: a sortable, +searchable table of declared traits, moods and drives, with reported probabilities, event-history traces, and a selected-channel +inspector. It reads any compatible persona from the strand; no preset character or agent integration is required. ```sh -probbit monitor --demo --open -probbit monitor --demo drives --open # synthetic goals/drive demo (0.8.1) +probbit monitor pip.strand --once --plain # native CLI snapshot; no browser or server +probbit monitor pip.strand --follow # live terminal bars +probbit monitor pip.strand --open # browser-rendered utility on 127.0.0.1 +probbit monitor --demo --open # no strand yet: the tutor's synthetic week, in under a minute +probbit monitor --demo drives --open # synthetic goals/drive scenario ``` -![probbit monitor: the tutor's week in the browser, the bars moving with every event](docs/probbit-monitor.gif) +Select a channel to inspect its units, range, distribution, source and the engine's `why`. Hold an event while new ones +arrive, or follow latest. History means **recorded events, not seconds**. This is declared engine state, not thoughts or +CPU/GPU/memory usage. Replay verification, stale event age, held view, paused/retired strand and lost connection are distinct. +Everything is embedded in the CLI: no dependencies, external assets, telemetry or Electron installation. +See [the monitor guide](docs/persona.md#58-monitor-watch-an-individuals-inner-state) for the data and replay boundaries. ## After any judge: `probbit evaluate` diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index 1c09108..014a388 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -59,8 +59,8 @@ The exact tiers run before the sampler: | `live PERSONA [--seed N \| --state FILE] [--strand FILE] [--events FILE]` | JSONL events in, one stance per event out | | `live PERSONA --demo week [--seed N] [--plain]` | a scripted week: learning to a cap, a night, rules that hold | | `live verify STRAND` | replay a strand; the earliest line that differs | -| `monitor STRAND [--follow] [--plain] [--serve] [--open] [--port N]` | the strand replayed as live bars: the terminal, or a page on 127.0.0.1 | -| `monitor --demo [--open]` | the tutor's week, paced, in under a minute | +| `monitor STRAND [--follow] [--once] [--plain] [--serve] [--open] [--port N]` | the strand replayed as live bars in the terminal, or the Activity Monitor page on 127.0.0.1 (channel table, event-history traces, inspector) | +| `monitor --demo [week\|drives] [--open]` | synthetic data: the tutor's week, paced, in under a minute; `drives`: a short goal-signal scenario | ## Testing a character - `lint`: contradicting habits; with `--props`, every rule proved or fuzzed; diff --git a/docs/agents.md b/docs/agents.md index 9d31278..b484e35 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -216,5 +216,5 @@ Practical notes for agent use: (on Linux and macOS) `--priority low` keep it from crowding the agent's own process. - `--top`, `demo --live` and the hero screen draw only on a terminal; an agent's pipes never see them. - A persona's character rules in CI: `probbit persona lint PERSONA --props rules.json` proves each rule or, when the bound - cannot decide it, fuzzes it (exit 1 when one breaks); `probbit persona fuzz` / `prove` with `--json` give the documents + cannot decide it, fuzzes it (exit 1 when one breaks; exit 3 when one stays unproven: unknown and no counterexample found); `probbit persona fuzz` / `prove` with `--json` give the documents (docs/persona.md §5.6). They test the stance a host gets, not the words a model writes. diff --git a/docs/persona.md b/docs/persona.md index 6f2f5d2..4612fdf 100644 --- a/docs/persona.md +++ b/docs/persona.md @@ -567,6 +567,15 @@ tie, the later-declared one), the turn is re-solved, and `habits.yielded` and `e previous level their `prev` restrictions read) for an empty stance, and reports each contradiction with its resolution. Run it before shipping a persona. +**Exit codes.** `lint` exits 0 when every contradiction is resolved and, with rules (`--never RULE` / `--props FILE`, section +5.6), every rule is held by construction or proved; 1 when a contradiction is +unresolved or a rule has a counterexample; 2 on a bad persona, rule or flag; 3 (since 0.8.2) when nothing is broken but some +rule is **unproven**: `prove` left it `unknown` and the default `fuzz` search found no counterexample. Unproven is not broken, +and it is not green either: the JSON's `ok` is false and `unproven` lists each such rule as `{id, rule, seeds_unknown, +searched}` (`seeds_unknown`: the individuals the bound could not decide; `searched`: `individuals`, `scripts`, `depth`, `beam`, +`grid`, `hours`), and stderr says "unproven, not broken". Exit 1 wins over 3. A CI gate that should accept unproven rules +checks for exit 3 explicitly; up to 0.8.1 the same case exited 0 with `ok: true`. + **Plan and mode.** A stance level is the joint plan's, so on coupled traits it can differ from the trait's own most likely level (section 4): the 0.5.0 tutor's seed 1, told the learner is upset, planned humour `playful` (odds 0.411) while `light` had 0.425, and valence `up` (0.386) while `even` had 0.457; the two are coupled (`align: 1.0`) and the plan took their high levels @@ -586,7 +595,10 @@ writes. Two commands test it over a population of individuals (`--seeds 0-99`); line of YAML), `--props FILE` several (`{"props": [...]}`, each with an optional `id`). **`fuzz`** searches event scripts built from the persona's declared inputs (flags, levels, numbers on `--grid` x their max, -idle hours on `--hours`): random scripts, then a beam guided by the exact odds (the next events that put the most odds on a +idle hours on `--hours`). The grid (default 0, 0.5, 1) also gets, after the given points and each once, every threshold the +rule's `when` and the habits' `when` read on a number input, as a fraction of that input's max (since 0.8.2): a rule in force +only at stakes 0.7 to 0.8 is searched at 0.7 and 0.8; `search.grid` in the JSON lists the grid used. The search runs random +scripts, then a beam guided by the exact odds (the next events that put the most odds on a level the rule forbids). Each individual's shortest counterexample is shrunk (drop events, then single inputs) and printed with the breaking turn's levels, odds, why and line, and the `replay` / `explain` commands that reproduce it. The search is Philox keyed by `--fuzz-seed` and the seed: the same inputs give the same report, byte for byte, on any number of threads. "None @@ -635,7 +647,8 @@ every turn in that range. On the drives fixture of the tests: "a chore due withi **`lint --props FILE`** (or `--never RULE`) runs both for CI: `prove` on every rule, then `fuzz` (the default search) on the ones it leaves unknown; the lint document gets `props` (each rule's `prove` entry, with a `fuzz` entry when it was unknown) and -`broken`; exit 1 when a contradiction is unresolved or a rule breaks. +`broken` and `unproven`; exit 1 when a contradiction is unresolved or a rule breaks, exit 3 when a rule stays unknown and the +search finds no counterexample (section 5.5). Exit codes: `fuzz` 0 nothing found, 1 a counterexample; `prove` 0 every rule held or proved, 1 some rule unknown; both 2 bad input. JSON: `--json` (`probbit_persona_fuzz: 1`, `probbit_persona_prove: 1`); timing goes to stderr. @@ -801,19 +814,19 @@ invalid move, 4 when the lock is held or the individual is retired. ### 5.8 Monitor: watch an individual's inner state -`probbit monitor STRAND` draws an individual's inner state as live horizontal bars, in the terminal or as a page in the browser. +`probbit monitor STRAND` draws an individual's declared state as live terminal bars, or as a compact browser Activity Monitor. It replays the strand with the rules of `live verify` (the two share one replay), recomputing every stance document from the strand alone: it needs no other log, works for any strand on any machine, and says "replay verified" while every line checks, or names the line that differs. ```sh probbit monitor pip.strand --follow # bars in the terminal, redrawn as the strand grows -probbit monitor pip.strand --open # the same board as a page at http://127.0.0.1:PORT/, in the browser +probbit monitor pip.strand --open # a channel table and inspector at http://127.0.0.1:PORT/, in the browser probbit monitor --demo --open # the tutor's week, paced, for someone without a strand yet probbit monitor --demo drives --open # eight synthetic events with goals and drives ``` -**What it shows.** The latest event. A header: persona and version, seed, the individual's digest, the event number, the hours +**What it shows.** The latest event (or a selected historical event in the browser). In the terminal, a header: persona and version, seed, the individual's digest, the event number, the hours since the event before, the replay badge. The stance: one row per trait, its levels as segments sized by their odds, the level taken in colour and the phrase it says. The moods, with a sparkline of the last 50 events. The senses: the event's inputs as the turn used them (flags lit, numbers as bars, levels by name, the clock) and the history features against their caps. The habits @@ -840,12 +853,61 @@ cannot read the documents: | route | answer | |---|---| -| `/` | the page: one file embedded in the binary (about 21 KB, styles and script inline; it fetches no fonts, styles or scripts), with a Content-Security-Policy that lets it connect to its own server and nowhere else | +| `/` | the page: one file embedded in the binary (under 64 KB, styles and script inline; it fetches no fonts, styles or scripts), with a Content-Security-Policy that lets it connect to its own server and nowhere else | | `/events` | server-sent events: `meta` (the layout: the strand, the engine, the persona, its traits, moods, inputs and history features in the persona's order), the latest `frame`, then a frame per event: `{n, doc, given, learned, spark, diverges, note}`, that is the stance document, the inputs as the strand logs them, the learned deltas, the moods' recent positions, the line that differs and a warning; a heartbeat comment after 15 quiet seconds; events that land within one poll (100 ms), or while a page falls behind, come as the latest frame | | `/doc/N` | event N's stance document, canonical JSON and a newline, as `probbit live` printed it (its sha256 is the `stance` digest of the strand's line N + 1); the latest 10,000 are kept | -The page draws the terminal's board in a dark theme, the bars moving as the odds do (`prefers-reduced-motion` respected), from -360 px wide up. No file is changed, and nothing leaves the machine. +The browser opens a compact **Activity Monitor for one individual**, not a fleet dashboard. Its utility-style toolbar, +channel table and inspector use system fonts and an automatic light/dark appearance (also selectable in the toolbar). +The desktop list scrolls inside the utility; at narrow widths the inspector sits below it. This is a browser-rendered CLI +surface, not a native Swift, WinUI or Electron app. For a native terminal snapshot, use `probbit monitor STRAND --once`; +`--plain` adds a portable text-only form. No separate application installation is needed. + +**Channels.** Every declared trait and mood gets a row; when present, `pursue` and each goal's wanting, afterglow and +prediction error become drive rows. All/Traits/Moods/Drives filters, a text filter, sortable columns and keyboard selection +work for arbitrary persona names and channel declarations. A persona-confidence hold is marked **held**, with pre-hold odds in the inspector; an engine-unreleased trait is separately marked **unreleased**. Missing +values are unreported, not zero. A channel's inspector shows its actual selected level/value, each reported probability, +units, declared range, source field, optional supplied phrase and the engine's event-level `why`. That `why` is not a +per-channel causal attribution or a model's thoughts. + +**Reading the graphs.** The table trace for a discrete channel follows the *currently selected level's probability* through +past documents; the inspector plots each declared level's probability separately. No semantic ordering is assumed for the +level names. Drive traces show the reported numeric values in engine units (wanting/afterglow) or reward error (prediction +error), against the declared cap (including `expectation.pe_cap` for prediction error). `Δ event` compares the same quantity to the immediately preceding +event, when its document is available; probabilities use percentage points. Unknown history is a gap, not a connecting line or fabricated zero. +Both chart axes and timeline refer to **event numbers**, not wall-clock timestamps. Quiet connections never create state, +mood decay or graph samples; SSE heartbeat comments are not events. + +**Inspecting history.** The timeline loads at most the latest 50 documents from `/doc/N`, with at most four concurrent +requests. Equal-sized marks represent recorded events, not workload. Select a mark, use previous/next, or use the +keyboard-accessible scrubber to hold an event. **Hold view** freezes only the browser's inspected event; the individual +and stream continue. **Follow latest (+N)** returns to the newest event. If a held event ages out of that 50-document +window, its displayed state is retained, the out-of-window condition is named, and the scrubber is hidden until a recent +event is selected. A checkpoint bounds available history. Demo restarts, changed metadata or observed strand resets clear +the cache; metadata arriving without a following frame immediately clears the prior individual's visible state. + +`/doc/N` contains a stance document, not a complete monitor frame. Historical documents therefore have no invented raw +clock inputs or learned deltas. These appear in the expandable event-evidence area only when that event's full frame was +actually received. Document inputs, history features and rules remain available. The JSON link follows the inspected event. + +**Status is separate from the graph.** Live/awaiting-event, synthetic replay, held view, paused/frozen strand, permanently +retired strand, replay divergence and disconnected/retrying are distinct. After 60 seconds without a newly numbered event +received in this browser, an otherwise connected active strand is labelled **stale event**. This is *receiver age*, not an +engine timestamp and not evidence that the host or agent is idle. Disconnects retain the last received state without claiming +live activity. The replay badge describes the current stream and its checkpoint scope, even while an older event is held. + +**Scope.** One endpoint has one individual. No agent fleet, thoughts, CPU, GPU, memory, token use or task activity is inferred. +To add multiple agents, a host adapter would need an explicit registry of endpoint/individual identities and independently +measured lifecycle and work telemetry, each with its own timestamp/source. Those are not supplied by the current strand or +monitor API. A future fleet surface must keep per-individual replay/history and source attribution separate; a list of +channels is not a list of agents. + +The page works from 360 px wide, respects `prefers-reduced-motion`, and loads no external assets or dependencies. +No strand is changed, and nothing leaves the machine. Real Chrome/Chromium regressions (Node 22+, no npm dependencies): +`cargo build --release -p probbit-cli`, then `node probbit-cli/tests/monitor_browser.mjs`. Only authored synthetic fixtures are +used, visibly labelled in screenshots. Generated evidence defaults to a temporary directory, not the repository. The checks +cover sorting/filtering/keyboard selection, arbitrary and traits-only personas, unreleased levels, append/hold/backfill, +missing history gaps, checkpoint bounds, 50 documents/four fetches, stale/frozen/retired/disconnected states and metadata reset. **Demo.** `probbit monitor --demo` replays the week of `probbit live examples/persona/tutor.yaml --seed 2 --demo week` (section 5.7; the week is written to a temporary file, read back and removed), paced 1 s per hour and each night in 2 s, under a minute: @@ -887,7 +949,7 @@ store the new state. On `refused` or `fallback`, use the line as given (habits o | `probbit persona replay PERSONA [--seed N] --script JSON\|FILE [--out TRACE] [--no-inertia] [--timing]` | `init`, then every turn of a script: one stance per line; stderr: the trace's sha256 and the final state digest | | `probbit persona explain PERSONA [--seed N] --script ... --turn K` | turn K in words: every contribution to every field, the joint odds, the habit-free twin, the line, the why | | `probbit persona diff PERSONA [--seed A] [--other PERSONA2] [--seed2 B] --script ...` | the distance between two individuals (section 5.4) and their most different turn | -| `probbit persona lint PERSONA [--props FILE \| --never RULE] [--seeds 0-99] [--threads N]` | contradicting habits and planned levels that are not their trait's most likely one (section 5.5); with rules, prove them and fuzz the unknown ones (section 5.6); with a drives block, `floors`: the habits that can exclude a floor goal and the floors whose lift can reach 50; exit 1 if a contradiction is unresolved or a rule breaks | +| `probbit persona lint PERSONA [--props FILE \| --never RULE] [--seeds 0-99] [--threads N]` | contradicting habits and planned levels that are not their trait's most likely one (section 5.5); with rules, prove them and fuzz the unknown ones (section 5.6); with a drives block, `floors`: the habits that can exclude a floor goal and the floors whose lift can reach 50; exit 1 if a contradiction is unresolved or a rule breaks, 3 if a rule is unproven (unknown, no counterexample found) | | `probbit persona fuzz PERSONA (--never RULE \| --props FILE) [--seeds 0-99] [--fuzz-seed N] [--scripts N] [--depth N] [--beam N] [--grid LIST] [--hours LIST] [--threads N] [--json]` | search event scripts for each individual's shortest counterexample to a character property, shrunk and replayable (section 5.6); exit 1 if one is found | | `probbit persona prove PERSONA (--never RULE \| --props FILE) [--seeds 0-99] [--threads N] [--json]` | per rule: held by construction, proved for every event sequence, or unknown with the cell that failed (section 5.6); exit 1 if one is unknown | | `probbit persona check PERSONA` / `describe PERSONA` | valid, its digest and sizes / its traits, moods, inputs, habits and agenda | diff --git a/install.sh b/install.sh index da1911a..14aa5bc 100755 --- a/install.sh +++ b/install.sh @@ -136,7 +136,13 @@ main() { *) cmd="$dir/probbit" say "" - case "${SHELL:-}" in */zsh) profile='~/.zshrc' ;; */bash) profile='~/.bashrc (or ~/.bash_profile for login shells)' ;; *) profile='your shell profile' ;; esac + # the file the user's shell reads: zsh ~/.zshrc; bash ~/.bash_profile on macOS (Terminal opens login shells), + # ~/.bashrc elsewhere; any other shell ~/.profile + case "${SHELL:-}" in + */zsh) profile='~/.zshrc' ;; + */bash) if [ "$(uname -s 2> /dev/null)" = Darwin ]; then profile='~/.bash_profile'; else profile='~/.bashrc'; fi ;; + *) profile='~/.profile' ;; + esac say "$dir is not on your PATH. Add it in $profile:" say " export PATH=\"$dir:\$PATH\"" ;; diff --git a/probbit-cli/src/fuzz.rs b/probbit-cli/src/fuzz.rs index 8c974c3..32e176e 100644 --- a/probbit-cli/src/fuzz.rs +++ b/probbit-cli/src/fuzz.rs @@ -19,6 +19,13 @@ type Event = Vec<(String, Json)>; /// The search settings (the flags of `probbit persona fuzz`) pub struct Search { pub seeds: Vec, pub fuzz_seed: u64, pub scripts: usize, pub depth: usize, pub beam: usize, pub grid: Vec, pub hours: Vec, pub threads: usize } +/// The search's grid: the given points in their order, then every threshold the rules and the habits read on a number input +/// (persona::thresholds) that is not one of them, so a condition that flips between two given points can still be put in force +pub fn grid(p: &Persona, props: &[Prop], given: &[f64]) -> Vec { + let mut g = given.to_vec(); + for t in persona::thresholds(p, props) { if !g.contains(&t) { g.push(t); } } + g +} /// One individual's shortest counterexample to one property, shrunk: the script, the breaking turn's stance document, what broke pub struct Found { pub script: Vec, pub doc: Json, pub broken: Vec } @@ -342,7 +349,7 @@ pub fn tool(args: &[(String, Json)], eng: SyncEngine) -> Result Option { h.push_str(&format!(" {f} {d}\n")); } if ["decide", "run", "evaluate", "stats"].contains(&cmd) { h.push_str("controls: flag > PROBBIT_* environment > probbit.json > default.\n"); } if cmd == "live" { h.push_str("exit: 0 done (live: every event answered; verify: every line replays; control: the line appended), 1 verify: a line differs\n (its number and what), 2 a bad persona, state, event (one {\"error\"} object on stdout per bad event; the run goes on), control\n line or flag, 4 refused: the strand's writer lock is held by another writer, or the individual is paused or retired (one\n {\"error\"} object per refused event; nothing is written).\n"); } - if cmd == "persona" { h.push_str("exit: 0 done (every turn status, refusals and fallbacks included, is an answer; fuzz: no counterexample found; prove: every rule\n held or proved), 1 lint found an unresolved contradiction or (with rules) a counterexample, fuzz found a counterexample or prove left a rule\n unknown, 2 bad persona / state / inputs / script / rule (one {\"error\"}\n object on stdout, code \"persona\") or bad flag (stderr).\n"); } + if cmd == "persona" { h.push_str("exit: 0 done (every turn status, refusals and fallbacks included, is an answer; fuzz: no counterexample found; prove: every rule\n held or proved), 1 lint found an unresolved contradiction or (with rules) a counterexample, fuzz found a counterexample or prove left a rule\n unknown, 2 bad persona / state / inputs / script / rule (one {\"error\"}\n object on stdout, code \"persona\") or bad flag (stderr), 3 lint (with rules): a rule the bound leaves unknown and the search\n cannot break (\"unproven\", not broken).\n"); } if ["decide", "run", "evaluate"].contains(&cmd) { h.push_str("exit: 0 answer (exact | diagnostics_passed | partial), 1 infeasible (a proof), 2 bad input (one {\"error\"} object\n on stdout) or bad flag (stderr), 3 refused / declined / non-finite result.\n"); } Some(h) } @@ -475,7 +475,7 @@ fn evaluate_cmd(args: &[String]) { let c = finish(evaluate::respond(&r, doc), code, &View::new(args, "evaluate"), &bars); if c != 0 { std::process::exit(c); } } -const PERSONA_HELP: &str = "The individuality layer (docs/persona.md): a persona file (YAML subset or JSON: traits with priors, moods with\n inertia, couplings, per-turn inputs, history, habits = hard rules) + an individual's state + this turn's inputs -> ONE probbit-ir\n program, run in process (`probbit run --op decide` at fixed work) -> the stance: a level per trait with exact odds, the habits\n in force and the ones that bound, a refusal when the engine cannot vouch, a why and a short stance line for any model's prompt.\n Documents are canonical JSON (keys sorted); a turn is a pure function of (persona, state, inputs, version).\n --pretty indents a single JSON document on stdout (init, turn, diff, lint, check, compile, describe, fuzz, prove);\n fuzz/prove --pretty implies --json. State files stay canonical. Replay stays JSONL; explain stays plain text.\nsubcommands:\n init PERSONA [--seed N] [--out STATE] a new individual (genes from the seed, resting stance); stdout or --out\n turn PERSONA --state STATE [--inputs JSON|FILE] [--out STATE2] [--timing] [--no-inertia]\n the stance on stdout; the new state replaces STATE (or goes to --out)\n replay PERSONA [--seed N] --script JSON|FILE [--out TRACE] [--no-inertia] [--timing]\n init, then every turn of the script: one stance per line (JSONL)\n explain PERSONA [--seed N] --script JSON|FILE --turn K turn K in words: every contribution, the odds, the twin\n diff PERSONA [--seed A] [--other PERSONA2] [--seed2 B] --script JSON|FILE distance between two individuals\n lint PERSONA [--never RULE | --props FILE] [--seeds 0-99] [--threads N]\n contradicting habits (every conditional habit and pair, every prev level);\n warnings: planned levels that are not their trait's most likely one;\n with rules: prove each one, fuzz the unknown ones; exit 1 on a counterexample\n fuzz PERSONA (--never RULE | --props FILE) [--seeds 0-99] [--fuzz-seed N] [--scripts N] [--depth N] [--beam N]\n [--grid LIST] [--hours LIST] [--threads N] [--json] [--fixture-src env:synthetic|human:synthetic]\n search event scripts for the shortest one whose stance breaks a rule\n (habit syntax: {when: {...}, then: {...}}); shrunk (§5.6). --fixture-src\n explicitly authorizes synthetic-source fixtures for strict replay; never\n use them to relabel production events. The persona must allow that source.\n prove PERSONA (--never RULE | --props FILE) [--seeds 0-99] [--threads N] [--json]\n per rule: held by construction (a habit implies it), proved for every\n event sequence (a sound bound), or unknown (the cell it fails; fuzz it) (§5.6)\n check PERSONA valid? digest and sizes\n compile PERSONA --state STATE [--inputs JSON|FILE] the turn's probbit-ir program (its sha256 = engine.program)\n describe PERSONA traits, moods, inputs, habits, agenda\n A script is a JSON list of per-turn input objects, or {\"turns\": [...]}. Resource controls as for run (PROBBIT_THREADS, ...)."; +const PERSONA_HELP: &str = "The individuality layer (docs/persona.md): a persona file (YAML subset or JSON: traits with priors, moods with\n inertia, couplings, per-turn inputs, history, habits = hard rules) + an individual's state + this turn's inputs -> ONE probbit-ir\n program, run in process (`probbit run --op decide` at fixed work) -> the stance: a level per trait with exact odds, the habits\n in force and the ones that bound, a refusal when the engine cannot vouch, a why and a short stance line for any model's prompt.\n Documents are canonical JSON (keys sorted); a turn is a pure function of (persona, state, inputs, version).\n --pretty indents a single JSON document on stdout (init, turn, diff, lint, check, compile, describe, fuzz, prove);\n fuzz/prove --pretty implies --json. State files stay canonical. Replay stays JSONL; explain stays plain text.\nsubcommands:\n init PERSONA [--seed N] [--out STATE] a new individual (genes from the seed, resting stance); stdout or --out\n turn PERSONA --state STATE [--inputs JSON|FILE] [--out STATE2] [--timing] [--no-inertia]\n the stance on stdout; the new state replaces STATE (or goes to --out)\n replay PERSONA [--seed N] --script JSON|FILE [--out TRACE] [--no-inertia] [--timing]\n init, then every turn of the script: one stance per line (JSONL)\n explain PERSONA [--seed N] --script JSON|FILE --turn K turn K in words: every contribution, the odds, the twin\n diff PERSONA [--seed A] [--other PERSONA2] [--seed2 B] --script JSON|FILE distance between two individuals\n lint PERSONA [--never RULE | --props FILE] [--seeds 0-99] [--threads N]\n contradicting habits (every conditional habit and pair, every prev level);\n warnings: planned levels that are not their trait's most likely one;\n with rules: prove each one, fuzz the unknown ones; exit 1 on a counterexample,\n exit 3 when a rule stays unknown and the search finds none (\"unproven\": not\n broken, not proved either)\n fuzz PERSONA (--never RULE | --props FILE) [--seeds 0-99] [--fuzz-seed N] [--scripts N] [--depth N] [--beam N]\n [--grid LIST] [--hours LIST] [--threads N] [--json] [--fixture-src env:synthetic|human:synthetic]\n search event scripts for the shortest one whose stance breaks a rule\n (habit syntax: {when: {...}, then: {...}}); shrunk (§5.6). --fixture-src\n explicitly authorizes synthetic-source fixtures for strict replay; never\n use them to relabel production events. The persona must allow that source.\n prove PERSONA (--never RULE | --props FILE) [--seeds 0-99] [--threads N] [--json]\n per rule: held by construction (a habit implies it), proved for every\n event sequence (a sound bound), or unknown (the cell it fails; fuzz it) (§5.6)\n check PERSONA valid? digest and sizes\n compile PERSONA --state STATE [--inputs JSON|FILE] the turn's probbit-ir program (its sha256 = engine.program)\n describe PERSONA traits, moods, inputs, habits, agenda\n A script is a JSON list of per-turn input objects, or {\"turns\": [...]}. Resource controls as for run (PROBBIT_THREADS, ...)."; /// The persona engine of this process: `persona::run_program` with the resource controls `probbit run` would use (flag > env > /// config > default; resolved once), and --priority low applied if asked for. They never change an answer at fixed work. pub(crate) fn persona_engine() -> impl Fn(&Json, &persona::Flags) -> Json { @@ -548,7 +548,7 @@ fn fuzz_cmd(args: &[String], path: &str, p: &persona::Persona, eng: fuzz::SyncEn if out.iter().any(|v| matches!(v, fuzz::Verdict::Unknown { .. })) { std::process::exit(1) } return; } let s = fuzz::Search { seeds: seeds_arg(args), fuzz_seed: arg(args, "--fuzz-seed", 0u64), scripts: bounded("--scripts", 60, 0, 1_000_000), depth: bounded("--depth", 8, 1, 64), - beam: bounded("--beam", 4, 0, 64), grid: list_arg(args, "--grid", &[0.0, 0.5, 1.0], 0.0, 1.0), hours: list_arg(args, "--hours", &[1.0, 12.0, 48.0], 0.0, 1e6), threads: bounded("--threads", cores, 1, 1024) }; + beam: bounded("--beam", 4, 0, 64), grid: fuzz::grid(p, &props, &list_arg(args, "--grid", &[0.0, 0.5, 1.0], 0.0, 1.0)), hours: list_arg(args, "--hours", &[1.0, 12.0, 48.0], 0.0, 1e6), threads: bounded("--threads", cores, 1, 1024) }; let t0 = std::time::Instant::now(); let (mut res, turns) = fuzz::fuzz(p, &props, &s, eng); let secs = t0.elapsed().as_secs_f64(); if let Some(i) = args.iter().position(|a| a == "--fixture-src") { fuzz::authorize_fixtures(p, &s.seeds, &mut res, &args[i + 1], eng).unwrap_or_else(|e| bad_input("persona fuzz: ", e)); @@ -694,7 +694,8 @@ fn persona_cmd(args: &[String]) { "fuzz" | "prove" => fuzz_cmd(args, path, &p, &engine), _ => { let found = persona::lint(&p, eng); let unresolved = found.iter().filter(|f| !f.get("resolution").and_then(Json::as_str).is_some_and(|r| r.starts_with("yield"))).count(); let seeds = seeds_arg(args); let warnings = persona::plan_warnings(&p, &seeds, eng); - let mut doc = vec![("persona", jstr(&p.name)), ("conflicts", Json::Arr(found)), ("unresolved", num(unresolved as f64)), ("warnings", Json::Arr(warnings))]; let mut broken = 0; + let mut doc = vec![("persona", jstr(&p.name)), ("conflicts", Json::Arr(found)), ("unresolved", num(unresolved as f64)), ("warnings", Json::Arr(warnings))]; + let (mut broken, mut unproven) = (0, vec![]); // drives (§2.9): habits that outrank a goal floor, floors whose lift can reach 50 if let Some(mut n) = persona::floor_notes(&p) { n.extend(persona::floor_reach(&p, &seeds)); doc.push(("floors", Json::Arr(n))); } // with rules (§5.6): prove each one, then fuzz the ones the bound leaves unknown @@ -702,17 +703,29 @@ fn persona_cmd(args: &[String]) { let props = rules_arg(args, &p); let threads = arg(args, "--threads", std::thread::available_parallelism().map_or(1, |n| n.get())).clamp(1, 1024); let verdicts = fuzz::prove(&p, &props, &seeds, threads, &engine); let unknown: Vec = props.iter().zip(&verdicts).filter(|(_, v)| matches!(v, fuzz::Verdict::Unknown { .. })).map(|(q, _)| q.clone()).collect(); - let s = fuzz::Search { seeds: seeds.clone(), fuzz_seed: 0, scripts: 60, depth: 8, beam: 4, grid: vec![0.0, 0.5, 1.0], hours: vec![1.0, 12.0, 48.0], threads }; + let s = fuzz::Search { seeds: seeds.clone(), fuzz_seed: 0, scripts: 60, depth: 8, beam: 4, grid: fuzz::grid(&p, &unknown, &[0.0, 0.5, 1.0]), hours: vec![1.0, 12.0, 48.0], threads }; let (res, turns) = if unknown.is_empty() { (vec![], 0) } else { fuzz::fuzz(&p, &unknown, &s, &engine) }; let fd = fuzz::doc(&p, path, &unknown, &s, &res, turns); let fp = fd.get("properties").and_then(Json::as_arr).unwrap_or(&[]).to_vec(); broken = fp.iter().filter(|x| x.get("verdict").and_then(Json::as_str) == Some("counterexample")).count(); let mut entries = fuzz::prove_doc(&p, &props, &seeds, &verdicts).get("properties").and_then(Json::as_arr).unwrap_or(&[]).to_vec(); for e in entries.iter_mut() { let id = e.get("id").cloned(); if let (Json::Obj(v), Some(f)) = (e, fp.iter().find(|x| x.get("id").cloned() == id)) { v.push(("fuzz".into(), f.clone())); } } - doc.push(("props", Json::Arr(entries))); doc.push(("broken", num(broken as f64))); } - doc.push(("ok", Json::Bool(unresolved == 0 && broken == 0))); + // unproven: the bound left it unknown and the search found no counterexample. Not broken, and not proved either. + for (q, v) in props.iter().zip(&verdicts) { + let fuzz::Verdict::Unknown { failed, .. } = v else { continue }; + if fp.iter().any(|x| x.get("id").and_then(Json::as_str) == Some(q.id.as_str()) && x.get("verdict").and_then(Json::as_str) == Some("counterexample")) { continue; } + let searched = obj(vec![("individuals", num(s.seeds.len() as f64)), ("scripts", num(s.scripts as f64)), ("depth", num(s.depth as f64)), ("beam", num(s.beam as f64)), + ("grid", Json::Arr(s.grid.iter().map(|x| num(*x)).collect())), ("hours", Json::Arr(s.hours.iter().map(|x| num(*x)).collect()))]); + unproven.push(obj(vec![("id", jstr(&q.id)), ("rule", q.rule.clone()), ("seeds_unknown", Json::Arr(failed.iter().map(|f| num(f.0 as f64)).collect())), ("searched", searched)])); } + doc.push(("props", Json::Arr(entries))); doc.push(("broken", num(broken as f64))); doc.push(("unproven", Json::Arr(unproven.clone()))); } + doc.push(("ok", Json::Bool(unresolved == 0 && broken == 0 && unproven.is_empty()))); emit(&persona_text(&obj(doc), pretty)); - if unresolved > 0 || broken > 0 { std::process::exit(1) } } + if !unproven.is_empty() { + let ids: Vec<&str> = unproven.iter().filter_map(|u| u.get("id").and_then(Json::as_str)).collect(); + err_line(&format!("lint: {} rule{} unproven, not broken: {} (the bound left {} unknown and the search found no counterexample; `probbit persona prove` names the cell)", + ids.len(), if ids.len() == 1 { "" } else { "s" }, ids.join(", "), if ids.len() == 1 { "it" } else { "them" })); } + if unresolved > 0 || broken > 0 { std::process::exit(1) } + if !unproven.is_empty() { std::process::exit(3) } } } } diff --git a/probbit-cli/src/mcp.rs b/probbit-cli/src/mcp.rs index b6aecac..5b8055d 100644 --- a/probbit-cli/src/mcp.rs +++ b/probbit-cli/src/mcp.rs @@ -231,7 +231,7 @@ fn tools() -> Json { "seeds": {{"type": ["string", "array"], "description": "the individuals: \"0-99\" (default), \"1,4,9\" or a list of integers"}}, "fixture_src": {{"enum": ["env:synthetic", "human:synthetic"], "description": "Explicitly authorize an allowed source for generated synthetic fixtures only. Never authenticates or relabels production events. Without it source-required counterexamples have null replay/explain commands."}}, "fuzz_seed": {{"type": "integer", "minimum": 0}}, "scripts": {{"type": "integer", "minimum": 0, "maximum": 1000000}}, "depth": {{"type": "integer", "minimum": 1, "maximum": 64}}, - "beam": {{"type": "integer", "minimum": 0, "maximum": 64}}, "grid": {{"type": "array", "items": {{"type": "number", "minimum": 0, "maximum": 1}}}}, + "beam": {{"type": "integer", "minimum": 0, "maximum": 64}}, "grid": {{"type": "array", "items": {{"type": "number", "minimum": 0, "maximum": 1}}, "description": "fractions of each number input's max (default [0, 0.5, 1]); the thresholds the rule and the habits read are added"}}, "hours": {{"type": "array", "items": {{"type": "number", "minimum": 0}}}}, "threads": {{"type": "integer", "minimum": 1, "maximum": 1024}}}}}}"#))), tool("probbit_live_event", "Feed one event to a resident individual", "probbit live, one event: a persona (inline document or a file path), the individual's state (or a seed for a new one) and one event (inputs keyed by input id, with elapsed_hours since the previous event from the host's clock; quantised to 1e-6 h) -> {stance, state}: moods decay by their half-lives over the elapsed hours, and with a learning block the reward / correction flags move the learned deltas within their caps (habits never move). With strand_path, the event's line is appended to that strand file on the server's disk (a new file gets the header; an existing one is continued, and state must be the state after its last line) and the result has strand {path, events, head}; the strand is locked while its line is appended (a second writer is refused, code locked), every 1,000th event adds a checkpoint line, and a strand paused or retired by its owner (probbit live control, not a tool) refuses every event (code paused or retired). With reward_from in the persona, an event's src says who produced it (human:ID, env:SENSOR, self, clock) and a reward from the individual itself, from no source or from an undeclared one is refused. Pass the returned state back with the next event.", p(&format!(r#"{{"type": "object", "additionalProperties": false, "properties": {{{PERSONA_ARG}, diff --git a/probbit-cli/src/monitor.html b/probbit-cli/src/monitor.html index 1a4b25c..20a4427 100644 --- a/probbit-cli/src/monitor.html +++ b/probbit-cli/src/monitor.html @@ -3,319 +3,247 @@ - -probbit monitor - - + +probbit · Activity Monitor + +
-
- -probbit monitor - - - -connecting -
- -

Reading the bars: width = reported probability; colour = chosen level, not necessarily the most likely one. Mood trails show mean level, low to high, over the last 50 events.

-

waiting for the strand

-
- - -
- - - - - -
-
+

Activity Monitor

probbit · declared agent state
Waiting for individual
+
+
CONNECTINGWaiting for the local monitor.
+

Waiting for a strand.

+ + + + + +
Read-only · one individual per endpointEngine state only · no CPU / GPU / memory telemetry
diff --git a/probbit-cli/src/monitor.rs b/probbit-cli/src/monitor.rs index 0ace64e..f308d64 100644 --- a/probbit-cli/src/monitor.rs +++ b/probbit-cli/src/monitor.rs @@ -49,14 +49,14 @@ struct Var { id: String, levels: Vec, say: Vec } struct Input { id: String, kind: String, max: f64 } /// What the layout reads from the strand's header struct Meta { name: String, version: String, seed: u64, individual: String, engine: String, traits: Vec, moods: Vec, inputs: Vec, - history: Vec<(String, f64)>, learning: Option<(Vec, f64)>, caps: (f64, f64), goals: Vec } + history: Vec<(String, f64)>, learning: Option<(Vec, f64)>, caps: (f64, f64, f64), goals: Vec } impl Meta { fn of(r: &Replay) -> Meta { let d = persona::describe(&r.live.p); let obj = |k: &str| d.get(k).and_then(Json::as_obj).map(|o| o.to_vec()).unwrap_or_default(); let decl = |k: &str| r.doc.get(k).and_then(Json::as_arr).map(|a| a.to_vec()).unwrap_or_default(); let max = |id: &str| decl("inputs").iter().find(|i| i.get("id").and_then(Json::as_str) == Some(id)).and_then(|i| i.get("max")).and_then(Json::as_f64).unwrap_or(1.0); - let cap = |b: &str, d: f64| r.doc.get("drives").and_then(|x| x.get(b)).and_then(|x| x.get("cap")).and_then(Json::as_f64).unwrap_or(d); + let cap = |b: &str, k: &str, d: f64| r.doc.get("drives").and_then(|x| x.get(b)).and_then(|x| x.get(k)).and_then(Json::as_f64).unwrap_or(d); Meta { name: r.live.p.name.clone(), version: r.live.p.version.clone(), seed: r.live.st.seed, individual: r.header.get("state").and_then(|s| s.get("digest")).and_then(Json::as_str).unwrap_or("").to_string(), engine: r.header.get("engine").and_then(Json::as_str).unwrap_or("").to_string(), @@ -65,7 +65,7 @@ impl Meta { inputs: obj("inputs").into_iter().map(|(id, x)| Input { kind: x.get("kind").and_then(Json::as_str).unwrap_or("").to_string(), max: max(&id), id }).collect(), history: decl("history").iter().filter_map(|h| Some((h.get("id")?.as_str()?.to_string(), h.get("cap").and_then(Json::as_f64).unwrap_or(5.0)))).collect(), learning: d.get("learning").map(|l| (strs(l.get("traits")), l.get("total_cap").and_then(Json::as_f64).unwrap_or(1.0))), - caps: (cap("wanting", 3.0), cap("afterglow", 2.0)), + caps: (cap("wanting", "cap", 3.0), cap("afterglow", "cap", 2.0), cap("expectation", "pe_cap", 1.0)), // a drives block's goals, in the order of the levels of `pursue` (describe hides that trait; its learned row needs them) goals: strs(d.get("drives").and_then(|x| x.get("goals"))) } } @@ -341,7 +341,7 @@ fn meta_json(w: Option<&Watch>, path: &str) -> Json { ("inputs", Json::Arr(m.inputs.iter().map(|x| o(vec![("id", s(&x.id)), ("kind", s(&x.kind)), ("max", Json::Num(x.max))])).collect())), ("history", Json::Arr(m.history.iter().map(|(id, cap)| o(vec![("id", s(id)), ("cap", Json::Num(*cap))])).collect())), ("learning", m.learning.as_ref().map_or(Json::Null, |(t, cap)| o(vec![("traits", strs(t)), ("total_cap", Json::Num(*cap))]))), - ("caps", o(vec![("wanting", Json::Num(m.caps.0)), ("afterglow", Json::Num(m.caps.1))])), ("goals", strs(&m.goals))]); + ("caps", o(vec![("wanting", Json::Num(m.caps.0)), ("afterglow", Json::Num(m.caps.1)), ("prediction_error", Json::Num(m.caps.2))])), ("goals", strs(&m.goals))]); } o(kv) } @@ -936,6 +936,23 @@ mod tests { assert!(positive && negative && glowing); } + /// Monitor scales come from this persona, including a non-default prediction-error cap. + #[test] + fn drive_metadata_preserves_declared_scales() { + let doc = json::parse(r#"{"probbit_persona":1,"identity":{"name":"Scale fixture","version":"1"}, + "traits":[{"id":"approach","levels":["consider","act"]}], + "drives":{"goals":[{"id":"repair"},{"id":"explore"}],"wanting":{"cap":7}, + "afterglow":{"cap":4},"expectation":{"pe_cap":2.5}}}"#).unwrap(); + let p = persona::build(&doc).unwrap(); + let (_, header) = Live::start(p.clone(), &doc, persona::init(&p, Some(2), true, &run), Clock::Fixed, "probbit test"); + let w = Watch::open(&header).unwrap(); + let meta = meta_json(Some(&w), "fixture.strand"); + let caps = meta.get("caps").unwrap(); + assert_eq!(caps.get("wanting").and_then(Json::as_f64), Some(7.0)); + assert_eq!(caps.get("afterglow").and_then(Json::as_f64), Some(4.0)); + assert_eq!(caps.get("prediction_error").and_then(Json::as_f64), Some(2.5)); + } + /// The page's layout lists the persona's variables in its order; a frame carries the event's document as replayed (drives /// fields included, as they are), the moods' recent positions, the line that diverges and the note #[test] diff --git a/probbit-cli/src/persona.rs b/probbit-cli/src/persona.rs index 6fdbcfd..35957d4 100644 --- a/probbit-cli/src/persona.rs +++ b/probbit-cli/src/persona.rs @@ -1806,6 +1806,17 @@ pub fn alphabet(p: &Persona, grid: &[f64], hours: &[f64]) -> Vec<(String, &'stat let h = pos_hours(); if !h.is_empty() { out.push((format!("goal.{}.deadline_hours", g.id), "goal", h)); } } } out } +/// The thresholds the rules and the habits read on number inputs, on the grid's scale (a fraction of the input's max), each once, +/// in ascending order: the values at which a condition flips, so the search can put an input exactly there +pub fn thresholds(p: &Persona, props: &[Prop]) -> Vec { + let mut out: Vec = vec![]; + for w in p.habits.iter().map(|h| &h.when).chain(props.iter().map(|q| &q.habit.when)) { for (k, c) in w { + // the host's number inputs (the drives' own inputs are not on the search's alphabet) + let Some(x) = p.input(k).filter(|x| x.kind == Kind::Number && !p.drives.as_ref().is_some_and(|d| d.synthetic.contains(&x.id))) else { continue }; + let ts: Vec = match c { Json::Num(t) => vec![*t], _ => ["at_least", "at_most"].iter().filter_map(|b| c.get(b).and_then(Json::as_f64)).collect() }; + for t in ts { if t >= 0.0 && t <= x.max { let g = t / x.max; if !out.contains(&g) { out.push(g); } } } } } + out.sort_by(|a, b| a.partial_cmp(b).unwrap()); out +} /// An event of the alphabet as turn inputs: `goal..` entries nested into `goals` (after the host's inputs) pub fn event_json(e: &[(String, Json)]) -> Json { let mut v: Vec<(String, Json)> = vec![]; let mut goals: Vec<(String, Json)> = vec![]; diff --git a/probbit-cli/tests/monitor.rs b/probbit-cli/tests/monitor.rs index c542cd6..c0cad02 100644 --- a/probbit-cli/tests/monitor.rs +++ b/probbit-cli/tests/monitor.rs @@ -210,7 +210,8 @@ fn serve_answers_the_page_the_events_and_the_documents() { let host = format!("127.0.0.1:{port}"); let (st, h, page) = get(port, "/", &host); assert_eq!(st, "HTTP/1.1 200 OK"); assert!(h.contains("Content-Type: text/html") && h.contains("connect-src 'self'"), "{h}"); - assert!(page.contains("probbit monitor") && page.contains("new EventSource('/events')") && page.len() < 40_000, "{} bytes", page.len()); + // The self-contained activity monitor keeps channel selection and document history offline. + assert!(page.contains("probbit · Activity Monitor") && page.contains("new EventSource('/events')") && page.len() < 64_000, "{} bytes", page.len()); // the page fetches nothing from anywhere else: no URL in it but the SVG namespace assert!(page.match_indices("http").all(|(i, _)| page[i..].starts_with("http://www.w3.org/2000/svg")), "a URL in the page"); for k in 1..=3 { let (st, _, d) = get(port, &format!("/doc/{k}"), &host); assert_eq!((st.as_str(), d.as_str()), ("HTTP/1.1 200 OK", format!("{}\n", docs[k - 1]).as_str()), "doc {k}"); } @@ -249,7 +250,7 @@ fn open_serves_the_page_the_demo_too() { let mut url = String::new(); std::io::BufRead::read_line(&mut std::io::BufReader::new(child.0.stdout.take().unwrap()), &mut url).unwrap(); let port: u16 = url.trim_end().strip_prefix("http://127.0.0.1:").and_then(|p| p.strip_suffix('/')).unwrap_or_else(|| panic!("the URL line: {url:?}")).parse().unwrap(); let host = format!("127.0.0.1:{port}"); - let (st, _, page) = get(port, "/", &host); assert_eq!(st, "HTTP/1.1 200 OK"); assert!(page.contains("probbit monitor")); + let (st, _, page) = get(port, "/", &host); assert_eq!(st, "HTTP/1.1 200 OK"); assert!(page.contains("probbit · Activity Monitor")); let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); s.set_read_timeout(Some(Duration::from_secs(20))).unwrap(); write!(s, "GET /events HTTP/1.1\r\nHost: {host}\r\n\r\n").unwrap(); let mut buf = String::new(); let evs = stream(&mut s, &mut buf, 2); diff --git a/probbit-cli/tests/monitor_browser.mjs b/probbit-cli/tests/monitor_browser.mjs index def0d21..46e7980 100644 --- a/probbit-cli/tests/monitor_browser.mjs +++ b/probbit-cli/tests/monitor_browser.mjs @@ -55,8 +55,9 @@ function command(args, input) { } async function server(args, tag) { - const child = spawn(bin, ['monitor', ...args, '--serve'], { - cwd: repo, env: {...process.env, PROBBIT_THREADS: '2'}, + const monitorArgs = args.map(arg => path.isAbsolute(arg) && path.dirname(arg) === out ? path.basename(arg) : arg); + const child = spawn(bin, ['monitor', ...monitorArgs, '--serve'], { + cwd: out, env: {...process.env, PROBBIT_THREADS: '2'}, }); children.push(child); let stdout = '', stderr = '', error; @@ -69,7 +70,7 @@ async function server(args, tag) { if (child.exitCode !== null) throw Error('Monitor exited: ' + stderr); return stdout.match(/http:\/\/127\.0\.0\.1:\d+\//)?.[0]; }, tag + ' server'); - return {url}; + return {url, child}; } async function json(url) { @@ -144,6 +145,32 @@ try { const send = await cdp(tabs.find(tab => tab.type === 'page').webSocketDebuggerUrl); await send('Page.enable'); await send('Runtime.enable'); + // Keep a test-only handle to the real SSE connection for the metadata-without-frame boundary. + // The product page has no test hooks and its normal EventSource behavior is unchanged. + await send('Page.addScriptToEvaluateOnNewDocument', {source:`{ + const NativeEventSource = window.EventSource; + window.EventSource = class extends NativeEventSource { + constructor(...args) { super(...args); window.monitorTestStream = this; } + }; + }`}); + await send('Page.addScriptToEvaluateOnNewDocument', {source:`{ + const nativeFetch = window.fetch; + window.monitorFetch = {active:0, maximum:0, docs:[]}; + window.fetch = async (...args) => { + if (!String(args[0]).startsWith('/doc/')) return nativeFetch(...args); + const stats = window.monitorFetch; stats.active++; stats.maximum = Math.max(stats.maximum,stats.active); stats.docs.push(args[0]); + try { + if (new URL(location.href).searchParams.has('gap') && args[0] === '/doc/2') return new Response('',{status:404}); + return await nativeFetch(...args); + } finally { stats.active--; } + }; + document.addEventListener('DOMContentLoaded', () => { + const label = document.createElement('span'); label.className='synthetic'; label.textContent='SYNTHETIC TEST'; + document.querySelector('.titlebar').insertBefore(label,document.querySelector('.identity')); + const sourceLabel=document.getElementById('synthetic'),syncLabel=()=>{label.hidden=!sourceLabel.hidden;}; + new MutationObserver(syncLabel).observe(sourceLabel,{attributes:true,attributeFilter:['hidden']});syncLabel(); + }); + }`}); await send('Emulation.setEmulatedMedia', {features: [{name: 'prefers-reduced-motion', value: 'reduce'}]}); const value = async expression => { const result = await send('Runtime.evaluate', {expression, returnByValue: true, awaitPromise: true}); @@ -157,6 +184,7 @@ try { }; const state = () => value(`({ event: document.getElementById('ev')?.textContent, + identity: document.getElementById('identity')?.textContent, note: document.getElementById('note')?.textContent, badge: document.getElementById('badge')?.textContent, line: document.getElementById('line')?.textContent, @@ -164,104 +192,199 @@ try { saidHidden: document.getElementById('said')?.hidden, emptyHidden: document.getElementById('empty')?.hidden, rawHidden: document.getElementById('raw')?.hidden, - drives: document.getElementById('drives')?.hidden, + stream: document.getElementById('stream-title')?.textContent, + mode: document.getElementById('stream')?.dataset.mode, + streamDetail: document.getElementById('stream-detail')?.textContent, + workspaceHidden: document.getElementById('workspace')?.hidden, + view: document.getElementById('view-mode')?.textContent, + historyCount: document.getElementById('history-count')?.textContent, + historyNote: document.getElementById('history-note')?.textContent, + historicalNoteHidden: document.getElementById('historical-note')?.hidden, + learnedHidden: document.getElementById('learned')?.hidden, + changes: document.getElementById('changes')?.textContent, + follow: document.getElementById('follow')?.textContent, + selected: document.getElementById('channel-title')?.textContent, + channelValue: document.getElementById('channel-value')?.textContent, + channelOdds: document.getElementById('channel-odds')?.textContent, scrollWidth: document.documentElement.scrollWidth, width: innerWidth, height: document.documentElement.scrollHeight })`); async function shot(name, width, height) { - await viewport(width, height); - await sleep(150); + await viewport(width, height); await sleep(120); const frame = await state(); assert.ok(frame.scrollWidth <= width, 'Horizontal overflow: ' + JSON.stringify(frame)); + assert.ok(!(await value(`document.getElementById('foot').textContent`)).includes(repo) && !(await value(`document.getElementById('foot').textContent`)).includes(out),'Screenshots must not embed private fixture paths'); const screenshot = await send('Page.captureScreenshot', { - format: 'png', captureBeyondViewport: true, - clip: {x: 0, y: 0, width, height: Math.max(height, frame.height), scale: 1}, + format:'png', captureBeyondViewport:true, + clip:{x:0,y:0,width,height:Math.max(height,frame.height),scale:1}, }); - write(name + '.png', Buffer.from(screenshot.data, 'base64')); - report.screenshots.push({name, ...frame}); + write(name+'.png',Buffer.from(screenshot.data,'base64')); + report.screenshots.push({name,...frame}); } - const event = number => until(async () => (await state()).event === 'event ' + number, 'event ' + number); - const learnError = () => value(`[...document.querySelectorAll('#drives .row')] - .find(row => row.querySelector('.id')?.textContent === 'learn')?.querySelector('.petxt')?.textContent`); + const event = number => until(async () => (await state()).event === 'event '+number, 'event '+number); + const click = selector => value(`document.querySelector(${JSON.stringify(selector)}).click()`); + const rows = () => value(`[...document.querySelectorAll('#channel-rows tr')].map(r=>({key:r.dataset.channel,name:r.querySelector('button').textContent,selected:r.getAttribute('aria-selected'),current:r.querySelector('.current-col').textContent,odds:r.querySelector('.odds-col').textContent}))`); + const selected = key => click('#channel-rows button[data-channel="'+key+'"]'); - const demo = await server(['--demo', 'drives'], 'drives'); - await viewport(1440, 1000); - await navigate(demo.url); - await event(3); + const demo = await server(['--demo','drives'],'drives'); + await viewport(1280,820); await navigate(demo.url); await event(3); report.demo.event3 = await state(); - assert.equal(report.demo.event3.drives, false); - await shot('monitor-drives-desktop', 1440, 1000); - await shot('monitor-drives-narrow', 390, 844); - await event(4); - report.demo.negativeError = await learnError(); - assert.ok(report.demo.negativeError.includes('below expectation')); - await shot('monitor-drives-negative', 1440, 1000); + assert.ok(report.demo.event3.stream.includes('SYNTHETIC')); + assert.equal((await rows()).length,13); + await value(`document.getElementById('theme').value='light';document.getElementById('theme').dispatchEvent(new Event('change'))`); + await selected('drive:want:learn'); + assert.equal((await state()).selected,'learn · wanting'); + assert.equal((await state()).channelOdds,'engine units'); + const doc3=await json(demo.url+'doc/3'); + assert.equal((await state()).channelValue,doc3.drives.want.learn.toFixed(2)); + assert.ok(await value(`document.getElementById('channel-facts').textContent.includes('0 to 3')`)); + assert.equal(await value(`getComputedStyle(document.querySelector('.spark')).transitionDuration`),'0s'); + await shot('activity-desktop-light',1280,820); + assert.ok((await state()).height<=820,'Useful desktop should fit one viewport'); + await value(`document.getElementById('theme').value='dark'; document.getElementById('theme').dispatchEvent(new Event('change'))`); + await shot('activity-desktop-dark',1280,820); + await event(4); await selected('drive:surprise:learn'); + report.demo.negativeError=await state(); + assert.ok(Number((await state()).channelValue)<0); + assert.ok(await value(`document.getElementById('channel-facts').textContent.includes('Below expectation')`)); await event(5); - report.demo.clearedError = await learnError(); - assert.equal(report.demo.clearedError, 'prediction error 0.00'); - await until(async () => (await state()).note?.includes('complete; restarting'), 'demo completion cue'); - report.demo.completed = await state(); - await shot('monitor-drives-completed', 1440, 1000); - await until(async () => (await state()).note?.includes('loop 2'), 'demo loop 2'); - report.demo.restarted = await state(); - assert.equal(report.demo.restarted.event, 'event 0'); - assert.equal(report.demo.restarted.line, ''); - assert.equal(report.demo.restarted.why, ''); - assert.equal(report.demo.restarted.saidHidden, true); - assert.equal(report.demo.restarted.emptyHidden, false); - assert.equal(report.demo.restarted.rawHidden, true); - assert.equal(report.demo.restarted.drives, true); + assert.equal((await state()).selected,'learn · prediction error','Selection survives new event'); + assert.equal((await state()).channelValue,'0.00'); + await until(async()=>(await state()).note.includes('complete; restarting'),'demo completion cue'); + await until(async()=>(await state()).note.includes('loop 2'),'demo loop 2'); + report.demo.restarted=await state(); + assert.ok(['event 0','event 1'].includes(report.demo.restarted.event)); + if(report.demo.restarted.event==='event 0')assert.equal(report.demo.restarted.workspaceHidden,true); + + // Author a different persona: no assumptions about the tutor, a preset character, or fixed mood/trait names. + const persona=path.join(out,'instrument-fixture.json'); + writeFileSync(persona,JSON.stringify({probbit_persona:1,identity:{name:'Field unit synthetic',version:'1'}, + traits:[{id:'response_depth',levels:['brief','detailed'],say:['be brief','provide detail']},{id:'release_gate',levels:['low','high'],prior:[0.2,0.8],vouch:0.99,hold:'low'}], + moods:[{id:'reserve',levels:['low','steady','full'],inertia:0.5,half_life_hours:4}], + inputs:[{id:'boost',kind:'flag',effects:{reserve:2,response_depth:1}}]})); + const full=path.join(out,'follow-full.strand'),growing=path.join(out,'follow-growing.strand'); + const outputs=command(['live',persona,'--seed','2','--clock','fixed','--strand',full], + '{"boost":true,"elapsed_hours":1}\n{}\n{"boost":true,"elapsed_hours":2}\n').trim().split('\n').map(JSON.parse); + const lines=readFileSync(full,'utf8').match(/.*\n/g);writeFileSync(growing,lines.slice(0,2).join('')); + const follow=await server([growing],'follow'); + await viewport(1280,820);await navigate(follow.url);await event(1); + assert.equal((await state()).identity,'Field unit synthetic'); + assert.equal(await value(`document.querySelectorAll('#identity synthetic').length`),0,'Identity uses textContent'); + assert.ok((await state()).changes.includes('First recorded event')); + assert.ok((await rows()).find(r=>r.key==='trait:release_gate').current.includes('held')); + await selected('trait:release_gate');assert.ok(await value(`document.getElementById('channel-sub').textContent.includes('Held at declared level')`));assert.ok(await value(`document.getElementById('channel-facts').textContent.includes('Before hold')`)); + await click('[data-filter="mood"]');assert.deepEqual((await rows()).map(r=>r.name),['reserve']); + assert.equal((await state()).selected,'reserve');assert.ok(await value(`document.getElementById('channel-sub').textContent.includes('No previous event')`)); + await click('[data-filter="all"]'); + await click('[data-sort="name"] button'); + assert.deepEqual((await rows()).map(r=>r.name),['release_gate','reserve','response_depth']); + await click('[data-sort="name"] button'); + assert.deepEqual((await rows()).map(r=>r.name),['response_depth','reserve','release_gate']); + await value(`document.getElementById('search').value='res';document.getElementById('search').dispatchEvent(new Event('input'))`); + assert.deepEqual((await rows()).map(r=>r.name),['response_depth','reserve']); + await value(`document.getElementById('search').value='no-such-channel';document.getElementById('search').dispatchEvent(new Event('input'))`); + assert.equal((await rows()).length,0);assert.equal(await value(`document.getElementById('inspector').hidden`),true); + await value(`document.getElementById('search').value='';document.getElementById('search').dispatchEvent(new Event('input'));document.querySelector('#channel-rows button').focus()`); + await send('Input.dispatchKeyEvent',{type:'keyDown',key:'ArrowDown',code:'ArrowDown',windowsVirtualKeyCode:40}); + await send('Input.dispatchKeyEvent',{type:'keyUp',key:'ArrowDown',code:'ArrowDown',windowsVirtualKeyCode:40}); + assert.equal((await state()).selected,'reserve'); + const firstOdds=outputs[0].mood.reserve.odds[outputs[0].mood.reserve.level]; + assert.equal((await state()).channelOdds,(firstOdds*100).toFixed(1)+'%'); + report.table={sort:true,filter:true,search:true,selection:true,keyboard:true,genericPersona:true,confidenceHold:true}; + appendFileSync(growing,lines[2].slice(0,-1));await sleep(350);assert.equal((await state()).event,'event 1'); + const appended=Date.now();appendFileSync(growing,'\n');await event(2);report.follow.appendLatencyMs=Date.now()-appended; + assert.ok(report.follow.appendLatencyMs<1000);assert.equal((await state()).selected,'reserve'); + assert.equal((await state()).line,outputs[1].line);assert.deepEqual(await json(follow.url+'doc/2'),outputs[1]); + await click('#hold');appendFileSync(growing,lines[3]); + await until(async()=>(await state()).streamDetail.includes('latest received event 3'),'held view receives event'); + assert.equal((await state()).event,'event 2');assert.equal((await state()).line,outputs[1].line);assert.ok((await state()).follow.includes('+1')); + await click('#previous');assert.equal((await state()).line,outputs[0].line); + await value(`document.getElementById('scrubber').focus()`); + await send('Input.dispatchKeyEvent',{type:'keyDown',key:'ArrowRight',code:'ArrowRight',windowsVirtualKeyCode:39}); + await send('Input.dispatchKeyEvent',{type:'keyUp',key:'ArrowRight',code:'ArrowRight',windowsVirtualKeyCode:39}); + assert.equal((await state()).event,'event 2');await click('#follow');await event(3); + report.follow.heldView=true; + // Advance only the browser's receiver-age clock. Channel state must not decay on a timer. + const stable=await value(`document.getElementById('detail-chart').innerHTML`); + await value(`window.realNow=Date.now;Date.now=()=>window.realNow()+61000`); + await until(async()=>(await state()).mode==='stale','explicit stale event threshold'); + assert.equal(await value(`document.getElementById('detail-chart').innerHTML`),stable,'Quiet state is not fabricated'); + assert.ok((await state()).streamDetail.includes('does not prove the agent is idle')); + await value(`Date.now=window.realNow`);report.follow.stale=true; + + // Reload only returns stance documents for history, not original raw clock/learned frames. + await navigate(follow.url+'?backfill');await event(3); + await until(async()=>(await state()).historyCount==='3/3 recent documents','history backfill'); + await click('[data-event="1"]');report.follow.documentOnly=await state(); + assert.equal((await state()).line,outputs[0].line);assert.equal((await state()).historicalNoteHidden,false);assert.equal((await state()).learnedHidden,true); + assert.ok(await value(`document.getElementById('input-facts').textContent.includes('Not captured in this frame')`)); + assert.equal(await value(`document.getElementById('raw').getAttribute('href')`),'/doc/1'); + await shot('activity-narrow-history',390,844); + await click('#follow');command(['live','control',growing,'pause','--by','human:fixture','--reason','synthetic browser check']); + await until(async()=>(await state()).mode==='paused','strand pause');assert.ok((await state()).stream.includes('STATE FROZEN')); + command(['live','control',growing,'retire','--by','human:fixture','--reason','synthetic browser check']); + await until(async()=>(await state()).mode==='retired','strand retired');assert.ok((await state()).streamDetail.includes('final')); + report.follow.retired=await state(); + follow.child.kill('SIGTERM');await until(async()=>(await state()).mode==='disconnected','disconnected status');assert.equal((await state()).line,outputs[2].line); + await shot('activity-disconnected',1280,820); + await until(()=>follow.child.exitCode!==null||follow.child.signalCode!==null,'server shutdown'); + await server([growing,'--port',new URL(follow.url).port],'reconnected'); + await until(async()=>(await state()).mode==='retired','SSE reconnect');assert.equal((await state()).line,outputs[2].line); - const week = await server(['--demo'], 'week'); - await navigate(week.url); - await event(3); - await shot('monitor-week-desktop', 1440, 1000); - await shot('monitor-week-narrow', 390, 844); + // A missing event remains a gap, disabled in history and unjoined in the selected channel plot. + await navigate(follow.url+'?gap');await event(3); + await until(async()=>(await state()).historyCount==='2/3 recent documents','history gap'); + assert.equal(await value(`document.querySelector('[data-event="2"]').disabled`),true); + await selected('mood:reserve'); + const paths=await value(`[...document.querySelectorAll('#detail-chart path')].map(p=>p.getAttribute('d'))`); + assert.ok(paths.length===3&&paths.every(d=>!d.includes('L')&&d.split('M').length===3),'Missing event is not interpolated: '+paths); + report.historyGap=true; - const full = path.join(out, 'follow-full-' + Date.now() + '.strand'); - const growing = path.join(out, 'follow-growing-' + Date.now() + '.strand'); - const outputs = command(['live', 'examples/persona/tutor.yaml', '--seed', '2', '--clock', 'fixed', '--strand', full], - '{"praise":true,"elapsed_hours":1}\n{"loss":true,"elapsed_hours":1}\n{}\n').trim().split('\n').map(JSON.parse); - const lines = readFileSync(full, 'utf8').match(/.*\n/g); - writeFileSync(growing, lines.slice(0, 2).join('')); - const follow = await server([growing], 'follow'); - await viewport(1440, 1000); - await navigate(follow.url); - await event(1); - report.follow.before = await state(); - await shot('monitor-follow-before', 1440, 1000); - appendFileSync(growing, lines[2].slice(0, -1)); - await sleep(350); - assert.equal((await state()).event, 'event 1'); - report.follow.incompleteHeld = true; - const appended = Date.now(); - appendFileSync(growing, '\n'); - await until(async () => (await state()).event === 'event 2', 'appended event', 2000); - report.follow.appendLatencyMs = Date.now() - appended; - assert.ok(report.follow.appendLatencyMs < 1000, 'Follow update exceeded one second'); - report.follow.after = await state(); - assert.equal(report.follow.after.line, outputs[1].line); - assert.deepEqual(await json(follow.url + 'doc/2'), outputs[1]); - await shot('monitor-follow-after', 1440, 1000); - appendFileSync(growing, lines[3]); - await event(3); - assert.ok((await state()).why.includes('prior state still applies')); - assert.equal((await json(follow.url + 'doc/3')).why, 'baseline (no live evidence)'); - report.follow.rawWhyUnchanged = true; - command(['live', 'control', growing, 'pause', '--by', 'human:demo', '--reason', 'synthetic browser check']); - await until(async () => (await state()).badge?.includes('paused'), 'paused status'); - report.follow.paused = await state(); - await shot('monitor-paused-narrow', 390, 844); + const checkpoint=path.join(out,'checkpoint.strand'); + command(['live',persona,'--seed','2','--clock','fixed','--checkpoint-every','2','--strand',checkpoint],'{}\n{}\n{}\n'); + const cp=await server([checkpoint],'checkpoint');await navigate(cp.url);await event(3); + await until(async()=>(await state()).historyCount==='2/2 recent documents','checkpoint bound'); + assert.ok((await state()).badge.includes('from checkpoint 2'));assert.equal(await value(`document.getElementById('scrubber').min`),'2');report.checkpoint=true; - const checkpoint = path.join(out, 'checkpoint-' + Date.now() + '.strand'); - command(['live', 'examples/persona/tutor.yaml', '--seed', '2', '--clock', 'fixed', '--checkpoint-every', '2', '--strand', checkpoint], '{}\n{}\n{}\n'); - const checkpointServer = await server([checkpoint], 'checkpoint'); - await navigate(checkpointServer.url); - await event(3); - report.checkpoint = await state(); - assert.ok(report.checkpoint.badge.includes('from checkpoint 2')); - assert.equal(failures.length, 0, JSON.stringify(failures)); - report.ok = true; + const longStrand=path.join(out,'long-history.strand'),longFull=path.join(out,'long-full.strand'); + command(['live',persona,'--seed','2','--clock','fixed','--strand',longFull],'{}\n'.repeat(111)); + const longLines=readFileSync(longFull,'utf8').match(/.*\n/g);writeFileSync(longStrand,longLines.slice(0,61).join('')); + const longServer=await server([longStrand],'long-history');await navigate(longServer.url);await event(60); + await until(async()=>(await state()).historyCount==='50/50 recent documents','bounded history'); + assert.equal(await value(`document.querySelectorAll('#history-strip button').length`),50);assert.equal(await value(`document.getElementById('scrubber').min`),'11'); + const fetches=await value(`window.monitorFetch`);assert.ok(fetches.maximum<=4&&fetches.maximum>0);assert.equal(fetches.docs.length,49);report.bounds={documents:50,maximumFetches:fetches.maximum}; + await click('#hold');appendFileSync(longStrand,longLines.slice(61).join('')); + await until(async()=>(await state()).streamDetail.includes('latest received event 111'),'held outside window'); + await until(async()=>(await state()).historyCount==='50/50 recent documents','new bounded window'); + assert.equal((await state()).event,'event 60');assert.ok((await state()).historyNote.includes('outside the recent window (#62–#111)')); + assert.equal(await value(`document.getElementById('scrubber').parentNode.hidden`),true);assert.equal(await value(`document.querySelectorAll('#history-strip [aria-current="true"]').length`),0); + await shot('activity-held-outside-window',360,800);await click('#follow');await event(111); + assert.equal(await value(`document.getElementById('scrubber').parentNode.hidden`),false); + assert.ok(await value(`performance.getEntriesByType('resource').every(r=>new URL(r.name).origin===location.origin)`),'No external resources'); + // Persona changes without an accompanying frame must erase prior visible state, including held views. + await value(`document.getElementById('hold').click();window.monitorTestStream.close();window.monitorTestStream.dispatchEvent(new MessageEvent('meta',{data:JSON.stringify({persona:{name:'Second fixture',version:'1',seed:9,individual:'synthetic-second'},path:'synthetic.strand',traits:[],moods:[],inputs:[],history:[]})}));`); + report.metadataOnly=await state();assert.equal(report.metadataOnly.identity,'Second fixture');assert.equal(report.metadataOnly.line,'');assert.equal(report.metadataOnly.why,'');assert.equal(report.metadataOnly.workspaceHidden,true);assert.equal(report.metadataOnly.rawHidden,true);assert.equal(report.metadataOnly.learnedHidden,true);assert.equal(report.metadataOnly.stream,'AWAITING FIRST EVENT'); + await value(`window.monitorTestStream.dispatchEvent(new Event('error'))`);assert.equal((await state()).mode,'disconnected');assert.equal((await state()).workspaceHidden,true); + // No moods or drives is a normal supported persona, not an empty app. + const bare=path.join(out,'traits-only.json'),bareStrand=path.join(out,'traits-only.strand'); + writeFileSync(bare,JSON.stringify({probbit_persona:1,identity:{name:'Minimal fixture',version:'1'},traits:[{id:'approach',levels:['consider','act']}]})); + command(['live',bare,'--clock','fixed','--strand',bareStrand],'{}\n'); + const bareServer=await server([bareStrand],'traits-only');await navigate(bareServer.url);await event(1);assert.equal((await rows()).length,1);assert.equal((await state()).selected,'approach'); + await click('[data-filter="drive"]');assert.equal((await rows()).length,0);assert.equal(await value(`document.getElementById('no-match').hidden`),false);report.traitsOnly=true; + const refusing=path.join(out,'unreleased.json'),refusingStrand=path.join(out,'unreleased.strand'); + const refusingPersona=JSON.parse(readFileSync(path.join(repo,'examples/persona/tutor.json'),'utf8')); + refusingPersona.engine={op:'sample',sweeps:8,on_conflict:'yield'};writeFileSync(refusing,JSON.stringify(refusingPersona)); + const refused=JSON.parse(command(['live',refusing,'--clock','fixed','--strand',refusingStrand],'{}\n').trim()); + const unreleased=Object.keys(refused.stance).find(k=>refused.stance[k].released===false);assert.ok(unreleased,'Fixture must genuinely withhold a trait'); + const refusingServer=await server([refusingStrand],'unreleased');await navigate(refusingServer.url);await event(1);await selected('trait:'+unreleased); + assert.ok((await rows()).find(r=>r.key==='trait:'+unreleased).current.includes('unreleased'));assert.ok(await value(`document.getElementById('channel-sub').textContent.includes('engine did not vouch')`));report.unreleased=true; + const capped=path.join(out,'declared-scales.json'),cappedStrand=path.join(out,'declared-scales.strand'); + writeFileSync(capped,JSON.stringify({probbit_persona:1,identity:{name:'Scale fixture',version:'1'},traits:[{id:'approach',levels:['consider','act']}],drives:{goals:[{id:'repair'},{id:'explore'}],wanting:{cap:7},afterglow:{cap:4},expectation:{pe_cap:2.5}}})); + command(['live',capped,'--clock','fixed','--strand',cappedStrand],'{}\n'); + const cappedServer=await server([cappedStrand],'declared-scales');await navigate(cappedServer.url);await event(1); + await selected('drive:surprise:repair');assert.ok(await value(`document.getElementById('channel-facts').textContent.includes('-2.5 to 2.5')`)); + assert.ok(await value(`document.getElementById('detail-chart').textContent.includes('2.50')`));report.declaredScales=true; + assert.equal(failures.length,0,JSON.stringify(failures));report.ok=true; } catch (error) { report.error = String(error.stack || error); process.exitCode = 1; diff --git a/probbit-cli/tests/persona.rs b/probbit-cli/tests/persona.rs index 1ac5969..cde9f40 100644 --- a/probbit-cli/tests/persona.rs +++ b/probbit-cli/tests/persona.rs @@ -514,6 +514,34 @@ fn fuzz_is_deterministic_and_never_breaks_a_habit() { } } +/// 0.8.2: the search's grid gains every threshold the rule and the habits read on a number input. "No jokes at stakes 0.7 to +/// 0.8" sits between the default grid points (0, 0.5, 1): the random search alone (`--beam 0`) never put it in force and found +/// nothing; now the grid reports [0, 0.5, 1, 0.7, 0.8] and the single event {"stakes": 0.7} breaks it. The thresholds go on the +/// grid's scale (a fraction of the input's max), from habits too, each once, after the given points. +#[test] +fn fuzz_grid_includes_the_thresholds_of_the_rule_and_the_habits() { + let rule = "{when: {stakes: {at_least: 0.7, at_most: 0.8}}, then: {humour: [none]}}"; + for grid in [None, Some("0,0.5,1")] { + let mut args = vec!["persona", "fuzz", "examples/persona/tutor.yaml", "--never", rule, "--seeds", "0-4", "--beam", "0", "--scripts", "40", "--json"]; + if let Some(g) = grid { args.extend(["--grid", g]); } + let (c, out, e) = probbit_at_root(&args); assert_eq!(c, 1, "{e}"); + let d = parse(&out); assert_eq!(jw(s(&d, &["search", "grid"])), "[0,0.5,1,0.7,0.8]"); + assert_eq!(jw(s(&d, &["properties"]).as_arr().unwrap()[0].get("shortest").and_then(|x| x.get("script")).unwrap()), r#"[{"stakes":0.7}]"#); + } + // a threshold already on the grid is not added twice + let (_, out, _) = probbit_at_root(&["persona", "fuzz", "examples/persona/tutor.yaml", "--never", "{when: {stakes: 0.5}, then: {humour: [none]}}", "--seeds", "0", "--scripts", "1", "--beam", "0", "--json"]); + assert_eq!(jw(s(&parse(&out), &["search", "grid"])), "[0,0.5,1]"); + // max 100: a habit's 65 and the rule's 42 become 0.65 and 0.42 (sorted, after the given points) + let file = tmp("grid-thresholds.json"); + std::fs::write(&file, r#"{"probbit_persona":1,"identity":{"name":"Fixture","version":"1"}, + "traits":[{"id":"tone","levels":["calm","sharp"],"prior":[0.6,0.4]}], + "inputs":[{"id":"pressure","kind":"number","max":100,"effects":{"tone":1}}], + "habits":[{"id":"calm_under_pressure","when":{"pressure":{"at_least":65}},"then":{"tone":["calm"]}}]}"#).unwrap(); + let (_, out, e) = probbit(&["persona", "fuzz", &file, "--never", r#"{"when":{"pressure":42},"then":{"tone":["calm"]}}"#, "--seeds", "0", "--scripts", "1", "--beam", "0", "--json"], ""); + assert_eq!(jw(s(&parse(&out), &["search", "grid"])), "[0,0.5,1,0.42,0.65]", "{e}"); + let _ = std::fs::remove_file(&file); +} + /// A bad rule is one error object at its path (exit 2), like a bad habit; a missing rule or a bad flag is a stderr line (exit 2) #[test] fn fuzz_refuses_bad_rules_and_flags() { @@ -565,7 +593,35 @@ fn lint_with_rules_proves_then_fuzzes_the_unknown_ones() { assert_eq!((s(&p[1], &["verdict"]).as_str(), s(&p[1], &["fuzz", "verdict"]).as_str()), (Some("unknown"), Some("counterexample"))); assert_eq!(jw(s(&p[1], &["fuzz", "shortest", "script"])), r#"[{"sentiment":"negative"}]"#); let (c, out, e) = probbit_at_root(&["persona", "lint", TUTOR_050, "--never", "{when: {sentiment: negative}, then: {directness: {at_most: balanced}}}", "--seeds", "0-2"]); assert_eq!(c, 0, "{e}"); - assert_eq!(s(&parse(&out), &["ok"]), &Json::Bool(true)); + let l = parse(&out); assert_eq!((s(&l, &["ok"]), jw(s(&l, &["unproven"]))), (&Json::Bool(true), "[]".to_string())); + let _ = std::fs::remove_file(&props); +} + +/// 0.8.2: a rule the bound leaves unknown and the search cannot break is not green. On the shipped tutor, "a loss makes it warm" +/// is unknown for seeds 0-2 and fuzz finds no counterexample: exit 3, `ok` false, `broken` 0, one `unproven` entry naming the +/// undecided seeds and what was searched, and "unproven, not broken" on stderr. A proved rule stays exit 0; a broken rule next to +/// an unproven one exits 1 and still lists the unproven one. +#[test] +fn lint_reports_an_unknown_unbroken_rule_as_unproven() { + let tutor = "examples/persona/tutor.yaml"; + let (c, out, e) = probbit_at_root(&["persona", "lint", tutor, "--never", "{when: {loss: true}, then: {warmth: [warm]}}", "--seeds", "0-2"]); + assert_eq!(c, 3, "{e}"); assert!(e.contains("1 rule unproven, not broken: never"), "{e}"); + let l = parse(&out); assert_eq!((s(&l, &["ok"]), s(&l, &["broken"]).as_f64()), (&Json::Bool(false), Some(0.0))); + let p = s(&l, &["props"]).as_arr().unwrap(); assert_eq!((s(&p[0], &["verdict"]).as_str(), s(&p[0], &["fuzz", "verdict"]).as_str()), (Some("unknown"), Some("none_found"))); + let u = s(&l, &["unproven"]).as_arr().unwrap(); assert_eq!(u.len(), 1, "{out}"); + assert_eq!((s(&u[0], &["id"]).as_str(), jw(s(&u[0], &["rule"])), jw(s(&u[0], &["seeds_unknown"]))), + (Some("never"), r#"{"then":{"warmth":["warm"]},"when":{"loss":true}}"#.to_string(), "[0,1,2]".to_string())); + assert_eq!(jw(s(&u[0], &["searched"])), r#"{"beam":4,"depth":8,"grid":[0,0.5,1],"hours":[1,12,48],"individuals":3,"scripts":60}"#); + // proved: exit 0, nothing unproven, nothing on stderr about it + let (c, out, e) = probbit_at_root(&["persona", "lint", tutor, "--never", "{when: {error: true}, then: {caution: {at_least: measured}}}", "--seeds", "0-2"]); + assert_eq!(c, 0, "{e}"); assert_eq!(jw(s(&parse(&out), &["unproven"])), "[]"); assert!(!e.contains("unproven"), "{e}"); + // broken next to unproven: exit 1, both reported + let props = tmp("lint-unproven.json"); + std::fs::write(&props, r#"{"props": [{"id": "praise_lifts", "when": {"praise": true}, "then": {"valence": {"at_least": "even"}}}, + {"id": "loss_warm", "when": {"loss": true}, "then": {"warmth": ["warm"]}}]}"#).unwrap(); + let (c, out, e) = probbit_at_root(&["persona", "lint", tutor, "--props", &props, "--seeds", "0-2"]); assert_eq!(c, 1, "{e}"); + let l = parse(&out); assert_eq!((s(&l, &["ok"]), s(&l, &["broken"]).as_f64()), (&Json::Bool(false), Some(1.0))); + assert_eq!(s(&l, &["unproven"]).as_arr().unwrap().iter().map(|u| s(u, &["id"]).as_str().unwrap()).collect::>(), ["loss_warm"]); let _ = std::fs::remove_file(&props); } diff --git a/scripts/bench/test_install_sh.sh b/scripts/bench/test_install_sh.sh index 595b086..1f6c10d 100755 --- a/scripts/bench/test_install_sh.sh +++ b/scripts/bench/test_install_sh.sh @@ -66,4 +66,13 @@ sh scripts/bench/package_like_release.sh "$WORK/unrunnable/probbit" "$TARGET" "$ if PROBBIT_DOWNLOAD_BASE="$BASE/unrunnable" PROBBIT_VERSION="$TAG" PROBBIT_INSTALL_DIR="$WORK/bin" "$SH" install.sh; then echo "FAIL: unrunnable candidate accepted"; exit 1; fi after=$("$PY" -c 'import hashlib,sys; print(hashlib.sha256(open(sys.argv[1],"rb").read()).hexdigest())' "$WORK/bin/probbit") [ "$before" = "$after" ] || { echo "FAIL: unrunnable candidate replaced working binary"; exit 1; } + +echo "== 8. the PATH hint names the rc file of the user's shell" +case "$(uname -s)" in Darwin) bashrc='~/.bash_profile' ;; *) bashrc='~/.bashrc' ;; esac +for pair in "/bin/zsh ~/.zshrc" "/usr/local/bin/zsh ~/.zshrc" "/bin/bash $bashrc" "/bin/dash ~/.profile" " ~/.profile"; do + shell=${pair% *} want=${pair##* } + out=$(SHELL="$shell" PROBBIT_DOWNLOAD_BASE="$BASE/good" PROBBIT_VERSION="$TAG" PROBBIT_INSTALL_DIR="$WORK/hint" "$SH" install.sh 2>&1) + case "$out" in *"$WORK/hint is not on your PATH. Add it in $want:"*) ;; *) echo "FAIL: SHELL=$shell: no hint naming $want"; echo "$out"; exit 1 ;; esac + echo "ok: SHELL=${shell:-(empty)} -> $want" +done echo "install.sh: all checks passed ($SH, $TARGET)"