Skip to content

examples: the rule-check demo behind the clip - #32

Merged
BitmapAsset merged 1 commit into
mainfrom
examples-rule-check-demo
Oct 14, 2026
Merged

BitmapAsset merged 1 commit into
mainfrom
examples-rule-check-demo

Conversation

@BitmapAsset

Copy link
Copy Markdown
Owner

The script behind the rule-check clip, on the shipped dispatch example, so anyone can reproduce every number in it.

examples/rule-check/demo.py (Python 3.9 standard library; PROBBIT_BIN or probbit on PATH; probbit 0.8.2 or later; uses only persona prove, live and live verify)

  • Derives the "before" policy by text: examples/persona/dispatch-policy.yaml minus its routine_acts habit (the list item and the comment above it), written to a temp dir.
  • 20 individuals (seeds 0-19): the day-0 rule check (persona prove --props dispatch-props.json, rule routine_work_acts), a day-0 test of 10 routine requests, 300 synthetic events in which users praise asking and criticise acting (live, fixed clock, src: human:synthetic), 20 routine requests on day 30, and live verify on every strand.
  • The population line over seeds 0-99, then the shipped policy (habit included) with the same check and the same events.
  • Flags --seeds 0-19, --events 300, --population 0-99. The output is identical between runs apart from the elapsed time on the last line.

Measured with probbit 0.8.2 (and with a build of main; same output):

day-0 test passed 17/20; by day 30, 4 of those ask about routine work every time.
rule check flagged 7/20 on day 0; drifted 7/20; flagged-and-drifted 7; drifted-but-not-flagged 0; flagged-not-drifted 0.
all strands replay byte for byte: True
population, seeds 0-99: flagged 24, drifted 24, both 24; drifted-but-not-flagged 0; passed day-0 test then drifted 15.
after one added habit: check says held_by_construction for 20/20; drifted 0/20; 10.1 s

Also: examples/rule-check/README.md; one README paragraph in "Check in CI" ("Reproduce the clip") with these numbers; a CHANGELOG 0.8.3 Added line; python/test_rule_check_demo.py, a smoke test that runs the demo with --seeds 0-3 --events 20 (about 2 s). Python suite: 45 OK. No Rust or workflow change; no tag, release or publish.

examples/rule-check/demo.py (Python 3.9 standard library, probbit 0.8.2 or later) runs the rule-check demo
on the shipped dispatch example: the policy minus its routine habit, the day-0 rule check and test, 300
synthetic rewarded events per individual, day 30, the population line over seeds 0-99, then the same
check and events with the habit back. README paragraph with the measured numbers, CHANGELOG line, and a
smoke test in the Python suite (4 individuals, 20 events, about 2 s).
@BitmapAsset
BitmapAsset merged commit e1d9660 into main Oct 14, 2026
15 checks passed
@BitmapAsset
BitmapAsset deleted the examples-rule-check-demo branch October 14, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant