feat: make the conformance checker one installable command

.github/workflows/ci.yml

@@ -66,15 +66,9 @@ jobs:
 
       - name: C++ client conforms to SPEC.md
         run: |
-          uv run --with websockets examples/conformance_server.py \
-            --port 8765 --steps 10 --horizon 4 --action-dim 7 --timeout 120 &
-          server=$!
-          for _ in $(seq 30); do
-            (echo > /dev/tcp/127.0.0.1/8765) 2>/dev/null && break
-            sleep 1
-          done
-          /tmp/plugrl_client 127.0.0.1 8765 10
-          wait $server
+          uv run --extra conformance plugrl-conformance \
+            --port 8765 --steps 10 --horizon 4 --action-dim 7 --timeout 120 \
+            --client "/tmp/plugrl_client 127.0.0.1 8765 10"
 
       # The action dtype is the environment's and is never renegotiated, so a
       # client has to read the typestr. A client that hard-codes float32 still
@@ -84,30 +78,19 @@ jobs:
       # so the client's own output says whether it decoded them.
       - name: ... and reads the action typestr rather than assuming float32
         run: |
-          uv run --with websockets examples/conformance_server.py \
+          # Without pipefail a failing check would be hidden by tee's status.
+          set -o pipefail
+          uv run --extra conformance plugrl-conformance \
             --port 8766 --steps 3 --horizon 2 --action-dim 3 \
-            --action-dtype float64 --timeout 120 &
-          server=$!
-          for _ in $(seq 30); do
-            (echo > /dev/tcp/127.0.0.1/8766) 2>/dev/null && break
-            sleep 1
-          done
-          /tmp/plugrl_client 127.0.0.1 8766 3 | tee /tmp/out.txt
-          wait $server
+            --action-dtype float64 --timeout 120 \
+            --client "/tmp/plugrl_client 127.0.0.1 8766 3" | tee /tmp/out.txt
           grep -q 'dtype=<f8' /tmp/out.txt
           # Time-major [H, n, da] with action[t, :, 0] = t, so the first six
           # values of a (2, 1, 3) chunk are 0 0 0 then 1 0 0.
           grep -q 'first values: 0 0 0 1 0 0' /tmp/out.txt
 
       - name: Python reference client conforms to SPEC.md
         run: |
-          uv run --with websockets examples/conformance_server.py \
-            --port 8767 --steps 10 --horizon 3 --action-dim 5 --timeout 120 &
-          server=$!
-          for _ in $(seq 30); do
-            (echo > /dev/tcp/127.0.0.1/8767) 2>/dev/null && break
-            sleep 1
-          done
-          uv run --with websockets examples/raw_client.py \
-            --host 127.0.0.1 --port 8767 --steps 10
-          wait $server
+          uv run --extra conformance plugrl-conformance \
+            --port 8767 --steps 10 --horizon 3 --action-dim 5 --timeout 120 \
+            --client "python examples/raw_client.py --host 127.0.0.1 --port 8767 --steps 10"

README.md

@@ -20,11 +20,25 @@ own known defects, and everything in it that can be checked by a test is.
 |---|---|
 | [`SPEC.md`](SPEC.md) | the protocol, in full |
 | [`src/plugrl_protocol/`](src/plugrl_protocol/) | the message types and the msgpack codec — 101 lines |
+| [`src/plugrl_protocol/conformance.py`](src/plugrl_protocol/conformance.py) | `plugrl-conformance`, which grades a client clause by clause |
 | [`examples/`](examples/) | two env clients written against the spec, sharing no code with PlugRL |
 | [`tests/`](tests/) | the spec's checkable clauses, as tests |
 
-The package is small on purpose. It is the piece both sides import, so
-anything that could live on one side does.
+The codec is small on purpose. It is the piece both sides import, so anything
+that could live on one side does; the checker sits beside it rather than in
+it, and is not imported by either side at runtime.
+
+## Checking a client
+
+One command, which starts your client once the socket is listening:
+
+```bash
+uv run --extra conformance plugrl-conformance \
+    --port 8000 --steps 20 --client "./my_client 127.0.0.1 8000 20"
+```
+
+It exits non-zero on a violation, so it can sit in a CI job. What it does not
+require, because SPEC.md does not, is listed at the top of the module.
 
 ## The protocol in one screen
 
@@ -87,8 +101,7 @@ pip install "plugrl-protocol @ git+https://github.com/PlugRL/plugrl-protocol.git
 
 Only Python clients that want the shared codec need this at all. A client in
 another language should implement [SPEC.md](SPEC.md) directly, which is what
-the C++ example does - and what `examples/conformance_server.py` will grade
-it against.
+the C++ example does - and what `plugrl-conformance` will grade it against.
 
 ## License