Specify the wire protocol, and check it on every change

[tactino]

Gives the wire protocol a written specification, a way to check an
implementation against it, and CI that does so on every change. The licence
and packaging work this branch started with is now the smaller half.

The protocol is written down

SPEC.md (530 lines) is the first statement of what the bytes between the
training server and an env client actually are: transport settings, the
msgpack ndarray convention, the four messages, the exchange rules, the close
reasons, a conformance checklist.

It is written from the implementation and says so. Where the
implementation is under-specified, or where two of our own servers disagree,
it says that in a box marked Gap rather than inventing a rule. Five of
those:

Gap
The two servers disagree	WebSocketAgentServer sends env_ids with the action; RayAgentServer sends none, ignores env_indices, and keeps one env's state per connection. An earlier single-env dialect that was never updated.
text leaks numpy	It ships as a <U array — fixed-width UTF-32, NUL-padded, width set by the longest string in that message. Measured, not assumed.
The two clients disagree about it	plugrl-env-client sends the <U array; examples/raw_client.py sends a msgpack string array. Both accepted, because the server never validates observation contents.
step_ids	Required, sent, read by nothing.
Authorization	Sent when configured, checked by no server. There is no authentication.

The metadata message used to be a sixth: always {}. plugrl-server now
fills it in, and section 5.1 documents the keys plus the rule that makes them
trustworthy — an absent key means the server does not know, never that the
value is a default.

It can be checked, not just read

examples/conformance_server.py speaks the protocol and grades a client
clause by clause, printing what it broke and exiting non-zero. plugrl-server
cannot do this job: it validates the message_type and hands the rest to the
policy, which is right for a training run and useless to someone bringing up
a client in a new language.

The report has two severities and keeping them apart is the point. A
violation is something the real server would reject or mishandle. A
note is something it accepts that differs from what the Python client
sends — a portability risk, not a breach. Collapsing the second into the
first would mean enforcing rules the protocol does not have.

I checked that the harness bites: four deliberately broken clients were run
against it. Two infers in a row, a fourth key in the observation, and
mismatched leading dimensions were all caught. A float64 reward was not — and
that turned out to be the specification's fault. It said <f4; the server
never inspects the width. The spec now says float, and the harness reports
the difference as a note.

CI checks the cross-language claim

This repository asserts that the protocol can be spoken by something that is
not this codebase. That was demonstrated once, by hand, and would have stayed
green in the README long after it stopped being true. The new job builds the
C++ client, fails if ldd shows anything but the C++ and C runtimes, and
drives the conformance server. The Python reference client gets the same.

One step is subtler: a client that hard-codes float32 sends perfectly valid
messages and passes every clause a server can observe — it only misreads what
comes back. So that step runs a server whose actions are <f8 and whose
values encode the horizon position, and asserts on the client's own output.
It is the regression test for a bug the C++ client had until this branch.

Bugs this found in the C++ reference client

• pack_f8 and pack_i8 memcpy'd the host layout while declaring <f8 and
  <i8. Correct on x86, silently wrong on a big-endian controller, and
  undetectable at the far end because the declared byte order still said
  little.
• The action decode assumed float32. Against an <f8 server the old binary
  printed 0 0 0 0 0 0 where the new one prints 0 0 0 1 0 0.
• A text frame where binary was expected was fed to the msgpack parser.
• A frame length field was used to allocate before being bounded.

The rest

Apache-2.0 licence; msgpack_numpy.py attributed to openpi in the file
header and in NOTICE (it is openpi's file, reformatted only, and the wire
format is deliberately identical); CITATION.cff; pre-commit.

Tests: 30, from 0. The private-dependency CI problem this branch opened
with is resolved — the repository is public.

---

🤖 Generated with Claude Code