feat(chat): moderated free-form chat (service + bridge) - #44
Open
ChronoFinale wants to merge 1 commit into
Open
Conversation
ChronoFinale
force-pushed
the
feat/chat-moderation
branch
8 times, most recently
from
August 7, 2026 02:08
153a459 to
e1fa79d
Compare
ChronoFinale
marked this pull request as ready for review
August 7, 2026 02:10
ChronoFinale
force-pushed
the
feat/chat-moderation
branch
from
August 7, 2026 02:35
e1fa79d to
43307db
Compare
Adds apps/moderation - a verdict service - and routes chat through it. **The relay is unchanged until MODERATION_SERVICE_URL is set.** Unset, the local obscenity filter runs exactly as before and the new response branches are unreachable. The dev compose does set it, so `docker compose up` gives you a moderated stack out of the box - that needs a model in the moderation volume, and without one the service reports not-ready and chat fails closed rather than going through unmoderated. Run with `MODERATION_SERVICE_URL=` to get the old behaviour back. Message in, verdict out, nothing persisted: no database, no per-player state beyond an in-memory rate-limit bucket. An opaque playerId is the only identity that crosses the boundary, so this box cannot answer "what did this Steam account say". A funnel sorted by certainty rather than severity. A transform tier strips links whose domain is not approved and applies vocabulary rewrites; the transformed text is what every later tier judges AND publishes. Then rate-limit, threats, allowlist fast-pass, blocklist, PII/contact. Only the ambiguous middle reaches the model. The allowlist alone fast-passes about 60% of real chat - "good luck have fun" goes from 1301ms to 1ms - so the word lists ship with the service rather than being copied onto a box by hand. It judges one message and never punishes anyone: bans, mutes and strikes belong to whatever calls it. A message that is not already an allowlisted preset is sent for a verdict and published only on an explicit allow. Any timeout, non-200, redirect, unparseable body or unrecognised verdict fails closed. The allowlist short-circuit runs before the call, so an outage degrades to preset-only chat rather than no chat. A rewritten message is published and returned to the sender as publishText so a client can show what others received, while the evidence buffer and reported-message record keep what the player typed. A rewrite must never launder the record. Details that are easy to get wrong, and why they are the way they are: - redirect: 'error' - a verdict must come from the configured origin, not wherever a redirect leads. - The client deadline exceeds the service's judgement deadline, or a slow but successful verdict is abandoned here while still occupying the service's single model lane, and the retry deepens the backlog that caused it. - HTTP 429 is the service shedding load, not this player being too fast, so it reports an outage. Per-player limiting arrives as a 200 with band rate_limited and is the only thing told to slow down. - guard_unavailable means the model was down; the player is not told they broke a rule. Two variables for the service (GUARD_MODEL, SHADOW_MODE) and one for the relay (MODERATION_SERVICE_URL). No bearer token: the service publishes no host port, so it is reached only by the relay over an internal network. No resource limits either - uncapped, the container sees every host core and the guard matches its thread count to them, so the two cannot disagree. Capping CPUs WITHOUT also setting GUARD_THREADS is the one configuration that fails badly, because llama.cpp threads spin-wait and oversubscription collapses throughput instead of degrading it. The model is not in the image or the repo - it is far past GitHub's file limit, and baking it in would publish a fine-tune with every pull. Put the .gguf in the model volume once; it survives restarts and rebuilds. Without one the service still starts, reports not-ready, and chat fails closed. Ships in shadow mode: the guard logs what it would block without blocking, and the deterministic tiers still enforce. Turning the bridge on with stock defaults is therefore MORE permissive than the local filter it replaces, not less - that is the intended first step, but it should be a deliberate one. 443 relay + 228 service unit tests, no infra required. The e2e suite drives the whole stack for real: a clean message reaches the other player over MQTT, a violent threat is refused and published to nobody, the refusal reads as something a player can understand, and a burst is rate-limited without being reported as an outage. Getting that suite honest meant fixing three things in the harness that made it report success it had not earned: the api build context could never rebuild the image, so a stale binary was under test; chat is disabled by default at two gates, so every message was refused before moderation ran and the reject tests passed on that; and database seeding trusted whatever answered on a port, which a stray tunnel to another host had been shadowing.
ChronoFinale
force-pushed
the
feat/chat-moderation
branch
from
August 7, 2026 02:53
43307db to
e25082a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Chat moderation, server side. Adds
apps/moderation(a verdict service) to the monorepo and routes chat through it.Try it
Moderated out of the box. It needs a guard model in the volume. Chrono has the
.gguf(tuned-v2.Q8_0.gguf, 767MB, md5f5c3697c18e20fc386aa1b1b367b3caf— worth checking, it is a big file):The filename does not matter:
GUARD_MODELdefaults to the/model-cachedirectory and loads whatever single.ggufis in it, logging which. Point it at a file if you keep several there.Without one the service still starts and reports not-ready, and what happens to chat depends on the mode. Shipping default is
SHADOW_MODE=1: a missing or still-loading model publishes (logged asreview) rather than taking chat down, because that mode already publishes everything the guard would have blocked — refusing when it cannot answer at all would be strictly harsher than the case where it did object. SetSHADOW_MODE=0and it fails closed instead: nothing unjudged is published.Either way this is never unfiltered — the deterministic tiers (threats, blocklist, PII/contact, link stripping, rate limit) run ahead of the model and still reject with no model present. A model that is merely slow (deadline/backlog) refuses in both modes, since a retry there actually succeeds. If chat looks broken, check
model_loadedinGET /healthfirst.MODERATION_SERVICE_URL= docker compose upturns the bridge off entirely and chat falls back to the local obscenity filter, exactly as today.What it is
Message in, verdict out. No database, no per-player state beyond an in-memory rate-limit bucket. An opaque
playerIdis the only identity that crosses the boundary, so this side can't answer "what did this Steam account say".A funnel sorted by certainty, not severity: strip unapproved links and apply vocabulary rewrites, then rate-limit → threats → allowlist fast-pass → blocklist → PII/contact. Only the ambiguous middle reaches the model. The allowlist alone fast-passes ~60% of real chat (
good luck have fungoes 1301ms → 1ms), which is why the word lists ship with the service instead of being copied onto a box by hand.It judges one message and punishes nobody — bans, mutes and strikes stay yours.
Resources
RAM is the binding constraint, not throughput — measured chat peaked at 7.2 msg/s in the worst minute on record.
If you benchmark smaller boxes: cap CPUs and set
GUARD_THREADSto the same number. llama.cpp threads spin-wait, so oversubscribing doesn't degrade gracefully, it collapses (~2s per judgement becomes 20–40s). Uncapped it self-tunes, which is why there are no limits in the compose by default. This is almost certainly what "the lower spec couldn't handle it" was.Note the image is debian-slim rather than alpine, unlike
apps/server— node-llama-cpp's prebuilt binaries need glibc.Shadow mode
Ships with
SHADOW_MODE=1: the guard logs what it would block without blocking, while the deterministic tiers still enforce. Combined with a blocklist deliberately softer than the relay's local filter, turning this on with stock defaults is more permissive than today, not less. That's the intended first step — setSHADOW_MODE=0to actually enforce.Tests
pnpm test— 591 relay + 244 service, no infra needed: no Docker, no model, no network. The guard is injected as data, so every tier of the pipeline is exercised with plain strings and a fake judge.There is an end-to-end suite too (two real clients over MQTT against the built stack), but it needs Docker plus a guard GGUF to assert anything, so it is not in this PR — it would only ever show up as a skip or a failure in review. Happy to bring it over separately if you want it.
Behaviour verified by hand in containers, since unit tests cannot see it: enforcing and shadow mode, each with the model present and absent; deterministic tiers still rejecting with no model loaded; and the bridge switched off falling back to the local obscenity filter unchanged.
Not included
Reports, appeals, sanctions and a moderator console are built but deliberately left out — this is blocking and delivery notices only.
Client half: Balatro-Multiplayer/BalatroMultiplayerAPI#19. Optional — without it a blocked message still shows the server's reason, just without the instant echo.