Skip to content

feat(crdt): grove_crdt Dart client, and x-forge-sync on the extension's sync routes - #34

Merged
juicycleff merged 55 commits into
mainfrom
feat/crdt-dart
Oct 8, 2026
Merged

juicycleff merged 55 commits into
mainfrom
feat/crdt-dart

Conversation

@juicycleff

@juicycleff juicycleff commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

This adds grove_crdt, a pure Dart port of the CRDT client, in crdt-dart/. It also marks the extension's sync routes with x-forge-sync, so forge's client generators can find them. Forge's Dart client (forge_client_grove) is built on top of it.

Trying it

You'll need the Flutter 3.47 toolchain (Dart 3.13). From crdt-dart/:

dart pub get
dart test
dart test -p node

The conformance and convergence tests under test/conformance/ build a real Go server from this repo's crdt package and run Dart clients against it, so they need Go 1.26 as well. They build with GOWORK=off on purpose: the server is its own module in crdt-dart/tool/conformance_server, which keeps modernc.org/sqlite and gorilla/websocket out of grove's root go.mod.

What's in it

The package covers what crdt-js covers: the HLC, every CRDT type (register, counter, set, list, document, text), merge and apply, compaction, the store with its pending queue and undo, plugins, presence and rooms, HTTP, SSE and WebSocket transports, and a sync engine. Where crdt-js and the Go server disagree, the Go server wins. crdt-dart/doc/go-parity.md lists every place that matters, and why.

Four checks hold it to Go:

  • Wire JSON is pinned by goldens that a Go program generates from the real crdt structs (cmd/golden).
  • Set keys are byte-identical to Go's json.Marshal, lone surrogates included.
  • Text offsets count runes, the way Go does.
  • Merge, apply and the text CRDT were fuzzed against Go across close to 200,000 random operations on three replicas, with no mismatches.

The sync engine adds the things an offline device needs and crdt-js doesn't have:

  • per-table paging that survives the server's page limit;
  • push batching;
  • rejection handling, so a bad change is marked and never blocks the queue;
  • clock-skew correction;
  • cancellation that stops cleanly when the signed-in account changes.

In total there are about 1,500 tests on the VM and on node.

On the Go side, WithCRDTEntity maps tables to entities, and every sync route (/sync/pull, /sync/push, /sync/stream, /sync/ws) now carries forge.WithSync with its role. The SSE stream also sends a keep-alive comment every 15 seconds while it's idle. Without it, a quiet stream got dropped every 45 seconds, and each drop wiped the node's presence.

Before this can merge

Nothing's blocking it now. The extension needed a forge release with forge.WithSync, and v1.12.2 has it. extension/go.mod and kv/extension/go.mod both require forge v1.12.2, and the extension builds without a go.work.

Nothing in crdt/ changes. #35 has merged into main since this was opened, and this branch still merges cleanly on top of it.

One thing the tests found in the server

A change pushed with an older HLC than a peer has already pulled past never reaches that peer, when the merge changes state (counters, sets, lists, text). ReadChangesSince pages by the row's stored HLC (crdt/metadata.go:208), and a state-changing merge keeps the older one. A device that was offline for a while loses its counter increments on every other device. There's a skipped test in test/conformance/convergence_test.dart that asserts the right behaviour. The fix belongs in the server: bump the row HLC to the server clock on a state-changing merge. It's left out here so crdt/ stays untouched. #37 fixes it on the server. Once that merges, un-skip the test and drop the LATE_STAMP_SKIP exemption from the CRDT Dart workflow in the same commit.

While we were in there, we noticed streamKeepAlive in crdt/server.go is set but never read. The extension now sends its own keep-alive, so nothing here depends on it.

Selects the conformance tests by suite path and fails when none ran, one failed, or one skipped for anything but the documented late-stamp limit.
The conformance server compiles grove/hook, so a hook change now runs the workflow. The assertion lists the files under test/conformance/ and fails when one of them ran no passing test.
MIN_CONFORMANCE_SUITES is the count of files under test/conformance/ today. Adding a suite needs no edit, and removing one means lowering the number on purpose.
@juicycleff
juicycleff marked this pull request as ready for review October 8, 2026 13:30
@juicycleff
juicycleff merged commit c98e352 into main Oct 8, 2026
9 checks passed
@juicycleff
juicycleff deleted the feat/crdt-dart branch October 8, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant