Skip to content

Share over HTTPS: signed static feeds, trust publish / follow / pull - #21

Open
harlantwood wants to merge 4 commits into
masterfrom
claude/https-feeds-bsuvj4
Open

harlantwood wants to merge 4 commits into
masterfrom
claude/https-feeds-bsuvj4

Conversation

@harlantwood

Copy link
Copy Markdown
Member

TL;DR

Roadmap Phase 3. You can now publish your signed atoms as a static feed that any web host can serve (GitHub Pages included), and follow other people's feeds over HTTPS. Every atom is verified before it reaches the store. The feed format and its verification live in the pure core and are exposed to JS as buildFeed / verifyFeed.

Feed format (core, trustgraph_core::feed, documented in doc/feeds.md)

  • A feed is a directory holding atoms.ndjson (signed Trust Atom credentials, one per line, as trust add stores them) and index.json (type: TrustFeedIndex, owner DID, updated passed in by the caller, atoms.count, atoms.digest = SHA2-256 multihash Qm… of the exact bytes of atoms.ndjson).
  • The index is signed with the same eddsa-jcs-2022 Data Integrity code as atoms (credential::sign / verify). All signing goes through two private functions, secure and check, so a later format PR can change it in one place.
  • feed::verify is all or nothing. It checks the index proof and that the signer is owner, the type, the digest, every atom's proof, that every atom was issued by owner, and the count.
  • feed::build drops duplicates and refuses unsigned atoms or atoms issued by anyone else.
  • New Error::InvalidFeed.

API and bindings

  • api::build_feed(credentials, secret, updated) -> {index, atoms} and api::verify_feed(index, atoms) -> {valid, owner?, updated?, ids?, atoms?, error?}. Like verify, verify_feed never throws.
  • Both are exposed identically in wasm (buildFeed, verifyFeed) and node, with types in bindings/trustgraph.d.ts. They are covered by tests/js/smoke.mjs and tests/js/types.ts.

CLI

  • trust publish [--out DIR] [--well-known] [--as KEY] (--as is an alias of --key) writes every signed atom in the store whose source is your DID. Unsigned atoms are skipped, with a note on stderr. --well-known writes DIR/.well-known/trust/ and also an empty DIR/.nojekyll, because GitHub Pages otherwise does not serve dot-directories. Files are written atoms first, each via a temp file and rename.
  • trust follow <feed> [--no-pull], trust unfollow <feed>, trust following, trust pull [feed].
    • Accepted feeds: an https URL, a bare domain (resolved to https://domain/.well-known/trust/index.json), a URL with no path (same well-known location), a directory URL (resolved to …/index.json), or a local path or file:// URL.
    • A feed is only followed if it verifies, unless --no-pull is given.
    • pull with several feeds reports each one and exits 1 if any failed.
  • HTTP uses ureq 3: blocking, rustls with webpki roots, no tokio. It takes proxies from the environment and limits sizes to 1 MiB for the index and 256 MiB for the atoms. Plain http:// is accepted because signatures, not TLS, protect the content. Bare domains always use https.
  • following.json in the CLI home records, per feed:
    • the ETag, sent back as If-None-Match, so an unchanged feed costs one 304;
    • the digest, so atoms.ndjson is not re-downloaded when the index is unchanged on hosts without ETags;
    • the pinned owner, so a feed later signed by someone else is rejected;
    • the last updated time, so an older index cannot be replayed (no rollbacks).
      This state is only updated after the whole feed verifies.

Docs

  • New doc/feeds.md. README has a new "Sharing" section with a GitHub Pages example and new rows in the command table. Feed functions were added to doc/architecture.md and the core README. One row was added to the roadmap's "What has shipped" table.

Verified

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets --locked and cargo clippy -p trustgraph-core --all-targets --locked, both with RUSTFLAGS=-D warnings
  • RUSTDOCFLAGS=-D warnings cargo doc --workspace --no-deps --locked
  • scripts/check-core-purity.sh
  • cargo test --workspace --locked on stable and on 1.85 (MSRV)
  • New tests: 8 core feed unit tests, an API test, CLI unit tests for location parsing and state, and 7 CLI integration tests in crates/trustgraph-cli/tests/feeds.rs.
    • They use a local path feed and a tiny in-process HTTP server (std TcpListener thread with ETag support). There is no external network.
    • They cover a tampered atom, including one under a re-signed index, digest mismatch, a foreign atom, the ETag 304 path, owner pinning, rollback, .well-known discovery and HTTP 404.
  • wasm job: wasm32 clippy, scripts/build-wasm-package.sh (574 KiB, under budget), and smoke tests on the node and web builds. Node job: pnpm run build:node && pnpm run test:node. pnpm run typecheck.
  • By hand: trust follow example.com through the session proxy gets a real HTTPS 404, which shows that TLS and the proxy work.

Not in this PR

  • did:web, so organisations can sign feeds with their own domain. It is only mentioned as a follow-up.
  • IPFS transport, incremental or paged feeds for very large feeds, and revocation of published atoms.
  • Using the OS certificate store (ureq's platform-verifier feature) instead of webpki roots, if TLS-intercepting corporate proxies turn out to matter.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68


Generated by Claude Code

claude added 4 commits October 8, 2026 15:06
A feed is a directory with atoms.ndjson (signed credentials, one per
line) and index.json (owner, updated, count, SHA2-256 digest of the
atoms), signed with the same eddsa-jcs-2022 Data Integrity proof as
atoms. The format and its all-or-nothing verification live in the pure
core (trustgraph_core::feed), exposed as api::build_feed/verify_feed and
buildFeed/verifyFeed in the WebAssembly and Node packages.

The CLI gains `trust publish [--out DIR] [--well-known] [--as KEY]`,
`trust follow`, `unfollow`, `following` and `pull`, fetching over HTTPS
with ureq 3 (rustls, blocking), or from local paths and file:// URLs.
Bare domains resolve to /.well-known/trust/index.json. following.json in
the home directory records ETags (If-None-Match), digests, the pinned
owner and the last `updated` time, so unchanged feeds cost one 304 and
old copies can't be replayed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
…bsuvj4

# Conflicts:
#	README.md
#	doc/plan/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants