Repository navigation
Share over HTTPS: signed static feeds, trust publish / follow / pull - #21
Open
harlantwood wants to merge 4 commits into
Open
harlantwood wants to merge 4 commits into
harlantwood wants to merge 4 commits into
Conversation
A feed is a directory with atoms.ndjson (signed credentials, one per line) and index.json (owner, updated, count, SHA2-256 digest of the atoms), signed with the same eddsa-jcs-2022 Data Integrity proof as atoms. The format and its all-or-nothing verification live in the pure core (trustgraph_core::feed), exposed as api::build_feed/verify_feed and buildFeed/verifyFeed in the WebAssembly and Node packages. The CLI gains `trust publish [--out DIR] [--well-known] [--as KEY]`, `trust follow`, `unfollow`, `following` and `pull`, fetching over HTTPS with ureq 3 (rustls, blocking), or from local paths and file:// URLs. Bare domains resolve to /.well-known/trust/index.json. following.json in the home directory records ETags (If-None-Match), digests, the pinned owner and the last `updated` time, so unchanged feeds cost one 304 and old copies can't be replayed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
…bsuvj4 # Conflicts: # README.md # doc/plan/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Roadmap Phase 3. You can now publish your signed atoms as a static feed that any web host can serve (GitHub Pages included), and follow other people's feeds over HTTPS. Every atom is verified before it reaches the store. The feed format and its verification live in the pure core and are exposed to JS as
buildFeed/verifyFeed.Feed format (core,
trustgraph_core::feed, documented in doc/feeds.md)atoms.ndjson(signed Trust Atom credentials, one per line, astrust addstores them) andindex.json(type: TrustFeedIndex,ownerDID,updatedpassed in by the caller,atoms.count,atoms.digest= SHA2-256 multihashQm…of the exact bytes ofatoms.ndjson).eddsa-jcs-2022Data Integrity code as atoms (credential::sign/verify). All signing goes through two private functions,secureandcheck, so a later format PR can change it in one place.feed::verifyis all or nothing. It checks the index proof and that the signer isowner, the type, the digest, every atom's proof, that every atom was issued byowner, and the count.feed::builddrops duplicates and refuses unsigned atoms or atoms issued by anyone else.Error::InvalidFeed.API and bindings
api::build_feed(credentials, secret, updated) -> {index, atoms}andapi::verify_feed(index, atoms) -> {valid, owner?, updated?, ids?, atoms?, error?}. Likeverify,verify_feednever throws.buildFeed,verifyFeed) and node, with types inbindings/trustgraph.d.ts. They are covered bytests/js/smoke.mjsandtests/js/types.ts.CLI
trust publish [--out DIR] [--well-known] [--as KEY](--asis an alias of--key) writes every signed atom in the store whose source is your DID. Unsigned atoms are skipped, with a note on stderr.--well-knownwritesDIR/.well-known/trust/and also an emptyDIR/.nojekyll, because GitHub Pages otherwise does not serve dot-directories. Files are written atoms first, each via a temp file and rename.trust follow <feed> [--no-pull],trust unfollow <feed>,trust following,trust pull [feed].https://domain/.well-known/trust/index.json), a URL with no path (same well-known location), a directory URL (resolved to…/index.json), or a local path orfile://URL.--no-pullis given.pullwith several feeds reports each one and exits 1 if any failed.ureq3: blocking, rustls with webpki roots, no tokio. It takes proxies from the environment and limits sizes to 1 MiB for the index and 256 MiB for the atoms. Plainhttp://is accepted because signatures, not TLS, protect the content. Bare domains always use https.following.jsonin the CLI home records, per feed:ETag, sent back asIf-None-Match, so an unchanged feed costs one 304;atoms.ndjsonis not re-downloaded when the index is unchanged on hosts without ETags;updatedtime, so an older index cannot be replayed (no rollbacks).This state is only updated after the whole feed verifies.
Docs
doc/architecture.mdand the core README. One row was added to the roadmap's "What has shipped" table.Verified
cargo fmt --all --checkcargo clippy --workspace --all-targets --lockedandcargo clippy -p trustgraph-core --all-targets --locked, both withRUSTFLAGS=-D warningsRUSTDOCFLAGS=-D warnings cargo doc --workspace --no-deps --lockedscripts/check-core-purity.shcargo test --workspace --lockedon stable and on 1.85 (MSRV)crates/trustgraph-cli/tests/feeds.rs.TcpListenerthread with ETag support). There is no external network..well-knowndiscovery and HTTP 404.scripts/build-wasm-package.sh(574 KiB, under budget), and smoke tests on the node and web builds. Node job:pnpm run build:node && pnpm run test:node.pnpm run typecheck.trust follow example.comthrough the session proxy gets a real HTTPS 404, which shows that TLS and the proxy work.Not in this PR
did:web, so organisations can sign feeds with their own domain. It is only mentioned as a follow-up.ureq'splatform-verifierfeature) instead of webpki roots, if TLS-intercepting corporate proxies turn out to matter.🤖 Generated with Claude Code
https://claude.ai/code/session_01UdKneRqUnsbv1t7NyvcT68
Generated by Claude Code