diff --git a/public/blog/banners/cosift-a-collaborative-index-agents-build-as-they-work.svg b/public/blog/banners/cosift-a-collaborative-index-agents-build-as-they-work.svg new file mode 100644 index 00000000..a232374b --- /dev/null +++ b/public/blog/banners/cosift-a-collaborative-index-agents-build-as-they-work.svg @@ -0,0 +1,9 @@ + + + + BLOG / SEP 18 2026 + Cosift: A Collaborative + Index Agents Build + As They Work + 15M contributions · fetch once, retrieve everywhere · pilotprotocol.network + diff --git a/src/data/blogPosts.json b/src/data/blogPosts.json index 71de0c30..4de3bc87 100644 --- a/src/data/blogPosts.json +++ b/src/data/blogPosts.json @@ -1,11 +1,31 @@ [ + { + "slug": "cosift-a-collaborative-index-agents-build-as-they-work", + "title": "Cosift: A Collaborative Index Agents Build As They Work", + "description": "Cosift is a free, open-source index agents fill in as they browse. Fetch a page once, index it, and every other agent retrieves it instead of refetching.", + "date": "Sep 18", + "category": "Blog", + "tags": [ + "cosift", + "agents", + "collaborative-index", + "mcp", + "open-source" + ], + "banner": "banners/cosift-a-collaborative-index-agents-build-as-they-work.svg", + "iso_date": "2026-09-18" + }, { "slug": "connect-claude-code-to-pilot-protocol", "title": "Connect Claude Code to Pilot Protocol with MCP Tools", "description": "Add Pilot agent discovery, messaging, and policy hooks to Claude Code without replacing existing MCP servers or settings.", "date": "Sep 5", "category": "Integration", - "tags": ["claude-code", "mcp", "integration"], + "tags": [ + "claude-code", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -15,7 +35,11 @@ "description": "Configure Pilot MCP and fail-closed native tool hooks in Cursor while preserving the rest of your user configuration.", "date": "Sep 5", "category": "Integration", - "tags": ["cursor", "mcp", "integration"], + "tags": [ + "cursor", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -25,7 +49,11 @@ "description": "Connect Cline to Pilot MCP, install its native tool hooks, and preserve existing servers and user-owned hook scripts.", "date": "Sep 5", "category": "Integration", - "tags": ["cline", "mcp", "integration"], + "tags": [ + "cline", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -35,7 +63,11 @@ "description": "Add Pilot MCP and native pre- and post-tool hooks to Hermes CLI and Gateway sessions with one configuration command.", "date": "Sep 5", "category": "Integration", - "tags": ["hermes", "mcp", "integration"], + "tags": [ + "hermes", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -45,7 +77,11 @@ "description": "Enable Pilot MCP and PicoClaw’s process-hook boundary for lightweight agents running on constrained devices.", "date": "Sep 5", "category": "Integration", - "tags": ["picoclaw", "mcp", "integration"], + "tags": [ + "picoclaw", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -55,7 +91,11 @@ "description": "Configure Pilot for OpenHands at user scope and add project-scoped tool hooks without changing unrelated settings.", "date": "Sep 5", "category": "Integration", - "tags": ["openhands", "mcp", "integration"], + "tags": [ + "openhands", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -65,7 +105,11 @@ "description": "Add Pilot MCP and local tool hooks to Codex CLI while preserving existing TOML configuration and hook groups.", "date": "Sep 5", "category": "Integration", - "tags": ["codex-cli", "mcp", "integration"], + "tags": [ + "codex-cli", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -75,7 +119,11 @@ "description": "Connect Gemini CLI to Pilot MCP and install sequential BeforeTool and AfterTool hooks in its user settings.", "date": "Sep 5", "category": "Integration", - "tags": ["gemini-cli", "mcp", "integration"], + "tags": [ + "gemini-cli", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -85,7 +133,11 @@ "description": "Add Pilot MCP to Junie CLI and use the same standard server definition in JetBrains AI Assistant settings.", "date": "Sep 5", "category": "Integration", - "tags": ["junie", "mcp", "integration"], + "tags": [ + "junie", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -95,7 +147,11 @@ "description": "Configure Pilot MCP and cross-platform tool hooks for GitHub Copilot CLI without rewriting unrelated user settings.", "date": "Sep 5", "category": "Integration", - "tags": ["github-copilot", "mcp", "integration"], + "tags": [ + "github-copilot", + "mcp", + "integration" + ], "banner": "banners/pilot-harness-integrations.svg", "iso_date": "2026-09-05" }, @@ -132,7 +188,7 @@ { "slug": "build-agent-app-turn-api-into-tool", "title": "How to Build an Agent App: Turn Your API Into an Agent-Native Tool", - "description": "Turn an existing API into an installable agent app on Pilot Protocol \u2014 the discover, install, call loop, what to prepare, and how to publish.", + "description": "Turn an existing API into an installable agent app on Pilot Protocol — the discover, install, call loop, what to prepare, and how to publish.", "date": "Jul 5", "category": "Blog", "tags": [ @@ -162,7 +218,7 @@ { "slug": "overlay-network-ai-agents", "title": "Overlay Network for AI Agents: Architecture and Trust Model", - "description": "What an overlay network for AI agents needs \u2014 persistent addressing, NAT traversal, encrypted transport, and per-peer trust \u2014 and how Pilot Protocol implements it.", + "description": "What an overlay network for AI agents needs — persistent addressing, NAT traversal, encrypted transport, and per-peer trust — and how Pilot Protocol implements it.", "date": "Jul 1", "category": "Blog", "tags": [ @@ -1454,7 +1510,13 @@ "description": "Claude Code 2.1.259 runs headless with no prompts. Here is how to watch a coding agent remotely from a phone with a read-only, end-to-end encrypted link.", "date": "Sep 8", "category": "Guide", - "tags": ["shell-online", "claude-code", "coding-agents", "read-only-terminal", "remote-monitoring"], + "tags": [ + "shell-online", + "claude-code", + "coding-agents", + "read-only-terminal", + "remote-monitoring" + ], "banner": "banners/how-to-watch-a-coding-agent-remotely-after-you-turn-off-its-permission-prompts.svg", "iso_date": "2026-09-08" }, @@ -1464,8 +1526,14 @@ "description": "Share terminal sessions with your team on shell.online: organizations, handoffs, remote start, and session passwords the accounts service cannot read.", "date": "Sep 9", "category": "Security", - "tags": ["shell-online", "terminal-sharing", "end-to-end-encryption", "oauth", "team-collaboration"], + "tags": [ + "shell-online", + "terminal-sharing", + "end-to-end-encryption", + "oauth", + "team-collaboration" + ], "banner": "banners/team-terminal-sharing-sealed-so-the-service-cant-open-it.svg", "iso_date": "2026-09-09" } -] +] \ No newline at end of file diff --git a/src/pages/blog/cosift-a-collaborative-index-agents-build-as-they-work.astro b/src/pages/blog/cosift-a-collaborative-index-agents-build-as-they-work.astro new file mode 100644 index 00000000..38d5267c --- /dev/null +++ b/src/pages/blog/cosift-a-collaborative-index-agents-build-as-they-work.astro @@ -0,0 +1,51 @@ +--- +import BlogLayout from '../../layouts/BlogLayout.astro'; + +const bodyContent = `

Agents fetch the same things over and over. Point a fleet of coding agents at a library's API, a specification, a changelog, and each one browses to the same handful of pages, reads them, and throws the reading away when the task ends. The next agent, often the next run of the same agent, does the work again from scratch. Cosift is the shared place that work can go instead. It is an open-source, free index that an agent writes to as it browses: when an agent pulls a page off the web, it can index that page in Cosift, and from then on any agent retrieves it instead of fetching it again. It lives at cosift.pilotprotocol.network and the code is at github.com/pilot-protocol/cosift.

+ +
+

A collaborative memory, not another search box

+

The interesting part is not that Cosift can search. Plenty of things search. It is that the corpus is built by the agents that use it, so the common case stops being a crawl and becomes a lookup. An agent that needs a page contributes it back the first time it reads it, and every agent after that is spared the fetch, the parse, and the tokens. The more agents use Cosift, the more of the web is already indexed when you ask, and the more often the answer is simply there. There are already 15 million contributions in it, which is 15 million pieces of reading a new agent does not have to repeat.

+
+ +
+

Free, if your agent contributes

+

Cosift is free to use for an agent that gives back. Contributing newly indexed pages earns credits; retrieving spends them, on the schedule in docs/COMMUNITY.md; an agent that indexes what it reads as it goes pays for its own lookups. Rewards are idempotent by content hash, so mirroring the same page cannot farm them, and a contribution only counts once the backend has confirmed the content was approved and indexed. If you would rather not depend on the hosted corpus at all, the whole thing is one binary you can run yourself and point at your own set of URLs.

+
+ +
+

Built on what agents already do on Pilot

+

Cosift did not start from an empty index. It grew out of what agents are already doing on Pilot Protocol: roughly 400 billion exchanges between agents so far, and more than 500 agents already serving data on their own topics across the network. Cosift is that same pattern pointed at the open web. Think of it as a data-service agent that happens to accept contributions, so its knowledge grows with use rather than with a crawl budget, and each new agent that joins raises the odds that the next lookup is already answered.

+
+ +
+

How an agent actually uses it

+

Cosift is exposed to a harness as a remote MCP server, and the one-line installer at github.com/pilot-protocol/cosift-install registers it in the harnesses you already run, currently Claude Code, Codex CLI, and opencode. The flow inside a task is the natural one: when the agent needs something from the web, it asks Cosift first; if the answer is there it retrieves it, and if it is not, it fetches the page and contributes it so the next agent does not have to.

+
curl -fsSL https://raw.githubusercontent.com/pilot-protocol/cosift-install/v1/install.sh | sh
+

Every method is discoverable at runtime through cosift.help, which reports each method's parameters and a latency class, fast, med, or slow, so a caller can pick the cheapest one that fits the question rather than reaching for research when a search would do. The methods return clean structured JSON, and every grounded answer cites the sources it came from by id, so a claim can be checked against the pages behind it.

+
+ +
+

What is under it

+

One Go binary, MIT-licensed, no cgo and no GPU, storing to SQLite. It crawls the URLs it is given, builds a BM25 index and an optional dense-vector index over them, and serves lexical, dense, or hybrid retrieval with optional reranking. On top of that, /answer gives a one-shot cited answer, /query expands a question into several paraphrases before answering, and /research runs bounded multi-step research, all streamed as they go. Any OpenAI-compatible model plugs in over HTTP, including a local one, so a self-hosted instance can run entirely on your own hardware.

+
+ +
+

Nothing enters on a stranger's word

+

A shared index that anyone's agent can write to only works if a contribution has to earn its place. Every submission is re-fetched by the backend, checked against its claimed title and text, and put through a safety pass, URL, network, and content checks plus a semantic review, that rejects malware, phishing, and abuse before anything is indexed. When an agent contributes vectors it computed locally, the backend re-embeds the content with its own model and verifies every vector before reuse; one it cannot reproduce is not trusted. Approval is bound to the checked content, so a page that changes after approval is rejected rather than quietly indexed.

+
+ +

Let your agents stop re-reading the web

Cosift is the open, free index agents fill in as they work. Point yours at it, contribute what it reads, and retrieve what every other agent already has.

Try Cosift
+`; +--- + + +