From 22f670e41078843710b8e7758b411f0def7ff9af Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Sun, 6 Sep 2026 14:39:30 +0000 Subject: [PATCH] Document the CLI, on the site and in the repo The CLI shipped, went on npm, got a cli-tools command and a moshcode entry, and was documented in none of the places someone would look for it: crawlproof.com had no page, and the repo README did not mention it existed. The only writeup was inside the published package, which you have to install before you can read. /docs/cli covers install and auth, the five dashboard screens and their keys, stats, running an ad, and the bearer-token routes underneath, so anything that is not this CLI can call them too. It also states what the ROI numbers mean, because those are the figures someone will quote at somebody else: self-deal is not revenue, personal money is not business cost, everything is a monthly rate prorated onto the window, and the bank window is not the traffic range. Along with what the dashboard refuses to claim -- a site that did not answer is missing rather than zero, and a visitor is any non-crawler hit, which is why the per-pageview figure sits beside it. The README section says where the package comes from and that it is a build artifact of lib/dashboard/* rather than a second copy, so nobody edits the wrong one. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD --- README.md | 37 +++++ app/(marketing)/docs/cli/page.tsx | 227 ++++++++++++++++++++++++++++++ app/(marketing)/docs/page.tsx | 12 +- 3 files changed, 273 insertions(+), 3 deletions(-) create mode 100644 app/(marketing)/docs/cli/page.tsx diff --git a/README.md b/README.md index 0a372eea..20f3bb87 100644 --- a/README.md +++ b/README.md @@ -127,6 +127,43 @@ https:///api/coinpay/webhook Credit purchases are created through `/api/credits/create-invoice`; successful webhook delivery finalizes the purchase and increments `profiles.credits_balance`. +## CLI + +`crawlproof` reads the account from a terminal: traffic across every site you +own, ad delivery, and the bank feed behind it. Published from +[`packages/cli`](packages/cli) as `@profullstack/crawlproof`, and documented at +[/docs/cli](https://crawlproof.com/docs/cli). + +```sh +npm install -g @profullstack/crawlproof +echo '{ "token": "crp_…" }' > ~/.crawlproof.json && chmod 600 ~/.crawlproof.json + +crawlproof dashboard # five live screens: ROI, Traffic, Ads, Money, Spend +crawlproof dashboard --range=1m --json | jq .roi.derived +crawlproof stats example.com # sources, referrers, top pages as text +crawlproof ad https://example.com/launch +crawlproof ads pause crawlproof-ad-144 +``` + +The package ships only the commands that are pure HTTP against a bearer token. +`audit` needs the audit engines and their model SDKs and `sweep` needs a cron +secret, so both stay in-repo behind `npm run cli --`. + +It is a **build artifact**, not a second copy: `packages/cli/build.mjs` +esbuild-bundles `lib/dashboard/*` and `cli/dashboard.ts` where they already +live and where the test suite already covers them, leaving hqtui and the +CoinPay SDK external. Rebuild with `node packages/cli/build.mjs`. + +The endpoints underneath are the same ones anything else can call: + +| Route | Answers | +| --- | --- | +| `GET /api/tracker/v1/sites` | the projects this token can read | +| `GET /api/tracker/v1/stats?site=&range=&who=` | who arrived and from where | +| `GET /api/ads/v1/earnings?days=` | ad delivery and money, both sides | +| `GET/POST /api/ads/v1/campaigns` | list and run campaigns | +| `GET/POST /api/ads/v1/slots` | list and create publisher slots | + ## Product flows ### Free audit diff --git a/app/(marketing)/docs/cli/page.tsx b/app/(marketing)/docs/cli/page.tsx new file mode 100644 index 00000000..584a3068 --- /dev/null +++ b/app/(marketing)/docs/cli/page.tsx @@ -0,0 +1,227 @@ +import Link from "next/link"; + +export const metadata = { + title: "CLI", + description: + "Install the CrawlProof CLI and read your traffic, ad delivery and spend from a terminal. Live dashboard, plain-text stats, and one-command ad campaigns.", + alternates: { canonical: "/docs/cli" }, +}; + +/** A command and what it is for, so the page reads as a reference rather than prose. */ +function Cmd({ children, note }: { children: string; note?: string }) { + return ( +
+ 
+        {children}
+
+ {note ?

{note}

: null} +
+ ); +} + +export default function CliDocsPage() { + return ( +
+

+ + ← Docs + +

+

CLI

+

+ crawlproof reads your account from a + terminal: who arrived on every site you own, what your ads delivered, + and — when a CoinPay merchant session is on the machine — what the bank + actually did. It is the same data the dashboard renders, without a + browser. +

+ +
+

Install

+ + npm install -g @profullstack/crawlproof + +

+ Authenticate with an API token from{" "} + Social → API tokens. Either export it, or drop it in{" "} + ~/.crawlproof.json so you never + have to think about it again. +

+ {`export CRAWLPROOF_TOKEN=crp_… + +# or, once: +echo '{ "token": "crp_…" }' > ~/.crawlproof.json && chmod 600 ~/.crawlproof.json`} +
+ +
+

The dashboard

+

+ Five live screens: ROI, Traffic,{" "} + Ads, Money and{" "} + Spend. It refreshes on a timer. +

+ + {`crawlproof dashboard +crawlproof dashboard --range=1m --who=all +crawlproof dashboard --sites=example.com,blog.example.com`} + +

+ Aliases: roi and{" "} + tui. On a machine with no terminal,{" "} + --json prints the same snapshot the + screens render. +

+ {`crawlproof dashboard --json | jq .roi.derived +crawlproof dashboard --json | jq '.sites[] | {site, visitors}'`} +
+ +
+

Traffic, as text

+

+ Sources, referrers and top pages for one site. Defaults to the last + day and humans only, because a launch is invisible inside a month of + crawler traffic. With a single project the site can be left out. +

+ {`crawlproof stats +crawlproof stats example.com --range=1w +crawlproof stats example.com --who=bots --json`} + 
{`example.com  1d  humans
+172 visitors, 58 pageviews
+
+Sources
+  Referral · example.com     163
+  Search · google              9
+
+Pages
+  /                           18
+  /pricing                     7`}
+
+ +
+

Run an ad

+

+ CrawlProof reads the page, writes the creatives and starts serving. A + URL that already has a live campaign gets that campaign back rather + than a duplicate, so running it twice is safe. +

+ + {`crawlproof ad https://example.com/launch +crawlproof ad https://example.com/launch --budget=500 --name "Launch" + +crawlproof ads +crawlproof ads show crawlproof-ad-144 +crawlproof ads pause crawlproof-ad-144 +crawlproof ads budget crawlproof-ad-144 250`} + +
+ +
+

What the numbers mean

+

+ Two rules run through the ROI arithmetic, and both exist because + breaking either produces a friendlier number that is false. +

+

+ Self-deal is not revenue. Where an account advertises + on its own slots, ad spend and ad earnings are one dollar moving + between two pockets. They appear under Internal and count as + neither cost nor revenue. +

+

+ Personal money is not business cost. A bank feed + carries groceries next to servers, so cost is the business scope only, + joined from each transaction’s account to that + account’s books. +

+

+ Everything is normalised to a monthly rate and then prorated onto the + window you picked, because burn is a rate. The bank window is not the + traffic range — bank data has no hourly resolution — so every panel + names its own basis. +

+

+ It also reports what it cannot know. A site that did not answer is + shown as missing rather than zero, a vendor list built from one page + of a longer ledger says so, and a fleet whose visits run far above its + pageviews says that too. A “visitor” is any hit not + classified as a crawler, which on a site with a machine-readable + endpoint runs orders of magnitude above the pages anyone read, so the + per-pageview figure sits beside the per-visitor one. +

+
+ +
+

Money screens

+

+ The Money and Spend screens read a bank and card feed through{" "} + + CoinPay + + . They need a merchant session, which{" "} + coinpay auth login writes to{" "} + ~/.coinpay.json. Without one the + other three screens still work and the money panels say what is + missing rather than showing zero. +

+ {`crawlproof dashboard --no-coinpay # skip them entirely`} +
+ +
+

Environment

+
+ + + + + + + + + + + + + + + + + + + + + +
VariableMeaning
CRAWLPROOF_TOKEN + API token. Falls back to the token{" "} + field of ~/.crawlproof.json;{" "} + --token beats both. +
CRAWLPROOF_SITE_URL + API base, default https://crawlproof.com. +
COINPAY_SESSION_TOKEN + Merchant JWT for the money screens. Defaults to{" "} + jwtToken in{" "} + ~/.coinpay.json. +
+
+
+ +
+

The API underneath

+

+ Every command is HTTP with a bearer token, so anything can call it. +

+ {`curl -H "Authorization: Bearer $CRAWLPROOF_TOKEN" \\ + https://crawlproof.com/api/tracker/v1/sites + +curl -H "Authorization: Bearer $CRAWLPROOF_TOKEN" \\ + "https://crawlproof.com/api/tracker/v1/stats?site=example.com&range=1d" + +curl -H "Authorization: Bearer $CRAWLPROOF_TOKEN" \\ + "https://crawlproof.com/api/ads/v1/earnings?days=30"`} +
+
+ ); +} diff --git a/app/(marketing)/docs/page.tsx b/app/(marketing)/docs/page.tsx index 8234d34f..2887ec12 100644 --- a/app/(marketing)/docs/page.tsx +++ b/app/(marketing)/docs/page.tsx @@ -3,7 +3,7 @@ import Link from "next/link"; export const metadata = { title: "Docs", description: - "CrawlProof developer documentation: AEO Score, drop-in stats tracker, autoblog webhook integration.", + "CrawlProof developer documentation: AEO Score, drop-in stats tracker, the crawlproof CLI, autoblog webhook integration.", alternates: { canonical: "/docs" }, }; @@ -26,6 +26,12 @@ const DOCS: DocCard[] = [ description: "Install the cookieless tracker, collect pageviews and interactions, and send custom events from any frontend stack. Also reachable at /docs/stats-tracker.", }, + { + href: "/docs/cli", + title: "CLI", + description: + "Read your traffic, ad delivery and spend from a terminal. Live five-screen dashboard, plain-text stats, and one-command ad campaigns over the same bearer-token API.", + }, { href: "/docs/autoblog-webhook", title: "Autoblog webhook", @@ -41,8 +47,8 @@ export default function DocsIndexPage() {

Developer-facing guides for the parts of CrawlProof you integrate with — the score that ranks your AEO health, the tracker that - watches your live AI traffic, and the webhook that delivers - autoblog posts to your CMS. + watches your live AI traffic, the CLI that reads all of it from a + terminal, and the webhook that delivers autoblog posts to your CMS.