Publish the read side as @profullstack/crawlproof, and make crawlproof a command - #241
Merged
Conversation
This was referenced Sep 6, 2026
Merged
The CLI had no bin and was not on PATH, so every command in its own help and in this PR's description was a lie unless you first cd'd into the checkout and typed `npm run cli --`. `crawlproof dashboard` now works from anywhere. bin/crawlproof.mjs resolves the repo from its own location and runs tsx against cli/index.ts, preferring the checkout's tsx and falling back to npx so a fresh clone still starts. It relays the child's signal rather than inventing an exit code, so a TUI killed with ctrl+c does not look like a clean exit. The token now also falls back to the `token` field of ~/.crawlproof.json, the same shape and the same reasoning as ~/.coinpay.json: using the CLI should not be conditional on remembering to export a secret first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD
A dashboard you can only run by cd'ing into a Next.js checkout is not installed anywhere. packages/cli publishes the two commands a box actually wants — `dashboard` and `stats` — so cli-tools can vendor it the way it vendors hqtui, and any machine can `npm i -g @profullstack/crawlproof`. Only those two, deliberately. Both are token-authed and pure HTTP, which is what makes them publishable: `audit` needs the audit engines and their model SDKs, and `sweep` needs a cron secret. Those stay in the repo. The package is a **build artifact**, not a second copy. lib/dashboard/* and cli/dashboard.ts stay where the suite already covers them and where the in-repo CLI already imports them; build.mjs bundles the same files with esbuild and leaves hqtui and the CoinPay SDK external, since both have their own release cadence. There is one implementation and it cannot drift. The stats printer moves to lib/dashboard/stats-text.ts and returns a string rather than writing to stdout, because two CLIs print it now and a printer that owns the process can be neither shared nor tested. Verified from the bundle against production: `crawlproof stats crawlproof.com` reads the token out of ~/.crawlproof.json with nothing exported, and the dashboard collects all 52 sites with zero source errors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD
ralyodio
force-pushed
the
crawlproof-cli-package
branch
from
September 6, 2026 09:37
bfeec68 to
988b04b
Compare
ThreatCrush Security Scan39 finding(s) HIGH/CRITICAL: 2 | MEDIUM: 28 | LOW: 9
Snippets are redacted; ThreatCrush never prints matched credential material. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if (site) query.set("site", site); | ||
|
|
||
| const res = await fetch(`${apiBase(args)}/api/tracker/v1/stats?${query.toString()}`, { | ||
| headers: { Authorization: `Bearer ${token}`, Accept: "application/json" }, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #240 — merge that first. This branch adds only the last two commits.
crawlproof dashboardwas a lie: there was nocrawlproofon PATH and the CLI had nobin, so every command in its own help needed you to cd into the checkout and typenpm run cli --. Two changes fix that.crawlproofis a command nowbin/crawlproof.mjsresolves the repo from its own location and runs tsx againstcli/index.ts, preferring the checkout's tsx and falling back to npx so a fresh clone still starts. It relays the child's signal rather than inventing an exit code, so a TUI killed with ctrl+c does not look like a clean exit.The token also falls back to the
tokenfield of~/.crawlproof.json— same shape and same reasoning as~/.coinpay.json. Using the CLI should not be conditional on remembering to export a secret first.packages/cli→@profullstack/crawlproofSo cli-tools can vendor it the way it vendors hqtui, and any box can
npm i -g @profullstack/crawlproof.Two commands only, deliberately:
dashboardandstats. Both are token-authed and pure HTTP, which is exactly what makes them publishable —auditneeds the audit engines and their model SDKs,sweepneeds a cron secret. Those stay in the repo.It is a build artifact, not a second copy.
lib/dashboard/*andcli/dashboard.tsstay where the suite already covers them and where the in-repo CLI already imports them;build.mjsbundles the same files with esbuild (44kb) and leaves hqtui and the CoinPay SDK external, since both have their own release cadence. One implementation, no drift.The stats printer moves to
lib/dashboard/stats-text.tsand returns a string instead of writing to stdout — two CLIs print it now, and a printer that owns the process can be neither shared nor tested.Verified
From the built bundle, not the source:
crawlproof stats crawlproof.comagainst production, reading the token from~/.crawlproof.jsonwith nothing exported: 172 visitors, 58 pageviews, sources and pages as expected.crawlproof dashboard --jsonagainst a local server on the production environment: all 52 sites, all three feeds, zero source errors.Full suite green: 2,125 tests (19 new),
tsc --noEmitclean.Note
A bare flag takes the next bare word as its value, so
crawlproof stats --json site.comswallows the site. That is the existing parser's behaviour, shared with the in-repo CLI; this PR pins it in a test so it stays a decision rather than becoming a surprise, but does not change it.🤖 Generated with Claude Code
https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD