fix(integrations): list the toolkits the public catalog publishes by samkujovich · Pull Request #1249 · ArcadeAI/docs

samkujovich · 2026-10-19T18:52:44Z

Part of GRO-615

What/why

The integrations index on docs.arcade.dev and arcade.dev/tools disagree about which MCP servers Arcade offers. arcade.dev/tools is right: it is built from the live public catalog. The docs index was built from the TOOLKITS list in @arcadeai/design-system, which is hand-maintained and pinned here at 7.11.0. So docs lists Twilio as a "Coming soon" Verified server after it launched, shows about 37 coming-soon toolkits that no one can use, and lists Math, which arcade.dev/tools hides as internal. After this change the docs index lists the same toolkits as arcade.dev/tools, with the same badges.

Codebase changes

The index now reads the per-toolkit files that generate-toolkit-docs.yml writes every night. That job reads experience-api's /api/public/*, the same catalog arcade.dev/tools snapshots, so both sites share one source. Each generated file already carries the card fields (label, icon, category, type, BYOC/Pro), so design-system no longer drives this page. The category sidebar (sync-toolkit-sidebar.ts) reads the same files for its labels, hidden flag and category, so the index and sidebar can't disagree. Partner toolkits (Nimble, Tavily) are still merged in from data/partner-toolkits.ts. Math is hidden through the catalog (ArcadeAI/monorepo#5848), so docs keeps no list of its own. The hand-written Workday "launched" override is gone, because it only patched a stale design-system flag.

Proof

Behavior	How it was proven
Index lists exactly the toolkits arcade.dev/tools lists	Real: resolved the index with `getToolkitsWithDocsLinks` + `resolveIndexToolkits` and diffed it against ArcadeAI/www `src/content/tools-catalog.json` On current `main`'s data: the same 84 curated toolkits, with Bright Data, Clickhouse, MongoDB and Postgres as `arcade`, Twilio listed, and no Math. The only gap is Google Forms, newly in the catalog, whose data is in the open nightly PR #1258
Every card links to a real page	Real: same run, 0 cards without a page; existing `tests/integration-index-links.test.ts` passes
Card fields come from the generated file; partners kept	Unit: new `tests/integration-catalog.test.ts`
Sidebar matches the generated data	Unit: `sync-toolkit-sidebar.test.ts` (58 passed). Real: before switching, all 125 design-system labels matched the generated files; regenerating the sidebar changed one line, removing Math
Built page renders the new list	Real: `bun run build` on the Fumadocs `main`; the prerendered `integrations.html` carries 123 catalog cards plus the 2 partners, with no Math or coming-soon cards, and no Math link or page
Deployed page shows the new list	Real: Playwright against this PR's Vercel preview and production at 1440px. All servers 169 → 125; search "google" 15 → 14 (Google Forms gone); Developer Tools 30 → 19 (Math, Arcade Engine API and the coming-soon cards gone). Screenshots below
Nothing else regressed	`pnpm test` (899 passed, 2 skipped), `pnpm lint`, `tsc --noEmit`

Before (docs.arcade.dev)	After (Vercel preview)
All servers: 169, including design-system-only entries	All servers: 125, the published catalog
Search "google": Google Forms shown as "Coming soon"	Search "google": only the published Google servers
Developer Tools: Math, Arcade Engine API, and coming-soon cards	Developer Tools: only published toolkits

Additional notes

Docs picks up catalog changes when the nightly generated-docs PR merges; that PR needs a human push before its required checks run.
toolkit-static-params.ts still falls back to design-system for a toolkit with no generated file. No such toolkit exists today.
Other Math mentions in docs: the CrewAI guide's next steps suggested a "Math" toolkit and now suggests GoogleCalendar or Linear; the nightly already removed Math's page from public/llms.txt. The deploy and Helm pages keep [arcade_math] as a toolPackages example. The package still ships, so it stays accurate.
Reviewer decision: dropping the coming-soon cards and hiding Math was chosen so that docs matches arcade.dev/tools exactly.

~ 🤖 sam-i-am, Sam’s Agent

Note

Medium Risk
Changes which integrations appear on docs (fewer “coming soon” and internal entries like Math) and how the sidebar is built; behavior is covered by new/updated tests but affects a high-traffic docs surface.

Overview
Aligns the docs integrations index and category sidebar with the same nightly-generated toolkit catalog that powers arcade.dev/tools, instead of the hand-maintained @arcadeai/design-system TOOLKITS list.

getToolkitsWithDocsLinks now loads per-toolkit JSON from toolkit-docs-generator via loadAllToolkitData, maps card fields with toIntegrationCatalog, and still appends partner toolkits. Design-system enrichment, docs-link backfill, and the Workday “launched” isComingSoon override are removed. sync-toolkit-sidebar.ts uses the same JSON for category, isHidden, and labels (no design-system priority or setToolkitsForTesting). Math is dropped from the public list by setting isHidden: true in math.json and regenerating sidebar/llms.txt; the CrewAI guide’s next steps now mention MCP servers like GoogleCalendar/Linear instead of Math.

Adds tests/integration-catalog.test.ts and updates sidebar tests for the JSON-only behavior.

^{Reviewed by Cursor Bugbot for commit 7b1d264. Bugbot is set up for automated code reviews on this repo. Configure here.}

The integrations index built its list from the design-system TOOLKITS array, so it drifted from production whenever the pinned package fell behind: Twilio showed as a coming-soon Verified server after it launched, and Workday needed a hand-written override. Build the index from the nightly toolkit-docs-generator output instead. That data comes from the same experience-api public catalog that arcade.dev/tools snapshots, so both sites list the same toolkits with the same badges. Coming-soon placeholders that exist only in the design system are no longer listed, and Math and ArcadeEngineApi are hidden to match arcade.dev/tools.

vercel · 2026-10-19T18:52:50Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Oct 22, 2026 10:19pm UTC

…x-from-public-catalog # Conflicts: # lib/integration-catalog.ts

laughnan

lgtm

sdserranog

Left a couple of small comments inline: one question about the internal list and one about the sidebar still using design-system.

sdserranog · 2026-10-20T20:52:36Z

-  LAUNCHED_TOOLKIT_IDS.has(normalizeToolkitId(toolkit.id))
-    ? { ...toolkit, isComingSoon: false }
-    : toolkit;
+const INTERNAL_TOOLKIT_IDS: ReadonlySet<string> = new Set([


ArcadeEngineApi isn't in the catalog, so I think we can remove it from this list.

For Math, should we keep its docs page? If not, we could mark it as hidden in the catalog and drop this list. Worth a Linear ticket either way, so docs stops keeping its own list.

Agreed, and the list is gone now (acbb4ad). ArcadeEngineApi isn't in the live catalog (129 toolkits, none named that). For Math, we're hiding it in the toolkit metadata instead: ArcadeAI/monorepo#5848 sets isHidden: true, and the index already skips hidden toolkits, so docs doesn't keep a list of its own. Its docs page goes away with it. That PR sits under GRO-615 rather than a separate ticket.

~ 🤖 sam-i-am, Sam’s Agent

sdserranog · 2026-10-20T20:52:36Z

+  partners: readonly ToolkitWithDocsLink[]
+): ToolkitWithDocsLink[] => [
+  ...[...generated]
+    .filter((toolkit) => !INTERNAL_TOOLKIT_IDS.has(toolkit.id))


This hides Math on the index, but the sidebar still reads from design-system, so Math keeps showing there and the two can disagree on labels too. Could we fix the sidebar in this PR as well so both match?

Done in acbb4ad. sync-toolkit-sidebar.ts now takes labels, the hidden flag and the category only from the generated toolkit files, the same source as the index, and no longer imports design-system. Before switching, I compared design-system and the generated files: all 125 labels match and nothing is hidden only in design-system. Regenerating the sidebar changed exactly one line: Math's entry is gone, since math.json now carries isHidden: true (what the nightly writes once ArcadeAI/monorepo#5848 is live).

~ 🤖 sam-i-am, Sam’s Agent

…ath there The sidebar took toolkit labels, the hidden flag and a fallback category from design-system. It now reads them from the generated toolkit files, the same source as the index, so the two cannot disagree. Math is hidden in the toolkit metadata (ArcadeAI/monorepo#5848), so the index no longer needs its own list of toolkits to leave out. math.json carries the flag now, which the nightly writes once the catalog serves it. Part of GRO-615

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5f2d747. Configure here.}

The CrewAI guide pointed readers at a "Math" toolkit, which the catalog now hides, and at "Search", which is not a current toolkit name. Part of GRO-615

…index-from-public-catalog

getToolkitFiles lowercases file names before reading them, so a fixture saved as GoogleSearch.json was never found on a case-sensitive file system: the fallback-label test failed on Linux CI, and the hidden test passed only because its file went missing. Fixtures now use lowercase file names with the toolkit ID inside, and the hidden test keeps a visible sibling so it fails if the flag stops working. Part of GRO-615

…x-from-public-catalog # Conflicts: # public/llms.txt

sdserranog

LGTM!

vercel Bot deployed to Preview October 19, 2026 18:56

View deployment

test(integrations): use a verified fixture in the catalog test

62a75a5

vercel Bot deployed to Preview October 19, 2026 20:47

View deployment

This was referenced Oct 19, 2026

refactor(integrations): stop relying on the coming-soon flag #1251

Closed

refactor(integrations): stop listing the community designation #1252

Open

samkujovich marked this pull request as ready for review October 20, 2026 02:24

Merge remote-tracking branch 'origin/main' into fix/integrations-inde…

7f35ce2

…x-from-public-catalog # Conflicts: # lib/integration-catalog.ts

vercel Bot deployed to Preview October 20, 2026 02:42

View deployment

samkujovich requested review from a team and sdserranog October 20, 2026 19:46

laughnan approved these changes Oct 20, 2026

View reviewed changes

sdserranog reviewed Oct 20, 2026

View reviewed changes

sdserranog requested changes Oct 20, 2026

View reviewed changes

samkujovich and others added 2 commits October 20, 2026 14:31

🤖 Regenerate LLMs.txt

5f2d747

vercel Bot deployed to Preview October 20, 2026 21:33

View deployment

cursor Bot reviewed Oct 20, 2026

View reviewed changes

Comment thread toolkit-docs-generator/tests/scripts/sync-toolkit-sidebar.test.ts

samkujovich added 2 commits October 20, 2026 15:42

docs(crewai): suggest real MCP servers instead of Math in next steps

fd685ae

The CrewAI guide pointed readers at a "Math" toolkit, which the catalog now hides, and at "Search", which is not a current toolkit name. Part of GRO-615

Merge remote-tracking branch 'origin/fix-1249' into fix/integrations-…

c0d33f2

…index-from-public-catalog

vercel Bot deployed to Preview October 20, 2026 22:43

View deployment

vercel Bot deployed to Preview October 20, 2026 22:51

View deployment

Merge remote-tracking branch 'origin/main' into fix/integrations-inde…

1df0b34

…x-from-public-catalog # Conflicts: # public/llms.txt

samkujovich requested a review from sdserranog October 22, 2026 22:18

vercel Bot deployed to Preview October 22, 2026 22:19

View deployment

sdserranog approved these changes Oct 23, 2026

View reviewed changes

Navigation Menu

Repository navigation

fix(integrations): list the toolkits the public catalog publishes - #1249

fix(integrations): list the toolkits the public catalog publishes#1249
samkujovich wants to merge 9 commits into
mainfrom
fix/integrations-index-from-public-catalog

samkujovich commented Oct 19, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

laughnan left a comment

Uh oh!

sdserranog left a comment

Uh oh!

sdserranog Oct 20, 2026

Uh oh!

samkujovich Oct 20, 2026

Uh oh!

sdserranog Oct 20, 2026

Uh oh!

samkujovich Oct 20, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

sdserranog left a comment

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

samkujovich commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What/why

Codebase changes

Proof

Additional notes

Uh oh!

vercel Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laughnan left a comment

Choose a reason for hiding this comment

Uh oh!

sdserranog left a comment

Choose a reason for hiding this comment

Uh oh!

sdserranog Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

samkujovich Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

sdserranog Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

samkujovich Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sdserranog left a comment

Choose a reason for hiding this comment

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

samkujovich commented Oct 19, 2026 •

edited

Loading

vercel Bot commented Oct 19, 2026 •

edited

Loading