test: cover stream handler, billing, limits, and permission evaluation; fix CI checks - #288
test: cover stream handler, billing, limits, and permission evaluation; fix CI checks#288usehoplite[bot] wants to merge 5 commits into
Conversation
Adds focused tests for the riskiest untested production paths in the CLI server, and fixes a bug they surfaced. - openai-compatible-stream: 28 tests for the SSE handler shared by every provider route (chunked content, reasoning deltas, streamed tool_calls assembly, dedup, error events, usage, non-streaming fallback, finish). Fixes a premature per-read flush that dropped tool calls whose arguments arrived across multiple SSE reads, returning a bogus empty-response error instead of executing the tool. - pricing: cost math, alias/fallback lookup, provider labels. - credit-meter: credit cost multipliers and soft-gate deduction paths (no balance, insufficient funds, decrement, DB fail-open). - token-budget: device id, daily token budget, opus caps, proxy usage aggregation and 48h pruning. - permission-manager: dangerous-command detection, read-only allowlist, session-level override, and agent ruleset precedence. - .hoplite/settings.json: record bun install as the setup command so fresh sandboxes get dependencies before tests run. Full suite with these files: 162 pass / 9 fail / 1 error. The 9 failures and 1 error are pre-existing (permission-prompt test drift vs the read-only allowlist, macOS/env-dependent speech tests, and subscription-check failing to load because the generated Prisma client is absent without DATABASE_URL); all 69 new tests pass. Co-authored-by: Yash Dewasthale <yashdev.yvd@gmail.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Fixes the three CI jobs (TypeScript Check, Lint, Test) that have been failing on every run, including on main: - bun.lock was stale (still referenced the deleted apps/video workspace), so 'bun install --frozen-lockfile' failed in ~2s on every job. Regenerated with the pinned bun 1.2.21 (pure lockfile repair). - Wire 'bun run db:generate' (new root script) with DATABASE_URL into the typecheck and test jobs: two packages import generated Prisma clients (CLI server -> src/generated, superdesign-db -> prisma/client) that were never generated in CI. - TypeScript errors (pre-existing): server (todowrite/question strict narrowing, missing MergeConnectorManager methods used by /connectors, test mock typing), cortex-sdk (v2/v3 AI SDK model types unified via GatewayModel plus defensive usage normalization). - Lint errors (pre-existing): supercode-cli-client (Link, entity escaping, purity-safe skeleton width, useSyncExternalStore for the mounted flag) and apps/web (unescaped entities in marketing pages, explicit any in chart/server-action code, setState-in-effect and purity fixes, motion slot component caching). - Test drift: permission-prompt tests asserted the pre-allowlist prompting behavior for read-only commands; speech tests re-imported the module mid-run with a replaced process.env (host-dependent ffmpeg check, TDZ flakes). Both rewritten to the intended contract, host-independent, static imports. Local verification with bun 1.2.21 (CI-pinned) and 'bun install --frozen-lockfile': - turbo typecheck: 18/18 packages - turbo lint: 17/17 packages - bun test: 288 pass, 0 fail, 0 error Co-authored-by: Yash Dewasthale <yashdev.yvd@gmail.com>
Review feedback on #288: - token-budget.test.ts: the daily-budget block test never awaited the rejects assertion, so a non-rejection would pass silently. Add await. - openai-compatible-stream.ts: after removing the in-loop tool-call flush, sawToolCalls is write-only. Remove the declaration and all assignments (emitToolCall and the two delta branches); flushing still happens on finish_reason tool_calls and at EOF. Co-authored-by: Yash Dewasthale <yashdev.yvd@gmail.com>
The module-scope setInterval for the ticking clock ran when the page module is evaluated during SSR/build, leaking a timer into build workers. Move the interval inside the useNow hook (useEffect + useState) so render stays pure and no module-level side effects exist. Co-authored-by: Yash Dewasthale <yashdev.yvd@gmail.com>
|
Status update on the checks: Review feedback — both threads (@tembo) are addressed in GitHub Actions (TypeScript Check / Lint / Test) — all three jobs are failing with: "The job was not started because your account is locked due to a billing issue." This is not a code failure: the runners never start, and it affects every run on this repo including Vercel – supercli (apps/web) — the deployment on |
🤖 Supercode AI ReviewSummaryThis PR fixes a broken CI pipeline (stale Walkthrough
Changes table
Findings
Risk assessmentMedium — The stream handler bug fix is a real behavioral change in a hot path (SSE tool-call assembly shared by 6+ provider routes); the fix is directionally correct and regression-tested, but any edge case in Test plan
Suggested PR descriptionWhatFixes failing CI on this repo (and Why
ChangesCI / lockfile
Bug fix
Tests (69 new)
Pre-existing fixes for CI green
How tested
Automated review by Supercode · leave a 👍/👎 reaction to rate this review |
Applies the actionable items from the Supercode review comment: - stream test: [DONE] only cuts off the current read batch's remaining lines (next read continues, trailing bytes drained at EOF); rename the test and comment to match that actual contract instead of implying it stops the whole stream. - credit-meter.test: use toBeCloseTo for the fractional MiniMax cost so IEEE-754 rounding across engines can't fail it. - profile-form: sync name/email on a stable profile.id instead of object identity, so a refetched profile object can't clobber in-flight edits. - pull-requests page: useNow via useSyncExternalStore with a server snapshot of 0 (no SSR/client Date.now() hydration mismatch); timeAgo renders a placeholder until mounted. - mergedev.connect: document that the session status is optimistic and real MCP health is verified by McpManager.reconnectServer (which throws on failure). Declined after verification: the node:os mock spread and ./types.ts imports are already correct (types.ts exports both shapes; the mock overrides both named and default homedir), mock.module is genuinely untyped in the resolved bun:test types so the cast stays, and the ffmpeg stub + motion cache concerns don't apply to the Linux-only CI and bounded element-type key set. Co-authored-by: Yash Dewasthale <yashdev.yvd@gmail.com>
|
Thanks for the thorough review — I worked through each finding. Addressed in Applied
Verified and declined (with evidence)
Verification after the changes: server |
What
Fixes the failing CI checks on this PR (
TypeScript Check,Lint,Test) — which were failing on every run, including onmain, for the same pre-existing reasons — and adds focused tests for the riskiest untested production paths.Root cause of the red checks
bun.lockwas stale: it still referenced the deletedapps/videoworkspace, sobun install --frozen-lockfile(step 1 of every job) failed in ~2 seconds. Once that is fixed, three layers of pre-existing debt surface: missing generated Prisma clients in CI, 30+ TypeScript errors, and ~90 lint errors.Changes
CI (
ci.yml,package.json,bun.lock)bun.lockwith the pinned bun 1.2.21 (removes only the staleapps/videosubtree + outdatedconfigVersion).db:generatescript; run it (withDATABASE_URL) before typecheck and test — two packages import generated Prisma clients that CI never generated (CLI server →src/generated,superdesign-db→prisma/client).Tests (69 new, all passing)
openai-compatible-stream.test.ts(28)tool_callsassembly, dedup, error events, usage, embedded MiniMax calls, non-streaming fallback,writeFinishpermission-rules.test.ts(11)token-budget.test.ts(12)credit-meter.test.ts(10)pricing.test.ts(8)Production bug fix surfaced by the tests —
streamOpenAICompatibleChatflushed and cleared pending tool calls after every SSE read, silently dropping tool calls whose arguments arrive across multiple reads and returning a bogus "empty response" error. Removed the in-loop flush (EOF flush still handles[DONE]); covered by a regression test.Pre-existing failures fixed to make the checks green
todowrite,question), missingMergeConnectorManagermethods used by/connectors(implemented), test mock typing; cortex-sdk v2/v3 AI SDK model types unified viaGatewayModel+ defensive usage normalization.useSyncExternalStoremounted flag) and apps/web (unescaped entities in marketing pages, explicitanyin chart/server-action code, setState-in-effect and purity fixes, motion slot component caching).permission-prompt.test.tsasserted the pre-allowlist prompting behavior for read-only commands;speech.test.tsre-imported the module mid-run with a replacedprocess.env(host-dependent ffmpeg check, TDZ flakes). Both rewritten to the intended contract, host-independent, static imports.Review feedback (both threads resolved) — the daily-budget
rejects.toThrowassertion is now awaited (79d2824); the write-onlysawToolCallsvariable was removed from the stream handler. The pull-requests page's module-scopesetInterval(which leaks a timer into SSR/build workers) was replaced with an effect-scoped hook (e3e0098).Verification
Run with bun 1.2.21 (CI-pinned) and
bun install --frozen-lockfile:turbo typecheck: 18/18 packages passturbo lint: 17/17 packages passbun test(root): 288 pass / 0 fail / 0 errornext build --webpack(apps/web): compiles cleanly (✓); the only local build error is a pre-existing, env-dependent Pinecone config requirement in/api/reviews/triggerand/api/webhooks/github(untouched by this PR)supercli,supercli-client, andsupercli-docsdeployments all successful on the head commit (e3e0098)main) fails immediately with "The job was not started because your account is locked due to a billing issue." This is a GitHub account billing lock on the repo owner's account (re-confirmed on run 358,e3e0098); only the account owner can clear it. The branch's code-side verification above is green with the exact CI toolchain, so the checks should pass once the lock is lifted.