Skip to content

fix: isolate profile variation submissions - #1631

Merged
Cedric Vidal (cedricvidal) merged 6 commits into
mainfrom
cedricvidal-fix-multi-profile-submission
Nov 5, 2026
Merged

Cedric Vidal (cedricvidal) merged 6 commits into
mainfrom
cedricvidal-fix-multi-profile-submission

Conversation

@cedricvidal

@cedricvidal Cedric Vidal (cedricvidal) commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Profile variation submissions could leak baseline profile-managed resource values into every variation, causing profiles with different bindings to conflict before comparative runs were created.

This change makes the API authoritative for profile fan-out. It rejects request-level profile-managed fields in variation mode, resolves each profile independently from the shared run intent, and keeps Portal and CLI submissions thin and consistent. It also documents the cross-client ownership invariant for future changes.

Closes #1613

PR stack

  1. fix: isolate profile variation submissions #1631 — Profile variation isolation (this PR)
  2. fix(portal): lock profile-managed run resources #1632 — Profile-managed Resources UI (depends on fix: isolate profile variation submissions #1631)

Demos

Before

Baseline profile resources are copied into shared request state and conflict with the comparison profile.

before-profile-variation.mp4

After

The API creates two isolated requests whose persisted resource values come from their respective profiles.

after-profile-variation.mp4

Testing

  • pnpm exec vitest run apps/api/src/endpoints.test.ts apps/cli/src/commands/run.test.ts apps/portal/src/pages/SubmitRun.variations.test.tsx --reporter=default - passed, 188 tests
  • pnpm --filter api build - passed
  • pnpm --filter cli build - passed
  • pnpm --filter portal build - passed
  • git diff --check - passed
  • Docker Compose API and Portal readiness checks - passed
  • Live API verification confirmed the baseline and comparison requests share one submission ID while persisting baseline/repository and comparison/repository independently

Documentation and compatibility

Updated docs/architecture/app-design.md, docs/architecture/resources.md, and AGENTS.md to document API-owned fan-out and profile isolation.

Variation-mode API submissions now reject request-level worker, model, reasoning effort, agent version, MCP server, skill, extension, and resource fields. This is an intentional contract tightening that prevents ambiguous shared/profile state. Single-profile submissions are unchanged. No migration, dependency, or upgrade steps are required.

Checklist

  • If Portal features changed, keep CLI capabilities in sync.
  • If Portal components changed, update their Storybook stories. (N/A - no reusable component changed; page submission behavior is covered by Vitest.)
  • If database changes require a migration, include up() / down() and keep it CosmosDB-compatible. (N/A - no database changes.)
  • If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt with pnpm notice as needed. (N/A - no dependency changes.)
  • Video showing the behavior before the suggested change
  • Video showing the behavior after the suggested change

Move profile fan-out ownership into the API so every profile resolves against fresh shared run intent. Keep Portal and CLI payloads free of profile-managed fields and document the cross-client ownership invariant.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added type: documentation Documentation additions, corrections, and improvements. area: cli Scope command-line interface and terminal workflows. language: javascript Work involving JavaScript code, tooling, or dependencies. area: portal Scope web portal, pages, and user-facing components. topic: testing Test coverage, test infrastructure, and validation quality. topic: ui Portal, visual presentation, layout, and interface components. labels Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #523

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3265 3265 0 0 0 0 0 1m16s

🎉 All tests passed!

Github Test Reporter

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #528

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3302 3302 0 0 0 0 0 58.7s

🎉 All tests passed!

Github Test Reporter

@cedricvidal
Cedric Vidal (cedricvidal) merged commit 53c6726 into main Nov 5, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Scope command-line interface and terminal workflows. area: portal Scope web portal, pages, and user-facing components. language: javascript Work involving JavaScript code, tooling, or dependencies. topic: testing Test coverage, test infrastructure, and validation quality. topic: ui Portal, visual presentation, layout, and interface components. type: documentation Documentation additions, corrections, and improvements.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Profile variation submissions leak baseline profile configuration

1 participant