Skip to content

Isolate agent execution from grading data - #104

Closed
dmorosanu wants to merge 2 commits into
mainfrom
feat/agent-grader-isolation
Closed

Isolate agent execution from grading data#104
dmorosanu wants to merge 2 commits into
mainfrom
feat/agent-grader-isolation

Conversation

@dmorosanu

Copy link
Copy Markdown
Contributor

Summary

  • Stage a sanitized agent task instead of passing the original task YAML into the agent container.
  • Keep success criteria, references, lifecycle metadata, and the original task directory on the trusted host.
  • Regrade the captured agent workspace after the container exits, using the original private task definition.
  • Stop automatically mounting task and reference sources into the agent container.

Why

The agent previously received the complete test case alongside the work it needed to perform. That allowed it to inspect private evaluation inputs such as criteria, solutions, references, and adjacent grading files. This change separates agent execution from grading while preserving the existing artifact-based result flow.

Validation

  • Ruff and whitespace checks passed.
  • Podman probes confirmed a representative troubleshooting task does not expose task-local grading files, the original task directory, or private evaluation metadata to the agent container.

Follow-up

The stacked public-input branch adds an explicit allowlist for plugin and template inputs, which closes the remaining exposure from raw source mounts.

@dmorosanu

Copy link
Copy Markdown
Contributor Author

Parking this line of work: the container UID/GID isolation in #87 covers the target cases - grading material stays out of reach of the agent identity during the agent phase. #87 now runs agents without container-isolation support in normal mode and lets dynamic criteria run in the grader phase, so a separate harness-level data-separation layer is not needed right now. Branch stays parked for potential later reuse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant