Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/repository-agents.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ jobs:
printf '# Review me\n\nA short document.\n' > "$RUNNER_TEMP/review-input.md"
uvx --from "$wheel" oar run \
"$RUNNER_TEMP/profiles/reviewer" \
--task review \
--task review-document \
--input "$RUNNER_TEMP/review-input.md" \
--output "$RUNNER_TEMP/review-output.md" \
--dry-run
Expand Down
48 changes: 43 additions & 5 deletions projects/openshell-agent-runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ printf '# Review me\n\nA short document.\n' > document.md
uvx --from openshell-agent-runner oar validate ./profiles/reviewer

uvx --from openshell-agent-runner oar run ./profiles/reviewer \
--task review \
--task review-document \
--gateway openshell \
--input document.md \
--output /tmp/oar-review.md \
Expand All @@ -58,17 +58,37 @@ for each run, such as the task, inputs, output path, gateway, and workspace.

```yaml
id: reviewer
description: Review an uploaded document.
description: Review an uploaded document or code repository.

sandbox:
policy: policy.yaml
upload: []
env: []

tasks:
review:
review-document:
required_input: document
prompt: prompt.md
prompt: prompt-document.md
prompt_variables:
focus:
description: Areas of the document that deserve special attention.
default: Review the complete document.
context:
description: Additional context that should inform the review.
default: No additional context was provided.
tools: [read, grep, find, ls, bash]
skills: []
extensions: []
review-repository:
required_input: repository
prompt: prompt-repository.md
prompt_variables:
focus:
description: Files or directories that deserve special attention.
default: Review the entire repository.
context:
description: Additional context that should inform the review.
default: No additional context was provided.
tools: [read, grep, find, ls, bash]
skills: []
extensions: []
Expand All @@ -93,6 +113,12 @@ schemas, and tools that are not built in or declared by a referenced extension.
The runtime also verifies that Pi actually registered every selected tool before
the first model request.

Prompts support literal runtime substitution. Tasks declare named
`prompt_variables` with optional defaults, and callers override them with a
repeatable `--prompt-var NAME=VALUE`. Variables without defaults are required.
OAR also supplies reserved input metadata such as `{{ oar.input_path }}` and
`{{ oar.input_name }}`. Templates do not execute expressions or shell syntax.

Add `output_schema` to a task when its result must be JSON. OAR exposes the
built-in Pi `submit_result` extension for that task, lets Pi correct invalid
submissions during the session, and validates the downloaded result against the
Expand All @@ -117,7 +143,19 @@ profile and task before `--help`:

```bash
uvx --from openshell-agent-runner oar run \
./profiles/reviewer --task review --help
./profiles/reviewer --task review-document --help
```

The reviewer also accepts a code repository directory:

```bash
uvx --from openshell-agent-runner oar run ./profiles/reviewer \
--task review-repository \
--gateway openshell \
--input ./my-project \
--prompt-var focus="src/auth and tests/auth" \
--prompt-var context="Pre-release security review" \
--output /tmp/oar-repository-review.md
```

## Documentation
Expand Down
75 changes: 71 additions & 4 deletions projects/openshell-agent-runner/docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ printf '# Review me\n\nA short document.\n' > document.md
uvx --from openshell-agent-runner oar validate ./profiles/reviewer

uvx --from openshell-agent-runner oar run ./profiles/reviewer \
--task review \
--task review-document \
--gateway openshell \
--input document.md \
--output /tmp/oar-review.md \
Expand Down Expand Up @@ -103,8 +103,13 @@ The CLI supplies run-specific values:
- `--task` selects a task from `profile.yaml`.
- `--upload SOURCE:DESTINATION` uploads a file or directory using OpenShell's
native mapping format. It may be repeated.
- `--input FILE` is an optional document-task convenience. OAR uploads the file
to `/workspace/input/document.md` and sets `REPOSITORY_ROOT=/workspace/input`.
- `--input PATH` supplies the file or directory required by the selected task.
A `document` input is uploaded beneath `/workspace/input` with its ordinary
file extension preserved. A `repository` input is uploaded beneath the same
directory. OAR sets `REPOSITORY_ROOT` to the resulting document or repository
directory.
- `--prompt-var NAME=VALUE` supplies a non-secret runtime prompt variable. It
may be repeated for tasks that declare more than one variable.
- `--env KEY=VALUE` adds a sandbox environment value.
- `--gateway` selects an existing OpenShell gateway.
- `--workspace` selects a gateway-side OpenShell namespace. It defaults to
Expand All @@ -116,6 +121,47 @@ Environment keys start with a letter or underscore and contain only letters,
digits, and underscores. They cannot start with OpenShell's reserved
`OPENSHELL_` prefix.

### Prompt variables

Tasks can declare string variables used by their prompt template:

```yaml
tasks:
review-repository:
required_input: repository
prompt: prompt-repository.md
prompt_variables:
focus:
description: Files or directories that deserve special attention.
default: Review the entire repository.
context:
description: Additional context that should inform the review.
```

Variables with defaults are optional; variables without defaults are required.
Callers can supply several independent values by repeating the option:

```bash
--prompt-var focus="src/auth and tests/auth" \
--prompt-var context="Pre-release security review"
```

Templates reference declared variables by name and OAR metadata through the
reserved `oar` namespace:

```markdown
Inspect `{{ oar.input_path }}`, originally provided as
`{{ oar.input_name }}`.

Focus: {{ focus }}
Context: {{ context }}
```

Tasks with required inputs receive `oar.input_path` and `oar.input_name`.
Substitution is literal: prompt templates do not support expressions,
conditionals, loops, or shell evaluation. Unknown, duplicated, missing, unused,
and malformed variables are rejected before the sandbox starts.

### Tools and extensions

Each task lists the tools Pi may use. OAR accepts Pi's built-in `bash`, `edit`,
Expand Down Expand Up @@ -185,12 +231,33 @@ OpenShell treats a directory destination like `cp`: it creates the source
directory beneath that destination. Uploads run in declaration order, so more
than one source can intentionally merge into the same destination.

The packaged reviewer uses task-specific required inputs:

```bash
oar run ./profiles/reviewer \
--task review-document \
--input ./document.md \
--output ./document-review.md

oar run ./profiles/reviewer \
--task review-repository \
--input ./repository \
--prompt-var focus="src/auth and tests/auth" \
--prompt-var context="Pre-release security review" \
--output ./repository-review.md
```

Document tasks require a file and repository tasks require a directory. For a
repository task, OAR makes the uploaded repository the agent's working
directory. The repository is an uploaded snapshot; changes inside the sandbox
are disposable and are not synchronized back to the host.

Uploads come from three places:

| Source | Contents |
| --- | --- |
| Profile | `sandbox.upload` mappings shared by every run |
| CLI | Repeatable `--upload` mappings and the optional `--input` document |
| CLI | Repeatable `--upload` mappings and the task's required `--input` |
| OAR | Prompt, Pi model settings, skills, extensions, and optional schema |

Caller uploads normally live under `/workspace`. OAR runtime uploads live under
Expand Down
47 changes: 40 additions & 7 deletions projects/openshell-agent-runner/src/openshell_agent_runner/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,16 @@ def run(
output: Annotated[
Path, typer.Option("--output", help="Host path for the agent result.")
],
input_document: Annotated[
input_path: Annotated[
Path | None,
typer.Option("--input", help="Host document required by document tasks."),
typer.Option("--input", help="Host input required by the selected task."),
] = None,
prompt_variable: Annotated[
list[str] | None,
typer.Option(
"--prompt-var",
help="Non-secret NAME=VALUE prompt variable. Repeat for several.",
),
] = None,
upload: Annotated[
list[str] | None,
Expand Down Expand Up @@ -159,7 +166,8 @@ def run(
profile_directory=profile,
task_id=task,
output=output,
input_document=input_document,
input_path=input_path,
prompt_variables=prompt_variable or (),
uploads=upload or (),
environments=environment or (),
gateway=gateway,
Expand Down Expand Up @@ -227,8 +235,10 @@ def _render_task_help(
_help_command(f" oar run {shlex.quote(str(profile_directory))} \\"),
_help_command(f" --task {shlex.quote(task_id)} \\"),
]
if task.required_input == "document":
usage_lines.append(_help_command(" --input DOCUMENT \\"))
if task.required_input is not None:
usage_lines.append(
_help_command(f" --input {task.required_input.upper()} \\")
)
usage_lines.append(_help_command(" --output OUTPUT"))

upload_lines = [_help_heading("Additional configured uploads:")]
Expand All @@ -244,6 +254,22 @@ def _render_task_help(
environment_lines.append(" None. Add values with --env KEY=VALUE.")

input_lines = _required_input_help(task.required_input)
prompt_variable_lines = [_help_heading("Prompt variables:")]
if task.prompt_variables:
for name, variable in task.prompt_variables.items():
requirement = (
f"Default: {variable.default}"
if variable.default is not None
else "Required."
)
prompt_variable_lines.extend(
[
_help_command(f" --prompt-var {name}=VALUE"),
f" {variable.description} {requirement}",
]
)
else:
prompt_variable_lines.append(" None.")

output_description = (
f"JSON validated against {task.output_schema}."
Expand All @@ -260,6 +286,8 @@ def _render_task_help(
"",
*input_lines,
"",
*prompt_variable_lines,
"",
*upload_lines,
"",
*environment_lines,
Expand All @@ -274,10 +302,15 @@ def _render_task_help(
def _required_input_help(required_input: str | None) -> list[str]:
if required_input is None:
return [_help_heading("Required input:"), " None."]
description = (
"Host document to review."
if required_input == "document"
else "Host code repository to review."
)
return [
_help_heading("Required argument:"),
_help_command(" --input DOCUMENT"),
" Host document to review.",
_help_command(f" --input {required_input.upper()}"),
f" {description}",
]


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@
)

from openshell_agent_runner.errors import ConfigurationError
from openshell_agent_runner.prompt_templates import (
BUILTIN_PROMPT_VARIABLES,
PROMPT_VARIABLE_NAME_PATTERN,
validate_prompt_template,
)

IDENTIFIER_PATTERN = r"^[a-z][a-z0-9-]{0,62}$"
RESOURCE_IDENTIFIER_PATTERN = r"^[a-z][a-z0-9_-]{0,62}$"
Expand Down Expand Up @@ -62,6 +67,7 @@ def validate_environment(cls, values: list[str]) -> list[str]:


ToolName = Annotated[str, Field(pattern=RESOURCE_IDENTIFIER_PATTERN)]
PromptVariableName = Annotated[str, Field(pattern=PROMPT_VARIABLE_NAME_PATTERN)]


class ExtensionConfig(StrictModel):
Expand All @@ -76,10 +82,18 @@ def require_unique_tools(cls, values: list[str]) -> list[str]:
return values


class PromptVariableConfig(StrictModel):
description: str = Field(min_length=1, max_length=1000)
default: str | None = Field(default=None, min_length=1)


class TaskConfig(StrictModel):
description: str | None = Field(default=None, min_length=1, max_length=1000)
required_input: Literal["document"] | None = None
required_input: Literal["document", "repository"] | None = None
prompt: Path
prompt_variables: dict[PromptVariableName, PromptVariableConfig] = Field(
default_factory=dict
)
output_schema: Path | None = None
tools: list[ToolName] = Field(default_factory=list)
skills: list[Path] = Field(default_factory=list)
Expand Down Expand Up @@ -345,7 +359,23 @@ def _validate_profile_resources(resolved: ResolvedProfile) -> None:
directory = resolved.profile_dir
_inside(directory, directory / resolved.profile.sandbox.policy, "sandbox policy")
for task_id, task in resolved.profile.tasks.items():
_inside(directory, directory / task.prompt, f"prompt for task {task_id}")
prompt = _inside(
directory, directory / task.prompt, f"prompt for task {task_id}"
)
available_builtins = (
BUILTIN_PROMPT_VARIABLES if task.required_input is not None else frozenset()
)
try:
template = prompt.read_text(encoding="utf-8")
validate_prompt_template(
template,
task.prompt_variables.keys(),
available_builtins,
)
except (OSError, UnicodeError, ValueError) as error:
raise ConfigurationError(
f"invalid prompt template for task {task_id}: {error}"
) from error
if task.output_schema is not None:
schema = _inside(
directory,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import json
import shutil
import tempfile
from collections.abc import Mapping
from importlib.resources import files
from pathlib import Path

Expand All @@ -15,6 +16,7 @@
ResolvedProfile,
)
from openshell_agent_runner.harnesses.resources import PreparedResources
from openshell_agent_runner.prompt_templates import render_prompt_template

SANDBOX_RUNTIME_ROOT = "/sandbox/oar-runtime"

Expand All @@ -23,13 +25,19 @@ def image_directory() -> Path:
return Path(str(files("openshell_agent_runner.harnesses.pi") / "runtime" / "image"))


def prepare_resources(resolved: ResolvedProfile, task_id: str) -> PreparedResources:
def prepare_resources(
resolved: ResolvedProfile,
task_id: str,
prompt_variables: Mapping[str, str] | None = None,
) -> PreparedResources:
temporary = tempfile.TemporaryDirectory(prefix="oar-pi-")
runtime = Path(temporary.name) / "runtime"
(runtime / "skills").mkdir(parents=True, exist_ok=True)
(runtime / "extensions").mkdir(parents=True, exist_ok=True)
task = resolved.profile.tasks[task_id]
shutil.copy2(resolved.profile_dir / task.prompt, runtime / "prompt.md")
template = (resolved.profile_dir / task.prompt).read_text(encoding="utf-8")
rendered_prompt = render_prompt_template(template, prompt_variables or {})
(runtime / "prompt.md").write_text(rendered_prompt, encoding="utf-8")
shutil.copy2(resolved.profile_dir / MODELS_FILENAME, runtime / MODELS_FILENAME)
shutil.copy2(resolved.profile_dir / SETTINGS_FILENAME, runtime / SETTINGS_FILENAME)
arguments = [
Expand Down
Loading
Loading