Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 42 additions & 3 deletions docs/reference/protocols/explore-result-from-path-delta-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,11 +100,50 @@ through `--explore-result-json result.json`. It additionally supplies
reusable result differs from the current route delta. Two sources must normalize
to identical results; a conflict fails before primary commit.

## Execution evidence on the same input

Both attachment forms accept the following `execution_observation` value
when the candidate ran on the finding's declared `input_revision`.

```json
{
"schema_version": "explore_execution_from_finding_v0",
"candidate_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"planned_method_id": "bounded-probe",
"executed_method_id": "bounded-probe",
"change_reason": null,
"elapsed_seconds": 0.125,
"limit_seconds": 10,
"search_iterations": 2,
"stop_reason": "iteration_limit",
"evidence_refs": "finding"
}
```

Use the actual candidate digest, method and measured resource values. Set
unknown measurements to null. The shorthand binds the required input revision
from this finding and rejects an explicit nested `input_revision`. Choose
`"finding"` only when all of the finding's validated evidence refs describe
the execution. Otherwise list its exact nonempty subset. With path-delta
capture this operates on the finding's selected refs, after excluding any local
file pointers. A released result still requires its `result_ref` and
matching `result_candidate_sha256`.

The Explore owner expands this input to the complete
`explore_execution_observation_v0` before the existing attribution checks
and durable write. That complete form still accepts explicit values and rejects
mismatches. Standalone observations, including `explore finding` and stored
records, use the complete form. The captured writeback plan and Explore
turn-context expose `optional_execution_observation_example` when relevant.

## Bounds and readback

A path-delta reference requires this same vision packet's top-level delta;
a previous stored vision, a nested delta or a separate result file alone cannot
supply it. Observation allows 320 characters and interpretation 1200, matching
the Goal observation budget and retaining every legal route item as plain list
text without JSON escaping expansion. The combined finding, including its input
supply it. Question allows 180 characters and applicability 200. Observation's
320-character limit matches the Goal observation budget. Interpretation allows
1200 characters and retains every legal route item as plain list text without
JSON escaping expansion. The combined finding, including its input
revision and applicability, fits within the 2000-character stored and next-turn
summary budget. This replaces the previous 300/300 attachment and 1200 summary
limits; the default three-result page and progressive reads are unchanged. No
Expand Down
4 changes: 4 additions & 0 deletions loopx/capabilities/explore/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ This records producer evidence, not verified execution or utility. Findings,
replan context and the existing research-history detail retain it; omission
preserves old events. See the [local probe producer](../../../benchmark/runtime/EXECUTION_OBSERVATIONS.md).

When the observed execution used the finding's tested input revision, result
attachments can bind it once and select the evidence to reuse. Readback and
standalone observations keep the complete observation form. [Execution evidence authoring](../../../docs/reference/protocols/explore-result-from-path-delta-v0.md#execution-evidence-on-the-same-input)

## Quantitative findings

An ordinary result attachment can include `measurements`, saved on its existing
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@ import {requireJsonObject} from "../runtime_decode.ts";
import {BARE_SHA256_PATTERN} from "../content_digest.ts";
import {isExploreEvidenceRef} from "./explore_evidence_ref.ts";

export const EXPLORE_EXECUTION_OBSERVATION_SCHEMA = "explore_execution_observation_v0";

export function normalizeExploreExecutionObservation(params: JsonObject): JsonObject {
const row = requireJsonObject(params.observation, "execution observation");
const required = ["schema_version", "candidate_sha256", "input_revision", "planned_method_id",
"executed_method_id", "change_reason", "elapsed_seconds", "limit_seconds",
"search_iterations", "stop_reason", "evidence_refs"];
if (row.schema_version !== "explore_execution_observation_v0"
if (row.schema_version !== EXPLORE_EXECUTION_OBSERVATION_SCHEMA
|| required.some(key => !Object.hasOwn(row, key))
|| Object.keys(row).some(key => ![...required, "exit_code", "result_ref", "result_candidate_sha256"].includes(key))) {
throw new EffectRuntimeRequestError("execution observation fields are missing or unsupported");
Expand Down
39 changes: 32 additions & 7 deletions loopx/control_plane/capabilities/explore_result_writeback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,17 @@ import {EffectRuntimeRequestError} from "../effect_runtime_errors.ts";
import {jsonObject, requireJsonObject, requireNonEmptyString, requireStringArray, requireStringLiteral} from "../runtime_decode.ts";
import {contextSourceIdentity} from "../quota/diagnostic_projection.ts";
import {normalizeGoalPathDelta} from "../goals/vision_checkpoint.ts";
import {normalizeExploreExecutionObservation} from "./explore_execution_observation.ts";
import {EXPLORE_EXECUTION_OBSERVATION_SCHEMA, normalizeExploreExecutionObservation} from "./explore_execution_observation.ts";
import {isExploreEvidenceRef} from "./explore_evidence_ref.ts";
import {normalizeWorkYardstickDecision} from "../work_items/yardstick_decision.ts";
import {normalizeExploreMeasurements} from "./explore_measurements.ts";

export const EXPLORE_RESULT_ATTACHMENT_SCHEMA = "explore_result_attachment_v0";
export const EXPLORE_PATH_DELTA_ATTACHMENT_SCHEMA = "explore_result_from_path_delta_v0";
const EXPLORE_EXECUTION_FROM_FINDING_SCHEMA = "explore_execution_from_finding_v0";
const FINDING_EVIDENCE_SELECTION = "finding";
const QUESTION_LIMIT = 180;
const APPLICABILITY_LIMIT = 200;
// The complete scoped finding fits the result-log owner: 160 revision + 200
// applicability + 320 observation + 1200 interpretation + four labels < 2000.
export const EXPLORE_WRITEBACK_SUMMARY_LIMIT = 2000;
Expand Down Expand Up @@ -62,6 +66,12 @@ export function exploreResultWritebackAffordance(): JsonObject {
unit: "ratio", higher_is_better: true, comparison_scope: "validation:fixed-budget-v1",
input_revision: "dataset-v1", evaluation_ref: "evaluator:validation-v1",
candidate_ref: "candidate:revision-2", evidence_ref: "result:validation-2"}]},
optional_execution_observation_example: {execution_observation: {
schema_version: EXPLORE_EXECUTION_FROM_FINDING_SCHEMA,
candidate_sha256: "a".repeat(64), planned_method_id: "bounded-probe", executed_method_id: "bounded-probe",
change_reason: null, elapsed_seconds: 0.125, limit_seconds: 10, search_iterations: 2,
stop_reason: "iteration_limit", evidence_refs: FINDING_EVIDENCE_SELECTION,
}},
guidance: [
"After validating reusable evidence, put explore_result in the vision JSON already submitted with --agent-vision-json.",
EXPLORE_BEFORE_WORK.openLease,
Expand Down Expand Up @@ -94,7 +104,8 @@ export function exploreResultWritebackAffordance(): JsonObject {
"To reuse explicitly verified conditions, optional measurement_context supplies comparison_scope, input_revision and evaluation_ref for observations that omit those fields; conflicting fields reject. CLI finding accepts the same context via --measurement-context-json. This copies declared conditions, not proof of comparability.",
EXPLORE_BEFORE_WORK.feedbackBeforeWork,
EXPLORE_BEFORE_WORK.executionEvidence,
"Observations allow 320 characters, interpretations 1200; the complete scoped summary is preserved within 2000 characters.",
`Use optional_execution_observation_example when the execution used this finding's tested input. Omit the nested input_revision and set its evidence_refs to "${FINDING_EVIDENCE_SELECTION}" only if all finding refs describe this execution, or list the exact opaque subset. Supply the actual candidate and execution facts. A released result still needs its result_ref and matching result_candidate_sha256.`,
`Questions allow ${QUESTION_LIMIT} characters, applicability ${APPLICABILITY_LIMIT}, observations 320, and interpretations 1200. The complete scoped summary is preserved within 2000 characters.`,
].join(" "),
// Blank evidence fields deliberately fail validation until the caller
// supplies observed facts. Goal/Agent/Todo/Turn bind in ordinary writeback;
Expand Down Expand Up @@ -163,6 +174,20 @@ function explicitNodeId(value: unknown): string {
}
return value.trim();
}
/** Expand an explicit same-finding binding before the canonical codec. */
function normalizeAttachedExecution(value: unknown, inputRevision: unknown, refs: string[]): JsonObject {
const authored = jsonObject(value);
if (authored?.schema_version !== EXPLORE_EXECUTION_FROM_FINDING_SCHEMA) {
return normalizeExploreExecutionObservation({observation: value});
}
if (Object.hasOwn(authored, "input_revision")) {
throw new EffectRuntimeRequestError("execution observation cannot override its explicit finding input revision");
}
return normalizeExploreExecutionObservation({observation: {...authored,
schema_version: EXPLORE_EXECUTION_OBSERVATION_SCHEMA, input_revision: inputRevision,
evidence_refs: authored.evidence_refs === FINDING_EVIDENCE_SELECTION ? refs : authored.evidence_refs,
}});
}
function normalizeAttachment(value: unknown, visionPacket?: unknown, linkedScope?: unknown): JsonObject {
const row = requireJsonObject(value, "Explore result attachment");
if (row.schema_version === EXPLORE_PATH_DELTA_ATTACHMENT_SCHEMA) {
Expand All @@ -172,8 +197,8 @@ function normalizeAttachment(value: unknown, visionPacket?: unknown, linkedScope
throw new EffectRuntimeRequestError("Explore result attachment contains unknown fields");
}
requireStringLiteral(row.schema_version, [EXPLORE_RESULT_ATTACHMENT_SCHEMA], "attachment schema");
const question = text(row.question, "question", 180);
const applicability = text(row.applicability, "applicability", 200);
const question = text(row.question, "question", QUESTION_LIMIT);
const applicability = text(row.applicability, "applicability", APPLICABILITY_LIMIT);
// Identity is mechanical, never inferred evidence. Exact scoped text, not
// revision/status/observation, binds successive results to the same question.
const node = Object.hasOwn(row, "node_id")
Expand All @@ -190,7 +215,7 @@ function normalizeAttachment(value: unknown, visionPacket?: unknown, linkedScope
return ref;
});
const execution = Object.hasOwn(row, "execution_observation")
? normalizeExploreExecutionObservation({observation: row.execution_observation}) : null;
? normalizeAttachedExecution(row.execution_observation, row.input_revision, refs) : null;
if (Object.hasOwn(row, "measurement_context") && !Object.hasOwn(row, "measurements")) {
throw new EffectRuntimeRequestError("measurement_context requires measurements");
}
Expand Down Expand Up @@ -260,8 +285,8 @@ function questionScope(row: JsonObject, value: unknown): JsonObject {
if (!refs.includes(nodeId) || matches.length !== 1) {
throw new EffectRuntimeRequestError("Explore capture requires an existing question linked to this Todo");
}
return {question: text(matches[0].title, "question", 180),
applicability: text(matches[0].summary, "applicability", 200)};
return {question: text(matches[0].title, "question", QUESTION_LIMIT),
applicability: text(matches[0].summary, "applicability", APPLICABILITY_LIMIT)};
}
export function normalizeExploreResultAttachment(params: JsonObject): JsonObject {
const result = normalizeAttachment(params.attachment, params.vision_packet, params.linked_scope);
Expand Down
3 changes: 2 additions & 1 deletion loopx/control_plane/capabilities/explore_turn_context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export function exploreReusableContextPaths(content: JsonObject = {}): (string |
return [
...["recent_nodes", "recent_findings", "writeback_results", "measurement_feedback", "work_yardsticks", "work_yardstick_capture",
"path_delta_attachment_template", "linked_question_attachment_template",
"optional_question_evolution_example", "result_writeback_guidance",
"optional_question_evolution_example", "optional_execution_observation_example", "result_writeback_guidance",
"result_attachment_template", "optional_measurement_example", "measurement_guidance",
"record_node_template", "record_finding_template", "guidance",
"result_writeback_option", "result_writeback_inline_option", "result_writeback_inline_field",
Expand Down Expand Up @@ -49,6 +49,7 @@ function resultWritebackFields(writeback: JsonObject): JsonObject {
path_delta_attachment_template: writeback.path_delta_attachment_template,
linked_question_attachment_template: writeback.linked_question_attachment_template,
optional_question_evolution_example: writeback.optional_question_evolution_example,
optional_execution_observation_example: writeback.optional_execution_observation_example,
result_writeback_guidance: writeback.guidance,
result_attachment_template: writeback.attachment_template,
optional_measurement_example: writeback.optional_measurement_example,
Expand Down
6 changes: 5 additions & 1 deletion loopx/control_plane/quota/context_delivery.ts
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,11 @@ function freshPaths(context: JsonObject, surface: Surface, legacyQuotaFields: re
detail.forEach((attachment, j) => {
const item = jsonObject(attachment), prefix = [...path, key, i, field, j];
if (!item) { paths.push(prefix); return; }
for (const property of Object.keys(item)) if (property !== "guidance") paths.push([...prefix, property]);
for (const property of Object.keys(item)) {
if (property === "guidance" || (property === "optional_execution_observation_example"
&& item.capability_id === "explore" && item.required === false)) continue;
paths.push([...prefix, property]);
}
});
} else paths.push([...path, key, i, field]);
}
Expand Down
3 changes: 2 additions & 1 deletion loopx/control_plane/quota/progressive_context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -537,7 +537,8 @@ export function initialContextReadPointers(context: JsonObject, decisionFields:
// Keep its identity and actual read/authoring routes discoverable.
for (const key of Object.keys(object(attachment)!)) {
if (["guidance", "attachment_template", "linked_question_attachment_template", "path_delta_attachment_template",
"optional_measurement_example", "optional_question_evolution_example"].includes(key)) continue;
"optional_measurement_example", "optional_question_evolution_example"].includes(key) ||
(key === "optional_execution_observation_example" && object(attachment)?.capability_id === "explore")) continue;
include(...path, key); // Unknown attachment controls are never hidden.
}
});
Expand Down
Loading
Loading