Skip to content

feat(productivity): add independent prose review to write-naturally - #8

Closed
christso wants to merge 1 commit into
mainfrom
feat/write-naturally-review
Closed

christso wants to merge 1 commit into
mainfrom
feat/write-naturally-review

Conversation

@christso

Copy link
Copy Markdown
Contributor

Summary

Add prose review as an operation of write-naturally, rather than introducing a second skill or duplicating its writing rules.

  • Write/rewrite still returns the requested prose; review returns grounded findings and local suggestions without an unsolicited full rewrite.
  • Substantial documents intended for publication receive a separate prose-compatible reviewer when the host supports one. The reviewer loads the same skill, does not recursively delegate, and cannot edit or publish.
  • Hosts without agent support use the same procedure locally without claiming independence.
  • Review separates wording corrections from business decisions, accepts an adequate draft without cosmetic findings, and does not treat facts absent from a briefing as unsupported commitments.
  • Strengthen the shared editing checks for overlapping terms, neighbouring labels, redundant context and explicit comparisons. The review procedure applies them in three complete passes; it does not maintain a separate style rubric.
  • Update usage documentation and skill discovery text.

The new reference defines the review procedure and report. The core skill remains the source of writing criteria. A publishing workflow can invoke review mode even when the drafting operation was skipped; this change does not claim to enforce invocation at the harness level.

Dogfooding

An initial independent review exposed an agent-selection problem: a specialized code-review role imposed a code-correctness verdict. The delegation entry point now explicitly selects a prose-compatible agent and reads the review procedure before dispatch.

A fresh general agent used the revised skill to review all four changed public files and reported no material issues. That review did not verify runtime enforcement or external claims.

The companion private review evaluation passed four of five cases with zero execution/grader errors. One whole-document case still misses required editorial findings. The regression remains failing; the criteria were not weakened. Increasing reviewer reasoning effort did not eliminate the miss, so the evaluation retains the existing medium-effort default. This PR is draft for review, not a claim that the reviewer can replace human judgement or the eval suite.

Independent grader calibration accepted both good reports and rejected all six deliberately bad reports on the intended metrics, with zero errors. Configured models: reviewer gpt-6.1-sol, judge gpt-5.6-luna; Promptfoo 0.124.1.

Verification

  • bun run validate: five plugins, both native catalogues and local Markdown references validated.
  • bun test: 21 passed, zero failed.
  • Existing private rewrite benchmark: 16 passed, five failed, zero errors in the full run. These are observed failures, not proven regressions introduced by this change; no baseline comparison was run.

Private evaluation cases and captured outputs remain in the companion private repository; none are included in this public change.

@christso

Copy link
Copy Markdown
Contributor Author

Closing at the author’s request. Keep the existing write-naturally skill and request independent review explicitly. The private evaluation suite will be decoupled from this unmerged proposal.

@christso christso closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant