Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 106 additions & 31 deletions .claude/skills/doc-review/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,124 @@
---
name: doc-review
description: Review documentation for quality, structure, and convention compliance. Use when the user wants to review docs, audit a page or section, check if content is clear, evaluate information architecture, or assess navigation. Do NOT use when the user wants to write or fix docs — use doc-write for that. Triggers on "review the docs", "audit this page", "is this getting-started page clear", "check the implementation guide section".
description: Review documentation for quality, structure, and convention compliance. Use when the user wants to review docs, audit a page or section, check if content is clear, evaluate information architecture, or assess navigation. Do NOT use when the user wants to write or fix docs — use doc-write for that. For a periodic whole-site navigation and coverage audit that files issues, use doc-structure. Triggers on "review the docs", "audit this page", "is this getting-started page clear", "check the implementation guide section".
---

# Doc Review
# Doc review

Evaluate technical docs against the Diátaxis framework, information-architecture best practices, and the project's CLAUDE.md.
Review changed documentation against this repo's CLAUDE.md, the Diátaxis framework, and the site's information architecture.

Most of this review's value comes from checks a human reviewer cannot do cheaply: verifying prose against generated reference pages, catching pages the change forgot to update, and questioning where a new page lives. Spend your turns there.

## What is already checked for you

Do not re-do this work:

- **Spelling and Vale rules** — the `Mintlify Validation (kosli) - vale-spellcheck` check runs on every PR and enforces `styles/Kosli/AmericanSpelling.yml`. Never report a spelling finding. Never report "American spelling: pass".
- **PR title format** — the `Validate PR Title` job enforces Conventional Commits.
- **Live-docs script behavior** — the `Test live-docs scripts` job runs `pytest tests/`.

**Link checking is the exception — verify it yourself.** `Mintlify Validation (kosli) - link-rot` is unreliable: check the PR's own check runs (`gh pr checks`), and if it reports `skipping` or is absent, nothing has validated the links. Treat that as the default and confirm every internal target resolves.

## Before reviewing

1. **Read the project's CLAUDE.md** — it defines all site-specific conventions. Use it as your compliance checklist for writing style, components, link format, and frontmatter. If no CLAUDE.md exists, read 3-4 existing pages to infer conventions.
2. **Read `docs.json`** to understand navigation structure.
3. **Determine review scope** — single page, section, or full site. If unclear, ask the user which scope they want.
1. **Read CLAUDE.md** — the compliance checklist for frontmatter, components, links, writing style, and the don'ts.
2. **Read `config/navigation.json`** for the navigation tree. Navigation lives there, not in `docs.json` — `docs.json` only holds a `$ref` to it.
3. **Read each changed file at the current branch head**, not just the diff. A finding that was already fixed in a later commit must not be reported.
4. **Determine scope** — a single page, a section, or the changed files in a PR. If unclear, ask.

## Generated pages: report upstream, never inline

Most of the diff volume in this repo is machine-generated. A finding on a generated page is still worth reporting — but **reporting it as an edit to this repo is wrong**, because the next release silently reverts it and the defect ships again.

### Two categories, two different failures

**Deterministically regenerated — a hand-edit is destroyed.** The generator deletes and rewrites these files:

| Page | Generated by | Fix it here |
|---|---|---|
| `client_reference/kosli*.md` | `kosli docs` from the release binary | **`kosli-dev/cli`** → `cmd/kosli/<command>.go` |
| `helm/k8s_reporter/*.mdx` | `helm-docs` | **`kosli-dev/cli`** → `charts/k8s-reporter/mintlify/<page>.md.gotmpl`, its `_templates.gotmpl` / `_mintlify_templates.gotmpl`, or `values.yaml` |
| `schemas/flow-template/v1.json`, `schemas/policy/v1.json` | `scripts/update_schemas.py` from the API | **`kosli-dev/server`** → the Pydantic models |
| The `kosli *` groups in `config/navigation.json` | `scripts/update-cli-nav.py` | This repo → the script |
| Live-docs sections under `client_reference/` | `scripts/add_livedocs.py` | This repo → the script and `scripts/live_docs_*_data.py` |

The filename maps to the source: `client_reference/kosli_attest_sonar.md` ← `cmd/kosli/attestSonar.go`; `kosli_create_attestation-type.md` ← `cmd/kosli/createAttestationType.go`; `helm/k8s_reporter/karpenter.mdx` ← `charts/k8s-reporter/mintlify/karpenter.md.gotmpl`.

**Agent-synced from an upstream source of truth — a hand-edit survives but drifts.** `terraform-reference/` (from `kosli-dev/terraform-provider-kosli`) and `github-action-reference/setup_cli_action.md` (from `kosli-dev/setup-cli-action`'s `README.md` and `action.yml`) are written by the `.mintlify/workflows/` agents. Editing them here works, but if the upstream source disagrees the next sync will undo it. Report both the page fix and the upstream mismatch.

**Not generated, despite the directory:** `client_reference/overview.md` and `client_reference/output_and_verbosity.md` are hand-authored — the regeneration step only removes `kosli*.md`. Only the version string in `overview.md` is stamped by the workflow. Review these normally.

### Rules

- **Do not post an inline comment on a deterministically regenerated page.** The line it anchors to will not exist after the next release.
- **Never emit a "Fix this" link scoped to `repo=kosli-dev/docs` for a generated page.** This has happened: a review correctly wrote *"the durable fix is in the `kosli-dev/cli` generator — a hand-edit here is overwritten by the next release"* and then attached a fix link telling an agent to edit the generated file anyway. If a fix link is warranted, scope it to the upstream repo.
- Report these findings **in the top-level comment**, grouped under a single heading, each naming the upstream repo and file to change.
- Say plainly that the finding does not block the merge. The regeneration PR is a faithful copy of upstream; blocking it does not fix the source and only delays the release.

### Tell a generator bug apart from a prose typo

In `kosli-dev/cli`, **`^` is the convention for a backtick** inside a command's long description (`^--jq^`, `^jq^`, `^'NAME=EXPRESSION'^`), and `kosli docs` substitutes it when generating markdown. So a literal `^` surviving into a published page is a **generator escaping bug**, not a typo in the Go string — the fix is in the doc generator's rendering path, not the description. A caret that reached an `<Accordion>` title while the same substitution worked in body prose is exactly this bug.

Distinguish it from genuine prose defects — British spelling, a contradictory sentence, an ambiguous clause — which do live in the Go string and are fixed by editing it.

## The checks that matter most

Run these first. They found the real defects in past reviews.

### 1. Cross-file consistency

A change is rarely confined to the files it touches. Grep the rest of the site for what the change contradicts.

- **Prose vs. generated reference.** Pages under `client_reference/`, `terraform-reference/`, and `helm/` are generated. When prose names a command, flag, resource, or attribute, verify it exists in the generated page. *Precedent: a changelog entry documented `kosli update attestation-type`, a command with no reference page — it did not exist.*
- **Prose vs. generated schema.** `schemas/flow-template/v1.json` and `schemas/policy/v1.json` come from the API. When a page enumerates valid values, diff that list against the schema. *Precedent: `template-reference/flow_template.md` listed an attestation-type set the regenerated schema had already outgrown.*
- **Incomplete sweeps.** When a change removes or renames a concept, grep the whole site for the old term and list every file still using it. *Precedent: an approvals-removal PR updated `glossary.md` and `controls.md` but left `understand_kosli/how_kosli_works.md:22` still calling approvals a built-in attestation type.*

### 2. Placement and navigation

- Every new page must appear in `config/navigation.json`. Missing entry is **Critical**.
- **Ask whether the page is in the right tab and group**, not just whether it is listed somewhere. Apply the placement table in the `doc-write` skill. A page whose content is complete factual lookup belongs in the **Reference** tab even when it documents an integration. *Precedent: a GitHub Action reference page was first authored into `integrations/`; a human reviewer had to ask for the move to Reference. That question should come from this review.*
- Flag a new group created to hold a single page, and any page nested more than three levels below its tab.

### 3. Redirects

Any PR that renames, moves, or deletes a page needs a `config/redirects.json` entry. This is one of the easiest things to forget, because the PR looks complete without it. A missing redirect for a page that was live is **Critical** — the URL is in customers' bookmarks, in CLI error output, and in the changelog.

### 4. Anchor stability

Headings are link targets. Before accepting a renamed heading, grep for its anchor across the repo. Some anchors are referenced from outside the docs — `faq/faq.md#boolean-flags` is emitted in CLI error messages. A renamed heading that breaks an inbound anchor is **Critical**.

### 5. Page-level quality

- **Diátaxis fit** — tutorial, how-to, reference, or explanation? Does the content match the form? Report a mismatch only when it would send a reader down the wrong path, not as a taxonomy note.
- **Frontmatter** — `title` and `description` present and accurate.
- **Links** — root-relative (`/getting_started/install`), never relative (`../install`). A relative link is **Critical**. Verify every internal target resolves to a file that exists.
- **Correctness** — commands, flags, output blocks, and screenshots that match the current product.

## Page-level review
## What not to report

1. **Diátaxis classification** — Identify the doc type (tutorial, how-to, reference, explanation). Does the content match? Common issue: tutorials mixed with reference material, or how-to guides that teach instead of solving.
2. **Content quality** — Clear title? Accurate description? Complete for its doc type? Factually correct and current?
3. **Structure** — Logical flow? Appropriate depth?
4. **CLAUDE.md compliance** — Check the page against every rule and convention in the project's CLAUDE.md (frontmatter, components, links, writing style, don'ts).
5. **Link validation** — Internal links resolve to existing files? (Use Glob/Grep to verify.)
The bar is: **would a reader be measurably better off after this change?** If not, leave it out. Specifically, never report:

## Section-level review
- A finding already fixed at the current branch head.
- Rewording that is a matter of taste. Grammar that is merely awkward is not a finding; grammar that is ambiguous or wrong is.
- Whitespace or column alignment inside pasted command output.
- Anything the automated checks above already cover.
- A finding you immediately talk yourself out of. If the recommendation ends in "which it already does" or "no change needed", it was never a finding.
- Sample-data churn in a regenerated page — shifted timestamps, fingerprints, commit SHAs, snapshot indices. Confirm it is only data churn and move on; do not itemise it.
- Any recommendation to hand-edit a deterministically regenerated page. If the only fix is upstream, report it upstream or not at all.
- Praise, "what looks good" sections, and tables of checks that passed. The author knows what they wrote. Reviews are re-rendered into a sticky comment on every push, so recital costs the reader on every read.

1. Read all pages in the section.
2. Check navigation order — logical progression?
3. Identify gaps — missing pages for common user tasks?
4. Check Diátaxis balance — right mix of doc types?
5. Verify cross-linking between related pages.
6. Assess consistency in voice, structure, and components.
Report at most **8 findings**. If more clear the bar, report the 8 that matter most and say how many were left out.

## Site-level review
## Output

1. Is the top-level navigation intuitive?
2. Are tutorials, how-to guides, reference, and explanation clearly separated?
3. Can new users find getting-started content quickly? Can experienced users find reference quickly?
4. Structural issues: orphaned pages, deep nesting (>3 levels), overloaded sections, missing landing pages.
Group findings by file. For each: **Location** (`file:line`), **Issue** (one or two sentences), **Recommendation** (concrete).

## Output format
Categorize:

Categorize findings as:
- **Critical** — Broken functionality, factual errors, missing required content.
- **Improvement** — Clarity issues, structural problems, missing best practices.
- **Suggestion** — Nice-to-have enhancements.
- **Critical** — missing nav entry, relative link, broken internal link or anchor, missing redirect, factually wrong instruction.
- **Improvement** — a reader is likely to be misled, blocked, or sent to the wrong page.
- **Suggestion** — a real but minor gain. If you have no Improvements, question whether the Suggestions are worth posting at all.

For each: **Location** (file:line or section), **Issue**, **Recommendation**.
Put findings on generated pages in their own section, headed **Upstream — does not block this merge**, with the repo and file to change. Never mark one Critical: nothing in the PR under review can fix it.

End with a summary: findings by category, overall assessment, top 3 priorities.
Close with one line: counts by category, and a merge verdict. When nothing clears the bar, say exactly that in one sentence and stop — a short review is a good review.
Loading