Repository navigation
Testing guidance prioritizes real behavior and APIs - #235
Merged
Merged
Conversation
6 tasks done
There was a problem hiding this comment.
🟢 Approved
No active review issues are recorded for the testing-guidance and plugin-skill changes.
No open issues 🎉
What changed in this PR
This PR makes the Testing standard the single owner of behavior-focused testing guidance and adds a plugin skill that points agents to it. It does not migrate existing suites or install the skill into consumers.
Changes:
- Prioritizes supported behavior, real implementations, independent expected results, and justified test doubles.
- Preserves a credential-free local loop while defining authorization, isolation, cleanup, and evidence requirements for protected live-API checks.
- Adds the testing skill pointer and synchronizes plugin versions to
0.1.2.
| File | Description |
|---|---|
src/docs/Ways-of-Working/Principles/Engineering-Practices.md |
Delegates local-testing requirements to the Testing standard. |
src/docs/Coding-Standards/Testing.md |
Defines behavior-focused testing policy and protected live-check requirements. |
src/docs/Coding-Standards/index.md |
Updates the Testing standard’s summary. |
.github/plugin/README.md |
Includes testing in the coding-skill scope. |
.github/plugin/msx/skills/msx-coding-testing/SKILL.md |
Adds a pointer to the canonical Testing standard. |
.github/plugin/msx/plugin.json |
Bumps the plugin version to 0.1.2. |
.github/plugin/marketplace.json |
Aligns the marketplace version with the plugin. |
Review effort: Balanced
Marius Storhaug (MariusStorhaug)
marked this pull request as ready for review
November 2, 2026 18:43
Marius Storhaug (MariusStorhaug)
added a commit
that referenced
this pull request
Nov 3, 2026
…rds (#208) Release guidance defines durable intent, phase-aware recovery, complete source-bound consumer records, and settings-driven decisions. It preserves the accepted keyed release routing, `.github/release.settings.yml` convention, solution-specific path exceptions, and release-range-first PR records while defining the lifecycle contract. This delivery changes documentation, not a producer runtime. It creates no settings file, legacy filename fallback, versioned release, live caller change, consumer installation, or governance change. Existing immutable-version consumers need no action. Implementation owners verify the invoked producer's actual interface and conformance before adopting any documented setting or lifecycle control. <!-- ANNOUNCEMENT STOP --> ## New: Owned moving-tag recovery keeps one explicit trust boundary Use the procedure only for an intentionally moved MSX-owned major or minor alias that the consumer's artifact-specific pinning standard permits. Ownership alone does not authorize mutable references: the owned Action/reusable-workflow major-tag exception does not permit minor aliases. Exact version tags are immutable; their movement is an integrity incident, not routine recovery. SHA- or exact-version-pinned consumers require no action for this documentation delivery. If a trusted, permitted alias is stale, verify the intended remote and back up needed local-only tags outside `refs/tags/` before refreshing. Renaming a tag within that namespace does not preserve it from pruning. A verified `git bundle create owned-tags-backup.bundle --tags` followed by `git bundle verify owned-tags-backup.bundle` can retain original tag objects and reachable history; keep the backup private and outside checked-in paths. The whole-namespace refresh is `git fetch --tags --force --prune --prune-tags origin`. Force replaces local tag values with that explicitly selected remote's values, and tag pruning removes local tags absent there. Accept that complete scope deliberately, review the output, and do not normalize an altered exact version tag. Naming `origin` prevents another upstream from choosing the source of this operation. For ongoing controlled movement, inspect `remote.origin.fetch` and add `+refs/tags/*:refs/tags/*` once only when absent. Set clone-local `remote.origin.prune` and `remote.origin.pruneTags` to true, not clone-wide pruning defaults that can affect other remotes. Fetch explicitly from origin. Compare the local alias's tag-object hash with `git ls-remote --refs origin refs/tags/<alias>`, then resolve its exact release and immutable source before use. A stale or unverifiable result blocks use. This is a general conditional operating procedure, not version-specific migration history. No actual consumer tag, configuration, or history is changed by this PR. <details> <summary>Technical details</summary> | Changed surface | Standards checked | Framework docs checked | Result | | --- | --- | --- | --- | | Accept Moved Release Tags guide | Guide structure, Natural Language, Markdown | Owned-alias trust, tag pruning, immutable-source verification | Aligned | | GitHub Actions owned-tag link | Owning coding standard, Markdown | Allowed major-tag exception and consumer recovery | Aligned | | Capability indexes and navigation | Front matter, generated indexes | Documentation Model | Aligned | The earlier isolated anonymous Git experiment reproduced implicit upstream selection and pruning of a renamed local-only tag. Complete 40-character identities confirmed that an independent bundle retains the pruned ref, explicit origin restores the intended alias, and origin-scoped pruning does not remove origin-owned tags when another remote is fetched. The verification repositories contained only empty commits and test tags; no repository tags or releases were changed. The accepted-main reconciliation leaves that operating procedure intact. </details> ## Changed: Durable lifecycle and native targets require verified conformance The numbered specification retains fixed approved source and notes, permanent reservations after a successful build or external visibility, one retained artifact through verification and publication, destination progress, phase-aware retries, retirement and withdrawal, stable/candidate authority, current discovery, optional aliases, and independent post-completion announcements. Version computation uses permanent reservations and the reviewed aggregate minimum bump. Consumer record coverage separately starts from the actual latest successful artifact publication for each product, target, and channel. Authors resolve publication coverage before writing a release-bound record: verify the actual artifact and publication receipt, its immutable source provenance, and the exact candidate; compare those source trees and retain the scoped PR-base comparison separately. Include earlier merged, skipped, direct, and otherwise unshipped changes in the applicable consumer range. Successful Gallery publication advances module note coverage even if later GitHub completion fails. Completing that older GitHub record preserves its original snapshot and does not make its already-published module changes new again. These authoring baselines do not change aggregate approval, version reservations, or completed-state discovery. Consumers compose every applicable source-bound record after their actual consumed baseline through the target in release order. Notes retain prerequisites, ordering, required intermediate operations, verification, or an explicit no-action result with its compatibility basis. Reading a record does not itself require installing that version. PRs and diffs provide optional depth; resource docs describe supported target-state usage, not another migration history. Missing publication provenance or required instructions blocks the affected publication rather than becoming a no-action result. Implementation owners verify the invoked immutable producer and actual controls before relying on them. A requirement or configuration example does not prove that an older runtime implements recovery, manual release creation, candidate routing, alias reconciliation, or announcement journaling. Exact producer adoption actions belong in its own PR/release record. Existing operators retain their released interface; an incomplete release is not repaired by rebuilding under an exposed version or re-resolving changed metadata. This documentation delivers no broad lifecycle engine, history backfill, downstream propagation, or Discord sender. A duplicate or phase-aware retry reuses its original logical intent, approved source, frozen version, and retained artifact. A separately approved corrective release has a distinct immutable approved request identity, even when reviewed source is unchanged; it receives a new intent/version and preserves its relationship to the earlier attempt. Do not reuse a retry key to suppress that newly approved request, supersede an exposed intent, or reuse a reserved version. An affected product integration template requires its repository, compatible immutable commit, producer-source compatibility evidence, and delivered work or a justified no-change result. This docs-only delivery changes no producer or integration template, so concrete runtime/template adoption values do not apply here. Native publishing identities remain explicit. A PowerShell prerelease uses numeric `ModuleVersion` plus `PrivateData.PSData.Prerelease`: canonical `2.5.0-alpha` is core `2.5.0` and label `alpha`, not prerelease text in `ModuleVersion`. Validate the selected client's suffix syntax and reject or explicitly map unsupported forms without collisions or lost ordering. Native bounded ranges, where supported, can express major/minor update policies without aliases. A field without a native range resolver can retain immutable patch updates when permitted bounded selection exists outside that field. A SHA-pinned Action or reusable workflow does not need or acquire permission to retain a minor alias for that policy. Git/container reference fields do not evaluate range-like strings. Require a permitted matching alias only when no other permitted resolver supplies the bound; if none can express it, fail without widening to latest or another boundary. Retain the exact version, full SHA, or digest required by the artifact's pinning standard and trust policy. The owning VS Code framework declares stable/prerelease native mappings together before enabling Marketplace publication. Numeric versions and a native channel do not imply direct conversion from every canonical SemVer. An odd-minor convention requires reserved stable space and proven series/counter allocation. Unsupported mappings or collisions fail before publication; an enabled required destination is not silently skipped. This PR specifies the conformance boundary, not a new allocator or Marketplace publisher. <details> <summary>Technical details</summary> | Changed surface | Standards checked | Framework docs checked | Result | | --- | --- | --- | --- | | Release Management spec/design and index | Specification/design separation, Natural Language, Markdown | Durable intent, aggregate minima, distinct publication/version baselines, complete records, conformance | Aligned | | Publishing-target contract | Owning target requirements, Markdown | Two-field module mapping, native coordinates, immutable bounded selection | Aligned | | Parent/capability indexes and navigation | Generated-index conventions | Documentation Model | Aligned | The earlier native PowerShell manifest verification established numeric `ModuleVersion` and separate prerelease data; prerelease text does not parse as `System.Version`. Official Git, PowerShell, and Marketplace documentation grounded the mapping and trust corrections. No runtime publication or invented allocator is claimed. Written consumer-policy cases cover omitted policy retaining the strongest immutable reference, native minor bounds without aliases, permitted SHA-pinned patch selection without changing major/minor or retaining a minor alias, the owned-major exception not authorizing minor aliases, unsupported resolution failing without widening, immutable external latest selection, opaque range strings, and patch-only pre-1.0 selection staying inside its chosen minor. These are written conformance cases, not evidence of a delivered resolver runtime. **Current immutable candidate:** `a37884419c6c5ec4da56222be27238ee192b4cd1`. **Scoped implementation base:** accepted main `bdb581f68fddd84bc129394a94388671d1d14356`, containing delivered #209, #235, #233, and #196. The complete base-to-head comparison contains eight existing lifecycle, target, guide, and navigation files. Current canonical format, naming, testing, solution-specific exceptions, repository policy, CI configuration, and VS Code framework guidance match accepted main without an additional PR delta. This repository's Docs route is documentation validation and Pages deployment, not a versioned Release Management artifact route; no release-publication baseline or `Release decisions` block is fabricated for this PR. Local validation of the composed candidate passes all **64 functional/tooling Pester tests in six suites**, normal index regeneration, relative links across 128 pages, seven cross-repository links, exact-file Markdown lint across eight relevant pages, patch checks, and a clean build with manifest-pinned Zensical **0.0.59**. The unavailable pinned build version was restored only in an isolated temporary environment, which was removed afterward. No dependency manifest was changed. The accepted testing policy is preserved; no prose-drift tests were added or restored. The inherited-routing scenario documents the requirement but does not claim an implemented resolver test. **New-head required CI was running at handoff:** [Lint](https://github.com/MSXOrg/docs/actions/runs/41675815774/job/126055996866), [Links](https://github.com/MSXOrg/docs/actions/runs/41675815774/job/126055996479), [Test](https://github.com/MSXOrg/docs/actions/runs/41675815774/job/126055996962), [Build](https://github.com/MSXOrg/docs/actions/runs/41675815774/job/126055996918), and [Cross-repository links](https://github.com/MSXOrg/docs/actions/runs/41675815839/job/126055998868). All five checks identify `a37884419c6c5ec4da56222be27238ee192b4cd1`. Earlier green checks are not represented as a result for this new head; current status is shown by the PR checks. The previous full automated review [5728246486](#208 (review)) approved `f68927e94a31520c4db482aa8f2359cc999b80aa` with no active concerns. It is an earlier checkpoint, not a review of this candidate. This tick began with all review threads resolved and no changes-requested review, but with approval blocked. No reviewer was requested or approval manufactured during conflict maintenance. The exact resolution review preserves lifecycle authority, reservations, immutable artifact recovery, source-bound records, and consumer policy while integrating the accepted keyed-routing rules. The closing Task's original implementation-plan evidence remains in the prior delivery history. This maintenance resolves the accepted-main composition gate and leaves new-head required CI and GitHub's remaining approval/protection requirements as acceptance gates. Producer implementation, consumer execution, date strategies, historical correction, and live governance remain outside this Task. No PR merge, merge enqueue, retarget, release, or artifact promotion was performed; automatic landing remains off. </details> <details> <summary>Related references</summary> - Resolves [#227](#227) - References [#166](#166) - References [#209](#209) </details> ## Changed: Keyed routing and settings retain their accepted boundaries The lifecycle retains `ReleaseType.Stable`, `ReleaseType.Prerelease`, and `ReleaseType.ReleaseCandidate`, each with at most one branch. The stable route inherits the default branch when the routing object or stable key is omitted. Validate the effective mapping after inheritance: unknown keys and duplicate branch assignments fail, including a candidate or prerelease route assigned to the inherited stable branch. Prerelease and release-candidate routes publish only their assigned prerelease series, not stable releases or stable discovery/aliases. Configured branches retain their push-driven series, and open PRs retain `release:prerelease` using one explicit owned bump label or the valid effective `DefaultBump`. Candidate versions retain `rc` and increasing counters; promotion to stable uses the stable version resolver without carrying either prerelease counter over. An unreviewed direct push to the stable line still validates without publishing. The release solution's optional path remains `.github/release.settings.yml` under the accepted Naming standard. The convention does not implement a reader, rename a public input, require a shared directory, or restrict otherwise supported explicit custom paths. Verify the immutable producer revision's declared settings input and default before adoption. `ConfigurationFile`, where exposed, remains that producer's settings-path input. PSModule retains `.github/PSModule.yml` through `SettingsPath`, including module release settings; module repositories do not need a separate release-solution settings file. The VS Code template retains `.github/vscode-extension.yml` and its settings keys. Both owning exceptions are preserved from accepted main. This documentation does not authorize a reader, template, or live-caller migration, and creates no legacy fallback. `DefaultBump` still accepts `patch`, `minor`, or `major`, with shared `patch` as the documented default. Repository settings and explicitly supplied supported caller overrides resolve before one owned bump-label override. Omitted caller inputs preserve repository policy; invalid supplied settings and conflicting signals fail even when another input supplies a valid level. A compatible effective default needs no per-PR bump label. Skip and out-of-scope work validate without publication. The owned set remains exactly `release:patch`, `release:minor`, `release:major`, `release:prerelease`, and `release:skip`. Prerelease uses the resolved bump; skip combines with no other signal. Candidate and announcement behavior uses supported settings rather than extra labels. A route without versioned release behavior carries neither release metadata nor any release label. Authors retain the finalized PR record, release-range-first procedure, complete source-bound notes, and scoped supporting evidence already delivered on main. The no-PR-template-file policy remains in Repository Standard and its owning governance guidance, not PR Format. Product integration template evidence and genuine repository-specific contribution instructions remain separate and intact. Existing runtime consumers do not rename live files or install software because this guidance changed. Producer support and any required adoption actions must be verified and delivered by the relevant implementation owner. <details> <summary>Technical details</summary> | Changed surface | Standards checked | Framework docs checked | Result | | --- | --- | --- | --- | | Lifecycle routing requirements and design | Specification/design separation, Natural Language, Markdown | Accepted fixed routing keys, inherited defaults, branch collisions, prerelease sources and stable-only authority | Aligned | | Lifecycle settings and configuration sections | Natural Language, Markdown, Naming | Optional release-solution file, verified reader/input boundary, unchanged defaults and typed overrides | Aligned | | Accepted-base composition | Current PR Format, Testing, Repository Standard | Release-range-first records, no-template ownership, preserved PSModule and VS Code exceptions | Aligned | Conflict maintenance merges only accepted main `bdb581f68fddd84bc129394a94388671d1d14356` into the existing branch. It resolves conflicts in release spec/design/index without importing an unmerged delivery or restoring the older `release-branches` schema. The lifecycle's artifact, aggregate-approval, recovery, discovery, announcement, version-reservation, settings-precedence, and optional-file contracts remain intact. Accepted keyed-routing requirements are expressed within the existing numbered specification and design; the inherited-default collision scenario makes the effective-mapping rule explicit. Shared canonical pages and the VS Code framework match accepted main exactly. The #233 and #196 sequential-composition gates are resolved against their delivered merges, not their superseded development candidates. No other branch, historical record, live label, consumer pin, governance setting, or producer runtime was changed. New-head CI and remaining GitHub acceptance gates are still required; Agent merge maintenance does not authorize automatic landing. </details> <details> <summary>Related references</summary> - References [#196](#196) - References [#233](#233) - References [#235](#235) </details> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing guidance prioritizes observable supported behavior, real implementations, and safe actual APIs instead of assertions that merely detect implementation edits. A fast credential-free local loop remains available; protected live checks carry explicit prerequisites, authorization, isolation, cleanup, and honest coverage evidence.
The shared plugin adds a thin testing-skill pointer. This is a standards/plugin-source delivery, not a test-suite migration or consumer rollout: existing installations are unchanged, and the focused skill requires updating the shared
msxplugin after this source is merged and available.New: A focused skill routes test work to the canonical policy
msx-coding-testingroutes test design, authoring, and review to the single Testing standard. The skill copies no policy. Consumers can continue using their existing general coding route; to gain the focused trigger, update their sharedmsxplugin after the source is available. This PR does not install it for them.Technical details
The actual plugin and marketplace entry advance together from
0.1.1to0.1.2. The independently managed marketplace metadata stays0.1.0. Existing plugin tests pass (4 tests, 49 skill directories), both changed plugin Markdown files lint cleanly, and the new pointer resolves when explicitly passed to the existing link tool (1 target in 1 file)..github/plugin/msx/skills/msx-coding-testing/SKILL.md.github/plugin/msx/plugin.jsonand.github/plugin/marketplace.json.github/plugin/README.mdChanged: Testing policy distinguishes functional evidence from edit detection
Contributors and agents choose tests around supported functions, commands, and APIs without an arbitrary mocked-unit-test ratio or unnecessary remote calls for pure logic. Mocks and fakes are justified exceptions with explicit limits; a fake or an unexecuted live check does not prove that an external API works. A behavior-preserving refactor must not fail tests solely because implementation text changed.
The engineering principle links to the owning policy instead of requiring a cloud-free full suite. Independent expected results, meaningful examined-unit counts, regression intent, linting, formatting, static analysis, link checks, builds, and temporary-fixture tooling checks remain valid. Production writes and unsafe, destructive, or paid calls gain no authorization from this standard. No runtime, CI, or existing-suite migration is included.
Technical details
Scoped comparison:
MSXOrg/docsbaseaa9b5cecc69f4cbe6aa04a5dd70ff9cdadeeac3dto immutable candidate607cca0010fa08b9fee43c6f5b07c17c5a051253, in two focused ordinary commits. The complete diff contains seven files; no tests, scripts, workflows, runtime dependencies, or settings-delivery changes are imported. The worktree is clean.src/docs/Coding-Standards/Testing.mdsrc/docs/Ways-of-Working/Principles/Engineering-Practices.mdsrc/docs/Coding-Standards/index.mdValidation on the committed tree:
0.0.59, restored in an isolated environment, no issues. The global0.0.50mismatch was detected and is not the pinned build evidence; no repository dependency or shared tool was changed.2026-11-02T18:38:24Z, covering all seven changed files. The explicit review loop returned clean with zero new comments; complete review-thread pagination returned no threads.The scoped Task records baseline/scenario reasoning, both commits, local checks, CI/review evidence, and completed standards/convergence review. Every implementation-plan item is complete; the Task remains open for its normal reviewed-merge completion path. Targeted open issue/PR searches for testing, mocks, and functional tests found no additional fully convergent issue; the settings and release deliveries remain separate. No policy-wording or structural-content tests were added, and no existing suites were changed or deleted.
Native Agent merge was verified before enabling and read back afterward:
enabled=true,address_reviews=true,fix_ci=true,resolve_conflicts=true,merge_pr=false. This is maintenance-only authorization. The PR remains draft by request with GitHub auto-merge unset; advancing readiness and any later merge require the appropriate user/owner decision. No merge, enqueue, service publication, permission/rule/label change, or consumer rollout was performed or authorized. No protected live-API experiment campaign was run or claimed. The inspected Docs route only publishes Pages on non-PR events and does not invoke Release Management; no release-decision block or labels are applied.Related references