[Marius Storhaug (MariusStorhaug)]

Focused framework checks now verify release-note content and authored-page transfer instead of fixed log sentences, documentation phrases, or a private helper parameter inventory. Version outputs, failure handling, recovery, WhatIf behavior, module fixtures, and the shipped validation suites remain covered.

This is a scoped test/CI cleanup. Production scripts, consumer inputs, documentation, coverage targets, permissions, and release labels are unchanged. The PR stays in draft because the existing producer-conformance and publication-evidence holds are not resolved here.

Technical details

Scope and immutable identities

Repository: PSModule/Process-PSModule. Default-branch base: 64ffb1d87f86bd5d487891fdb140b78385a00531. Reviewed head: 7a00cbb9cd72baddf14d990019cf5da3c9c3600e. Seven small ordinary commits change five files. The worktree is clean and the head is pushed. No open PR branch is a source.

Changed file	Result
.github/actions/Release-PSModule/tests/Release-PSModule.WhatIf.Tests.ps1	Replace a diagnostic-phrase assertion with real Git commit selection and notes-file content observed at an intercepted GitHub write boundary. Use two different commits, verify the selected target, reject unexpected WhatIf calls, and clean up the shim after each case.
.github/actions/Publish-PSModule/tests/Publish-PSModule.Helpers.Tests.ps1	Remove the helper parameter-inventory guard. All six SemVer output/invalid-input cases remain.
.github/workflows/Workflow-Test-Default.yml	Compare the generated root landing page with its nonempty authored input instead of three fixed phrases.
.github/workflows/Workflow-Test-WithManifest.yml	Apply the same input/output contract to the manifest fixture.
.github/workflows/Test-Actions.yml	Remove tautological configuration echoes and the later configuration reset. Run the configured shuffle/seed/start markers, with explicit serial file execution because the suites share process environment variables. Keep discovery and failure gates.

Complete audit inventory

PSModule/Process-PSModule#550 contains the complete source-bound classification of every assertion and invoked test/discovery entry point, including counts, real entry points, expectations, and policy reasons. Counts below describe the audited source; no fixed repository-inventory assertion is added.

Surface	Disposition and contract
Get-PSModuleSettings (13 cases)	Retain real event routing, exact merged-commit association, controlled tree/compare outputs, and incomplete-response failures.
Resolve-PSModuleVersion (45 cases)	Retain real JSON normalization, version baselines/composition, next versions, context, and label-conflict failures. Version strings and controlled response counts are functional expectations, not snapshots. Existing anonymous Gallery reads do not independently prove the Gallery service contract.
Publish helpers (6 cases)	Retain stable/prerelease SemVer, whitespace handling, and invalid empty input. Remove only the parameter-absence guard.
Publish recovery (3 cases)	Retain real script branching for simulated existing/absent packages and lookup failures. Unsafe vendor writes remain intercepted.
Release (4 cases)	Retain action outputs, dry-run target, and simulated resume inputs. Rewrite manual dispatch to observe notes from the selected real commit rather than log prose.
update-index (1 case)	Retain actual catalog/page generation and links from controlled repository metadata. Owned remote metadata boundaries remain simulated.
Shipped SourceCode suite (10 definitions, data-driven links)	Retain all documented static-analysis IDs. This is a supported validation product, not an edit-detection assertion against its own source.
Shipped Module suite (7 definitions, data-driven types)	Retain actual import/manifest/version-folder validation, type registration, and OnRemove cleanup. A verified pre-existing discovery/oracle gap is tracked separately in #553.
Both containers and configurations	Retain the actual Test-PSModule/Invoke-Pester discovery, report, and coverage wiring.
Both fixture module suites (5 cases each)	Retain real exported greeting-function and SolarSystems variable behavior.
Both fixture Environment suites (2 cases each), TestData.Assertions, BeforeAll/AfterAll	Retain exact controlled environment transport expectations in tests and phases. Fixture values/lengths are input data, not source snapshots.
Two generated-index workflow checks	Rewrite six frozen phrase checks as an authored-input transfer contract. Different headings/prose are accepted; lost content and empty inputs fail.
Other workflow assertions and quality gates	Retain test-result/coverage, schema, lint, build, setup/teardown, and missing/unexecuted-result controls. Existing fixture skip settings are unchanged. Guidance benchmarks/runtime examples and site JavaScript add no undiscovered assertion runner.

Validation and evidence limits

Check	Result and boundary
Exact-head action CI	Run 41827042422: 72 passed, 0 failed, 0 skipped, Pester 6.2.0 within the unchanged [6.1.0,7.0.0) range. The baseline had 73 cases; the sole removed case was the parameter inventory.
Local action loop	The actual workflow test step passes with installed Pester 6.1.0, shuffled order, and serial files. Pester rejects an empty selection rather than reporting a pass.
Release fault checks	Wrong notes fail. Reading HEAD instead of CommitSha fails with the different newer commit's text. A diagnostic-only wording change passes. All experimental production edits were restored.
Documentation contract	The real generator and the exact two workflow assertion scripts pass 12 controlled checks: original/changed authored wording accepted for both layouts, lost content rejected, empty input rejected. Hashes compare input and output files, not a stored snapshot or generator source.
Retained SourceCode validator	A valid isolated source fixture passes 10/10; an injected OutNull defect produces the intended single failure.
Retained Module validator	A real source-bound CI artifact passes 26/26. Removing registration in an isolated copy produces 21 intended failures. A comment-driven false green is documented in #553; the shipped suite is not removed or weakened.
Parallel-isolation probe	The actual Pester 6.1 parallel runner uses the same process for both controlled workers; one worker observes the other's environment value and fails. Serial execution preserves the baseline's actual mode without suppressing a test.
Script/workflow lint	Clean-profile, Pester-aware PSScriptAnalyzer 1.25.0: 0 findings. Actionlint 1.7.12: pass. Exact-head LF blobs with the pinned CI YAML rules: 0 errors, 23 warnings. No lint configuration or suppression is changed.
Manual Zizmor	The same four existing broad-permission findings occur on base and head. They are tracked in #552; no permission change is authorized in this PR.
Exact-head repository CI	52 successful checks, 7 existing conditional skips, no failed or pending checks. Both fixture workflows, root-index checks, test-result/coverage collection, lint, and CodeQL succeed.

The seven conditional skips are the two fixture publication jobs of each kind, the default fixture's repository lint, and the manifest fixture's source test/lint jobs. These are not new passes. Live Gallery publication, real GitHub release writes, publication provenance, and safety-owner handback remain unexecuted here. Test doubles prove local caller handling only; they do not prove an upstream service, authentication, or safe publication ownership.

The compiled-fixture check first detected a missing ThreadJob 1.x dependency. Only that declared dependency was restored to isolated scratch storage; no global installation changed. An early Windows scratch-build attempt hit formatter line-ending limits, so the successful retained-Module evidence uses the real Ubuntu CI artifact from run 41824863677/source dbd162f0001b73cfa03207dd18f0ce607adc6114. The compiler and shipped validator are unchanged between that source and the final head. An inherited-shell analyzer run also emitted an intermittent Context/Get-Context resolution warning; baseline/candidate rechecks returned 0/0 and clean-profile analysis returned 0, without a suppression.

Review disposition

Author self-review covers the complete scoped diff. The first independent automated review identified two valid gaps: the selected commit was also HEAD, and parallel execution could race on process environment. Both were fixed, exercised with deliberate counterexamples, replied to, and resolved.

Copilot review 5768041720 is APPROVED for the exact final head 7a00cbb9cd72baddf14d990019cf5da3c9c3600e; its complete summary reports no active issues and both prior findings resolved. No new inline finding remains. This is an automated approval, not a claimed human approval. The author-side reply records are not counted as independent reviews.

Standards and framework alignment

Changed surface	Standards checked	Framework documentation checked	Result
Release/Publish action tests	MSX Testing, PowerShell Testing, naming, functions, error handling, natural language	Existing action contracts/help and version/release behavior	Fixed in this PR: real behavior replaces incidental phrase/signature checks; simulation limits are explicit.
Generated-index workflow assertions	MSX Testing, YAML, GitHub Actions	Actual Build-Docs route and Build-PSModuleDocumentation authored-Markdown transfer	Fixed in this PR: complete input/output preservation replaces frozen phrases.
Test-Actions runner	MSX Testing, PowerShell Testing, GitHub Actions	Writing module tests and current Pester/action contract	Fixed in this PR: real configured execution; serial files retain isolation.
Existing fixture-workflow permissions	GitHub Actions least privilege	Actual called workflow permission contract	Exception: #552. No permission or required-control change is included.
Retained shipped type validation	MSX Testing independent oracle and meaningful examined cases	Existing Module validation contract	Exception: #553. Separate supported-contract repair, not a compiler/publisher rewrite here.

Relevant canonical policy files were unchanged when refreshed through MSXOrg/docs@1b0d4893c8fefabe604d04393068a8ff6f848082. Existing guidance already describes the scoped runtime contracts; no documentation/native-help path needs a cleanup edit.

The Task's audit, focused implementation, local verification, CI, and exact-head automated review steps are complete. The issue-convergence search covered functional-test alignment, snapshot/change-detection terms, touched components, and the two new follow-ups. Only #550 is fully addressed. #184 and all unrelated/held proposals remain open and unchanged.

Producer hold

The actual repository release route is .github/workflows/Release.yml calling PSModule/Release-GHRepository@750abed0ecab02394554b610dc3aa191e75e3248 (v2.0.3). Its immutable released interface uses legacy labels; no adapter to the canonical release-decision contract is verified. Publication/source baselines for this versioned workflow producer are not established by this scoped diff.

The draft therefore makes no release decision, release-label, final version, consumer-publication coverage, or ready-for-merge claim. The existing producer-conformance/publication-evidence holds remain intact. No merge, auto-merge, label removal, publication, permission change, or held-branch integration was performed. Original-tested-artifact and safety-owner decisions are unchanged.

Related references

• Resolves [functional-test-alignment] Prove framework behavior without freezing implementation text #550
• References Scope fixture workflow permissions to their calling jobs #552
• References Module validation hides missing type registration when an exporter comment changes #553
• References MSX Testing standard

[Copilot]

🟡 Changes recommended

Parallel test suites can race on shared environment variables, and the release-notes test cannot detect whether commit-message selection ignores CommitSha.

2 open issues

• Test does not catch CommitSha being ignored for release notes · New
• Parallel tests race on shared GitHub environment variables · New

What changed in this PR

This PR advances the framework’s test-alignment audit by checking observable release behavior and generated documentation rather than fixed wording or implementation details. Production scripts and consumer inputs are unchanged.

Changes:

• Verify release-note contents and target commits at an intercepted GitHub command boundary, and reject GitHub calls in WhatIf cases.
• Compare generated Functions indexes with authored fixtures using file hashes.
• Remove implementation-focused assertions and preserve the configured Pester options through execution.

File	Description
.github/​workflows/​Workflow-Test-WithManifest.yml	Verify authored Markdown is preserved in manifest-based builds.
.github/​workflows/​Workflow-Test-Default.yml	Verify authored Markdown is preserved in default builds.
.github/​workflows/​Test-Actions.yml	Remove configuration self-checks and retain Pester execution settings.
.github/​actions/​Release-PSModule/​tests/​Release-PSModule.WhatIf.Tests.ps1	Assert release notes and target arguments using a real Git fixture and intercepted GitHub calls.
.github/​actions/​Publish-PSModule/​tests/​Publish-PSModule.Helpers.Tests.ps1	Remove a parameter-introspection assertion.

_{Review effort: Balanced}

[Copilot]

🟢 Approved

No active review issues are identified for the changes aligning tests with observable behavior.

No open issues 🎉

2 resolved since last review

• Parallel tests race on shared GitHub environment variables
• Test does not catch CommitSha being ignored for release notes

_{Review effort: Balanced}