Repository navigation
ci: add release publishing scripts and an agent runbook - #2628
Rana Singh (ranadeepsingh) wants to merge 12 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Hey Rana Singh (@ranadeepsingh) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
c763701 to
2822d0b
Compare
## Summary Refresh the release automation on current master, align it with the Fabric release guide and the live Publish-Official pipeline, and add proof-oriented coverage for release identifiers, artifact verification, GitHub workflows, and BBC-VHD edits. ## Prompting Intent Refresh microsoft/SynapseML PR microsoft#2628 using the SynapseML PR readiness loop, follow the internal Fabric release guide, and derive automation from actual prior OSS, Internal, Publish-Official, and BBC-VHD releases rather than relying on stale examples. ## Linked Sources - Fabric release guide: https://msdata.visualstudio.com/A365/_wiki/wikis/Osmos%20Team%20Wiki/130638/SynapseML-Fabric-Release-Guide-v2 - Release automation PR: microsoft#2628 - Derivative tag automation: microsoft#2540 - Live v1.1.3 release: https://github.com/microsoft/SynapseML/releases/tag/v1.1.3 - Publish-Official pipeline: https://msdata.visualstudio.com/A365/_build?definitionId=35879 - Historical BBC-VHD release PR: https://msdata.visualstudio.com/A365/_git/BBC-VHD/pullrequest/1805064 ## Rationale Keep ESRP, review, White-Glove, and train decisions human-gated while automating deterministic mechanics. Tag the exact reviewed merge, dispatch downstream GitHub automation explicitly because GITHUB_TOKEN pushes do not recurse, gate release notes on public artifacts, use the live pipeline parameter contract, fail loudly on incomplete network evidence, and roll back paired BBC-VHD writes rather than leaving partial release state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
This PR adds release-engineering automation to make SynapseML-to-Fabric releases more deterministic and reviewable, including a GitHub Actions “prepare → tag merged commit → publish notes” flow plus supporting Python tooling to derive/verify release identifiers and safely bump BBC-VHD pins.
Changes:
- Introduces a release matrix generator (
release_matrix.py) and an end-to-end verifier (verify_release.py) with accompanying tests. - Adds
bump_bbcvhd.py(plus tests) to safely update BBC-VHD component pins while preserving line endings and rolling back on failure. - Adds/updates GitHub workflows for release prepare/tagging and manual, artifact-gated release notes; enables
workflow_dispatchfor PR validation so release PRs created viaGITHUB_TOKENcan be validated.
Show a summary per file
| File | Description |
|---|---|
| scripts/test_bump_version.py | Extends bump-version tests for denylisted repo-relative paths; normalizes path keys to POSIX. |
| scripts/bump-version.py | Improves Windows console encoding robustness; adds path-based denylist support; normalizes path handling via as_posix(). |
| scripts/release/release_matrix.py | Adds a single-source-of-truth release matrix for tags and artifact versions + CLI rendering. |
| scripts/release/verify_release.py | Adds live verification of GitHub/ADO tags and Maven/PyPI/UPack/Azure Artifacts presence. |
| scripts/release/bump_bbcvhd.py | Adds deterministic, rollback-safe BBC-VHD component pin updater with CRLF/LF preservation. |
| scripts/release/README.md | Documents how to use the new release tooling and where it fits in the guide. |
| scripts/release/test_release_matrix.py | Adds regression/contract tests for the release matrix derivations and CLI validation. |
| scripts/release/test_verify_release.py | Adds unit tests for verifier networking/error handling and run plan behavior. |
| scripts/release/test_bump_bbcvhd.py | Adds tests for BBC-VHD bump idempotency, rollback, and newline preservation. |
| scripts/release/test_release_workflows.py | Adds tests asserting key workflow contract properties (manual gating, dispatch behavior). |
| scripts/release/test_prev_tag.sh | Adds a repo-taglist regression script for “previous primary tag” selection logic. |
| .github/workflows/release-prepare.yml | Adds a reviewed release PR generator and a merged-commit tagger that dispatches downstream orchestration. |
| .github/workflows/release-notes.yml | Adds a manual, artifact-gated GitHub Release publisher with explicit previous-tag selection. |
| .github/workflows/pr-validation.yml | Enables workflow_dispatch so validations can be dispatched for bot-opened release PR branches. |
Review details
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Suppressed comments (1)
scripts/release/verify_release.py:260
--skip internaldoes not currently skip the internal PyPI feed checks (thesynapseml_internalpackage); it only skips internal git tag checks. This makes--skip internalbehave inconsistently compared with--skip public.
def pip(self, package: str, version: str) -> str:
if "pip" in self.skip or "ado" in self.skip:
return SKIPPED
# Azure Artifacts normalises pypi names: synapseml_internal -> synapseml-internal
return (
OK
if version
in self._feed_versions("Synapse-Conda", "pypi", package.replace("_", "-"))
else MISSING
)
- Files reviewed: 14/14 changed files
- Comments generated: 2
- Review effort level: Lite
## Summary Make --skip internal consistently skip SynapseML-Internal tags, UPacks, and wheels while retaining OSS artifact checks, and document every skip scope with regression coverage. ## Prompting Intent Resolve all current-head automated review findings on microsoft/SynapseML PR microsoft#2628 and preserve an explicit, safe release-verification CLI contract. ## Linked Sources - Release automation PR: microsoft#2628 - Internal artifact skip review: microsoft#2628 (comment) - Skip help review: microsoft#2628 (comment) ## Rationale A release operator who opts out of Internal verification must not still query or fail on Internal packages. Passing artifact scope explicitly keeps OSS checks active, avoids hiding public release gaps, and makes combined skip behavior predictable from both CLI help and maintainer documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
scripts/release/verify_release.py:75
- On Windows,
subprocess.run(..., shell=True)expects a command string, but this code passes a list. That combination can fail (or behave differently) on win32, which would break the fallback ADO auth path even whenazis installed. Build a platform-appropriate command (string forshell=True, list otherwise).
out = subprocess.run(
[
"az",
"account",
"get-access-token",
"--resource",
ADO_RESOURCE,
"--query",
"accessToken",
"-o",
"tsv",
],
capture_output=True,
text=True,
shell=(sys.platform == "win32"),
)
- Files reviewed: 14/14 changed files
- Comments generated: 0 new
- Review effort level: Lite
## Summary Build the Azure CLI authentication command as a string only when Windows requires shell execution, while retaining an argument list and shell-free execution on other platforms. Add regression coverage for both command shapes. ## Prompting Intent Resolve the current-head suppressed Copilot review finding on microsoft/SynapseML PR microsoft#2628 and prove the fallback authentication path used by release verification on Windows. ## Linked Sources - Release automation PR: microsoft#2628 - Fabric release guide: https://msdata.visualstudio.com/A365/_wiki/wikis/Osmos%20Team%20Wiki/130638/SynapseML-Fabric-Release-Guide-v2 ## Rationale Windows resolves the Azure CLI through a command shim, so the verifier needs shell execution there; subprocess requires that shell command to be a string. Keeping list-based execution elsewhere preserves safe argument boundaries, while an actual token-less Windows replay proves the fallback can authenticate and inspect the live release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed the current-head suppressed Windows auth finding in 6e80f27.
|
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
scripts/release/verify_release.py:313
- The Maven publication gate here verifies only
synapseml-core_{scala}(viapublic_maven()), but the workflow-generated release notes advertisecom.microsoft.azure:synapseml_2.12:${VERSION}(see.github/workflows/release-notes.yml:137) and the docs use thesynapseml_2.12coordinate (website/doctest.py:25). This can letverify_release.pyreport COMPLETE even if the primary install coordinate is missing (or vice versa). Consider aligning the verification to check the same Maven artifact(s) that users are instructed to consume (e.g., verify bothsynapseml_{scala}andsynapseml-core_{scala}, or switch to the canonical one).
"maven",
tp.key,
f"synapseml-core_{tp.scala}",
tp.oss_maven_version,
c.public_maven(tp.scala, tp.oss_maven_version),
- Files reviewed: 14/14 changed files
- Comments generated: 0 new
- Review effort level: Lite
## Summary Gate release completeness on both the user-facing synapseml_<scala> Maven coordinate and the release-guide synapseml-core_<scala> coordinate for every selected Spark target. Add a failing-install-coordinate regression and document the two-coordinate contract. ## Prompting Intent Resolve the current-head suppressed Copilot review finding on microsoft/SynapseML PR microsoft#2628 by aligning artifact verification with generated release notes, installation docs, and actual published releases. ## Linked Sources - Release automation PR: microsoft#2628 - Fabric release guide: https://msdata.visualstudio.com/A365/_wiki/wikis/Osmos%20Team%20Wiki/130638/SynapseML-Fabric-Release-Guide-v2 - Live v1.1.3 release: https://github.com/microsoft/SynapseML/releases/tag/v1.1.3 ## Rationale The aggregate synapseml artifact is the coordinate users install and the one release notes advertise, while the guide explicitly points maintainers to synapseml-core. Requiring both prevents a GitHub Release from reporting complete when either the public install contract or the guide's Maven evidence is absent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Audited the current-head review overview's concern about a missing queue-time
Relevant current-head code: I executed the real guard against an isolated temporary Git repository with a The four actual review threads are fixed and resolved in |
|
Current-head CI is complete for
This is not yet approval to merge or publish. Both repository release-App |
## Summary Extend the existing SynapseML release skill with an agent checkpoint runbook and a cross-platform, credential-free rehearsal command. Preserve explicit human authority for live preparation, immutable tags, publication, signing, release notes and merges. ## Prompting Intent The engineer chose the agent-runbook route for publishing, together with the existing automation/scripts and dry-run tests. Implement that route without interpreting the request as approval to publish a production release. ## Linked Sources - Release automation PR: microsoft#2628 - Existing operator guide: scripts/release/README.md - Existing release skill: .github/skills/synapseml-release/SKILL.md - Existing guards and ledger: scripts/release/bootstrap_release.py and release_ops.py - Prior exact-head CI/readiness: microsoft#2628 (comment) - User request: "sure let's do the agent run book route for publishign then along with the automations and scripts and dry run tests" ## Rationale Reuse the guarded publication engine and its test doubles instead of adding another approval or publishing state machine. Offline tests exercise real plan/ledger/bootstrap/receipt code with simulated services and disposable Git repositories, and explicitly deny live network and non-Git subprocess calls. The runner has no production inputs, preserves existing report/ledger files, and fails on missing tools, timeout, zero tests, failures or skips. Distinguish pretag bootstrap preview from posttag publication preflight and retain one authoritative ledger/operator across agent sessions. Both native Windows and Linux rehearsals passed all 12 scenarios. The full Linux release/pipeline suite passed 1227 tests, with the existing optional SBT skip and Windows-worktree notebook deselection. Native Windows covered the actual notebook case and runner edge cases with 74 passes. Pinned Black and link checks passed. A harness-selected Low incremental review reported no significant issues; this is not a new full-PR multi-model review. This is release infrastructure, labeled area/build, with no linked ADO task. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command. |
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
|
The current-head review reports no inline findings, but its overview mentions
That is a tag reference, not a link to a separate GitHub Release object. The offline rehearsal also executes real bootstrap tag creation and idempotent No code change is justified by the overview as written. Current-head Azure |
|
Agent runbook and offline rehearsal are implemented in
Use the agent runbook |
## Summary Close full-PR review findings in the agent-operated release process. Bind public artifact bytes to separate CDN and Central producer receipts, preserve ambiguous submissions and existing release branches, and reject partial-coordinate republication and inconsistent runtime or timeline evidence. Restore source-bound primary API docs before package upload, disclose the public wheel in approvals, and cover version preparation in CI. ## Prompting Intent The engineer requested a fleet review of all microsoft#2628 changes, an up-to-date branch and PR description, and reliable automation scripts and an agent runbook for publishing new versions. This work does not authorize production publication, signing, or merges. ## Linked Sources - Automation PR: microsoft#2628 - Notebook archive issue: microsoft#2346 - Current validation baseline: microsoft#2752 - Operator guide: scripts/release/README.md - Agent runbook: .github/skills/synapseml-release/references/agent-runbook.md - Six fleet findings and resolutions: reviews/pr-2628/task-pr2628-fleet-attempt-1-stage-attempt-1-review-{1,2,3,4,5,6}-harness.md ## Rationale Retain existing plan approval identities and one publisher instead of adding a separate agent publishing engine. Store destination-specific producer hashes because independently built Maven outputs need not be byte-identical. Verify downloaded bytes rather than treating availability as provenance. Preserve failed and ambiguous state rather than deleting history or retrying immutable Maven coordinates. Keep same-wheel qualification and production authority explicit human gates. Preserve all review findings and append their resolution evidence, including the withdrawn unproven qualification claim. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
## Summary Run the rehearsal child with an explicit environment allowlist rather than inheriting release credentials, production inputs or Python overrides. Add a regression using synthetic credentials and document the boundary. ## Prompting Intent The engineer requested reliable agent-operated publishing automation, offline dry runs and resolution of all review findings on microsoft#2628. The current-head hosted review identified inherited credentials in the otherwise credential-free rehearsal. ## Linked Sources - PR: microsoft#2628 - Environment finding: microsoft#2628 (comment) - Rebutted JSX finding: microsoft#2628 (comment) - Passing website build: https://github.com/microsoft/SynapseML/actions/runs/37855557257/job/113578741269 - Resolution evidence: reviews/pr-2628/task-pr2628-fleet-attempt-1-stage-attempt-1-review-2-harness.md ## Rationale An allowlist excludes future credential names without relying on an incomplete secret-name denylist. Preserve the process and filesystem environment needed by Python and disposable Git fixtures on Windows and Linux. Keep the actual service isolation and no-publication-authority contract unchanged. Do not change valid literal JSX text to satisfy an incorrect identifier finding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
|
Fleet review and corrective validation are complete at The six committed review reports retain the findings and resolutions. Corrections Final evidence:
Engineering work is complete; production publication is not authorized by |
Type: infrastructure PR label: area/build ## Summary Remove duplicate release implementations, test setup and operator instructions. Keep one runbook and a short discovery skill, deleting four overlapping references and the shadow predecessor-tag script. Preserve the original review evidence and record the six fresh audits with their resolutions. Fix the audit's reproduced defects: predecessor selection under pipefail, overbroad archive version replacement, malformed PyPI metadata, unbounded JSON responses, missing token timeout, independent PyPI visibility in tests, and native Windows bootstrap fixture assumptions. ## Prompting Intent The engineer asked to rebase microsoft#2628 onto master before a fresh fleet reviewed and revised every changed file to reduce PR bloat. Keep the publishing scripts and agent-runbook route reliable, preserve approval and immutable-publication safeguards, update the PR, and push the validated changes without publishing a production release. ## Linked Sources - Release automation: microsoft#2628 - Notebook archives: microsoft#2346 - Release operator guide: scripts/release/README.md - Fresh audit and resolution reports: reviews/pr-2628/task-pr2628-trim-attempt-1-stage-attempt-1-review-{1..6}-harness.md - Repository contribution and review rules: AGENTS.md and .github/skills/synapseml-pr-loop/ ## Rationale Reuse the existing inventory description and strict JSON parser instead of maintaining parallel implementations. Consolidate repeated fixtures and assertions while retaining actual workflow execution, destination-specific download proof, realistic evidence budgets and saved-plan identities. Do not retire supported private operations, discard historical reviews or weaken immutable-package recovery simply to reduce the line count. The combined source, test and documentation changes remove 1,248 net lines; required new review reports are accounted for separately. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Type: infrastructure PR label: area/build ## Summary Normalize the older pathlib RuntimeError for cyclic profile symlinks at the existing profile-loading boundary. Keep the matrix CLI's controlled exit 2 and sanitized error, and preserve public-only operation without a profile. Add portable exception-normalization and real cyclic-link CLI regressions. Append the reproduced finding and resolution to the existing contract review. ## Prompting Intent The engineer requested a complete review and reduction of the release automation PR, with reliable publishing scripts and an agent runbook. Investigate the current-head automated review's unresolved overview claims rather than treating zero inline comments as proof that no defect remains. ## Linked Sources - Release automation and current review: microsoft#2628 - Contract audit and resolution: reviews/pr-2628/task-pr2628-trim-attempt-1-stage-attempt-1-review-2-harness.md - Shared profile input boundary: scripts/release/release_config.py ## Rationale Missing files, directories, invalid JSON and invalid encoding already return controlled errors. On older Python, a cyclic symlink instead raises RuntimeError before the matrix CLI's ValueError handler. Fix the shared input boundary, not a broad outer CLI catch. Other runtime failures remain visible, and no publication behavior or approved-plan identity changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Checked both concerns named in the The three previously reported Maven wiring cases remain fixed. Release jobs do not invoke The profile concern exposed one real edge case on older Python: resolving a cyclic symlink raised The updated head is being revalidated. This work has not published a production release. |
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
|
The bloat review is complete at The revision removes 1,189 net implementation, test and documentation lines and five duplicate files. Release guidance shrinks from 1,218 to 605 lines in one runbook plus its discovery skill. The 43 historical reports remain intact; six new reports add 659 lines, so the complete PR shrinks by 530 net lines rather than 1,189. Supported private operations, saved-plan identities and immutable-publication safeguards remain in scope. Final validation
Azure validated merge The current review's SBT finding was a source-binding test misread and is resolved with fixture and execution evidence. The earlier vague profile claim exposed a real cyclic-symlink error path, which was fixed and revalidated. The PR title and template-based description now reflect the reduced scope and current evidence. Maintainer approval, App/publisher setup, candidate qualification and the explicitly approved pre-merge live trial remain separate gates. No production release, signing approval or merge was performed. |


Related Issues/PRs
Related to #2346. New releases include notebook archives;
historical backfill remains separate. Includes #2752.
What changes are proposed in this pull request?
Add scripts and an operator runbook
for publishing SynapseML from master and Spark 4.1, with Spark 4.0 available
explicitly. An agent can prepare and track a release, but cannot approve itself
to publish.
PRs and immutable tags without replacing existing branches.
notebook archives. Require exact-plan approval and retain submission state
so interrupted runs cannot blindly resubmit immutable packages.
completing a release. Keep existing saved-plan identities readable.
regressions in CI, fix aggregate-wheel image-byte conversion, and correct
installation examples used during version preparation.
No release has been published. Keep this PR unmerged until the approved
pre-merge trial is complete. App configuration, publisher access, candidate
qualification on every selected runtime, exact-plan approval, signing, and
maintainer merges remain separate gates. Offline rehearsal does not clear them.
The final website artifact-lock update follows verified publication.
How is this patch tested?
Head:
25bad376c0a1b1c99ba0d3f4d1fb8f93cd0e9cdc. Current-head GitHubvalidation passed, including 1,702 release tests with SBT enabled.
Azure build 239752260
succeeded: all 65 jobs passed, with 3,695 tests passed and 22 skips matching
the preceding validated build by suite and count. All 377 test runs completed.
The build validated merge
b285060052872f6a4321e694cedda2b0f2788675, whoseparents are this head and current master
861c3a1e14.No required checks are missing, failed or pending. No review threads or
suppressed findings remain unresolved. The bot's overview still links the
resolved source-binding false positive;
the fixture and executed SBT test establish why no code change was needed.
Maintainer approval and the live release gates above remain outstanding.
Validation commands and results
python -m pytest scripts/release scripts/test_bump_version.py tools/ci/tests/test_pipeline_yaml.py tools/ci/tests/test_e2e_impact.py -q --tb=short -p no:cacheprovider -k 'not TestHistoricalReplay and not current_committed_notebooks_are_admissible'python -m pytest scripts/test_bump_version.py scripts/release/test_release_dbc.py -q --tb=short -p no:cacheprovider -k 'TestHistoricalReplay or current_committed_notebooks_are_admissible'python scripts/release/release_dry_run.py --report "$REPORT"python -m pytest scripts/release/test_release_dry_run.py scripts/release/test_release_rehearsal.py scripts/release/test_public_release_docs.py scripts/release/test_release_config.py -q --tb=short -p no:cacheprovidernpm --prefix website testSYNAPSEML_DOCS_PREVIEW=true.$env:SYNAPSEML_DOCS_PREVIEW='true'; npm --prefix website run buildpython -m black --check --extend-exclude 'docs/' .python -m pytest scripts/release scripts/test_bump_version.py tools/ci/tests/test_pipeline_yaml.py -qReview history and earlier validation
Review reports and resolutions
retain the original findings. Six fresh, disjoint audits covered every changed
file before the simplification pass. Reviewer model selection was left to the
runtime, not claimed as a complementary or three-model review.
The simplification removes 1,189 net implementation, test and documentation
lines, including five duplicate files. Release instructions shrink from 1,218
to 605 lines. The 43 historical reports remain intact; six new reports document
the audit. These records explain why the complete PR diff remains larger than
the executable changes.
Earlier validation
belongs to head
800589ac8dand is historical evidence only.Does this PR change any dependencies?
Add
actions/create-github-app-tokenpinned tobcd2ba49218906704ab6c1aa796996da409d3eb1and verify the SBT launcher with acommitted checksum. Release CI installs the existing Hypothesis dependency;
API-doc publication restores Graphviz/Doxygen prerequisites. Runtime dependency
pins are unchanged.
Does this PR add a new feature? If so, have you added samples on website?
Release infrastructure only. Operator commands and installation examples are
documented; no estimator or transformer API is added.
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.npm startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.