Skip to content

ci: add release publishing scripts and an agent runbook - #2628

Open
Rana Singh (ranadeepsingh) wants to merge 12 commits into
microsoft:masterfrom
ranadeepsingh:release-automation-plan
Open

Rana Singh (ranadeepsingh) wants to merge 12 commits into
microsoft:masterfrom
ranadeepsingh:release-automation-plan

Conversation

@ranadeepsingh

@ranadeepsingh Rana Singh (ranadeepsingh) commented Aug 14, 2026 •

Copy link
Copy Markdown
Collaborator

Related Issues/PRs

Related to #2346. New releases include notebook archives;
historical backfill remains separate. Includes #2752.

What changes are proposed in this pull request?

Add scripts and an operator runbook
for publishing SynapseML from master and Spark 4.1, with Spark 4.0 available
explicitly. An agent can prepare and track a release, but cannot approve itself
to publish.

  • Prepare version changes, validate source/runtime bindings, and create release
    PRs and immutable tags without replacing existing branches.
  • Publish Maven packages, the primary PyPI wheel, API docs, and validated
    notebook archives. Require exact-plan approval and retain submission state
    so interrupted runs cannot blindly resubmit immutable packages.
  • Verify downloaded public artifacts against their producer receipts before
    completing a release. Keep existing saved-plan identities readable.
  • Exercise the process offline without inherited credentials. Include release
    regressions in CI, fix aggregate-wheel image-byte conversion, and correct
    installation examples used during version preparation.

No release has been published. Keep this PR unmerged until the approved
pre-merge trial is complete.
App configuration, publisher access, candidate
qualification on every selected runtime, exact-plan approval, signing, and
maintainer merges remain separate gates. Offline rehearsal does not clear them.
The final website artifact-lock update follows verified publication.

How is this patch tested?

  • I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Head: 25bad376c0a1b1c99ba0d3f4d1fb8f93cd0e9cdc. Current-head GitHub
validation passed, including 1,702 release tests with SBT enabled.
Azure build 239752260
succeeded: all 65 jobs passed, with 3,695 tests passed and 22 skips matching
the preceding validated build by suite and count
. All 377 test runs completed.
The build validated merge b285060052872f6a4321e694cedda2b0f2788675, whose
parents are this head and current master 861c3a1e14.

No required checks are missing, failed or pending. No review threads or
suppressed findings remain unresolved. The bot's overview still links the
resolved source-binding false positive;
the fixture and executed SBT test establish why no code change was needed.
Maintainer approval and the live release gates above remain outstanding.

Validation commands and results
Command Result on the reduced patch
python -m pytest scripts/release scripts/test_bump_version.py tools/ci/tests/test_pipeline_yaml.py tools/ci/tests/test_e2e_impact.py -q --tb=short -p no:cacheprovider -k 'not TestHistoricalReplay and not current_committed_notebooks_are_admissible' Linux: 1,811 passed. One opt-in SBT test skipped locally and executed in GitHub CI; 11 Git-dependent cases covered below.
python -m pytest scripts/test_bump_version.py scripts/release/test_release_dbc.py -q --tb=short -p no:cacheprovider -k 'TestHistoricalReplay or current_committed_notebooks_are_admissible' Windows: 11 passed.
python scripts/release/release_dry_run.py --report "$REPORT" Windows and Linux: 12 scenarios passed each, zero skips, separate new report files.
python -m pytest scripts/release/test_release_dry_run.py scripts/release/test_release_rehearsal.py scripts/release/test_public_release_docs.py scripts/release/test_release_config.py -q --tb=short -p no:cacheprovider Windows: 110 passed; also included in the Linux suite.
npm --prefix website test 39 passed normally and with SYNAPSEML_DOCS_PREVIEW=true.
$env:SYNAPSEML_DOCS_PREVIEW='true'; npm --prefix website run build Passed after normal docgen preparation and locked website dependency restoration.
python -m black --check --extend-exclude 'docs/' . Black 22.3.0 passed, 238 files unchanged.
python -m pytest scripts/release scripts/test_bump_version.py tools/ci/tests/test_pipeline_yaml.py -q Current-head GitHub CI: 1,702 passed, including the real SBT and historical Git cases.
Review history and earlier validation

Review reports and resolutions
retain the original findings. Six fresh, disjoint audits covered every changed
file before the simplification pass. Reviewer model selection was left to the
runtime, not claimed as a complementary or three-model review.

The simplification removes 1,189 net implementation, test and documentation
lines
, including five duplicate files. Release instructions shrink from 1,218
to 605 lines. The 43 historical reports remain intact; six new reports document
the audit. These records explain why the complete PR diff remains larger than
the executable changes.

Earlier validation
belongs to head 800589ac8d and is historical evidence only.

Does this PR change any dependencies?

  • No. You can skip this section.
  • Yes. Make sure the dependencies are resolved correctly, and list changes here.

Add actions/create-github-app-token pinned to
bcd2ba49218906704ab6c1aa796996da409d3eb1 and verify the SBT launcher with a
committed checksum. Release CI installs the existing Hypothesis dependency;
API-doc publication restores Graphviz/Doxygen prerequisites. Runtime dependency
pins are unchanged.

Does this PR add a new feature? If so, have you added samples on website?

  • No. You can skip this section.
  • Yes. Make sure you have added samples following below steps.

Release infrastructure only. Operator commands and installation examples are
documented; no estimator or transformer API is added.

  1. Find the corresponding markdown file for your new feature in website/docs/documentation folder.
    Make sure you choose the correct class estimators/transformers and namespace.
  2. Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
  3. Make sure the DocTable points to correct API link.
  4. Navigate to website folder, and run npm start to make sure the website renders correctly.
  5. Don't forget to add <!--pytest-codeblocks:cont--> before each python code blocks to enable auto-tests for python samples.
  6. Make sure the WebsiteSamplesTests job pass in the pipeline.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

Copy link
Copy Markdown

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 22, 2026
## Summary
Refresh the release automation on current master, align it with the Fabric release guide and the live Publish-Official pipeline, and add proof-oriented coverage for release identifiers, artifact verification, GitHub workflows, and BBC-VHD edits.

## Prompting Intent
Refresh microsoft/SynapseML PR microsoft#2628 using the SynapseML PR readiness loop, follow the internal Fabric release guide, and derive automation from actual prior OSS, Internal, Publish-Official, and BBC-VHD releases rather than relying on stale examples.

## Linked Sources
- Fabric release guide: https://msdata.visualstudio.com/A365/_wiki/wikis/Osmos%20Team%20Wiki/130638/SynapseML-Fabric-Release-Guide-v2
- Release automation PR: microsoft#2628
- Derivative tag automation: microsoft#2540
- Live v1.1.3 release: https://github.com/microsoft/SynapseML/releases/tag/v1.1.3
- Publish-Official pipeline: https://msdata.visualstudio.com/A365/_build?definitionId=35879
- Historical BBC-VHD release PR: https://msdata.visualstudio.com/A365/_git/BBC-VHD/pullrequest/1805064

## Rationale
Keep ESRP, review, White-Glove, and train decisions human-gated while automating deterministic mechanics. Tag the exact reviewed merge, dispatch downstream GitHub automation explicitly because GITHUB_TOKEN pushes do not recurse, gate release notes on public artifacts, use the live pipeline parameter contract, fail loudly on incomplete network evidence, and roll back paired BBC-VHD writes rather than leaving partial release state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh Rana Singh (ranadeepsingh) added enhancement github_actions Pull requests that update GitHub Actions code labels Aug 22, 2026
@ranadeepsingh Rana Singh (ranadeepsingh) changed the title feat: Automate release preparation, notes, and artifact verification feat: Automate reviewed SynapseML-to-Fabric releases Aug 22, 2026
@ranadeepsingh
Rana Singh (ranadeepsingh) marked this pull request as ready for review August 22, 2026 03:24
Copilot AI lite review requested due to automatic review settings August 22, 2026 03:24
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds release-engineering automation to make SynapseML-to-Fabric releases more deterministic and reviewable, including a GitHub Actions “prepare → tag merged commit → publish notes” flow plus supporting Python tooling to derive/verify release identifiers and safely bump BBC-VHD pins.

Changes:

  • Introduces a release matrix generator (release_matrix.py) and an end-to-end verifier (verify_release.py) with accompanying tests.
  • Adds bump_bbcvhd.py (plus tests) to safely update BBC-VHD component pins while preserving line endings and rolling back on failure.
  • Adds/updates GitHub workflows for release prepare/tagging and manual, artifact-gated release notes; enables workflow_dispatch for PR validation so release PRs created via GITHUB_TOKEN can be validated.
Show a summary per file
File Description
scripts/test_bump_version.py Extends bump-version tests for denylisted repo-relative paths; normalizes path keys to POSIX.
scripts/bump-version.py Improves Windows console encoding robustness; adds path-based denylist support; normalizes path handling via as_posix().
scripts/release/release_matrix.py Adds a single-source-of-truth release matrix for tags and artifact versions + CLI rendering.
scripts/release/verify_release.py Adds live verification of GitHub/ADO tags and Maven/PyPI/UPack/Azure Artifacts presence.
scripts/release/bump_bbcvhd.py Adds deterministic, rollback-safe BBC-VHD component pin updater with CRLF/LF preservation.
scripts/release/README.md Documents how to use the new release tooling and where it fits in the guide.
scripts/release/test_release_matrix.py Adds regression/contract tests for the release matrix derivations and CLI validation.
scripts/release/test_verify_release.py Adds unit tests for verifier networking/error handling and run plan behavior.
scripts/release/test_bump_bbcvhd.py Adds tests for BBC-VHD bump idempotency, rollback, and newline preservation.
scripts/release/test_release_workflows.py Adds tests asserting key workflow contract properties (manual gating, dispatch behavior).
scripts/release/test_prev_tag.sh Adds a repo-taglist regression script for “previous primary tag” selection logic.
.github/workflows/release-prepare.yml Adds a reviewed release PR generator and a merged-commit tagger that dispatches downstream orchestration.
.github/workflows/release-notes.yml Adds a manual, artifact-gated GitHub Release publisher with explicit previous-tag selection.
.github/workflows/pr-validation.yml Enables workflow_dispatch so validations can be dispatched for bot-opened release PR branches.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (1)

scripts/release/verify_release.py:260

  • --skip internal does not currently skip the internal PyPI feed checks (the synapseml_internal package); it only skips internal git tag checks. This makes --skip internal behave inconsistently compared with --skip public.
    def pip(self, package: str, version: str) -> str:
        if "pip" in self.skip or "ado" in self.skip:
            return SKIPPED
        # Azure Artifacts normalises pypi names: synapseml_internal -> synapseml-internal
        return (
            OK
            if version
            in self._feed_versions("Synapse-Conda", "pypi", package.replace("_", "-"))
            else MISSING
        )
  • Files reviewed: 14/14 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread scripts/release/verify_release.py Outdated
Comment thread scripts/release/verify_release.py
Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 22, 2026
## Summary
Make --skip internal consistently skip SynapseML-Internal tags, UPacks, and wheels while retaining OSS artifact checks, and document every skip scope with regression coverage.

## Prompting Intent
Resolve all current-head automated review findings on microsoft/SynapseML PR microsoft#2628 and preserve an explicit, safe release-verification CLI contract.

## Linked Sources
- Release automation PR: microsoft#2628
- Internal artifact skip review: microsoft#2628 (comment)
- Skip help review: microsoft#2628 (comment)

## Rationale
A release operator who opts out of Internal verification must not still query or fail on Internal packages. Passing artifact scope explicitly keeps OSS checks active, avoids hiding public release gaps, and makes combined skip behavior predictable from both CLI help and maintainer documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 22, 2026 03:34
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/release/verify_release.py:75

  • On Windows, subprocess.run(..., shell=True) expects a command string, but this code passes a list. That combination can fail (or behave differently) on win32, which would break the fallback ADO auth path even when az is installed. Build a platform-appropriate command (string for shell=True, list otherwise).
    out = subprocess.run(
        [
            "az",
            "account",
            "get-access-token",
            "--resource",
            ADO_RESOURCE,
            "--query",
            "accessToken",
            "-o",
            "tsv",
        ],
        capture_output=True,
        text=True,
        shell=(sys.platform == "win32"),
    )
  • Files reviewed: 14/14 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 22, 2026
## Summary
Build the Azure CLI authentication command as a string only when Windows requires shell execution, while retaining an argument list and shell-free execution on other platforms. Add regression coverage for both command shapes.

## Prompting Intent
Resolve the current-head suppressed Copilot review finding on microsoft/SynapseML PR microsoft#2628 and prove the fallback authentication path used by release verification on Windows.

## Linked Sources
- Release automation PR: microsoft#2628
- Fabric release guide: https://msdata.visualstudio.com/A365/_wiki/wikis/Osmos%20Team%20Wiki/130638/SynapseML-Fabric-Release-Guide-v2

## Rationale
Windows resolves the Azure CLI through a command shim, so the verifier needs shell execution there; subprocess requires that shell command to be a string. Keeping list-based execution elsewhere preserves safe argument boundaries, while an actual token-less Windows replay proves the fallback can authenticate and inspect the live release.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 22, 2026 03:40
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Addressed the current-head suppressed Windows auth finding in 6e80f27.

  • Windows now gives subprocess.run(..., shell=True) a command string; non-Windows retains an argument list with shell=False.
  • Parameterized regression coverage locks both command shapes.
  • A token-less Windows fallback replay authenticated through �z and found all 12 selected v1.1.3 master artifacts; the complete release-tooling suite is 279 passed.

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/release/verify_release.py:313

  • The Maven publication gate here verifies only synapseml-core_{scala} (via public_maven()), but the workflow-generated release notes advertise com.microsoft.azure:synapseml_2.12:${VERSION} (see .github/workflows/release-notes.yml:137) and the docs use the synapseml_2.12 coordinate (website/doctest.py:25). This can let verify_release.py report COMPLETE even if the primary install coordinate is missing (or vice versa). Consider aligning the verification to check the same Maven artifact(s) that users are instructed to consume (e.g., verify both synapseml_{scala} and synapseml-core_{scala}, or switch to the canonical one).
            "maven",
            tp.key,
            f"synapseml-core_{tp.scala}",
            tp.oss_maven_version,
            c.public_maven(tp.scala, tp.oss_maven_version),
  • Files reviewed: 14/14 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 22, 2026
## Summary
Gate release completeness on both the user-facing synapseml_<scala> Maven coordinate and the release-guide synapseml-core_<scala> coordinate for every selected Spark target. Add a failing-install-coordinate regression and document the two-coordinate contract.

## Prompting Intent
Resolve the current-head suppressed Copilot review finding on microsoft/SynapseML PR microsoft#2628 by aligning artifact verification with generated release notes, installation docs, and actual published releases.

## Linked Sources
- Release automation PR: microsoft#2628
- Fabric release guide: https://msdata.visualstudio.com/A365/_wiki/wikis/Osmos%20Team%20Wiki/130638/SynapseML-Fabric-Release-Guide-v2
- Live v1.1.3 release: https://github.com/microsoft/SynapseML/releases/tag/v1.1.3

## Rationale
The aggregate synapseml artifact is the coordinate users install and the one release notes advertise, while the guide explicitly points maintainers to synapseml-core. Requiring both prevents a GitHub Release from reporting complete when either the public install contract or the guide's Maven evidence is absent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The release queue omits the required approved source-tag variable, blocking immutable tagged publication.

Review effort: Lite
Findings: None

Resolved since last review (4)

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Audited the current-head review overview's concern about a missing queue-time
SYNAPSEML_RELEASE_TAG. No code change is needed for that concern.

release_ops.py queues the approved plan, commit and tagged branch. Each of
BuildAndCacheSbt, Publish and Release runs release_guard.py maven
before its SBT cache/bootstrap step. The guard validates the plan approval,
selected source tag, checked-out commit, local tag and clean working tree, then
emits the Azure task variable SYNAPSEML_RELEASE_TAG, along with the approved
commit and plan ID. Later tasks inherit those job-scoped variables.

Relevant current-head code:

I executed the real guard against an isolated temporary Git repository with a
synthetic source commit and local 0.0.0 tags. For master, Spark 4.0 and Spark
4.1, the queued request had no tag variable and the guard still emitted the
exact approved tag, commit and plan ID. Parsed workflow checks confirmed that
the guard precedes SBT in all three producer jobs. No remote tags or packages
were created. Existing release-version tests also require the explicit tag and
reject inconsistent source bindings.

The four actual review threads are fixed and resolved in 9c63630f27.
Azure build 239370788 is validating that head's merge result. The release App
settings are still absent, and the approved pre-merge release trial and human
approval remain separate gates.

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Current-head CI is complete for 9c63630f27bafc73a34fb463639a7a44bd24d343.

  • Azure build 239370788
    succeeded on PR merge 16bbca427383064f2856b9cb575590d9ebb6f010, with
    reason=pullRequest. All 65 jobs succeeded.
  • Published test results show 3,695 passed and 22 skipped. The skipped suite
    names and counts exactly match preceding green build 239243494. The new head
    adds 16 passing tests, and the OpenCV Python log confirms that all 16 new
    image-conversion cases ran, with 20 total passes.
  • The separate CI-helper log reports 306 passes and 63 passing subtests.
    Conditional task skips cover cache reuse, pre-job hooks, master-only
    publication and a component scan that had already run.
  • The final readiness snapshot has no failed, pending or missing required
    checks, no unresolved threads and no suppressed current-head comments.
    The latest automated review covers this head. Its source-tag overview concern
    is addressed with executed evidence in the
    previous comment.
  • A fresh fetch confirms zero commits behind master 861c3a1e14a9, matching
    local/remote heads and a clean worktree. Website deployment is intentionally
    skipped for PRs. Aggregate CodeQL remains neutral due to its missing
    base-branch API-upload configuration.

This is not yet approval to merge or publish. Both repository release-App
settings are absent. The remaining gates are App installation/configuration
and a live App-created PR check, publisher/Databricks/storage access, final
source-bound candidate qualification, exact-plan and signing approval, the
requested approved pre-merge release trial, and maintainer review approval.
No immutable release tags, production packages, signing approvals or merges
were created by this readiness pass.

## Summary
Extend the existing SynapseML release skill with an agent checkpoint runbook
and a cross-platform, credential-free rehearsal command. Preserve explicit
human authority for live preparation, immutable tags, publication, signing,
release notes and merges.

## Prompting Intent
The engineer chose the agent-runbook route for publishing, together with the
existing automation/scripts and dry-run tests. Implement that route without
interpreting the request as approval to publish a production release.

## Linked Sources
- Release automation PR: microsoft#2628
- Existing operator guide: scripts/release/README.md
- Existing release skill: .github/skills/synapseml-release/SKILL.md
- Existing guards and ledger: scripts/release/bootstrap_release.py and release_ops.py
- Prior exact-head CI/readiness: microsoft#2628 (comment)
- User request: "sure let's do the agent run book route for publishign then along with the automations and scripts and dry run tests"

## Rationale
Reuse the guarded publication engine and its test doubles instead of adding
another approval or publishing state machine. Offline tests exercise real
plan/ledger/bootstrap/receipt code with simulated services and disposable Git
repositories, and explicitly deny live network and non-Git subprocess calls.
The runner has no production inputs, preserves existing report/ledger files,
and fails on missing tools, timeout, zero tests, failures or skips.
Distinguish pretag bootstrap preview from posttag publication preflight and
retain one authoritative ledger/operator across agent sessions.

Both native Windows and Linux rehearsals passed all 12 scenarios. The full
Linux release/pipeline suite passed 1227 tests, with the existing optional SBT
skip and Windows-worktree notebook deselection. Native Windows covered the
actual notebook case and runner edge cases with 74 passes. Pinned Black and
link checks passed. A harness-selected Low incremental review reported no
significant issues; this is not a new full-PR multi-model review.

This is release infrastructure, labeled area/build, with no linked ADO task.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Release automation is high-impact and has a moderate unresolved Spark 3.5 release-notes tag-link defect requiring correction and human validation.

Review effort: Lite
Findings: None

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

The current-head review reports no inline findings, but its overview mentions
an unresolved Spark 3.5 release-notes tag-link defect. I traced and executed
that path on 7bc3019409; I could not reproduce a tag mismatch.

notes_installation
renders the Spark 3.5 row as:

| 3.5 | 3.11 | `v1.2.0-spark3.5` |

That is a tag reference, not a link to a separate GitHub Release object.
The primary plan includes v1.2.0, v1.2.0-spark3.5 and
v1.2.0-python3.11; only the primary tag receives a GitHub Release.
An executed rendering check for both the default two-runtime plan and the
optional three-runtime plan confirmed that every notes-table tag belongs to
its target's approved oss_tags.

The offline rehearsal also executes real bootstrap tag creation and idempotent
recovery against a disposable local Git repository, then checks every planned
tag against its approved commit. It passed on Windows and Linux. None of
these checks creates canonical release tags or substitutes for live release
qualification.

No code change is justified by the overview as written. Current-head Azure
build 239396928
is still running. App setup, final candidate qualification and human release
approvals remain separate requirements.

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Agent runbook and offline rehearsal are implemented in 7bc3019409.

  • Native Windows and Linux each passed all 12 offline rehearsal scenarios.
  • The full release/pipeline test suite passed 1,227 cases, with the existing
    opt-in SBT skip and WSL worktree-pointer deselection. Native Windows covered
    the actual committed-notebook case and runner edge cases with 74 passes.
  • Azure build 239396928
    completed on PR merge a721079061860a602bdd2d1102aefbed1c6b46f7.
    GitHub reports success. Azure's raw result is partiallySucceeded:
    64 jobs succeeded, and Publish succeeded with an advisory cache timeout.
    Its Use cached Anaconda environment task timed out during cache download;
    fallback Create Anaconda environment and Publish Artifacts both succeeded.
    No job failed, was canceled or was skipped.
  • Published results contain 3,695 passing tests and 22 skips, with skipped
    suite names/counts identical to preceding build 239370788.
  • Current-head automated review has no inline or suppressed findings and all
    threads are resolved. Its overview tag concern has an
    executed evidence response.
    Final readiness reports no failed, pending or missing required checks.

Use the agent runbook
with the existing release scripts. Offline rehearsal is not live-service
qualification or publication approval. App configuration, final candidates,
the approved pre-merge release trial, signing and maintainer approval remain
external gates. No production release tags, packages or merges were created.

## Summary
Close full-PR review findings in the agent-operated release process.
Bind public artifact bytes to separate CDN and Central producer receipts,
preserve ambiguous submissions and existing release branches, and reject
partial-coordinate republication and inconsistent runtime or timeline evidence.
Restore source-bound primary API docs before package upload, disclose the
public wheel in approvals, and cover version preparation in CI.

## Prompting Intent
The engineer requested a fleet review of all microsoft#2628 changes,
an up-to-date branch and PR description, and reliable automation scripts and
an agent runbook for publishing new versions. This work does not authorize
production publication, signing, or merges.

## Linked Sources
- Automation PR: microsoft#2628
- Notebook archive issue: microsoft#2346
- Current validation baseline: microsoft#2752
- Operator guide: scripts/release/README.md
- Agent runbook: .github/skills/synapseml-release/references/agent-runbook.md
- Six fleet findings and resolutions: reviews/pr-2628/task-pr2628-fleet-attempt-1-stage-attempt-1-review-{1,2,3,4,5,6}-harness.md

## Rationale
Retain existing plan approval identities and one publisher instead of adding
a separate agent publishing engine. Store destination-specific producer hashes
because independently built Maven outputs need not be byte-identical.
Verify downloaded bytes rather than treating availability as provenance.
Preserve failed and ambiguous state rather than deleting history or retrying
immutable Maven coordinates. Keep same-wheel qualification and production
authority explicit human gates. Preserve all review findings and append their
resolution evidence, including the withdrawn unproven qualification claim.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One or more issues must be addressed before approval.

2 open findings

🧠 Review effort: Lite

Comment thread scripts/release/release_dry_run.py Outdated
Comment thread website/src/pages/index.js
## Summary
Run the rehearsal child with an explicit environment allowlist rather than
inheriting release credentials, production inputs or Python overrides.
Add a regression using synthetic credentials and document the boundary.

## Prompting Intent
The engineer requested reliable agent-operated publishing automation, offline
dry runs and resolution of all review findings on microsoft#2628.
The current-head hosted review identified inherited credentials in the
otherwise credential-free rehearsal.

## Linked Sources
- PR: microsoft#2628
- Environment finding: microsoft#2628 (comment)
- Rebutted JSX finding: microsoft#2628 (comment)
- Passing website build: https://github.com/microsoft/SynapseML/actions/runs/37855557257/job/113578741269
- Resolution evidence: reviews/pr-2628/task-pr2628-fleet-attempt-1-stage-attempt-1-review-2-harness.md

## Rationale
An allowlist excludes future credential names without relying on an incomplete
secret-name denylist. Preserve the process and filesystem environment needed
by Python and disposable Git fixtures on Windows and Linux. Keep the actual
service isolation and no-publication-authority contract unchanged.
Do not change valid literal JSX text to satisfy an incorrect identifier finding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Unresolved moderate release-gating findings remain in a high-risk publishing workflow.

0 open findings

2 resolved since last review

🧠 Review effort: Lite

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Fleet review and corrective validation are complete at
800589ac8dfe4bd23596cb174c9ec997e574b6ae, based on current master
861c3a1e14a9511b5604563ff1e3976cefa82e90. The PR title and template-based
description now describe the current implementation rather than accumulated
historical checkpoints.

The six committed review reports retain the findings and resolutions. Corrections
cover runtime and branch safety, partial-publication recovery, stale adoption,
state initialization, authoritative read failures, warning time bounds,
destination-specific artifact receipts and public-byte verification, API-doc
publication, approval visibility, CI coverage and runtime-specific install pins.
The follow-up hosted review led to an explicit environment allowlist for the
offline rehearsal. Its homepage identifier comment was rebutted with literal JSX
source and a successful actual website build.

Final evidence:

  • Local combined regression: 1,778 passed, one opt-in SBT skip and 11
    Git-dependent cases covered by a separate 11-pass native Windows run.
  • Hosted release tooling: 1,669 passed, including that SBT check and the
    historical Git cases. Compile, style, website and notebook checks passed.
  • Offline rehearsal: 12 scenarios passed on each of Windows and Linux,
    with zero skipped scenarios and no live-service or publication authority.
  • Azure 239694763
    validated exact merge e31c343fe33d43a3650546c0cce3a055b8dacf16.
    3,695 passed, 22 baseline-identical skips, all 377 test runs complete.
    Its raw result is partiallySucceeded: three dependency-cache restores had
    TLS errors in UnitTests language, whose 98 actual tests and result publication
    all passed. The other 64 jobs succeeded. This is a documented non-gating cache
    warning, not a suppressed test failure.
  • Final readiness reports no unresolved or suppressed findings and no
    failed, pending or missing required checks. The latest automated review covers
    this exact head. Its generic overview mentions unspecified moderate findings
    while listing zero; every concrete finding has a recorded disposition.

Engineering work is complete; production publication is not authorized by
these results.
The two GitHub App settings were still absent when checked.
App installation/configuration, publisher and Databricks/storage access, final
source-bound same-wheel qualification, exact-plan approval, signing, the
explicitly approved pre-merge trial and maintainer merges remain separate gates.
No production tags, packages, signing approvals or merges were created here.

Type: infrastructure
PR label: area/build

## Summary
Remove duplicate release implementations, test setup and operator instructions.
Keep one runbook and a short discovery skill, deleting four overlapping
references and the shadow predecessor-tag script. Preserve the original
review evidence and record the six fresh audits with their resolutions.

Fix the audit's reproduced defects: predecessor selection under pipefail,
overbroad archive version replacement, malformed PyPI metadata, unbounded
JSON responses, missing token timeout, independent PyPI visibility in tests,
and native Windows bootstrap fixture assumptions.

## Prompting Intent
The engineer asked to rebase microsoft#2628 onto master before a
fresh fleet reviewed and revised every changed file to reduce PR bloat.
Keep the publishing scripts and agent-runbook route reliable, preserve
approval and immutable-publication safeguards, update the PR, and push the
validated changes without publishing a production release.

## Linked Sources
- Release automation: microsoft#2628
- Notebook archives: microsoft#2346
- Release operator guide: scripts/release/README.md
- Fresh audit and resolution reports: reviews/pr-2628/task-pr2628-trim-attempt-1-stage-attempt-1-review-{1..6}-harness.md
- Repository contribution and review rules: AGENTS.md and .github/skills/synapseml-pr-loop/

## Rationale
Reuse the existing inventory description and strict JSON parser instead of
maintaining parallel implementations. Consolidate repeated fixtures and
assertions while retaining actual workflow execution, destination-specific
download proof, realistic evidence budgets and saved-plan identities.

Do not retire supported private operations, discard historical reviews or
weaken immutable-package recovery simply to reduce the line count. The
combined source, test and documentation changes remove 1,248 net lines;
required new review reports are accounted for separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The critical Maven release wiring issue and moderate malformed-profile path handling issue remain unresolved.

0 open findings

🧠 Review effort: Lite

Type: infrastructure
PR label: area/build

## Summary
Normalize the older pathlib RuntimeError for cyclic profile symlinks at the
existing profile-loading boundary. Keep the matrix CLI's controlled exit 2
and sanitized error, and preserve public-only operation without a profile.

Add portable exception-normalization and real cyclic-link CLI regressions.
Append the reproduced finding and resolution to the existing contract review.

## Prompting Intent
The engineer requested a complete review and reduction of the release
automation PR, with reliable publishing scripts and an agent runbook.
Investigate the current-head automated review's unresolved overview claims
rather than treating zero inline comments as proof that no defect remains.

## Linked Sources
- Release automation and current review: microsoft#2628
- Contract audit and resolution: reviews/pr-2628/task-pr2628-trim-attempt-1-stage-attempt-1-review-2-harness.md
- Shared profile input boundary: scripts/release/release_config.py

## Rationale
Missing files, directories, invalid JSON and invalid encoding already return
controlled errors. On older Python, a cyclic symlink instead raises
RuntimeError before the matrix CLI's ValueError handler. Fix the shared
input boundary, not a broad outer CLI catch. Other runtime failures remain
visible, and no publication behavior or approved-plan identity changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Checked both concerns named in the 5d0277fae1 review overview rather than treating its zero inline findings as an all-clear.

The three previously reported Maven wiring cases remain fixed. Release jobs do not invoke publishR, module publishPython, or notebook uploads; primary API-doc publication is intentional. Disabled optional test jobs are omitted from the dependency list, required gates remain enforced, and Release exists only when both publication flags define its Publish dependency. Current-YAML execution and assertions passed 34 cases, plus six required-gate probes. See pipeline.yaml at lines 200-219, 298-324 and 631-647.

The profile concern exposed one real edge case on older Python: resolving a cyclic symlink raised RuntimeError and escaped the matrix CLI's controlled error path. Fixed in 25bad376c0a1b1c99ba0d3f4d1fb8f93cd0e9cdc at the existing shared profile-loading boundary. The real CLI now exits 2 with sanitized stderr, no traceback and no output artifact. Portable regressions cover both pathlib exception forms; public-only generation still ignores private profiles. All 135 configuration, matrix, defaults and saved-plan tests passed.

The updated head is being revalidated. This work has not published a production release.

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One or more issues must be addressed before approval.

1 open finding

🧠 Review effort: Lite

Comment thread project/ReleaseVersion.scala
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

The bloat review is complete at 25bad376c0a1b1c99ba0d3f4d1fb8f93cd0e9cdc, based on current master 861c3a1e14a9511b5604563ff1e3976cefa82e90. The requested rebase ran before the six fresh audits and was a no-op. The final local, fork and PR heads match.

The revision removes 1,189 net implementation, test and documentation lines and five duplicate files. Release guidance shrinks from 1,218 to 605 lines in one runbook plus its discovery skill. The 43 historical reports remain intact; six new reports add 659 lines, so the complete PR shrinks by 530 net lines rather than 1,189. Supported private operations, saved-plan identities and immutable-publication safeguards remain in scope.

Final validation

Evidence Result
Linux release, version-bump and CI-helper suite 1,811 passed. The one local opt-in SBT skip ran in GitHub CI; the 11 WSL-incompatible Git cases passed natively.
Windows runner, rehearsal, docs and profile suite 110 passed.
Actual offline runner 12 scenarios passed on each OS, with zero skips.
GitHub PR validation 1,702 release tests passed with SBT enabled; compile and Scala/Python style passed.
Website 39 tests passed normally and in preview; normal docgen preparation and the full preview build passed.
Azure 239752260 succeeded, all 65 jobs passed. All 377 test runs completed: 3,695 passed, 22 not-applicable outcomes matching baseline 239694763 by suite and count. No failed, incomplete or unanalyzed tests.

Azure validated merge b285060052872f6a4321e694cedda2b0f2788675, with this exact PR head and current master as its parents. The final readiness snapshot has zero missing, failed or pending required checks, zero unresolved threads and zero suppressed current-head findings. The frozen 44-file revision manifest still matches the committed tree, and the checkout is clean.

The current review's SBT finding was a source-binding test misread and is resolved with fixture and execution evidence. The earlier vague profile claim exposed a real cyclic-symlink error path, which was fixed and revalidated.

The PR title and template-based description now reflect the reduced scope and current evidence. Maintainer approval, App/publisher setup, candidate qualification and the explicitly approved pre-merge live trial remain separate gates. No production release, signing approval or merge was performed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build enhancement github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants