Repository navigation
ci: remove Spark 4.1 compatibility replay - #2752
Merged
Rana Singh (ranadeepsingh) merged 1 commit intoOct 6, 2026
Merged
Rana Singh (ranadeepsingh) merged 1 commit into
Rana Singh (ranadeepsingh) merged 1 commit into
Conversation
Label: area/build ## Summary Remove the advisory ReleaseBranchCompat job, its unused prerequisite list, and replay-only tests. Update CI guidance and protect the retained Spark 4.1 push and PR triggers with regression tests. ## Prompting Intent The maintainer requested a separate PR retiring the noisy Spark 4.1 compatibility replay job. Keep normal Spark 4.1 validation, Internal compatibility, and the release automation work in PR microsoft#2628 unchanged. ## Linked Sources - Maintainer request to remove the compatibility job in a new PR. - Failure context: microsoft#2628 - Pipeline baseline: https://github.com/microsoft/SynapseML/blob/6ce51f704f8e8e403a111195ed886cb82d62502c/pipeline.yaml ## Rationale Delete the retired job rather than leave a disabled matrix and dead prerequisite configuration. Validate port changes on their actual branch instead of automatically applying every master PR patch to Spark 4.1. The parsed pipeline is unchanged apart from this job's removal, including all other jobs and their conditions. All retained test functions are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 879c5a1c-4efa-4620-93ea-f8f0052112f1
Rana Singh (ranadeepsingh)
requested a review
from Brendan Walsh (BrendanWalsh)
as a code owner
October 6, 2026 07:17
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Hey Rana Singh (@ranadeepsingh) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
Copilot started reviewing on behalf of
Rana Singh (ranadeepsingh)
October 6, 2026 07:17
View session
Collaborator
Author
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The patch is consistent, but it changes shared Azure pipeline behavior and live Azure validation remains pending.
Review effort: Balanced
Findings: None
What changed in this PR
Removes the advisory Spark 4.1 patch-replay job while preserving direct port-branch validation.
Changes:
- Deletes
ReleaseBranchCompatand its prerequisite metadata. - Removes obsolete replay tests and adds regression coverage.
- Updates CI and branch-validation guidance.
| File | Description |
|---|---|
pipeline.yaml |
Removes the compatibility replay job. |
.pipelines/release-compat-prerequisites.txt |
Deletes obsolete prerequisite metadata. |
tools/ci/tests/test_pipeline_yaml.py |
Replaces replay tests with removal and trigger checks. |
tools/ci/README.md |
Documents direct port validation. |
.github/skills/synapseml-pr-loop/references/ci-triage.md |
Updates CI triage guidance. |
.github/skills/synapseml-branches/SKILL.md |
Revises CI source-of-truth terminology. |
.github/skills/synapseml-branches/references/branch-spark4-common.md |
Clarifies branch-specific toolchain selection. |
.github/skills/synapseml-branches/references/branch-spark3p5.md |
Documents the removal and remaining validation responsibilities. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
3 of 5 tasks
Rana Singh (ranadeepsingh)
added a commit
to leninworld/SynapseML
that referenced
this pull request
Oct 7, 2026
## Summary Add synthetic public-estimator coverage for dense and sparse streaming with a wider ambient OpenMP team. Run each case in a fresh child JVM and compare evaluated predictions with bulk ingestion. Confine temporary files and native crash reports to the test directory, remove inherited credentials, enforce a timeout, and release cached data and native model handles. Clarify that Linux process affinity after OpenMP binding is only a hint about native team width. Integrate current master without rewriting contributor commits. Retain the additive pre-commit review evidence. ## Prompting Intent The maintainer requested a committed crash regression for the external contribution, with no confidential data or secrets, review-thread triage, before/after memory and timing measurements, and a six-round final review after the less expensive gates. Keep the contributor's production fix intact and do not merge the pull request. ## Linked Sources - Contribution: microsoft#2751 - Related reported crash: microsoft#2333 - Review dispositions: microsoft#2751 (comment) - Current master integration: microsoft#2752 ## Rationale A native crash in the shared test JVM could terminate unrelated suites. Fresh child JVMs also prevent earlier fits from narrowing the OpenMP teams and hiding the regression. Synthetic data exercises the bundled native library without using the reporter's private data. The target-production-source replay crashes on the dense path. The sparse replay passed and is retained as a separate correctness control, not claimed as sparse fail-before evidence. The relationship to the reporter's exact incident remains conditional. This is an additive test/documentation follow-up, not a new production fix. Final-head Azure validation and the final six-round review are separate post-publication gates; this commit does not claim those gates passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rana Singh (ranadeepsingh)
added a commit
to ranadeepsingh/SynapseML
that referenced
this pull request
Oct 8, 2026
## Summary Close full-PR review findings in the agent-operated release process. Bind public artifact bytes to separate CDN and Central producer receipts, preserve ambiguous submissions and existing release branches, and reject partial-coordinate republication and inconsistent runtime or timeline evidence. Restore source-bound primary API docs before package upload, disclose the public wheel in approvals, and cover version preparation in CI. ## Prompting Intent The engineer requested a fleet review of all microsoft#2628 changes, an up-to-date branch and PR description, and reliable automation scripts and an agent runbook for publishing new versions. This work does not authorize production publication, signing, or merges. ## Linked Sources - Automation PR: microsoft#2628 - Notebook archive issue: microsoft#2346 - Current validation baseline: microsoft#2752 - Operator guide: scripts/release/README.md - Agent runbook: .github/skills/synapseml-release/references/agent-runbook.md - Six fleet findings and resolutions: reviews/pr-2628/task-pr2628-fleet-attempt-1-stage-attempt-1-review-{1,2,3,4,5,6}-harness.md ## Rationale Retain existing plan approval identities and one publisher instead of adding a separate agent publishing engine. Store destination-specific producer hashes because independently built Maven outputs need not be byte-identical. Verify downloaded bytes rather than treating availability as provenance. Preserve failed and ambiguous state rather than deleting history or retrying immutable Maven coordinates. Keep same-wheel qualification and production authority explicit human gates. Preserve all review findings and append their resolution evidence, including the withdrawn unproven qualification claim. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Remove the advisory Release Branch Compatibility Check spark4.1 job from master PR builds. It applies PR patches onto the port branch before compiling them, which can fail on replay conflicts independently of the PR's own runtime validation.
ReleaseBranchCompat, its unused prerequisite list, and its dedicated tests.Port changes and resolved syncs still need validation on the actual port branch. This PR stops automatic patch replay; it does not establish Spark 4.1 compatibility.
Separate from the release automation changes in #2628. No release code, dependency versions, or branch policies are changed.
Validation
6ce51f7confirmed that the only semantic pipeline change is deletingReleaseBranchCompat.