You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds the preview useSeparateTestAction setting, defaulting to false, which moves normal test execution from RunPipeline into a new RunTests action using al runtests.
Flow
RunPipeline builds and publishes the apps, creates the container, and keeps it alive.
It exports the container name, generated credentials, and a runtime signal.
The workflow conditionally invokes RunTests.
RunTests selects the test apps, executes them with AlTool, and produces JUnit results for the existing analysis step.
The existing cleanup step removes the container.
Notes
Only normal tests are moved. BCPT, page scripting, and multi-country tests remain in RunPipeline.
runTestsInAllInstalledTestApps uses the existing installTestAppsJson handoff rather than discovering every installed server app.
The optional testType setting supports UnitTest, IntegrationTest, and Uncategorized.
A custom RunTestsInBcContainer override can still replace the built-in AlTool runner.
BCH is still used for container creation, app installation and metadata, server and company discovery, test enumeration, and event-log collection. AlTool owns the actual test execution.
✅ Checklist
Add tests (E2E, unit tests)
Update RELEASENOTES.md
Update documentation (e.g. for new settings or scenarios)
Introduces a new opt-in RunTests action that moves normal test execution
(testFolders) out of the RunPipeline action. When useSeparateTestAction is
true, RunPipeline compiles, publishes and installs apps and keeps the build
container alive, but skips normal tests; the RunTests action then runs those
tests against the same container and produces TestResults.xml. BCPT and page
scripting tests are unchanged. Behavior is unchanged when the setting is false.
Includes the new setting (ReadSettings, settings.schema.json, settings.md),
the RunPipeline keep-alive guard, template _BuildALGoProject wiring, release
notes and Pester tests.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The separate RunTests action now runs normal tests through Microsoft's
headless al runtests (AlTool) CLI by default instead of BcContainerHelper.
The runner resolves the kept-alive container's connection settings host-side
from the container name, installs the AL developer tools as a dotnet global
tool, runs the test codeunits batched (with an isolated fallback/rerun pass)
and emits the same TestResults.xml (JUnit) schema, so downstream test result
analysis is unchanged. No RunPipeline change is required.
BcContainerHelper remains available via a RunTestsInBcContainer override
script, which fully replaces the built-in AlTool runner (this is how, for
example, BCApps supplies its own runner and how Legacy test-type tests are
handled). The AlTool runner does not run Legacy test types or tests that
require UI/client-callback interaction; this is documented as a known
limitation.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The official released al runtests tool does not support --testplan, so the
batch-first path in AlToolTestRunner hard-set �llPassed=false whenever the
batch failed to connect and never reset it, making every run falsely report
FAILED. Remove the batching path entirely and run each test codeunit in its
own �l runtests <codeunitId> --testmethods --raw invocation, with the
existing rerun-on-failure pass. �llPassed now derives solely from real
per-method outcomes via Add-JUnitTestSuite.
- Delete Invoke-AlBatchRunTests, ConvertFrom-AlBatchOutput,
ConvertTo-AlTestPlanJson and their Export-ModuleMember entries.
- Update README and RELEASENOTES to per-codeunit wording.
- Drop the two batch-specific unit test contexts.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The eligibility check still enables RunTests when doNotBuildTests is true and doNotRunTests is false. BcContainerHelper disables test execution internally for doNotBuildTests, but that does not update $settings or this workflow signal. The separate runner can therefore select downloaded test apps or existing compiled test artifacts and attempt tests that were explicitly disabled, potentially without the test toolkit installed. Exclude doNotBuildTests from eligibility and add a regression case for this combination.
Preserve test-data import when all test categories are disabled
Actions/RunPipeline/RunPipeline.ps1:534
When doNotRunBcptTests and doNotRunPageScriptingTests are both true, setting doNotRunTests here also skips BcContainerHelper's ImportTestDataInBcContainer stage: that stage requires at least one enabled test category. Configured configPackages and custom test-data imports are then omitted, and RunTests invokes the runner without performing that preparation. Preserve the import stage before separate test execution, or retain the existing pipeline path for builds that need it.
Honor BeforeEachTestApp database restoration in separate runner
Actions/RunTests/RunTests.psm1:314
This loop does not honor restoreDatabases: ["BeforeEachTestApp"]. RunPipeline still passes that setting to BcContainerHelper, but normal tests are now skipped there, so its per-app restore never runs. The separate runner executes each app against state left by earlier apps or by BCPT/page scripting, violating the configured isolation. Preserve the baseline backup and restore it before each normal test app, or leave builds using this option on the existing RunPipeline path.
The reason will be displayed to describe this comment to others. Learn more.
How about moving the al-related calling out of AL-Go helper into its own module?
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Maria Zhelezova (mazhelez)
left a comment
Summary
Adds the preview
useSeparateTestActionsetting, defaulting tofalse, which moves normal test execution fromRunPipelineinto a newRunTestsaction usingal runtests.Flow
RunPipelinebuilds and publishes the apps, creates the container, and keeps it alive.RunTests.RunTestsselects the test apps, executes them with AlTool, and produces JUnit results for the existing analysis step.Notes
RunPipeline.runTestsInAllInstalledTestAppsuses the existinginstallTestAppsJsonhandoff rather than discovering every installed server app.testTypesetting supportsUnitTest,IntegrationTest, andUncategorized.RunTestsInBcContaineroverride can still replace the built-in AlTool runner.✅ Checklist