Enhance customALGoFiles feature

Actions/.Modules/settings.schema.json

@@ -744,6 +744,7 @@
             "type": "object",
             "properties": {
                 "filesToInclude": {
+                    "description": "An array of file specifications to include in the update. Files that match these specifications are copied from the template to the repository. When used in a custom template's settings, inclusions are also propagated from the original template to consumer repos even if the files no longer exist in the custom template.",
                     "type": "array",
                     "items": {
                         "type": "object",
@@ -760,6 +761,10 @@
                                 "type": "string",
                                 "description": "The destination folder where the files should be updated, relative to the repository root. If not specified, defaults to the same as the source file folder."
                             },
+                            "destinationName": {
+                                "type": "string",
+                                "description": "The filename to use at the destination. If specified, overrides the source filename, allowing the file to be renamed when copied. Should be used together with a filter that matches a single file."
+                            },
                             "perProject": {
                                 "type": "boolean",
                                 "description": "Indicates whether the file update should be applied per project. In that case, the destinationFolder is considered relative to each project folder."
@@ -768,6 +773,7 @@
                     }
                 },
                 "filesToExclude": {
+                    "description": "An array of file specifications to exclude from the update. Files that match these specifications are not copied from the template to the repository. When used in a custom template's settings, exclusions are also propagated from the original template to consumer repos even if the files no longer exist in the custom template.",
                     "type": "array",
                     "items": {
                         "type": "object",

Actions/CheckForUpdates/CheckForUpdates.ps1

@@ -1,4 +1,4 @@
-Param(
+Param(
     [Parameter(HelpMessage = "The GitHub actor running the action", Mandatory = $false)]
     [string] $actor,
     [Parameter(HelpMessage = "Base64 encoded GhTokenWorkflow secret", Mandatory = $false)]
@@ -51,8 +51,9 @@ if ($token) {
 # if $update is set to N, CheckForUpdates will only check for updates and output a warning if there are updates available
 # if $downloadLatest is set to true, CheckForUpdates will download the latest version of the template repository, else it will use the templateSha setting in the .github/AL-Go-Settings file
 
-# Get Repo settings as a hashtable (do NOT read any specific project settings, nor any specific workflow, user or branch settings)
-$repoSettings = ReadSettings -buildMode '' -project '' -workflowName '' -userName '' -branchName '' | ConvertTo-HashTable -recurse
+# Get repo settings independent of the build, project, workflow, user, branch, and trigger running this update.
+# Repository-scoped and unconditional settings still apply.
+$repoSettings = ReadSettings -buildMode '' -project '' -workflowName '' -userName '' -branchName '' -trigger '' | ConvertTo-HashTable -recurse
 $templateSha = $repoSettings.templateSha
 
 # If templateUrl has changed, download latest version of the template repository (ignore templateSha)
@@ -113,6 +114,12 @@ if (-not $isDirectALGo) {
 
 # Get the list of projects in the current repository
 $baseFolder = $ENV:GITHUB_WORKSPACE
+
+if ($originalTemplateFolder) {
+    # Use current custom template settings for this run without changing the workspace before comparison.
+    $repoSettings = ReadSettingsWithCurrentCustomTemplateRepoSettings -baseFolder $baseFolder -customTemplateFolder $templateFolder
+}
+
 $projects = @(GetProjectsFromRepository -baseFolder $baseFolder -projectsFromSettings $repoSettings.projects)
 
 $filesToInclude, $filesToExclude = GetFilesToUpdate -settings $repoSettings -projects $projects -baseFolder $baseFolder -templateFolder $templateFolder -originalTemplateFolder $originalTemplateFolder
@@ -133,6 +140,14 @@ if ($projects.Count -gt 1) {
     Write-Host "Calculated dependency depth to be $depth"
 }
 
+# Prepare the list of template folders to be used for verification
+$templateFolders = @($templateFolder)
+$customTemplateFolder = ''
+if ($originalTemplateFolder) {
+    $templateFolders += $originalTemplateFolder
+    $customTemplateFolder = $templateFolder
+}
+
 # Loop through all folders in CheckFiles and check if there are any files that needs to be updated
 foreach($fileToInclude in $filesToInclude) {
     $type = $fileToInclude.type
@@ -141,17 +156,37 @@ foreach($fileToInclude in $filesToInclude) {
     if(-not $originalSrcPath) {
         $originalSrcPath = $srcPath
     }
+    $hasDistinctOriginalSource = -not (GetPathStringComparer).Equals($originalSrcPath, $srcPath)
+
+    # Skip files that do not physically resolve to themselves within the template folders
+    if (-not (Test-PathPhysicallyEqual -Path $srcPath -AnchorPaths $templateFolders)) {
+        OutputWarning "Skipping update for source file '$srcPath': source does not physically resolve to itself within the template folder(s). This may indicate a symlink/junction redirect."
+        continue
+    }
+    if ($hasDistinctOriginalSource) {
+        # Skip files with original files that do not physically resolve to themselves within the template folders
+        if (-not (Test-PathPhysicallyEqual -Path $originalSrcPath -AnchorPaths $templateFolders)) {
+            OutputWarning "Skipping update for source file '$srcPath': original source '$originalSrcPath' does not physically resolve to itself within the template folder(s). This may indicate a symlink/junction redirect."
+            continue
+        }
+    }
 
     $dstPath = $fileToInclude.destinationFullPath
 
-    $dstFileExists = Test-Path -Path $dstPath -PathType Leaf
+    # Skip files with destinations that do not physically resolve to themselves within the base folder
+    if (-not (Test-PathPhysicallyEqual -Path $dstPath -AnchorPaths @($baseFolder))) {
+        OutputWarning "Skipping update for source file '$srcPath': destination '$dstPath' does not physically resolve to itself. This may indicate a symlink/junction redirect."
+        continue
+    }
+
+    $dstFileExists = Test-Path -LiteralPath $dstPath -PathType Leaf
 
     Write-Host "Processing file: $srcPath -> $dstPath (type: $type)"
 
     switch ($type) {
         "workflow" {
             # For workflow files, we might need to modify the file based on the settings
-            $srcContent = GetWorkflowContentWithChangesFromSettings -srcFile $originalSrcPath -repoSettings $repoSettings -depth $depth
+            $srcContent = GetWorkflowContentWithChangesFromSettings -srcFile $originalSrcPath -repoSettings $repoSettings -depth $depth -customTemplateFolder $customTemplateFolder
             # Replace static placeholders
             $srcContent = $srcContent.Replace('{TEMPLATEURL}', $templateUrl)
         }
@@ -170,7 +205,7 @@ foreach($fileToInclude in $filesToInclude) {
         ReplaceOwnerRepoAndBranch -srcContent ([ref]$srcContent) -templateOwner $templateOwner -templateBranch $templateBranch
     }
 
-    if ($type -eq 'workflow' -and $originalSrcPath -ne $srcPath) {
+    if ($type -eq 'workflow' -and $hasDistinctOriginalSource) {
         # Apply customizations from custom template repository
         Write-Host "Apply customizations from custom template repository, file: $srcPath"
         [Yaml]::ApplyTemplateCustomizations([ref] $srcContent, $srcPath)
@@ -204,8 +239,8 @@ foreach($fileToInclude in $filesToInclude) {
 
 Push-Location -Path $baseFolder
 # Remove files that are in $filesToExclude and exist in the repository
-$removeFiles = $filesToExclude | Where-Object { $_ -and (Test-Path -Path $_.destinationFullPath -PathType Leaf) } | ForEach-Object {
-    $relativePath = Resolve-Path -Path $_.destinationFullPath -Relative
+$removeFiles = $filesToExclude | Where-Object { $_ -and (Test-Path -LiteralPath $_.destinationFullPath -PathType Leaf) } | ForEach-Object {
+    $relativePath = Resolve-Path -LiteralPath $_.destinationFullPath -Relative
     Write-Host "File marked for removal: $relativePath"
     $relativePath
 }
@@ -256,16 +291,24 @@ else {
 
         invoke-git status
 
+        $dstRoot = (Get-Location).Path
+
         # Update the files
         # Calculate the release notes, while updating
         $releaseNotes = ""
         $updateFiles | ForEach-Object {
+            # Skip files that do not physically resolve to themselves within the destination root folder
+            if (-not (Test-PathPhysicallyEqual -Path (Join-Path $dstRoot $_.DstFile) -AnchorPaths @($dstRoot))) {
+                OutputWarning "Skipping update of '$($_.DstFile)': destination does not physically resolve to itself. This may indicate a symlink/junction redirect."
+                return
+            }
+
             # Create the destination folder if it doesn't exist
             $path = [System.IO.Path]::GetDirectoryName($_.DstFile)
-            if ($path -and -not (Test-Path -path $path -PathType Container)) {
+            if ($path -and -not (Test-Path -LiteralPath $path -PathType Container)) {
                 New-Item -Path $path -ItemType Directory | Out-Null
             }
-            if (([System.IO.Path]::GetFileName($_.DstFile) -eq "RELEASENOTES.copy.md") -and (Test-Path $_.DstFile)) {
+            if (([System.IO.Path]::GetFileName($_.DstFile) -eq "RELEASENOTES.copy.md") -and (Test-Path -LiteralPath $_.DstFile)) {
                 # Read the release notes of the version currently installed
                 $oldReleaseNotes = Get-ContentLF -Path $_.DstFile
                 # Get the release notes of the new version (for the PR body)
@@ -287,8 +330,14 @@ else {
             $releaseNotes = "No release notes available!"
         }
         $removeFiles | ForEach-Object {
+            # Skip files that do not physically resolve to themselves within the destination root folder
+            if (-not (Test-PathPhysicallyEqual -Path (Join-Path $dstRoot $_) -AnchorPaths @($dstRoot))) {
+                OutputWarning "Skipping removal of '$_': destination does not physically resolve to itself. This may indicate a symlink/junction redirect."
+                return
+            }
+
             Write-Host "Remove $_"
-            Remove-Item (Join-Path (Get-Location).Path $_) -Force
+            Remove-Item -LiteralPath $_ -Force
         }
 
         # Update the templateUrl and templateSha in the repo settings file

Actions/CheckForUpdates/yamlclass.ps1

@@ -19,7 +19,7 @@ class Yaml {
 
     # Static load function to load a Yaml file into a Yaml class
     static [Yaml] Load([string] $filename) {
-        $fileContent = Get-Content -Path $filename -Encoding UTF8
+        $fileContent = Get-Content -LiteralPath $filename -Encoding UTF8
         return [Yaml]::new($fileContent)
     }
 

Actions/Github-Helper.psm1

@@ -859,7 +859,7 @@ function Get-ContentLF {
     )
 
     Process {
-        (Get-Content -Path $path -Encoding UTF8 -Raw).Replace("`r", "").TrimEnd("`n")
+        (Get-Content -LiteralPath $path -Encoding UTF8 -Raw).Replace("`r", "").TrimEnd("`n")
     }
 }
 

RELEASENOTES.md

@@ -1,3 +1,13 @@
+### Enhanced `customALGoFiles` setting
+
+The `customALGoFiles` setting of a custom template was only applied on the next Update (from `AL-Go-TemplateRepoSettings.doNotEdit.json`). Now the up-to-date settings of the custom template are used directly during "Update AL-Go System Files". The template's `filesToInclude` and `filesToExclude` settings are merged with the consumer repo's settings before resolution.
+
+- **`filesToInclude`** now also resolves files from the original AL-Go template. Files present in the official template are propagated even when they are absent from your custom template. When a file exists in both, the official template supplies the base content; for workflow files, customizations from the custom template are reapplied.
+- **`filesToExclude`** now also resolves files from the original AL-Go template (same dual-resolution as `filesToInclude`). Files resolved by `filesToInclude` whose source matches a `filesToExclude` entry are not copied to consumer repos, and existing copies are removed.
+- **`destinationName`** (new property on `filesToInclude`): Allows renaming a file at the destination. When set, the file is written to `<destinationFolder>/<destinationName>` instead of keeping the source filename.
+
+Read more at [Customizing AL-Go for GitHub](Scenarios/CustomizingALGoForGitHub.md#Using-custom-template-files).
+
 ### Allow pre-release packages as NuGet dependencies
 
 AL-Go now supports including pre-release versions of Business Central app packages from NuGet feeds. To enable this, append `-allowPrerelease` to the `nuGetFeedSelectMode` setting in your project configuration. For example, `LatestMatching-allowPrerelease` will select the latest matching version of the package, including pre-release versions.

Scenarios/CustomizingALGoForGitHub.md

@@ -235,14 +235,30 @@ In order to instruct AL-Go which files to look for at the template repository, y
 
 `filesToInclude`, as the name suggests, is an array of file configurations that will instruct AL-Go which files to include (create/update). Every item in the array may contain the following properties:
 
-- `sourceFolder`: A path to a folder, relative to the template, where to look for files. If not specified the root folder is implied. `*` characters are not supported. _Example_: `src/scripts`.
+- `sourceFolder`: A literal path to a folder, relative to the template, where to look for files. If not specified the root folder is implied. Wildcards are not supported; characters such as `[` and `]` are treated literally. _Example_: `src/scripts`.
 - `filter`: A string to use for filtering in the specified source path. It can contain `*` and `?` wildcards. _Example_: `*.ps1` or `fileToUpdate.ps1`.
 - `destinationFolder`: A path to a folder, relative to repository that is being updated, where the files should be placed. If not specified, defaults to the same as the source file folder. _Example_: `src/templateScripts`.
 - `perProject`: A boolean that indicates whether the matched files should be propagated for all available AL-Go projects. In that case, `destinationFolder` is relative to the project folder. _Example_: `.AL-Go/scripts`.
+- `destinationName`: The filename to use at the destination. If specified, overrides the source filename, allowing the file to be renamed when copied. Should be used together with a `filter` that matches a single file. _Example_: `customScript.ps1`.
 
 > [!NOTE]
 > `filesToInclude` is used to define all the template files that will be used by AL-Go for GitHub. If a template file is not matched, it will be ignored. Please pay attention, when changing the file configurations: there might be template files that were previously propagated to your repositories. In case these files are no longer matched via `filesToInclude`, AL-Go for GitHub will ignore them and you might have to remove them manually.
 
+When using a custom template repository, `filesToInclude` also resolves files from the **original** AL-Go template (i.e. the official [AL-Go-PTE](https://github.com/microsoft/AL-Go-PTE) or [AL-Go-AppSource](https://github.com/microsoft/AL-Go-AppSource) template). This means files present in the official AL-Go template that are not overridden by your custom template are still propagated to consumer repositories. When a file exists in both the original template and your custom template, how the file's **content** is resolved depends on the file's type:
+
+- **Workflow files** (`.github/workflows/*.yaml`/`*.yml`): the content is based on the original template's file, with customizations from your custom template's copy (see [Adding custom jobs](#adding-custom-jobs)) re-applied on top.
+- **Settings files**: if the destination file already exists, its settings are preserved and only its `$schema` is updated from the original template. If it does not exist, the original template's settings are used. Current custom template repository settings are also read when updating workflows.
+- **All other files** (e.g. PowerShell scripts, `.copy.md`, `.agent.md`): the original template's file content is used as-is; changes made to that same file in your custom template are not applied in this case.
+
+The following table summarizes how `filesToInclude` resolves files when a custom template is in use:
+
+| File is present in original template | File is present in custom template | File is matched by `filesToInclude` | Result |
+|---|---|---|---|
+| Yes | No | Yes | File from **original template** is propagated |
+| No | Yes | Yes | File from **custom template** is propagated |
+| Yes | Yes | Yes | File from **original template** is propagated; for Workflow files, customizations from the **custom template** are also applied |
+| Yes/No | Yes/No | No | File is **ignored** |
+
 `filesToExclude` is an array of file configurations that will instruct AL-Go which files to exclude (remove) from `filesToInclude`. Every item in the array may contain the following properties:
 
 - `sourceFolder`: A path to a folder, relative to the template, where to look for files. If not specified the root folder is implied. _Example_: `src/scripts`.
@@ -251,13 +267,16 @@ In order to instruct AL-Go which files to look for at the template repository, y
 > [!NOTE] `filesToExclude` is an array of file configurations already included in `filesToInclude`. These files are specifically marked to be excluded from the update process.
 > This mechanism allows for fine-grained control over which files are propagated to the end repository and which should be explicitly removed, ensuring that unwanted files are not carried forward during updates.
 
+> [!TIP]
+> When using a custom template repository, you can use `filesToExclude` in the custom template's settings to prevent files from the original AL-Go template from being propagated to consumer repos. For example, if the original template includes a workflow you don't want in your consumer repos, adding it to `filesToExclude` in your custom template's settings will remove it during the next update.
+
 The following table summarizes how AL-Go for GitHub manages file updates and exclusions when using custom template files. Say, there is a file (e.g. `file.ps1`) in the template repository.
 
 | File is present in end repo | File is matched by `filesToInclude` | File is matched by `filesToExclude` | Result |
 |---|---|---|---|
 | Yes/No | Yes | No | The file is **updated/created** in the end repo |
 | Yes | Yes | Yes | The file is **removed** from the end repo, as it's matched for exclusion |
-| Yes | No | Yes | The files is **_not_** removed as it was not matched as update |
+| Yes | No | Yes | The file is **_not_** removed as it was not matched as update |
 | No | Yes/No | Yes | The file is **_not_ created** in the end repo, as it's matched for exclusion |
 
 ### Examples of using custom template files