How to use output of a powershell command as parameters in Azure pipeline? - parameter-passing

Question

I have a PowerShell script task that gets the names of some files from a folder in my git repo and puts them into a variable. I want to use those file names in parameters and use "each" condition in another task (task: HelmDeploy#0) in order to run that task each time with one of the file names as valueFile variable.
Here is what I have tried, however it gives an error Template-Yaml/deploy-jobs.yaml#pipelinetemplates Expected a sequence or mapping. Actual value '$[dependencies.A.outputs['fileoutput.valuefiles']]' in line ${{each file in parameters.files}}
deploy-jobs.yaml
parameters:
files: []
jobs:
- job: Deploy
pool:
vmImage: 'ubuntu-latest'
variables:
filenames: ${{ parameters.files }}
steps:
- task: HelmInstaller#1
displayName: 'Installing Helm'
inputs:
helmVersionToInstall: '2.15.1'
- task: HelmDeploy#0
displayName: 'Initializing Helm'
inputs:
connectionType: 'Azure Resource Manager'
azureSubscription: $(azureSubscription)
azureResourceGroup: $(azureResourceGroup)
kubernetesCluster: $(kubernetesCluster)
command: 'init'
- task: AzureCLI#2
inputs:
azureSubscription: $(azureSubscription)
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript:
echo "##vso[task.setvariable variable=imgtag]$(az acr repository show-tags --name myacr --repository myrepo --orderby time_desc --top 1 | awk ' /[[:digit:]]/ { print $0 } ' | tr -d '[:space:]')"
- task: Bash#3
displayName: 'Fetching repo-tag'
inputs:
targetType: 'inline'
script: |
echo tag=$(imgtag)
echo filenames=$(filenames) ### output is: **/myfolder/dev/file1.yaml,**/myfolder/dev/file2.yaml
- ${{each file in parameters.files}}: ##Error
- task: HelmDeploy#0
displayName: 'Upgrading helmchart'
inputs:
connectionType: 'Azure Resource Manager'
azureSubscription: $(azureSubscription)
azureResourceGroup: $(azureResourceGroup)
kubernetesCluster: $(kubernetesCluster)
command: 'upgrade'
chartType: 'FilePath'
chartPath: $(chartPath)
install: true
releaseName: $(releaseName)
##valueFile: $(valuefiles)
valueFile: ${{ file }}
arguments: '--set image.tag=$(imgtag) --set domain=$(domain)'
azure-pipeline.yaml file is as following:
trigger:
branches:
include:
- master
- refs/tags/v*
paths:
exclude:
- readme.md
variables:
azureSubscription: 'myazuresubscription'
chartPath: '**/mychart'
containerRegistry: 'mysc'
repository: 'myrepo'
resources:
repositories:
- repository: pipelinetemplates
type: github
name: 'mygitorg/myrepo'
endpoint: 'mygitorg'
stages:
- stage: Deploy_Cluster
variables:
azureResourceGroup: 'myresourcegroup'
kubernetesCluster: 'mycluster'
releaseName: 'mychartreleasename'
#valueFile: '**/mychart/values.yaml'
domain: 'mydomain'
jobs:
- job: A
pool:
vmImage: 'ubuntu-latest'
steps:
- task: PowerShell#2
displayName: 'Fetching ValueFiles'
inputs:
targetType: 'inline'
script: |
Write-Host "Fetching value files"
cd myfolder
$a=git ls-files
$List = $a | foreach {'**/myfolder/dev/' + $_}
Write-Host $List
$d = '"{0}"' -f ($List -join '","')
Write-Host $d ### output is: "**/myfolder/dev/file1.yaml","**/myfolder/dev/file2.yaml"
Write-Host "##vso[task.setvariable variable=valuefiles;isOutput=true]$d"
name: fileoutput
- template: Template-Yaml/deploy-jobs.yaml#pipelinetemplates ##Error expected a sequence or mapping
parameters:
files : $[dependencies.Deploy.outputs['fileoutput.valuefiles']]
I got some idea from this page: https://www.aaron-powell.com/posts/2019-05-24-azure-pipeline-templates-and-parameters/ regarding using dependencies.
I googled a lot, however I couldn't find a solution to this issue so far, any help would be appreciated.
Tested the replied suggested by Levi:
parameters:
files: []
jobs:
#- ${{each file in parameters.files}}:
- job: Deploy
dependsOn: A
pool:
vmImage: 'ubuntu-latest'
variables:
filenames: ${{ parameters.file }}
steps:
- task: Bash#3
displayName: 'Fetching repo-tag'
inputs:
targetType: 'inline'
script: |
##echo files=$(filenames) #output is files=file1.yaml,file2.yaml
for i in $(filenames)
do
echo "valuefiles= $i "
done
OutPut is valuefiles= files=file1.yaml,file2.yaml
Testing with PowerShell:
- task: PowerShell#2
displayName: 'Fetching ValueFiles'
inputs:
targetType: 'inline'
script: |
foreach ($i in ${{ parameters.files }}) {
Write-Host "filenames=$i"
}
Error: ObjectNotFound: ($[dependencies.A.ou\u2026output.valuefiles]]:String) [], ParentContainsErrorRecordException
- task: PowerShell#2
displayName: 'Fetching ValueFiles'
inputs:
targetType: 'inline'
script: |
foreach ($i in $(filenames)) {
Write-Host "filenames=$i"
}
Error: foreach ($i in ) {
+ ~
Unexpected token ')' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException

Answer 1

The error occurred is because the files can only be caculated at runtime, but - ${{each file in parameters.files}} is valuated at compile time. Check here for more information about variable syntax.
- ${{each file in parameters.files}} won't work if the dynamic variables have been passed through parameters.
I doesnot know much about kubernetes, if you can manage to use powershell/bash script to do the HelmDeploy task. You can foreach the files inside script and deploy each file.
- powershell: |
foreach ($i in $(filenames)) {helm.exe upgrade ....}
A similar issue has been submitted to Microsoft Develop team, and it is determined that above issue is by design. But you can still submit a feature request by clicking "Suggest a feature " and choose "azure devops"
Update:
azure-pipeline.yaml
trigger:
branches:
include:
- master
stages:
- stage: Deploy_Cluster
jobs:
- job: A
pool:
vmImage: 'ubuntu-latest'
steps:
- task: PowerShell#2
displayName: 'Fetching ValueFiles'
inputs:
targetType: 'inline'
script: |
Write-Host "Fetching value files"
cd '$(system.defaultworkingdirectory)'
$a=git ls-files
$List = $a | foreach {'**/myfolder/dev/' + $_}
Write-Host $List
$d = '"{0}"' -f ($List -join '","')
Write-Host $d
Write-Host "##vso[task.setvariable variable=valuefiles;isOutput=true]$d"
name: fileoutput
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-latest'
variables:
allfiles: $[dependencies.A.outputs['fileoutput.valuefiles']]
steps:
- template: deploy-jobs.yaml
parameters:
files : $(allfiles)
deploy-jobs.yaml
parameters:
files: []
steps :
- task: PowerShell#2
displayName: 'Fetching ValueFiles'
inputs:
targetType: 'inline'
script: >
foreach ($i in ${{parameters.files}})
{
Write-Host "filenames=$i"
}