Use of ResultTemplate in vsts extension tasks - json

I created a task that talks to a REST API to retreive the values for 2 picklists.
Filling the first dropdown box works fine, when using just the jsonpath.
Based on the first picklist I'd like to retreive values of the second list.
I've tried some variations and I'm trying something like this:
The json which I receive in the first rest call is similar to:
{
"id": "45",
"href": "https://selfservice/api/configurations/45/",
"name": "Type",
"description": "",
"version": "0.0.4",
"attributes": [
{
"value": {
"sdk-object": {
"type": "ConfigurationElement",
"id": "56"
}
},
"type": "ConfigurationElement",
"name": "win"
},
{
"value": {
"sdk-object": {
"type": "ConfigurationElement",
"id": "57"
}
},
"type": "ConfigurationElement",
"name": "lin"
}
]
}
I try to show the attributes name in the list and need the id of the attribute in the second picklist.
I created the following datasourcebindings in the task.json. Of course, the targets exist in the task.
task.json:
{
"id": "GUID",
"name": "Spinup",
"friendlyName": "Create environment",
"description": "Starts Workflow to create an environment. ___ The following placeholders are created deplending on the template and can be used in the rest of the release:**XLDEnvironment** and **testAgentHostname**",
"category": "Deploy",
"author": "***",
"version": {
"Major": 0,
"Minor": 0,
"Patch": 33
},
"minimumAgentVersion": "1.95.3",
"inputs": [
{
"label": "Endpoint",
"name": "connectedServiceName",
"required": true,
"type": "connectedService:server",
"helpMarkDown": "endpoint to connect to."
},
{
"name": "stage",
"type": "string",
"label": "Stage",
"defaultValue": "$(Release.EnvironmentName)",
"required": true,
"helpMarkDown": "Stage of the release, default value is based on the pipeline."
},
{
"name": "releaseName",
"type": "string",
"label": "Environment name",
"defaultValue": "$(Release.ReleaseName)",
"required": true,
"helpMarkDown": "Name of the environment that will be created."
},
{
"name": "owner",
"type": "string",
"label": "Owner of the environment",
"defaultValue": "***",
"required": true,
"helpMarkDown": "It is common to use the initials of the product owner in this field. There has to be a valid AD user owner of each environment. This can also be set by using a variable through the pipeline."
},
{
"name": "group",
"type": "string",
"label": "Group of the owner",
"defaultValue": "",
"required": true,
"helpMarkDown": "group that owns the environment. groups can be requested by the infrastructure department. Example: ****"
},
{
"name": "platform",
"type": "pickList",
"label": "Platform (OS)",
"defaultValue": "",
"required": true,
"helpMarkDown": "Choose the type of the target platform."
},
{
"name": "size",
"type": "pickList",
"label": "Environment Template",
"defaultValue": "",
"required": true,
"helpMarkDown": "Size of the environment to create."
}
],
"dataSourceBindings": [
{
"dataSourceName": "GetCEPlatformType",
"endpointId": "$(connectedServiceName)",
"target": "platform",
"selector": "jsonpath:$.attributes[*].name",
"keySelector": "jsonpath:$.attributes[*].value.sdk-object.id"
},
{
"dataSourceName": "GetCEPlatformSize",
"endpointId": "$(connectedServiceName)",
"target": "size",
"parameters": {
"platformTypeId": "$(platform)"
}
}
],
"instanceNameFormat": "Spin up $(size) $(platform) environment",
"execution": {
"PowerShell3": {
"target": "$(currentDirectory)\\task.ps1",
"argumentFormat": "",
"workingDirectory": "$(currentDirectory)"
}
}
}
vss-extension.json:
{
"manifestVersion": 1,
"id": "*****",
"name": "Release Tasks",
"version": "1.0.1",
"publisher": "***",
"targets": [
{
"id": "Microsoft.VisualStudio.Services"
}
],
"description": "Tools to contact ***. Includes a task to spin up a platform and a task to remove the platform.",
"categories": [
"Build and release"
],
"icons": {
"default": "images/extension-icon.png"
},
"files": [
{
"path": "RemoveEnvironment"
},
{
"path": "SpinUpEnvironment"
}
],
"contributions": [
{
"id": "******",
"type": "ms.vss-distributed-task.task",
"targets": [
"ms.vss-distributed-task.tasks"
],
"properties": {
"name": "RemoveEnvironment"
}
},
{
"id": "*********",
"type": "ms.vss-distributed-task.task",
"targets": [
"ms.vss-distributed-task.tasks"
],
"properties": {
"name": "SpinUpEnvironment"
}
},
{
"description": "Service Endpoint type for all connections",
"id": "endpoint-type",
"properties": {
"authenticationSchemes": [
{
"inputDescriptors": [
{
"description": "Username",
"id": "username",
"inputMode": "textbox",
"name": "Username",
"validation": {
"dataType": "string",
"isRequired": true
}
},
{
"description": "Password",
"id": "password",
"inputMode": "passwordbox",
"isConfidential": true,
"name": "Password",
"validation": {
"dataType": "string",
"isRequired": false
}
}
],
"type": "ms.vss-endpoint.endpoint-auth-scheme-basic"
}
],
"dataSources": [
{
"endpointUrl": "api/configurations/*****/",
"name": "GetCEPlatformType",
"resultSelector": "jsonpath:$.attributes[*].name"
},
{
"endpointUrl": "api/configurations/$(platformTypeId)/",
"name": "GetCEPlatformSize",
"resultSelector": "jsonpath:$.attributes[*].name"
}
],
"displayName": "*****",
"helpMarkDown": "Enter the url and credentials to connect to the endpoint.",
"name": "server",
"url": {
"displayName": "Server URL",
"helpText": "Url for the server to connect to."
}
},
"targets": [
"ms.vss-endpoint.endpoint-types"
],
"type": "ms.vss-endpoint.service-endpoint-type"
}
]
}
Sadly I can't get this to work. The datasources exist in the endpoint, but I'm not sure what to do with the resultSelector.
Does anyone have an idea on how to get this to work? The documentation on this isn't too good.
I'm not familiar with mustache, but in the mustache test tool this seems to work.
Thoughts are appreciated!

Try with this in the first data source:
{
"dataSourceName": "dsList1",
"endpointId": "$(connectedServiceName)",
"target": "list1",
"selector": "jsonpath:$.attributes[*].name",
"keySelector": "jsonpath:$.attributes[*].value.sdk-object.id"
}
Update:
You were trying to use "platformTypeId" in vss-extension.json file while it was defined in task.json, this cause the error message you mentioned in the comment.
By the way, I just noticed that you are using "dataSourceBindings", then you cannot use "selector" and "KeySelector" to parse the result since they are used for "sourceDefinitions".
To achieve the feature you want with "dataSourceBindings", you can definition the endpoint url in the task.json directly instead of defining datasource in endpoint contribution.
So you can move the datasources section in the vss-extension.json file first, and then in the task.json file, change to following:
"dataSourceBindings": [
{
"endpointId": "$(connectedServiceName)",
"endpointURL": "api/configurations/*****/",
"target": "platform",
"resultselector": "jsonpath:$.attributes[*]",
"resultTemplate": "{ \"Value\" : \"{{{value.id}}}\", \"DisplayValue\" : \"{{{name}}}\" }"
},
{
"endpointId": "$(connectedServiceName)",
"endpointURL": "api/configurations/$(platformTypeId)/",
"target": "size"
}
],

Related

Map step function result and extract certain keys

I have the following output coming from a step function task: ListObjectsV2
{
"Contents": [
{
"ETag": "\"86c12c034bc6c30cb89b500b954c188f\"",
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_1.csv",
"LastModified": "2023-02-09T13:46:20Z",
"Size": 796014,
"StorageClass": "STANDARD"
},
{
"ETag": "\"58e4a770e0f66073b00d185df500f07f\"",
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_2.csv",
"LastModified": "2023-02-09T13:47:20Z",
"Size": 934038,
"StorageClass": "STANDARD"
},
{
"ETag": "\"460abd0de64d5cb67e8f0d46878cb1ef\"",
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_3.csv",
"LastModified": "2023-02-09T13:46:57Z",
"Size": 794264,
"StorageClass": "STANDARD"
},
{
"ETag": "\"1bfedc3dc92e4ba8d04e24b9b5a0ed58\"",
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_4.csv",
"LastModified": "2023-02-09T13:46:24Z",
"Size": 788756,
"StorageClass": "STANDARD"
},
{
"ETag": "\"9d6c434ce5ebdf203a790fbcf19338dc\"",
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_5.csv",
"LastModified": "2023-02-09T13:47:07Z",
"Size": 831156,
"StorageClass": "STANDARD"
}
],
"IsTruncated": false,
"KeyCount": 5,
"MaxKeys": 1000,
"Name": "vita-internal-text-classification-dev-183576513728",
"Prefix": "55271f52fffe4461a2ee3228ebb97157"
}
I want to have an array containing only the Key key, to pass to the next state, like so:
[
{
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_1.csv",
},
{
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_2.csv",
},
{
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_3.csv",
},
{
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_4.csv",
},
{
"Key": "55271f52fffe4461a2ee3228ebb97157/input/batch_5.csv",
}
]
So far I've tried setting the ResultPath to:
$.Contents[*].Key
$.Contents[*].['Key']
What I get is:
[
"55271f52fffe4461a2ee3228ebb97157/input/batch_1.csv",
"55271f52fffe4461a2ee3228ebb97157/input/batch_2.csv",
"55271f52fffe4461a2ee3228ebb97157/input/batch_3.csv",
"55271f52fffe4461a2ee3228ebb97157/input/batch_4.csv",
"55271f52fffe4461a2ee3228ebb97157/input/batch_5.csv",
]
But I've gotten bad output from that, any help?
The way I've solved this is to use an Inline Map state with a Pass state to build the necessary format. You can see this pattern in an example here for how to use Step Functions Distributed Map to bulk delete objects from S3. You can see this in the inner Create Object Identifier Array Map state. If you were doing this in Standard Workflows, this could be a cost concern given the number of state transitions involved. But since in the Item Processor I'm using Express Workflows, which are billed by duration (and these are super fast), it works pretty well.
{
"Comment": "A state machine to bulk delete objects from S3 using Distributed Map",
"StartAt": "Confirm Bucket Provided",
"States": {
"Confirm Bucket Provided": {
"Type": "Choice",
"Choices": [
{
"Not": {
"Variable": "$.bucket",
"IsPresent": true
},
"Next": "Fail - No Bucket"
}
],
"Default": "Check for Prefix"
},
"Check for Prefix": {
"Type": "Choice",
"Choices": [
{
"Not": {
"Variable": "$.prefix",
"IsPresent": true
},
"Next": "Generate Parameters - Without Prefix"
}
],
"Default": "Generate Parameters - With Prefix"
},
"Generate Parameters - Without Prefix": {
"Type": "Pass",
"Parameters": {
"Bucket.$": "$.bucket",
"Prefix": ""
},
"ResultPath": "$.list_parameters",
"Next": "Delete Objects from S3 Bucket"
},
"Fail - No Bucket": {
"Type": "Fail",
"Error": "InsuffcientArguments",
"Cause": "No Bucket was provided"
},
"Generate Parameters - With Prefix": {
"Type": "Pass",
"Next": "Delete Objects from S3 Bucket",
"Parameters": {
"Bucket.$": "$.bucket",
"Prefix.$": "$.prefix"
},
"ResultPath": "$.list_parameters"
},
"Delete Objects from S3 Bucket": {
"Type": "Map",
"ItemProcessor": {
"ProcessorConfig": {
"Mode": "DISTRIBUTED",
"ExecutionType": "EXPRESS"
},
"StartAt": "Create Object Identifier Array",
"States": {
"Create Object Identifier Array": {
"Type": "Map",
"ItemProcessor": {
"ProcessorConfig": {
"Mode": "INLINE"
},
"StartAt": "Create Object Identifier",
"States": {
"Create Object Identifier": {
"Type": "Pass",
"End": true,
"Parameters": {
"Key.$": "$.Key"
}
}
}
},
"ItemsPath": "$.Items",
"ResultPath": "$.object_identifiers",
"Next": "Delete Objects"
},
"Delete Objects": {
"Type": "Task",
"Next": "Clear Output",
"Parameters": {
"Bucket.$": "$.BatchInput.bucket",
"Delete": {
"Objects.$": "$.object_identifiers"
}
},
"Resource": "arn:aws:states:::aws-sdk:s3:deleteObjects",
"Retry": [
{
"ErrorEquals": [
"States.ALL"
],
"BackoffRate": 2,
"IntervalSeconds": 1,
"MaxAttempts": 6
}
],
"ResultSelector": {
"Deleted.$": "$.Deleted",
"RetryCount.$": "$$.State.RetryCount"
}
},
"Clear Output": {
"Type": "Pass",
"End": true,
"Result": {}
}
}
},
"ItemReader": {
"Resource": "arn:aws:states:::s3:listObjectsV2",
"Parameters": {
"Bucket.$": "$.list_parameters.Bucket",
"Prefix.$": "$.list_parameters.Prefix"
}
},
"MaxConcurrency": 5,
"Label": "S3objectkeys",
"ItemBatcher": {
"MaxInputBytesPerBatch": 204800,
"MaxItemsPerBatch": 1000,
"BatchInput": {
"bucket.$": "$.list_parameters.Bucket"
}
},
"ResultSelector": {},
"End": true
}
}
}

I am trying to deploy an Azure SQL VM with ARM template. I am getting error as The template reference 'ion5eddb999' is ambiguous

The full error is below.
New-AzResourceGroupDeployment : 23:35:36 - Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The template reference 'ion5eddb999' is ambiguous: there are multiple template
resources '/subscriptions/d2143d4c-a258-4a38-8fbf-de0e42756e22/resourceGroups/sumantest/providers/Microsoft.Compute/virtualMachines/ion5eddb999,/subscriptions/d2143d4c-a258-4a38-8fbf-de0e42756e22/resou
rceGroups/sumantest/providers/Microsoft.SqlVirtualMachine/SqlVirtualMachines/ion5eddb999' defined with this name. Please use fully qualified resource identity instead. Please see
https://aka.ms/arm-template-expressions/#reference for usage details.'.
Below is the ARM template. I have tried using hard coded sqlserver name as well, but same error.
Please let me know what is wrong I am doing here. Any help is appreciated.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"virtualMachineName": {
"type": "String",
"defaultValue": "ion5eddb999",
"metadata": {
"description": "The name of the VM"
}
},
"virtualMachineSize": {
"type": "String",
"defaultValue": "Standard_B4ms",
"metadata": {
"description": "The virtual machine size."
}
},
"ipAddress":{
"type":"string",
"defaultValue": "172.31.172.99",
"metadata": {
"description": "The virtual machine ip address"
}
},
"existingVirtualNetworkName": {
"type": "String",
"defaultValue": "ion5ed-vnet",
"metadata": {
"description": "Specify the name of an existing VNet in the same resource group"
}
},
"existingVnetResourceGroup": {
"type": "String",
"defaultValue": "ion5ed-gateway",
"metadata": {
"description": "Specify the resrouce group of the existing VNet"
}
},
"existingSubnetName": {
"type": "String",
"defaultValue": "ion5ed-sub-devtest",
"metadata": {
"description": "Specify the name of the Subnet Name"
}
},
"imageOffer": {
"type": "String",
"defaultValue": "sql2019-ws2019",
"allowedValues": [
"sql2019-ws2019",
"sql2017-ws2019",
"SQL2017-WS2016",
"SQL2016SP1-WS2016",
"SQL2016SP2-WS2016",
"SQL2014SP3-WS2012R2",
"SQL2014SP2-WS2012R2"
],
"metadata": {
"description": "Windows Server and SQL Offer"
}
},
"sqlSku": {
"type": "String",
"defaultValue": "Standard",
"allowedValues": [
"Standard",
"Enterprise",
"SQLDEV",
"Web",
"Express"
],
"metadata": {
"description": "SQL Server Sku"
}
},
"adminUsername": {
"type": "String",
"metadata": {
"description": "The admin user name of the VM"
}
},
"adminPassword": {
"type": "SecureString",
"metadata": {
"description": "The admin password of the VM"
}
},
"storageWorkloadType": {
"type": "String",
"defaultValue": "General",
"allowedValues": [
"General",
"OLTP",
"DW"
],
"metadata": {
"description": "SQL Server Workload Type"
}
},
"sqlVirtualMachineName": {
"type": "string",
"defaultValue": "ion5eddb999"
},
"sqlDataDisksCount": {
"type": "int",
"defaultValue": 1,
"minValue": 1,
"maxValue": 8,
"metadata": {
"description": "Amount of data disks (1TB each) for SQL Data files"
}
},
"dataPath": {
"type": "String",
"defaultValue": "F:\\SQLData",
"metadata": {
"description": "Path for SQL Data files. Please choose drive letter from F to Z, and other drives from A to E are reserved for system"
}
},
"sqlLogDisksCount": {
"type": "int",
"defaultValue": 1,
"minValue": 1,
"maxValue": 8,
"metadata": {
"description": "Amount of data disks (1TB each) for SQL Log files"
}
},
"logPath": {
"type": "String",
"defaultValue": "G:\\SQLLog",
"metadata": {
"description": "Path for SQL Log files. Please choose drive letter from F to Z and different than the one used for SQL data. Drive letter from A to E are reserved for system"
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
}
},
"variables": {
"networkInterfaceName": "[concat(parameters('virtualMachineName'), '-nic')]",
//"networkSecurityGroupName": "[concat(parameters('virtualMachineName'), '-nsg')]",
"diskConfigurationType": "NEW",
"subnetRef": "[resourceID(parameters('existingVNetResourceGroup'), 'Microsoft.Network/virtualNetWorks/subnets', parameters('existingVirtualNetworkName'), parameters('existingSubNetName'))]",
"dataDisksLuns": "[array(range(0 ,parameters('sqlDataDisksCount')))]",
"logDisksLuns": "[array(range(parameters('sqlDataDisksCount'), parameters('sqlLogDisksCount')))]",
"dataDisks": {
"createOption": "empty",
"caching": "ReadOnly",
"writeAcceleratorEnabled": false,
"storageAccountType": "StandardSSD_LRS",
"diskSizeGB": 100
},
"tempDbPath": "D:\\SQLTemp"
},
"resources": [
{
"type": "Microsoft.Network/networkInterfaces",
"apiVersion": "2020-06-01",
"name": "[variables('networkInterfaceName')]",
"location": "[parameters('location')]",
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
},
"privateIPAllocationMethod": "Static",
"privateIPAddress": "[parameters('ipAddress')]",
"privateIPAddressVersion": "IPv4"
}
}
],
"enableAcceleratedNetworking": false
}
},
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2020-06-01",
"name": "[parameters('virtualMachineName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkInterfaces/', variables('networkInterfaceName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('virtualMachineSize')]"
},
"storageProfile": {
"osDisk": {
"createOption": "fromImage",
"managedDisk": {
"storageAccountType": "StandardSSD_LRS"
}
},
"imageReference": {
"publisher": "MicrosoftSQLServer",
"offer": "[parameters('imageOffer')]",
"sku": "[parameters('sqlSku')]",
"version": "latest"
},
"copy": [
{
"name": "dataDisks",
"count": "[add(parameters('sqlDataDisksCount'), parameters('sqlLogDisksCount'))]",
"input": {
"lun": "[copyIndex('dataDisks')]",
"createOption": "[variables('dataDisks').createOption]",
"caching": "[if(greaterOrEquals(copyIndex('dataDisks'), parameters('sqlDataDisksCount')) ,'None', variables('dataDisks').caching )]",
"writeAcceleratorEnabled": "[variables('dataDisks').writeAcceleratorEnabled]",
"diskSizeGB": "[variables('dataDisks').diskSizeGB]",
"managedDisk": {
"storageAccountType": "[variables('dataDisks').storageAccountType]"
}
}
}
]
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkInterfaceName'))]"
}
]
},
"osProfile": {
"computerName": "[parameters('virtualMachineName')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]",
"windowsConfiguration": {
"enableAutomaticUpdates": true,
"provisionVmAgent": true
}
}
}
},
{
"type": "Microsoft.SqlVirtualMachine/SqlVirtualMachines",
"apiVersion": "2017-03-01-preview",
"name": "[last(split(resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName')),'/'))]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName'))]"
],
"properties": {
"virtualMachineResourceId": "[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName'))]",
"sqlManagement": "Full",
"SqlServerLicenseType": "PAYG",
"StorageConfigurationSettings": {
"DiskConfigurationType": "[variables('diskConfigurationType')]",
"StorageWorkloadType": "[parameters('storageWorkloadType')]",
"SQLDataSettings": {
"LUNs": "[variables('dataDisksLUNs')]",
"DefaultFilePath": "[parameters('dataPath')]"
},
"SQLLogSettings": {
"Luns": "[variables('logDisksLUNs')]",
"DefaultFilePath": "[parameters('logPath')]"
},
"SQLTempDbSettings": {
"DefaultFilePath": "[variables('tempDbPath')]"
}
}
}
}
],
"outputs": {
"virtualMachine": {
"type": "object",
"value": "[reference(parameters('virtualMachineName'))]"
}
}
}
For your output use:
"outputs": {
"virtualMachine": {
"type": "object",
"value": "[reference(resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName')))]"
},
"sqlVirtualMachine": {
"type": "object",
"value": "[reference(resourceId('Microsoft.SqlVirtualMachine/SqlVirtualMachines', parameters('virtualMachineName')))]"
}
}
Depending on what you're after... You can name them the same, but any reference to them (dependsOn, reference()) needs to be unambiguous.

How to extend choice in adaptive card when option in ChoiceSet selected?

I have no idea how to use "Only show when" property in my ChoiceSet. I want "Version" ChoiceSet to be displayed when second option in the first ChoiceSet is selected. Maybe I need to save responce from the first ChoiceSet to a variable but I also have no idea how to do it.
{
"type": "AdaptiveCard",
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.5",
"body": [
{
"type": "Input.ChoiceSet",
"choices": [
{
"title": "Online",
"value": "Online"
},
{
"title": "On-Premise",
"value": "OnPremise"
}
],
"placeholder": "Placeholder text",
"style": "expanded",
"id": "EnvironmentType"
},
{
"type": "TextBlock",
"text": "choose version",
"wrap": true
},
{
"type": "Input.ChoiceSet",
"choices": [
{
"title": "2019",
"value": "2019"
},
{
"title": "2016",
"value": "2016"
}
],
"placeholder": "Version",
"isVisible": false,
"id": "ChoiceSetId"
}
]
}

Azure ARM template - using array variable

I am defining an alertrule inside my template.json with custom eMails which shall be alerted in case of an error. So the JSON snippet looks like this:
"resources": [
{
"type": "microsoft.insights/alertrules",
"properties": {
"action": {
"customEmails": [
"user01#contoso.com",
"user02#contoso.com"
]
}
}
}
]
Now I'd like to store these emails as an array-variable, something like this:
"variables": {
"alertEmails": [
"user01#contoso.com",
"user02#contoso.com"
]
},
"resources": [
{
"type": "microsoft.insights/alertrules",
"properties": {
"action": {
"customEmails": "[variables('alertEmails')]"
}
}
}
]
This doesn't work, but I didn't found out what the correct syntax is. Can someone tell me?
If you want to use array, maybe we can use json like this:
"parameters": {
"customEmailAddresses": {
"type": "array",
"defaultValue": ["one#microsoft.com",
"two#microsoft.com",
"three#microsoft.com"]
}
},
and in action, like this:
"customEmails": "[array(parameters('customEmailAddresses'))]"
This doesn't work, but I didn't found out what the correct syntax is. Can someone tell me?
I test it with your supplied code, it works correctly on my side. The following is my used template. Please have a try to test with the following demo code.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"serverFarmName": {
"type": "string",
"defaultValue": "serviceplan"
},
"resourceId": {
"type": "string",
"defaultValue": "/subscriptions/{subscriptionId}/resourceGroups/{resourceName}/providers/Microsoft.Web/sites/websiteName",
"metadata": {
"description": "Resource ID of the resource emitting the metric that will be used for the comparison."
}
},
"metricName": {
"type": "string",
"defaultValue": "BytesReceived",
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "1",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"aggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Last",
"Maximum",
"Minimum",
"Total"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be between five minutes and one day. ISO 8601 duration format."
}
},
"sendToServiceOwners": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether alerts are sent to service owners"
}
},
"webhookUrl": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "URL of a webhook that will receive an HTTP POST when the alert activates."
}
},
"serverFarmResourceGroup": {
"type": "string",
"defaultValue": "resourceGroup"
}
},
"variables": {
"alertEmails": [
"sunguiguan#hotmail.com",
"user02#contoso.com"
],
"TomARMtestName": "[concat('TomARMtest', uniqueString(resourceGroup().id))]"
},
"resources": [
{
"type": "Microsoft.Insights/alertRules",
"name": "newalert",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-01",
"properties": {
"name": "newalert",
"description": "newalert",
"isEnabled": true,
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ThresholdRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleMetricDataSource",
"resourceUri": "[parameters('resourceId')]",
"metricName": "[parameters('metricName')]"
},
"operator": "[parameters('operator')]",
"threshold": "[parameters('threshold')]",
"windowSize": "[parameters('windowSize')]",
"timeAggregation": "[parameters('aggregation')]"
},
"actions": [
{
"customEmails": "[variables('alertEmails')]"
}
]
}
}
]
,
"outputs": {
"out": {
"type": "array",
"value": "[variables('alertEmails')]"
}
}
}
And I follow the azure document to use customEmails": "[split(parameters('customEmailAddresses'), ',')]" code, it also works correctly on my side.

Azure Data Factory Copy Activity

I have been working on this for a couple days and cannot get past this error. I have 2 activities in this pipeline. The first activity copies data from an ODBC connection to an Azure database, which is successful. The 2nd activity transfers the data from Azure table to another Azure table and keeps failing.
The error message is:
Copy activity met invalid parameters: 'UnknownParameterName', Detailed message: An item with the same key has already been added..
I do not see any invalid parameters or unknown parameter names. I have rewritten this multiple times using their add activity code template and by myself, but do not receive any errors when deploying on when it is running. Below is the JSON pipeline code.
Only the 2nd activity is receiving an error.
Thanks.
Source Data set
{
"name": "AnalyticsDB-SHIPUPS_06shp-01src_AZ-915PM",
"properties": {
"structure": [
{
"name": "UPSD_BOL",
"type": "String"
},
{
"name": "UPSD_ORDN",
"type": "String"
}
],
"published": false,
"type": "AzureSqlTable",
"linkedServiceName": "Source-SQLAzure",
"typeProperties": {},
"availability": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"external": true,
"policy": {}
}
}
Destination Data set
{
"name": "AnalyticsDB-SHIPUPS_06shp-02dst_AZ-915PM",
"properties": {
"structure": [
{
"name": "SHIP_SYS_TRACK_NUM",
"type": "String"
},
{
"name": "SHIP_TRACK_NUM",
"type": "String"
}
],
"published": false,
"type": "AzureSqlTable",
"linkedServiceName": "Destination-Azure-AnalyticsDB",
"typeProperties": {
"tableName": "[olcm].[SHIP_Tracking]"
},
"availability": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"external": false,
"policy": {}
}
}
Pipeline
{
"name": "SHIPUPS_FC_COPY-915PM",
"properties": {
"description": "copy shipments ",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "RelationalSource",
"query": "$$Text.Format('SELECT COMPANY, UPSD_ORDN, UPSD_BOL FROM \"orupsd - UPS interface Dtl\" WHERE COMPANY = \\'01\\'', WindowStart, WindowEnd)"
},
"sink": {
"type": "SqlSink",
"sqlWriterCleanupScript": "$$Text.Format('delete imp_fc.SHIP_UPS_IntDtl_Tracking', WindowStart, WindowEnd)",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
},
"translator": {
"type": "TabularTranslator",
"columnMappings": "COMPANY:COMPANY, UPSD_ORDN:UPSD_ORDN, UPSD_BOL:UPSD_BOL"
}
},
"inputs": [
{
"name": "AnalyticsDB-SHIPUPS_03shp-01src_FC-915PM"
}
],
"outputs": [
{
"name": "AnalyticsDB-SHIPUPS_03shp-02dst_AZ-915PM"
}
],
"policy": {
"timeout": "1.00:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval",
"retry": 3,
"longRetry": 0,
"longRetryInterval": "00:00:00"
},
"scheduler": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"name": "915PM-SHIPUPS-fc-copy->[imp_fc]_[SHIP_UPS_IntDtl_Tracking]"
},
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "SqlSource",
"sqlReaderQuery": "$$Text.Format('select distinct ups.UPSD_BOL, ups.UPSD_BOL from imp_fc.SHIP_UPS_IntDtl_Tracking ups LEFT JOIN olcm.SHIP_Tracking st ON ups.UPSD_BOL = st.SHIP_SYS_TRACK_NUM WHERE st.SHIP_SYS_TRACK_NUM IS NULL', WindowStart, WindowEnd)"
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
},
"translator": {
"type": "TabularTranslator",
"columnMappings": "UPSD_BOL:SHIP_SYS_TRACK_NUM, UPSD_BOL:SHIP_TRACK_NUM"
}
},
"inputs": [
{
"name": "AnalyticsDB-SHIPUPS_06shp-01src_AZ-915PM"
}
],
"outputs": [
{
"name": "AnalyticsDB-SHIPUPS_06shp-02dst_AZ-915PM"
}
],
"policy": {
"timeout": "1.00:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval",
"retry": 3,
"longRetryInterval": "00:00:00"
},
"scheduler": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"name": "915PM-SHIPUPS-AZ-update->[olcm]_[SHIP_Tracking]"
}
],
"start": "2017-08-22T03:00:00Z",
"end": "2099-12-31T08:00:00Z",
"isPaused": false,
"hubName": "adf-tm-prod-01_hub",
"pipelineMode": "Scheduled"
}
}
Have you seen this link?
They get the same error message and suggest using AzureTableSink instead of SqlSink
"sink": {
"type": "AzureTableSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
}
It would make sense for you too since your 2nd copy activity is Azure to Azure
It could be a red herring but I'm pretty sure "tableName" is a require entry in the typeProperties for a sqlSource. Yours is missing this for the input dataset. Appreciate you have a join in the sqlReaderQuery so probably best to put a dummy (but real) table name in there.
Btw, not clear why you are using $$Text.Format and WindowStart/WindowEnd on your queries if you're not transposing these values into the query; you could just put the query between double quotes.