I am writing a template in JSON, and i am trying to make one parameter dependent on another. For example, there is parameter "ImageOSType" with allowedValues in choosing OSType (linux, windows), and when linux is selected i would like in another parameter called "OperatingSystem" to allowedValues change accordingly (when linux then show ubuntu and redhat OS, when windows show windows servers).
I tried to do this with "if" statement on many different methods but did not work, the closest i am is with "if" statement and "effect": "modify", but it does not change the allowedValues and I don't know why.
Below is my code
{
"$schema":"https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion":"1.0.0.0",
"parameters":{
"ImageOSType":{
"type":"string",
"allowedValues":[
"windows",
"linux"
],
"metadata":{
"description":"Select the OS type of image"
}
},
"operatingSystem":{
"type":"String",
"allowedValues":[
"CIS-Red Hat Enterprise Linux 7-L1",
"CIS-Red Hat Enterprise Linux 8-L1",
"CIS-Ubuntu18.04-L1",
"CIS-Ubuntu20.04-L1"
],
"if": {
"field": "[parameters('ImageOSType')]",
"notLike": "linux"
},
"then": {
"effect": "modify",
"details": {
"operations": [{
"operation": "add",
"field": "[parameters('ImageOSType').allowedValues()]",
"value": "CISWindowsServer2016-L1"
}]
}
},
"metadata":{
"description":"Name of the gallery image definition OS, choose from available"
}
},
},
variables{}
}
Related
I was searching a way to run ecs task. I already have a cluster and task definition settings. I just wanted to trigger a task using CloudFormation template. I know that I can run a task by clicking on the console and it works fine. For cfn, approach needs to be define properly.
Check the attached screenshots. I wanted to run that task using CloudFormation and pass container override environment variables. As per my current templates, it is not allowing me to do same like I can do using console. Using console I just need to select the following options
1. Launch type
2. Task Definition
Family
Revision
3. VPC and security groups
4. Environment variable overrides rest of the things automatically selected
It starts working with console but with cloudformaton template how can we do that. Is it possible to do or there is no such feature?
"taskdefinition": {
"Type" : "AWS::ECS::TaskDefinition",
"DependsOn": "DatabaseMaster",
"Properties" : {
"ContainerDefinitions" : [{
"Environment" : [
{
"Name" : "TARGET_DATABASE",
"Value" : {"Ref":"DBName"}
},
{
"Name" : "TARGET_HOST",
"Value" : {"Fn::GetAtt": ["DatabaseMaster", "Endpoint.Address"]}
}
]
}],
"ExecutionRoleArn" : "arn:aws:iam::xxxxxxxxxx:role/ecsTaskExecutionRole",
"Family" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"TaskRoleArn" : "arn:aws:iam::xxxxxxxxxxxxxxx:role/xxxxxxxxxxxxxxx-XXXXXXXXX"
}
},
"EcsService": {
"Type" : "AWS::ECS::Service",
"Properties" : {
"Cluster" : "xxxxxxxxxxxxxxxxx",
"LaunchType" : "FARGATE",
"NetworkConfiguration" : {
"AwsvpcConfiguration" : {
"SecurityGroups" : ["sg-xxxxxxxxxxx"],
"Subnets" : ["subnet-xxxxxxxxxxxxxx"]
}
},
"TaskDefinition" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
}
There is no validity error in the code however, I am talking about the approach. I added image name container name but now it is asking for memory and cpu, it should not ask as it is already defined we just need to run a task.
Edited
I wanted to run a task after creation of my database and wanted to pass those database values to the task to run and complete a job.
Here is the working example of what you can do if you wanted to pass variable and run a task. In my case, I wanted to run a task after creation of my database but with environment variables, directly AWS does not provide any feature to do so, this is the solution which can help to trigger you ecs task.
"IAMRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Description": "Allow CloudWatch Events to trigger ECS task",
"Policies": [
{
"PolicyName": "Allow-ECS-Access",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecs:*",
"iam:PassRole",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
}
],
"RoleName": { "Fn::Join": [ "", ["CloudWatchTriggerECSRole-", { "Ref": "DBInstanceIdentifier" }]]}
}
},
"DummyParameter": {
"Type" : "AWS::SSM::Parameter",
"Properties" : {
"Name" : {"Fn::Sub": "${AWS::StackName}-${DatabaseMaster}-EndpointAddress"},
"Type" : "String",
"Value" : {"Fn::GetAtt": "DatabaseMaster.Endpoint.Address"}
},
"DependsOn": "TaskSchedule"
},
"TaskSchedule": {
"Type": "AWS::Events::Rule",
"Properties": {
"Description": "Trigger ECS task upon creation of DB instance",
"Name": { "Fn::Join": [ "", ["ECSTaskTrigger-", { "Ref": "DBName" }]]},
"RoleArn": {"Fn::GetAtt": "IAMRole.Arn"},
"EventPattern": {
"source": [ "aws.ssm" ],
"detail-type": ["Parameter Store Change"] ,
"resources": [{"Fn::Sub":"arn:aws:ssm:eu-west-1:XXXXXXX:parameter/${AWS::StackName}-${DatabaseMaster}-EndpointAddress"}],
"detail": {
"operation": ["Create"],
"name": [{"Fn::Sub": "${AWS::StackName}-${DatabaseMaster}-EndpointAddress"}],
"type": ["String"]
}
},
"State": "ENABLED",
"Targets": [
{
"Arn": "arn:aws:ecs:eu-west-1:xxxxxxxx:cluster/NameOf-demo",
"Id": "NameOf-demo",
"RoleArn": {"Fn::GetAtt": "IAMRole.Arn"},
"EcsParameters": {
"LaunchType": "FARGATE",
"NetworkConfiguration": {
"AwsVpcConfiguration": {
"SecurityGroups": {"Ref":"VPCSecurityGroups"},
"Subnets": {"Ref":"DBSubnetName"}
}
},
"PlatformVersion": "LATEST",
"TaskDefinitionArn": "arn:aws:ecs:eu-west-1:XXXXXXXX:task-definition/NameXXXXXXXXX:1"
},
"Input": {"Fn::Sub": [
"{\"containerOverrides\":[{\"name\":\"MyContainerName\",\"environment\":[{\"name\":\"VAR1\",\"value\":\"${TargetDatabase}\"},{\"name\":\"VAR2\",\"value\":\"${TargetHost}\"},{\"name\":\"VAR3\",\"value\":\"${TargetHostPassword}\"},{\"name\":\"VAR4\",\"value\":\"${TargetPort}\"},{\"name\":\"VAR5\",\"value\":\"${TargetUser}\"},{\"name\":\"VAR6\",\"value\":\"${TargetLocation}\"},{\"name\":\"VAR7\",\"value\":\"${TargetRegion}\"}]}]}",
{
"VAR1": {"Ref":"DBName"},
"VAR2": {"Fn::GetAtt": ["DatabaseMaster", "Endpoint.Address"]},
"VAR3": {"Ref":"DBPassword"},
"VAR4": "5432",
"VAR5": {"Ref":"DBUser"},
"VAR6": "value6",
"VAR7": "eu-west-2"
}
]}
}
]
}
}
For Fargate task, we need to specify in CPU in Task Definition. and memory or memory reservation in either task or container definition.
and environment variables should be passed to each container as ContainerDefinitions and overrided when task is run from ecs task-run from console or cli.
{
"ContainerTaskdefinition": {
"Type": "AWS::ECS::TaskDefinition",
"Properties": {
"Family": "SomeFamily",
"ExecutionRoleArn": !Ref RoleArn,
"TaskRoleArn": !Ref TaskRoleArn,
"Cpu": "256",
"Memory": "1GB",
"NetworkMode": "awsvpc",
"RequiresCompatibilities": [
"EC2",
"FARGATE"
],
"ContainerDefinitions": [
{
"Name": "container name",
"Cpu": 256,
"Essential": "true",
"Image": !Ref EcsImage,
"Memory": "1024",
"LogConfiguration": {
"LogDriver": "awslogs",
"Options": {
"awslogs-group": null,
"awslogs-region": null,
"awslogs-stream-prefix": "ecs"
}
},
"Environment": [
{
"Name": "ENV_ONE_KEY",
"Value": "Valu1"
},
{
"Name": "ENV_TWO_KEY",
"Value": "Valu2"
}
]
}
]
}
}
}
EDIT(from discussion in comments):
ECS Task Run is not a cloud-formation resource, it can only be run from console or CLI.
But if we choose to run from a cloudformation resource, it can be done using cloudformation custom resource. But once task ends, we now have a resource in cloudformation without an actual resource behind. So, custom resource needs to do:
on create: run the task.
on delete: do nothing.
on update: re-run the task
Force an update by changing an attribute or logical id, every time we need to run the task.
I have noticed that upon querying Orion CB, while it is working with provisioned devices and having IoT Agent receive HTTP and MQTT messages, it will always output all the values written in the quotation marks:
{
"id": "sensor_data",
"type": "Sensor",
"ActiveTime": {
"type": "Seconds",
"value": "17703",
"metadata": {
"TimeInstant": {
"type": "ISO8601",
"value": "2018-07-04T13:32:27.357Z"
}
}
},
"Distance": {
"type": "Number",
"value": "312",
"metadata": {
"TimeInstant": {
"type": "ISO8601",
"value": "2018-07-04T13:32:27.413Z"
}
}
}
}
However, if to work with only entities in Orion CB, it is possible to receive actual values (like in the example in the manual):
{
"id": "Room1",
"pressure": {
"metadata": {},
"type": "Integer",
"value": 720
},
"temperature": {
"metadata": {},
"type": "Float",
"value": 23
},
"type": "Room"
}
Sometimes, I need to receive the actual value from my sensor in order to format it and use in further applications, but they are in quotation marks, which makes it a little difficult.
Is it possible to somehow change?(maybe in device provisioning), or it really should be that way regarding devices?
Thanks in advance!
EDIT 1
This is the way I provisioned the device:
{
"devices": [
{
"device_id": "sensor_data",
"entity_name": "sensor_data",
"entity_type": "Sensor",
"transport": "MQTT",
"timezone": "Europe/Helsinki",
"attributes": [
{ "object_id": "act", "name": "ActiveTime", "type": "Seconds"},
{ "object_id": "dst", "name": "Distance", "type": "Number"}
]
}
]
}
And this is how the MQTT messages are sent from my sensor (I have set up the topics for IoT Agent to understand them)
/123456789/sensor_data/attrs/act 12
/123456789/sensor_data/attrs/dst 322
123456789 is the API Key I have set here.
This situation tipycally happens when IoT Agents uses NGSIv1 to push data to Context Broker, given that NGSIv1 always "string-fy" any attribute value. Recently, the ability to use NGSIv2 (which doesn't have this limitatino) was introduced in IoT Agents.
In order to solve your problem you have to:
Use a recent IOTA-UL version (the current one from master branch will work)
Enable NGSIv2 in configuration as explained in documentation. This is done in the config.js file:
config.iota = {
...
contextBroker: {
...
ngsiVersion: 'v2'
}
...
}
or using environament variable IOTA_CB_NGSI_VERSION=v2 for the IOTA-UL process.
Enable autocast as explained in documentation. This is done in config.js file:
config.iota = {
...
autocast: true,
...
}
or using environament variable IOTA_AUTOCAST=true for the IOTA-UL process.
Set the right type for each attribute at provision time. The documentation here) provides the right types:
Type "Number" for integer or float numbers
Type "Boolean" for boolean
Type "None" for null
Thus, in your case the provisioning for Distance is ok, but for ActiveTime you should use also Number as type.
is it possible to check, in an ARM Template, if the name for my Virtual Machine already exists?
I am developing a Solution Template for the Azure Marketplace. Maybe it is possible to set a paramter in the UiDefinition uniqe?
The goal is to reproduce this green Hook
A couple notes...
VM Names only need to be unique within a resourceGroup, not within the subscription
Solution Templates must be deployed to empty resourceGroups, so collisions with existing resources aren't possible
For solution templates the preference is that you simply name the VMs for the user, rather than asking - use something that is appropriate for the workload (e.g. jumpbox) - not all solutions do this but we're trying to improve that experience
Given that it's not likely we'll ever build a control that checks for naming collisions on resources without globally unique constraints.
That help?
This looks impossible, according to the documentation.
There are no validation scenarious.
I assume that you should be using the Microsoft.Common.TextBox UI element in your createUiDefinition.json.
I have tried to reproduce a green check by creating a simple createUiDefinition.json as below with a Microsoft.Common.TextBox UI element as shown below.
{
"$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json",
"handler": "Microsoft.Compute.MultiVm",
"version": "0.1.2-preview",
"parameters": {
"basics": [
{
"name": "textBoxA",
"type": "Microsoft.Common.TextBox",
"label": "VM Name",
"defaultValue": "",
"toolTip": "Please enter a VM name",
"constraints": {
"required": true
},
"visible": true
}
],
"steps": [],
"outputs": {}
}
}
I am able to reproduce the green check beside the VM Name textbox as shown below:
However, this green check DOES NOT imply the VM Name is Available.
This is because based on my testing, even if I use an existing VM Name in the same subscription, it is still showing the green check.
Based on the official documented constraints that are supported by the Microsoft.Common.TextBox UI element, it DOES NOT VALIDATE Name Availability.
Hope this helps!
While bmoore's point is correct that it's unlikely you would ever need this for a VM (nor is there an API for it), there are other compute resources that do have global naming requirements.
As of 2022 this concept is possible now with the use of the ArmApiControl UI element. It allows you to call ARM apis as part of validation in the createUiDefinition.json. Here is an example using the check name API for an Azure App service.
{
"$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
"handler": "Microsoft.Azure.CreateUIDef",
"version": "0.1.2-preview",
"parameters": {
"basics": [
{}
],
"steps": [
{
"name": "domain",
"label": "Domain Names",
"elements": [
{
"name": "domainInfo",
"type": "Microsoft.Common.InfoBox",
"visible": true,
"options": {
"icon": "Info",
"text": "Pick the domain name that you want to use for your app."
}
},
{
"name": "appServiceAvailabilityApi",
"type": "Microsoft.Solutions.ArmApiControl",
"request": {
"method": "POST",
"path": "[concat(subscription().id, '/providers/Microsoft.Web/checknameavailability?api-version=2021-02-01')]",
"body": "[parse(concat('{\"name\":\"', concat('', steps('domain').domainName), '\", \"type\": \"Microsoft.Web/sites\"}'))]"
}
},
{
"name": "domainName",
"type": "Microsoft.Common.TextBox",
"label": "Domain Name Word",
"toolTip": "The name of your app service",
"placeholder": "yourcompanyname",
"constraints": {
"validations": [
{
"regex": "^[a-zA-Z0-9]{4,30}$",
"message": "Alphanumeric, between 4 and 30 characters."
},
{
"isValid": "[not(equals(steps('domain').appServiceAvailabilityApi.nameAvailable, false))]",
"message": "[concat('Error with the url: ', steps('domain').domainName, '. Reason: ', steps('domain').appServiceAvailabilityApi.reason)]"
},
{
"isValid": "[greater(length(steps('domain').domainName), 4)]",
"message": "The unique domain suffix should be longer than 4 characters."
},
{
"isValid": "[less(length(steps('domain').domainName), 30)]",
"message": "The unique domain suffix should be shorter than 30 characters."
}
]
}
},
{
"name": "section1",
"type": "Microsoft.Common.Section",
"label": "URLs to be created:",
"elements": [
{
"name": "domainExamplePortal",
"type": "Microsoft.Common.TextBlock",
"visible": true,
"options": {
"text": "[concat('https://', steps('domain').domainName, '.azurewebsites.net - The main app service URL')]"
}
}
],
"visible": true
}
]
}
],
"outputs": {
"desiredDomainName": "[steps('domain').domainName]"
}
}
}
You can copy the above code and test it in the createUiDefinition.json sandbox azure provides.
We are upgrading our Data pipeline version from 3.3.2 to 5.8, so those bootstrap actions on old AMI release have changed to be setup using configuration and specifying them under classification / property definition.
So my Json looks like below
{
"enableDebugging": "true",
"taskInstanceBidPrice": "1",
"terminateAfter": "2 Hours",
"name": "ExportCluster",
"taskInstanceType": "m1.xlarge",
"schedule": {
"ref": "Default"
},
"emrLogUri": "s3://emr-script-logs/",
"coreInstanceType": "m1.xlarge",
"coreInstanceCount": "1",
"taskInstanceCount": "4",
"masterInstanceType": "m3.xlarge",
"keyPair": "XXXX",
"applications": ["hadoop","hive", "tez"],
"subnetId": "XXXXX",
"logUri": "s3://pipelinedata/XXX",
"releaseLabel": "emr-5.8.0",
"type": "EmrCluster",
"id": "EmrClusterWithNewEMRVersion",
"configuration": [
{ "ref": "configureEmrHiveSite" }
]
},
{
"myComment": "This object configures hive-site xml.",
"name": "HiveSite Configuration",
"type": "HiveSiteConfiguration",
"id": "configureEmrHiveSite",
"classification": "hive-site",
"property": [
{"ref": "hive-exec-compress-output" }
]
},
{
"myComment": "This object sets a hive-site configuration
property value.",
"name":"hive-exec-compress-output",
"type": "Property",
"id": "hive-exec-compress-output",
"key": "hive.exec.compress.output",
"value": "true"
}
],
"parameters": []
With the above Json file it gets loaded into Data Pipeline but throws an error saying
Object:HiveSite Configuration
ERROR: 'HiveSiteConfiguration'
Object:ExportCluster
ERROR: 'configuration' values must be of type 'null'. Found values of type 'null'
I am not sure what this really means and could you please let me know if i am specifying this correctly which i think i am according to http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html
The below block should have the name as "EMR Configuration" only then its recognized correctly by the AWS Data pipeline and the Hive-site.xml is being set accordingly.
{
"myComment": "This object configures hive-site xml.",
"name": "EMR Configuration",
"type": "EmrConfiguration",
"id": "configureEmrHiveSite",
"classification": "hive-site",
"property": [
{"ref": "hive-exec-compress-output" }
]
},
I'm trying to conditionally provide resource property values through translation of runtime resource properties within a copyIndex loop..
Upon deploying the following ARM template, I receive the error:
Unable to process template language expressions for resource '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachines/{vm-name}/extensions/Microsoft.EnterpriseCloud.Monitoring' at line '30' and column '10'. 'The template resource '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachines/{vm-name}' is not found.' (Code: InvalidTemplate)
"type": "[variables('extensionType')[reference(concat('Microsoft.Compute/virtualMachines/', parameters('virtualMachines')[copyIndex()].name)).storageProfile.osDisk.osType]]",
However, the VM exists with the ID it provides, so it doesn't make sense that the engine cannot find it. If I hard-code the Extension Type, there are no errors and the Extension is installed on the VM with the same ID.
Unfortunately, I don't know if this is a bug within ARM or if I'm just doing something wrong..
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceResourceId": { "type": "string" },
"virtualMachines": { "type": "array" }
},
"variables": {
"extensionType": {
"Windows": "MicrosoftMonitoringAgent",
"Linux": "OmsAgentForLinux"
}
},
"resources": [
{
"copy": {
"name": "VMMonitoringExtensionsCopy",
"count": "[length(parameters('virtualMachines'))]"
},
"type": "Microsoft.Compute/virtualMachines/extensions",
"apiVersion": "2015-05-01-preview",
"location": "[parameters('virtualMachines')[copyIndex()].location]",
"name": "[concat(parameters('virtualMachines')[copyIndex()].name, '/Microsoft.EnterpriseCloud.Monitoring')]",
"properties": {
"publisher": "Microsoft.EnterpriseCloud.Monitoring",
"type": "[variables('extensionType')[reference(concat('Microsoft.Compute/virtualMachines/', parameters('virtualMachines')[copyIndex()].name)).storageProfile.osDisk.osType]]",
"typeHandlerVersion": "1.0",
"autoUpgradeMinorVersion": true,
"settings": {
"workspaceId": "[reference(parameters('workspaceResourceId'), '2015-11-01-preview').customerId]"
},
"protectedSettings": {
"workspaceKey": "[listKeys(parameters('workspaceResourceId'), '2015-11-01-preview').primarySharedKey]"
}
}
}
]
}
The object array being passed in for virtualMachines looks like this:
[
{ "name": "vm-name", "location": "azure-region" }
]
A couple things you can try:
1) Assuming the VM is not defined in the same template try using the "full" resourceId in the reference function. See the last example in this doc:
https://azure.microsoft.com/en-us/documentation/articles/resource-group-template-functions/#reference - it seems like the error already knows the full resourceId, but it's worth trying
2) the other thought is that the reference function is evaluated at runtime and the resource provider doesn't like the expression but that's a swag.
I will do some more poking and see if we can't nail this down.