I am trying to select the cluster name for a specific virtual machine using Ansible's json_query which uses JMESPATH in the backend.
I have followed the tutorial and examples at http://jmespath.org but am unable to construct the correct query.
My datastructure is as follows:
{
"datacenters": [
{
"name": "dc-a",
"clusters": [
{ "name": "cluster-a",
"hosts": [
{
"name": "host-a",
"vms": [
{
"name": "vm-a",
"summary": {
"mem": "8",
"diskGB": "78.00",
"annotation": "",
"state": "poweredOn",
"ostype": "Microsoft Windows Server 2008 R2 (64-bit)",
"cpu": "2"
}
},
{
"name": "vm-b",
"summary": {
"mem": "24",
"diskGB": "114.00",
"annotation": "",
"state": "poweredOn",
"ostype": "Microsoft Windows Server 2008 R2 (64-bit)",
"cpu": "4"
}
}]
}]
}]
}]
}
I am able to select the VM ('vm-a') using the query below but I am looking for the cluster (i.e. 'cluster-a').
datacenters[].clusters[].hosts[].vms[?name=='vm-a'].name[]
datacenters[].clusters[?contains(hosts[].vms[].name, 'vm-a')].name[]
Related
Using an AWS step function, I'm attempting to select only certain data from the results of a task. For some unfathomable reason, AWS have chosen not to allow Query in step functions, so I'm using ResultsSelector. However, I'm struggling with the JSONPath that is required.
How can I use the ResultsSelector to construct my desired JSON object?
Take this result -
{
"IsTruncated": false,
"KeyMarker": "",
"MaxKeys": 1000,
"Name": "some-bucket-name",
"Prefix": "some/prefix/",
"VersionIdMarker": "",
"Versions": [
{
"ETag": "\"02e9c20b7cd36fcf6e47926c26f0b39e\"",
"IsLatest": true,
"Key": "some/prefix/my.file",
"LastModified": "2021-09-30T15:34:59Z",
"Owner": {
"Id": "1fd170056d1480a7c1c9b43f5bf0603d91cbabc4ec77eefdcaa10218c3a920f6"
},
"Size": 69606,
"StorageClass": "STANDARD",
"VersionId": "y6XzRsCUZcXMPHqwwnhAGLwTlmPoj9dj"
},
{
"ETag": "\"01bc5b65afe6b0cc0722fc5da32a8a44\"",
"IsLatest": false,
"Key": "some/prefix/my.file",
"LastModified": "2021-09-30T15:34:21Z",
"Owner": {
"Id": "1fd170056d1480a7c1c9b43f5bf0603d91cbabc4ec77eefdcaa10218c3a920f6"
},
"Size": 69407,
"StorageClass": "STANDARD",
"VersionId": "jPdKeUqnYlf0_eNXzHaYCvDfdHLOvRX7"
}
]
}
What I'd like is to use the ResultsSelector to construct this JSON object -
{
"Objects": [
{
"Key": "some/prefix/my.file",
"VersionId": "y6XzRsCUZcXMPHqwwnhAGLwTlmPoj9dj"
},
{
"Key": "some/prefix/my.file",
"VersionId": "jPdKeUqnYlf0_eNXzHaYCvDfdHLOvRX7"
}
]
}
However, the closest I've been able to get so far is by using this -
{
"Key.$": "$.Versions[*].Key",
"VersionId.$": "$.Versions[*].VersionId"
}
Which gets me this -
{
"VersionId": [
"y6XzRsCUZcXMPHqwwnhAGLwTlmPoj9dj",
"jPdKeUqnYlf0_eNXzHaYCvDfdHLOvRX7",
],
"Key": [
"some/prefix/my.file",
"some/prefix/my.file"
]
}
The Jayway JsonPath implementation gives the output with the Key. You need to verify if AWS Step function supports the below jsonpath.
Tool : https://jsonpath.herokuapp.com/
$.Versions[*].['Key','VersionId']
If it works then you can do something like
"Objects.$": "$.Versions[*].['Key','VersionId']",
I need to extract only the servers whose status is online (in this case server2, server 4) from the below json. Can anyone help me with this? I need solution in jq with 1.5 version
[
{
"name": "core.vm",
"machines": [{
"name": "server1",
"max_load": 220,
"status": "offline"
},
{
"name": "server2",
"max_load": 220,
"status": "online"
}
]
},
{
"name": "cust.vm",
"machines": [{
"name": "server1",
"max_load": 20,
"status": "offline"
},
{
"name": "server2",
"max_load": 20,
"status": "online"
}
]
},
{
"name": "intl.vm",
"machines": [{
"name": "server3",
"max_load": 20,
"status": "offline"
},
{
"name": "server4",
"max_load": 20,
"status": "online"
}
]
}
]
You said you wanted server2 and server 4 for output (though you presumably meant server2 and server4), but there are two server2 that are online, and you didn't explain how you chose to return server2.
If you want to include servers if any instance of it is online,
jq -r '[ .[].machines[] | select(.status == "online") | .name ] | unique[]'
jqplay
If you want to include servers if all instance of it are online, that would require something more complex.
I'm have a requirement to build 3 different types of environment each having different numbers of instances to be launched using single CF template in JSON. For example: 1st ENV needs 2 instances, 2nd ENV needs 4 instances and 3rd ENV needs 8 instances. Please advice
im getting input from user to choose the ENV type and based on that the number of instances should be launched.
unable to mention instance count under EC2 properties and unable to use Fn::If to change the condition
You can use Mappings in conjunction with Parameters as following:
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "ALB and ASG",
"Mappings": {
"Environment": {
"i": {
"InstanceType": "t2.small",
"Count": 2
},
"s": {
"InstanceType": "t3.small",
"Count": 3
},
"p": {
"InstanceType": "m1.small",
"Count": 4
}
}
},
"Parameters": {
"ShortEnv": {
"AllowedValues": [
"i",
"p",
"s"
],
"ConstraintDescription": "must define the current environment",
"Description": "Environment type",
"Type": "String"
}
}
And then can use it as following in the relevant resource properties:
"InstanceType": {
"Fn::FindInMap": [
"Environment",
{
"Ref": "ShortEnv"
},
"InstanceType"
]
},
"DesiredCapacity": {
"Fn::FindInMap": [
"Environment",
{
"Ref": "ShortEnv"
},
"InstanceType"
]
}
I want to automate the cluster creation task in EMR. I have a json file
which contains configurations which need to be applied on new cluster and I want to write a shell script which automates this task for me.
Is it possible to create cluster in EMR by giving all the configurations from json file?
For example I have this file
{
"Cluster": {
"Ec2InstanceAttributes": {
"EmrManagedMasterSecurityGroup": "sg-00b10b71",
"RequestedEc2AvailabilityZones": [],
"AdditionalSlaveSecurityGroups": [],
"AdditionalMasterSecurityGroups": [],
"RequestedEc2SubnetIds": [
"subnet-02291b3e"
],
"Ec2SubnetId": "subnet-02291b3e",
"IamInstanceProfile": "EMR_EC2_DefaultRole",
"Ec2KeyName": "perf_key_pair",
"Ec2AvailabilityZone": "us-east-1e",
"EmrManagedSlaveSecurityGroup": "sg-f2b30983"
},
"Name": "NitinJ-Perf",
"ServiceRole": "EMR_DefaultRole",
"Tags": [
{
"Value": "Perf-Nitink",
"Key": "Qubole"
}
],
"Applications": [
{
"Version": "3.7.2",
"Name": "Ganglia"
},
{
"Version": "2.7.3",
"Name": "Hadoop"
},
{
"Version": "2.1.1",
"Name": "Hive"
},
{
"Version": "0.16.0",
"Name": "Pig"
},
{
"Version": "0.8.4",
"Name": "Tez"
}
],
"MasterPublicDnsName": "ec2-34-229-254-217.compute-1.amazonaws.com",
"ScaleDownBehavior": "TERMINATE_AT_INSTANCE_HOUR",
"InstanceGroups": [
{
"RequestedInstanceCount": 4,
"Status": {
"Timeline": {
"ReadyDateTime": 1499150835.979,
"CreationDateTime": 1499150533.99
},
"State": "RUNNING",
"StateChangeReason": {
"Message": ""
}
},
"Name": "Core Instance Group",
"InstanceGroupType": "CORE",
"EbsBlockDevices": [],
"ShrinkPolicy": {},
"Id": "ig-34P3CVF8ZL5CW",
"Configurations": [],
"InstanceType": "r3.4xlarge",
"Market": "ON_DEMAND",
"RunningInstanceCount": 4
},
{
"RequestedInstanceCount": 1,
"Status": {
"Timeline": {
"ReadyDateTime": 1499150804.591,
"CreationDateTime": 1499150533.99
},
"State": "RUNNING",
"StateChangeReason": {
"Message": ""
}
},
"Name": "Master Instance Group",
"InstanceGroupType": "MASTER",
"EbsBlockDevices": [],
"ShrinkPolicy": {},
"Id": "ig-3V7EHQ36187PY",
"Configurations": [],
"InstanceType": "r3.4xlarge",
"Market": "ON_DEMAND",
"RunningInstanceCount": 1
}
],
"Configurations": [
{
"Properties": {
"hive.vectorized.execution.enabled": "true"
},
"Classification": "hive-site"
}
]
}
}
Can I create a cluster on EMR by using some command like
aws emr create-cluster --cli-input-json file://'pwd'/emr_cluster_up.json
There is no such option through the AWS CLI as per the AWS CLI documentation. But if you want to automate the EMR cluster creation using a JSON file. You can use Cloud formation and automate the cluster creation.
Getting Started with AWS CloudFormation
I have created the ARM template for azure web app. I need to publish the ARM template to azure market place. I have used the azure publish portal https://publish.windowsazure.com/workspace/multi-resource-solutions for publishing the ARM template.
To on-board the ARM template to azure market place zip file must contain a mainTemplate.json and createUiDefinition.json. I found some samples for createUiDefination.json file in https://github.com/Azure/azure-quickstart-templates but all the createUiDefination.json is for VM. I am unable to find the samples or tutorials for createUiDefination.json for Azure web app.
I need to validate the azure web app site name is already exists or not. Also need to create or use the app service plan.
Is there is any tutorial or sample for creating createUiDefination.json for azure web app?
I need to validate the azure web app site name is already exists or
not.
This is not possible, you need to add to the site name a unique String, to ensure that the site name is globally unique.
For example you could use in your ARM template this function: uniqueString()
Similar question was answered by a Microsoft employee.
Also need to create or use the app service plan.
Add an App Service Plan to your Azure Resource Manager Template. For example like this:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"hostingPlanName": {
"type": "string",
"minLength": 1
},
"skuName": {
"type": "string",
"defaultValue": "F1",
"allowedValues": [
"F1",
"D1",
"B1",
"B2",
"B3",
"S1",
"S2",
"S3",
"P1",
"P2",
"P3",
"P4"
],
"metadata": {
"description": "Describes plan's pricing tier and capacity. Check details at https://azure.microsoft.com/en-us/pricing/details/app-service/"
}
},
"skuCapacity": {
"type": "int",
"defaultValue": 1,
"minValue": 1,
"metadata": {
"description": "Describes plan's instance count"
}
}
},
"variables": {
"webSiteName": "[concat('webSite', uniqueString(resourceGroup().id))]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('hostingPlanName')]",
"type": "Microsoft.Web/serverfarms",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "HostingPlan"
},
"sku": {
"name": "[parameters('skuName')]",
"capacity": "[parameters('skuCapacity')]"
},
"properties": {
"name": "[parameters('hostingPlanName')]"
}
},
{
"apiVersion": "2015-08-01",
"name": "[variables('webSiteName')]",
"type": "Microsoft.Web/sites",
"location": "[resourceGroup().location]",
"tags": {
"[concat('hidden-related:', resourceGroup().id, '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]": "Resource",
"displayName": "Website"
},
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]"
],
"properties": {
"name": "[variables('webSiteName')]",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('hostingPlanName'))]"
}
}
]
}
Validating an app service's name is possible in createUiDefinition.json.
The crux of it is the Microsoft.Solutions.ArmApiControl, which can be used to call ARM apis as part of the validation of a text box. Here is an example:
{
"$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
"handler": "Microsoft.Azure.CreateUIDef",
"version": "0.1.2-preview",
"parameters": {
"basics": [
{}
],
"steps": [
{
"name": "domain",
"label": "Domain Names",
"elements": [
{
"name": "domainInfo",
"type": "Microsoft.Common.InfoBox",
"visible": true,
"options": {
"icon": "Info",
"text": "Pick the domain name that you want to use for your app."
}
},
{
"name": "appServiceAvailabilityApi",
"type": "Microsoft.Solutions.ArmApiControl",
"request": {
"method": "POST",
"path": "[concat(subscription().id, '/providers/Microsoft.Web/checknameavailability?api-version=2021-02-01')]",
"body": "[parse(concat('{\"name\":\"', concat('', steps('domain').domainName), '\", \"type\": \"Microsoft.Web/sites\"}'))]"
}
},
{
"name": "domainName",
"type": "Microsoft.Common.TextBox",
"label": "Domain Name Word",
"toolTip": "The name of your app service",
"placeholder": "yourcompanyname",
"constraints": {
"validations": [
{
"regex": "^[a-zA-Z0-9]{4,30}$",
"message": "Alphanumeric, between 4 and 30 characters."
},
{
"isValid": "[not(equals(steps('domain').appServiceAvailabilityApi.nameAvailable, false))]",
"message": "[concat('Error with the url: ', steps('domain').domainName, '. Reason: ', steps('domain').appServiceAvailabilityApi.reason)]"
},
{
"isValid": "[greater(length(steps('domain').domainName), 4)]",
"message": "The unique domain suffix should be longer than 4 characters."
},
{
"isValid": "[less(length(steps('domain').domainName), 30)]",
"message": "The unique domain suffix should be shorter than 30 characters."
}
]
}
},
{
"name": "section1",
"type": "Microsoft.Common.Section",
"label": "URLs to be created:",
"elements": [
{
"name": "domainExamplePortal",
"type": "Microsoft.Common.TextBlock",
"visible": true,
"options": {
"text": "[concat('https://', steps('domain').domainName, '.azurewebsites.net - The main app service URL')]"
}
}
],
"visible": true
}
]
}
],
"outputs": {
"desiredDomainName": "[steps('domain').domainName]"
}
}
}
You can paste that in to the createUiDefinition.json sandbox azure provides to test it out.
To look at how to do an Azure App Service instead of a VM, look in to documentation like this and use it as the maintemplate.json in the app.zip package.