I want to filter out two types in the query expression
Json file :
[
{
"name": "name0",
"tags": {
"env": "dev"
},
"type": "Microsoft.OperationsManagement/solutions"
},
{
"name": "name1",
"tags": {
"env": "dev"
},
"type": "Microsoft.Web/sites"
},
{
"name": "name2",
"tags": {
"env": "dev"
},
"type": "Microsoft.Web/serverFarms"
},
{
"name": "name4",
"tags": null,
"type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks"
}
]
Expression
az resource list --resource-group MYRG --query '[? type != `"Microsoft.OperationsManagement/solutions"` && `"Microsoft.Network/privateDnsZones/virtualNetworkLinks"`]'
But result is including 2nd type in the result
[
{
"name": "name1",
"tags": {
"env": "dev"
},
"type": "Microsoft.Web/sites"
},
{
"name": "name2",
"tags": {
"env": "dev"
},
"type": "Microsoft.Web/serverFarms"
},
{
"name": "name4",
"tags": null,
"type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks"
}
]
I have tried other expression but no luck. Could someone tell me what exactly I am missing.
You can try adding another type != after &&:
az resource list --resource-group MYRG --query '[? type != `"Microsoft.OperationsManagement/solutions"` && type != `"Microsoft.Network/privateDnsZones/virtualNetworkLinks"`]'
You can also filter using slice to exclude "Microsoft.OperationsManagement/solutions" and "Microsoft.Network/privateDnsZones/virtualNetworkLinks":
az resource list --resource-group MYRG --query [1:3].type
You can refer to How to query Azure resources using the Azure CLI and JMESPATH queries
Related
Info
I have a terraform state file (json) with some deprecated attributes.
I would like to remove theses deprecated attributes.
I try to use jq and select() && del() but did not succeed to get back my full json without the deprecated attribue timeouts.
Problem
How to get my full json without the attribute timeouts for only one type of resources google_dns_record_set.
Data
{
"version": 4,
"terraform_version": "1.0.6",
"serial": 635,
"lineage": "6a9c2392-fdae-2b54-adcc-7366f262ffa4",
"outputs": {"test":"test1"},
"resources": [
{
"module": "module.resources",
"mode": "data",
"type": "google_client_config"
},
{
"module": "module.xxx.module.module1[\"cluster\"]",
"mode": "managed",
"type": "google_dns_record_set",
"name": "public_ip_ic_dns",
"provider": "module.xxx.provider[\"registry.terraform.io/hashicorp/google\"]",
"instances": [
{
"schema_version": 0,
"attributes": {
"id": "projects/xxx-xxx/managedZones/xxx--public/rrsets/*.net1.cluster.xxx--public.net.com./A",
"managed_zone": "xxx--public",
"name": "*.net1.cluster.xxx--public.net.com.",
"project": "xxx-xxx",
"rrdatas": [
"11.22.33.44"
],
"timeouts": null,
"ttl": 300,
"type": "A"
},
"sensitive_attributes": [],
"private": "xxx",
"dependencies": [
"xxx"
]
}
]
}
]
}
Command
jq -r '.resources[] | select(.type=="google_dns_record_set").instances[].attributes | del(.timeouts)' data.json
Pull the del command up front to include the whole selection as its own filter
del(.resources[] | select(.type=="google_dns_record_set").instances[].attributes.timeouts)
Demo
I am using aws ecs query to get list of properties being used by the current running task.
command -
cft = "aws ecs describe-tasks --cluster arn:aws:ecs:us-west-2:4984314772:cluster/secrets --tasks arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b
I am storing this in an output variable
output= $( eval $cft)
Output:
"tasks": [
{
"attachments": [
{
"id": "da8a1312-8278-46d5-8e3b-6b6a1d96f820",
"type": "ElasticNetworkInterface",
"status": "ATTACHED",
"details": [
{
"name": "subnetId",
"value": "subnet-0a151f2eb959ad4"
},
{
"name": "networkInterfaceId",
"value": "eni-081948e3666253f"
},
{
"name": "macAddress",
"value": "02:2a:9i:5c:4a:77"
},
{
"name": "privateDnsName",
"value": "ip-172-56-17-177.us-west-2.compute.internal"
},
{
"name": "privateIPv4Address",
"value": "172.56.17.177"
}
]
}
],
"availabilityZone": "us-west-2a",
"clusterArn": "arn:aws:ecs:us-west-2:4984314772:cluster/secrets",
"containers": [
{
"taskArn": "arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b",
"name": "nginx",
"image": "nginx",
"lastStatus": "PENDING",
"networkInterfaces": [
{
"attachmentId": "da8a1312-8278-46d5-6b6a1d96f820",
"privateIpv4Address": "172.31.17.176"
}
],
"healthStatus": "UNKNOWN",
"cpu": "0"
}
],
"cpu": "256",
"createdAt": "2020-12-10T18:00:16.320000+05:30",
"desiredStatus": "RUNNING",
"group": "family:nginx",
"healthStatus": "UNKNOWN",
"lastStatus": "PENDING",
"launchType": "FARGATE",
"memory": "512",
"overrides": {
"containerOverrides": [
{
"name": "nginx"
}
],
"inferenceAcceleratorOverrides": []
},
"platformVersion": "1.4.0",
"tags": [],
"taskArn": "arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b",
"taskDefinitionArn": "arn:aws:ecs:us-west-2:4984314772:task-definition/nginx:17",
"version": 2
}
],
"failures": []
}
now if do an echo of $output.tasks[0].containers[0] nothing happens it prints the entire thing again, i want to store the result in output variable and refer different parameter like we do in json format.
You will need to use a json parser such as jq and so:
eval $cft | jq '.tasks[].containers[]'
To avoid using eval you could simple pipe the aws command into jq and so:
aws ecs describe-tasks --cluster arn:aws:ecs:us-west-2:4984314772:cluster/secrets --tasks arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b | jq '.tasks[].containers[]'
or:
cft=$(aws ecs describe-tasks --cluster arn:aws:ecs:us-west-2:4984314772:cluster/secrets --tasks arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b | jq '.tasks[].containers[]')
echo $cft | jq '.tasks[].containers[]'
Using the command aws resourcegroupstaggingapi get-resources --profile (profile_name) returns an array of JSON object with the Resource ARNs value and their tags (another array of JSON of objects with tag's key and value.
Here's a anonymized example:
{
"ResourceTagMappingList": [
{
"ResourceARN": "arn:aws:acm:eu-west-1:123456789000:certificate/XXXXX-YYYY-8888-9999-CCCCCCCCCCCCC",
"Tags": [
{
"Key": "Environment",
"Value": "BAR"
}
]
},
{
"ResourceARN": "arn:aws:acm:eu-west-1:123456789000:certificate/XXXXX-YYYY-8888-9999-CCCCCCCCCCCCC",
"Tags": [
{
"Key": "Environment",
"Value": "FOO"
}
]
},
{
"ResourceARN": "arn:aws:ec2:eu-west-1:123456789000:elastic-ip/eipalloc-112345440809463",
"Tags": [
{
"Key": "Component",
"Value": "somethingCool"
},
{
"Key": "DeployID",
"Value": "di-01"
},
{
"Key": "Name",
"Value": "eip-nat-somethingCool-di-01"
}
]
},
{
"ResourceARN": "arn:aws:ec2:eu-west-1:123456789000:elastic-ip/eipalloc-19853410278439394i3",
"Tags": [
{
"Key": "Component",
"Value": "somethingCool"
},
{
"Key": "DeployID",
"Value": "bla-internal-goku"
},
{
"Key": "Name",
"Value": "eip-nat-somethingCool-bla-internal-goku"
}
]
},
{
"ResourceARN": "arn:aws:elasticloadbalancing:eu-west-1:123456789000:targetgroup/tf-20190624192221842800000004/oisajhiuweniçqej82u23948u3",
"Tags": [
{
"Key": "Component",
"Value": "somethingCool"
},
{
"Key": "DeployID",
"Value": "env01-bla00"
},
{
"Key": "Environment",
"Value": "env01"
},
{
"Key": "Estate",
"Value": "something"
},
{
"Key": "Name",
"Value": "target-group-lala-somethingCool-env01-bla00-vsquad"
}
]
}
]
}
So I would like to know if I can use the cli tool jq to filter the objects based on a specific tag Value? And if it is possible to list out all Value of a specific Tag Key?
Filter objects based on a specific tag value:
aws resourcegroupstaggingapi get-resources --region REGIONCODE | jq '.ResourceTagMappingList[] | select(.Tags[0].Value == "VALUETYPE")'
List all values of a specific tag key:
aws resourcegroupstaggingapi get-tag-values --key KEYNAME
You can query for all tag values for the specified key in the specified region for the AWS account, by calling the same API, but get-tag-values.
https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/API_GetTagValues.html
I'm trying to use jq to parse a JSON (the output of the OpenShift oc process ... command, actually), and add/update the env array of a container with a new key/value pair.
Sample input:
{
"kind": "List",
"apiVersion": "v1",
"metadata": {},
"items": [
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"annotations": {
"description": "Exposes and load balances the node.js application pods"
},
"name": "myapp-web"
},
"spec": {
"ports": [
{
"name": "web",
"port": 3000,
"protocol": "TCP",
"targetPort": 3000
}
],
"selector": {
"name": "myapp"
}
}
},
{
"apiVersion": "v1",
"kind": "Route",
"metadata": {
"name": "myapp-web"
},
"spec": {
"host": "app.internal.io",
"port": {
"targetPort": "web"
},
"to": {
"kind": "Service",
"name": "myapp-web"
}
}
},
{
"apiVersion": "v1",
"kind": "DeploymentConfig",
"metadata": {
"annotations": {
"description": "Defines how to deploy the application server"
},
"name": "myapp"
},
"spec": {
"replicas": 1,
"selector": {
"name": "myapp"
},
"strategy": {
"type": "Rolling"
},
"template": {
"metadata": {
"labels": {
"name": "myapp"
},
"name": "myapp"
},
"spec": {
"containers": [
{
"env": [
{
"name": "A_ENV",
"value": "a-value"
}
],
"image": "node",
"name": "myapp-node",
"ports": [
{
"containerPort": 3000,
"name": "app",
"protocol": "TCP"
}
]
}
]
}
},
"triggers": [
{
"type": "ConfigChange"
}
]
}
}
]
}
In this JSON, I want to do the following:
Find the DeploymentConfig object
Check if it has the env array in the first container
If it does, add a new object {"name": "B_ENV", "value": "b-value"} in it
If it does not, add the env array, with the object {"name": "B_ENV", "value": "b-value"} in it
So far, I'm able to tackle part of this, where I'm able to find the concerned object, and add the new env var to the container:
oc process -f <dc.yaml> -o json | jq '.items | map(if .kind == "DeploymentConfig"
then .spec.template.spec.containers[0].env |= .+ [{"name": "B_ENV", "value": "b-value"}]
else .
end)'
This is able to insert the new env var as expected, but the output is an array as shown below. Also, it doesn't handle the part where the env array may not be there at all.
I want to be able to produce the same output as the input, but with the new env var added.
Sample output:
[
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"annotations": {
"description": "Exposes and load balances the node.js application pods"
},
"name": "myapp-web"
},
"spec": {
"ports": [
{
"name": "web",
"port": 3000,
"protocol": "TCP",
"targetPort": 3000
}
],
"selector": {
"name": "myapp"
}
}
},
{
"apiVersion": "v1",
"kind": "Route",
"metadata": {
"name": "myapp-web"
},
"spec": {
"host": "app.internal.io",
"port": {
"targetPort": "web"
},
"to": {
"kind": "Service",
"name": "myapp-web"
}
}
},
{
"apiVersion": "v1",
"kind": "DeploymentConfig",
"metadata": {
"annotations": {
"description": "Defines how to deploy the application server"
},
"name": "myapp"
},
"spec": {
"replicas": 1,
"selector": {
"name": "myapp"
},
"strategy": {
"type": "Rolling"
},
"template": {
"metadata": {
"labels": {
"name": "myapp"
},
"name": "myapp"
},
"spec": {
"containers": [
{
"env": [
{
"name": "A_ENV",
"value": "a-value"
},
{
"name": "B_ENV",
"value": "b-value"
}
],
"image": "node",
"name": "myapp-node",
"ports": [
{
"containerPort": 3000,
"name": "app",
"protocol": "TCP"
}
]
}
]
}
},
"triggers": [
{
"type": "ConfigChange"
}
]
}
}
]
Is this doable, or is it too much to do with jq, and I should probably do this in python or node?
EDIT 1:
I just realized that conditional add/update of env array is already handled by the |= syntax!
So, I basically just need to be able to get the same structure back as the input, with the relevant env var added in the concerned array.
You pretty much had it, but you'll want to restructure your filter to preserve the full results. You want to make sure that none of the filters changes the context. By starting off with .items, you changed it from your root object to the items array. That in of itself is not so much a problem but what you do with that matters. Keep in mind that assignments/updates preserves the original context as a result with the change applied. So if you wrote your filter in terms of an update, it will work for you.
So to do this, you'll want to find the env array of the first container in the DeploymentConfig item. First let's find that:
.items[] | select(.kind == "DeploymentConfig").spec.template.spec.containers[0].env
You don't need to do anything else in terms of error handling since select will simply produce no result. From there, you just need to update the array by adding the new value.
(.items[] | select(.kind == "DeploymentConfig").spec.template.spec.containers[0].env) +=
[{name:"B_ENV",value:"b-value"}]
If the array exists, it will add the new item. If not, it will create a new env array. If env is not an array, that would be a different problem.
I am starting with elasticsearch now and i don't know anything about it.
I have folowing .JSON:
[
{
"label": "Admin Law",
"tags": [
"#admin"
],
"owner": "generalTopicTagText"
},
{
"label": "Judicial review",
"tags": [
"#JR"
],
"owner": "generalTopicTagText"
},
{
"label": "Admiralty/Shipping",
"tags": [
"#shipping"
],
"owner": "generalTopicTagText"
}
]
My mapping is this:
{
"topic_tax": {
"properties": {
"label": {
"type": "string",
"index": "not_analyzed"
},
"tags": {
"type": "string",
"index_name": "tag"
},
"owner": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
I need to put the first .Json into Elasticsearch, but it does not work.
All I know is that i am defining only 1 of this:
{
"label": "Judicial review",
"tags": [
"#JR"
],
"owner": "generalTopicTagText"
}
So when i try to put all of them with my elasticsearch.init, it will not work.
But I really don't know how to declare the mapping.Json to put the all .Json, it is like i need something like a for there.
You have to insert them json after json. But what you should do is use the bulk api of elasticsearch to insert multiple documents in one request. Check this api doc to see how it works
You can do something like this
curl -XPUT 'localhost:9000/es/post/1?version=2' -d '{
"text" : "your test message!"
}'
here is the documentation for index json with elasticsearch