In a JSON array, I want to select an array element on basis of a node's value, then update a different node in the same array element. E.g. in the JSON below:
{
"apiVersion": "vlabs",
"properties": {
"orchestratorProfile": {
"orchestratorType": "Kubernetes",
"orchestratorRelease": "1.7",
"orchestratorVersion": "1.7.10",
"kubernetesConfig": {
"kubernetesImageBase": "gcrio.azureedge.net/google_containers/",
"clusterSubnet": "10.105.208.0/20",
"networkPolicy": "calico",
"nonMasqueradeCidr": "10.0.0.0/8",
"maxPods": 110,
"dockerBridgeSubnet": "172.17.0.1/16"
"addons": [
{
"name": "tiller",
"enabled": true
},
{
"name": "aci-connector",
"enabled": true
},
{
"name": "kubernetes-dashboard",
"enabled": true
},
{
"name": "rescheduler",
"enabled": true
}
]
}
}
}
}
I want to disable all addons which are not "rescheduler", i.e. set .enabled = false for elements of array .properties.orchestratorProfile.kubernetesConfig.addons[] where .name != "rescheduler". Closest I could work out was
jq -r '.properties.orchestratorProfile.kubernetesConfig.addons[] |
select (.name != "rescheduler" ) | .enabled = false'
but this, or any other ways I tried, I always lose the data outside of the array.
The expected outcome is:
{
"apiVersion": "vlabs",
"properties": {
"orchestratorProfile": {
"orchestratorType": "Kubernetes",
"orchestratorRelease": "1.7",
"orchestratorVersion": "1.7.10",
"kubernetesConfig": {
"kubernetesImageBase": "gcrio.azureedge.net/google_containers/",
"clusterSubnet": "10.105.208.0/20",
"networkPolicy": "calico",
"nonMasqueradeCidr": "10.0.0.0/8",
"maxPods": 110,
"dockerBridgeSubnet": "172.17.0.1/16"
"addons": [
{
"name": "tiller",
"enabled": false
},
{
"name": "aci-connector",
"enabled": false
},
{
"name": "kubernetes-dashboard",
"enabled": false
},
{
"name": "rescheduler",
"enabled": true
}
]
}
}
}
}
How do I go about doing this? Any idea or help or guidance is appreciated in advance.
Your jq query is spot-on except essentially for a missing pair of parentheses:
(.properties.orchestratorProfile.kubernetesConfig.addons[]
| select (.name != "rescheduler" ).enabled) = false
That is, on the LHS of the assignment, you need to specify the paths of the values that need to be updated.
jq solution:
jq '.properties.orchestratorProfile.kubernetesConfig.addons =
[.[] | if .name != "rescheduler" then .enabled = false else . end]' file
Related
I am trying to update a boolean value in a json file using jq.
My json file is like this:
{
"kind": "KubeletConfiguration",
"apiVersion": "kubelet.config.k8s.io/v1beta1",
"address": "0.0.0.0",
"authentication": {
"anonymous": {
"enabled": true
},
"webhook": {
"cacheTTL": "2m0s",
"enabled": true
},
"x509": {
"clientCAFile": "/etc/kubernetes/pki/ca.crt"
}
},
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
....omitted
and I want to update the .authentication.anonymous.enabled value with the boolean false
I have tried the following:
jq --arg new false '.authentication.anonymous.enabled |= $new' config.json
This updates the value to false but it does so as a string, rather than a boolean. As below:
{
"kind": "KubeletConfiguration",
"apiVersion": "kubelet.config.k8s.io/v1beta1",
"address": "0.0.0.0",
"authentication": {
"anonymous": {
"enabled": "false"
},
"webhook": {
"cacheTTL": "2m0s",
"enabled": true
},
"x509": {
"clientCAFile": "/etc/kubernetes/pki/ca.crt"
}
},
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
....omitted
How do I get this to update as a boolean (no quotes around the value)?
Use --argjson for JSON parameters. Also, you only need the assignment operator = here as the evaluation of the RHS doesn't rely on the LHS context.
jq --argjson new false '.authentication.anonymous.enabled = $new' config.json
Demo
If you only wanted to toggle that field, you could do it without parameters and variables:
jq '.authentication.anonymous.enabled |= not' config.json
Demo
I know this has to be simple, but for some reason it's eluding me how to find an element given a condition and modify one of its fields. The doc should be fully output (sed style) with the edit made.
{
"state": "wait",
"steps": {
"step1": [
{ "name":"Foo", "state":"wait" },
{ "name":"Bar", "state":"wait" }
],
"step2": [
{ "name":"Foo", "state":"wait" },
{ "name":"Zoinks", "state":"ready" }
],
"step3": [
{ "name":"Foo", "state":"cancel" }
]
}
}
I'm expecting something like this should be workable.
jq '. | (select(.steps[][].name=="Foo" and .steps[][].state=="wait") |= . + {.state:"Ready"}'
or
jq '. | (select(.steps[][]) | if (.name=="Foo" and .state=="wait") then (.state="Ready") else . end)
The desired output, of course, would be
{
"state": "wait",
"steps": {
"step1": [
{ "name":"Foo", "state":"ready" },
{ "name":"Bar", "state":"wait" }
],
"step2": [
{ "name":"Foo", "state":"ready" },
{ "name":"Zoinks", "state":"ready" }
],
"step3": [
{ "name":"Foo", "state":"cancel" }
]
}
}
Instead, when I'm not getting cryptic errors, I'm either modifying a top-level field in the document or modifying the field for all the elements or repeated the entire doc multiple times.
Any insights greatly appreciated.
Thanks.
p.s. is there a better syntax than [] to wildcard the named-elements under steps? Or after the pipe to identify the indices discovered by the select?
Pipe the output of .steps[][] into a select call that chooses the objects with the desired name and state values, then set the state value on the result.
$ jq '(.steps[][] | select(.name == "Foo" and .state == "wait")).state = "ready"' tmp.json
{
"state": "wait",
"steps": {
"step1": [
{
"name": "Foo",
"state": "ready"
},
{
"name": "Bar",
"state": "wait"
}
],
"step2": [
{
"name": "Foo",
"state": "ready"
},
{
"name": "Zoinks",
"state": "ready"
}
],
"step3": [
{
"name": "Foo",
"state": "cancel"
}
]
}
}
You can help confirm this using diff (the first jq just normalizes the formatting so that only the changes made by the second one show up in the diff):
$ diff <(jq . tmp.json) <(jq '...' tmp.json)
7c7
< "state": "wait"
---
> "state": "ready"
17c17
< "state": "wait"
---
> "state": "ready"
My input is like below. I want to search for SearchString key (you can see that we can't use a fixed index for it) and when the key appears decrypt its value from base64 (perhaps using #base64d filter). Is this possible with JQ? If so, how?
[
{
"Name": "searchblock",
"Priority": 3,
"Statement": {
"RateBasedStatement": {
"Limit": 100,
"AggregateKeyType": "IP",
"ScopeDownStatement": {
"ByteMatchStatement": {
"SearchString": "Y2F0YWxvZ3NlYXJjaA==",
"FieldToMatch": {
"UriPath": {}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "LOWERCASE"
}
],
"PositionalConstraint": "CONTAINS"
}
}
}
},
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "searchblock"
}
},
{
"Name": "bot-block",
"Priority": 4,
"Statement": {
"ByteMatchStatement": {
"SearchString": "Ym90",
"FieldToMatch": {
"SingleHeader": {
"Name": "user-agent"
}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "LOWERCASE"
}
],
"PositionalConstraint": "CONTAINS"
}
},
"Action": {
"Allow": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "user-agent"
}
}
]
We use path, paths, getpath, and setpath built-ins for such operations when a fixed path is not available.
getpath(paths | select(.[-1] == "SearchString")) |= #base64d
Online demo
walk is quite intuitive for this kind of task:
walk(if type == "object" and .SearchString
then .SearchString |= #base64d else . end)
Using this approach, it's also trivial to modify the program to make it more robust, e.g. to check that .SearchString is a string:
walk(if type == "object" and (.SearchString|type) == "string"
then .SearchString |= #base64d else . end)
Note: if your jq does not include walk, you can simply copy its def from any reputable web site, or from https://github.com/stedolan/jq/blob/master/src/builtin.jq
I'm failry new to jq and I've learned to do most things on my own but I'm hitting my head on my keyboard for this one. Look at the following json
JSON:
{
"importType": "Upsert",
"immediateDeployment": false,
"isIgnoreNulls": false,
"isOverwriteNullsOnly": false,
"isPreferredandSync": false,
"outputLayout": {
"fields": [
{
"name": "TEMP_KEY",
"type": "String",
"length": "2000",
"displayName": "TEMP_KEY"
},
{
"name": "LoadSeqNum",
"type": "String",
"length": "2000",
"displayName": "LoadSeqNum"
}
]
}
}
What I'm trying to do is inside of .outputLayout.fields[] I want to create a new pair called "isIdentifier" where it is true if the .outputLayout.fields[].name is LoadSeqNum and false if it's not but I need to keep the rest of the json just as it is. So target should look as following:
Goal:
{
"importType": "Upsert",
"immediateDeployment": false,
"isIgnoreNulls": false,
"isOverwriteNullsOnly": false,
"isPreferredandSync": false,
"outputLayout": {
"fields": [
{
"name": "TEMP_KEY",
"type": "String",
"length": "2000",
"displayName": "TEMP_KEY"
"isIdentifier": false
},
{
"name": "LoadSeqNum",
"type": "String",
"length": "2000",
"displayName": "LoadSeqNum"
"isIdentifier": true
}
]
}
}
I tried this:
jq '.outputLayout.fields[] | . + {"isIdentifier": (if (.name)=="LoadSeqNum" then true else false end)}'
But of course I'm missing all the higher level things. When I try to do:
.outputLayout.fields[].isIdentifier=(if (.outputLayout.fields[].name)=="LoadSeqNum" then true else false end)
I get the whole thing twice, once with both true and the other one with both false. I understand why it's doing that but I'm having a tough time figuring out what would work. Any help or point in right direction?
.outputLayout.fields[] |= (.isIdentifier = (.displayName == "LoadSeqNum") )
Or equivalently but perhaps a little less cryptically:
.outputLayout.fields |= map( .isIdentifier = (.displayName == "LoadSeqNum") )
I'm trying to write a JQ-filter for filtering specific resources from an AWS cloudformation template based on resource properties.
For example, when starting from the following (shortened) cloudformation template:
{
"Resources": {
"vpc001": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.1.0.0/16",
"InstanceTenancy": "default",
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true"
}
},
"ig001": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
"Tags": [
{
"Key": "Name",
"Value": "ig001"
}
]
}
}
}
}
I would like to construct a jq-filter enabling me to filter out specific resources based on (one or multiple) of their property fields.
For example:
when filtering for Type="AWS::EC2::InternetGateway" the result should be
{
"Resources": {
"ig001": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
"Tags": [
{
"Key": "Name",
"Value": "ig001"
}
]
}
}
}
}
An added bonus would be to be able to filter on a 'OR'-ed combination of values.
As such a filter for "AWS::EC2::InternetGateway" OR "AWS::EC2::VPC" should yield the original document.
Any suggestion or insight would be greatly appreciated.
Tx!
#hek2mgl's suggestion may be sufficient for your purposes, but it doesn't quite produce the answer you requested. Here's one very similar solution that does. It uses a generalization of jq's map() and map_values() filters that is often useful anyway:
def mapper(f):
if type == "array" then map(f)
elif type == "object" then
. as $in
| reduce keys[] as $key
({};
[$in[$key] | f ] as $value
| if $value | length == 0 then . else . + {($key): $value[0]}
end)
else .
end;
.Resources |= mapper(select(.Type=="AWS::EC2::VPC"))
Using your example input:
$ jq -f resources.jq resources.json
{
"Resources": {
"vpc001": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.1.0.0/16",
"InstanceTenancy": "default",
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true"
}
}
}
As #hek2mgl pointed out, it's now trivial to specify a more complex selection criterion.
}
Here is a solution which uses a separate function to select all resources matching a specified condition which is passed a {key,value} pair for each resource.
def condition:
.value.Type == "AWS::EC2::VPC"
;
{
Resources: .Resources | with_entries(select(condition))
}
Output from sample data:
{
"Resources": {
"vpc001": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.1.0.0/16",
"InstanceTenancy": "default",
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true"
}
}
}
}
Use the select() function:
jq '.Resources[]|select(.Type=="AWS::EC2::VPC")' aws.json
You can use or if you want to filter by multiple conditions, like this:
jq '.Resources[]|select(.Type=="AWS::EC2::VPC" or .Type=="foo")' aws.json
Use aws cli's --query parameter.
Completely eliminates the need for jq.
http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html#controlling-output-filter
I found one way to do this without defining a function:
jq '.Resources | to_entries[] | select(.value.Type == "AWS::EC2::InternetGateway")|[{key: .key, value: .value}]|from_entries' example.json
{
"ig001": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
"Tags": [
{
"Key": "Name",
"Value": "ig001"
}
]
}
}
}