I am trying to update a boolean value in a json file using jq.
My json file is like this:
{
"kind": "KubeletConfiguration",
"apiVersion": "kubelet.config.k8s.io/v1beta1",
"address": "0.0.0.0",
"authentication": {
"anonymous": {
"enabled": true
},
"webhook": {
"cacheTTL": "2m0s",
"enabled": true
},
"x509": {
"clientCAFile": "/etc/kubernetes/pki/ca.crt"
}
},
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
....omitted
and I want to update the .authentication.anonymous.enabled value with the boolean false
I have tried the following:
jq --arg new false '.authentication.anonymous.enabled |= $new' config.json
This updates the value to false but it does so as a string, rather than a boolean. As below:
{
"kind": "KubeletConfiguration",
"apiVersion": "kubelet.config.k8s.io/v1beta1",
"address": "0.0.0.0",
"authentication": {
"anonymous": {
"enabled": "false"
},
"webhook": {
"cacheTTL": "2m0s",
"enabled": true
},
"x509": {
"clientCAFile": "/etc/kubernetes/pki/ca.crt"
}
},
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
....omitted
How do I get this to update as a boolean (no quotes around the value)?
Use --argjson for JSON parameters. Also, you only need the assignment operator = here as the evaluation of the RHS doesn't rely on the LHS context.
jq --argjson new false '.authentication.anonymous.enabled = $new' config.json
Demo
If you only wanted to toggle that field, you could do it without parameters and variables:
jq '.authentication.anonymous.enabled |= not' config.json
Demo
Related
My input is like below. I want to search for SearchString key (you can see that we can't use a fixed index for it) and when the key appears decrypt its value from base64 (perhaps using #base64d filter). Is this possible with JQ? If so, how?
[
{
"Name": "searchblock",
"Priority": 3,
"Statement": {
"RateBasedStatement": {
"Limit": 100,
"AggregateKeyType": "IP",
"ScopeDownStatement": {
"ByteMatchStatement": {
"SearchString": "Y2F0YWxvZ3NlYXJjaA==",
"FieldToMatch": {
"UriPath": {}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "LOWERCASE"
}
],
"PositionalConstraint": "CONTAINS"
}
}
}
},
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "searchblock"
}
},
{
"Name": "bot-block",
"Priority": 4,
"Statement": {
"ByteMatchStatement": {
"SearchString": "Ym90",
"FieldToMatch": {
"SingleHeader": {
"Name": "user-agent"
}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "LOWERCASE"
}
],
"PositionalConstraint": "CONTAINS"
}
},
"Action": {
"Allow": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "user-agent"
}
}
]
We use path, paths, getpath, and setpath built-ins for such operations when a fixed path is not available.
getpath(paths | select(.[-1] == "SearchString")) |= #base64d
Online demo
walk is quite intuitive for this kind of task:
walk(if type == "object" and .SearchString
then .SearchString |= #base64d else . end)
Using this approach, it's also trivial to modify the program to make it more robust, e.g. to check that .SearchString is a string:
walk(if type == "object" and (.SearchString|type) == "string"
then .SearchString |= #base64d else . end)
Note: if your jq does not include walk, you can simply copy its def from any reputable web site, or from https://github.com/stedolan/jq/blob/master/src/builtin.jq
I asked a question about getting a key and value from a JSON structure here: Using jq to get key and value from JSON
This is the JSON:
{
"63": {
"state": {
"on": false,
"alert": "select",
"mode": "automation",
"reachable": true
},
"swupdate": {
"state": "notupdatable",
"lastinstall": "2019-09-15T11:19:15"
},
"type": "plug",
"name": "Tree",
"modelid": "XXX",
"manufacturername": "XXX",
"productname": "plug",
"capabilities": {
"certified": false,
"control": {},
"streaming": {
"renderer": false,
"proxy": false
}
},
"config": {
"archetype": "plug",
"function": "functional",
"direction": "omnidirectional"
},
"uniqueid": "00:0d:6f:ff:fe:da:c9:dc-01",
"swversion": "2.0.022"
}
}
I want to emit only the parent key for an object based on the contents of .name: where .name == "Tree", return "63".
I can emit the whole object with:
jq -r '.[] | select(.name == "Tree")'
or a list of key and name with:
jq -r 'map_values(.name)'
You can format the JSON in a key/value format using to_entries() and get the key corresponding to a certain value
jq --raw-output 'to_entries[] | select(.value.name == "Tree").key'
I have to filter JSON by IsNew parameter (true or false)
Part of the JSON is below:
{
"data": [
{
"type": "users",
"attributes": {
"first-name": "student21",
"last-name": "student21",
"username": "student21",
"role": "student",
"IsNew": true
},
"relationships": {
"groups": {
"data": [
{
"type": "groups",
"id": "123f"
}
]
}
}
},
{
"type": "users",
"attributes": {
"first-name": "student23",
"last-name": "student23",
"email": "",
"avatar-url": null,
"username": "student23",
"role": "student",
"IsNew": false
},
"relationships": {
"groups": {
"data": [
{
"type": "groups",
"id": "456"
}
]
}
}
}
]
}
I've tried the following expressions:
$..data..[?(#.IsNew == true)].username,
$..data..[?(#.IsNew == 'true')].username,
$..data..[?(#.IsNew == "true")].username
All those expressions don't return any result.
I need to extract usernames for students with "IsNew" == true and "IsNew" == false separetely.
To extract usernames for students with "IsNew" == true, use a JSON Extractor with following settings:
JSON Path Expression :$.data..attributes[?(#.IsNew =~ /.*true/i)].username
Match No. -1 [to get all the matches for multiple students]
To extract usernames for students with "IsNew" == false, use a JSON Extractor with following settings:
JSON Path Expression :$.data..attributes[?(#.IsNew =~ /.*false/i)].username
Match No. -1 [to get all the matches for multiple students]
Use following variables to process further:
${FalseStudent_matchNr} or ${FalseStudent} if you have used Match No other than -1 in JSON Extractor
${TrueStudent_matchNr} or ${TrueStudent} if you have used Match No other than -1 in JSON Extractor
In a JSON array, I want to select an array element on basis of a node's value, then update a different node in the same array element. E.g. in the JSON below:
{
"apiVersion": "vlabs",
"properties": {
"orchestratorProfile": {
"orchestratorType": "Kubernetes",
"orchestratorRelease": "1.7",
"orchestratorVersion": "1.7.10",
"kubernetesConfig": {
"kubernetesImageBase": "gcrio.azureedge.net/google_containers/",
"clusterSubnet": "10.105.208.0/20",
"networkPolicy": "calico",
"nonMasqueradeCidr": "10.0.0.0/8",
"maxPods": 110,
"dockerBridgeSubnet": "172.17.0.1/16"
"addons": [
{
"name": "tiller",
"enabled": true
},
{
"name": "aci-connector",
"enabled": true
},
{
"name": "kubernetes-dashboard",
"enabled": true
},
{
"name": "rescheduler",
"enabled": true
}
]
}
}
}
}
I want to disable all addons which are not "rescheduler", i.e. set .enabled = false for elements of array .properties.orchestratorProfile.kubernetesConfig.addons[] where .name != "rescheduler". Closest I could work out was
jq -r '.properties.orchestratorProfile.kubernetesConfig.addons[] |
select (.name != "rescheduler" ) | .enabled = false'
but this, or any other ways I tried, I always lose the data outside of the array.
The expected outcome is:
{
"apiVersion": "vlabs",
"properties": {
"orchestratorProfile": {
"orchestratorType": "Kubernetes",
"orchestratorRelease": "1.7",
"orchestratorVersion": "1.7.10",
"kubernetesConfig": {
"kubernetesImageBase": "gcrio.azureedge.net/google_containers/",
"clusterSubnet": "10.105.208.0/20",
"networkPolicy": "calico",
"nonMasqueradeCidr": "10.0.0.0/8",
"maxPods": 110,
"dockerBridgeSubnet": "172.17.0.1/16"
"addons": [
{
"name": "tiller",
"enabled": false
},
{
"name": "aci-connector",
"enabled": false
},
{
"name": "kubernetes-dashboard",
"enabled": false
},
{
"name": "rescheduler",
"enabled": true
}
]
}
}
}
}
How do I go about doing this? Any idea or help or guidance is appreciated in advance.
Your jq query is spot-on except essentially for a missing pair of parentheses:
(.properties.orchestratorProfile.kubernetesConfig.addons[]
| select (.name != "rescheduler" ).enabled) = false
That is, on the LHS of the assignment, you need to specify the paths of the values that need to be updated.
jq solution:
jq '.properties.orchestratorProfile.kubernetesConfig.addons =
[.[] | if .name != "rescheduler" then .enabled = false else . end]' file
I have three JSON files:
vault.json:
{
"aws":
{
"access_key_id": "My-Key-id",
"secret_access_key": "My-Access-Key"
},
"ssl":
{
"crt": "My-Crt",
"key": "My-Key",
"req": "My-Req"
}
}
input.json:
{
".cloud_cpi_key": {
"type": "wildcard_domain",
"configurable": true,
"credential": false,
"value": "vault-supplied-value",
"optional": false
},
".cloud_cpi_secret": {
"type": "wildcard_domain",
"configurable": true,
"credential": false,
"value": "vault-supplied-value",
"optional": false
},
".properties.networking_point_of_entry": {
"type": "selector",
"configurable": true,
"credential": false,
"value": "external_ssl",
"optional": false
},
".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate": {
"type": "rsa_cert_credentials",
"configurable": true,
"credential": true,
"value": {
"private_key_pem": "vault-supplied-value",
"cert_pem": "vault-supplied-value"
},
"optional": false
}
}
keyfile.json
{
".cloud_cpi_key.value": "aws.access_key_id",
".cloud_cpi_secret": "secret_access_key",
".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate.value.private_key_pem": "ssl.key",
".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate.value.cert_pem": "ssl.crt"
}
I'd like to update the second json file, with the values from the first json, based on the third json.
can this be done via JQ to provide output.json?
output.json:
{
".cloud_cpi_key": {
"type": "string",
"configurable": true,
"credential": true,
"value": "My-Key-id",
"optional": false
},
".cloud_cpi_secret": {
"type": "string",
"configurable": true,
"credential": true,
"value": "My-Access-Key",
"optional": false
},
".properties.networking_point_of_entry": {
"type": "selector",
"configurable": true,
"credential": false,
"value": "external_ssl",
"optional": false
},
".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate": {
"type": "rsa_cert_credentials",
"configurable": true,
"credential": true,
"value": {
"private_key_pem": "My-Key",
"cert_pem": "My-Crt"
},
"optional": false
}
}
I can modify keyfile.json any way I like to make things easier, like
{
"fromkey": "aws.access_key_id"
"tokey": ".cloud_cpi_key.value"
},
{ "fromkey": ....
}
But no values may be placed in keyfile.json, only key names.
And I can modify vault.json, to put things in arrays, or what-have-you, but I cannot change the lowest levels, i.e. I cannot change:
{
"access_key_id": "My-Key-id",
"secret_access_key": "My-Access-Key"
}
I cannot modify input.json.
How can I accomplish this with JQ?
You don't give details about the transformation you have in mind, but if you can specify the algorithm, then rest assured, it can be done in jq.
Perhaps the thing you're missing is how to get jq to read the three files. One way to proceed would be to use the invocation:
jq --argfile keyfile keyfile.json --argfile vault vault.json -f vault.jq input.json
where vault.jq is the file containing your jq program, in which you would refer to the contents of keyfile.json as $keyfile, and similarly for the contents of vault.json
Simplifying keyfile.json
Since you indicate you have some flexibility about the format of keyfile.json, and since it appears to hold path information, I would recommend considering adopting path specifications that can be used directly with the jq builtins getpath and setpath.
For example, consider this format for keyfile.json:
[ [<path in input.json>], [<path in vault.json> ], ... ]
The first two entries corresponding to your example would thus be:
[
[ [".cloud_cpi_key","value"], ["aws","access_key_id"]],
[ [".cloud_cpi_secret"], ["aws", "secret_access_key"]]
]
To use the "vault" as the basis for updates, your jq program would then be the one-liner:
reduce $keyfile[] as $p (.; setpath(($p|.[0]); $vault|getpath($p|.[1])))