I have an irregular json, e.g.:
[
{ "key": 123 },
[
{ "key": null },
{ "not key": "blah-blah" }
]
]
What's the most efficient way to add a placeholder record ("new key": null) into all objects which have "key" record? this is an expected output:
[
{ "key": 123, "new key": null },
[
{ "key": null, "new key": null },
{ "not key": "blah-blah" }
]
]
Using jq 1.6 and its walk function to modify JSON objects recursively :
jq 'walk(if type == "object" and has("key") then . + { "new key" : null } else . end)'
You can try it here.
The following will work on jq 1.5 :
jq 'def addNewKey: map_values(if type == "object" and has("key") then . + { "new key" : null } elif type=="array" or type=="object" then addNewKey else . end); addNewKey'
You can try it here.
Related
I have some json that I want to add to based on a walk. A simple example of the json I have is below:
{
"valueSource": "memory",
"dataType": "Boolean",
"alarms": [
{
"setpointA": 1.0,
"name": "Alarm",
"priority": "Diagnostic",
"ackMode": "Auto",
}
],
"name": "Test Alarm",
"value": false,
"tagType": "AtomicTag"
}
I want to add to each object in the "alarms" key's array the following key:
{
"bindType": "Tag",
"value": "[.]<parent.name>.Name"
}
where <parent.name> is "Test Alarm" in this example which is the parent-of-the-alarm-array-item's "name" key.
I've gotten this jq filter so far that adds the object, but the value key value is wrong (it's getting the alarm array item's name instead of its parent's name):
walk( if type == "object" and .setpointA then .label = {"bindType":"Tag", "value": "[.]\(.name).Name"} else . end)
Essentially I want this:
{
"valueSource": "memory",
"dataType": "Boolean",
"alarms": [
{
"setpointA": 1.0,
"name": "Alarm",
"priority": "Diagnostic",
"ackMode": "Auto",
"label": {
"bindType": "Tag",
"value": "[.]Test Alarm.Name"
}
}
],
"name": "Test Alarm",
"value": false,
"tagType": "AtomicTag"
}
Here is my jqplay below. It has the final result in the JSON section, where the Result should match this but doesn't at the moment.
https://jqplay.org/s/-qHFIWolrD
Follow up question:
How would I add this displayPath key as well?
You cannot reference to parent, you have to save the reference in a variable beforehand, and descend with having access to that variable.
.tags[] |= (
.name as $name
| # rest of your code using $name
walk(
if type == "object" and .setpointA
then .label = {"bindType":"Tag", "value": "[.]\($name).Name"}
else . end
)
)
Demo
As you happen to know that the objects are located in the .alarms array, you could also just iterate over the items, select only those matching the condition and then assign to their .label whatever you want (including $name)
.tags[] |= (
.name as $name
| (.alarms[] | select(has("setpointA"))).label = {
bindType: "Tag", value: "[.]\($name).Name"
}
)
Demo
Edit responding to OP's follow-up question
I actually don't care about the setpointA key, I just want to add the label key (as well as another displayPath key) to each item in all alarms arrays.
.tags[] |= (.name as $name | .alarms[] += (
{bindType: "Tag", value: "[.]\($name)."} | {
label: (.value += "Name"),
displayPath: (.value += "Documentation")
}
))
Demo
As jq does not allow you to refer to parent object, you need to work on parent level :
jq 'walk(if type == "object" and .alarms and ( .alarms | arrays )
then (.alarms[] | select(.setpointA)).label =
{ bindType: "Tag", value: "[.]\(.name).Name"}
else . end
)' data.json
I need to remove elements from a json file based on certain key values. Here is the file I am trying to process.
{
"element1": "Test Element 1",
"element2": {
"tags": "internal",
"data": {
"data1": "Test Data 1",
"data2": "Test Data 2"
}
},
"element3": {
"function1": {
"tags": [
"new",
"internal"
]
},
"data3": "Test Data 3",
"data4": "Test Data 4"
},
"element4": {
"function2": {
"tags": "new"
},
"data5": "Test Data 5"
}
}
I want to remove all elements that have a "tag" with value "internal". So the result should look like this:
{
"element1": "Test Element 1",
"element4": {
"function2": {
"tags": "new"
},
"data5": "Test Data 5"
}
}
I tried various approaches but I just don't get it done using jq. Any ideas? Thanks.
Just to add some more complexity. Let's assume the json is:
{
"element1": "Test Element 1",
"element2": {
"tags": "internal",
"data": {
"data1": "Test Data 1",
"data2": "Test Data 2"
}
},
"element3": {
"function1": {
"tags": [
"new",
"internal"
]
},
"data3": "Test Data 3",
"data4": "Test Data 4"
},
"element4": {
"function2": {
"tags": "new"
},
"data5": "Test Data 5"
},
"structure1" : {
"substructure1": {
"element5": "Test Element 5",
"element6": {
"tags": "internal",
"data6": "Test Data 6"
}
}
}
}
and I want to get
{
"element1": "Test Element 1",
"element4": {
"function2": {
"tags": "new"
},
"data5": "Test Data 5"
},
"structure1" : {
"substructure1": {
"element5": "Test Element 5",
}
}
}
Not easy, finding elements which have a tags key somewhere whose value is either the string internal, or an array of which an element is the string internal in a reliable way is only possible with a complex boolean expression as below.
Once found, deleting them can be done using the del built-in.
del(.[] | first(select(recurse
| objects
| has("tags") and (.tags
| . == "internal" or (
type == "array" and index("internal")
)
)
)))
Online demo
I think I figured out how to also solve the more complex case. I am now running:
walk(if type == "object" and has("tags") and (.tags | . == "internal" or (type == "array" and index("internal"))) then del(.) else . end) | delpaths([paths as $path | select(getpath($path) == null) | $path])
This will remove all elements that contain 'internal' as 'tag'.
The following solution is written with a helper function for clarity. The helper function uses any for efficiency and is defined so as to add a dash of generality.
To understand the solution, it will be helpful to know about with_entries and the infix // operator, both of which are explained in the jq manual.
# Does the incoming JSON value contain an object which has a .tags
# value that is equal to $value or to an array containing $value ?
def hasTag($value):
any(.. | select(type=="object") | .tags;
. == $value or (type == "array" and index($value)));
Assuming the top-level JSON entity is a JSON object, we can now simply write:
with_entries( select( .value | hasTag("internal") | not) )
I have a JSON file that looks like this:
{
"InstanceId": "i-9KwoRGF6jbhYdZi823aE4qN",
"Tags": [
{
"Key": "blah",
"Value": "server-blah"
},
{
"Key": "environment",
"Value": "ops"
},
{
"Key": "server_role",
"Value": "appserver"
},
{
"Key": "Name",
"Value": "some_name"
},
{
"Key": "product",
"Value": "some_server"
}
]
}
{
...more objects like the above...
}
I need to display the InstanceId where "Key" == "environment" and "Value" == "ops".
I have jq-1.6.
If I say:
cat source.json | jq '
{ InstanceId, Tags } |
(.Tags[] | select( .Key == "environment" ))
'
I get some of what I want, but I cannot figure out how to include InstanceId in the output nor how to incorporate the "and" part of the select.
Here is a simple but efficient approach using any:
select( any(.Tags[]; .Key=="environment" and .Value == "ops") )
| .InstanceId
An alternative approach that avoids .Tags[]:
{"Key": "environment", "Value": "ops"} as $object
| select( .Tags | index($object) )
| .InstanceId
I'm not sure if this is the exact output you're looking for (comment if it isn't), but this will output the InstanceIds of JSON objects that contain a Tag with Key environment and Value ops.
jq 'select( .Tags[] | (.Key == "environment" and .Value == "ops")) | .InstanceId' < source.json
I have a json structure and would like to replace strings in 2 fields that are in a seperate text file.
Here is the json file with 2 records:
{
"events" : {
"-KKQQIUR7FAVxBOPOFhr" : {
"dateAdded" : 1487592568926,
"owner" : "62e6aaa0-a50c-4448-a381-f02efde2316d",
"type" : "boycott"
},
"-KKjjM-pAXvTuEjDjoj_" : {
"dateAdded" : 1487933370561,
"owner" : "62e6aaa0-a50c-4448-a381-f02efde2316d",
"type" : "boycott"
}
},
"geo" : {
"-KKQQIUR7FAVxBOPOFhr" : {
".priority" : "qw3yttz1k9",
"g" : "qw3yttz1k9",
"l" : [ 40.762632, -73.973837 ]
},
"-KKjjM-pAXvTuEjDjoj_" : {
".priority" : "qw3yttx6bv",
"g" : "qw3yttx6bv",
"l" : [ 41.889019, -87.626291 ]
}
},
"log" : "null",
"users" : {
"62e6aaa0-a50c-4448-a381-f02efde2316d" : {
"events" : {
"-KKQQIUR7FAVxBOPOFhr" : {
"type" : "boycott"
},
"-KKjjM-pAXvTuEjDjoj_" : {
"type" : "boycott"
}
}
}
}
}
And here is the txt file that I want to substitue in:
49.287130, -123.124026
36.129770, -115.172811
There are lots more records but I kept this to 2 for brevity.
Any help would be appreciated. Thank you.
The problem description seems to assume that the ordering of the key-value pairs within a JSON object is fixed. Different JSON-oriented tools (and indeed different versions of jq) have different takes on this. In any case, the following assumes a version of jq that respects the ordering (e.g. jq 1.5); it also assumes that inputs is available, though that is inessential.
The key to the following solution is the helper function, map_nth_value/2, which modifies the value of the nth key in a JSON object:
def map_nth_value(n; filter):
to_entries
| (.[n] |= {"key": .key, "value": (.value | filter)} )
| from_entries ;
[inputs | select(length > 0) | split(",") | map(tonumber)] as $lists
| reduce range(0; $lists|length) as $i
( $object;
.geo |= map_nth_value($i; .l = $lists[$i] ) )
With the above jq program in a file (say program.jq), and with the text file in a file (say input.txt) and the JSON object in a file (say object.json), the following invocation:
jq -R -n --argfile object object.json -f program.jq input.txt
produces:
{
"events": {
"-KKQQIUR7FAVxBOPOFhr": {
"dateAdded": 1487592568926,
"owner": "62e6aaa0-a50c-4448-a381-f02efde2316d",
"type": "boycott"
},
"-KKjjM-pAXvTuEjDjoj_": {
"dateAdded": 1487933370561,
"owner": "62e6aaa0-a50c-4448-a381-f02efde2316d",
"type": "boycott"
}
},
"geo": {
"-KKQQIUR7FAVxBOPOFhr": {
".priority": "qw3yttz1k9",
"g": "qw3yttz1k9",
"l": [
49.28713,
-123.124026
]
},
"-KKjjM-pAXvTuEjDjoj_": {
".priority": "qw3yttx6bv",
"g": "qw3yttx6bv",
"l": [
36.12977,
-115.172811
]
}
},
"log": "null",
"users": {
"62e6aaa0-a50c-4448-a381-f02efde2316d": {
"events": {
"-KKQQIUR7FAVxBOPOFhr": {
"type": "boycott"
},
"-KKjjM-pAXvTuEjDjoj_": {
"type": "boycott"
}
}
}
}
}
Here's some example JSON:
{
"Tags": [
{
"Key": "Name",
"Value": "foo"
},
{
"Key": "Type",
"Value": "C"
}
]
}
I want to print the value of "Value" only when "Key" is "Type". So it should print out "C". This is what I have so far.
echo $MY_TAGS | jq 'if .Tags[].Key == "Type" then .Tags[].Value else empty end'
But it prints out:
"foo"
"C"
Is there a way to do this?
Try this:
.Tags[] | select(.Key == "Type") | .Value