I have this sample of JSON:
[
{
"name": "val1",
"expire": { "$value": 10 }
},
{
"name": "val2",
"expire": 20
},
{
"name": "val3"
}
]
And I want to transform it to this form with jq:
[
{
"name": "val1",
"expire": 10
},
{
"name": "val2",
"expire": 20
},
{
"name": "val3",
"expire": null
}
]
All that I've found it's if-then-else, but it looks like I have no clue how to build right expression.
Condition based on type check, it looks like a right way, but just return "compile"-time error, I don't know how to fix it:
.[] | { name, expire: (if .expire then (if type(.expire) == "number" then .expire else .expire."$value" end) else null end) }
Condition based on "$value" check, somehow filter out second object:
.[] | { name, expire: (if .expire then (if .expire."$value"? then .expire."$value" else .expire end) else null end) }
As I understand, the problem here in internal if, where second object checked with .expire."$value"?, error was thrown and object removed from result because of error.
Try this filter:
map( {name,
"expire": (.expire | if type == "object" then .["$value"] elif type == "number" then . else null end) } )
or (with significantly different semantics in edge cases):
map(.expire |= if type == "object" then .["$value"]
elif type == "number" then . else null end)
You can think of it another way as updating each object's expire property with the "$value" if present, or the current value. If the value doesn't exist, it's just simply null.
.[].expire |= (."$value"? // .)
Related
I have some json that I want to add to based on a walk. A simple example of the json I have is below:
{
"valueSource": "memory",
"dataType": "Boolean",
"alarms": [
{
"setpointA": 1.0,
"name": "Alarm",
"priority": "Diagnostic",
"ackMode": "Auto",
}
],
"name": "Test Alarm",
"value": false,
"tagType": "AtomicTag"
}
I want to add to each object in the "alarms" key's array the following key:
{
"bindType": "Tag",
"value": "[.]<parent.name>.Name"
}
where <parent.name> is "Test Alarm" in this example which is the parent-of-the-alarm-array-item's "name" key.
I've gotten this jq filter so far that adds the object, but the value key value is wrong (it's getting the alarm array item's name instead of its parent's name):
walk( if type == "object" and .setpointA then .label = {"bindType":"Tag", "value": "[.]\(.name).Name"} else . end)
Essentially I want this:
{
"valueSource": "memory",
"dataType": "Boolean",
"alarms": [
{
"setpointA": 1.0,
"name": "Alarm",
"priority": "Diagnostic",
"ackMode": "Auto",
"label": {
"bindType": "Tag",
"value": "[.]Test Alarm.Name"
}
}
],
"name": "Test Alarm",
"value": false,
"tagType": "AtomicTag"
}
Here is my jqplay below. It has the final result in the JSON section, where the Result should match this but doesn't at the moment.
https://jqplay.org/s/-qHFIWolrD
Follow up question:
How would I add this displayPath key as well?
You cannot reference to parent, you have to save the reference in a variable beforehand, and descend with having access to that variable.
.tags[] |= (
.name as $name
| # rest of your code using $name
walk(
if type == "object" and .setpointA
then .label = {"bindType":"Tag", "value": "[.]\($name).Name"}
else . end
)
)
Demo
As you happen to know that the objects are located in the .alarms array, you could also just iterate over the items, select only those matching the condition and then assign to their .label whatever you want (including $name)
.tags[] |= (
.name as $name
| (.alarms[] | select(has("setpointA"))).label = {
bindType: "Tag", value: "[.]\($name).Name"
}
)
Demo
Edit responding to OP's follow-up question
I actually don't care about the setpointA key, I just want to add the label key (as well as another displayPath key) to each item in all alarms arrays.
.tags[] |= (.name as $name | .alarms[] += (
{bindType: "Tag", value: "[.]\($name)."} | {
label: (.value += "Name"),
displayPath: (.value += "Documentation")
}
))
Demo
As jq does not allow you to refer to parent object, you need to work on parent level :
jq 'walk(if type == "object" and .alarms and ( .alarms | arrays )
then (.alarms[] | select(.setpointA)).label =
{ bindType: "Tag", value: "[.]\(.name).Name"}
else . end
)' data.json
Looking to change all the values of name to Lower Case.
I am currently running
map( .[].data_node.name |= ascii_downcase)
and receive jq: error (at data3.json:538): explode input must be a string error.
Sample input:
[
{
"data_node":{
"name":"FRODO BAGGINS",
"race":"hobbit",
"existence":"middle earth"
},
"parent":"bilbo baggins"
},
{
"data_node":{
"name":"SAMWISE GAMJEE",
"race":"hobbit",
"existence":"middle earth"
},
"parent":"gamjee lord"
},
{
"data_node":{
"name":null,
"race":"hobbit",
"existence":"middle earth"
},
"parent":"bilbo baggins"
}
]
Sample output:
[
{
"data_node":{
"name":"frodo baggins",
"race":"hobbit",
"existence":"middle earth"
},
"parent":"bilbo baggins"
},
{
"data_node":{
"name":"samwise gamjee",
"race":"hobbit",
"existence":"middle earth"
},
"parent":"gamjee lord"
},
{
"data_node":{
"name": null,
"race":"hobbit",
"existence":"middle earth"
},
"parent":"bilbo baggins"
}
]
What's wrong with my current line of code and what's the solution to doing this right.
More briefly and more robustly:
map(.data_node.name |= if type == "string" then ascii_downcase else . end)
Or still more briefly:
map(.data_node.name |= ascii_downcase? // .)
(These assume that jq is invoked without the -s option.)
Add an if condition to do the case conversion only if the name field is not null as
map( if (.data_node.name != null) then .data_node.name |= ascii_downcase else . end )
As far as the error you are seeing, I'd assume that the case conversion we apply does not apply to the null datatype in jq and only for string types.
To run it on the command-line directly without running as a script
jq 'map( if (.data_node.name != null) then .data_node.name |= ascii_downcase else . end )' < data3.json
jqplay-URL
I have an API that returns JSON - big blocks of it. Some of the key value pairs have more blocks of JSON as the value associated with a key. jq does a great job of parsing the main JSON levels. But I can't find a way to get it to 'recurse' into the values associated with the keys and pretty print them as well.
Here is the start of one of the JSON returns. Note it is only a small percent of the full return:
{
"code": 200,
"status": "OK",
"data": {
"PlayFabId": "xxxxxxx",
"InfoResultPayload": {
"AccountInfo": {
"PlayFabId": "xxxxxxxx",
"Created": "2018-03-22T19:23:29.018Z",
"TitleInfo": {
"Origination": "IOS",
"Created": "2018-03-22T19:23:29.033Z",
"LastLogin": "2018-03-22T19:23:29.033Z",
"FirstLogin": "2018-03-22T19:23:29.033Z",
"isBanned": false
},
"PrivateInfo": {},
"IosDeviceInfo": {
"IosDeviceId": "xxxxxxxxx"
}
},
"UserVirtualCurrency": {
"GT": 10,
"MB": 70
},
"UserVirtualCurrencyRechargeTimes": {},
"UserData": {},
"UserDataVersion": 15,
"UserReadOnlyData": {
"DataVersion": {
"Value": "6",
"LastUpdated": "2018-03-22T19:48:59.543Z",
"Permission": "Public"
},
"achievements": {
"Value": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50,\"achievements\":[{\"id\":2,\"name\":\"Correct Round 4\",\"description\":\"Round 4 answered correctly\",\"maxValue\":10,\"increment\":1,\"currentValue\":3,\"valueUnit\":\"unit\",\"awardOnIncrement\":true,\"marbles\":10,\"image\":\"https://www.jamandcandy.com/kissinkuzzins/achievements/icons/sphinx\",\"SuccessKey\":[\"0_3_4_0\",\"0_5_4_0\",\"0_6_4_0\",\"0_7_4_0\",\"0_8_4_0\",\"0_9_4_0\",\"0_10_4_0\"],\"event\":\"Player_answered_round\",\"achieved\":false},{\"id\":0,\"name\":\"Complete
This was parsed using jq but as you can see when you get to the
"achievements": { "Vales": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50,\
lq does no further parse the value at is also JSON.
Is there a filter I am missing to get it to parse the values as well as the higher level structure?
Is there a filter I am missing ...?
The filter you'll need is fromjson, but it should only be applied to the stringified JSON; consider therefore using |= as illustrated using your fragment:
echo '{"achievements": { "Vales": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50}]"}}' |
jq '.achievements.Vales |= fromjson'
{
"achievements": {
"Vales": [
{
"id": 0,
"gamePack": "GAME.PACK.0.KK",
"marblesAmount": 50
}
]
}
}
recursively/1
If you want to apply fromjson recursively wherever possible, then recursively is your friend:
def recursively(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | recursively(f) )} )
elif type == "array" then map( recursively(f) )
else try (f as $f | if $f == . then . else ($f | recursively(f)) end) catch $in
end;
This would be applied as follows:
recursively(fromjson)
Example
{a: ({b: "xyzzy"}) | tojson} | tojson
| recursively(fromjson)
yields:
{
"a": {
"b": "xyzzy"
}
}
I would like to use jq to remove all dictionaries within a JSON "object" (I used that term generally to refer to either an Array or a Dictionary) that
a) contain a key named "delete_me", AND
b) where the key "delete_me" meets some predetermined condition (null, non-zero, true, etc)
Basically, the logic I want to implement is: walk the input, and at each node, if that node is not an Array or an Object, then keep it and move on, otherwise, keep it but remove from it any children that are dictionaries for which either condition a) or b) fail.
Any suggestions?
Sample input:
{
"a": { "foo": "bar" },
"b": {
"i": {
"A": {
"i": [
{
"foo": {},
"bar": {
"delete_if_this_is_null": false,
"an_array": [],
"another_array": [
{
"delete_if_this_is_null": null,
"foo": "bar"
}
],
"etc": ""
},
"foo2": "s"
},
{
"foo": {
"an_array": [
{
"delete_if_this_is_null": "ok",
"foo":"bar",
"another_object": { "a":1 }
},
{
"delete_if_this_is_null": null,
"foo2":"bar2",
"another_object": { "a":1 },
"name": null
}
],
"an_object": {
"delete_if_this_is_null":null,
"foo3":"bar3"
}
},
"zero": 0,
"b": "b"
}
]
}
}
}
}
should yield, if the "delete_me" key is delete_if_this_is_null and the predetermined condition is delete_if_this_is_null == null:
{
"a": { "foo": "bar" },
"b": {
"i": {
"A": {
"i": [
{
"foo": {},
"bar": {
"delete_if_this_is_null": false,
"an_array": [],
"another_array": [],
"etc": ""
},
"foo2": "s"
},
{
"foo": {
"an_array": [
{
"delete_if_this_is_null": "ok",
"foo":"bar",
"another_object": { "a":1 }
}
]
},
"zero": 0,
"b": "b"
}
]
}
}
}
}
UPDATE: Here's the solution: Assume the input is in a file 'input.json':
jq 'def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
def mapper(f):
if type == "array" then map(f)
elif type == "object" then
. as $in
| reduce keys[] as $key
({};
[$in[$key] | f ] as $value
| if $value | length == 0 then .
else . + {($key): $value[0]} end)
else .
end;
walk( mapper(select((type == "object" and .delete_if_this_is_null == null) | not)) )' < input.json
Jeff's solution may zap too much. For example, using:
def data: [1,2, {"hello": {"delete_me": true, "a":3 }, "there": 4} ]; ];
Jeff's solution yields empty (i.e. nothing).
The following may therefore be closer to what you're looking for:
walk(if (type == "object" and .delete_me) then del(.) else . end )
For data, this yields:
[1,2,{"hello":null,"there":4}]
Alternative Solution
If a solution that eliminates the "hello":null in the above example is required, then a variant of jq's map_values/1 is needed. Here's one approach:
def mapper(f):
if type == "array" then map(f)
elif type == "object" then
. as $in
| reduce keys[] as $key
({};
[$in[$key] | f ] as $value
| if $value | length == 0 then .
else . + {($key): $value[0]} end)
else .
end;
data | walk( mapper(select((type == "object" and .delete_me) | not)) )
The result is:
[1,2,{"there":4}]
Here is a solution which uses a recursive function:
def clean(condition):
if type == "object" then
if condition
then empty
else
with_entries(
if (.value|type) == "object" and (.value|condition)
then empty
else .value |= clean(condition)
end
)
end
elif type == "array" then
map(
if type == "object" and condition
then empty
else clean(condition)
end
)
else .
end
;
clean(
has("delete_if_this_is_null") and (.delete_if_this_is_null == null)
)
I'm not sure what exactly you're trying to accomplish in your question but I'm assuming you want to recursively search through a json response and remove json objects that satisfy some condition.
You can do this rather easily with the help of the walk filter that will be coming up in a future version of jq, see the implementation in the source.
# Apply f to composite entities recursively, and to atoms
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
With that, you can filter them out like so:
def filter_objects(predicate): # removes objects that satisfies some predicate
walk(
if (type == "object") and (predicate) then
empty
else
.
end
)
;
filter_objects(.delete_me) # remove objects that has a truthy property "delete_me"
How would I parse this object, removing all entries where "field"="status", but at the same time retaining the validity of the JSON object (i.e. also removing the extraneous closing braces or brackets at the end of the object?
{"logic":"and","filters":[{"filters":[{"field":"name","operator":"contains","value":"JOHNSON"},{"field":"city","operator":"contains","value":"MILWAUKEE"}],"logic":"and"},{"logic":"or","filters":[{"field":"status","operator":"eq","value":"A"},{"field":"status","operator":"eq","value":"G"},{"field":"status","operator":"eq","value":"O"},{"field":"status","operator":"eq","value":"P"},{"field":"status","operator":"eq","value":"S"}]}]
Here is a solution using jq based on my answer to a similar question
def clean(condition):
if type == "object" then
if condition
then empty
else
with_entries(
if (.value|type) == "object" and (.value|condition)
then empty
else .value |= clean(condition)
end
)
end
elif type == "array" then
map(
if type == "object" and condition
then empty
else clean(condition)
end
)
else .
end
;
clean(
has("field") and (.field == "status")
)
If filter.jq contains this filter and data.json contains the sample data then the command
$ jq -M -f filter.jq data.json
produces
{
"logic": "and",
"filters": [
{
"filters": [
{
"field": "name",
"operator": "contains",
"value": "JOHNSON"
},
{
"field": "city",
"operator": "contains",
"value": "MILWAUKEE"
}
],
"logic": "and"
},
{
"logic": "or",
"filters": []
}
]
}