I am learning how to use jq to manipulate json data,
I'm having a little trouble with it.
This is my input JSON
{
"user":{
"advertisingID":"617a68"
},
"deviceTs":1575387020137,
"activies":[
{
"ts":1575617868326,
"appsUsage":{
"isFull":true,
"data":[
{
"com.orange.phone":44009
}
],
"startTs":1575617281541
}
},
{
"ts":1575618968326,
"appsUsage":{
"isFull":true,
"data":[
{
"uk.green.launcher2":4354
},
{
"com.black.phone":1232
}
],
"startTs":1575617281541
}
}
]
}
I want to replace all keys containing "dots" by "dashes"
and expected output:
{
"user":{
"advertisingID":"617a68"
},
"deviceTs":1575387020137,
"activies":[
{
"ts":1575617868326,
"appsUsage":{
"isFull":true,
"data":[
{
"com-orange-phone":44009 <----
}
],
"startTs":1575617281541
}
},
{
"ts":1575618968326,
"appsUsage":{
"isFull":true,
"data":[
{
"uk-green-launcher2":4354 <----
},
{
"com-black-phone":1232 <----
}
],
"startTs":1575617281541
}
}
]
}
I have tried with
.activies |= map( with_entries(if .key == "appsUsage" then ... else . end) )
...
(split(".")|join("-")) but without success,
Thanks in advance.
I don't think you need regex for this; I believe a conjuction of split and join builtins would be more effective and clean.
.activies |= walk(
if type == "object" then
reduce (keys_unsorted[] | select(index("."))) as $k (.;
(.[$k | split(".") | join("-")] = .[$k])
| del(.[$k])
)
else . end
)
Online demo
Upon peak's recommendation, a more readable solution:
.activies |= walk(
if type == "object" then
with_entries(
.key |= gsub("\\.";"-")
)
else . end
)
Online demo
Related
Consider the below JSON:
[
{
"vd":[
{
"key":1
},
{
"key":2
}
]
}
]
Now I would like to filter based on the filter key==1. So the result should be:
[
{
"vd": [
{
"key": 1
}
]
}
]
Is there a way to achieve this using jq?
Traverse to the array in question .[].vd, and update |= with a map based on a select using your criteria .key == 1:
jq '.[].vd |= map(select(.key == 1))'
[
{
"vd": [
{
"key": 1
}
]
}
]
Demo
My json template is as of this:
{
"interface_settings": [
{
"name": "lan",
"status": "$status",
...
},
{
"name": "lte1",
"status": "$status",
...
},
{
...
}
],
...
}
And my jq command:
jq '.interface_settings[].status="up"' <my_json_template file>
will update both status values within the interface_settings section. How may I just have one changed ?
let's say I want to update the status where the name is "lan"
One way to update all such objects would be:
.interface_settings |= map( if .name == "lan" then .status = "up" else . end)
Just the first such
.interface_settings |= (reduce .[] as $x (null;
if .done
then .ans += [$x]
elif $x.name == "lan"
then .ans += [$x | .status = "up"] | .done = true
else .ans += [$x]
end) | .ans)
I'm starting with a JSON file that has multiple objects
{
"name": "foo",
"url": "https://zombo.com"
}
{
"name": "bar",
"url": "https://acme.com"
}
and I'm trying to combine those into a single object with the name attribute as the key:
{
"widgets": {
"foo": {
"url": "https://zombo.com"
},
"bar": {
"url": "https://acme.com"
}
}
}
I've been banging my head against this for a while and I think I'm fairly close with the following query:
{ widgets: (reduce . as $item ({}; . + {($item.name): {url: $item.url}})) }
However, this results in multiple objects and I'm running out of ideas.
jqplay available here: https://jqplay.org/s/SI7XEMb5l9.
I'd do it like this:
jq -s '{ widgets: map( { (.name): {url} } ) | add }'
-s (--slurp) combines the objects to an array of objects
map( { (.name): { url } } ) reshapes each object
add concatenates the array elements into a single object
and finally, I build an object with { widgets: ... }.
Use reduce with inputs:
jq -n '{ widgets : (reduce inputs as $p ({}; . + ($p | { (.name) : { url } }))) }' file
I have an API that returns JSON - big blocks of it. Some of the key value pairs have more blocks of JSON as the value associated with a key. jq does a great job of parsing the main JSON levels. But I can't find a way to get it to 'recurse' into the values associated with the keys and pretty print them as well.
Here is the start of one of the JSON returns. Note it is only a small percent of the full return:
{
"code": 200,
"status": "OK",
"data": {
"PlayFabId": "xxxxxxx",
"InfoResultPayload": {
"AccountInfo": {
"PlayFabId": "xxxxxxxx",
"Created": "2018-03-22T19:23:29.018Z",
"TitleInfo": {
"Origination": "IOS",
"Created": "2018-03-22T19:23:29.033Z",
"LastLogin": "2018-03-22T19:23:29.033Z",
"FirstLogin": "2018-03-22T19:23:29.033Z",
"isBanned": false
},
"PrivateInfo": {},
"IosDeviceInfo": {
"IosDeviceId": "xxxxxxxxx"
}
},
"UserVirtualCurrency": {
"GT": 10,
"MB": 70
},
"UserVirtualCurrencyRechargeTimes": {},
"UserData": {},
"UserDataVersion": 15,
"UserReadOnlyData": {
"DataVersion": {
"Value": "6",
"LastUpdated": "2018-03-22T19:48:59.543Z",
"Permission": "Public"
},
"achievements": {
"Value": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50,\"achievements\":[{\"id\":2,\"name\":\"Correct Round 4\",\"description\":\"Round 4 answered correctly\",\"maxValue\":10,\"increment\":1,\"currentValue\":3,\"valueUnit\":\"unit\",\"awardOnIncrement\":true,\"marbles\":10,\"image\":\"https://www.jamandcandy.com/kissinkuzzins/achievements/icons/sphinx\",\"SuccessKey\":[\"0_3_4_0\",\"0_5_4_0\",\"0_6_4_0\",\"0_7_4_0\",\"0_8_4_0\",\"0_9_4_0\",\"0_10_4_0\"],\"event\":\"Player_answered_round\",\"achieved\":false},{\"id\":0,\"name\":\"Complete
This was parsed using jq but as you can see when you get to the
"achievements": { "Vales": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50,\
lq does no further parse the value at is also JSON.
Is there a filter I am missing to get it to parse the values as well as the higher level structure?
Is there a filter I am missing ...?
The filter you'll need is fromjson, but it should only be applied to the stringified JSON; consider therefore using |= as illustrated using your fragment:
echo '{"achievements": { "Vales": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50}]"}}' |
jq '.achievements.Vales |= fromjson'
{
"achievements": {
"Vales": [
{
"id": 0,
"gamePack": "GAME.PACK.0.KK",
"marblesAmount": 50
}
]
}
}
recursively/1
If you want to apply fromjson recursively wherever possible, then recursively is your friend:
def recursively(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | recursively(f) )} )
elif type == "array" then map( recursively(f) )
else try (f as $f | if $f == . then . else ($f | recursively(f)) end) catch $in
end;
This would be applied as follows:
recursively(fromjson)
Example
{a: ({b: "xyzzy"}) | tojson} | tojson
| recursively(fromjson)
yields:
{
"a": {
"b": "xyzzy"
}
}
I would like to use jq to remove all dictionaries within a JSON "object" (I used that term generally to refer to either an Array or a Dictionary) that
a) contain a key named "delete_me", AND
b) where the key "delete_me" meets some predetermined condition (null, non-zero, true, etc)
Basically, the logic I want to implement is: walk the input, and at each node, if that node is not an Array or an Object, then keep it and move on, otherwise, keep it but remove from it any children that are dictionaries for which either condition a) or b) fail.
Any suggestions?
Sample input:
{
"a": { "foo": "bar" },
"b": {
"i": {
"A": {
"i": [
{
"foo": {},
"bar": {
"delete_if_this_is_null": false,
"an_array": [],
"another_array": [
{
"delete_if_this_is_null": null,
"foo": "bar"
}
],
"etc": ""
},
"foo2": "s"
},
{
"foo": {
"an_array": [
{
"delete_if_this_is_null": "ok",
"foo":"bar",
"another_object": { "a":1 }
},
{
"delete_if_this_is_null": null,
"foo2":"bar2",
"another_object": { "a":1 },
"name": null
}
],
"an_object": {
"delete_if_this_is_null":null,
"foo3":"bar3"
}
},
"zero": 0,
"b": "b"
}
]
}
}
}
}
should yield, if the "delete_me" key is delete_if_this_is_null and the predetermined condition is delete_if_this_is_null == null:
{
"a": { "foo": "bar" },
"b": {
"i": {
"A": {
"i": [
{
"foo": {},
"bar": {
"delete_if_this_is_null": false,
"an_array": [],
"another_array": [],
"etc": ""
},
"foo2": "s"
},
{
"foo": {
"an_array": [
{
"delete_if_this_is_null": "ok",
"foo":"bar",
"another_object": { "a":1 }
}
]
},
"zero": 0,
"b": "b"
}
]
}
}
}
}
UPDATE: Here's the solution: Assume the input is in a file 'input.json':
jq 'def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
def mapper(f):
if type == "array" then map(f)
elif type == "object" then
. as $in
| reduce keys[] as $key
({};
[$in[$key] | f ] as $value
| if $value | length == 0 then .
else . + {($key): $value[0]} end)
else .
end;
walk( mapper(select((type == "object" and .delete_if_this_is_null == null) | not)) )' < input.json
Jeff's solution may zap too much. For example, using:
def data: [1,2, {"hello": {"delete_me": true, "a":3 }, "there": 4} ]; ];
Jeff's solution yields empty (i.e. nothing).
The following may therefore be closer to what you're looking for:
walk(if (type == "object" and .delete_me) then del(.) else . end )
For data, this yields:
[1,2,{"hello":null,"there":4}]
Alternative Solution
If a solution that eliminates the "hello":null in the above example is required, then a variant of jq's map_values/1 is needed. Here's one approach:
def mapper(f):
if type == "array" then map(f)
elif type == "object" then
. as $in
| reduce keys[] as $key
({};
[$in[$key] | f ] as $value
| if $value | length == 0 then .
else . + {($key): $value[0]} end)
else .
end;
data | walk( mapper(select((type == "object" and .delete_me) | not)) )
The result is:
[1,2,{"there":4}]
Here is a solution which uses a recursive function:
def clean(condition):
if type == "object" then
if condition
then empty
else
with_entries(
if (.value|type) == "object" and (.value|condition)
then empty
else .value |= clean(condition)
end
)
end
elif type == "array" then
map(
if type == "object" and condition
then empty
else clean(condition)
end
)
else .
end
;
clean(
has("delete_if_this_is_null") and (.delete_if_this_is_null == null)
)
I'm not sure what exactly you're trying to accomplish in your question but I'm assuming you want to recursively search through a json response and remove json objects that satisfy some condition.
You can do this rather easily with the help of the walk filter that will be coming up in a future version of jq, see the implementation in the source.
# Apply f to composite entities recursively, and to atoms
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
With that, you can filter them out like so:
def filter_objects(predicate): # removes objects that satisfies some predicate
walk(
if (type == "object") and (predicate) then
empty
else
.
end
)
;
filter_objects(.delete_me) # remove objects that has a truthy property "delete_me"