How to get the unique keys from attributes key with JQ
{"id":1, "attributes":{"a": 1, "b": 2, "c": 3}}
{"id":2, "attributes":{"a": 4, "b": 5, "d": 6}}
{"id":3, "name":"ABC"}
Result like this
[
"a",
"b",
"c",
"d"
]
I'm try like this
jq '.attributes' test.json | jq -r '[inputs | keys[]] | unique | sort'
or
jq -r '[inputs.attributes | keys[]] | unique | sort' test.json
but getting error
jq: error (at :11): null (null) has no keys
One way could be using reduce on subsequent inputs:
jq 'reduce inputs.attributes as $a (.attributes; . + $a) | keys'
[
"a",
"b",
"c",
"d"
]
Demo
Along the lines of your second attempt:
jq -n '[inputs.attributes // empty | keys_unsorted[]] | unique'
The important point is that we have to take care of the case where there is no "attributes" key.
Note also that unique sorts, so (unless you're using gojq) we can use keys_unsorted to avoid redundant sorting.
With slurp:
jq -s 'map(.attributes|keys?)|add|unique' test.json
-s loads the input file as array
map(.attributes|keys?) extracts only the keys (ignoring errors, such as trying to get keys of null)
add merges all nested arrays into a single array ([[1,2],[2,3]] becomes [1,2,2,3])
unique sorts and removes duplicates
Related
I am trying to update a json file using jq. My json file looks like
{
"A": "123",
"B": "456",
"C": "789",
"D": []
}
Here the value for key D is empty so I am adding some values to it. And this is working
Now, if for some reason the key doesn't exist then I need to first create the key D. And I am not able to achieve this
{
"A": "123",
"B": "456",
"C": "789",
}
cat test.json | jq 'has("D")' = false && cat test.json jq --argjson addobj '{"D": "[]"}'
I am getting the error
jq: error: Could not open file =: No such file or directory
jq: error: Could not open file false: No such file or directory
expected output
{
"A": "123",
"B": "456",
"C": "789",
"D": []
}
Can anyone please let me know what is the issue here and how to resolve it?
Thanks in advance.
P.S: Please let me know if any info is missing here
Your problem is not (only) with jq, but with shell syntax.
But if all you are trying to do is to update the value of key D whether it exists or not, then you don't need any checks and can simply assign the new value:
$ jq '.D = ["new value"]' test.json
{
"A": "123",
"B": "456",
"C": "789",
"D": [
"new value"
]
}
If you want to modify the current value, the operator |= might be helpful.
An alternative, equivalent program would be '. + { D: ["new value"] }'
If you really want to fix your script, here's a working version of it:
if jq -e 'has("D")' test.json >/dev/null; then
# key exists
jq --argjson addobj '{"D": "[]"}' 'your program here' test.json
else
# key doesn't exist
jq 'your other program here'
fi
But this is arguably easier in jq directly:
jq --argjson addobj '{"D": "[]"}' '
if has("D") then
# D exists
. # <- your jq program
else
# D doesn't exist
. # <- your other jq program
end
' test.json
If your goal is to simply insert the key with a default value if it doesn't exist, but keep any existing value, the following simple jq program (and nothing else) should take care of that:
jq '{D: []} + .' test.json
(keys in the RHS overwrite keys from the LHS – {a:1}+{a:2} becomes {a:2})
Objects in JavaScript (and by extension JSON), are a bag of unordered key-value pairs and {a:1,b:2} is the same object as {b:2,a:1}.
However, jq mostly keeps order of keys (although I don't think this is specified/guaranteed). So, a slightly more complicated version which puts D at the end of the object, but keeps existing values would be:
jq '.D |= (. // [])' test.json
I'm trying to create a massive CSV file converted from each *.json file. This snippet works until it faces the file that doesn't have the key (hobby).
Original
{
"name": "bob",
"hobby": [
"baseball",
"baseketball"
]
}
jq snippet
cat *.json | jq '.name as $n | .hobby | to_entries[] | [ $n, .value]'
It works
[][]... is a pre-format when creating CSV with jq
[
"bob",
"baseball"
]
[
"bob",
"baseketball"
]
https://jqplay.org/s/L-SmqiN-jw
However if the .hobby key doesn't exist it fails miserably.
jq: error (at <stdin>:6): null (null) has no keys
exit status 5
https://jqplay.org/s/gapUv1Tpmb
I tried to use if block but it seems not correct. How can we do such a thing either
return [] (empty array)
skip jq excution for the current working file with this problem and go to the next
One of many possibilities would be to use try:
.name as $n | .hobby | try to_entries[] | [ $n, .value]
try EXP is equivalent to try EXP catch empty.
Enter the following:
{
"name": "bob",
"hobby": [
"baseball",
"baseketball"
]
}
$ cat file | jq -c "{name,hobby:.hobby[]}|[.[]]"
["bob","baseball"]
["bob","baseketball"]
I have this json:
[
[
"0.99980000",
"3813863.19000000"
],
[
"0.99970000",
"3813412.11000000"
],
[
"0.99960000",
"1188619.16000000"
],
[
"0.99950000",
"1992659.70000000"
],
[
"0.99940000",
"3258570.31000000"
]
]
I would like to select the "highest of first" values, being 0.99980000 - how do I go about it with jq?
I wouldn't like to use shell's tail/head for that (i.e. "jq -r '.[] | .[]' | head -1" would output 0.99980000 - but it doesn't look that great and will not even work if the json data is not sorted already, at least not without yet more shell commands).
Use max_by:
jq -r 'max_by(.[0] | tonumber) | .[0]' file.json
I start from a jsonlines file similar to this
{ "kw": "foo", "age": 1}
{ "kw": "foo", "age": 1}
{ "kw": "foo", "age": 1}
{ "kw": "bar", "age": 1}
{ "kw": "bar", "age": 1}
Please note each line is a valid json, but the whole file is not.
The output I'm seeking is an ordered list of keywords sorted by its occurrence. Like this:
[
{"kw": "foo", "count": 3},
{"kw": "bar", "count": 2}
]
I'm able to group and count the keywords using the slurp option
jq --slurp '. | group_by(.kw) | .[] | {kw: .[0].kw, count: . | length }'
Output:
{"kw":"bar","count":2}
{"kw":"foo","count":3}
But:
This is not sorted
This is not valid JSON array
A very stupid solution I've found, is to pass twice via jq :)
jq --slurp --compact-output '. | group_by(.kw) | .[] | {kw: .[0].kw, count: . | length }' sample.json \
| jq --slurp --compact-output '. | sort_by(.count)'
But I'm pretty sure someone smarter than me can find a more elegant solution.
This is not sorted
That is not quite correct, group_by(.foo) internally does a sort(.foo), so the results are shown in the sorted order of the field. See jq Manual - group_by(path_expression)
This is not valid JSON array
Just enclose the operation within [..] and also the leading . is optional. So just do
jq --slurp --compact-output '[ group_by(.kw)[] | {kw: .[0].kw, count: length } ]'
If you are referring to sort by the .count you can do a ascending sort and reverse
jq --slurp --compact-output '[ group_by(.kw)[] | {kw: .[0].kw, count: length }] | sort_by(.count) | reverse'
How can I extract all key names, even in nested objects with jq?
For example, I have json:
{
"a": 1,
"b": {
"c": 2
}
}
and I want to get list:
a, b, b.c
I know that for top level keys I can get this, with:
. | to_entries[] | .key, but what about keys in nested objects?
Short jq solution:
jq -r '[paths | join(".")]' jsonfile
The output:
[
"a",
"b",
"b.c"
]
paths function outputs the paths to all the elements in its input
join(".") - to concatenate keys within hierarchical paths
Given input foo.json
{"a":1,"b":[{"c":2}]}
jq '[
paths |
map(select(type!="number")) |
select(length > 0) |
join(".")
] | unique' foo.json
outputs
[
"a",
"b",
"b.c"
]