I am trying to update a json file using jq. My json file looks like
{
"A": "123",
"B": "456",
"C": "789",
"D": []
}
Here the value for key D is empty so I am adding some values to it. And this is working
Now, if for some reason the key doesn't exist then I need to first create the key D. And I am not able to achieve this
{
"A": "123",
"B": "456",
"C": "789",
}
cat test.json | jq 'has("D")' = false && cat test.json jq --argjson addobj '{"D": "[]"}'
I am getting the error
jq: error: Could not open file =: No such file or directory
jq: error: Could not open file false: No such file or directory
expected output
{
"A": "123",
"B": "456",
"C": "789",
"D": []
}
Can anyone please let me know what is the issue here and how to resolve it?
Thanks in advance.
P.S: Please let me know if any info is missing here
Your problem is not (only) with jq, but with shell syntax.
But if all you are trying to do is to update the value of key D whether it exists or not, then you don't need any checks and can simply assign the new value:
$ jq '.D = ["new value"]' test.json
{
"A": "123",
"B": "456",
"C": "789",
"D": [
"new value"
]
}
If you want to modify the current value, the operator |= might be helpful.
An alternative, equivalent program would be '. + { D: ["new value"] }'
If you really want to fix your script, here's a working version of it:
if jq -e 'has("D")' test.json >/dev/null; then
# key exists
jq --argjson addobj '{"D": "[]"}' 'your program here' test.json
else
# key doesn't exist
jq 'your other program here'
fi
But this is arguably easier in jq directly:
jq --argjson addobj '{"D": "[]"}' '
if has("D") then
# D exists
. # <- your jq program
else
# D doesn't exist
. # <- your other jq program
end
' test.json
If your goal is to simply insert the key with a default value if it doesn't exist, but keep any existing value, the following simple jq program (and nothing else) should take care of that:
jq '{D: []} + .' test.json
(keys in the RHS overwrite keys from the LHS – {a:1}+{a:2} becomes {a:2})
Objects in JavaScript (and by extension JSON), are a bag of unordered key-value pairs and {a:1,b:2} is the same object as {b:2,a:1}.
However, jq mostly keeps order of keys (although I don't think this is specified/guaranteed). So, a slightly more complicated version which puts D at the end of the object, but keeps existing values would be:
jq '.D |= (. // [])' test.json
Related
I am trying to process a large json file for testing purposes that has a few thousand entries. The json contains a long list of data to is too large for me to process in one go. Using a jq, is there an easy way to get a valid snippet of the json that only contains the first few entries from the data list? For example is there a query that would look at the whole json file and return to me a valid json that only contains the first 4 entries from data? Thank you!
{
"info":{
"name":"some-name"
},
"data":[
{...},
{...},
{...},
{...}
}
Based on your snippet, the relevant jq would be:
.data |= .[:4]
Here's an example using the --stream option:
$ cat input.json
{
"info": {"name": "some-name"},
"data": [
{"a":1},
{"b":2},
{"c":3},
{"d":4},
{"e":5},
{"f":6},
{"g":7}
]
}
jq --stream -n '
reduce (
inputs | select(has(1) and (.[0] | .[0] == "data" and .[1] < 4))
) as $in (
{}; .[$in[0][-1]] = $in[1]
)
' input.json
{
"a": 1,
"b": 2,
"c": 3,
"d": 4
}
Note: Using limit would have been more efficient in this case, but I tried to be more generic for the purpose of scalability.
My input JSON is of the format:
{
"a": "apple",
"b": "banana",
"c": 5,
"d": ["this", "is", "an", "array"],
"e": false
}
What I want is:
a="apple"
b="banana"
c=5
d=["this", "is", "an", "array"]
e=false
Note that only strings in the input JSON have quotes in the output.
By using jq -r 'to_entries[] | "\(.key)=\"\(.value)\""' I could generate an output like
a="apple"
b="banana"
c="5"
d="["this", "is", "an", "array"]"
e="false"
So my question is:
Is using jq the right way to approach this problem? Or should I use regex?
If jq is the correct direction, how do I fix what I've come up with?
You can test whether .value is a string or not using type (manual entry).
jq -r 'to_entries[]
| "\(.key) = \( .value
| if type == "string"
then "\"\(.)\""
else .
end
)"'
I have json as a string "Str"
"{
"A": {
"id": 4
},
"B": {//Something},
"C": {
"A": {
"id": 2
}
},
"E": {
"A": null
},
"F": {//Something}
}"
I wanted all non null values of "A" which can be repeated anywhere in json. I wanted output like all contents of "A"
{"id": 4}
{"id": 2}
Can you please help me with Linux command to get this ?
Instead of line oriented ones use a tool which is capable of parsing JSON values syntax wise. An example using jq:
$ json_value='{"A":{"id":4},"B":{"foo":0},"C":{"A":{"id":2}},"E":{"A":null},"F":{"foo":0}}'
$
$ jq -c '..|objects|.A//empty' <<< "$json_value"
{"id":4}
{"id":2}
.. # list nodes recursively
| objects # select objects
| .A // empty # print A's value if present.
I have JSON like this that I'm parsing with jq:
{
"data": [
{
"item": {
"name": "string 1"
},
"item": {
"name": "string 2"
},
"item": {
"name": "string 3"
}
}
]
}
...and I'm trying to get "string 1" "string 2" and "string 3" into a Bash array, but I can't find a solution that ignores the whitespace in them. Is there a method in jq that I'm missing, or perhaps an elegant solution in Bash for it?
Current method:
json_names=$(cat file.json | jq ".data[] .item .name")
read -a name_array <<< $json_names
The below assume your JSON text is in a string named s. That is:
s='{
"data": [
{
"item1": {
"name": "string 1"
},
"item2": {
"name": "string 2"
},
"item3": {
"name": "string 3"
}
}
]
}'
Unfortunately, both of the below will misbehave with strings containing literal newlines; since jq doesn't have support for NUL-delimited output, this is difficult to work around.
On bash 4 (with slightly sloppy error handling, but tersely):
readarray -t name_array < <(jq -r '.data[] | .[] | .name' <<<"$s")
...or on bash 3.x or newer (with very comprehensive error handling, but verbosely):
# -d '' tells read to process up to a NUL, and will exit with a nonzero exit status if that
# NUL is not seen; thus, this causes the read to pass through any error which occurred in
# jq.
IFS=$'\n' read -r -d '' -a name_array \
< <(jq -r '.data[] | .[] | .name' <<<"$s" && printf '\0')
This populates a bash array, contents of which can be displayed with:
declare -p name_array
Arrays are assigned in the form:
NAME=(VALUE1 VALUE2 ... )
where NAME is the name of the variable, VALUE1, VALUE2, and the rest are fields separated with characters that are present in the $IFS (input field separator) variable.
Since jq outputs the string values as lines (sequences separated with the new line character), then you can temporarily override $IFS, e.g.:
# Disable globbing, remember current -f flag value
[[ "$-" == *f* ]] || globbing_disabled=1
set -f
IFS=$'\n' a=( $(jq --raw-output '.data[].item.name' file.json) )
# Restore globbing
test -n "$globbing_disabled" && set +f
The above will create an array of three items for the following file.json:
{
"data": [
{"item": {
"name": "string 1"
}},
{"item": {
"name": "string 2"
}},
{"item": {
"name": "string 3"
}}
]
}
The following shows how to create a bash array consisting of arbitrary JSON texts produced by a run of jq.
In the following, I'll assume input.json is a file with the following:
["string 1", "new\nline", {"x": 1}, ["one\ttab", 4]]
With this input, the jq filter .[] produces four JSON texts -- two JSON strings, a JSON object, and a JSON array.
The following bash script can then be used to set x to be a bash array of the JSON texts:
#!/bin/bash
x=()
while read -r value
do
x+=("$value")
done < <(jq -c '.[]' input.json)
For example, adding this bash expression to the script:
for a in "${x[#]}" ; do echo a="$a"; done
would yield:
a="string 1"
a="new\nline"
a={"x":1}
a=["one\ttab",4]
Notice how (encoded) newlines and (encoded) tabs are handled properly.
I have the following in a file (which I will call "myfile"):
[{
"id": 123,
"name": "John",
"aux": [{
"abc": "random",
"def": "I want this"
}],
"blah": 23.11
}]
I could parse it if the file did not have the first [ and last ] as follows:
$ cat myfile | jq -r '.aux[] | .def'
I want this
$
but with the [ and ] I get:
$ cat myfile | jq -r '.aux[] | .def'
jq: error: Cannot index array with string
How can I deal with the [ and ] using jq? (I'm sure I could parse them off with a different tool but I want to learn correct usage of jq.
It should be:
jq '.[].aux[].def' file.json
.[] iterates over the outer array, .aux[] then iterates over the the aux array of every node and .def prints their .def property.
This will output:
"I want this"
If you want to get rid of the double quotes pass -r (--raw) to jq:
jq -r '.[].aux[].def' file.json
Output:
I want this