I have the below JSON string with multiple JSON objects. I would like to extract just the id and name from each object and print it.
{
"users": [
{
"id": "1",
"name": "John Wick",
"location": "USA"
},
{
"id": "2",
"name": "Walter White",
"location": "USA"
}
]
}
I am using the below code to extract the id and name using 'jq'
for key in $(jq -c '.users | .[]' sample.json); do
id=$(jq -r '.id' <<< "$key");
name=$(jq -r '.name' <<< "$key")
echo $id $name
done
But I am getting parsing errors like below. Can someone help me with this?
parse error: Unfinished string at EOF at line 2, column 0
I tried replacing spaces with a combination of special chars and replace again special chars with spaces. It worked for me but I need a better solution than this.
You can use the csv or tsv features of jq - lots of great examples in the man page!
man jq
Try this:
jq -r '.users[] | [.id , .name] | #csv' sample.json
Example output:
$ jq -r '.users[] | [.id , .name] | #csv' sample.json
"1","John Wick"
"2","Walter White"
Or using string interpolation:
$ jq -r '.users[] | [.id, .name] | "\(.[0]) \(.[1])"' sample.json
1 John Wick
2 Walter White
Using 2 read's to capture the values, we can let JQ loop over the objects, and output the id and name. :
#!/bin/bash
jq -r -c '.users[] | .id, .name' /tmp/input3 | while read -r id && read -r name; do
echo -e "ID: ${id}\t Name: ${name}"
done
Output:
ID: 1 Name: John Wick
ID: 2 Name: Walter White
Related
I'm trying to create a massive CSV file converted from each *.json file. This snippet works until it faces the file that doesn't have the key (hobby).
Original
{
"name": "bob",
"hobby": [
"baseball",
"baseketball"
]
}
jq snippet
cat *.json | jq '.name as $n | .hobby | to_entries[] | [ $n, .value]'
It works
[][]... is a pre-format when creating CSV with jq
[
"bob",
"baseball"
]
[
"bob",
"baseketball"
]
https://jqplay.org/s/L-SmqiN-jw
However if the .hobby key doesn't exist it fails miserably.
jq: error (at <stdin>:6): null (null) has no keys
exit status 5
https://jqplay.org/s/gapUv1Tpmb
I tried to use if block but it seems not correct. How can we do such a thing either
return [] (empty array)
skip jq excution for the current working file with this problem and go to the next
One of many possibilities would be to use try:
.name as $n | .hobby | try to_entries[] | [ $n, .value]
try EXP is equivalent to try EXP catch empty.
Enter the following:
{
"name": "bob",
"hobby": [
"baseball",
"baseketball"
]
}
$ cat file | jq -c "{name,hobby:.hobby[]}|[.[]]"
["bob","baseball"]
["bob","baseketball"]
curl http://testhost.test.com:8080/application/app/version | jq '.version' | jq '.[]'
The above command outputs only the values as below:
"madireddy#test.com"
"2323"
"test"
"02-03-2014-13:41"
"application"
How can I get the key names instead like the below:
email
versionID
context
date
versionName
To get the keys in the order they appear in the original JSON use:
jq 'keys_unsorted' file.json
If you want the keys sorted alphanumerically, you can use:
jq 'keys' file.json
Complete example
$ cat file.json
{ "Created-By" : "Apache Maven", "Build-Number" : "", "Archiver-Version" : "Plexus Archiver", "Build-Id" : "", "Build-Tag" : "", "Built-By" : "cporter"}
$ jq 'keys_unsorted' file.json
[
"Created-By",
"Build-Number",
"Archiver-Version",
"Build-Id",
"Build-Tag",
"Built-By"
]
$ jq 'keys' file.json
[
"Archiver-Version",
"Build-Id",
"Build-Number",
"Build-Tag",
"Built-By",
"Created-By"
]
To get the keys on a deeper node in a JSON:
echo '{"data": "1", "user": { "name": 2, "phone": 3 } }' | jq '.user | keys[]'
"name"
"phone"
You need to use jq 'keys[]'. For example:
echo '{"example1" : 1, "example2" : 2, "example3" : 3}' | jq 'keys[]'
Will output a line separated list:
"example1"
"example2"
"example3"
In combination with the above answer, you want to ask jq for raw output, so your last filter should be eg.:
cat input.json | jq -r 'keys'
From jq help:
-r output raw strings, not JSON texts;
To print keys on one line as csv:
echo '{"b":"2","a":"1"}' | jq -r 'keys | [ .[] | tostring ] | #csv'
Output:
"a","b"
For csv completeness ... to print values on one line as csv:
echo '{"b":"2","a":"1"}' | jq -rS . | jq -r '. | [ .[] | tostring ] | #csv'
Output:
"1","2"
If your input is an array of objects,
[
{
"a01" : { "name" : "A", "user" : "B" }
},
{
"a02" : { "name" : "C", "user" : "D" }
}
]
try with:
jq '.[] | keys[]'
Oddly enough, the accepted answer doesn’t actually answer the Q exactly, so for reference, here is a solution that does:
$ jq -r 'keys_unsorted[]' file.json
echo '{"ab": 1, "cd": 2}' | jq -r 'keys[]' prints all keys one key per line without quotes.
ab
cd
Here's another way of getting a Bash array with the example JSON given by #anubhava in his answer:
arr=($(jq --raw-output 'keys_unsorted | #sh' file.json))
echo ${arr[0]} # 'Archiver-Version'
echo ${arr[1]} # 'Build-Id'
echo ${arr[2]} # 'Build-Jdk'
I have a json content (output.json)
{"project": {"id": "A", "content": [{"name": "XYZ", "location": "Berlin", "comments":""}, {"name": "ABC", "location": "NewYork", "comments": "Hwllo"}, {"name": "DEF", "location": "Paris", "comments": "Success"}]}}
I would like to extract location key with value when name matches say ABC from the above json using bash or shell commands
I tried something like below which gives be content within curly braces. but not sure on searching specific key.
cat output.json | grep -o -e "{.*}"
Output expectations:
if name matches ABC, get output as "location":"NewYork"
Any suggestions on processing further?
For extracting from json you should use jq if you can. According to authors "jq is like sed for JSON data" (source).
In your case it should be:
$ jq -r '.project' output.json | jq -r '.content' | jq '.[] | select(.name=="ABC")' | jq -r '.location'
Output will be:
NewYork
To get output which you required so:
"location":"NewYork"
You can use:
echo "\"location\":$(jq -r '.project' output.json | jq -r '.content' | jq '.[] | select(.name=="ABC")' | jq '.location')"
Before you use jq you should install it on Debian and Ubuntu it will be:
$ sudo apt install jq
for other OS you should check this site.
There may be better ways to do it, in a quick twisted way is here.
cat output.json | sed 's/"name"/\n"name"/g' | grep '"name"' | awk -F',' '{print $2}'
Add | grep <preferred name> also if need to filter based on name.
Use Perl
$ perl -0777 -lne ' while(/"name":\s+"ABC",\s+"location":\s+(\S+)/msg) { print "$1\n" } ' output.json
"NewYork",
$ cat output.json
{"project": {"id": "A", "content": [{"name": "XYZ", "location": "Berlin", "comments":""}, {"name": "ABC", "location": "NewYork", "comments": "Hwllo"}, {"name": "DEF", "location": "Paris", "comments": "Success"}]}}
$
Please use a JSON parser for processing JSON.
With Xidel it's as simple as:
xidel -s output.json -e '($json//content)()[name="ABC"]/location'
Alternatively:
xidel -s output.json -e '$json/(.//content)()[name="ABC"]/location'
or in full:
xidel -s output.json -e '$json/project/(content)()[name="ABC"]/location'
The above is XPath notation (example #11, Reading JSON). Dot notation (like jq) is also possible:
xidel -s output.json -e '($json).project.content()[name="ABC"].location'
[edit]
Commands above put out NewYork and I just realized you require the output to be "location":"NewYork". Xidel can do that too:
xidel -s output.json -e '($json//content)()[name="ABC"]/concat("""location"":""",location,"""")'
[/edit]
I have the following in a file (which I will call "myfile"):
[{
"id": 123,
"name": "John",
"aux": [{
"abc": "random",
"def": "I want this"
}],
"blah": 23.11
}]
I could parse it if the file did not have the first [ and last ] as follows:
$ cat myfile | jq -r '.aux[] | .def'
I want this
$
but with the [ and ] I get:
$ cat myfile | jq -r '.aux[] | .def'
jq: error: Cannot index array with string
How can I deal with the [ and ] using jq? (I'm sure I could parse them off with a different tool but I want to learn correct usage of jq.
It should be:
jq '.[].aux[].def' file.json
.[] iterates over the outer array, .aux[] then iterates over the the aux array of every node and .def prints their .def property.
This will output:
"I want this"
If you want to get rid of the double quotes pass -r (--raw) to jq:
jq -r '.[].aux[].def' file.json
Output:
I want this
curl http://testhost.test.com:8080/application/app/version | jq '.version' | jq '.[]'
The above command outputs only the values as below:
"madireddy#test.com"
"2323"
"test"
"02-03-2014-13:41"
"application"
How can I get the key names instead like the below:
email
versionID
context
date
versionName
To get the keys in the order they appear in the original JSON use:
jq 'keys_unsorted' file.json
If you want the keys sorted alphanumerically, you can use:
jq 'keys' file.json
Complete example
$ cat file.json
{ "Created-By" : "Apache Maven", "Build-Number" : "", "Archiver-Version" : "Plexus Archiver", "Build-Id" : "", "Build-Tag" : "", "Built-By" : "cporter"}
$ jq 'keys_unsorted' file.json
[
"Created-By",
"Build-Number",
"Archiver-Version",
"Build-Id",
"Build-Tag",
"Built-By"
]
$ jq 'keys' file.json
[
"Archiver-Version",
"Build-Id",
"Build-Number",
"Build-Tag",
"Built-By",
"Created-By"
]
To get the keys on a deeper node in a JSON:
echo '{"data": "1", "user": { "name": 2, "phone": 3 } }' | jq '.user | keys[]'
"name"
"phone"
You need to use jq 'keys[]'. For example:
echo '{"example1" : 1, "example2" : 2, "example3" : 3}' | jq 'keys[]'
Will output a line separated list:
"example1"
"example2"
"example3"
In combination with the above answer, you want to ask jq for raw output, so your last filter should be eg.:
cat input.json | jq -r 'keys'
From jq help:
-r output raw strings, not JSON texts;
To print keys on one line as csv:
echo '{"b":"2","a":"1"}' | jq -r 'keys | [ .[] | tostring ] | #csv'
Output:
"a","b"
For csv completeness ... to print values on one line as csv:
echo '{"b":"2","a":"1"}' | jq -rS . | jq -r '. | [ .[] | tostring ] | #csv'
Output:
"1","2"
If your input is an array of objects,
[
{
"a01" : { "name" : "A", "user" : "B" }
},
{
"a02" : { "name" : "C", "user" : "D" }
}
]
try with:
jq '.[] | keys[]'
Oddly enough, the accepted answer doesn’t actually answer the Q exactly, so for reference, here is a solution that does:
$ jq -r 'keys_unsorted[]' file.json
echo '{"ab": 1, "cd": 2}' | jq -r 'keys[]' prints all keys one key per line without quotes.
ab
cd
Here's another way of getting a Bash array with the example JSON given by #anubhava in his answer:
arr=($(jq --raw-output 'keys_unsorted | #sh' file.json))
echo ${arr[0]} # 'Archiver-Version'
echo ${arr[1]} # 'Build-Id'
echo ${arr[2]} # 'Build-Jdk'