I want to use a "prettified" JSON with a bash cat argument. Due to its used with another function I need to have it "un-prettified" (all white spaces and linebreaks removed). Is there a simple way to do it?
To give you a simple example its like this:
Working
file.json:
{ "name":"John", "age":30, "city":"New York"}
foo.sh:
#!/bin/sh
myfunc cat file.json //..
Not Working
file.json:
{
"name":"John",
"age":30,
"city":"New York"
}
foo.sh:
#!/bin/sh
myfunc cat file.json //..
Any suggestions?
You could use jq with -c parameter
jq -c '.' file.json
will print
{"name":"John","age":30,"city":"New York"}
Probably jq could help you.
file.json
{
"name": "John",
"age": 30,
"city": "New York"
}
Do:
cat file.json | jq -c
Will return:
{"name":"John","age":30,"city":"New York"}
Even better as #Charles Duffy comment to only spin up one process, not two:
jq -c < file.json
Related
I have a json content (output.json)
{"project": {"id": "A", "content": [{"name": "XYZ", "location": "Berlin", "comments":""}, {"name": "ABC", "location": "NewYork", "comments": "Hwllo"}, {"name": "DEF", "location": "Paris", "comments": "Success"}]}}
I would like to extract location key with value when name matches say ABC from the above json using bash or shell commands
I tried something like below which gives be content within curly braces. but not sure on searching specific key.
cat output.json | grep -o -e "{.*}"
Output expectations:
if name matches ABC, get output as "location":"NewYork"
Any suggestions on processing further?
For extracting from json you should use jq if you can. According to authors "jq is like sed for JSON data" (source).
In your case it should be:
$ jq -r '.project' output.json | jq -r '.content' | jq '.[] | select(.name=="ABC")' | jq -r '.location'
Output will be:
NewYork
To get output which you required so:
"location":"NewYork"
You can use:
echo "\"location\":$(jq -r '.project' output.json | jq -r '.content' | jq '.[] | select(.name=="ABC")' | jq '.location')"
Before you use jq you should install it on Debian and Ubuntu it will be:
$ sudo apt install jq
for other OS you should check this site.
There may be better ways to do it, in a quick twisted way is here.
cat output.json | sed 's/"name"/\n"name"/g' | grep '"name"' | awk -F',' '{print $2}'
Add | grep <preferred name> also if need to filter based on name.
Use Perl
$ perl -0777 -lne ' while(/"name":\s+"ABC",\s+"location":\s+(\S+)/msg) { print "$1\n" } ' output.json
"NewYork",
$ cat output.json
{"project": {"id": "A", "content": [{"name": "XYZ", "location": "Berlin", "comments":""}, {"name": "ABC", "location": "NewYork", "comments": "Hwllo"}, {"name": "DEF", "location": "Paris", "comments": "Success"}]}}
$
Please use a JSON parser for processing JSON.
With Xidel it's as simple as:
xidel -s output.json -e '($json//content)()[name="ABC"]/location'
Alternatively:
xidel -s output.json -e '$json/(.//content)()[name="ABC"]/location'
or in full:
xidel -s output.json -e '$json/project/(content)()[name="ABC"]/location'
The above is XPath notation (example #11, Reading JSON). Dot notation (like jq) is also possible:
xidel -s output.json -e '($json).project.content()[name="ABC"].location'
[edit]
Commands above put out NewYork and I just realized you require the output to be "location":"NewYork". Xidel can do that too:
xidel -s output.json -e '($json//content)()[name="ABC"]/concat("""location"":""",location,"""")'
[/edit]
I have this JSON:
{
"blocks" : {
"xrb_1111111111111111111111111111111111111111111111111117353trpda": {
"142A538F36833D1CC78B94E11C766F75818F8B940771335C6C1B8AB880C5BB1D": "6000000000000000000000000000000"
},
"xrb_3t6k35gi95xu6tergt6p69ck76ogmitsa8mnijtpxm9fkcm736xtoncuohr3": {
"4C1FEEF0BEA7F50BE35489A1233FE002B212DEA554B55B1B470D78BD8F210C74": "106370018000000000000000000000000"
}
}
I am having trouble using jq in bash to read out:
xrb_1111111111111111111111111111111111111111111111111117353trpda
xrb_3t6k35gi95xu6tergt6p69ck76ogmitsa8mnijtpxm9fkcm736xtoncuohr3
I try to do it on jqplay but not having much luck with it.
I have tried with jqplay to get the results and manage find but not when its nested this way.
This jq script might work for you:
$ cat file
{
"blocks": {
"foo_b": {
"aaa": "bbb"
},
"bar_b": {
"ccc": "ddd"
}
}
}
$ jq -r '.blocks | to_entries[].key' file
foo_b
bar_b
keys_unsorted gives an array of keys. Whence:
jq -r '.blocks | keys_unsorted[]' input.json
xrb_1111111111111111111111111111111111111111111111111117353trpda
xrb_3t6k35gi95xu6tergt6p69ck76ogmitsa8mnijtpxm9fkcm736xtoncuohr3
First of all, your JSON is invalid. It's missing a closing bracket.
$ cat input.json
{
"blocks": {
"xrb_1111111111111111111111111111111111111111111111111117353trpda": {
"142A538F36833D1CC78B94E11C766F75818F8B940771335C6C1B8AB880C5BB1D": "6000000000000000000000000000000"
},
"xrb_3t6k35gi95xu6tergt6p69ck76ogmitsa8mnijtpxm9fkcm736xtoncuohr3": {
"4C1FEEF0BEA7F50BE35489A1233FE002B212DEA554B55B1B470D78BD8F210C74": "106370018000000000000000000000000"
}
}
}
If you're open to use something else than jq, then I'd suggest the Swiss knife tool Xidel.
Dot notation:
$ xidel -s input.json -e '($json).blocks()'
XPath notation:
$ xidel -s input.json -e '$json/(blocks)()'
Both with the output:
xrb_1111111111111111111111111111111111111111111111111117353trpda
xrb_3t6k35gi95xu6tergt6p69ck76ogmitsa8mnijtpxm9fkcm736xtoncuohr3
Looks like it's not actual for jq 1.4, could you provide any other ways to join JSON files by key?
e.g
{
"key": "874102296-1",
"que_lat": "40"
}
{
"key": "874102296-2",
"que_lat": "406790"
}
and
{
"key": "874102296-1",
"in_time": "1530874104733",
"latency": "12864258288242"
}
{
"key": "874102296-2",
"in_time": "1530874104746"
}
As a result, i'd like to have something like this:
{
"key": "874102296-1",
"in_time": "1530874104733",
"full_latency": "12864258288242",
"que_lat": "40"
}
{
"key": "874102296-2",
"in_time": "1530874104746",
"que_lat": "406790"
}
Thanks!
The problem can easily be solved using the def of hashJoin given in the SO page that you cite.
Solution using jq 1.5 or higher
If you have jq 1.5 or higher, you could use this invocation:
jq -n --slurpfile f1 file1.json --slurpfile f2 file2.json -f join.jq
where join.jq contains the second def of hashJoin, together with:
hashJoin($f1; $f2; .key)[]
Solution using jq 1.4
If you have jq 1.4, the trickiness is to read each of the two files separately as an array. Here's one approach that assumes bash:
jq -n --argfile f1 <(jq -s . file1.json) --argfile f2 <(jq -s . file2.json) -f join.jq
where join.jq is as above.
If you cannot use bash, then it might be simplest to create temporary files.
I'm trying to create a JSON file by executing the following command:
jq --arg greeting world '{"hello":"$greeting"}' > file.json
This command stuck without any input. While
jq -n --arg greeting world '{"hello":"$greeting"}' > file.json
doesn't parse correctly. I'm just wondering is really possible to create a JSON file.
So your code doesn't work because included the variable inside double quotes which gets treated as string. That is why it is not working
As #Jeff Mercado, pointed out the solution is
jq -n --arg greeting world '{"hello":$greeting}' > file.json
About the - in a name. This is actually possible. But as of now this is not available in released version of jq. If you compile the master branch of jq on your system. There is a new variable called $ARGS.named which can be used to access the information.
I just compiled and check the below command and it works like a charm
./jq -n --arg name-is tarun '{"name": $ARGS.named["name-is"]}'
{
"name": "tarun"
}
$ARGS provides access to named (--arg name value) and positional (--args one two three) arguments from the jq command line, and allows you to build up objects easily & safely.
Named arguments:
$ jq -n '{christmas: $ARGS.named}' \
--arg one 'partridge in a "pear" tree' \
--arg two 'turtle doves'
{
"christmas": {
"one": "partridge in a \"pear\" tree",
"two": "turtle doves"
}
}
Positional arguments:
$ jq -n '{numbers: $ARGS.positional}' --args 1 2 3
{
"numbers": [
"1",
"2",
"3"
]
}
Note you can access individual items of the positional array, and that the named arguments are directly available as variables:
jq -n '{first: {name: $one, count: $ARGS.positional[0]}, all: $ARGS}' \
--arg one 'partridge in a "pear" tree' \
--arg two 'turtle doves' \
--args 1 2 3
{
"first": {
"name": "partridge in a \"pear\" tree",
"count": "1"
},
"all": {
"positional": [
"1",
"2",
"3"
],
"named": {
"one": "partridge in a \"pear\" tree",
"two": "turtle doves"
}
}
}
To add to what Jeff and Tarun have already said, you might want to use the \() string interpolation syntax in your command. eg.
jq -n --arg greeting world '{"hello":"\($greeting)"}'
for me this produces
{
"hello": "world"
}
Regarding your reply to Jeff's comment, the argument name you choose has to be a valid jq variable name so an arg like greeting-for-you won't work but you could use underscores so greeting_for_you would be ok. Or you could use the version Tarun described.
Here is my json:
[
{
"ReferringUrl": "N",
"OpenAccess": "0",
"ItmId": "1694738780"
},
{
"ReferringUrl": "L",
"OpenAccess": "1",
"ItmId": "1347809133"
}
]
I want it back to the original json format:
[{"ReferringUrl": "N","OpenAccess": "0","ItmId": "1694738780"},{"ReferringUrl": "L","OpenAccess": "1","ItmId": "1347809133"}]
How to use jq to do this? :) Thank you!
Just use the --compact-output / -c option:
cat file | jq -c
(or, without feline abuse: jq -c '.' file)