How to create a json file with jq - json

Now I am trying to make a json file.
I found an example which is with jq.
echo "$(jq -n '{Test1: $ARGS.named}' \
--arg one 'Apple' \
--arg two 'Banana')" >> config.json
I can get the result and save it into config.json
{
"Test1": {
"one": "Apple",
"two": "Banana"
}
}
Now, how to make the following result and save it.
{
"Test1": {
"one": "Apple",
"two": "Banana"
},
"Test2": {
"one": "Kiwi",
"two": "Tomato"
}
}
Thanks

Create a JSON file:
$ jq -n --arg one 'Apple' --arg two 'Banana' \
'{Test1: $ARGS.named}' > config.json
View the JSON file:
$ cat config.json
{
"Test1": {
"one": "Apple",
"two": "Banana"
}
}
Create another JSON file based on the first one (using jq's . for the input object, and + to add (merge) two objects):
$ jq --arg one 'Kiwi' --arg two 'Tomato' \
'. + {Test2: $ARGS.named}' config.json > config2.json
View that other JSON file:
$ cat config2.json
{
"Test1": {
"one": "Apple",
"two": "Banana"
},
"Test2": {
"one": "Kiwi",
"two": "Tomato"
}
}
Overwrite the first one with the second one:
$ mv config2.json config.json
Now the first one has the content of the second one:
$ cat config.json
{
"Test1": {
"one": "Apple",
"two": "Banana"
},
"Test2": {
"one": "Kiwi",
"two": "Tomato"
}
}

Without creating a temporary file, assuming your shell is POSIX-based:
{
jq -cn --arg one 'Apple' --arg two 'Banana' '{Test1: $ARGS.named}'
jq -cn --arg one 'Kiwi' --arg two 'Tomato' '{Test2: $ARGS.named}'
} | jq -s add > output.json
Or
jq -n \
--argjson Test1 "$(jq -n --arg one 'Apple' --arg two 'Banana' '$ARGS.named')" \
--argjson Test2 "$(jq -n --arg one 'Kiwi' --arg two 'Tomato' '$ARGS.named')" \
'$ARGS.named' > output.json
If this is a prelude to a list of "TestN" objects, we'll get a bit more programmitic, here with bash syntax:
ones=( Apple Kiwi )
twos=( Banana Tomato )
for ((i=0; i < ${#ones[#]}; i++)); do
jq -cn \
--arg one "${ones[i]}" \
--arg two "${twos[i]}" \
--arg key "Test$((i + 1))" \
'{($key): {$one, $two}}'
done | jq -s add > output.json
Or, use a different tool: jo
jo Test1="$(jo one=Apple two=Banana)" Test2="$(jo one=Kiwi two=Tomato)"
{"Test1":{"one":"Apple","two":"Banana"},"Test2":{"one":"Kiwi","two":"Tomato"}}

Related

Construct json through jq tool in shell script loop

json can be created using the following command.
jq -n \
--arg v1 "Value1" \
--arg v2 "Value2" \
'{k1: "$v1", k2:$v2'}
But when my key is mutable, how should I loop? For example, the script I execute is
test.sh k1=v1 k2=v2 k3=v3
test.sh is as follows
index=1
while ((index <= "$#")); do
data_i_key=$(echo ${!index} | awk -F "=" '{print $1}')
data_i_value=$(echo ${!index} | awk -F "=" '{print $2}')
let index++
JSON_STRING=$(jq -n \
--arg value "$data_i_value" \
'{'"$data_i_key"': $value'})
echo $JSON_STRING
The above print result is
{ "K3": "V3" }
if I replace it with
JSON_STRING+=$(jq -n \
--arg val_value "$dataValue" \
'{'"$data_i_key"': $val_value'})
The print result is
{ "k1": "v1" }{ "k2": "v2" }{ "K3": "V3" }
The above two methods have not achieved the goal, do you have a good way to deal with it? My desired output is
{ "k1": "v1" , "k2": "v2" ,"K3": "V3" }
hope you can help me.
I'd suggest a totally different, but simpler, approach:
for kv; do
echo "$kv" | jq -R './"=" | {key:first,value:last}'
done | jq -s 'from_entries'
It builds {key: …, value: …} objects from your positional parameters (splitting them by the equal sign) and then slurping all those objects in a second jq process, converting them into a single object via from_entries.
Alternatively, using -n (--null-input), --args and then accessing via $ARGS.positional. This avoids the loop and the second jq call altogether.
jq -n '$ARGS.positional | map(./"=" | {key:first,value:last}) | from_entries' --args "$#"
If your values can contain = themselves, you have to join all but the first value of the array:
jq -n '$ARGS.positional
| map(./"=" | {key:first,value:.[1:]|join("=")})
| from_entries' --args "$#"
Use parentheses around the key expression to have it evaluated:
jq -n --arg k1 "Key1" --arg v1 "Value1" '{($k1): $v1}'
{
"Key1": "Value1"
}
Perhaps this, which will handle an arbitrary number of key=value arguments:
#!/usr/bin/env bash
args=()
for arg; do
IFS="=" read -r k v <<<"$arg"
args+=("$k" "$v")
done
jq -n --args '
$ARGS.positional as $a
| reduce range(0; $a | length; 2) as $i ({}; .[$a[$i]] = $a[$i + 1])
' "${args[#]}"
Produces:
$ ./test.sh k1=v1 k2=v2 k3=v3
{
"k1": "v1",
"k2": "v2",
"k3": "v3"
}

Compose dynamic JSON file with arguments

I found that using jq I can compose a dynamic JSON file with arguments.
For example profile.jq:
{
"name": $name,
"age": $age,
"secretIdentity": $id,
"powers":{
"sid": $something
}
}
jq -n --arg name "FL" \
--arg age 60 \
--arg id 1234 \
--something "ss" \
-f profile.jq > out.json
so I can get a json file out.json
{
"name": "FL",
"age": 60,
"secretIdentity": 1234,
"powers":{
"sid": "ss"
}
}
However, there are many args, is there any easier way than passing each arg using --arg? Like, is that possible to give all args in a JSON file, something like:
somecommand --argfile arg-file.json --inputfile template-json.json
PS: I noticed that jq has an option --args, but I couldn't find any example of how to use it.

generate JSON out of command line arguments

I want to create JSON output with jq that looks like this:
{
"records": [
{
"id": "1234",
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
I assumed I have to twiddle with the "filter" of jq whose concept I don't fully get after reading the doc.
This is what I got so far:
$ jq --arg id 1234 \
--arg song Yesterday \
--arg artist "The Beatles" \
'.' \
<<<'{ "records" : [{ "id":"$id", "song":"$song", "artist":"$artist" }] }'
which prints
{
"records": [
{
"id" : "$id",
"song" : "$song",
"artist" : "$artist"
}
]
}
Do I modify the filter? Do I change the input?
An alternate way to your original attempt, on jq-1.6 you can use the $ARGS.positional attribute to construct your JSON from scratch
jq -n '
$ARGS.positional | {
records: [
{
id: .[0],
song: .[1],
artist: .[2]
}
]
}' --args 1234 Yesterday "The Beatles"
As for why your original attempt didn't work, looks you are not modifying your json at all, with your filter '.' you are basically just reading in and printing out "untouched". The arguments set using --arg need to be set to the object inside the filter.
You are looking for something like this:
jq --null-input \
--arg id 1234 \
--arg song Yesterday \
--arg artist "The Beatles" \
'.records[0] = {$id, $song, $artist}'
Each variable reference between curly brackets is converted to a key-value pair where its name is the key, and its value is the value. And assigning the resulting object to .records[0] forces the creation of the surrounding structure.
jq --null-input\
--argjson id 1234\
--arg song Yesterday\
--arg artist "The Beatles"\
'{ "records" : [{ $id, $song, $artist }] }'
gives
{
"records": [
{
"id": 1234,
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
I think you got the JSON/JQ the wrong way round:
This should be your JQ script:
rec.jq
{
records: [
{
id: $id,
song: $song,
artist: $artist
}
]
}
And this should be your JSON (empty):
rec.json
{}
Then:
jq --arg id 123 --arg song "Yesterday" --arg artist "The Beatles" -f rec.jq rec.json
Which produces:
{
"records": [
{
"id": "123",
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
Start with an empty JSON and add the missing bits:
$ jq --arg id 1234 \
--arg song Yesterday \
--arg artist "The Beatles" \
'. | .records[0].id=$id | .records[0].song=$song | .records[0].artist=$artist' \
<<<'{}'
Outputs
{
"records": [
{
"id": "1234",
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
Another, cleaner, approach based on the answer of #Inian could be
jq -n \
--arg id 1234
--arg song Yesterday
--arg artist "The Beatles"
'{records: [{id:$id, song:$song, artist:$artist}]}'

JQ Join JSON files by key

Looks like it's not actual for jq 1.4, could you provide any other ways to join JSON files by key?
e.g
{
"key": "874102296-1",
"que_lat": "40"
}
{
"key": "874102296-2",
"que_lat": "406790"
}
and
{
"key": "874102296-1",
"in_time": "1530874104733",
"latency": "12864258288242"
}
{
"key": "874102296-2",
"in_time": "1530874104746"
}
As a result, i'd like to have something like this:
{
"key": "874102296-1",
"in_time": "1530874104733",
"full_latency": "12864258288242",
"que_lat": "40"
}
{
"key": "874102296-2",
"in_time": "1530874104746",
"que_lat": "406790"
}
Thanks!
The problem can easily be solved using the def of hashJoin given in the SO page that you cite.
Solution using jq 1.5 or higher
If you have jq 1.5 or higher, you could use this invocation:
jq -n --slurpfile f1 file1.json --slurpfile f2 file2.json -f join.jq
where join.jq contains the second def of hashJoin, together with:
hashJoin($f1; $f2; .key)[]
Solution using jq 1.4
If you have jq 1.4, the trickiness is to read each of the two files separately as an array. Here's one approach that assumes bash:
jq -n --argfile f1 <(jq -s . file1.json) --argfile f2 <(jq -s . file2.json) -f join.jq
where join.jq is as above.
If you cannot use bash, then it might be simplest to create temporary files.

Create JSON file using jq

I'm trying to create a JSON file by executing the following command:
jq --arg greeting world '{"hello":"$greeting"}' > file.json
This command stuck without any input. While
jq -n --arg greeting world '{"hello":"$greeting"}' > file.json
doesn't parse correctly. I'm just wondering is really possible to create a JSON file.
So your code doesn't work because included the variable inside double quotes which gets treated as string. That is why it is not working
As #Jeff Mercado, pointed out the solution is
jq -n --arg greeting world '{"hello":$greeting}' > file.json
About the - in a name. This is actually possible. But as of now this is not available in released version of jq. If you compile the master branch of jq on your system. There is a new variable called $ARGS.named which can be used to access the information.
I just compiled and check the below command and it works like a charm
./jq -n --arg name-is tarun '{"name": $ARGS.named["name-is"]}'
{
"name": "tarun"
}
$ARGS provides access to named (--arg name value) and positional (--args one two three) arguments from the jq command line, and allows you to build up objects easily & safely.
Named arguments:
$ jq -n '{christmas: $ARGS.named}' \
--arg one 'partridge in a "pear" tree' \
--arg two 'turtle doves'
{
"christmas": {
"one": "partridge in a \"pear\" tree",
"two": "turtle doves"
}
}
Positional arguments:
$ jq -n '{numbers: $ARGS.positional}' --args 1 2 3
{
"numbers": [
"1",
"2",
"3"
]
}
Note you can access individual items of the positional array, and that the named arguments are directly available as variables:
jq -n '{first: {name: $one, count: $ARGS.positional[0]}, all: $ARGS}' \
--arg one 'partridge in a "pear" tree' \
--arg two 'turtle doves' \
--args 1 2 3
{
"first": {
"name": "partridge in a \"pear\" tree",
"count": "1"
},
"all": {
"positional": [
"1",
"2",
"3"
],
"named": {
"one": "partridge in a \"pear\" tree",
"two": "turtle doves"
}
}
}
To add to what Jeff and Tarun have already said, you might want to use the \() string interpolation syntax in your command. eg.
jq -n --arg greeting world '{"hello":"\($greeting)"}'
for me this produces
{
"hello": "world"
}
Regarding your reply to Jeff's comment, the argument name you choose has to be a valid jq variable name so an arg like greeting-for-you won't work but you could use underscores so greeting_for_you would be ok. Or you could use the version Tarun described.