Create JSON with jq - json

I am trying to create a JSON file with jq from the result of the command "lsb_release"
What i have done :
if [ -x "$(command -v lsb_release)" ]; then
lsb_release -a | jq --raw-input 'split("\t") | { (.[0]) : .[1] }' > ubuntu_release.json
fi
the result is
{
"Distributor ID:": "Ubuntu"
}
{
"Description:": "Ubuntu 20.04.3 LTS"
}
{
"Release:": "20.04"
}
{
"Codename:": "focal"
}
but i want the result
[
{
"Distributor ID:": "Ubuntu"
},
{
"Description:": "Ubuntu 20.04.3 LTS"
},
{
"Release:": "20.04"
},
{
"Codename:": "focal"
}
]
can anybody body help me ? :)

Usually, when we want to create an array from a stream of inputs, we can use --slurp/-s. But when combined with --raw-input/-R, this causes the entire input to be provided as a single string (that contains line feeds).
Slurping can also be achieved using --null-input/-n and [ inputs | ... ]. And this works as desired for text files.
jq -nR '[ inputs | split("\t") | { (.[0]) : .[1] } ]'
Demo on jqplay
That said, I suspect you will find the following output format more useful:
{
"Distributor ID": "Ubuntu",
"Description": "Ubuntu 20.04.3 LTS",
"Release": "20.04",
"Codename": "focal"
}
This can be achieved by simply adding | add.
jq -nR '[ inputs | split(":\t") | { (.[0]) : .[1] } ] | add'
Demo on jqplay
One can also use reduce.
jq -nR 'reduce ( inputs | split(":\t") ) as [ $k, $v ] ( {}; . + { ($k): $v } )'
Demo on jqplay

Filter
reduce (inputs / ":\t") as [$key, $value] ({}; .+{($key): $value})
Input
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal
Output
{
"Distributor ID": "Ubuntu",
"Description": "Ubuntu 20.04.3 LTS",
"Release": "20.04",
"Codename": "focal"
}
Note that each line of $key and $value from inputs is processed and combined by reduce.
Demo
https://jqplay.org/s/ZBvKf6vQ0F

Related

Append multiple dummy objects to json array with jq

Lets say this is my array :
[
{
"name": "Matias",
"age": "33"
}
]
I can do this :
echo "$response" | jq '[ .[] | select(.name | test("M.*"))] | . += [.[]]'
And it will output :
[
{
"name": "Matias",
"age": "33"
},
{
"name": "Matias",
"age": "33"
}
]
But I cant do this :
echo "$response" | jq '[ .[] | select(.name | test("M.*"))] | . += [.[] * 3]'
jq: error (at <stdin>:7): object ({"name":"Ma...) and number (3) cannot be multiplied
I need to extend an array to create a dummy array with 100 values. And I cant do it. Also, I would like to have a random age on the objects. ( So later on I can filter the file to measure performance of an app .
Currently jq does not have a built-in randomization function, but it's easy enough to generate random numbers that jq can use. The following solution uses awk but in a way that some other PRNG can easily be used.
#!/bin/bash
function template {
cat<<EOF
[
{
"name": "Matias",
"age": "33"
}
]
EOF
}
function randoms {
awk -v n=$1 'BEGIN { for(i=0;i<n;i++) {print int(100*rand())} }'
}
randoms 100 | jq -n --argfile template <(template) '
first($template[] | select(.name | test("M.*"))) as $t
| [ $t | .age = inputs]
'
Note on performance
Even though the above uses awk and jq together, this combination is about 10 times faster than the posted jtc solution using -eu:
jq+awk: u+s = 0.012s
jtc with -eu: u+s = 0.192s
Using jtc in conjunction with awk as above, however, gives u+s == 0.008s on the same machine.

Using jq with 'contains' in a 'select' inside a 'del' is not working

i try to remove some entries from a dict in a json. It works by using == but with contains it doesn't work.
Jq call working:
jq 'del(.entries[] | select(.var == "foo"))' input.json
Jq call not working:
jq 'del(.entries[] | select(.var | contains("foo")))' input.json
input.json:
{
"entries": [
{
"name": "test1",
"var": "foo"
},
{
"name": "test2",
"var": "bar"
}
]
}
Output:
{
"entries": [
{
"name": "test2",
"var": "bar"
}
]
}
The result of jq '.entries[] | select(.var == "foo")' input.json and jq '.entries[] | select(.var | contains("foo"))' input.json is the same, so I think the two del-calls should also work.
Is this a bug in jq or did I something wrong?
This must be a bug as it seems to work perfectly on jq 1.6 (try it here).
If you're unable to update to jq 1.6 you should be able to use the following command instead, which I've successfully tested on jq 1.5 :
jq '.entries |= map(select(.var | contains("foo") | not))' file.json

How to merge objects with InstanceId unique in bash shell?

I have two json files as below:
I wanna merge objects in tmp1.json and tmp2.json with InstanceId unique value in bash shell.
I have tried jq with argjson option but my jq 1.4 version not support this option. Sorry, I unable update jq to 1.5 version.
#cat tmp1.json
{
"VolumeId": "vol-046e0be08ac95095a",
"Instances": [
{
"InstanceId": "i-020ce1b2ad08fa6bd"
}
]
}
{
"VolumeId": "vol-007253a7d24c1c668",
"Instances": [
{
"InstanceId": "i-0c0650c15b099b993"
}
]
}
#cat tmp2.json
{
"InstanceId": "i-0c0650c15b099b993",
"InstanceName": "Test1"
}
{
"InstanceId": "i-020ce1b2ad08fa6bd",
"InstanceName": "Test"
}
My desired is:
{
"VolumeId": "vol-046e0be08ac95095a",
"Instances": [
{
"InstanceId": "i-020ce1b2ad08fa6bd"
"InstanceName": "Test"
}
]
}
{
"VolumeId": "vol-007253a7d24c1c668",
"Instances": [
{
"InstanceId": "i-0c0650c15b099b993"
"InstanceName": "Test1"
}
]
}
#!/bin/bash
JQ=jq-1.4
# For ease of understanding, the following is a bit more verbose than
# necessary.
# One way to get around the constraints of using jq 1.4 is
# to use the "slurp" option so that the contents of the two files can
# be kept separately.
# Note that jq 1.6 includes the following def of INDEX, but we can use it with jq 1.4.
($JQ -s . tmp1.json ; $JQ -s . tmp2.json) | $JQ -s '
def INDEX(stream; idx_expr):
reduce stream as $row ({};
.[$row|idx_expr|
if type != "string" then tojson
else .
end] |= $row);
.[0] as $tmp1
| .[1] as $tmp2
| INDEX($tmp2[]; .InstanceId) as $dict
| $tmp1
| map( .Instances |= map(.InstanceName = $dict[.InstanceId].InstanceName))
| .[]
'
Streamlined
INDEX(.[1][]; .InstanceId) as $dict
| .[0][]
| .Instances |= map(.InstanceName = $dict[.InstanceId].InstanceName)
minify the two json files
try the following command:
cat tmp2.json|jq -r '"\(.InstanceId) \(.InstanceName)"'|xargs -n2 sh -c 'cat tmp1.json|jq "if .Instances[0].InstanceId==\"$0\" then .Instances[0].InstanceName=\"$1\" else empty end"'
Here is the output:
{
"VolumeId": "vol-007253a7d24c1c668",
"Instances": [
{
"InstanceId": "i-0c0650c15b099b993",
"InstanceName": "Test1"
}
]
}
{
"VolumeId": "vol-046e0be08ac95095a",
"Instances": [
{
"InstanceId": "i-020ce1b2ad08fa6bd",
"InstanceName": "Test"
}
]
}

Convert .txt to json using Shell scripting

I have a text file data.txt:
Framework1,Version1
Framework2,Version2
Framework3,Version3
I need to convert it to data.json which would look like:
[
{"FrameworkName":"Framework1", "VersionName":"Version1"},
{"FrameworkName":"Framework3", "VersionName":"Version2"},
{"FrameworkName":"Framework3", "VersionName":"Version3"}
]
I have tried using awk but it hasn't helped me much. Any help would be appreciated.
jq solution:
jq -Rs '[ split("\n")[] | select(length > 0)
| split(",") | {FrameworkName: .[0], VersionName: .[1]} ]' data.txt
The output:
[
{
"FrameworkName": "Framework1",
"VersionName": "Version1"
},
{
"FrameworkName": "Framework2",
"VersionName": "Version2"
},
{
"FrameworkName": "Framework3",
"VersionName": "Version3"
}
]
https://stedolan.github.io/jq/manual/v1.5/

How to capture values from JSON parent and sub-parent parameter using JQ?

I have this below JSON file which I need to capture using JQ, but so far I only manage to capture the parent parameter (SUBSCRIBER_ID), but unable to capture the sub-parent parameter which is "Offer".
Need your guys help on providing a correct JQ filter to capture both "SUBSCRIBER_ID" and "Offer" value.
JSON
{"Data1": [
{"Data2": {
"SUBSCRIBER_ID" : "999050280010099",
"MSISDN" : "999050280010099",
"EMAIL" : "john#email.com",
"OFFERS" : [
{
"Offer" : 12344,
"EffectiveDate" : "1488787236",
"ExpiryDate" : "4070869200"
} ],
"IsGroup" : "false",
}}
]}
My JQ Filter which is not working
'.Data1 | .[] | .Data2 | to_entries | map(.value) | #csv' -r
Expected output:
SUBSCRIBER_ID,Offer
999050280010099,12344
You can try this jq:
jq -r '.Data1|.[]|.Data2|[.SUBSCRIBER_ID, .OFFERS[].Offer]|#csv' file > out.csv
(OR) As suggested by #peak,
jq -r '.Data1[].Data2|[.SUBSCRIBER_ID, .OFFERS[].Offer]|#csv' file
Another one method:
jq -r '.[]|.[]|map([.SUBSCRIBER_ID, .OFFERS[].Offer])|.[]|#csv' file
Input:
$ cat file.json
{
"Data1": [
{
"Data2": {
"SUBSCRIBER_ID": "999050280010099",
"MSISDN": "999050280010099",
"EMAIL": "john#email.com",
"OFFERS": [
{
"Offer": 12344,
"EffectiveDate": "1488787236",
"ExpiryDate": "4070869200"
}
],
"IsGroup": "false"
}
}
]
}
Test:
$ jq -r '.Data1|.[]|.Data2|[.SUBSCRIBER_ID, .OFFERS[].Offer]|#csv' file.json
"999050280010099",12344
$ jq -r '.[]|.[]|map([.SUBSCRIBER_ID, .OFFERS[].Offer])|.[]|#csv' file.json
"999050280010099",12344