Convert .txt to json using Shell scripting - json

I have a text file data.txt:
Framework1,Version1
Framework2,Version2
Framework3,Version3
I need to convert it to data.json which would look like:
[
{"FrameworkName":"Framework1", "VersionName":"Version1"},
{"FrameworkName":"Framework3", "VersionName":"Version2"},
{"FrameworkName":"Framework3", "VersionName":"Version3"}
]
I have tried using awk but it hasn't helped me much. Any help would be appreciated.


jq solution:
jq -Rs '[ split("\n")[] | select(length > 0)
| split(",") | {FrameworkName: .[0], VersionName: .[1]} ]' data.txt
The output:
[
{
"FrameworkName": "Framework1",
"VersionName": "Version1"
},
{
"FrameworkName": "Framework2",
"VersionName": "Version2"
},
{
"FrameworkName": "Framework3",
"VersionName": "Version3"
}
]
https://stedolan.github.io/jq/manual/v1.5/

Related

BASH/Linux - How do you parse JSON with jq when array doesn't have any keys? [duplicate]

curl http://testhost.test.com:8080/application/app/version | jq '.version' | jq '.[]'
The above command outputs only the values as below:
"madireddy#test.com"
"2323"
"test"
"02-03-2014-13:41"
"application"
How can I get the key names instead like the below:
email
versionID
context
date
versionName

To get the keys in the order they appear in the original JSON use:
jq 'keys_unsorted' file.json
If you want the keys sorted alphanumerically, you can use:
jq 'keys' file.json
Complete example
$ cat file.json
{ "Created-By" : "Apache Maven", "Build-Number" : "", "Archiver-Version" : "Plexus Archiver", "Build-Id" : "", "Build-Tag" : "", "Built-By" : "cporter"}
$ jq 'keys_unsorted' file.json
[
"Created-By",
"Build-Number",
"Archiver-Version",
"Build-Id",
"Build-Tag",
"Built-By"
]
$ jq 'keys' file.json
[
"Archiver-Version",
"Build-Id",
"Build-Number",
"Build-Tag",
"Built-By",
"Created-By"
]

To get the keys on a deeper node in a JSON:
echo '{"data": "1", "user": { "name": 2, "phone": 3 } }' | jq '.user | keys[]'
"name"
"phone"

You need to use jq 'keys[]'. For example:
echo '{"example1" : 1, "example2" : 2, "example3" : 3}' | jq 'keys[]'
Will output a line separated list:
"example1"
"example2"
"example3"

In combination with the above answer, you want to ask jq for raw output, so your last filter should be eg.:
cat input.json | jq -r 'keys'
From jq help:
-r output raw strings, not JSON texts;

To print keys on one line as csv:
echo '{"b":"2","a":"1"}' | jq -r 'keys | [ .[] | tostring ] | #csv'
Output:
"a","b"
For csv completeness ... to print values on one line as csv:
echo '{"b":"2","a":"1"}' | jq -rS . | jq -r '. | [ .[] | tostring ] | #csv'
Output:
"1","2"

If your input is an array of objects,
[
{
"a01" : { "name" : "A", "user" : "B" }
},
{
"a02" : { "name" : "C", "user" : "D" }
}
]
try with:
jq '.[] | keys[]'

Oddly enough, the accepted answer doesn’t actually answer the Q exactly, so for reference, here is a solution that does:
$ jq -r 'keys_unsorted[]' file.json

echo '{"ab": 1, "cd": 2}' | jq -r 'keys[]' prints all keys one key per line without quotes.
ab
cd

Here's another way of getting a Bash array with the example JSON given by #anubhava in his answer:
arr=($(jq --raw-output 'keys_unsorted | #sh' file.json))
echo ${arr[0]} # 'Archiver-Version'
echo ${arr[1]} # 'Build-Id'
echo ${arr[2]} # 'Build-Jdk'

Transforming a list containing key/value strings with jq

So, I basically have a file test.json
[
"Name=TestName",
"Tag=TestTag"
]
Which I'd like to transform into
[
{
"ParameterKey": "Name",
"ParameterValue": "TestName",
},
{
"ParameterKey": "Tag",
"ParameterValue": "TestTag",
}
]
With jq. Any idea?

You don't need to use split() call twice but just once and access the results directly with the Array/Object Value Iterator: .[] and specifying the index inside
jq -n '[ inputs[] | split("=") | {ParameterKey: .[0], ParameterValue: .[1]} ]'

You can try JQ Play
I tried with the following jq. It should work as long as you are sure of the format of the array.
[.[] | {ParameterKey: split("=")[0], ParameterValue: split("=")[1]}]
If you are using from terminal, you can use the following option
cat test.json | jq '[.[] | {ParameterKey: split("=")[0], ParameterValue: split("=")[1]}]'

Using jq with 'contains' in a 'select' inside a 'del' is not working

i try to remove some entries from a dict in a json. It works by using == but with contains it doesn't work.
Jq call working:
jq 'del(.entries[] | select(.var == "foo"))' input.json
Jq call not working:
jq 'del(.entries[] | select(.var | contains("foo")))' input.json
input.json:
{
"entries": [
{
"name": "test1",
"var": "foo"
},
{
"name": "test2",
"var": "bar"
}
]
}
Output:
{
"entries": [
{
"name": "test2",
"var": "bar"
}
]
}
The result of jq '.entries[] | select(.var == "foo")' input.json and jq '.entries[] | select(.var | contains("foo"))' input.json is the same, so I think the two del-calls should also work.
Is this a bug in jq or did I something wrong?

This must be a bug as it seems to work perfectly on jq 1.6 (try it here).
If you're unable to update to jq 1.6 you should be able to use the following command instead, which I've successfully tested on jq 1.5 :
jq '.entries |= map(select(.var | contains("foo") | not))' file.json

How to capture values from JSON parent and sub-parent parameter using JQ?

I have this below JSON file which I need to capture using JQ, but so far I only manage to capture the parent parameter (SUBSCRIBER_ID), but unable to capture the sub-parent parameter which is "Offer".
Need your guys help on providing a correct JQ filter to capture both "SUBSCRIBER_ID" and "Offer" value.
JSON
{"Data1": [
{"Data2": {
"SUBSCRIBER_ID" : "999050280010099",
"MSISDN" : "999050280010099",
"EMAIL" : "john#email.com",
"OFFERS" : [
{
"Offer" : 12344,
"EffectiveDate" : "1488787236",
"ExpiryDate" : "4070869200"
} ],
"IsGroup" : "false",
}}
]}
My JQ Filter which is not working
'.Data1 | .[] | .Data2 | to_entries | map(.value) | #csv' -r
Expected output:
SUBSCRIBER_ID,Offer
999050280010099,12344

You can try this jq:
jq -r '.Data1|.[]|.Data2|[.SUBSCRIBER_ID, .OFFERS[].Offer]|#csv' file > out.csv
(OR) As suggested by #peak,
jq -r '.Data1[].Data2|[.SUBSCRIBER_ID, .OFFERS[].Offer]|#csv' file
Another one method:
jq -r '.[]|.[]|map([.SUBSCRIBER_ID, .OFFERS[].Offer])|.[]|#csv' file
Input:
$ cat file.json
{
"Data1": [
{
"Data2": {
"SUBSCRIBER_ID": "999050280010099",
"MSISDN": "999050280010099",
"EMAIL": "john#email.com",
"OFFERS": [
{
"Offer": 12344,
"EffectiveDate": "1488787236",
"ExpiryDate": "4070869200"
}
],
"IsGroup": "false"
}
}
]
}
Test:
$ jq -r '.Data1|.[]|.Data2|[.SUBSCRIBER_ID, .OFFERS[].Offer]|#csv' file.json
"999050280010099",12344
$ jq -r '.[]|.[]|map([.SUBSCRIBER_ID, .OFFERS[].Offer])|.[]|#csv' file.json
"999050280010099",12344

How to get key names from JSON using jq

curl http://testhost.test.com:8080/application/app/version | jq '.version' | jq '.[]'
The above command outputs only the values as below:
"madireddy#test.com"
"2323"
"test"
"02-03-2014-13:41"
"application"
How can I get the key names instead like the below:
email
versionID
context
date
versionName

To get the keys in the order they appear in the original JSON use:
jq 'keys_unsorted' file.json
If you want the keys sorted alphanumerically, you can use:
jq 'keys' file.json
Complete example
$ cat file.json
{ "Created-By" : "Apache Maven", "Build-Number" : "", "Archiver-Version" : "Plexus Archiver", "Build-Id" : "", "Build-Tag" : "", "Built-By" : "cporter"}
$ jq 'keys_unsorted' file.json
[
"Created-By",
"Build-Number",
"Archiver-Version",
"Build-Id",
"Build-Tag",
"Built-By"
]
$ jq 'keys' file.json
[
"Archiver-Version",
"Build-Id",
"Build-Number",
"Build-Tag",
"Built-By",
"Created-By"
]

To get the keys on a deeper node in a JSON:
echo '{"data": "1", "user": { "name": 2, "phone": 3 } }' | jq '.user | keys[]'
"name"
"phone"

You need to use jq 'keys[]'. For example:
echo '{"example1" : 1, "example2" : 2, "example3" : 3}' | jq 'keys[]'
Will output a line separated list:
"example1"
"example2"
"example3"

In combination with the above answer, you want to ask jq for raw output, so your last filter should be eg.:
cat input.json | jq -r 'keys'
From jq help:
-r output raw strings, not JSON texts;

To print keys on one line as csv:
echo '{"b":"2","a":"1"}' | jq -r 'keys | [ .[] | tostring ] | #csv'
Output:
"a","b"
For csv completeness ... to print values on one line as csv:
echo '{"b":"2","a":"1"}' | jq -rS . | jq -r '. | [ .[] | tostring ] | #csv'
Output:
"1","2"

If your input is an array of objects,
[
{
"a01" : { "name" : "A", "user" : "B" }
},
{
"a02" : { "name" : "C", "user" : "D" }
}
]
try with:
jq '.[] | keys[]'

Oddly enough, the accepted answer doesn’t actually answer the Q exactly, so for reference, here is a solution that does:
$ jq -r 'keys_unsorted[]' file.json

echo '{"ab": 1, "cd": 2}' | jq -r 'keys[]' prints all keys one key per line without quotes.
ab
cd

Here's another way of getting a Bash array with the example JSON given by #anubhava in his answer:
arr=($(jq --raw-output 'keys_unsorted | #sh' file.json))
echo ${arr[0]} # 'Archiver-Version'
echo ${arr[1]} # 'Build-Id'
echo ${arr[2]} # 'Build-Jdk'