How to query my JSON file for specific key - json

I am working on a project where I want to query a JSon using Jquery but I am getting an error:
jq: error: syntax error, unexpected IDENT, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
..id
jq: 1 compile error
Error: Process completed with exit code 3.
I am using the command:
jq '..id' new_file.json
To query my JSON file which is named new_file.json
Here is my JSON file:
[
{
"type": "users_export",
"status": "pending",
"connection_id": "con_kmZIjREZWYzt39JI",
"format": "json",
"limit": 5,
"fields": [
{
"name": "user_id"
},
{
"name": "user_id",
"export_as": "provider"
},
{
"name": "username"
},
{
"name": "username",
"export_as": "provider"
},
{
"name": "nickname"
},
{
"name": "email"
},
{
"name": "email"
},
{
"name": "identities[0].connection",
"export_as": "provider"
},
{
"name": "email_verified",
"export_as": "provider"
}
],
"connection": "dev-default-evoyanbs",
"created_at": "2022-11-16T17:45:58.429Z",
"id": "job_aztDgKXWT8g8iZ5T"
}
]
I want the 'job_aztDgKXWT8g8iZ5T' as my output but I am getting the above mentioned error, can someone please help me out, thanks.

As you have an array, the command should be:
jq '.[].id' new_file.json
And the result will be: "job_aztDgKXWT8g8iZ5T".
If you want to get rid of double quotes in your output, you can add raw-output flag in your command, like this:
jq -r '.[].id' new_file.json
or
jq --raw-output '.[].id' new_file.json

Related

Bash script - unable to replace string with double quotes and curly braces

I'm struggling for some time and I would need some help with the following operation.
I have a JSON file and I would like to replace a string with something a bit more complex.
This is a snippet of my json file:
{ "AWS679f53fac002430cb0da5b7982bd22872D164C4C": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "hnb659fds-assets-xxccddff",
"S3Key": "68b4ffa1c39cb3733535725f85311791c09eab53b7ab8efa5152e68f8abdb005.zip"
},
"Role": {
"Fn::GetAtt": [
"AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2",
"Arn"
]
},
"Handler": "index.handler",
"Runtime": "nodejs12.x",
"Timeout": 120
},
"DependsOn": [
"AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2"
],
"Metadata": {
"aws:cdk:path": "CODE/AWS679f53fac002430cb0da5b7982bd2287/Resource",
"aws:asset:path": "asset.68b4ffa1c39cb3733535725f85311791c09eab53b7ab8efa5152e68f8abdb005",
"aws:asset:is-bundled": false,
"aws:asset:property": "Code"
}
}
}
What I need is to replace this part
"S3Bucket": "hnb659fds-assets-xxccddff",
and have the following result
"S3Bucket": {"Fn::Sub": "AAA-${AWS::Region}" },
I don't know the AWS679f53fac002430cb0da5b7982bd22872D164C4C. It is generated randomly and the string to replace is present several times in my json file.
The initial values to be replaced is stored in a variable along with the new value to be used in the replaced version as following:
cdk_bucket_name=hnb659fds-assets-xxccddff
OUTPUT_BUCKET=AAA
I need these variables because this is part of a bigger script
So I tried some sed but does not work
new_bucket_name="{"Fn::Sub\": \"$OUTPUT_BUCKET-${AWS::Region}\" }"
sed -i "s#$cdk_bucket_name#$new_bucket_name#g" my.template.json
One issue that I have is that ${AWS::Region} gets interpreted so is empty.
And second, I cannot manage the quotes in order to have my desired result.
Using sed
$ output_bucket=AAA
$ new_bucket_name="{\"Fn::Sub\": \"$output_bucket-\${AWS::Region}\" }"
$ cdk_bucket_name=hnb659fds-assets-xxccddff
$ sed s"/\"$cdk_bucket_name\"/$new_bucket_name/" input_file
{ "AWS679f53fac002430cb0da5b7982bd22872D164C4C": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": {"Fn::Sub": "AAA-${AWS::Region}" },
"S3Key": "68b4ffa1c39cb3733535725f85311791c09eab53b7ab8efa5152e68f8abdb005.zip"
},
"Role": {
"Fn::GetAtt": [
"AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2",
"Arn"
]
},
"Handler": "index.handler",
"Runtime": "nodejs12.x",
"Timeout": 120
},
"DependsOn": [
"AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2"
],
"Metadata": {
"aws:cdk:path": "CODE/AWS679f53fac002430cb0da5b7982bd2287/Resource",
"aws:asset:path": "asset.68b4ffa1c39cb3733535725f85311791c09eab53b7ab8efa5152e68f8abdb005",
"aws:asset:is-bundled": false,
"aws:asset:property": "Code"
}
}
}
Using a proper JSON parser shell tool like jq:
jq '
(
.[].Properties.Code.S3Bucket |
select(. == "hnb659fds-assets-xxccddff")
) = $newS3Bucket
' input_file.json \
--argjson newS3Bucket '{"Fn::Sub":"AAA-${AWS::Region}"}'

How to parse this boolean contained JSON output with jq?

The JSON output I am trying to parse:
{
"success": true,
"data": {
"aa": [
{
"timestamp": 123456,
"price": 1
},
{
"timestamp": 123457,
"price": 2
],
"bb": [
{
"timestamp": 123456,
"price": 3
},
{
"timestamp": 123457,
"price": 4
}
]
}
}
So after banging my head against the wall a million times, I just removed the "success": true", line from the output and I could easily do jq stuff with it. Otherwise if I ran for example:
cat jsonfile.json | jq -c .[].aa
I would get:
Cannot index boolean with string "aa"
Which makes sense, since the first key is boolean. But I have no clue how to skip it while processing with jq.
Goal is to filter only timestamp and price of "aa", without giving any care about the "success": true key/value pair.
You need to select the data field first: jq .data.aa[]

JQ newbie trouble selecting nested keys

Hi I am new to JQ and Json. I am using
$ jq --version
jq-1.5
I am having a heck of a time trying to figure out how to select the values for id, attributes.name, attributes.albumName, and attributes.artistName
I am using the terminal app on a mac. I am running into some sort strange parsing problem
$ jq '.results.songs.data[0] | {id, attributes.name } ' t
jq: error: syntax error, unexpected FIELD, expecting '}' (Unix shell quoting issues?) at <top-level>, line 1:
.results.songs.data[0] | {id, attributes.name }
jq: 1 compile error
$
This example shows the structure of the data I am trying to filter looks like
$ jq '.results.songs.data[0] | {id, attributes } ' t
{
"id": "152471393",
"attributes": {
"previews": [
{
"url": "https://audio-ssl.itunes.apple.com/apple-assets-us-std-000001/AudioPreview71/v4/7d/c5/68/7dc56849-29b8-bd90-2bb1-51750e479569/mzaf_4742389090778091050.plus.aac.p.m4a"
}
],
"artwork": {
"width": 1449,
"height": 1449,
"url": "https://is5-ssl.mzstatic.com/image/thumb/Music/v4/7d/01/56/7d0156be-12cd-8724-a0ca-727b1013a81d/source/{w}x{h}bb.jpeg",
"bgColor": "ddcfc4",
"textColor1": "010100",
"textColor2": "422f10",
"textColor3": "2d2a27",
"textColor4": "614f34"
},
"artistName": "Gnarls Barkley",
"url": "https://itunes.apple.com/us/album/crazy/152471339?i=152471393",
"discNumber": 1,
"genreNames": [
"Alternative",
"Music",
"R&B/Soul",
"Rock",
"Soul",
"Hip-Hop/Rap",
"Rap",
"Hip-Hop",
"Adult Alternative",
"Neo-Soul",
"Alternative Rap",
"Underground Rap"
],
"durationInMillis": 178387,
"releaseDate": "2006-03-13",
"name": "Crazy",
"isrc": "USAT20611041",
"albumName": "St. Elsewhere",
"playParams": {
"id": "152471393",
"kind": "song"
},
"trackNumber": 2
}
}
Thanks
Andy
With your sample JSON as input, the following invocation:
jq '{id, name: .attributes.name }' input.json
produces:
{
"id": "152471393",
"name": "Crazy"
}
The filter above is short for:
{"id" : .id, "name": .attributes.name }
In any case, the keys must be appropriately specified.
For future reference, when asking questions on stackoverflow.com, please adhere to the http://stackoverflow.com/help/mcve guidelines as much as possible.

Bash JQ getting multiple values Issue in JSON file

I'm trying to parse a JSON file for getting multiple values. I know how to parse the specific values ( "A"/"B"/"C") in the array (.info.file.hashes[]).
For Example : When issuing the following command over the file b.json
jq -r '.info.file.hashes[] | select(.name == ("A","B","C")).value' b.json
Result :
f34d5f2d4577ed6d9ceec516c1f5a744
66031dad95dfe6ad10b35f06c4342faa
9df25fa4e379837e42aaf6d05d92012018d4b659
Where b.json:
{
"Finish": 1475668827,
"Start": 1475668826,
"info": {
"file": {
"Score": 4,
"file_subtype": "None",
"file_type": "Image",
"hashes": [
{
"name": "A",
"value": "f34d5f2d4577ed6d9ceec516c1f5a744"
},
{
"name": "B",
"value": "66031dad95dfe6ad10b35f06c4342faa"
},
{
"name": "C",
"value": "9df25fa4e379837e42aaf6d05d92012018d4b659"
},
{
"name": "D",
"value": "4a51cc531082d216a3cf292f4c39869b462bf6aa"
},
{
"name": "E",
"value": "e445f412f92b25f3343d5f7adc3c94bdc950601521d5b91e7ce77c21a18259c9"
}
],
"size": 500
}
}
}
Now, how can i get multiple values with "Finish", "Start" along with the hash values? I have tried issuing the command.
jq -r '.info.file.hashes[] | select(.name == ("A","B","C")).value','.Finish','.Start' b.json
and Im getting the result as:
f34d5f2d4577ed6d9ceec516c1f5a744
null
66031dad95dfe6ad10b35f06c4342faa
null
9df25fa4e379837e42aaf6d05d92012018d4b659
null
null
null
Expected Result :
f34d5f2d4577ed6d9ceec516c1f5a744
66031dad95dfe6ad10b35f06c4342faa
9df25fa4e379837e42aaf6d05d92012018d4b659
1475668827
1475668826
Literally just downloaded and read the manual
Try
jq '(.info.file.hashes[] |select(.name == ("A","B","C")).value), .Finish, .Start' b.json
"f34d5f2d4577ed6d9ceec516c1f5a744"
"66031dad95dfe6ad10b35f06c4342faa"
"9df25fa4e379837e42aaf6d05d92012018d4b659"
1475668827
1475668826
Note the brackets used for grouping the pipe separately from the Finish and Start values.

Get formatted data in shell script which is read from file containing JSON data

Writting one shell script to automatically get list of name, current and latest available version from raw json data.
I am trying to format JSON data stored in file using shell script. I tried using JQ command line JSON parser.
I want to get formatted JSON data in script. Their is advanced option provided in JQ for same scenario. I am not able to use it properly.
Example: File containing Following JSON
{
"endpoint": {
"name": "test-plugin",
"version": "0.0.1"
},
"dependencies": {
"plugin1": {
"main": {
"name": "plugin1name",
"description": "Dummy text"
},
"pkgMeta": {
"name": "plugin1name",
"version": "0.0.1"
},
"dependencies": {},
"versions": [
"0.0.5",
"0.0.4",
"0.0.3",
"0.0.2",
"0.0.1"
],
"update": {
"latest": "0.0.5"
}
},
"plugin2": {
"main": {
"name": "plugin2name",
"description": "Dummy text"
},
"pkgMeta": {
"name": "plugin2name",
"version": "0.1.1"
},
"dependencies": {},
"versions": [
"0.1.5",
"0.1.4",
"0.1.3",
"0.1.2",
"0.1.1"
],
"update": {
"latest": "0.1.5"
}
}
}
}
Trying to get result in format
[{name: "plugin1name",
c_version: "0.0.1",
n_version: "0.0.5"
},
{name: "plugin2name",
c_version: "0.1.1",
n_version: "0.1.5"}]
Can someone suggest anything ?
Your json file is not valid at: .dependencies.pkgMeta.version.
After fixing your json file, try this command:
jq '
.dependencies |
to_entries |
map(.value |
{
name: .main.name,
c_version: .pkgMeta.version,
n_version: .update.latest
}
)' input.json
The result is:
[
{
"name": "plugin1name",
"c_version": "0.0.1",
"n_version": "0.0.5"
},
{
"name": "plugin2name",
"c_version": "0.1.1",
"n_version": "0.1.5"
}
]