A command that I run from my terminal return the following:
{
"attributes": {
"env": "prod"
},
"created_by": "email#email.com",
"id": "612",
"state": "published",
"version": "0.22.0"
}
{
"attributes": {
"env": "prod"
},
"created_by": "email#email.com",
"id": "611",
"state": "published",
"version": "0.22.0"
}
And I just want to get the "version" from the first object. Theres is a way to do that? Everything I tried with Array returns a error...
The data you've extracted is a stream of JSON objects. You could extract the .version from the first of these by piping the data to:
jq -n input.version
Related
I am trying to extract values from a json that I obtained using the curl command for api testing. My json looks as below. I need some help extracting the value "20456" from here?
{
"meta": {
"status": "OK",
"timestamp": "2022-09-16T14:45:55.076+0000"
},
"links": {},
"data": {
"id": 24843,
"username": "abcd",
"firstName": "abc",
"lastName": "xyz",
"email": "abc#abc.com",
"phone": "",
"title": "",
"location": "",
"licenseType": "FLOATING",
"active": true,
"uid": "u24843",
"type": "users"
}
}
{
"meta": {
"status": "OK",
"timestamp": "2022-09-16T14:45:55.282+0000",
"pageInfo": {
"startIndex": 0,
"resultCount": 1,
"totalResults": 1
}
},
"links": {
"data.createdBy": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.createdBy}"
},
"data.fields.user1": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.fields.user1}"
},
"data.modifiedBy": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.modifiedBy}"
},
"data.fields.projectManager": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.fields.projectManager}"
},
"data.parent": {
"type": "projects",
"href": "https://abc#abc.com/rest/v1/projects/{data.parent}"
}
},
"data": [
{
"id": 20456,
"projectKey": "Stratus",
"parent": 20303,
"isFolder": false,
"createdDate": "2018-03-12T23:46:59.000+0000",
"modifiedDate": "2020-04-28T22:14:35.000+0000",
"createdBy": 18994,
"modifiedBy": 18865,
"fields": {
"projectManager": 18373,
"user1": 18628,
"projectKey": "Stratus",
"text1": "",
"name": "Stratus",
"description": "",
"date2": "2019-03-12",
"date1": "2018-03-12"
},
"type": "projects"
}
]
}
I have tried the following, but end up getting error:
▶ cat jqTrial.txt | jq '.data[].id'
jq: error (at <stdin>:21): Cannot index number with string "id"
20456
Also tried this but I get strings outside the object that I am not sure how to remove:
cat jqTrial.txt | jq '.data[]'
Assuming you want the project id not the user id:
jq '
.data
| if type == "object" then . else .[] end
| select(.type == "projects")
| .id
' file.json
There's probably a better way to write the 2nd expression
Indeed, thanks to #pmf
.data | objects // arrays[] | select(.type == "projects").id
Your input consists of two JSON documents; both have a data field on top level. But while the first one is itself an object which has an .id field, the second one is an array with one object item, which also has an .id field.
To retrieve both, you could use the --slurp (or -s) option which wraps both top-level objects into an array, then you can address them separately by index:
jq --slurp '.[0].data.id, .[1].data[].id' jqTrial.txt
24843
20456
Demo
I am using aws ecs query to get list of properties being used by the current running task.
command -
cft = "aws ecs describe-tasks --cluster arn:aws:ecs:us-west-2:4984314772:cluster/secrets --tasks arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b
I am storing this in an output variable
output= $( eval $cft)
Output:
"tasks": [
{
"attachments": [
{
"id": "da8a1312-8278-46d5-8e3b-6b6a1d96f820",
"type": "ElasticNetworkInterface",
"status": "ATTACHED",
"details": [
{
"name": "subnetId",
"value": "subnet-0a151f2eb959ad4"
},
{
"name": "networkInterfaceId",
"value": "eni-081948e3666253f"
},
{
"name": "macAddress",
"value": "02:2a:9i:5c:4a:77"
},
{
"name": "privateDnsName",
"value": "ip-172-56-17-177.us-west-2.compute.internal"
},
{
"name": "privateIPv4Address",
"value": "172.56.17.177"
}
]
}
],
"availabilityZone": "us-west-2a",
"clusterArn": "arn:aws:ecs:us-west-2:4984314772:cluster/secrets",
"containers": [
{
"taskArn": "arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b",
"name": "nginx",
"image": "nginx",
"lastStatus": "PENDING",
"networkInterfaces": [
{
"attachmentId": "da8a1312-8278-46d5-6b6a1d96f820",
"privateIpv4Address": "172.31.17.176"
}
],
"healthStatus": "UNKNOWN",
"cpu": "0"
}
],
"cpu": "256",
"createdAt": "2020-12-10T18:00:16.320000+05:30",
"desiredStatus": "RUNNING",
"group": "family:nginx",
"healthStatus": "UNKNOWN",
"lastStatus": "PENDING",
"launchType": "FARGATE",
"memory": "512",
"overrides": {
"containerOverrides": [
{
"name": "nginx"
}
],
"inferenceAcceleratorOverrides": []
},
"platformVersion": "1.4.0",
"tags": [],
"taskArn": "arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b",
"taskDefinitionArn": "arn:aws:ecs:us-west-2:4984314772:task-definition/nginx:17",
"version": 2
}
],
"failures": []
}
now if do an echo of $output.tasks[0].containers[0] nothing happens it prints the entire thing again, i want to store the result in output variable and refer different parameter like we do in json format.
You will need to use a json parser such as jq and so:
eval $cft | jq '.tasks[].containers[]'
To avoid using eval you could simple pipe the aws command into jq and so:
aws ecs describe-tasks --cluster arn:aws:ecs:us-west-2:4984314772:cluster/secrets --tasks arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b | jq '.tasks[].containers[]'
or:
cft=$(aws ecs describe-tasks --cluster arn:aws:ecs:us-west-2:4984314772:cluster/secrets --tasks arn:aws:ecs:us-west-2:4984314772:task/secrets/86855757eec4487f9d4475a1f7c4cb0b | jq '.tasks[].containers[]')
echo $cft | jq '.tasks[].containers[]'
noob here. I am trying to program my homebridge to accept a new dyson fan. Thanks to joe-ng who wrote it. https://github.com/joe-ng/homebridge-dyson-link
This is what I'm inputting and keep getting a syntax error warning:
{
"bridge": {
"name": "Homebridge C81F",
"username": "mac:address:of:rasberrypi",
"port": 88888,
"pin": "031-45-154"
},
"accessories": [],
"platforms": [
{
"platform": "DysonPlatform",
"name": "DysonPlatform",
"email": "my-email",
"password": "my-password",
"country": "UK",
"accessories": [
{
"ip": "192.168.0.88",
"displayName": "Bedroom Fan",
"serialNumber": "VS7-UK-SERIALNO",
}
]
}
]
I've tried using JSONLint and it is telling me expecting string, so I thought it was commas but I'm not very experienced so would appreciate any help. Thanks.
Suppose you have it in a file.json, then you can display the file with the jq program.
For your json, the jq program returns an error:
> jq '.' file.json
parse error: Expected another key-value pair at line 21, column 13
After you remove comma (,), you have next error:
➜ ~ jq '.' file.json
parse error: Unfinished JSON term at EOF at line 25, column 0
After you add missing bracket (}), you get:
{
"bridge": {
"name": "Homebridge C81F",
"username": "mac:address:of:rasberrypi",
"port": 88888,
"pin": "031-45-154"
},
"accessories": [],
"platforms": [
{
"platform": "DysonPlatform",
"name": "DysonPlatform",
"email": "my-email",
"password": "my-password",
"country": "UK",
"accessories": [
{
"ip": "192.168.0.88",
"displayName": "Bedroom Fan",
"serialNumber": "VS7-UK-SERIALNO"
}
]
}
]
}
I have a list of dictionaries (basically JSON Response of an endpoint)
I would need to Parse this 16000 lines of json objects and fitler the documents/objects which match criteria that
whose leaf element/field : statusInfo/status in not "UP" and of those filtered objects, just return "name" , "serviceUrl","status"
example :
"ADMIN-V1" "http://aws-ec2.aws.com:4435" "Warning"
I have been researching about JSONPath module , but there is no good documentation about it, and I could not find any easier way.
Any guidance is highly appreciated.
here is a snippet from long 16000 lines of JSON response.
[
{
"id": "9c108ec5",
"name": "USER-V2",
"managementUrl": "http://aws-ec2.aws.com:5784/",
"healthUrl": "http://aws-ec2.aws.com:5784/health",
"serviceUrl": "http://aws-ec2.aws.com:5784/",
"statusInfo": {
"status": "UP",
"timestamp": 1566663146681,
"details": {
"description": " Eureka Discovery Client",
"status": "UP"
}
},
"source": "discovery",
"metadata": {},
"info": {
"component": "user",
"description": "User REST Resource",
"version": "2.2.1",
"git": {
"commit": {
"time": "07/27/2018 # 15:06:55 CDT",
"id": "b2a1b37"
},
"branch": "refs/tags/v2.2.1"
}
}
},
{
"id": "1a381f20",
"name": "ADMIN-V1",
"managementUrl": "http://aws-ec2.aws.com:4435/",
"healthUrl": "http://aws-ec2.aws.com:4435/health",
"serviceUrl": "http://aws-ec2.aws.com:4435/",
"statusInfo": {
"status": "Warning",
"timestamp": 1566663146682,
"details": {
"description": "Spring Cloud Eureka Discovery Client",
"status": "Warning"
}
},
"source": "discovery",
"metadata": {},
"info": {
"description": "Exchange Admin REST Resource",
"api": {
"version": "1.2.1",
"name": "admin",
"link": "https://app.swaggerhub.com/apis/AWSExchange/admin/1.2.1"
},
"implementation": "admin",
"version": "1.1.0",
"git": {
"commit": {
"time": "01/04/2019 # 15:36:48 UTC",
"id": "39d5551"
},
"branch": "refs/tags/v1.1.0"
}
}
}
]
If your json file contains one big array, you'll want to stream that file in truncating out the array. Then use fromstream/1 to rebuild the objects and filtering them out as you go.
I don't have a representative file to test out the performance myself, but give this a try:
$ jq --stream -n 'fromstream(1|truncate_stream(inputs))
| select(.statusInfo.status != "UP")
| .name, .serviceUrl, .statusInfo.status
' input.json
I wish to parse individual elements of inner JSON object to build / load in the database.
The following is the JSON object. How can I parse elements like id, name queue etc? I will iterate it in loop and work and build the insert query.
{
"apps": {
"app": [
{
"id": "application_1540378900448_18838",
"user": "hive",
"name": "insert overwrite tabl...summary_view_stg_etl(Stage-2)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
},
{
"id": "application_1540378900448_18833",
"user": "hive",
"name": "insert into SNOW_WORK...metric_definitions')(Stage-13)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
}
]
}
}
You're better off converting the data to a format easily consumed by a database processor, like csv, then do something about it.
$ jq -r '(.apps.app[0] | keys_unsorted) as $k
| $k, (.apps.app[] | [.[$k[]]])
| #csv
' input.json
its pretty simple just fetch elment which is having an array of values.
var JSONOBJ={
"apps": {
"app": [
{
"id": "application_1540378900448_18838",
"user": "hive",
"name": "insert overwrite tabl...summary_view_stg_etl(Stage-2)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
},
{
"id": "application_1540378900448_18833",
"user": "hive",
"name": "insert into SNOW_WORK...metric_definitions')(Stage-13)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
}
]
}
}
JSONOBJ.apps.app.forEach(function(o){console.log(o.id);console.log(o.user);console.log(o.name);})