Trying to convert a JSON array into a CSV file using jq but not able to succeed. Following is my JSON output from curl command:
{
"requestID": "463aeb25-f4c3-40ba-a031-e62d698afc6e",
"signature": {
"id": "json",
"ph_no": "json",
"status": "json"
},
"results": [
{
"id": "9f34-66758813073c",
"ph_no": "343434325",
"status": "active"
},
{
"id": "b1a2-30a14a68c576",
"ph_no": "6767666764",
"status": "active"
},
{
"id": "9af4-5b231f05ce37",
"ph_no": "546745435",
"status": "active"
},
{
"id": "99bd-ed67fd139074",
"ph_no": "323246566",
"status": "active"
},
{
"id": "9ecc-8277c3ffa274",
"ph_no": "6753643554",
"status": "active"
}
],
"status": "success",
"metrics": {
"elapsedTime": "29.461027ms",
"executionTime": "29.364961ms",
"resultCount": 146,
"resultSize": 13856
}
}
I have tried using following referring to some solutions online but not working.
jq -r '["id","ph_no","status"],(to_entries|.[]|[.key,.value.id,.value.ph_no,.value.status)|#csv' temp.json
How should i modify the jq command to convert JSON to CSV ?
If you just want the results array of objects:
jq -r '(["id","ph_no","status"], (.results[] | [.id, .ph_no, .status])) | #csv' temp.json
Related
I am trying to extract values from a json that I obtained using the curl command for api testing. My json looks as below. I need some help extracting the value "20456" from here?
{
"meta": {
"status": "OK",
"timestamp": "2022-09-16T14:45:55.076+0000"
},
"links": {},
"data": {
"id": 24843,
"username": "abcd",
"firstName": "abc",
"lastName": "xyz",
"email": "abc#abc.com",
"phone": "",
"title": "",
"location": "",
"licenseType": "FLOATING",
"active": true,
"uid": "u24843",
"type": "users"
}
}
{
"meta": {
"status": "OK",
"timestamp": "2022-09-16T14:45:55.282+0000",
"pageInfo": {
"startIndex": 0,
"resultCount": 1,
"totalResults": 1
}
},
"links": {
"data.createdBy": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.createdBy}"
},
"data.fields.user1": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.fields.user1}"
},
"data.modifiedBy": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.modifiedBy}"
},
"data.fields.projectManager": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.fields.projectManager}"
},
"data.parent": {
"type": "projects",
"href": "https://abc#abc.com/rest/v1/projects/{data.parent}"
}
},
"data": [
{
"id": 20456,
"projectKey": "Stratus",
"parent": 20303,
"isFolder": false,
"createdDate": "2018-03-12T23:46:59.000+0000",
"modifiedDate": "2020-04-28T22:14:35.000+0000",
"createdBy": 18994,
"modifiedBy": 18865,
"fields": {
"projectManager": 18373,
"user1": 18628,
"projectKey": "Stratus",
"text1": "",
"name": "Stratus",
"description": "",
"date2": "2019-03-12",
"date1": "2018-03-12"
},
"type": "projects"
}
]
}
I have tried the following, but end up getting error:
▶ cat jqTrial.txt | jq '.data[].id'
jq: error (at <stdin>:21): Cannot index number with string "id"
20456
Also tried this but I get strings outside the object that I am not sure how to remove:
cat jqTrial.txt | jq '.data[]'
Assuming you want the project id not the user id:
jq '
.data
| if type == "object" then . else .[] end
| select(.type == "projects")
| .id
' file.json
There's probably a better way to write the 2nd expression
Indeed, thanks to #pmf
.data | objects // arrays[] | select(.type == "projects").id
Your input consists of two JSON documents; both have a data field on top level. But while the first one is itself an object which has an .id field, the second one is an array with one object item, which also has an .id field.
To retrieve both, you could use the --slurp (or -s) option which wraps both top-level objects into an array, then you can address them separately by index:
jq --slurp '.[0].data.id, .[1].data[].id' jqTrial.txt
24843
20456
Demo
A command that I run from my terminal return the following:
{
"attributes": {
"env": "prod"
},
"created_by": "email#email.com",
"id": "612",
"state": "published",
"version": "0.22.0"
}
{
"attributes": {
"env": "prod"
},
"created_by": "email#email.com",
"id": "611",
"state": "published",
"version": "0.22.0"
}
And I just want to get the "version" from the first object. Theres is a way to do that? Everything I tried with Array returns a error...
The data you've extracted is a stream of JSON objects. You could extract the .version from the first of these by piping the data to:
jq -n input.version
Using jq, how can I extract the host, name, and date_happened fields from the json output below? I read another thread regarding this here at stackoverflow, but still can't seem to get it.
-Thanks
{
"events": [
{
"alert_type": "success",
"children": [
{
"alert_type": "error",
"date_happened": 1573502725,
"id": "5188183926379101887"
},
{
"alert_type": "success",
"date_happened": 1573503145,
"id": "5188190972457497744"
}
],
"comments": [],
"date_happened": 1573502725,
"device_name": null,
"host": "i-0e4b192579a9b423b",
"id": 5188183933173874377,
"is_aggregate": true,
"priority": "normal",
"resource": "/api/v1/events/5188183933173874377",
"source": "Monitor Alert",
"tags": [
"autoscaling_group:app2_backend-asg-prod",
"availability-zone:us-east-1b",
"datadog-agent:true",
"environment:prod",
"host:i-0e6b192579a9b423b",
"iam_profile:app2_backend_instance_profile",
"image:ami-2769055d",
"instance-type:m4.large",
"monitor",
"name:app2_backend-prod",
"region:us-east-1",
"role:app2_backend",
The jq filter:
.events[]
| [.host,
(.tags[] | select( test("^name:") ) | sub("name:";"")),
.date_happened]
produces
["i-0e4b192579a9b423b","app2_backend-prod",1573502725]
I have a json file and I want to add some value from top in another place in json.
I am trying to use jq command line.
{
"channel": "mychannel",
"videos": [
{
"id": "10",
"url": "youtube.com"
},
{
"id": "20",
"url": "youtube.com"
}
]
}
The output would be:
{
"channel": "mychannel",
"videos": [
{
"channel": "mychannel",
"id": "10",
"url": "youtube.com"
},
{
"channel": "mychannel",
"id": "20",
"url": "youtube.com"
}
]
}
in my json the "channel" is static, same value always. I need a way to concatenate always in each video array.
Someone can help me?
jq .videos + channel
Use a variable to remember .channel in the later stages of the pipeline.
$ jq '.channel as $ch | .videos[].channel = $ch' tmp.json
{
"channel": "mychannel",
"videos": [
{
"id": "10",
"url": "youtube.com",
"channel": "mychannel"
},
{
"id": "20",
"url": "youtube.com",
"channel": "mychannel"
}
]
}
I have a json file to store my data and I convert it to CSV to edit my data. But when i convert it to json again it all goes unconstructed. How can i convert my csv to same structure as my old json.
JSON
{
"product": [
{
"id": "item0001",
"category": "12",
"name": "Name1",
"tag": "tag1",
"more": [
{
"id": "1",
"name": "AL"
},
{
"id": "1",
"name": "BS"
}
],
"active": true
},
{
"id": "item0002",
"categoryId": "13",
"name": "Name2",
"tag": "tag2",
"size": "2",
"more": [
{
"id": "2",
"name": "DL"
},
{
"id": "2",
"name": "AS"
}
],
"active": true
}
]
}
CSV
id,categoryId,name,shortcut,more/0/optionId,more/0/price,more/1/optionId,more/1/price,active,more/2/optionId,more/2/price,spanSize
item0001,ab92d2c6-010e-4182-844d-65050e746617,Name1,Shortcut1,1,60,1,70,TRUE,,,
item0002,ab92d2c6-010e-4182-844d-65050e746617,Name2,Shortcut2,2,60,2,70,TRUE,2,2,4
You can use Miller (mlr) to convert you file both ways
https://miller.readthedocs.io/en/latest/flatten-unflatten/
first from JSON to CSV
mlr --ijson --ocsv cat test.json > test.csv
then edit CSV (Visidata is a very nice command line tool for the job)
then convert it back to CSV
mlr --icsv --ojson cat test.csv > test_v2.json
If you want to have some JSON lines structure instead, use --ojonl