Here is my sample.json:
{
"process" : {
"pid" : "1462",
"path" : "\/Applications\/Google Chrome.app\/Contents\/Frameworks\/Google Chrome Framework.framework\/Versions\/108.0.5359.98\/Helpers\/Google Chrome Helper.app\/Contents\/MacOS\/Google Chrome Helper",
"signature(s)" : {
"signatureIdentifier" : "com.google.Chrome.helper",
"signatureStatus" : 0,
"signatureSigner" : 3,
"signatureAuthorities" : [
"Developer ID Application: Google LLC (EQHXZ8M8AV)",
"Developer ID Certification Authority",
"Apple Root CA"
]
}
},
"connections" : [
{
"remoteHostName" : "n\/a",
"protocol" : "UDP",
"interface" : "",
"localAddress" : "::",
"state" : "n\/a",
"remotePort" : "0",
"localPort" : "5353",
"remoteAddress" : "::"
},
{
"remoteHostName" : "n\/a",
"protocol" : "TCP",
"interface" : "en0",
"localAddress" : "2a02:560:5424:b200:359c:f801:abab:cd28",
"state" : "Established",
"remotePort" : "443",
"localPort" : "50190",
"remoteAddress" : "2600:1f18:60d5:4e03:ffe8:813e:6d1a:d379"
}
]
}
I would like to create a custom CSV from this data to see all connections by process id (pid), but I don't get it.
What I have so far:
cat sample.json | jq '[.process.pid], (.connections | .[])'
Thanks in advance for your help!
jq -r '{pid: .process.pid} + .connections[] | to_entries | map(.value) | #csv' input.json
Output
"1462","n/a","UDP","","::","n/a","0","5353","::"
"1462","n/a","TCP","en0","2a02:560:5424:b200:359c:f801:abab:cd28","Established","443","50190","2600:1f18:60d5:4e03:ffe8:813e:6d1a:d379"
Related
how can I have transformed my json
{
"clients": [
{
"id" : "qwerty",
"accounts" : [{"number" : "6666"}, {"number" : "7777"}]
},
{
"id" : "zxcvb",
"accounts" : [{"number" : "1111"}, {"number" : "2222"}]
}
]
}
into following type of json? using JQ
{
"items": [
{
"id" : "qwerty",
"number" : "6666"
},{
"id" : "qwerty",
"number" : "7777"
},{
"id" : "zxcvb",
"number" : "1111"
},{
"id" : "zxcvb",
"number" : "2222"
}]
}
What kind of tools from JQ can help me? I can't choose any possible way to do it
Something like this should do the trick:
{items: [.clients[] | {id} + .accounts[]]}
Online demo
I wish to reduce with jq
[ {
"context" : "app:swagger,dev:8080",
"parent" : null,
"beans" : [ {
"bean" : "app",
"aliases" : [ ],
"scope" : "singleton",
"type" : "com.example.App",
"resource" : "null",
"dependencies" : [ "environment" ]
}, {
"bean" : "environment",
"aliases" : [ ],
"scope" : "singleton",
"type" : "com.example.Environment",
"resource" : "null",
"dependencies" : [ ]
}
},
...
}]
to
app --> environment
...
The problem statement appears to be under-specified, but the following seems to be either a solution or very close to one:
jq -r '.[] | .beans[] | "\(.bean) --> \(.dependencies[])"' input.json
I found the solution for json to csv conversion. Below is the sample json and solution.
{
"took" : 111,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "alerts",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"alertID" : "639387c3-0fbe-4c2b-9387-c30fbe7c2bc6",
"alertCategory" : "Server Alert",
"description" : "Successfully started.",
"logId" : null
}
},
{
"_index" : "alerts",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"alertID" : "2",
"alertCategory" : "Server Alert",
"description" : "Successfully stoped.",
"logId" : null
}
}
]
}
}
The solution :
jq -r '.hits.hits[]._source | [ "alertID" , "alertCategory" , "description", "logId" ], ([."alertID",."alertCategory",."description",."logId" // "null"]) | #csv' < /root/events.json
The problem with this solution is that I have to hard code the column names. What If my json gets a few additions under _source tag later? I need a solution which can handle the dynamic data under _source. I am open to any other tool or command in shell.
Simply use keys_unsorted (or keys if you want them sorted). See e.g. Convert JSON array into CSV using jq or How to convert arbitrary simple JSON to CSV using jq? for two SO examples. There are many others too.
I am trying to convert my json file to a csv file using jq. Below is the sample input events.json file.
{
"took" : 111,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "alerts",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"alertID" : "639387c3-0fbe-4c2b-9387-c30fbe7c2bc6",
"alertCategory" : "Server Alert",
"description" : "Successfully started.",
"logId" : null
}
},
{
"_index" : "alerts",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"alertID" : "2",
"alertCategory" : "Server Alert",
"description" : "Successfully stoped.",
"logId" : null
}
}
]
}
}
My rows in csv should have the data inside each _source tag. So my columns would be alertId , alertCategory , description and logId with its respective data.
I tried the below command :
jq --raw-output '.hits[] | [."alertId",."alertCategory",."description",."logId"] | #csv' < /root/events.json
and its not working.
Can anyone help me with this?
Your path-expression is not right, you have a hits array inside an object named hits and the fields you trying to put in CSV is present under __source object.
So your expression should have been below. Use it along with -r flag to put the output in raw output format
.hits.hits[]._source | [ .alertID, .alertCategory, .description, .logId ] | #csv
If your fields are null, the string representation of your null field value results in just "". If you want an explicit "null" string representation, use the alternate operator along with the field you expect to be null, e.g. instead of .logId, you can do (.logId // "null")
To add the column name as the header in the output CSV format, you could use the #csv or the join(",") function in raw output format -r
[ "alertId" , "alertCategory" , "description", "logId" ],
( .hits.hits[]._source | [ .alertID, .alertCategory, .description, .logId // "null" ]) | #csv
or
[ "alertId" , "alertCategory" , "description", "logId" ],
( .hits.hits[]._source | [ .alertID, .alertCategory, .description, .logId // "null" ]) | join(",")
I am working with a JSON file similar to the one below:
{ "Response" : {
"TimeUnit" : [ 1576126800000 ],
"metaData" : {
"errors" : [ ],
"notices" : [ "query served by:1"]
},
"stats" : {
"data" : [ {
"identifier" : {
"names" : [ "apiproxy", "response_status_code", "target_response_code", "target_ip" ],
"values" : [ "IO", "502", "502", "7.1.143.6" ]
},
"metric" : [ {
"env" : "dev",
"name" : "sum(message_count)",
"values" : [ 0.0]
} ]
} ]
} } }
My object is to display a mapping of the identifier and values like :
apiproxy=IO
response_status_code=502
target_response_code=502
target_ip=7.1.143.6
I have been able to parse both names and values with
.[].stats.data[] | (.identifier.names[]) and .[].stats.data[] | (.identifier.values[])
but I need help with the jq way to map the values.
The whole thing can be done in jq using the -r command-line option:
.[].stats.data[]
| [.identifier.names, .identifier.values]
| transpose[]
| "\(.[0])=\(.[1])"