I have the input JSON data at the bottom.
I'd like to generate a TSV output like the following. <TAB> is the TAB character. How can I do it in jq?
timestamp<TAB>open<TAB>high<TAB>dividends
1623072600<TAB>4229.33984375<TAB>4232.33984375<TAB>
1623159000<TAB>4233.81005859375<TAB>4236.740234375<TAB>
1623245400<TAB>4232.990234375<TAB>4237.08984375<TAB>0.42
1623331800<TAB>4228.56005859375<TAB>4249.740234375<TAB>
1623418200<TAB>4242.89990234375<TAB>4248.3798828125<TAB>
{
"timestamp": [
1623072600,
1623159000,
1623245400,
1623331800,
1623418200
],
"indicators": {
"quote": [
{
"open": [
4229.33984375,
4233.81005859375,
4232.990234375,
4228.56005859375,
4242.89990234375
],
"high": [
4232.33984375,
4236.740234375,
4237.08984375,
4249.740234375,
4248.3798828125
]
}
]
},
"events": {
"dividends": {
"1623245400": {
"amount": 0.42,
"date": 1623245400
}
}
}
}
Using jq with the -r command-line option:
(.events.dividends|map_values(.amount)) as $dividends
| ["timestamp", "open", "high", "dividends"],
( [.timestamp, (.indicators.quote[0] | .open, .high),
[$dividends[.timestamp[]|tostring]]]
| transpose[])
| #tsv
Notice how the dividends column is computed using the provided $dividends dictionary:
$dividends[.timestamp[]|tostring]
Related
I want to transform JSON data using jq filter
Json data:
{
"main": [
{
"firstKey": "ABCD",
"id": "12345",
"data": [
{
"name": "first_id",
"value": "first_id_value"
},
{
"name": "second_id",
"value": "second_id_value"
},
{
"name": "third_id",
"value": "third_id_value"
}
]
}
]
}
Expected OUTPUT:
{
"firstKey": "ABCD",
"id": "12345",
"data.name.first_id": "first_id_value",
"data.name.second_id": "second_id_value",
"data.name.third_id": "third_id_value"
}
After many trials and errors, I was near to expected output using following filter expression
[.main[]|{"firstKey", "id"},foreach .data[] as $item (0; "data.name.\($item.name)" as $a|$item.value as $b| {($a): $b})][]
Used foreach as objects under "data" are dynamic. the number of objects can differ.
The output for the above expression is:
{
"firstKey": "ABCD",
"id": "12345"
}
{
"data.name.first_id": "first_id_value"
}
{
"data.name.second_id": "second_id_value"
}
{
"data.name.third_id": "third_id_value"
}
But I want the objects of data to be under the same braces as 'firstKey' and 'id'.
LINK to JqPlay
Any suggestions will be helpful.
Since your structure is so rigid, you can cheat and use the built-in from_entries, which takes a list of {key, value} pairs and constructs an object:
.main[] |
{firstKey, id} +
(.data | map({key: "data.name.\(.name)", value}) |
from_entries)
I have json data like below:
{
"Items": [
{
"id": {
"S": "c921e4eb-5958-424a-ae3a-b9cada0d9481"
},
"type": {
"S": "transaction.1612878877726"
}
},
{
"id": {
"S": "355057f0-4327-49c7-979f-5a27410d81ba"
},
"type": {
"S": "transaction.1612345630260"
}
},
{
"id": {
"S": "664dc02f-0ad8-484a-98a5-a403beea775b"
},
"type": {
"S": "transaction.1612164919232"
}
},
...
]
}
I'd like to print the value id and type in one line per item from the Items array, e.g.
c921e4eb-5958-424a-ae3a-b9cada0d9481, transaction.1612878877726
355057f0-4327-49c7-979f-5a27410d81ba, transaction.1612345630260
...
I tried cat file | jq '.Items[].id.S, .Items[].type.S' but it prints id and type in separate lines. How can I achieve it with jq?
I would just use string manipulation, either adding 3 strings :
jq --raw-output '.Items[] | .id.S + ", " + .type.S' file
or using string interpolation :
jq --raw-output '.Items[] | "\(.id.S), \(.type.S)"' file
You can try it here.
I'm pasting here a JSON example data which would require some manipulation to get a desired output which is mentioned in the next section to be read after this piece of JSON code.
I want to use jq for parsing my desired data.
{
"MetricAlarms": [
{
"EvaluationPeriods": 3,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"AlarmActions": [
"Unimportant:Random:alarm:ELK2[10.1.1.2]-Root-Disk-Alert"
],
"AlarmName": "Unimportant:Random:alarm:ELK1[10.1.1.0]-Root-Alert",
"Dimensions": [
{
"Name": "path",
"Value": "/"
},
{
"Name": "InstanceType",
"Value": "m5.2xlarge"
},
{
"Name": "fstype",
"Value": "ext4"
}
],
"DatapointsToAlarm": 3,
"MetricName": "disk_used_percent"
},
{
"EvaluationPeriods": 3,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"AlarmActions": [
"Unimportant:Random:alarm:ELK2[10.1.1.2]"
],
"AlarmName": "Unimportant:Random:alarm:ELK2[10.1.1.2]",
"Dimensions": [
{
"Name": "path",
"Value": "/"
},
{
"Name": "InstanceType",
"Value": "r5.2xlarge"
},
{
"Name": "fstype",
"Value": "ext4"
}
],
"DatapointsToAlarm": 3,
"MetricName": "disk_used_percent"
}
]
}
So when I Pass some Key1 & value1 as a parameter "Name": "InstanceType", to the JQ probably using cat | jq and output expected should be as below
m5.2xlarge
r5.2xlarge
A generic approach to search for a key-value pair (sk-sv) in input recursively and extract another key's value (pv) from objects found:
jq -r --arg sk Name \
--arg sv InstanceType \
--arg pv Value \
'.. | objects | select(contains({($sk): $sv})) | .[$pv]' file
I have two JSON files a.json and b.json. The contents in a.json file is a JSON object and inside b.json its an array.I want to add/update status field in each mappings in a.json by retrieving the value from b.json file.
a.json:
{
"title": 25886,
"data": {
"request": {
"c": 46369,
"t1": 1562050127.376641
},
},
"rs": {
"mappings": {
"12345": {
"id": "12345",
"name": "test",
"customer_id": "11228",
},
"45678": {
"id": "45678",
"name": "abc",
"customer_id": "11206",
}
}
}}
b.json:
[
{
"status": "pending",
"extra": {
"name": "test"
},
"enabled": true,
"id": "12345"
},
{
"status": "not_started",
"extra": {
"name": "abc"
},
"enabled": true,
"id": "45678"
}
]
Below is my expected output:
{
"title": 25886,
"data": {
"request": {
"c": 46369,
"t1": 1562050127.376641
},
},
"rs": {
"mappings": {
"12345": {
"id": "12345",
"name": "test",
"customer_id": "11228",
"status":"pending"
},
"45678": {
"id": "45678",
"name": "abc",
"customer_id": "11206",
"status":"not_started"
}
}
}}
In this expected JSON file we have status field whose value is retrieved from b.json file based on a matching id value. How to do this using jq ?
For the purposes of this problem, b.json essentially defines a dictionary, so for simplicity, efficiency and perhaps elegance,
it make sense to start by using the builtin function INDEX to create the relevant dictionary:
INDEX( $b[] | {id, status}; .id )
This assumes an invocation of jq along the lines of:
jq --argfile b b.json -f update.jq a.json
(Yes, I know --argfile has been deprecated. Feel free to choose another way to set $b to the contents of b.json.)
Now, to perform the update, it will be simplest to use the "update" operator, |=, in conjunction with map_values. (Feel free to check the jq manual :-)
Putting everything together:
INDEX( $b[] | {id, status}; .id ) as $dict
| .rs.mappings |= map_values( .status = $dict[.id].status )
Hi I have the below JSON file with nested object:
{
"Maps": {
"Campus": [
{
"name": "nus",
"Building": [
{
"name": "sde1",
"Floor": [
{
"name": "floor1"
},
{
"name": "floor2"
}
]
},
{
"name": "sde2"
}
]
},
{
"name": "ntu",
"Building": [
{
"name": "ece1",
"Floor": [
{
"name": "floor1"
},
{
"name": "floor2"
}
]
},
{
"name": "ece2"
}
]
}
]
}
}
I want to use jq to parse the above JSON file and get the below format:
nus>sde1>floor1
nus>sde1>floor2
ntu>ece1>floor1
ntu>ece1>floor2
basically I have to concatenate the Campus Name with Building Name and Floor name and put a < symbol in between.
If the nested object field Floor is not exist, ignore the parse and continue the next child object.
How to achieve that? thanks.
You can use the following jq command:
jq '.Maps.Campus[]|"\(.name)>\(.Building[]|"\(.name)>\(.Floor[]?.name)")"' file.json
jq is smart enough to print the combinations of .name and .Building[].name since .Building is an array. The same action get's applied to .Building[].name and Floor[]?.name. ? because floor is not always set.
Here is a solution which uses jq variables
.Maps.Campus[]
| .name as $campus
| .Building[]
| .name as $bldg
| .Floor[]?
| .name as $floor
| "\($campus)>\($bldg)>\($floor)"