I have the following JSON;
[
{
"id": 1,
"start": "2022-06-20",
"result": 24
},
{
"id": 2,
"start": "2022-06-21",
"result": 56
},
{
"id": 3,
"start": "2022-06-21",
"result": 78
}
]
I'm wanting to add 2 new values to each array above using JQ, dimension and date, but date needs to be a copy of the existing key value start. The expected output is as below;
[
{
"id": 1,
"start": "2022-06-20",
"result": 24,
"date": "2022-06-20",
"dimension": "new"
},
{
"id": 2,
"start": "2022-06-21",
"result": 56,
"date": "2022-06-21",
"dimension": "new"
},
{
"id": 3,
"start": "2022-06-21",
"result": 78,
"date": "2022-06-21",
"dimension": "new"
}
]
The jq I have at present can add the new key dimension, but I can't figure out how to copy start -> date
jq '.[] += {"dimension": "new"}' input.json
Thanks for any help
Just create the new key / value pairs.
jq 'map(.date = .start | .dimension = "new")' input.json
You might have tried the following:
.[] += { date: .start, dimension: "new" } // WRONG
But it's not quite right since . is the array, not the element of the array. You can use |= as a topicalizer.
.[] |= ( . += { date: .start, dimension: "new" } )
But I'd use map instead.
map( . += { date: .start, dimension: "new" } )
Alternatively,
. += { date: .start, dimension: "new" }
can also also be achieved using
.date = .start | .dimension: "new"
So you could use
.[] |= ( .date = .start | .dimension: "new" )
or
map( .date = .start | .dimension = "new" )
I have json data like below:
{
"Items": [
{
"id": {
"S": "c921e4eb-5958-424a-ae3a-b9cada0d9481"
},
"type": {
"S": "transaction.1612878877726"
}
},
{
"id": {
"S": "355057f0-4327-49c7-979f-5a27410d81ba"
},
"type": {
"S": "transaction.1612345630260"
}
},
{
"id": {
"S": "664dc02f-0ad8-484a-98a5-a403beea775b"
},
"type": {
"S": "transaction.1612164919232"
}
},
...
]
}
I'd like to print the value id and type in one line per item from the Items array, e.g.
c921e4eb-5958-424a-ae3a-b9cada0d9481, transaction.1612878877726
355057f0-4327-49c7-979f-5a27410d81ba, transaction.1612345630260
...
I tried cat file | jq '.Items[].id.S, .Items[].type.S' but it prints id and type in separate lines. How can I achieve it with jq?
I would just use string manipulation, either adding 3 strings :
jq --raw-output '.Items[] | .id.S + ", " + .type.S' file
or using string interpolation :
jq --raw-output '.Items[] | "\(.id.S), \(.type.S)"' file
You can try it here.
This is how my input looks:
{
"text" : "Some text here"
}
{
"usage": {
"text_units": 1,
"text_characters": 101,
"features": 1
},
"language": "en",
"categories": [
{
"score": 0.655041,
"label": "/technology law, govt and politics/espionage and intelligence/surveillance"
},
{
"score": 0.639809,
"label": "/technology and computing/computer security/network security"
},
{
"score": 0.624533,
"label": "/business and industrial/business operations"
}
]
}
Using JQ, if the first element of array category in the second object contains /technology, I want to add a new field named relevant with 1 as value (which I managed), and copy the text field from the first object.
So, the expected output is:
{
"usage": {
"text_units": 1,
"text_characters": 101,
"features": 1
},
"language": "en",
"categories": [
{
"score": 0.655041,
"label": "/technology law, govt and politics/espionage and intelligence/surveillance"
},
{
"score": 0.639809,
"label": "/technology and computing/computer security/network security"
},
{
"score": 0.624533,
"label": "/business and industrial/business operations"
}
],
"relevant": 1,
"text": "Some text here"
}
And this is what I have done so far:
if .categories[0].label | test("/technology"; "i") then . |=( . + {"relevant": 1} + {"text": .text}) else . |= . + {"relevant": 0} end
Link to a demo on jqplay
Your input consists of two separate objects. In order to be able to access the first while processing the second, you could save the first into a variable.
. as {$text} | input | if .categories[0].label | test("/technology"; "i") then . + {relevant: 1, $text} else . + {relevant: 0} end
Online demo
How to check for null or empty in jq and substitute for empty string in jq transformation.
Example in below JSON, this is the JQ
JQ:
.amazon.items[] | select(.name | contains ("shoes")) as $item |
{
activeItem: .amazon.activeitem,
item : {
id : $item.id,
state : $item.state,
status : if [[ $item.status = "" or $item.status = null ]];
then 'IN PROCESS' ; else $item.status end
}
}
JSON:
{
"amazon": {
"activeitem": 2,
"items": [
{
"id": 1,
"name": "harry potter",
"state": "sold"
},
{
"id": 2,
"name": "adidas shoes",
"state": "in inventory"
},
{
"id": 3,
"name": "watch",
"state": "returned"
},{
"id": 4,
"name": "Nike shoes",
"state": "in inventory"
}
]
}
}
I want to add a default string "In Process" if the status is empty or Null.
Based on Item condition, using the query below and take the first object from the filtered results.
code
.amazon.items[] | select(.name | contains ("shoes"))
code
Expected Output:
{
"activeitem": 2,
"item": {
"id": 2,
"name": "adidas shoes",
"state": "in inventory",
"status": "IN PROCESS"
}
}
The key here is to use |=:
.amazon.item.status |=
if . == null or . == ""
then "IN PROCESS"
else .
end
I have a json file that I need to convert to a csv file, but I am a little wary of trusting a json-to-csv converter site as the outputted data seems to be incorrect... so I was hoping to get some help here!
I have the following json file structure:
{
"GroupName": "GrpName13",
"Number": 3,
"Notes": "Test Group ",
"Units": [
{
"UnitNumber": "TestUnit13",
"DataSource": "Factory",
"ContractNumber": "TestContract13",
"CarNumber": "2",
"ControllerTypeMessageId" : 4,
"NumberOfLandings": 4,
"CreatedBy": "user1",
"CommissionModeMessageId": 2,
"Details": [
{
"DetailName": "TestFloor13",
"DetailNumber": "5"
}
],
"UnitDevices": [
{
"DeviceTypeMessageId": 1,
"CreatedBy": "user1"
}
]
}
]
}
The issue I think Im seeing is that the converters seem to not be able to comprehend the many nested data values. And the reason I think the converters are wrong is because when I try to convert back to json using them, I dont receive the same structure.
Does anyone know how to manually format this json into csv format, or know of a reliable converter than can handle nested values?
Try
www.json-buddy.com/convert-json-csv-xml.htm
if not working for you then you can try this tool
http://download.cnet.com/JSON-to-CSV/3000-2383_4-76680683.html
should be helpful!
I have tried your json on this for url:
http://www.convertcsv.com/json-to-csv.htm
As a result:
UnitNumber,DataSource,ContractNumber,CarNumber,ControllerTypeMessageId,NumberOfLandings,CreatedBy,CommissionModeMessageId,Details/0/DetailName,Details/0/DetailNumber,UnitDevices/0/DeviceTypeMessageId,UnitDevices/0/CreatedBy
TestUnit13,Factory,TestContract13,2,4,4,user1,2,TestFloor13,5,1,user1
Because it could save the path of the key,like the 'DeviceTypeMessageId' in list 'UnitDevices': it will named the columns name with 'UnitDevices/0/DeviceTypeMessageId', this could avoid the same name mistake, so you can get the columns name by its converter rules.
Hope helpful.
Here is a solution using jq
If the file filter.jq contains
def denormalize:
def headers($p):
keys_unsorted[] as $k
| if .[$k]|type == "array" then (.[$k]|first|headers("\($p)\($k)_"))
else "\($p)\($k)"
end
;
def setup:
[
keys_unsorted[] as $k
| if .[$k]|type == "array" then [ .[$k][]| setup ]
else .[$k]
end
]
;
def iter:
if length == 0 then []
elif .[0]|type != "array" then
[.[0]] + (.[1:] | iter)
else
(.[0][] | iter) as $x
| (.[1:] | iter) as $y
| [$x[]] + $y
end
;
[ headers("") ], (setup | iter)
;
denormalize | #csv
and data.json contains (note extra samples added)
{
"GroupName": "GrpName13",
"Notes": "Test Group ",
"Number": 3,
"Units": [
{
"CarNumber": "2",
"CommissionModeMessageId": 2,
"ContractNumber": "TestContract13",
"ControllerTypeMessageId": 4,
"CreatedBy": "user1",
"DataSource": "Factory",
"Details": [
{
"DetailName": "TestFloor13",
"DetailNumber": "5"
}
],
"NumberOfLandings": 4,
"UnitDevices": [
{
"CreatedBy": "user1",
"DeviceTypeMessageId": 1
},
{
"CreatedBy": "user10",
"DeviceTypeMessageId": 10
}
],
"UnitNumber": "TestUnit13"
},
{
"CarNumber": "99",
"CommissionModeMessageId": 99,
"ContractNumber": "Contract99",
"ControllerTypeMessageId": 99,
"CreatedBy": "user99",
"DataSource": "Another Factory",
"Details": [
{
"DetailName": "TestFloor99",
"DetailNumber": "99"
}
],
"NumberOfLandings": 99,
"UnitDevices": [
{
"CreatedBy": "user99",
"DeviceTypeMessageId": 99
}
],
"UnitNumber": "Unit99"
}
]
}
then the command
jq -M -r -f filter.jq data.json
will produce
"GroupName","Notes","Number","Units_CarNumber","Units_CommissionModeMessageId","Units_ContractNumber","Units_ControllerTypeMessageId","Units_CreatedBy","Units_DataSource","Units_Details_DetailName","Units_Details_DetailNumber","Units_NumberOfLandings","Units_UnitDevices_CreatedBy","Units_UnitDevices_DeviceTypeMessageId","Units_UnitNumber"
"GrpName13","Test Group ",3,"2",2,"TestContract13",4,"user1","Factory","TestFloor13","5",4,"user1",1,"TestUnit13"
"GrpName13","Test Group ",3,"2",2,"TestContract13",4,"user1","Factory","TestFloor13","5",4,"user10",10,"TestUnit13"
"GrpName13","Test Group ",3,"99",99,"Contract99",99,"user99","Another Factory","TestFloor99","99",99,"user99",99,"Unit99"