I am trying to transform my JSON to different structure using JQ. I am able to partially achieve my new structure, however i get a additional blocks of data.
As i iterate i am able to get the structure in new format. But additional structures are coming.
Code Snippet- https://jqplay.org/s/3gulSlZiWz
JSON
{
"amazon": {
"activeitem": 2,
"createdDate": "2019-01-15T17:36:31.588Z",
"lastModifiedDate": "2019-01-15T17:36:31.588Z",
"user": "net",
"userType": "new",
"items": [
{
"id": 1,
"name": "harry potter",
"state": "sold",
"type": {
"branded": false,
"description": "artwork",
"contentLevel": "season"
}
},
{
"id": 2,
"name": "adidas shoes",
"state": "in inventory",
"type": {
"branded": false,
"description": "Spprts",
"contentLevel": "season"
}
},
{
"id": 3,
"name": "watch",
"state": "returned",
"type": {
"branded": false,
"description": "walking",
"contentLevel": "special"
}
},
{
"id": 4,
"name": "adidas shoes",
"state": "in inventory",
"type": {
"branded": false,
"description": "running",
"contentLevel": "winter"
}
}
],
"product": {
"id": 4,
"name": "adidas shoes",
"source": "dealer",
"destination": "resident"
}
}
}
JQ Query:
.amazon|
{
userType: .userType,
userName: .user,
itemCatalog: {
itemId: .items[].id,
name: .items[].name
}
}
Expected Response:
{
"userType": "new",
"userName": "net",
"itemCatalog": {
"itemId": 1,
"name": "harry potter"
},{
"itemId": 2,
"name": "adidas shoes"
}, {
"itemId": 3,
"name": "watch"
},{
"itemId": 4,
"name": "adidas shoes"
}
}
But getting something weird long duplicated response.
As already pointed out in a comment, the "expected response" is not JSON and probably not what you want anyway. The following would make sense and in any case illustrates how to iterate appropriately:
.amazon
| { userType: .userType,
userName: .user,
itemCatalog: (.items | map({ itemId: .id, name} ))
}
Output
{
"userType": "new",
"userName": "net",
"itemCatalog": [
{
"itemId": 1,
"name": "harry potter"
},
{
"itemId": 2,
"name": "adidas shoes"
},
{
"itemId": 3,
"name": "watch"
},
{
"itemId": 4,
"name": "adidas shoes"
}
]
}
Related
I have a lot of data I need to parse though.
I need to pull all pid's and price's.
`
[
{
"id": 159817,
"price": "10.69",
"stocked": true,
"store": {
"id": 809,
"nsn": "22036-0",
"pricingSource": "manual",
"lastUpdated": "2022-12-05T15:24:33.908Z"
},
"sharedFields": {
"type": "PRODUCT",
"id": 24549,
"pid": "12079",
"labels": [
{
"type": "default",
"value": "Chicken Sandwich",
"locale": "en"
},
{
"type": "fresh",
"value": "Chicken",
"locale": "en"
},
{
"type": "product_json",
"value": "Chicken",
"locale": "en"
}
],
"calMin": 600,
"calMax": 600,
"lastUpdated": "2021-12-31T13:49:22.794Z"
}
},
{
"id": 159818,
"price": "9.29",
"stocked": true,
"store": {
"id": 809,
"nsn": "22036-0",
"pricingSource": "manual",
"lastUpdated": "2022-12-05T15:24:33.908Z"
},
"sharedFields": {
"type": "PRODUCT",
"id": 25,
"pid": "1",
"labels": [
{
"type": "default",
"value": "Ham Sandwich",
"locale": "en"
},
{
"type": "fresh",
"value": "Ham",
"locale": "en"
}
],
"calMin": 540,
"calMax": 540,
"lastUpdated": "2021-07-09T19:30:00.326Z"
}
}
]
`
and I need to place them into a string like this, but on a scale of 150 products. I'd also need to change "pid" to "productId"
[{ "productId": "46238", "price": 6.09 }, { "productId": "40240", "price": 1.49 }]
I need to add a string before this data, but I'm pretty confident I can figure that part out.
I am pretty open to the easiest suggestion, whether that be VBS, Excel macro, etc.
I have a big file that has lines as below.
{
"total": 320,
"assets": [
{
"audit": {
"created": {
"date": "2019-09-30T12:38:01.421Z"
},
"updated": {}
},
"organizationId": "12345678",
"id": 211123898,
"name": "groupId:760c47ad-c9f2958be:assetId:8o-api",
"exchangeAssetName": "8O API",
"groupId": "760c47ad-c9f2958be",
"assetId": "8o-api",
"apis": [
{
"audit": {
"created": {
"date": "2019-09-30T12:38:03.139Z"
},
"updated": {
"date": "2020-03-09T21:37:55.745Z"
}
},
"organizationId": "12345678",
"id": 15822364,
"groupId": "760c47ad-c9f2958be",
"assetId": "8o-api",
"assetVersion": "1.0.0",
"productVersion": "v1",
"description": null,
"tags": [],
"order": 1,
"providerId": null,
"deprecated": false,
"lastActiveDate": "2021-01-15T22:43:33.881Z",
"isPublic": false,
"stage": "release",
"lastActiveDelta": 7,
"pinned": false,
"activeContractsCount": 6,
"autodiscoveryInstanceName": "v1:15822364"
}
],
"totalApis": 1,
"autodiscoveryApiName": "groupId:760c47ad-c9f2958be:assetId:8o-api"
},
{
"audit": {
"created": {
"date": "2018-06-22T19:41:35.760Z"
},
"updated": {
"date": "2018-09-13T06:20:51.151Z"
}
},
"organizationId": "760c47ad-c9f2958be",
"id": 210914379,
"name": "hips-ts",
"exchangeAssetName": "hips-ts",
"groupId": "760c47ad-c9f2958be",
"assetId": "hips-ts",
"apis": [
{
"audit": {
"created": {
"date": "2018-06-22T19:41:35.759Z"
},
"updated": {
"date": "2020-03-09T21:37:55.745Z"
}
},
"organizationId": "760c47ad-c9f2958be",
"id": 15470738,
"groupId": "760c47ad-c9f2958be",
"assetId": "hips-ts",
"assetVersion": "1.0.0",
"productVersion": "v1",
"description": null,
"tags": [],
"order": 1,
"providerId": null,
"deprecated": false,
"lastActiveDate": "2021-01-15T22:43:30.004Z",
"endpointUri": null,
"isPublic": false,
"stage": "release",
"lastActiveDelta": 11,
"pinned": false,
"activeContractsCount": 1,
"autodiscoveryInstanceName": "1-test"
}
],
"totalApis": 1,
"autodiscoveryApiName": "hips-ts"
}
]
}
I am trying to use jq to just get the assetId and the activeContractsCount from this in a comma separated way.
So my output for this text should be
8o-api, 6
hips-ts, 1
I tried the following wiht jq jq -r '[.assets[].assetId, .assets[].apis[].activeContractsCount]|#csv' and I tried mapping too but nothing seems to stick.
Can you help me here? Any help is appreciated.
The following will produce exactly two columns, one for the "assetID" and one for the count:
.assets[].apis[]
| [.assetId, .activeContractsCount]
| #csv
I figured it out as soon as I posted the question. But please let me know if there's a better way to do this.
jq -r '.assets[]|[.apis[].assetId, .apis[].activeContractsCount]|#csv'
I have an array of JSON objects formatted as follows:
[
{
"id": 1,
"names": [
{
"name": "Bulbasaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 1,
"types": [
{
"slot": 1,
"type": {
"name": "grass",
"url": "http://myserver.com:8000/api/v2/type/12/"
}
},
{
"slot": 2,
"type": {
"name": "poison",
"url": "http://myserver.com:8000/api/v2/type/4/"
}
}
]
},
{
"id": 2,
"names": [
{
"name": "Ivysaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 2,
"types": [
{
"slot": 1,
"type": {
"name": "ice",
"url": "http://myserver.com:8000/api/v2/type/10/"
}
},
{
"slot": 2,
"type": {
"name": "electric",
"url": "http://myserver.com:8000/api/v2/type/8/"
}
}
]
},
{
"id": 3,
"names": [
{
"name": "Venusaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 3,
"types": [
{
"slot": 1,
"type": {
"name": "ground",
"url": "http://myserver.com:8000/api/v2/type/2/"
}
},
{
"slot": 2,
"type": {
"name": "rock",
"url": "http://myserver.com:8000/api/v2/type/3/"
}
}
]
}
]
Note that these are pairs of separate objects that appear sequentially in a JSON array, with each pair sharing an id field. This pattern repeats several hundred times in the array. What I need to accomplish is to "merge" each id-sharing pair into one object. So, the resultant output would be
[
{
"id": 1,
"names": [
{
"name": "Bulbasaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "grass",
"url": "http://myserver.com:8000/api/v2/type/12/"
}
},
{
"slot": 2,
"type": {
"name": "poison",
"url": "http://myserver.com:8000/api/v2/type/4/"
}
}
]
},
{
"id": 2,
"names": [
{
"name": "Ivysaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "ice",
"url": "http://myserver.com:8000/api/v2/type/10/"
}
},
{
"slot": 2,
"type": {
"name": "electric",
"url": "http://myserver.com:8000/api/v2/type/8/"
}
}
]
},
{
"id": 3,
"names": [
{
"name": "Venusaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "ground",
"url": "http://myserver.com:8000/api/v2/type/2/"
}
},
{
"slot": 2,
"type": {
"name": "rock",
"url": "http://myserver.com:8000/api/v2/type/3/"
}
}
]
}
]
I've gotten these objects to appear next to each other via the group_by(.id) command, but I'm at a loss as to how I should actually combine them. I'm very much still a novice with jq so I'm a bit overwhelmed with the amount of possible solutions.
[Note: The following assumes that the data shown in the Q have been corrected so that they are valid JSON.]
The merging you want can be achieved by object addition (x + y). For example, given the two JSON objects as shown in the question (i.e., as a stream), you could write:
jq -s '.[0] + .[1]'
However, since the question also indicates these objects are actually in an array, let's next consider the case of an array with two objects. In that case, you could simply write:
jq add
Finally, if you have an array of arrays each of which is an array of objects, you could use map(add). Since you don't have a very large array, you could simply write:
group_by(.id) | map(add)
Please note that jq defines object addition in a non-commutative way. Specifically, there is a bias towards the right-most key.
I am trying to transform my JSON to different structure using JQ. I am able to achieve my new structure, however i am getting feilds with Null objects if they are not present in Source, my client wants to remove the fields if they are having null values..
As i iterate i am able to get the structure in new format. But additional structures are coming.
Code Snippet- https://jqplay.org/s/w2N_Ozg9Ag
JSON
{
"amazon": {
"activeitem": 2,
"createdDate": "2019-01-15T17:36:31.588Z",
"lastModifiedDate": "2019-01-15T17:36:31.588Z",
"user": "net",
"userType": "new",
"items": [
{
"id": 1,
"name": "harry potter",
"state": "sold",
"type": {
"branded": false,
"description": "artwork",
"contentLevel": "season"
}
},
{
"id": 2,
"name": "adidas shoes",
"state": null ,
"type": {
"branded": false,
"description": "Spprts",
"contentLevel": "season"
}
},
{
"id": 3,
"name": "watch",
"type": {
"branded": false,
"description": "walking",
"contentLevel": "special"
}
},
{
"id": 4,
"name": "adidas shoes",
"state": "in inventory",
"type": {
"branded": false,
"description": "running",
"contentLevel": "winter"
}
}
],
"product": {
"id": 4,
"name": "adidas shoes",
"source": "dealer",
"destination": "resident"
}
}
}
JQ Query:
.amazon | { userType: .userType, userName: .user, itemCatalog: (.items | map({ itemId: .id, name, state} )) }
Expected Response:
{
"userType": "new",
"userName": "net",
"itemCatalog": [
{
"itemId": 1,
"name": "harry potter",
"state": "sold"
},
{
"itemId": 2,
"name": "adidas shoes"
},
{
"itemId": 3,
"name": "watch"
},
{
"itemId": 4,
"name": "adidas shoes",
"state": "in inventory"
}
]
}
With the query i have, i am getting state : null for the entries which has empty or null values. I want to hide the field itself in these cases.
Horrible solution, delete them after the query. There must be a neater way? (I started using jq today)
.amazon | { userType: .userType, userName: .user, itemCatalog: (.items | map({ itemId: .id, name, state} )) }| del(.itemCatalog[].state |select(. == null))
https://jqplay.org/s/jlNYmJNi25
I am trying to get json converted from:
{
"Devices": [
{
"Udid": "7a2b0e6c928f2321a75e423ba23ae93d",
"SerialNumber": "RF1D232ZLEE",
"MacAddress": "40F232726FC8",
"Imei": "3576342323280150",
"EasId": "SEC1BC252327E92B",
"AssetNumber": "7a2b0e23223928f2321a75e423ba23ae93d",
"DeviceFriendlyName": "gel1 Android Android 5.0.1 ZLEE ",
"LocationGroupId": {
"Id": {
"Value": 19529
},
"Name": "Group Express"
},
"LocationGroupName": "Group Express",
"UserId": {
"Name": ""
},
"UserName": "",
"UserEmailAddress": "",
"Ownership": "S",
"PlatformId": {
"Id": {
"Value": 5
},
"Name": "Android"
},
"Platform": "Android",
"ModelId": {
"Id": {
"Value": 5
},
"Name": "samsung GT-I9505"
},
"Model": "samsung GT-I9505",
"OperatingSystem": "5.0.1",
"PhoneNumber": "+447881867010",
"LastSeen": "2016-07-06T14:01:03.590",
"EnrollmentStatus": "Unenrolled",
"ComplianceStatus": "NotAvailable",
"CompromisedStatus": false,
"LastEnrolledOn": "2016-06-15T16:01:38.763",
"LastComplianceCheckOn": "0001-01-01T00:00:00.000",
"LastCompromisedCheckOn": "2016-07-06T13:58:26.183",
"IsSupervised": false,
"DeviceMCC": {
"SIMMCC": "234",
"CurrentMCC": "234"
},
"AcLineStatus": 0,
"VirtualMemory": 0,
"Id": {
"Value": 23459
}
},
{
"Udid": "c5f94db71d406dae7f881d3edf059e",
"SerialNumber": "",
"MacAddress": "000C300F9108",
"Imei": "",
"EasId": "D80DB85EC411C8E9B28BC292A603F05C2C0EEEC8",
"AssetNumber": "c592f93db71d406dae7f881d3edf059e",
"DeviceFriendlyName": "user Windows 10 WinRT 10.0.10240 ",
"LocationGroupId": {
"Id": {
"Value": 18498
},
"Name": "Business Solutions"
},
"LocationGroupName": "Business Solutions",
"UserId": {
"Name": ""
},
"UserName": "",
"UserEmailAddress": "",
"Ownership": "C",
"PlatformId": {
"Id": {
"Value": 12
},
"Name": "WinRT"
},
"Platform": "WinRT",
"ModelId": {
"Id": {
"Value": 50
},
"Name": "Windows 10"
},
"Model": "Windows 10",
"OperatingSystem": "10.0.10240",
"PhoneNumber": "",
"LastSeen": "2016-05-03T10:54:07.650",
"EnrollmentStatus": "Unenrolled",
"ComplianceStatus": "NotAvailable",
"CompromisedStatus": false,
"LastEnrolledOn": "2016-01-29T16:41:57.760",
"LastComplianceCheckOn": "0001-01-01T00:00:00.000",
"LastCompromisedCheckOn": "0001-01-01T00:00:00.000",
"IsSupervised": false,
"DeviceMCC": {
"SIMMCC": "",
"CurrentMCC": ""
},
"AcLineStatus": 0,
"VirtualMemory": 0,
"Id": {
"Value": 23545
}
}
],
"Page": 0,
"PageSize": 500,
"Total": 13}
To something like:
{"name": "Devices",
"children": [
{"name":"Udid", "size":"7f0dsda63274692ea4f0b66fec67a020158"},
{"name":"SerialNumber", "size":"P988KJSPQF938"},
{"name":"MacAddress", "size":"1HJUSUD031C4"},
{"name":"Imei", "size":""},
{"name":"EasId", "size":"ApKJSPQF193"},
{"name":"AssetNumber", "size":"7f0cda636b3305fea4f0b66fec9997267a020158"},
{"name":"DeviceFriendlyName", "size":"TMcKenz iPad iOS 7.1.4 F193 "},
{"name":"LocationGroupId",
"children": [
{"name":"Id","size":7488},
{"name":"Name","size":"MCM"}
]
},
{"name":"UserId",
"children": [
{"name":"Id","size":6418},
{"name":"Name","size":"Tom McKenz"}
]
},
{"name":"UserName", "size":"TMcKenz"},
{"name":"UserEmailAddress", "size":"TMcKenz#awaw.com"}
]
}
Not sure what is the best practice here: is it possible to use D3.nest or do you need to iterate through all the nodes and change to 'name' and 'children' accordingly.