Getting unique values from nested Array using jq - json

Trying to get unique values stored in items array for each group. somehow it's always mixed...
My JSON looks like this:
{
"start": 1534425916,
"stop": 1535030716,
"groups": [
{
"group": "transmission",
"data": {
"events": 665762,
},
"items": [
{
"item": "manualni",
"data": {
"events": 389158,
}
},
{
"item": "automaticka",
"data": {
"events": 276604,
}
}
]
},
{
"group": "vat",
"data": {
"events": 671924,
},
"items": [
{
"item": "ne",
"data": {
"events": 346221,
}
},
{
"item": "ano",
"data": {
"events": 325703,
}
}
]
}
]
}
Desired result is the following:
{
"id": "transmission",
"value": [
"manualni",
"automaticka",
]
}
{
"id": "vat",
"value": [
"ne",
"ano"
]
}
Tried with this filter on command line:
| jq '{id: .groups[].group, value: [.groups[].items[].item]}'
Which results in the above mentioned mixed up result:
{
"id": "transmission",
"value": [
"manualni",
"automaticka",
"ne",
"ano"
]
}
{
"id": "vat",
"value": [
"manualni",
"automaticka",
"ne",
"ano"
]
}
Any idea how to receive the uniquified values here? Thanks in advance!

This gets the desired result. I think the manual entry under .[] explains why it works.
jq '.groups[] | {"id": .group, "value": [.items[].item]}'

Related

How to remove empty json object from nested json structure

In my current JSON, I am getting an empty JSON object {} inside dummy_var5. The empty object is inside an array which is inside an object itself.
{
"dummy_var1": "abc",
"dummy_var2": [
{
"item": {
"action": "test",
"po": {
"id": "abc"
},
"ot": "test1",
"id": "1"
}
}
],
"dummy_var3": {
"dummy_var4": [
{
"name": "test",
"value": "test1"
},
{
"name": "test",
"value": "test1"
}
],
"name": "test2"
},
"dummy_var5": [
{
"ref": "test",
"name": "test1",
"type": null
},
{}
],
"dummy_var6": [
{
"role": "test",
"ref": "test1",
"partyDescription": "test2"
}
]
}
Considering this structure does not change, Is there any way to remove this via jolt. We tried using third party tool 'atlasmap' but couldn't achieve the desired result.
Expected output :-
{
"dummy_var1": "abc",
"dummy_var2": [
{
"item": {
"action": "test",
"po": {
"id": "abc"
},
"ot": "test1",
"id": "1"
}
}
],
"dummy_var3": {
"dummy_var4": [
{
"name": "test",
"value": "test1"
},
{
"name": "test",
"value": "test1"
}
],
"name": "test2"
},
"dummy_var5": [
{
"ref": "test",
"name": "test1",
"type": null
}
],
"dummy_var6": [
{
"role": "test",
"ref": "test1",
"partyDescription": "test2"
}
]
}
You can use this single shift transformation spec
[
{
"operation": "shift",
"spec": {
"*": "&", // the attributes other than "dummy_var5"
"dummy_var5": {
"*": {
"*": "&2.[&1].&"
}
}
}
}
]
the match "*":"&" of the line "*": "&2.[&1].&" returns the null value from the leaf node for this level, and so removes the innermost null object {}

find and replace in json file using jq filter

I've below json file env.json and I want to search for "[\"res\",\"q3\"]" and replace it with a variable var1 value "[\"res\"]"
{
"idsb": "marqd",
"data": {
"name": "bcon-dv-alert"
},
"ingress": {
"args": {
"params": [
{
"name": "spt",
"value": "cld"
},
{
"name": "scv",
"value": "sdv"
},
{
"name": "scr",
"value": "ord"
}
{
"name": "srm",
"value": "[\"res\",\"q3\"]"
},
{
"name": "tgo",
"value": "pbc"
}
]
},
"wfr": {
"name": "t-r-e"
},
"snm": "as-r"
}
}
I tried the below way but it's not working
var1="[\"res\"]"
jq '.ingress.args.params[] | select(.name=="srm").value |= ["'${var1}'"]' env.json
where am making mistake? what's the right way to do it?
The final result will be
{
"idsb": "marqd",
"data": {
"name": "bcon-dv-alert"
},
"ingress": {
"args": {
"params": [
{
"name": "spt",
"value": "cld"
},
{
"name": "scv",
"value": "sdv"
},
{
"name": "scr",
"value": "ord"
}
{
"name": "srm",
"value": "[\"res\"]"
},
{
"name": "tgo",
"value": "pbc"
}
]
},
"wfr": {
"name": "t-r-e"
},
"snm": "as-r"
}
}
Since you want to update ingress, and not return only the result of the loops, use:
.ingress.args.params |= map(select(.name=="srm").value |= "new-value")
Try it online

Is there any way to select all child array with just JsonPath

I have a json as below
{
"Shop": [
{
"id": "1",
"Items": [
{
"Item": "Item1"
}
]
},
{
"id": "2",
"Items": [
{
"Item": "Item2"
}
]
},
{
"id": "3",
"Items": [
{
"Item": "Item3"
}
]
}
]
}
I would like to select all Items with just JsonPath. I have tried as following combinations but I did not get any values
$.[Shop[0], Shop[1], Shop[2]].Items
$.[Shop[0].Items, Shop[1].Items, Shop[2].Items]
Thank you in advance
If I understand correctly you are looking for * wildcard to select all elements in an array:
$.Shop[*].Items
Gives me:
[ [
{
"Item": "Item1"
} ], [
{
"Item": "Item2"
} ], [
{
"Item": "Item3"
} ] ]

JOLT - Transform Array of JSON objects containing nested Arrays into their respective Array group

I want to transform a JSON input file into expected JSON output file:
My input file:
[
{
"category": [
{
"id": "id1",
"name": "Connected Home & Housewares1"
},
{
"id": "id2",
"name": "Housewares1"
},
{
"id": "id3",
"name": "Household Batteries1"
},
{
"id": "id4",
"name": "Alkaline Batteries1"
}
]
},
{
"category": [
{
"id": "id1",
"name": "Connected Home & Housewares2"
},
{
"id": "id2",
"name": "Housewares2"
},
{
"id": "id3",
"name": "Household Batteries2"
},
{
"id": "id4",
"name": "Alkaline Batteries2"
}
]
}
]
(Above is just an example of 2 records, but it contains ~50k records)
Expected Output:
[
{
"childSKUs": [
{
"divisionName": "Connected Home & Housewares1",
"deptName": "Housewares1",
"className": "Household Batteries1"
}
]
},
{
"childSKUs": [
{
"divisionName": "Connected Home & Housewares2",
"deptName": "Housewares2",
"className": "Household Batteries2"
}
]
}
]
My current JOLT Spec:
[
{
"operation": "shift",
"spec": {
"*": {
"category": {
"0": {
"name": "[0].childSKUs[&1].divisionName"
},
"1": {
"name": "[0].childSKUs[&1].deptName"
},
"2": {
"name": "[0].childSKUs[&1].className"
}
}
}
}
}
]
But it is only giving following output:
[
{
"childSKUs": [
{
"divisionName": [
"Connected Home & Housewares1",
"Connected Home & Housewares2"
]
},
{
"deptName": [
"Housewares1",
"Housewares2"
]
},
{
"className": [
"Household Batteries1",
"Household Batteries2"
]
}
]
}
]
I tried various options by playing in https://jolt-demo.appspot.com but no luck so far. Any help is appreciated. Would also be great if you explain your SPEC how was it done.
Ok, I figured it out myself.
Thanks to #Pokuri's answer here which gave me the idea to arrive at the solution: https://stackoverflow.com/a/55861132/248847
Correct JOLT SPEC for above question:
[
{
"operation": "shift",
"spec": {
"*": {
"category": {
"0": {
"name": "[&3].childSKUs[&1].divisionName"
},
"1": {
"name": "[&3].childSKUs[0].deptName"
},
"2": {
"name": "[&3].childSKUs[0].className"
}
}
}
}
}
]

denormalizing JSON with jq

I have JSON that looks like this:
[
{
"fields": {
"versions": [
{
"id": "36143",
"name": "ST card"
},
{
"id": "36144",
"description": "Acceptance test card",
"name": "AT card"
}
],
"severity": {
"value": "B-Serious",
"id": "14231"
}
}
},
{
"fields": {
"versions": [
{
"id": "36145",
"name": "ST card"
}
],
"severity": {
"value": "C-Limited",
"id": "14235"
}
}
}
]
I want to convert it with jq to this:
[
{
"id": "36143",
"name": "ST card"
"value": "B-Serious"
},
{
"id": "36144",
"name": "AT card"
"value": "B-Serious"
},
{
"id": "36145",
"name": "ST card"
"value": "C-Limited"
}
]
Note that the first object has 2 versions, and the same severity. I have tried jq's group_by and map functions but haven't been too successful. Please help :)
This should work. You wouldn't want to use a group_by here, you would do that if you were trying to go from more to less, we're going the other way.
You're combining the different versions with the corresponding severity. Here's how you could do that.
map(.fields | (.versions[] | { id, name }) + { value: .severity.value })