My mule code is hitting two tables and getting some details.
First one is order details, which I am storing in a flow variable i.e order and another database is returning order item details which I am storing in orderitem variable.
I want to aggregate both the payload based on one condition. Every orderId has order items (which is stored in flowVars.orderitem) and map these order items to respective orderID.
flowVars.order value is as below
[{partnerId=e83185e9f33e4234ba9eaa81dba515ad, orderId=12345, orderDate=2017-02-28 16:41:41.0, id=22}, {partnerId=e83185e9f33e4234ba9eaa81dba515ad, orderId=123456, orderDate=2017-02-28 16:41:41.0, id=23}, {partnerId=e83185e9f33e4234ba9eaa81dba515ad, orderId=11111, orderDate=2017-02-28 16:41:41.0, id=24}, {partnerId=e83185e9f33e4234ba9eaa81dba515ad, orderId=321123, orderDate=2017-05-19 15:25:41.0, id=26}]
and flowVars.orderitem value is as below
[{productCode=ELT-LP-ICND1-020067, orderId=12345, quantity=10, id=14}, {productCode=ELT-IP-ICND1-1.0, orderId=12345, quantity=11, id=15}, {productCode=ELT-LP-ICND1-020067, orderId=123456, quantity=12, id=16}, {productCode=ELT-IP-ICND1-1.0, orderId=123456, quantity=13, id=17}, {productCode=ELT-LP-ICND1-020067, orderId=11111, quantity=14, id=18}, {productCode=ELT-IP-ICND1-1.0, orderId=11111, quantity=15, id=19}, {productCode=ELT-LP-ICND2-020067, orderId=321123, quantity=5, id=20}]
Expected Output
[
{
"orderId": "12345",
"orderDate": "2017-02-28T16:41:41",
"partnerId": "e83185e9f33e4234ba9eaa81dba515ad",
"orderItems": [
{
"productCode": "ELT-LP-ICND1-020067",
"quantity": "10"
},
{
"productCode": "ELT-IP-ICND1-1.0",
"quantity": "11"
}
]
},
{
"orderId": "123456",
"orderDate": "2017-02-28T16:41:41",
"partnerId": "e83185e9f33e4234ba9eaa81dba515ad",
"orderItems": [
{
"productCode": "ELT-LP-ICND1-020067",
"quantity": "12"
},
{
"productCode": "ELT-IP-ICND1-1.0",
"quantity": "13"
}
]
},
{
"orderId": "11111",
"orderDate": "2017-02-28T16:41:41",
"partnerId": "e83185e9f33e4234ba9eaa81dba515ad",
"orderItems": [
{
"productCode": "ELT-LP-ICND1-020067",
"quantity": "14"
},
{
"productCode": "ELT-IP-ICND1-1.0",
"quantity": "15"
}
]
},
{
"orderId": "321123",
"orderDate": "2017-05-19T15:25:41",
"partnerId": "e83185e9f33e4234ba9eaa81dba515ad",
"orderItems": [
{
"productCode": "ELT-LP-ICND1-020067",
"quantity": "5"
}
]
}
]
Here I need to show respective order item details of an order. So basically I need to combine both the payloads.
I tried using dataweave but not luck.
Dataweave code:
%dw 1.0
%output application/json
%var mergeddata = flowVars.orderitem groupBy $.orderId
---
flowVars.order map ((data,index) ->
{
orderid: data.orderId,
partnerid: data.partnerId,
orderdate: data.orderDate,
order: flowVars.orderitem default [] map ((data1 ,indexOf) ->
{
(productcode: data1.productCode) when (data1.orderId == data.orderId),
(quantity: data1.quantity) when (data1.orderId == data.orderId) ,
(id: data1.id) when (data1.orderId == data.orderId)
}
)})
And output after transformation:
{
"orderid": "12345",
"partnerid": "e83185e9f33e4234ba9eaa81dba515ad",
"orderdate": "2017-02-28T16:41:41",
"order": [
{
"productcode": "ELT-LP-ICND1-020067",
"quantity": 10,
"id": 14
},
{
"productcode": "ELT-IP-ICND1-1.0",
"quantity": 11,
"id": 15
},
{
},
{
},
{
},
{
},
{
}
]
},
{
"orderid": "123456",
"partnerid": "e83185e9f33e4234ba9eaa81dba515ad",
"orderdate": "2017-02-28T16:41:41",
"order": [
{
},
{
},
{
"productcode": "ELT-LP-ICND1-020067",
"quantity": 12,
"id": 16
},
{
"productcode": "ELT-IP-ICND1-1.0",
"quantity": 13,
"id": 17
},
{
},
{
},
{
}
]
},
{
"orderid": "11111",
"partnerid": "e83185e9f33e4234ba9eaa81dba515ad",
"orderdate": "2017-02-28T16:41:41",
"order": [
{
},
{
},
{
},
{
},
{
"productcode": "ELT-LP-ICND1-020067",
"quantity": 14,
"id": 18
},
{
"productcode": "ELT-IP-ICND1-1.0",
"quantity": 15,
"id": 19
},
{
}
]
},
{
"orderid": "321123",
"partnerid": "e83185e9f33e4234ba9eaa81dba515ad",
"orderdate": "2017-05-19T15:25:41",
"order": [
{
},
{
},
{
},
{
},
{
},
{
},
{
"productcode": "ELT-LP-ICND2-020067",
"quantity": 5,
"id": 20
}
]
}
]
As you can see that I am almost there and able to map order item details with respective orderId but still I am getting some blank values after transformation.
Can anyone help me to achieve expected output. Thanks in advance!!!
You need to filter the flowVars.orderitem map, rather than iterate it in full and only print values when the orderId matches.
order: ((flowVars.orderitem default []) filter (data.orderId == $.orderId)) map ((data1 ,indexOf) -> {
productcode: data1.productCode,
quantity: data1.quantity
id: data1.id
})
You can then remove all of those 'when' statements too.
Related
Now I have a new situation Version 3.0.
I have this fake json:
[
{
"type":"PF",
"code":12345,
"Name":"Darth Vader",
"currency":"BRL",
"status":"ACTIVE",
"localization":"NABOO",
"createDate":1627990848665,
"olderAdress":[
{
"localization":"DEATH STAR",
"status":"BLOCKED",
"createDate":1627990848665
},
{
"localization":"TATOOINE",
"status":"CANCELLED",
"createDate":1627990555665
},
{
"localization":"ALDERAAN",
"status":"INACTIVED",
"createDate":1627990555665
}
]
},
{
"type":"PF",
"code":12345,
"Name":"Anakin Skywalker",
"currency":"BRL",
"status":"ACTIVE",
"localization":"NABOO",
"createDate":1627990848665,
"olderAdress":null
}
]
And I need to add a new field in each array element ONLY IF THE ARRAY IS NOT NULL. But I need to do that by aggregate because I'm using this result in Spring before sending to the users.
I need this result:
[
{
"type": "PF",
"code": 12345,
"Name": "Darth Vader",
"currency": "BRL",
"status": "ACTIVE",
"localization": "NABOO",
"createDate": 1627990848665,
"olderAddress": [
{
"localization": "DEATH STAR",
"status": "BLOCKED",
"createDate": 1627990848665,
"isItemOfOlderAddress" : true
},
{
"localization": "TATOOINE",
"status": "CANCELLED",
"createDate": 1627990555665,
"isItemOfOlderAddress" : true
},
{
"localization": "ALDERAAN",
"status": "INACTIVED",
"createDate": 1627990555665,
"isItemOfOlderAddress" : true
},
]
},
{
"type": "PF",
"code": 12345,
"Name": "Anakin Skywalker",
"currency": "BRL",
"status": "ACTIVE",
"localization": "NABOO",
"createDate": 1627990848665,
"olderAdress": null
},
]
So I added the field isItemOfOlderAddress only where olderAddress is not null and where olderAddress is null I only show the default information. How can I do that?
Query
if olderAdress is an array(so not null also), add "isItemOfOlderAddress": true field to all members
else keep the old value(so keep the null also)
Test code here
db.collection.aggregate([
{
"$set": {
"olderAdress": {
"$cond": [
{
"$isArray": [
"$olderAdress"
]
},
{
"$map": {
"input": "$olderAdress",
"in": {
"$mergeObjects": [
"$$this",
{
"isItemOfOlderAddress": true
}
]
}
}
},
"$olderAdress"
]
}
}
}
])
I am new to python, using python3. I have json data like:
{
"message": {
"count": 46,
"limit": 1000,
"schools": [
{
"class": "1",
"class_id": "1c8***",
"charges": [
{
"cost": 10,
"breakdown": [
{
"books": "1",
"unitQuantity": "10"
}
]
}
],
"area": "maccau"
},
{
"class": "2",
"class_id": "1c3***",
"charges": [
{
"cost": 100,
"breakdown": [
{
"books": "1",
"unitQuantity": "100"
}
]
}
],
"area": "maccau"
},
{
"class": "1",
"class_id": "1c3***",
"charges": [
{
"cost": 10,
"breakdown": [
{
"books": "1",
"unitQuantity": "10"
}
]
}
],
"area": "maccau"
},
{
"class": "2",
"class_id": "1c8***",
"charges": [
{
"cost": 50,
"breakdown": [
{
"books": "1",
"unitQuantity": "50"
}
]
}
],
"area": "maccau"
}
],
"url": {
"link": "/"
}
}
}
I was able to use json.loads to load data and I am trying to get results like:
class Cost
1 20
2 150
I tried converting json to a dictionary:
item_dict = json.load(json_data)
Tried to get data out using for loop and checking if class = 1 and then summing up the cost. But I feel like that is not the best approach. Can someone please tell me what would be the best way of doing this?
I have json stored in a column (oid) with the following structure:
{
"fullName": "test test",
"personDetails": {
"address": "Advisor",
"phoneNumber": "clare.railton#heptonstalls.co.uk"
},
"id": "6765788-yt67",
"submittedDocument": {
"answers": [
{
"questionId": "2",
"responses": [
{
"value": "123456"
}
]
},
{
"questionId": "2.1",
"responses": [
{
"IdA": 1,
"IdB": 1,
"value": "false"
},
{
"IdA": 1,
"IdB": 2,
"value": "false"
},
{
"IdA": 1,
"IdB": 3,
"value": "false"
},
{
"IdA": 1,
"IdB": 4,
"value": "true"
}
]
}
]
},
"date": "2018-11-22",
"PeriodId": 123456
}
How would i get the value of the response to all question numbers? I have managed to get the json structure from the oid column using the lo_get function but i am struggling to capture the values i need.
Many thanks
Are you sure you want a large object to store the json data?
Postgres can handle json columntypes in tables:
CREATE TABLE test_json (id INTEGER PRIMARY KEY, json json);
INSERT INTO test_json VALUES(1,'{
"fullName": "test test",
"personDetails": {
"address": "Advisor",
"phoneNumber": "clare.railton#heptonstalls.co.uk"
},
"id": "6765788-yt67",
"submittedDocument": {
"answers": [
{
"questionId": "2",
"responses": [
{
"value": "123456"
}
]
},
{
"questionId": "2.1",
"responses": [
{
"IdA": 1,
"IdB": 1,
"value": "false"
},
{
"IdA": 1,
"IdB": 2,
"value": "false"
},
{
"IdA": 1,
"IdB": 3,
"value": "false"
},
{
"IdA": 1,
"IdB": 4,
"value": "true"
}
]
}
]
},
"date": "2018-11-22",
"PeriodId": 123456
}');
SELECT json_extract_path(
json_array_elements(
json_extract_path(
json_array_elements(
json_extract_path((json),'submittedDocument','answers')
),'responses')
),'value'
)FROM test_json;
See:
https://www.postgresql.org/docs/current/functions-json.html
My json is like below. I am trying to group my objects based on the role id for target id equal to 1083.
My json:
[ {
"id" :1,
"role": {
"id": "25",
},
"target": {
"id": "1083",
}
},
{
"id" :2,
"role": {
"id": "25",
},
"target": {
"id": "1083",
}
},
{
"id" :3,
"role": {
"id": "25",
},
"target": {
"id": "1084",
}
},
{
"id" :4,
"role": {
"id": "3",
},
"target": {
"id": "1083",
}
}
]
Expected result:
{
"25" : [
{
"id": 1
"role": {
"id": "25",
},
"target": {
"id": "1083",
}
},
{ "id": 2
"role": {
"id": "25",
},
"target": {
"id": "1083",
}
}
],
"3": [
{
"id" :4,
"role": {
"id": "3",
},
"target": {
"id": "1083",
}
}]
}
So for my result json, it's grouped by 25 and 3. The third record doesn't exist in my expected result as it's target id is 1084
I tried filter to get only target id 1083, but when I tried to use grouby, I am getting errors
data.filter(o => {
return o.target.id === "1083"
})
With lodash via filter and groupBy:
var data = [{ "id": 1, "role": { "id": "25", }, "target": { "id": "1083", } }, { "id": 2, "role": { "id": "25", }, "target": { "id": "1083", } }, { "id": 3, "role": { "id": "25", }, "target": { "id": "1084", } }, { "id": 4, "role": { "id": "3", }, "target": { "id": "1083", } } ]
const groupIt = (targetId) => _.chain(data)
.filter(x => x.target.id ===targetId)
.groupBy('role.id')
.value()
console.log(groupIt('1083'))
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.11/lodash.min.js"></script>
You could achieve the same without lodash via filter and then reduce:
var data = [{ "id": 1, "role": { "id": "25", }, "target": { "id": "1083", } }, { "id": 2, "role": { "id": "25", }, "target": { "id": "1083", } }, { "id": 3, "role": { "id": "25", }, "target": { "id": "1084", } }, { "id": 4, "role": { "id": "3", }, "target": { "id": "1083", } } ]
const groupIt = (targetId) => data.filter(x => x.target.id === targetId)
.reduce((r, c) => (r[c.role.id] = [...r[c.role.id] || [], c], r), {})
console.log(groupIt('1083'))
I would like to exclude the items which don't have productModel property in the below JSON. How can we achieve this in groovy
I tried using hasProperty but not worked for me as expected. If possible can I get some sample snippet
I tried below code - but didn't work as I expected.
response.getAt('myData').getAt('data').getAt('product').hasProperty('productModel').each { println "result ${it}" }
Any help would be really appreciated.
{
"myData": [{
"data": {
"product": {
"productId": "apple",
"productName": "iPhone",
"productModel": "6s"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {}
},
{
"data": {
"product": {
"productId": "apple",
"productName": "iPhone",
"productModel": "7"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {}
},
{
"data": {
"product": {
"productId": "apple",
"productName": "Macbook"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {}
}
],
"metadata": {
"count": 3,
"offset": 0
}
}
If you want to exclude specific fields from JSON object then you have to recreate it using filtered data. The crucial part takes these two lines (assuming that json variable in the below example stores your JSON as text):
def root = new JsonSlurper().parseText(json)
def myData = root.myData.findAll { it.data.product.containsKey('productModel') }
What happens here is we access root.myData list and we filter it using findAll(predicate) method and predicate in this case says that only objects that have key productModel in path data.product are accepted. This findAll() method does not mutate existing list and that is why we store the result in variable myData - after running this method we will end up with a list of size 2.
In next step you have to recreate the object you want to represent as a JSON:
def newJsonObject = [
myData: myData,
metadata: [
count: myData.size(),
offset: 0
]
]
println JsonOutput.prettyPrint(JsonOutput.toJson(newJsonObject))
In this part we create newJsonObject and in the end we convert it to a JSON representation.
Here is the full example:
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
def json = '''{
"myData": [{
"data": {
"product": {
"productId": "apple",
"productName": "iPhone",
"productModel": "6s"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {}
},
{
"data": {
"product": {
"productId": "apple",
"productName": "iPhone",
"productModel": "7"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {}
},
{
"data": {
"product": {
"productId": "apple",
"productName": "Macbook"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {}
}
],
"metadata": {
"count": 3,
"offset": 0
}
}'''
def root = new JsonSlurper().parseText(json)
def myData = root.myData.findAll { it.data.product.containsKey('productModel') }
def newJsonObject = [
myData: myData,
metadata: [
count: myData.size(),
offset: 0
]
]
println JsonOutput.prettyPrint(JsonOutput.toJson(newJsonObject))
And here is the output it produces:
{
"myData": [
{
"data": {
"product": {
"productId": "apple",
"productName": "iPhone",
"productModel": "6s"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [
{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {
}
},
{
"data": {
"product": {
"productId": "apple",
"productName": "iPhone",
"productModel": "7"
},
"statusCode": "active",
"date": "2018-08-07T00:00:00.000Z"
},
"links": [
{
"productUrl": "test"
},
{
"productImage": "test"
}
],
"info": {
}
}
],
"metadata": {
"count": 2,
"offset": 0
}
}