I need to fetch the name of the array while traversing the child array items.
for example, if my input looks like
{"title": [
{
"value": "18724-100",
"locale": "en-GB"
},
{
"value": "18724-5",
"locale": "en-GB"
},
{
"value": "18724-99",
"locale": "fr-FR"
}
]}
I need output as
{
"data": [
{
"locale": "en-GB",
"metadata": [
{
"key": "title",
"value": "18724-100"
},
{
"key": "title",
"value": "18724-5"
}
]
},
{
"locale": "fr-FR",
"metadata": {
"key": "title",
"value": "18724-99"
}
}
]
}
I tried following spec in JSONata
{
"data": title{locale: value[]} ~> $each(function($v, $k) {
{
"locale": $k,
"metadata": $v.{"key": ???,"value": $}
}
})
}
Please help me to fill "???" so that I can get the array name
Assuming that the input object will always have a single root-level key you can write your expression like this:
{
"data": title{locale: value[]} ~> $each(function($v, $k) {
{
"locale": $k,
"metadata": $v.{"key": $keys($$)[0],"value": $}
}
})
}
$keys returns an array containing keys in the object. $keys($$) will return all keys in root-level of this array (in this case: "title").
Note that for a following input object:
{"title": [
{
"value": "18724-100",
"locale": "en-GB"
},
{
"value": "18724-5",
"locale": "en-GB"
},
{
"value": "18724-99",
"locale": "fr-FR"
}
],
"foo": 123
}
$keys($$) would return an array of two elements (["title", "foo"]).
I want to read hyperopt parameters from a JSON file.
My JSON file would be like:
[
{
"id": "121",
"model": [
{
"model_name": "power",
"estimator_type": [
{
"type": "Polynomial",
"degree": [2, 3, 4]
},
{
"type": "svm",
"C": [0, 1],
"kernel": [
{
"ktype": "linear"
},
{
"ktype": "RBF",
"width": [0, 1]
}
]
}
],
"cut_values": {
"qids": ["1234"]
}
},
{
"model_name": "speed",
"estimator_type": [
{
"type": "Polynomial",
"degree": ["quniform", 2, 3]
}
],
"cut_values": null
}
]
},
{
"id": "123",
"model": [
{
"model_name": "power",
"estimator_type": [
{
"type": "LinearRegression"
}
],
"cut_values": null
}
]
}
]
I have checked this post but with no success for more complex JSON like the one above.
I want to be able to create a space like 2.2 A Search Space Example: scikit-learn.
I've explored couple of existing JSON query language such JMESPath, JsonPath and JSONiq. Unfortunately, none of them seem to be able to support my use case in a generic way.
Basically, I'm receiving different type of responses from different web services. I need to give the ability to the user to remap the response in a 2 dimensional array in other to leverage our visualization tool. Based on the new format, the user can decide how to display his data between existing widgets. Pretty much like a customisable dashboard entirely managed on the UI.
Anyway my input looks like:
{
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
}
expected output:
[
{
"name": "medium",
"count": 10,
"category": "1"
},
{
"name": "high",
"count": 20,
"category": "1"
},
{
"name": "medium",
"count": 30,
"category": "2"
},
{
"name": "high",
"count": 40,
"category": "2"
}
]
The closer I went is with JMESPath but my query isn't dynamic at all. The user needs to be aware of possible category of grouping.
The query looks like: [ category_1[].{name: name, count: count, category: '1'}, category_2[].{name: name, count: count, category: '2'} ] | []
In other words, I need an enough powerful JSON query language to perform this JavaScript code:
const output = flatMap(input, (value, key) => {
return value.map(x => {
return { ...x, category: key };
});
});
Any thoughts?
This is indeed not currently possible in JMESPath (0.15.x). There are other spec compliant JMESPath packages that (with a bit of extra effort) will do what you require. Using NPM package #metrichor/jmespath (a typescript implementation) you could extend it with the functions you require as follows:
import {
registerFunction,
search,
TYPE_ARRAY,
TYPE_OBJECT
} from '#metrichor/jmespath';
registerFunction(
'flatMapValues',
([inputObject]) => {
return Object.entries(inputObject).reduce((flattened, entry) => {
const [key, value]: [string, any] = entry;
if (Array.isArray(value)) {
return [...flattened, ...value.map(v => [key, v])];
}
return [...flattened, [key, value]];
}, [] as any[]);
},
[{ types: [TYPE_OBJECT, TYPE_ARRAY] }],
);
With these extended functions a JMESPath expression would now look like this to remap the key into every value:
search("flatMapValues(#)[*].merge([1], {category: [0]})", {
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
});
// OUTPUTS:
[
{
category: 'category_1',
count: 10,
name: 'medium',
},
{
category: 'category_1',
count: 20,
name: 'high',
},
{
category: 'category_2',
count: 30,
name: 'medium',
},
{
category: 'category_2',
count: 40,
name: 'high',
},
]
That said you could just register the function you wrote above and use it
Finally, managed a way with JSONiq using Zorba implementation. Definitively the way to go if you need powerful JSON queries. Apparently this has been integrated in Apache Spark with Rumble
Anyway, here's my solution:
jsoniq version "1.0";
let $categories :=
{
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
}
for $key in keys($categories), $row in flatten($categories.$key)
return {"count": $row.count, "name": $row.name, "category": $key}
output:
{ "count" : 10, "name" : "medium", "category" : "category_1" }{ "count" : 20, "name" : "high", "category" : "category_1" }{ "count" : 30, "name" : "medium", "category" : "category_2" }{ "count" : 40, "name" : "high", "category" : "category_2" }
You can try Zorba here.
This is an alternative possibility in JSONiq that does not explicitly list the keys in each row, with the merge constructor {| |}:
jsoniq version "1.0";
let $categories :=
{
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
}
for $key in keys($categories),
$row in members($categories.$key)
return {|
$row,
{ "category": $key }
|}
For the sake of completeness, this is the reverse query that would turn the output back into the original input (which uses a group by clause):
jsoniq version "1.0";
let $output :=
(
{ "count" : 10, "name" : "medium", "category" : "category_1" },
{ "count" : 20, "name" : "high", "category" : "category_1" },
{ "count" : 30, "name" : "medium", "category" : "category_2" },
{ "count" : 40, "name" : "high", "category" : "category_2" }
)
return
{|
for $row in $output
group by $category := $row.category
return { $category : [ $row ] }
|}
This is simple with ~Q (disclaimer: I'm the developer).
{
"results:{}:[]": [{
"{}:":".",
"category":"$key"
}]
}
Output:
{
"results": [
{
"name": "medium",
"count": 10,
"category": "category_1"
},
{
"name": "high",
"count": 20,
"category": "category_1"
},
{
"name": "medium",
"count": 30,
"category": "category_2"
},
{
"name": "high",
"count": 40,
"category": "category_2"
}
]
}
Edit: some more info to explain the syntax:
"results:{}:[]"
The :{} part means "iterate over all keys in the object", :[] means "iterate over all array elements".
"{}:":"."
This copies each field in the current object to the output.
"category":"$key"
Add a field called "category", with the current traversed key as value.
If we wanted to get the numbers (i.e. 1,2,... instead of category_1, category_2, etc), we can use substr:
"category": "$key substr(9)"
You actually don't need any additional libs for that. Here is a small function which does the trick. You only need to split the key.
const transform = (obj) => {
const ret = [];
for (let key in obj) {
const tmp = key.split('_');
for (let item of obj[key]) {
ret.push({
...item,
[tmp[0]]: tmp[1],
});
}
}
return ret;
};
const result = transform(obj);
I am trying to use dataweave in Mule to read specific data values from an incoming payload. My sample payload looks like below:
{
"source": [
{
"uri": "entities/1R6xV",
"createdBy": "API_USER",
"createdTime": 1562504739146,
"attributes": {
"label": "000000000002659654",
"value": {
"Name": [
{
}
],
"Id": [
{
}
],
"Number": [
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
}
}
}
]
}
If I need to read the "label", I can achieve that by
label: payload.source.attributes.label
Similarly, how can I read the "value" under attributes > Number. It doesn't work by:
Value: payload.source.attributes.Number.value
I am new to Dataweave. Please advise.
The problem is that the dot selector (.) works on object and on array of objects. When it is applied to an array it will apply the dot selector to all the elements of the array that are of type object and return that result.
Lets go part by part
payload.source
Returns
[
{
"uri": "entities/1R6xV",
"createdBy": "API_USER",
"createdTime": 1562504739146,
"attributes": {
"label": "000000000002659654",
"value": {
"Name": [
{
}
],
"Id": [
{
}
],
"Number": [
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
}
}
}
]
So far so good as payload is an Object it returns the value of source that is an array
payload.source.attributes
Returns
[
{
"label": "000000000002659654",
"value": {
"Name": [
{
}
],
"Id": [
{
}
],
"Number": [
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
}
}
]
Works ok because the result of payload.source was ended an Array of object so it will do that selection over those objects.
Now when you execute
payload.source.attributes.value.Number
It returns
[
[
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
]
That is an array of arrays and here is where it is broken.
My Solution
You have two alternatives here
Use flatten function
flatten(payload.source.attributes.value.Number).value
Use descendant selector
payload.source.attributes.value.Number..value
Since Number is an array, you need to specify the index you want. In this case, the zeroth element:
Value: payload.source[0].attributes.value.Number[0].value
If you have multiple numbers, it would look something like this:
%dw 1.0
%output application/json
---
values: payload.source[0].attributes.value.Number map {
value: $.value
}
What is the best way to serialize a dictionary when key is a complex type. For example consider this invalid json:
[
{ParentId:1, ParentName:'X'}:
[{'ChildId':'1', 'ChildName':'a'}, {'ChildId':'2', 'ChildName':'b'}],
{ParentId:2, ParentName:'Y'}:
[{'ChildId':"3", 'ChildName':'c'}]}
]
Is there any way to correctly serialize this?
You can wrap the json with keys as strings and values as your complex object.
In addition, your keys of the complex type should be strings as well.
For example:
[
{
"name": {
"ParentId": 1,
"ParentName": "X"
},
"value": [
{
"ChildId": "1",
"ChildName": "a"
},
{
"ChildId": "2",
"ChildName": "b"
}
]
},
{
"name": {
"ParentId": 2,
"ParentName": "Y"
},
"value": [
{
"ChildId": "3",
"ChildName": "c"
}
]
}
]
Unfortunately this is not a valid json, so the answer would be no :-( keys have to be strings. You could reorganize your data structure to wrap your children list into a "parent" object which has an id and a name !
Basically it would look like :
[
{
"ParentId": 1,
"ParentName": "X",
"children": [
{
"ChildId": 1,
"ChildName": "a"
},
{
"ChildId": 2,
"ChildName": "b"
}
]
},
{
"ParentId": 2,
"ParentName": "Y",
"children": [
{
"ChildId": 3,
"ChildName": "c"
}
]
}
]
You can find the specifications of JSON here : https://www.rfc-editor.org/rfc/rfc4627