I am trying to use dataweave in Mule to read specific data values from an incoming payload. My sample payload looks like below:
{
"source": [
{
"uri": "entities/1R6xV",
"createdBy": "API_USER",
"createdTime": 1562504739146,
"attributes": {
"label": "000000000002659654",
"value": {
"Name": [
{
}
],
"Id": [
{
}
],
"Number": [
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
}
}
}
]
}
If I need to read the "label", I can achieve that by
label: payload.source.attributes.label
Similarly, how can I read the "value" under attributes > Number. It doesn't work by:
Value: payload.source.attributes.Number.value
I am new to Dataweave. Please advise.
The problem is that the dot selector (.) works on object and on array of objects. When it is applied to an array it will apply the dot selector to all the elements of the array that are of type object and return that result.
Lets go part by part
payload.source
Returns
[
{
"uri": "entities/1R6xV",
"createdBy": "API_USER",
"createdTime": 1562504739146,
"attributes": {
"label": "000000000002659654",
"value": {
"Name": [
{
}
],
"Id": [
{
}
],
"Number": [
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
}
}
}
]
So far so good as payload is an Object it returns the value of source that is an array
payload.source.attributes
Returns
[
{
"label": "000000000002659654",
"value": {
"Name": [
{
}
],
"Id": [
{
}
],
"Number": [
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
}
}
]
Works ok because the result of payload.source was ended an Array of object so it will do that selection over those objects.
Now when you execute
payload.source.attributes.value.Number
It returns
[
[
{
"type": "config/Types/Number/attributes/Number",
"ov": true,
"value": "000000000002659654",
"uri": "entities/1R6xV/attributes/Num/1ZtyT/Number/60pvN6"
}
]
]
That is an array of arrays and here is where it is broken.
My Solution
You have two alternatives here
Use flatten function
flatten(payload.source.attributes.value.Number).value
Use descendant selector
payload.source.attributes.value.Number..value
Since Number is an array, you need to specify the index you want. In this case, the zeroth element:
Value: payload.source[0].attributes.value.Number[0].value
If you have multiple numbers, it would look something like this:
%dw 1.0
%output application/json
---
values: payload.source[0].attributes.value.Number map {
value: $.value
}
Related
I have a requirement to roll a collection of nodes that uses the current node name (within the collection) and for the value take each child nodes value (single node) into a string array, then use the parents key as the key.
Given.
{
"client": {
"addresses": [
{
"id": "27ef465ef60d2705",
"type": "RegisteredOfficeAddress"
},
{
"id": "b7affb035be3f984",
"type": "PlaceOfBusiness"
},
{
"id": "a8a3bef166141206",
"type": "EmailAddress"
}
],
"links": [
{
"id": "29a9de859e70799e",
"type": "Director",
"name": "Bob the Builder"
},
{
"id": "22493ad4c4fd8ac5",
"type": "Secretary",
"name": "Jennifer"
}
],
"Names": [
{
"id": "53977967eadfffcd",
"type": "EntityName",
"name": "Banjo"
}
]
}
}
from this the output needs to be
{
"client": {
"addresses": [
"RegisteredOfficeAddress",
"PlaceOfBusiness",
"EmailAddress"
],
"links": [
"Director",
"Secretary"
],
"Names": [
"EntityName"
]
}
}
What is the best way to achieve this? Any pointers to what/how to do this would be greatly appreciated.
Ron.
You can iterate over entries of your client object first with the help of the $each function, then get types for each of them, and combine via $merge:
{
"client": client
~> $each(function($list, $key) {{ $key: $list.type }})
~> $merge
}
Live playground: https://stedi.link/OpuRdE9
I need to fetch the name of the array while traversing the child array items.
for example, if my input looks like
{"title": [
{
"value": "18724-100",
"locale": "en-GB"
},
{
"value": "18724-5",
"locale": "en-GB"
},
{
"value": "18724-99",
"locale": "fr-FR"
}
]}
I need output as
{
"data": [
{
"locale": "en-GB",
"metadata": [
{
"key": "title",
"value": "18724-100"
},
{
"key": "title",
"value": "18724-5"
}
]
},
{
"locale": "fr-FR",
"metadata": {
"key": "title",
"value": "18724-99"
}
}
]
}
I tried following spec in JSONata
{
"data": title{locale: value[]} ~> $each(function($v, $k) {
{
"locale": $k,
"metadata": $v.{"key": ???,"value": $}
}
})
}
Please help me to fill "???" so that I can get the array name
Assuming that the input object will always have a single root-level key you can write your expression like this:
{
"data": title{locale: value[]} ~> $each(function($v, $k) {
{
"locale": $k,
"metadata": $v.{"key": $keys($$)[0],"value": $}
}
})
}
$keys returns an array containing keys in the object. $keys($$) will return all keys in root-level of this array (in this case: "title").
Note that for a following input object:
{"title": [
{
"value": "18724-100",
"locale": "en-GB"
},
{
"value": "18724-5",
"locale": "en-GB"
},
{
"value": "18724-99",
"locale": "fr-FR"
}
],
"foo": 123
}
$keys($$) would return an array of two elements (["title", "foo"]).
I am trying to read a json string using Li Haoyi's ujson. This is the string:
{
"dataflows": [
{
"name": "test",
"sources": [
{
"name": "person_inputs",
"path": "/data/input/events/person/*",
"format": "JSON"
}
],
"transformations": [
{
"name": "validation",
"type": "validate_fields",
"params": {
"input": "person_inputs",
"validations": [
{
"field": "office",
"validations": [
"notEmpty"
]
},
{
"field": "age",
"validations": [
"notNull"
]
}
]
}
},
{
"name": "ok_with_date",
"type": "add_fields",
"params": {
"input": "validation_ok",
"addFields": [
{
"name": "dt",
"function": "current_timestamp"
}
]
}
}
],
"sinks": [
{
"input": "ok_with_date",
"name": "raw-ok",
"paths": [
"/data/output/events/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
},
{
"input": "validation_ko",
"name": "raw-ko",
"paths": [
"/data/output/discards/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
}
]
}
]
}
And this is how I read it:
val j = os.read(os.pwd/RelPath("src/main/scala/metadata.json"))
val jsonData = ujson.read(j)
But, the return type is ujson.Obj, and not Arr(ArrayBuffer(Obj), as expected, such that when I try to get jsonData(0), what I get is json.Value$InvalidData: Expected ujson.Arr.
I am asking this question because I have tried to use the ujson object to create a upickle object, but I cannot, and I suspect it is because of this initial error.
Any ideas of why this happens? Any help would be greatly appreciated! Thanks in advance!!
The outer element of your JSON is not an array, it is an object with a single element dataflows whose value is an array. Try jsonData("dataflows")(0).
I was wondering if there is any 'cleaner' way of looping through nested JSON in ruby?
This is my JSON object:
{
"data": [
{
"file": "test/test_project_js/jquery.js",
"results": [
{
"vulnerabilities": [
{
"severity": "high"
},
{
"severity": "medium"
},
{
"severity": "none"
},
{
"severity": "high"
}
]
}
]
},
{
"file": "test/test_project_js/jquery.js",
"results": [
{
"vulnerabilities": [
{
"severity": "none"
},
{
"severity": "none"
},
{
"severity": "none"
},
{
"severity": "high"
}
]
}
]
}
]
}
I want to extract severity under each vulnerability present inside each results[] which is under data[]
Current code approach is
severity_arr = raw['data'].each do |data|
data['results'].each do |result|
result['vulnerabilities'].map {|vulnerability| vulnerability['severity']}
end
end
You can use flat_map and dig:
data[:data].flat_map { |datum| datum.dig(:results, 0, :vulnerabilities) }
# [{:severity=>"high"}, {:severity=>"medium"}, {:severity=>"none"}, {:severity=>"high"}, {:severity=>"none"}, {:severity=>"none"}, {:severity=>"none"}, {:severity=>"high"}]
What's maybe not convenient, is that data.results holds an array with a single hash. Maybe a hash is enough for that.
I've written a JQ search that outputs the following, but I cannot work out how to get into the detail and extract specific information from this.
{
"https://www.example.org/rest/relation/node/recording/revision_uid": [
{
"_links": {
"self": {
"href": "https://www.example.org/user/37?_format=hal_json"
},
"type": {
"href": "https://www.example.org/rest/type/user/user"
}
},
"uuid": [
{
"value": "d40684cf-2321-42d2-9194"
}
]
}
],
"https://www.example.org/rest/relation/node/recording/uid": [
{
"_links": {
"self": {
"href": "https://www.example.org/user/37?_format=hal_json"
},
"type": {
"href": "https://www.example.org/rest/type/user/user"
}
},
"uuid": [
{
"value": "d40684cf-2321-42d2-9194"
}
],
"lang": "en"
}
],
"https://www.example.org/rest/relation/node/recording/field_category": [
{
"_links": {
"self": {
"href": "https://www.example.org/simplecategory?_format=hal_json"
},
"type": {
"href": "https://www.example.org/rest/type/taxonomy_term/tags"
}
},
"uuid": [
{
"value": "3fef93d5-926a-41aa-95cb"
}
]
}
],
"https://www.example.org/rest/relation/node/recording/field_part1_speaker": [
{
"_links": {
"self": {
"href": "https://www.example.org/by/speakername?_format=hal_json"
},
"type": {
"href": "https://www.example.org/rest/type/taxonomy_term/author"
}
},
"uuid": [
{
"value": "fb6c806f-fa6a-4aa0-89ef"
}
]
}
]
}
How do I write a query that returns 'https://www.example.org/simplecategory?_format=hal_json'?
And I'd then want a similar query that returns 'https://www.example.org/by/speakername?_format=hal_json'
So jq '._embedded' gets me the data above.
I've then tried various combinations to get to the value of https://www.example.org/rest/relation/node/recording/field_category.
- jq '._embedded.https://www.example.org/rest/relation/node/recording/field_category - but of course the URL has special characters in it.
jq .["https://www.example.org/rest/relation/node/recording/field_category"]
jq ."https://www.example.org/rest/relation/node/recording/field_category$"
I've also messed around with some of JQs built in functions, like flatten and to_entries, from_entries. I've also tried regular expressions but my efforts return Cannot iterate over null (null).
How do I write a query that returns 'https://www.example.org/simplecategory?_format=hal_json'?
If you want to specify the top-level key explicitly, the follow-on query would be:
.["https://www.example.org/rest/relation/node/recording/revision_uid"][]
| ._links.self.href
That is, the entire query would be:
._embedded
| .["https://www.example.org/rest/relation/node/recording/revision_uid"][]
| ._links.self.href
And I'd then want a similar query
An alternative to specifying the top-level key explicitly might be to select the href of interest from the array of all of them:
._embedded
| [.[][]._links.self.href]
This would yield:
[
"https://www.example.org/user/37?_format=hal_json",
"https://www.example.org/user/37?_format=hal_json",
"https://www.example.org/simplecategory?_format=hal_json",
"https://www.example.org/by/speakername?_format=hal_json"
]