Using apply on an array of objects in a json - json

I have a json which I am reading in R and converting to a list object. For a key "metrics", there is an array of multiple objects of the same type.
Json structure:
{"metrics":[{"metricName":"abc",
"metricType":"def"
},
{"metricName":"ghi",
"metricType":"jkl"
}]
}
This is how my list object looks like:
$metrics
$metrics[[1]]
$metrics[[1]]$metricName
[1] "abc"
$metrics[[1]]$metricType
[1] "def"
$metrics[[2]]
$metrics[[2]]$metricName
[1] "ghi"
$metrics[[2]]$metricType
[1] "jkl"
I want to apply a function (someFunc) to each object of the array. $metrics[[1]],$metrics[[2]]. How can this be done using apply family of functions?
somefunc(x){return(list(x$metricName,x$metricType)}
I tried concatenating like this:
lapply(lapply(metrics,"["),someFunc)
This does not throw an error but gives empty lists as output. The someFunc expects x$metricName,x$metricType objects to process. But using "[" does not render that kind of object I guess. Can this be handled using the apply functions?

Can you explici what your function someFunc is doing (and its parameters)?
This is working for me
someFunc=function(metricName,metricType){
return(paste(metricName,metricType))
}
metrics=list(list(metricName="abc",metricType="m"),
list(metricName="gg",metricType="L"))
lapply(metrics,FUN=function(el){someFunc(el$metricName,el$metricType)})
(use sapply eventually if you want to have a vector.

Related

How to parse nested json-array in streaming fashion with zio-json

For a json array like this:
[
my-json-obj1,
my-json-obj2,
my-json-obj3,
....
my-json-objN
]
And MyJsonObj class that represents a mapping of single object in array I can say:
val myJson = '''[...]'''
ZStream
.fromIterable(myJson.toSeq)
.via(JsonDecoder[MyJsonObj].decodeJsonPipeline(JsonStreamDelimiter.Array))
to parse that array in a "streaming" way, i.e. emit mapped objects as they are parsed form the input as opposed to reading all the input first and then extracting the objects.
How can I do the same if the array is nested inside a json object say like this?:
{
"hugeArray":
[
my-json-obj1,
my-json-obj2,
my-json-obj3,
....
my-json-objN
]
}
I trawled through zio-json source code, but I can't find any foothole there for this use case. I guess I could carve out that array from the json document and feed that to decodeJsonPipeline. Is there any better, json-syntax aware way of doing this? If not directly in zio-json perhaps with help of some other open source json libraries?

Selecting property from nested JSON array stored within Logic App variable

I am struggling to select a property from a JSON array that I've stored in a variable (as an array). I receive the following error:
The execution of template action 'Select' failed: The evaluation of 'query' action >'where' expression '{
"Response": "#variables('UserEvents').responseStatus.response",
"trest": ""
}' failed: 'The template language expression >'variables('UserEvents').responseStatus.response' cannot be evaluated because property >'responseStatus' cannot be selected. Array elements can only be selected using an > integer index. Please see https://aka.ms/logicexpressions for usage details.'.
The JSON is below:
[
{
"value": [
{
"responseStatus": {
"response": "notResponded",
"time": "0001-01-01T00:00:00Z"
}
}
]
}
]
I'm trying to select the 'response' property only by using the following expression:
#variables('UserEvents').responseStatus.response
I have tried adding [0] to various parts of the above expression but it doesn't seem to make any difference. I am sure it's pretty simple and I'm just getting the syntax wrong, but am completely stuck!
Any help appreciated - thanks.
Adrian
#{
variables('UserEvents') ? ['value'] ? [0] ? ['response']
}
Try that. The integer index mentioned in the error is the way around this. May need to play around with the structure to make it match your data but [0] will make it select the first object within your value array.
There are 2 ways to get the properties inside a nested JSON
WAY - 1 (Try to get properties using For-each loop)
For this we need to use 2 for-each loops to retrieve the property inside the nested JSON After Parsing the JSON. Below is the screenshot of my logic app for your reference.
WAY - 2 (Using Dynamic Expression)
Here is the Expression that worked for me.
body('Parse_JSON')?[0]?['value']?[0]?['responseStatus']?['response']
Since it is in Array of Array format we need to mention the index of the array. Below is the screenshot of my logic app for your reference :
output :

Azure Data Factory - forEach - JSON to array errors

My aim is to use Azure Data Factory to copy data from one place to another using REST API.
The first part of the copying is using the ForEach activity to select parameters from a nested JSON/array. I have had problems with this nested JSON because of various error messages in the ForEach activity.
My JSON is of the following form:
(
{
"key_1": "value_1",
"key_2": [
"value_2_1",
"value_2_2"
]
}
)
and first I'm setting the #json conversion function in front of it:
#json(
'{
"key_1": "value_1",
"key_2": [
"value_2_1",
"value_2_2"
]
}'
)
Here you can see the Execute Pipeline object and its parameters:
I am setting this JSON as a parameter in the Execute Pipeline object. I am setting its type as an "Array". (For the record, I have still had similar error messages even though I have tried to change the parameter type to "String" or "Object".)
The ForEach activity is used to select an item from the nested JSON, which is written into a parameter.
This immediately produces the following error in the Execute Pipeline activity:
Operation on target... ...failed: The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'.
So, even though I set the JSON parameter type to "Array", it is changed into "Object" when I debug the pipeline activity. See the error below:
Next, I tried to use the ADF #createArray function before my JSON text.
#createArray(
'{
"key_1": "value_1",
"key_2": [
"value_2_1",
"value_2_2"
]
}'
)
When debugging, the forEach activity throws me an error in the first Copy data activity:
The expression 'concat(item().SELECTING_key_1_FROM_MY_JSON))' cannot be evaluated because property 'key_1' cannot be selected. Property selection is not supported on values of type 'String'.
Please help me, what I am doing wrong in my attempts of converting the JSON to an array? What should I change in my code?
So far I have tried changing the parameter type and using various functions in the JSON dynamic content but with no luck.
update:
If you want change the filename in Copy Data --> Sink.
You can key in the dynamic content #concat(pipeline().parameters.Pip_Object.key_1,'.json') to rename the file.
Please correct me if I understand you wrong.
First, we should use Parameters to store the Json array as follows. Because Variables are not support to store the Json array. Variables are only support to store simple data type such as ["1","2","3"...].
Json array format should as follows:
[{"key_1": "value_1"},{"key_2": ["value_2_1","value_2_2"]}]
I created a simple test here. Then ForEach the Json array:
Inside ForEach1 activity, pass #item() to the object type parameter Pip_Object.
The Input is as follows:
Add dynamic content above using any combination of expressions, functions and system variables at for each settings
#activity('Get Metadata1').output.childItems
We ended up using two pipelines: a generic one and a specific one.
The generic one has the JSON as a parameter (JSON_PARAMETER), and we set it a default value in the following form
[{"key_1":"value_1","key_2":["value_2"]},{"key_1":"value_3","key_2":["value_2"]}, ...etc. ...}]
The generic pipeline has a forEach loop, in which that mentioned JSON parameter is called in Settings -> Items:
#JSON(pipeline().parameters.JSON_PARAMETER)
In the specific pipeline, there is an Execute Pipeline activity, where the JSON_PARAMETER is found in Settings -> Parameters. If the default value of JSON_PARAMETER is used, the field is left blank. If we want to change the parameter, before Execute Pipeline, we put a Set Variable activity where we change the Variables -> Value to:
#concat('
[
{"key_1":"value_1",
"key_2":
["value_2",
"value_3"
]
},
{"key_1":"value_3",
...and so on...
}
]
')

How can I serialize a nested object with singleton values to JSON in R?

How I can use jsonlite's toJSON() function to produce the following JSON?
{"data":{"foo":"bar"}}
I'm new to R. I was able to produce the following, but "bar" is inside an array which I don't want:
> library(jsonlite)
> toJSON(list(data=list(foo = "bar")))
{"data":{"foo":["bar"]}}
I can't figure out how to get "bar" out of the array, and I can't find an obvious analog to a hashmap/dictionary in the examples of toJSON I've found.
you can acheive that with the auto_unbox argument. See ?toJSON for more info.
> toJSON(list(data=list(foo = "bar")),auto_unbox = TRUE)
{"data":{"foo":"bar"}}

R: Un-escape JSON string and build JSON object

I am just learning to use the jsonlite library to fetch json data from a server. However in the received json response (whose structure I have no control over), there seems to be a node that I can only describe as a chunk of 'escaped' JSON data, right in the middle of the json object. How do I build a JSON object out of it? I am able to extract each such value OK but then I can't use it as is without turning it into a true JSON object.
example:
library(jsonlite)
myFakeJSON <- '"{"country": "UK","ranking": "45"}"'
json <- toJSON(myFakeJSON)
but:
> json
[1] "\"{\"country\": \"UK\",\"ranking\": \"45\"}\""
The result is not a json object.. What am I doing wrong? How do I escape (or un-escape??) the received data? Seems like something obvious, but not to me :(
I think you are making two errors. First: too many quotes, second: wrong test for JSON-hood. If you wanted to use toJSON, then you would give it an R object for conversion rather than a effort at as JSON string.
> myFakeJSON <- '{"country": "UK", "ranking": "45"}'
> fromJSON(myFakeJSON)
$country
[1] "UK"
$ranking
[1] "45"
If you just need to remove the extra double-quotes on the "outside" of the curly braces, then this regex replacement succeeds on this small example:
> json <- fromJSON(gsub("\\}\\\"", "}", gsub("\\\"\\{","{", myFakeJSON))); json
$country
[1] "UK"
$ranking
[1] "45"