Deserialize multidimensional JSON API Response with JSMSerializerBundle - json

I work with Symfony2/JSMSerializerBundle.
Serializing flat json objects to PHP objects works great. But the API I use, gives a multidimensional Json response:
{
"Webmessage": {
"#version": "1.0",
"Header": {
"Country": "NL",
"Language": "NL"
},
"Content": {
"Filters": {
"Sizes": {
"Size": [
{
"#id": "241",
"#text": "3,5"
},
{
"#id": "55",
"#text": "36"
}
]
},
"Colours": {
"Colour": [
{
"#id": "159",
"#text": "wit"
},
{
"#id": "54",
"#text": "zwart"
}
]
}
}
}
}
}
As deserialized PHP I want something like this:
Array
(
[sizes] => Array
(
[0] => AppBundle\Entity\Filter Object
(
[id:AppBundle\Entity\Filter:private] => 1
[text:AppBundle\Entity\Filter:private] => Heren
)
[1] => AppBundle\Entity\Filter Object
(
[id:AppBundle\Entity\Filter:private] => 2
[text:AppBundle\Entity\Filter:private] => Dames
)
)
[colour] => Array
(
[0] => AppBundle\Entity\Filter Object
(
[id:AppBundle\Entity\Filter:private] =>56
[text:AppBundle\Entity\Filter:private] => Black
)
[1] => AppBundle\Entity\Filter Object
(
[id:AppBundle\Entity\Filter:private] => 212
[text:AppBundle\Entity\Filter:private] => Yellow
)
)
)
Who has tips how i can do this?
Thanks!

Maybe you can decode it first and then use the Normalizer to create the entities. Something like this:
$array= json_decode($json, true);
$valueToDenormalize = $array['value'];
$normalizer = new GetSetMethodNormalizer();
$entity = $normalizer->denormalize($valueToDenormalize, 'Your\Class');
Be aware, I have not tried this. I don't know if the normalizer will work this way, but I know it's used to normalize and denormalize between arrays and Symfony's entities.
For further investigation you could take a look at the Serializer docs:
http://symfony.com/doc/current/components/serializer.html
Or the Normalizer:
http://api.symfony.com/2.3/Symfony/Component/Serializer/Normalizer/GetSetMethodNormalizer.html
Here's something about json_encode:
http://php.net/manual/en/function.json-decode.php

Related

Parsing JSON from CouchDB to ElasticSearch via Logstash

I am not able to parse JSON from CouchDB to Elasticsearch index in the desired way.
My CouchDB data looks like this:
{
"_id": "56161609157031561692637",
"_rev": "4-4119e8df293a6354be4c9fd7e8b12e68",
"deleteFlag": "N",
"entryUser": "John",
"parameter": "{\"id\":\"14188\",\"rcs_p\":null,\"rcs_e\":null,\"dep_p\":null,\"dep_e\":null,\"dep_place\":null,\"rcf_p\":null,\"rcf_e\":null,\"rcf_place\":null,\"dlv_p\":\"3810\",\"dlv_e\":\"1569\",\"seg_no\":null,\"trans_type\":\"incoming\",\"trans_service\":\"delivery\"}",
"physicalId": "0",
"recordDate": "2020-12-28T17:50:16+05:45",
"tag": "CARGO",
"uId": "56161609157031561692637",
"~version": "CgMBKgA="
}
What I am trying to do is be able to search using the nested field of the parameter of the above JSON.
When I put the data in ES index it is stored like this:
{
"_index": "del3",
"_type": "_doc",
"_id": "XRCV9XYBx5PRwauO--qO",
"_version": 1,
"_score": 0,
"_source": {
"#version": "1",
"doc_as_upsert": true,
"doc": {
"physicalId": "0",
"recordDate": "2020-12-27T12:56:45+05:45",
"tag": "CARGO",
"~version": "CgMBGgA=",
"uId": "48541609052212485430933",
"_rev": "3-937bf92e6010afec13664b1d9d06844b",
"deleteFlag": "N",
"entryUser": "John",
"parameter": "{\"id\":\"4038\",\"rcs_p\":null,\"rcs_e\":null,\"dep_p\":null,\"dep_e\":null,\"dep_place\":null,\"rcf_p\":null,\"rcf_e\":null,\"rcf_place\":null,\"dlv_p\":\"5070\",\"dlv_e\":\"2015\",\"seg_no\":null,\"trans_type\":\"incoming\",\"trans_service\":\"delivery\"}"
},
"#timestamp": "2021-01-12T07:53:33.978Z"
},
"fields": {
"#timestamp": [
"2021-01-12T07:53:33.978Z"
],
"doc.recordDate": [
"2020-12-27T07:11:45.000Z"
]
}
}
I want to be able to access the fields inside the parameter (id, rcs_p, rcs_e, ..) in Elasticsearch.
Here is my logstash.conf file:
input {
couchdb_changes {
host => "<host_name>"
port => 5984
db => "mychannel_asset$management"
keep_id => false
keep_revision => true
#initial_sequence => 0
always_reconnect => true
sequence_path => "/usr/share/logstash/config/seqfile"
}
}
filter {
json {
source => "[parameter]"
remove_field => ["[parameter]"]
}
}
output {
if([doc][tag] == "CARGO") {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "del3"
user => elastic
password => changeme
}
}
}
How do I achieve my desired result? I also tried to do by creating a custom template by defining a nested type for parameter but no luck yet. Any help would be appreciated.
I think you did almost everything right. I'm not too sure about the actual structure, but one of these might work:
filter {
json {
source => "parameter"
target => "parameter"
}
}
filter {
json {
source => "[doc][parameter]"
target => "[doc][parameter]"
}
}
I don't know how CouchDB source input plugins works but it seems to be putting everything under doc object.

API POST request in Julia

I am trying to convert some Python code to Julia. Here is the Python code:
url = "http://api.scb.se/OV0104/v1/doris/sv/ssd/START/BE/BE0101/BE0101G/BefUtvKon1749"
json = {
"query": [
{
"code": "Kon",
"selection": {
"filter": "item",
"values": [
"1",
"2"
]
}
},
{
"code": "ContentsCode",
"selection": {
"filter": "item",
"values": [
"000000LV"
]
}
}
],
"response": {
"format": "px"
}
}
r = requests.post(url=url, json=json)
Below is the Julia code, that is not working, with this error message:
syntax: { } vector syntax is discontinued around path:8
top-level scope at population_data.jl:8
using DataFrames, DataFramesMeta, HTTP, JSON3
url = "http://api.scb.se/OV0104/v1/doris/sv/ssd/START/BE/BE0101/BE0101G/BefUtvKon1749"
json = {
"query": [
{
"code": "Kon",
"selection": {
"filter": "item",
"values": [
"1",
"2",
"1+2"
]
}
},
{
"code": "ContentsCode",
"selection": {
"filter": "item",
"values": [
"000000LV"
]
}
}
],
"response": {
"format": "px"
}
}
r = HTTP.post(url, json)
My attempts to solve this are the following:
Convert the json variable to a string using """ around it.
Converting the JSON string to Julia data types, using JSON3.read()
Passing the converted JSON string to the POST request. This gives the following error:
IOError(Base.IOError("read: connection reset by peer (ECONNRESET)", -54) during request(http://api.scb.se/OV0104/v1/doris/sv/ssd/START/BE/BE0101/BE0101G/BefUtvKon1749)
None of it works, and I am not even sure that it is about the JSON format. It could be that I am passing the wrong parameters to the POST request. What should I do?
One way of solving this consists in building the parameters as native julia data structures, and use JSON to convert and use them as the body of your PUT request:
Dictionaries in julia are built using a syntax like Dict(key => value). Arrays are built using a standard syntax: [a, b, c]. The julia native data structure equivalent to your parameters would look like this:
params = Dict(
"query" => [
Dict("code" => "Kon",
"selection" => Dict(
"filter" => "item",
"values" => [
"1",
"2",
"1+2"
]),
),
Dict("code"=> "ContentsCode",
"selection" => Dict(
"filter" => "item",
"values" => [
"000000LV"
]),
),
],
"response" => Dict(
"format" => "px"
))
Then, you can use JSON.json() to build the JSON representation of it as a string and pass it to the HTTP request:
using HTTP
using JSON
url = "http://api.scb.se/OV0104/v1/doris/sv/ssd/START/BE/BE0101/BE0101G/BefUtvKon1749"
# send the request
r = HTTP.request("POST", url,
["Content-Type" => "application/json"],
JSON.json(params))
# retrieve the response body as a string
b = String(r.body)

JSON/Kotlin - handling JSON objects and arrays

First of all, I'm very new to JSON concept so sorry if my question is silly or very simple to answer.
I want to use Oxford Dictionary API for simple app which I'm writing in Kotlin, here is the response from API as JSON:
{
"metadata": {
"provider": "Oxford University Press"
},
"results": [
{
"id": "hello",
"language": "en",
"lexicalEntries": [
{
"entries": [
{
"etymologies": [
"early 19th century: variant of earlier hollo; related to holla"
],
"homographNumber": "000",
"senses": [
{
"definitions": [
"used as a greeting or to begin a telephone conversation"
],
"examples": [
{
"text": "hello there, Katie!"
}
],
"id": "m_en_gbus0460730.012",
"short_definitions": [
"used as greeting"
],
"subsenses": [
{
"definitions": [
"used to express surprise"
],
"examples": [
{
"text": "hello, what's all this then?"
}
],
"id": "m_en_gbus0460730.017",
"regions": [
"British"
],
"short_definitions": [
"used to express surprise"
]
},
{
"definitions": [
"used as a cry to attract someone's attention"
],
"examples": [
{
"text": "‘Hello below!’ he cried"
}
],
"id": "m_en_gbus0460730.018",
"short_definitions": [
"used attract attention"
]
},
{
"definitions": [
"used informally to express sarcasm or anger"
],
"examples": [
{
"text": "Hello! Did you even get what the play was about?"
}
],
"id": "m_en_gbus0460730.019",
"short_definitions": [
"used informally to express sarcasm or anger"
]
}
]
And now, I would like to extract only "definitions" from this JSON object but as you can see it is nested within other JSON arrays, my code so far looks like this:
var resultJSON = JSONObject(result)
var JSON_results = resultJSON.getJSONArray("results")
var JSON_lexical = JSON_results.getJSONObject(0).getJSONArray("lexicalEntries")
var JSON_entries = JSON_lexical.getJSONObject(0).getJSONArray("entries")
var JSON_senses = JSON_entries.getJSONObject(0).getJSONArray("senses")
var JSON_definitions = JSON_senses.getJSONObject(0).getJSONArray("definitions")
Log.i("JSON", JSON_definitions.getString(0))
I know that there needs to be a better way of doing this but I can't find how.
Kotlin actually makes it easier to map such responses with something called "data classes". So you can simply paste the JSON response in an online JSON to Kotlin Data Class Generator e.g. https://json2kotlin.com
It churns out .kt files like this:
data class Json4Kotlin_Base (
val metadata : Metadata,
val results : List<Results>
)
and thn you can simply pass on the response JSON to the Data class mapping like this:
val json = getJson() // your json value here
val topic = Gson().fromJson(json, Json4Kotlin_Base::class.java)
In case you're looking for GSON annotations in the generated models, chose the option when you generate those.
Here's a video tutorial for step by step process about it.
https://www.youtube.com/watch?v=n46WbgNoEnE
Try as follow
val justDefinitions = mutableListOf<String>()
JSON_senses.forEach {
val definitions = it.getJSONArray("definitions")
for (i in 0 until definitions.length()) { {
justDefinitions.add(it.getString(i))
}
}

JsonPath with Newtonsoft.JSON

I tried for nearly an hour different approaches, but I don't get it ;(
my JSON object is this:
"typeOfHair": {
"value": [
{
"code": "Dry Hair",
"values": [
{
"value": "DryHair",
"language": "en"
},
{
"value": "TrockenesHaar",
"language": "de"
}
]
},
{
"code": "Any Type of Hair",
"values": [
{
"value": "AnyTypeOfHair",
"language": "en"
},
{
"value": "JedenHaartyp",
"language": "de"
}
]
}
]
}
And my task is to get with Newtonsoft.JSON all values where the language is "de".
My current approach is:
JsonObject.SelectTokens("typeOfHair.value.values[?(#.language == 'de')].value").ToList()
Can someone help me with this?
Kind regards
You're very close. You need to account for the outer value array typeOfHair.value[] by using the JsonPATH wildcard operator [*]:
var values = JsonObject.SelectTokens("typeOfHair.value[*].values[?(#.language == 'de')].value")
// Convert from JValue to string
.Select(v => (string)v)
// Save in a list
.ToList();
And, the result is:
["TrockenesHaar","JedenHaartyp"]
Sample fiddle.
I know the OP specified JSONPath explicitly but for the sake of completeness below is how to achieve the same with LINQ to JSON:
var values = jObject["typeOfHair"]["value"]
.SelectMany(v => v["values"])
.Where(v => (string)v["language"] == "de")
.Select(v => (string)v["value"])
.ToList();
Demo: https://dotnetfiddle.net/1S4sT4

Should JSON Api attributes element contain nested objects?

This is the first time we are using JSON API in our projects and according to specification on their web, this is what a regular JSON API response should look like
HTTP/1.1 200 OK
Content-Type: application/vnd.api+json
{
"data": [{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!",
"body": "The shortest article. Ever.",
"created": "2015-05-22T14:56:29.000Z",
"updated": "2015-05-22T14:56:28.000Z"
},
"relationships": {
"author": {
"data": {"id": "42", "type": "people"}
}
}
}],
"included": [
{
"type": "people",
"id": "42",
"attributes": {
"name": "John",
"age": 80,
"gender": "male"
}
}
]
}
We are not sure, should attributes in data be always flat, or attributes could also contain nested objects such as location for example
"data": [{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!",
"body": "The shortest article. Ever.",
"created": "2015-05-22T14:56:29.000Z",
"updated": "2015-05-22T14:56:28.000Z",
"location":
{
"lat": "0.00",
"long": "0.00"}
},
Yes, take a look to : http://jsonapi.org/format/#document-resource-object-attributes
Complex data structures involving JSON objects and arrays are allowed
as attribute values. However, any object that constitutes or is
contained in an attribute MUST NOT contain a relationships or links
member, as those members are reserved by this specification for future
use.
After Decoding your JSON, Result is -
Array
(
[data] => Array
(
[0] => Array
(
[type] => articles
[id] => 1
[attributes] => Array
(
[title] => JSON API paints my bikeshed!
[body] => The shortest article. Ever.
[created] => 2015-05-22T14:56:29.000Z
[updated] => 2015-05-22T14:56:28.000Z
[location] => Array
(
[lat] => 0.00
[long] => 0.00
)
)
[relationships] => Array
(
[author] => Array
(
[data] => Array
(
[id] => 42
[type] => people
)
)
)
)
)
[included] => Array
(
[0] => Array
(
[type] => people
[id] => 42
[attributes] => Array
(
[name] => John
[age] => 80
[gender] => male
)
)
)
)
And here location contains an array, so this will be an nested object.