In Elm, how to decode a JSON object inside nested JSON - json

I use Elm 0.19.1 with NoRedInk/elm-json-decode-pipeline/1.0.0
I have an Aircraft type which is
type alias Aircraft = {name:String}
For this, I have the following decoder:
aircraftDecoder : Json.Decode.Decoder Aircraft
aircraftDecoder =
Json.Decode.succeed Aircraft
|> Json.Decode.Pipeline.required "name" Json.Decode.string
Unfortunately, the decoder is complaining me telling: "BadBody "Problem with the given value: (...)"
This is because actually my area of interest if full of noise around it (from an HATEOAS api), like this:
{
"_embedded" : {
"aircrafts" : [ {
"name" : "AC01",
"_links" : {
"self" : {
"href" : "http://localhost:8080/aircrafts/1"
},
"aircraft" : {
"href" : "http://localhost:8080/aircrafts/1"
}
}
}, {
"name" : "AC01",
"_links" : {
"self" : {
"href" : "http://localhost:8080/aircrafts/2"
},
"aircraft" : {
"href" : "http://localhost:8080/aircrafts/2"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "http://localhost:8080/aircrafts{?page,size,sort}",
"templated" : true
},
"profile" : {
"href" : "http://localhost:8080/profile/aircrafts"
}
},
"page" : {
"size" : 20,
"totalElements" : 4,
"totalPages" : 1,
"number" : 0
}
}
How can I change the code, and keep using the Pipeline, so that the decoder doesn't get lost by all this noise?
I heard something about using Json.Decode.at, but the documentation is not good enough to let me get the correct code.

As far as I can tell, the problem is that:
at this moment, your decoder might be well fit to decode a single Aircraft
however, your JSON has a list of Aircrafts
also, you decoder does not know where, in the JSON, it might find Aircrafts
And you know, for Elm Aircraft and List Aircraft are completely different beats (as they should be). Anyway, the solution would be a two-step thing:
Telling the decoder where, within the JSON structure, are the Aircrafts
Parse a List Aircraft instead of a single Aircraft
Following your code and importing at and list from Json.Decode, the code might look like that:
listAircraftDecoder : Json.Decode.Decoder List Aircraft
listAircraftDecoder =
at ["_embedded", "aircrafts"] (list aircraftDecoder)
This means we're now aiming at a list of Aircrafts and that this list is an array within the JSON. Starting at the JSON root, take the property "_embedded" and, inside it, the property "aircrafts". This is a array, and Elm's list knows how to deal with it.
Finally, we just need to tell Elm's list to pass each element of the JSON array to a specific decoder – your aircraftDecoder in our case.
Does that make sense?

The following should work:
aircraftNameDecoder : Json.Decode.Decoder String
aircraftNameDecoder =
Json.Decode.map (Maybe.withDefault "" << List.head) <|
Json.Decode.at [ "_embedded", "aircrafts" ] <|
Json.Decode.list (Json.Decode.field "name" Json.Decode.string)
aircraftDecoder : Json.Decode.Decoder Aircraft
aircraftDecoder =
Json.Decode.succeed Aircraft
|> Json.Decode.Pipeline.custom aircraftNameDecoder
And please see elm/json for more documentation on Json.Decode.at.

Related

Elasticsearch mapping nested vs flat.What is the better type for searching?

I'm going to store some data to Elasticsearch so what is the best type for mapping.What type is most effect for searching speed?I mean nested type and flat type mapping.I'll put the example data here.I can't find any document for this question.If you guys have something please comment here.
Nested type
{
"User_name" : "John",
"Contact" : {
"contact_01" : "+777000...",
"contact_02" : "+888000..."
}
}
Flat type
{
"User_name" : "John",
"contact_01" : "+777000...",
"contact_02" : "+888000..."
}

EvaluateJsonPath unable to return a scalar

I'm trying to extract a value from JSON to a flowfile-attribute. When I run the EvaluateJsonPath processor I get an error stating
"Unable to get a scalar value for expression $..fields.storyBoard.stringValue.
Input JSON looks like this:
{
"name" : "projects/fakedims-0000/databases/(default)/documents/device/0000",
"fields" : {
"reportKey" : {
"stringValue" : "abc123"
},
"dateOccured" : {
"timestampValue" : "2018-10-14T04:00:00Z"
},
"storyBoard" : {
"stringValue" : "https://path/to/media"
},
"new" : {
"integerValue" : "25"
},
"name" : {
"stringValue" : "device one"
},
"location" : {
"geoPointValue" : {
"latitude" : -78.413751,
"longitude" : 38.156487
}
}
},
"createTime" : "2018-10-19T00:02:26.209335Z",
"updateTime" : "2018-10-19T22:22:24.382136Z"
}
The JSONPath expression is $..fields.storyBoard.stringValue
What I think is happening is that the processor is returning ["https://path/to/media"] rather than just the string.
This is what I get if a evaluate to flowfile-content rather than an attribute. Why? What can I do to fix it?
Change the Return Type property value to json in EvaluateJsonPath processor, if you are extracting as flowfile-attribute
Return Type property description:
Indicates the desired return type of the JSON Path expressions.
Selecting 'auto-detect' will set the return type to 'json' for a
Destination of 'flowfile-content', and 'scalar' for a Destination of
'flowfile-attribute'.
As you are trying to extract nested key not the key on the root level(ex:name,createTime..), that's the reason why we need to configure the Return Type as Json not as scalar.
In Addition you can use FlattenJson processor(seperator '_') to flatten-out all nested json then use Return Type as auto detect in EvaluateJsonPath processor.
EvaluateJsonConfigs:
Output:
We are going to have attribute value without enclosing in an array

JSON.stringify - accessing data

So I have Json obj which I convert via JSON.stringfy and the result is:
{
"data" : [ {
"id" : 417206355511802,
"name" : "test01"
}, {
"id" : 421211003974634,
"name" : "test02"
}, {
"id" : 403713622404901,
"name" : "test03"
}]
}
How can I access each name value? I was trying:
var test = result[0].name;
alert(test);
You can't access anything from the result of stringify() - it produces a string, hence its name. Rather, it sounds like you started with a string and converted it to an object via JSON.parse().
If that's not the case, and you already have the object, you don't need to stringify if you want to access properties.
That out of the way, you're missing the data step.
myobj.data[0].name; //<-- where myobj is the variable holding the object
JSON.stringify will not help you.becouse it's give a string as output.you can directly access the object elements by
var arr = myObj.data;
arr.forEach(function(elem) {
console.log(elem.id);
console.log(elem.name);
});

Flat csv data to Json

I am trying to process a Json data in Java. I have the data in below format (it is nested data structure with arrays etc.)
person.name,person.friend[0],person.friend[1],person.address.city,person.address.country
1,x,y,kolkata,india
2,a,b,london,uk
The first line is header denoting the nested object hierarchy. I want a json in below format,
{
"data" : [
{
"name" : "1",
"friend" : ["x","y"],
"address" : { "city" : "kolkata", "country" : "india" }
},
{
"name" : "2",
"friend" : ["a","b"],
"address" : { "city" : "london", "country" : "uk" }
} ]
}
The object structure is dynamic and I dont know the columns or header in advance, i.e. I can not use any predefined POJO to get populated with the data. In this example, it "Person" object but it may be any object structure.
I have gone through Jackson or Gson API, but none seems to fulfill this requirement. Is there any API that can help? or any other wayout?
Thanks
You need to do it in 2 steps.
First, you have to parse your CSV. I recommend superCSV. Parsing CSV may be fancy sometimes, so I really recommend you to use a library for that.
Second, you can serialize into JSON. Then you can use GSON, jackson, flexjson, whatever.
After a long Google...I found that the only option is to represent a collection based object structure in flat file is repeated rows,
person.name,person.friends,person.address.city,person.address.country
1,x,kolkata,india
1,y,kolkata,india
2,a,london,uk
2,b,london,uk
where the non-array elements repeats. We need to form a json from this, then need to filter or club the same object by its ID (here person.name)

Transform a List[(T, Double)] on a Json value with the play framework

I'm writing a Play 2.3.2 application, using Scala.
I use ReactiveMongo to access a my mongoDB database.
I've a collection named recommendation.advices that store all the suggestion to the user.
A document has the following form:
{
"_id" : ObjectId("54475f434be669af141677b7"),
"id" : "b8e84cb9-4b3a-4c6c-b01d-af276747e4ad",
"user" : {
"id" : "",
"email" : "luigi#gmail.com"
},
"output" : [
{
"tag" : "Vegetable:Carrots - alberto#gmail.com",
"score" : 0
},
{
"tag" : "Paper Goods:Liners - Baking Cups",
"score" : 0
},
{
"tag" : "Vegetable:Carrots - Jumbo",
"score" : 0
},
{
"tag" : "Paper Goods:Lialberto- Baking Cups",
"score" : 0
}
],
"date" : 1413963587328,
"type" : "system",
"clicked" : true
}
Now I'm writing a method in my controller that returns all the data where "clicked": true, and I want to return a Json in the following form:
{"idR": "b8e84cb9-4b3a-4c6c-b01d-af276747e4ad",
"user" : {
"id" : "",
"email" : "luigi#gmail.com"
},
"tags" : [
{
"tag" : "Vegetable:Carrots - alberto#gmail.com",
"score" : 0
},
{
"tag" : "Paper Goods:Liners - Baking Cups",
"score" : 0
},
{
"tag" : "Vegetable:Carrots - Jumbo",
"score" : 0
},
{
"tag" : "Paper Goods:Lialberto- Baking Cups",
"score" : 0
}
]
}
How can I make that in play??
I try to implement the method as the following:
def clickedAdvises = Action.async {
val query = Json.obj("clicked" -> true)
Advices.find(query).toList flatMap { advices =>
val results = for(el <- advices) yield Json.obj("idR" -> el.id, "user" -> el.user, "tags" -> el.output)
Future{Ok(Json.obj("results" -> results))}
}
}
The el.output is a List[(Tag, Double)], and for the Tag I've defined a formatter.
But the compiler gives me the following errors:
[error] /Users/alberto/git/bdrim/modules/recommendation-system/app/recommendationsystem/controllers/StatisticsController.scala:111: type mismatch;
[error] found : List[(recommendationsystem.models.Tag, Double)]
[error] required: play.api.libs.json.Json.JsValueWrapper
[error] val results = for(el <- advices) yield Json.obj("idR" -> el.id, "user" -> el.user, "tags" -> el.output)
[error] ^
[error] /Users/alberto/git/bdrim/modules/recommendation-system/app/recommendationsystem/controllers/StatisticsController.scala:111: type mismatch;
[error] found : List[(recommendationsystem.models.Tag, Double)]
[error] required: play.api.libs.json.Json.JsValueWrapper
[error] val results = for(el <- advices) yield Json.obj("idR" -> el.id, "user" -> el.user, "tags" -> el.output)
[error] ^
[error] one error found
How can I fixed that??
The error is self explanatory:
You need JsValue, instead you are giving a List[(recommendationsystem.models.Tag, Double)] which apparently is the type of output field in your Advice class.
The fact that you have a formatter for your Tag class does not change the situation, first because what you have is a List of tuples, not a single tag, second because formatter only works when you use toJson and fromJson macros.
So what you need is somehow convert this List of tuples to a JsValue.
JsArray is corresponding to JSON list. If you look at the code of it
case class JsArray(value: Seq[JsValue] = List()) extends JsValue
you see that you can pass a Seq of JsValues to it. So you need to somehow convert a tuple of (recommendationsystem.models.Tag, Double) to a JsObject (which is corresponding to a JSON object). You can do this simply with the following code:
JsObject(Seq(
("tag", tag),
("score", score)
))
or by using the Json.obj which is a helper factory mehtod for constructing a JsObject.
So the code will be:
val results = for {
el <- advices
} yield
Json.obj(
"idR" -> el.id,
"user" -> el.user,
"tags" -> Json.arr(el.output.map(it => Json.obj((it._1, it._2)).asInstanceOf[JsValueWrapper]): _*)
)
Easier way!
If you refer to Play Doc about JSON you will see that you can convert objects to/from json by using play.api.libs.json.Json.toJson method.
In order to do that you need an implicit writer for your classes which is pretty straightforward.
There are some macro helpers, for example Json.writes macro helps you create a writer for your class. This machinery is recursive so if you have implicit writers for all classes needed, then you can simply call Json.toJson(advice) and get the result.
So I would suggest to start writing an implicit writer for your advice class:
import play.api.libs.json._
implicit def adviceWriter = Json.writes[Advice]
then try
Json.toJson(advice)
Then compile the code, you will get compile errors because you need writer for all needed classes. Continue by providing a writer for them till you get no compile error!
This makes the code cleaner and more concise and focused, as the actual conversion to Json will not pollute the actual code. In addition, you can reuse the code without explicitly doing anything. You need just to bring implicit writers and readers to your context by importing them and using Json.toJson and Json.fromJson methods.
I think you should transform el.output to JsObject before insert it in you jsObject
Are you sure that el object can be called by something like el.id? It's not JavaScript, even it's JsValue, it should called by (el \ "id").
I'm no pretty sure, but try using el(id) to call it and insert as a value of your JsObject like
Json.obj("idR" -> el(id))