Validate JSON in Play framework without JSPath - json

I have pre-defined class MyClass the following JSON structure:
{
"key1": [
// Objects of MyClass
],
"key2": [
// Objects of MyClass
],
"key3": [
// Objects of MyClass
]
}
key1, key2, key3 are optional (at least one must present but I do not need to validate it)
According to the Play framework documentation I need to do the following:
implicit val myClassReads: Reads[] = (
(JsPath \ "key1").read[List[MyClass]] and
(JsPath \ "key2").read[List[MyClass]] and
(JsPath \ "key3").read[List[MyClass]]
)
But this approach has some disadvantages :
The data type of key1, key2, key3 is always MyClass). Repeating read[MyClass] seems redundant.
What if the JSON schema changes in away that there are key1, key2, ... key100 ? The code becomes very messy.
How can I write a custom validator that apply read[MyClass] to all fields?

If you are using Play Json 2.8.x, you can just do:
val result = json.validate[Map[String, List[MyClass]]]
Is it possible to validate whether the JSON keys are in ALLOWED_KEYS: List[String] with your approach?
You can also use another type for the key of the Map, as long as you define a KeyReads instance for that type. So you could use an Enum, with only a limited number of possible values. Or you can use refinement types (e.g. using the "refined" library) and so on.
The easiest way to restrict the possible values of the keys is to check the resulting Map after conversion:
val ALLOWED_KEYS = Set("key1", "key2", ...)
val result = json.validate(
Reads.verifying[Map[String, List[MyClass]]](_.keySet.subsetOf(ALLOWED_KEYS))
)
or you can also check the keys upfront if you want:
val result = json.validate(
Reads.verifying[JsObject](_.keys.subsetOf(ALLOWED_KEYS))
.andThen(Reads.of[Map[String, List[MyClass]]])
)

Something like this ?
def convertJsonToListModel[T](json: JsValue)(implicit reads: Reads[T]): List[T] = {
val conversionResult: JsResult[List[T]] = json.validate[List[T]]
conversionResult match {
case s: JsSuccess[List[T]] => s.get
case e: JsError => e.get
}
}

Play JSON has automatic format derivation for product type's (e.g. case classes).
More details you can find in the documentation: https://www.playframework.com/documentation/latest/ScalaJsonAutomated
I guess in your case this would like something like next:
import play.api.libs.json._
case class MyClass(value: String)
object MyClass {
implicit format = Json.format[Resident]
}
case class MyClassContainer(key1: List[MyClass], key2: List[MyClass], key3: List[MyClass])
object MyClassContainer {
implicit format = Json.format[MyClassContainer]
}
Hope this will be helpful.

Related

Play framework (Scala) - Getting subset of json that contains arrays

I have many very large json-objects that I return from Play Framework with Scala.
In most cases the user doesn't need all the data in the objects, only a few fields. So I want to pass in the paths I need (as query parameters), and return a subset of the json object.
I have looked at using JSON Transformers for this task.
Filter code
def filterByPaths(paths: List[JsPath], inputObject: JsObject) : JsObject = {
paths
.map(_.json.pick)
.map(inputObject.transform)
.filter(_.isSuccess)
.map { case JsSuccess(value, path) => (value, path) }
.foldLeft(Json.obj()) { (obj, jsValueAndPath) =>
val(jsValue, path) = jsValueAndPath
val transformer = __.json.update(path.json.put(jsValue))
obj.transform(transformer).get
}
}
Usage:
val input = Json.obj(
"field1" -> Json.obj(
"field2" -> "right result"
),
"field4" -> Json.obj(
"field5" -> "not included"
),
)
val result = filterByPaths(List(JsPath \ "field1" \ "field2"), input)
// {"field1":{"field2":"right result"}}
Problem
This code works fine for JsObjects. But I can't make it work if there are JsArrays in the strucure. I had hoped that my JsPath could contain an index to look up the field, but that's not the case. (Don't know why I assumed that, maybe my head was too far in the JavaScript-world)
So this would fail to return the first entry in the Array:
val input: JsObject = Json.parse("""
{
"arr1" : [{
"field1" : "value1"
}]
}
""").as[JsObject]
val result = filterByPaths(List(JsPath \ "arr1" \ "0"), input)
// {}
Question
My question is: How can I return a subset of a json structure that contains arrays?
Alternative solution
I have the data as a case class first, and I serialize it to Json, and then run filterByPaths on it. Having a Reader that only creates the json I need in the first place might be a better solution, but creating a Reader on the fly, with configuration from queryparams seamed a more difficult task, then just stripping down the json afterwards.
The example of the returning array element:
val input: JsValue = Json.parse("""
{
"arr1" : [{
"field1" : "value1"
}]
}
""")
val firstElement = (input \ "arr1" \ 0).get
val firstElementAnotherWay = input("arr1")(0)
More about this in the Play Framework documentation: https://www.playframework.com/documentation/2.6.x/ScalaJson
Update
It looks like you got the old issue RuntimeException: expected KeyPathNode. JsPath.json.put, JsPath.json.update can't past an object to a nesting array.
https://github.com/playframework/playframework/issues/943
https://github.com/playframework/play-json/issues/82
What you can do:
Use the JSZipper: https://github.com/mandubian/play-json-zipper
Create a script to update arrays "manually"
If you can afford it, strip array in a resulting object
Example of stripping array (point 3):
def filterByPaths(paths: List[JsPath], inputObject: JsObject) : JsObject = {
paths
.map(_.json.pick)
.map(inputObject.transform)
.filter(_.isSuccess)
.map { case JsSuccess(value, path) => (value, path)}
.foldLeft(Json.obj()) { (obj, jsValueAndPath) =>
val (jsValue, path) = jsValueAndPath
val arrayStrippedPath = JsPath(path.path.filter(n => !(n.toJsonString matches """\[\d+\]""")))
val transformer = __.json.update(arrayStrippedPath.json.put(jsValue))
obj.transform(transformer).get
}
}
val result = filterByPaths(List(JsPath \ "arr1" \ "0"), input)
// {"arr1":{"field1":"value1"}}
The example
The best to handle JSON objects is by using case classes and create implicit Reads and Writes, by that you can handle errors every fields directly. Don't make it complicated.
Don't use .get() much recommended to use .getOrElse() because scala is a type-safe programming language.
Don't just use any Libraries except you know the process behind it, much better to create your own parsing method with simplified solution to save memory.
I hope it will help you..

play framework json reads from empty string to empty list

Hi everyone recently I faced an issue in converting json into my own data model.
I have a json format message which may contain an empty string:
{
"name" : "John Doe",
"hobbies": ""
}
or a list of hobby types:
{
"name" : "John Doe",
"hobbies": [{"name":"basketball"}]
}
And the following is my case class data model in scala play framework:
case class Person(name: String, hobbies: List[Hobby])
case class Hobby(name: String)
Right now I'm using the default json formatter but of course it's not working well when we have empty string as value.
implicit val HobbyJson= Json.format[Hobby]
implicit val PersonJson = Json.format[Person]
it will throw exception if the hobbies has a empty string. I want to convert it into an empty list when it's the empty string. I search the document Play provides but couldn't find infomation. Can anyone give some suggestions?
Thanks in advance.
As you mentioned, the default Format macros won't work for you here because of the inconsistent treatment of hobbies. So you need to implement your own Reads[Person] - here's how I'd do it:
object PersonJson {
implicit val hobbyConverter = Json.format[Hobby]
val personReads = new Reads[Person] {
override def reads(json: JsValue): JsResult[Person] = {
for {
personName <- (json \ "name").validate[String]
hobbies <- (json \ "hobbies").validate[JsValue]
} yield {
val maybeHobbyList = hobbies.validate[List[Hobby]].asOpt
Person(personName, maybeHobbyList.getOrElse(Nil))
}
}
}
implicit val personConverter = Format(personReads, Json.writes[Person])
}
The key thing to note here is surrounding the whole thing in a JsResult courtesy of the for-comprehension and the yield. This gives us all the necessary checking (like the name field being there and being a String, and the hobbies field being there).
The code within the yield block only runs if we've got something that looks pretty close to a Person. Then we can safely try validating the hobbies as a List[Hobby], and convert the result to an Option[List[Hobby]]. It'll be a None if it didn't work (thus it must have been a string) and so we default it to the empty list as required.
Thanks #millhouse answer, it definitely works. Like he said we need a custom Reads[Person] to properly convert it.
I also post my code as reference.
implicit val personJsonReads: Reads[Person] = (
(__ \ "name").read[String] and
(__ \ "hobbies").read[List[Hobby]].orElse(Reads.pure(List()))
) (Person.apply _)
read[List[Hobby]].orElse(Reads.pure(List())) will generate the empty list when the value cannot convert to List[Hobby].

Argonaut: Generic method to encode/decode array of objects

I am trying to implement a generic pattern with which to generate marshallers and unmarshallers for an Akka HTTP REST service using Argonaut, handling both entity and collection level requests and responses. I have no issues in implementing the entity level as such:
case class Foo(foo: String)
object Foo {
implicit val FooJsonCodec = CodecJson.derive[Foo]
implicit val EntityEncodeJson = FooJson.Encoder
implicit val EntityDecodeJson = FooJson.Decoder
}
I am running into issues attempting to provide encoders and decoders for the following:
[
{ "foo": "1" },
{ "foo": "2" }
]
I have attempted adding the following to my companion:
object Foo {
implicit val FooCollectionJsonCodec = CodecJson.derive[HashSet[Foo]]
}
However, I am receiving the following error:
Error:(33, 90) value jencode0L is not a member of object argonaut.EncodeJson
I see this method truly does not exist but is there any other generic method to generate my expected result. I'm strongly avoiding using an additional case class to describe the collection since I am using reflection heavily in my use case.
At this point, I'd even be fine with a manually constructed Encoder and Decoder, however, I've found no documentation on how to construct it with the expected structure.
Argonaut have predefined encoders and decoders for Scala's immutable lists, sets, streams and vectors. If your type is not supported explicitly, as in the case of java.util.HashSet, you can easily add EncodeJson and DecodeJson for the type:
import argonaut._, Argonaut._
import scala.collection.JavaConverters._
implicit def hashSetEncode[A](
implicit element: EncodeJson[A]
): EncodeJson[java.util.HashSet[A]] =
EncodeJson(set => EncodeJson.SetEncodeJson[A].apply(set.asScala.toSet))
implicit def hashSetDecode[A](
implicit element: DecodeJson[A]
): DecodeJson[java.util.HashSet[A]] =
DecodeJson(cursor => DecodeJson.SetDecodeJson[A]
.apply(cursor)
.map(set => new java.util.HashSet(set.asJava)))
// Usage:
val set = new java.util.HashSet[Int]
set.add(1)
set.add(3)
val jsonSet = set.asJson // [1, 3]
jsonSet.jdecode[java.util.HashSet[Int]] // DecodeResult(Right([1, 3]))
case class A(set: java.util.HashSet[Int])
implicit val codec = CodecJson.derive[A]
val a = A(set)
val jsonA = a.asJson // { "set": [1, 3] }
jsonA.jdecode[A] // DecodeResult(Right(A([1, 3])))
Sample is checked on Scala 2.12.1 and Argonaut 6.2-RC2, but as far as I know it shouldn't depend on some latest changes.
Approach like this works with any linear or unordered homogenous data structure that you want to represent as JSON array. Also, this is preferable to creating a CodecJson: latter can be inferred automatically from JsonEncode and JsonDecode, but not vice versa. This way, your set will serialize and deserialize both when used independently or within other data type, as shown in example.
I don't use Argonaut but use spray-json and suspect solution can be similar.
Have you tried something like this ?
implicit def HashSetJsonCodec[T : CodecJson] = CodecJson.derive[Set[T]]
if it doesn't work I'd probably try creating more verbose implicit function like
implicit def SetJsonCodec[T: CodecJson](implicit codec: CodecJson[T]): CodecJson[Set[T]] = {
CodecJson(
{
case value => JArray(value.map(codec.encode).toList)
},
c => c.as[JsonArray].flatMap {
case arr: Json.JsonArray =>
val items = arr.map(codec.Decoder.decodeJson)
items.find(_.isError) match {
case Some(error) => DecodeResult.fail[Set[T]](error.toString(), c.history)
case None => DecodeResult.ok[Set[T]](items.flatMap(_.value).toSet[T])
}
}
)
}
PS. I didn't test this but hopefully it leads you to the right direction :)

immutable Map (de)serialization to/from Play JSON

I have following (simplified) structure:
case class MyKey(key: String)
case class MyValue(value: String)
Let's assume that I have Play JSON formatters for both case classes.
As an example I have:
val myNewMessage = collection.immutable.Map(MyKey("key1") -> MyValue("value1"), MyKey("key2") -> MyValue("value2"))
As a result of following transformation
play.api.libs.json.Json.toJson(myNewMessage)
I'm expecting something like:
{ "key1": "value1", "key2": "value2" }
I have tried writing the formatter, but somehow I can not succeed:
implicit lazy val mapMyKeyMyValueFormat: Format[collection.immutable.Map[MyKey, MyValue]] = new Format[collection.immutable.Map[MyKey, MyValue]] {
override def writes(obj: collection.immutable.Map[MyKey, MyValue]): JsValue = Json.toJson(obj.map {
case (key, value) ⇒ Json.toJson(key) -> Json.toJson(value)
})
override def reads(json: JsValue): JsResult[collection.immutable.Map[MyKey, MyValue]] = ???
}
I have no idea how to write proper reads function. Is there any simpler way of doing it? I'm also not satisfied with my writes function.
Thx!
The reason the writes method is not working is because you're transforming the Map[MyKey, MyValue] into a Map[JsValue, JsValue], but you can't serialize that to JSON. The JSON keys need to be strings, so you need some way of transforming MyKey to some unique String value. Otherwise you'd be trying to serialize something like this:
{"key": "keyName"} : {"value": "myValue"}
Which is not valid JSON.
If MyKey is as simple as stated in your question, this can work:
def writes(obj: Map[MyKey, MyValue]): JsValue = Json.toJson(obj.map {
case (key, value) => key.key -> Json.toJson(value)
}) // ^ must be a String
Play will then know how to serialize a Map[String, MyValue], given the appropriate Writes[MyValue].
But I'm not certain that's what you want. Because it produces this:
scala> Json.toJson(myNewMessage)
res0: play.api.libs.json.JsValue = {"key1":{"value":"value1"},"key2":{"value":"value2"}}
If this is the output you want:
{ "key1": "value1", "key2": "value2" }
Then your Writes should look more like this:
def writes(obj: Map[MyKey, MyValue]): JsValue = {
obj.foldLeft(JsObject(Nil)) { case (js, (key, value)) =>
js ++ Json.obj(key.key -> value.value)
}
}
Which produces this:
scala> writes(myNewMessage)
res5: play.api.libs.json.JsValue = {"key1":"value1","key2":"value2"}
Reads are easy so long as the structure of MyKey and MyValue are the same, otherwise I have no idea what you'd want it to do. It's very dependent on the actual structure you want. As is, I would suggest leveraging existing Reads[Map[String, String]] and transforming it to the type you want.
def reads(js: JsValue): JsResult[Map[MyKey, MyValue]] = {
js.validate[Map[String, String]].map { case kvMap =>
kvMap.map { case (key, value) => MyKey(key) -> MyValue(value) }
}
}
It's hard to see much else without knowing the actual structure of the data. In general I stay away from having to serialize and deserialize Maps.

Handle multidimensional JSON with scala Play framework

I am trying to send data from the client to the server using a JSON request. The body of the JSON request looks like this:
[
[
{"x":"0","y":"0","player":0},
{"x":"0","y":"1","player":0},
{"x":"0","y":"2","player":1}
],
[
{"x":"1","y":"0","player":0},
{"x":"1","y":"1","player":2},
{"x":"1","y":"2","player":0}
],
[
{"x":"2","y":"0","player":0},
{"x":"2","y":"1","player":1},
{"x":"2","y":"2","player":2}
]
]
On server side I would like to transform data with Play 2 framework to Scala 2D list like this:
List(
List(0,0,1),
List(0,2,0),
List(0,1,2)
)
this is 3x3 but it can be variable like 50x50 or so.
Thanks for any help.
It might be incomplete (don't know if you want to modelize the square matrix contraint as well) but something like that could be a good start:
First here is what the controller (and model) part can define
import play.api.libs.json.Json._
import play.api.libs.json._
type PlayerT = (String, String, Int)
implicit val playerTripleReads:Reads[PlayerT] = (
(__ \ "x").read[String] and
(__ \ "y").read[String] and
(__ \ "player").read[Int]
tupled
)
def getJson = Action(parse.json) { request =>
request.body.validate[List[List[PlayerT]]].map{
case xs => Ok(xs.mkString("\n"))
}.recoverTotal{
e => BadRequest("Detected error:"+ JsError.toFlatJson(e))
}
}
In this version, you'll get a list of list holding validated tuples of the form (String, String, Int) which has been aliased with the PlayerT type to save some typing.
As you may saw, the reader as been created "by-hand" by composing (using the and combinator) three basic blocks and the result is flattened using the tupled operator.
With this solution you're now on track to play with those tuples, but IMO the code will suffer from bad readability, because of the usage of _1, _2 and _3 along the way.
So here is a different approach (which is in fact even easier...) that tackles this problem of sane coding, this will simply defined a `case class that models your atomic data
case class Player(x:String, y:String, player:Int)
implicit val playerReads = Json.reads[Player]
def getJson = Action(parse.json) { request =>
request.body.validate[List[List[Player]]].map{
case xs => Ok(xs.mkString("\n"))
}.recoverTotal{
e => BadRequest("Detected error:"+ JsError.toFlatJson(e))
}
}
Note that, the reader will always follow further changes in your data representation, that is the case class's fields thanks to the use of the implicit creation of the reader at compile time.
Now, you'll be able to use x, y and player fields rather than _1, _2 and _3.