Create a generic Json serialization function - json

Is it possible to create a generic function in Scala, using Play Framework 2.2, that will serialize an arbitrary object to JSON, without having to be supplied a writer or formatter?
For instance, this non-generic code will create a JSON response given a Customer:
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class Customer(id: Int, name: String)
object scratch {
val p = Customer(1, "n")
//> p : Customer = Customer(1,n)
def createJsonResponseCustomer(data: Customer) = {
implicit val formatter = Json.format[Customer]
Json.obj("success" -> true, "data" -> Json.toJson[Customer](data))
}
createJsonResponseCustomer(p)
//> res0: play.api.libs.json.JsObject = {"success":true,"data":{"id":1,"name":"n"}}
}
To avoid having to define the formatter for each different object, I'd like to create a generic function like this:
def createJsonResponse[T](data: T) = {
implicit val formatter = Json.format[T]
Json.obj("success" -> true, "data" -> Json.toJson[T](data))
}
But this attempt produces the error No unapply function found at Json.format[T].
In other words, this works:
def getFormatter(c: Customer) = Json.format[Customer]
but this doesn't:
def getFormatterGeneric[T](c: T) = Json.format[T]
Is there any way around this?

You need to define the formatter somewhere, for each type you wish to read or write. This is because the formatter instances are resolved at compile time, not at runtime. This is a good thing, because it means trying to serialize a type that does not have a serializer becomes a compile-time error, not a runtime one.
Instead of defining the formatters on the fly, define them in a module that you can reuse, e.g.
object JsonFormatters {
implicit val customerWrites: Format[Customer] = Json.format[Customer]
}
Then import JsonFormatters._ in the scope that you want to write some JSON.
Now, you can write a generic method similar to what you wanted: you just have to specify the requirement for a formatter in the signature of your method. In practice, this is an implicit paramter of type Writes[T].
def createJsonResponse[T](data: T)(implicit writes: Writes[T]) =
Json.obj("success" -> true, "data" -> Json.toJson[T](data))
You can also write this method signature using context bound syntax, i.e.
def createJsonResponse[T : Writes](data: T) = ...
This requires that there is an instance of Writes[T] in scope; but the compiler will choose the correct instance for you based on the type T, rather than you resolving it explicitly.
Note that Writes[T] is a supertype of Format[T]; since you are only writing JSON in this method, there's no need to specify a requirement for Format[T], which would also give you Reads[T].

Related

Implicit conversion of generic type T to JsValue in Play! Framework

I have a problem with declaring of generic method in Play 2.6 application that converts JSON to instance of one of the case class models.
All models declared with helper objects and formatters:
import play.api.libs.json.{Json, OFormat}
case class Shot(id: Long, likes_count: Long)
object Shot {
implicit val format: OFormat[Shot] = Json.format[Shot]
}
val s1: Shot = Json.toJson(f).as[Shot] // Works great
def testJsonGeneric[T](js: JsValue)(implicit ev: OFormat[T]): T = {
js.as[T](ev)
}
val s2: Shot = testJsonGeneric(Json.toJson(f)) // could not find implicit value for parameter ev: play.api.libs.json.OFormat[T]. Compilation failed
The last line of code throws
could not find implicit value for parameter ev: play.api.libs.json.OFormat[T]
But if I call my generic method like this (with explicit formatter) it works just fine:
val s2: Shot = testJsonGeneric(Json.toJson(f))(Shot.format)
However, it looks like if I expect my JSON to return a list of objects I have to define an extra formatter for List[Shot] to pass explicitly to the method when default Play's json.as[List[Shot]] could easily allow me to do this with a single existing formatter like the one that already defined in the helper object.
So, is it even possible to provide formatters implicitly for generic type T in my case?
Thank you
You can definitely do this, you just have to change the declaration a bit.
Move the case class and companion declaration outside the method, and then explicitly import Shot._ to bring the implicit in scope:
import play.api.libs.json.{JsValue, Json, OFormat}
object Foo {
case class Shot(id: Long, likes_count: Long)
object Shot {
implicit def format: OFormat[Shot] = Json.format[Shot]
}
def main(args: Array[String]): Unit = {
import Shot._
val f = Shot(1, 2)
def testJsonGeneric[T](js: JsValue)(implicit ev: OFormat[T]): T = {
js.as[T](ev)
}
val s2: Shot = testJsonGeneric(Json.toJson(f))
}
}

Json "Validate" for Play

For the validate method on request.body it matches the attribute name and value type of the json object to those defined in the model definition. Now if I were to add an extra attribute to the json object and try to validate it, it passes as a JsSuccess when it shouldn't.
{
"Name": "Bob",
"Age": 20,
"Random_Field_Not_Defined_in_Models": "Test"
}
My Person Class is defined as follows
case class Person(name: String, age: Int)
I'm assuming you've been using the built-in Reads[T] or Format[T] converters that Play gives you via Json.reads[T], e.g.:
import play.api.libs.json._
val standardReads = Json.reads[Person]
While these are super-handy, if you need additional validation, you'll have to define a custom Reads[Person] class; but fortunately we can still leverage the built-in JSON-to-case-class macro to do the basic checking and conversion, and then add an extra layer of custom checks if things seem OK:
val standardReads = Json.reads[Person]
val strictReads = new Reads[Person] {
val expectedKeys = Set("name", "age")
def reads(jsv:JsValue):JsResult[Person] = {
standardReads.reads(jsv).flatMap { person =>
checkUnwantedKeys(jsv, person)
}
}
private def checkUnwantedKeys(jsv:JsValue, p:Person):JsResult[Person] = {
val obj = jsv.asInstanceOf[JsObject]
val keys = obj.keys
val unwanted = keys.diff(expectedKeys)
if (unwanted.isEmpty) {
JsSuccess(p)
} else {
JsError(s"Keys: ${unwanted.mkString(",")} found in the incoming JSON")
}
}
}
Note how we utilize standardReads first, to make sure we're dealing with something that can be converted to a Person. No need to reinvent the wheel here.
We use flatMap to effectively short-circuit the conversion if we get a JsError from standardReads - i.e. we only call checkUnwantedKeys if needed.
checkUnwantedKeys just uses the fact that a JsObject is really just a wrapper around a Map, so we can easily check the names of the keys against a whitelist.
Note that you could also write that flatMap using a for-comprehension, which starts to look a lot cleaner if you need even more checking stages:
for {
p <- standardReads.reads(jsv)
r1 <- checkUnexpectedFields(jsv, p)
r2 <- checkSomeOtherStuff(jsv, r1)
r3 <- checkEvenMoreStuff(jsv, r2)
} yield r3
If you want to avoid too much boilerplate it is possible to make a more generic solution using a little bit of scala reflection:
import play.api.libs.json._
import scala.reflect.runtime.universe._
def checkedReads[T](underlyingReads: Reads[T])(implicit typeTag: TypeTag[T]): Reads[T] = new Reads[T] {
def classFields[T: TypeTag]: Set[String] = typeOf[T].members.collect {
case m: MethodSymbol if m.isCaseAccessor => m.name.decodedName.toString
}.toSet
def reads(json: JsValue): JsResult[T] = {
val caseClassFields = classFields[T]
json match {
case JsObject(fields) if (fields.keySet -- caseClassFields).nonEmpty =>
JsError(s"Unexpected fields provided: ${(fields.keySet -- caseClassFields).mkString(", ")}")
case _ => underlyingReads.reads(json)
}
}
}
Then you can specify your reads instances as:
implicit val reads = checkedReads(Json.reads[Person])
This leverages a fair bit of Scala type magic and also the reflection library (that lets you look at fields on classes).
Rather than relying on a fixed set of fields the classFields method gets all of the fields dynamically for the case class (type param T). It looks at all of the members and collects only the case class accessors (otherwise we'd pick up methods like toString). It returns a Set[String] of field names.
You'll notice that the checkedReads takes an implicit TypeTag[T]. This is supplied by the compiler at compile time and used by the typeOf method.
The remaining code is fairly self explanatory. If the incoming json matches our first case (it is a JsObject and there are fields not on the case class) then we return a JsError. Otherwise we pass it on to the underlying reader.

How to fail on json with unused fields in play-json scala? [duplicate]

For the validate method on request.body it matches the attribute name and value type of the json object to those defined in the model definition. Now if I were to add an extra attribute to the json object and try to validate it, it passes as a JsSuccess when it shouldn't.
{
"Name": "Bob",
"Age": 20,
"Random_Field_Not_Defined_in_Models": "Test"
}
My Person Class is defined as follows
case class Person(name: String, age: Int)
I'm assuming you've been using the built-in Reads[T] or Format[T] converters that Play gives you via Json.reads[T], e.g.:
import play.api.libs.json._
val standardReads = Json.reads[Person]
While these are super-handy, if you need additional validation, you'll have to define a custom Reads[Person] class; but fortunately we can still leverage the built-in JSON-to-case-class macro to do the basic checking and conversion, and then add an extra layer of custom checks if things seem OK:
val standardReads = Json.reads[Person]
val strictReads = new Reads[Person] {
val expectedKeys = Set("name", "age")
def reads(jsv:JsValue):JsResult[Person] = {
standardReads.reads(jsv).flatMap { person =>
checkUnwantedKeys(jsv, person)
}
}
private def checkUnwantedKeys(jsv:JsValue, p:Person):JsResult[Person] = {
val obj = jsv.asInstanceOf[JsObject]
val keys = obj.keys
val unwanted = keys.diff(expectedKeys)
if (unwanted.isEmpty) {
JsSuccess(p)
} else {
JsError(s"Keys: ${unwanted.mkString(",")} found in the incoming JSON")
}
}
}
Note how we utilize standardReads first, to make sure we're dealing with something that can be converted to a Person. No need to reinvent the wheel here.
We use flatMap to effectively short-circuit the conversion if we get a JsError from standardReads - i.e. we only call checkUnwantedKeys if needed.
checkUnwantedKeys just uses the fact that a JsObject is really just a wrapper around a Map, so we can easily check the names of the keys against a whitelist.
Note that you could also write that flatMap using a for-comprehension, which starts to look a lot cleaner if you need even more checking stages:
for {
p <- standardReads.reads(jsv)
r1 <- checkUnexpectedFields(jsv, p)
r2 <- checkSomeOtherStuff(jsv, r1)
r3 <- checkEvenMoreStuff(jsv, r2)
} yield r3
If you want to avoid too much boilerplate it is possible to make a more generic solution using a little bit of scala reflection:
import play.api.libs.json._
import scala.reflect.runtime.universe._
def checkedReads[T](underlyingReads: Reads[T])(implicit typeTag: TypeTag[T]): Reads[T] = new Reads[T] {
def classFields[T: TypeTag]: Set[String] = typeOf[T].members.collect {
case m: MethodSymbol if m.isCaseAccessor => m.name.decodedName.toString
}.toSet
def reads(json: JsValue): JsResult[T] = {
val caseClassFields = classFields[T]
json match {
case JsObject(fields) if (fields.keySet -- caseClassFields).nonEmpty =>
JsError(s"Unexpected fields provided: ${(fields.keySet -- caseClassFields).mkString(", ")}")
case _ => underlyingReads.reads(json)
}
}
}
Then you can specify your reads instances as:
implicit val reads = checkedReads(Json.reads[Person])
This leverages a fair bit of Scala type magic and also the reflection library (that lets you look at fields on classes).
Rather than relying on a fixed set of fields the classFields method gets all of the fields dynamically for the case class (type param T). It looks at all of the members and collects only the case class accessors (otherwise we'd pick up methods like toString). It returns a Set[String] of field names.
You'll notice that the checkedReads takes an implicit TypeTag[T]. This is supplied by the compiler at compile time and used by the typeOf method.
The remaining code is fairly self explanatory. If the incoming json matches our first case (it is a JsObject and there are fields not on the case class) then we return a JsError. Otherwise we pass it on to the underlying reader.

What is the easest way return objects as json response from action in play framework (scala)?

I'm new to scala and play framework.
Why does scala not have something like this??
class Customer (idx: Int, emailx: String) {
val id: Int = idx
val email: String = emailx
}
....
def customers = Action {
val customer = new Customer(1, "Customer1")
Ok(Json.toJson(customer))
}
I like play frameowrk (with scala, its productivity)
But,
Why should I map each field of my object manulay to json field?? Was it so hard for scala to implement this future like in Java or C#, even php has json_encode.
Is there any way to achieve this simple goal (return object as json) without any additional manipulations?
Macros are slick and perfect for generating simple case class formats
implicit val jsonFormat = Json.format[Customer]
Typically you put this declaration in your companion object to the type you are generating a format for. This way it is implicitly in scope in any file that you import your type (Customer). Like this:
case class Customer(...)
object Customer {
implicit val jsonFormat = Json.format[Customer]
}
Then in your controller you can do
Json.toJson(customer)
which will produce the JsValue type expected by Play.
For my classes I define a convertor, so that I can write exactly what you have written, e.g. Json.toJson(customer), but the convertor, though simple, does currently have to be written once. E.g.
implicit val customerWrites = new Writes[Customer] {
def writes(customer:Customer) = Json.obj(
"id" -> customer.id,
"email" -> customer.email
)
}
Perhaps macros, into which I have not delved, could do this more automatically...

Handling Pk[Int] values in spray-json

[edit]
So, i got a quick and dirty solution, thanks to Edmondo1984, I don't know if it's the best solution. I don't handle null values with pattern matching at the write function. You can read more details about my problem after this editing. Here is my code now:
object DBNames extends DefaultJsonProtocol {
implicit val pkFormat: JsonFormat[Pk[Int]] = new JsonFormat[Pk[Int]] {
def write(obj: Pk[Int]): JsValue = JsNumber(obj.get)
def read(json: JsValue): Pk[Int] = json.asJsObject.getFields("id") match {
case Seq(JsNumber(id)) => new Pk[Int] { id.toInt }
case _ => throw new DeserializationException("Int expected")
}
}
implicit val nameFormat = jsonFormat2(Name)
jsonFormat2 will implicitly use pkFormat to parse Pk[Int] values.
In my controller class I have this:
def listNames() = Action {
val names = DBNames.findAll()
implicit val writer = DBNames.nameFormat
var json = names.toJson
Ok(json.toString()).as("application/json")
}
I had to get the nameFormat from my model and make it implicit, so bars.toJson could use it to parse the Seq[Name] names.
[/edit]
I'm trying to use Play! Framework with Scala, I'm new to Scala programming and Play Framework, and everything seems nice, but I'm working on this problem during several hours and didn't find a solution.
I have a Case Class:
case class Name (id: Pk[Int], name: String)
And an object to deal with MySql. I created a implicit val nameFormat = jsonFormat2(Name) to deal with JSON.
object DBNames extends DefaultJsonProtocol {
implicit val nameFormat = jsonFormat2(Name)
var parser =
{
get[Pk[Int]]("id") ~
get[String]("name") map {
case id ~ name => Name(id,name)
}
}
def findAll():Seq[Name] =
{
DB.withConnection {
implicit connection =>
SQL("select * from names").as(DBNames.parser *)
}
}
def create(name: Name){
DB.withConnection {
implicit connection =>
SQL("insert into names (name) values ({name})").on(
'name -> name.name
).executeUpdate()
}
}
}
But when I try to compile it, Play! gives me this result:
[error] D:\ProjetosJVM\TaskList\app\models\Names.scala:20: could not find implicit value for evidence parameter of type models.DBNames.JF[anorm.Pk[Int]]
It seems like he couldn't find a way to parse the id value, since it is a Pk[Int] value.
So, by reading this: https://github.com/spray/spray-json I didn't found a way to parse it without creating a complete object parser like they show in the documentation:
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object ColorJsonFormat extends RootJsonFormat[Color] {
def write(c: Color) = JsObject(
"name" -> JsString(c.name),
"red" -> JsNumber(c.red),
"green" -> JsNumber(c.green),
"blue" -> JsNumber(c.blue)
)
def read(value: JsValue) = {
value.asJsObject.getFields("name", "red", "green", "blue") match {
case Seq(JsString(name), JsNumber(red), JsNumber(green), JsNumber(blue)) =>
new Color(name, red.toInt, green.toInt, blue.toInt)
case _ => throw new DeserializationException("Color expected")
}
}
}
}
I have a "big" (actually small) project where I want to make most of things work with Ajax, so I think this is not a good way to do it.
How can I deal with JSON objects in this project, where almost all case classes will have a "JSON parser", without creating large ammounts of code, like the snippet above? And also, how can I make it work with an Seq[Name]?
You don't need to write a complete parser. The compiler says:
[error] D:\ProjetosJVM\TaskList\app\models\Names.scala:20: could not find implicit
value for evidence parameter of type models.DBNames.JF[anorm.Pk[Int]]
The scala compiler is looking for an implicit parameter of type JF[anorm.Pk[Int]] and there is no such an implicit parameter in scope. What is JF[anorm.Pk[Int]]? Well, you need to know the library and I didn't, so I had browsed spray-json source and found out:
trait StandardFormats {
this: AdditionalFormats =>
private[json] type JF[T] = JsonFormat[T] // simple alias for reduced verbosity
so JF[T] is just an alias for JsonFormat[T]. It all make sense: PK[Int] is a class coming from Anorm and spray-json provides out-of-the-box json support for standard types, but does not even know Anorm exists So you have to code your support for Pk[Int] and make it implicit in scope.
You will have code like the following:
object DBNames extends DefaultJsonProtocol {
implicit val pkFormat : JsonFormat[Pk[Int]] = new JsonFormat[Pk[Int]] {
//implementation
}
// rest of your code
}
If you have just started with Scala, you would probably have to read more about implicits and their resolution. I am providing you with a minimal answer: once you have provided the right implementation, your code will compile. I suggest you to refer to the javadoc of anorm.Pk and of JsonFormat to understand how to implement it correctly for your type.
Pk looks like scala.Option and in StandardFormats source code inside spray-json you find the JsonFormat implementation for Option, from which you can copy