I am trying to create a dynamic parser which allows me to parse json content into different classes depending on a class name.
I will get the json and the class name (as String) and I would like to do something like this:
val theCaseClassName = "com.ardlema.JDBCDataProviderProperties"
val myCaseClass = Class.forName(theCaseClassName)
val jsonJdbcProperties = """{"url":"myUrl","userName":"theUser","password":"thePassword"}"""
val json = Json.parse(jsonJdbcProperties)
val value = Try(json.as[myClass])
The above code obviously does not compile because the json.as[] method tries to convert the node into a "T" (I have an implicit Reads[T] defined for my case class)
What would be the best way to get a proper "T" to pass in to the json.as[] method from the original String?
A great solution that might work would be to do polymorphic deserialization. This allows you to add a field (like "type") to your json and allow Jackson (assuming you're using an awesome json parser like Jackson) to figure out the proper type on your behalf. It looks like you might not be using Jackson; I promise it's worth using.
This post gives a great introduction to polymorphic types. It covers many useful cases including the case where you can't modify 3rd party code (here you add a Mixin to annotate the type hierarchy).
The simplest case ends up looking like this (and all of this works great with Scala objects too -- jackson even has a great scala module):
object Test {
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type"
)
#JsonSubTypes(Array(
new Type(value = classOf[Cat], name = "cat"),
new Type(value = classOf[Dog], name = "dog")
))
trait Animal
case class Dog(name: String, breed: String, leash_color: String) extends Animal
case class Cat(name: String, favorite_toy: String) extends Animal
def main(args: Array[String]): Unit = {
val objectMapper = new ObjectMapper with ScalaObjectMapper
objectMapper.registerModule(DefaultScalaModule)
val dogStr = """{"type": "dog", "name": "Spike", "breed": "mutt", "leash_color": "red"}"""
val catStr = """{"type": "cat", "name": "Fluffy", "favorite_toy": "spider ring"}"""
val animal1 = objectMapper.readValue[Animal](dogStr)
val animal2 = objectMapper.readValue[Animal](catStr)
println(animal1)
println(animal2)
}
}
This generates this output:
// Dog(Spike,mutt,red)
// Cat(Fluffy,spider ring)
You can also avoid listing the subtype mapping, but it requires that the json "type" field is a bit more complex. Experiment with it; you might like it. Define Animal like this:
#JsonTypeInfo(
use = JsonTypeInfo.Id.CLASS,
include = JsonTypeInfo.As.PROPERTY,
property = "type"
)
trait Animal
And it produces (and consumes) json like this:
/*
{
"breed": "mutt",
"leash_color": "red",
"name": "Spike",
"type": "classpath.to.Test$Dog"
}
{
"favorite_toy": "spider ring",
"name": "Fluffy",
"type": "classpath.to.Test$Cat"
}
*/
You should select your Reads[T] based on the class name. Unfortunately this will probably have to be a manual pattern match:
val r: Reads[_] = theCaseClassName match {
case "com.ardlema.JDBCDataProviderProperties" => JDBCReads
case ... => ...
}
val value = json.as(r).asInstanceOf[...]
Alternately, look at the implementation of json.as; at some point it's probably requiring a classTag and then calling .runtimeClass on it. Assuming that's so, you can just do whatever it is and pass your own myCaseClass there.
Related
I'm trying to write a method that will allow Jackson ObjectMapper readValue on a json string to a parameterized object type. Something like this
case class MyObj(field1: String, field2: String)
val objectMapper: ObjectMapper = new ObjectMapper().registerModule(new DefaultScalaModule)
def fromJson[T](jsonString: String, objTyp: T): T = {
objectMapper.readValue(jsonString, classOf[T])
}
val x = fromJson("""{"field1": "something", "field2": "something"}""", MyObj)
This of course returns an error of
class type required but T found
i've looked at this issue Scala classOf for type parameter
but it doesn't seem to help. It seems like this is possible to do somehow. Looking for any help
You have to give it the actual runtime class to parse into, not just a type parameter.
One way to do it is passing the class directly:
def fromJson[T](json: String, clazz: Class[T]) = objectMapper.readValue[T](json, clazz)
val x = fromJson("""...""", classOf[MyObj])
Alternatively, you can use ClassTag, which looks a bit messier in implementation, but kinda prettier at call site:
def fromJson[T : ClassTag](json: String): T = objectMapper.readValue[T](
json,
implicitly[ClassTag[T]].runtimeClass.asInstanceOf[Class[T]]
)
val x = fromJson[MyObj]("""{"field1": "something", "field2": "something"}""")
i've looked at this issue Scala classOf for type parameter but it doesn't seem to help.
In the very first answer there it's written classTag[T].runtimeClass as a replacement of classOf[T]. This should help.
Regarding the signature
def fromJson[T](jsonString: String, objTyp: T): T
You should notice that MyObj has type MyObj.type (companion-object type), not MyObj (case-class type).
Class companion object vs. case class itself
So if you call fromJson("""...""", MyObj) then the types in these two places
def fromJson[...](jsonString: String, objTyp: ???): ???
^^^ ^^^ <--- HERE
can't be the same.
If it's enough for you to call
fromJson("""...""", classOf[MyObj])
or
fromJson[MyObj]("""...""")
(normally it should be enough) then please see #Dima's answer, you should prefer those options, they're easier.
Just in case, if you really want to call like fromJson("""...""", MyObj) then for example you can use the type class ToCompanion (this is more complicated) from
Invoke construcotr based on passed parameter
Get companion object of class by given generic type Scala (answer)
// ToCompanion should be defined in a different subproject
def fromJson[C, T](jsonString: String, objTyp: C)(implicit
toCompanion: ToCompanion.Aux[C, T],
classTag: ClassTag[T]
): T =
objectMapper.readValue(jsonString, classTag.runtimeClass.asInstanceOf[Class[T]])
val x = fromJson("""{"field1": "something", "field2": "something"}""", MyObj)
// MyObj(something,something)
How to configure the spray-json parsing on parsing options?
Similarly as Jackson Parsing Features.
For example, I am parsing a json that has a field that my case class has not, and it is breaking:
spray.json.DeserializationException: Object is missing required member 'myfield'
UPDATE :
A simple example:
case class MyClass(a: String, b: Long);
and try to parse an incomplete json like
val data = "{a: \"hi\"}"
with a spray-json format like:
jsonFormat2(MyClass.apply)
// ...
data.parseJson.convertTo[MyClass]
(simplified code).
But the question goes further, I want to ask about configuration options like in other parsers. More examples:
Be able to ignore fields that exist in the JSON but not in the case class.
Ways of managing nulls or nonexistent values.
etc.
SprayJson allows you to define custom parsers like so:
case class Foo(a: String, b: Int)
implicit object FooJsonFormat extends RootJsonFormat[Foo] {
override def read(json: JsValue): Foo = {
json.asJsObject.getFields("name", "id") match {
case Seq(JsString(name), id) =>
Foo(name, id.convertTo[Int])
}
}
override def write(obj: Foo): JsValue = obj.toJson
}
This allows you to parse any arbitrary payload and pull out the fields "name" and "id" - other fields are ignored. If those fields are not guaranteed you can add something like:
case Seq(JsString(name), JsNull) =>
Foo(name, 0)
You should look at what's available in JsValue.scala - in particular JsArray may come in handy if you're getting payloads with anonymous arrays (i.e. the root is [{...}] instead of {"field":"value"...})
Spray Json doesn't support default parameters. So You cannot have a case class like
case class MyClass(a: String, b: Int = 0)
and then parse json like {"a":"foo"}
However if you make the second parameter as Option. then it works.
import spray.json._
case class MyClass(a: String, b: Option[Int] = None)
object MyProtocol extends DefaultJsonProtocol {
implicit val f = jsonFormat2(MyClass)
}
import MyProtocol.f
val mc1 = MyClass("foo", Some(10))
val strJson = mc1.toJson.toString
val strJson2 = """{"a": "foo"}"""
val mc2 = strJson2.parseJson.convertTo[MyClass]
println(mc2)
This question already has answers here:
How to use jackson to deserialize to Kotlin collections
(3 answers)
Closed 7 years ago.
What is the correct syntax to deserialize the following JSON:
[ {
"id" : "1",
"name" : "Blues"
}, {
"id" : "0",
"name" : "Rock"
} ]
I tried:
//Works OK
val dtos = mapper.readValue(json, List::class.java)
However I want:
val dtos : List<GenreDTO> = mapper.readValue(json,
List<GenreDTO>::class.java)
The above syntax is not correct and gives: only classes are allowed on the left hand side of a class literal
NOTE: The answer from #IRus is also correct, it was being modified at the same time I wrote this to fill in more details.
You should use the Jackson + Kotlin module or you will have other problems deserializing into Kotlin objects when you do no have a default constructor.
Your first sample of the code:
val dtos = mapper.readValue(json, List::class.java)
Is returning an inferred type of List<*> since you did not specify more type information, and it is actually a List<Map<String,Any>> which is not really "working OK" but is not producing any errors. It is unsafe, not typed.
The second code should be:
import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue
val mapper = jacksonObjectMapper()
// ...
val genres: List<GenreDTO> = mapper.readValue(json)
You do not need anything else on the right side of the assignment, the Kotlin module for Jackson will reify the generics and create the TypeReference for Jackson internally. Notice the readValue import, you need that or .* for the com.fasterxml.jackson.module.kotlin package to have the extension functions that do all of the magic.
A slightly different alternative that also works:
val genres = mapper.readValue<List<GenreDTO>>(json)
There is no reason to NOT use the extension functions and the add-on module for Jackson. It is small and solves other issues that would require you to jump through hoops to make a default constructor, or use a bunch of annotations. With the module, your class can be normal Kotlin (optional to be data class):
class GenreDTO(val id: Int, val name: String)
The error you're getting is about following expression:
List<GenreDTO>::class.java
Because of how jvm treats generics there's no separate class for List<GenreDTO> thus compiler complains. Similarly in Java the following will not compile:
List<GenreDTO>.getClass()
Here's a sample that will deserialize the list properly:
val value:List<GenreDTO> = mapper.readValue(json, object : TypeReference<List<GenreDTO>>() {})
As #JaysonMinard has pointed out you can use jackson-module-kotlin to simplify the invocation to:
val genres: List<GenreDTO> = mapper.readValue(json)
// or
val genres = mapper.readValue<List<GenreDTO>>(json)
This is possible because of reified type parameters. Consider looking at Extensions to find out details.
Following code works well for me:
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue
import com.fasterxml.jackson.module.kotlin.registerKotlinModule
val json = """[ {
"id" : "1",
"name" : "Blues"
}, {
"id" : "0",
"name" : "Rock"
} ]"""
data class GenreDTO(val id: Int, val name: String)
val mapper = ObjectMapper().registerKotlinModule()
fun main(args: Array<String>) {
val obj: List<GenreDTO> = mapper.readValue(json)
obj.forEach {
println(it)
}
}
This work because of extension function defined inside jackson-kotlin-module (that used reified generics):
public inline fun <reified T: Any> ObjectMapper.readValue(content: String): T = readValue(content, object: TypeReference<T>() {})
Thanks #JaysonMinard for notify me about it.
Output:
GenreDTO(id=1, name=Blues)
GenreDTO(id=0, name=Rock)
I'm new to scala and play framework.
Why does scala not have something like this??
class Customer (idx: Int, emailx: String) {
val id: Int = idx
val email: String = emailx
}
....
def customers = Action {
val customer = new Customer(1, "Customer1")
Ok(Json.toJson(customer))
}
I like play frameowrk (with scala, its productivity)
But,
Why should I map each field of my object manulay to json field?? Was it so hard for scala to implement this future like in Java or C#, even php has json_encode.
Is there any way to achieve this simple goal (return object as json) without any additional manipulations?
Macros are slick and perfect for generating simple case class formats
implicit val jsonFormat = Json.format[Customer]
Typically you put this declaration in your companion object to the type you are generating a format for. This way it is implicitly in scope in any file that you import your type (Customer). Like this:
case class Customer(...)
object Customer {
implicit val jsonFormat = Json.format[Customer]
}
Then in your controller you can do
Json.toJson(customer)
which will produce the JsValue type expected by Play.
For my classes I define a convertor, so that I can write exactly what you have written, e.g. Json.toJson(customer), but the convertor, though simple, does currently have to be written once. E.g.
implicit val customerWrites = new Writes[Customer] {
def writes(customer:Customer) = Json.obj(
"id" -> customer.id,
"email" -> customer.email
)
}
Perhaps macros, into which I have not delved, could do this more automatically...
I've a case class similar to this found here :
case class WebCategory(topGroupName: String,
topGroupID: String,
webCategoryName : String,
webCategoryID : String,
subWebCats:Seq[SubWebCat])
case class SubWebCat(name:String, id:String)
And my request body json is having the exact same key names as of case class. For ex:
{
"webCategoryID" : "blah",
"webCategoryName" : "abcabc",
"topGroupID" : "blah",
"topGroupName" : "namehere",
"subWebCats" : [
{
"name" : "blah",
"id" : "idblah"
},
{
"name" : "another blah",
"id" : "another idblah"
}
]
}
The case class & req body keys are same then is it possible to directly build the case class object from request json? If it is possible then how can I do this? Any references would help. If it is not possible then this means I've to define my custom implicit converter explained in the answer in which I don't have any problem implementing it.
Note: I'm using Play 2.3 & Scala 11 for my development
You can use Play's built-in JSON validation for this quite easily. You don't need to add any third-party dependencies for this.
case class WebCategory(topGroupName: String,
topGroupID: String,
webCategoryName : String,
webCategoryID : String,
subWebCats:Seq[SubWebCat])
object WebCategory {
implicit val fmt = Json.format[WebCategory]
}
case class SubWebCat(name:String, id:String)
object SubWebCat {
implicit val fmt = Json.format[SubWebCat]
}
Then, in your controller action:
def save: Action(parse.json) { implicit request =>
request.body.validate[WebCategory].fold(
errors => BadRequest(errors.mkString),
category => Ok("saved category")
)
}
We use FasterXml for serialization and deserialization as follows.
include this dependency in your build.sbt
"com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.4.0-rc2"
Create a two helper functions toJson and fromJson to serialize and deserialize
object JsonProvider {
//create mapper and register scala module
private val mapper = new ObjectMapper
mapper.registerModule(DefaultScalaModule)
def toJson(obj: Object): String = {
val writer = new StringWriter
mapper.writeValue(writer, obj)
writer.toString
}
def fromJson[T: scala.reflect.Manifest](json: String): T = {
mapper.readValue(json, scala.reflect.classTag[T].runtimeClass).asInstanceOf[T]
}
}
Use it as follows to cast your request body into case class.
JsonProvider.fromJson[WebCategory](request.body.toString())
to convert caseclass to json use it like this.
JsonProvider.toJson(obj);
where "obj" is the object of case class.
Problem with Play-Json default.
Suppose you have case class A with three parameters as follows
case class A(id:String,name:String,roll:Int)
and if your Json you want to parse is as follows
{
name:"XYZ",
roll:22
}
you can't parse this Json with play-json because of the missing field or one way is to define your Read and write functions which is very cumbersome.
but with fasterXML you can easily parse this Json like :
val a = JsonProvider.fromJsonA
and you can assign Id to case class A later like val a1 = a.copy(id="xyz")
i had this problem then i switched from play-json to fasterXML the problem was
How to send Json from client with missing fields for its corresponding Case Class after using Json.format function