I have scala class like class A(b:Seq[String])
My problem is when I deserialize it from text which have no b field my class contains null. It is possible to force deserealizer to fill with empty Seq?
I use com.fasterxml.jackson.databind.ObjectMapper with com.fasterxml.jackson.module.scala.DefaultScalaModule.
EDIT:
I want solution that fix all such fields without explicitly mentioning full list of them. Ir changing all declarations.
This is not currently supported by Jackson, unfortunately.
You can see the relevant GitHub ticket here: https://github.com/FasterXML/jackson-databind/issues/347
Your best bet is to map null into an empty Seq in the class constructor or in an accessor method:
class A(_b: Seq[String]) {
val b = _b match {
case null => Nil
case bs => bs
}
}
(See also https://stackoverflow.com/a/20655330/8261 for other options)
If you use Spray JSON, then a simple example which doesn't handle the absence of the b field would look like:
import spray.json._
case class A(b: Seq[String])
object Protocol extends DefaultJsonProtocol {
implicit def aFormat = jsonFormat1(A)
}
import Protocol._
val str1 = """{ "b" : [] }""""
val str2 = """{ "b" : ["a", "b", "c"] }""""
val str3 = """{}"""
str1.parseJson.convertTo[A]//successful
str2.parseJson.convertTo[A]//successful
str3.parseJson.convertTo[A]//Deserialization error
In Spray JSON, this can be solved by writing a more detailed protocol format for class A:
import spray.json._
case class A(b: Seq[String])
object Protocol extends DefaultJsonProtocol {
implicit object aFormat extends RootJsonFormat[A] {
override def write(obj: A): JsValue = JsObject("b" -> obj.b.toJson)
override def read(json: JsValue): A = json match {
//Case where the object has exactly one member, 'b'
case JsObject(map) if map.contains("b") && map.size == 1 => A(map("b").convertTo[Seq[String]])
//Case where the object has no members
case JsObject(map) if map.isEmpty => A(Seq())
//Any other json value, which cannot be converted to an instance of A
case _ => deserializationError("Malformed")
}
}
}
import Protocol._
val str1 = """{ "b" : [] }""""
val str2 = """{ "b" : ["a", "b", "c"] }""""
val str3 = """{}"""
str1.parseJson.convertTo[A]//successful
str2.parseJson.convertTo[A]//successful
str3.parseJson.convertTo[A]//successful
I had a similar problem deserializing String arrays with jackson, and found that jackson apparently (have not tried yet) fixed this in jackson-module 2.1.2. It might work for Seq.
https://github.com/FasterXML/jackson-module-scala/issues/48
Related
How to configure the spray-json parsing on parsing options?
Similarly as Jackson Parsing Features.
For example, I am parsing a json that has a field that my case class has not, and it is breaking:
spray.json.DeserializationException: Object is missing required member 'myfield'
UPDATE :
A simple example:
case class MyClass(a: String, b: Long);
and try to parse an incomplete json like
val data = "{a: \"hi\"}"
with a spray-json format like:
jsonFormat2(MyClass.apply)
// ...
data.parseJson.convertTo[MyClass]
(simplified code).
But the question goes further, I want to ask about configuration options like in other parsers. More examples:
Be able to ignore fields that exist in the JSON but not in the case class.
Ways of managing nulls or nonexistent values.
etc.
SprayJson allows you to define custom parsers like so:
case class Foo(a: String, b: Int)
implicit object FooJsonFormat extends RootJsonFormat[Foo] {
override def read(json: JsValue): Foo = {
json.asJsObject.getFields("name", "id") match {
case Seq(JsString(name), id) =>
Foo(name, id.convertTo[Int])
}
}
override def write(obj: Foo): JsValue = obj.toJson
}
This allows you to parse any arbitrary payload and pull out the fields "name" and "id" - other fields are ignored. If those fields are not guaranteed you can add something like:
case Seq(JsString(name), JsNull) =>
Foo(name, 0)
You should look at what's available in JsValue.scala - in particular JsArray may come in handy if you're getting payloads with anonymous arrays (i.e. the root is [{...}] instead of {"field":"value"...})
Spray Json doesn't support default parameters. So You cannot have a case class like
case class MyClass(a: String, b: Int = 0)
and then parse json like {"a":"foo"}
However if you make the second parameter as Option. then it works.
import spray.json._
case class MyClass(a: String, b: Option[Int] = None)
object MyProtocol extends DefaultJsonProtocol {
implicit val f = jsonFormat2(MyClass)
}
import MyProtocol.f
val mc1 = MyClass("foo", Some(10))
val strJson = mc1.toJson.toString
val strJson2 = """{"a": "foo"}"""
val mc2 = strJson2.parseJson.convertTo[MyClass]
println(mc2)
I have a lot of different external JSON entities that I want to parse to different internal case classs via json4s (scala). Everything works fine via the extract function from json4s. I have implemented a parse function which takes a type and a json string and parses the string to the type / the case class. To map the correct json string to the correct case class I have implemented a pattern matching function, which looks like this
entityName match {
case "entity1" => JsonParser.parse[Entity1](jsonString)
case "entity2" => JsonParser.parse[Entity2](jsonString)
....
I don't like the repetition here and would like to do this mapping via a map like this:
val mapping = Map(
"entity1" -> Entity1,
"entity2" -> Entity2
....
With this map in place I could implement the JsonParser.parse function only once like this
JsonParser.parse[mapping(entityName)](jsonString)
This is not working, because the map is referencing to the object and not to the class type. I also tried classOf[Entity1], but this also is not working. Is there a way to do this?
Thanks!
The way you want your JsonParser.parse to work is not possible in Scala. Scala is a strongly and statically typed language. It means that the compiler should know the types of the values at the compile time to be able to verify that you accesss only valid fields and methods on them and/or pass them as valid parameters to methods. Assuming your classes are
case class Entity1(value:Int, unique1:Int)
case class Entity2(value:String, unique2:String)
and you write
val parsed = JsonParser.parse[mapping("entity1")](jsonString)
how the compiler could know the type of parsed to know the type of parsed.value or to know that parsed.unique1 is a valid field while parsed.unique2 is not? The best type compiler could assign to such parsed is something very generic like Any. Of course you can downcast that Any to the specific type later but this means you still have to specify that type explicitly in the asInstanceOf which kind of defeats the whole purpose. Still, if somehow returning Any is OK for you, you may try to do something like this:
import org.json4s.jackson.JsonMethods
implicit val formats = org.json4s.DefaultFormats // or whatever Formats you actually need
val typeMap: Map[String, scala.reflect.Manifest[_]] = Map(
"String" -> implicitly[scala.reflect.Manifest[String]],
"Int" -> implicitly[scala.reflect.Manifest[Int]]
)
def parseAny(typeName: String, jsonString: String): Any = {
val jValue = JsonMethods.parse(jsonString)
jValue.extract(formats, typeMap(typeName))
}
and then do something like this:
def testParseByTypeName(typeName: String, jsonString: String): Unit = {
try {
val parsed = parseAny(typeName, jsonString)
println(s"parsed by name $typeName => ${parsed.getClass} - '$parsed'")
} catch {
case e => println(e)
}
}
def test() = {
testParseByTypeName("String", "\"abc\"")
testParseByTypeName("Int", "123")
}
P.S. If your entityName doesn't come from the outside (i.e. you don't analyze data to find out actual type), you don't actually need it at all. It is enough to use type (without a need for match/case) such as:
def parse[T](jsonString: String)(implicit mf: scala.reflect.Manifest[T]): T = {
val jValue = JsonMethods.parse(jsonString)
jValue.extract[T]
}
def testParse[T](prefix: String, jsonString: String)(implicit mf: scala.reflect.Manifest[T]): Unit = {
try {
val parsed = parse[T](jsonString)
println(s"$prefix => ${parsed.getClass} - '$parsed'")
} catch {
case e => println(e)
}
}
def test() = {
testParse[String]("parse String", "\"abc\"")
testParse[Int]("parse Int", "123")
}
Following idea from #SergGr, as a snippet to paste on Ammonite REPL:
{
import $ivy.`org.json4s::json4s-native:3.6.0-M2`
import org.json4s.native.JsonMethods.parse
import org.json4s.DefaultFormats
import org.json4s.JValue
case class Entity1(name : String, value : Int)
case class Entity2(name : String, value : Long)
implicit val formats = DefaultFormats
def extract[T](input : JValue)(implicit m : Manifest[T]) = input.extract[T]
val mapping: Map[String, Manifest[_]] = Map(
"entity1" -> implicitly[Manifest[Entity1]],
"entity2" -> implicitly[Manifest[Entity2]]
)
val input = parse(""" { "name" : "abu", "value" : 1 } """)
extract(input)(mapping("entity1")) //Entity1("abu", 1)
extract(input)(mapping("entity2")) //Entity2("abu", 1L)
}
I would like to design a base trait/class in Scala that can produce the following json:
trait GenericResource {
val singularName: String
val pluralName: String
}
I would inherit this trait in a case class:
case class Product(name: String) extends GenericResource {
override val singularName = "product"
override val pluralName = "products"
}
val car = Product("car")
val jsonString = serialize(car)
the output should look like: {"product":{"name":"car"}}
A Seq[Product] should produce {"products":[{"name":"car"},{"name":"truck"}]} etc...
I'm struggling with the proper abstractions to accomplish this. I am open to solutions using any JSON library (available in Scala).
Here's about the simplest way I can think of to do the singular part generically with circe:
import io.circe.{ Decoder, Encoder, Json }
import io.circe.generic.encoding.DerivedObjectEncoder
trait GenericResource {
val singularName: String
val pluralName: String
}
object GenericResource {
implicit def encodeResource[A <: GenericResource](implicit
derived: DerivedObjectEncoder[A]
): Encoder[A] = Encoder.instance { a =>
Json.obj(a.singularName -> derived(a))
}
}
And then if you have some case class extending GenericResource like this:
case class Product(name: String) extends GenericResource {
val singularName = "product"
val pluralName = "products"
}
You can do this (assuming all the members of the case class are encodeable):
scala> import io.circe.syntax._
import io.circe.syntax._
scala> Product("car").asJson.noSpaces
res0: String = {"product":{"name":"car"}}
No boilerplate, no extra imports, etc.
The Seq case is a little trickier, since circe automatically provides a Seq[A] encoder for any A that has an Encoder, but it doesn't do what you want—it just encodes the items and sticks them in a JSON array. You can write something like this:
implicit def encodeResources[A <: GenericResource](implicit
derived: DerivedObjectEncoder[A]
): Encoder[Seq[A]] = Encoder.instance {
case values # (head +: _) =>
Json.obj(head.pluralName -> Encoder.encodeList(derived)(values.toList))
case Nil => Json.obj()
}
And use it like this:
scala> Seq(Product("car"), Product("truck")).asJson.noSpaces
res1: String = {"products":[{"name":"car"},{"name":"truck"}]}
But you can't just stick it in the companion object and expect everything to work—you have to put it somewhere and import it when you need it (otherwise it has the same priority as the default Seq[A] instances).
Another issue with this encodeResources implementation is that it just returns an empty object if the Seq is empty:
scala> Seq.empty[Product].asJson.noSpaces
res2: String = {}
This is because the plural name is attached to the resource at the instance level, and if you don't have an instance there's no way to get it (short of reflection). You could of course conjure up a fake instance by passing nulls to the constructor or whatever, but that seems out of the scope of this question.
This issue (the resource names being attached to instances) is also going to be trouble if you need to decode this JSON you've encoded. If that is the case, I'd suggest considering a slightly different approach where you have something like a GenericResourceCompanion trait that you mix into the companion object for the specific resource type, and to indicate the names there. If that's not an option, you're probably stuck with reflection or fake instances, or both (but again, probably not in scope for this question).
Here is spray-json example. Here is NullOptions trait.
The problem is when I declare a case class say
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val some: RootJsonFormat[Some] = jsonFormat2(Some)
}
case class Some (
name:String,
age:Int
)
and json do not contains a field for example:
{
"name":"John"
}
I get: java.util.NoSuchElementException: key not found: age
So I have to add an Option and NullOption trait like that:
object MyJsonProtocol extends DefaultJsonProtocol with NullOptions {
implicit val some: RootJsonFormat[Some] = jsonFormat2(Some)
}
case class Some (
name:String,
age:Option[Int]
)
Everything works. But I do not want to have a case classes where all member are Option. Is there a way to configure spray json unmarshalling to just set nulls without additional Option type?
P.S.
I understand that in general Option is better then null check, but in my case it is just monkey code.
Also complete example of marshalling during response processing is here
The only way I can think of is to implement your own Protocol via read/write, which might be cumbersome. Below is a simplified example. Note that I changed the age to be an Integer instead of an Int since Int is an AnyVal, which is not nullable by default. Furthermore, I only consider the age field to be nullable, so you might need to adopt as necessary. Hope it helps.
case class Foo (name:String, age: Integer)
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object FooJsonFormat extends RootJsonFormat[Foo] {
def write(foo: Foo) =
JsObject("name" -> JsString(foo.name),
"age" -> Option(foo.age).map(JsNumber(_)).getOrElse(JsNull))
def read(value: JsValue) = value match {
case JsObject(fields) =>
val ageOpt: Option[Integer] = fields.get("age").map(_.toString().toInt) // implicit conversion from Int to Integer
val age: Integer = ageOpt.orNull[Integer]
Foo(fields.get("name").get.toString(), age)
case _ => deserializationError("Foo expected")
}
}
}
import MyJsonProtocol._
import spray.json._
val json = """{ "name": "Meh" }""".parseJson
println(json.convertTo[Foo]) // prints Foo("Meh",null)
It seems you're out of luck
From the doc you linked:
spray-json will always read missing optional members as well as null optional members as None
You can customize the json writing, but not the reading.
I have a json format that consists of map -> map -> ... -> int for an arbitrary number of maps per key. The keys are always strings and the leaf types are always ints. The depth of the map structure varies per key in the map. For example, key "A" could be type Map[String, Int] and key "B" could be type Map[String, Map[String, Int]]. I know I can parse this format successfully as Map[String, Any], but I'm trying to preserve the types to make these structures easier to merge later in the code.
I can't seem to define my nested structure in such a way as to not throw an error on the json4s extract. I'm not quite sure if the problem is in my structure definition or if I'm not doing the json extract correctly.
Here is the code
sealed trait NestedMap[A]
case class Elem[A](val e : A) extends NestedMap[A]
case class NMap[A](val e : Map[String, NestedMap[A]]) extends NestedMap[A]
// XXX this next line doesn't seem to help or hurt
case class Empty extends NestedMap[Nothing]
implicit val formats = DefaultFormats
val s = "{\"1\": 1, \"2\": 1, \"3\": {\"4\": 1}}"
val b = parse(s).extract[NMap[Int]]
Here is the error that always comes up
org.json4s.package$MappingException: No usable value for e
Expected object but got JNothing
Do I need to add another value that extends NestedMap? Am I going about this entirely wrong? Any help is much appreciated.
By default, a tree like yours is expanded to to a different json.
import org.json4s._
import org.json4s.native.Serialization
import org.json4s.native.Serialization.{read, write}
import org.json4s.native.JsonMethods._
implicit val formats = Serialization.formats(NoTypeHints)
sealed trait Elem
case class Leaf(val e:Int) extends Elem
case class Tree(val e:Map[String, Elem]) extends Elem
scala> val t = Tree(Map("1"->Leaf(1),"2"->Leaf(2),"3"->Tree(Map("4"->Leaf(4)))))
t: Tree = Tree(Map(1 -> Leaf(1), 2 -> Leaf(2), 3 -> Tree(Map(4 -> Leaf(4)))))
scala> write(t)
res0: String = {"e":{"1":{"e":1},"2":{"e":2},"3":{"e":{"4":{"e":4}}}}}
scala> val jt = parse(res0)
jt: org.json4s.JValue = JObject(List((e,JObject(List((1,JObject(List((e,JInt(1))))), (2,JObject(List((e,JInt(2))))), (3,JObject(List((e,JObject(List((4,JObject(List((e,JInt(4))))))))))))))))
scala> val s = """{"1":1,"2":2, "3":{"4":1}}"""
s: String = {"1":1,"2":2, "3":{"4":1}}
scala> val st = parse(s)
st: org.json4s.JValue = JObject(List((1,JInt(1)), (2,JInt(2)) (3,JObject(List((4,JInt(1)))))))