Which JSON serialization library would fit to the following case? - json

I've got following case:
I'd like to serialize Scala case classes that extend parent class with var of type java.util.UUID.
Serialization of this case classes should happen without any configuration of them - no annotations and definition of custom formats. Any serialization hints may be situated in parent class.
I tried sjson, but Reflection based serialization can't serialize UUID types and type based serialization forces me to define formats for every case class.
Which json serialization library would best fit this case?

Here's one solution with Lift JSON.
import java.util.UUID
import net.liftweb.json._
import net.liftweb.json.JsonAST._
import net.liftweb.json.JsonDSL._
import net.liftweb.json.Serialization._
sealed abstract class Parent {
def uuid: UUID
}
case class Foo(uuid: UUID, name: String) extends Parent
object UUIDTest extends Application {
implicit val formats = Serialization.formats(NoTypeHints) + new UUIDSerializer
val f = Foo(UUID.randomUUID, "foo")
val ser = write(f)
println(ser)
val f2 = read[Foo](ser)
assert(f == f2)
// Special serializer for UUID type
class UUIDSerializer extends Serializer[UUID] {
private val Class = classOf[UUID]
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), UUID] = {
case (TypeInfo(Class, _), json) => json match {
case JObject(JField("mostSig", JInt(m)) :: JField("leastSig", JInt(l)) :: Nil) =>
new UUID(m.longValue, l.longValue)
case x => throw new MappingException("Can't convert " + x + " to UUID")
}
}
def serialize(implicit format: Formats): PartialFunction[Any, JValue] = {
case x: UUID =>
("mostSig" -> x.getMostSignificantBits) ~ ("leastSig" -> x.getLeastSignificantBits)
}
}
}
It prints:
{"uuid":{"mostSig":-8054689529719995935,"leastSig":-5722404370736228056},"name":"foo"}'
Another solution which uses a custom serializer for Parent type.
sealed abstract class Parent {
var uuid: UUID = UUID.randomUUID
}
case class Foo(name: String) extends Parent
object UUIDTest extends Application {
implicit val formats =
Serialization.formats(NoTypeHints) + new ParentSerializer
val f = Foo("foo")
val ser = write(f)
println(ser)
val f2 = read[Foo](ser)
assert(f == f2)
// Special serializer for Parent type
class ParentSerializer extends Serializer[Parent] {
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Parent] = {
case (t#TypeInfo(cl, _), json) if (classOf[Parent].isAssignableFrom(cl)) =>
val x = Extraction.extract(json, t)(DefaultFormats).asInstanceOf[Parent]
x.uuid = (for {
JField("mostSig", JInt(m)) <- json
JField("leastSig", JInt(l)) <- json
} yield new UUID(m.longValue, l.longValue)).head
x
}
def serialize(implicit format: Formats): PartialFunction[Any, JValue] = {
case x: Parent =>
Extraction.decompose(x)(DefaultFormats) ++
JField("mostSig", x.uuid.getMostSignificantBits) ++
JField("leastSig", x.uuid.getLeastSignificantBits)
}
}
}

If the type is important, you should take a look at YAML.
http://www.google.fr/search?q=java+yaml
It's a subset of json with improved stuff, like variable typing.

You could try jerkson: https://github.com/codahale/jerkson
Its working good for my use, but that is mostly list/map structures. Would not be surprised if it supports your needs though..
Edit: Tried it with the following example (inspired by the lift example in another answer). Seems to work fine.
import java.util.UUID
import com.codahale.jerkson.Json
import org.scalatest.FunSuite
sealed abstract class Parent {
def uuid: UUID
}
case class Foo(uuid: UUID, name: String) extends Parent
class TmpJsonTest extends FunSuite {
test("Json case class serialize") {
val f = Foo(UUID.randomUUID, "foo")
val ser = Json.generate(f)
println(ser)
val f2 = Json.parse[Foo](ser)
assert(f === f2)
}
}

Try the XStream library which includes JSON support. I have used this successfully in a few projects. It has a number of default converters, including one for java.util.UUID. A full list of default converters is located here: http://x-stream.github.io/converters.html.
A brief tutorial on using XStream for JSON reading and writing is located here: http://x-stream.github.io/json-tutorial.html. The tutorial code is written for Java but it should work just the same for Scala since reflection is being used behind the scenes.
Keep in mind that serializing and then deserializing arbitrary graphs of objects is not always possible with this library. In particular, loops in your data cannot be handled, i.e. your data must be a purely hierarchical tree. This is a reasonable limitation given the intentions of the JSON format.
Referenced links:
XStream JSON tutorial: http://x-stream.github.io/json-tutorial.html
XStream default converters: http://x-stream.github.io/converters.html

Related

Configure spray-json for non strict parsing deserialization

How to configure the spray-json parsing on parsing options?
Similarly as Jackson Parsing Features.
For example, I am parsing a json that has a field that my case class has not, and it is breaking:
spray.json.DeserializationException: Object is missing required member 'myfield'
UPDATE :
A simple example:
case class MyClass(a: String, b: Long);
and try to parse an incomplete json like
val data = "{a: \"hi\"}"
with a spray-json format like:
jsonFormat2(MyClass.apply)
// ...
data.parseJson.convertTo[MyClass]
(simplified code).
But the question goes further, I want to ask about configuration options like in other parsers. More examples:
Be able to ignore fields that exist in the JSON but not in the case class.
Ways of managing nulls or nonexistent values.
etc.
SprayJson allows you to define custom parsers like so:
case class Foo(a: String, b: Int)
implicit object FooJsonFormat extends RootJsonFormat[Foo] {
override def read(json: JsValue): Foo = {
json.asJsObject.getFields("name", "id") match {
case Seq(JsString(name), id) =>
Foo(name, id.convertTo[Int])
}
}
override def write(obj: Foo): JsValue = obj.toJson
}
This allows you to parse any arbitrary payload and pull out the fields "name" and "id" - other fields are ignored. If those fields are not guaranteed you can add something like:
case Seq(JsString(name), JsNull) =>
Foo(name, 0)
You should look at what's available in JsValue.scala - in particular JsArray may come in handy if you're getting payloads with anonymous arrays (i.e. the root is [{...}] instead of {"field":"value"...})
Spray Json doesn't support default parameters. So You cannot have a case class like
case class MyClass(a: String, b: Int = 0)
and then parse json like {"a":"foo"}
However if you make the second parameter as Option. then it works.
import spray.json._
case class MyClass(a: String, b: Option[Int] = None)
object MyProtocol extends DefaultJsonProtocol {
implicit val f = jsonFormat2(MyClass)
}
import MyProtocol.f
val mc1 = MyClass("foo", Some(10))
val strJson = mc1.toJson.toString
val strJson2 = """{"a": "foo"}"""
val mc2 = strJson2.parseJson.convertTo[MyClass]
println(mc2)

How can I define a dynamic base json object?

I would like to design a base trait/class in Scala that can produce the following json:
trait GenericResource {
val singularName: String
val pluralName: String
}
I would inherit this trait in a case class:
case class Product(name: String) extends GenericResource {
override val singularName = "product"
override val pluralName = "products"
}
val car = Product("car")
val jsonString = serialize(car)
the output should look like: {"product":{"name":"car"}}
A Seq[Product] should produce {"products":[{"name":"car"},{"name":"truck"}]} etc...
I'm struggling with the proper abstractions to accomplish this. I am open to solutions using any JSON library (available in Scala).
Here's about the simplest way I can think of to do the singular part generically with circe:
import io.circe.{ Decoder, Encoder, Json }
import io.circe.generic.encoding.DerivedObjectEncoder
trait GenericResource {
val singularName: String
val pluralName: String
}
object GenericResource {
implicit def encodeResource[A <: GenericResource](implicit
derived: DerivedObjectEncoder[A]
): Encoder[A] = Encoder.instance { a =>
Json.obj(a.singularName -> derived(a))
}
}
And then if you have some case class extending GenericResource like this:
case class Product(name: String) extends GenericResource {
val singularName = "product"
val pluralName = "products"
}
You can do this (assuming all the members of the case class are encodeable):
scala> import io.circe.syntax._
import io.circe.syntax._
scala> Product("car").asJson.noSpaces
res0: String = {"product":{"name":"car"}}
No boilerplate, no extra imports, etc.
The Seq case is a little trickier, since circe automatically provides a Seq[A] encoder for any A that has an Encoder, but it doesn't do what you want—it just encodes the items and sticks them in a JSON array. You can write something like this:
implicit def encodeResources[A <: GenericResource](implicit
derived: DerivedObjectEncoder[A]
): Encoder[Seq[A]] = Encoder.instance {
case values # (head +: _) =>
Json.obj(head.pluralName -> Encoder.encodeList(derived)(values.toList))
case Nil => Json.obj()
}
And use it like this:
scala> Seq(Product("car"), Product("truck")).asJson.noSpaces
res1: String = {"products":[{"name":"car"},{"name":"truck"}]}
But you can't just stick it in the companion object and expect everything to work—you have to put it somewhere and import it when you need it (otherwise it has the same priority as the default Seq[A] instances).
Another issue with this encodeResources implementation is that it just returns an empty object if the Seq is empty:
scala> Seq.empty[Product].asJson.noSpaces
res2: String = {}
This is because the plural name is attached to the resource at the instance level, and if you don't have an instance there's no way to get it (short of reflection). You could of course conjure up a fake instance by passing nulls to the constructor or whatever, but that seems out of the scope of this question.
This issue (the resource names being attached to instances) is also going to be trouble if you need to decode this JSON you've encoded. If that is the case, I'd suggest considering a slightly different approach where you have something like a GenericResourceCompanion trait that you mix into the companion object for the specific resource type, and to indicate the names there. If that's not an option, you're probably stuck with reflection or fake instances, or both (but again, probably not in scope for this question).

Serialize extended class to JSON in Scala with Play Framework serializer

I would like to serialize an extended class in scala and
i have some test code..
import org.specs2.mutable._
import org.specs2.runner._
import org.junit.runner._
import play.api.libs.json.Json
#RunWith(classOf[JUnitRunner])
class JsonSerializerTest extends Specification {
class A(val s1: String)
case class B(s2: String) extends A("a")
"Application" should {
"serialize class to JSON" in {
implicit val bWrites = Json.writes[B]
implicit val bReads = Json.reads[B]
val bClass = B("b")
println(bClass.s1 + " " + bClass.s2)
val serialized = Json.toJson[B](bClass)
val s1 = (serialized \ "s1").asOpt[String]
s1 should beSome[String]
}
}
}
In this case test print:
a b
Application should
'None' is not Some
java.lang.Exception: 'None' is not Some
It means that s1 field from parent class were not serialized.
The solution
class A(val s1: String)
case class B(override val s1: String, s2: String) extends A(s1)
mostly unacceptable because in real application classes have a lot of fields and specifying them explicitly every time when i extend class complicates the code.
Is there any other solution for this case?
you can manually create the json serializers (described here: https://www.playframework.com/documentation/2.5.x/ScalaJsonCombinators)
the problem with your version is that Json.writes and Json.reads are macros that are looking specifically at the constructor of the case class, and building the serializer from that (so superclass arguments aren't captured). you could copy and roll your own version of the macro: https://github.com/playframework/playframework/blob/d6c2673d91d85fd37de424951ee5ad9f4f4cce98/framework/src/play-json/src/main/scala/play/api/libs/json/JsMacroImpl.scala
lastly, you can make a function that takes the result of Json.writes and Json.reads, and adds on the shared fields you want. something like:
object A {
def writesSubclass[T](writer: Writes[T]): Writes[T] = new Writes[T] {
def writes(t: T) = Json.obj("s1" -> t.s1) ++ writer.writes(t).as[JsObject]
}
}
implicit val bWrites = A.writesSubclass(Json.writes[B])
depending on how often your A gets extended, this might be your best bet.

Spray-json convert part of the json to Map[String,List[Double]]

I have the following json
{
"op":{
"samples":{
"ep_mem_high_wat":[ 0,0,0,0,0,0,0],
"ep_mem_low_wat":[ 0,0,0,0,0,0,0]
},
"samplesCount":60,
"isPersistent":true,
"lastTStamp":1415619627689,
"interval":1000
},
"hot_keys":[
{
"name":"counter::F03E91E2A4B9C25F",
"ops":0.11010372549516878
}
]
}
I would like to parse "samples" property of this Json as Map[String,List[Double]] as
Map[String,List[Double]]("ep_mem_high_wat" -> [ 0,0,0,0,0,0,0],"ep_mem_low_wat" -> [0,0,0,0,0,0,0])
For this purpose I perform the following: I create my case classes
case class Samples(properties:Map[String,List[Double]])
case class Op(samples:Samples,samplesCount:Int,isPersistent:Boolean,lastTStamp:Long,interval:Int)
case class Key(name:String,ops:Double)
case class BucketStatisticResponse(op:Op,hot_keys:List[Key])
and then I create custom Json protocol
object BucketStatisticJsonProtocol extends DefaultJsonProtocol {
implicit object SamplesJsonFormat extends RootJsonFormat[Samples] {
def write(obj: Samples) = obj.toJson
def read(value: JsValue) = {
val props = value.asJsObject.fields
Samples(value.convertTo[Map[String,List[Double]]])
}
}
implicit val keyFormat = jsonFormat2(Key)
implicit val opFormat = jsonFormat5(Op)
implicit val bucketStatisticFormat= jsonFormat2(BucketStatisticResponse)
}
And then I'm trying to parse json
import BucketStatisticJsonProtocol._
val res = json.toJson.convertTo[BucketStatisticResponse]
As the result I'm gettign the folloing exception:
Exception in thread "main" spray.json.DeserializationException: Object expected in field 'op'
at spray.json.package$.deserializationError(package.scala:23)
at spray.json.ProductFormats$class.fromField(ProductFormats.scala:54)
at high.availability.poc.DynamicJson$BucketStatisticJsonProtocol$.fromField(DynamicJson.scala:23)
What am I doing wrong?
Look at the error message: It's complaining about the type of value it's finding in the AST for the op field. So the document you're actually parsing probably doesn't match your sample above.
Other than that, your code looks OK to me. Just as a matter of taste, I typically don't extend DefaultJsonProtocol; I put my JSON formats in the package object, and import DefaultJsonProtocol._ to handle any of the simplistic conversions as necessary.

Json Serialization for Trait with Multiple Case Classes (Sum Types) in Scala's Play

I often have to serialize/deserialize sum types (like Either[S,T]), and I haven't yet found a general or elegant way to do it. Here's an example type (essentially equivalent to Either)
sealed trait OutcomeType
case class NumericOutcome(units: String) extends OutcomeType
case class QualitativeOutcome(outcomes: List[String]) extends OutcomeType
Here's my best effort at a companion object that implements serialization. It works, but it's very tiresome to write these sorts of things over and over for every sum type. Are there any suggestions for making it nicer and/or more general?
import play.api.libs.json._
import play.api.libs.functional.syntax._
object OutcomeType {
val fmtNumeric = Json.format[NumericOutcome]
val fmtQualitative = Json.format[QualitativeOutcome]
implicit object FormatOutcomeType extends Format[OutcomeType] {
def writes(o: OutcomeType) = o match {
case n#NumericOutcome(_) => Json.obj("NumericOutcome" -> Json.toJson(n)(fmtNumeric))
case q#QualitativeOutcome(_) => Json.obj("QualitativeOutcome" -> Json.toJson(q)(fmtQualitative))
}
def reads(json: JsValue) = (
Json.fromJson(json \ "NumericOutcome")(fmtNumeric) orElse
Json.fromJson(json \ "QualitativeOutcome")(fmtQualitative)
)
}
}
I think that is about as simple as you can make it, if you want to avoid writing the code for each explicit subtype maybe you could do it with reflection, use jackson directly or some other json library with reflection support. Or write your own macro to generate the Format from a list of subtypes.
I have a systematic solution for the problem of serializing sum-types in my json pickling library Prickle. Similar ideas could be employed with Play. There is still some config code required, but its high signal/noise, eg final code like:
implicit val fruitPickler = CompositePickler[Fruit].concreteType[Apple].concreteType[Lemon]
CompositePicklers associated with a supertype are configured with one PicklerPair for each known subtype (ie sum type option). The associations are setup at configure time.
During pickling a descriptor is emitted into the json stream describing which subtype the record is.
During unpickling, the descriptor is read out of the json and then used to locate the appropriate Unpickler for the subtype
An example updated for play 2.5:
object TestContact extends App {
sealed trait Shape
object Shape {
val rectFormat = Json.format[Rect]
val circleFormat = Json.format[Circle]
implicit object ShapeFormat extends Format[Shape] {
override def writes(shape: Shape): JsValue = shape match {
case rect: Rect =>
Json.obj("Shape" ->
Json.obj("Rect" ->
Json.toJson(rect)(rectFormat)))
case circle: Circle =>
Json.obj("Shape" ->
Json.obj("Circle" ->
Json.toJson(circle)(circleFormat)))
}
override def reads(json: JsValue): JsResult[Shape] = {
json \ "Shape" \ "Rect" match {
case JsDefined(rectJson) => rectJson.validate[Rect](rectFormat)
case _ => json \ "Shape" \ "Circle" match {
case JsDefined(circleJson) => circleJson.validate[Circle](circleFormat)
case _ => JsError("Not a valide Shape object.")
}
}
}
}
}
case class Rect(width: Double, height: Double) extends Shape
case class Circle(radius: Double) extends Shape
val circle = Circle(2.1)
println(Json.toJson(circle))
val rect = Rect(1.3, 8.9)
println(Json.toJson(rect))
var json = Json.obj("Shape" -> Json.obj("Circle" -> Json.obj("radius" -> 4.13)))
println(json.validate[Shape])
json =
Json.obj("Shape" ->
Json.obj("Rect" ->
Json.obj("width" -> 23.1, "height" -> 34.7)))
println(json.validate[Shape])
}