I have HTTP client written in Scala that uses json4s/jackson to serialize and deserialize HTTP payloads. For now I was using only Scala case classes as model and everything was working fine, but now I have to communicate with third party service. They provided me with their own model but its written in Java, so now I need to deserialize jsons also to Java classes. It seams to work fine with simple classes but when class contains collections like Lists or Maps json4s has problems and sets all such fields to null.
Is there any way to handle such cases? Maybe I should use different formats (I'm using DefaultFormats + few custom ones). Example of problem with test:
import org.json4s.DefaultFormats
import org.json4s.jackson.Serialization.read
import org.scalatest.{FlatSpec, Matchers}
class JavaListTest extends FlatSpec with Matchers{
implicit val formats = DefaultFormats
"Java List" should "be deserialized properly" in {
val input = """{"list":["a", "b", "c"]}"""
val output = read[ObjectWithList](input)
output.list.size() shouldBe 3
}
}
And sample Java class:
import java.util.List;
public class ObjectWithList {
List<String> list;
}
I have also noticed that when I'll try to deserialize to Scala case class that contains java.util.List[String] type of field I'll get an exception of type: org.json4s.package$MappingException: Expected collection but got List[String]
Key for solving your issue, is composition of formatters. Basically you want to define JList formatter as list formatter composed with toJList function.
Unfortunately, json4s Formatters are extremely difficult to compose, so I used the Readers for you to get an idea. I also simplified an example, to having only java list:
import DefaultReaders._
import scala.collection.JavaConverters._
implicit def javaListReader[A: Reader]: Reader[java.util.List[A]] = new Reader[util.List[A]] {
override def read(value: JValue) = DefaultReaders.traversableReader[List, A].read(value).asJava
}
val input = """["a", "b", "c"]"""
val output = Formats.read[java.util.List[String]](parse(input))
To my knowledge json4s readers will not work with java classes out of the box, so you might either need to implement the Serializer[JList[_]] the same way, or mirror your java classes with case classes and use them inside your domain.
P.S.
Highly recommend you to switch to circe or argonaut, then you will forget about the most problems with jsons.
Related
I have a problem when trying to serialize sequences of AnyVal using json4s in scala.
Here is a test using FunSuite that reproduces the problem:
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.jackson.Serialization._
import org.scalatest.{FunSuite, Matchers}
case class MyId(id: String) extends AnyVal
case class MyModel(ids: Seq[MyId])
class AnyValTest extends FunSuite with Matchers {
test("should serialize correctly") {
implicit val formats = DefaultFormats
val model = MyModel(Seq(MyId("1"), MyId("2")))
val text = write(model)
parse(text).extract[MyModel] shouldBe model
}
}
The test fails when trying to extract MyModel from the JValue because it can not find a suitable value for the ids field.
I notice that it AnyVal are working fine when used directly though with something like:
case class AnotherModel(id: MyId)
Then I am able to serialise and deserialise correctly.
I know this question is one year old but I ran into the same issue. Writing what I did in case it helps someone else. You will need a custom serializer.
case class Id(asString: String) extends AnyVal
class NotificationSerializer extends CustomSerializer[Id](format ⇒ (
{case JString(s) => Id(s)},
{case Id(s) => JString(s)}))
Without above serialization, your JSON will look something like
{"ids":[[{"asString":"testId1"},{"asString":"testId2"}]]}
I am not entirely sure why AnyVal case class serialization works fine when it is a part of another case class but not standalone. My best guess is that the behavior is due to the allocation behavior of JVM for array containing value classes. See http://docs.scala-lang.org/overviews/core/value-classes.html for 'when allocation is necessary' section.
I have my relevant actors' messages (de)serializable to/from Play! JSON. I'd like to use JSON (de)serializers for akka persistance system (if possbile).
In akka persistance documentation there is possibility to use our own serializers. Further more here are the instructions how to write custom serializers. Since akka.serialization.Serializer is expecting toBinary and fromBinary is there any way to use Play JSON serializers with akka persistence?
Thank you!
Best!
Where do you like to serialize data to?
I'm looking for a mongodb based akka persistence store with serialized objects using the json formats on my own. Maybe the following driver might be interesting for you as well:
https://github.com/scullxbones/akka-persistence-mongo/issues/16
Integrating play json into akka-persistence is complicated since play json uses instances of Format that are gathered via implicits. Akka provides just java.lang.Object for serialization and a java.lang.Class[_] for deserialization which makes resolving the correct implicit Format impossible.
What you could do is write a custom akka.serialization.Serializer that has a Map from Class[A] to Format[A]. This map can be used to find the correct format for a java.lang.Object / java.lang.Class[_]:
class JsonSerializer(serializers: Map[Class[_], Format[_]]) extends Serializer {
val charset: Charset = StandardCharsets.UTF_8
val identifier: Int = "play-json-serializer".##
val includeManifest: Boolean = true
def serializer[A](c: Class[_]): GenericFormat[A] = serializers.get(c) match {
case Some(format) => format.asInstanceOf[GenericFormat[A]]
case None => throw new RuntimeException("No Format available for " + c.getName)
}
def toBinary(o: AnyRef): Array[Byte] = jsonSerialize(o).getBytes(charset)
def fromBinary(bytes: Array[Byte], manifest: Option[Class[_]]): AnyRef = jsonDeserialize(bytes, manifest.get)
def jsonSerialize[A](a: A): String = {
implicit val format: GenericFormat[A] = serializer[A](a.getClass)
Json.stringify(Json.toJson(a))
}
def jsonDeserialize[A](bytes: Array[Byte], manifest: Class[_]): A = {
implicit val format: GenericFormat[A] = serializer[A](manifest)
Json.fromJson[A](Json.parse(new String(bytes, charset))).get
}
}
You can now inherit this class and pass play formats for all types that your akka-serializer should be able to (de)serialize to the constructor. This serializer must be configured in the akka configuration as described in the documentation:
class MyJsonSerializer extends JsonSerializer(Map(
Serializer[Foo], Serializer[...], ...
))
// Just a utility class for the pretty syntax above
object Serializer {
def apply[A](implicit format: Format[A], ctag: ClassTag[A]): (Class[A], Format[A]) =
(ctag.runtimeClass.asInstanceOf[Class[A]], format)
}
Is there a way and/or library to automatically create Kotlin Data class from Json like it is works in Scala Json.Spray?
Something like this:
data class User(id: Int, name: String)
class DataClassFactory(val json: String) {
fun getUser(): User {
//some reflection
return User(10, "Kirill")
}
}
fun main(args: Array<String>): Unit {
val json = "{id: 10, name: Kirill}"
val usr = DataClassFactory(json).getUser()
println(usr)
}
You can use the Jackson module for Kotlin to serialize/deserialize easily from any format that Jackson supports (including JSON). This is the easiest way, and supports Kotlin data classes without annotations. See https://github.com/FasterXML/jackson-module-kotlin for the module which includes the latest information for using from Maven and Gradle (you can infer IVY and download JARs from the Maven repositories as well)
Alternatives exist such as Boon, but it has no specific support for Kotlin (usually a problem with not having a default constructor) and uses some unsafe direct access to internal JVM classes for performance. In doing so, it can crash on some VM's, and in cases where you extend Boon from Kotlin with custom serializer/deserializer it makes assumptions about the classes that do not hold true in Kotlin (the String class is wrapped for example) which I have seen core dump. Boon is lightening fast, just be careful of these issues and test first before using.
(note: I am the creator of the Jackson-Kotlin module)
This is very clean and easy in Kotlin.
import com.fasterxml.jackson.module.kotlin.*
data class User(val id: Int, val name: String)
fun main(args: Array<String>) {
val mapper = jacksonObjectMapper()
val json = """{"id": 10, "name": "Kirill"}"""
val user = mapper.readValue<User>(json)
println(user)
}
produces this output:
User(id=10, name=Kirill)
you only have to add this to your pom.xml
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-kotlin</artifactId>
<version>2.6.3-4</version>
</dependency>
Why not use Jackson or any other serializer? It should work..
What about this
This is a translator that translate JSON string into kotlin data class ,it make it throught a plugin,see next
https://plugins.jetbrains.com/plugin/9960-jsontokotlinclass
http://www.json2kotlin.com converts your json response to kotlin data classes online, without the need to install any plugin. Optionally, you can generate gson annotations too. (Disclosure: I created this utility)
[
{"fname":"Foo","lname":"Pacman"},
{"fname":"Bar","lname":"Mario"},
{"fname":"Poo","lname":"Wario"}
]
Well I have JSON string in this format,
Now what I need is to convert each tuples -> {"fname":"Foo","lname":"Pacman"}
To a Person object,
for e.g. lets assume I have a case class
case class Person(fname:String,lname:String)
Now how am I to get, List<person>
If I had a JSON containing data for single tuple, then I could,
val o:Person = parse[Person](jsonString)// I am actually using Jerkson Lib
But since there are more than one tuples, how am i to parse them individually and create objects and create a list.
Jerkson supports deserializing lists of objects out of the box, so all you should need to do is:
val people = parse[List[Person]](personJson)
You can use json4s (which is a wrapper around either jackson or lift-json) where you also get such parsing capabilities out of the box.
import org.json4s._
import org.json4s.jackson.JsonMethods._
implicit val formats = DefaultFormats
val personJson = """
[
{"fname":"Foo","lname":"Pacman"},
{"fname":"Bar","lname":"Mario"},
{"fname":"Poo","lname":"Wario"}
]"""
case class Person(fname:String,lname:String)
val people = parse(personJson).extract[List[Person]]
I'm attempting basic serialization/hydration with lift-json, but without success. As near as I can tell from the package readme, this should work. Help?
I'm using Scala 2.8.0 and Lift 2.2 cross-built for 2.8 with sbt ("net.liftweb" %% "lift-json" % "2.2").
import net.liftweb.json._
import net.liftweb.json.Serialization.{read, write}
implicit val formats = Serialization.formats(NoTypeHints)
case class Route(title: String)
val rt = new Route("x277a1")
val ser = write(rt)
// ser: String = {} ...
val deser = read[Route]("""{"title":"Some Title"}""")
// net.liftweb.json.MappingException: Parsed JSON values do not match with class constructor
Lift JSON's serialization does not work for case classes defined in REPL (paranamer can't find the bytecode to read the type metadata). Compile Route with scalac and then the above example works.
The same problem applies every time when the (de)serialuzed class is not on the classpath. In such case, paranamer can't read the parameter names. It is necessary to provide a custom ParameterNameReader.
Such problem applies for e.g.:
REPL (as mentioned) - unless you define the class outside the REPL and add via classpath.
Play Framework - unless you provide a simple custom ParameterNameReader (see below) or load the (de)serialized class as a Maven/Play/... dependency
Feel free to add another situation (you can edit this post).
The PlayParameterNameReader:
import net.liftweb.json.ParameterNameReader
import java.lang.reflect.Constructor
import play.classloading.enhancers.LocalvariablesNamesEnhancer
import scala.collection.JavaConversions._
object PlayParameterReader extends ParameterNameReader{
def lookupParameterNames(constructor: Constructor[_]) = LocalvariablesNamesEnhancer.lookupParameterNames(constructor)
}