Serializing Java Path with kotlinx.serialization - json

With kotlinx.serialization, this code will throw an error:
println(Json.encodeToString(Path.of("value")))
saying kotlinx.serialization.SerializationException: Class 'WindowsPath' is not registered for polymorphic serialization in the scope of 'Path'.
WindowsPath is internal, therefore I can not register it as a polymorphic subclass (as in this example), only with Path itself, and even a custom KSerializer for Path throws the same exact error.
Is there any way to have Path properly serialize/deserialize without having to store it as a string?

Path is an interface, so it's implicitly serializable with the PolymorphicSerializer strategy. This strategy requires you to register serializers for subclasses implementing it, but as you know, in this case it's impossible.
There is a default polymorphic serializer, but it affects only deserialization process and it works only when deserializable value is a JSONObject.
And for the following serializer
object PathAsStringSerializer : KSerializer<Path> {
override val descriptor = PrimitiveSerialDescriptor("Path", PrimitiveKind.STRING)
override fun serialize(encoder: Encoder, value: Path) = encoder.encodeString(value.toAbsolutePath().toString())
override fun deserialize(decoder: Decoder): Path = Path.of(decoder.decodeString())
}
\\Not working
val module = SerializersModule { polymorphicDefault(Path::class) { PathAsStringSerializer } }
val decoded : Path = Json { serializersModule = module }.decodeFromString("C:\\Temp")
it will throw runtime exception kotlinx.serialization.json.internal.JsonDecodingException: Expected class kotlinx.serialization.json.JsonObject as the serialized body of kotlinx.serialization.Polymorphic<Path>, but had class kotlinx.serialization.json.JsonLiteral
So, it can't be serialized in a common way, and there are 3 cases of its serialization/deserialization, which need to be handled:
1. Serialization of simple Path variable
In this case you need to explicitly pass your custom serializer:
val path = Path.of("C:\\Temp")
val message1 = Json.encodeToString(PathAsStringSerializer, path).also { println(it) }
println(Json.decodeFromString(PathAsStringSerializer, message1))
2. Serialization of classes, which use Path as a generic parameter
In this case you need to define separate serializers (you may reference original PathAsStringSerializer) and also explicitly pass them:
object ListOfPathsAsStringSerializer : KSerializer<List<Path>> by ListSerializer(PathAsStringSerializer)
val message2 = Json.encodeToString(ListOfPathsAsStringSerializer, listOf(path)).also { println(it) }
println(Json.decodeFromString(ListOfPathsAsStringSerializer, message2))
#Serializable
data class Box<T>(val item: T)
object BoxOfPathSerializer : KSerializer<Box<Path>> by Box.serializer(PathAsStringSerializer)
val message3 = Json.encodeToString(BoxOfPathSerializer, Box(path)).also { println(it) }
println(Json.decodeFromString(BoxOfPathSerializer, message3))
3. Serialization of classes, which have fields of aforementioned types
In this case you need to add respectful #Serializable(with = ...) annotations for these fields:
#Serializable
data class InnerObject(
#Serializable(with = ListOfPathsAsStringSerializer::class)
val list: MutableList<Path> = mutableListOf(),
#Serializable(with = PathAsStringSerializer::class)
val path: Path,
#Serializable(with = BoxOfPathSerializer::class)
val box: Box<Path>
)
Or just list them once for a whole file:
#file: UseSerializers(PathAsStringSerializer::class, ListOfPathsAsStringSerializer::class, BoxOfPathSerializer::class)
Plugin-generated serializer for this case would be good enough:
val message4 = Json.encodeToString(InnerObject(mutableListOf(path), path, Box(path))).also { println(it) }
println(Json.decodeFromString<InnerObject>(message4))

Related

How to serialize a Kotlin Set to a JSON object whose values are empty objects?

I would like to make a Kotlin class serializable using Kotlin serialization.
The class is very simple, something like this:
#Serializable(with = CustomSerializer::class)
data class MyObject(val keys: Set<String>)
Now, I need the serialization format to be a JSON object where the keys are given by the Set<String> and the values are always empty JSON objects.
Example:
val example = MyObject(setOf("abc", "def"))
Should serialize to:
{ "abc": {}, "def": {} }
The reason is that this object is being sent to an API where that's how they want the JSON to look like... the empty objects could contain some directives but I don't want or need to use those.
Having trouble doing that by just reading the documentation.
I've found one way to do it... and it seems simple enough!
I realized that I can get a serializer of empty Objects almost for free with this:
#Serializable
private object EmptyMap
Now, I can write a custom serializer in a straightforward way:
object MyObjectSerializer : KSerializer<MyObject> {
private val _delegate = MapSerializer(String.serializer(), EmptyMap.serializer())
override val descriptor: SerialDescriptor = _delegate.descriptor
override fun serialize(encoder: Encoder, value: MyObject) {
val data = value.keys.associateWith { EmptyMap }
encoder.encodeSerializableValue(_delegate, data)
}
override fun deserialize(decoder: Decoder): MyObject {
val value = decoder.decodeSerializableValue(_delegate)
return MyObject(value.keys)
}
}
Now all that's left to do is to apply the serializer on the type, which can be done with:
#Serializer(with = MyObjectSerializer)
data class MyObject(val keys: Set<String>)
Running Json.encodeToString(example) on the examples works perfectly.

Using a KClass reference as a reified parameter to deserialize from JSON

I'm trying to implement a general serialization framework to convert outgoing and incoming messages to json using the kotlinx serialialization. I'm developing a multiplatform app, so I'm trying to get it to run on KotlinJVM and KotlinJS.
For this, I add a type field to every message and use a map that maps each type string to a KClass. What's the type for that map? It contains KClass<> objects whose classes extend the Message class, therefore in java I'd specify my map as
Map<KClass<? extends Message>, String>.
How can I do that in Kotlin?
Afterwards I need to serialize and deserialize the message based on its key and therefore type. Java frameworks take a Class parameter for the type of the object I want to deserialize/instantiate (e.g. gson.fromJson(ClientMessage.class)). In Kotlin this is done using reified parameters Json.decodeFromString<Type>. I do not know the type of the message at compile time though and just have a reference to a KClass, how can I instantiate an object based on that?
#Serializable
open class Message(val type: String) {
companion object {
val messageTypes: Map<KClass<out Message>, String> = mapOf(
ClientLoginMessage::class to "clientLoginMessage",
Message::class to "message"
)
inline fun <reified T> getMessageTypeByClass(): String = messageTypes[T::class]!! // utility for defining the type in the constructors of the individual messages
}
fun toJson() = Json.encodeToString(this)
fun fromJson(json: String): Message? {
val plainMessage = Json.decodeFromString<Message>(json) // get type string from json
return messageTypes.entries.find { it.value == plainMessage.type }?.let {
// how can I use the KClass from it.key as reified parameter?
Json.decodeFromString<?????>(json)
}
}
}
#Serializable
class ClientLoginMessage
: Message(Message.getMessageTypeByClass<ClientLoginMessage>()) {}
Create a map of serializers like for types:
val serializers: Map<KClass<out Message>, KSerializer<out Message>> = mapOf(
ClientLoginMessage::class to ClientLoginMessage.serializer(),
Message::class to Message.serializer()
)
Pass in the serializer needed to Json.decodeFromString like this:
fun fromJson(json: String): Message? {
val plainMessage = Json.decodeFromString<Message>(json) // get type string from json
return messageTypes.entries.find { it.value == plainMessage.type }?.let {
// how can I use the KClass from it.key as reified parameter?
Json.decodeFromString(serializers.get(plainMessage.type)!!, json)
}
}
You might also want to have a look at the Kotlin built in handling of polymorphic classes: https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md

Jackson Kotlin - Deserialize JsonNode

Problem
I have JSON content in the form of a string, which I first want to traverse programmatically with Jackson. Then, when I have the node of interest, I want to deserialize it.
What I have tried
I have successfully deserialized strings using mapper.readValue, but now I want to perform such an operation on a jsonNode instead of a string.
Libraries
jackson-core:2.9.9
jackson-module-kotlin:2.9.9
Kotlin 1.3.41
kotlin-stdlib-jdk8:1.3.41
Code
package somepackage
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.kotlin.KotlinModule
import com.fasterxml.jackson.module.kotlin.readValue
import com.fasterxml.jackson.module.kotlin.treeToValue
fun main() {
val mapper = ObjectMapper().registerModule(KotlinModule())
readValueWorksFine(mapper)
treeToValueFails(mapper)
}
fun treeToValueFails(mapper: ObjectMapper) {
val fullJsonContent = """
[{
"product_id":123,
"Comments":
[{
"comment_id": 23,
"message": "Hello World!"
}]
}]
""".trimIndent()
// Traverse to get the node of interest
val commentsNode: JsonNode = mapper.readTree(fullJsonContent).get(0).get("Comments")
// Deserialize
val comments: List<Comment> = mapper.treeToValue<List<Comment>>(commentsNode)
// The line below fails. (I would have expected the exception to be thrown in the line above instead.
// Exception:
// Exception in thread "main" java.lang.ClassCastException: class
// java.util.LinkedHashMap cannot be cast to class somepackage.Comment (java.util.LinkedHashMap is in module
// java.base of loader 'bootstrap'; somepackage.Comment is in unnamed module of loader 'app')
for (comment: Comment in comments) { // This line fails
println(comment.comment_id)
println(comment.message)
}
}
fun readValueWorksFine(mapper: ObjectMapper) {
val commentsJsonContent = """
[{
"comment_id": 23,
"message": "Hello World!"
}]
""".trimIndent()
val comments1: List<Comment> = mapper.readValue<List<Comment>>(commentsJsonContent)
for (comment in comments1) {
println(comment)
}
}
data class Comment(val comment_id: Long, val message: String)
Exception/Output
The code above results in the following exception/output:
Comment(comment_id=23, message=Hello World!)
Exception in thread "main" java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class somepackage.Comment (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; somepackage.Comment is in unnamed module of loader 'app')
at somepackage.TKt.treeToValueFails(T.kt:39)
at somepackage.TKt.main(T.kt:13)
at somepackage.TKt.main(T.kt)
The problem cause
Even though ObjectMapper.treeToValue is a Kotlin inline extension function with a reified generic parameter (which means that generics are preserved at runtime), it calls the Java ObjectMapper.treeToValue(TreeNode, Class<T>) method. The value passed as Class<T> will loose generic type information for generic types such as List<Comment>, because of type erasure.
So treeToValue can be used for:
mapper.treeToValue<Comment>(commentNode)
but not for:
mapper.treeToValue<List<Comment>>(commentsNode)
Also note that ObjectMapper contains multiple methods that have #SuppressWarnings annotations, which causes some problems not to appear at compile-time, but at run-time.
Solution 1 - use convertValue()
This is the best solution. It uses the Kotlin extension function ObjectMapper.convertValue.
val commentsNode = mapper.readTree(fullJsonContent).get(0).get("Comments")
val comments = mapper.convertValue<List<Comment>>(commentsNode)
Solution 2 - use an ObjectReader
This solution doesn't use jackson-module-kotlin extension functions.
val reader = mapper.readerFor(object : TypeReference<List<Comment>>() {})
val comments: List<Comment> = reader.readValue(commentsNode)
Solution 3 - deserialize in map
Because treeToValue (Kotlin extension function) does work for non-generic types, you can first get the nodes as as list of JsonNodes, and then map each JsonNode to a Comment.
But it's cumbersome that you cannot simply return mapper.treeToValue(it), because that causes type inference compile errors.
val commentsNode = mapper.readTree(fullJsonContent).get(0).get("Comments")
val comments = commentsNode.elements().asSequence().toList().map {
val comment: Comment = mapper.treeToValue(it)
comment
}

Configure spray-json for non strict parsing deserialization

How to configure the spray-json parsing on parsing options?
Similarly as Jackson Parsing Features.
For example, I am parsing a json that has a field that my case class has not, and it is breaking:
spray.json.DeserializationException: Object is missing required member 'myfield'
UPDATE :
A simple example:
case class MyClass(a: String, b: Long);
and try to parse an incomplete json like
val data = "{a: \"hi\"}"
with a spray-json format like:
jsonFormat2(MyClass.apply)
// ...
data.parseJson.convertTo[MyClass]
(simplified code).
But the question goes further, I want to ask about configuration options like in other parsers. More examples:
Be able to ignore fields that exist in the JSON but not in the case class.
Ways of managing nulls or nonexistent values.
etc.
SprayJson allows you to define custom parsers like so:
case class Foo(a: String, b: Int)
implicit object FooJsonFormat extends RootJsonFormat[Foo] {
override def read(json: JsValue): Foo = {
json.asJsObject.getFields("name", "id") match {
case Seq(JsString(name), id) =>
Foo(name, id.convertTo[Int])
}
}
override def write(obj: Foo): JsValue = obj.toJson
}
This allows you to parse any arbitrary payload and pull out the fields "name" and "id" - other fields are ignored. If those fields are not guaranteed you can add something like:
case Seq(JsString(name), JsNull) =>
Foo(name, 0)
You should look at what's available in JsValue.scala - in particular JsArray may come in handy if you're getting payloads with anonymous arrays (i.e. the root is [{...}] instead of {"field":"value"...})
Spray Json doesn't support default parameters. So You cannot have a case class like
case class MyClass(a: String, b: Int = 0)
and then parse json like {"a":"foo"}
However if you make the second parameter as Option. then it works.
import spray.json._
case class MyClass(a: String, b: Option[Int] = None)
object MyProtocol extends DefaultJsonProtocol {
implicit val f = jsonFormat2(MyClass)
}
import MyProtocol.f
val mc1 = MyClass("foo", Some(10))
val strJson = mc1.toJson.toString
val strJson2 = """{"a": "foo"}"""
val mc2 = strJson2.parseJson.convertTo[MyClass]
println(mc2)

Which JSON serialization library would fit to the following case?

I've got following case:
I'd like to serialize Scala case classes that extend parent class with var of type java.util.UUID.
Serialization of this case classes should happen without any configuration of them - no annotations and definition of custom formats. Any serialization hints may be situated in parent class.
I tried sjson, but Reflection based serialization can't serialize UUID types and type based serialization forces me to define formats for every case class.
Which json serialization library would best fit this case?
Here's one solution with Lift JSON.
import java.util.UUID
import net.liftweb.json._
import net.liftweb.json.JsonAST._
import net.liftweb.json.JsonDSL._
import net.liftweb.json.Serialization._
sealed abstract class Parent {
def uuid: UUID
}
case class Foo(uuid: UUID, name: String) extends Parent
object UUIDTest extends Application {
implicit val formats = Serialization.formats(NoTypeHints) + new UUIDSerializer
val f = Foo(UUID.randomUUID, "foo")
val ser = write(f)
println(ser)
val f2 = read[Foo](ser)
assert(f == f2)
// Special serializer for UUID type
class UUIDSerializer extends Serializer[UUID] {
private val Class = classOf[UUID]
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), UUID] = {
case (TypeInfo(Class, _), json) => json match {
case JObject(JField("mostSig", JInt(m)) :: JField("leastSig", JInt(l)) :: Nil) =>
new UUID(m.longValue, l.longValue)
case x => throw new MappingException("Can't convert " + x + " to UUID")
}
}
def serialize(implicit format: Formats): PartialFunction[Any, JValue] = {
case x: UUID =>
("mostSig" -> x.getMostSignificantBits) ~ ("leastSig" -> x.getLeastSignificantBits)
}
}
}
It prints:
{"uuid":{"mostSig":-8054689529719995935,"leastSig":-5722404370736228056},"name":"foo"}'
Another solution which uses a custom serializer for Parent type.
sealed abstract class Parent {
var uuid: UUID = UUID.randomUUID
}
case class Foo(name: String) extends Parent
object UUIDTest extends Application {
implicit val formats =
Serialization.formats(NoTypeHints) + new ParentSerializer
val f = Foo("foo")
val ser = write(f)
println(ser)
val f2 = read[Foo](ser)
assert(f == f2)
// Special serializer for Parent type
class ParentSerializer extends Serializer[Parent] {
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Parent] = {
case (t#TypeInfo(cl, _), json) if (classOf[Parent].isAssignableFrom(cl)) =>
val x = Extraction.extract(json, t)(DefaultFormats).asInstanceOf[Parent]
x.uuid = (for {
JField("mostSig", JInt(m)) <- json
JField("leastSig", JInt(l)) <- json
} yield new UUID(m.longValue, l.longValue)).head
x
}
def serialize(implicit format: Formats): PartialFunction[Any, JValue] = {
case x: Parent =>
Extraction.decompose(x)(DefaultFormats) ++
JField("mostSig", x.uuid.getMostSignificantBits) ++
JField("leastSig", x.uuid.getLeastSignificantBits)
}
}
}
If the type is important, you should take a look at YAML.
http://www.google.fr/search?q=java+yaml
It's a subset of json with improved stuff, like variable typing.
You could try jerkson: https://github.com/codahale/jerkson
Its working good for my use, but that is mostly list/map structures. Would not be surprised if it supports your needs though..
Edit: Tried it with the following example (inspired by the lift example in another answer). Seems to work fine.
import java.util.UUID
import com.codahale.jerkson.Json
import org.scalatest.FunSuite
sealed abstract class Parent {
def uuid: UUID
}
case class Foo(uuid: UUID, name: String) extends Parent
class TmpJsonTest extends FunSuite {
test("Json case class serialize") {
val f = Foo(UUID.randomUUID, "foo")
val ser = Json.generate(f)
println(ser)
val f2 = Json.parse[Foo](ser)
assert(f === f2)
}
}
Try the XStream library which includes JSON support. I have used this successfully in a few projects. It has a number of default converters, including one for java.util.UUID. A full list of default converters is located here: http://x-stream.github.io/converters.html.
A brief tutorial on using XStream for JSON reading and writing is located here: http://x-stream.github.io/json-tutorial.html. The tutorial code is written for Java but it should work just the same for Scala since reflection is being used behind the scenes.
Keep in mind that serializing and then deserializing arbitrary graphs of objects is not always possible with this library. In particular, loops in your data cannot be handled, i.e. your data must be a purely hierarchical tree. This is a reasonable limitation given the intentions of the JSON format.
Referenced links:
XStream JSON tutorial: http://x-stream.github.io/json-tutorial.html
XStream default converters: http://x-stream.github.io/converters.html