Polymorphic deserialization with kotlinx.serialization in Kotlin/Native - json

I want to decode a json string containing a list of objects in a polymorphic class structure using kotlinx.serialization in a Kotlin Multiplatform project, but it works only on JVM, not on Native. Here is a minimum reproducible example:
#Serializable
abstract class Project {
abstract val name: String
}
#Serializable
#SerialName("BasicProject")
data class BasicProject(override val name: String): Project()
#Serializable
#SerialName("OwnedProject")
data class OwnedProject(override val name: String, val owner: String) : Project()
fun main() {
val data = Json.decodeFromString<List<Project>>("""
[
{"type":"BasicProject","name":"example"},
{"type":"OwnedProject","name":"kotlinx.serialization","owner":"kotlin"}
]
"""))
}
This works on JVM but throws the following exception on Native:
kotlinx.serialization.SerializationException: Serializer for class ‘Project’ is not found.
Mark the class as #Serializable or provide the serializer explicitly.
On Kotlin/Native explicitly declared serializer should be used for interfaces and enums without #Serializable annotation.message
This problem has been discussed before in the context of encoding and some workarounds have been suggested, e.g. here, but my problem is decoding. Is there a workaround, or do I simply have to implement my own json parser?

You need to explicitly pass respectful serializer and serializersModule:
object ListOfProjectSerializer : KSerializer<List<Project>> by ListSerializer(Project.serializer())
val module = SerializersModule {
polymorphic(Project::class) {
subclass(BasicProject::class)
subclass(OwnedProject::class)
}
}
fun main() {
val data = Json { serializersModule = module }.decodeFromString(
ListOfProjectSerializer,
"""
[
{"type":"BasicProject","name":"example"},
{"type":"OwnedProject","name":"kotlinx.serialization","owner":"kotlin"}
]
"""
)
}

Related

Kotlin Serialization issues: Class is not registered for polymorphic serialization in the scope of its interface

I am facing issues with serialization using Kotlin. I've followed through the steps here https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serialization-guide.md but unfortunately, no luck...
This is my code:
sealed interface Convertible {
fun convertUserInput(value : String): String
}
#Serializable
#SerialName("CustomConvertible")
class CustomConvertible(): Convertible {
override fun convertUserInput(value : String): String {
return ""
}
}
#Serializable
class DTOAttribute(val convertibles : List<Convertible> = emptyList())
Later on, I'd like to encode the DTOAttribute with val string = Json.encodeToString(dtoAttr)
Calling this, gives me the following exception:
kotlinx.serialization.SerializationException: Class 'CustomConvertible' is not registered for polymorphic serialization in the scope of 'Convertible'.
Mark the base class as 'sealed' or register the serializer explicitly.
This confuses me, as I've marked the interface as sealed and used #Serializable.
Versions build.gradle.kts
plugins{
kotlin("jvm")
kotlin("plugin.serialization") version("1.6.10")
// ...
}
sourceSets{
named("main") {
dependencies {
// ...
api("org.jetbrains.kotlin:kotlin-reflect:1.6.10")
api("org.jetbrains.kotlinx:kotlinx-serialization-json:1.3.2")
}
}
}
What else am I missing then?
Update: you're using Kotlin 1.6.10, but KxS didn't
support serializing sealed interfaces until 1.6.20
If you can update to 1.6.20+, then adding #Serializable to Convertible works.
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
#Serializable
sealed interface Convertible {
fun convertUserInput(value: String): String
}
#Serializable
#SerialName("CustomConvertible")
class CustomConvertible : Convertible {
override fun convertUserInput(value: String): String {
return ""
}
}
#Serializable
class DTOAttribute(
val convertibles: List<Convertible> = emptyList()
)
fun main() {
val dtoAttribute = DTOAttribute(listOf(CustomConvertible()))
val string = Json.encodeToString(dtoAttribute)
println(string)
// {"convertibles":[{"type":"CustomConvertible"}]}
}
If I remove #Serializable I get the same error that you report
//#Serializable
sealed interface Convertible {
fun convertUserInput(value: String): String
}
Exception in thread "main" kotlinx.serialization.SerializationException: Class 'CustomConvertible' is not registered for polymorphic serialization in the scope of 'Convertible'.
Mark the base class as 'sealed' or register the serializer explicitly.
Versions
Kotlinx Serialization 1.3.3
Kotlin/JVM 1.7.10
Workaround for 1.6.10 - sealed class
If you can't update your version of Kotlin then you can convert Convertible to be a sealed class.
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
#Serializable
sealed class Convertible {
abstract fun convertUserInput(value: String): String
}
#Serializable
#SerialName("CustomConvertible")
class CustomConvertible : Convertible() {
override fun convertUserInput(value: String): String {
return ""
}
}
#Serializable
class DTOAttribute(
val convertibles: List<Convertible> = emptyList()
)
fun main() {
val dtoAttribute = DTOAttribute(listOf(CustomConvertible()))
val string = Json.encodeToString(dtoAttribute)
println(string)
// {"convertibles":[{"type":"CustomConvertible"}]}
}
Setup
Here's how to setup using Gradle Kotlin DSL (from the README):
// build.gradle.kts
plugins {
kotlin("jvm") version "1.7.10" // or kotlin("multiplatform") or any other kotlin plugin
kotlin("plugin.serialization") version "1.7.10"
}
dependencies {
implementation(platform("org.jetbrains.kotlinx:kotlinx-serialization-bom:1.3.3"))
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json")
}

kotlinx serialization — best way to do polymorphic child deserialization

I have a Json input like:
{
"type": "type_1",
"data": {
// ...
}
}
data field can vary depending on type.
So, I need a deserializer, that looks on type (enum) and deserializes data respectively (for instance, for type_1 value it's Type1 class, for type_2 — Type2, etc).
I thought about a fully-custom deserializer (extending a KSerializer<T>), but it looks like an overkill.
What's the best (kotlin) way to do such deserialization?
Kotlin way for polymorphic deserialization is to have a plain JSON (with all data fields on the same level as type field):
{
"type": "type_1",
// ...
}
and register all subclasses of abstract superclass with serializers module (this step could be skipped if superclass is a sealed class).
No need for enums - just mark subclasses declarations with respectful #SerialName("type_1") annotations if its name in JSON differs from fully-qualified class name.
If original JSON shape is a strict requirement, then you may transform it on the fly to a plain one, reducing the task to the previous one.
#Serializable(with = CommonAbstractSuperClassDeserializer::class)
abstract class CommonAbstractSuperClass
#Serializable
#SerialName("type_1")
data class Type1(val x: Int, val y: Int) : CommonAbstractSuperClass()
#Serializable
#SerialName("type_2")
data class Type2(val a: String, val b: Type1) : CommonAbstractSuperClass()
object CommonAbstractSuperClassDeserializer :
JsonTransformingSerializer<CommonAbstractSuperClass>(PolymorphicSerializer(CommonAbstractSuperClass::class)) {
override fun transformDeserialize(element: JsonElement): JsonElement {
val type = element.jsonObject["type"]!!
val data = element.jsonObject["data"] ?: return element
return JsonObject(data.jsonObject.toMutableMap().also { it["type"] = type })
}
}
fun main() {
val kotlinx = Json {
serializersModule = SerializersModule {
polymorphic(CommonAbstractSuperClass::class) {
subclass(Type1::class)
subclass(Type2::class)
}
}
}
val str1 = "{\"type\":\"type_1\",\"data\":{\"x\":1,\"y\":1}}"
val obj1 = kotlinx.decodeFromString<CommonAbstractSuperClass>(str1)
println(obj1) //Type1(x=1, y=1)
val str2 = "{\"type\":\"type_2\",\"data\":{\"a\":\"1\",\"b\":{\"x\":1,\"y\":1}}}"
val obj2 = kotlinx.decodeFromString<CommonAbstractSuperClass>(str2)
println(obj2) //Type2(a=1, b=Type1(x=1, y=1))
//Works for plain JSON shape as well:
val str0 = "{\"type\":\"type_1\",\"x\":1,\"y\":1}"
val obj0 = kotlinx.decodeFromString<CommonAbstractSuperClass>(str0)
println(obj0) //Type1(x=1, y=1)
}

Exception when deserializing JSON in Kotlin JS app

I'm trying to get a Kotlin JS app working, and when consuming data from a server using this code:
val client = HttpClient {
install(JsonFeature) {
serializer = KotlinxSerializer()
}
}
#Serializable
data class Entry(val start: String, val end: String)
suspend fun loadData() {
val data = client.get<List<Entry>>("http://localhost:8080/data") {
accept(ContentType.Application.Json)
}
console.log(data)
}
I get exceptions like this:
Serializer for class 'Entry' is not found.
Mark the class as #Serializable or provide the serializer explicitly.
On Kotlin/JS explicitly declared serializer should be used for interfaces and enums without #Serializable annotation
even though the class is marked as #Serializable.
If I change it to client.get<List<Map<String,String>>> then I get a valid result.
What am I doing wrong?
I needed to add this to build.gradle.kts:
kotlin("plugin.serialization") version "1.4.31"

How do you register the KotlinModule to the AWS lambda Jackson Object Mapper?

I'm using Kotlin to write an AWS Lambda. I have a Kotlin data class
class MessageObject(
val id: String,
val name: String,
val otherId: String
)
This data class is used as the input to the required interface implementation
class Handler : RequestHandler<MessageObject, Output> {
...
override fun handleRequest(msg: MessageObject, ctx: Context) {
...
}
}
When I test this lambda in the aws console, and pass it a proper JSON message, I get this:
An error occurred during JSON parsing: java.lang.RuntimeException
java.lang.RuntimeException: An error occurred during JSON parsing
Caused by: java.io.UncheckedIOException:
com.amazonaws.lambda.thirdparty.com.fasterxml.jackson.databind.exc.InvalidDefinitionException:
Cannot construct instance of 'com.mycode.MessageObject'(no Creators, like default construct, exist):
cannot deserialize from Object value (no delegate- or property-based Creator)
I'm almost certain this is fixed by saying:
ObjectMapper().registerModule(KotlinModule())
but in the world of AWS Lambda how do I edit the object mapper provided by AWS?
If you haven't gotten it to work with KotlinModule, since the problem you're having is that Jackson requires a default empty constructor and you currently don't have one. You could just change your MessageObject as follows and it should work:
data class MessageObject(
var id: String = "",
var name: String = "",
var otherId: String = ""
)
I created this repo with a fully functional kotlin lambda template using the Serverless Framework. Have a look for some other tidbits you might need: https://github.com/crafton/sls-aws-lambda-kotlin-gradlekt
You cannot use data class with provided RequestHandler<I, O> unfortunately, because you need register the kotlin module for your jackson mapper in order to work with data classes. But you can write you own RequestHandler, which will like this one.
Here's the code:
interface MyRequestStreamHandler<I : Any, O : Any?> : RequestStreamHandler {
val inputType: Class<I>
fun handleRequest(input: I, context: Context): O?
override fun handleRequest(inputStream: InputStream, outputStream: OutputStream, context: Context) {
handleRequest(inputStream.readJson(inputType), context).writeJsonNullable(outputStream)
}
interface MessageObjectRequestHandler : MyRequestStreamHandler< MessageObject, Output> {
override val inputType: Class<MessageObject >
get() = MessageObject::class.java
}
}
And jackson util:
private val objectMapper = jacksonObjectMapper()
.configure(JsonParser.Feature.ALLOW_COMMENTS, true)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.registerKotlinModule()
private val writer: ObjectWriter = objectMapper.writer()
fun <T : Any> readJson(clazz: Class<T>, stream: InputStream): T =
objectMapper.readValue(stream, clazz)
fun <T : Any> InputStream.readJson(clazz: Class<T>): T =
readJson(clazz, this)
fun Any?.writeJsonNullable(outputStream: OutputStream) {
if (this != null) writer.writeValue(outputStream, this)
}
Now, you can keep your MessageObject class to be data class, and your handler will look something like:
class LambdaMain : MessageObjectRequestHandler {
override fun handleRequest(input: MessageObject, context: Context): Output {
//...
}
}

Serialize Sealed class within a data class using Gson in kotlin

I have created a sealed class for the json field Value under CustomAttribute data class. This field can return String or Array of Strings.
How can we deserialize this sealed class from json?
data class CustomAttribute (
val attributeCode: String,
val value: Value
)
sealed class Value {
class StringArrayValue(val value: List<String>) : Value()
class StringValue(val value: String) : Value()
}
One solution is to use a RuntimeTypeAdapterFactory as per the instructions in this answer
val valueTypeAdapter = RuntimeTypeAdapter.of(Value::class.java)
.registerSubtype(StringArrayValue::class.java)
.registerSubtype(StringValue::class.java)
val gson = GsonBuilder().registerTypeAdapter(valueTypeAdapter).create()
RuntimeTypeAdapter is included in the source code for Gson but not exposed as a Maven artifact.
It is designed to be copy/pasted into your project from here
I created a TypeAdapterFactory implementation specifically to support sealed classes and their subtypes. This works similarly to the RuntimeTypeAdapterFactory (and I used it as a guide to write my class), but will specifically only support sealed types, and will deserialize using object instances of objects with a sealed class supertype (RuntimeTypeAdapterFactory will create a new instance of object types, which breaks equality checks when a single instance is the expectation).
private class SealedTypeAdapterFactory<T : Any> private constructor(
private val baseType: KClass<T>,
private val typeFieldName: String
) : TypeAdapterFactory {
private val subclasses = baseType.sealedSubclasses
private val nameToSubclass = subclasses.associateBy { it.simpleName!! }
init {
if (!baseType.isSealed) throw IllegalArgumentException("$baseType is not a sealed class")
}
override fun <R : Any> create(gson: Gson, type: TypeToken<R>?): TypeAdapter<R>? {
if (type == null || subclasses.isEmpty() || subclasses.none { type.rawType.isAssignableFrom(it.java) }) return null
val elementTypeAdapter = gson.getAdapter(JsonElement::class.java)
val subclassToDelegate: Map<KClass<*>, TypeAdapter<*>> = subclasses.associateWith {
gson.getDelegateAdapter(this, TypeToken.get(it.java))
}
return object : TypeAdapter<R>() {
override fun write(writer: JsonWriter, value: R) {
val srcType = value::class
val label = srcType.simpleName!!
#Suppress("UNCHECKED_CAST") val delegate = subclassToDelegate[srcType] as TypeAdapter<R>
val jsonObject = delegate.toJsonTree(value).asJsonObject
if (jsonObject.has(typeFieldName)) {
throw JsonParseException("cannot serialize $label because it already defines a field named $typeFieldName")
}
val clone = JsonObject()
clone.add(typeFieldName, JsonPrimitive(label))
jsonObject.entrySet().forEach {
clone.add(it.key, it.value)
}
elementTypeAdapter.write(writer, clone)
}
override fun read(reader: JsonReader): R {
val element = elementTypeAdapter.read(reader)
val labelElement = element.asJsonObject.remove(typeFieldName) ?: throw JsonParseException(
"cannot deserialize $baseType because it does not define a field named $typeFieldName"
)
val name = labelElement.asString
val subclass = nameToSubclass[name] ?: throw JsonParseException("cannot find $name subclass of $baseType")
#Suppress("UNCHECKED_CAST")
return (subclass.objectInstance as? R) ?: (subclassToDelegate[subclass]!!.fromJsonTree(element) as R)
}
}
}
companion object {
fun <T : Any> of(clz: KClass<T>) = SealedTypeAdapterFactory(clz, "type")
}
}
Usage:
GsonBuilder().registerTypeAdapter(SealedTypeAdapterFactory.of(Value::class)).create()
I have successfully serialized and de-serialized a sealed class in the past, with a disclaimer of using Jackson, not Gson as my serialization engine.
My sealed class has been defined as:
#JsonTypeInfo(use = JsonTypeInfo.Id.MINIMAL_CLASS, include = JsonTypeInfo.As.PROPERTY, visible = true)
sealed class FlexibleResponseModel
class SnapshotResponse(val collection: List<EntityModel>): FlexibleResponseModel()
class DifferentialResponse(val collection: List<EntityModel>): FlexibleResponseModel()
class EventDrivenResponse(val collection: List<EntityEventModel>): FlexibleResponseModel()
class ErrorResponse(val error: String): FlexibleResponseModel()
With the annotations used, it required no further configuration for the Jackson instance to properly serialize and de-serialize instances of this sealed class granted that both sides of the communication possessed a uniform definition of the sealed class.
While I recognise that JsonTypeInfo is a Jackson-specific annotation, perhaps you might consider switching over from Gson if this feature is a must - or you might be able to find an equivalent configuration for Gson which would also include the class identifier in your serialized data.