Deserialize JSON with "non-stable" values using kotlin serialization - json

I have not very well structured JSON from BE (that I cannot change) which was formerly handled with moshi (custom adapter for the issue). Now I am trying to use pure kotlin serialization instead, but as said, JSON structure doesn't help me much.
{
"foo": {
"version":1,
"mask": [
{
"values": [
{ "bar": 1, ... }
]
},
{
"values": [
"important text i guess"
]
},
]
}
}
Now as you can see my issue is with values that can contains both object as well as string. All this should be parsed into kotlin where values looks like this:
data class Values(
val bar: Int? = null,
val text: String? = null,
...
)
Theoretically I can change local implementation, e.g. split the values class or something.
I've already tried to apply Polymorphic deserialization, but as I understand it was not able to recognise difference between two descendants of values without classDiscriminator set in JSON.
Any good advice?
Update
The polymorphic version I tried (in case I just made some error)
#Polymorphic
#Serializable
sealed class Values{
#Serializable
data class ObjectValues(
val bar: Int? = null,
...
)
#Serializable
data class TextValues(
val text: String? = null
)
}
and use it:
Json {
ignoreUnknownKeys = true
serializersModule = SerializersModule {
polymorphic(Values::class) {
subclass(Values.ObjectValues::class, Values.ObjectValues.serializer())
subclass(Values.TextValues::class, Values.TextValues.serializer())
}
}
error:
Polymorphic serializer was not found for missing class discriminator ('null')
JSON input: {"bar":42,...}
Disclaimer
I know that this would fix it all :)
"values": [ { "text":"important text i guess" } ]

Having classDiscriminator is not a mandatory requirement for polymorphic deserialization. You can use content-based polymorphic deserialization in this case.
All you need is to define a JsonContentPolymorphicSerializer for Values class with some logic for exact subclass serializer selection:
object ValuesSerializer : JsonContentPolymorphicSerializer<Values>(Values::class) {
override fun selectDeserializer(element: JsonElement) = when {
element.jsonObject["values"]!!.jsonArray.first() is JsonObject -> ObjectValues.serializer()
else -> TextValues.serializer()
}
}
and wire it as a serializer for Values class:
#Serializable(with = ValuesSerializer::class)
sealed class Values
#Serializable
data class TextValues(val values: List<String>) : Values()
#Serializable
data class ObjectValues(val values: List<Bar>) : Values()
#Serializable
data class Bar(val bar: Int)
#Serializable
data class MyJson(val foo: Foo)
#Serializable
data class Foo(val version: Int, val mask: List<Values>)
No need for serializersModule:
val result = Json.decodeFromString<MyJson>(json)

Related

Kotlinx Deserialization Object or List of Object to List of Object

I am currently implementing an API client with Ktor. The API I am requesting does not return a consistent JSON format.
for Example:
sometimes the JSON looks like this:
{
"description": {
"lang": "en",
"value": "an English description..."
},
...
}
and sometimes like this:
{
"description": [
{
"lang": "en",
"value": "an English description..."
},
{
"lang": "fr",
"value": "a French description..."
}
],
...
}
Now my Question:
How can I implement a Custom Kotlinx Deserializer to Decode an Object of T or a List<T> to a List<T>
My classes look like this:
#Serializable
class ResourceResponse(
#SerialName("description")
val descriptions: List<Description>
) {
#Serializable
data class Description(
#SerialName("value")
val value: String,
#SerialName("lang")
val language: String,
)
}
I want that a Json with only one Description-Object will be deserialized to a List with one Object and not specifically for the description, but in general for classes.
I've found nothing really helpful in the Web.
One solution is to first deserialize it to JsonElement, introspect and then decide how to deserialize it further into ResourceResponse:
fun decode(s: String): ResourceResponse {
val json = Json.parseToJsonElement(s).jsonObject
return when (val desc = json["description"]) {
is JsonArray -> Json.decodeFromJsonElement(json)
is JsonObject -> {
val json2 = json.toMutableMap()
json2["description"] = JsonArray(listOf(desc))
Json.decodeFromJsonElement(JsonObject(json2))
}
else -> throw IllegalArgumentException("Invalid value for \"description\": $desc")
}
}
This solution is definitely not ideal. It may be potentially less performant as we need to deserialize the whole tree into the tree of JsonElement objects only to transform it to the final types (although, maybe the library does this internally anyway). It works only for json and it is tricky to use this solution if ResourceResponse is somewhere deep into the data structure.
You can use a JsonContentPolymorphicSerializer to choose a deserializer based on the form of the JSON.
This one should work:
#Suppress("UNCHECKED_CAST")
class DescriptionsSerializer : JsonContentPolymorphicSerializer<List<ResourceResponse.Description>>(
List::class as KClass<List<ResourceResponse.Description>>
) {
// Here we check the form of the JSON we are decoding, and choose
// the serializer accordingly
override fun selectDeserializer(element: JsonElement): DeserializationStrategy<out List<ResourceResponse.Description>> {
return if (element is JsonArray)
ListSerializer(ResourceResponse.Description.serializer())
else
SingleDescriptionAsList()
}
class SingleDescriptionAsList : KSerializer<List<ResourceResponse.Description>> {
override val descriptor: SerialDescriptor
get() = ResourceResponse.Description.serializer().descriptor
override fun deserialize(decoder: Decoder): List<ResourceResponse.Description> {
return listOf(ResourceResponse.Description.serializer().deserialize(decoder))
}
override fun serialize(encoder: Encoder, value: List<ResourceResponse.Description>) {
throw Exception("Not in use")
}
}
}
You must also amend your original class to tell it to use this serializer:
#Serializable
class ResourceResponse(
#SerialName("description")
#Serializable(with = DescriptionsSerializer::class) val descriptions: List<Description>
) {
#Serializable
data class Description(
#SerialName("value")
val value: String,
#SerialName("lang")
val language: String,
)
}
Then you will be able to decode JSON objects with the single key "descriptions" using the ResourceResponse serializer.
For avoidance of doubt, if there are other keys in the JSON (it's not entirely clear from the question) then those should also be written into ResourceResponse definition.
After my research, I have now come up with a solution. For this you need a wrapper class. (here GenericResponse). I hope I can help others who have the same problem.
This is the Wrapper-Class
#Serializable(with = ListOrObjectSerializer::class)
class GenericResponse<T>(
val data: List<T> = emptyList()
) {
private var _isNothing : Boolean = false
val isNothing: Boolean
get() {
return this._isNothing
}
companion object {
fun <T> nothing(): GenericResponse<T> {
val o = GenericResponse(emptyList<T>())
o._isNothing = true
return o
}
}
}
And the Serializer looks like:
import kotlinx.serialization.KSerializer
import kotlinx.serialization.builtins.ListSerializer
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.*
class ListOrObjectSerializer<T : Any>(private val tSerializer: KSerializer<T>): KSerializer<GenericResponse<T>> {
override val descriptor: SerialDescriptor
get() = tSerializer.descriptor
override fun deserialize(decoder: Decoder): GenericResponse<T> {
val input = decoder as JsonDecoder
val jsonObj = input.decodeJsonElement()
return when(jsonObj) {
is JsonObject -> GenericResponse(listOf(Json.decodeFromJsonElement(tSerializer, jsonObj)))
is JsonArray -> GenericResponse(Json.decodeFromJsonElement(ListSerializer(tSerializer), jsonObj))
else -> return GenericResponse.nothing()
}
}
override fun serialize(encoder: Encoder, value: GenericResponse<T>) {
throw IllegalAccessError("serialize not supported")
}
}
My Data-Class look now like:
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
#Serializable
class ResourceResponse(
#SerialName("description")
val descriptions: GenericResponse<Description>? = null,
) {
#Serializable
data class Description(
#SerialName("value")
val value: String? = null,
#SerialName("lang")
val language: String? = null,
)
}
data class ResourceResponse(
#SerializedName("description") val descriptions: List<Description>,
)
data class Description(
#SerializedName("value") val value: String,
#SerializedName("lang") val language: String,
)
it should be like that

How to convert a JSON string to an Object in KMM

Previously, I asked this question: Implementing generic method in interface that uses implementors class which allowed for an object to be converted to a JSON string.
But, now I would like to reverse the process. Ideally this would look like:
interface Domain {
constructor(json: String) {
/*...*/
}
}
#Serializable
class User(val a: Int, val b: Int): Domain {}
val user = User("{a: 3, b: 4}")
But, I'm unable to figure out how to construct an object directly from a JSON string.
A next best option would be create a static generator method:
interface Domain {
companion object {
inline fun <reified T> fromJSON(json: String): T {
return Json.decodeFromString(json)
}
}
}
val user = User.fromJSON("{a: 3, b: 4}")
But, this doesn't work at all because User does not inherit Domain's companion object. The 3rd best option:
val user = Domain.fromJSON<User>("{a: 3, b: 4}")
This does work from the Android side, however since fromJSON is declared inline and reified it is not exposed to iOS at all from the KMM.
Which brings me to my current solution:
#Serializable
class User(val a: Int, val b: Int): Domain {
companion object {
fun fromJSON(json: String): User { return Json.decodeFromString(json) }
}
}
val user = User.fromJSON("{a: 3, b: 4}")
This works, however it requires the above boilerplate code to be added to each and every 'Domain' object.
Is there anyway to improve on my current solution? (Of course, the higher up the chain the better.)
I think you have Object then you should need to convert it as :--
import com.google.gson.Gson
import com.google.gson.GsonBuilder
import com.codewithfun.kotlin.jsonparser.models.Tutorial
fun main(args: Array<String>) {
val gson = Gson()
val gsonPretty = GsonBuilder().setPrettyPrinting().create()
val tutsList: List<Tutorial> = listOf(
Tutorial("Tut #1", "bezkoder", listOf("cat1", "cat2")),
Tutorial("Tut #2", "zkoder", listOf("cat3", "cat4"))
);
val jsonTutsList: String = gson.toJson(tutsList)
println(jsonTutsList)
val jsonTutsListPretty: String = gsonPretty.toJson(tutsList)
println(jsonTutsListPretty)
}
Then the output be like :--
[{"title":"Tut #1","author":"bezkoder","categories":["cat1","cat2"]},{"title":"Tut #2","author":"zkoder","categories":["cat3","cat4"]}]
[
{
"title": "Tut #1",
"author": "bezkoder",
"categories": [
"cat1",
"cat2"
]
},
{
"title": "Tut #2",
"author": "zkoder",
"categories": [
"cat3",
"cat4"
]
}
]

Kotlin Serialization of Generic Type that is Serializable

Kotlin serialization is hard! How do I do get Kotlin to believe that the values in my properties map are either primitives or classes annotated with #Serializable?
I'm trying to turn a class like this: class Entity(val id: String, val type: String, val properties: Map<String, *>) where I know that * is primitive or #Serializable or String into JSON, e.g.: { id: "0", type: "falcon", max-flight-range-nm: 10 }
import kotlinx.serialization.*
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.encodeToJsonElement
fun main() {
val json = Json {
prettyPrint = true
}
val falcon = Entity("0", "falcon", mapOf("max-flight-range-nm" to 10))
println(json.encodeToString(falcon))
// Desired JSON (no nesting of 'Entity.parameters', they bump-up into parent intentionally
// { id: "0", type: "falcon", max-flight-range-nm: 10 }
// Avoiding extra nesting is conceptually similar to Jackson's #JsonAnyGetter annotation: https://www.baeldung.com/jackson-annotations#1-jsonanygetter
}
#Serializable
class Entity(val id: String, val type: String, #Contextual val properties: Map<String, *>) {
#Serializer(forClass = Entity::class)
companion object : KSerializer<Entity> {
override fun serialize(encoder: Encoder, value: Entity) {
encoder.encodeString(value.id)
encoder.encodeString(value.type)
// attempted hack to encode properties at this level
for ((mapKey, mapValue) in value.properties) {
val valueJson = Json.encodeToJsonElement(mapValue)
val jsonObject = JsonObject(mapOf(mapKey to valueJson))
encoder.encodeString(jsonObject.toString())
}
}
}
}
I have full control over all code, so if needed I could go so far as to write a bunch of sealed class PropertyValue subclasses and mark them as #Serializable - so IntPropertyValue, StringPropertyValue, FooPropertyValue, etc. Maybe that's the only way?
Unfortunately the above fails at compile-time (no errors in Intelli-J though):
org.jetbrains.kotlin.codegen.CompilationException: Back-end (JVM) Internal error: Serializer for element of type Any? has not been found.
I have full control over all code, so if needed I could go so far as to write a bunch of sealed class PropertyValue subclasses and mark them as #Serializable - so IntPropertyValue, StringPropertyValue, FooPropertyValue, etc. Maybe that's the only way?
You need to make PropertyValue itself #Serializable (see https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md) and of course use val properties: Map<String, PropertyValue> instead of val properties: Map<String, *> (since the error message says "Serializer for element of type Any?" you might have forgotten this step).
Also note that by default you won't get
{ id: "0", type: "falcon", max-flight-range-nm: 10 }
but e.g. (omitting key quotes)
{ id: "0", type: "falcon", properties: {
max-flight-range-nm: {
type: "your_package.IntPropertyValue", value: 10
}
} }
If you want to parse/produce that particular JSON, write a custom serializer, and then you might get away with Map<String, *> too (but that's still usually a bad idea).

Deserializing a json object property to a String using kotlinx.serialization

Given json as follows where the structure of the payload object will vary:
{
"id": 1,
"displayName": "Success",
"payload": {
"someProperty": "example",
"someOtherProperty": {
"someNestedProperty": "example"
}
}
}
...using kotlinx.serialization how can I deserialize this into the following data class, where the value of payload should be the raw json string of the payload object.
#Serializable
data class Stub(
val id: Int,
val displayName: String,
val payload: String
)
Struggled to find a way of doing this with Serializers, but it was simple enough to implement manually using JsonElement.
val jsonObject = Json.parseToJsonElement(jsonString).jsonObject
val stub = Stub(
jsonObject["id"]!!.jsonPrimitive.int,
jsonObject["displayName"]!!.jsonPrimitive.content,
jsonObject["payload"]!!.toString()
)
There is a way to handle using JSONTransformingSerializer. It allows you to transform the json prior to deserialization. In this case from a jsonElement into a jsonPrimitive (of type String).
First create a transformer as follows:
object JsonAsStringSerializer: JsonTransformingSerializer<String>(tSerializer = String.serializer()) {
override fun transformDeserialize(element: JsonElement): JsonElement {
return JsonPrimitive(value = element.toString())
}
}
Now apply this transfer to the specific element in your data class by adding...
#Serializable(with = JsonAsStringSerializer::class)
just above the property you want to transform. Like this...
#Serializable
data class Stub(
val id: Int,
val displayName: String,
#Serializable(with = JsonAsStringSerializer::class)
val payload: String
)
The value of payload will be a string:
"{'someProperty': 'example','someOtherProperty': {'someNestedProperty':'example'}"
If you are later trying to deserialize this into different models depending on the structure, check out the JsonContentPolymorphicSerializer feature.

How to deserialize JSON with dynamic object?

I have a simple json, but the containing field has dynamic object. For instance, json can look like
{
"fixedField1": "value1",
"dynamicField1": {
"f1": "abc",
"f2": 123
}
}
or
{
"fixedField1": "value2",
"dynamicField1": {
"g1": "abc",
"g2": { "h1": "valueh1"}
}
}
I am trying to serialize this object, but not sure how to map the dynamic field
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
val dynamicField: Map<String, Any> // ???? what should be the type?
)
Above code fails with following error
Backend Internal error: Exception during code generation Cause:
Back-end (JVM) Internal error: Serializer for element of type Any has
not been found.
I ran into a similar problem when I had to serialize arbitrary Map<String, Any?>
The only way I managed to do this so far was to use the JsonObject/JsonElement API and combining it with the #ImplicitReflectionSerializer
The major downside is the use of reflection which will only work properly in JVM and is not a good solution for kotlin-multiplatform.
#ImplicitReflectionSerializer
fun Map<*, *>.toJsonObject(): JsonObject = JsonObject(map {
it.key.toString() to it.value.toJsonElement()
}.toMap())
#ImplicitReflectionSerializer
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Boolean -> JsonPrimitive(this)
is Map<*, *> -> this.toJsonObject()
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
is Array<*> -> JsonArray(this.map { it.toJsonElement() })
else -> {
//supporting classes that declare serializers
val jsonParser = Json(JsonConfiguration.Stable)
val serializer = jsonParser.context.getContextualOrDefault(this)
jsonParser.toJson(serializer, this)
}
}
Then, to serialize you would use:
val response = mapOf(
"fixedField1" to "value1",
"dynamicField1" to mapOf (
"f1" to "abc",
"f2" to 123
)
)
val serialized = Json.stringify(JsonObjectSerializer, response.toJsonObject())
Note
This reflection based serialization is only necessary if you are constrained to use Map<String, Any?>
If you are free to use your own DSL to build the responses, then you can use the json DSL directly, which is very similar to mapOf
val response1 = json {
"fixedField1" to "value1",
"dynamicField1" to json (
"f1" to "abc",
"f2" to 123
)
}
val serialized1 = Json.stringify(JsonObjectSerializer, response1)
val response 2 = json {
"fixedField1" to "value2",
"dynamicField1" to json {
"g1" to "abc",
"g2" to json { "h1" to "valueh1"}
}
}
val serialized2 = Json.stringify(JsonObjectSerializer, response2)
If, however you are constrained to define a data type, and do serialization as well as deserialization you probably can't use the json DSL so you'll have to define a #Serializer using the above methods.
An example of such a serializer, under Apache 2 license, is here: ArbitraryMapSerializer.kt
Then you can use it on classes that have arbitrary Maps. In your example it would be:
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
#Serializable(with = ArbitraryMapSerializer::class)
val dynamicField: Map<String, Any>
)