I have a simple json, but the containing field has dynamic object. For instance, json can look like
{
"fixedField1": "value1",
"dynamicField1": {
"f1": "abc",
"f2": 123
}
}
or
{
"fixedField1": "value2",
"dynamicField1": {
"g1": "abc",
"g2": { "h1": "valueh1"}
}
}
I am trying to serialize this object, but not sure how to map the dynamic field
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
val dynamicField: Map<String, Any> // ???? what should be the type?
)
Above code fails with following error
Backend Internal error: Exception during code generation Cause:
Back-end (JVM) Internal error: Serializer for element of type Any has
not been found.
I ran into a similar problem when I had to serialize arbitrary Map<String, Any?>
The only way I managed to do this so far was to use the JsonObject/JsonElement API and combining it with the #ImplicitReflectionSerializer
The major downside is the use of reflection which will only work properly in JVM and is not a good solution for kotlin-multiplatform.
#ImplicitReflectionSerializer
fun Map<*, *>.toJsonObject(): JsonObject = JsonObject(map {
it.key.toString() to it.value.toJsonElement()
}.toMap())
#ImplicitReflectionSerializer
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Boolean -> JsonPrimitive(this)
is Map<*, *> -> this.toJsonObject()
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
is Array<*> -> JsonArray(this.map { it.toJsonElement() })
else -> {
//supporting classes that declare serializers
val jsonParser = Json(JsonConfiguration.Stable)
val serializer = jsonParser.context.getContextualOrDefault(this)
jsonParser.toJson(serializer, this)
}
}
Then, to serialize you would use:
val response = mapOf(
"fixedField1" to "value1",
"dynamicField1" to mapOf (
"f1" to "abc",
"f2" to 123
)
)
val serialized = Json.stringify(JsonObjectSerializer, response.toJsonObject())
Note
This reflection based serialization is only necessary if you are constrained to use Map<String, Any?>
If you are free to use your own DSL to build the responses, then you can use the json DSL directly, which is very similar to mapOf
val response1 = json {
"fixedField1" to "value1",
"dynamicField1" to json (
"f1" to "abc",
"f2" to 123
)
}
val serialized1 = Json.stringify(JsonObjectSerializer, response1)
val response 2 = json {
"fixedField1" to "value2",
"dynamicField1" to json {
"g1" to "abc",
"g2" to json { "h1" to "valueh1"}
}
}
val serialized2 = Json.stringify(JsonObjectSerializer, response2)
If, however you are constrained to define a data type, and do serialization as well as deserialization you probably can't use the json DSL so you'll have to define a #Serializer using the above methods.
An example of such a serializer, under Apache 2 license, is here: ArbitraryMapSerializer.kt
Then you can use it on classes that have arbitrary Maps. In your example it would be:
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
#Serializable(with = ArbitraryMapSerializer::class)
val dynamicField: Map<String, Any>
)
Related
I am currently implementing an API client with Ktor. The API I am requesting does not return a consistent JSON format.
for Example:
sometimes the JSON looks like this:
{
"description": {
"lang": "en",
"value": "an English description..."
},
...
}
and sometimes like this:
{
"description": [
{
"lang": "en",
"value": "an English description..."
},
{
"lang": "fr",
"value": "a French description..."
}
],
...
}
Now my Question:
How can I implement a Custom Kotlinx Deserializer to Decode an Object of T or a List<T> to a List<T>
My classes look like this:
#Serializable
class ResourceResponse(
#SerialName("description")
val descriptions: List<Description>
) {
#Serializable
data class Description(
#SerialName("value")
val value: String,
#SerialName("lang")
val language: String,
)
}
I want that a Json with only one Description-Object will be deserialized to a List with one Object and not specifically for the description, but in general for classes.
I've found nothing really helpful in the Web.
One solution is to first deserialize it to JsonElement, introspect and then decide how to deserialize it further into ResourceResponse:
fun decode(s: String): ResourceResponse {
val json = Json.parseToJsonElement(s).jsonObject
return when (val desc = json["description"]) {
is JsonArray -> Json.decodeFromJsonElement(json)
is JsonObject -> {
val json2 = json.toMutableMap()
json2["description"] = JsonArray(listOf(desc))
Json.decodeFromJsonElement(JsonObject(json2))
}
else -> throw IllegalArgumentException("Invalid value for \"description\": $desc")
}
}
This solution is definitely not ideal. It may be potentially less performant as we need to deserialize the whole tree into the tree of JsonElement objects only to transform it to the final types (although, maybe the library does this internally anyway). It works only for json and it is tricky to use this solution if ResourceResponse is somewhere deep into the data structure.
You can use a JsonContentPolymorphicSerializer to choose a deserializer based on the form of the JSON.
This one should work:
#Suppress("UNCHECKED_CAST")
class DescriptionsSerializer : JsonContentPolymorphicSerializer<List<ResourceResponse.Description>>(
List::class as KClass<List<ResourceResponse.Description>>
) {
// Here we check the form of the JSON we are decoding, and choose
// the serializer accordingly
override fun selectDeserializer(element: JsonElement): DeserializationStrategy<out List<ResourceResponse.Description>> {
return if (element is JsonArray)
ListSerializer(ResourceResponse.Description.serializer())
else
SingleDescriptionAsList()
}
class SingleDescriptionAsList : KSerializer<List<ResourceResponse.Description>> {
override val descriptor: SerialDescriptor
get() = ResourceResponse.Description.serializer().descriptor
override fun deserialize(decoder: Decoder): List<ResourceResponse.Description> {
return listOf(ResourceResponse.Description.serializer().deserialize(decoder))
}
override fun serialize(encoder: Encoder, value: List<ResourceResponse.Description>) {
throw Exception("Not in use")
}
}
}
You must also amend your original class to tell it to use this serializer:
#Serializable
class ResourceResponse(
#SerialName("description")
#Serializable(with = DescriptionsSerializer::class) val descriptions: List<Description>
) {
#Serializable
data class Description(
#SerialName("value")
val value: String,
#SerialName("lang")
val language: String,
)
}
Then you will be able to decode JSON objects with the single key "descriptions" using the ResourceResponse serializer.
For avoidance of doubt, if there are other keys in the JSON (it's not entirely clear from the question) then those should also be written into ResourceResponse definition.
After my research, I have now come up with a solution. For this you need a wrapper class. (here GenericResponse). I hope I can help others who have the same problem.
This is the Wrapper-Class
#Serializable(with = ListOrObjectSerializer::class)
class GenericResponse<T>(
val data: List<T> = emptyList()
) {
private var _isNothing : Boolean = false
val isNothing: Boolean
get() {
return this._isNothing
}
companion object {
fun <T> nothing(): GenericResponse<T> {
val o = GenericResponse(emptyList<T>())
o._isNothing = true
return o
}
}
}
And the Serializer looks like:
import kotlinx.serialization.KSerializer
import kotlinx.serialization.builtins.ListSerializer
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.*
class ListOrObjectSerializer<T : Any>(private val tSerializer: KSerializer<T>): KSerializer<GenericResponse<T>> {
override val descriptor: SerialDescriptor
get() = tSerializer.descriptor
override fun deserialize(decoder: Decoder): GenericResponse<T> {
val input = decoder as JsonDecoder
val jsonObj = input.decodeJsonElement()
return when(jsonObj) {
is JsonObject -> GenericResponse(listOf(Json.decodeFromJsonElement(tSerializer, jsonObj)))
is JsonArray -> GenericResponse(Json.decodeFromJsonElement(ListSerializer(tSerializer), jsonObj))
else -> return GenericResponse.nothing()
}
}
override fun serialize(encoder: Encoder, value: GenericResponse<T>) {
throw IllegalAccessError("serialize not supported")
}
}
My Data-Class look now like:
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
#Serializable
class ResourceResponse(
#SerialName("description")
val descriptions: GenericResponse<Description>? = null,
) {
#Serializable
data class Description(
#SerialName("value")
val value: String? = null,
#SerialName("lang")
val language: String? = null,
)
}
data class ResourceResponse(
#SerializedName("description") val descriptions: List<Description>,
)
data class Description(
#SerializedName("value") val value: String,
#SerializedName("lang") val language: String,
)
it should be like that
I have not very well structured JSON from BE (that I cannot change) which was formerly handled with moshi (custom adapter for the issue). Now I am trying to use pure kotlin serialization instead, but as said, JSON structure doesn't help me much.
{
"foo": {
"version":1,
"mask": [
{
"values": [
{ "bar": 1, ... }
]
},
{
"values": [
"important text i guess"
]
},
]
}
}
Now as you can see my issue is with values that can contains both object as well as string. All this should be parsed into kotlin where values looks like this:
data class Values(
val bar: Int? = null,
val text: String? = null,
...
)
Theoretically I can change local implementation, e.g. split the values class or something.
I've already tried to apply Polymorphic deserialization, but as I understand it was not able to recognise difference between two descendants of values without classDiscriminator set in JSON.
Any good advice?
Update
The polymorphic version I tried (in case I just made some error)
#Polymorphic
#Serializable
sealed class Values{
#Serializable
data class ObjectValues(
val bar: Int? = null,
...
)
#Serializable
data class TextValues(
val text: String? = null
)
}
and use it:
Json {
ignoreUnknownKeys = true
serializersModule = SerializersModule {
polymorphic(Values::class) {
subclass(Values.ObjectValues::class, Values.ObjectValues.serializer())
subclass(Values.TextValues::class, Values.TextValues.serializer())
}
}
error:
Polymorphic serializer was not found for missing class discriminator ('null')
JSON input: {"bar":42,...}
Disclaimer
I know that this would fix it all :)
"values": [ { "text":"important text i guess" } ]
Having classDiscriminator is not a mandatory requirement for polymorphic deserialization. You can use content-based polymorphic deserialization in this case.
All you need is to define a JsonContentPolymorphicSerializer for Values class with some logic for exact subclass serializer selection:
object ValuesSerializer : JsonContentPolymorphicSerializer<Values>(Values::class) {
override fun selectDeserializer(element: JsonElement) = when {
element.jsonObject["values"]!!.jsonArray.first() is JsonObject -> ObjectValues.serializer()
else -> TextValues.serializer()
}
}
and wire it as a serializer for Values class:
#Serializable(with = ValuesSerializer::class)
sealed class Values
#Serializable
data class TextValues(val values: List<String>) : Values()
#Serializable
data class ObjectValues(val values: List<Bar>) : Values()
#Serializable
data class Bar(val bar: Int)
#Serializable
data class MyJson(val foo: Foo)
#Serializable
data class Foo(val version: Int, val mask: List<Values>)
No need for serializersModule:
val result = Json.decodeFromString<MyJson>(json)
The problem I am trying to solve is perfectly described by the following text got from this link:
For a concrete example of when this could be useful, consider an API that supports partial updates of objects. Using this API, a JSON object would be used to communicate a patch for some long-lived object. Any included property specifies that the corresponding value of the object should be updated, while the values for any omitted properties should remain unchanged. If any of the object’s properties are nullable, then a value of null being sent for a property is fundamentally different than a property that is missing, so these cases must be distinguished.
That post presents a solution but using the kotlinx.serialization library, however, I must use gson library for now.
So I am trying to implement my own solution as I didn't find anything that could suit my use case (please let me know if there is).
data class MyObject(
val fieldOne: OptionalProperty<String> = OptionalProperty.NotPresent,
val fieldTwo: OptionalProperty<String?> = OptionalProperty.NotPresent,
val fieldThree: OptionalProperty<Int> = OptionalProperty.NotPresent
)
fun main() {
val gson = GsonBuilder()
.registerTypeHierarchyAdapter(OptionalProperty::class.java, OptionalPropertyDeserializer())
.create()
val json1 = """{
"fieldOne": "some string",
"fieldTwo": "another string",
"fieldThree": 18
}
"""
println("json1 result object: ${gson.fromJson(json1, MyObject::class.java)}")
val json2 = """{
"fieldOne": "some string",
"fieldThree": 18
}
"""
println("json2 result object: ${gson.fromJson(json2, MyObject::class.java)}")
val json3 = """{
"fieldOne": "some string",
"fieldTwo": null,
"fieldThree": 18
}
"""
println("json3 result object: ${gson.fromJson(json3, MyObject::class.java)}")
}
sealed class OptionalProperty<out T> {
object NotPresent : OptionalProperty<Nothing>()
data class Present<T>(val value: T) : OptionalProperty<T>()
}
class OptionalPropertyDeserializer : JsonDeserializer<OptionalProperty<*>> {
private val gson: Gson = Gson()
override fun deserialize(
json: JsonElement?,
typeOfT: Type?,
context: JsonDeserializationContext?
): OptionalProperty<*> {
println("Inside OptionalPropertyDeserializer.deserialize json:$json")
return when {
// Is it a JsonObject? Bingo!
json?.isJsonObject == true ||
json?.isJsonPrimitive == true-> {
// Let's try to extract the type in order
// to deserialize this object
val parameterizedType = typeOfT as ParameterizedType
// Returns an Present with the value deserialized
return OptionalProperty.Present(
context?.deserialize<Any>(
json,
parameterizedType.actualTypeArguments[0]
)!!
)
}
// Wow, is it an array of objects?
json?.isJsonArray == true -> {
// First, let's try to get the array type
val parameterizedType = typeOfT as ParameterizedType
// check if the array contains a generic type too,
// for example, List<Result<T, E>>
if (parameterizedType.actualTypeArguments[0] is WildcardType) {
// In case of yes, let's try to get the type from the
// wildcard type (*)
val internalListType = (parameterizedType.actualTypeArguments[0] as WildcardType).upperBounds[0] as ParameterizedType
// Deserialize the array with the base type Any
// It will give us an array full of linkedTreeMaps (the json)
val arr = context?.deserialize<Any>(json, parameterizedType.actualTypeArguments[0]) as ArrayList<*>
// Iterate the array and
// this time, try to deserialize each member with the discovered
// wildcard type and create new array with these values
val result = arr.map { linkedTreeMap ->
val jsonElement = gson.toJsonTree(linkedTreeMap as LinkedTreeMap<*, *>).asJsonObject
return#map context.deserialize<Any>(jsonElement, internalListType.actualTypeArguments[0])
}
// Return the result inside the Ok state
return OptionalProperty.Present(result)
} else {
// Fortunately it is a simple list, like Array<String>
// Just get the type as with a JsonObject and return an Ok
return OptionalProperty.Present(
context?.deserialize<Any>(
json,
parameterizedType.actualTypeArguments[0]
)!!
)
}
}
// It is not a JsonObject or JsonArray
// Let's returns the default state NotPresent.
else -> OptionalProperty.NotPresent
}
}
}
I got most of the code for the custom deserializer from here.
This is the output when I run the main function:
Inside OptionalPropertyDeserializer.deserialize json:"some string"
Inside OptionalPropertyDeserializer.deserialize json:"another string"
Inside OptionalPropertyDeserializer.deserialize json:18
json1 result object: MyObject(fieldOne=Present(value=some string), fieldTwo=Present(value=another string), fieldThree=Present(value=18))
Inside OptionalPropertyDeserializer.deserialize json:"some string"
Inside OptionalPropertyDeserializer.deserialize json:18
json2 result object: MyObject(fieldOne=Present(value=some string), fieldTwo=my.package.OptionalProperty$NotPresent#573fd745, fieldThree=Present(value=18))
Inside OptionalPropertyDeserializer.deserialize json:"some string"
Inside OptionalPropertyDeserializer.deserialize json:18
json3 result object: MyObject(fieldOne=Present(value=some string), fieldTwo=null, fieldThree=Present(value=18))
I am testing the different options for the fieldTwo and it is almost fully working, with the exception of the 3rd json, where I would expect that fieldTwo should be fieldTwo=Present(value=null) instead of fieldTwo=null.
And I see that in this situation, the custom deserializer is not even called for fieldTwo.
Can anyone spot what I am missing here? Any tip would be very appreciated!
I ended giving up of gson and move to moshi.
I implemented this behavior based on the solution presented in this comment.
Given json as follows where the structure of the payload object will vary:
{
"id": 1,
"displayName": "Success",
"payload": {
"someProperty": "example",
"someOtherProperty": {
"someNestedProperty": "example"
}
}
}
...using kotlinx.serialization how can I deserialize this into the following data class, where the value of payload should be the raw json string of the payload object.
#Serializable
data class Stub(
val id: Int,
val displayName: String,
val payload: String
)
Struggled to find a way of doing this with Serializers, but it was simple enough to implement manually using JsonElement.
val jsonObject = Json.parseToJsonElement(jsonString).jsonObject
val stub = Stub(
jsonObject["id"]!!.jsonPrimitive.int,
jsonObject["displayName"]!!.jsonPrimitive.content,
jsonObject["payload"]!!.toString()
)
There is a way to handle using JSONTransformingSerializer. It allows you to transform the json prior to deserialization. In this case from a jsonElement into a jsonPrimitive (of type String).
First create a transformer as follows:
object JsonAsStringSerializer: JsonTransformingSerializer<String>(tSerializer = String.serializer()) {
override fun transformDeserialize(element: JsonElement): JsonElement {
return JsonPrimitive(value = element.toString())
}
}
Now apply this transfer to the specific element in your data class by adding...
#Serializable(with = JsonAsStringSerializer::class)
just above the property you want to transform. Like this...
#Serializable
data class Stub(
val id: Int,
val displayName: String,
#Serializable(with = JsonAsStringSerializer::class)
val payload: String
)
The value of payload will be a string:
"{'someProperty': 'example','someOtherProperty': {'someNestedProperty':'example'}"
If you are later trying to deserialize this into different models depending on the structure, check out the JsonContentPolymorphicSerializer feature.
The JSON file I'm pulling from unfortunately has a node with the same variable name but could have two different data types randomly. When I make a network call (using gson) I get the error:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a BEGIN_ARRAY but was int at line 1 column 5344 path $[1].medium
the JSON looks like
{
"title": "Live JSON generator",
"url": google.com,
"medium": ["chicken", "radio", "room"]
}
//However sometimes medium can be:
"medium": 259
My Serialized class looks like:
data class SearchItem(
#SerializedName("title") var title: String,
#SerializedName("url") var urlStr: String,
#SerializedName("medium") val medium: List<String>? = null
) : Serializable {}
The way I'm making the network call is like this:
private val api: P1Api
fun onItemClicked(searchItem: SearchItem) {
api.getCollections { response, error ->
response.toString()
val searchItems: List<SearchItem> = Util.gson?.fromJson<List<SearchItem>>(
response.get("results").toString()
, object : TypeToken<List<SearchItem>>() {}.type)?.toList()!!
...
doStuffWithSearchItems(searchItems)
}
How do I handle both cases where "medium" can either be an array of strings or it could be an Int?
You could write custom JsonDeserializer for this case:
class SearchItemCustomDeserializer: JsonDeserializer<SearchItem> {
override fun deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): SearchItem {
val obj = json.asJsonObject
val title = obj.get("title").asString
val url = obj.get("url").asString
val mediumProp = obj.get("medium")
val medium = if(mediumProp.isJsonArray) {
mediumProp.asJsonArray.map { it.asString }
} else {
listOf(mediumProp.asString)
}
return SearchItem(
title = title,
urlStr = url,
medium = medium
)
}
}
With this class you "manually" deserialize json to object. For medium property we check is this array or simple json primitive with function mediumProp.isJsonArray. And if answer is yes - then deserialize field as json array of strings mediumProp.asJsonArray.map { it.asString } Else deserialize the field as string.
And then we register our custom SearchItemCustomDeserializer on GsonBuilder using method registerTypeAdapter
val gson = GsonBuilder()
.registerTypeAdapter(SearchItem::class.java, SearchItemCustomDeserializer())
.create()
And after this you can use this gson instance to deserialize yours objects