Deserializing a json object property to a String using kotlinx.serialization - json

Given json as follows where the structure of the payload object will vary:
{
"id": 1,
"displayName": "Success",
"payload": {
"someProperty": "example",
"someOtherProperty": {
"someNestedProperty": "example"
}
}
}
...using kotlinx.serialization how can I deserialize this into the following data class, where the value of payload should be the raw json string of the payload object.
#Serializable
data class Stub(
val id: Int,
val displayName: String,
val payload: String
)

Struggled to find a way of doing this with Serializers, but it was simple enough to implement manually using JsonElement.
val jsonObject = Json.parseToJsonElement(jsonString).jsonObject
val stub = Stub(
jsonObject["id"]!!.jsonPrimitive.int,
jsonObject["displayName"]!!.jsonPrimitive.content,
jsonObject["payload"]!!.toString()
)

There is a way to handle using JSONTransformingSerializer. It allows you to transform the json prior to deserialization. In this case from a jsonElement into a jsonPrimitive (of type String).
First create a transformer as follows:
object JsonAsStringSerializer: JsonTransformingSerializer<String>(tSerializer = String.serializer()) {
override fun transformDeserialize(element: JsonElement): JsonElement {
return JsonPrimitive(value = element.toString())
}
}
Now apply this transfer to the specific element in your data class by adding...
#Serializable(with = JsonAsStringSerializer::class)
just above the property you want to transform. Like this...
#Serializable
data class Stub(
val id: Int,
val displayName: String,
#Serializable(with = JsonAsStringSerializer::class)
val payload: String
)
The value of payload will be a string:
"{'someProperty': 'example','someOtherProperty': {'someNestedProperty':'example'}"
If you are later trying to deserialize this into different models depending on the structure, check out the JsonContentPolymorphicSerializer feature.

Related

Deserialize JSON with "non-stable" values using kotlin serialization

I have not very well structured JSON from BE (that I cannot change) which was formerly handled with moshi (custom adapter for the issue). Now I am trying to use pure kotlin serialization instead, but as said, JSON structure doesn't help me much.
{
"foo": {
"version":1,
"mask": [
{
"values": [
{ "bar": 1, ... }
]
},
{
"values": [
"important text i guess"
]
},
]
}
}
Now as you can see my issue is with values that can contains both object as well as string. All this should be parsed into kotlin where values looks like this:
data class Values(
val bar: Int? = null,
val text: String? = null,
...
)
Theoretically I can change local implementation, e.g. split the values class or something.
I've already tried to apply Polymorphic deserialization, but as I understand it was not able to recognise difference between two descendants of values without classDiscriminator set in JSON.
Any good advice?
Update
The polymorphic version I tried (in case I just made some error)
#Polymorphic
#Serializable
sealed class Values{
#Serializable
data class ObjectValues(
val bar: Int? = null,
...
)
#Serializable
data class TextValues(
val text: String? = null
)
}
and use it:
Json {
ignoreUnknownKeys = true
serializersModule = SerializersModule {
polymorphic(Values::class) {
subclass(Values.ObjectValues::class, Values.ObjectValues.serializer())
subclass(Values.TextValues::class, Values.TextValues.serializer())
}
}
error:
Polymorphic serializer was not found for missing class discriminator ('null')
JSON input: {"bar":42,...}
Disclaimer
I know that this would fix it all :)
"values": [ { "text":"important text i guess" } ]
Having classDiscriminator is not a mandatory requirement for polymorphic deserialization. You can use content-based polymorphic deserialization in this case.
All you need is to define a JsonContentPolymorphicSerializer for Values class with some logic for exact subclass serializer selection:
object ValuesSerializer : JsonContentPolymorphicSerializer<Values>(Values::class) {
override fun selectDeserializer(element: JsonElement) = when {
element.jsonObject["values"]!!.jsonArray.first() is JsonObject -> ObjectValues.serializer()
else -> TextValues.serializer()
}
}
and wire it as a serializer for Values class:
#Serializable(with = ValuesSerializer::class)
sealed class Values
#Serializable
data class TextValues(val values: List<String>) : Values()
#Serializable
data class ObjectValues(val values: List<Bar>) : Values()
#Serializable
data class Bar(val bar: Int)
#Serializable
data class MyJson(val foo: Foo)
#Serializable
data class Foo(val version: Int, val mask: List<Values>)
No need for serializersModule:
val result = Json.decodeFromString<MyJson>(json)

JSON decode nested field as Map[String, String] in Scala using circe

A circe noob here. I am trying to decode a JSON string to case class in Scala using circe. I want one of the nested fields in the input JSON to be decoded as a Map[String, String] instead of creating a separate case class for it.
Sample code:
import io.circe.parser
import io.circe.generic.semiauto.deriveDecoder
case class Event(
action: String,
key: String,
attributes: Map[String, String],
session: String,
ts: Long
)
case class Parsed(
events: Seq[Event]
)
Decoder[Map[String, String]]
val jsonStr = """{
"events": [{
"ts": 1593474773,
"key": "abc",
"action": "hello",
"session": "def",
"attributes": {
"north_lat": -32.34375,
"south_lat": -33.75,
"west_long": -73.125,
"east_long": -70.3125
}
}]
}""".stripMargin
implicit val eventDecoder = deriveDecoder[Event]
implicit val payloadDecoder = deriveDecoder[Parsed]
val decodeResult = parser.decode[Parsed](jsonStr)
val res = decodeResult match {
case Right(staff) => staff
case Left(error) => error
}
I am ending up with a decoding error on attributes field as follows:
DecodingFailure(String, List(DownField(north_lat), DownField(attributes), DownArray, DownField(events)))
I found an interesting link here on how to decode JSON string to a map here: Convert Json to a Map[String, String]
But I'm having little luck as to how to go about it.
If someone can point me in the right direction or help me out on this that will be awesome.
Let's parse the error :
DecodingFailure(String, List(DownField(geotile_north_lat), DownField(attributes), DownArray, DownField(events)))
It means we should look in "events" for an array named "attributes", and in this a field named "geotile_north_lat". This final error is that this field couldn't be read as a String. And indeed, in the payload you provide, this field is not a String, it's a Double.
So your problem has nothing to do with Map decoding. Just use a Map[String, Double] and it should work.
So you can do something like this:
final case class Attribute(
key: String,
value: String
)
object Attribute {
implicit val attributesDecoder: Decoder[List[Attribute]] =
Decoder.instance { cursor =>
cursor
.value
.asObject
.toRight(
left = DecodingFailure(
message = "The attributes field was not an object",
ops = cursor.history
)
).map { obj =>
obj.toList.map {
case (key, value) =>
Attribute(key, value.toString)
}
}
}
}
final case class Event(
action: String,
key: String,
attributes: List[Attribute],
session: String,
ts: Long
)
object Event {
implicit val eventDecoder: Decoder[Event] = deriveDecoder
}
Which you can use like this:
val result = for {
json <- parser.parse(jsonStr).left.map(_.toString)
obj <- json.asObject.toRight(left = "The input json was not an object")
eventsRaw <- obj("events").toRight(left = "The input json did not have the events field")
events <- eventsRaw.as[List[Event]].left.map(_.toString)
} yield events
// result: Either[String, List[Event]] = Right(
// List(Event("hello", "abc", List(Attribute("north_lat", "-32.34375"), Attribute("south_lat", "-33.75"), Attribute("west_long", "-73.125"), Attribute("east_long", "-70.3125")), "def", 1593474773L))
// )
You can customize the Attribute class and its Decoder, so their values are Doubles or Jsons.

Same SerializedName for two different data types sometimes, but with Kotlin

The JSON file I'm pulling from unfortunately has a node with the same variable name but could have two different data types randomly. When I make a network call (using gson) I get the error:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a BEGIN_ARRAY but was int at line 1 column 5344 path $[1].medium
the JSON looks like
{
"title": "Live JSON generator",
"url": google.com,
"medium": ["chicken", "radio", "room"]
}
//However sometimes medium can be:
"medium": 259
My Serialized class looks like:
data class SearchItem(
#SerializedName("title") var title: String,
#SerializedName("url") var urlStr: String,
#SerializedName("medium") val medium: List<String>? = null
) : Serializable {}
The way I'm making the network call is like this:
private val api: P1Api
fun onItemClicked(searchItem: SearchItem) {
api.getCollections { response, error ->
response.toString()
val searchItems: List<SearchItem> = Util.gson?.fromJson<List<SearchItem>>(
response.get("results").toString()
, object : TypeToken<List<SearchItem>>() {}.type)?.toList()!!
...
doStuffWithSearchItems(searchItems)
}
How do I handle both cases where "medium" can either be an array of strings or it could be an Int?
You could write custom JsonDeserializer for this case:
class SearchItemCustomDeserializer: JsonDeserializer<SearchItem> {
override fun deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): SearchItem {
val obj = json.asJsonObject
val title = obj.get("title").asString
val url = obj.get("url").asString
val mediumProp = obj.get("medium")
val medium = if(mediumProp.isJsonArray) {
mediumProp.asJsonArray.map { it.asString }
} else {
listOf(mediumProp.asString)
}
return SearchItem(
title = title,
urlStr = url,
medium = medium
)
}
}
With this class you "manually" deserialize json to object. For medium property we check is this array or simple json primitive with function mediumProp.isJsonArray. And if answer is yes - then deserialize field as json array of strings mediumProp.asJsonArray.map { it.asString } Else deserialize the field as string.
And then we register our custom SearchItemCustomDeserializer on GsonBuilder using method registerTypeAdapter
val gson = GsonBuilder()
.registerTypeAdapter(SearchItem::class.java, SearchItemCustomDeserializer())
.create()
And after this you can use this gson instance to deserialize yours objects

How to deserialize JSON with dynamic object?

I have a simple json, but the containing field has dynamic object. For instance, json can look like
{
"fixedField1": "value1",
"dynamicField1": {
"f1": "abc",
"f2": 123
}
}
or
{
"fixedField1": "value2",
"dynamicField1": {
"g1": "abc",
"g2": { "h1": "valueh1"}
}
}
I am trying to serialize this object, but not sure how to map the dynamic field
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
val dynamicField: Map<String, Any> // ???? what should be the type?
)
Above code fails with following error
Backend Internal error: Exception during code generation Cause:
Back-end (JVM) Internal error: Serializer for element of type Any has
not been found.
I ran into a similar problem when I had to serialize arbitrary Map<String, Any?>
The only way I managed to do this so far was to use the JsonObject/JsonElement API and combining it with the #ImplicitReflectionSerializer
The major downside is the use of reflection which will only work properly in JVM and is not a good solution for kotlin-multiplatform.
#ImplicitReflectionSerializer
fun Map<*, *>.toJsonObject(): JsonObject = JsonObject(map {
it.key.toString() to it.value.toJsonElement()
}.toMap())
#ImplicitReflectionSerializer
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Boolean -> JsonPrimitive(this)
is Map<*, *> -> this.toJsonObject()
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
is Array<*> -> JsonArray(this.map { it.toJsonElement() })
else -> {
//supporting classes that declare serializers
val jsonParser = Json(JsonConfiguration.Stable)
val serializer = jsonParser.context.getContextualOrDefault(this)
jsonParser.toJson(serializer, this)
}
}
Then, to serialize you would use:
val response = mapOf(
"fixedField1" to "value1",
"dynamicField1" to mapOf (
"f1" to "abc",
"f2" to 123
)
)
val serialized = Json.stringify(JsonObjectSerializer, response.toJsonObject())
Note
This reflection based serialization is only necessary if you are constrained to use Map<String, Any?>
If you are free to use your own DSL to build the responses, then you can use the json DSL directly, which is very similar to mapOf
val response1 = json {
"fixedField1" to "value1",
"dynamicField1" to json (
"f1" to "abc",
"f2" to 123
)
}
val serialized1 = Json.stringify(JsonObjectSerializer, response1)
val response 2 = json {
"fixedField1" to "value2",
"dynamicField1" to json {
"g1" to "abc",
"g2" to json { "h1" to "valueh1"}
}
}
val serialized2 = Json.stringify(JsonObjectSerializer, response2)
If, however you are constrained to define a data type, and do serialization as well as deserialization you probably can't use the json DSL so you'll have to define a #Serializer using the above methods.
An example of such a serializer, under Apache 2 license, is here: ArbitraryMapSerializer.kt
Then you can use it on classes that have arbitrary Maps. In your example it would be:
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
#Serializable(with = ArbitraryMapSerializer::class)
val dynamicField: Map<String, Any>
)

Jackson Serialize JSON array as String property

I am trying to ingest payloads from a remote web service using Jackson where the returned JSON contains properties that are defined as Arrays sometimes and plain Strings at other times.
I tried annotating the field using JsonRawValue but I am still getting the error:
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_ARRAY token
The following Kotlin classes are used to serialize the said JSON payload:
data class Device(
#get:JsonProperty("common.uuid")
val commonUuid: String? = null,
#field:JsonRawValue
#get:JsonProperty("user.ldap.groups.dn")
val userLdapGroupsDn: String? = null
)
data class DeviceSearchResults(
#JsonProperty("resultCount")
val resultCount: Int = 0,
#JsonProperty("totalCount")
val totalCount: Int = 0,
#JsonProperty("results")
val results: List<Device> = listOf()
)
.. and I put together the following unit test to show the error:
stubs-get-devices.json
{
"results": [
{
"common.uuid": "848ba0e8-d313-4df5-9a9e-3d9ea28951fa",
"user.ldap.groups.dn": [
"cn=qss-role-megamall-internet,ou=foo,ou=groups,dc=bar"
]
}
]
}
#Test
fun testJsonSerialization() {
val objectMapper = ObjectMapper().apply {
setSerializationInclusion(JsonInclude.Include.NON_EMPTY)
configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
registerModule(KotlinModule())
}
val file = File("stubs/get-devices.json")
val value = objectMapper.readValue(file, DeviceSearchResults::class.java)
assertThat(value).isNotNull
println(value)
}
Is there a way to coerce a JSON array value into a String property within the serialized Object?