I am trying to ingest payloads from a remote web service using Jackson where the returned JSON contains properties that are defined as Arrays sometimes and plain Strings at other times.
I tried annotating the field using JsonRawValue but I am still getting the error:
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_ARRAY token
The following Kotlin classes are used to serialize the said JSON payload:
data class Device(
#get:JsonProperty("common.uuid")
val commonUuid: String? = null,
#field:JsonRawValue
#get:JsonProperty("user.ldap.groups.dn")
val userLdapGroupsDn: String? = null
)
data class DeviceSearchResults(
#JsonProperty("resultCount")
val resultCount: Int = 0,
#JsonProperty("totalCount")
val totalCount: Int = 0,
#JsonProperty("results")
val results: List<Device> = listOf()
)
.. and I put together the following unit test to show the error:
stubs-get-devices.json
{
"results": [
{
"common.uuid": "848ba0e8-d313-4df5-9a9e-3d9ea28951fa",
"user.ldap.groups.dn": [
"cn=qss-role-megamall-internet,ou=foo,ou=groups,dc=bar"
]
}
]
}
#Test
fun testJsonSerialization() {
val objectMapper = ObjectMapper().apply {
setSerializationInclusion(JsonInclude.Include.NON_EMPTY)
configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
registerModule(KotlinModule())
}
val file = File("stubs/get-devices.json")
val value = objectMapper.readValue(file, DeviceSearchResults::class.java)
assertThat(value).isNotNull
println(value)
}
Is there a way to coerce a JSON array value into a String property within the serialized Object?
Related
I have this JSON to deserialize:
"data": {
"type": 18,
"msg": "firstName,lastName,15"
"timestamp": 1551770400000
}
I want to get this data in my model:
class DataDto(
type: String,
timestamp: Long,
msg: DataMsgDto?
) {
#JsonFormat(shape = JsonFormat.Shape.STRING)
#JsonPropertyOrder("firstName", "lastName", "age")
class DataMsgDto(
firstName: String,
lastName: String,
age: Long
)
}
I use this code to get data:
DataBuffer payload //this is what I get from server
val jsonNode = objectMapper.readTree(payload.toString(StandardCharsets.UTF_8))
objectMapper.treeToValue(jsonNode, DataDto::class.java)
But this does not work because in msg I don't have fields. So, how can I do this?
You have a JSON property that contains comma-separated values. The JSON message is valid, and can be deserialized by Jackson.
It's possible to write custom Jackson code that can handle this case automatically
How to deserialize a string separated by comma to list with Jackson commonly?
Jackson json property (string) to instance
But the most practical way to solve this is unlikely to be digging around trying to make Jackson understand msg, or writing a custom deserializer. Instead, allow msg to be a String
class DataDto(
val type: String,
val timestamp: Long,
// just use a normal String for msg
val msg: String?
)
And then manually decode it in Kotlin
fun parseDataDtoMsg(dto: DataDto): DataMsgDto? {
val msgElements = dto.msg?.split(",") ?: return null
// parse each message - you can chose whether to
// throw an exception if the `msg` does not match what you expect,
// or ignore invalid data and return `null`
val firstName = ...
val lastName = ...
val age = ...
return DataDtoMsg(
firstName = firstName,
lastName = lastName,
age = age,
)
}
data class DataMsgDto(
val firstName: String
val lastName: String
val age: Long,
)
The problem I am trying to solve is perfectly described by the following text got from this link:
For a concrete example of when this could be useful, consider an API that supports partial updates of objects. Using this API, a JSON object would be used to communicate a patch for some long-lived object. Any included property specifies that the corresponding value of the object should be updated, while the values for any omitted properties should remain unchanged. If any of the object’s properties are nullable, then a value of null being sent for a property is fundamentally different than a property that is missing, so these cases must be distinguished.
That post presents a solution but using the kotlinx.serialization library, however, I must use gson library for now.
So I am trying to implement my own solution as I didn't find anything that could suit my use case (please let me know if there is).
data class MyObject(
val fieldOne: OptionalProperty<String> = OptionalProperty.NotPresent,
val fieldTwo: OptionalProperty<String?> = OptionalProperty.NotPresent,
val fieldThree: OptionalProperty<Int> = OptionalProperty.NotPresent
)
fun main() {
val gson = GsonBuilder()
.registerTypeHierarchyAdapter(OptionalProperty::class.java, OptionalPropertyDeserializer())
.create()
val json1 = """{
"fieldOne": "some string",
"fieldTwo": "another string",
"fieldThree": 18
}
"""
println("json1 result object: ${gson.fromJson(json1, MyObject::class.java)}")
val json2 = """{
"fieldOne": "some string",
"fieldThree": 18
}
"""
println("json2 result object: ${gson.fromJson(json2, MyObject::class.java)}")
val json3 = """{
"fieldOne": "some string",
"fieldTwo": null,
"fieldThree": 18
}
"""
println("json3 result object: ${gson.fromJson(json3, MyObject::class.java)}")
}
sealed class OptionalProperty<out T> {
object NotPresent : OptionalProperty<Nothing>()
data class Present<T>(val value: T) : OptionalProperty<T>()
}
class OptionalPropertyDeserializer : JsonDeserializer<OptionalProperty<*>> {
private val gson: Gson = Gson()
override fun deserialize(
json: JsonElement?,
typeOfT: Type?,
context: JsonDeserializationContext?
): OptionalProperty<*> {
println("Inside OptionalPropertyDeserializer.deserialize json:$json")
return when {
// Is it a JsonObject? Bingo!
json?.isJsonObject == true ||
json?.isJsonPrimitive == true-> {
// Let's try to extract the type in order
// to deserialize this object
val parameterizedType = typeOfT as ParameterizedType
// Returns an Present with the value deserialized
return OptionalProperty.Present(
context?.deserialize<Any>(
json,
parameterizedType.actualTypeArguments[0]
)!!
)
}
// Wow, is it an array of objects?
json?.isJsonArray == true -> {
// First, let's try to get the array type
val parameterizedType = typeOfT as ParameterizedType
// check if the array contains a generic type too,
// for example, List<Result<T, E>>
if (parameterizedType.actualTypeArguments[0] is WildcardType) {
// In case of yes, let's try to get the type from the
// wildcard type (*)
val internalListType = (parameterizedType.actualTypeArguments[0] as WildcardType).upperBounds[0] as ParameterizedType
// Deserialize the array with the base type Any
// It will give us an array full of linkedTreeMaps (the json)
val arr = context?.deserialize<Any>(json, parameterizedType.actualTypeArguments[0]) as ArrayList<*>
// Iterate the array and
// this time, try to deserialize each member with the discovered
// wildcard type and create new array with these values
val result = arr.map { linkedTreeMap ->
val jsonElement = gson.toJsonTree(linkedTreeMap as LinkedTreeMap<*, *>).asJsonObject
return#map context.deserialize<Any>(jsonElement, internalListType.actualTypeArguments[0])
}
// Return the result inside the Ok state
return OptionalProperty.Present(result)
} else {
// Fortunately it is a simple list, like Array<String>
// Just get the type as with a JsonObject and return an Ok
return OptionalProperty.Present(
context?.deserialize<Any>(
json,
parameterizedType.actualTypeArguments[0]
)!!
)
}
}
// It is not a JsonObject or JsonArray
// Let's returns the default state NotPresent.
else -> OptionalProperty.NotPresent
}
}
}
I got most of the code for the custom deserializer from here.
This is the output when I run the main function:
Inside OptionalPropertyDeserializer.deserialize json:"some string"
Inside OptionalPropertyDeserializer.deserialize json:"another string"
Inside OptionalPropertyDeserializer.deserialize json:18
json1 result object: MyObject(fieldOne=Present(value=some string), fieldTwo=Present(value=another string), fieldThree=Present(value=18))
Inside OptionalPropertyDeserializer.deserialize json:"some string"
Inside OptionalPropertyDeserializer.deserialize json:18
json2 result object: MyObject(fieldOne=Present(value=some string), fieldTwo=my.package.OptionalProperty$NotPresent#573fd745, fieldThree=Present(value=18))
Inside OptionalPropertyDeserializer.deserialize json:"some string"
Inside OptionalPropertyDeserializer.deserialize json:18
json3 result object: MyObject(fieldOne=Present(value=some string), fieldTwo=null, fieldThree=Present(value=18))
I am testing the different options for the fieldTwo and it is almost fully working, with the exception of the 3rd json, where I would expect that fieldTwo should be fieldTwo=Present(value=null) instead of fieldTwo=null.
And I see that in this situation, the custom deserializer is not even called for fieldTwo.
Can anyone spot what I am missing here? Any tip would be very appreciated!
I ended giving up of gson and move to moshi.
I implemented this behavior based on the solution presented in this comment.
Given json as follows where the structure of the payload object will vary:
{
"id": 1,
"displayName": "Success",
"payload": {
"someProperty": "example",
"someOtherProperty": {
"someNestedProperty": "example"
}
}
}
...using kotlinx.serialization how can I deserialize this into the following data class, where the value of payload should be the raw json string of the payload object.
#Serializable
data class Stub(
val id: Int,
val displayName: String,
val payload: String
)
Struggled to find a way of doing this with Serializers, but it was simple enough to implement manually using JsonElement.
val jsonObject = Json.parseToJsonElement(jsonString).jsonObject
val stub = Stub(
jsonObject["id"]!!.jsonPrimitive.int,
jsonObject["displayName"]!!.jsonPrimitive.content,
jsonObject["payload"]!!.toString()
)
There is a way to handle using JSONTransformingSerializer. It allows you to transform the json prior to deserialization. In this case from a jsonElement into a jsonPrimitive (of type String).
First create a transformer as follows:
object JsonAsStringSerializer: JsonTransformingSerializer<String>(tSerializer = String.serializer()) {
override fun transformDeserialize(element: JsonElement): JsonElement {
return JsonPrimitive(value = element.toString())
}
}
Now apply this transfer to the specific element in your data class by adding...
#Serializable(with = JsonAsStringSerializer::class)
just above the property you want to transform. Like this...
#Serializable
data class Stub(
val id: Int,
val displayName: String,
#Serializable(with = JsonAsStringSerializer::class)
val payload: String
)
The value of payload will be a string:
"{'someProperty': 'example','someOtherProperty': {'someNestedProperty':'example'}"
If you are later trying to deserialize this into different models depending on the structure, check out the JsonContentPolymorphicSerializer feature.
The JSON file I'm pulling from unfortunately has a node with the same variable name but could have two different data types randomly. When I make a network call (using gson) I get the error:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a BEGIN_ARRAY but was int at line 1 column 5344 path $[1].medium
the JSON looks like
{
"title": "Live JSON generator",
"url": google.com,
"medium": ["chicken", "radio", "room"]
}
//However sometimes medium can be:
"medium": 259
My Serialized class looks like:
data class SearchItem(
#SerializedName("title") var title: String,
#SerializedName("url") var urlStr: String,
#SerializedName("medium") val medium: List<String>? = null
) : Serializable {}
The way I'm making the network call is like this:
private val api: P1Api
fun onItemClicked(searchItem: SearchItem) {
api.getCollections { response, error ->
response.toString()
val searchItems: List<SearchItem> = Util.gson?.fromJson<List<SearchItem>>(
response.get("results").toString()
, object : TypeToken<List<SearchItem>>() {}.type)?.toList()!!
...
doStuffWithSearchItems(searchItems)
}
How do I handle both cases where "medium" can either be an array of strings or it could be an Int?
You could write custom JsonDeserializer for this case:
class SearchItemCustomDeserializer: JsonDeserializer<SearchItem> {
override fun deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): SearchItem {
val obj = json.asJsonObject
val title = obj.get("title").asString
val url = obj.get("url").asString
val mediumProp = obj.get("medium")
val medium = if(mediumProp.isJsonArray) {
mediumProp.asJsonArray.map { it.asString }
} else {
listOf(mediumProp.asString)
}
return SearchItem(
title = title,
urlStr = url,
medium = medium
)
}
}
With this class you "manually" deserialize json to object. For medium property we check is this array or simple json primitive with function mediumProp.isJsonArray. And if answer is yes - then deserialize field as json array of strings mediumProp.asJsonArray.map { it.asString } Else deserialize the field as string.
And then we register our custom SearchItemCustomDeserializer on GsonBuilder using method registerTypeAdapter
val gson = GsonBuilder()
.registerTypeAdapter(SearchItem::class.java, SearchItemCustomDeserializer())
.create()
And after this you can use this gson instance to deserialize yours objects
I have a simple json, but the containing field has dynamic object. For instance, json can look like
{
"fixedField1": "value1",
"dynamicField1": {
"f1": "abc",
"f2": 123
}
}
or
{
"fixedField1": "value2",
"dynamicField1": {
"g1": "abc",
"g2": { "h1": "valueh1"}
}
}
I am trying to serialize this object, but not sure how to map the dynamic field
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
val dynamicField: Map<String, Any> // ???? what should be the type?
)
Above code fails with following error
Backend Internal error: Exception during code generation Cause:
Back-end (JVM) Internal error: Serializer for element of type Any has
not been found.
I ran into a similar problem when I had to serialize arbitrary Map<String, Any?>
The only way I managed to do this so far was to use the JsonObject/JsonElement API and combining it with the #ImplicitReflectionSerializer
The major downside is the use of reflection which will only work properly in JVM and is not a good solution for kotlin-multiplatform.
#ImplicitReflectionSerializer
fun Map<*, *>.toJsonObject(): JsonObject = JsonObject(map {
it.key.toString() to it.value.toJsonElement()
}.toMap())
#ImplicitReflectionSerializer
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Boolean -> JsonPrimitive(this)
is Map<*, *> -> this.toJsonObject()
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
is Array<*> -> JsonArray(this.map { it.toJsonElement() })
else -> {
//supporting classes that declare serializers
val jsonParser = Json(JsonConfiguration.Stable)
val serializer = jsonParser.context.getContextualOrDefault(this)
jsonParser.toJson(serializer, this)
}
}
Then, to serialize you would use:
val response = mapOf(
"fixedField1" to "value1",
"dynamicField1" to mapOf (
"f1" to "abc",
"f2" to 123
)
)
val serialized = Json.stringify(JsonObjectSerializer, response.toJsonObject())
Note
This reflection based serialization is only necessary if you are constrained to use Map<String, Any?>
If you are free to use your own DSL to build the responses, then you can use the json DSL directly, which is very similar to mapOf
val response1 = json {
"fixedField1" to "value1",
"dynamicField1" to json (
"f1" to "abc",
"f2" to 123
)
}
val serialized1 = Json.stringify(JsonObjectSerializer, response1)
val response 2 = json {
"fixedField1" to "value2",
"dynamicField1" to json {
"g1" to "abc",
"g2" to json { "h1" to "valueh1"}
}
}
val serialized2 = Json.stringify(JsonObjectSerializer, response2)
If, however you are constrained to define a data type, and do serialization as well as deserialization you probably can't use the json DSL so you'll have to define a #Serializer using the above methods.
An example of such a serializer, under Apache 2 license, is here: ArbitraryMapSerializer.kt
Then you can use it on classes that have arbitrary Maps. In your example it would be:
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
#Serializable(with = ArbitraryMapSerializer::class)
val dynamicField: Map<String, Any>
)