I am trying to serialise GeneralResponse:
case class GeneralResponse[T](succeeded: Boolean, payload: Option[T])
and the payload is GroupsForUserResult:
case class GroupsForUserResult(groups: Seq[UUID]).
I am using mapper.readValue(response.body, classOf[GeneralResponse[GroupsForUserResult]]) but unfortunately the payload is serialised as a Map and not as the desired case class (GroupForUserResult).
Because of Java Erasure - Jackson can't know at runtime about the generic type T from the line -
mapper.readValue(response.body, classOf[GeneralResponse[GroupsForUserResult]])
A solution to this problem will be
import com.fasterxml.jackson.core.`type`.TypeReference
mapper.readValue(json, new TypeReference[GeneralResponse[GroupsForUserResult]] {})
This way you provide an instance of TypeReference with all the needed Type information.
The accepted answer is close enough but you also have to provide the Type Parameter to .readValue method,
Working example with test,
import com.fasterxml.jackson.core.`type`.TypeReference
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import org.scalatest.{FunSuite, Matchers}
case class Customer[T](name: String, address: String, metadata: T)
case class Privileged(desc: String)
class ObjectMapperSpecs extends FunSuite with Matchers {
test("deserialises to case class") {
val objectMapper = new ObjectMapper()
.registerModule(DefaultScalaModule)
val value1 = new TypeReference[Customer[Privileged]] {}
val response = objectMapper.readValue[Customer[Privileged]](
"""{
"name": "prayagupd",
"address": "myaddress",
"metadata": { "desc" : "some description" }
}
""".stripMargin, new TypeReference[Customer[Privileged]] {})
response.metadata.getClass shouldBe classOf[Privileged]
response.metadata.desc shouldBe "some description"
}
}
The signature of com.fasterxml.jackson.databind.ObjectMapper#readValue,
public <T> T readValue(String content, TypeReference valueTypeRef)
throws IOException, JsonParseException, JsonMappingException
{
return (T) _readMapAndClose(_jsonFactory.createParser(content), _typeFactory.constructType(valueTypeRef));
}
If you don't provide the type parameter, it will blow up with error Customer cannot be cast to scala.runtime.Nothing$
Related
I'm using Kotlin to write an AWS Lambda. I have a Kotlin data class
class MessageObject(
val id: String,
val name: String,
val otherId: String
)
This data class is used as the input to the required interface implementation
class Handler : RequestHandler<MessageObject, Output> {
...
override fun handleRequest(msg: MessageObject, ctx: Context) {
...
}
}
When I test this lambda in the aws console, and pass it a proper JSON message, I get this:
An error occurred during JSON parsing: java.lang.RuntimeException
java.lang.RuntimeException: An error occurred during JSON parsing
Caused by: java.io.UncheckedIOException:
com.amazonaws.lambda.thirdparty.com.fasterxml.jackson.databind.exc.InvalidDefinitionException:
Cannot construct instance of 'com.mycode.MessageObject'(no Creators, like default construct, exist):
cannot deserialize from Object value (no delegate- or property-based Creator)
I'm almost certain this is fixed by saying:
ObjectMapper().registerModule(KotlinModule())
but in the world of AWS Lambda how do I edit the object mapper provided by AWS?
If you haven't gotten it to work with KotlinModule, since the problem you're having is that Jackson requires a default empty constructor and you currently don't have one. You could just change your MessageObject as follows and it should work:
data class MessageObject(
var id: String = "",
var name: String = "",
var otherId: String = ""
)
I created this repo with a fully functional kotlin lambda template using the Serverless Framework. Have a look for some other tidbits you might need: https://github.com/crafton/sls-aws-lambda-kotlin-gradlekt
You cannot use data class with provided RequestHandler<I, O> unfortunately, because you need register the kotlin module for your jackson mapper in order to work with data classes. But you can write you own RequestHandler, which will like this one.
Here's the code:
interface MyRequestStreamHandler<I : Any, O : Any?> : RequestStreamHandler {
val inputType: Class<I>
fun handleRequest(input: I, context: Context): O?
override fun handleRequest(inputStream: InputStream, outputStream: OutputStream, context: Context) {
handleRequest(inputStream.readJson(inputType), context).writeJsonNullable(outputStream)
}
interface MessageObjectRequestHandler : MyRequestStreamHandler< MessageObject, Output> {
override val inputType: Class<MessageObject >
get() = MessageObject::class.java
}
}
And jackson util:
private val objectMapper = jacksonObjectMapper()
.configure(JsonParser.Feature.ALLOW_COMMENTS, true)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.registerKotlinModule()
private val writer: ObjectWriter = objectMapper.writer()
fun <T : Any> readJson(clazz: Class<T>, stream: InputStream): T =
objectMapper.readValue(stream, clazz)
fun <T : Any> InputStream.readJson(clazz: Class<T>): T =
readJson(clazz, this)
fun Any?.writeJsonNullable(outputStream: OutputStream) {
if (this != null) writer.writeValue(outputStream, this)
}
Now, you can keep your MessageObject class to be data class, and your handler will look something like:
class LambdaMain : MessageObjectRequestHandler {
override fun handleRequest(input: MessageObject, context: Context): Output {
//...
}
}
I have created a sealed class for the json field Value under CustomAttribute data class. This field can return String or Array of Strings.
How can we deserialize this sealed class from json?
data class CustomAttribute (
val attributeCode: String,
val value: Value
)
sealed class Value {
class StringArrayValue(val value: List<String>) : Value()
class StringValue(val value: String) : Value()
}
One solution is to use a RuntimeTypeAdapterFactory as per the instructions in this answer
val valueTypeAdapter = RuntimeTypeAdapter.of(Value::class.java)
.registerSubtype(StringArrayValue::class.java)
.registerSubtype(StringValue::class.java)
val gson = GsonBuilder().registerTypeAdapter(valueTypeAdapter).create()
RuntimeTypeAdapter is included in the source code for Gson but not exposed as a Maven artifact.
It is designed to be copy/pasted into your project from here
I created a TypeAdapterFactory implementation specifically to support sealed classes and their subtypes. This works similarly to the RuntimeTypeAdapterFactory (and I used it as a guide to write my class), but will specifically only support sealed types, and will deserialize using object instances of objects with a sealed class supertype (RuntimeTypeAdapterFactory will create a new instance of object types, which breaks equality checks when a single instance is the expectation).
private class SealedTypeAdapterFactory<T : Any> private constructor(
private val baseType: KClass<T>,
private val typeFieldName: String
) : TypeAdapterFactory {
private val subclasses = baseType.sealedSubclasses
private val nameToSubclass = subclasses.associateBy { it.simpleName!! }
init {
if (!baseType.isSealed) throw IllegalArgumentException("$baseType is not a sealed class")
}
override fun <R : Any> create(gson: Gson, type: TypeToken<R>?): TypeAdapter<R>? {
if (type == null || subclasses.isEmpty() || subclasses.none { type.rawType.isAssignableFrom(it.java) }) return null
val elementTypeAdapter = gson.getAdapter(JsonElement::class.java)
val subclassToDelegate: Map<KClass<*>, TypeAdapter<*>> = subclasses.associateWith {
gson.getDelegateAdapter(this, TypeToken.get(it.java))
}
return object : TypeAdapter<R>() {
override fun write(writer: JsonWriter, value: R) {
val srcType = value::class
val label = srcType.simpleName!!
#Suppress("UNCHECKED_CAST") val delegate = subclassToDelegate[srcType] as TypeAdapter<R>
val jsonObject = delegate.toJsonTree(value).asJsonObject
if (jsonObject.has(typeFieldName)) {
throw JsonParseException("cannot serialize $label because it already defines a field named $typeFieldName")
}
val clone = JsonObject()
clone.add(typeFieldName, JsonPrimitive(label))
jsonObject.entrySet().forEach {
clone.add(it.key, it.value)
}
elementTypeAdapter.write(writer, clone)
}
override fun read(reader: JsonReader): R {
val element = elementTypeAdapter.read(reader)
val labelElement = element.asJsonObject.remove(typeFieldName) ?: throw JsonParseException(
"cannot deserialize $baseType because it does not define a field named $typeFieldName"
)
val name = labelElement.asString
val subclass = nameToSubclass[name] ?: throw JsonParseException("cannot find $name subclass of $baseType")
#Suppress("UNCHECKED_CAST")
return (subclass.objectInstance as? R) ?: (subclassToDelegate[subclass]!!.fromJsonTree(element) as R)
}
}
}
companion object {
fun <T : Any> of(clz: KClass<T>) = SealedTypeAdapterFactory(clz, "type")
}
}
Usage:
GsonBuilder().registerTypeAdapter(SealedTypeAdapterFactory.of(Value::class)).create()
I have successfully serialized and de-serialized a sealed class in the past, with a disclaimer of using Jackson, not Gson as my serialization engine.
My sealed class has been defined as:
#JsonTypeInfo(use = JsonTypeInfo.Id.MINIMAL_CLASS, include = JsonTypeInfo.As.PROPERTY, visible = true)
sealed class FlexibleResponseModel
class SnapshotResponse(val collection: List<EntityModel>): FlexibleResponseModel()
class DifferentialResponse(val collection: List<EntityModel>): FlexibleResponseModel()
class EventDrivenResponse(val collection: List<EntityEventModel>): FlexibleResponseModel()
class ErrorResponse(val error: String): FlexibleResponseModel()
With the annotations used, it required no further configuration for the Jackson instance to properly serialize and de-serialize instances of this sealed class granted that both sides of the communication possessed a uniform definition of the sealed class.
While I recognise that JsonTypeInfo is a Jackson-specific annotation, perhaps you might consider switching over from Gson if this feature is a must - or you might be able to find an equivalent configuration for Gson which would also include the class identifier in your serialized data.
I am struggling to understand how to parse an empty object {} with the experimental kotlinx.serialization library. The complication arises when in fact an API response can be one of;
{
"id": "ABC1",
"status": "A_STATUS"
}
or
{}
The data structure I have used as my serializer is;
data class Thing(val id: String = "", val status: String = "")
This is annotated with #kotlinx.serialization.Serializable and used within an API client library to marshall between the raw API response and the data model. The default values tell the serialisation library that the field is optional and replaces the #Optional approach of pre-Kotlin 1.3.30.
Finally, the kotlinx.serialization.json.Json parser I am using has the configuration applied by using the nonstrict template.
How do I define a serializer that can parse both an empty object and the expected data type with kotlinx.serialization? Do I need to write my own KSerialiser or is there config I am missing. Ideally, the empty object should be ignored/parsed as a null?
The error I get when parsing an empty object with my Thing data class is;
Field 'id' is required, but it was missing
So this was down to the kotlinCompilerClasspath having a different version kotlin (1.3.21, not 1.3.31).
Interestingly this was owing to advice I followed when configuring my gradle plugin project to not specify a version for the kotlin-dsl plugin.
Explicitly relying on the version I needed fixed the kotlinx.serialisation behavior (no changes to the mainline code)
Yes, ideally null instead of {} is way more convenient to parse but sometimes you just need to consume what backend sends you
There are 2 solutions that come to my mind.
Simpler, specific to your case using map:
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
class ThingMapSerializerTest {
#Test
fun `should deserialize to non empty map`() {
val thingMap: Map<String, String> =
Json.decodeFromString("""{"id":"ABC1","status":"A_STATUS"}""")
assertTrue(thingMap.isNotEmpty())
assertEquals("ABC1", thingMap["id"])
assertEquals("A_STATUS", thingMap["status"])
}
#Test
fun `should deserialize to empty map`() {
val thingMap: Map<String, String> = Json.decodeFromString("{}")
assertTrue(thingMap.isEmpty())
}
}
More complex but more general that works for any combinations of value types. I recommend sealed class with explicit empty value instead of data class with empty defaults:
import kotlinx.serialization.KSerializer
import kotlinx.serialization.SerializationException
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.descriptors.buildClassSerialDescriptor
import kotlinx.serialization.descriptors.serialDescriptor
import kotlinx.serialization.encoding.CompositeDecoder
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.encoding.decodeStructure
import kotlinx.serialization.json.Json
import org.junit.Assert.assertEquals
import org.junit.Test
class ThingSerializerTest {
#Test
fun `should deserialize to thing`() {
val thing: OptionalThing =
Json.decodeFromString(
OptionalThing.ThingSerializer,
"""{"id":"ABC1","status":"A_STATUS"}"""
)
assertEquals(OptionalThing.Thing(id = "ABC1", status = "A_STATUS"), thing)
}
#Test
fun `should deserialize to empty`() {
val thing: OptionalThing =
Json.decodeFromString(OptionalThing.ThingSerializer, "{}")
assertEquals(OptionalThing.Empty, thing)
}
sealed class OptionalThing {
data class Thing(val id: String = "", val status: String = "") : OptionalThing()
object Empty : OptionalThing()
object ThingSerializer : KSerializer<OptionalThing> {
override val descriptor: SerialDescriptor =
buildClassSerialDescriptor("your.app.package.OptionalThing") {
element("id", serialDescriptor<String>(), isOptional = true)
element("status", serialDescriptor<String>(), isOptional = true)
}
override fun deserialize(decoder: Decoder): OptionalThing {
decoder.decodeStructure(descriptor) {
var id: String? = null
var status: String? = null
loop# while (true) {
when (val index = decodeElementIndex(descriptor)) {
CompositeDecoder.DECODE_DONE -> break#loop
0 -> id = decodeStringElement(descriptor, index = 0)
1 -> status = decodeStringElement(descriptor, index = 1)
else -> throw SerializationException("Unexpected index $index")
}
}
return if (id != null && status != null) Thing(id, status)
else Empty
}
}
override fun serialize(encoder: Encoder, value: OptionalThing) {
TODO("Not implemented, not needed")
}
}
}
}
When 'Thing' is a field within json object:
"thing":{"id":"ABC1","status":"A_STATUS"} // could be {}
you can annotate property like that:
#Serializable(with = OptionalThing.ThingSerializer::class)
val thing: OptionalThing
Tested for:
classpath "org.jetbrains.kotlin:kotlin-serialization:1.4.10"
implementation "org.jetbrains.kotlinx:kotlinx-serialization-json:1.0.1"
I am trying to exclude the _passthroughFields property in the example below. When I use the debugger, it looks like my PropertyFilter is never used. What am I doing wrong?
import java.util
import com.fasterxml.jackson.databind.ser.impl.SimpleBeanPropertyFilter.SerializeExceptFilter
import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider
import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper, ObjectWriter}
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
import org.scalatest.{Matchers, WordSpec}
import scala.collection.immutable.Map
class PassthroughFieldsSpec extends WordSpec with Matchers {
"JacksonParser" when {
"given an Object and undesired fields" should {
"not include those fields in the json response" in {
trait Foo {
def id: String
def _passthroughFields: Map[String, String] = Map.empty
}
class Bar(val id: String, override val _passthroughFields: Map[String, String]) extends Foo
val item = new Bar("abcd", Map.empty)
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
val excludes = new util.HashSet[String](1)
excludes.add("_passthroughFields")
excludes.add("get_passthroughFields")
val filters = new SimpleFilterProvider()
.addFilter("filter properties by name", new SerializeExceptFilter(excludes))
val writer: ObjectWriter = mapper.writer(filters)
val json = writer.writeValueAsString(item)
json.contains("_passthroughFields") shouldBe false
}
}
}
}
I think you can exclude it using something like #JsonIgnore or make the field transient.
Otherwise if you need to define it outside of the case class code (like in your example) you can do it with Genson.
import com.owlike.genson._
import com.owlike.genson.reflect.VisibilityFilter
// Note that if you use case classes you don't need the visibility filter stuff
// it is used to indicate that private fields should be ser/de
val genson = new GensonBuilder()
.withBundle(ScalaBundle())
.useFields(true, VisibilityFilter.PRIVATE)
.exclude("_passthroughFields")
.create()
genson.toJson(item)
Disclaimer: I am Gensons author.
I'm new to Jackson. I am encountering the following error when using Jackson to deserialize a JSON string if I don't explicitly specify a class type during deserialization:
com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate abstract type [simple type, class scala.runtime.Nothing$] (need to add/enable type information?)
Is there a way to serialize / deserialize JSON in Jackson without having to specify the class type?
Here is my test code:
import java.lang.reflect.{Type, ParameterizedType}
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.`type`.TypeReference;
import org.junit._
import org.junit.Assert._
case class Person (name:String)
case class Address (city:String, state:String)
class JSONTest {
#Test
def jsonTest () = {
val p = new Person ("Bob")
val json = JacksonWrapper.serialize(p)
println ("json= " + json)
val obj = JacksonWrapper.deserialize[Person](json)
println ("obj = " + obj)
// fails since class type isn't explictly specified.
// is there a way to do it so that class type is automatically determined?
val obj2 = JacksonWrapper.deserialize(json)
println ("obj= " + obj2)
}
}
//http://stackoverflow.com/questions/12591457/scala-2-10-json-serialization-and-deserialization
object JacksonWrapper {
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
def serialize(value: Any): String = {
import java.io.StringWriter
val writer = new StringWriter()
mapper.writeValue(writer, value)
writer.toString
}
def deserialize[T: Manifest](value: String) : T =
mapper.readValue(value, typeReference[T])
private [this] def typeReference[T: Manifest] = new TypeReference[T] {
override def getType = typeFromManifest(manifest[T])
}
private [this] def typeFromManifest(m: Manifest[_]): Type = {
if (m.typeArguments.isEmpty) { m.erasure }
else new ParameterizedType {
def getRawType = m.erasure
def getActualTypeArguments = m.typeArguments.map(typeFromManifest).toArray
def getOwnerType = null
}
}
}
You have to provide the type information somehow. Jackson serializes to JSON only your object's fields, it doesn't store additional information about the type anywhere, so for your Person class, the JSON probably looks like this: { "name": "Bob"}. Without you providing the information that you want to deserialize as an instance of Person, Jackson can't known the type. You could have another class with the field name as well, and JSON doesn't say which one you need. That's why you need to provide the type when deserializing - in Java by passing an argument of type Class and in Scala by providing the type parameter to ObjectMapper and friends.