I want to use a class discriminator in Kotlin to get polymorphic deserialization of JSON messages. The problem is that the class discriminator in my messages is nested one level deep:
{
"header": {
"message_type": "FooBar"
}
"data": {...}
}
The contents of data depend on message_type. If message_type were one level higher, this would be trivial by setting #JsonClassDiscriminator("message_type") on the base class and #SerialName("FooBar") on the subclass. But since it is nested it is unclear how to solve this without introducing a ton of boilerplate serialization code.
Not the prettiest solution, but you can hack around this as such:
write your #Serializable object as if the discriminator is at the top level
when parsing, first use decodeFromString<JsonElement> to get a generic JSON object
use that object to move the discriminator to top-level
use decodeFromJsonElement to parse that
What step 3 looks like
// if json object has a message type in its header,
// return an object with that at top level so we can use it as a class discriminator
fun hoistDiscriminator(json: JsonElement): JsonElement {
if (json is JsonObject) {
val header = json["header"]
if (header is JsonObject) {
val discriminator = header["message_type"]
if (discriminator is JsonPrimitive && discriminator.isString) {
val map = json.toMutableMap()
map["message_type"] = discriminator
return JsonObject(map)
}
}
}
return json
}
and then the full parsing looks like this
json.decodeFromJsonElement(hoistDiscriminator(json.decodeFromString(response)))
There is a corner case in step 3: what if message_type is already defined at top-level? Handling that is left as an exercise to the reader.
Related
I would like to make a Kotlin class serializable using Kotlin serialization.
The class is very simple, something like this:
#Serializable(with = CustomSerializer::class)
data class MyObject(val keys: Set<String>)
Now, I need the serialization format to be a JSON object where the keys are given by the Set<String> and the values are always empty JSON objects.
Example:
val example = MyObject(setOf("abc", "def"))
Should serialize to:
{ "abc": {}, "def": {} }
The reason is that this object is being sent to an API where that's how they want the JSON to look like... the empty objects could contain some directives but I don't want or need to use those.
Having trouble doing that by just reading the documentation.
I've found one way to do it... and it seems simple enough!
I realized that I can get a serializer of empty Objects almost for free with this:
#Serializable
private object EmptyMap
Now, I can write a custom serializer in a straightforward way:
object MyObjectSerializer : KSerializer<MyObject> {
private val _delegate = MapSerializer(String.serializer(), EmptyMap.serializer())
override val descriptor: SerialDescriptor = _delegate.descriptor
override fun serialize(encoder: Encoder, value: MyObject) {
val data = value.keys.associateWith { EmptyMap }
encoder.encodeSerializableValue(_delegate, data)
}
override fun deserialize(decoder: Decoder): MyObject {
val value = decoder.decodeSerializableValue(_delegate)
return MyObject(value.keys)
}
}
Now all that's left to do is to apply the serializer on the type, which can be done with:
#Serializer(with = MyObjectSerializer)
data class MyObject(val keys: Set<String>)
Running Json.encodeToString(example) on the examples works perfectly.
Intro
I'm sending JSON messages between two backend servers that use different languages. The producing
server creates a variety of JSON messages, wrapped inside a message with metadata.
The wrapping class is Message, The consuming server has to determine which type of message its
receiving based solely on the message contents.
When I try to use a star-projection to
deserialize the message, I get an error.
Example
import kotlinx.serialization.json.Json
#Language("JSON")
val carJson = """
{
"message_type": "some message",
"data": {
"info_type": "Car",
"name": "Toyota"
}
}
""".trimIndent()
// normally I wouldn't know what the Json message would be - so the type is Message<*>
val actualCarMessage = Json.decodeFromString<Message<*>>(carJson)
Error message
Exception in thread "main" java.lang.IllegalArgumentException: Star projections in type arguments are not allowed, but Message<*>
at kotlinx.serialization.SerializersKt__SerializersKt.serializerByKTypeImpl$SerializersKt__SerializersKt(Serializers.kt:81)
at kotlinx.serialization.SerializersKt__SerializersKt.serializer(Serializers.kt:59)
at kotlinx.serialization.SerializersKt.serializer(Unknown Source)
at ExampleKt.main(example.kt:96)
at ExampleKt.main(example.kt)
Class structure
I want to deserialize JSON into a data class, Message, that has a field with a generic type.
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
#Serializable
data class Message<out DataType : SpecificInformation>(
#SerialName("message_type")
val type: String,
#SerialName("data")
val data: DataType,
)
The field is constrained by a sealed interface, SpecificInformation, with some implementations.
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.JsonClassDiscriminator
#JsonClassDiscriminator("info_type")
sealed interface SpecificInformation {
#SerialName("info_type")
val infoType: String
}
#Serializable
#SerialName("User")
data class UserInformation(
#SerialName("info_type")
override val infoType: String,
val name: String,
) : SpecificInformation
// there are more implementations...
Workaround?
This is a known
issue (kotlinx.serialization/issues/944)
,
so I'm looking for workarounds.
I have control over the JSON structure and libraries - though I have a preference for
kotlinx.serialization.
I can't change that there are two JSON objects, one is inside the other, and the discriminator is
inside the inner-class.
A custom serializer would be great. But I'd prefer to have this configured on the class or file
(with #Serializable(with = ...) or #file:UseSerializers(...)) as using a
custom SerializersModule is not as seamless.
Attempt: JsonContentPolymorphicSerializer
I've written a custom serializer, which only if it's used specifically (which is something I'd like
to avoid). It's also quite clunky, breaks if the data classes change or a new one is added, and
doesn't benefit from the sealed interface.
Can this be improved so that
It can be used generically? Json.decodeFromString<Message<*>>(carJson)
It doesn't have any hard-coded strings?
class MessageCustomSerializer : JsonContentPolymorphicSerializer<Message<*>>(Message::class) {
override fun selectDeserializer(element: JsonElement): DeserializationStrategy<out Message<*>> {
val discriminator = element
.jsonObject["data"]
?.jsonObject?.get("info_type")
?.jsonPrimitive?.contentOrNull
println("found discriminator $discriminator")
val subclassSerializer = when (discriminator?.lowercase()) {
"user" -> UserInformation.serializer()
"car" -> CarInformation.serializer()
else -> throw IllegalStateException("could not find serializer for $discriminator")
}
println("found subclassSerializer $subclassSerializer")
return Message.serializer(subclassSerializer)
}
}
fun main() {
#Language("JSON")
val carJson = """
{
"message_type": "another message",
"data": {
"info_type": "Car",
"brand": "Toyota"
}
}
""".trimIndent()
val actualCarMessage =
Json.decodeFromString(MessageCustomSerializer(), carJson)
val expectedCarMessage = Message("another message", CarInformation("Car", "Toyota"))
require(actualCarMessage == expectedCarMessage) {
println("car json parsing ❌")
}
println("car json parsing ✅")
}
#Serializable(with = ... - infinite loop
I tried applying MessageCustomSerializer directly to Message...
#Serializable(with = MessageCustomSerializer::class)
data class Message<out T : SpecificInformation>(
//...
But then I couldn't access the plugin-generated serializer, and this causes an infinite loop.
return Message.serializer(subclassSerializer) // calls 'MessageCustomSerializer', causes infinite loop
#Serializer(forClass = ...) - not generic
In addition to annotating Message with #Serializable(with = MessageCustomSerializer::class), I
tried
deriving a plugin-generated serializer:
#Serializer(forClass = Message::class)
object MessagePluginGeneratedSerializer : KSerializer<Message<*>>
But this serializer is not generic, and causes an error
java.lang.AssertionError: No such value argument slot in IrConstructorCallImpl: 0 (total=0).
Symbol: MessageCustomSerializer.<init>|-5645683436151566731[0]
at org.jetbrains.kotlin.ir.expressions.IrMemberAccessExpressionKt.throwNoSuchArgumentSlotException(IrMemberAccessExpression.kt:66)
at org.jetbrains.kotlin.ir.expressions.IrFunctionAccessExpression.putValueArgument(IrFunctionAccessExpression.kt:31)
at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.irInvoke(GeneratorHelpers.kt:210)
at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializableCompanionIrGenerator.irInvoke(SerializableCompanionIrGenerator.kt:35)
You are asking many things here, so I will simply try to give some pointers in regards to the errors you are making which you seem to be stuck on. With those in mind, and reading the documentation I link to, I believe you should be able to resolve the rest yourself.
Polymorphic serialization
Acquaint yourself with kotlinx.serialization polymorphic serialization. When you are trying to serialize Message<*> and DataType you are trying to use polymorphic serialization.
In case you are serializing Message<*> as the root object, specifying PolymorphicSerializer explicitly (as I also posted in the bug report you link to) should work. E.g., Json.decodeFromString( PolymorphicSerializer( Message::class ), carJson ).
P.s. I'm not 100% certain what you are trying to do here is the same as in the bug report. Either way, specifying the serializer explicitely should work, whether or not it is a bug that you shouldn't be required to do so.
The message_type and info_type fields you have in Message and DataType respectively are class discriminators. You need to configure this in your Json settings, and set the correct SerialName on your concrete classes for them to work. Using a different class discriminator per hierarchy is only possible starting from kotlinx.serialization 1.3.0 using #JsonClassDiscriminator.
Overriding plugin-generated serializer
But then I couldn't access the plugin-generated serializer, and this causes an infinite loop.
#Serializable(with = ...) overrides the plugin-generated serializer. If you want to retain the plugin-generated serializer, do not apply with.
When you are serializing the object directly (as the root object), you can still pass a different serializer to use as the first parameter to encode/decode. When you want to override the serializer to use for a specific property nested somewhere in the root object, use #Serializable on the property.
Polymorphism and generic classes
The "No such value argument slot in IrConstructorCallImpl: 0" error is to be expected.
You need to do more work in case you want to specify a serializer for polymorphic generic classes.
Hello I have a problem with my JSON. I am using OkHTTPClient to get JSON from web - to get objects from JSON using kotlinx.serialization via method which contains this and return value from method should be Result :
private suspend inline fun <reified T> OkHttpClient.get(webUrl: HttpUrl): Result<T> =
try {
//Builder defined here ... but skipping this line of code
val data = Json { ignoreUnknownKeys = true }.decodeFromString<T (result.body!!.string())
Result.Success(data)
} catch (e: Exception) {
Result.Failure(e)
}
suspend fun getFact(): Result<Fact> =
client.httpGet("myURL".toHttpUrl())
Json from myURL:
{"status":"success","data":[{"fact":"This is random information i need to get"}],"message":"Retrieved Fact"}
My serializer and Serializable data classes:
#Serializable
data class Fact(
#Serializable(with = FactListSerializer::class)
val data: String) java.io.Serializable
object FactListSerializer : JsonTransformingSerializer<List<String>>(ListSerializer(String.serializer())) {
override fun transformDeserialize(element: JsonElement): JsonElement {
return if (element is JsonArray) {
JsonArray(listOf(element)).first()
} else {
element
}
}
}
To be honest I am not sure what I am doing, but I am getting this error all the time when I print val fact = api.getFact():
Fact: Failure(error=kotlinx.serialization.json.internal.JsonDecodingException: Expected JsonPrimitive at 0, found {"fact":"This is random information i need to get"}
What I need to return is only first element of array fact, because JSON obtain always only 1 fact inside array. So I don't want to return from Serializer/Json List but only Fact object.
But as you see I am obtaining always Result Fauilure, don't know why. My goal is to obtain Result Success and obtaining from that JSON object Fact (only one), but I am not sure if I am doing it correct (obviously not) and even if it is even possible to return from JSONArray only one object (element of type Fact).
So what I expect is something like this:
Fact: Success(value=Fact(fact=This is random information i need to get))
I think the deserializer definition should be changed on 3 levels. The example of how to use JsonTransformingDeserializer in the docs actually describes most of what you need.
JsonArray(listOf(element)).first() should just be element.first(). Here you're building a JsonArray containing your initial JsonArray as only element, and then taking the first, so you basically get back the exact same element.
The type parameter T of JsonTransformingSerializer is supposed to be the type of the property it's applied to, so you should at least get a warning in the code because yours is defined to work on List<String> but is applied to a String property. It should be JsonTransformingSerializer<String>(String.serializer()).
You not only need to unwrap the data array, you also need to extract the value of the fact key within the element of that array.
So with all these changes, it should give something like this:
object FactListSerializer : JsonTransformingSerializer<String>(String.serializer()) {
override fun transformDeserialize(element: JsonElement): JsonElement {
val unwrappedData = if (element is JsonArray) element.first() else element
return unwrappedData.jsonObject["fact"] ?: error("missing 'fact' key in 'data' array")
}
}
I am a Scala newbie, extending someone else's code. The code uses the Play framework's JSON libraries. I am accessing objects of class Future[Option[A]] and Future[Option[List[B]]. The classes A and B each have their own JSON writes method, so each can return JSON as a response to a web request. I'm trying to combine these into a single JSON response that I can return as an HTTP response.
I thought creating a class which composes A and B into a single class would allow me to do this, something along these lines:
case class AAndB(a: Future[Option[A]], b: Future[Option[List[B]]])
object AAndB {
implicit val implicitAAndBWrites = Json.writes[AAndB]
}
But that fails all over the place. A and B are both structured like this:
sealed trait A extends SuperClass {
val a1: String = "identifier"
}
case class SubA(a2: ClassA2) extends A {
override val a1: String = "sub identifier"
}
object SubA {
val writes = Writes[SubA] { aa =>
Json.obj(
"a1" -> aa.a1
"a2" -> aa.a2
)
}
}
Since B is accessed as a List, the expected output would be along these lines:
{
"a":{
"a1":"val1",
"a2":"val2"
},
"b":[
{
"b1":"val 3",
"b2":"val 4"
},
{
"b1":"val 5",
"b2":"val 6"
},
{
"b1":"val 7",
"b2":"val 8"
}
]
}
Your help is appreciated.
As #cchantep mentioned in the comments on your question, having Futures as part of a case class declaration is highly unusual - case classes are great for encapsulating immutable domain objects (i.e that don't change over time) but as soon as you involve a Future[T] you potentially have multiple outcomes:
The Future hasn't completed yet
The Future failed
The Future completed successfully, and contains a T instance
You don't want to tangle up this temporal stuff with the act of converting to JSON. For this reason you should model your wrapper class with the Futures removed:
case class AAndB(a: Option[A], b: Option[List[B]])
object AAndB {
implicit val implicitAAndBWrites = Json.writes[AAndB]
}
and instead use Scala/Play's very concise handling of them in your Controller class to access the contents of each. In the below example, assume the existence of injected service classes as follows:
class AService {
def findA(id:Int):Future[Option[A]] = ...
}
class BListService {
def findBs(id:Int):Option[Future[List[B]]] = ...
}
Here's what our controller method might look like:
def showCombinedJson(id:Int) = Action.async {
val fMaybeA = aService.findA(id)
val fMaybeBs = bService.findBs(id)
for {
maybeA <- fMaybeA
maybeBs <- fMaybeBs
} yield {
Ok(Json.toJson(AAndB(maybeA, maybeBs)))
}
}
So here we launch both the A- and B-queries in parallel (we have to do this outside the for-comprehension to achieve this parallelism). The yield block of the for-comprehension will be executed only if/when both the Futures complete successfully - at which point it is safe to access the contents within. Then it's a simple matter of building an instance of the wrapper class, converting to JSON and returning an Ok result to Play.
Note that the result of the yield block will itself be inside a Future (in this case it's a Future[Result]) so we use Play's Action.async Action builder to handle this - letting Play deal with all of the actual waiting-for-things-to-happen.
In cases where I want to parse from JSON into a domain object I have defined to contain a collection of enums I have found that Groovy does not coerce the contents of the collection automatically, which I suppose shouldn't be expected anyway as generics are a compile-time concern.
If I naively do type coercion on the parsed JSON my collections will contain strings at runtime, which will make comparisons of the collection elements with enums fail regardless of value.
An alternative is to override the setter for the enum collection and do coercion on each element. This is illustrated in the below example.
import groovy.json.*
enum Hero {
BATMAN, ROBIN
}
class AClass {
Collection<Hero> heroes
}
class BClass {
Collection<Hero> heroes
void setHeroes(Collection heroes){
this.heroes = heroes.collect { it as Hero }
}
}
class CClass {
AClass a
BClass b
}
def json = '''
{
"a":
{
"heroes":["BATMAN", "ROBIN"]
},
"b":
{
"heroes":["BATMAN", "ROBIN"]
}
}
'''
def c = new JsonSlurper().parseText(json) as CClass
assert c.a.heroes[0].class == String
assert c.b.heroes[0].class == Hero
The overridden setter approach solves my problem, but it seems that it's a bit vanilla, and I was wondering if Groovy supports a smarter way of propagating the type coercion.
I don't know of a better way currently. We could write an external visitor kind of processor for the JsonSlurper result, but that would be more code