kotlinx deserialization: different types && scalar && arrays - json

I'm trying to deserialize a JSON like this (much more complex, but this is the essential part):
[
{
"field": "field1",
"value": [1000, 2000]
},
{
"field": "field2",
"value": 1
},
{
"field": "field2",
"value":["strval2","strval3"]
},
{
"field": "field4",
"value": "strval1"
}
]
I've tried to figure out how to use JsonContentPolymorphicSerializer in different variants but it all ends up the same:
class java.util.ArrayList cannot be cast to class myorg.ConditionValue (java.util.ArrayList is in module java.base of loader 'bootstrap'; myorg.ConditionValue is in unnamed module of loader 'app')
#Serializable
sealed class ConditionValue
#Serializable(with = StringValueSerializer::class)
data class StringValue(val value: String) : ConditionValue()
#Serializable(with = StringListValueSerializer::class)
data class StringListValue(val value: List<StringValue>) : ConditionValue()
object ConditionSerializer : JsonContentPolymorphicSerializer<Any>(Any::class) {
override fun selectDeserializer(element: JsonElement) = when (element) {
is JsonPrimitive -> StringValueSerializer
is JsonArray -> ListSerializer(StringValueSerializer)
else -> StringValueSerializer
}
}
object StringValueSerializer : KSerializer<StringValue> {
override val descriptor: SerialDescriptor = buildClassSerialDescriptor("StringValue")
override fun deserialize(decoder: Decoder): StringValue {
require(decoder is JsonDecoder)
val element = decoder.decodeJsonElement()
return StringValue(element.jsonPrimitive.content)
}
override fun serialize(encoder: Encoder, value: StringValue) {
encoder.encodeString(value.value)
}
}
What am I missing? And how to approach it?

This is indeed a difficult problem.
Probably the quickest and clearest way is to avoid getting bogged down with the 'correct' Kotlinx Serializer way and just decode the polymorphic type to a JsonElement
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
#Serializable
data class MyData(
val field: String,
val value: JsonElement, // polymorphism is hard, JsonElement is easy
)
The following code produces the correct output
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json
fun main() {
val json = /*language=json*/ """
[
{
"field": "field1",
"value": [1000, 2000]
},
{
"field": "field2",
"value": 1
},
{
"field": "field2",
"value":["strval2","strval3"]
},
{
"field": "field4",
"value": "strval1"
}
]
""".trimIndent()
val result = Json.decodeFromString<List<MyData>>(json)
println(result)
}
[
MyData(field=field1, value=[1000,2000]),
MyData(field=field2, value=1),
MyData(field=field2, value=["strval2","strval3"]),
MyData(field=field4, value="strval1")
]
Now you can manually convert MyData to a more correct instance.
val converted = results.map { result ->
val convertedValue: ConditionValue = when (val value = result.value) {
is JsonPrimitive -> convertPrimitive(value)
is JsonArray -> convertJsonArray(value)
else -> error("cannot convert $value")
}
MyDataConverted(
field = result.field,
value = convertedValue
)
}
...
fun convertJsonArray(array: JsonArray): ConditionValueList<*> =
TODO()
fun convertPrimitive(primitive: JsonPrimitive): ConditionValuePrimitive =
TODO()
As a final note, I can recommend using inline classes to represent your values. If you do want to work with Kotlinx Serialization, then they work better than creating custom serializers for primitive types.
Here's how I'd model the data in your example:
sealed interface ConditionValue
sealed interface ConditionValuePrimitive : ConditionValue
sealed interface ConditionValueCollection<T : ConditionValuePrimitive> : ConditionValue
#JvmInline
value class StringValue(val value: String) : ConditionValuePrimitive
#JvmInline
value class IntegerValue(val value: Int) : ConditionValuePrimitive
#JvmInline
value class ConditionValueList<T : ConditionValuePrimitive>(
val value: List<T>
) : ConditionValueCollection<T>
data class MyDataConverted(
val field: String,
val value: ConditionValue,
)
Versions:
Kotlin 1.7.10
Kotlinx Serialization 1.3.3

Related

How to serialize fields with varying type?

I have the following data classes to parse JSON. I can parse it easily with the decodeFromString method. However, the Info classes could contain the List<Int> type from time to time along with the Int type so that both are included in a single JSON. How can I handle this variation in serialization?
#Serializable
data class Node (#SerialName("nodeContent") val nodeContent: List<Info>)
#Serializable
data class Info (#SerialName("info") val info: Int)
p.s. The closest question to mine is this one: Kotlinx Serialization, avoid crashes on other datatype. I wonder if there are other ways?
EDIT:
An example is given below.
"nodeContent": [
{
"info": {
"name": "1",
},
},
{
"info": [
{
"name": "1"
},
{
"name": "2"
},
],
},
{
"info": {
"name": "2",
},
}
]
Here is an approach with a custom serializer similar to the link you provided. The idea is to return a list with just a single element.
// Can delete these two lines, they are only for Kotlin scripts
#file:DependsOn("org.jetbrains.kotlinx:kotlinx-serialization-json:1.2.0")
#file:CompilerOptions("-Xplugin=/snap/kotlin/current/lib/kotlinx-serialization-compiler-plugin.jar")
import kotlinx.serialization.*
import kotlinx.serialization.json.*
import kotlinx.serialization.encoding.Decoder
#Serializable
data class Node (val nodeContent: List<Info>)
#Serializable(with = InfoSerializer::class)
data class Info (val info: List<Name>)
#Serializable
data class Name (val name: Int)
#Serializer(forClass = Info::class)
object InfoSerializer : KSerializer<Info> {
override fun deserialize(decoder: Decoder): Info {
val json = ((decoder as JsonDecoder).decodeJsonElement() as JsonObject)
return Info(parseInfo(json))
}
private fun parseInfo(json: JsonObject): List<Name> {
val info = json["info"] ?: return emptyList()
return try {
listOf(Json.decodeFromString<Name>(info.toString()))
} catch (e: Exception) {
(info as JsonArray).map { Json.decodeFromString<Name>(it.toString()) }
}
}
}
Usage:
val ss2 = """
{
"nodeContent": [
{
"info":
{"name": 1}
},
{
"info": [
{"name": 1},
{"name": 2}
]
},
{
"info":
{"name": 2}
}
]
}
"""
val h = Json.decodeFromString<Node>(ss2)
println(h)
Result:
Node(nodeContent=[Info(info=[Name(name=1)]), Info(info=[Name(name=1), Name(name=2)]), Info(info=[Name(name=2)])])

Convert List to Json

How do I create a Json (Circe) looking like this:
{
"items": [{
"field1": "somevalue",
"field2": "somevalue2"
},
{
"field1": "abc",
"field2": "123abc"
}]
}
val result = Json.fromFields(List("items" -> ???))
You can do so using Circe's built in list typeclasses for encoding JSON. This code will work:
import io.circe.{Encoder, Json}
import io.circe.syntax._
case class Field(field1: String, field2: String)
object Field {
implicit val encodeFoo: Encoder[Field] = new Encoder[Field] {
final def apply(a: Field): Json = Json.obj(
("field1", Json.fromString(a.field1)),
("field2", Json.fromString(a.field2))
)
}
}
class Encoding(items: List[Field]) {
def getJson: Json = {
Json.obj(
(
"items",
items.asJson
)
)
}
}
If we instantiate an "Encoding" class and call getJson it will give you back the desired JSON. It works because with circe all you need to do to encode a list is provide an encoder for whatever is inside the list. Thus, if we provide an encoder for Field it will encode it inside a list when we call asJson on it.
If we run this:
val items = new Encoding(List(Field("jf", "fj"), Field("jfl", "fjl")))
println(items.getJson)
we get:
{
"items" : [
{
"field1" : "jf",
"field2" : "fj"
},
{
"field1" : "jfl",
"field2" : "fjl"
}
]
}

Alternative to parsing json with option [ either [A,B ] ] in scala

For example, here payload is optional and it has 3 variants:
How can I parse the json with types like option[either[A,B,C]] but to use abstract data type using things sealed trait or sum type?
Below is a minimal example with some boiler plate:
https://scalafiddle.io/sf/K6RUWqk/1
// Start writing your ScalaFiddle code here
val json =
"""[
{
"id": 1,
"payload" : "data"
},
{
"id": 2.1,
"payload" : {
"field1" : "field1",
"field2" : 5,
"field3" : true
}
},
{
"id": 2.2,
"payload" : {
"field1" : "field1",
}
},
{
"id": 3,
payload" : 4
},
{
"id":4,
"
}
]"""
final case class Data(field1: String, field2: Option[Int])
type Payload = Either[String, Data]
final case class Record(id: Int, payload: Option[Payload])
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
implicit final val dataDecoder: Decoder[Data] = deriveDecoder
implicit final val payloadDecoder: Decoder[Payload] = Decoder[String] either Decoder[Data]
implicit final val recordDecoder: Decoder[Record] = deriveDecoder
val result = io.circe.parser.decode[List[Record]](json)
println(result)
Your code is almost fine, you have just syntax issues in your json and Record.id should be Double instead of Int - because it is how this field present in your json ("id": 2.1). Please, find fixed version below:
val json =
s"""[
{
"id": 1,
"payload" : "data"
},
{
"id": 2.1,
"payload" : {
"field1" : "field1",
"field2" : 5,
"field3" : true
}
},
{
"id": 2.2,
"payload" : {
"field1" : "field1"
}
},
{
"id": 3,
"payload" : 4
},
{
"id": 4
}
]"""
type Payload = Either[String, Data]
final case class Data(field1: String, field2: Option[Int])
final case class Record(id: Double, payload: Option[Payload]) // id is a Double in your json in some cases
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
implicit val dataDecoder: Decoder[Data] = deriveDecoder
implicit val payloadDecoder: Decoder[Payload] = Decoder[String] either Decoder[Data]
implicit val recordDecoder: Decoder[Record] = deriveDecoder
val result = io.circe.parser.decode[List[Record]](json)
println(result)
Which produced in my case:
Right(List(Record(1.0,Some(Left(data))), Record(2.1,Some(Right(Data(field1,Some(5))))), Record(2.2,Some(Right(Data(field1,None)))), Record(3.0,Some(Left(4))), Record(4.0,None)))
UPDATE:
The more general approach would be to use so-called Sum Types or in simple words - general sealed trait with several different implementations. Please, see for more details next Circe doc page : https://circe.github.io/circe/codecs/adt.html
In your case it can be achieved something like this:
import cats.syntax.functor._
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
sealed trait Payload
object Payload {
implicit val decoder: Decoder[Payload] = {
List[Decoder[Payload]](
Decoder[StringPayload].widen,
Decoder[IntPayload].widen,
Decoder[ObjectPayload].widen
).reduce(_ or _)
}
}
case class StringPayload(value: String) extends Payload
object StringPayload {
implicit val decoder: Decoder[StringPayload] = Decoder[String].map(StringPayload.apply)
}
case class IntPayload(value: Int) extends Payload
object IntPayload {
implicit val decoder: Decoder[IntPayload] = Decoder[Int].map(IntPayload.apply)
}
case class ObjectPayload(field1: String, field2: Option[Int]) extends Payload
object ObjectPayload {
implicit val decoder: Decoder[ObjectPayload] = deriveDecoder
}
final case class Record(id: Double, payload: Option[Payload])
object Record {
implicit val decoder: Decoder[Record] = deriveDecoder
}
def main(args: Array[String]): Unit = {
val json =
s"""[
{
"id": 1,
"payload" : "data"
},
{
"id": 2.1,
"payload" : {
"field1" : "field1",
"field2" : 5,
"field3" : true
}
},
{
"id": 2.2,
"payload" : {
"field1" : "field1"
}
},
{
"id": 3,
"payload" : "4"
},
{
"id": 4
}
]"""
val result = io.circe.parser.decode[List[Record]](json)
println(result)
}
which produced in my case next output:
Right(List(Record(1.0,Some(StringPayload(data))), Record(2.1,Some(ObjectPayload(field1,Some(5)))), Record(2.2,Some(ObjectPayload(field1,None))), Record(3.0,Some(StringPayload(4))), Record(4.0,None)))
Hope this helps!

Unable to create converter for class when using sealed class or an interface with Moshi

I am trying to parse a json data from a server.It has dynamic keys so I am trying to have like a parent class that have the shared keys and child class for each specific node. I wrote a kotlin code using retrofit and Moshi but it's not working. I tried with a sealed class and interface without success. Actually I would prefer that works with sealed class but I don't know what I am doing wrong
interface MyApi {
#GET("/...")
fun fetchMyFeed(): Call<MyResponse>
}
data class MyResponse(
val data: List<ParentResponse>
)
interface ParentResponse{
val name: String
}
data class Child1Response(
val age: String,
val kids: List<KidsResponse>,
val cars: List<CarsResponse>
)
data class Child2Response(
val job: String,
val address: List<AddressResponse>
)
fun fetchAllFeed(): List<Any>? =
try {
val response = api.fetchMyFeed().execute()
if (response.isSuccessful) {
Log.d("check",${response.body()?.data?})
null
} else null
} catch (e: IOException) {
null
} catch (e: RuntimeException) {
null
}```
and the json file is :
{
"data": [
{
"name": "string",
"job": "string",
"address": [
{
"avenue": "string",
"imageUrl": "string",
"description": "string"
}
]
},
{
"name": "string",
"age": "string",
"kids": {
"count": "string",
"working": "string"
},
"cars": [
{
"brand": "string",
"age": "string",
"imageUrl": "string"
}
]
}
]
}
Unable to create converter for class
You can make use of JsonAdapter from moshi to parse different JSON Models if you can differentiate them by foreseeing some value in the json.
for example, consider json response having two schemas,
{
"root": {
"subroot": {
"prop" : "hello",
"type" : "String"
}
}
}
(or)
{
"root": {
"subroot": {
"prop" : 100,
"type" : "Integer"
}
}
}
Here, subroot has different schemas (one containing string property and another containg a integer property) which can be identified by "type"
You can create a parent sealed class with common keys and derive few child classes with varying keys. Write a adapter to select the type of class to be used while json serialization and add that adapter to moshi builder.
Model classes:
class Response {
#Json(name = "root")
val root: Root? = null
}
class Root {
#Json(name = "subroot")
val subroot: HybridModel? = null
}
sealed class HybridModel {
#Json(name = "type")
val type: String? = null
class StringModel : HybridModel() {
#Json(name = "prop")
val prop: String? = null
}
class IntegerModel : HybridModel() {
#Json(name = "prop")
val prop: Int? = null
}
}
Few extension methods to JsonReader,
inline fun JsonReader.readObject(process: () -> Unit) {
beginObject()
while (hasNext()) {
process()
}
endObject()
}
fun JsonReader.skipNameAndValue() {
skipName()
skipValue()
}
HybridAdapter to select type of class for "subroot" key
class HybridAdapter : JsonAdapter<HybridModel>() {
#FromJson
override fun fromJson(reader: JsonReader): HybridModel {
var type: String = ""
// copy reader and foresee type
val copy = reader.peekJson()
copy.readObject {
when (copy.selectName(JsonReader.Options.of("type"))) {
0 -> {
type = copy.nextString()
}
else -> copy.skipNameAndValue()
}
}
//handle exception if type cannot be identified
if (type.isEmpty()) throw JsonDataException("missing type")
// build model based on type
val moshi = Moshi.Builder().build()
return if (type == "String")
moshi.adapter(HybridModel.StringModel::class.java).fromJson(reader)!!
else
moshi.adapter(HybridModel.IntegerModel::class.java).fromJson(reader)!!
}
#ToJson
override fun toJson(p0: JsonWriter, p1: HybridModel?) {
// serialization logic
}
}
Finally build Moshi with the HybridAdapter to serialize HybridModel,
fun printProp(response: Response?) {
val subroot = response?.root?.subroot
when (subroot) {
is HybridModel.StringModel -> println("string model: ${subroot.prop}")
is HybridModel.IntegerModel -> println("Integer model: ${subroot.prop}")
}
}
fun main() {
val jsonWithStringSubroot =
"""
{
"root": {
"subroot": {
"prop" : "hello",
"type" : "String"
}
}
}
"""
val jsonWithIntegerSubroot =
"""
{
"root": {
"subroot": {
"prop" : 1,
"type" : "Integer"
}
}
}
"""
val moshi = Moshi.Builder().add(HybridAdapter()).build()
val response1 = moshi.adapter(Response::class.java).fromJson(jsonWithStringSubroot)
printProp(response1) // contains HybridModel.StringModel
val response2 = moshi.adapter(Response::class.java).fromJson(jsonWithIntegerSubroot)
printProp(response2) // contains HybridModel.IntegerModel
}

How to deserialize json with json-lenses

What is the best practice to deserialize JSON to a Scala case class using json-lenses?
some.json :
[
{
"id": 1,
"name": "Alice"
},
{
"id": 2,
"name": "Bob"
},
{
"id": 3,
"name": "Chris"
}
]
some case class :
case class Foo(id: Long, name: String)
What's best way to convert the json in some.json to List[Foo] ?
json-lenses supports spray-json and with spray-json you could do:
import spray.json._
case class Foo(id: Long, name: String)
object JsonProtocol extends DefaultJsonProtocol {
implicit val FooFormat = jsonFormat2(Foo)
}
import JsonProtocol._
val source = scala.io.Source.fromFile("some.json")
val json = try source.mkString.parseJson finally source.close()
json.convertTo[List[Foo]]
// List[Foo] = List(Foo(1,Alice), Foo(2,Bob), Foo(3,Chris))