Scala/Play: JSON serialization issue - json

I have a simple custom data structure which I use to map the results from the database:
case class Filter(id: Int, table: String, name: String, Type: String, structure: String)
The resulting object type is List[Filter] and if converted to JSON, it should look something like this:
[
{
"id": 1,
"table": "table1",
"name": "name1",
"Type": "type1",
"structure": "structure1"
},
{
"id": 2,
"table": "table2",
"name": "name2",
"Type": "type2",
"structure": "structure2"
}
]
Now when I try to serialize my object into JSON
val result: String = Json.toJson(filters)
I am getting something like
No Json deserializer found for type List[Filter]. Try to implement an implicit Writes or Format for this type.
How do I solve this seemingly simple problem without writing some ridiculous amount of boilerplate?
My stack is Play 2.2.1, Scala 2.10.3, Java 8 64bit

Short answer:
Just add:
implicit val filterWrites = Json.writes[Filter]
Longer answer:
If you look at the definition of Json.toJson, you will see that its complete signature is:
def toJson[T](o: T)(implicit tjs: Writes[T]): JsValue = tjs.writes(o)
Writes[T] knows how to take a T and transform it to a JsValue. You will need to have an implicit Writes[Filter] around that knows how to serialize your Filter instance. The good news is that Play's JSON library comes with a macro that can instantiate those Writes[_] for you, so you don't have to write boring code that transforms your case class's fields into JSON values. To invoke this macro and have its value picked up by implicit search add the line above to your scope.

Related

It is really tedious to write a bunch of data classes for parsing a simple JSON using Kotlin's seriallization library. Any better way?

I tried parsing JSON using Kotlin's default serialization library. However, I found it really overwhelming to write a bunch of data classes to deserialize a simple JSON string.
To illustrate,
{
"artists": {
"items": [
{
"genres": [
"desi pop",
"filmi",
"modern bollywood"
],
"images": [
{
"url": "https://i.scdn.co/image/ab6761610000e5ebb2b70762d89a9d76c772b3b6"
}
],
"name": "Arijit Singh",
"type": "artist"
}
]
}
}
for this data, I had to write these many classes,
#Serializable
data class Root(val artists: SubRoot)
#Serializable
data class SubRoot(val items: List<Artist>)
#Serializable
data class Artist(
val genres: List<String>,
val images: List<Image>,
val name: String,
val type: String
)
#Serializable
data class Image(val url: String)
Does anybody know a better way? Some library with in-built magic that does these kind of stuff for me?
If you don't want to use the automatic mapping you can just parse them as JsonElements and do your own thing instead of letting the library map them to those data classes.
https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/json.md#json-elements
For example, if you want to get that url, you could do:
val root = Json.parseToJsonElement(json)
return root.
jsonObject["artists"]?.
jsonObject?.get("items")?.
jsonArray?.get(0)?.
jsonObject?.get("images")?.
jsonArray?.get(0)?.
jsonObject?.get("url")?.
jsonPrimitive.toString()
)
This specific example will return null if any field couldn't be found while traversing the tree. It will give an IllegalArgumentException if any of the casts fail.

Kotlin - Array property in data class error

I'm modelling some JSON - and using the following lines
data class Metadata(
val id: String,
val creators: Array<CreatorsModel>
)
along with:
data class CreatorsModel (
val role: String,
val name: String
)
However keep seeing the error: Array property in data class error.
Any ideas why this is?
FYI, the JSON looks like:
{
"id": "123",
"creators": [{
"role": "Author",
"name": "Marie"
}
]
}
In Kotlin you should aim to use List instead of Array where possible. Array has some JVM implications, and although the compiler will let you, the IDE may prompt you to override equals and hashcode manually. Using List will make things much simpler.
You can find out more about the difference here: Difference between List and Array types in Kotlin

Circe Couldn't convert raw json to case class Error: could not find Lazy implicit value of type io.circe.generic.decoding.DerivedDecoder

I have defined a couple of case classes for JSON representation but I am not sure whether I did it properly as there a lot of nested case classes.
Entities like spec, meta and so on are of type JSONObject as well as the Custom object itself.
Here is all the classes I have defined:
case class CustomObject(apiVersion: String,kind: String, metadata: Metadata,spec: Spec,labels: Object,version: String)
case class Metadata(creationTimestamp: String, generation: Int, uid: String,resourceVersion: String,name: String,namespace: String,selfLink: String)
case class Spec(mode: String,image: String,imagePullPolicy: String, mainApplicationFile: String,mainClass: String,deps: Deps,driver: Driver,executor: Executor,subresources: Subresources)
case class Driver(cores: Double,coreLimit: String,memory: String,serviceAccount: String,labels: Labels)
case class Executor(cores: Double,instances: Double,memory: String,labels: Labels)
case class Labels(version: String)
case class Subresources(status: Status)
case class Status()
case class Deps()
And this is a JSON structure for the custom K8s object I need to transform:
{
"apiVersion": "sparkoperator.k8s.io/v1alpha1",
"kind": "SparkApplication",
"metadata": {
"creationTimestamp": "2019-01-11T15:58:45Z",
"generation": 1,
"name": "spark-example",
"namespace": "default",
"resourceVersion": "268972",
"selfLink": "/apis/sparkoperator.k8s.io/v1alpha1/namespaces/default/sparkapplications/spark-example",
"uid": "uid"
},
"spec": {
"deps": {},
"driver": {
"coreLimit": "1000m",
"cores": 0.1,
"labels": {
"version": "2.4.0"
},
"memory": "1024m",
"serviceAccount": "default"
},
"executor": {
"cores": 1,
"instances": 1,
"labels": {
"version": "2.4.0"
},
"memory": "1024m"
},
"image": "gcr.io/ynli-k8s/spark:v2.4.0,
"imagePullPolicy": "Always",
"mainApplicationFile": "http://localhost:8089/spark_k8s_airflow.jar",
"mainClass": "org.apache.spark.examples.SparkExample",
"mode": "cluster",
"subresources": {
"status": {}
},
"type": "Scala"
}
}
UPDATE:
I want to convert JSON into case classes with Circe, however, with such classes I face this error:
Error: could not find Lazy implicit value of type io.circe.generic.decoding.DerivedDecoder[dataModel.CustomObject]
implicit val customObjectDecoder: Decoder[CustomObject] = deriveDecoder[CustomObject]
I have defined implicit decoders for all case classes:
implicit val customObjectLabelsDecoder: Decoder[Labels] = deriveDecoder[Labels]
implicit val customObjectSubresourcesDecoder: Decoder[Subresources] = deriveDecoder[Subresources]
implicit val customObjectDepsDecoder: Decoder[Deps] = deriveDecoder[Deps]
implicit val customObjectStatusDecoder: Decoder[Status] = deriveDecoder[Status]
implicit val customObjectExecutorDecoder: Decoder[Executor] = deriveDecoder[Executor]
implicit val customObjectDriverDecoder: Decoder[Driver] = deriveDecoder[Driver]
implicit val customObjectSpecDecoder: Decoder[Spec] = deriveDecoder[Spec]
implicit val customObjectMetadataDecoder: Decoder[Metadata] = deriveDecoder[Metadata]
implicit val customObjectDecoder: Decoder[CustomObject] = deriveDecoder[CustomObject]
The reason you can't derive a decode for CustomObject is because of the labels: Object member.
In circe all decoding is driven by static types, and circe does not provide encoders or decoders for types like Object or Any, which have no useful static information.
If you change that case class to something like the following:
case class CustomObject(apiVersion: String, kind: String, metadata: Metadata, spec: Spec)
…and leave the rest of your code as is, with the import:
import io.circe.Decoder, io.circe.generic.semiauto.deriveDecoder
And define your JSON document as doc (after adding a quotation mark to the "image": "gcr.io/ynli-k8s/spark:v2.4.0, line to make it valid JSON), the following should work just fine:
scala> io.circe.jawn.decode[CustomObject](doc)
res0: Either[io.circe.Error,CustomObject] = Right(CustomObject(sparkoperator.k8s.io/v1alpha1,SparkApplication,Metadata(2019-01-11T15:58:45Z,1,uid,268972,spark-example,default,/apis/sparkoperator.k8s.io/v1alpha1/namespaces/default/sparkapplications/spark-example),Spec(cluster,gcr.io/ynli-k8s/spark:v2.4.0,Always,http://localhost:8089/spark_k8s_airflow.jar,org.apache.spark.examples.SparkExample,Deps(),Driver(0.1,1000m,1024m,default,Labels(2.4.0)),Executor(1.0,1.0,1024m,Labels(2.4.0)),Subresources(Status()))))
Despite what one of the other answers says, circe can definitely derive encoders and decoders for case classes with no members—that's definitely not the problem here.
As a side note, I wish it were possible to have better error messages than this:
Error: could not find Lazy implicit value of type io.circe.generic.decoding.DerivedDecoder[dataModel.CustomObject
But given the way circe-generic has to use Shapeless's Lazy right now, this is the best we can get. You can try circe-derivation for a mostly drop-in alternative for circe-generic's semi-automatic derivation that has better error messages (and some other advantages), or you can use a compiler plugin like splain that's specifically designed to give better error messages even in the presence of things like shapeless.Lazy.
As one final note, you can clean up your semi-automatic definitions a bit by letting the type parameter on deriveDecoder be inferred:
implicit val customObjectLabelsDecoder: Decoder[Labels] = deriveDecoder
This is entirely a matter of taste, but I find it a little less noisy to read.

Parse JSON array using Scala Argonaut

I'm using Scala & Argonaut, trying to parse the following JSON:
[
{
"name": "apple",
"type": "fruit",
"size": 3
},
{
"name": "jam",
"type": "condiment",
"size": 5
},
{
"name": "beef",
"type": "meat",
"size": 1
}
]
And struggling to work out how to iterate and extract the values into a List[MyType] where MyType will have name, type and size properties.
I will post more specific code soon (i have tried many things), but basically I'm looking to understand how the cursor works, and how to iterate through arrays etc. I have tried using \\ (downArray) to move to the head of the array, then :->- to iterate through the array, then --\ (downField) is not available (at least IntelliJ doesn't think so).
So the question is how do i:
navigate to the array
iterate through the array (and know when I'm done)
extract string, integer etc. values for each field - jdecode[String]? as[String]?
The easiest way to do this is to define a codec for MyType. The compiler will then happily construct a decoder for List[MyType], etc. I'll use a plain class here (not a case class) to make it clear what's happening:
class MyType(val name: String, val tpe: String, val size: Int)
import argonaut._, Argonaut._
implicit def MyTypeCodec: CodecJson[MyType] = codec3(
(name: String, tpe: String, size: Int) => new MyType(name, tpe, size),
(myType: MyType) => (myType.name, myType.tpe, myType.size)
)("name", "type", "size")
codec3 takes two parameter lists. The first has two parameters, which allow you to tell how to create an instance of MyType from a Tuple3 and vice versa. The second parameter list lets you specify the names of the fields.
Now you can just write something like the following (if json is your string):
Parse.decodeValidation[List[MyType]](json)
And you're done.
Since you don't need to encode and are only looking at decoding, you can do as suggested by Travis, but by implementing another implicit: MyTypeDecodeJson
implicit def MyTypeDecodeJson: DecodeJson[MyType] = DecodeJson(
raw => for {
name <- raw.get[String]("name")
type <- raw.get[String]("type")
size <- raw.get[Int]("size")
} yield MyType(name, type, size))
Then to parse your list:
Parse.decodeValidation[List[MyType]](jsonString)
Assuming MyType is a case class, the following works too:
case class MyType(name: String, type: String, size: Int)
object MyType {
implicit val createCodecJson: CodecJson[MyType] = CodecJson.casecodec3(apply, unapply)(
"name",
"type",
"size"
)
}

Parsing bad Json in Scala

I'm trying to parse some problematic Json in Scala using Play Json and using implicit, but not sure how to proceed...
The Json looks like this:
"rules": {
"Some_random_text": {
"item_1": "Some_random_text",
"item_2": "text",
"item_n": "MoreText",
"disabled": false,
"Other_Item": "thing",
"score": 1
},
"Some_other_text": {
"item_1": "Some_random_text",
"item_2": "text",
"item_n": "MoreText",
"disabled": false,
"Other_Item": "thing",
"score": 1
},
"Some_more_text": {
"item_1": "Some_random_text",
"item_2": "text",
"item_n": "MoreText",
"disabled": false,
"Other_Item": "thing",
"score": 1
}
}
I'm using an implicit reader but because each top level item in rules is effectively a different thing I don't know how to address that...
I'm trying to build a case class and I don't actually need the random text heading for each item but I do need each item.
To make my life even harder after these items are lots of things in other formats which I really don't need. They are unnamed items which just start:
{
random legal Json...
},
{
more Json...
}
I need to end up with the Json I'm parsing in a seq of case classes.
Thanks for your thoughts.
I'm using an implicit reader but because each top level item in rules is effectively a different thing I don't know how to address that...
Play JSON readers depend on knowing names of fields in advance. That goes for manually constructed readers and also for macro generated readers. You cannot use an implicit reader in this case. You need to do some traversing first and extract pieces of Json that do have regular structure with known names and types of fields. E.g. like this:
case class Item(item_1: String, item_2: String, item_n: String, disabled: Boolean, Other_Item: String, score: Int)
implicit val itemReader: Reads[Item] = Json.reads[Item]
def main(args: Array[String]): Unit = {
// parse JSON text and assume, that there is a JSON object under the "rules" field
val rules: JsObject = Json.parse(jsonText).asInstanceOf[JsObject]("rules").asInstanceOf[JsObject]
// traverse all fields, filter according to field name, collect values
val itemResults = rules.fields.collect {
case (heading, jsValue) if heading.startsWith("Some_") => Json.fromJson[Item](jsValue) // use implicit reader here
}
// silently ignore read errors and just collect sucessfully read items
val items = itemResults.flatMap(_.asOpt)
items.foreach(println)
}
Prints:
Item(Some_random_text,text,MoreText,false,thing,1)
Item(Some_random_text,text,MoreText,false,thing,1)
Item(Some_random_text,text,MoreText,false,thing,1)