Decode incomplete ADT with Circe - json

For a case class Apple(color:String, sweetness:Double) I can define a Decoder[String => Apple] via generic.(semi)auto or generic.extras.(semi)auto,
However for a sealed trait hierarchy (an ADT) I cannot:
sealed trait Fruit {
def color:String
}
case class Apple(color:String, sweetness:Double) extends Fruit
sealed trait SpecialFruit extends Fruit
case class Camachile(color:String, burstyness:Double) extends SpecialFruit
case class Langsat(color:String, transparency:Double) extends SpecialFruit
Decoder[String => Fruit] // <--- wont compile
How do I create such a Decoder?
Update
The reason I need such a decoder, is because
- The json I'm parsing does not contain all fields.
- The obtaining a decoder for the missing fields is non-trivial.
The last point makes it infeasible to go via Decoder[Fruit]

With decode[Fruit](jsonString), here is the example:
https://scalafiddle.io/sf/jvySm0B/0
There is a similar example on circe's home page: https://circe.github.io/circe/

Here is my attempt at an implementation. Warning: it's probably a bit off the coding standards in Circe, but it seems to work alright for my purposes. Nested sealed traits are also supported.
package no.kodeworks.kvarg.json
import io.circe.generic.extras.Configuration
import io.circe.{Decoder, HCursor}
import no.kodeworks.kvarg.util._
import shapeless.ops.function.FnFromProduct
import shapeless.ops.union.UnzipFields
import shapeless.{Coproduct, HList, LabelledGeneric, _}
trait AdtConfiguredIncompleteDecoders {
implicit def decodeIncompleteAdt[
Missing <: HList
, Adt
, Func
, Subtypes <: Coproduct
, SubtypeKeys <: HList
, SubtypeValues <: Coproduct
, ParamsSubtypes <: HList
, SubtypeFuncs <: HList
]
(implicit
func: FnFromProduct.Aux[Missing => Adt, Func],
sub: LabelledGeneric.Aux[Adt, Subtypes],
uz: UnzipFields.Aux[Subtypes, SubtypeKeys, SubtypeValues],
subtypeFuncs: SubtypeFunc.Aux[Missing, SubtypeValues, SubtypeFuncs],
configuration: Configuration = null
): Decoder[Func] = {
val conf = Option(configuration).getOrElse(Configuration.default)
val disc = conf.discriminator.getOrElse("type")
val keys = hlistToList[Symbol](uz.keys()).map(_.name)
.map(conf.transformConstructorNames)
val subtypeFuncs0 = hlistToList[Decoder[Func]](subtypeFuncs())
Decoder.withReattempt {
case h: HCursor =>
h.downField(disc).as[String] match {
case Right(decodedType) =>
val subtypeFuncDecoder = subtypeFuncs0(keys.indexOf(decodedType))
subtypeFuncDecoder(h)
}
}
}
trait SubtypeFunc[P <: HList, A <: Coproduct] extends DepFn0 with Serializable {
type Out <: HList
}
object SubtypeFunc {
type Aux[P <: HList, A <: Coproduct, Out0 <: HList] = SubtypeFunc[P, A] {
type Out = Out0}
implicit def cnilSubtypeFunc[P <: HList]: Aux[P, CNil, HNil] = new SubtypeFunc[P, CNil] {
type Out = HNil
override def apply(): HNil = HNil
}
implicit def cconsSubtypeFunc[
Missing <: HList
, CaseClass
, Func
, Rest <: Coproduct]
(implicit
func: FnFromProduct.Aux[Missing => CaseClass, Func],
subtypefuncHead: Decoder[Func],
subtypeFuncRest: SubtypeFunc[Missing, Rest],
): SubtypeFunc.Aux[Missing, CaseClass :+: Rest, Decoder[Func] :: subtypeFuncRest.Out] = {
new SubtypeFunc[Missing, CaseClass :+: Rest] {
type Out = Decoder[Func] :: subtypeFuncRest.Out
override def apply() =
subtypefuncHead :: subtypeFuncRest()
}
}
}
}
Snip from no.kodeworks.kvarg.util package object:
def hlistToList[T](hlist: shapeless.HList): List[T] = {
import shapeless._
hlist match {
case HNil => Nil
case head :: tail =>
collection.immutable.::(head.asInstanceOf[T], hlistToList[T](tail))
}
}

Related

json Generic decoder with default values using scala with circe

I have encountered a weird situation.
I m trying to build a method that takes a type and a JSON
and build it into a case class instance and if needed auto-complete missing key values.
So far I managed to do everything separately but not altogether.
The case class with its defaults:
case class Foo(a: String = "empty String", b: Option[Int] = Some(1))
and when I do the conversion:
import io.circe.generic.extras.auto._
import io.circe.generic.extras.Configuration
import io.circe.parser.decode
implicit val customConfig: Configuration = Configuration.default.withDefaults
println(decode[Foo]("{}"))
this is the output I get:
Right(Foo(empty String,Some(1)))
and this is working as I expected
but when I put it into a generic method it required a to be an option due to the error:
Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: DecodingFailure(Attempt to decode value on failed cursor, List(DownField(a)))
so I`m changing the case class to be
case class Foo(a: Option[String] = Some("empty String"), b: Option[Int] = Some(1))
and add the decoder:
object Foo{
implicit val decoder:Decoder[Foo] = deriveDecoder[Foo]
}
to the method:
import io.circe.Decoder
import io.circe.parser.decode
def convertToObj[T](jsonStr: String)(implicit decoder: Decoder[T]): T = {
decode[T](jsonStr)
match {
case Right(value) => value
case Left(error) => throw error
}
}
println(convertToObj[Foo]("{}"))
and the output is:
Foo(None,None)
so now I have lost my default values that I put and not able to use the automatic decoder as well.
How can I combine my two wishes into one approach?
You would need to do something like:
package foo.bar
import io.circe.Decoder
import io.circe.generic.extras.semiauto
import io.circe.generic.extras.Configuration
import io.circe.parser.decode
case class Foo(a: String = "empty String", b: Option[Int] = Some(1))
object Foo {
implicit val customConfig: Configuration = Configuration.default.withDefaults
implicit val decoder: Decoder[Foo] = semiauto.deriveConfiguredDecoder[Foo]
}
object TestApp extends App {
def convertToObj[T](jsonStr: String)(implicit decoder: Decoder[T]): T =
decode[T](jsonStr) match {
case Right(value) => value
case Left(error) => throw error
}
println(convertToObj[Foo]("{}"))
}
However, you can have circe automatically derive your decoder for you, so you can get away with less boilerplate:
package foo.bar
import io.circe.Decoder
import io.circe.generic.extras.auto._
import io.circe.generic.extras.Configuration
import io.circe.parser.decode
case class Foo(a: String = "empty String", b: Option[Int] = Some(1))
object TestApp extends App {
implicit val customConfig: Configuration = Configuration.default.withDefaults
def convertToObj[T](jsonStr: String)(implicit decoder: Decoder[T]): T =
decode[T](jsonStr) match {
case Right(value) => value
case Left(error) => throw error
}
println(convertToObj[Foo]("{}"))
}
Both of these examples give me output: Foo(empty String,Some(1))
NOTE:
method deriveDecoder in object semiauto is deprecated (since 0.12.0): Use deriveConfiguredDecoder

convert json to array of scala objects using spray json

I am not much familier with spray json, but I have to convert the below json into Array[myTest]
Below is the code, but it doesnt work. It throws the following errors: How do I fix them?
Error:(19, 54) Cannot find JsonReader or JsonFormat type class for Array[A$A61.this.myTest]
lazy val converted= trainingDataRef.toJson.convertTo[Array[myTest]]
^
Error:(19, 54) not enough arguments for method convertTo: (implicit evidence$1: spray.json.JsonReader[Array[A$A61.this.myTest]])Array[A$A61.this.myTest].
Unspecified value parameter evidence$1.
lazy val converted= trainingDataRef.toJson.convertTo[Array[myTest]]
^
Error:(10, 61) could not find implicit value for evidence parameter of type spray.json.DefaultJsonProtocol.JF[Map[String,Any]]
implicit val format: RootJsonFormat[myTest] = jsonFormat3(myTest.apply)
^
Code: ^
import spray.json.DefaultJsonProtocol._
import spray.json._
case class myTest (
id: String,
classDetails: Map[String, Any],
school: Map[String, Any])
object myTest {
implicit val format: RootJsonFormat[myTest] = jsonFormat3(myTest.apply)
}
val trainingDataRef = """[{"id":"my-id","classDetails":{"sec":"2","teacher":"John"},"school":{"name":"newschool"}}]"""
println(trainingDataRef.getClass)
val converted= trainingDataRef.toJson.convertTo[Array[myTest]]
println(converted)
spray-json has good documentation, try take a look there. Basically, you have to define your case classes and implement JsonFormat for them:
import spray.json.DefaultJsonProtocol._
import spray.json._
case class ClassDetails(sec: String, teacher: String)
object ClassDetails {
implicit val format: RootJsonFormat[ClassDetails] = jsonFormat2(ClassDetails.apply)
}
case class School(name: String)
object School {
implicit val format: RootJsonFormat[School] = jsonFormat1(School.apply)
}
case class ClassInfo
(
id: String,
classDetails: ClassDetails,
school: School
)
object ClassInfo {
implicit object ClassInfoFormat extends RootJsonFormat[ClassInfo] {
def write(c: ClassInfo): JsValue = JsObject(
"id" -> JsString(c.id),
"classDetails" -> c.classDetails.toJson,
"school" -> c.school.toJson
)
def read(value: JsValue): ClassInfo = {
value.asJsObject.getFields("id", "classDetails", "school") match {
case Seq(JsString(name), details, school) =>
new ClassInfo(name, details.convertTo[ClassDetails], school.convertTo[School])
case _ => throw new DeserializationException("ClassInfo expected")
}
}
}
}
val json = """[{"id":"my-id","classDetails":{"sec":"2","teacher":"John"},"school":{"name":"newschool"}}]"""
// JSON string to case classes
val classInfos = json.parseJson.convertTo[Seq[ClassInfo]]
classInfos.zipWithIndex.foreach { case (c, idx) =>
println(s"$idx => $c")
}
println
// Seq[ClassInfo] to JSON
println(s"$classInfos: ")
println(classInfos.toJson.prettyPrint)

No instance of play.api.libs.json.Format is available for scala.Predef.Map[java.lang.String, scala.Option[scala.Double]]

Trying to write a json format for an entity which contains a Map of Option. It throws following error
Error:(8, 68) No instance of play.api.libs.json.Format is available for scala.Predef.Map[java.lang.String, scala.Option[scala.Double]] in the implicit scope (Hint: if declared in the same file, make sure it's declared before)
Code snippet:
import play.api.libs.json.{Json, OFormat}
val a: Map[String, Option[Double]] = Map("a" -> None)
case class Person(x: Map[String, Option[Double]])
object Person {
implicit val personFormat: OFormat[Person] = Json.format[Person]
}
Json.toJson(Person(a))
You can define implicits:
import scala.collection.Map
object Person {
implicit object MapReads extends Reads[Map[String, Option[Double]]] {
def reads(jsValue: JsValue): JsResult[Map[String, Option[Double]]] = jsValue match {
case JsObject(map) => JsSuccess(
map.mapValues {
case JsNumber(d) => Some(d.toDouble)
case _ => None
}
)
case _ => JsError()
}
}
implicit object MapWrites extends Writes[Map[String, Option[Double]]] {
def writes(map: Map[String, Option[Double]]): JsValue =
JsObject(map.mapValues(optd => JsNumber(optd.getOrElse[Double](0.0))))
}
implicit val personFormat: OFormat[Person] = Json.format[Person]
}
println(Json.toJson(Person(a)))//{"x":{"a":0}}
Based on answer.
The problem seems to be the inability of play-json macros to work with nested type variables:
Map[String, Option[Double]]
You could use an intermediate type wrapping Option
import play.api.libs.json.{Json, OFormat}
case class OptionDouble(value: Option[Double])
case class Person(x: Map[String, OptionDouble])
implicit val optionDoubleFormat = Json.format[OptionDouble]
implicit val personFormat = Json.format[Person]
val a: Map[String, OptionDouble] = Map("a" -> OptionDouble(None))
Json.toJson(Person(a))
Another option could be to write the formatter by hand.

Play Framework / Scala: abstract repository and Json de/serialization

This question is maybe more about Scala than Play, but here it is: I am trying to achieve an abstraction of a repository for common DB operations.
trait Entity {
def id: UUID
}
trait Repository[T <: Entity] {
val JSON_KEY_ID = "_id"
def collection: JSONCollection
def insert(t: T): Future[Either[String, T]] = {
collection.insert(t).map(wr => if (wr.ok) Right(t) else Left(wr.getMessage()))
.recover { case t => Left(t.getMessage) }
}
def update(t: T): Future[Either[String, T]] = {
val selector = Json.obj(JSON_KEY_ID -> t.id.toString)
collection.update(selector, t).map(wr => if (wr.ok) Right(t) else Left(wr.getMessage()))
.recover { case t => Left(t.getMessage) }
}
}
Then I have the objects I would like to use this with:
case class UserDAO(id: UUID) extends Entity[UserDAO]
object UserDAO {
val JSON_KEY_ID = "_id"
implicit val userDaoWrites: OWrites[UserDAO] = new OWrites[UserDAO] {
def writes(user: UserDAO): JsObject = Json.obj(
JSON_KEY_ID -> JsString(user.id.toString)
)
}
implicit val userDaoReads: Reads[UserDAO] = (
(__ \ JSON_KEY_ID).read[UUID]
)(UserDAO.apply _)
}
and then I define its repository like this:
class UserRepository #Inject()(val reactiveMongoApi: ReactiveMongoApi) extends Repository[UserDAO] {
val db = reactiveMongoApi.db
override def collection: JSONCollection = db.collection[JSONCollection]("users")
}
The error I get is
No Json serializer as JsObject found for type T. Try to implement an implicit OWrites or OFormat for this type.
collection.insert(t).map(wr => if (wr.ok) Right(t) else Left(wr.getMessage()))
^
I tried to provide implicit OWrites[T], or even implicit OWrites[_] to no avail. Maybe what I am trying to achieve is impossible. If not, how could I solve this? Thank you very much.
You should be able to just use a context bound.
trait Entity {
def id: UUID
}
class Repository[T <: Entity : Writes] {
...
}
That will ensure that if there exists an implicit Writes[T] in scope, it will be available for your insert and update functions.

Scala (spray) json serialization and deserialization issue

I'm using my own implicit implementation of JSON serializer and deserializer for my case object
My case class looks like (it's just a code snippet)
sealed trait MyTrait
case object MyCaseClass extends MyTrait
I want to write my own ser. and deser. of JSON for MyTrait
implicit val myTraitFormat = new JsonFormat[MyTrait] {
override def read(json: JsValue): MyTrait = json.asJsObject.getFields("value") match {
case Seq(JsString("TEST")) ⇒ MyCaseClass
case _ ⇒ throw new DeserializationException(s"$json is not a valid extension of my trait")
}
override def write(myTrait: MyTrait): JsValue = {
myTrait match {
case MyCaseClass => JsObject("value" -> JsString("TEST"))
}
}
}
Now my test is failing by throwing DeserializationException:
"The my JSON format" when {
"deserializing a JSON" must {
"return correct object" in {
val json = """{"value": "TEST"}""".asJson
json.convertTo[MyTrait] must equal (MyCaseClass)
}
}
}
Obviously json.asJsObject.getFields("value")can not be matched to Seq(JsString("TEST")). Maybe this is related to using traits?
But I have found example on official spray-json site https://github.com/spray/spray-json#providing-jsonformats-for-other-types
Any ideas how to properly match on field in JsObject?
Thanks!
Best
Write variable name instead of string:
override def read(json: JsValue): MyCaseClass = json.asJsObject.getFields("value") match {
case Seq(JsString(value)) ⇒ MyCaseClass
case _ ⇒ throw new DeserializationException(s"$json is not a valid case class")
}
Does that work for you?
Update:
Yes, your test is wrong. I tried it with the following (note parseJson instead of asJson) and it worked:
scala> val json = """{"value": "TEST"}""".parseJson
json: spray.json.JsValue = {"value":"TEST"}
scala> json.convertTo[MyCaseClass]
res2: MyCaseClass = MyCaseClass()
Update 2: Tried it with trait:
scala> import spray.json._
import spray.json._
scala> import spray.json.DefaultJsonProtocol._
import spray.json.DefaultJsonProtocol._
[Your code]
scala> val json = """{"value": "TEST"}""".parseJson
json: spray.json.JsValue = {"value":"TEST"}
scala> json.convertTo[MyTrait]
res1: MyTrait = MyCaseClass