Consider a code:
val myList : List[Map[String, AnyRef]] = ...
//during spray routing
import spray.json._
complete(myList.toJson)
And I got en error:
Cannot find JsonWriter or JsonFormat type class for List[Map[String,AnyRef]]
First of all why??? Spray spray.json.CollectionFormats sources contains:
implicit def listFormat[T :JsonFormat] = new RootJsonFormat[List[T]] {
def write(list: List[T]) = JsArray(list.map(_.toJson).toVector)
def read(value: JsValue): List[T] = value match {
case JsArray(elements) => elements.map(_.convertTo[T])(collection.breakOut)
case x => deserializationError("Expected List as JsArray, but got " + x)
}
}
So every list item theoretically can be converted with _.toJson call. _ is Map[String,AnyRef], but Map types are also supported. Why my example is not compiled? How to get it work?
Related
I'm paraphrasing a question from the circe Gitter channel here.
Suppose I've got a Scala sealed trait hierarchy (or ADT) like this:
sealed trait Item
case class Cake(flavor: String, height: Int) extends Item
case class Hat(shape: String, material: String, color: String) extends Item
…and I want to be able to map back and forth between this ADT and a JSON representation like the following:
{ "tag": "Cake", "contents": ["cherry", 100] }
{ "tag": "Hat", "contents": ["cowboy", "felt", "black"] }
By default circe's generic derivation uses a different representation:
scala> val item1: Item = Cake("cherry", 100)
item1: Item = Cake(cherry,100)
scala> val item2: Item = Hat("cowboy", "felt", "brown")
item2: Item = Hat(cowboy,felt,brown)
scala> import io.circe.generic.auto._, io.circe.syntax._
import io.circe.generic.auto._
import io.circe.syntax._
scala> item1.asJson.noSpaces
res0: String = {"Cake":{"flavor":"cherry","height":100}}
scala> item2.asJson.noSpaces
res1: String = {"Hat":{"shape":"cowboy","material":"felt","color":"brown"}}
We can get a little closer with circe-generic-extras:
import io.circe.generic.extras.Configuration
import io.circe.generic.extras.auto._
implicit val configuration: Configuration =
Configuration.default.withDiscriminator("tag")
And then:
scala> item1.asJson.noSpaces
res2: String = {"flavor":"cherry","height":100,"tag":"Cake"}
scala> item2.asJson.noSpaces
res3: String = {"shape":"cowboy","material":"felt","color":"brown","tag":"Hat"}
…but it's still not what we want.
What's the best way to use circe to derive instances like this generically for ADTs in Scala?
Representing case classes as JSON arrays
The first thing to note is that the circe-shapes module provides instances for Shapeless's HLists that use an array representation like the one we want for our case classes. For example:
scala> import io.circe.shapes._
import io.circe.shapes._
scala> import shapeless._
import shapeless._
scala> ("foo" :: 1 :: List(true, false) :: HNil).asJson.noSpaces
res4: String = ["foo",1,[true,false]]
…and Shapeless itself provides a generic mapping between case classes and HLists. We can combine these two to get the generic instances we want for case classes:
import io.circe.{ Decoder, Encoder }
import io.circe.shapes.HListInstances
import shapeless.{ Generic, HList }
trait FlatCaseClassCodecs extends HListInstances {
implicit def encodeCaseClassFlat[A, Repr <: HList](implicit
gen: Generic.Aux[A, Repr],
encodeRepr: Encoder[Repr]
): Encoder[A] = encodeRepr.contramap(gen.to)
implicit def decodeCaseClassFlat[A, Repr <: HList](implicit
gen: Generic.Aux[A, Repr],
decodeRepr: Decoder[Repr]
): Decoder[A] = decodeRepr.map(gen.from)
}
object FlatCaseClassCodecs extends FlatCaseClassCodecs
And then:
scala> import FlatCaseClassCodecs._
import FlatCaseClassCodecs._
scala> Cake("cherry", 100).asJson.noSpaces
res5: String = ["cherry",100]
scala> Hat("cowboy", "felt", "brown").asJson.noSpaces
res6: String = ["cowboy","felt","brown"]
Note that I'm using io.circe.shapes.HListInstances to bundle up just the instances we need from circe-shapes together with our custom case class instances, in order to minimize the number of things our users have to import (both as a matter of ergonomics and for the sake of keeping down compile times).
Encoding the generic representation of our ADTs
That's a good first step, but it doesn't get us the representation we want for Item itself. To do that we need some more complex machinery:
import io.circe.{ JsonObject, ObjectEncoder }
import shapeless.{ :+:, CNil, Coproduct, Inl, Inr, Witness }
import shapeless.labelled.FieldType
trait ReprEncoder[C <: Coproduct] extends ObjectEncoder[C]
object ReprEncoder {
def wrap[A <: Coproduct](encodeA: ObjectEncoder[A]): ReprEncoder[A] =
new ReprEncoder[A] {
def encodeObject(a: A): JsonObject = encodeA.encodeObject(a)
}
implicit val encodeCNil: ReprEncoder[CNil] = wrap(
ObjectEncoder.instance[CNil](_ => sys.error("Cannot encode CNil"))
)
implicit def encodeCCons[K <: Symbol, L, R <: Coproduct](implicit
witK: Witness.Aux[K],
encodeL: Encoder[L],
encodeR: ReprEncoder[R]
): ReprEncoder[FieldType[K, L] :+: R] = wrap[FieldType[K, L] :+: R](
ObjectEncoder.instance {
case Inl(l) => JsonObject("tag" := witK.value.name, "contents" := (l: L))
case Inr(r) => encodeR.encodeObject(r)
}
)
}
This tells us how to encode instances of Coproduct, which Shapeless uses as a generic representation of sealed trait hierarchies in Scala. The code may be intimidating at first, but it's a very common pattern, and if you spend much time working with Shapeless you'll recognize that 90% of this code is essentially boilerplate that you see any time you build up instances inductively like this.
Decoding these coproducts
The decoding implementation is a little worse, even, but follows the same pattern:
import io.circe.{ DecodingFailure, HCursor }
import shapeless.labelled.field
trait ReprDecoder[C <: Coproduct] extends Decoder[C]
object ReprDecoder {
def wrap[A <: Coproduct](decodeA: Decoder[A]): ReprDecoder[A] =
new ReprDecoder[A] {
def apply(c: HCursor): Decoder.Result[A] = decodeA(c)
}
implicit val decodeCNil: ReprDecoder[CNil] = wrap(
Decoder.failed(DecodingFailure("CNil", Nil))
)
implicit def decodeCCons[K <: Symbol, L, R <: Coproduct](implicit
witK: Witness.Aux[K],
decodeL: Decoder[L],
decodeR: ReprDecoder[R]
): ReprDecoder[FieldType[K, L] :+: R] = wrap(
decodeL.prepare(_.downField("contents")).validate(
_.downField("tag").focus
.flatMap(_.as[String].right.toOption)
.contains(witK.value.name),
witK.value.name
)
.map(l => Inl[FieldType[K, L], R](field[K](l)))
.or(decodeR.map[FieldType[K, L] :+: R](Inr(_)))
)
}
In general there will be a little more logic involved in our Decoder implementations, since each decoding step can fail.
Our ADT representation
Now we can wrap it all together:
import shapeless.{ LabelledGeneric, Lazy }
object Derivation extends FlatCaseClassCodecs {
implicit def encodeAdt[A, Repr <: Coproduct](implicit
gen: LabelledGeneric.Aux[A, Repr],
encodeRepr: Lazy[ReprEncoder[Repr]]
): ObjectEncoder[A] = encodeRepr.value.contramapObject(gen.to)
implicit def decodeAdt[A, Repr <: Coproduct](implicit
gen: LabelledGeneric.Aux[A, Repr],
decodeRepr: Lazy[ReprDecoder[Repr]]
): Decoder[A] = decodeRepr.value.map(gen.from)
}
This looks very similar to the definitions in our FlatCaseClassCodecs above, and the idea is the same: we're defining instances for our data type (either case classes or ADTs) by building on the instances for the generic representations of those data types. Note that I'm extending FlatCaseClassCodecs, again to minimize imports for the user.
In action
Now we can use these instances like this:
scala> import Derivation._
import Derivation._
scala> item1.asJson.noSpaces
res7: String = {"tag":"Cake","contents":["cherry",100]}
scala> item2.asJson.noSpaces
res8: String = {"tag":"Hat","contents":["cowboy","felt","brown"]}
…which is exactly what we wanted. And the best part is that this will work for any sealed trait hierarchy in Scala, no matter how many case classes it has or how many members those case classes have (although compile times will start to hurt once you're into the dozens of either), assuming all of the member types have JSON representations.
I am trying to follow the tutorial https://www.jamesward.com/2012/02/21/play-framework-2-with-scala-anorm-json-coffeescript-jquery-heroku but of course play-scala has changed since the tutorial (as seems to be the case with every tutorial I find). I am using 2.4.3 This requires I actually learn how things work, not necessarily a bad thing.
One thing that is giving me trouble is the getOrElse method.
Here is my Bar.scala model
package models
import play.api.db._
import play.api.Play.current
import anorm._
import anorm.SqlParser._
case class Bar(id: Option[Long], name: String)
object Bar {
val simple = {
get[Option[Long]]("id") ~
get[String]("name") map {
case id~name => Bar(id, name)
}
}
def findAll(): Seq[Bar] = {
DB.withConnection { implicit connection =>
SQL("select * from bar").as(Bar.simple *)
}
}
def create(bar: Bar): Unit = {
DB.withConnection { implicit connection =>
SQL("insert into bar(name) values ({name})").on(
'name -> bar.name
).executeUpdate()
}
}
}
and my BarFormat.scala Json formatter
package models
import play.api.libs.json._
import anorm._
package object Implicits {
implicit object BarFormat extends Format[Bar] {
def reads(json: JsValue):JsResult[Bar] = JsSuccess(Bar(
Option((json \ "id").as[Long]),
(json \ "name").as[String]
))
def writes(bar: Bar) = JsObject(Seq(
"id" -> JsNumber(bar.id.getOrElse(0L)),
"name" -> JsString(bar.name)
))
}
}
and for completeness my Application.scala controller:
package controllers
import play.api.mvc._
import play.api.data._
import play.api.data.Forms._
import javax.inject.Inject
import javax.inject._
import play.api.i18n.{ I18nSupport, MessagesApi, Messages, Lang }
import play.api.libs.json._
import views._
import models.Bar
import models.Implicits._
class Application #Inject()(val messagesApi: MessagesApi) extends Controller with I18nSupport {
val barForm = Form(
single("name" -> nonEmptyText)
)
def index = Action {
Ok(views.html.index(barForm))
}
def addBar() = Action { implicit request =>
barForm.bindFromRequest.fold(
errors => BadRequest,
{
case (name) =>
Bar.create(Bar(None, name))
Redirect(routes.Application.index())
}
)
}
def listBars() = Action { implicit request =>
val bars = Bar.findAll()
val json = Json.toJson(bars)
Ok(json).as("application/json")
}
and routes
# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~
# Home page
POST /addBar controllers.Application.addBar
GET / controllers.Application.index
GET /listBars controllers.Application.listBars
# Map static resources from the /public folder to the /assets URL path
GET /assets/*file controllers.Assets.versioned(path="/public", file: Asset)
When I try to run my project I get the following error:
now bar.id is defined as an Option[Long] so bar.id.getOrElse(0L) should return a Long as far as I can tell, but it is clearly returning an Any. Can anyone help me understand why?
Thank You!
That's the way type inference works in Scala...
First of all there is an implicit conversion from Int to BigDecimal:
scala> (1 : Int) : BigDecimal
res0: BigDecimal = 1
That conversion allows for Int to be converted before the option is constructed:
scala> Some(1) : Option[BigDecimal]
res1: Option[BigDecimal] = Some(1)
If we try getOrElse on its own where the type can get fixed we get expected type Int:
scala> Some(1).getOrElse(2)
res2: Int = 1
However, this does not work (the problem you have):
scala> Some(1).getOrElse(2) : BigDecimal
<console>:11: error: type mismatch;
found : Any
required: BigDecimal
Some(1).getOrElse(2) : BigDecimal
^
Scala's implicit conversions kick in last, after type inference is performed. That makes sense, because if you don't know the type how would you know what conversions need to be applied. Scala can see that BigDecimal is expected, but it has an Int result based on the type of the Option it has. So it tries to widen the type, can't find anything that matches BigDecimal in Int's type hierarchy and fails with the error.
This works, however because the type is fixed in the variable declaration:
scala> val v = Some(1).getOrElse(2)
v: Int = 1
scala> v: BigDecimal
res4: BigDecimal = 1
So we need to help the compiler somehow - any type annotation or explicit conversion would work. Pick any one you like:
scala> (Some(1).getOrElse(2) : Int) : BigDecimal
res5: BigDecimal = 1
scala> Some(1).getOrElse[Int](2) : BigDecimal
res6: BigDecimal = 1
scala> BigDecimal(Some(1).getOrElse(2))
res7: scala.math.BigDecimal = 1
Here is the signature for Option.getOrElse method:
getOrElse[B >: A](default: ⇒ B): B
The term B >: A expresses that the type parameter B or the abstract type B refer to a supertype of type A, in this case, Any being the supertype of Long:
val l: Long = 10
val a: Any = l
So, we can do something very similar here with getOrElse:
val some: Option[Long] = Some(1)
val value: Any = option.getOrElse("potatos")
val none: Option[Long] = None
val elseValue: Any = none.getOrElse("potatos")
Which brings us to your scenario: the returned type from getOrElse will be a Any and not a BigDecimal, so you will need another way to handle this situation, like using fold, per instance:
def writes(bar: Bar) = {
val defaultValue = BigDecimal(0)
JsObject(Seq(
"id" -> JsNumber(bar.id.fold(defaultValue)(BigDecimal(_))),
"name" -> JsString(bar.name)
))
}
Some other discussions that can help you:
Why is Some(1).getOrElse(Some(1)) not of type Option[Int]?
Option getOrElse type mismatch error
I have following class (simplified a bit there) that would extend the JSON-format for certain objects that represent the Database-level with the ID-field:
import play.api.libs.json._
import play.api.libs.functional.syntax._
class EntityFormat[T <: Entity](entityFormatter: Format[T]) extends Format[T] {
val extendedFormat: Format[T] = (
__.format[T](entityFormatter) ~
(__ \ "id").format[Option[Long]]
)(tupleToEntity, entityToTuple)
private def tupleToEntity(e: T, id: Option[Long]) = {
e.id = id
e
}
private def entityToTuple(e: T) = (e, e.id)
def writes(o: T): JsValue = extendedFormat.writes(o)
def reads(json: JsValue): JsResult[T] = extendedFormat.reads(json)
}
abstract class Entity {
var id: Option[Long] = None
}
With Play 2.3, I could then write
implicit val userFormat: Format[User] = new EntityFormat(Json.format[User])
Which would then work with with the ID-field being in the generated JSON. However, with Play 2.4 I'm getting following compile-time issues:
No Json formatter found for type Option[Long]. Try to implement an implicit Format for this type. (__ \ "id").format[Option[Long]]
missing arguments for method tupleToEntity in class DomainEntityFormat; follow this method with `_' if you want to treat it as a partially applied function )(tupleToEntity, entityToTuple)
^
missing arguments for method tupleToEntity in class DomainEntityFormat; follow this method with `_' if you want to treat it as a partially applied function )(tupleToEntity, entityToTuple)
^
How should you do the extending with Play 2.4 to make that kind of JSON Format work?
Instead of:
(__ \ "id").format[Option[Long]]
Try:
(__ \ "id").formatNullable[Long]
I have json rest api application based on play framework and want to receive information about validation errors when I parse incoming requests. Everything is working fine except conversion from json array to json value.
Json structure I want to achieve:
{
"errors": {
"name": ["invalid", "tooshort"],
"email": ["invalid"]
}
}
When I tried to implement a sample it worked perfectly:
def error = Action {
BadRequest(obj(
"errors" -> obj(
"name" -> arr("invalid", "tooshort"),
"email" -> arr("invalid")
)
))
}
When I tried to extract the changing part like this:
def error = Action {
val e = Seq("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
// in real app it will be Seq[(JsPath, Seq[ValidationError])]
BadRequest(obj(
"errors" -> obj(e: _*)
))
}
I got a compiler error:
type mismatch; found : Seq[(String, play.api.libs.json.JsArray)] required: Seq[(String, play.api.libs.json.Json.JsValueWrapper)]
Maybe there is some implicit conversion I'm missing from JsArray to JsValueWrapper? But then, why does the sample works fine in the same file with the same imports?
Play 2.1.1, Scala 2.10.0
UPDATE:
Problem resolved thanks to Julien Lafont, the final code:
implicit val errorsWrites = new Writes[Seq[(JsPath, Seq[ValidationError])]] {
/**
* Maps validation result of Ember.js json request to json validation object, which Ember can understand and parse as DS.Model 'errors' field.
*
* #param errors errors collected after json validation
* #return json in following format:
*
* {
* "errors": {
* "error1": ["message1", "message2", ...],
* "error2": ["message3", "message4", ...],
* ...
* }
* }
*/
def writes(errors: Seq[(JsPath, Seq[ValidationError])]) = {
val mappedErrors = errors.map {
e =>
val fieldName = e._1.toString().split("/").last // take only last part of the path, which contains real field name
val messages = e._2.map(_.message)
fieldName -> messages
}
obj("errors" -> toJson(mappedErrors.toMap)) // Ember requires root "errors" object
}
}
Usage:
def create = Action(parse.json) { // method in play controller
request =>
fromJson(request.body) match {
case JsSuccess(pet, path) => Ok(obj("pet" -> Pets.create(pet)))
case JsError(errors) => UnprocessableEntity(toJson(errors)) // Ember.js expects 422 error code in case of errors
}
}
You can simply define your errors in a Map[String, Seq[String]], and transform it in Json with Json.toJson (there are built-in writers for Map[String,Y] and Seq[X])
scala> val e = Map("name" -> Seq("invalid", "tooshort"), "email" -> Seq("invalid"))
e: scala.collection.immutable.Map[String,Seq[String]] = Map(name -> List(invalid, tooshort), email -> List(invalid))
scala> Json.toJson(e)
res0: play.api.libs.json.JsValue = {"name":["invalid","tooshort"],"email":["invalid"]}
scala> Json.obj("errors" -> Json.toJson(e))
res1: play.api.libs.json.JsObject = {"errors":{"name":["invalid","tooshort"],"email":["invalid"]}}
The reason the long handed version works is because scala's automatic type inference triggers an implicit conversion toJsFieldJsValueWrapper.
For example
scala> import play.api.libs.json._
import play.api.libs.functional.syntax._
import play.api.libs.json._
scala> import play.api.libs.functional.syntax._
import play.api.libs.functional.syntax._
scala> import Json._
import Json._
scala> arr("invalid", "tooshort")
res0: play.api.libs.json.JsArray = ["invalid","tooshort"]
scala> obj("name" -> arr("invalid", "tooshort"))
res1: play.api.libs.json.JsObject = {"name":["invalid","tooshort"]}
scala> obj _
res3: Seq[(String, play.api.libs.json.Json.JsValueWrapper)] => play.api.libs.json.JsObject = <function1>
Notice that arr returns a JsArray, however obj requires a JsValueWrapper. Scala is able to convert the JsArray to JsValueWrapper when it constructs the arguments for obj. However it is not able to convert a Seq[(String, JsArray)] to a `Seq[(String, JsValueWrapper)].
If you provide the expected type when of the sequence in advance, the Scala compiler's type inferencer will perform the conversion as it creates the sequence.
scala> Seq[(String, JsValueWrapper)]("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
res4: Seq[(String, play.api.libs.json.Json.JsValueWrapper)] = List((name,JsValueWrapperImpl(["invalid","tooshort"])), (email,JsValueWrapperImpl(["invalid"])))
However once the sequence is created it cannot be converted unless there is an implicit conversion in scope that can convert sequences.
The final code snippet looks like:
def error = Action {
val e = Seq[(String, JsValueWrapper)]("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
// in real app it will be Seq[(JsPath, Seq[ValidationError])]
BadRequest(obj(
"errors" -> obj(e: _*)
))
}
This is my code:
import play.api.mvc._
import play.api.libs.json._
import play.api.libs.json.Json._
import play.api.libs.json.Writes._
class BaseController extends Controller with Secured with DefaultWrites {
private implicit def str2json(str: String) = new {
def asSuccessJson = toJson(Map("success" -> true, "message" -> str)) // (*)
def asFailedJson = toJson(Map("success" -> false, "message" -> str)) // (*)
}
}
But it can't be compiled on two (*) lines. The error message is:
Multiple markers at this line
- No Json deserializer found for type scala.collection.immutable.Map[java.lang.String,Any]. Try
to implement an implicit Writes or Format for this type.
- not enough arguments for method toJson: (implicit tjs:
play.api.libs.json.Writes[scala.collection.immutable.Map[java.lang.String,Any]])
play.api.libs.json.JsValue.Unspecified value parameter tjs.
I have to write it as:
def asSuccessJson = toJson(Map("success" -> true.toString, "message" -> str))
Notice true.toString. It works but boring.
How to fix it?
It's quite logical: you try to convert an heterogeneous map into a JsValue:
Map("success" -> true, "message" -> str) is a Map[String, Any].
There is no implicit writer able to convert a Map[String, Any] into a JsValue (and there can't be any).
When you write Map("success" -> true.toString, "message" -> str), you create a Map[String, String] and there is a writer for this.
I would write:
def asSuccessJson = JsObject(Seq("success" -> JsBoolean(true), "message" -> JsString(str))) // (*)
BTW, the JSON API will certainly be "beautified" a bit in Play 2's next releases...