Scala Compiler (2.11.7) anomaly with Play JSON Writes - json

So I have the following code:
Base:
import play.api.libs.json.{JsNull, Json, JsValue, Writes}
case class Cost(cost: Option[Double])
This compiles:
case object Cost {
def writes = new Writes[Cost] {
override def writes(r: Cost): JsValue = {
val cost = r.cost.map(Json.toJson(_)).getOrElse(JsNull)
Json.obj(
"cost" -> cost
)
}
}
}
But this doesn't compile
case object Cost {
def writes = new Writes[Cost] {
override def writes(r: Cost): JsValue = {
Json.obj(
"cost" -> r.cost.map(Json.toJson(_)).getOrElse(JsNull)
)
}
}
}
Compiler error is the following:
type mismatch;
[error] found : Object
[error] required: play.api.libs.json.Json.JsValueWrapper
[error] "cost" -> r.cost.map(Json.toJson(_)).getOrElse(JsNull)
In the latter if I use .asInstanceOf[JsValue] it works, but with IntelliJ grays it out saying it's unnecessary as it can't be anything else that JsValue anyway. What might be the reason that Scala compiler (2.11.7) doesn't detect the Class properly?

I am pretty sure the issue comes from .getOrElse(JsNull)
I have successfully compiled this code:
import play.api.libs.json.{JsNull, Json, JsValue, Writes}
case class Cost(cost: Option[Double])
case object Cost {
def writes = new Writes[Cost] {
override def writes(r: Cost): JsValue = {
Json.obj(
"cost" -> r.cost.map(Json.toJson(_))
)
}
}
}
and parsed the output:
scala> Cost(Some(5))
res2: Cost = Cost(Some(5.0))
scala> Json.toJson(res2)(Cost.writes)
res5: play.api.libs.json.JsValue = {"cost":5}
Looking for the source of the issue you can check couple of additional solutions, assuming the writes function:
val cost = r.cost.map(t => Json.toJson(t))
Json.obj(
"cost" -> cost
)
if you want to resolve the Option of cost value with getOrElse you can either cast (as you have tried) or provide the type:
cost.getOrElse[JsValue](JsNull)
cost.getOrElse(JsNull).asInstanceOf[JsValue]
and without the type specification sbt always gives an error saying:
[error] (...) type mismatch;
[error] found : Object
[error] required: play.api.libs.json.Json.JsValueWrapper
which must come from some SBT's compiler bug.

Related

Slick dynamic groupby

I have code like so:
def query(buckets: List[String]): Future[Seq[(List[Option[String]], Option[Double])]] = {
database.run {
groupBy(row => buckets.map(bucket => customBucketer(row.metadata, bucket)))
.map { grouping =>
val bucket = grouping._1
val group = grouping._2
(bucket, group.map(_.value).avg)
}
.result
}
}
private def customBucketer(metadata: Rep[Option[String]], bucket: String): Rep[Option[String]] = {
...
}
I am wanting to be able to create queries in slick which groupby and collect on a given list of columns.
Error I am getting upon compilation is:
[error] Slick does not know how to map the given types.
[error] Possible causes: T in Table[T] does not match your * projection,
[error] you use an unsupported type in a Query (e.g. scala List),
[error] or you forgot to import a driver api into scope.
[error] Required level: slick.lifted.FlatShapeLevel
[error] Source type: List[slick.lifted.Rep[Option[String]]]
[error] Unpacked type: T
[error] Packed type: G
[error] groupBy(row => buckets.map(bucket => customBucketer(row.metadata, bucket)))
[error] ^
Here's a workaround for Slick 3.2.3 (and some background on my approach):
You may have noticed dynamically selecting columns is easy as long as you can assume a fixed type, e.g:
columnNames = List("col1", "col2")
tableQuery.map( r => columnNames.map(name => r.column[String](name)) )
But if you try a similar approach with a groupBy operation, Slick will complain that it "does not know how to map the given types".
So, while this is hardly an elegant solution, you can at least satisfy Slick's type-safety by statically defining both:
groupby column type
Upper/lower bound on the quantity of groupBy columns
A simple way of implementing these two constraints is to again assume a fixed type and to branch the code for all possible quantities of groupBy columns.
Here's the full working Scala REPL session to give you an idea:
import java.io.File
import akka.actor.ActorSystem
import com.typesafe.config.ConfigFactory
import slick.jdbc.H2Profile.api._
import scala.concurrent.{Await, Future}
import scala.concurrent.duration._
val confPath = getClass.getResource("/application.conf")
val config = ConfigFactory.parseFile(new File(confPath.getPath)).resolve()
val db = Database.forConfig("slick.db", config)
implicit val system = ActorSystem("testSystem")
implicit val executionContext = system.dispatcher
case class AnyData(a: String, b: String)
case class GroupByFields(a: Option[String], b: Option[String])
class AnyTable(tag: Tag) extends Table[AnyData](tag, "macro"){
def a = column[String]("a")
def b = column[String]("b")
def * = (a, b) <> ((AnyData.apply _).tupled, AnyData.unapply)
}
val table = TableQuery[AnyTable]
def groupByDynamically(groupBys: Seq[String]): DBIO[Seq[GroupByFields]] = {
// ensures columns are returned in the right order
def selectGroups(g: Map[String, Rep[Option[String]]]) = {
(g.getOrElse("a", Rep.None[String]), g.getOrElse("b", Rep.None[String])).mapTo[GroupByFields]
}
val grouped = if (groupBys.lengthCompare(2) == 0) {
table
.groupBy( cols => (cols.column[String](groupBys(0)), cols.column[String](groupBys(1))) )
.map{ case (groups, _) => selectGroups(Map(groupBys(0) -> Rep.Some(groups._1), groupBys(1) -> Rep.Some(groups._2))) }
}
else {
// there should always be at least one group by specified
table
.groupBy(cols => cols.column[String](groupBys.head))
.map{ case (groups, _) => selectGroups(Map(groupBys.head -> Rep.Some(groups))) }
}
grouped.result
}
val actions = for {
_ <- table.schema.create
_ <- table.map(a => (a.column[String]("a"), a.column[String]("b"))) += ("a1", "b1")
_ <- table.map(a => (a.column[String]("a"), a.column[String]("b"))) += ("a2", "b2")
_ <- table.map(a => (a.column[String]("a"), a.column[String]("b"))) += ("a2", "b3")
queryResult <- groupByDynamically(Seq("b", "a"))
} yield queryResult
val result: Future[Seq[GroupByFields]] = db.run(actions.transactionally)
result.foreach(println)
Await.ready(result, Duration.Inf)
Where this gets ugly is when you can have upwards of a few groupBy columns (i.e. having a separate if branch for 10+ cases would get monotonous). Hopefully someone will swoop in and edit this answer for how to hide that boilerplate behind some syntactic sugar or abstraction layer.

Why does this getOrElse statement return type ANY?

I am trying to follow the tutorial https://www.jamesward.com/2012/02/21/play-framework-2-with-scala-anorm-json-coffeescript-jquery-heroku but of course play-scala has changed since the tutorial (as seems to be the case with every tutorial I find). I am using 2.4.3 This requires I actually learn how things work, not necessarily a bad thing.
One thing that is giving me trouble is the getOrElse method.
Here is my Bar.scala model
package models
import play.api.db._
import play.api.Play.current
import anorm._
import anorm.SqlParser._
case class Bar(id: Option[Long], name: String)
object Bar {
val simple = {
get[Option[Long]]("id") ~
get[String]("name") map {
case id~name => Bar(id, name)
}
}
def findAll(): Seq[Bar] = {
DB.withConnection { implicit connection =>
SQL("select * from bar").as(Bar.simple *)
}
}
def create(bar: Bar): Unit = {
DB.withConnection { implicit connection =>
SQL("insert into bar(name) values ({name})").on(
'name -> bar.name
).executeUpdate()
}
}
}
and my BarFormat.scala Json formatter
package models
import play.api.libs.json._
import anorm._
package object Implicits {
implicit object BarFormat extends Format[Bar] {
def reads(json: JsValue):JsResult[Bar] = JsSuccess(Bar(
Option((json \ "id").as[Long]),
(json \ "name").as[String]
))
def writes(bar: Bar) = JsObject(Seq(
"id" -> JsNumber(bar.id.getOrElse(0L)),
"name" -> JsString(bar.name)
))
}
}
and for completeness my Application.scala controller:
package controllers
import play.api.mvc._
import play.api.data._
import play.api.data.Forms._
import javax.inject.Inject
import javax.inject._
import play.api.i18n.{ I18nSupport, MessagesApi, Messages, Lang }
import play.api.libs.json._
import views._
import models.Bar
import models.Implicits._
class Application #Inject()(val messagesApi: MessagesApi) extends Controller with I18nSupport {
val barForm = Form(
single("name" -> nonEmptyText)
)
def index = Action {
Ok(views.html.index(barForm))
}
def addBar() = Action { implicit request =>
barForm.bindFromRequest.fold(
errors => BadRequest,
{
case (name) =>
Bar.create(Bar(None, name))
Redirect(routes.Application.index())
}
)
}
def listBars() = Action { implicit request =>
val bars = Bar.findAll()
val json = Json.toJson(bars)
Ok(json).as("application/json")
}
and routes
# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~
# Home page
POST /addBar controllers.Application.addBar
GET / controllers.Application.index
GET /listBars controllers.Application.listBars
# Map static resources from the /public folder to the /assets URL path
GET /assets/*file controllers.Assets.versioned(path="/public", file: Asset)
When I try to run my project I get the following error:
now bar.id is defined as an Option[Long] so bar.id.getOrElse(0L) should return a Long as far as I can tell, but it is clearly returning an Any. Can anyone help me understand why?
Thank You!
That's the way type inference works in Scala...
First of all there is an implicit conversion from Int to BigDecimal:
scala> (1 : Int) : BigDecimal
res0: BigDecimal = 1
That conversion allows for Int to be converted before the option is constructed:
scala> Some(1) : Option[BigDecimal]
res1: Option[BigDecimal] = Some(1)
If we try getOrElse on its own where the type can get fixed we get expected type Int:
scala> Some(1).getOrElse(2)
res2: Int = 1
However, this does not work (the problem you have):
scala> Some(1).getOrElse(2) : BigDecimal
<console>:11: error: type mismatch;
found : Any
required: BigDecimal
Some(1).getOrElse(2) : BigDecimal
^
Scala's implicit conversions kick in last, after type inference is performed. That makes sense, because if you don't know the type how would you know what conversions need to be applied. Scala can see that BigDecimal is expected, but it has an Int result based on the type of the Option it has. So it tries to widen the type, can't find anything that matches BigDecimal in Int's type hierarchy and fails with the error.
This works, however because the type is fixed in the variable declaration:
scala> val v = Some(1).getOrElse(2)
v: Int = 1
scala> v: BigDecimal
res4: BigDecimal = 1
So we need to help the compiler somehow - any type annotation or explicit conversion would work. Pick any one you like:
scala> (Some(1).getOrElse(2) : Int) : BigDecimal
res5: BigDecimal = 1
scala> Some(1).getOrElse[Int](2) : BigDecimal
res6: BigDecimal = 1
scala> BigDecimal(Some(1).getOrElse(2))
res7: scala.math.BigDecimal = 1
Here is the signature for Option.getOrElse method:
getOrElse[B >: A](default: ⇒ B): B
The term B >: A expresses that the type parameter B or the abstract type B refer to a supertype of type A, in this case, Any being the supertype of Long:
val l: Long = 10
val a: Any = l
So, we can do something very similar here with getOrElse:
val some: Option[Long] = Some(1)
val value: Any = option.getOrElse("potatos")
val none: Option[Long] = None
val elseValue: Any = none.getOrElse("potatos")
Which brings us to your scenario: the returned type from getOrElse will be a Any and not a BigDecimal, so you will need another way to handle this situation, like using fold, per instance:
def writes(bar: Bar) = {
val defaultValue = BigDecimal(0)
JsObject(Seq(
"id" -> JsNumber(bar.id.fold(defaultValue)(BigDecimal(_))),
"name" -> JsString(bar.name)
))
}
Some other discussions that can help you:
Why is Some(1).getOrElse(Some(1)) not of type Option[Int]?
Option getOrElse type mismatch error

Play 2.3 Scala: Explicitly passing Writer - needs to match Option{T] vs T; implicit writer can handle both

I have a case class with a json.Writes[T] defined on it.
If I have an Option[T], and with an implicit write in scope, I can call Json.toJson(Option[T]); and this works
However if I call Json.toJson(T)(json.Writes[T]) - I get a compile error
type mismatch;
found : play.api.libs.json.Writes[models.WorkflowTemplateMilestone]{def writes(o: models.WorkflowTemplateMilestone): play.api.libs.json.JsObject}
required: play.api.libs.json.Writes[Option[models.WorkflowTemplateMilestone]]
Note: implicit value workflowTemplateMilestoneAPIWrites is not applicable here because it comes after the application point and it lacks an explicit result type
Am I passing the Writer in incorrectly? How does it work between T and Option[T] with the implicit write?
The actual code is below; in case I've mis diagnosed the issue
Custom writer
implicit val workflowTemplateMilestoneAPIWrites = new Writes[WorkflowTemplateMilestone] {
def writes(o: WorkflowTemplateMilestone) = Json.obj(
"id" -> o.id,
"name" -> o.name,
"order" -> o.order
)
}
WORKS
implicit val workflowTemplateMilestoneAPIWrites = new Writes[List[(WorkflowTemplate, Option[WorkflowTemplateMilestone])]] {
def writes(l: List[(WorkflowTemplate, Option[WorkflowTemplateMilestone])]) = Json.obj(
"id" -> l.head._1.id,
"name" -> l.head._1.name,
"milestones" ->
l.map(o =>
Json.toJson(o._2) **// Writer is picked up through Implict this WORKS**
)
)
}
GIVES COMPILE ERROR
implicit val workflowTemplateMilestoneAPIWrites = new Writes[List[(WorkflowTemplate, Option[WorkflowTemplateMilestone])]] {
def writes(l: List[(WorkflowTemplate, Option[WorkflowTemplateMilestone])]) = Json.obj(
"id" -> l.head._1.id,
"name" -> l.head._1.name,
"milestones" ->
l.map(o =>
Json.toJson(o._2)**(WorkflowTemplateMilestone.workflowTemplateMilestoneAPIWrites)** // But if I explicitly pass in the Writer, I get the compile error
)
)
}
Thanks,
Brent

Scala play json api: can't implicitly convert JsArray to JsValueWrapper

I have json rest api application based on play framework and want to receive information about validation errors when I parse incoming requests. Everything is working fine except conversion from json array to json value.
Json structure I want to achieve:
{
"errors": {
"name": ["invalid", "tooshort"],
"email": ["invalid"]
}
}
When I tried to implement a sample it worked perfectly:
def error = Action {
BadRequest(obj(
"errors" -> obj(
"name" -> arr("invalid", "tooshort"),
"email" -> arr("invalid")
)
))
}
When I tried to extract the changing part like this:
def error = Action {
val e = Seq("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
// in real app it will be Seq[(JsPath, Seq[ValidationError])]
BadRequest(obj(
"errors" -> obj(e: _*)
))
}
I got a compiler error:
type mismatch; found : Seq[(String, play.api.libs.json.JsArray)] required: Seq[(String, play.api.libs.json.Json.JsValueWrapper)]
Maybe there is some implicit conversion I'm missing from JsArray to JsValueWrapper? But then, why does the sample works fine in the same file with the same imports?
Play 2.1.1, Scala 2.10.0
UPDATE:
Problem resolved thanks to Julien Lafont, the final code:
implicit val errorsWrites = new Writes[Seq[(JsPath, Seq[ValidationError])]] {
/**
* Maps validation result of Ember.js json request to json validation object, which Ember can understand and parse as DS.Model 'errors' field.
*
* #param errors errors collected after json validation
* #return json in following format:
*
* {
* "errors": {
* "error1": ["message1", "message2", ...],
* "error2": ["message3", "message4", ...],
* ...
* }
* }
*/
def writes(errors: Seq[(JsPath, Seq[ValidationError])]) = {
val mappedErrors = errors.map {
e =>
val fieldName = e._1.toString().split("/").last // take only last part of the path, which contains real field name
val messages = e._2.map(_.message)
fieldName -> messages
}
obj("errors" -> toJson(mappedErrors.toMap)) // Ember requires root "errors" object
}
}
Usage:
def create = Action(parse.json) { // method in play controller
request =>
fromJson(request.body) match {
case JsSuccess(pet, path) => Ok(obj("pet" -> Pets.create(pet)))
case JsError(errors) => UnprocessableEntity(toJson(errors)) // Ember.js expects 422 error code in case of errors
}
}
You can simply define your errors in a Map[String, Seq[String]], and transform it in Json with Json.toJson (there are built-in writers for Map[String,Y] and Seq[X])
scala> val e = Map("name" -> Seq("invalid", "tooshort"), "email" -> Seq("invalid"))
e: scala.collection.immutable.Map[String,Seq[String]] = Map(name -> List(invalid, tooshort), email -> List(invalid))
scala> Json.toJson(e)
res0: play.api.libs.json.JsValue = {"name":["invalid","tooshort"],"email":["invalid"]}
scala> Json.obj("errors" -> Json.toJson(e))
res1: play.api.libs.json.JsObject = {"errors":{"name":["invalid","tooshort"],"email":["invalid"]}}
The reason the long handed version works is because scala's automatic type inference triggers an implicit conversion toJsFieldJsValueWrapper.
For example
scala> import play.api.libs.json._
import play.api.libs.functional.syntax._
import play.api.libs.json._
scala> import play.api.libs.functional.syntax._
import play.api.libs.functional.syntax._
scala> import Json._
import Json._
scala> arr("invalid", "tooshort")
res0: play.api.libs.json.JsArray = ["invalid","tooshort"]
scala> obj("name" -> arr("invalid", "tooshort"))
res1: play.api.libs.json.JsObject = {"name":["invalid","tooshort"]}
scala> obj _
res3: Seq[(String, play.api.libs.json.Json.JsValueWrapper)] => play.api.libs.json.JsObject = <function1>
Notice that arr returns a JsArray, however obj requires a JsValueWrapper. Scala is able to convert the JsArray to JsValueWrapper when it constructs the arguments for obj. However it is not able to convert a Seq[(String, JsArray)] to a `Seq[(String, JsValueWrapper)].
If you provide the expected type when of the sequence in advance, the Scala compiler's type inferencer will perform the conversion as it creates the sequence.
scala> Seq[(String, JsValueWrapper)]("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
res4: Seq[(String, play.api.libs.json.Json.JsValueWrapper)] = List((name,JsValueWrapperImpl(["invalid","tooshort"])), (email,JsValueWrapperImpl(["invalid"])))
However once the sequence is created it cannot be converted unless there is an implicit conversion in scope that can convert sequences.
The final code snippet looks like:
def error = Action {
val e = Seq[(String, JsValueWrapper)]("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
// in real app it will be Seq[(JsPath, Seq[ValidationError])]
BadRequest(obj(
"errors" -> obj(e: _*)
))
}

toJson(Map("success"->true, "message"->str)) can't be compiled in play2

This is my code:
import play.api.mvc._
import play.api.libs.json._
import play.api.libs.json.Json._
import play.api.libs.json.Writes._
class BaseController extends Controller with Secured with DefaultWrites {
private implicit def str2json(str: String) = new {
def asSuccessJson = toJson(Map("success" -> true, "message" -> str)) // (*)
def asFailedJson = toJson(Map("success" -> false, "message" -> str)) // (*)
}
}
But it can't be compiled on two (*) lines. The error message is:
Multiple markers at this line
- No Json deserializer found for type scala.collection.immutable.Map[java.lang.String,Any]. Try
to implement an implicit Writes or Format for this type.
- not enough arguments for method toJson: (implicit tjs:
play.api.libs.json.Writes[scala.collection.immutable.Map[java.lang.String,Any]])
play.api.libs.json.JsValue.Unspecified value parameter tjs.
I have to write it as:
def asSuccessJson = toJson(Map("success" -> true.toString, "message" -> str))
Notice true.toString. It works but boring.
How to fix it?
It's quite logical: you try to convert an heterogeneous map into a JsValue:
Map("success" -> true, "message" -> str) is a Map[String, Any].
There is no implicit writer able to convert a Map[String, Any] into a JsValue (and there can't be any).
When you write Map("success" -> true.toString, "message" -> str), you create a Map[String, String] and there is a writer for this.
I would write:
def asSuccessJson = JsObject(Seq("success" -> JsBoolean(true), "message" -> JsString(str))) // (*)
BTW, the JSON API will certainly be "beautified" a bit in Play 2's next releases...