I am querying the spotify api for a list of tracks for a given query using ws, when I come to transform the JSON data into a case class I'm getting an error that I've yet to figure out...
class SearchController #Inject() (
val ws: WSClient
) extends Controller {
case class TrackSearch(href: String)
implicit val trackResultsReads: Reads[TrackSearch] = (
(__ \ "tracks" \ "href").read[String]
)(TrackSearch.apply _)
def index = Action.async { implicit request =>
search("track", param(request, "q")).map { r =>
val ts = r.json.as[TrackSearch]
println(ts)
Ok
}
}
private def search(category: String, query: String): Future[Try[WSResponse]] = {
ws.url("https://api.spotify.com/v1/search")
.withQueryString("q" -> query, "type" -> category)
.get()
.map(Success(_))
.recover { case x => Failure(x) }
}
private def param(request: Request[AnyContent], name: String): String = {
request.queryString.get(name).flatMap(_.headOption).getOrElse("")
}
}
The error I am getting is:
Overloaded method value [read] cannot be applied to (String => SearchController.this.TrackSearch)
implicit val trackResultsReads: Reads[TrackSearch]
> (__ \ "tracks" \ "href").read[String]
)(TrackSearch.apply _)
If I query the JSPath in my action, I can get the "href" string back fine, so it is not that:
println(r._2.json \ "tracks" \ "href")
The issue is that there is just a single field. If you added a second field it would compile. I don't fully understand why it shouldn't compile with a single field. So in the single field case, try the following:
implicit val trackResultsReads: Reads[TrackSearch] = {
((__ \ "tracks" \ "href").read[String])
.map(TrackSearch(_))
}
Here is quite an old link where I found the above. See also this link for a similar SO question with a different approach.
Related
I'm trying to go create some formats for this:
case class Work[T](todo: Seq[T], failed: Seq[T], success: Seq[T])
object Work {
implicit def format[T](implicit r: Reads[T], w: Writes[T]): Format[Work[T]] = Json.format[Work[T]]
}
object InternalMessage {
implicit def format[D, R](implicit
rD: Reads[D],
wD: Writes[D],
rR: Reads[R],
wR: Writes[R]
): Format[InternalMessage[D, R]] = Json.format[InternalMessage[D, R]]
}
case class InternalMessage[D, R](
download: List[Work[D]],
refine: List[Work[R]],
numberOfTries: Int
)
This doesn't work and I don't understand why. The error is
[error] /home/oleber/develop/data-platform/modules/importerTemplate/src/main/scala/template/TemplateModel.scala:46: No apply function found matching unapply parameters
[error] implicit def format[T](implicit r: Reads[T], w: Writes[T]): Format[Work[T]] = Json.format[Work[T]]
[error] ^
[error] /home/oleber/develop/data-platform/modules/importerTemplate/src/main/scala/template/TemplateModel.scala:55: No apply function found matching unapply parameters
[error] ): Format[InternalMessage[D, R]] = Json.format[InternalMessage[D, R]]
Thanks for any help
AFAIU you can't generate JSON Format for such data types using Play 2.5 macros. What is limiting you is that you use Seq[T] i.e. you want JSON serialization for a generic class that has fields that use some complex type built from that generic type rather than just raw generic T. Relevant code seems to be in the JsMacroImpl lines 135-141. In those lines inside paramsMatch inner method of maybeApply calculation the macro checks if there are an apply and unapply methods with matching (complement) signatures:
val maybeApply = applies.collectFirst {
case (apply: MethodSymbol) if hasVarArgs && {
// Option[List[c.universe.Type]]
val someApplyTypes = apply.paramLists.headOption.map(_.map(_.asTerm.typeSignature))
val someInitApply = someApplyTypes.map(_.init)
val someApplyLast = someApplyTypes.map(_.last)
val someInitUnapply = unapplyReturnTypes.map(_.init)
val someUnapplyLast = unapplyReturnTypes.map(_.last)
val initsMatch = someInitApply == someInitUnapply
val lastMatch = (for {
lastApply <- someApplyLast
lastUnapply <- someUnapplyLast
} yield lastApply <:< lastUnapply).getOrElse(false)
initsMatch && lastMatch
} => apply
case (apply: MethodSymbol) if {
val applyParams = apply.paramLists.headOption.
toList.flatten.map(_.typeSignature)
val unapplyParams = unapplyReturnTypes.toList.flatten
def paramsMatch = (applyParams, unapplyParams).zipped.forall {
case (TypeRef(NoPrefix, applyParam, _),
TypeRef(NoPrefix, unapplyParam, _)) => // for generic parameter
applyParam.fullName == unapplyParam.fullName
case (applyParam, unapplyParam) => applyParam =:= unapplyParam
}
(applyParams.size == unapplyParams.size && paramsMatch)
} => apply
}
As you can see in the relevant (non-hasVarArgs branch), this code only handles two cases: when the types in apply and unapply match exactly or when the same raw generic type is used. Your case of Seq[T] (as well as other complex generic types) is not handled here and the error you get is generated just a few lines down when maybeApply is empty:
val (tparams, params) = maybeApply match {
case Some(apply) => {
apply.typeParams -> apply.paramLists.head
// assume there is a single parameter group
}
case None => c.abort(c.enclosingPosition, "No apply function found matching unapply parameters")
}
In Play-JSON 2.6 this code was significantly re-worked and now it seems to support your case as well (see conforms inner method).
If upgrading to (not released yet) Play-JSON 2.6 is not acceptable, you can create Formats object(s) yourself using JSON Reads/Writes/Format Combinators doc. Something like this should work for you:
implicit def format[T](implicit r: Reads[T], w: Writes[T]): Format[Work[T]] = {
val workReads = (
(JsPath \ "todo").read[Seq[T]] and
(JsPath \ "failed").read[Seq[T]] and
(JsPath \ "success").read[Seq[T]]
) (Work.apply[T] _)
val workWrites = (
(JsPath \ "todo").write[Seq[T]] and
(JsPath \ "failed").write[Seq[T]] and
(JsPath \ "success").write[Seq[T]]
) (unlift(Work.unapply[T]))
new Format[Work[T]] {
override def reads(json: JsValue): JsResult[Work[T]] = workReads.reads(json)
override def writes(o: Work[T]): JsValue = workWrites.writes(o)
}
}
I'm trying to create Json readers in my Play framework application (Scala). The problem is, part of my Json is a little funky, and requires further processing to retrieve the values. For example:
{
"field1":"value1",
"field2":"value/1",
"num2":2
}
with case classes:
case class Field1(text: String, fields: Field2)
case class Field2(text: String, num: Int, num2: Int)
Basically the text and num fields for Field2 are derived from the value value/1, by splitting the text. Here's the splitter function:
def splitter(path: String, num2: Int): Field2 = {
val split = path.split("\\")
Field2(split(0), split(1).toInt, num2)
}
This is fairly straightforward, the actual splitter function is far more complex. Basically the only way to construct this object Field2 is to pass a single string to a function that spits out the required object.
How do I go about creating a reader for Field2 (and by extension for Field1)?
Here's what I have so far:
object Field1 {
implicit val reader = (
(__ \ "field1").read[String] and
(__).read[Field2]
) (Field1.apply _)
}
object Field2 {
implicit val reader = (
splitter((__ \ "field2").read[String], (__ \ "num2"))
) // Obviously incorrect syntax + type mismatch, but this is roughly what I'm trying to accomplish.
}
If you use the non-functional combinator syntax it becomes a little simpler I think:
object Field2 {
implicit val reader = Reads[Field2] { json =>
for {
path <- (json \ "field2").validate[String]
num2 <- (json \ "num2").validate[Int]
} yield splitter(path, num2)
}
}
Additionally if you want splitter to further validate the input have it return a JsResult[Field2] like this:
def splitter(path: String, num2: Int): JsResult[Field2] = {
val split = path.split("\\")
if (split.size != 2) {
JsError(s"$path must be of the form: {field}\\{num}")
} else {
Try(Field2(split(0), split(1).toInt, num2)).map(JsSuccess(_)).getOrElse {
JsError(s"${split(1)} is not a valid Int")
}
}
}
And then modify the reader:
object Field2 {
implicit val reader = Reads[Field2] { json =>
for {
path <- (json \ "field2").validate[String]
num2 <- (json \ "num2").validate[Int]
field2 <- splitter(path, num2)
} yield field2
}
}
If you still prefer using the functional syntax and you don't mind the lack of validation that splitter does try this:
def splitter(path: String, num2: Int): Field2 = {
val split = path.split("\\")
Field2(split(0), split(1).toInt, num2)
}
implicit val reader = (
(__ \ "field2").read[String] and
(__ \ "num2").read[Int]
)(splitter _)
I'd recommend against this, it's unsafe (split(1) and toInt both might throw exceptions) and the functional syntax can have performance issues. See https://www.lucidchart.com/techblog/2016/08/29/speeding-up-restful-services-in-play-framework/.
I don't know why do you need case classes, but you can also transform json for your needs with
(__ \ "field2" \ "num2").json.copyFrom((__ \ "num2").json.pick) and
(__ \ "field2").json.update(
of[String].map { o =>
val split = o.split("/")
Json.obj(
"text" -> split(0),
"num" -> split(1).toInt
)
}
)
).reduce andThen (__ \ "num2").json.prune
scala> j: play.api.libs.json.JsResult[play.api.libs.json.JsObject] = JsSuccess({"field2":{"num2":2,"text":"value","num":1},"field1":"value1"},/num2)
and then you can use your Reads[Field1]
Play's JSON serialization is by default permissive when serializing from JSON into a case class. For example.
case class Stuff(name: String, value: Option[Boolean])
implicit val stuffReads: Reads[Stuff] = (
( __ \ 'name).read[String] and
( __ \ 'value).readNullable[Boolean]
)(Stuff.apply _)
If the following JSON was received:
{name: "My Stuff", value: true, extraField: "this shouldn't be here"}
It will succeed with a 'JsSuccess' and discard the 'extraField'.
Is there a way to construct the Json Reads function to have it return a JsError if there are 'unhandled' fields?
You can verify that the object doesn't contain extra keys before performing your own decoding:
import play.api.data.validation.ValidationError
def onlyFields(allowed: String*): Reads[JsObject] = Reads.filter(
ValidationError("One or more extra fields!")
)(_.keys.forall(allowed.contains))
Or if you don't care about error messages (and that one's not very helpful, anyway):
def onlyFields(allowed: String*): Reads[JsObject] =
Reads.verifying(_.keys.forall(allowed.contains))
And then:
implicit val stuffReads: Reads[Stuff] = onlyFields("name", "value") andThen (
(__ \ 'name).read[String] and
(__ \ 'value).readNullable[Boolean]
)(Stuff)
The repetition isn't very nice, but it works.
Inspired from Travis' comment to use LabelledGeneric I was able achieve compile time safe solution.
object toStringName extends Poly1 {
implicit def keyToStrName[A] = at[Symbol with A](_.name)
}
case class Foo(bar: String, boo: Boolean)
val labl = LabelledGeneric[Foo]
val keys = Keys[labl.Repr].apply
now keys.map (toStringName).toList will give you
res0: List[String] = List(bar, boo)
I'm using play-json's macros to define implicit Writes for serializing JSON. However, it seems like by default play-json omits fields for which Option fields are set to None. Is there a way to change the default so that it outputs null instead? I know this is possible if I define my own Writes definition, but I'm interested in doing it via macros to reduce boilerplate code.
Example
case class Person(name: String, address: Option[String])
implicit val personWrites = Json.writes[Person]
Json.toJson(Person("John Smith", None))
// Outputs: {"name":"John Smith"}
// Instead want to output: {"name":"John Smith", "address": null}
The Json.writes macro generates a writeNullable[T] for optional fields. Like you know (or not), writeNullable[T] omits the field if the value is None, whereas write[Option[T]] generates a null field.
Defining a custom writer is the only option you have to get this behavior.
(
(__ \ 'name).write[String] and
(__ \ 'address).write[Option[String]]
)(unlift(Person.unapply _))
You can use a custom implicit JsonConfiguration, see Customize the macro to output null
implicit val config = JsonConfiguration(optionHandlers = OptionHandlers.WritesNull)
implicit val personWrites = Json.writes[Person]
Json.toJson(Person("John Smith", None))
Not a real solution for you situation. But slightly better than having to manually write the writes
I created a helper class that can "ensure" fields.
implicit class WritesOps[A](val self: Writes[A]) extends AnyVal {
def ensureField(fieldName: String, path: JsPath = __, value: JsValue = JsNull): Writes[A] = {
val update = path.json.update(
__.read[JsObject].map( o => if(o.keys.contains(fieldName)) o else o ++ Json.obj(fieldName -> value))
)
self.transform(js => js.validate(update) match {
case JsSuccess(v,_) => v
case err: JsError => throw new JsResultException(err.errors)
})
}
def ensureFields(fieldNames: String*)(value: JsValue = JsNull, path: JsPath = __): Writes[A] =
fieldNames.foldLeft(self)((w, fn) => w.ensureField(fn, path, value))
}
so that you can write
Json.writes[Person].ensureFields("address")()
Similar answer to above, but another syntax for this:
implicit val personWrites = new Writes[Person] {
override def writes(p: Person) = Json.obj(
"name" -> p.name,
"address" -> p.address,
)
}
This is simple:
implicit val personWrites = new Writes[Person] {
override def writes(p: Person) = Json.obj(
"name" -> p.name,
"address" -> noneToString(p.address),
)
}
def optToString[T](opt: Option[T]) =
if (opt.isDefined) opt.get.toString else "null"
You can define something like this :
implicit class JsPathExtended(path: JsPath) {
def writeJsonOption[T](implicit w: Writes[T]): OWrites[Option[T]] = OWrites[Option[T]] { option =>
option.map(value =>
JsPath.createObj(path -> w.writes(value))
).getOrElse(JsPath.createObj(path -> JsNull))
}
}
And if you are using play framework :
implicit val totoWrites: Writes[Toto] = (
(JsPath \ "titre").write[String] and
(JsPath \ "option").writeJsonOption[String] and
(JsPath \ "descriptionPoste").writeNullable[String]
) (unlift(Toto.unapply))
implicit val totoReads: Reads[Toto] = (
(JsPath \ "titre").read[String] and
(JsPath \ "option").readNullable[String] and
(JsPath \ "descriptionPoste").readNullable[String]
) (Toto.apply _)
You may wrap your option and redefine serialization behavior:
case class MyOption[T](o: Option[T])
implicit def myOptWrites[T: Writes] = Writes[MyOption[T]](_.o.map(Json.toJson).getOrElse(JsNull))
CAVEATS:
1)this approach is good only for case classes used solely as a serialization protocol definition. If you are reusing some data model classes in controllers - define custom serializers for them not to pollute the model.
2)(related only to Option) if you use the same class for writes and reads. Play will require the wrapped fields to be present (possibly null) during deserialization.
P.S.: Failed to do the same with type tagging. Compiler error is like
No instance of play.api.libs.json.Writes is available for tag.<refinement> (given the required writes were explicitly defined).
Looks like Play's macro fault.
I am attempting to validate JSON when unmarshalling to an object in Play2.1. The Format object I have defined only validates when a field is absent in the JSON, but I want to validate that fields are nonEmpty strings. Is this possible? I've tried specifying the minLength() constraint (as seen here) in the reads() call, but I get a compiler error saying minLength can't be found. Is that only for the tuple approach?
See the following Specs2 Junit test which fails now, but should pass when the constraint is defined properly:
import org.specs2.mutable._
import play.api.libs.json._
class SimpleValidation extends SpecificationWithJUnit{
private val badPayload: JsValue = Json.obj(
"simpleValue1" -> "mySimpleValue", // Comment this line out to pass test
"simpleValue2" -> ""
)
"An IssueFormat" should {
"validate when unmarshalling" in {
badPayload.validate[SimpleObj].fold(
valid = (res => {
// Fail if valid
failure("Payload should have been invalid")
}),
invalid = (e => {
// Should be one error
e.length mustBeEqualTo(1)
}))
}
}
}
import play.api.libs.functional.syntax._
case class SimpleObj(simpleValue1: String, simpleValue2: String)
object SimpleObj {
val simpleReads = (
(__ \ "simpleValue1").read[String] and
(__ \ "simpleValue2").read[String])(SimpleObj.apply _) // read[String](minLength(0)) yields compiler error
val simpleWrites = (
(__ \ "simpleValue1").write[String] and
(__ \ "simpleValue2").write[String])(unlift(SimpleObj.unapply))
implicit val simpleFormat: Format[SimpleObj] = Format(simpleReads, simpleWrites)
}
You can use the minLength in Reads:
import play.api.libs.json.Reads._
Then minLength should be available, however, try this format instead:
implicit val simpleReads = (
(__ \ "simpleValue1").read(minLength[String](1)) and
(__ \ "simpleValue2").read(minLength[String](1))(SimpleObj.apply _)
After looking through the Play2.1 documentation some more, I was able to add a custome read validator. If you replace the SimpleObj from the original question, with the following, the test case will pass. Not sure if there is a simpler way to do this, but this definitely works:
object SimpleObj {
// defines a custom reads to be reused
// a reads that verifies your value is not equal to a give value
def notEqual[T](v: T)(implicit r: Reads[T]): Reads[T] = Reads.filterNot(ValidationError("validate.error.unexpected.value", v))(_ == v)
implicit val simpleReads = (
(__ \ "simpleValue1").read[String](notEqual("")) and
(__ \ "simpleValue2").read[String](notEqual("")))(SimpleObj.apply _)
val simpleWrites = (
(__ \ "simpleValue1").write[String] and
(__ \ "simpleValue2").write[String])(unlift(SimpleObj.unapply))
implicit val simpleFormat: Format[SimpleObj] = Format(simpleReads, simpleWrites)
}