I've been using this json combinator for several basic / standard cases without really understanding how it works. All was fine.
Now I want to get myself prepared for whatever advanced cases might come; I need to understand the code.
Ref.: https://www.playframework.com/documentation/2.3.x/ScalaJsonCombinators
I think I can understand the Reads:
implicit val locationReads: Reads[Location] = (
(JsPath \ "lat").read[Double] and
(JsPath \ "long").read[Double]
)(Location.apply _)
It creates a Reads that:
First -- when given a JsValue (through its "reads" method) -- it pulls the "lat", followed by the "long". Out of those two it creates a tuple (Double, Double). -- https://www.playframework.com/documentation/2.3.x/api/scala/index.html#play.api.libs.json.Reads
That tuple is then assigned to the partial function of that Reads..., which in this case is whatever returned by "Location.apply _". I tried it in repl:
...
scala> val yowMan = Location.apply _
yowMan: (Double, Double) => Location = <function2>
scala> yowMan(1, 2)
res14: Location = Location(1.0,2.0)
That partial function takes the a tuple of (Double, Double) as input. So..., the outcome of step 1 is channeled to step 2, and we get an instance of Location as the return of "reads".
Now for the Writes:
implicit val locationWrites: Writes[Location] = (
(JsPath \ "lat").write[Double] and
(JsPath \ "long").write[Double]
)(unlift(Location.unapply))
First the "unapply". I tried in repl:
scala> val heyDude = Location.unapply
<console>:16: error: missing arguments for method unapply in object Location;
follow this method with `_' if you want to treat it as a partially applied function
val heyDude = Location.unapply
Oops, ok, I followed the instruction:
scala> val heyDude = Location.unapply _
heyDude: Location => Option[(Double, Double)] = <function1>
Ok, so we get a partial function that transforms an instance of Location to an (optional) tuple of (Double, Double).
Next, the "unlift":
scala> val hohoho = unlift(heyDude)
hohoho: Location => (Double, Double) = <function1>
scala> val loc = Location(1, 2)
loc: Location = Location(1.0,2.0)
scala> hohoho(loc)
res16: (Double, Double) = (1.0,2.0)
Ok, so... unlift simply throws away the "Option", and takes us directly to the tuple.
Ok... so... I guess... this "writes" of the Writes... *) when given an instance of Location, it will:
Pass that object through that partial function produced by unlift(Location.unapply).
The tuple (Double, Double) returned by that partial function is then channeled to whatever is produced by this:
(JsPath \ "lat").write[Double] and
(JsPath \ "long").write[Double]
What exactly is that "whatever"? Following the API doc of JsPath, I think it is OWrites: https://www.playframework.com/documentation/2.3.x/api/scala/index.html#play.api.libs.json.OWrites
But... I can't see there's a method named "and" in OWrites. Where is this "and" declared? And what does it do? Is it: "oWrites1 and oWrites2" produces "oWrites3"? And this "oWrites3" is a special type of OWrites that takes tuple as input? ... If that's the case... the tuple doesn't have information about the name of the property in the case class ("lat" and "long"). How does it know that the produced json string should be {"lat": 1, "long": 2} then?
Sorry for the train of questions. Please help me obtaining a clear understanding of this. Thanks!
*) https://www.playframework.com/documentation/2.3.x/api/scala/index.html#play.api.libs.json.Writes
UPDATES:
Adding related question: Syntax and meaning of a Scala/Play! code sample
When in doubt, decompile it. This showed that there is an implicit toFunctionalBuilderOps, which then you can see in FunctionalBuilderOps that there is your and method
Related
I am unable to understand example code of JsPath and Read in the documentation
https://www.playframework.com/documentation/2.2.1/ScalaJsonCombinators
import play.api.libs.json._
import play.api.libs.functional.syntax._
Question 1 - We create a custom reader. Reads should be able to read a structure of data consisting of String, Float and a List. But in the example below, we pass it a Json! How is the Json getting converted to (String, Fload and List)?
Question 2 - we use JsPath \ "key1" but where have we passed the JSON?
val customReads: Reads[(String, Float, List[String])] =
(JsPath \ "key1").read[String](email keepAnd minLength(5)) and
(JsPath \ "key2").read[Float](min(45)) and
(JsPath \ "key3").read[List[String]]
tupled
import play.api.libs.json.Json
val js = Json.obj(
"key1" -> "alpha",
"key2" -> 123.345F,
"key3" -> Json.arr("alpha", "beta")
)
res5: JsSuccess(("alpha", 123.345F, List("alpha", "beta")))
scala> customReads.reads(js)
customReads.reads(js).fold(
invalid = { errors => ... },
valid = { res =>
val (s, f, l): (String, Float, List[String]) = res
...
}
)
Question 1
We create a custom reader.
Yes
Reads should be able to read a structure of data consisting of String, Float and a List.
No
val customReads: Reads[(String, Float, List[String])]= ...
Means that customReads is the variable of type Reads, and that Reads.reads must return tuple of types (String, Float, List[String])
But in the example below, we pass it a JSON! How is the JSON getting converted to (String, Float and List)?
Reads.reads is a function that takes JSON as a parameter and returns some value, extracted from the JSON by defined rules. In our case, the rules are:
(JsPath \ "key1").read[String](email keepAnd minLength(5))
and
(JsPath \ "key2").read[Float](min(45))
and
(JsPath \ "key3").read[List[String]]
all these values must be
tupled
so our Reads.reads function returns (String, Float, List[String]) tuple.
Question 2
we use JsPath \ "key1" but where have we passed the JSON?
JsPath \ "key1" is not the actual code to process the JSON, it's a rule to process a JSON. i.e it is like XPath expression /key1
I have the following case class:
case class User(name: String).
I am trying to implement a JSON Reads converter for it, so I can do the following:
val user = userJson.validate[User]
… but the incoming JSON has slightly different structure:
{ "firstName": "Bob", "lastName": "Dylan" }.
How can I implement my JSON Reads converter to combine the JSON fields firstName and lastName into a name property on my class?
This should do the trick:
implicit val userReads: Reads[User] =
for {
first <- (__ \ "firstName").read[String]
last <- (__ \ "lastName").read[String]
} yield User(s"$first $last")
EDIT
Without using a for comprehension
implicit val userReads =
{
(__ \ "firstName").read[String] and
(__ \ "lastName"
}.read[String] ).tupled.map(t => User(s"${t._1} ${t._2}"))
Bringing userReads in scope where you want to use it will let you parse the JSON you provided.
Reads is essentially a function from JsValue to JsResult, meaning userReads represents a function from JsValue -> JsResult. Within the function, it first inspects the provided JSON & tries to read out a property named "firstName" from the current JSON path (__ is shorthand for this). \ indicates that the field its looking for is one level beneath the root, and read[String] means the value associated with the "firstName" key should be read as a string. Same follows for "lastName".
Edit
In the version without the for comprehension, it first creates an intermediary object FunctionalBuilder[Reads]#CanBuild[String, String], which is a complicated way of saying it reads two distinct strings from the Json. Next it converts that complex object into a Reads[(String, String)] by way of tupled. Finally it maps the pair of strings into a User.
Were you to try validating some JSON without "firstName" & "lastName", this will fail with a validation error for a missing path.
I have a case class
case class Foo(bar: Int, baz: String)
and I am trying to write a JSON serialization for it. But I have an additional requirement, to add one more field to the generated Json (say "greeting": "Hello")
I am trying something along these lines:
val writes = ((JsPath \ "bar").write[Int] and
(JsPath \ "baz").write[String] and
(JsPath \ "greeting").write[String])(unlift(Foo.unapply))
But, how should I pass the static "Hello" string to the above Writes?
And how can I use this writes to create a Format for my Foo class?
I would do it like this:
val writes = Writes[Foo](f => {
Json.obj(
"bar" -> f.bar,
"baz" -> f.baz,
"greeting" -> "Hello")
})
Drawback is that you have to specify the members twice, but it's handy for special cases like this where you need additional control.
In a one-off scenario, where you don't want to define yet another Writes, you could do this:
Json.toJson(foo).as[JsObject] ++ Json.obj("greeting" -> "hello")
I have ended up with another solution:
val writes = ((JsPath \ "bar").write[Int] and
(JsPath \ "baz").write[String] and
(JsPath \ "greeting").write[String])((f: Foo) => (f.bar, f.baz, "Hello"))
So, I've just recently started learning Scala. Sorry for my incompetence in advance.
I tried to look up my answer on stackoverflow. I was able to find several related topics, but I didn't spot my problem.
I'm trying to send a json response based on a Scala object. I have an Action and I'm doing the following:
def oneCredential = Action {
val cred = Credential("John", "Temp", "5437437")
Ok(Json.toJson(cred))
}
I've created a case class and appropriate implicit Writes[T] for it
import play.api.libs.json._
import play.api.libs.functional.syntax._
import play.api.libs.json.util._
case class Credential(name: String, account: String, password: String)
object Credential{
implicit val credentialWrites = (
(__ \ "name").write[String] and
(__ \ "account").write[String] and
(__ \ "password").write[String]
)(Credential)
}
When I'm trying to run this, I've the following error: "Overloaded method value [apply] cannot be applied to (models.Credential.type)". Also, I tried this
implicit val credentialWrites = (
(__ \ "name").write[String] and
(__ \ "account").write[String] and
(__ \ "password").write[String]
)(Credential.apply _)
Fail. The error: could not find implicit value for parameter fu: play.api.libs.functional.Functor[play.api.libs.json.OWrites]
Then this:
implicit val credentialWrites = (
(__ \ "name").writes[String] and
(__ \ "account").writes[String] and
(__ \ "password").writes[String]
)(Credential)
Another fail: "value writes is not a member of play.api.libs.json.JsPath Note: implicit value credentialWrites is not applicable here because it comes after the application point and it lacks an explicit result type". Right, I understood the first part of an error, but not the second.
Finally I found a shorthand solution:
implicit val credentialWrites = Json.writes[Credential]
With this I've got no errors and the code finally worked. I've found the solution on this blog. It's said that the shorthand form is exactly the same as the one with "writes" above. But this "long" form didn't work for me.
Why is shorthand version working, while the long one isn't? Can somebody explain this?
Thank you!
PS Scala version: 2.10.2
The definitions you've given would work for Reads, but Writes needs a different kind of argument at the end. Take the following example:
case class Baz(foo: Int, bar: String)
val r = (__ \ 'foo).read[Int] and (__ \ 'bar).read[String]
val w = (__ \ 'foo).write[Int] and (__ \ 'bar).write[String]
r can be applied to a function (Int, String) => A to get a Reads[A], which means we can use it as follows (these are all equivalent):
val bazReader1 = r((foo: Int, bar: String) => Baz(foo, bar))
val bazReader2 = r(Baz.apply _)
val bazReader3 = r(Baz)
What we're doing is lifting the function into the applicative functor for Reads so that we can apply it to our Reads[Int] and Reads[String] (but you don't need to care about that if you don't want to).
w takes a different kind of argument (again, you don't need to care, but this is because Writes has a contravariant functor—it doesn't have an applicative functor):
val bazWriter1 = w((b: Baz) => (b.foo, b.bar))
We could write this equivalently as the following:
val bazWriter2 = w(unlift(Baz.unapply))
Here we're using the case class's automatically generated extractor, unapply, which returns an Option[(Int, String)]. We know in this case that it'll always return a Some, so we can use unlift (which comes from the functional syntax package, and just calls the standard library's Function.unlift) to turn the Baz => Option[(Int, String)] into the required Baz => (Int, String).
So just change your final line to )(unlift(Credential.unapply)) and you're good to go.
I have an equivalent of the following model in play scala :
case class Foo(id:Int,value:String)
object Foo{
import play.api.libs.json.Json
implicit val fooFormats = Json.format[Foo]
}
For the following Foo instance
Foo(1, "foo")
I would get the following JSON document:
{"id":1, "value": "foo"}
This JSON is persisted and read from a datastore. Now my requirements have changed and I need to add a property to Foo. The property has a default value :
case class Foo(id:String,value:String, status:String="pending")
Writing to JSON is not a problem :
{"id":1, "value": "foo", "status":"pending"}
Reading from it however yields a JsError for missing the "/status" path.
How can I provide a default with the least possible noise ?
(ps: I have an answer which I will post below but I am not really satisfied with it and would upvote and accept any better option)
Play 2.6+
As per #CanardMoussant's answer, starting with Play 2.6 the play-json macro has been improved and proposes multiple new features including using the default values as placeholders when deserializing :
implicit def jsonFormat = Json.using[Json.WithDefaultValues].format[Foo]
For play below 2.6 the best option remains using one of the options below :
play-json-extra
I found out about a much better solution to most of the shortcomings I had with play-json including the one in the question:
play-json-extra which uses [play-json-extensions] internally to solve the particular issue in this question.
It includes a macro which will automatically include the missing defaults in the serializer/deserializer, making refactors much less error prone !
import play.json.extra.Jsonx
implicit def jsonFormat = Jsonx.formatCaseClass[Foo]
there is more to the library you may want to check: play-json-extra
Json transformers
My current solution is to create a JSON Transformer and combine it with the Reads generated by the macro. The transformer is generated by the following method:
object JsonExtensions{
def withDefault[A](key:String, default:A)(implicit writes:Writes[A]) = __.json.update((__ \ key).json.copyFrom((__ \ key).json.pick orElse Reads.pure(Json.toJson(default))))
}
The format definition then becomes :
implicit val fooformats: Format[Foo] = new Format[Foo]{
import JsonExtensions._
val base = Json.format[Foo]
def reads(json: JsValue): JsResult[Foo] = base.compose(withDefault("status","bidon")).reads(json)
def writes(o: Foo): JsValue = base.writes(o)
}
and
Json.parse("""{"id":"1", "value":"foo"}""").validate[Foo]
will indeed generate an instance of Foo with the default value applied.
This has 2 major flaws in my opinion:
The defaulter key name is in a string and won't get picked up by a refactoring
The value of the default is duplicated and if changed at one place will need to be changed manually at the other
The cleanest approach that I've found is to use "or pure", e.g.,
...
((JsPath \ "notes").read[String] or Reads.pure("")) and
((JsPath \ "title").read[String] or Reads.pure("")) and
...
This can be used in the normal implicit way when the default is a constant. When it's dynamic, then you need to write a method to create the Reads, and then introduce it in-scope, a la
implicit val packageReader = makeJsonReads(jobId, url)
An alternative solution is to use formatNullable[T] combined with inmap from InvariantFunctor.
import play.api.libs.functional.syntax._
import play.api.libs.json._
implicit val fooFormats =
((__ \ "id").format[Int] ~
(__ \ "value").format[String] ~
(__ \ "status").formatNullable[String].inmap[String](_.getOrElse("pending"), Some(_))
)(Foo.apply, unlift(Foo.unapply))
I think the official answer should now be to use the WithDefaultValues coming along Play Json 2.6:
implicit def jsonFormat = Json.using[Json.WithDefaultValues].format[Foo]
Edit:
It is important to note that the behavior differs from the play-json-extra library. For instance if you have a DateTime parameter that has a default value to DateTime.Now, then you will now get the startup time of the process - probably not what you want - whereas with play-json-extra you had the time of the creation from the JSON.
I was just faced with the case where I wanted all JSON fields to be optional (i.e. optional on user side) but internally I want all fields to be non-optional with precisely defined default values in case the user does not specify a certain field. This should be similar to your use case.
I'm currently considering an approach which simply wraps the construction of Foo with fully optional arguments:
case class Foo(id: Int, value: String, status: String)
object FooBuilder {
def apply(id: Option[Int], value: Option[String], status: Option[String]) = Foo(
id getOrElse 0,
value getOrElse "nothing",
status getOrElse "pending"
)
val fooReader: Reads[Foo] = (
(__ \ "id").readNullable[Int] and
(__ \ "value").readNullable[String] and
(__ \ "status").readNullable[String]
)(FooBuilder.apply _)
}
implicit val fooReader = FooBuilder.fooReader
val foo = Json.parse("""{"id": 1, "value": "foo"}""")
.validate[Foo]
.get // returns Foo(1, "foo", "pending")
Unfortunately, it requires writing explicit Reads[Foo] and Writes[Foo], which is probably what you wanted to avoid? One further drawback is that the default value will only be used if the key is missing or the value is null. However if the key contains a value of the wrong type, then again the whole validation returns a ValidationError.
Nesting such optional JSON structures is not a problem, for instance:
case class Bar(id1: Int, id2: Int)
object BarBuilder {
def apply(id1: Option[Int], id2: Option[Int]) = Bar(
id1 getOrElse 0,
id2 getOrElse 0
)
val reader: Reads[Bar] = (
(__ \ "id1").readNullable[Int] and
(__ \ "id2").readNullable[Int]
)(BarBuilder.apply _)
val writer: Writes[Bar] = (
(__ \ "id1").write[Int] and
(__ \ "id2").write[Int]
)(unlift(Bar.unapply))
}
case class Foo(id: Int, value: String, status: String, bar: Bar)
object FooBuilder {
implicit val barReader = BarBuilder.reader
implicit val barWriter = BarBuilder.writer
def apply(id: Option[Int], value: Option[String], status: Option[String], bar: Option[Bar]) = Foo(
id getOrElse 0,
value getOrElse "nothing",
status getOrElse "pending",
bar getOrElse BarBuilder.apply(None, None)
)
val reader: Reads[Foo] = (
(__ \ "id").readNullable[Int] and
(__ \ "value").readNullable[String] and
(__ \ "status").readNullable[String] and
(__ \ "bar").readNullable[Bar]
)(FooBuilder.apply _)
val writer: Writes[Foo] = (
(__ \ "id").write[Int] and
(__ \ "value").write[String] and
(__ \ "status").write[String] and
(__ \ "bar").write[Bar]
)(unlift(Foo.unapply))
}
This probably won't satisfy the "least possible noise" requirement, but why not introduce the new parameter as an Option[String]?
case class Foo(id:String,value:String, status:Option[String] = Some("pending"))
When reading a Foo from an old client, you'll get a None, which I'd then handle (with a getOrElse) in your consumer code.
Or, if you don't like this, introduce an BackwardsCompatibleFoo:
case class BackwardsCompatibleFoo(id:String,value:String, status:Option[String] = "pending")
case class Foo(id:String,value:String, status: String = "pending")
and then turn that one into a Foo to work with further on, avoiding to have to deal with this kind of data gymnastics all along in the code.
You may define status as an Option
case class Foo(id:String, value:String, status: Option[String])
use JsPath like so:
(JsPath \ "gender").readNullable[String]