Using play-json 2.3
How do i parse this
{i: 0}
into this
case class A(i: Int, s: Seq[Int])
i would very much like to reuse Json.format macro
at the moment it gives me "missing path s"
UPD:
i ended up writing a custom format and pimping it onto JsPath:
implicit class JsPathExtensions(path: JsPath) {
//return mzero if the path is missing
def lazyFormatNullableM[T: Monoid](f: => Format[T]): OFormat[T] = OFormat(path.lazyReadNullable(f).map(_.orZero), new OWrites[T] {
override def writes(o: T) = path.lazyWriteNullable(f).writes(o.some)
})
}
If the path is missing, I don't see any way to utilize the macro without changing the class. In which case, you can define Reads using json combinators and use the macro for Writes.
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val reads: Reads[A] = (
(__ \ "i").read[Int] and
(__ \ "s").read[Seq[Int]].orElse(Reads.pure(Nil))
)(A.apply _)
implicit val writes: Writes[A] = Json.writes[A]
The json macros only cover a very limited range of use cases, and it's inevitable that you'll run into a case where you must write your own Reads or Writes.
The easiest way to utilise the Json.format[A] macro would be to slightly alter your definition of A:
case class A(i: Int, s: Option[Seq[Int]])
The usage of an Option here is pretty clear and you can map over it and do lots of good things, but if you really want to treat s like it's an empty collection, you could enhance your case class a little more:
case class Foo(i: Int, s: Option[Seq[Int]]) {
lazy val theSeq:Seq[Int] = s.getOrElse(Seq[Int]())
}
Now if you access theSeq, there is absolutely no difference between incoming JSON that omitted s and JSON that supplied an empty array for s.
Related
I can't convert list in specific case: when type is extends from trait.
When I can convert:
import play.api.libs.functional.syntax._
import play.api.libs.json._
sealed trait Item
case class Id(id: Long) extends Item
case class MyList(list: List[Id])
object MyFormat {
implicit lazy val idFormat = Json.format[Id]
implicit lazy val myListFormat = Json.format[MyList]
}
When I can not convert:
sealed trait Item
case class Id(id: Long) extends Item
case class MyList(list: List[Id])
object MyFormat {
implicit lazy val itemFormat = new Format[Item] {
override def writes(o: Item): JsValue = o match {
case i: Id => idFormat.writes(i)
}
override def reads(json: JsValue): JsResult[Item] = {
idFormat.reads(json)
}
}
implicit lazy val idFormat = Json.format[Id]
implicit lazy val myListFormat = Json.format[MyList]
}
Error:
Error:(33, 49) No instance of play.api.libs.json.Format is available for scala.collection.immutable.List[Main2.Id] in the implicit scope (Hint: if declared in the same file, make sure it's declared before)
implicit lazy val myListFormat = Json.format[MyList]
Why I can't format in 2nd case?
If I add formatter for list:
implicit lazy val idsFormat = Json.format[List[Id]]
then I got Error:(33, 46) No instance of Reads is available for scala.collection.immutable.Nil in the implicit scope (Hint: if declared in the same file, make sure it's declared before)
implicit lazy val idsFormat = Json.format[List[Id]]
PS:
The only one solution than I found:
Define custom format for List[Id]
When read or write, use format for Id
When read, use
def flatten[T](xs: Seq[JsResult[T]]): JsResult[List[T]] = {
val (ss: Seq[JsSuccess[T]], fs: Seq[JsError]) = xs.partition(_.isSuccess)
if (fs.isEmpty) JsSuccess(ss.map(_.get).toList) else fs.head
}
If play JSON features automatic/semiautomatic codec instances derivation, then it will use implicit to enable such a mechanism. It means that codecs for complex things should be introduced after their components.
In your case, it seems like play JSON tries to derive codec for List as for case class ,i. e. as for List(a1, List(a2, ..., List(an, Nil))) and when it hits Nil, it doesn't know how to derive codec for that.
I believe you want your list encoded not like a chain of folded objects but as JSON array.
Then you should search play sources for default List[T] codec and try to use it by specializing it for Id.
General tool for debugging missing implicits is compiler option "-Xlog-implicits". It will log all failed implicit searches to console, and it is possible to figure out what's missing by these messages.
It is also strongly advised to know how implicit works before working with libraries that use this feautre extensively.
And last, but not least:
Have you ever tried using circe? It features automatic and semi-automatic JSON derivation for families of sealed traits and standard scala classes. It even has integration with play framework. Circe derivation removes the most headache of writing codec code, but requires strong knowledge of implicit precedence to work properly. The way it works described here and here.
You can also try to create your own derivation for play-json if there's no adequate standard one with morphling.
As a disclaimer: I'm very new to Scala and functional programming in general.
I have the following classes:
case class A(
field1: String,
field2: DateTime
)
case class B(
listOfStuff: Seq[A]
)
object A{
def create(field1: String, field2: DateTime): A = A(field1, field2)
}
object B{
def create(listOfStuff: Seq[A]): B = B(listOfStuff)
}
(The create() functions exist because I sometimes had issues in my code when using apply(). Let's ignore this, I doubt it's relevant.)
I get these objects in JSON format and I try to parse them using the Play-JSON library. An important aspect is that the list (Seq) can be missing from the JSON text, it's an optional field.
With that in mind, this is how I wrote that particular line:
private implicit val messageReader = (
//...
(__ \ "stuff").readNullable[Seq[A]].map(_.getOrElse(Seq()))
//...
)(B.create _)
When compiling, I get the following error:
Error:(!line!, !column!) No Json deserializer found for type Seq[A].
Try to implement an implicit Reads or Format for this type.
From what I saw in this question, apparently you need to have an implicit instance of Reads[T] for every type T that is not part of the language (or something like that, did I mention I'm new to this?)
So I also added a secondary Reads[T] in the same scope, my code now looked like this:
private implicit val messageReader = (
(__ \ "stuff").readNullable[Seq[A]].map(_.getOrElse(Seq()))
)(B.create _)
private implicit val anotherReader = (
(__ \ "field1").read[String] and
(__ \ "field2").read[String].map(org.joda.time.DateTime.parse)
)(A.create _)
However I still get that same error on the same line.
I feel like there's some very easy and obvious problem here, but I can't figure out what it is.
How do I fix this?
if B.create accepts a single argument then its reads can look like this:
private implicit val messageReader : Reads[B] =
(__ \ "stuff").readNullable[Seq[A]].map(_.getOrElse(Seq())).map(B.create)
I have a use case where I want to serialize a User entity to json. This user entity includes private fields I don't want to expose, such as password.
I'm using a custom OWrites to deal with this when I return a single user:
val userSafeWrites: OWrites[User] = (
(__ \ EMAIL_FIELD).write[String] ~
(__ \ FIRST_NAME_FIELD).write[String] ~
(__ \ LAST_NAME_FIELD).write[String] ~
(__ \ PHONE_NUMBER_FIELD).write[Option[String]] ~
(__ \ ID_FIELD).write[Long]
)(p => (p.email, p.firstName, p.lastName, p.phoneNumber, p._id.get))
and then I specify the OWrites instead of using the implicit:
Ok(Json.toJson(user)(User.userSafeWrites))
However, I have now to return a Set[User].
How can I do that? Do I need to implement a OWrites[Set[User]]? I can understand how to do that if I was to return an object with a field name with the results, such as:
{
"users": [{user1}, {user2}]
}
However, I want to return simply an array, to comform to the output in other endpoints:
[{user1}, {user2}]
Or I should map each element of the set to a JsObject and apply the custom OWrites to each object? What would be the most efficient way to do that?
I feel this is something pretty simple and I'm just being a moron for not finding the answer myself.
You are reimplementing writes for traversable that are trivial anyway but still. Another option is to reuse that Writes
val users: Set[User] = ???
Json.toJson(users)(Writes.traversableWrites(userSafeWrites))
Writes for Traversable are defined as follows:
implicit def traversableWrites[A: Writes] = Writes[Traversable[A]] { as =>
JsArray(as.map(toJson(_)).toSeq)
}
Which is just what you did.
It takes as implicit parameter of type Writes[A] and uses it to write each member. You can pass this parameter explicitly to obtain desired Writes.
Another option as #cchantep mentioned is just to import your implicit where you need it.
For example, having some model with default writes
case class User(num: Int)
object User {
implicit val writes = Json.writes[User]
}
and another object with the custom writes
object OtherWrites {
implicit val custom: Writes[User] = new Writes[User] {
override def writes(o: User): JsValue = JsNull
}
}
in the class that is the client
object Client extends App {
val obj = List(User(1))
print(Json.toJson(obj))
}
You would get [{"num":1}] printed as default writes are present in the implicit scope because they are defined in companion object of User.
You can import custom writes where you need them and those from companion object will be overriden
object Client extends App {
import test.OtherWrites._
val obj = List(User(1))
print(Json.toJson(obj))
}
And you get [null] printed.
Meh, it's as simple as:
Ok(JsArray(userSet.map(user => Json.toJson(user)(User.userSafeWrites)).toSeq))
Spent over one hour googling and trying to sort it out in the most ridiculous ways before posting the question.
Solved it in one minute after posting the question.
Having one of those days...
unless there's a better way to do it
For the validate method on request.body it matches the attribute name and value type of the json object to those defined in the model definition. Now if I were to add an extra attribute to the json object and try to validate it, it passes as a JsSuccess when it shouldn't.
{
"Name": "Bob",
"Age": 20,
"Random_Field_Not_Defined_in_Models": "Test"
}
My Person Class is defined as follows
case class Person(name: String, age: Int)
I'm assuming you've been using the built-in Reads[T] or Format[T] converters that Play gives you via Json.reads[T], e.g.:
import play.api.libs.json._
val standardReads = Json.reads[Person]
While these are super-handy, if you need additional validation, you'll have to define a custom Reads[Person] class; but fortunately we can still leverage the built-in JSON-to-case-class macro to do the basic checking and conversion, and then add an extra layer of custom checks if things seem OK:
val standardReads = Json.reads[Person]
val strictReads = new Reads[Person] {
val expectedKeys = Set("name", "age")
def reads(jsv:JsValue):JsResult[Person] = {
standardReads.reads(jsv).flatMap { person =>
checkUnwantedKeys(jsv, person)
}
}
private def checkUnwantedKeys(jsv:JsValue, p:Person):JsResult[Person] = {
val obj = jsv.asInstanceOf[JsObject]
val keys = obj.keys
val unwanted = keys.diff(expectedKeys)
if (unwanted.isEmpty) {
JsSuccess(p)
} else {
JsError(s"Keys: ${unwanted.mkString(",")} found in the incoming JSON")
}
}
}
Note how we utilize standardReads first, to make sure we're dealing with something that can be converted to a Person. No need to reinvent the wheel here.
We use flatMap to effectively short-circuit the conversion if we get a JsError from standardReads - i.e. we only call checkUnwantedKeys if needed.
checkUnwantedKeys just uses the fact that a JsObject is really just a wrapper around a Map, so we can easily check the names of the keys against a whitelist.
Note that you could also write that flatMap using a for-comprehension, which starts to look a lot cleaner if you need even more checking stages:
for {
p <- standardReads.reads(jsv)
r1 <- checkUnexpectedFields(jsv, p)
r2 <- checkSomeOtherStuff(jsv, r1)
r3 <- checkEvenMoreStuff(jsv, r2)
} yield r3
If you want to avoid too much boilerplate it is possible to make a more generic solution using a little bit of scala reflection:
import play.api.libs.json._
import scala.reflect.runtime.universe._
def checkedReads[T](underlyingReads: Reads[T])(implicit typeTag: TypeTag[T]): Reads[T] = new Reads[T] {
def classFields[T: TypeTag]: Set[String] = typeOf[T].members.collect {
case m: MethodSymbol if m.isCaseAccessor => m.name.decodedName.toString
}.toSet
def reads(json: JsValue): JsResult[T] = {
val caseClassFields = classFields[T]
json match {
case JsObject(fields) if (fields.keySet -- caseClassFields).nonEmpty =>
JsError(s"Unexpected fields provided: ${(fields.keySet -- caseClassFields).mkString(", ")}")
case _ => underlyingReads.reads(json)
}
}
}
Then you can specify your reads instances as:
implicit val reads = checkedReads(Json.reads[Person])
This leverages a fair bit of Scala type magic and also the reflection library (that lets you look at fields on classes).
Rather than relying on a fixed set of fields the classFields method gets all of the fields dynamically for the case class (type param T). It looks at all of the members and collects only the case class accessors (otherwise we'd pick up methods like toString). It returns a Set[String] of field names.
You'll notice that the checkedReads takes an implicit TypeTag[T]. This is supplied by the compiler at compile time and used by the typeOf method.
The remaining code is fairly self explanatory. If the incoming json matches our first case (it is a JsObject and there are fields not on the case class) then we return a JsError. Otherwise we pass it on to the underlying reader.
I'm looking for suggestions or libraries that can help me convert JSON (with nested structure) from one format to another in Scala.
I saw there are a few JavaScript and Java based solutions. Anything in Scala ?
I really like the Play JSON library. It's API is very clean and it's very fast even if some parts have a slightly steeper learning curve. You can also use the Play JSON library even if you aren't using the rest of Play.
https://playframework.com/documentation/2.3.x/ScalaJson
To convert JSON to scala objects (and vice versa), Play uses implicits. There is a Reads type which specifies how to convert JSON to a scala type, and a Writes type which specifies how to convert a scala object to JSON.
For example:
case class Foo(a: Int, b: String)
There are a few different routes you can take to convert Foo to JSON. If your object is simple (like Foo), Play JSON can create a conversion function for you:
implicit val fooReads = Json.reads[Foo]
or you can create a custom conversion function if you want more control or if your type is more complex. The below examples uses the name id for the property a in Foo:
implicit val fooReads = (
(__ \ "id").read[Int] ~
(__ \ "name").read[String]
)(Foo)
The Writes type has similar capabilities:
implicit val fooWrites = Json.writes[Foo]
or
implicit val fooWrites = (
(JsPath \ "id").write[Int] and
(JsPath \ "name").write[String]
)(unlift(Foo.unapply))
You can read more about Reads/Writes (and all the imports you will need) here: https://playframework.com/documentation/2.3.x/ScalaJsonCombinators
You can also transform your JSON without mapping JSON to/from scala types. This is fast and often requires less boilerplate. A simple example:
import play.api.libs.json._
// Only take a single branch from the input json
// This transformer takes the entire JSON subtree pointed to by
// key bar (no matter what it is)
val pickFoo = (__ \ 'foo).json.pickBranch
// Parse JSON from a string and apply the transformer
val input = """{"foo": {"id": 10, "name": "x"}, "foobar": 100}"""
val baz: JsValue = Json.parse(input)
val foo: JsValue = baz.transform(pickFoo)
You can read more about transforming JSON directly here: https://playframework.com/documentation/2.3.x/ScalaJsonTransformers
You can use Json4s Jackson. With PlayJson, you have to write Implicit conversions for all the case classes. If the no. of classes are small, and will not have frequent changes while development, PlayJson seems to be okay. But, if the case classes are more, I recommend using json4s.
You need to add implicit conversion for different types, so that json4s will understand while converting to json.
You can add the below dependency to your project to get json4s-jackson
"org.json4s" %% "json4s-jackson" % "3.2.11"
A sample code is given below (with both serialization and deserialization):
import java.util.Date
import java.text.SimpleDateFormat
import org.json4s.DefaultFormats
import org.json4s.jackson.JsonMethods._
import org.json4s.jackson.{Serialization}
/**
* Created by krishna on 19/5/15.
*/
case class Parent(id:Long, name:String, children:List[Child])
case class Child(id:Long, name:String, simple: Simple)
case class Simple(id:Long, name:String, date:Date)
object MainClass extends App {
implicit val formats = (new DefaultFormats {
override def dateFormatter = new SimpleDateFormat("yyyy-MM-dd")
}.preservingEmptyValues)
val d = new Date()
val simple = Simple(1L, "Simple", d)
val child1 = Child(1L, "Child1", simple)
val child2 = Child(2L, "Child2", simple)
val parent = Parent(1L, "Parent", List(child1, child2))
//Conversion from Case Class to Json
val json = Serialization.write(parent)
println(json)
//Conversion from Json to Case Class
val parentFromJson = parse(json).extract[Parent]
println(parentFromJson)
}