How to get rid of this implicit conversion? - json

Suppose I am using json4s to parse JSON:
val str = """{"a":"aaaa", "x": 0}"""
val json = JsonMethods.parse(str)
val a = for(JObject(fields) <- json; JField("a", JString(a)) <- fields) yield a
The type of a is List[String] but I need Option[String], so I am calling headOption:
val a = (
for(JObject(fields) <- json; JField("a", JString(a)) <- fields) yield a
).headOption
Since I found myself calling headOption again and again I tried an implicit conversion:
object X { implicit def foo[A](as: List[A]): Option[A] = as.headOption }
import X.foo
val a: Option[String] =
for(JObject(fields) <- json; JField("a", JString(a)) <- fields) yield a
The implicit conversion is working but I don't like it. What would you suggest ?

One approach is to use json4's typeclasses, e.g. json4s-scalaz has those:
trait JSONR[A] {
def read(json: JValue): Result[A]
}
trait JSONW[A] {
def write(value: A): JValue
}
source
For syntactic simplicity one can define extensions methods for JValue:
implicit class JValueOps(value: JValue) {
def validate[A: JSONR]: ValidationNel[Error, A] = implicitly[JSONR[A]].read(value)
def read[A: JSONR]: Error \/ A = implicitly[JSONR[A]].read(value).disjunction.leftMap(_.head)
}
And then do a traversal and also parse the resulting JValue of the traversal like this:
val str =
"""
|{
| "a": "aaaa",
| "x": 0
|}""".stripMargin
val json = parseJson(str)
(json \ "a").read[Option[String]]
// \/-(Some(aaaa))
(json \ "b").read[Option[String]]
// \/-(None)
(json \ "a").validate[Option[String]]
// Success(Some(aaaa))
(json \ "b").validate[Option[String]]
// Success(None)
Defining your own JSONR[A]/JSONW[A] instances (and putting them in implicit scope) is possible like this:
case class MyA(a: Option[String], x: Int)
implicit val myARead: JSONR[MyA] = JSON.readE[MyA] { json =>
for {
a <- (json \ "a").read[Option[String]]
x <- (json \ "x").read[Int]
} yield MyA(a, x)
}
implicit val myAWrite: JSONW[MyA] = JSON.write[MyA] { myA =>
("a" -> myA.a) ~
("x" -> myA.x)
}
json.read[MyA]
// \/-(MyA(Some(aaaa),0))
json.validate[MyA]
// Success(MyA(Some(aaaa),0))
MyA(Some("aaaa"), 0).toJson
// JObject(List((a,JString(aaaa)), (x,JInt(0))))
Note that the read[A] and write[a] methods are glue code which is not in json4s-scalaz available yet, you can find the source here. There are also more examples.
Then json.read[A] returns a Error \/ A and json.validate[A] yields a Validation types from scalaz. There are similar types in cats.
myARead is an example of monadic parsing style (compose via flatMap). An alternative uses applicative parsing. This has the benefit that all validation errors are accumulated:
val myARead2: JSONR[MyA] = JSON.read[MyA] { json =>
(
(json \ "a").validate[Option[String]] |#|
(json \ "x").validate[Int]
).tupled.map(MyA.tupled)
}
val myARead3: JSONR[MyA] = JSON.read[MyA] {
for {
a <- field[Option[String]]("a") _
x <- field[Int]("x") _
} yield (a |#| x).tupled.map(MyA.tupled)
}
There is also https://github.com/json4s/json4s/blob/3.4/core/src/main/scala/org/json4s/JsonFormat.scala

Related

validate json with type value

I wand to validate json that has a type value
I have a case class SearchRequestMessage with a companion object with json writer
Value T can be Long or UUID
case class SearchRequestMessage[T](jobId: UUID, offerId: UUID, search: T, searchByType: SearchByType)
object SearchRequestMessage{
implicit def searchResultsWrites[T: Writes]: Writes[SearchRequestMessage[T]] = new Writes[SearchRequestMessage[T]] {
def writes(searchSesult: SearchRequestMessage[T]) =
JsObject(
Seq(
"jobId" -> JsString(searchSesult.jobId.toString),
"offerId" -> JsString(searchSesult.offerId.toString),
"search" -> JsString(searchSesult.search.toString),
"searchByType" -> JsString(searchSesult.searchByType.toString)
))
}
}
Also I have SearchByTypes enum
object SearchByTypes extends Enumeration {
type SearchByType = Value
val Emails: SearchByType = Value(0, "EMAILS")
val Offer: SearchByType = Value(1, "OFFER")
implicit val SearchByTypeFormat: Format[SearchByTypes.Value] = JsonFormatter.enumFormat(SearchByTypes)
}
enum formatter
import play.api.libs.json._
object JsonFormatter {
def enumFormat[T <: Enumeration](enum: T): Format[T#Value] = new EnumFormatter[T](enum)
def enumWithIdsFormat[T <: Enumeration](enum: T): Format[T#Value] = new EnumWithIdsFormatter[T](enum)
class EnumFormatter[T <: Enumeration](enum: T) extends Format[T#Value] {
override def writes(o: T#Value): JsValue = o match {
case null => JsNull
case _ => JsString(o.toString)
}
override def reads(json: JsValue): JsResult[T#Value] = json match {
case JsString(value) => {
try {
JsSuccess(enum.withName(value))
} catch {
case _: NoSuchElementException =>
JsError(
s"Enumeration expected of type: '${enum.getClass}', but it does not appear to contain the value: '$value'")
}
}
case _ => JsError(s"Invalid JSON: $json. Error in '${enum.getClass}' field. Possible values: ${enum.values}")
}
}
class EnumWithIdsFormatter[T <: Enumeration](enum: T) extends Format[T#Value] {
private val nameField = "name"
private val idField = "id"
override def writes(o: T#Value): JsValue = JsObject(Map(idField -> JsNumber(o.id), nameField -> JsString(o.toString)))
override def reads(json: JsValue): JsResult[T#Value] = json match {
case JsObject(values) =>
values.get(idField) match {
case Some(JsNumber(value)) if value <= enum.maxId && value >= 0 ⇒
try {
JsSuccess(enum(value.toInt))
} catch {
case _: NoSuchElementException =>
JsError(
s"Enumeration expected of type: '${enum.getClass}', but it does not appear to contain the value: '$value'")
}
case Some(JsNumber(x)) ⇒
JsError(s"Invalid JSON:$json. Field '$idField': '$x' is out of range. Possible values:${enum.values.map(_.id)}")
case Some(_) ⇒
JsError(s"Invalid JSON:$json. Field '$idField' isn't number. Possible values:${enum.values.map(_.id)}")
case None ⇒
JsError(s"Invalid JSON:$json. Missing field '$idField'")
}
case _ =>
JsError(s"Invalid JSON: $json")
}
}
}
Now I want to validate json
import play.api.libs.json.Json
import play.api.libs.functional.syntax._
val obj = SearchRequestMessage(UUID.randomUUID(), UUID.randomUUID(), "email", SearchByTypes.Emails)
val json = Json.toJson(obj)
val result = ((json \ "jobId").validate[UUID] and
(json \ "offerId").validate[UUID] and
(json \ "search").validate[String] and
(json \ "searchByType").validate[SearchByType])(SearchRequestMessage[String].apply(_,_,_,_))
and an error appear:
Error:(41, 75) missing argument list for method apply in object SearchRequestMessage
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing `apply _` or `apply(_,_,_,_)` instead of `apply`.
(json \ "searchByType").validate[SearchByType])(SearchRequestMessage[String].apply(_,_,_,_))
How I can fix this error?
try
(json \ "searchByType").validate[SearchByType])(SearchRequestMessage[String].apply)
or (which is a bit verbose)
(json \ "searchByType").validate[SearchByType])((a,b,c,d) => SearchRequestMessage[String].apply(a,b,c,d))
This works
((json \ "jobId").validate[UUID] and
(json \ "offerId").validate[UUID] and
(json \ "search").validate[String] and
(json \ "searchByType").validate[SearchByType])((a: UUID, b: UUID, c: String, d: SearchByType) => SearchRequestMessage.apply[String](a,b,c,d))

Scala Play - Iterate over JsArray/JsObject to retrive key and count number of occurance of a key whose value is null

I am using play JSON library to validate a JSON schema.
I have the following JSON data. And I need to validate a schema to this JSON.
// Scala Code
package com.base
import scala.io.Source
import play.api.libs.json._
import play.api.libs.json.JsNull
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
import com.utils.ReadJsonFile._
case class JsValueToModel(web_pages: Array[String], name: String, alpha_two_code: String , state_province: Option[String], domains: Array[String], country: String)
object ValidateJSON extends App {
// Parsing Json file to JsArray
val json = Json.parse(readJson1)
// Using Recursive path by each key to traverse json
val web_pages = json \\ "web_pages"
val name = json \\ "name"
val alpha_two_code = json \\ "alpha_two_code"
val state_province = json \\ "state-province"
val domains = json \\ "domains"
val country = json \\ "country"
// Converting JsValue to Model to read the json key value
implicit val JsValueToModelReads: Reads[JsValueToModel] = (
(JsPath \ "web_pages").read[Array[String]] and
(JsPath \ "name").read[String] and
(JsPath \ "alpha_two_code").read[String] and
(JsPath \ "state-province").readNullable[String] and
(JsPath \ "domains").read[Array[String]] and
(JsPath \ "country").read[String]
)(JsValueToModel.apply _)
// Validation Json Object with JsonSchemaModel
val validateJson = json.validate[List[JsValueToModel]] match {
case s: JsSuccess[List[JsValueToModel]] => {
// val v: List[JsValueToModel] = s.get
println("Validation Success")
}
case e: JsError => {
println("Validation Errors: " + JsError.toJson(e).toString)
}
}
// Length of json
println(json.as[JsArray].value.size)
I am getting following error -
json.as[JsObject].map((a,b) => (a == null))
value map is not a member of play.api.libs.json.JsObject
[error] json.as[JsObject].map((a,b) => (a == null))
json.map((a,b) => (a == null))
value map is not a member of play.api.libs.json.JsValue
[error] json.map((a,b) => (a == null))
I am not sure what to change. I tried to do some pattern matching but not able to do - something like -
def findAndCountNull(a: JsValue):(String,Option[Any]) = a match {
case (x,y) => (x == null) // ???
case _ => a
}
Thanks to getting some help regarding this.
I saw some other libraries like json4s, JsZipper as well as from lift but want to use play json library to understand.
Use JSON automated mapping:
import play.api.libs.json.{JsError, JsSuccess, Json}
import scala.io.{Codec, Source}
object Data {
implicit val jsonFormat = Json.format[Data]
}
case class Data(web_pages: Seq[String], name: String, alpha_two_code: String, `state-province`: Option[String], domains: Seq[String], country: String)
val str = Source.fromURL("https://raw.githubusercontent.com/Hipo/university-domains-list/master/world_universities_and_domains.json")(Codec.UTF8).mkString
Json.parse(str).validate[Seq[Data]] match {
case JsSuccess(x, _) =>
Right(x)
case JsError(errors) =>
Left(errors)
}
Result:
res0: scala.util.Either[Seq[(play.api.libs.json.JsPath, Seq[play.api.libs.json.JsonValidationError])],Seq[Data]] =
Right(List(Data(List(https://www.cstj.qc.ca, https://ccmt.cstj.qc.ca, https://ccml.cstj.qc.ca),Cégep de Saint-Jérôme,CA,None,List(cstj.qc.ca),Canada), Data(List(http://www.lindenwood.edu/),Lindenwood University,US,None,List(lindenwood.edu),United States), Data(List(http://www.davietjal.org/),DAV Institute of Engineering & Technology,IN,Some(Punjab),List(davietjal.org),India), Data(List(http://www.lpu.in/),Lovely Professional University,IN,Some(Punjab),List(lpu.in),India), Data(List(https://sullivan.edu/),Sullivan University,US,None,List(sullivan.edu),United States), Data(List(https://www.fscj.edu/),Florida State College at Jacksonville,US,None,List...
For just counting null - use pattern matching (simplified - you should check JSArray too):
val parsed = Json.parse(str)
val seq = parsed.as[JsArray].value.map(_ \\ "state-province").map(x => x.head)
val nulls = seq.count {
case JsNull => true
case _ => false
}
println(s"$nulls")

Scala Graph JSON with Play Framework

I am trying to pass in a POST request to a REST API developped with Play! (2.5) an object that I would like to use a Scala Graph (from the graph-core dependency).
It looks like the graph already has JSON serialization/deserialization methods based on lift-json, but I am not sure how to "plug" that into Play Json library. Until now I was using implicit converters (with Reads/Writes methods) but I would like to avoid having to write my own methods for the graph part since it is already part of the library itself.
For instance, let's say I have this code:
import java.util.UUID
import scalax.collection.Graph
case class Task(
id: UUID,
status: String)
case class Stuff(
id: UUID = UUID.randomUUID(),
name: String,
tasks: Option[Graph[Task, DiEdge]])
implicit val stuffWrites: Writes[Stuff] = (
(JsPath \ "id").write[UUID] and
(JsPath \ "name").write[String]
)(unlift(Stuff.unapply))
implicit val stuffReads: Reads[Stuff] = (
(JsPath \ "id").read[UUID] and
(JsPath \ "name").read[String]
)(Stuff.apply _)
implicit val taskWrite: Writes[Task] = (
(JsPath \ "id").write[UUID] and
(JsPath \ "status").write[String]
)(unlift(Task.unapply))
implicit val taskReads: Reads[Task] = (
(JsPath \ "id").read[UUID] and
(JsPath \ "status").read[String]
)(Task.apply _)
I miss the part to serialize the graph and the parenting. Should I rewrite everything from scratch, or can I rely on methods toJson/fromJson from scalax.collection.io.json ?
Since I struggled a bit to get this working, I thought I would share the code:
class UUIDSerializer extends Serializer[UUID] {
private val UUIDClass = classOf[UUID]
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), UUID] = {
case (TypeInfo(UUIDClass, _), json) => json match {
case JString(id) => UUID.fromString(id)
case x => throw new MappingException("Can't convert " + x + " to UUID")
}
}
def serialize(implicit format: Formats): PartialFunction[Any, JValue] = {
case x: UUID => JString(x.toString)
}
}
val extraSerializers = new UUIDSerializer :: Nil
implicit val formats = Serialization.formats(NoTypeHints) ++ extraSerializers
val taskDescriptor = new NodeDescriptor[Task](typeId = "Tasks", customSerializers=extraSerializers) {
def id(node: Any) = node match {
case Task(id, _) => id.toString
}
}
val quickJson = new Descriptor[Task](
defaultNodeDescriptor = taskDescriptor,
defaultEdgeDescriptor = Di.descriptor[Task]()
)
implicit val tasksWrites = new Writes[Graph[Task, DiEdge]] {
def writes(graph: Graph[Task, DiEdge]): JsValue = {
val json = graph.toJson(quickJson)
Json.parse(json.toString)
}
}
implicit val tasksReads = new Reads[Graph[Task, DiEdge]] {
def reads(json: JsValue): JsResult[Graph[Task, DiEdge]] = {
try {
val graph = Graph.fromJson[Task, DiEdge](json.toString, quickJson)
JsSuccess(graph)
}
catch {
case e: Exception =>
JsError(e.toString)
}
}
}
implicit def stuffModelFormat = Jsonx.formatCaseClass[Stuff]
You can try writing companion objects for yours case classes where you specify the formatting.
Example:
object Task {
implicit val taskModelFormat = Json.format[Task]
}
object Stuff {
implicit val staffModelFormat = Json.format[Stuff]
}
instead of the above implicits. With this solution compiler will resolve the known formatters for you and you could be only required to specify the missing/unknown types instead of the whole structure.

Play/Scala how to prevent Json serialization of empty arrays?

I want to recursively write a class to Json, so I'm using the following implicit writes:
implicit val writesObject : Writes[Object] = (
(__ \ "id").writeNullable[String] ~
(__ \ "list").lazyWriteNullable(Writes.traversableWrites[Object](writesObject))
)(unlift(Object.unapply)
where Object is a class like this:
case class Object(id: Option[String], list: Option[Seq[Object]])
It works, however I would like to prevent it from printing anything if "list" is empty. For example:
I want:
{ id: "someID",
list: [
{
id: "someOtherId"
}
]
}
I currently get(but don't want):
{ id: "someID",
list: [
{
id: "someOtherId"
list: []
}
]
}
How can I achieve this? I'm new to Play/Scala and not sure exactly what should I be looking at so any pointers would be helpful. I'm using Scala 2.2.1.
PS: I've checked Scala Json Combinators but didn't see any reference on how to get this done.
Update:
So my issue is not that list is null, but that list is empty. That's why lazyWriteNullable wasn't working.
Testing johanandren answer I came up with the following extension to JsPath that returns Option[T] and supports the lazy format for recursive writes:
def lazyWriteNullableIterable[T <: Iterable[_]](w: => Writes[T]): OWrites[Option[T]] = OWrites((t: Option[T]) => {
if(t != null) {
t.getOrElse(Seq.empty).size match {
case 0 => Json.obj()
case _ => Writes.nullable[T](path)(w).writes(t)
}
}
else {
Json.obj()
}
})
Thanks
You can create a custom OFormat that will do this. By implicitly decorating JsPath with it you can include it in your json combinator definitions:
implicit class PathAdditions(path: JsPath) {
def readNullableIterable[A <: Iterable[_]](implicit reads: Reads[A]): Reads[A] =
Reads((json: JsValue) => path.applyTillLast(json).fold(
error => error,
result => result.fold(
invalid = (_) => reads.reads(JsArray()),
valid = {
case JsNull => reads.reads(JsArray())
case js => reads.reads(js).repath(path)
})
))
def writeNullableIterable[A <: Iterable[_]](implicit writes: Writes[A]): OWrites[A] =
OWrites[A]{ (a: A) =>
if (a.isEmpty) Json.obj()
else JsPath.createObj(path -> writes.writes(a))
}
/** When writing it ignores the property when the collection is empty,
* when reading undefined and empty jsarray becomes an empty collection */
def formatNullableIterable[A <: Iterable[_]](implicit format: Format[A]): OFormat[A] =
OFormat[A](r = readNullableIterable(format), w = writeNullableIterable(format))
}
This would allow you to create formats/reads/writes using the json combinator syntax like this:
case class Something(as: List[String], v: String)
import somewhere.PathAdditions
val reads: Reads[Something] = (
(__ \ "possiblyMissing").readNullableIterable[List[String]] and
(__ \ "somethingElse").read[String]
)(Something)
val writes: Writes[Something] = (
(__ \ "possiblyMissing").writeNullableIterable[List[String]] and
(__ \ "somethingElse").write[String]
)(unlift(Something.unapply))
val format: Format[Something] = (
(__ \ "possiblyMissing").formatNullableIterable[List[String]] and
(__ \ "somethingElse").format[String]
)(Something, unlift(Something.unapply))

How to convert an anonymous class to json?

I have the following code:
def test = Action {
val Anon = new {
val foo = "foo"
val bar = "bar"
}
Ok(Json.toJson(Anon))
}
And I get this compilation error:
No Json deserializer found for type Object{val foo: String; val bar:
String}. Try to implement an implicit Writes or Format for this type.
What is the quick fix for this problem? I already found another question here that relates to this error, but perhaps it was more specific/complex.
As far as I can tell the only way is to introduce a structural type:
type AnonType = {
def foo:String
def bar:String
}
Then you can either do
implicit val writeAnon1 =
((__ \ "foo").write[String] and
(__ \ "bar").write[String])
{anon:AnonType => (anon.foo, anon.bar)}
or
implicit val writeAnon2 = new Writes[AnonType] {
def writes(o:AnonType) =
Json toJson Map(
"foo" -> o.foo,
"bar" -> o.bar)
}