I am receiving a JSON from an external service and my goal is to parse it exactly as it is.
The main issue is this: a value can be nullable or it can be absent BUT null has different meaning of absent. So I want to catch this somehow.
For example this JSON:
{
"a": null,
"b": 1
}
is different from this one:
{
"b": 1
}
Can you help me please?
UPDATE:
Sorry for the delay in the update. Anyway: you are right, I have a implicit custom reads in the middle and currently I use "a".readNullable[Double] and "a".write[Option[Double]] and case class is something like:
case class Example(a: Option[Double])
Just laying out what #mfirry was talking about with a detailed example (play-json 2.6):
scala> import play.api.libs.json._
import play.api.libs.json._
scala> val json1 = Json.parse("""{"a": null, "b": 1}""")
json1: play.api.libs.json.JsValue = {"a":null,"b":1}
scala> val json2 = Json.parse("""{"b": 1}""")
json2: play.api.libs.json.JsValue = {"b":1}
scala> (json1 \ "a").isDefined
res8: Boolean = true
scala> (json1 \ "a") == JsDefined(JsNull)
res3: Boolean = true
scala> (json2 \ "a").isDefined
res7: Boolean = false
scala> (json2 \ "a")
res5: play.api.libs.json.JsLookupResult = JsUndefined('a' is undefined on object: {"b":1})
Related
This question has come up a few times recently, so I'm FAQ-ing it here. Suppose I've got some case classes like this:
import io.circe._, io.circe.generic.semiauto._
object model {
case class A(a: String)
case class B(a: String, i: Int)
case class C(i: Int, b: Boolean)
implicit val encodeA: Encoder[A] = deriveEncoder
implicit val encodeB: Encoder[B] = deriveEncoder
implicit val encodeC: Encoder[C] = deriveEncoder
implicit val decodeA: Decoder[A] = deriveDecoder
implicit val decodeB: Decoder[B] = deriveDecoder
implicit val decodeC: Decoder[C] = deriveDecoder
}
And I want to encode a value that could be any one of these as JSON using circe and Shapeless coproducts.
import io.circe.shapes._, io.circe.syntax._
import shapeless._
import model._
type ABC = A :+: B :+: C :+: CNil
val c: ABC = Coproduct[ABC](C(123, false))
This looks fine at first:
scala> c.asJson
res0: io.circe.Json =
{
"i" : 123,
"b" : false
}
But the problem is that I can never decode a coproduct containing a B element, since any valid JSON document that could be decoded as B can also be decoded as A, and the coproduct decoders provided by circe-shapes try the elements in the order they appear in the coproduct.
scala> val b: ABC = Coproduct[ABC](B("xyz", 123))
b: ABC = Inr(Inl(B(xyz,123)))
scala> val json = b.asJson
json: io.circe.Json =
{
"a" : "xyz",
"i" : 123
}
scala> io.circe.jawn.decode[ABC](json.noSpaces)
res1: Either[io.circe.Error,ABC] = Right(Inl(A(xyz)))
How can I disambiguate the elements of my coproduct in my encoding?
The circe-shapes module encodes ordinary hlists and coproducts without labels, as seen above: a coproduct as the bare JSON representation of the element, and an hlist ends up as just a JSON array (the same as the default tuple encoding):
scala> ("xyz" :: List(1, 2, 3) :: false :: HNil).asJson.noSpaces
res2: String = ["xyz",[1,2,3],false]
In the case of hlists there's no danger of ambiguity because of overlapping names, but for coproducts there is. In both cases, though, you can add labels to the JSON representation using Shapeless's labeling mechanism, which tags values with a type-level symbol.
An hlist with labels is called a "record", and a coproduct with labels is a "union". Shapeless provides special syntax and operations for both of these, and circe-shapes treats both differently from unlabeled hlists or coproducts. For example (assuming the definitions and imports above):
scala> import shapeless.union._, shapeless.syntax.singleton._
import shapeless.union._
import shapeless.syntax.singleton._
scala> type ABCL = Union.`'A -> A, 'B -> B, 'C -> C`.T
defined type alias ABCL
scala> val bL: ABCL = Coproduct[ABCL]('B ->> B("xyz", 123))
bL: ABCL = Inr(Inl(B(xyz,123)))
scala> val jsonL = bL.asJson
jsonL: io.circe.Json =
{
"B" : {
"a" : "xyz",
"i" : 123
}
}
scala> io.circe.jawn.decode[ABCL](jsonL.noSpaces)
res3: Either[io.circe.Error,ABCL] = Right(Inr(Inl(B(xyz,123))))
The encoding for records similarly includes the member names:
scala> ('a ->> "xyz" :: 'b ->> List(1) :: 'c ->> false :: HNil).asJson.noSpaces
res4: String = {"c":false,"b":[1],"a":"xyz"}
In general if you want your hlist and coproduct encodings to include labels (and look like the encodings you'd get from circe-generic for case classes and sealed trait hierarchies), you can use records or coproducts to tell circe-shapes to do this with a minimal amount of extra syntactic and runtime overhead.
I am trying to create a validator for reads that will only allow a valid email or an empty string.
What I tried until now seem to work only partially.
Here is my reads defenition:
case class EmailFieldValueForm(value: String) extends FieldValueForm
val emailFieldValueFormReads: Reads[EmailFieldValueForm] =
(__ \ "value").read[String](email or maxLength(0)).map(EmailFieldValueForm.apply _)
When I test it, I am getting the following error:
diverging implicit expansion for type play.api.libs.json.Reads[V]
[error] starting with value uuidReads in trait DefaultReads [error]
(__ \ "value").read[String](email or
maxLength(0)).map(EmailFieldValueForm.apply _)
Also, tried doing it with regex:
val emailFieldValueFormReads: Reads[EmailFieldValueForm] =
(__ \ "value").read[String](email or pattern("""^$"""r)).map(EmailFieldValueForm.apply)
And in this case, any json I provide is passing.For example:
val invalidJson = Json.parse(
"""
|{
| "value": "boo"
|}
""".stripMargin
Simply gives me an empty value, but does not fail validation.
What am I doing wrong?
Thanks,
SOLUTION
Seems like I was not doing the testing correctly. The following worked:
val invalidJson = Json.parse(
"""
|{
| "value": "boo"
|}
""".stripMargin
)
(EmailFieldValueForm.emailFieldValueFormReads reads invalidJson match {
case JsSuccess(value, _) => value
case JsError(e) => throw JsResultException(e)
}) must throwA[JsResultException]
The problem lies in implicit reads: Reads[M] for def maxLength[M](m: Int)(implicit reads: Reads[M], p: M => scala.collection.TraversableLike[_, M])
You may specify type M explicitly and all will be fine
import play.api.libs.functional.syntax._
import play.api.libs.json.Reads._
import play.api.libs.json._
val emailFieldValueFormReads: Reads[EmailFieldValueForm] =
((__ \ "value").read[String](email or maxLength[String](0))).map(EmailFieldValueForm.apply _)
Json.parse("""{"value": "sss"}""").validate[EmailFieldValueForm](emailFieldValueFormReads)
Json.parse("""{"value": ""}""").validate[EmailFieldValueForm](emailFieldValueFormReads)
Json.parse("""{"value": "a#a.com"}""").validate[EmailFieldValueForm](emailFieldValueFormReads)
scala> res0: play.api.libs.json.JsResult[EmailFieldValueForm] = JsError(List((/value,List(ValidationError(List(error.email),WrappedArray()), ValidationError(List(error.maxLength),WrappedArray(0))))))
scala> res1: play.api.libs.json.JsResult[EmailFieldValueForm] = JsSuccess(EmailFieldValueForm(),/value)
scala> res2: play.api.libs.json.JsResult[EmailFieldValueForm] = JsSuccess(EmailFieldValueForm(a#a.com),/value)
For testing with scalatest (PlaySpec) you may use
Json.parse("""{"value": "sss"}""").validate[EmailFieldValueForm](emailFieldValueFormReads) must be an 'error
Json.parse("""{"value": "a#a.com"}""").validate[EmailFieldValueForm](emailFieldValueFormReads) must be an 'success
Json.parse("""{"value": ""}""").validate[EmailFieldValueForm](emailFieldValueFormReads) must be an 'success
Let's say I've got a reads that creates an object from JSON with two optional fields:
implicit val rd: Reads[MyObject] = (
(__ \ "field1").readNullable[String] and
(__ \ "field2").readNullable[String]
)(MyObject.apply _)
I want to check to make sure that the value of field1 is one of the values in the list:
List("foo", "bar")
I can do that after the fact, by creating a new MyObject and mapping the values through a function to transform them, but I feel like there should be a way to do this more elegantly using JSON transformers or something.
Ideally, I want the Reads to read the nullable value of field1 and transform it if it is defined, without the need to post-process it. Is there some way of sneaking a transform in there?
You can use this approach:
case class MyObject(a: Option[String], b: Option[String])
val allowedValues = Seq("foo", "bar")
implicit val reads: Reads[MyObject] = new Reads[MyObject] {
override def reads(json: JsValue): JsResult[MyObject] = {
val a = (json \ "a").asOpt[String].filter(allowedValues.contains)
val b = (json \ "b").asOpt[String]
JsSuccess(MyObject(a, b))
}
}
Usage examples:
scala> Json.parse(""" { "a": "bar", "b": "whatever"} """).validate[MyObject]
res2: play.api.libs.json.JsResult[MyObject] = JsSuccess(MyObject(Some(bar),Some(whatever)),)
scala> Json.parse(""" { "a": "other", "b": "whatever"} """).validate[MyObject]
res3: play.api.libs.json.JsResult[MyObject] = JsSuccess(MyObject(None,Some(whatever)),)
scala> Json.parse(""" {} """).validate[MyObject]
res4: play.api.libs.json.JsResult[MyObject] = JsSuccess(MyObject(None,None),)
Okay, after doing some more research, I came up with the following:
In play.api.libs.json.ConstraintReads there is a function called verifying(cond: A => Boolean) that returns a Reads[A]. This can be passed as a parameter to JsPath.readNullable[A] like so:
implicit val rd: Reads[MyObject] = (
(__ \ "field1").readNullable[String](verifying(allowedValues.contains)) and
(__ \ "field2").readNullable[String]
)(MyObject.apply _)
This will return a JsResponse, either a JsSuccess if "field1" validates, or a JsError if it doesn't validate. It actually fails on an invalid input, rather than just ignoring the input. That's more like the behaviour I wanted.
There are a number of other constraint functions that perform similar tests on the read value, as well.
Is there a way to perform conditional logic while parsing json using Scala/Play?
For example, I would like to do something like the following:
implicit val playlistItemInfo: Reads[PlaylistItemInfo] = (
(if(( (JsPath \ "type1").readNullable[String]) != null){ (JsPath \ "type1" \ "id").read[String]} else {(JsPath \ "type2" \ "id").read[String]}) and
(JsPath \ "name").readNullable[String]
)(PlaylistItemInfo.apply _)
In my hypothetical JSON parsing example, there are two possible ways to parse the JSON. If the item is of "type1", then there will be a value for "type1" in the JSON. If this is not present in the JSON or its value is null/empty, then I would like to read the JSON node "type2" instead.
The above example does not work, but it gives you the idea of what I am trying to do.
Is this possible?
The proper way to do this with JSON combinators is to use orElse. Each piece of the combinator must be a Reads[YourType], so if/else doesn't quite work because your if clause doesn't return a Boolean, it returns Reads[PlaylistItemInfo] checked against null which will always be true. orElse let's us combine one Reads that looks for the type1 field, and a second one that looks for the type2 field as a fallback.
This might not follow your exact structure, but here's the idea:
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class PlaylistItemInfo(id: Option[String], tpe: String)
object PlaylistItemInfo {
implicit val reads: Reads[PlaylistItemInfo] = (
(__ \ "id").readNullable[String] and
(__ \ "type1").read[String].orElse((__ \ "type2").read[String])
)(PlaylistItemInfo.apply _)
}
// Read type 1 over type 2
val js = Json.parse("""{"id": "test", "type1": "111", "type2": "2222"}""")
scala> js.validate[PlaylistItemInfo]
res1: play.api.libs.json.JsResult[PlaylistItemInfo] = JsSuccess(PlaylistItemInfo(Some(test),111),)
// Read type 2 when type 1 is unavailable
val js = Json.parse("""{"id": "test", "type2": "22222"}""")
scala> js.validate[PlaylistItemInfo]
res2: play.api.libs.json.JsResult[PlaylistItemInfo] = JsSuccess(PlaylistItemInfo(Some(test),22222),)
// Error from neither
val js = Json.parse("""{"id": "test", "type100": "fake"}""")
scala> js.validate[PlaylistItemInfo]
res3: play.api.libs.json.JsResult[PlaylistItemInfo] = JsError(List((/type2,List(ValidationError(error.path.missing,WrappedArray())))))
I have the following Read defined:
import org.joda.time.DateTime;
implicit val userInfoRead: Reads[UserInfo] = (
(JsPath \ "userName").readNullable[String] and
] (JsPath \ "startDate").readNullable[DateTime]
(UserInfo.apply _)
With the following JSON object being passed in:
"userInfo" : {
"userName": "joeuser",
"startDate": "2006-02-28"
}
When I validate this data I get the following error:
(/startDate,List(ValidationError(validate.error.expected.jodadate.format,WrappedArray(yyyy-MM-dd))))))
Any suggestions on what I'm missing in the formatting?
As far as I can see, the issue is probably just the format not matching what Joda is expecting. I simplified a bit, and this worked for me:
scala> import org.joda.time.DateTime
import org.joda.time.DateTime
scala> case class UserInfo(userName: String, startDate: DateTime)
defined class UserInfo
scala> implicit val dateReads = Reads.jodaDateReads("yyyy-MM-dd")
dateReads: play.api.libs.json.Reads[org.joda.time.DateTime] = play.api.libs.json.DefaultReads$$anon$10#22db02cb
scala> implicit val userInfoReads = Json.reads[UserInfo]
userInfoReads: play.api.libs.json.Reads[UserInfo] = play.api.libs.json.Reads$$anon$8#52bcbd5d
scala> val json = Json.parse("""{
| "userName": "joeuser",
| "startDate": "2006-02-28"
| }""")
json: play.api.libs.json.JsValue = {"userName":"joeuser","startDate":"2006-02-28"}
scala> json.validate[UserInfo]
res12: play.api.libs.json.JsResult[UserInfo] = JsSuccess(UserInfo(joeuser,2006-02-28T00:00:00.000-05:00),)