I'm trying to write a reads for the following class
case class User(id: String, imageId: Option[String])
The JSON I have looks like this:
{
"id": 1
"image": { "id" : 2 }
}
However the 'image' field may not exist, or it may be null.
My reads looks like this:
implicit val userReader: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "image" \ "id").readNullable[String]
) (User.apply _)
But this does not seem to work, I still get an error when it finds a sample with no image. 'ValidationError [...] error.path.missing [...] /image/id'
How can this be solved? I'd prefer not to create an Image class, which I'm not going to use anywhere.
I think the issue comes from the fact that you are trying to read a String whereas id is an Int. You should mind your types in your json / reader.
Two options
if id is really an Int, read it as an Int
if id is a String, change your JSON
If you still want to have a String in your case class, you can do :
(JsPath \ "image" \ "id").readNullable[Int].map(_.toString)
You forgot a comma between the two elements ("id" and "image"). That's why the /image/id path seems to be missing for the parser.
There are all sorts of tools for validating a JSON structure, for instance https://jsonlint.com/ .
So your JSON structure should look like this:
{
"id": 1,
"image": {
"id": 2
}
}
i may be late but here is a workaround:
implicit val userReader: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "image" \ "id").readNullable[String].orElse((__ \ "dummy").readNullable[String])
) (User.apply _)
Not perfect but it work well
better solution would be to directly cast the Js Reads(JsSuccess(None:Option[Int])) inside the orElse but i haven't been able to do it...
I want to validate a password field as follows:
First check that the password has only lower case
Then check that its length is min 8 characters.
I have written the two validation codes as follows:
/*check lowercase*/
def checkPasswordCase[T]: Reads[String] = {
Reads.StringReads.filter(ValidationError("Password not in lowercase"))(str => {
(str.matches("""[a-z]+"""))
})
}
/*check length*/
def checkPasswordLength[T](min:Int): Reads[String] = {
Reads.StringReads.filter(ValidationError("Invalid password length"))(str => {
(str.length >= min)
})
}
I called the code as follows but got compilation error Cannot prove that String <:< String => C.
(JsPath \ "user" \ "password").read[String](checkPasswordCase)(checkPasswordLength(8)) and ...
I tried using Reads.minLength(8) but got a different error.
Couldn't I use two validation codes back to back?
You can use either the keepAnd or andKeep validators to do this. These combinators will run both Reads but only keep the successful result of either the left or right operands:
val r = (JsPath \ "user" \ "password").read[String](
checkPasswordCase keepAnd checkPasswordLength(8))
val js = Json.obj("user" -> Json.obj("password" -> "FOO"))
println(js.validate[String](r))
// JsError(List((/user/password,List(
// ValidationError(List(Password not in lowercase),WrappedArray()),
// ValidationError(List(Invalid password length),WrappedArray())))))
I have the following reads function for parsing JSON files.
case class tables(col1 : Option[List[another case class]], col2 : Option[List[another case class]], col3 : Option[List[another case class]], col4 : Option[List[another case class]])
implicit val tablesRead: Reads[tables] = (
(JsPath \ "col1").read(Reads.optionWithNull[List[data1]]).filterNot(_.get.isEmpty) and
(JsPath \ "col2").read(Reads.optionWithNull[List[data2]]).filterNot(_.get.isEmpty) and
(JsPath \ "col3").read(Reads.optionWithNull[List[data3]]).filterNot(_.get.isEmpty) and
(JsPath \ "col4").read(Reads.optionWithNull[List[data4]]).filterNot(_.get.isEmpty)
) (tables.apply _)
I want to then insert the JSON into a database after having validated it. I have therefore declared the following function.
def createFromJson = Action.async(parse.json) { request =>
request.body.validate[jsonWrapper] match {
case JsSuccess(data, _) =>
for {
dbFuture <- dataFuture(data.userID)
lastError <- dbFuture.insert(data.tables)
} yield {
Ok("Success\n")
}
case JsError(errors) => Future.successful(BadRequest("Failed :" + Error.show(errors)))
}
}
This works and correctly rejects JSONs looking like this:
{"tables":{"col1":[],"col2":[],"col3":[],"col4":[]}, "userID":"irrelavent"}
and accepts JSONs with actual data in, like so:
{"tables":{"col1":[{data1}],"col2":[{data2}],"col3":[{data3}],"col4":[{data4}]}, "userID":"irrelavent"}
But want i need is something that does this but also accepts a JSON with missing fields
{"tables":{"col1":[{data1}],"col2":[],"col3":[{data3}],"col4":[{data4}]}, "userID":"irrelavent"}
And preferable ignore them (i.e. return something like :
{"tables":{"col1":[{data1}],"col3":[{data3}],"col4":[{data4}]}, "userID":"irrelavent"})
Is this possible to do?
Many thanks,
Peter M.
You can automatically generate a Reads[tables] using Json.reads macro with the behavior you want:
implicit val tablesRead: Reads[tables] = Json.reads[tables]
If the fields is missing from the JSON the column will be None.
On a minor note, the common form in scala is to start a class name with a capital letter so you should rename tables to Tables.
I'm reading a nested JSON document using several Reads[T] implementations, however, I'm stuck with the following sub-object:
{
...,
"attributes": {
"keyA": [1.68, 5.47, 3.57],
"KeyB": [true],
"keyC": ["Lorem", "Ipsum"]
},
...
}
The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary. The values of the keys are always JsArray instances, but of different size and type (however, all elements of a particular array must have the same JsValue type).
The Scala representation of one single attribute:
case class Attribute[A](name: String, values: Seq[A])
// 'A' can only be String, Boolean or Double
The goal is to create a Reads[Seq[Attribute]] that can be used for the "attributes"-field when transforming the whole document (remember, "attributes" is just a sub-document).
Then there is a simple map that contains allowed combinations of keys and array types that should be used to validate attributes. Edit: This map is specific for each request (or rather specific for every type of json document). But you can assume that it is always available in the scope.
val required = Map(
"KeyA" -> "Double",
"KeyB" -> "String",
"KeyD" -> "String",
)
So in the case of the JSON shown above, the Reads should create two errors:
"keyB" does exist, but has the wrong type (expected String, was boolean).
"keyD" is missing (whereas keyC is not needed and can be ignored).
I'm having trouble creating the necessary Reads. The first thing I tried as a first step, from the perspective of the outer Reads:
...
(__ \ "attributes").reads[Map[String, JsArray]]...
...
I thought this is a nice first step because if the JSON structure is not an object containing Strings and JsArrays as key-value pairs, then the Reads fails with proper error messages. It works, but: I don't know how to go on from there. Of course I just could create a method that transforms the Map into a Seq[Attribute], but this method somehow should return a JsResult, since there are further validations to do.
The second thing I tried:
val attributeSeqReads = new Reads[Seq[Attribute]] {
def reads(json: JsValue) = json match {
case JsObject(fields) => processAttributes(fields)
case _ => JsError("attributes not an object")
}
def processAttributes(fields: Map[String, JsValue]): JsResult[Seq[Attribute]] = {
// ...
}
}
The idea was to validate each element of the map manually within processAttributes. But I think this is too complicated. Any help is appreciated.
edit for clarification:
At the beginning of the post I said that the keys (keyA, keyB...) are unknown at compile time. Later on I said that those keys are part of the map required which is used for validation. This sounds like a contradiction, but the thing is: required is specific for each document/request and is also not known at compile time. But you don't need to worry about that, just assume that for every request the correct required is already available in the scope.
You are too confused by the task
The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary
So the number of keys and their types are known in advance and the final?
So in the case of the JSON shown above, the Reads should create two
errors:
"keyB" does exist, but has the wrong type (expected String, was
boolean).
"keyD" is missing (whereas keyC is not needed and can be ignored).
Your main task is just to check the availability and compliance?
You may implement Reads[Attribute] for every your key with Reads.list(Reads.of[A]) (this Reads will check type and required) and skip omitted (if not required) with Reads.pure(Attribute[A]). Then tuple convert to list (_.productIterator.toList) and you will get Seq[Attribute]
val r = (
(__ \ "attributes" \ "keyA").read[Attribute[Double]](list(of[Double]).map(Attribute("keyA", _))) and
(__ \ "attributes" \ "keyB").read[Attribute[Boolean]](list(of[Boolean]).map(Attribute("keyB", _))) and
((__ \ "attributes" \ "keyC").read[Attribute[String]](list(of[String]).map(Attribute("keyC", _))) or Reads.pure(Attribute[String]("keyC", List()))) and
(__ \ "attributes" \ "keyD").read[Attribute[String]](list(of[String]).map(Attribute("keyD", _)))
).tupled.map(_.productIterator.toList)
scala>json1: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}
scala>res37: play.api.libs.json.JsResult[List[Any]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57)), Attribute(KeyB,List(true)), Attribute(keyC,List()), Attribute(KeyD,List(Lorem, Ipsum))),)
scala>json2: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyC":["Lorem","Ipsum"]}}
scala>res38: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray())))))
scala>json3: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":["Lorem"],"keyC":["Lorem","Ipsum"]}}
scala>res42: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray()))), (/attributes/keyB(0),List(ValidationError(List(error.expected.jsboolean),WrappedArray())))))
If you will have more than 22 attributes, you will have another problem: Tuple with more than 22 properties.
for dynamic properties in runtime
inspired by 'Reads.traversableReads[F[_], A]'
def attributesReads(required: Map[String, String]) = Reads {json =>
type Errors = Seq[(JsPath, Seq[ValidationError])]
def locate(e: Errors, idx: Int) = e.map { case (p, valerr) => (JsPath(idx)) ++ p -> valerr }
required.map{
case (key, "Double") => (__ \ key).read[Attribute[Double]](list(of[Double]).map(Attribute(key, _))).reads(json)
case (key, "String") => (__ \ key).read[Attribute[String]](list(of[String]).map(Attribute(key, _))).reads(json)
case (key, "Boolean") => (__ \ key).read[Attribute[Boolean]](list(of[Boolean]).map(Attribute(key, _))).reads(json)
case _ => JsError("")
}.iterator.zipWithIndex.foldLeft(Right(Vector.empty): Either[Errors, Vector[Attribute[_ >: Double with String with Boolean]]]) {
case (Right(vs), (JsSuccess(v, _), _)) => Right(vs :+ v)
case (Right(_), (JsError(e), idx)) => Left(locate(e, idx))
case (Left(e), (_: JsSuccess[_], _)) => Left(e)
case (Left(e1), (JsError(e2), idx)) => Left(e1 ++ locate(e2, idx))
}
.fold(JsError.apply, { res =>
JsSuccess(res.toList)
})
}
(__ \ "attributes").read(attributesReads(Map("keyA" -> "Double"))).reads(json)
scala> json: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}
scala> res0: play.api.libs.json.JsResult[List[Attribute[_ >: Double with String with Boolean]]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57))),/attributes)
I'm trying to parse the data from GovTrack, for example, https://www.govtrack.us/api/v2/bill/74369 . But titles is in a peculiar format:
"titles": [
[
"short",
"introduced",
"Public Online Information Act of 2011"
],
[
"official",
"introduced",
"To establish an advisory committee to issue nonbinding governmentwide guidelines..."
]
]
titles is an array of each title type, with fields in a particular order. I want to read this into a more standard JSON format:
{
'short_title': "Public Online Information Act of 2011",
'official_title': "To establish an advisory committee to issue nonbinding governmentwide guidelines..."
}
The short title or official title may or may not be there, and there could actually be several short titles.
How do I make a Reads for this? Right now I've got:
implicit val billReads: Reads[Bill] = (
(JsPath \ "id").read[Int] and
(JsPath \ "display_number").read[String] and
(JsPath \ "current_status").read[String] and
(JsPath \ "titles")(0)(2).read[String]
)(Bill.apply _)
How do I specify "The member of the array that has a first element equal to 'official'"?
As far as I know, there is no out of the box way to do it, but I would do it with additional custom reader, like this:
val officialReads = new Reads[String] {
override def reads(json: JsValue): JsResult[String] = (json \ "titles") match {
case JsArray(titles) => titles.collectFirst({
case JsArray(values) if (values.headOption.map(v => "official".equals(v.as[String])).getOrElse(false)) =>
JsSuccess(values.tail.tail.head.as[String])
}).getOrElse(JsError("No official title"))
case _ => JsError("Can't read official title")
}
}
And your Bill reader would look like this:
val implicitReads: Reads[Bill] = (
(__ \ "id").read[Int] and
(__ \ "display_number").read[String] and
(__ \ "current_status").read[String] and
officialReads
)(Bill.apply _)
I've tested, this works :)