parsing a Json Array in play framework JsObject - json

I have the following Json:
{
"web-category" : "macaroons",
"sub-categories" : [
{ "name" : "pink" },
{ "name" : "blue" },
{ "name" : "green" }
]
}
I have got it in Play as a JsObject. So I can now successfully do the following:
//(o is the JsObject)
val webCat:Option[String] = (o \ "web-category").asOpt[String]
println(webCat.toString)
>> Some(macaroons)
So far, so good. But how do I access the array Json objects? I have this...
val subCats:Option[JsArray] = (o \ "sub-categories").asOpt[JsArray]
println(subCats.toString)
>> Some([{"name" : "blue"},{"name" : "green"},{"name" : "pink"}])
but what I need is to take the JsArray and get a List of all the names something like this:
List("blue", "green", "pink")
Don't know how to access the JsArray thusly.
my thanks for your help in this.

I'd argue that it's generally a good idea to move from JSON-land to native-Scala-representation-land as early as possible. If obj is your JsObject, for example, you can write this:
val subCategories = (obj \ "sub-categories").as[List[Map[String, String]]]
val names = subCategories.map(_("name"))
Or even:
case class Category(name: String, subs: List[String])
import play.api.libs.functional.syntax._
implicit val categoryReader = (
(__ \ "web-category").read[String] and
(__ \ "sub-categories").read[List[Map[String, String]]].map(_.map(_("name")))
)(Category)
And then:
obj.as[Category]
This latter approach makes error handling even cleaner (e.g. you can just replace as with asOpt at this top level) and composes nicely with other Reads type class instances—if you have a JsArray of these objects, for example, you can just write array.as[List[Category]] and get what you expect.

What Peter said, or:
(o \ "sub-categories" \\ "name").map(_.as[String]).toList

Something like this:
subCats.map( jsarray => jsarray.value.map(jsvalue => (jsvalue \ "name").as[String]).toList)
This will normally return a Option[List[String]]

Related

Scala reads object with nested optional property

I'm trying to write a reads for the following class
case class User(id: String, imageId: Option[String])
The JSON I have looks like this:
{
"id": 1
"image": { "id" : 2 }
}
However the 'image' field may not exist, or it may be null.
My reads looks like this:
implicit val userReader: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "image" \ "id").readNullable[String]
) (User.apply _)
But this does not seem to work, I still get an error when it finds a sample with no image. 'ValidationError [...] error.path.missing [...] /image/id'
How can this be solved? I'd prefer not to create an Image class, which I'm not going to use anywhere.
I think the issue comes from the fact that you are trying to read a String whereas id is an Int. You should mind your types in your json / reader.
Two options
if id is really an Int, read it as an Int
if id is a String, change your JSON
If you still want to have a String in your case class, you can do :
(JsPath \ "image" \ "id").readNullable[Int].map(_.toString)
You forgot a comma between the two elements ("id" and "image"). That's why the /image/id path seems to be missing for the parser.
There are all sorts of tools for validating a JSON structure, for instance https://jsonlint.com/ .
So your JSON structure should look like this:
{
"id": 1,
"image": {
"id": 2
}
}
i may be late but here is a workaround:
implicit val userReader: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "image" \ "id").readNullable[String].orElse((__ \ "dummy").readNullable[String])
) (User.apply _)
Not perfect but it work well
better solution would be to directly cast the Js Reads(JsSuccess(None:Option[Int])) inside the orElse but i haven't been able to do it...

Play framework 2.5 reads optional filter

I have the following reads function for parsing JSON files.
case class tables(col1 : Option[List[another case class]], col2 : Option[List[another case class]], col3 : Option[List[another case class]], col4 : Option[List[another case class]])
implicit val tablesRead: Reads[tables] = (
(JsPath \ "col1").read(Reads.optionWithNull[List[data1]]).filterNot(_.get.isEmpty) and
(JsPath \ "col2").read(Reads.optionWithNull[List[data2]]).filterNot(_.get.isEmpty) and
(JsPath \ "col3").read(Reads.optionWithNull[List[data3]]).filterNot(_.get.isEmpty) and
(JsPath \ "col4").read(Reads.optionWithNull[List[data4]]).filterNot(_.get.isEmpty)
) (tables.apply _)
I want to then insert the JSON into a database after having validated it. I have therefore declared the following function.
def createFromJson = Action.async(parse.json) { request =>
request.body.validate[jsonWrapper] match {
case JsSuccess(data, _) =>
for {
dbFuture <- dataFuture(data.userID)
lastError <- dbFuture.insert(data.tables)
} yield {
Ok("Success\n")
}
case JsError(errors) => Future.successful(BadRequest("Failed :" + Error.show(errors)))
}
}
This works and correctly rejects JSONs looking like this:
{"tables":{"col1":[],"col2":[],"col3":[],"col4":[]}, "userID":"irrelavent"}
and accepts JSONs with actual data in, like so:
{"tables":{"col1":[{data1}],"col2":[{data2}],"col3":[{data3}],"col4":[{data4}]}, "userID":"irrelavent"}
But want i need is something that does this but also accepts a JSON with missing fields
{"tables":{"col1":[{data1}],"col2":[],"col3":[{data3}],"col4":[{data4}]}, "userID":"irrelavent"}
And preferable ignore them (i.e. return something like :
{"tables":{"col1":[{data1}],"col3":[{data3}],"col4":[{data4}]}, "userID":"irrelavent"})
Is this possible to do?
Many thanks,
Peter M.
You can automatically generate a Reads[tables] using Json.reads macro with the behavior you want:
implicit val tablesRead: Reads[tables] = Json.reads[tables]
If the fields is missing from the JSON the column will be None.
On a minor note, the common form in scala is to start a class name with a capital letter so you should rename tables to Tables.

Play JSON: Reading and validating a JsObject with unknown keys

I'm reading a nested JSON document using several Reads[T] implementations, however, I'm stuck with the following sub-object:
{
...,
"attributes": {
"keyA": [1.68, 5.47, 3.57],
"KeyB": [true],
"keyC": ["Lorem", "Ipsum"]
},
...
}
The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary. The values of the keys are always JsArray instances, but of different size and type (however, all elements of a particular array must have the same JsValue type).
The Scala representation of one single attribute:
case class Attribute[A](name: String, values: Seq[A])
// 'A' can only be String, Boolean or Double
The goal is to create a Reads[Seq[Attribute]] that can be used for the "attributes"-field when transforming the whole document (remember, "attributes" is just a sub-document).
Then there is a simple map that contains allowed combinations of keys and array types that should be used to validate attributes. Edit: This map is specific for each request (or rather specific for every type of json document). But you can assume that it is always available in the scope.
val required = Map(
"KeyA" -> "Double",
"KeyB" -> "String",
"KeyD" -> "String",
)
So in the case of the JSON shown above, the Reads should create two errors:
"keyB" does exist, but has the wrong type (expected String, was boolean).
"keyD" is missing (whereas keyC is not needed and can be ignored).
I'm having trouble creating the necessary Reads. The first thing I tried as a first step, from the perspective of the outer Reads:
...
(__ \ "attributes").reads[Map[String, JsArray]]...
...
I thought this is a nice first step because if the JSON structure is not an object containing Strings and JsArrays as key-value pairs, then the Reads fails with proper error messages. It works, but: I don't know how to go on from there. Of course I just could create a method that transforms the Map into a Seq[Attribute], but this method somehow should return a JsResult, since there are further validations to do.
The second thing I tried:
val attributeSeqReads = new Reads[Seq[Attribute]] {
def reads(json: JsValue) = json match {
case JsObject(fields) => processAttributes(fields)
case _ => JsError("attributes not an object")
}
def processAttributes(fields: Map[String, JsValue]): JsResult[Seq[Attribute]] = {
// ...
}
}
The idea was to validate each element of the map manually within processAttributes. But I think this is too complicated. Any help is appreciated.
edit for clarification:
At the beginning of the post I said that the keys (keyA, keyB...) are unknown at compile time. Later on I said that those keys are part of the map required which is used for validation. This sounds like a contradiction, but the thing is: required is specific for each document/request and is also not known at compile time. But you don't need to worry about that, just assume that for every request the correct required is already available in the scope.
You are too confused by the task
The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary
So the number of keys and their types are known in advance and the final?
So in the case of the JSON shown above, the Reads should create two
errors:
"keyB" does exist, but has the wrong type (expected String, was
boolean).
"keyD" is missing (whereas keyC is not needed and can be ignored).
Your main task is just to check the availability and compliance?
You may implement Reads[Attribute] for every your key with Reads.list(Reads.of[A]) (this Reads will check type and required) and skip omitted (if not required) with Reads.pure(Attribute[A]). Then tuple convert to list (_.productIterator.toList) and you will get Seq[Attribute]
val r = (
(__ \ "attributes" \ "keyA").read[Attribute[Double]](list(of[Double]).map(Attribute("keyA", _))) and
(__ \ "attributes" \ "keyB").read[Attribute[Boolean]](list(of[Boolean]).map(Attribute("keyB", _))) and
((__ \ "attributes" \ "keyC").read[Attribute[String]](list(of[String]).map(Attribute("keyC", _))) or Reads.pure(Attribute[String]("keyC", List()))) and
(__ \ "attributes" \ "keyD").read[Attribute[String]](list(of[String]).map(Attribute("keyD", _)))
).tupled.map(_.productIterator.toList)
scala>json1: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}
scala>res37: play.api.libs.json.JsResult[List[Any]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57)), Attribute(KeyB,List(true)), Attribute(keyC,List()), Attribute(KeyD,List(Lorem, Ipsum))),)
scala>json2: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyC":["Lorem","Ipsum"]}}
scala>res38: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray())))))
scala>json3: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":["Lorem"],"keyC":["Lorem","Ipsum"]}}
scala>res42: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray()))), (/attributes/keyB(0),List(ValidationError(List(error.expected.jsboolean),WrappedArray())))))
If you will have more than 22 attributes, you will have another problem: Tuple with more than 22 properties.
for dynamic properties in runtime
inspired by 'Reads.traversableReads[F[_], A]'
def attributesReads(required: Map[String, String]) = Reads {json =>
type Errors = Seq[(JsPath, Seq[ValidationError])]
def locate(e: Errors, idx: Int) = e.map { case (p, valerr) => (JsPath(idx)) ++ p -> valerr }
required.map{
case (key, "Double") => (__ \ key).read[Attribute[Double]](list(of[Double]).map(Attribute(key, _))).reads(json)
case (key, "String") => (__ \ key).read[Attribute[String]](list(of[String]).map(Attribute(key, _))).reads(json)
case (key, "Boolean") => (__ \ key).read[Attribute[Boolean]](list(of[Boolean]).map(Attribute(key, _))).reads(json)
case _ => JsError("")
}.iterator.zipWithIndex.foldLeft(Right(Vector.empty): Either[Errors, Vector[Attribute[_ >: Double with String with Boolean]]]) {
case (Right(vs), (JsSuccess(v, _), _)) => Right(vs :+ v)
case (Right(_), (JsError(e), idx)) => Left(locate(e, idx))
case (Left(e), (_: JsSuccess[_], _)) => Left(e)
case (Left(e1), (JsError(e2), idx)) => Left(e1 ++ locate(e2, idx))
}
.fold(JsError.apply, { res =>
JsSuccess(res.toList)
})
}
(__ \ "attributes").read(attributesReads(Map("keyA" -> "Double"))).reads(json)
scala> json: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}
scala> res0: play.api.libs.json.JsResult[List[Attribute[_ >: Double with String with Boolean]]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57))),/attributes)

Scala : Read JSON file with Play

I'm trying to extract the data from a JSON file that looks like that :
val json: JsValue = Json.parse("""
{
"item": {
"id" : "11111111",
"name" : "box",
"categories" : [{
"name" : "blue box",
"id" : "12345",
},
{
"name" : "fancy box",
"id" : "98765",
}]
}
}
""")
I would like to do it using the play JSON library. I came up with this code :
//I define my class and reader for one category
case class ItemCategory(id: Option[String], name: String)
implicit val categoryRead: Reads[ItemCategory] = (
(JsPath \ "item" \ "categories" \ "id").readNullable[String] and
(JsPath \ "item" \ "categories" \ "name").read[String]
)(ItemCategory.apply _)
//I define my class and reader for one item
case class MyItem(categories : Option[List[ItemCategory]], id : Option[String], name : Option[String])
implicit val myItemRead: Reads[MyItem] = (
(JsPath \ "item" \ "categories").readNullable[List[ItemCategory]] and
(JsPath \ "item" \ "id").readNullable[String] and
(JsPath \ "item" \ "name").readNullable[String]
)(MyItem.apply _)
//I then try to read :
var item: JsResult[MyItem] = json.validate[MyItem](myItemRead)
println(item)
However this code gives a JsError :List(ValidationError(validate.error.missing-path,WrappedArray())).
Which to my understanding simply means that some path went missing. I tried to read just one category and it worked fine, I try to read an item without trying to get the categories and here again it went fine. Hence I think the problem is on reading a list of items. I would really appreciate if you could help me with this.
Path is relative. categoryRead js path should be relative. Such as _ \ xxx

How to use different names when mapping JSON array to Scala object using combinators

Given a JSON array like this one:
{
"success": true,
"data": [
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
],
"additional_data": {
"pagination": {
"start": 0,
"limit": 100,
"more_items_in_collection": false
}
}
}
In my Play 2.2.2 application, using the Scala JSON Reads Combinator, everything works going this way:
implicit val entityReader = Json.reads[Entity]
val futureJson: Future[List[Entity]] = futureResponse.map(
response => (response.json \ "data").validate[List[Entity]].get
The problem now is the key named 'e54cbe3a434d8e6' which I would like to name 'value' in my object:
// This doesn't work, as one might expect
case class Entity(id: Long, title: String, e54cbe3a434d8e6: Long)
// I would like to use 'value' instead of 'e54cbe3a434d8e6'
case class Entity(id: Long, title: String, value: Long)
There is vast information about the combinators here and here but I only want to use a fieldname which is different from the key name in the JSON array. Can some one help me to find a simple way?
I suppose it has got something to do with JSON.writes?!
One simple way without trying to apply transformations on json itself is to define a custom Reads in such a way to handle this:
val json = obj(
"data" -> obj(
"id" -> 600,
"title" -> "test deal",
"e54cbe3a434d8e6" -> 54))
case class Data(id: Long, title: String, value: Int)
val reads = (
(__ \ "id").read[Long] ~
(__ \ "title").read[String] ~
(__ \ "e54cbe3a434d8e6").read[Int] // here you get mapping from your json to Scala case class
)(Data)
def index = Action {
val res = (json \ "data").validate(reads)
println(res) // prints "JsSuccess(Data(600,test deal,54),)"
Ok(json)
}
Another way is to use combinators like this:
... the same json and case class
implicit val generatedReads = reads[Data]
def index = Action {
val res = (json \ "data").validate(
// here we pick value at 'e54cbe3a434d8e6' and put into brand new 'value' branch
__.json.update((__ \ "value").json.copyFrom((__ \ "e54cbe3a434d8e6").json.pick)) andThen
// here we remove 'e54cbe3a434d8e6' branch
(__ \ "e54cbe3a434d8e6").json.prune andThen
// here we validate result with generated reads for our case class
generatedReads)
println(res) // prints "JsSuccess(Data(600,test deal,54),/e54cbe3a434d8e6/e54cbe3a434d8e6)"
Ok(prettyPrint(json))
}