I'm trying to extract the data from a JSON file that looks like that :
val json: JsValue = Json.parse("""
{
"item": {
"id" : "11111111",
"name" : "box",
"categories" : [{
"name" : "blue box",
"id" : "12345",
},
{
"name" : "fancy box",
"id" : "98765",
}]
}
}
""")
I would like to do it using the play JSON library. I came up with this code :
//I define my class and reader for one category
case class ItemCategory(id: Option[String], name: String)
implicit val categoryRead: Reads[ItemCategory] = (
(JsPath \ "item" \ "categories" \ "id").readNullable[String] and
(JsPath \ "item" \ "categories" \ "name").read[String]
)(ItemCategory.apply _)
//I define my class and reader for one item
case class MyItem(categories : Option[List[ItemCategory]], id : Option[String], name : Option[String])
implicit val myItemRead: Reads[MyItem] = (
(JsPath \ "item" \ "categories").readNullable[List[ItemCategory]] and
(JsPath \ "item" \ "id").readNullable[String] and
(JsPath \ "item" \ "name").readNullable[String]
)(MyItem.apply _)
//I then try to read :
var item: JsResult[MyItem] = json.validate[MyItem](myItemRead)
println(item)
However this code gives a JsError :List(ValidationError(validate.error.missing-path,WrappedArray())).
Which to my understanding simply means that some path went missing. I tried to read just one category and it worked fine, I try to read an item without trying to get the categories and here again it went fine. Hence I think the problem is on reading a list of items. I would really appreciate if you could help me with this.
Path is relative. categoryRead js path should be relative. Such as _ \ xxx
Related
I'm trying to write a reads for the following class
case class User(id: String, imageId: Option[String])
The JSON I have looks like this:
{
"id": 1
"image": { "id" : 2 }
}
However the 'image' field may not exist, or it may be null.
My reads looks like this:
implicit val userReader: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "image" \ "id").readNullable[String]
) (User.apply _)
But this does not seem to work, I still get an error when it finds a sample with no image. 'ValidationError [...] error.path.missing [...] /image/id'
How can this be solved? I'd prefer not to create an Image class, which I'm not going to use anywhere.
I think the issue comes from the fact that you are trying to read a String whereas id is an Int. You should mind your types in your json / reader.
Two options
if id is really an Int, read it as an Int
if id is a String, change your JSON
If you still want to have a String in your case class, you can do :
(JsPath \ "image" \ "id").readNullable[Int].map(_.toString)
You forgot a comma between the two elements ("id" and "image"). That's why the /image/id path seems to be missing for the parser.
There are all sorts of tools for validating a JSON structure, for instance https://jsonlint.com/ .
So your JSON structure should look like this:
{
"id": 1,
"image": {
"id": 2
}
}
i may be late but here is a workaround:
implicit val userReader: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "image" \ "id").readNullable[String].orElse((__ \ "dummy").readNullable[String])
) (User.apply _)
Not perfect but it work well
better solution would be to directly cast the Js Reads(JsSuccess(None:Option[Int])) inside the orElse but i haven't been able to do it...
I can not come up with solution for Patch semantic:
if json has no property, I need skip modification
if json property has null, I need to remove this property (for not required properties)
in other cases I need to set the value
I need transformation into mongo.db update command ("$unset" for 2, "$set" for 3)
For example I need store json with required property "summary". So:
{"summary": "modified by patch", "description": null}
must be transformed to:
{
"$set" : {
"summary": "modified by patch"
},
"$unset": {
"description": ""
}
}
this json
{"description": null}
must be transformed to ("summary" is skipped):
{
"$unset" : {
"description": ""
}
}
and for this
{"summary": null}
I need transformation error (can't remove required properties)
My solution is
def patch(path: JsPath)(r: Reads[JsObject]) = Reads{json =>
path.asSingleJsResult(json).fold(
_ => JsSuccess(Json.obj()),
_ => r.reads(json)
)
}
and for requiered properties
def requiredError = ValidationError("error.remove.required")
val summaryPatch = patch(__ \ "summary")(
(__ \ "$set" \ "summary").json.copyFrom(
(__ \ "summary").json.pick.filterNot(requiredError)(_ == JsNull)
)
)
for other
val descriptionPatch = patch(__ \ "description")(
(__ \ "$set" \ "description").json.copyFrom(
(__ \ "description").json.pick.filterNot(_ == JsNull)
) orElse
(__ \ "$unset" \ "description").json.copyFrom(
(__ \ "description").json.pick)
)
)
to mongo.db trasformer:
toMongoPatch = (summaryPatch and descriptionPatch).reduce
I'm trying to parse the data from GovTrack, for example, https://www.govtrack.us/api/v2/bill/74369 . But titles is in a peculiar format:
"titles": [
[
"short",
"introduced",
"Public Online Information Act of 2011"
],
[
"official",
"introduced",
"To establish an advisory committee to issue nonbinding governmentwide guidelines..."
]
]
titles is an array of each title type, with fields in a particular order. I want to read this into a more standard JSON format:
{
'short_title': "Public Online Information Act of 2011",
'official_title': "To establish an advisory committee to issue nonbinding governmentwide guidelines..."
}
The short title or official title may or may not be there, and there could actually be several short titles.
How do I make a Reads for this? Right now I've got:
implicit val billReads: Reads[Bill] = (
(JsPath \ "id").read[Int] and
(JsPath \ "display_number").read[String] and
(JsPath \ "current_status").read[String] and
(JsPath \ "titles")(0)(2).read[String]
)(Bill.apply _)
How do I specify "The member of the array that has a first element equal to 'official'"?
As far as I know, there is no out of the box way to do it, but I would do it with additional custom reader, like this:
val officialReads = new Reads[String] {
override def reads(json: JsValue): JsResult[String] = (json \ "titles") match {
case JsArray(titles) => titles.collectFirst({
case JsArray(values) if (values.headOption.map(v => "official".equals(v.as[String])).getOrElse(false)) =>
JsSuccess(values.tail.tail.head.as[String])
}).getOrElse(JsError("No official title"))
case _ => JsError("Can't read official title")
}
}
And your Bill reader would look like this:
val implicitReads: Reads[Bill] = (
(__ \ "id").read[Int] and
(__ \ "display_number").read[String] and
(__ \ "current_status").read[String] and
officialReads
)(Bill.apply _)
I've tested, this works :)
Given a JSON array like this one:
{
"success": true,
"data": [
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
],
"additional_data": {
"pagination": {
"start": 0,
"limit": 100,
"more_items_in_collection": false
}
}
}
In my Play 2.2.2 application, using the Scala JSON Reads Combinator, everything works going this way:
implicit val entityReader = Json.reads[Entity]
val futureJson: Future[List[Entity]] = futureResponse.map(
response => (response.json \ "data").validate[List[Entity]].get
The problem now is the key named 'e54cbe3a434d8e6' which I would like to name 'value' in my object:
// This doesn't work, as one might expect
case class Entity(id: Long, title: String, e54cbe3a434d8e6: Long)
// I would like to use 'value' instead of 'e54cbe3a434d8e6'
case class Entity(id: Long, title: String, value: Long)
There is vast information about the combinators here and here but I only want to use a fieldname which is different from the key name in the JSON array. Can some one help me to find a simple way?
I suppose it has got something to do with JSON.writes?!
One simple way without trying to apply transformations on json itself is to define a custom Reads in such a way to handle this:
val json = obj(
"data" -> obj(
"id" -> 600,
"title" -> "test deal",
"e54cbe3a434d8e6" -> 54))
case class Data(id: Long, title: String, value: Int)
val reads = (
(__ \ "id").read[Long] ~
(__ \ "title").read[String] ~
(__ \ "e54cbe3a434d8e6").read[Int] // here you get mapping from your json to Scala case class
)(Data)
def index = Action {
val res = (json \ "data").validate(reads)
println(res) // prints "JsSuccess(Data(600,test deal,54),)"
Ok(json)
}
Another way is to use combinators like this:
... the same json and case class
implicit val generatedReads = reads[Data]
def index = Action {
val res = (json \ "data").validate(
// here we pick value at 'e54cbe3a434d8e6' and put into brand new 'value' branch
__.json.update((__ \ "value").json.copyFrom((__ \ "e54cbe3a434d8e6").json.pick)) andThen
// here we remove 'e54cbe3a434d8e6' branch
(__ \ "e54cbe3a434d8e6").json.prune andThen
// here we validate result with generated reads for our case class
generatedReads)
println(res) // prints "JsSuccess(Data(600,test deal,54),/e54cbe3a434d8e6/e54cbe3a434d8e6)"
Ok(prettyPrint(json))
}
I have the following Json:
{
"web-category" : "macaroons",
"sub-categories" : [
{ "name" : "pink" },
{ "name" : "blue" },
{ "name" : "green" }
]
}
I have got it in Play as a JsObject. So I can now successfully do the following:
//(o is the JsObject)
val webCat:Option[String] = (o \ "web-category").asOpt[String]
println(webCat.toString)
>> Some(macaroons)
So far, so good. But how do I access the array Json objects? I have this...
val subCats:Option[JsArray] = (o \ "sub-categories").asOpt[JsArray]
println(subCats.toString)
>> Some([{"name" : "blue"},{"name" : "green"},{"name" : "pink"}])
but what I need is to take the JsArray and get a List of all the names something like this:
List("blue", "green", "pink")
Don't know how to access the JsArray thusly.
my thanks for your help in this.
I'd argue that it's generally a good idea to move from JSON-land to native-Scala-representation-land as early as possible. If obj is your JsObject, for example, you can write this:
val subCategories = (obj \ "sub-categories").as[List[Map[String, String]]]
val names = subCategories.map(_("name"))
Or even:
case class Category(name: String, subs: List[String])
import play.api.libs.functional.syntax._
implicit val categoryReader = (
(__ \ "web-category").read[String] and
(__ \ "sub-categories").read[List[Map[String, String]]].map(_.map(_("name")))
)(Category)
And then:
obj.as[Category]
This latter approach makes error handling even cleaner (e.g. you can just replace as with asOpt at this top level) and composes nicely with other Reads type class instances—if you have a JsArray of these objects, for example, you can just write array.as[List[Category]] and get what you expect.
What Peter said, or:
(o \ "sub-categories" \\ "name").map(_.as[String]).toList
Something like this:
subCats.map( jsarray => jsarray.value.map(jsvalue => (jsvalue \ "name").as[String]).toList)
This will normally return a Option[List[String]]