How to traverse JSON object fields using JsPath? - json

Take into account the following JSON provided by a vendor API:
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
val json = Json.parse(
"""
|{
| "returns": {
| "markets" : {
| "ABC" : {
| "label": "ABC",
| "id":1
| },
| "DEF" : {
| "label": "DEF",
| "id":2
| }
| }
| }
|}
""".stripMargin)
How to extract a sequence of pairs related to "label" and "id" fields.
From this piece of JSON the result I'm expecting is:
Seq((1,"ABC"),(2,"DEF"))
I'm failing with constructing a correct JsPath extractor because it expects a single match e.g.
val jsonTransformer = (__ \ 'returns \ 'markets).json.pick
json.transform(jsonTransformer)

Here's how I'd build this parser out of nice composable pieces. First for a general purpose object-to-array transformer that throws away keys:
val objToArray: Reads[JsArray] =
JsPath.json.pickBranch.map(obj => JsArray(obj.fields.map(_._2)))
Now for a Reads that can process market objects:
val marketReads: Reads[(Int, String)] =
((__ \ 'id).read[Int] and (__ \ 'label).read[String]).tupled
And now we tie it all together:
val pairsReads: Reads[List[(Int, String)]] =
(__ \ 'returns \ 'markets).read(objToArray) andThen list(marketReads)
And finally:
scala> json.validate(pairsReads).foreach(println)
List((1,ABC), (2,DEF))
Which is what you want. Note that I've specified the types above for clarity, but that's not necessary here—I'd probably leave them out in real code because the pieces are pretty small and straightforward.

Related

Accessing attributes in objects Scala

An example of the sort of objects I need to grab from my json can be found in the following example(src):
{
"test": {
"attra": "2017-10-12T11:17:52.971Z",
"attrb": "2017-10-12T11:20:58.374Z"
},
"dummyCheck": false,
"type": "object",
"ruleOne": {
"default": 2557
},
"ruleTwo": {
"default": 2557
}
}
From the example above I want to access the default value under "ruleOne".
I've tried messing about with several different things below but I seem to be struggling. I can grab values like "dummyCheck" ok. What's the best way to key into where I need to go?
Example of how I am trying to get the value below:
import org.json4s._
import org.json4s.native.JsonMethods._
import org.json4s.DefaultFormats
implicit val formats = DefaultFormats
val test = parse(src)
println((test \ "ruleOne.default").extract[Integer])
Edit:
To further extend what is above:
def extractData(data: java.io.File) = {
val json = parse(data)
val result = (json \ "ruleOne" \ "default").extract[Int]
result
}
If I was to extend the above into a function that is called by passing in:
extractData(src)
That would only ever give me RuleOne.default.. is there a way I could extend it so that I could dynamically pass it multiple string arguments to parse (like a splat)
def extractData(data: java.io.File, path: String*) = {
val json = parse(data)
val result = (json \ path: _*).extract[Int]
result
}
so consuming it would be like
extractData(src, "ruleOne", "default")
This here works with "json4s-jackson" % "3.6.0-M2", but it should work in exactly the same way with native backend.
val src = """
|{
| "test": {
| "attra": "2017-10-12T11:17:52.971Z",
| "attrb": "2017-10-12T11:20:58.374Z"
| },
| "dummyCheck": false,
| "type": "object",
| "ruleOne": {
| "default": 2557
| },
| "ruleTwo": {
| "default": 2557
| }
|}""".stripMargin
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.DefaultFormats
implicit val formats = DefaultFormats
val test = parse(src)
println((test \ "ruleOne" \ "default").extract[Int])
Output:
2557
To make it work with native, simply replace
import org.json4s.jackson.JsonMethods._
by
import org.json4s.native.JsonMethods._
and make sure that you have the right dependencies.
EDIT
Here is a vararg method that transforms string parameters into a path:
def extract(json: JValue, path: String*): Int = {
path.foldLeft(json)(_ \ _).extract[Int]
}
With this, you can now do:
println(extract(test, "ruleOne", "default"))
println(extract(test, "ruleTwo", "default"))
Note that it accepts a JValue, not a File, because the version with File would be unnecessarily painful to test, whereas JValue-version can be tested with parsed string constants.

parsing json in scala which contains map

I'm trying to parse json string with scala and playframework, I've read doc but still I'm stuck.
I got:
val jsonStr = """{
| "metric": "java.lang.Memory.HeapMemoryUsage.committed",
| "tags": {
| "instanceId": "ubuntu",
| "runId": "B_name_of_the_app-c4m8_0_2016-01-01_23-31-34"
| },
| "aggregateTags": [],
| "dps": {
| "1455711498": 8.71890944E8,
| "1455711558": 9.10688256E8,
| "1455711618": 9.24319744E8,
| "1455711678": 8.47773696E8,
| "1455711738": 9.35329792E8,
| "1455711798": 9.53679872E8,
| "1455714981": 1.983905792E9,
| "1455715041": 2.054684672E9,
| "1455715101": 2.05520896E9
| }
| }""".stripMargin
according to playframework doc I created classes to parse this thing:
// start of the scala file
import play.api.libs.json.{JsPath, Json, Reads}
import play.api.libs.functional.syntax._
case class Metric(metricName: String, tags: Tags, aggregateTags: Option[Seq[String]], dps: Seq[Map[String,Double]])
object Metric{
implicit val metricReads: Reads[Metric] = (
(JsPath \ "metric").read[String] and
(JsPath \ "tags").read[Tags] and
(JsPath \ "aggreagateTags").readNullable[Seq[String]] and
(JsPath \ "dps").read[Seq[Map[String,Double]]] //this one is tricky
)(Metric.apply _)
}
case class Tags(instanceId:String, runId: String)
object Tags{
implicit val tagsReads: Reads[Tags] = (
(JsPath \ "instanceId").read[String] and (JsPath \ "runId").read[String]
)(Tags.apply _)
}
Json.parse(jsonStr).validate[Metric]
// end of the scala file
Unfortunately validation results with:
res0: play.api.libs.json.JsResult[Metric] = JsError(List((//dps,List(ValidationError(List(error.expected.jsarray),WrappedArray())))))
I'm not sure how to solve this problem, also tried parse dps' as a separate class but also didn't work..Any tips?
You have (JsPath \ "dps").read[Seq[Map[String,Double]]] but dps in your JSON is not a Seq but a single entry - changing that part of the reader to (JsPath \ "dps").read[Map[String,Double]] will fix the problem.

Play Framework: How to Validate Optional Fields When Parsing JSON

Given the following JSON...
{
"nickname": "mj",
"firstname": "Mike",
"lastName": "Jordan",
"trash": "ignore"
}
... I need to parse and validate it so that
any field except nickname, firstName, and lastName are filtered out – in my example above trash has to be removed
if defined, nickname (which is optional) must be at least 3 characters long
Here below is my code:
import play.api.libs.json._
import play.api.libs.functional.syntax._
def orEmpty = reads | __.json.put(Json.obj())
val js = Json.parse("""{ "nickname": "mj", "firstname": "Mike", "lastName": "Jordan" }""")
val validateUser = (
((__ \ 'nickname).json.pickBranch(Reads.of[JsString] <~ Reads.minLength[String](3)) orEmpty) ~
((__ \ 'firstName).json.pickBranch) ~
((__ \ 'lastName).json.pickBranch)
)
validateUser.reads(js).fold(
valid = { validated => JsSuccess(js) },
invalid => { errors => JsError(errors) }
)
The problem is that if nickname is invalid because shorter than 3 characters, orEmpty applies and no error is reported. What I need is to keep nickname optional (that's why I defined orEmpty), but when defined the validation should succeed if and only if nickanme passes the Reads.minLength check.
Assuming Play Framework 2.4.x. I believe reading nullable fields is a little different in 2.3.x but it's the same idea.
It would be easier to reason about this if you assume that JSON validators and transformers are really just Reads[T <: JsValue].
Overall, what you need to consider is that the nickname field is actually optional and you need to represent it as such. Basically, what you're looking for is to compose the nullableReads with Reads.minLength to read nickname as Option[String].
So, you can represent the structure you want to validate as:
case class ToValidate(nickname: Option[String], firstname: String, lastname: String)
And define you're Reader:
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
implicit val toValidateReads: Reads[ToValidate] = (
(__ \ "nickname").readNullable[String](minLength(3)) ~
(__ \ "firstname").read[String] ~
(__ \ "lastname").read[String]
)(ToValidate.apply _)
Then you can vaildate your input:
val js = Json.parse("""{ "nickname": "mj", "firstname": "Mike", "lastname": "Jordan" }""")
val v = js.validate[ToValidate]
println(v) // JsError(List((/nickname,List(ValidationError(List(error.minLength),WrappedArray(3))))))

If statements within Play/Scala JSON parsing?

Is there a way to perform conditional logic while parsing json using Scala/Play?
For example, I would like to do something like the following:
implicit val playlistItemInfo: Reads[PlaylistItemInfo] = (
(if(( (JsPath \ "type1").readNullable[String]) != null){ (JsPath \ "type1" \ "id").read[String]} else {(JsPath \ "type2" \ "id").read[String]}) and
(JsPath \ "name").readNullable[String]
)(PlaylistItemInfo.apply _)
In my hypothetical JSON parsing example, there are two possible ways to parse the JSON. If the item is of "type1", then there will be a value for "type1" in the JSON. If this is not present in the JSON or its value is null/empty, then I would like to read the JSON node "type2" instead.
The above example does not work, but it gives you the idea of what I am trying to do.
Is this possible?
The proper way to do this with JSON combinators is to use orElse. Each piece of the combinator must be a Reads[YourType], so if/else doesn't quite work because your if clause doesn't return a Boolean, it returns Reads[PlaylistItemInfo] checked against null which will always be true. orElse let's us combine one Reads that looks for the type1 field, and a second one that looks for the type2 field as a fallback.
This might not follow your exact structure, but here's the idea:
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class PlaylistItemInfo(id: Option[String], tpe: String)
object PlaylistItemInfo {
implicit val reads: Reads[PlaylistItemInfo] = (
(__ \ "id").readNullable[String] and
(__ \ "type1").read[String].orElse((__ \ "type2").read[String])
)(PlaylistItemInfo.apply _)
}
// Read type 1 over type 2
val js = Json.parse("""{"id": "test", "type1": "111", "type2": "2222"}""")
scala> js.validate[PlaylistItemInfo]
res1: play.api.libs.json.JsResult[PlaylistItemInfo] = JsSuccess(PlaylistItemInfo(Some(test),111),)
// Read type 2 when type 1 is unavailable
val js = Json.parse("""{"id": "test", "type2": "22222"}""")
scala> js.validate[PlaylistItemInfo]
res2: play.api.libs.json.JsResult[PlaylistItemInfo] = JsSuccess(PlaylistItemInfo(Some(test),22222),)
// Error from neither
val js = Json.parse("""{"id": "test", "type100": "fake"}""")
scala> js.validate[PlaylistItemInfo]
res3: play.api.libs.json.JsResult[PlaylistItemInfo] = JsError(List((/type2,List(ValidationError(error.path.missing,WrappedArray())))))

Parsing JSON Date Time in Scala/Play

I have the following Read defined:
import org.joda.time.DateTime;
implicit val userInfoRead: Reads[UserInfo] = (
(JsPath \ "userName").readNullable[String] and
] (JsPath \ "startDate").readNullable[DateTime]
(UserInfo.apply _)
With the following JSON object being passed in:
"userInfo" : {
"userName": "joeuser",
"startDate": "2006-02-28"
}
When I validate this data I get the following error:
(/startDate,List(ValidationError(validate.error.expected.jodadate.format,WrappedArray(yyyy-MM-dd))))))
Any suggestions on what I'm missing in the formatting?
As far as I can see, the issue is probably just the format not matching what Joda is expecting. I simplified a bit, and this worked for me:
scala> import org.joda.time.DateTime
import org.joda.time.DateTime
scala> case class UserInfo(userName: String, startDate: DateTime)
defined class UserInfo
scala> implicit val dateReads = Reads.jodaDateReads("yyyy-MM-dd")
dateReads: play.api.libs.json.Reads[org.joda.time.DateTime] = play.api.libs.json.DefaultReads$$anon$10#22db02cb
scala> implicit val userInfoReads = Json.reads[UserInfo]
userInfoReads: play.api.libs.json.Reads[UserInfo] = play.api.libs.json.Reads$$anon$8#52bcbd5d
scala> val json = Json.parse("""{
| "userName": "joeuser",
| "startDate": "2006-02-28"
| }""")
json: play.api.libs.json.JsValue = {"userName":"joeuser","startDate":"2006-02-28"}
scala> json.validate[UserInfo]
res12: play.api.libs.json.JsResult[UserInfo] = JsSuccess(UserInfo(joeuser,2006-02-28T00:00:00.000-05:00),)