Play JSON: reading optional nested properties - json

I have the following case classes and JSON combinators:
case class Commit(
sha: String,
username: String,
message: String
)
object Commit {
implicit val format = Json.format[Commit]
}
case class Build(
projectName: String,
parentNumber: String,
commits: List[Commit]
)
val buildReads: Reads[Build] =
for {
projectName <- (__ \ "buildType" \ "projectName").read[String]
name <- (__ \ "buildType" \ "name").read[String]
parentNumber <- ((__ \ "artifact-dependencies" \ "build")(0) \ "number").read[String]
changes <- (__ \ "changes" \ "change").read[List[Map[String, String]]]
} yield {
val commits = for {
change <- changes
sha <- change.get("version")
username <- change.get("username")
comment <- change.get("comment")
} yield Commit(sha, username, comment)
Build(s"$projectName::$name", parentNumber, commits)
}
My JSON reads combinator for Build will handle incoming JSON such as:
{
"buildType": {
"projectName": "foo",
"name": "bar"
},
"artifact-dependencies": {
"build": [{
"number": "1"
}]
},
"changes": {
"change": [{
"verison": "1",
"username": "bob",
"comment": "foo"
}]
}
}
However, if artifact-dependencies is missing, it will fall over. I would like this to be optional.
Should I use readNullable? I have tried to do so, but this fails because it is a nested property.
Does this look pragmatic, or am I abusing JSON combinators to parse my JSON into a case class?

Currently the Format[Commit] in its companion object isn't being used. There's no reason we can't use simple combinators for that, and separate the logic.
case class Commit(sha: String, username: String, message: String)
object Commit {
implicit val reads: Reads[Commit] = (
(__ \ "version").read[String] and
(__ \ "username").read[String] and
(__ \ "comment").read[String]
)(Commit.apply _)
}
Then, if "artifact-dependencies" can be missing, we should make parentNumber an Option[String] in Build.
case class Build(projectName: String, parentNumber: Option[String], commits: List[Commit])
I split the Reads that combines project names into a separate one to make the Reads[Build] look a little more clean.
val nameReads: Reads[String] = for {
projectName <- (__ \ "projectName").read[String]
name <- (__ \ "name").read[String]
} yield s"$projectName::$name"
Then, for when "artifact-dependencies" is missing, we can use orElse and Reads.pure(None) to fill it with None when that entire branch (or sub-branch) is not there. In this case, that would be simpler than mapping each step of the way.
implicit val buildReads: Reads[Build] = (
(__ \ "buildType").read[String](nameReads) and
((__ \ "artifact-dependencies" \ "build")(0) \ "number").readNullable[String].orElse(Reads.pure(None)) and
(__ \ "changes" \ "change").read[List[Commit]]
)(Build.apply _)
val js2 = Json.parse("""
{
"buildType": {
"projectName": "foo",
"name": "bar"
},
"changes": {
"change": [{
"version": "1",
"username": "bob",
"comment": "foo"
}]
}
}
""")
scala> js2.validate[Build]
res6: play.api.libs.json.JsResult[Build] = JsSuccess(Build(foo::bar,None,List(Commit(1,bob,foo))),)

I try to have my formats match the json as closely as possible. Admittedly, in this case it's a bit awkward, but that's because the json schema is kind of weird. Here's how I would do it given those limitations:
import play.api.libs.functional.syntax._
import play.api.libs.json._
case class Build(buildType: BuildType, `artifact-dependencies`: Option[ArtifactDependencies], changes: Changes)
case class BuildType(projectName: String, name: String)
case class ArtifactDependencies(build: List[DependencyInfo])
case class DependencyInfo(number: String)
case class Changes(change: List[Commit])
case class Commit(version: String, username: String, comment: String)
object BuildType {
implicit val buildTypeReads: Reads[BuildType] = (
(JsPath \ "projectName").read[String] and
(JsPath \ "name").read[String]
)(BuildType.apply _)
}
object ArtifactDependencies {
implicit val artifactDependencyReads: Reads[ArtifactDependencies] =
(JsPath \ "build").read[List[DependencyInfo]].map(ArtifactDependencies.apply)
}
object DependencyInfo {
implicit val dependencyInfoReads: Reads[DependencyInfo] =
(JsPath \ "number").read[String].map(DependencyInfo.apply)
}
object Changes {
implicit val changesReads: Reads[Changes] =
(JsPath \ "change").read[List[Commit]].map(Changes.apply)
}
object Commit {
implicit val commitReads: Reads[Commit] = (
(JsPath \ "version").read[String] and
(JsPath \ "username").read[String] and
(JsPath \ "comment").read[String]
)(Commit.apply _)
}
object Build {
implicit val buildReads: Reads[Build] = (
(JsPath \ "buildType").read[BuildType] and
(JsPath \ "artifact-dependencies").readNullable[ArtifactDependencies] and
(JsPath \ "changes").read[Changes]
)(Build.apply _)
def test() = {
val js = Json.parse(
"""
|{
| "buildType": {
| "projectName": "foo",
| "name": "bar"
| },
| "changes": {
| "change": [{
| "version": "1",
| "username": "bob",
| "comment": "foo"
| }]
| }
|}
""".stripMargin)
println(js.validate[Build])
val js1 = Json.parse(
"""
|{
| "buildType": {
| "projectName": "foo",
| "name": "bar"
| },
| "artifact-dependencies": {
| "build": [{
| "number": "1"
| }]
| },
| "changes": {
| "change": [{
| "version": "1",
| "username": "bob",
| "comment": "foo"
| }]
| }
|}
""".stripMargin)
println(js1.validate[Build])
}
}
The output is:
[info] JsSuccess(Build(BuildType(foo,bar),None,Changes(List(Commit(1,bob,foo)))),)
[info] JsSuccess(Build(BuildType(foo,bar),Some(ArtifactDependencies(List(DependencyInfo(1)))),Changes(List(Commit(1,bob,foo)))),)
Note that the slightly awkward
(JsPath \ "change").read[List[Commit]].map(Changes.apply)
is necessary for single argument case classes.
EDIT:
The crucial part I missed is that parentNumber now becomes a method defined on Build as follows:
case class Build(buildType: BuildType, `artifact-dependencies`: Option[ArtifactDependencies], changes: Changes) {
def parentNumber: Option[String] = `artifact-dependencies`.flatMap(_.build.headOption.map(_.number))
}

Related

Conditional filtering of JSON before deserialisation to case class model

How to parse a JSON conditionally before deserialisation to the following case class:
case class UserInfo(id: String, startDate: String, endDate: String)
I have an implicit reads
object UserInfo {
implicit val reads: Reads[UserInfo] = (
(__ \ "id").read[String] and
(__ \ "startDate").read[String] and
(__ \ "endDate").read[String]
)(UserInfo.apply _)
}
I can parse the following json using above implicit reads
val jsonString = """
{
"users":[
{
"id":"123",
"startDate":"2019-06-07",
"endDate":"2019-06-17"
},
{
"id":"333",
"startDate":"2019-06-07",
"endDate":"2019-06-27"
}
]
}"""
val userInfoList = (Json.parse(jsonString) \ "users").as[List[UserInfo]]
but sometimes the web service returns a JSON with no startDate and endDate, for example:
{
"users":[
{
"id":"123",
"startDate":"2019-06-07",
"endDate":"2019-06-17"
},
{
"id":"333",
"startDate":"2019-06-07"
},
{
"id":"444"
}
]
}
How to conditionally parse json to ignore objects that don't have startDate or endDate without making those fields optional in UserInfo model?
To avoid changing the model to optional fields we could define coast-to-coast transformer which filters out users with missing dates like so
val filterUsersWithMissingDatesTransformer = (__ \ 'users).json.update(__.read[JsArray].map {
case JsArray(values) => JsArray(values.filter { user =>
val startDateOpt = (user \ "startDate").asOpt[String]
val endDateOpt = (user \ "endDate").asOpt[String]
startDateOpt.isDefined && endDateOpt.isDefined
})
})
which given
val jsonString =
"""
|{
| "users":[
| {
| "id":"123",
| "startDate":"2019-06-07",
| "endDate":"2019-06-17"
| },
| {
| "id":"333",
| "startDate":"2019-06-07"
| },
| {
| "id":"444"
| }
| ]
|}
""".stripMargin
val filteredUsers = Json.parse(jsonString).transform(filterUsersWithMissingDatesTransformer)
println(filteredUsers.get)
outputs
{
"users": [
{
"id": "123",
"startDate": "2019-06-07",
"endDate": "2019-06-17"
}
]
}
meaning we can deserialise to the existing model without making startDate and endDate optional.
case class UserInfo(id: String, startDate: String, endDate: String)
You can use Option for this:
case class UserInfo(id: String, startDate: Option[String], endDate: Option[String])
object UserInfo {
implicit val reads: Reads[UserInfo] = (
(__ \ "id").read[String] and
(__ \ "startDate").readNullable[String] and
(__ \ "endDate").readNullable[String]
)(UserInfo.apply _)
}
This would work when startDate and endDate are not provided.

Specify a nested object in scala's play json

Given this code:
case class SocialUser(firstName: String, lastName: String)
case class UserDetails(avatarUrl: String, phone: String)
// I want to avoid having to specify each SocialUser field one by one but just use the implicit write as stated below
implicit val socialUserWrites = Json.writes[SocialUser]
implicit val userDetailsWrites = Json.writes[UserDetails]
Now, how could I output the json in this format?
{"user": {
"firstName: "",
"lastName": "",
"details": {
"avatarUrl": "",
"phone": "",
}
}}
You miss "user" in "UserDetail" writes:
implicit val combinedUserWrites: Writes[CombinedUser] = (
(__ \ "user").write[SocialUser] and
(__ \ "user" \ "userDetails").write[UserDetails]
)(unlift(CombinedUser.unapply))
x: CombinedUser = CombinedUser(SocialUser(f,l),UserDetails(a,p))
scala> res4: play.api.libs.json.JsValue = {"user":{"firstName":"f","lastName":"l","userDetails":{"avatarUrl":"a","phone":"p"}}}

Play Framework and Scala Json, parsing for json containing JSArray and JSObject

My sample json is either with a country object
Json sample 1
"#version": "1.0",
"country": {
"#country": "US",
"day": {
"#date": "2016-02-15",
"#value": "1"
}
}
or with country array:
Json sample 2
"#version": "1.0",
"country": [{
"#country": "US",
"day": {
"#date": "2016-02-15",
"#value": "1"
}
}, {
"#country": "UK",
"day": {
"#date": "2016-02-15",
"#value": "5"
}]
}
To read the json
implicit val dayJsonReads: Reads[DayJson] = (
(JsPath \ "#date").read[DateTime](dateReads) and
((JsPath \ "#value").read[Int] orElse (JsPath \ "#value").read[String].map(_.toInt))
)(DayJson.apply _)
implicit val countryJsonReads: Reads[CountryJson] = (
(JsPath \ "#country").read[String] and
(JsPath \ "day").read[DayJson]
)(CountryJson.apply _)
implicit val newUserJsonReads: Reads[NewUserJson] = (
(JsPath \ "#version").read[String] and
(JsPath \ "country").readNullable[Seq[CountryJson]]
)(NewUserJsonParent.apply _)
The above code reads sample json 2 however fails for sample json 1. Is it possible to use readNullable to read either JS Value or JS Object or can we convert it from JS Value to JS Object. Thank you.
You can do something like this:
object NewUserJson{
implicit val newUserJsonReads: Reads[NewUserJson] = (
(JsPath \ "#version").read[String] and
(JsPath \ "country").read[JsValue].map{
case arr: JsArray => arr.as[Seq[CountryJson]]
case obj: JsObject => Seq(obj.as[CountryJson])
}
)(NewUserJson.apply _)
}
This should work for this case class:
case class NewUserJson(`#version`: String, country: Seq[CountryJson])
But I don't like it, can't you just use the same structure, and when you have only one country just send a list that hold only one country, instead of object?
Working on Tomer's solution, below is a working sample. It would be nice if I can make it more compact.
Case class
case class NewUserJson(version: String, country: Option[Seq[CountryJson]])
Json parsing object
object NewUserJson{
implicit val newUserJsonReads: Reads[NewUserJson] = (
(JsPath \ "#version").read[String] and
(JsPath \ "country").readNullable[JsValue].map {
arr => {
if (!arr.isEmpty){
arr.get match {
case arr: JsArray => Option(arr.as[Seq[CountryJson]])
case arr: JsObject => Option(Seq(arr.as[CountryJson]))
}
}else {
None
}
}
}
)(NewUserJson.apply _)
}

How to parse this JSON using Play Framework?

I have the following JSON being returned from a web service:
{
"hits": [
{
"created_at": "2016-02-01T15:01:03.000Z",
"title": "title",
"num_comments": 778,
"parent_id": null,
"_tags": [
"story",
"author",
"story_11012044"
],
"objectID": "11012044",
"_highlightResult": {
"title": {
"value": "title",
"matchLevel": "full",
"matchedWords": [
"title"
]
},
"author": {
"value": "author",
"matchLevel": "none",
"matchedWords": [
]
},
"story_text": {
"value": "Please lead",
"matchLevel": "none",
"matchedWords": [
]
}
}
}
]
}
and I am trying to parse it using the JSON parsing libs in Play Framework. I have the following code:
import play.api.libs.functional.syntax._
import play.api.libs.json._
case class Post(id: Long, date: String, count: Int)
object Post {
implicit val postFormat = Json.format[Post]
implicit val writes: Writes[Post] = (
(JsPath \ "id").write[Long] and
(JsPath \"date").write[String] and
(JsPath \ "count").write[Int]
)(unlift(Post.unapply))
implicit val reads: Reads[Post] = (
(JsPath \ "objectID").read[Long] and
(JsPath \ "created_at").read[String] and
(JsPath \ "num_comments").read[Int]
)(Post.apply _)
}
import play.api.libs.json._
class PostRepo {
val request: WSRequest = ws.url(MY_URL)
def getPosts: Future[Seq[Post]] =
val result: Future[JsValue] = request.get().map(response =>
response.status match {
case 200 => Json.parse(response.body)
case _ => throw new Exception("Web service call failed: " + response.body)
})
result.map( {
jsonvalue => println("JSARRAY: " + jsonvalue);
(jsonvalue \ "hits").as[Seq[Post]]
})
result
}
Now, when I run the code, I am getting the following error:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[JsResultException:
JsResultException(errors:List(((0)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((0)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((0)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/id,List(ValidationError(List(error.path.missing),WrappedArray())))
Obviously something is wrong with the way I'm trying to parse the JSON but I have now spent a few hours trying to figure out the problem and I'm well and truly stuck.
Some refactoring code with Reads.seq
val r = (__ \ "hits").read[Seq[Post]](Reads.seq[Post])
def getPosts: Future[Seq[Post]] = {
WS.url(MY_URL).get().map(response =>
response.status match {
case 200 => r.reads(response.json) match {
case JsError(e) => throw new Exception("Json read fails. Response body:" + response.json.toString() + "\nRead error:" + e.toString())
case JsSuccess(x, _) => x
}
case _ => throw new Exception("Web service call failed: " + response.body)
})
}

Play Scala JSON Reads converter: mapping nested properties

I have the following case class:
case class User(name: String, age: String)
I am trying to implement a JSON Reads converter for it, so I can do the following:
val user = userJson.validate[User]
… but the incoming JSON has slightly different structure:
{ "age": "12", "details": { "name": "Bob" } }
How can I implement my JSON Reads converter?
You can do this using combinators to parse sub-paths.
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class User(name: String, age: String)
val js = Json.parse("""
{ "age": "12", "details": { "name": "Bob" } }
""")
implicit val reads: Reads[User] = (
(__ \ "details" \ "name").read[String] and
(__ \ "age").read[String]
)(User.apply _)
scala> js.validate[User]
res2: play.api.libs.json.JsResult[User] = JsSuccess(User(Bob,12),)