How to flatten a js array on Play using JSON transformers? - json

I have a Json of the following format:
{
"user": {
"id": "1",
"name": "Some User",
"permGroups": [
{
"id": "group1",
"name": "Group 1",
"actions": [
{
"id": "action1",
"name": "Action 1"
}
]
},
{
"id": "group2",
"name": "Group 2",
"actions": [
{
"id": "action2",
"name": "Action 2"
},
{
"id": "action3",
"name": "Action 3"
}
]
}
]
},
"title": "New Role",
"role_id": "56fea66c"
}
How can I make a JSON Transformer in Play! 2.1, that will turn that into:
{
"name": "New Role",
"id" : "56fea66c",
"permGroupIds": ["group1","group2"]
"actions": ["action1", "action2", "action3"]
}
I have this working so far:
import play.api.libs.json._
import play.api.libs.functional.syntax._
import play.api.libs.json.Reads._
val jsonStr = """{"user":{"id":"1","name":"Some User","permGroups":[{"id":"group1","name":"Group 1","actions":[{"id":"action1","name":"Action 1"}]},{"id":"group2","name":"Group 2","actions":[{"id":"action3","name":"Action 3"},{"id":"action3","name":"Action 3"}]}]},"title":"New Role","role_id":"56fea66c"}"""
val jsonVal = Json.parse(jsonStr)
val jsonTransformer = (
(__ \ 'name).json.copyFrom((__ \ 'title).json.pick) and
(__ \ 'id).json.copyFrom((__ \ 'role_id).json.pick)
).reduce
jsonVal.transform(jsonTransformer)
this produces:
{
"name": "New Role",
"id" : "56fea66c",
}

Here is one way to do it. This will ignore any perm groups or actions that don't have an id.
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
val jsonStr = """{"user":{"id":"1","name":"Some User","permGroups":[{"id":"group1","name":"Group 1","actions":[{"id":"action1","name":"Action 1"}]},{"id":"group2","name":"Group 2","actions":[{"id":"action3","name":"Action 3"},{"id":"action3","name":"Action 3"}]}]},"title":"New Role","role_id":"56fea66c"}"""
val jsonTransformer = (
(__ \ 'name).json.copyFrom((__ \ 'user \ 'name).json.pick) and
(__ \ 'id).json.copyFrom((__ \ 'role_id).json.pick) and
(__ \ 'permGroupIds).json.copyFrom((__ \ 'user \ 'permGroups).read[List[JsObject]].map(permGroups =>
JsArray(permGroups.flatMap(permGroup => (permGroup \ "id").toOption))
)) and
(__ \ 'actions).json.copyFrom((__ \ 'user \ 'permGroups).read[List[JsObject]].map(permGroups =>
JsArray(permGroups.flatMap(permGroup => (permGroup \ "actions").asOpt[JsArray]).flatMap(_ \\ "id"))
))
).reduce
val transformed = Json.parse(jsonStr).transform(jsonTransformer)
Json.prettyPrint(transformed.get)
This produces
{
"name" : "Some User",
"id" : "56fea66c",
"permGroupIds" : [ "group1", "group2" ],
"actions" : [ "action1", "action3", "action3" ]
}

Related

How to update nested JSON array in scala

I'm a newbie to scala/play and I've been trying to update (add a new id to) the query array in the sections[1] of the JSON but I've had no success traversing the JSON as I have little knowledge of transformers and how to use it.
"definitions": [
{
"sections": [
{
"priority": 1,
"content": {
"title": "Driver",
"links": [
{
"url": "https://blabla.com",
"text": "See all"
}
]
},
"SearchQuery": {
"options": {
"aggregate": true,
"size": 20,
},
"query": "{\"id\":{\"include\":[\"0wxZ4Nr2\", \"0wxZbNr2\", \"6WZOPMw1\"}}"
}
},
{
"priority": 2,
"content": {
"title": "Deliver",
"links": [
{
"url": "https://blabla.com",
"text": "See all"
}
]
},
"SearchQuery": {
"options": {
"aggregate": true,
"size": 20,
},
"query": "{\"id\":{\"include\":[\"2W12Q2wq\", \"Nwq09lW3\", \"QweNN2d9\"]}}"
}
}
]
}
Any suggestions on how I can achieve this. My goal is to put values inside the specific fields of JSON array. I am using play JSON library throughout my application?
As you noticed if you use PlayJSON you can use Json Transformers
Updating a field would work like this:
val queryUpdater = (__ \ "definitions" \ 1 \ "SearchQuery" \ "query").json.update(
of[JsString].map {
case JsString(value) =>
val newValue: String = ... // calculate new value
JsString(newValue)
}
)
json.transform(queryUpdater)
If you needed to update all queries it would be more like:
val updateQuery = (__ \ "SearchQuery" \ "query").json.update(
of[JsString].map {
case JsString(value) =>
val newValue: String = ... // calculate new value
JsString(newValue)
}
)
val updateQueries = (__ \ "definitions").json.update(
of[JsArray].map {
case JsArray(arr) =>
JsArray(arr.map(_.transform(updateQuery)))
}
)
json.transform(updateQueries)

Scala Sorting with Another Json's Key

I am new to Scala and please help me out.
I have 2 Json file. I want to sort first json with a key from the second json.
For eg:
First Json
{
"id": 1,
"response" : [{
"user_id" : 1,
"products" : [
{
"product_id": 10,
"price": 200
},
{
"product_id": 13,
"price": 210
},
{
"product_id": 9,
"price": 320
}
]
},{
"user_id" : 2,
"products" : [
{
"product_id": 15,
"price": 200
},
{
"product_id": 13,
"price": 210
},
{
"product_id": 8,
"price": 320
}
]
}
]
}
And My Second Json
{
"sort": [
{
"user_id": 1,
"products": [
{
"id": 8,
"rank": 5
},
{
"id": 9,
"rank": 1
},
{
"id": 10,
"rank": 3
},
{
"id": 13,
"rank": 2
},{
"id": 15,
"rank": 6
},{
"id": 17,
"rank": 4
},{
"id": 20,
"rank": 7
},{
"id": 21,
"rank": 8
},{
"id": 23,
"rank": 9
}
]
},{
"user_id": 2,
"products": [
{
"id": 8,
"rank": 5
},
{
"id": 9,
"rank": 1
},
{
"id": 10,
"rank": 3
},
{
"id": 13,
"rank": 2
},{
"id": 15,
"rank": 6
},{
"id": 17,
"rank": 4
},{
"id": 20,
"rank": 7
},{
"id": 21,
"rank": 8
},{
"id": 23,
"rank": 9
}
]
}
]
}
I want to sort my first json with respect to the rank I have in second Json.
Output should be like each user should have his products in sorted order based on the rank that is specified for each user on the second JSON.
This is so far what I have tried
def sortedRes() = Action.async {
val url = //1st json url
val sortUrl = //2nd json url
ws.url(url).get().map { response =>
val value: JsValue = Json.parse(response.body)
val result: Either[Exception, SampleResponses] = value.validate[SampleResponses] match {
case JsSuccess(searchResponse, _) =>
Right(searchResponse)
case JsError(errors) =>
Left(new Exception("Couldn't parse Search API response"))
}
val values: List[SampleResponse] = result.right.get.responses
ws.url(sortUrl).get().map { response =>
val sorted: JsValue = Json.parse(response.body)
val sortRespResult: Either[Exception, Sort] = sorted.validate[Sort] match {
case JsSuccess(sortResponse, _) =>
Right(sortResponse)
case JsError(errors) =>
Left(new Exception("Couldn't parse because of these errors : " + errors))
}
val prodRankDetails: List[SampleRank] = sortRespResult.right.get.sort
println("prod = " + prodRankDetails.head.products.sortWith(_.rank > _.rank))
}
Ok(Json.toJson(result.right.get))
}
}
In the last print statement I got the second json's first users product sorted. What I am trying to get is my first json sorted based on the second user.
Here is my model class
package models
import play.api.libs.functional.syntax._
import play.api.libs.json.Reads._
import play.api.libs.json._ // Combinator syntax
object sample {
case class SampleProduct(productId:Int, price: Int)
case class SampleResponse(userId: Int, products: List[SampleProduct])
case class SampleResponses(id: Int, responses: List[SampleResponse])
case class SampleRankedProduct(id: Int, rank: Int)
case class SampleRank(userId: Int, products: List[SampleRankedProduct])
case class Sort(sort: List[SampleRank])
implicit val productReads: Reads[SampleProduct] = (
(JsPath \ "product_id").read[Int] and
(JsPath \ "price").read[Int]
)(SampleProduct.apply _)
implicit val productWrites: Writes[SampleProduct] = (
(JsPath \ "product_id").write[Int] and
(JsPath \ "price ").write[Int]
)(unlift(SampleProduct.unapply))
implicit val responseReads: Reads[SampleResponse] = (
(JsPath \ "user_id").read[Int] and
(JsPath \ "products").read[List[SampleProduct]]
)(SampleResponse.apply _)
implicit val responseWrites: Writes[SampleResponse] = (
(JsPath \ "user_id").write[Int] and
(JsPath \ "products").write[List[SampleProduct]]
)(unlift(SampleResponse.unapply))
implicit val responsesReads: Reads[SampleResponses] = (
(JsPath \ "id").read[Int] and
(JsPath \ "response").read[List[SampleResponse]]
)(SampleResponses.apply _)
implicit val responsesWrites: Writes[SampleResponses] = (
(JsPath \ "id").write[Int] and
(JsPath \ "response").write[List[SampleResponse]]
)(unlift(SampleResponses.unapply))
implicit val rankedProductReads: Reads[SampleRankedProduct] = (
(JsPath \ "id").read[Int] and
(JsPath \ "rank").read[Int]
)(SampleRankedProduct.apply _)
implicit val rankedProductWrites: Writes[SampleRankedProduct] = (
(JsPath \ "id").write[Int] and
(JsPath \ "rank ").write[Int]
)(unlift(SampleRankedProduct.unapply))
implicit val rankReads: Reads[SampleRank] = (
(JsPath \ "user_id").read[Int] and
(JsPath \ "products").read[List[SampleRankedProduct]]
)(SampleRank.apply _)
implicit val rankWrites: Writes[SampleRank] = (
(JsPath \ "user_id").write[Int] and
(JsPath \ "products").write[List[SampleRankedProduct]]
)(unlift(SampleRank.unapply))
implicit val sortReads: Reads[Sort] = (JsPath \ "sort").read[List[SampleRank]].map(x ⇒ Sort(x))
implicit val sortWrites: Writes[Sort] = (__ \ "sort").write[List[SampleRank]].contramap { (person: Sort) => person.sort }
}
An approach for sorting the first JSON using the ranking from the second one, is something like that:
....
val prodRankDetails: List[SampleRank] = sortRespResult.right.get.sort
val sortedResults = values.copy(
responses = values.responses.map { resultForUser =>
val rankingForUser = prodRankDetails
.find(_.userId == resultForUser.userId)
.getOrElse(throw new RuntimeException(s"Rank not found for user ${resultForUser.userId}"))
.products
.map(product => (product.id, product.rank))
.toMap
resultForUser.copy(
products = resultForUser.products.sortWith((a, b) =>
rankingForUser.getOrElse(a.productId, Int.MaxValue) < rankingForUser.getOrElse(b.productId, Int.MaxValue))
)
}
)
Ok(Json.toJson(sortedResults))
that should be performed just after you got prodRankDetails. And also the response will be available inside the .map as you are managing a Future.
So you have to move also the Ok(...) inside that map and to flatMap the outer Future.
Hope this helps.

Scala how to read json path by array

My main aim is to make a transformation i.e from "data/message" to "posts/content".
What's wrong with my code? I can't even access the value of "message" in an array.
<code>
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
def mapping(): JsObject = {
val sjson =
"""
{
"data": [
{"message": "A", "created_time": "a" },
{"message": "B", "created_time": "b" }
],
"page": "test"
}
""".stripMargin
val json = Json.parse(sjson)
val jsonReads =
(__ \ 'data \\ 'message ).read[String]
json.validate(jsonReads).map {
case( message ) =>
Json.obj("content" -> message)
}.get
}
</code>
In response to your comment on this post, here's my updated answer:
val sjson =
"""
{
"data": [
{"message": "A", "created_time": "a" },
{"message": "B", "created_time": "b" }
],
"page": "test"
}
""".stripMargin
// Read the whole object
val json: JsObject = Json.parse(sjson).as[JsObject]
// Read the "data array"
val data: Seq[JsObject] = (json \ "data").as[Seq[JsObject]]
// Get final result my changing "data" to "posts",
// and all instances of "message" to "content" and "created_time" to "created"
val result =
Json.obj(
"posts" -> (data map (d => JsObject(d.value map {
case ("message", m) =>
"content" -> m
case ("created_time", t) =>
"created" -> t
})))
)

How to parse this JSON using Play Framework?

I have the following JSON being returned from a web service:
{
"hits": [
{
"created_at": "2016-02-01T15:01:03.000Z",
"title": "title",
"num_comments": 778,
"parent_id": null,
"_tags": [
"story",
"author",
"story_11012044"
],
"objectID": "11012044",
"_highlightResult": {
"title": {
"value": "title",
"matchLevel": "full",
"matchedWords": [
"title"
]
},
"author": {
"value": "author",
"matchLevel": "none",
"matchedWords": [
]
},
"story_text": {
"value": "Please lead",
"matchLevel": "none",
"matchedWords": [
]
}
}
}
]
}
and I am trying to parse it using the JSON parsing libs in Play Framework. I have the following code:
import play.api.libs.functional.syntax._
import play.api.libs.json._
case class Post(id: Long, date: String, count: Int)
object Post {
implicit val postFormat = Json.format[Post]
implicit val writes: Writes[Post] = (
(JsPath \ "id").write[Long] and
(JsPath \"date").write[String] and
(JsPath \ "count").write[Int]
)(unlift(Post.unapply))
implicit val reads: Reads[Post] = (
(JsPath \ "objectID").read[Long] and
(JsPath \ "created_at").read[String] and
(JsPath \ "num_comments").read[Int]
)(Post.apply _)
}
import play.api.libs.json._
class PostRepo {
val request: WSRequest = ws.url(MY_URL)
def getPosts: Future[Seq[Post]] =
val result: Future[JsValue] = request.get().map(response =>
response.status match {
case 200 => Json.parse(response.body)
case _ => throw new Exception("Web service call failed: " + response.body)
})
result.map( {
jsonvalue => println("JSARRAY: " + jsonvalue);
(jsonvalue \ "hits").as[Seq[Post]]
})
result
}
Now, when I run the code, I am getting the following error:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[JsResultException:
JsResultException(errors:List(((0)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((0)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((0)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/id,List(ValidationError(List(error.path.missing),WrappedArray())))
Obviously something is wrong with the way I'm trying to parse the JSON but I have now spent a few hours trying to figure out the problem and I'm well and truly stuck.
Some refactoring code with Reads.seq
val r = (__ \ "hits").read[Seq[Post]](Reads.seq[Post])
def getPosts: Future[Seq[Post]] = {
WS.url(MY_URL).get().map(response =>
response.status match {
case 200 => r.reads(response.json) match {
case JsError(e) => throw new Exception("Json read fails. Response body:" + response.json.toString() + "\nRead error:" + e.toString())
case JsSuccess(x, _) => x
}
case _ => throw new Exception("Web service call failed: " + response.body)
})
}

How to properly validate and transform a json in play scala?

I am trying to parse and transform a json file so that it can fit a DB structure. I have the following json file.
{
"request": {
"Target": "AR",
"Format": "jsonp",
"Service": "HasOff",
"Version": "2",
"NetworkId": "xyz",
"Method": "getConversions",
"api_key": "xyz",
"fields": [
"Offer.name",
"Stat.datetime",
"Stat.conversion_status",
"Stat.approved_payout",
"Stat.ip",
"Stat.ad_id",
"Stat.affiliate_info1",
"Stat.offer_id"
],
"data_start": "2015-04-16 23:59:59",
"data_end": "2015-04-18 00:00:00",
"callback": "angular.callbacks._3",
"_ga": "GA1.2.1142892596.1429262595",
"_gat": "1"
},
"response": {
"status": 1,
"httpStatus": 200,
"data": {
"page": 1,
"current": 100,
"count": 521,
"pageCount": 6,
"data": [
{
"Offer": {
"name": "ABC Company"
},
"Stat": {
"datetime": "2015-04-19 12:09:01",
"conversion_status": "approved",
"approved_payout": "26.94000",
"ip": "111.11.11.1",
"ad_id": "123456",
"affiliate_info1": "123456",
"offer_id": "260"
}
},
{
"Offer": {
"name": "ABC Company"
},
"Stat": {
"datetime": "2015-04-11 01:01:38",
"conversion_status": "approved",
"approved_payout": "44.94000",
"ip": "49.204.222.117",
"ad_id": "123456",
"affiliate_info1": "123456",
"offer_id": "260"
}
},
To process it I wrote a case class, model the class and try to convert the json into the model as given in -- https://www.playframework.com/documentation/2.2.x/ScalaJson. The code snippet is given below. I have reduced the number of parameters in the case class for sake of brevity.
case class ConversionReport(offerName: String, date: DateTime)
implicit val readConversion: Reads[ConversionReport] = (
(__ \ "response" \ "data" \ "data" \ "Offer" \ "name").read[String] and
(__ \ "response" \ "data" \ "data" \ "Stat" \ "datetime").read[DateTime]
) (ConversionReport)
def getConversionReport(){
val conversionUrl: String = "http://some_link"
val holder = WS.url(conversionUrl).get()
val result = Await.result(holder, Duration(10, "second")).json
val conversionResult: JsResult[ConversionReport] = result.validate[ConversionReport]
println ("xxxx The conversion result is " +conversionResult)
}
When I execute the code, I am getting the error --
xxxx The conversion result is JsError(List((/offerName,List(ValidationError(error.path.missing,WrappedArray()))), (/date,List(ValidationError(error.path.missing,WrappedArray())))))
I guess, I am not able to properly model the json file.
Any help is appreciated.
In this case data is an array, so your approach will not work. You will have to change few things.
import play.api.libs.json._
import play.api.libs.json.Reads._
case class ConversionReport(offerName: String, date: DateTime)
implicit val readConversion: Reads[ConversionReport] = (
(__\ "Offer" \ "name").read[String] and
(__ \ "Stat" \ "datetime").read[DateTime]
) (ConversionReport)
case class FullReport( reportList: Seq[ ConversionReport ] )
implicit val readFullReport: Reads[ FullReport ] = (
(__ \ "response" \ "data" \ "data").lazyRead( Reads.seq[ ConversionReport ]( readConversion ) )
) ( FullReport )
def getConversionReport(){
val conversionUrl: String = "http://some_link"
val holder = WS.url(conversionUrl).get()
val result = Await.result(holder, Duration(10, "second")).json
val conversionResult: JsResult[FullReport] = result.validate[FullReport]
println ("xxxx The conversion result is " +conversionResult)
}