How to update nested JSON array in scala - json

I'm a newbie to scala/play and I've been trying to update (add a new id to) the query array in the sections[1] of the JSON but I've had no success traversing the JSON as I have little knowledge of transformers and how to use it.
"definitions": [
{
"sections": [
{
"priority": 1,
"content": {
"title": "Driver",
"links": [
{
"url": "https://blabla.com",
"text": "See all"
}
]
},
"SearchQuery": {
"options": {
"aggregate": true,
"size": 20,
},
"query": "{\"id\":{\"include\":[\"0wxZ4Nr2\", \"0wxZbNr2\", \"6WZOPMw1\"}}"
}
},
{
"priority": 2,
"content": {
"title": "Deliver",
"links": [
{
"url": "https://blabla.com",
"text": "See all"
}
]
},
"SearchQuery": {
"options": {
"aggregate": true,
"size": 20,
},
"query": "{\"id\":{\"include\":[\"2W12Q2wq\", \"Nwq09lW3\", \"QweNN2d9\"]}}"
}
}
]
}
Any suggestions on how I can achieve this. My goal is to put values inside the specific fields of JSON array. I am using play JSON library throughout my application?

As you noticed if you use PlayJSON you can use Json Transformers
Updating a field would work like this:
val queryUpdater = (__ \ "definitions" \ 1 \ "SearchQuery" \ "query").json.update(
of[JsString].map {
case JsString(value) =>
val newValue: String = ... // calculate new value
JsString(newValue)
}
)
json.transform(queryUpdater)
If you needed to update all queries it would be more like:
val updateQuery = (__ \ "SearchQuery" \ "query").json.update(
of[JsString].map {
case JsString(value) =>
val newValue: String = ... // calculate new value
JsString(newValue)
}
)
val updateQueries = (__ \ "definitions").json.update(
of[JsArray].map {
case JsArray(arr) =>
JsArray(arr.map(_.transform(updateQuery)))
}
)
json.transform(updateQueries)

Related

How to get the All index values in Groovy JSON xpath

Please find the attached Groovy code which I am using to get the particular filed from the response body.
Query 1 :
It is retrieving the results when the I am using the correct Index value like if the data.RenewalDetails[o], will give output as Value 1 and if the data.RenewalDetails[1], output as Value 2.
But in my real case, I will never know about number of blocks in the response, so I want to get all the values that are satisficing the condition, I tried data.RenewalDetails[*] but it is not working. Can you please help ?
Query 2:
Apart from the above condition, I want to add one more filter, where "FamilyCode": "PREMIUM" in the Itemdetails, Can you help on the same ?
def BoundId = new groovy.json.JsonSlurper().parseText('{"data":{"RenewalDetails":[{"ExpiryDetails":{"duration":"xxxxx","destination":"LHR","from":"AUH","value":2,"segments":[{"valudeid":"xxx-xx6262-xxxyyy-1111-11-11-1111"}]},"Itemdetails":[{"BoundId":"Value1","isexpired":true,"FamilyCode":"PREMIUM","availabilityDetails":[{"travelID":"AAA-AB1234-AAABBB-2022-11-10-1111","quota":"X","scale":"XXX","class":"X"}]}]},{"ExpiryDetails":{"duration":"xxxxx","destination":"LHR","from":"AUH","value":2,"segments":[{"valudeid":"xxx-xx6262-xxxyyy-1111-11-11-1111"}]},"Itemdetails":[{"BoundId":"Value2","isexpired":true,"FamilyCode":"PREMIUM","availabilityDetails":[{"travelID":"AAA-AB1234-AAABBB-2022-11-10-1111","quota":"X","scale":"XXX","class":"X"}]}]}]},"warnings":[{"code":"xxxx","detail":"xxxxxxxx","title":"xxxxxxxx"}]}')
.data.RenewalDetails[0].Itemdetails.find { itemDetail ->
itemDetail.availabilityDetails[0].travelID.length() == 33
}?.BoundId
println "Hello " + BoundId
Something like this:
def txt = '''\
{
"data": {
"RenewalDetails": [
{
"ExpiryDetails": {
"duration": "xxxxx",
"destination": "LHR",
"from": "AUH",
"value": 2,
"segments": [
{
"valudeid": "xxx-xx6262-xxxyyy-1111-11-11-1111"
}
]
},
"Itemdetails": [
{
"BoundId": "Value1",
"isexpired": true,
"FamilyCode": "PREMIUM",
"availabilityDetails": [
{
"travelID": "AAA-AB1234-AAABBB-2022-11-10-1111",
"quota": "X",
"scale": "XXX",
"class": "X"
}
]
}
]
},
{
"ExpiryDetails": {
"duration": "xxxxx",
"destination": "LHR",
"from": "AUH",
"value": 2,
"segments": [
{
"valudeid": "xxx-xx6262-xxxyyy-1111-11-11-1111"
}
]
},
"Itemdetails": [
{
"BoundId": "Value2",
"isexpired": true,
"FamilyCode": "PREMIUM",
"availabilityDetails": [
{
"travelID": "AAA-AB1234-AAABBB-2022-11-10-1111",
"quota": "X",
"scale": "XXX",
"class": "X"
}
]
}
]
}
]
},
"warnings": [
{
"code": "xxxx",
"detail": "xxxxxxxx",
"title": "xxxxxxxx"
}
]
}'''
def json = new groovy.json.JsonSlurper().parseText txt
List<String> BoundIds = json.data.RenewalDetails.Itemdetails*.find { itemDetail ->
itemDetail.availabilityDetails[0].travelID.size() == 33 && itemDetail.FamilyCode == 'PREMIUM'
}?.BoundId
assert BoundIds.toString() == '[Value1, Value2]'
Note, that you will get the BoundIds as a List
If you amend your code like this:
def json = new groovy.json.JsonSlurper().parse(prev.getResponseData()
you would be able to access the number of returned items as:
def size = json.data.RenewalDetails.size()
as RenewalDetails represents a List
Just add as many queries you want using Groovy's && operator:
find { itemDetail ->
itemDetail.availabilityDetails[0].travelID.length() == 33 &&
itemDetail.FamilyCode.equals('PREMIUM')
}
More information:
Apache Groovy - Parsing and producing JSON
Apache Groovy: What Is Groovy Used For?

Scala Json parsing

This is the input json which I am getting which is nested json structure and I don't want to map directly to class, need custom parsing of some the objects as I have made the case classes
{
"uuid": "b547e13e-b32d-11ec-b909-0242ac120002",
"log": {
"Response": {
"info": {
"receivedTime": "2022-02-09T00:30:00Z",
"isSecure": "Yes",
"Data": [{
"id": "75641",
"type": "vendor",
"sourceId": "3",
"size": 53
}],
"Group": [{
"isActive": "yes",
"metadata": {
"owner": "owner1",
"compressionType": "gz",
"comments": "someComment",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "1"
},
{
"isActive": "yes",
"metadata": {
"owner": "owner12",
"compressionType": "snappy",
"comments": "someComment",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "2"
},
{
"isActive": "yes",
"metadata": {
"owner": "owner123",
"compressionType": "snappy",
"comments": "someComment",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "4"
},
{
"isActive": "yes",
"metadata": {
"owner": "owner124",
"compressionType": "snappy",
"comments": "someComments",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "4"
}
]
}
}
}
}
Code that I am trying play json also tried circe . Please help ..New to scala world
below is object and case class
case class DataCatalog(uuid: String, data: Seq[Data], metadata: Seq[Metadata])
object DataCatalog {
case class Data(id: String,
type: String,
sourceId: Option[Int],
size: Int)
case class Metadata(isActive: String,
owner: String,
compressionType: String,
comments: String,
createdDate: String,
updatedDate: String
)
def convertJson(inputjsonLine: String): Option[DataCatalog] = {
val result = Try {
//val doc: Json = parse(line).getOrElse(Json.Null)
//val cursor: HCursor = doc.hcursor
//val uuid: Decoder.Result[String] = cursor.downField("uuid").as[String]
val lat = (inputjsonLine \ "uuid").get
DataCatalog(uuid, data, group)
}
//do pattern matching
result match {
case Success(dataCatalog) => Some(dataCatalog)
case Failure(exception) =>
}
}
}
Any parsing api is fine.
If you use Scala Play, for each case class you should have an companion object which will help you a lot with read/write object in/from json:
object Data {
import play.api.libs.json._
implicit val read = Json.reads[Data ]
implicit val write = Json.writes[Data ]
def tupled = (Data.apply _).tupled
}
object Metadata {
import play.api.libs.json._
implicit val read = Json.reads[Metadata ]
implicit val write = Json.writes[Metadata ]
def tupled = (Metadata.apply _).tupled
}
Is required as each companion object to be in same file as the case class. For your json example, you need more case classes because you have a lot of nested objects there (log, Response, info, each of it)
or, you can read the field which you're interested in as:
(jsonData \ "fieldName").as[CaseClassName]
You can try to access the Data value as:
(jsonData \ "log" \ "Response" \ "info" \ "Data").as[Data]
same for Metadata

Filtering and updating a JSON in Scala

I have the following JSON
Json
{
"components": {
"id": "application",
"actions": [{
"action": "/show-dashboard",
"text": {
"en": "Dashboard",
"es": "Dashboard-ES"
}
}, {
"action": "/showContact",
"text": {
"en": "Personal",
"es": "Personal-ES"
}
}, {
"action": "/showSummary",
"text": {
"en": "Summary",
"es": "Summary-ES"
}
}]
}
}
And I need to filter the actions. Only two are possible (this json is smaller the original have more than 20 actions)
So I have a JsObject called components with this json. Then I used this
Scala
val filteredActions = (components \\ "actions").head.as[List[JsValue]].filter{ _.\("action").as[String] match {
case "/showContact" => true
case "/showSummary" => true
case _ => false
}}
So I have in filteredActions a json string with the actions filtered, now I need to update components and replace the current actions with this one.
I have try this:
Scala
val jsonTransformer2 = (__ \ 'components \ 'actions).json.update((__ \ 'components).json.put(Json.arr(filteredActions)))
val filtered = components.transform(jsonTransformer2)
But is not working..
Any advice please.
Thanks in advance
what results do you expect ?
if you just want to remove action /show-dashboard in actions
val jsonTransformer2 = (__ \ 'components \ 'actions).json.put(JsArray(filteredActions))
val filtered = components.transform(jsonTransformer2)
println(filtered.get)
// result
{
"components": {
"actions": [
{
"action": "/showContact",
"text": {
"en": "Personal",
"es": "Personal-ES"
}
},
{
"action": "/showSummary",
"text": {
"en": "Summary",
"es": "Summary-ES"
}
}
]
}
}
Thanks for the help. It really helped me to get a better understand of how this works.
I found the solution.
Just need it to change the transformer to this:
Scala
val jsonTransformer2 = (__ \ 'components \ 'actions).json.update((__ \ 'components \ 'actions).json.put(JsArray(filteredActions)))
Again thanks to everyone.

Put Data in mutlple branch of Array : Json Transformer ,Scala Play

i want to add values to all the arrays in json object.
For eg:
value array [4,2.5,2.5,1.5]
json =
{
"items": [
{
"id": 1,
"name": "one",
"price": {}
},
{
"id": 2,
"name": "two"
},
{
"id": 3,
"name": "three",
"price": {}
},
{
"id": 4,
"name": "four",
"price": {
"value": 1.5
}
}
]
}
i want to transform the above json in
{
"items": [
{
"id": 1,
"name": "one",
"price": {
"value": 4
}
},
{
"id": 2,
"name": "two",
"price": {
"value": 2.5
}
},
{
"id": 3,
"name": "three",
"price": {
"value": 2.5
}
},
{
"id": 4,
"name": "four",
"price": {
"value": 1.5
}
}
]
}
Any suggestions on how do i achieve this. My goal is to put values inside the specific fields of json array. I am using play json library throughout my application. What other options do i have instead of using json transformers.
You may use simple transformation like
val prices = List[Double](4,2.5,2.5,1.5).map(price => Json.obj("price" -> Json.obj("value" -> price)))
val t = (__ \ "items").json.update(
of[List[JsObject]]
.map(_.zip(prices).map(o => _._1 ++ _._2))
.map(JsArray)
)
res5: play.api.libs.json.JsResult[play.api.libs.json.JsObject] = JsSuccess({"items":[{"id":1,"name":"one","price":{"value":4}},{"id":2,"name":"two","price":{"value":2.5}},{"id":3,"name":"three","price":{"value":2.5}},{"id":4,"name":"four","price":{"value":1.5}}]},/items)
I suggest using classes, but not sure this fits to your project because it's hard to guess how your whole codes look like.
I put new Item manually for simplicity. You can create items using Json library :)
class Price(val value:Double) {
override def toString = s"{value:${value}}"
}
class Item(val id: Int, val name: String, val price: Price) {
def this(id: Int, name: String) {
this(id, name, null)
}
override def toString = s"{id:${id}, name:${name}, price:${price}}"
}
val price = Array(4, 2.5, 2.5, 1.5)
/** You might convert Json data to List[Item] using Json library instead. */
val items: List[Item] = List(
new Item(1, "one"),
new Item(2, "two"),
new Item(3, "three"),
new Item(4, "four", new Price(1.5))
)
val valueMappedItems = items.zipWithIndex.map{case (item, index) =>
if (item.price == null) {
new Item(item.id, item.name, new Price(price(index)))
} else {
item
}
}

How to properly validate and transform a json in play scala?

I am trying to parse and transform a json file so that it can fit a DB structure. I have the following json file.
{
"request": {
"Target": "AR",
"Format": "jsonp",
"Service": "HasOff",
"Version": "2",
"NetworkId": "xyz",
"Method": "getConversions",
"api_key": "xyz",
"fields": [
"Offer.name",
"Stat.datetime",
"Stat.conversion_status",
"Stat.approved_payout",
"Stat.ip",
"Stat.ad_id",
"Stat.affiliate_info1",
"Stat.offer_id"
],
"data_start": "2015-04-16 23:59:59",
"data_end": "2015-04-18 00:00:00",
"callback": "angular.callbacks._3",
"_ga": "GA1.2.1142892596.1429262595",
"_gat": "1"
},
"response": {
"status": 1,
"httpStatus": 200,
"data": {
"page": 1,
"current": 100,
"count": 521,
"pageCount": 6,
"data": [
{
"Offer": {
"name": "ABC Company"
},
"Stat": {
"datetime": "2015-04-19 12:09:01",
"conversion_status": "approved",
"approved_payout": "26.94000",
"ip": "111.11.11.1",
"ad_id": "123456",
"affiliate_info1": "123456",
"offer_id": "260"
}
},
{
"Offer": {
"name": "ABC Company"
},
"Stat": {
"datetime": "2015-04-11 01:01:38",
"conversion_status": "approved",
"approved_payout": "44.94000",
"ip": "49.204.222.117",
"ad_id": "123456",
"affiliate_info1": "123456",
"offer_id": "260"
}
},
To process it I wrote a case class, model the class and try to convert the json into the model as given in -- https://www.playframework.com/documentation/2.2.x/ScalaJson. The code snippet is given below. I have reduced the number of parameters in the case class for sake of brevity.
case class ConversionReport(offerName: String, date: DateTime)
implicit val readConversion: Reads[ConversionReport] = (
(__ \ "response" \ "data" \ "data" \ "Offer" \ "name").read[String] and
(__ \ "response" \ "data" \ "data" \ "Stat" \ "datetime").read[DateTime]
) (ConversionReport)
def getConversionReport(){
val conversionUrl: String = "http://some_link"
val holder = WS.url(conversionUrl).get()
val result = Await.result(holder, Duration(10, "second")).json
val conversionResult: JsResult[ConversionReport] = result.validate[ConversionReport]
println ("xxxx The conversion result is " +conversionResult)
}
When I execute the code, I am getting the error --
xxxx The conversion result is JsError(List((/offerName,List(ValidationError(error.path.missing,WrappedArray()))), (/date,List(ValidationError(error.path.missing,WrappedArray())))))
I guess, I am not able to properly model the json file.
Any help is appreciated.
In this case data is an array, so your approach will not work. You will have to change few things.
import play.api.libs.json._
import play.api.libs.json.Reads._
case class ConversionReport(offerName: String, date: DateTime)
implicit val readConversion: Reads[ConversionReport] = (
(__\ "Offer" \ "name").read[String] and
(__ \ "Stat" \ "datetime").read[DateTime]
) (ConversionReport)
case class FullReport( reportList: Seq[ ConversionReport ] )
implicit val readFullReport: Reads[ FullReport ] = (
(__ \ "response" \ "data" \ "data").lazyRead( Reads.seq[ ConversionReport ]( readConversion ) )
) ( FullReport )
def getConversionReport(){
val conversionUrl: String = "http://some_link"
val holder = WS.url(conversionUrl).get()
val result = Await.result(holder, Duration(10, "second")).json
val conversionResult: JsResult[FullReport] = result.validate[FullReport]
println ("xxxx The conversion result is " +conversionResult)
}