Find the maximum value from JSON data in Scala - json

I am very new to programming in Scala. I am writing a test program to get maximum value from JSON data. I have following code:
import scala.io.Source
import scala.util.parsing.json._
object jsonParsing{
//Id int `json:"id"`
//Price int `json:"price"`
def main(args: Array[String]): Unit = {
val file_name = "jsonData.txt"
val json_string = scala.io.Source.fromFile("jsonData.txt").getLines.mkString
val json_arr = json_string.split(",")
json_arr.foreach {println}
}
}
The json_arr.foreach {println} prints following data:
[{ "id":1
"price":4629}
{ "id":2
"price":7126}
{ "id":3
"price":8862}
{ "id":4
"price":8999}
{ "id":5
"price":1095}]
I am stuck at the part of figuring out how to find the maximum price from such JSON data? That is, in this case the output should be '8999'.

you can try something like this below:
package com.x.x.integration.commons
import collection.immutable.IndexedSeq
import com.google.gson.Gson
import com.google.gson.JsonObject
import com.google.gson.JsonParser
case class wrapperObject(val json_string: Array[MyJsonObject])
case class MyJsonObject(val id:Int ,val price:Int)
object Demo {
val gson = new Gson()
def main(args: Array[String])={
val json_string = scala.io.Source.fromFile("jsonData.txt").getLines.mkString
//val json_string= """{"json_string":[{"id":1,"price":4629},{"id":2,"price":7126},{"id":3,"price":8862},{"id":4,"price":8999},{"id":5,"price":1095}]}"""
val jsonStringAsObject= new JsonParser().parse(json_string).getAsJsonObject
val objectThatYouCanPlayWith:wrapperObject = gson.fromJson(jsonStringAsObject, classOf[wrapperObject])
var maxPrice:Int = 0
for(i <- objectThatYouCanPlayWith.json_string if i.price>maxPrice)
{
maxPrice= i.price
}
println(maxPrice)
}
}
check if it helps you

I also recommend to use Json4s or playJson.
But you could do without any libraries as such.
val json = """[{"id":1,"price":100},{"id":2, "price": 200}]"""
val priceRegex = """"price"\s*:\s*(\d+)""".r
val maxPrice = priceRegex.findAllIn(json).map({
case priceRegex(price) => price.toInt
}).max
println(maxPrice) // print 200

Although Play JSON is handy, you could use Regex as well.
import scala.io.Source
import scala.util.matching.Regex._
val jsonString = Source
.fromFile("jsonData.txt")
.getLines.mkString.split(",")
var maxPrice = 0
jsonString.foreach(each => {
val price: Option[Match] = ("\"price\":(\\d+)").r.findFirstMatchIn(each)
if (price.isDefined) {
maxPrice = Math.max(maxPrice, price.get.group(1).toInt)
}
})

Related

How I can sort list of JObject (Scala, json4s)

I read a JSON file and use json4s to parse JSON.
How I can sort an array of JObject that I read from the file?
package org.example
import org.json4s._
import org.json4s.native.JsonMethods._
object Main extends App {
val file = scala.io.Source.fromURL("https://raw.githubusercontent.com/mledoze/countries/master/countries.json")
val text = file.mkString
val json = parse(text)
val countries = json.children
countries.sortBy(...)
}
Each country has a field "area" and I need to sort the list by this field.
I don't know how to use sortBy in this situation because the country is not just an object of some class. This is JObject and the field "area" has a type JInt.
Can you help me? Or maybe do you know other solution?
Thanks, Gaƫl J for Hint.
I solved my problem as follows:
package org.example
import org.json4s._
import org.json4s.native.JsonMethods._
object Main extends App {
val file = scala.io.Source.fromURL("https://raw.githubusercontent.com/mledoze/countries/master/countries.json")
val text = file.mkString
val json = parse(text)
val countries = json.children
val countriesSorted = countries.sortWith((x: JValue, y: JValue) => {
//File has a few countries with area that has type of Double
def getValue(value: JValue): Double = {
value \ "area" match {
case JDouble(value) => value
case JInt(value) => value.toDouble
case _ => 0
}
}
val xValue = getValue(x)
val yValue = getValue(y)
xValue > yValue
})
for (country <- countriesSorted)
println(country \ "area")
}

Compare key and values in json string using scala

Want to compare the first json string with the other 2 json string.
First the keys should match . If they match , then compare the nested key and values.
val of1 = "{\"keyA\":{\"1\":13,\"0\":202}}"
val of2 = "{\"keyA\":{\"1\":12,\"0\":201}}"
val of3 = "{\"keyB\":{\"1\":12}}"
Should throw Error for key mismatch.
val of1 = "{\"keyA\":{\"1\":13,\"0\":202}}"
val of2 = "{\"keyA\":{\"1\":12,\"0\":201}}"
val of2 = "{\"keyA\":{\"1\":11,\"0\":200}}"
This should return true, as both keys match and also sub keys 1 and 0 have more values than sub key of json 2 and json 3.The numbers are Long values.
Please help.
Below is my try.
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
val of1 = "{\"keyA\":{\"1\":13,\"0\":202}}"
val of2 = "{\"keyA\":{\"1\":12,\"0\":201}}"
val of3 = "{\"keyB\":{\"1\":12}}"
def OffsetComparator(json1: String, json2: String, json3:String): Boolean = {
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
val jsonObj1 = mapper.readValue(json1, classOf[Map[String, Map[String, Long]]])
val jsonObj2 = mapper.readValue(json2, classOf[Map[String, Map[String, Long]]])
val jsonObj3 = mapper.readValue(json3, classOf[Map[String, Map[String, Long]]])
//Trying to get the key and compare first
val mapA = jsonObj1.keySet.foreach(i=>jsonObj1.keySet(i).toString)
val mapB = jsonObj2.keySet
val mapC = jsonObj3.keySet
println( (jsonObj1.keySet == jsonObj3.keySet) )
if (mapA.keySet != mapB.keySet || mapA.keySet != mapC.keySet) throw new Exception("partitions mismatch")
mapA.keys.forall(k => (mapA(k).asInstanceOf[Long] > mapB(k).asInstanceOf[Long] && mapA(k).asInstanceOf[Long] > mapC(k).asInstanceOf[Long]))
// getting error :java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long when i am casting as Long.Not su
}
println(OffsetComparator(of1, of2,of3))
}
You can try with https://github.com/gnieh/diffson. ITs available for circe, spray-json and play-json.
Your example with Circe:
import diffson._
import diffson.lcs._
import diffson.jsonpatch.lcsdiff._
import io.circe._
import diffson.circe._
import diffson.jsonpatch._
import io.circe.parser._
val decoder = Decoder[JsonPatch[Json]]
val encoder = Encoder[JsonPatch[Json]]
implicit val lcs = new Patience[Json]
val json1 = parse(of1)
val json2 = parse(of2)
val patch =
for {
json1 <- json1
json2 <- json2
} yield diff(json1, json2)
print(patch)
That gives:
Right(JsonPatch(List(Replace(Chain(Left(keyA), Left(0)),201,None), Replace(Chain(Left(keyA), Left(1)),12,None))))
take a look to see how it works https://index.scala-lang.org/gnieh/diffson/diffson-circe/4.0.3?target=_2.13
For Circe, indlude the dependence:
"org.gnieh" %% f"diffson-circe" % "4.0.3"

How to edit existing JSON object with sprayJSON

I am using akka with spray json support for which I need to edit value in the recieved json.
import akka.http.scaladsl.server.Directives
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import spray.json._
final case class Item(name: String, id: Long)
final case class Order(items: List[Item],orderTag:String)
trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val itemFormat = jsonFormat2(Item)
implicit val orderFormat = jsonFormat2(Order)
}
In my use case I recieve the json with orderTag value as null, all I need to do is edit the orderTag value with and then use it as entity value.Is it possible to write/edit jsonObject and How to do that ?
class MyJsonService extends Directives with JsonSupport {
// format: OFF
val route =
get {
pathSingleSlash {
complete(Item("thing", 42)) // will render as JSON
}
} ~
post {
entity(as[Order]) { order => // will unmarshal JSON to Order
val itemsCount = order.items.size
val itemNames = order.items.map(_.name).mkString(", ")
complete(s"Ordered $itemsCount items: $itemNames")
}
}
}
You can just edit the json AST like ..
val json = """{"orderTag":null}"""
val jsVal = json.parseJson
val updatedJs = if (jsObj.fields.get("orderTag") == Some(JsNull)) {
JsObject(jsObj.fields + ("orderTag" -> JsString("new tag")))
} else {
jsObj
}
updatedJs.compactPrint
res26: String = """
{"orderTag":"new tag"}
"""

How to implement implicit Json Writes of Future object in Play Framework 2.x

I am new with play framework and i want to ask periodically to amazon about some products in order to insert them into a kafka topic, an error happens when i try to compile the code.
This is the code of the KafkaProducer:
file example.model.AmazonProducerExample
//UPDATED method with the suggestions from the users, thank you guys!
package example.utils
import jodd.lagarto.dom.{NodeSelector, LagartoDOMBuilder}
import example.model.AmazonProduct
import scala.collection.JavaConversions._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import play.api.libs.json._
import example.utils._
import example.producer._
object AmazonPageParser {
private val topicName = "amazonRatingsTopic"
private val producer = Producer[String](topicName)
def parse(productId: String): Future[AmazonProduct] = {
val url = s"http://www.amazon.com/dp/$productId"
HttpClient.fetchUrl(url) map {
httpResponse =>
if (httpResponse.getStatusCode == 200) {
val body = httpResponse.getResponseBody
val domBuilder = new LagartoDOMBuilder()
val doc = domBuilder.parse(body)
val responseUrl = httpResponse.getUri.toString
val nodeSelector = new NodeSelector(doc)
val title = nodeSelector.select("span#productTitle").head.getTextContent
val img = nodeSelector.select("div#main-image-container img").head.getAttribute("src")
val description = nodeSelector.select("div#feature-bullets").headOption.map(_.getHtml).mkString
val amazonProduct = AmazonProduct(productId, title, responseUrl, img, description)
println("amazonProduct is " + amazonProduct.toString)
amazonProduct
} else {
println("An error happened!")
throw new RuntimeException(s"Invalid url $url")
}
}//map
}//parse method
def main(args: Array[String]): Unit = {
//Scala Puzzlers...
AmazonPageParser.parse("0981531679").onSuccess { case amazonProduct =>
implicit val amazonFormat = Json.format[AmazonProduct]
producer.send(Json.toJson(amazonProduct).toString)
println("amazon product sent to kafka cluster..." + amazonProduct.toString)
}
}
}
file example.model.Models
package example.model
import play.api.libs.json.Json
import reactivemongo.bson.Macros
case class AmazonProduct(itemId: String, title: String, url: String, img: String, description: String)
case class AmazonRating(userId: String, productId: String, rating: Double)
case class AmazonProductAndRating(product: AmazonProduct, rating: AmazonRating)
// For MongoDB
object AmazonRating {
implicit val amazonRatingHandler = Macros.handler[AmazonRating]
implicit val amazonRatingFormat = Json.format[AmazonRating]
}
file example.utils.AmazonPageParser
The compiler returns me this error:
[error] /Users/aironman/my-recommendation-spark-engine/src/main/scala/example/producer/AmazonProducerExample.scala:25: No Json serializer found for type scala.concurrent.Future[example.model.AmazonProduct]. Try to implement an implicit Writes or Format for this type.
[error] producer.send(Json.toJson(amazonProduct).toString)
[error] ^
I have readed this post with most votes but it does not work for me.
Can anybody help me?
Writes[T] produces Json. You can't produce it directly from Future without blocking.
However you can add "callback" to this future, like this:
amazonPageParser.parse(productId).onSuccess { case amazonProduct =>
producer.send(Json.toJson(amazonProduct).toString)
}
Or with other Future methods, like map or foreach.

How to parse json with arbitrary schema, update/create one field and write it back as json(Scala)

how do one update/create field in JSON object with arbitrary schema and write it back as JSON in Scala?
I tried with spray-json with something like that:
import spray.json._
import DefaultJsonProtocol._
val jsonAst = """{"anyfield":"1234", "sought_optional_field":5.0}""".parse
val newValue = jsonAst.asJsObject.fields.getOrElse("sought_optional_field", 1)
val newMap = jsonAst.asJsObject.fields + ("sought_optional_field" -> newValue)
JSONObject(newMap).toJson
but it gives weird result: "{"anyfield"[ : "1234", "sought_optional_field" : ]1}
You were almost there :
import spray.json._
import DefaultJsonProtocol._
def changeField(json: String) = {
val jsonAst = JsonParser(json)
val map = jsonAst.asJsObject.fields
val sought = map.getOrElse("sought_optional_field", 1.toJson)
map.updated("sought_optional_field", sought).toJson
}
val jsonA = """{"anyfield":"1234", "sought_optional_field":5.0}"""
val jsonB = """{"anyfield":"1234"}"""
changeField(jsonA)
// spray.json.JsValue = {"anyfield":"1234","sought_optional_field":5.0}
changeField(jsonB)
// spray.json.JsValue = {"anyfield":"1234","sought_optional_field":1}
Using Argonaut:
import argonaut._, Argonaut._
def changeField2(json: String) =
json.parseOption.map( parsed =>
parsed.withObject(o =>
o + ("sought_optional_field", o("sought_optional_field").getOrElse(jNumber(1)))
)
)
changeField2(jsonA).map(_.nospaces)
// Option[String] = Some({"anyfield":"1234","sought_optional_field":5})
changeField2(jsonB).map(_.nospaces)
// Option[String] = Some({"anyfield":"1234","sought_optional_field":1})