Restructure a json in scala play - json

The following code gets the request body and validates and creates a json:
object ValidateDBConfigJson {
implicit val reads: Reads[ValidateDetails] = (
(JsPath \ "name").read[String].filter(JsonValidationError("Invalid name"))(_.length > 0) and
(JsPath \ "email").read[String].filter(JsonValidationError("Invalid email"))(_.length > 0) and
)(ValidateDetails.apply _)
}
def index() = Action { implicit request =>
val bodyAsJson = request.body.asJson.get
bodyAsJson.validate[ValidateDetails] match {
case success: JsSuccess[ValidateDetails] => {
Ok(Json.parse("succeeded!"))
}
case JsError(error) =>
BadRequest(JsError.toJson(error))
}
}
The json looks like this:
{
"obj.name": [
{
"msg": [
"error.expected.jsstring"
],
"args": []
}
],
"obj.email": [
{
"msg": [
"Invalid email"
],
"args": []
}
]
}
I want structured in the following format:
{
"ErrorMessages" :
[
"error.expected.jsstring",
"Invalid email"
]
}

Preamble: when I am parsing JSON using Play I prefer to use case class/objects rather than implicit reads, so this answer will cover that method of doing this. There may be a simpler method of doing this with implicit reads but I am not as familiar with implicit reads.
Firstly, define a case class for everything you will be taking from JSON:
object Input {
case class Err(msg: Seq[String], args: Seq[String])
object Err {
implicit val format: OFormat[Err] = Json.format[Err]
}
case class ValidateDetails(`obj.name`: Seq[Err], `obj.email`: Seq[Err])
object ValidateDetails {
implicit val format: OFormat[ValidateDetails] = Json.format[ValidateDetails]
}
}
Note: Play won't know how to handle user-defined case classes so I've made one for Err as well. The implicit val format: OFormat[ValidateDetails] = Json.format[ValidateDetails] and implicit val format: OFormat[Err] = Json.format[Err] lines are magic and do all of the reads/writes for you.
Next, define a case class for your output JSON and define a function which will turn your input case class into your output one:
object Output {
case class OutputJson(`ErrorMessages`: Seq[String])
object OutputJson {
implicit val format: OFormat[OutputJson] = Json.format[OutputJson]
}
// take msg Seq from name & email and add together into single Seq
def inputToOutput(input: Input.ValidateDetails): OutputJson = {
OutputJson(input.`obj.name`.flatMap(_.msg) ++ input.`obj.email`.flatMap(_.msg))
}
}
Finally, put this into a method which maps to a POST route in your routes file:
def index() = Action { implicit request =>
val bodyAsJson = request.body.asJson.get
bodyAsJson.validate[Input.ValidateDetails] match {
case success: JsSuccess[Input.ValidateDetails] =>
// turn the JSON into the Output case class and parse that as JSON
val output: JsValue = Json.toJson(Output.inputToOutput(success.value))
Ok(output)
case JsError(error) =>
BadRequest(JsError.toJson(error))
}
}
Now, if you run the Play app on port 9000 and POST to http://localhost:9000/ with the below JSON body...
{
"obj.name": [
{
"msg": [
"error.expected.jsstring"
],
"args": []
}
],
"obj.email": [
{
"msg": [
"Invalid email"
],
"args": []
}
]
}
...the output will be:
{
"ErrorMessages": [
"error.expected.jsstring",
"Invalid email"
]
}
I hope this answers your question.

Related

Parsing/extracting information from JSON in Scala

I'm trying to extract (timestamp, value) tuples from this third party JSON message:
{
"data": {
"1496845320000": [
0.14
],
"1496845380000": [
0.14
],
"1496845560000": [
0.14
],
"1497013740000": [
"undef"
],
"1497013800000": [
"undef"
],
"1497013860000": [
"undef"
]
},
"status": "ok"
}
This is how I'm extracting the JSON entity:
val batchReadings = http.singleRequest(httpRequest) map {
case HttpResponse(StatusCodes.OK, headers, entity, _) =>
entity.dataBytes.runFold(ByteString(""))(_ ++ _).foreach { body =>
JSON.parseFull(body.utf8String) match {
case Some(e: Map[String, Any]) =>
e.get("status") match {
case Some("ok") => println("ok!")
e.get("data") match {
case Some(f: Map[String, List[Any]]) =>
println("entered second case")
new EfergyReadingsData(f.toList).getReadings
}
case _ => log.info("EfergyReaderWorkerActor: Received unknown message\n" + body.utf8String)
}
}
}
case HttpResponse(code, _, entity, _) =>
//entity.toStrict(100, materializer) N.B. Not having this, will freeze after 4 calls!
s"Response code: $code"
case unknown =>
s"unknown response: $unknown"
}
And this is the EfergyReadingsData class:
class EfergyReadingsData(readings: List[(String, List[Any])]) {
def getReadings: List[(Long, Double)] = {
println(readings)
// List((1123, 34.4))
val filteredReadings = readings filter {
f =>
f._2 match {
case t:List[Any] =>
t.head match {
case Double => true
case _ => false
}
}
}
val theRead = filteredReadings map {
p => (p._1.toLong, p._2.head)
}
println(theRead)
List((1123, 34.4))
}
}
As you can see I'm not returning the List of tuples that I'm supposed to in getReadings, I just created a list to be returned so I could test the code.
Besides not being able to filter the list to remove only the undefentries (the filteredReadings is empty), the first obvious problem with this approach is that I'm pattern matching for types inside List and Map and therefore getting a warning about those being unchecked because they are eliminated by erasure.
Although not represented above for simplicity, I tried to mitigate this by using case classes for the Lists:
sealed abstract class AnyReading {
val value: List[Any]
}
case class SuccessfulReading(override val value: List[Double]) extends AnyReading
case class FailedReading(override val value: List[String]) extends AnyReading
So, to conclude, what is the best way to go about this? I also thought of creating few case classes to represent the the JSON structure and try to deserialise the JSON message and then filter to remove the undef values, and ultimately get the list of tuples I wanted.
Thank you in advance for your help.

JSON validation using Scala

I am trying to write a scala application for JSON validation. I have a Animals.scala class that defines the following:
case class Animals (id: Int, type: String, targets: String)
object Animals {
implicit val reads: Reads[Animals] = (
(JsPath \ "id").read[Int] and
(JsPath \ "type").read[String] and
(JsPath \ "targets").read[String])(Animals.apply _)
}
I have Application.scala where I have tried to validate an incoming JSON against the case class.
object Application extends Controller {
// action for JSON validation
def validateRequest = Action { implicit request =>
// this will fail if the request body is not a valid json value
val bodyAsJson = request.body.asJson.get
bodyAsJson.validate[Animals] match {
case success: JsSuccess[Animals] => {
val id = success.get.id
Ok("Validation passed! id is "+ id)
}
case JsError(error) => BadRequest("Validation failed!")
}
}
}
And finally here's my JSON input:
{
"id" : 1,
"type" : "domestic",
"targets": {
"AND": [
{
"breed": ["greyhound", "dalmatian"]
},
{
"NOT": {
"color": ["amber", "pale_amber", "black"]
}
},
{
"zipcode": ["90210", "90211"]
}
]
}
}
And I get the following error:
JsError(List((/targets,List(ValidationError(error.expected.jsarray,WrappedArray())))))
I do realize that the error is thrown because targets field is not as simple as a String compared to my JSON. How do I wrap it so that the validation passes? Should I do List[List[String]] or something along those lines?
If you don't care about the structure of targets read it as a JsObject. It will parse any internal structure that way.

Scala Pickling doesn't seem to work with Point2D.Double

I'm working on a Scala program that uses the Scala Pickling library to serialize and deserialize a Map object that contains a String and a Point2D.Double object from the java.awt.geom package.
Here's the relevant logic:
contents +=
new Button("Save Config") {
reactions += {
case ButtonClicked(_) => {
var m: Map[String, Point2D.Double] = Map()
nodeFields.foreach(x => {
m += (x._1 -> new Point2D.Double(x._2._1.text.toDouble, x._2._2.text.toDouble))
})
val pkl = m.pickle
fc.showSaveDialog(null)
val outputFile = fc.selectedFile
val writer = new PrintWriter(outputFile)
writer.write(pkl.value)
writer.close()
Dialog.showMessage(null, "Success!")
}
}
}
If you need to see more, here's the commit with the offending logic
As it stands, the JSON formatted string output from pkl.value is a working serialized Map[String, Point2D.Double], except that the values of Point2D.Double are dropped!
Here's a snippet of the output:
{
"$type": "scala.collection.mutable.Map[java.lang.String,java.awt.geom.Point2D.Double]",
"elems": [
{
"$type": "scala.Tuple2[java.lang.String,java.awt.geom.Point2D.Double]",
"_1": "BOTTOMLANE\r",
"_2": {
}
},
{
"$type": "scala.Tuple2[java.lang.String,java.awt.geom.Point2D.Double]",
"_1": "UPPERLANESECOND_0\r",
"_2": {
}
},
{
"$type": "scala.Tuple2[java.lang.String,java.awt.geom.Point2D.Double]",
"_1": "upperSecondTower_1",
"_2": {
}
},
...
]
}
What can I do to fix this?
scala-pickling can not directly pickle/unpickle Point2D.Double because it has no public fields (the x and y values are accessible through the getX and getY getters).
A possible Pickler / Unpickler for Point2D.Double would be :
object Point2DPickler {
import scala.pickling._
import scala.pickling.Defaults._
import java.awt.geom.Point2D
type DoublePoint = java.awt.geom.Point2D.Double
implicit object Point2DDoublePickle extends Pickler[DoublePoint] with Unpickler[DoublePoint] {
private val doubleUnpickler = implicitly[Unpickler[Double]]
override def tag = FastTypeTag[java.awt.geom.Point2D.Double]
override def pickle(point: DoublePoint, builder: PBuilder) = {
builder.beginEntry(point)
builder.putField("x",
b => b.hintTag(FastTypeTag.Double).beginEntry(point.getX).endEntry()
)
builder.putField("y",
b => b.hintTag(FastTypeTag.Double).beginEntry(point.getY).endEntry()
)
builder.endEntry()
}
override def unpickle(tag: String, reader: PReader): DoublePoint = {
val x = doubleUnpickler.unpickleEntry(reader.readField("x")).asInstanceOf[Double]
val y = doubleUnpickler.unpickleEntry(reader.readField("y")).asInstanceOf[Double]
new Point2D.Double(x, y)
}
}
}
Which could be used as :
import scala.pickling.Defaults._
import scala.pickling.json._
import java.awt.geom.Point2D
import Point2DPickler._
val dpoint = new Point2D.Double(1d, 2d)
scala> val json = dpoint.pickle
json: pickling.json.pickleFormat.PickleType =
JSONPickle({
"$type": "java.awt.geom.Point2D.Double",
"x": {
"$type": "scala.Double",
"value": 1.0
},
"y": {
"$type": "scala.Double",
"value": 2.0
}
})
scala> val dpoint2 = json.value.unpickle[java.awt.geom.Point2D.Double]
dpoint2: java.awt.geom.Point2D.Double = Point2D.Double[1.0, 2.0]

How to serialize object type to JSON in Scalatra?

I am a newbie in Scalatra. I have a servlet with JacksonJsonSupport which serves REST endpoint with list of objects.
class OperationsController extends MyappStack with JacksonJsonSupport {
before() {
contentType = formats("json")
}
get("/") {
Data.operations
}
}
The Operation is implemented by either Adding or Removing case classes.
How do I add to the GET / response the specific class to the value? I would like to get as a response:
[
{
"operation": "Adding",
"value": 100
}
]
Instead of
[
{
"value": 100
}
]
Where Adding is a class that extends Operation.
For polymorphic values json4s can add the concrete type as an additional field. This is called a "type hint":
[{
"jsonClass": "Adding",
"value": 10
}, {
"jsonClass": "Adding",
"value": 20
}, {
"jsonClass": "Removing",
"value": 20
}]
This is for example using the ShortTypeHints:
import org.json4s.{ShortTypeHints, DefaultFormats}
import org.scalatra.ScalatraServlet
import org.scalatra.json.JacksonJsonSupport
import org.scalatra.test.specs2.MutableScalatraSpec
sealed trait Operation
case class Adding(value: Int) extends Operation
case class Removing(value: Int) extends Operation
class json1 extends MutableScalatraSpec {
mount(new ScalatraServlet with JacksonJsonSupport {
def typeHints = new ShortTypeHints(List(
classOf[Adding], classOf[Removing]
))
implicit lazy val jsonFormats = DefaultFormats + typeHints
before() {
contentType = formats("json")
}
get("/") {
List(
Adding(10),
Adding(20),
Removing(20)
)
}
}, "/*")
"Should return a list of operations" in {
get("/", headers = Seq("Content-type" -> "application/json")) {
println(body)
status should beEqualTo(200)
}
}
}
I think, that the easiest way is to update your case classes like
case class Adding(value: Int, operation: String = "Adding")
case class Removing (value: Int, operation: String = "Removing")
Another way is to update your jsonFormats with custom serializer, I found example of jsons custom serialization here
json_conversion.scala file we created the trait SimpleMongoDbJsonConversion and we use this in MyScalatraServlet.scala file, see the example below.
json_conversion.scala
package com.example.app
import org.scalatra._
import com.mongodb.casbah.Imports._
trait SimpleMongoDbJsonConversion extends ScalatraBase with ApiFormats {
def renderMongo = {
case dbo: DBObject =>
contentType = formats("json")
dbo.toString
case xs: TraversableOnce[_] =>
contentType = formats("json")
val l = xs map (x => x.toString) mkString(",")
"[" + l + "]"
}: RenderPipeline
override protected def renderPipeline = renderMongo orElse super.renderPipeline
}
MyScalatraServlet.scala
package com.example.app
import org.scalatra._
import com.mongodb.casbah.Imports._
class MyScalatraMongoServlet(mongoColl: MongoCollection) extends MyScalatraWebAppStack with SimpleMongoDbJsonConversion {
get("/") {
<html>
<body>
<h1>Hello, world!</h1>
Say hello to Scalate.
</body>
</html>
}
post("/insert") {
val key = params("key")
val value = params("value")
val newObj = MongoDBObject(key->value)
mongoColl += newObj
}
get("/users") {
mongoColl.find()
for { x <- mongoColl } yield x
}
}

Cryptic Spray Json error message

I'm receiving the following error message when trying to parse some json:
[info] The future returned an exception of type: spray.httpx.PipelineException, with message:
Vector("eba760a81b177051b0520418b4e10596955adb98196c15367a2467ab66a19b5c", 600, "AN51SPP6iZBHFJ3aux1jtn6MMMD13Gh3t7", 500,
["1BXVXP82f7x9YWdWuCaCYwad8ZoYayyRYt"], "76a91473758c13a91699376abb8fe76931bdd9bdc04ee388ac", false)
(of class scala.collection.immutable.Vector). (AddressUnspentTXORequestTest.scala:14)
and I'm not really sure what that means. Here is the piece of json that I am trying to parse:
[
{
"transaction_hash": "eba760a81b177051b0520418b4e10596955adb98196c15367a2467ab66a19b5c",
"output_index": 1,
"value": 600,
"asset_id": "AN51SPP6iZBHFJ3aux1jtn6MMMD13Gh3t7",
"asset_quantity": 500,
"addresses": [
"1BXVXP82f7x9YWdWuCaCYwad8ZoYayyRYt"
],
"script_hex": "76a91473758c13a91699376abb8fe76931bdd9bdc04ee388ac",
"spent": false,
"confirmations": 31674
},
{
"transaction_hash": "1f9f6224bee8813135aba622693c78a33b3460e4efdb340174f87fdd8c9d4148",
"output_index": 1,
"value": 600,
"asset_id": "AS6tDJJ3oWrcE1Kk3T14mD8q6ycHYVzyYQ",
"asset_quantity": 200000,
"addresses": [
"1BXVXP82f7x9YWdWuCaCYwad8ZoYayyRYt"
],
"script_hex": "76a91473758c13a91699376abb8fe76931bdd9bdc04ee388ac",
"spent": false,
"confirmations": 35895
}
]
and here is the case class that I am trying to parse it into:
case class UnspentTXO(transaction_hash: String, output_index: Int, value: Long,
asset_id: Option[String], asset_quantity: Option[Long], addresses: List[BitcoinAddress],
script_hex: String, spent: Boolean)
The method initiating the request is here :
def getUnspentTXOs(address: Address): Future[List[UnspentTXO]] = {
val pipeline: HttpRequest => Future[List[UnspentTXO]] =
sendReceive ~> unmarshal[List[UnspentTXO]]
pipeline(Get(host + path + address.value + "/unspents"))
}
and finally this is how I am parsing that Json Request:
override def read(value: JsValue): UnspentTXO = {
val Seq(transaction_hash, output_index, locked_satoshies, asset_id, asset_quantity, addresses, script_hex, spent) =
value.asJsObject.getFields("transaction_hash", "value", "asset_id", "asset_quantity", "addresses", "script_hex", "spent")
val assetId = asset_id match {
case JsString(s) => Some(s)
case JsNull => None
case _ => throw new RuntimeException("Asset id should be of type JsString or JsNull, got something else")
}
val assetQuantity = asset_quantity match {
case JsNumber(n) => Some(n.toLong)
case JsNull => None
case _ => throw new RuntimeException("Asset quantity should be JsNull or a JsNumber")
}
// convert JsArray to List[ BitcoinAdress ]
val addressList = addresses match {
case ja: JsArray => {
ja.elements.toList.map( e => BitcoinAddress(e.convertTo[String]))
}
case _ => throw new RuntimeException("address list should be of type JsArray, got something else")
}
UnspentTXO(transaction_hash.convertTo[String], output_index.convertTo[Int], locked_satoshies.convertTo[Long],
assetId, assetQuantity, addressList,
script_hex.convertTo[String], spent.convertTo[Boolean])
}
I think the problem might be that the request is returning a json array instead of just a JSON object, so I am not sure if I am handling that correctly inside of my getUnspentTXOs. The error message seems to be very vague. It seems that spray is trying to wrap the json fields inside of a Vector instead of inside of a UnspentTXO case class. I'm not sure of why this is happening though.
You can not just call convertTo[ Option[ Long ] ] and convertTo[ List [ BitcointAddress ] ].
This is how convertTo is defined,
def convertTo[T :JsonReader]: T = jsonReader[T].read(this)
Which means... only types T for which an implicit evidence of typeclass JsonReader[ T ] is available can be used with convertTo.
Unless you provide appropriate implicit typeclass evidence, you will have to specially handle few cases.
Other than this, spray-json is just too minimalistic... so that so JsObject is just a wrapper on top of Map[ String, JsValue] and getFields is defined as,
def getFields(fieldNames: String*): immutable.Seq[JsValue] =
fieldNames.flatMap(fields.get)(collection.breakOut)
Which means... it will just ignore asked fields which are not present in them map, and only return a sequence of JsValue's corresponding to the asked fields which are present.
Hence optional values have to be checked to be present in the map. Or we will have to pattern-match for each possible case ( which can result in a lot of cases).
override def read(value: JsValue): UnspentTXO = {
val jsObject = value.asJsObject
// get only non-optional values here
val Seq( transaction_hash, output_index, locked_satoshies, addresses,
script_hex, spent ) =
jsObject.getFields( "transaction_hash", "output_index", "value", "addresses", "script_hex", "spent" )
// Assuming you have imported spray.json._, simple types will work.
// have to handle options differently
// or 2 optional values would mean 4 patterns-matchings of sequences like above.
// jsObject.fields is just a Map[ String, JsValue ]
val assetId = jsObject.fields.get( "asset_id" ) match {
case Some( JsString( s ) ) => Some( s )
case None => None
}
val assetQuantity = jsObject.fields.get( "asset_quantity" ) match {
case Some( JsNumber( n ) ) => Some( n.toLong )
case None => None
}
// convert JsArray to List[ BitcoinAdress ]
val addressList = addresses match {
case ja : JsArray => {
ja.elements.toList.map( BitcoinAddress( _.convertTo[ String ] ) )
}
}
UnspentTXO( transaction_hash.convertTo[ String ], output_index.convertTo[ Int ],
locked_satoshies.convertTo[ Long ], assetId, assetQuantity,
addressList, script_hex.convertTo[ String ], spent.convertTo[ Boolean ] )
}
Another way to get convertTo[ Option[ T ] ] and convertTo[ List[ T ] ] working is by importing spray.json.DefaultJsonProtocol._ which provides json-formats for most generally used types. But even then, you must have an implicit evidence of typeclass JsonReader[ T ] in your scope.