How to know which field is missed while parsing json in Json4s - json

I have made a generic method which parses json to case class and it also works fine. But if tries to parse big json which have one or two mandatory field then I am not able to figure out which particular mandatory f ield is missing. I am only able to handle it with IllegalArgumentException. Is there a way to handle to know which is field is missing while parsing Json by using json4s.
Here is my code ->
object JsonHelper {
implicit val formats: DefaultFormats = DefaultFormats
def write[T <: AnyRef](value: T): String = jWrite(value)
def parse(value: String): JValue = jParser(value)
}
And this is the method I am using to parse Json and handle failed case ->
def parseJson[M](json: String)(implicit m: Manifest[M]): Either[ErrorResponse, M] = {
try
Right(JsonHelper.parse(json).extract[M])
catch {
case NonFatal(th) =>
th.getCause.getCause match {
case e: java.lang.IllegalArgumentException =>
error(s"Invalid JSON - $json", e)
Left(handle(exception = EmptyFieldException(e.getMessage.split(":").last)))
case _ =>
error(s"Invalid JSON - $json", th)
Left(handle(exception = new IllegalArgumentException("Invalid Json", th)))
}
}
}
Like for a Json ->
{
"name": "Json"
}
And case class ->
case class(name: String, profession: String)
if I try to parse above json into case class currently I am getting Invalid JSON - IllegalArgumentException. But is there a way that the exception tells which is field is missing like in above example "profession" is missing.

Is there a way to handle to know which is field is missing while parsing Json by using json4s.
Maybe you have more complicated setting, but for example for
import org.json4s._
import org.json4s.jackson.JsonMethods._
val str = """{
| "name": "Json"
|}""".stripMargin
val json = parse(str) // JObject(List((name,JString(Json))))
implicit val formats: Formats = DefaultFormats
case class MyClass(name: String, profession: String)
json.extract[MyClass]
it produces
org.json4s.MappingException: No usable value for profession
Did not find value which can be converted into java.lang.String
at org.json4s.reflect.package$.fail(package.scala:56)
at ...
with the name of missing field and if the class is just case class MyClass(name: String) then this produces MyClass(Json).
If the class is case class MyClass(name: String, profession: Option[String]) then this produces MyClass(Json,None).
So normally IllegalArgumentException should be followed by Caused by: org.json4s.MappingException with the field name. I guess now you're swallowing json4s MappingException somewhere. Maybe in th.getCause.getCause match .... It's hard to say without MCVE.

Related

PlayJSON in Scala

I am trying to familiarize myself with the PlayJSON library. I have a JSON formatted file like this:
{
"Person": [
{
"name": "Jonathon",
"age": 24,
"job": "Accountant"
}
]
}
However, I'm having difficulty with parsing it properly due to the file having different types (name is a String but age is an Int). I could technically make it so the age is a String and call .toInt on it later but for my purposes, it is by default an integer.
I know how to parse some of it:
import play.api.libs.json.{JsValue, Json}
val parsed: JsValue = Json.parse(jsonFile) //assuming jsonFile is that entire JSON String as shown above
val person: List[Map[String, String]] = (parsed \ "Person").as[List[Map[String, String]]]
Creating that person value throws an error. I know Scala is a strongly-typed language but I'm sure there is something I am missing here. I feel like this is an obvious fix too but I'm not quite sure.
The error produced is:
JsResultException(errors:List(((0)/age,List(JsonValidationError(List(error.expected.jsstring),WrappedArray())))
The error you are having, as explained in the error you are getting, is in casting to the map of string to string. The data you provided does not align with it, because the age is a string. If you want to keep in with this approach, you need to parse it into a type that will handle both strings and ints. For example:
(parsed \ "Person").validate[List[Map[String, Any]]]
Having said that, as #Luis wrote in a comment, you can just use case class to parse it. Lets declare 2 case classes:
case class JsonParsingExample(Person: Seq[Person])
case class Person(name: String, age: Int, job: String)
Now we will create a formatter for each of them on their corresponding companion object:
object Person {
implicit val format: OFormat[Person] = Json.format[Person]
}
object JsonParsingExample {
implicit val format: OFormat[JsonParsingExample] = Json.format[JsonParsingExample]
}
Now we can just do:
Json.parse(jsonFile).validate[JsonParsingExample]
Code run at Scastie.

json deserializer with support for parameterized Case Classes

Just encountered that liftweb.json does not work with parameterized Case Classes.
The following fails at runtime:
case class ResponseOrError[R](status: String, responseData: Option[R], exception: Option[Error]) {
}
val answer = json.extract[ResponseOrError[Response]]
with:
do not know how to get type parameter from R
Is there any JSON deserializer, which actually works with parameterized Case Classes?
json4s works the way you expect. Here is an example:
import org.json4s.{DefaultFormats, Formats}
import org.json4s.jackson.JsonMethods.parse
case class Z(str: String)
case class X[R](z: Option[R])
val json =
"""
|{
| "z": {
| "str" : "test"
| }
|}
""".stripMargin
implicit val formats: Formats = DefaultFormats.withStrictArrayExtraction
val result = parse(json).extract[X[Z]]
println(result)
output
X(Some(Z(test)))

Rename JSON fields with circe

I want to have different names of fields in my case classes and in my JSON, therefore I need a comfortable way of renaming in both, encoding and decoding.
Does someone have a good solution ?
You can use Custom key mappings via annotations. The most generic way is the JsonKey annotation from io.circe.generic.extras._. Example from the docs:
import io.circe.generic.extras._, io.circe.syntax._
implicit val config: Configuration = Configuration.default
#ConfiguredJsonCodec case class Bar(#JsonKey("my-int") i: Int, s: String)
Bar(13, "Qux").asJson
// res5: io.circe.Json = JObject(object[my-int -> 13,s -> "Qux"])
This requires the package circe-generic-extras.
Here's a code sample for Decoder (bit verbose since it won't remove the old field):
val pimpedDecoder = deriveDecoder[PimpClass].prepare {
_.withFocus {
_.mapObject { x =>
val value = x("old-field")
value.map(x.add("new-field", _)).getOrElse(x)
}
}
}
implicit val decodeFieldType: Decoder[FieldType] =
Decoder.forProduct5("nth", "isVLEncoded", "isSerialized", "isSigningField", "type")
(FieldType.apply)
This is a simple way if you have lots of different field names.
https://circe.github.io/circe/codecs/custom-codecs.html
You can use the mapJson function on Encoder to derive an encoder from the generic one and remap your field name.
And you can use the prepare function on Decoder to transform the JSON passed to a generic Decoder.
You could also write both from scratch, but it may be a ton of boilerplate, those solutions should both be a handful of lines max each.
The following function can be used to rename a circe's JSON field:
import io.circe._
object CirceUtil {
def renameField(json: Json, fieldToRename: String, newName: String): Json =
(for {
value <- json.hcursor.downField(fieldToRename).focus
newJson <- json.mapObject(_.add(newName, value)).hcursor.downField(fieldToRename).delete.top
} yield newJson).getOrElse(json)
}
You can use it in an Encoder like so:
implicit val circeEncoder: Encoder[YourCaseClass] = deriveEncoder[YourCaseClass].mapJson(
CirceUtil.renameField(_, "old_field_name", "new_field_name")
)
Extra
Unit tests
import io.circe.parser._
import org.specs2.mutable.Specification
class CirceUtilSpec extends Specification {
"CirceUtil" should {
"renameField" should {
"correctly rename field" in {
val json = parse("""{ "oldFieldName": 1 }""").toOption.get
val resultJson = CirceUtil.renameField(json, "oldFieldName", "newFieldName")
resultJson.hcursor.downField("oldFieldName").focus must beNone
resultJson.hcursor.downField("newFieldName").focus must beSome
}
"return unchanged json if field is not found" in {
val json = parse("""{ "oldFieldName": 1 }""").toOption.get
val resultJson = CirceUtil.renameField(json, "nonExistentField", "newFieldName")
resultJson must be equalTo json
}
}
}
}

How to take 5 elements of JsArray in Scala?

The below code compiles, but throws an error: Exception in thread "main" scala.MatchError:[{"id":6430758,"name":...] (of class play.api.libs.json.JsArray). How can I read JSON for the given link by taking the items list in it and only 5 elements?
import play.api.libs.json._
def getProjects: List[Map[String, Any]] = {
val iter = getJSON("https://api.github.com/search/repositories?q=scala")
val json: JsValue = Json.parse(iter.get mkString "\n")
val projects = (json \ "items") match {
case l: List[Map[String, Any]] => l take 5
}
projects
}
def getJSON(url: String): Try[Iterator[String]] =
Try(Source.fromURL(url).getLines) recover {
case e: FileNotFoundException =>
throw new AppException(s"Requested page does not exist: ${e.getMessage}.")
case e: MalformedURLException =>
throw new AppException(s"Please make sure to enter a valid URL: ${e.getMessage}.")
case _ => throw new AppException("An unexpected error has occurred.")
}
Since you're using Play, you should work within its JsValue abstraction rather than jumping out to a Map[String, Any].
The reason your match is failing is because json \ "items" isn't a Map[String, Any], it's a JsValue. Ideally, you know the structure of your JSON (what your schema for project is) and you can deserialize to that:
case class Project(id: Long, name: String, ...)
object Project {
implicit val fmt = Json.format[Project]
}
val projects = WS.get("https://api.github.com/search/repositories?q=scala").map { response =>
response.json.validate[Map[String, Project]].map(_ take 5)
}
That leaves you with a Future[JsResult[Map[String, Project]]]. The outer type is Future because the operation is inherently asynchronous, JsResult will be either a JsSuccess with your Map[String, Project] or a JsError containing the reason(s) your JSON couldn't be validated.
It feels quick and dirty, but if that's really what you're wanting to do then you can try:
val listOfMaps: Seq[Map[String, String]] =
(res1 \ "items").as[JsArray].value.map { jsobj =>
jsobj.as[JsObject].value.map { case (key, value) =>
key -> value.toString
}
}.take(5)
A better option would be to create a case class with they keys and types that you are expecting and write a Reads to parse the Json to that case class. See https://www.playframework.com/documentation/2.3.x/ScalaJsonCombinators. Then you would have a list of your case class and you can easily take 5 from there.

Can I use Lift's FieldSerializer to change a field on serialization?

I'm trying to html-escape all strings in my objects when serializing them with Lift's json library. I thought I could do this by passing an escape (partial) function to the objects' FieldSerializer, but this seems to add fields to the json-object - not replace them. You can see what I mean by running the following simple example and note that it outputs
{"y":"test","x":"test"} while I wanted {"y":"test"}:
import net.liftweb.json.Serialization
import net.liftweb.json.FieldSerializer
import net.liftweb.json.FieldSerializer._
import net.liftweb.json.DefaultFormats
case class Simple(x: String)
implicit val formats = DefaultFormats +
FieldSerializer[Simple](renameTo("x","y"),renameFrom("y", "x"))
Serialization.write(Simple("test"))
Is there a way I can tell FieldSerializer to replace the field matched by my partial function?
Try CoustomeSerializer:
implicit val formats = DefaultFormats +
new CustomSerializer[Simple](ser => ( {
case JObject(JField("y", JString(x)) :: Nil) => Simple(x)
}, {
case simple: Simple => JObject(JField("y", JString(simple.x)) :: Nil)
}))
ps.I asked this on lift's google group,and got this answer,checkout https://groups.google.com/d/msg/liftweb/ShRrGNrsu6Y/sNw4JGdSU6sJ