While trying to learn JSON-parsing in Scala using Play Framework's ScalaJson through JSON automated mapping using case classes, I'm getting the said error.
com.fasterxml.jackson.core.JsonParseException: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source:
"column" : {
"name" : "column_name"
}; line: 2, column: 11]
at com.fasterxml.jackson.core.JsonParser._constructError(scalaPlayJson.sc:1577)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(scalaPlayJson.sc:529)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(scalaPlayJson.sc:458)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(scalaPlayJson.sc:1620)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(scalaPlayJson.sc:685)
at play.api.libs.json.jackson.JsValueDeserializer.deserialize(scalaPlayJson.sc:175)
at play.api.libs.json.jackson.JsValueDeserializer.deserialize(scalaPlayJson.sc:124)
at play.api.libs.json.jackson.JsValueDeserializer.deserialize(scalaPlayJson.sc:119)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(scalaPlayJson.sc:3704)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(scalaPlayJson.sc:2001)
at play.api.libs.json.jackson.JacksonJson$.parseJsValue(scalaPlayJson.sc:231)
at play.api.libs.json.StaticBinding$.parseJsValue(scalaPlayJson.sc:12)
at play.api.libs.json.Json$.parse(scalaPlayJson.sc:167)
at #worksheet#.#worksheet#(scalaPlayJson.sc:50)
The input in question is the simplest JSON one could think of [file: column.json]
{
"column": {
"name": "column_name"
}
}
I'm using the following code to parse the given Json file.
case class Column(
tableId: Option[Int] = None,
id: Option[Int] = None,
name: String,
shouldCopy: Option[Boolean] = None,
dataType: Option[String] = None,
defaultValue: Option[String] = None
) {
def showColumnInfo: Unit = println(s"Name = $name")
}
object Column {
implicit val reads = Json.reads[Column]
implicit val writes = Json.writes[Column]
implicit val format = Json.format[Column]
}
Json.parse(Source.fromFile(s"$path/column.json").mkString("")).
asOpt[Column].
getOrElse(Column(name = "Could not read")).
showColumnInfo
Things that I've tried without success:
Change the key "column" to "Column" in JSON (capitalize 'C' to match case class's name)
Provide the above JSON as String instead of reading it from file
My questions are:
What's causing the error?
Where can I find list of all JsonParseException error codes with their meanings?
Framework versions:
Scala v2.11.11
Play-Json v2.6.6 [SBT: "com.typesafe.play" %% "play-json" % "2.6.6"]
SBT v1.0.3
Turns out my input JSON was wrong (it was a valid JSON, but incorrect as per provided case class). I had wrapped it inside an extra pair of braces {} where it should have been
{
"name": "column_name"
}
For anyone starting with Play-JSON, I would suggest them to use JsValue.as[T] instead of JsValue.asOpt[T] because the latter doesn't report any errors and you'll keep banging your head (I wasted over 5 hours :-(). The docs warn about it in advance:
Although the asOpt method is safer, any error information is lost.
Related
I have made a generic method which parses json to case class and it also works fine. But if tries to parse big json which have one or two mandatory field then I am not able to figure out which particular mandatory f ield is missing. I am only able to handle it with IllegalArgumentException. Is there a way to handle to know which is field is missing while parsing Json by using json4s.
Here is my code ->
object JsonHelper {
implicit val formats: DefaultFormats = DefaultFormats
def write[T <: AnyRef](value: T): String = jWrite(value)
def parse(value: String): JValue = jParser(value)
}
And this is the method I am using to parse Json and handle failed case ->
def parseJson[M](json: String)(implicit m: Manifest[M]): Either[ErrorResponse, M] = {
try
Right(JsonHelper.parse(json).extract[M])
catch {
case NonFatal(th) =>
th.getCause.getCause match {
case e: java.lang.IllegalArgumentException =>
error(s"Invalid JSON - $json", e)
Left(handle(exception = EmptyFieldException(e.getMessage.split(":").last)))
case _ =>
error(s"Invalid JSON - $json", th)
Left(handle(exception = new IllegalArgumentException("Invalid Json", th)))
}
}
}
Like for a Json ->
{
"name": "Json"
}
And case class ->
case class(name: String, profession: String)
if I try to parse above json into case class currently I am getting Invalid JSON - IllegalArgumentException. But is there a way that the exception tells which is field is missing like in above example "profession" is missing.
Is there a way to handle to know which is field is missing while parsing Json by using json4s.
Maybe you have more complicated setting, but for example for
import org.json4s._
import org.json4s.jackson.JsonMethods._
val str = """{
| "name": "Json"
|}""".stripMargin
val json = parse(str) // JObject(List((name,JString(Json))))
implicit val formats: Formats = DefaultFormats
case class MyClass(name: String, profession: String)
json.extract[MyClass]
it produces
org.json4s.MappingException: No usable value for profession
Did not find value which can be converted into java.lang.String
at org.json4s.reflect.package$.fail(package.scala:56)
at ...
with the name of missing field and if the class is just case class MyClass(name: String) then this produces MyClass(Json).
If the class is case class MyClass(name: String, profession: Option[String]) then this produces MyClass(Json,None).
So normally IllegalArgumentException should be followed by Caused by: org.json4s.MappingException with the field name. I guess now you're swallowing json4s MappingException somewhere. Maybe in th.getCause.getCause match .... It's hard to say without MCVE.
I am trying to familiarize myself with the PlayJSON library. I have a JSON formatted file like this:
{
"Person": [
{
"name": "Jonathon",
"age": 24,
"job": "Accountant"
}
]
}
However, I'm having difficulty with parsing it properly due to the file having different types (name is a String but age is an Int). I could technically make it so the age is a String and call .toInt on it later but for my purposes, it is by default an integer.
I know how to parse some of it:
import play.api.libs.json.{JsValue, Json}
val parsed: JsValue = Json.parse(jsonFile) //assuming jsonFile is that entire JSON String as shown above
val person: List[Map[String, String]] = (parsed \ "Person").as[List[Map[String, String]]]
Creating that person value throws an error. I know Scala is a strongly-typed language but I'm sure there is something I am missing here. I feel like this is an obvious fix too but I'm not quite sure.
The error produced is:
JsResultException(errors:List(((0)/age,List(JsonValidationError(List(error.expected.jsstring),WrappedArray())))
The error you are having, as explained in the error you are getting, is in casting to the map of string to string. The data you provided does not align with it, because the age is a string. If you want to keep in with this approach, you need to parse it into a type that will handle both strings and ints. For example:
(parsed \ "Person").validate[List[Map[String, Any]]]
Having said that, as #Luis wrote in a comment, you can just use case class to parse it. Lets declare 2 case classes:
case class JsonParsingExample(Person: Seq[Person])
case class Person(name: String, age: Int, job: String)
Now we will create a formatter for each of them on their corresponding companion object:
object Person {
implicit val format: OFormat[Person] = Json.format[Person]
}
object JsonParsingExample {
implicit val format: OFormat[JsonParsingExample] = Json.format[JsonParsingExample]
}
Now we can just do:
Json.parse(jsonFile).validate[JsonParsingExample]
Code run at Scastie.
During my fresh adventures with kotlin-react I hit a hard stop when trying to parse some data from my backend which contains enum values.
Spring-Boot sends the object in JSON form like this:
{
"id": 1,
"username": "Johnny",
"role": "CLIENT"
}
role in this case is the enum value and can have the two values CLIENT and LECTURER. If I were to parse this with a java library or let this be handled by Spring-Boot, role would be parsed to the corresponding enum value.
With kotlin-js' JSON.parse, that wouldn't work and I would have a simple string value in there.
After some testing, I came up with this snippet
val json = """{
"id": 1,
"username": "Johnny",
"role": "CLIENT",
}"""
val member: Member = JSON.parse(json) { key: String, value: Any? ->
if (key == "role") Member.Role.valueOf(value.toString())
else value
}
in which I manually have to define the conversion from the string value to the enum.
Is there something I am missing that would simplify this behaviour?
(I am not referring to using ids for the JSON and the looking those up, etc. I am curious about some method in Kotlin-JS)
I have the assumption there is not because the "original" JSON.parse in JS doesn't do this and Kotlin does not add any additional stuff in there but I still have hope!
As far as I know, no.
The problem
Kotlin.JS produces an incredibly weird type situation when deserializing using the embedded JSON class, which actually is a mirror for JavaScript's JSON class. While I haven't done much JavaScript, its type handling is near non-existent. Only manual throws can enforce it, so JSON.parse doesn't care if it returns a SomeCustomObject or a newly created object with the exact same fields.
As an example of that, if you have two different classes with the same field names (no inheritance), and have a function that accepts a variable, it doesn't care which of those (or a third for that matter) it receives as long as the variables it tries accessing on the class exists.
The type issues manifest themselves into Kotlin. Now wrapping it back to Kotlin, consider this code:
val json = """{
"x": 1, "y": "yes", "z": {
"x": 42, "y": 314159, "z": 444
}
}""".trimIndent()
data class SomeClass(val x: Int, val y: String, val z: Struct)
data class Struct(val x: Int, val y: Int, val z: Int)
fun main(args: Array<String>) {
val someInstance = JSON.parse<SomeClass>(json)
if(someInstance.z::class != Struct::class) {
println("Incompatible types: Required ${Struct::class}, found ${someInstance.z::class}");
}
}
What would you expect this to print? The natural would be to expect a Struct. The type is also explicitly declared
Unfortunately, that is not the case. Instead, it prints:
Incompatible types: Required class Struct, found class Any
The point
The embedded JSON de/serializer isn't good with types. You might be able to fix this by using a different serializing library, but I'll avoid turning this into a "use [this] library".
Essentially, JSON.parse fails to parse objects as expected. If you entirely remove the arguments and try a raw JSON.parse(json); on the JSON in your question, you'll get a role that is a String and not a Role, which you might expect. And with JSON.parse doing no type conversion what so ever, that means you have two options: using a library, or using your approach.
Your approach will unfortunately get complicated if you have nested objects, but with the types being changed, the only option you appear to have left is explicitly parsing the objects manually.
TL;DR: your approach is fine.
Im trying to implement validation by using this article ScalaJsonCombinators
Basically i want to get the value if exist and if not return null
val nextPage: JsResult[JsValue] = (ch.\("paging").\("next")).validate[JsValue]
val nextUrl: String = nextPage match {
case s: JsSuccess[String] => s.get
case e: JsError => null
}
I have tow issue`s
the first is a warning
Warning:(99, 19) non-variable type argument String in type pattern play.api.libs.json.JsSuccess[String] is unchecked since it is eliminated by erasure
case s: JsSuccess[String] => s.get
^
the second is an error because the string is a URI with special
characters im getting a scheme error
val nextPage: JsResult[JsValue] = (ch.("paging").("next")).validate[JsValue]
val nextUrl: String = nextPage match {
case s: JsSuccess[String] => s.toString
case e: JsError => null
}
java.lang.IllegalArgumentException: Illegal character in scheme name at index 9: JsSuccess(https://graph.facebook.com/v2.2/396697410351933/feed?limit=200&access_token=623501864400497%7CGumAg3k6Eu
What will be to correct way to validate this element without using serialization.??
thanks,
miki
You're validating incorrectly. calling validate[JsValue] on a JsValue is meaningless, because it will always result in JsSuccess[JsValue]. The other problem is you're then trying to pattern match a JsSuccess[String] from a JsSuccess[JsValue], which you cannot do for many reasons. One, the type argument is erased at runtime as noted by the compiler warning. And two, a JsSuccess[String] can't be a JsSuccess[JsValue], the types aren't related.
What you really need is validate[String].
Probably impossible to debug unless you provide the relevant JSON.
There's a more elegant way of doing this, though. Since you don't seem to care about the failures (you're discarding them for null), you can just use asOpt. Here is where I say that if there's a chance there is no next, then nextUrl should be Option[String] instead of String, and you should never ever ever use null in Scala.
val nextUrl: Option[String] = (ch \ "paging" \ "next").asOpt[String]
If you for some reason must use null:
val nextUrl: String = (ch \ "paging" \ "next").asOpt[String].getOrElse(null)
When my app is fed syntactically incorrect JSON I want to be able to report the error to the user with some useful detail that will allow the problem area to be located.
So in this example j will be None because of the trailing comma after "File1". Is there a way to obtain details of last parse error?
val badSyntax = """
{
"menu1": {
"id": "file1",
"value": "File1",
},
"menu2": {
"id": "file2",
"value": "File2",
}
}"""
val j = JSON.parseFull(badSyntax)
When you get a parse error, use JSON.lastNoSuccess to get the last error. It is of type JSON.NoSuccess of which thare are two subclasses, JSON.Error and JSON.Failure, both containing a msg: String member detailing the error.
Note that JSON.lastNoSuccess is not thread safe (it is a mere global variable) and is now deprecated (bound to disappear in scala 2.11)
UPDATE: Apparently, I was wrong about it not being thread-safe: it was indeed not thread-safe before scala 2.10, but now lastNoSuccess is backed by a thread-local variable (and is thus safe to use in a multi-threaded context).
After seing this, you'd be forgiven to think that as long as you read right after a parsing failure in the same thread as the one that was used to do the parsing (the thread where you called parseFull), then everything will work as expected. Unfortunately, during this refactor they also changed how they use lastNoSuccess internally inside Parsers.phrase (which is called by JSON.parseFull.
See https://github.com/scala/scala/commit/1a4faa92faaf495f75a6dd2c58376f6bb3fbf44c
Since this refactor, lastNoSuccess is reset to None at the end of Parsers.phrase. This is no problem in parsers in general, as lastNoSuccess is used as a temporary value that is returned as the result of Parsers.phrase anyway.
The problem here is that we don't call Parsers.phrase, but JSON.parseFull, which drops any error info (see case None => None inside method JSON.parseRaw at https://github.com/scala/scala/blob/v2.10.0/src/library/scala/util/parsing/json/JSON.scala).
The fact that JSON.parseFull drops any error info could easily be circumvented prior to scala 2.10 by directly reading JSON.lastNoSuccess as I advised before, but now that this value is reset at the end of Parsers.phrase, there is not much you can do to get the error information out of JSON.
Any solution? Yes. What you can do is to create your very own version of JSON that will not drop the error information:
import util.parsing.json._
object MyJSON extends Parser {
def parseRaw(input : String) : Either[NoSuccess, JSONType] = {
phrase(root)(new lexical.Scanner(input)) match {
case Success(result, _) => Right(result)
case ns: NoSuccess => Left(ns)
}
}
def parseFull(input: String): Either[NoSuccess, Any] = {
parseRaw(input).right.map(resolveType)
}
def resolveType(input: Any): Any = input match {
case JSONObject(data) => data.transform {
case (k,v) => resolveType(v)
}
case JSONArray(data) => data.map(resolveType)
case x => x
}
}
I just changed Option to Either as the result type, so that I can return parsing errors as an Left. Some test in the REPL:
scala> MyJSON.parseFull("[1,2,3]")
res11: Either[MyJSON.NoSuccess,Any] = Right(List(1.0, 2.0, 3.0))
scala> MyJSON.parseFull("[1,2,3")
res12: Either[MyJSON.NoSuccess,Any] =
Left([1.7] failure: end of input
[1,2,3
^)