How to parse JSON in Scala using standard Scala classes? - json

Question

I am using the build in JSON class in Scala 2.8 to parse JSON code. I don't want to use the Liftweb one or any other due to minimizing dependencies.
The way I am doing it seems too imperative, is there a better way to do it?
import scala.util.parsing.json._
...
val json:Option[Any] = JSON.parseFull(jsonString)
val map:Map[String,Any] = json.get.asInstanceOf[Map[String, Any]]
val languages:List[Any] = map.get("languages").get.asInstanceOf[List[Any]]
languages.foreach( langMap => {
val language:Map[String,Any] = langMap.asInstanceOf[Map[String,Any]]
val name:String = language.get("name").get.asInstanceOf[String]
val isActive:Boolean = language.get("is_active").get.asInstanceOf[Boolean]
val completeness:Double = language.get("completeness").get.asInstanceOf[Double]
}

Answer 1

This is a solution based on extractors which will do the class cast:
class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]
val jsonString =
"""
{
"languages": [{
"name": "English",
"is_active": true,
"completeness": 2.5
}, {
"name": "Latin",
"is_active": false,
"completeness": 0.9
}]
}
""".stripMargin
val result = for {
Some(M(map)) <- List(JSON.parseFull(jsonString))
L(languages) = map("languages")
M(language) <- languages
S(name) = language("name")
B(active) = language("is_active")
D(completeness) = language("completeness")
} yield {
(name, active, completeness)
}
assert( result == List(("English",true,2.5), ("Latin",false,0.9)))
At the start of the for loop I artificially wrap the result in a list so that it yields a list at the end. Then in the rest of the for loop I use the fact that generators (using <-) and value definitions (using =) will make use of the unapply methods.
(Older answer edited away - check edit history if you're curious)

Answer 2

This is the way I do the pattern match:
val result = JSON.parseFull(jsonStr)
result match {
// Matches if jsonStr is valid JSON and represents a Map of Strings to Any
case Some(map: Map[String, Any]) => println(map)
case None => println("Parsing failed")
case other => println("Unknown data structure: " + other)
}

Answer 3

I like #huynhjl's answer, it led me down the right path. However, it isn't great at handling error conditions. If the desired node does not exist, you get a cast exception. I've adapted this slightly to make use of Option to better handle this.
class CC[T] {
def unapply(a:Option[Any]):Option[T] = if (a.isEmpty) {
None
} else {
Some(a.get.asInstanceOf[T])
}
}
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]
for {
M(map) <- List(JSON.parseFull(jsonString))
L(languages) = map.get("languages")
language <- languages
M(lang) = Some(language)
S(name) = lang.get("name")
B(active) = lang.get("is_active")
D(completeness) = lang.get("completeness")
} yield {
(name, active, completeness)
}
Of course, this doesn't handle errors so much as avoid them. This will yield an empty list if any of the json nodes are missing. You can use a match to check for the presence of a node before acting...
for {
M(map) <- Some(JSON.parseFull(jsonString))
} yield {
map.get("languages") match {
case L(languages) => {
for {
language <- languages
M(lang) = Some(language)
S(name) = lang.get("name")
B(active) = lang.get("is_active")
D(completeness) = lang.get("completeness")
} yield {
(name, active, completeness)
}
}
case None => "bad json"
}
}

Answer 4

I tried a few things, favouring pattern matching as a way of avoiding casting but ran into trouble with type erasure on the collection types.
The main problem seems to be that the complete type of the parse result mirrors the structure of the JSON data and is either cumbersome or impossible to fully state. I guess that is why Any is used to truncate the type definitions. Using Any leads to the need for casting.
I've hacked something below which is concise but is extremely specific to the JSON data implied by the code in the question. Something more general would be more satisfactory but I'm not sure if it would be very elegant.
implicit def any2string(a: Any) = a.toString
implicit def any2boolean(a: Any) = a.asInstanceOf[Boolean]
implicit def any2double(a: Any) = a.asInstanceOf[Double]
case class Language(name: String, isActive: Boolean, completeness: Double)
val languages = JSON.parseFull(jstr) match {
case Some(x) => {
val m = x.asInstanceOf[Map[String, List[Map[String, Any]]]]
m("languages") map {l => Language(l("name"), l("isActive"), l("completeness"))}
}
case None => Nil
}
languages foreach {println}

Answer 5

val jsonString =
"""
|{
| "languages": [{
| "name": "English",
| "is_active": true,
| "completeness": 2.5
| }, {
| "name": "Latin",
| "is_active": false,
| "completeness": 0.9
| }]
|}
""".stripMargin
val result = JSON.parseFull(jsonString).map {
case json: Map[String, List[Map[String, Any]]] =>
json("languages").map(l => (l("name"), l("is_active"), l("completeness")))
}.get
println(result)
assert( result == List(("English", true, 2.5), ("Latin", false, 0.9)) )

Answer 6

You can do like this! Very easy to parse JSON code :P
package org.sqkb.service.common.bean
import java.text.SimpleDateFormat
import org.json4s
import org.json4s.JValue
import org.json4s.jackson.JsonMethods._
//import org.sqkb.service.common.kit.{IsvCode}
import scala.util.Try
/**
*
*/
case class Order(log: String) {
implicit lazy val formats = org.json4s.DefaultFormats
lazy val json: json4s.JValue = parse(log)
lazy val create_time: String = (json \ "create_time").extractOrElse("1970-01-01 00:00:00")
lazy val site_id: String = (json \ "site_id").extractOrElse("")
lazy val alipay_total_price: Double = (json \ "alipay_total_price").extractOpt[String].filter(_.nonEmpty).getOrElse("0").toDouble
lazy val gmv: Double = alipay_total_price
lazy val pub_share_pre_fee: Double = (json \ "pub_share_pre_fee").extractOpt[String].filter(_.nonEmpty).getOrElse("0").toDouble
lazy val profit: Double = pub_share_pre_fee
lazy val trade_id: String = (json \ "trade_id").extractOrElse("")
lazy val unid: Long = Try((json \ "unid").extractOpt[String].filter(_.nonEmpty).get.toLong).getOrElse(0L)
lazy val cate_id1: Int = (json \ "cate_id").extractOrElse(0)
lazy val cate_id2: Int = (json \ "subcate_id").extractOrElse(0)
lazy val cate_id3: Int = (json \ "cate_id3").extractOrElse(0)
lazy val cate_id4: Int = (json \ "cate_id4").extractOrElse(0)
lazy val coupon_id: Long = (json \ "coupon_id").extractOrElse(0)
lazy val platform: Option[String] = Order.siteMap.get(site_id)
def time_fmt(fmt: String = "yyyy-MM-dd HH:mm:ss"): String = {
val dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
val date = dateFormat.parse(this.create_time)
new SimpleDateFormat(fmt).format(date)
}
}

Answer 7

This is the way I do the Scala Parser Combinator Library:
import scala.util.parsing.combinator._
class ImprovedJsonParser extends JavaTokenParsers {
def obj: Parser[Map[String, Any]] =
"{" ~> repsep(member, ",") <~ "}" ^^ (Map() ++ _)
def array: Parser[List[Any]] =
"[" ~> repsep(value, ",") <~ "]"
def member: Parser[(String, Any)] =
stringLiteral ~ ":" ~ value ^^ { case name ~ ":" ~ value => (name, value) }
def value: Parser[Any] = (
obj
| array
| stringLiteral
| floatingPointNumber ^^ (_.toDouble)
|"true"
|"false"
)
}
object ImprovedJsonParserTest extends ImprovedJsonParser {
def main(args: Array[String]) {
val jsonString =
"""
{
"languages": [{
"name": "English",
"is_active": true,
"completeness": 2.5
}, {
"name": "Latin",
"is_active": false,
"completeness": 0.9
}]
}
""".stripMargin
val result = parseAll(value, jsonString)
println(result)
}
}

Answer 8

scala.util.parsing.json.JSON is deprecated.
Here is another approach with circe. FYI documentation: https://circe.github.io/circe/cursors.html
Add the dependency in build.sbt, I used scala 2.13.4, note the scala version must align with the library version.
val circeVersion = "0.14.0-M2"
libraryDependencies ++= Seq(
"io.circe" %% "circe-core" % circeVersion,
"io.circe" %% "circe-generic" % circeVersion,
"io.circe" %% "circe-parser" % circeVersion
)
Example 1:
case class Person(name: String, age: Int)
object Main {
def main(args: Array[String]): Unit = {
val input =
"""
|{
| "kind": "Listing",
| "data": [
| {
| "name": "Frodo",
| "age": 51
| },
| {
| "name": "Bilbo",
| "age": 60
| }
| ]
|}
|""".stripMargin
implicit val decoderPerson: Decoder[Person] = deriveDecoder[Person] // decoder required to parse to custom object
val parseResult: Json = circe.parser.parse(input).getOrElse(Json.Null)
val data: ACursor = parseResult.hcursor.downField("data") // get the data field
val personList: List[Person] = data.as[List[Person]].getOrElse(null) // parse the dataField to a list of Person
for {
person <- personList
} println(person.name + " is " + person.age)
}
}
Example 2, json has an object within an object:
case class Person(name: String, age: Int, position: Position)
case class Position(x: Int, y: Int)
object Main {
def main(args: Array[String]): Unit = {
val input =
"""
|{
| "kind": "Listing",
| "data": [
| {
| "name": "Frodo",
| "age": 51,
| "position": {
| "x": 10,
| "y": 20
| }
| },
| {
| "name": "Bilbo",
| "age": 60,
| "position": {
| "x": 75,
| "y": 85
| }
| }
| ]
|}
|""".stripMargin
implicit val decoderPosition: Decoder[Position] = deriveDecoder[Position] // must be defined before the Person decoder
implicit val decoderPerson: Decoder[Person] = deriveDecoder[Person]
val parseResult = circe.parser.parse(input).getOrElse(Json.Null)
val data = parseResult.hcursor.downField("data")
val personList = data.as[List[Person]].getOrElse(null)
for {
person <- personList
} println(person.name + " is " + person.age + " at " + person.position)
}
}