How to extract List and Map in json4s - json

I'm new in Scala and do not know how to deal with this Json with json4s:
After parsing the json and extracting the following json through:
val data = json \\ "someKey"
I have a Json like this:
[{"Id":14706061,
"Rcvr":1,
"HasSig":true,
"Sig":80},
{"Id":3425490,
"Rcvr":1,
"HasSig":false,
"Sig": 80}]
Printing it to the console, it returns:
JArray(List(JObject(List((Id,JInt(14706061)), (Rcvr,JInt(1)), (HasSig,JBool(true)), (Sig,JInt(80), Id,JInt(3425490)), (Rcvr,JInt(1)), (HasSig,JBool(false)), (Sig,JInt(80) ))
So, after that I used:
println("show values: " + data.values)
And had:
List(Map(Id -> 14706061, Rcvr -> 1, HasSig -> true, Sig -> 80), Map(Id -> 3425490, Rcvr -> 1, HasSig -> false, Sig -> 80))
But I don't know how to extract each Map from each position of the List.
I also tried to extract to a case class but I had 0 entries:
case class Example (Id: BigInt, Rcvr: Int, HasSig: Boolean, Sig: Int)
case class ExampleList (examples: List[Example])
implicit val formats = DefaultFormats.strict
val dataList = data.extract[ExampleList]
Thanks in advance for your help
PD. If I assign:
val dataList = data.values
The type of dataList (with getClass) is: class scala.collection.immutable.$colon$colon
PD2. SOLUTION
After the:
val data = json \\ "someKey"
I put:
val dataList = data.extract[JArray]
val examples = dataList.values
It returns an iterable Array with its Maps iterable, so fixed.
Checked with:
println("number of elements: " + examples.length)
and
println("show each item: " + examples.foreach(println))
Thanks for taking your time in reading.

If you want to extract into a Case Class instead of a Map, the correct type for extraction is List[Example], instead of ExampleList.
ExampleList has an attribute examples, your json doesn't. That is why you got an empty list.
import org.json4s.native.JsonMethods._
import org.json4s._
implicit val formats = DefaultFormats
val str = """[{"Id":14706061,
"Rcvr":1,
"HasSig":true,
"Sig":80},
{"Id":3425490,
"Rcvr":1,
"HasSig":false,
"Sig": 80}]"""
case class Example (Id: BigInt, Rcvr: Int, HasSig: Boolean, Sig: Int)
val json = parse(str)
val examples = json.extract[List[Example]]

Hope the below code helps.
enter code here
val iList = List(Map("Id" -> 14706061, "Rcvr" -> 1, "HasSig" -> "true", "Sig" -> 80),
Map("Id" -> 3425490, "Rcvr" -> 1, "HasSig" -> false, "Sig" -> 80))
for(i <-0 until iList.size){val lMap = iList(i)println("Id: " + lMap("Id"))}

Related

Turn string into simple JSON in scala

I have a string in scala which in terms of formatting, it is a json, for example
{"name":"John", "surname":"Doe"}
But when I generate this value it is initally a string. I need to convert this string into a json but I cannot change the output of the source. So how can I do this conversion in Scala? (I cannot use the Play Json library.)
If you have strings as
{"name":"John", "surname":"Doe"}
and if you want to save to elastic as mentioned here then you should use parseRaw instead of parseFull.
parseRaw will return you JSONType and parseFull will return you map
You can do as following
import scala.util.parsing.json._
val jsonString = "{\"name\":\"John\", \"surname\":\"Doe\"}"
val parsed = JSON.parseRaw(jsonString).get.toString()
And then use the jsonToEs api as
sc.makeRDD(Seq(parsed)).saveJsonToEs("spark/json-trips")
Edited
As #Aivean pointed out, when you already have json string from source, you won't be needing to convert to json, you can just do
if jsonString is {"name":"John", "surname":"Doe"}
sc.makeRDD(Seq(jsonString)).saveJsonToEs("spark/json-trips")
You can use scala.util.parsing.json to convert JSON in string format to JSON (which is basically HashMap datastructure),
eg.
scala> import scala.util.parsing.json._
import scala.util.parsing.json._
scala> val json = JSON.parseFull("""{"name":"John", "surname":"Doe"}""")
json: Option[Any] = Some(Map(name -> John, surname -> Doe))
To navigate the json format,
scala> json match { case Some(jsonMap : Map[String, Any]) => println(jsonMap("name")) case _ => println("json is empty") }
John
nested json example,
scala> val userJsonString = """{"name":"John", "address": { "perm" : "abc", "temp" : "zyx" }}"""
userJsonString: String = {"name":"John", "address": { "perm" : "abc", "temp" : "zyx" }}
scala> val json = JSON.parseFull(userJsonString)
json: Option[Any] = Some(Map(name -> John, address -> Map(perm -> abc, temp -> zyx)))
scala> json.map(_.asInstanceOf[Map[String, Any]]("address")).map(_.asInstanceOf[Map[String, String]]("perm"))
res7: Option[String] = Some(abc)

My coproduct encoding is ambiguous

This question has come up a few times recently, so I'm FAQ-ing it here. Suppose I've got some case classes like this:
import io.circe._, io.circe.generic.semiauto._
object model {
case class A(a: String)
case class B(a: String, i: Int)
case class C(i: Int, b: Boolean)
implicit val encodeA: Encoder[A] = deriveEncoder
implicit val encodeB: Encoder[B] = deriveEncoder
implicit val encodeC: Encoder[C] = deriveEncoder
implicit val decodeA: Decoder[A] = deriveDecoder
implicit val decodeB: Decoder[B] = deriveDecoder
implicit val decodeC: Decoder[C] = deriveDecoder
}
And I want to encode a value that could be any one of these as JSON using circe and Shapeless coproducts.
import io.circe.shapes._, io.circe.syntax._
import shapeless._
import model._
type ABC = A :+: B :+: C :+: CNil
val c: ABC = Coproduct[ABC](C(123, false))
This looks fine at first:
scala> c.asJson
res0: io.circe.Json =
{
"i" : 123,
"b" : false
}
But the problem is that I can never decode a coproduct containing a B element, since any valid JSON document that could be decoded as B can also be decoded as A, and the coproduct decoders provided by circe-shapes try the elements in the order they appear in the coproduct.
scala> val b: ABC = Coproduct[ABC](B("xyz", 123))
b: ABC = Inr(Inl(B(xyz,123)))
scala> val json = b.asJson
json: io.circe.Json =
{
"a" : "xyz",
"i" : 123
}
scala> io.circe.jawn.decode[ABC](json.noSpaces)
res1: Either[io.circe.Error,ABC] = Right(Inl(A(xyz)))
How can I disambiguate the elements of my coproduct in my encoding?
The circe-shapes module encodes ordinary hlists and coproducts without labels, as seen above: a coproduct as the bare JSON representation of the element, and an hlist ends up as just a JSON array (the same as the default tuple encoding):
scala> ("xyz" :: List(1, 2, 3) :: false :: HNil).asJson.noSpaces
res2: String = ["xyz",[1,2,3],false]
In the case of hlists there's no danger of ambiguity because of overlapping names, but for coproducts there is. In both cases, though, you can add labels to the JSON representation using Shapeless's labeling mechanism, which tags values with a type-level symbol.
An hlist with labels is called a "record", and a coproduct with labels is a "union". Shapeless provides special syntax and operations for both of these, and circe-shapes treats both differently from unlabeled hlists or coproducts. For example (assuming the definitions and imports above):
scala> import shapeless.union._, shapeless.syntax.singleton._
import shapeless.union._
import shapeless.syntax.singleton._
scala> type ABCL = Union.`'A -> A, 'B -> B, 'C -> C`.T
defined type alias ABCL
scala> val bL: ABCL = Coproduct[ABCL]('B ->> B("xyz", 123))
bL: ABCL = Inr(Inl(B(xyz,123)))
scala> val jsonL = bL.asJson
jsonL: io.circe.Json =
{
"B" : {
"a" : "xyz",
"i" : 123
}
}
scala> io.circe.jawn.decode[ABCL](jsonL.noSpaces)
res3: Either[io.circe.Error,ABCL] = Right(Inr(Inl(B(xyz,123))))
The encoding for records similarly includes the member names:
scala> ('a ->> "xyz" :: 'b ->> List(1) :: 'c ->> false :: HNil).asJson.noSpaces
res4: String = {"c":false,"b":[1],"a":"xyz"}
In general if you want your hlist and coproduct encodings to include labels (and look like the encodings you'd get from circe-generic for case classes and sealed trait hierarchies), you can use records or coproducts to tell circe-shapes to do this with a minimal amount of extra syntactic and runtime overhead.

Play Framework 2.3 Scala - Serialize nested objects to JSon with implicit Writes converters

I need for a frontend jquery-component a specific json object like this (ajax response):
[
{"division":"IT", "contacts":[
{“firstname”:”Carl”, “surname”:”Smith”, “empID”:1},
{“firstname”:”Henry”, “surname”:”Miller”, “empID”:2}
]},
{"division":"Sales", "contacts":[
{“firstname”:”Nancy”, “surname”:”McDonald”, “empID”:3},
{“firstname”:”Susan”, “surname”:”McBright”, “empID”:4}
]}
]
In Backend the data is read via anorm (MySQL) and transformed in following object:
List(Map("division" -> "IT", "contacts" -> List(c3,c4)), Map("division" -> "Sales", "contacts" -> List(c3,c4)))
Then I try to serialize the Object to JSon but without success (including implicit Writes converters). Below I made a simplified test case Idea-Worksheet in the same manner:
import play.api.libs.json.{JsValue, Json, Writes}
case class Contact(firstname: String, surname: String, empID: Option[Int])
case class ContactDivisionList(division: String, contacts: Seq[Contact])
implicit val ctWrites = new Writes[Contact] {
def writes(ct: Contact) = Json.obj(
"firstname" -> ct.firstname,
"surname" -> ct.surname,
"empid" -> ct.empID
)
}
implicit val sdlWrites = new Writes[ContactDivisionList] {
def writes(dlist: ContactDivisionList) = Json.obj(
"division" -> dlist.division,
"contacts" -> Json.toJson(dlist.contacts)
)
}
/*
Example
*/
val c1 = Contact("Carl","Smith",Option(1))
val c2 = Contact("Henry","Miller",Option(2))
val c3 = Contact("Nancy","McDonald",Option(3))
val c4 = Contact("Susan","McBright",Option(4))
//Test case 1 ->OK
Json.toJson(List(c1,c2,c3,c4))
//Test case 2 ->OK
val c_comp1=List(Map("contacts" -> List(c1,c2)),Map("contacts" -> List(c3,c4)))
//RESULT --> c_comp1: List[scala.collection.immutable.Map[String,List[Contact]]] = List(Map(contacts -> List(Contact(Carl,Smith,Some(1)), Contact(Henry,Miller,Some(2)))), Map(contacts -> List(Contact(Nancy,McDonald,Some(3)), Contact(Susan,McBright,Some(4)))))
Json.toJson(c_comp1)
//res1: play.api.libs.json.JsValue = [{"contacts": [{"firstname":"Carl","surname":"Smith","empid":1},{"firstname":"Henry","surname":"Miller","empid":2}]},{"contacts":[{"firstname":"Nancy","surname":"McDonald","empid":3},{"firstname":"Susan","surname":"McBright","empid":4}]}]
//Test case 3 ->Fail!!!
val c_comp2 = List(Map("division" -> "IT", "contacts" -> List(c1,c2)),Map("division" -> "Sales", "contacts" -> List(c3,c4)))
//sdlWrites: play.api.libs.json.Writes[ContactDivisionList]{def writes(dlist: ContactDivisionList): play.api.libs.json.JsObject} = $anon$2#3738baec
Json.toJson(c_comp2)
//!!!!!Error messages
/*Error:(39, 13) No Json serializer found for type List[scala.collection.immutable.Map[String,java.io.Serializable]]. Try to implement an implicit Writes or Format for this type.
Json.toJson(c_comp2)
^
Error:(39, 13) not enough arguments for method toJson: (implicit tjs: play.api.libs.json.Writes[List[scala.collection.immutable.Map[String,java.io.Serializable]] ])play.api.libs.json.JsValue.
Unspecified value parameter tjs.
Json.toJson(c_comp2)
^
*/
At the end of the script you can see "Test case 3" that got an error when i execute Json.toJson(c_comp2) -->"No Json serializer found for type..". I try a lot of things but i don't get it right. The only difference to successful "Test case 2" is that i extend the Map with a String-Tuppel.
I hope sombody can help me with that issue, Thx
Best regards
Karsten
Your problem is that the Map you have there is mapping String to the least upper bound of String (your division name) and List[Contact], which happens to be java.io.Serializable.
scala> case class Contact(firstname: String, surname: String, empID: Option[Int])
defined class Contact
scala> case class ContactDivisionList(division: String, contacts: Seq[Contact])
defined class ContactDivisionList
scala> val c1 = Contact("Carl","Smith",Option(1))
c1: Contact = Contact(Carl,Smith,Some(1))
scala> val c2 = Contact("Henry","Miller",Option(2))
c2: Contact = Contact(Henry,Miller,Some(2))
scala> val c3 = Contact("Nancy","McDonald",Option(3))
c3: Contact = Contact(Nancy,McDonald,Some(3))
scala> val c4 = Contact("Susan","McBright",Option(4))
c4: Contact = Contact(Susan,McBright,Some(4))
scala> Map("division" -> "IT", "contacts" -> List(c1,c2))
res10: scala.collection.immutable.Map[String,java.io.Serializable] = Map(division -> IT, contacts -> List(Contact(Carl,Smith,Some(1)), Contact(Henry,Miller,Some(2))))
I'm not entirely sure the nature of your problem, but if you already have List[ContactDivisionList], it's pretty straightforward to serialize that to JSON:
scala> implicit val contactWrites = Json.writes[Contact]
contactWrites: play.api.libs.json.OWrites[Contact] = play.api.libs.json.OWrites$$anon$2#3676af92
scala> implicit val contactDivisionListWrites = Json.writes[ContactDivisionList]
contactDivisionListWrites: play.api.libs.json.OWrites[ContactDivisionList] = play.api.libs.json.OWrites$$anon$2#2999d17d
scala> Json.toJson(List(ContactDivisionList("IT", List(c1,c2)), ContactDivisionList("Sales", List(c3, c4))))
res2: play.api.libs.json.JsValue = [{"division":"IT","contacts":[{"firstname":"Carl","surname":"Smith","empID":1},{"firstname":"Henry","surname":"Miller","empID":2}]},{"division":"Sales","contacts":[{"firstname":"Nancy","surname":"McDonald","empID":3},{"firstname":"Susan","surname":"McBright","empID":4}]}]
It seems to me that you should avoid having that Map in the first place. I've never worked with anorm before, but I think where you need to be looking at is the code that data structure, because at that point, you've lost typesafety. You should ideally work with ContactDivisionList or construct your object using the JsValue cases directly.

Automatic serialization/deserialization of generic case classes to/from JSON in Play2.2 Scala

This question is based on the following example, which is an attempt to deserialize a case class Node[Bird] from JSON.
import play.api.libs.json._
import play.api.libs.functional.syntax._
object ValidationTest {
case class Node[T](_id: Int, thing: T)
case class Bird(name: String, color: String)
// ERROR -- No apply function found matching unapply parameters
implicit val birdNodeFormat = Json.format[Node[Bird]]
val birdNodesJson: JsValue = Json.arr(
Json.obj(
"_id" -> 0,
"thing" -> Json.obj(
"name" -> "Cardinal",
"color" -> "Red"
)
),
Json.obj(
"_id" -> 1,
"thing" -> Json.obj(
"name" -> "Bluejay",
"color" -> "Blue"
)
)
)
val birdNodesResult = birdNodesJson.validate[Seq[Node[Bird]]]
}
In the preceding example, Scala is unable to resolve the proper apply/unapply functions for Node[Bird] for the format macro.
// ERROR -- No apply function found matching unapply parameters
implicit val birdNodeFormat = Json.format[Node[Bird]]
However, there is no problem with using a non-generic case class such as BirdNode.
case class BirdNode(_id: Int, thing: Bird)
// WORKS
implicit val birdNodeFormat = Json.format[BirdNode]
...
// WORKS
val birdNodesResult = birdNodesJson.validate[Seq[BirdNode]]
What is the best way to serialize/deserialize something like a Node[Bird] to/from JSON in Play 2.2?
You might have to define the format for Node[T] without using the macro, but this works:
implicit val birdFormat: Format[Bird] = Json.format[Bird]
implicit def nodeFormat[T : Format]: Format[Node[T]] =
((__ \ "_id").format[Int] ~
(__ \ "thing").format[T]
)(Node.apply, unlift(Node.unapply))

Scala play json api: can't implicitly convert JsArray to JsValueWrapper

I have json rest api application based on play framework and want to receive information about validation errors when I parse incoming requests. Everything is working fine except conversion from json array to json value.
Json structure I want to achieve:
{
"errors": {
"name": ["invalid", "tooshort"],
"email": ["invalid"]
}
}
When I tried to implement a sample it worked perfectly:
def error = Action {
BadRequest(obj(
"errors" -> obj(
"name" -> arr("invalid", "tooshort"),
"email" -> arr("invalid")
)
))
}
When I tried to extract the changing part like this:
def error = Action {
val e = Seq("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
// in real app it will be Seq[(JsPath, Seq[ValidationError])]
BadRequest(obj(
"errors" -> obj(e: _*)
))
}
I got a compiler error:
type mismatch; found : Seq[(String, play.api.libs.json.JsArray)] required: Seq[(String, play.api.libs.json.Json.JsValueWrapper)]
Maybe there is some implicit conversion I'm missing from JsArray to JsValueWrapper? But then, why does the sample works fine in the same file with the same imports?
Play 2.1.1, Scala 2.10.0
UPDATE:
Problem resolved thanks to Julien Lafont, the final code:
implicit val errorsWrites = new Writes[Seq[(JsPath, Seq[ValidationError])]] {
/**
* Maps validation result of Ember.js json request to json validation object, which Ember can understand and parse as DS.Model 'errors' field.
*
* #param errors errors collected after json validation
* #return json in following format:
*
* {
* "errors": {
* "error1": ["message1", "message2", ...],
* "error2": ["message3", "message4", ...],
* ...
* }
* }
*/
def writes(errors: Seq[(JsPath, Seq[ValidationError])]) = {
val mappedErrors = errors.map {
e =>
val fieldName = e._1.toString().split("/").last // take only last part of the path, which contains real field name
val messages = e._2.map(_.message)
fieldName -> messages
}
obj("errors" -> toJson(mappedErrors.toMap)) // Ember requires root "errors" object
}
}
Usage:
def create = Action(parse.json) { // method in play controller
request =>
fromJson(request.body) match {
case JsSuccess(pet, path) => Ok(obj("pet" -> Pets.create(pet)))
case JsError(errors) => UnprocessableEntity(toJson(errors)) // Ember.js expects 422 error code in case of errors
}
}
You can simply define your errors in a Map[String, Seq[String]], and transform it in Json with Json.toJson (there are built-in writers for Map[String,Y] and Seq[X])
scala> val e = Map("name" -> Seq("invalid", "tooshort"), "email" -> Seq("invalid"))
e: scala.collection.immutable.Map[String,Seq[String]] = Map(name -> List(invalid, tooshort), email -> List(invalid))
scala> Json.toJson(e)
res0: play.api.libs.json.JsValue = {"name":["invalid","tooshort"],"email":["invalid"]}
scala> Json.obj("errors" -> Json.toJson(e))
res1: play.api.libs.json.JsObject = {"errors":{"name":["invalid","tooshort"],"email":["invalid"]}}
The reason the long handed version works is because scala's automatic type inference triggers an implicit conversion toJsFieldJsValueWrapper.
For example
scala> import play.api.libs.json._
import play.api.libs.functional.syntax._
import play.api.libs.json._
scala> import play.api.libs.functional.syntax._
import play.api.libs.functional.syntax._
scala> import Json._
import Json._
scala> arr("invalid", "tooshort")
res0: play.api.libs.json.JsArray = ["invalid","tooshort"]
scala> obj("name" -> arr("invalid", "tooshort"))
res1: play.api.libs.json.JsObject = {"name":["invalid","tooshort"]}
scala> obj _
res3: Seq[(String, play.api.libs.json.Json.JsValueWrapper)] => play.api.libs.json.JsObject = <function1>
Notice that arr returns a JsArray, however obj requires a JsValueWrapper. Scala is able to convert the JsArray to JsValueWrapper when it constructs the arguments for obj. However it is not able to convert a Seq[(String, JsArray)] to a `Seq[(String, JsValueWrapper)].
If you provide the expected type when of the sequence in advance, the Scala compiler's type inferencer will perform the conversion as it creates the sequence.
scala> Seq[(String, JsValueWrapper)]("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
res4: Seq[(String, play.api.libs.json.Json.JsValueWrapper)] = List((name,JsValueWrapperImpl(["invalid","tooshort"])), (email,JsValueWrapperImpl(["invalid"])))
However once the sequence is created it cannot be converted unless there is an implicit conversion in scope that can convert sequences.
The final code snippet looks like:
def error = Action {
val e = Seq[(String, JsValueWrapper)]("name" -> arr("invalid", "tooshort"), "email" -> arr("invalid"))
// in real app it will be Seq[(JsPath, Seq[ValidationError])]
BadRequest(obj(
"errors" -> obj(e: _*)
))
}