converting multiple json objects into scala objects - json

How do I read multiple json objects from a request and convert them into my own custom object. For example, lets say we are retrieving a list of users with the following json:
{
"users":[
{
"name": "Bob",
"age": 31.0,
"email": "bob#gmail.com"
},
{
"name": "Kiki",
"age": 25.0,
"email": null
}
]
}
and my case class looks like the following
case class User(firstName: String, age: Double, email: String)
In this case the firstName value is differnt from the "name" json value. How do I get a Seq[User] from the given json file with the different names. I can't find any examples where someone is reading from a json file with multiple objects.
Thanks in advance.

Play's type class-based JSON library provides reader and writer instances for Seq[A] for any A with the appropriate instances, so all you have to do is provide a reader instance for your user type and you get a reader for a collection of users for free.
We can start with the following case class (note that I've made the fact that the email address is optional explicit with Option):
case class User(firstName: String, age: Double, email: Option[String])
Next we define our reader. I'm using 2.1's combinators, but there are other (more verbose) options.
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val userReads = (
(__ \ 'name).read[String] and
(__ \ 'age).read[Double] and
(__ \ 'email).read[Option[String]]
)(User.apply _)
We can test it:
val userJson = """{
"users": [
{ "name": "Bob", "age": 31.0, "email": "bob#gmail.com" },
{ "name": "Kiki", "age": 25.0, "email": null }
]
}"""
val users = (Json.parse(userJson) \ "users").as[Seq[User]]
And then:
scala> users foreach println
User(Bob,31.0,Some(bob#gmail.com))
User(Kiki,25.0,None)
As expected.

Related

Read a JSON and parse the contents and display the result in Scala

I am a newbie in scala. i try to read a json and parse it using json4s library.
Already written the case class and code for reading and parsing the sample json file.
I need to iterate the json and print the details of each attribute's.
Case Class
case class VehicleDetails(
name: String,
manufacturer: String,
model: String,
year: String,
color: String,
seat: Int,
variants: Seq[String],
engine: Int,
dealer: Map[String, String],
franchise: Map[String, String])
The json data and the code i tried is given below.
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.DefaultFormats
object CarDetails extends App {
val json = parse("""{
"vehicle_details": [
{
"CAR": {
"name": "Brezza",
"manufacturer": "Maruti",
"model": "LDI",
"year": 2019,
"color": "Blue",
"seat": 5,
"engine": 1,
"cylinder": 4,
"variants": [
"LDI",
"LDI(O)",
"VDI",
"VDI(O)",
"ZDI",
"ZDI+"
],
"dealer": {
"kerala": "Popular"
},
"franchise": {
"ekm": "popular_ekm"
}
},
"SUV": {
"name": "Scross",
"manufacturer": "Maruti",
"model": "LDI",
"year": 2020,
"color": "Blue",
"variants": [
"LDI",
"VDI",
"ZDI"
],
"dealer": {
"kerala": "Popular"
},
"franchise": {
"ekm": "popular_ekm"
}
}
}
]
}""")
implicit val formats = DefaultFormats
val definition = json.extract[VehicleDetails.Definition]
val elements = (json \\ "vehicle_details").children
This pretty close, just a few small changes needed.
First, create a class that encapsulates all the JSON data:
case class AllDetails(vehicle_details: List[Map[String, VehicleDetails]])
Then just extract that class from the json
implicit val formats = DefaultFormats
val details = Extraction.extract[AllDetails](json)
With this particular JSON the seat and engine fields are not present in all the records so you need to modify VehicleDetails to make these Option values:
case class VehicleDetails(
name: String,
manufacturer: String,
model: String,
year: String,
color: String,
seat: Option[Int],
variants: Seq[String],
engine: Option[Int],
dealer: Map[String, String],
franchise: Map[String, String]
)
[ Other values that might be omitted in other records will also need to be Option values ]
You can unpick the result using standard Scala methods. For example
res.vehicle_details.headOption.foreach { cars =>
val typeNames = cars.keys.mkString(", ")
println(s"Car types: $typeNames")
cars.foreach { case (car, details) =>
println(s"Car type: $car")
println(s"\tName: ${details.name}")
val variants = details.variants.mkString("[", ", ", "]")
println(s"\tVariants: $variants")
}
}
To get back to the raw JSON, use Serialization:
import org.json4s.jackson.Serialization
val newJson = Serialization.write(res)
println(newJson)

How do I ignore decoding failures in a JSON array?

Suppose I want to decode some values from a JSON array into a case class with circe. The following works just fine:
scala> import io.circe.generic.auto._, io.circe.jawn.decode
import io.circe.generic.auto._
import io.circe.jawn.decode
scala> case class Foo(name: String)
defined class Foo
scala> val goodDoc = """[{ "name": "abc" }, { "name": "xyz" }]"""
goodDoc: String = [{ "name": "abc" }, { "name": "xyz" }]
scala> decode[List[Foo]](goodDoc)
res0: Either[io.circe.Error,List[Foo]] = Right(List(Foo(abc), Foo(xyz)))
It's sometimes the case that the JSON array I'm decoding contains other, non-Foo-shaped stuff, though, which results in a decoding error:
scala> val badDoc =
| """[{ "name": "abc" }, { "id": 1 }, true, "garbage", { "name": "xyz" }]"""
badDoc: String = [{ "name": "abc" }, { "id": 1 }, true, "garbage", { "name": "xyz" }]
scala> decode[List[Foo]](badDoc)
res1: Either[io.circe.Error,List[Foo]] = Left(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(name), MoveRight, DownArray)))
How can I write a decoder that ignores anything in the array that can't be decoded into my case class?
The most straightforward way to solve this problem is to use a decoder that first tries to decode each value as a Foo, and then falls back to the identity decoder if the Foo decoder fails. The new either method in circe 0.9 makes the generic version of this practically a one-liner:
import io.circe.{ Decoder, Json }
def decodeListTolerantly[A: Decoder]: Decoder[List[A]] =
Decoder.decodeList(Decoder[A].either(Decoder[Json])).map(
_.flatMap(_.left.toOption)
)
It works like this:
scala> val myTolerantFooDecoder = decodeListTolerantly[Foo]
myTolerantFooDecoder: io.circe.Decoder[List[Foo]] = io.circe.Decoder$$anon$21#2b48626b
scala> decode(badDoc)(myTolerantFooDecoder)
res2: Either[io.circe.Error,List[Foo]] = Right(List(Foo(abc), Foo(xyz)))
To break down the steps:
Decoder.decodeList says "define a list decoder that tries to use the given decoder to decode each JSON array value".
Decoder[A].either(Decoder[Json] says "first try to decode the value as an A, and if that fails decode it as a Json value (which will always succeed), and return the result (if any) as a Either[A, Json]".
.map(_.flatMap(_.left.toOption)) says "take the resulting list of Either[A, Json] values and remove all the Rights".
…which does what we want in a fairly concise, compositional way. At some point we might want to bundle this up into a utility method in circe itself, but for now writing out this explicit version isn't too bad.

Convert json array to Seq of model class in Scala Play framework

I'm using Scala Play framework and Instagram API, and I want to extract a json array to my model class User:
case class User(val userId: String, val username: String, val profilePhoto: String, val name: String)
An json array example from the API is something like this:
{
"pagination": {},
"meta": {},
"data": [
{
"username": "carolinabentocb",
"profile_picture": "https://igcdn-photos-f-a.akamaihd.net/hphotos-ak-xfa1/t51.2885-19/s150x150/11429783_1673078532912085_1496721162_a.jpg",
"id": "363753337",
"full_name": "Carolina Bento"
},
{
"username": "pereira3044",
"profile_picture": "https://igcdn-photos-e-a.akamaihd.net/hphotos-ak-xaf1/t51.2885-19/s150x150/11351764_1662987433917180_971708049_a.jpg",
"id": "2141448590",
"full_name": "Alex"
}
]
}
In this link it is explained on how to map a json object to a model class, but how can I map the json array to a Seq/List/Array of Users?
The Json inception code is really great and it my preferred way to deserialize json. You will have to modify your User class to fit the instagram model API. Alternatively you could make a case class like InstagramApiUser or something to do the deserialization and copy to your own class later if you decide that is better for your flow. Here is the code and it works in a scala repl.
import play.api.libs.json.{Json, Format}
val js = Json.parse("""{
"pagination": {},
"meta": {},
"data": [
{
"username": "carolinabentocb",
"profile_picture": "https://igcdn-photos-f-a.akamaihd.net/hphotos-ak-xfa1/t51.2885-19/s150x150/11429783_1673078532912085_1496721162_a.jpg",
"id": "363753337",
"full_name": "Carolina Bento"
},
{
"username": "pereira3044",
"profile_picture": "https://igcdn-photos-e-a.akamaihd.net/hphotos-ak-xaf1/t51.2885-19/s150x150/11351764_1662987433917180_971708049_a.jpg",
"id": "2141448590",
"full_name": "Alex"
}
]
}""")
case class User(id: String, username: String, profile_picture: String, full_name: String)
object User {
implicit val jsonFormat: Format[User] = Json.format[User]
}
val result = (js \ "data").as[Seq[User]]
There are three methods to deserialize Json in the Play Json library, and as is the least idiomatic one in my opinion as it throws an exception if it fails to parse. You could try using asOpt[A] which will produce an Option[A] or better validate[A] which will produce a JsResult[A] and then you can log an error with the reason(s) that parsing your Json failed.
If you don't like naming your case class members to match the API names you can write the Reads manually like
import play.api.libs.json.{Json, Reads, JsPath}
import play.api.libs.functional.syntax._
case class User(val userId: String, val username: String, val profilePhoto: String, val name: String)
object User {
implicit val jsonReads: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "username").read[String] and
(JsPath \ "profile_picture").read[String] and
(JsPath \ "full_name").read[String]
)(User.apply _)
}
And it works the same way otherwise.
The solution I've found is the following:
val users: Seq[User] = (json \ "data").as[JsArray].value.map(j => j.validate[User].get)
It may exist a more beautiful approach, but I will stick with this one until other answers.

Play Scala JSON Reads converter: mapping nested properties

I have the following case class:
case class User(name: String, age: String)
I am trying to implement a JSON Reads converter for it, so I can do the following:
val user = userJson.validate[User]
… but the incoming JSON has slightly different structure:
{ "age": "12", "details": { "name": "Bob" } }
How can I implement my JSON Reads converter?
You can do this using combinators to parse sub-paths.
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class User(name: String, age: String)
val js = Json.parse("""
{ "age": "12", "details": { "name": "Bob" } }
""")
implicit val reads: Reads[User] = (
(__ \ "details" \ "name").read[String] and
(__ \ "age").read[String]
)(User.apply _)
scala> js.validate[User]
res2: play.api.libs.json.JsResult[User] = JsSuccess(User(Bob,12),)

How to parse this JSON into a Map[String, User]?

I want to go from the following JSON:
[
{
"name": "Cat",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png",
"location": "San Francisco, CA",
"id_str": "799346"
},
{
"name": "Dog",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174758/v65oai7fxn47qv9nectx_normal.png",
"location": "San Francisco, CA",
"id_str": "783214"
}
]
to a Map[String, User], where the key is the name, and the User is the object that looks like this:
case class User(name: String, profileImage: String, id: String)
I'm using Play and Scala but am struggling to figure out how to do the JSON parsing.
First you can define an instance of the Reads type class for your type:
import play.api.libs.functional.syntax._
import play.api.libs.json._
implicit val userReads = (
(__ \ 'name).read[String] and
(__ \ 'profile_image_url).read[String] and
(__ \ 'id_str).read[String]
)(User.apply _)
And then you get a Reads[List[User]] instance for free that will decode your JSON:
Json.parse(jsonString).validate[List[User]]
That's not exactly what you want, but you can turn the list into a map pretty easily:
Json.parse(jsonString).validate[List[User]].map(_.map(u => u.name -> u).toMap)
And now you've got a JsResult[Map[String, User]].
Alternatively, maybe you can use argonaut, it provides more friendly api.
val json: String = """[
{
"name": "Cat",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png",
"location": "San Francisco, CA",
"id_str": "799346"
},
{
"name": "Dog",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174758/v65oai7fxn47qv9nectx_normal.png",
"location": "San Francisco, CA",
"id_str": "783214"
}
]"""
import argonaut._, Argonaut._
case class User(name: String, url: String, location: String, idStr: String)
implicit def UserCodecJson =
casecodec4(User.apply, User.unapply)("name", "profile_image_url", "location", "id_str")
val result = json.decodeValidation[List[User]]
println(result) //Success(List(User(Cat,http://a0.twimg.com/profile ....
The good thing about this result is Scalaz Validation, because Validation is monad, so you can bind (flatmap) it with other Validation.
You can try this way:
1 Define you Case class with format:
case class User(name: String, profileImage: String, id: String) {
implicit val jsonFormat: Format[User] = Json.format[User]
}
2 parsing your json as List[User]
val jsonStr: String = ...
val users = Json.parse(jsonStr).as[List[User]]
3 group your List[User] to Map[String, User]
val whatYourWant:Map[String, User] = users.groupBy(_.name)