How to parse this JSON into a Map[String, User]? - json

I want to go from the following JSON:
[
{
"name": "Cat",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png",
"location": "San Francisco, CA",
"id_str": "799346"
},
{
"name": "Dog",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174758/v65oai7fxn47qv9nectx_normal.png",
"location": "San Francisco, CA",
"id_str": "783214"
}
]
to a Map[String, User], where the key is the name, and the User is the object that looks like this:
case class User(name: String, profileImage: String, id: String)
I'm using Play and Scala but am struggling to figure out how to do the JSON parsing.

First you can define an instance of the Reads type class for your type:
import play.api.libs.functional.syntax._
import play.api.libs.json._
implicit val userReads = (
(__ \ 'name).read[String] and
(__ \ 'profile_image_url).read[String] and
(__ \ 'id_str).read[String]
)(User.apply _)
And then you get a Reads[List[User]] instance for free that will decode your JSON:
Json.parse(jsonString).validate[List[User]]
That's not exactly what you want, but you can turn the list into a map pretty easily:
Json.parse(jsonString).validate[List[User]].map(_.map(u => u.name -> u).toMap)
And now you've got a JsResult[Map[String, User]].

Alternatively, maybe you can use argonaut, it provides more friendly api.
val json: String = """[
{
"name": "Cat",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png",
"location": "San Francisco, CA",
"id_str": "799346"
},
{
"name": "Dog",
"profile_image_url": "http://a0.twimg.com/profile_images/2284174758/v65oai7fxn47qv9nectx_normal.png",
"location": "San Francisco, CA",
"id_str": "783214"
}
]"""
import argonaut._, Argonaut._
case class User(name: String, url: String, location: String, idStr: String)
implicit def UserCodecJson =
casecodec4(User.apply, User.unapply)("name", "profile_image_url", "location", "id_str")
val result = json.decodeValidation[List[User]]
println(result) //Success(List(User(Cat,http://a0.twimg.com/profile ....
The good thing about this result is Scalaz Validation, because Validation is monad, so you can bind (flatmap) it with other Validation.

You can try this way:
1 Define you Case class with format:
case class User(name: String, profileImage: String, id: String) {
implicit val jsonFormat: Format[User] = Json.format[User]
}
2 parsing your json as List[User]
val jsonStr: String = ...
val users = Json.parse(jsonStr).as[List[User]]
3 group your List[User] to Map[String, User]
val whatYourWant:Map[String, User] = users.groupBy(_.name)

Related

Read a JSON and parse the contents and display the result in Scala

I am a newbie in scala. i try to read a json and parse it using json4s library.
Already written the case class and code for reading and parsing the sample json file.
I need to iterate the json and print the details of each attribute's.
Case Class
case class VehicleDetails(
name: String,
manufacturer: String,
model: String,
year: String,
color: String,
seat: Int,
variants: Seq[String],
engine: Int,
dealer: Map[String, String],
franchise: Map[String, String])
The json data and the code i tried is given below.
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.DefaultFormats
object CarDetails extends App {
val json = parse("""{
"vehicle_details": [
{
"CAR": {
"name": "Brezza",
"manufacturer": "Maruti",
"model": "LDI",
"year": 2019,
"color": "Blue",
"seat": 5,
"engine": 1,
"cylinder": 4,
"variants": [
"LDI",
"LDI(O)",
"VDI",
"VDI(O)",
"ZDI",
"ZDI+"
],
"dealer": {
"kerala": "Popular"
},
"franchise": {
"ekm": "popular_ekm"
}
},
"SUV": {
"name": "Scross",
"manufacturer": "Maruti",
"model": "LDI",
"year": 2020,
"color": "Blue",
"variants": [
"LDI",
"VDI",
"ZDI"
],
"dealer": {
"kerala": "Popular"
},
"franchise": {
"ekm": "popular_ekm"
}
}
}
]
}""")
implicit val formats = DefaultFormats
val definition = json.extract[VehicleDetails.Definition]
val elements = (json \\ "vehicle_details").children
This pretty close, just a few small changes needed.
First, create a class that encapsulates all the JSON data:
case class AllDetails(vehicle_details: List[Map[String, VehicleDetails]])
Then just extract that class from the json
implicit val formats = DefaultFormats
val details = Extraction.extract[AllDetails](json)
With this particular JSON the seat and engine fields are not present in all the records so you need to modify VehicleDetails to make these Option values:
case class VehicleDetails(
name: String,
manufacturer: String,
model: String,
year: String,
color: String,
seat: Option[Int],
variants: Seq[String],
engine: Option[Int],
dealer: Map[String, String],
franchise: Map[String, String]
)
[ Other values that might be omitted in other records will also need to be Option values ]
You can unpick the result using standard Scala methods. For example
res.vehicle_details.headOption.foreach { cars =>
val typeNames = cars.keys.mkString(", ")
println(s"Car types: $typeNames")
cars.foreach { case (car, details) =>
println(s"Car type: $car")
println(s"\tName: ${details.name}")
val variants = details.variants.mkString("[", ", ", "]")
println(s"\tVariants: $variants")
}
}
To get back to the raw JSON, use Serialization:
import org.json4s.jackson.Serialization
val newJson = Serialization.write(res)
println(newJson)

how to parse a complicated json in Scala

I am a real newbie in Scala (Using Play framework) and trying to figure out what would be the best way to parse a complicated JSON.
This is an example I have:
{
"id":"test1",
"tmax":270,
"at":2,
"bcat":[
"IAB26",
"IAB25",
"IAB24"
],
"imp":[
{
"id":"1",
"banner":{
"w":320,
"h":480,
"battr":[
10
],
"api":[
]
},
"bidfloor":0.69
}
],
"app":{
"id":"1234",
"name":"Video Games",
"bundle":"www.testapp.com",
"cat":[
"IAB1"
],
"publisher":{
"id":"1111"
}
},
"device":{
"dnt":0,
"connectiontype":2,
"carrier":"Ellijay Telephone Company",
"dpidsha1":"e5f61ae0597d8abee94860d66f7d512aa68d0985",
"dpidmd5":"c1827fe90bae819017dfdb30db7e84fa",
"didmd5":"1f6cb9dc519db8e48cf9592b31cce04e",
"didsha1":"2e6a5d7f5fd1b2b5dea56a80f2b9dc24902a0ca7",
"ifa":"422aeb3a-6507-40b5-9e6f-42a0e14b51be",
"osv":"4.4",
"os":"Android",
"ua":"Mozilla/5.0 (Linux; Android 4.4.2; DL1010Q Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Safari/537.36",
"ip":"24.75.160.74",
"devicetype":1,
"geo":{
"type":1,
"country":"USA",
"city":"Jasper"
}
},
"user":{
"id":"9c2be7c2a3dfe19070f193910e92b2e0"
},
"cur":[
"USD"
]
}
Thank you very much!
I would go with case classes as described in http://json4s.org/#extracting-values e.g.
scala> import org.json4s._
scala> import org.json4s.jackson.JsonMethods._
scala> implicit val formats = DefaultFormats // Brings in default date formats etc.
scala> case class Child(name: String, age: Int, birthdate: Option[java.util.Date])
scala> case class Address(street: String, city: String)
scala> case class Person(name: String, address: Address, children: List[Child])
scala> val json = parse("""
{ "name": "joe",
"address": {
"street": "Bulevard",
"city": "Helsinki"
},
"children": [
{
"name": "Mary",
"age": 5,
"birthdate": "2004-09-04T18:06:22Z"
},
{
"name": "Mazy",
"age": 3
}
]
}
""")
scala> json.extract[Person]
res0: Person = Person(joe,Address(Bulevard,Helsinki),List(Child(Mary,5,Some(Sat Sep 04 18:06:22 EEST 2004)), Child(Mazy,3,None)))
Use implicit to automatically fetch json value with the same name.
if you have a string like
val jsonString =
{ "company": {
"name": "google",
"department": [
{
"name": "IT",
"members": [
{
"firstName": "Max",
"lastName": "Joe",
"age" : 25
},
{
"firstName": "Jean",
"lastName": "Nick",
"age" : 55,
"salary": 100,000
}
]
},
{
"name": "Marketing"
"members": [
{
"firstName": "Mike",
"lastName": "Lucas",
"age" : 43
}
]
},
]
}
}
Then we should do as following to parse it into scala object
val json: JsValue = Json.parse(jsonString) // string to JsValue
case class Person(firstName: String, lastName: String, age: Int, salary: Option[Int]) // keep the lower level object above since you will use them below
implicit personR = Json.reads[Person]
// implicit will read the json attribute if you use the same name as it is in json String. For example "name" : "google" will be read if you have Company(name: String). See the "name" and name:String .
case class Department(name: String, members: Seq[Person])
implicit departmentR = Json.reads[Department]
case class Company(name: String, department: Seq[Department])
implicit val companyR = Json.reads[Company]
val result = json.validate[Company] match {
case s: JsSuccess[Company] => s.get // s.get will return a Company object
case e: JsError => println("error")
}
Don't know if it can be used standalone, but Play Framework has great tools to work with JSON. Take a look here

Convert json array to Seq of model class in Scala Play framework

I'm using Scala Play framework and Instagram API, and I want to extract a json array to my model class User:
case class User(val userId: String, val username: String, val profilePhoto: String, val name: String)
An json array example from the API is something like this:
{
"pagination": {},
"meta": {},
"data": [
{
"username": "carolinabentocb",
"profile_picture": "https://igcdn-photos-f-a.akamaihd.net/hphotos-ak-xfa1/t51.2885-19/s150x150/11429783_1673078532912085_1496721162_a.jpg",
"id": "363753337",
"full_name": "Carolina Bento"
},
{
"username": "pereira3044",
"profile_picture": "https://igcdn-photos-e-a.akamaihd.net/hphotos-ak-xaf1/t51.2885-19/s150x150/11351764_1662987433917180_971708049_a.jpg",
"id": "2141448590",
"full_name": "Alex"
}
]
}
In this link it is explained on how to map a json object to a model class, but how can I map the json array to a Seq/List/Array of Users?
The Json inception code is really great and it my preferred way to deserialize json. You will have to modify your User class to fit the instagram model API. Alternatively you could make a case class like InstagramApiUser or something to do the deserialization and copy to your own class later if you decide that is better for your flow. Here is the code and it works in a scala repl.
import play.api.libs.json.{Json, Format}
val js = Json.parse("""{
"pagination": {},
"meta": {},
"data": [
{
"username": "carolinabentocb",
"profile_picture": "https://igcdn-photos-f-a.akamaihd.net/hphotos-ak-xfa1/t51.2885-19/s150x150/11429783_1673078532912085_1496721162_a.jpg",
"id": "363753337",
"full_name": "Carolina Bento"
},
{
"username": "pereira3044",
"profile_picture": "https://igcdn-photos-e-a.akamaihd.net/hphotos-ak-xaf1/t51.2885-19/s150x150/11351764_1662987433917180_971708049_a.jpg",
"id": "2141448590",
"full_name": "Alex"
}
]
}""")
case class User(id: String, username: String, profile_picture: String, full_name: String)
object User {
implicit val jsonFormat: Format[User] = Json.format[User]
}
val result = (js \ "data").as[Seq[User]]
There are three methods to deserialize Json in the Play Json library, and as is the least idiomatic one in my opinion as it throws an exception if it fails to parse. You could try using asOpt[A] which will produce an Option[A] or better validate[A] which will produce a JsResult[A] and then you can log an error with the reason(s) that parsing your Json failed.
If you don't like naming your case class members to match the API names you can write the Reads manually like
import play.api.libs.json.{Json, Reads, JsPath}
import play.api.libs.functional.syntax._
case class User(val userId: String, val username: String, val profilePhoto: String, val name: String)
object User {
implicit val jsonReads: Reads[User] = (
(JsPath \ "id").read[String] and
(JsPath \ "username").read[String] and
(JsPath \ "profile_picture").read[String] and
(JsPath \ "full_name").read[String]
)(User.apply _)
}
And it works the same way otherwise.
The solution I've found is the following:
val users: Seq[User] = (json \ "data").as[JsArray].value.map(j => j.validate[User].get)
It may exist a more beautiful approach, but I will stick with this one until other answers.

How to deserialize json with json-lenses

What is the best practice to deserialize JSON to a Scala case class using json-lenses?
some.json :
[
{
"id": 1,
"name": "Alice"
},
{
"id": 2,
"name": "Bob"
},
{
"id": 3,
"name": "Chris"
}
]
some case class :
case class Foo(id: Long, name: String)
What's best way to convert the json in some.json to List[Foo] ?
json-lenses supports spray-json and with spray-json you could do:
import spray.json._
case class Foo(id: Long, name: String)
object JsonProtocol extends DefaultJsonProtocol {
implicit val FooFormat = jsonFormat2(Foo)
}
import JsonProtocol._
val source = scala.io.Source.fromFile("some.json")
val json = try source.mkString.parseJson finally source.close()
json.convertTo[List[Foo]]
// List[Foo] = List(Foo(1,Alice), Foo(2,Bob), Foo(3,Chris))

Play Scala JSON Reads converter: mapping nested properties

I have the following case class:
case class User(name: String, age: String)
I am trying to implement a JSON Reads converter for it, so I can do the following:
val user = userJson.validate[User]
… but the incoming JSON has slightly different structure:
{ "age": "12", "details": { "name": "Bob" } }
How can I implement my JSON Reads converter?
You can do this using combinators to parse sub-paths.
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class User(name: String, age: String)
val js = Json.parse("""
{ "age": "12", "details": { "name": "Bob" } }
""")
implicit val reads: Reads[User] = (
(__ \ "details" \ "name").read[String] and
(__ \ "age").read[String]
)(User.apply _)
scala> js.validate[User]
res2: play.api.libs.json.JsResult[User] = JsSuccess(User(Bob,12),)