What is the best practice to deserialize JSON to a Scala case class using json-lenses?
some.json :
[
{
"id": 1,
"name": "Alice"
},
{
"id": 2,
"name": "Bob"
},
{
"id": 3,
"name": "Chris"
}
]
some case class :
case class Foo(id: Long, name: String)
What's best way to convert the json in some.json to List[Foo] ?
json-lenses supports spray-json and with spray-json you could do:
import spray.json._
case class Foo(id: Long, name: String)
object JsonProtocol extends DefaultJsonProtocol {
implicit val FooFormat = jsonFormat2(Foo)
}
import JsonProtocol._
val source = scala.io.Source.fromFile("some.json")
val json = try source.mkString.parseJson finally source.close()
json.convertTo[List[Foo]]
// List[Foo] = List(Foo(1,Alice), Foo(2,Bob), Foo(3,Chris))
Related
I am a newbie in scala. i try to read a json and parse it using json4s library.
Already written the case class and code for reading and parsing the sample json file.
I need to iterate the json and print the details of each attribute's.
Case Class
case class VehicleDetails(
name: String,
manufacturer: String,
model: String,
year: String,
color: String,
seat: Int,
variants: Seq[String],
engine: Int,
dealer: Map[String, String],
franchise: Map[String, String])
The json data and the code i tried is given below.
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.DefaultFormats
object CarDetails extends App {
val json = parse("""{
"vehicle_details": [
{
"CAR": {
"name": "Brezza",
"manufacturer": "Maruti",
"model": "LDI",
"year": 2019,
"color": "Blue",
"seat": 5,
"engine": 1,
"cylinder": 4,
"variants": [
"LDI",
"LDI(O)",
"VDI",
"VDI(O)",
"ZDI",
"ZDI+"
],
"dealer": {
"kerala": "Popular"
},
"franchise": {
"ekm": "popular_ekm"
}
},
"SUV": {
"name": "Scross",
"manufacturer": "Maruti",
"model": "LDI",
"year": 2020,
"color": "Blue",
"variants": [
"LDI",
"VDI",
"ZDI"
],
"dealer": {
"kerala": "Popular"
},
"franchise": {
"ekm": "popular_ekm"
}
}
}
]
}""")
implicit val formats = DefaultFormats
val definition = json.extract[VehicleDetails.Definition]
val elements = (json \\ "vehicle_details").children
This pretty close, just a few small changes needed.
First, create a class that encapsulates all the JSON data:
case class AllDetails(vehicle_details: List[Map[String, VehicleDetails]])
Then just extract that class from the json
implicit val formats = DefaultFormats
val details = Extraction.extract[AllDetails](json)
With this particular JSON the seat and engine fields are not present in all the records so you need to modify VehicleDetails to make these Option values:
case class VehicleDetails(
name: String,
manufacturer: String,
model: String,
year: String,
color: String,
seat: Option[Int],
variants: Seq[String],
engine: Option[Int],
dealer: Map[String, String],
franchise: Map[String, String]
)
[ Other values that might be omitted in other records will also need to be Option values ]
You can unpick the result using standard Scala methods. For example
res.vehicle_details.headOption.foreach { cars =>
val typeNames = cars.keys.mkString(", ")
println(s"Car types: $typeNames")
cars.foreach { case (car, details) =>
println(s"Car type: $car")
println(s"\tName: ${details.name}")
val variants = details.variants.mkString("[", ", ", "]")
println(s"\tVariants: $variants")
}
}
To get back to the raw JSON, use Serialization:
import org.json4s.jackson.Serialization
val newJson = Serialization.write(res)
println(newJson)
For example, here payload is optional and it has 3 variants:
How can I parse the json with types like option[either[A,B,C]] but to use abstract data type using things sealed trait or sum type?
Below is a minimal example with some boiler plate:
https://scalafiddle.io/sf/K6RUWqk/1
// Start writing your ScalaFiddle code here
val json =
"""[
{
"id": 1,
"payload" : "data"
},
{
"id": 2.1,
"payload" : {
"field1" : "field1",
"field2" : 5,
"field3" : true
}
},
{
"id": 2.2,
"payload" : {
"field1" : "field1",
}
},
{
"id": 3,
payload" : 4
},
{
"id":4,
"
}
]"""
final case class Data(field1: String, field2: Option[Int])
type Payload = Either[String, Data]
final case class Record(id: Int, payload: Option[Payload])
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
implicit final val dataDecoder: Decoder[Data] = deriveDecoder
implicit final val payloadDecoder: Decoder[Payload] = Decoder[String] either Decoder[Data]
implicit final val recordDecoder: Decoder[Record] = deriveDecoder
val result = io.circe.parser.decode[List[Record]](json)
println(result)
Your code is almost fine, you have just syntax issues in your json and Record.id should be Double instead of Int - because it is how this field present in your json ("id": 2.1). Please, find fixed version below:
val json =
s"""[
{
"id": 1,
"payload" : "data"
},
{
"id": 2.1,
"payload" : {
"field1" : "field1",
"field2" : 5,
"field3" : true
}
},
{
"id": 2.2,
"payload" : {
"field1" : "field1"
}
},
{
"id": 3,
"payload" : 4
},
{
"id": 4
}
]"""
type Payload = Either[String, Data]
final case class Data(field1: String, field2: Option[Int])
final case class Record(id: Double, payload: Option[Payload]) // id is a Double in your json in some cases
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
implicit val dataDecoder: Decoder[Data] = deriveDecoder
implicit val payloadDecoder: Decoder[Payload] = Decoder[String] either Decoder[Data]
implicit val recordDecoder: Decoder[Record] = deriveDecoder
val result = io.circe.parser.decode[List[Record]](json)
println(result)
Which produced in my case:
Right(List(Record(1.0,Some(Left(data))), Record(2.1,Some(Right(Data(field1,Some(5))))), Record(2.2,Some(Right(Data(field1,None)))), Record(3.0,Some(Left(4))), Record(4.0,None)))
UPDATE:
The more general approach would be to use so-called Sum Types or in simple words - general sealed trait with several different implementations. Please, see for more details next Circe doc page : https://circe.github.io/circe/codecs/adt.html
In your case it can be achieved something like this:
import cats.syntax.functor._
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
sealed trait Payload
object Payload {
implicit val decoder: Decoder[Payload] = {
List[Decoder[Payload]](
Decoder[StringPayload].widen,
Decoder[IntPayload].widen,
Decoder[ObjectPayload].widen
).reduce(_ or _)
}
}
case class StringPayload(value: String) extends Payload
object StringPayload {
implicit val decoder: Decoder[StringPayload] = Decoder[String].map(StringPayload.apply)
}
case class IntPayload(value: Int) extends Payload
object IntPayload {
implicit val decoder: Decoder[IntPayload] = Decoder[Int].map(IntPayload.apply)
}
case class ObjectPayload(field1: String, field2: Option[Int]) extends Payload
object ObjectPayload {
implicit val decoder: Decoder[ObjectPayload] = deriveDecoder
}
final case class Record(id: Double, payload: Option[Payload])
object Record {
implicit val decoder: Decoder[Record] = deriveDecoder
}
def main(args: Array[String]): Unit = {
val json =
s"""[
{
"id": 1,
"payload" : "data"
},
{
"id": 2.1,
"payload" : {
"field1" : "field1",
"field2" : 5,
"field3" : true
}
},
{
"id": 2.2,
"payload" : {
"field1" : "field1"
}
},
{
"id": 3,
"payload" : "4"
},
{
"id": 4
}
]"""
val result = io.circe.parser.decode[List[Record]](json)
println(result)
}
which produced in my case next output:
Right(List(Record(1.0,Some(StringPayload(data))), Record(2.1,Some(ObjectPayload(field1,Some(5)))), Record(2.2,Some(ObjectPayload(field1,None))), Record(3.0,Some(StringPayload(4))), Record(4.0,None)))
Hope this helps!
my json file looks like :
{
"array":[
{
"name": ["na"],
"age": "15"
},
{
"name": ["aa","bb"],
"age": "12"
},
{
"name": ["rr,yy"],
"age": "22"
},
} ]
i try to read a config file and to get the name and age
I tried this but i didn't get a result:
val defaultSvaipConfig = ConfigFactory.load()
val config = ConfigFactory.parseFile(new File("config.json"))
val pathFileConfig = config.getConfig("array")
I tried also this :
ConfigFactory.parseFile(new File("application.json"))
config.getConfigList("mapConfig").asScala.map { conf =>
val name = config.getString("name")
val value = config.getString("age")
ConfigData(name, age)
This is probably not the answer you're looking for but for something like this I like to use circe(https://github.com/circe/circe-config):
import com.typesafe.config.ConfigFactory
import io.circe.generic.auto._
import io.circe.config.syntax._
case class Something(name: List[String], age: String)
case class WholeConfig(array: List[Something])
val config: Either[Error, WholeConfig] = ConfigFactory.load.as[WholeConfig]
I have a similar question with Deserialize json based on fields in .Net (C#), but do it in Scala.
I have an app which streams in 2 types of json objects (Account and User).
Account:
{
"data_type": "account",
"id": 1,
"type": "Trial",
"created_at": 1523982003,
}
User:
{
"data_type": "user",
"id": 1,
"account_id": 1,
"department": "Finance"
"created_at": 1523982122
}
I need to deserialize the two above json objects based on the field data_type in Scala with help of Circe library.
How can I do this?
This snippet works for me in Ammonite:
import $ivy.`io.circe:circe-core_2.12:0.9.3`, io.circe._
import $ivy.`io.circe:circe-generic_2.12:0.9.3`, io.circe.generic._
import $ivy.`io.circe:circe-generic-extras_2.12:0.9.3`, io.circe.generic.extras._
interp.load.plugin.ivy("org.scalamacros" % "paradise_2.12.4" % "2.1.1")
implicit val config: Configuration = Configuration.
default.
withSnakeCaseMemberNames.
withDiscriminator("data_type").
copy(transformConstructorNames = _.toLowerCase)
{
#ConfiguredJsonCodec
sealed trait InputEntity
object InputEntity {
#ConfiguredJsonCodec case class Account(id: Long, `type`: String, createdAt: Long) extends InputEntity
#ConfiguredJsonCodec case class User(id: Long, accountId: Long, department: String, createdAt: Long) extends InputEntity
}
}
import $ivy.`io.circe:circe-parser_2.12:0.9.3`, io.circe.parser._
val accountJson = """
{
"data_type": "account",
"id": 1,
"type": "Trial",
"created_at": 1523982003
}
"""
val account = decode[InputEntity](accountJson)
// account: Either[Error, InputEntity] = Right(Account(1L, "Trial", 1523982003L)
val userJson = """
{
"data_type": "user",
"id": 1,
"account_id": 1,
"department": "Finance",
"created_at": 1523982122
}
"""
val user = decode[InputEntity](userJson)
// user: Either[Error, InputEntity] = Right(User(1L, 1L, "Finance", 1523982122L))
(BTW: you had syntax errors in your JSON examples that would make the parser fail, so I fixed them in code above).
The most important here is
Configuration from io.circe.generic.extras._ that defines discriminator field,
keeping classes as sum type,
if you use annotations for generating codecs replace #JsonCodec with #ConfiguredJsonCodec.
Actually, you could also replace these Strings with enums, and read createdAt as LocalDateTime or similar, but that would be out of scope of this question.
The JSON object for which I'm trying to write a DecodeJson[T] contains an array of different "types" (meaning the JSON structure of its elements is varying). The only common feature is the type field which can be used to distinguish between the types. All other fields are different. Example:
{
...,
array: [
{ type: "a", a1: ..., a2: ...},
{ type: "b", b1: ...},
{ type: "c", c1: ..., c2: ..., c3: ...},
{ type: "a", a1: ..., a2: ...},
...
],
...
}
Using argonaut, is it possible to map the JSON array to a Scala Seq[Element] where Element is a supertype of suitable case classes of type ElementA, ElementB and so on?
I did the same thing with play-json and it was quite easy (basically a Reads[Element] that evaluates the type field and accordingly forwards to more specific Reads). However, I couldn't find a way to do this with argonaut.
edit: example
Scala types (I wish to use):
case class Container(id: Int, events: List[Event])
sealed trait Event
case class Birthday(name: String, age: Int) extends Event
case class Appointment(start: Long, participants: List[String]) extends Event
case class ... extends Event
JSON instance (not under my control):
{
"id":1674,
"events": {
"data": [
{
"type": "birthday",
"name": "Jones Clinton",
"age": 34
},
{
"type": "appointment",
"start": 1675156665555,
"participants": [
"John Doe",
"Jane Doe",
"Foo Bar"
]
}
]
}
}
You can create a small function to help you build a decoder that handles this format.
See below for an example.
import argonaut._, Argonaut._
def decodeByType[T](encoders: (String, DecodeJson[_ <: T])*) = {
val encMap = encoders.toMap
def decoder(h: CursorHistory, s: String) =
encMap.get(s).fold(DecodeResult.fail[DecodeJson[_ <: T]](s"Unknown type: $s", h))(d => DecodeResult.ok(d))
DecodeJson[T] { c: HCursor =>
val tf = c.downField("type")
for {
tv <- tf.as[String]
dec <- decoder(tf.history, tv)
data <- dec(c).map[T](identity)
} yield data
}
}
case class Container(id: Int, events: ContainerData)
case class ContainerData(data: List[Event])
sealed trait Event
case class Birthday(name: String, age: Int) extends Event
case class Appointment(start: Long, participants: List[String]) extends Event
implicit val eventDecoder: DecodeJson[Event] = decodeByType[Event](
"birthday" -> DecodeJson.derive[Birthday],
"appointment" -> DecodeJson.derive[Appointment]
)
implicit val containerDataDecoder: DecodeJson[ContainerData] = DecodeJson.derive[ContainerData]
implicit val containerDecoder: DecodeJson[Container] = DecodeJson.derive[Container]
val goodJsonStr =
"""
{
"id":1674,
"events": {
"data": [
{
"type": "birthday",
"name": "Jones Clinton",
"age": 34
},
{
"type": "appointment",
"start": 1675156665555,
"participants": [
"John Doe",
"Jane Doe",
"Foo Bar"
]
}
]
}
}
"""
def main(args: Array[String]) = {
println(goodJsonStr.decode[Container])
// \/-(Container(1674,ContainerData(List(Birthday(Jones Clinton,34), Appointment(1675156665555,List(John Doe, Jane Doe, Foo Bar))))))
}