Spray-json deserializing nested object - json

How to deserialize nested objects correctly in spray-json?
import spray.json._
case class Person(name: String)
case class Color(n: String, r: Int, g: Int, b: Int, p: Person)
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object ColorJsonFormat extends RootJsonFormat[Color] {
def write(c: Color) = JsObject(
"color-name" -> JsString(c.n),
"Green" -> JsNumber(c.g),
"Red" -> JsNumber(c.r),
"Blue" -> JsNumber(c.b),
"person-field" -> JsObject("p-name" -> JsString(c.p.name))
)
def read(value: JsValue) = {
value.asJsObject.getFields("color-name", "Red", "Green", "Blue", "person-field") match {
case Seq(JsString(name), JsNumber(red), JsNumber(green), JsNumber(blue), JsObject(person)) =>
Color(name, red.toInt, green.toInt, blue.toInt, null) //gotta replace null with correct deserializer
case _ => throw new DeserializationException("Color expected")
}
}
}
}
import MyJsonProtocol._
val jsValue = Color("CadetBlue", 95, 158, 160, Person("guest")).toJson
jsValue.prettyPrint
val color = jsValue.convertTo[Color] //person is missing of course
On a side-note, how to spray-json help serializing to a map of fields (with nested map for nested objects)?

The example below demonstrates JSON -> Abstract Syntax Tree -> Scala Case Classes and back with custom field names and support for optional case class members. The example is derived from the spray-json documentation at https://github.com/spray/spray-json for version 1.2.5.
package rando
import spray.json._
case class Color(name: String, red: Int, green: Int, blue: Int)
case class Team(name: String, color: Option[Color])
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val colorFormat = jsonFormat(Color, "name", "r", "g", "b")
implicit val teamFormat = jsonFormat(Team, "name", "jersey")
}
import MyJsonProtocol._
object GoSox extends App {
val obj = Team("Red Sox", Some(Color("Red", 255, 0, 0)))
val ast = obj.toJson
println(obj)
println(ast.prettyPrint)
println(ast.convertTo[Team])
println("""{ "name": "Red Sox", "jersey": null }""".asJson.convertTo[Team])
println("""{ "name": "Red Sox" }""".asJson.convertTo[Team])
}
The example outputs the following when executed:
Team(Red Sox,Some(Color(Red,255,0,0)))
{
"name": "Red Sox",
"jersey": {
"name": "Red",
"r": 255,
"g": 0,
"b": 0
}
}
Team(Red Sox,Some(Color(Red,255,0,0)))
Team(Red Sox,None)
Team(Red Sox,None)

To your remaining question - how to reuse JSON conversions within a wrapping type:
"person-field" -> JsObject("p-name" -> JsString(c.p.name))
I would change this to:
"person-field" -> p.toJson
This way, you are letting the Person case class JSONify itself, instead of introducing another way in the wrapping object. DRY and simpler.
And:
case Seq(JsString(name), JsNumber(red), JsNumber(green), JsNumber(blue), JsObject(person)) =>
Color(name, red.toInt, green.toInt, blue.toInt, null)
Use the .convertTo[Person] here:
case Seq(JsString(name), JsNumber(red), JsNumber(green), JsNumber(blue), jsv) =>
Color(name, red.toInt, green.toInt, blue.toInt, jsv.convertTo[Person])
If there are problems, please ask for more help. I have similar structure in my own project but didn't try to run them in this context.

import models.{Address, Patient}
import spray.json._
object Hospital {
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val address = jsonFormat(Address, "country", "state", "zip")
implicit val patient = jsonFormat(Patient, "name", "regNumber", "address")
}
import MyJsonProtocol._
def main(args: Array[String]): Unit = {
val p1 = Patient(name = "Amar", regNumber = 234, address = Address("IN", "KA", 49))
println(p1.toJson.sortedPrint)
}
}
Output
{
"address": {
"country": "IN",
"state": "KA",
"zip": 49
},
"name": "Amar",
"regNumber": 234
}

Related

spray json in scala: deserializing json with unknown fields without losing them

I'm looking for a solution to this but for json spray and my searches and attempts to get this working with json spray have failed thus far.
If I have the following json:
{
"personalData": {
"person": {
"first": "first_name",
"last": "last_name"
},
"contact": {
"phone": "1111111",
"address": {
"line": "123 Main St",
"city": "New York"
}
},
"xx": "yy", // unknown in advanced
"zz": { // unknown in advanced
"aa": "aa",
"bb": "bb",
"cc": {
"dd": "dd",
"ee": "ee"
}
}
}
}
We know for sure that the json will contain person and contact but we don't know what other fields may be created upstream from us that we don't care about/use.
I want to serialize this JSON into a case class containing person and contact but on the other hand, I don't want to lose the other fields (save them in a map so the class will be deserialized to the same json as received).
This is how far I've made it:
case class Address(
line: String,
city: String,
postalCode: Option[String]
)
case class Contact(
phone: String,
address: Address
)
case class Person(
first: String,
last: String
)
case class PersonalData(
person: Person,
contact: Contact,
extra: Map[String, JsValue]
)
implicit val personFormat = jsonFormat2(Person)
implicit val addressFormat = jsonFormat3(Address)
implicit val contactFormat = jsonFormat2(Contact)
implicit val personalDataFormat = new RootJsonFormat[PersonalData] {
def write(personalData: PersonalData): JsValue = {
JsObject(
"person" -> personalData.person.toJson,
"contact" -> personalData.contact.toJson,
// NOT SURE HOW TO REPRESENT extra input
)
}
def read(value: JsValue): CAERequestBEP = ???
}
Can someone help me do this with spray.json instead of play? I've spent such a long time trying to do this and can't seem to make it work.
In order to do that, you need to write your own formatter for PersonalDataFormat:
case class Person(first: String, last: String)
case class Address(line: String, city: String)
case class Contact(phone: String, address: Address)
case class PersonalData(person: Person, contact: Contact, extra: Map[String, JsValue])
case class Entity(personalData: PersonalData)
implicit val personFormat = jsonFormat2(Person)
implicit val addressFormat = jsonFormat2(Address)
implicit val contactFormat = jsonFormat2(Contact)
implicit object PersonalDataFormat extends RootJsonFormat[PersonalData] {
override def read(json: JsValue): PersonalData = {
val fields = json.asJsObject.fields
val person = fields.get("person").map(_.convertTo[Person]).getOrElse(???) // Do error handling instead of ???
val contact = fields.get("contact").map(_.convertTo[Contact]).getOrElse(???) // Do error handling instead of ???
PersonalData(person, contact, fields - "person" - "contact")
}
override def write(personalData: PersonalData): JsValue = {
JsObject(personalData.extra ++ ("person" -> personalData.person.toJson, "contact" -> personalData.contact.toJson))
}
}
implicit val entityFormat = jsonFormat1(Entity)
val jsonResult = jsonString.parseJson.convertTo[Entity]
The result is:
Entity(PersonalData(Person(first_name,last_name),Contact(1111111,Address(123 Main St,New York)),Map(xx -> "yy", zz -> {"aa":"aa","bb":"bb","cc":{}})))
(Assuming the json is not exactly the json above, but a valid similar one)
Code run in Scastie

Scala parses non-canonical JSON

The following data can be seen with different value types. How can I get the desired output?
package ceshi
import scala.util.parsing.json.JSON
object ceshi1212 {
def main(args: Array[String]): Unit = {
class CC[T] {
def unapply(a: Any): Option[T] = Some(a.asInstanceOf[T])
}
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
val jsonString =
"""
{
"languages": [
{
"name": "English",
"is_active": "true",
"completeness": "asdf"
},
{
"name": "Latin",
"is_active": "asdf",
"completeness": "232"
}
,{
"name": "Latin",
"is_active": "0009"
}
,
"error"
]
}
""".stripMargin
// 不规则json error和并列的数组类型不同 怎么解析自动跳过?
val result = for {
Some(M(map)) <- List(JSON.parseFull(jsonString))
L(languages) = map("languages")
M(language) <- languages
S(name) = language("name")
S(active) = language("is_active")
S(completeness) = language.getOrElse("completeness","--")
} yield {
(name, active, completeness)
}
println(result)
//i want result is: List((English,true,asdf), (Latin,asdf,232),(Latain,0009,""))
}
}
i want get result is List((English,true,asdf), (Latin,asdf,232),(Latain,0009,""))
note: 1 The string is not always at the end of the array, and the position is indeterminate
2 The three keys I need may not be complete
As said in the comments there are other libraries to be recommended for working with json have a look at this post to get an overview: What JSON library to use in Scala?
Answer to your question with specific framework (play-json)
Personally I can recommend to use the play json framework.
To obtain the result you have described with play json, your code might look like this:
import play.api.libs.json._
val json: JsValue = Json.parse(jsonString)
val list = (json \ "languages").as[Seq[JsValue]]
val names = list.map(x => ((x\"name").validate[String] match {
case JsSuccess(v, p ) => v
case _ => ""
}
))
val isActives = list.map(x => ((x\"is_active").validate[String] match {
case JsSuccess(v, p ) => v
case _ => ""
}
))
val completeness = list.map(x => ((x\"completeness").validate[String] match {
case JsSuccess(v, p ) => v
case _ => ""
}
))
// need to know in advance what is your max length of your tuple (tmax)
// since 3rd value "completeness" can be missing, so we just take "" instead
val tmax = 3
val res = for(idx <-0 to tmax-1) yield (names(idx),isActives(idx),completeness(idx))
res.toList
// List[(String, String, String)] = List((English,true,asdf), (Latin,asdf,232), (Latin,0009,""))
There's also a very good documentation for the play json framework, just check it out yourself: https://www.playframework.com/documentation/2.8.x/ScalaJson
If you can switch parser library to circe, you can deal with this types of bad data.
Given you have data model
import io.circe.generic.semiauto._
import io.circe.parser.decode
import io.circe.{Decoder, Json}
case class Languages(languages: Seq[Language])
case class Language(name: String, is_active: String, completeness: Option[String])
You can define a fault-tolerant seq decoder that would skip bad data rather than crash whole parse
def tolerantSeqDecoder[A: Decoder]: Decoder[Seq[A]] = Decoder.decodeSeq(Decoder[A]
.either(Decoder[Json])).map(_.flatMap(_.left.toOption))
and the rest...
val jsonString = """
{
"languages": [
{
"name": "English",
"is_active": "true",
"completeness": "asdf"
},
{
"name": "Latin",
"is_active": "asdf",
"completeness": "232"
},
{
"name": "Latin",
"is_active": "0009"
},
"error"
]
}
"""
val languageDecoder = deriveDecoder[Language]
implicit val tolerantDecoder = tolerantSeqDecoder[Language](languageDecoder)
implicit val languagesDecoder = deriveDecoder[Languages]
val parsed = decode[Languages](jsonString)
println(parsed)
out:
Right(Languages(List(Language(English,true,Some(asdf)), Language(Latin,asdf,Some(232)), Language(Latin,0009,None))))
This approach was suggested by one of circe developers: How do I ignore decoding failures in a JSON array?

Is there a simple way (one liner) to add an extra properties to the json output without having to write my own Json.writes

import play.api.libs.json.JsonConfiguration.Aux
import play.api.libs.json._
case class EmailStats(id: Int, providerId: String, stats: Option[String])
case class Email(id: Int = 0, createdAt: Option[String], updatedAt: Option[String]) {
implicit val config: Aux[Json.MacroOptions] = JsonConfiguration(SnakeCase)
implicit def format: OFormat[Email] = Json.using[Json.WithDefaultValues].format[Email]
implicit val writes: OWrites[Email] = Json.writes[Email]
implicit val reads: Reads[Email] = Json.reads[Email]
private var emailStats: Seq[EmailStats] = Seq[EmailStats]()
def getEmailStats(): Seq[EmailStats] = emailStats
}
Output Serialization:
{"id": 0, "created_at": "01/08/2020 00:00:12","created_at": "01/09/2020 01:56:05"}
Is there a way to add an extra property or method to the writes so that it renders the extra JSON?
{"id": 0, "created_at": "01/08/2020 00:00:12","created_at": "01/09/2020 01:56:05", "email_stats": [...]}
I don't want to do the following:
implicit val writes: Writes[Email] = (email: Email) => Json.obj(
"id" -> email.id,
"created_at" -> email.createdAt,
"updated_at" -> email.updatedAt
"email_stats" -> email.getEmailStats.map(stat=>Json.toJson(stat))
)
Thank you.
implicit val emailFormat: OFormat[Email] = {
val f = Json.using[Json.WithDefaultValues].format[Email]
OFormat(f, email => {
f.writes(email) + ("email_stats" -> Json.toJson(email.getEmailStats))
}
}
BTW, usually the implicits are added to the companion object instead of to the case class itself.
Here is a solution if you can adjust the Case Classes:
import play.api.libs.json._
case class EmailStats(id: Int, providerId: String, stats: Option[String] = None)
object EmailStats {
implicit def format: OFormat[EmailStats] = Json.using[Json.WithDefaultValues].format[EmailStats]
}
case class Email(id: Int = 0, createdAt: Option[String], updatedAt: Option[String], emailStats: Seq[EmailStats]) {
}
object Email {
implicit def format: OFormat[Email] = Json.using[Json.WithDefaultValues].format[Email]
}
Adjust the Constructors (added emailStats).
Add the Formatters in the Companion Objects.
Here the example:
val jsonStr = """{"id": 0, "createdAt": "01/08/2020 00:00:12","updatedAt": "01/09/2020 01:56:05", "emailStats": [{"id": 3, "providerId": "provider" }]}
|""".stripMargin
Json.parse(jsonStr).validate[Email] // -> JsSuccess(Email(0,Some(01/08/2020 00:00:12),Some(01/09/2020 01:56:05),List(EmailStats(3,provider,None))),)
Let me know if you need more explanation.
Be aware that I changed some of the properties to CamelCase, to simplify the example.

No Json deserializer found for type List Option

How can I use the play json OFormat macro to get a list of option?
val text = """[{"name": "John", "age": 30}, null, {"name": "Steve", "age": 34}]"""
import play.api.libs.json.Json
case class Person(name: String, age: Int)
implicit val personFormat = Json.format[Person]
val data = Json.parse(text).validate[List[Option[Person]]]
// Error: No Json deserializer found for type List[Option[Person]]. Try to implement an implicit Reads or Format for this type.
I am doing as follows, as a workaround:
val data = Json.parse(text).as[Array[JsValue]].toList.map {
case JsNull => None
case x => Some(x.validate[Person].get)
}
println(data)
// List(Some(Person(John,30)), None, Some(Person(Steve,34)))
How do I achieve the same without this workaround, using only the OFormat macro?
Not sure that it is possible directly, but could be done so for example (used this answer):
val text = """[{"name": "John", "age": 30}, null, {"name": "Steve", "age": 34}]"""
import play.api.libs.json._
implicit def optionFormat[T: Format]: Format[Option[T]] = new Format[Option[T]]{
override def reads(json: JsValue): JsResult[Option[T]] = json.validateOpt[T]
override def writes(o: Option[T]) = o match {
case Some(t) ⇒ implicitly[Writes[T]].writes(t)
case None ⇒ JsNull
}
}
case class Person(name: String, age: Int)
implicit val personFormat= {
implicit val f = Json.format[Person]
implicitly[Format[Option[Person]]]
}
val data = Json.parse(text).validate[List[Option[Person]]] // JsSuccess(List(Some(Person(John,30)), None, Some(Person(Steve,34))),)
Json.toJson(data.get) // [{"name":"John","age":30},null,{"name":"Steve","age":34}]
it just cannot translate by its own Reads[Person] -> Reads[Option[Person]] -> Reads[List[Option[Person]]]. I do help to get Reads[Option[Person]] with helper general method. Probably, analogue method is available in play lib..

How do I avoid nested Array when serializing a Map with Play Writes?

I am trying to serialize a map using the Json library from Play. I wrote my own Writes since there is none for Maps.
import play.api.libs.json.Json._
import play.api.libs.json._
object NestedArray extends App {
val m: Map[Int, String] = Map(1 -> "one", 2 -> "two")
implicit val mWrites = new Writes[Map[Int, String]] {
def writes(m: Map[Int, String]) = arr(
m.keys.map(k => {
obj("key" -> k.toString,
"value" -> m(k)
)
})
)
}
val j = toJson[Map[Int, String]](m)
println(prettyPrint(j))
}
The output is this:
[ [ {
"key" : "1",
"value" : "one"
}, {
"key" : "2",
"value" : "two"
} ] ]
As you can see there are two pairs of [ ] around the items. When I use a Wrapper class around the map I only get one pair of [ ].
case class Wrap(m: Map[Int, String])
val w = new Wrap(m)
implicit val wrapWrites = new Writes[Wrap] {
def writes(w: Wrap) = obj(
"m" -> w.m.keys.map(k => {
obj("key" -> k.toString,
"value" -> w.m(k)
)
})
)
}
val j2 = toJson[Wrap](w)
println(prettyPrint(j2))
Output:
{
"m" : [ {
"key" : "1",
"value" : "one"
}, {
"key" : "2",
"value" : "two"
} ]
}
Is there a way to achieve that without a wrapper class?
Json.arr makes a JSON array from it's argument list. Since the first argument is itself a sequence, the result is a sequence of a sequence.
E.g.
scala> Json.arr(1,2,3)
res1: play.api.libs.json.JsArray = [1,2,3]
scala> Json.arr(List(1,2,3))
res2: play.api.libs.json.JsArray = [[1,2,3]]
Removing the call to arr and converting the Iterable directly to JSON using toJson removes the nested array
import play.api.libs.json.Json._
import play.api.libs.json._
object NestedArray extends App {
val m: Map[Int, String] = Map(1 -> "one", 2 -> "two")
implicit val mWrites = new Writes[Map[Int, String]] {
def writes(m: Map[Int, String]): JsValue =
Json.toJson(m.keys.map(k => {
obj("key" -> k.toString,
"value" -> m(k)
)
}))
}
val j = toJson[Map[Int, String]](m)
println(prettyPrint(j))
}
Play does provide a Writes[Map[String, _]], which you can probably adapt for your uses. It seems unnecessary to create structure to represent a sequence of key-value pairs, when that's what a JSON object is already. Please see my answer to a similar question: How to serialize a Map[CustomType, String] to JSON