Akka HTTP Json Marshalling - json

I'm building simple REST API using Akka-HTTP and MongoDB for persistence. For Marshalling and Unmarshalling HTTP requests I'm using spray json. Following MongoDB documentation I defined entities like that:
package entities
import org.mongodb.scala.bson.ObjectId
case class Person(
_id: ObjectId,
firstName: String,
lastName: String,
email: String,
emailConfirmed: Boolean,
telephone: String,
...
)
object Person {
def apply(
firstName: String,
lastName: String,
email: String,
telephone: String,
...
): Publisher = new Publisher(
new ObjectId(),
firstName,
lastName,
email,
false,
telephone,
...
)
}
The problem with this approach is that I have to write a lot of boilerplate code to enable Marshalling-Unmarshalling:
implicit object PublisherItemFormat extends RootJsonFormat[Publisher] {
def write(publisher: Publisher) = JsObject(
"_id" -> JsString(publisher._id.toString),
"firstName" -> JsString(publisher.firstName),
"lastName" -> JsString(publisher.lastName),
"email" -> JsString(publisher.email),
"telephone" -> JsString(publisher.telephone),
...
)
def read(json: JsValue) = {
val jsObject = json.asJsObject
jsObject.getFields(
"_id",
"firstName",
"lastName",
"email",
"telephone",
...
) match {
case Seq(
_id,
firstName,
lastName,
email,
telephone,
...
) ⇒ Publisher(
firstName.convertTo[String],
lastName.convertTo[String],
email.convertTo[String],
telephone.convertTo[String],
...
)
}
}
}
I have approx 10 different entities that have to be saved in the Mongo, and each of them have 5-15 fields. With this number of entities, this kind of approach will produce too much boilerplate.
What could be a better way of doing it, with less boilerplate ? May-be there is another Json-serialization library instead of "spray-json" which can figure things out automatically with less code ?
Or how can I define Marshalling class for org.mongodb.scala.bson.ObjectId using Spray-Json ? This will also help to get rid of boilerplate code.

Use automatic derivation:
object EntitesJsonProtocol extends SprayJsonSupport with DefaultJsonProtocol {
implicit val placeResponseFormat = jsonFormat19(Person.apply)
def personToJson(p: Person) = p.toJson
}
the personToJson method is implemented so you don't have to contaminate your code with json imports, note that the order of declaration is important if a person is embedded inside a publisher you will have to declare personFormat before publisherFormat

Related

Can't use case class to convert JSON key:value pair when value is empty

In my Play+Scala (2.5.x, 2.11.11) application I am using Case classes to convert JSON to and from (client<->application and application<->DB). Using inbuilt Reads/Writes the conversion becomes completely automatic.
The issue comes when DB returns a null value for a field and the corresponding JSON output does not include that key:value pair. For example:
def getUserDetails (id: Option[String]) = Action.async {
dbch.dbProd.withConnection { implicit connection =>
val result: Option[SignupUserDetails] =
SQL"""select * from #$USR_LISTUSERS where (User_Pk = ${id})""".as(SignupUserDetailsParser.singleOpt)
......
}
}
Case class:
case class SignupUserDetails(User_Pk: Option[String], Emailid: Option[String], First_Name: Option[String], Last_Name: Option[String], Country: Option[String])
Parser:
implicit val SignupUserDetailsParser = Macro.namedParser[SignupUserDetails]
I found a similar question where it is mentioned that this behaviour should occur only for custom types but not for String type:
Explicitly output JSON null in case of missing optional value
But it does not work for String type as well.

Extracting certain fields from scala object <-> Json

I'm trying to extract certain fields from Scala object before converting to Json. Is there an easy way to do this.
It would also work if i could make a new Json with certain fields from a Json.
You can simply extract out the value of a Json and scala gives you the corresponding map. Example:
var myJson = Json.obj(
"customerId" -> "xyz",
"addressId" -> "xyz",
"firstName" -> "xyz",
"lastName" -> "xyz",
"address" -> "xyz"
)
Suppose you have the Json of above type. To convert it into map simply do:
var mapFromJson = myJson.value
This gives you a map of type : scala.collection.immutable.HashMap$HashTrieMap
Hard to say without more details. Suppose that you have the following Scala case class...
case class SomeObject(customerId: Long, addressId: Long, firstName: String, lastName: String, address: String)
...and that you wanted to extract the 'firstName', 'lastName', and address fields and then convert the object to Json. Using play-json you could define an implicit conversion on the companion object for the SomeObject class...
object SomeObject {
implicit val someObjectWrites = new Writes[SomeObject] {
def writes(object: SomeObject) = Json.obj(
"firstName" -> object.firstName,
"lastName" -> object.lastName,
"address" -> object.address
)
}
}
Then you could just use the code as follows:
val obj = SomeObject(12345, 678910, "John", "Doe", "My Address")
val json = Json.toJson(obj)
Note that there are probably other JSON libraries, besides play-json, that support similar functionality.

json4s parse json partially

I have a json model, where contents of certain attribute depend on the other attribute. Something like this:
"paymentMethod": "CREDIT_CARD",
"metaData": {
"cardType": "VISA",
"panPrefix": "",
"panSuffix": "",
"cardHolder": "",
"expiryDate": ""
}
So when paymentMethod equals to CREDIT_CARD, the metadata object will contain attributes as described. In case of other payment method, there'll be different metadata.
I want to handle this situation in a future-proof way. What I'm trying to do is to not parse the metadata field right away, but keep it somehow "unparsed" until I've parsed the paymentMethod field. Then I'd take the metadata and applied appropriate parsing approach.
However I don't know which type to use for a Scala class field for such "late parsed" attributes. I've tried String, JsonInput, JObject, and they all are not suitable (either don't compile or can't be parsed). Any ideas which type can I use? Or, in other words:
case class CreditCardMetadata(
cardType: String,
panPrefix: String,
panSuffix: String,
cardHolder: String,
expiryDate: String)
case class PaypalMetadata(...) // etc.
case class PaymentGatewayResponse(
paymentMethod: String,
metadata: ???)
You could create a CustomSerializer to parse the metadata directly. Something like :
case class PaymentResponse(payment: Payment, otherField: String)
sealed trait Payment
case class CreditCardPayment(cardType: String, expiryDate: String) extends Payment
case class PayPalPayment(email: String) extends Payment
object PaymentResponseSerializer extends CustomSerializer[PaymentResponse]( format => (
{
case JObject(List(
JField("paymentMethod", JString(method)),
JField("metaData", metadata),
JField("otherField", JString(otherField))
)) =>
implicit val formats = DefaultFormats
val payment = method match {
case "CREDIT_CARD" => metadata.extract[CreditCardPayment]
case "PAYPAL" => metadata.extract[PayPalPayment]
}
PaymentResponse(payment, otherField)
},
{ case _ => throw new UnsupportedOperationException } // no serialization to json
))
Which can be used as:
implicit val formats = DefaultFormats + PaymentResponseSerializer
val json = parse("""
{
"paymentMethod": "CREDIT_CARD",
"metaData": {
"cardType": "VISA",
"expiryDate": "2015"
},
"otherField": "hello"
}
""")
val json2 = parse("""
{
"paymentMethod": "PAYPAL",
"metaData": {
"email": "foo#bar.com"
},
"otherField": "world"
}
""")
val cc = json.extract[PaymentResponse]
// PaymentResponse(CreditCardPayment(VISA,2015),hello)
val pp = json2.extract[PaymentResponse]
// PaymentResponse(PayPalPayment(foo#bar.com),world)
You can use a Map[String, String].
It will contain anything you may need.
The answer by Peter Neyens has inspired me to implement my own solution. It's not as generic as his, but in my case I needed something really simple and ad-hoc. Here's what I've done:
It's possible to define a case class with the field of unknown type is represented by a JObject type. Something like this:
case class PaymentGatewayResponse(
default: Boolean,
paymentMethod: String,
visibleForCustomer: Boolean,
active: Boolean,
metaData: JObject)
When such json is parsed into such case class, this field is not parsed immediately, but contains all the necessary information. Then it's possible parse it in a separate step:
case class CreditCardMetadata(
cardType: String,
cardObfuscatedNumber: String,
cardHolder: String,
expiryDate: String)
val response: PaymentGatewayResponse = doRequest(...)
response.map { r =>
r.paymentMethod match {
case "CREDIT_CARD" => r.metaData.extract[CreditCardMetadata]
case unsupportedType: String => throw new UnsupportedPaymentMethodException(unsupportedType)
}
}

How to convert a JSON to Class - scala?

There is a http response which is a JSON string
{"id":"12345","dob":"01\/01\/1991","first_name":"Joe","gender":"male"}
Which needs to be instantiated into this class
case class UserRow(id: Long, firstName: String, lastName: String, dob: Long, gender: String)
I tried parsing the JSON into a map
val result = parseFull(response)
println(result)
Output
Some(Map(dob -> 01/01/1991, id -> 12345, first_name -> Joe, gender -> male))
Trying to get
map.get("id").toString().toLong //Throws a NumberFormatException
Dob should be converted to millis (EPOC) of type Long. Help is appreciated
Basic Answer
You can use https://github.com/json4s/json4s or another JSON-lib. And use a serializer.
I needed to adapt some parts of the JSON and the case class.
The id now really is a number
Fields that are optional (lastName) is not provided can be made optional by changing the type to Option in the case class
the names need to match exactly : last_name -> lastName
For handling the time in the dob field you could try this extension:
// Joda Time
implicit val formats = org.json4s.DefaultFormats ++ org.json4s.ext.JodaTimeSerializers.all
Some code examples
import org.json4s._
import org.json4s.native.Serialization
implicit val formats = Serialization.formats(NoTypeHints)
val jsonExample = """{"id":12345,"firstName":"Joe","gender":"male"}"""
case class UserRow(id: Long, firstName: String, lastName: Option[String], dob: Option[Long], gender: String)
Usage Example
scala> Serialization.read[UserRow](jsonExample)
res5: UserRow = UserRow(12345,Joe,None,None,male)
Extended Answer
The JSON provided leads to several problems, that could only be solved using a handcrafted deserializer https://github.com/json4s/json4s#serialization

Deserializing Json array into Scala object

I have been having major problems trying to deserialize a JSON array to a Scala object
[{"name":"Cool","city":"College Park","address":"806","id":1},{"name":"Mars ","city":"Durham","address":"12","id":2},{"name":"Something","city":"Raleigh
","address":"","id":3},{"name":"test","city":"","address":"","id":5}]
I have tried gson, jerkson(jackson scala wrapper), sjson, flexjson. None of them have worked. What I have here is a List of Customers. List[Customer].
This is the closest I've got:
val array = new JsonParser().parse( customers ).getAsJsonArray()
This gave me an 4 arrays. It obviously didn't give me a customer object though. I tried Jerkson.
val array = parse[List[Customer]](customers)
But I get this.
GenericSignatureFormatError occured : null
I'm just trying to find a simple way like I would in Java.
Here is my Scala class.
case class Customer(
id : Pk[ Int ],
name : String,
address : Option[ String ],
city : Option[ String ],
state : Option[ String ],
user_id : Int )
object Customer extends Magic[ Customer ]( Option( "Customer" ) ) {
def apply( name : String, address : String, city : String, state : String, user_id : Int ) = {
new Customer( NotAssigned, name, Some( address ), Some( city ), Some( state ), user_id )
}
def delete( id : Int ) = {
SQL( "DELETE from Customer where id = {id}" ).onParams( id ).executeUpdate()
}
}
Thanks for any help.
With gson, you could write your own json reader:
case class Customer(id: Int, name: String, address: Option[String],
city: Option[String], state: Option[String], user_id: Int)
object CustomerJsonReader {
def read(in: Reader) = readCustomers(new JsonReader(in))
def readCustomers(reader: JsonReader) = {
val customers = new ListBuffer[Customer]
reader.beginArray()
while (reader.hasNext()) {
customers += readCustomer(reader)
}
reader.endArray()
customers toList
}
def readCustomer(reader: JsonReader): Customer = {
var id = 0
var customerName = ""
var address: Option[String] = None
var city: Option[String] = None
var state: Option[String] = None
var userId = 0
reader.beginObject()
while (reader.hasNext()) {
val name = reader.nextName()
name match {
case "id" => id = reader.nextInt()
case "address" => address = Some(reader.nextString())
case "state" => state = Some(reader.nextString())
case "user_id" => userId = reader.nextInt()
case "name" => customerName = reader.nextString()
case "city" => city = Some(reader.nextString())
case _ => reader.skipValue()
}
}
reader.endObject()
Customer(id, customerName, address, city, state, userId)
}
}
val json =
"""
[{"name":"Cool","city":"College Park","address":"806","id":1},
{"name":"Mars ","city":"Durham","address":"12","id":2},
{"name":"Something","city":"Raleigh ","address":"","id":3},
{"name":"test","city":"","address":"","id":5}]
"""
val customers: List[Customer] =
CustomerJsonReader.read(new StringReader(json))
I know that with gson, you would need Array instead of a scala.List. I would suggest giving that a shot. You should use that with gson.fromJson, I think.
You can also try Jerkson = Jackson + Scala
Quite easy to use even if I had problems with special JSON fields containing "-"
A small tuto I saw on twitter recently: http://logician.free.fr/index.php/2011/09/16/play-scala-and-json/
I've been driven insane by this now and went through trying GSON, Lift-Json, Sjson and finally Jerkson, and found peace with that one.
Here's how I use it in combination with Play:
http://logician.eu/index.php/2011/09/16/play-scala-and-json/
http://logician.eu/index.php/2011/11/01/writing-custom-deserializers-for-jerkson/
I use Lift's json library for this purpose, it easily lets you parse JSON and extract values into case classes. It's packaged as a separate jar so you don't need the whole lift framework to use it.
import net.liftweb.json._
import net.liftweb.json.JsonDSL._
implicit val formats = DefaultFormats
val json: String = "[{...."
val parsed: JValue = parse(json)
val customers: List[Customer] = parsed.extract[List[Customer]]
Just make sure any optional fields are defined in the case class using Option. I noticed in your code the objects are missing the user_id field, which would cause a parse error if the user_id field is declared as Int instead of Option[Int].
Aside from trying to make Jerkson (which is a great library to use from what I have heard), you could also try Jackson's Scala module -- modules are the official way Jackson is extended to deal with 3rd party datatypes as well as native datatypes and constructs of other JVM languages.
(this is not to say this is more official than Jerkson, just that there are many useful Jackson extension modules that many developers are not familiar with)
Issues with Scala module are discussed on main Jackson mailing lists (user#jackson.codehaus.org); you may have found an edge case that could be fixed.
I have written a parser/validator dsl, which allows you to explicitly resolve any type erasure issue. Out of the box it handles case classes, tuples, Option, Either, List, Map, joda DatetTime, piping to functions, multiple key mapping and key name remapping.
It uses the Jackson parser
https://github.com/HigherState/jameson
Its pretty simple to with scala and with play libraries and the code
import play.api.libs.json.{Format, Json}
case class Customer(name:String, city:String, address:String, id:Int)
object Customer {
implicit val jsonFormat: Format[Customer] = Json.format(Customer)
}
val jsonDef = Json.parse(<json-string>)
val customers = jsonDef.as[List[Customer]]
customers is list of Customer objects.