Parse mongoDB Document JSON to scala case class [duplicate] - json

I am using MongoDB scala driver. I have a problem with fetching record from MongoDB.
Following is my MongoDB initialization
private val client: MongoClient = MongoClient()
private val database: MongoDatabase = client.getDatabase(“rulemgntdb”)
val WorkOrdercollection: MongoCollection[Document] = database.getCollection("workOrder")
Find query :
MongoFactory.WorkOrdercollection.find().collect().subscribe(
(results: Seq[Document]) =>
println(s”Found: #${results}“)
)
Results printed like this :
Found: #List(Document((_id,BsonString{value=‘5af153f49547a205f9798129’}), (workOrderId,BsonString{value=‘9a9e1ce8-c576-4a15-a1ff-4af780b14b7f’}), (thingId,BsonString{value=‘Mumbai_Robot_3’}), (alertId,BsonString{value=‘Alert_1’}), (description,BsonString{value=‘Robot is not in good condition’}), (lastViewedDate,BsonDateTime{value=1525781377952}), (suggestedMaintenanceDate,BsonDateTime{value=1525781377952}), (startDate,BsonDateTime{value=1525781377952})))
I want to map this Document to my Case class.
Case class is like :
case class WorkOrder (
var id : String = (new ObjectId()).toString(),
var workOrderId: String,
var thingId : String,
var alertId : String,
var description : String,
val lastViewedDate : Date,
val suggestedMaintenanceDate : Date,
val startDate : Date
)
If I do following for getting JSON string from Document :
MongoFactory.WorkOrdercollection.find(query).subscribe(
(user: Document) => println(user.toJson()), // onNext
(error: Throwable) => println(s"Query failed: ${error.getMessage}"), // onError
() => println("Done") // onComplete
)
Then I will get Following JSON String:
{ “_id” : “5af153f49547a205f9798129", “workOrderId” : “9a9e1ce8-c576-4a15-a1ff-4af780b14b7f”, “thingId” : “Mumbai_Robot_3", “alertId” : “Alert_1", “description” : “Robot is not in good condition”, “lastViewedDate” : { “$date” : 1525781377952 }, “suggestedMaintenanceDate” : { “$date” : 1525781377952 }, “startDate” : { “$date” : 1525781377952 } }
I can Parse JSON string to case class but...Look at “startDate” : { “$date” : 1525781377952 } I am not able to Parse MongoDB Date to scala Date
How can I map Document to Case class?

You need to provide a custom codec for $date field. The following shows how it is done in play-json but the concept is similar in other JSON libraries:
object WorkOrder {
implicit val dateRead: Reads[Date] =
(__ \ "$date").read[Long].map(date => new Date(date))
implicit val dateWrite: Writes[Date] = new Writes[Date] {
def writes(date: Date): JsValue = Json.obj("$date" -> date.getTime)
}
implicit val codec = Json.format[WorkOrder]
}

You could use a JSON library.
In play-json
case class WorkOrder (
id: String,
workOrderId: String,
thingId: String,
alertId: String,
description: String,
lastViewedDate: Date,
suggestedMaintenanceDate: Date,
startDate: Date
)
object WorkOrder {
implicit lazy val fmt = Json.format[WorkOrder]
}
def documentToWorkOrder(doc: Document): WorkOrder = {
Json.parse(user.toJson().toString).validate[WorkOrder] match {
case JsSuccess(_, workOrderObj) => workOrderObj
case JsError(throwable) => throw throwable
}
}
//then in your code
MongoFactory.WorkOrdercollection.find(query).subscribe(
(user: Document) => documentToWorkOrder(user),
(error: Throwable) => println(s"Query failed: ${error.getMessage}"),
() => println("Done")
)

Related

How to convert Scala Document to JSON in Scala

I want to convert variable message which is of type scala.Seq[Scala.Document] to JSON format in following code:
path("getMessages"){
get {
parameters('roomname.as[String]) {
(roomname) =>
try {
val messagesByGroupName = MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)).toFuture()
val messages = Await.result(messagesByGroupName,60.seconds)
println("Messages:"+messages)
complete(messages)
}
catch {
case e:TimeoutException =>
complete("Reading file timeout.")
}
}
}
But it is giving me error on complete(messages) line. It is not accepting message of that type.
I tried to convert it into JSON by using following :
import play.api.libs.json._
object MyJsonProtocol{
implicit object ChatFormat extends Format[Chat] {
def writes(c: Chat) : JsValue = {
val chatSeq = Seq (
"sender" -> JsString(c.sender),
"receiver" -> JsString(c.receiver),
"message" -> JsString(c.message),
"groupChatName" -> JsString(c.groupChatName),
)
JsObject(chatSeq)
}
def reads(value: JsValue) = {
JsSuccess(Chat("","","",""))
}
}
}
But it is not working.
My Chat.scala class is as follows:
import play.api.libs.json.{Json, Reads, Writes}
class Chat(var sender:String,var receiver:String,var message:String, var groupChatName:String){
def setSenderName(senderName:String) = {
sender = senderName
}
def setReceiverName(receiverName:String) = {
receiver = receiverName
}
def setMessage(getMessage:String) = {
message = getMessage
}
def setGroupChatName(chatName:String) = {
groupChatName = chatName
}
}
object Chat {
def apply(sender: String, receiver: String, message: String, groupname: String): Chat
= new Chat(sender, receiver, message,groupname)
def unapply(arg: Chat): Option[(String, String, String,String)] = ???
implicit val requestReads: Reads[Chat] = Json.reads[Chat]
implicit val requestWrites: Writes[Chat] = Json.writes[Chat]
}
I am also not able to figure out what to write in unapply method.
I am new to scala and akka.
EDIT:
My MongoDatabase.scala which has collection is as follows:
object MongoDatabase {
val chatCodecProvider = Macros.createCodecProvider[Chat]()
val codecRegistry = CodecRegistries.fromRegistries(
CodecRegistries.fromProviders(chatCodecProvider),
DEFAULT_CODEC_REGISTRY
)
implicit val system = ActorSystem("Scala_jwt-App")
implicit val executor: ExecutionContext = system.dispatcher
val mongoClient: MongoClient = MongoClient()
val databaseName = sys.env("database_name")
// Getting mongodb database
val database: MongoDatabase = mongoClient.getDatabase(databaseName).withCodecRegistry(codecRegistry)
val registrationCollection = sys.env("register_collection_name")
val chatCollection = sys.env("chat_collection")
// Getting mongodb collection
val collectionForUserRegistration: MongoCollection[Document] = database.getCollection(registrationCollection)
collectionForUserRegistration.drop()
val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
collectionForChat.drop()
}
And if try to change val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
to
val collectionForChat: MongoCollection[Chat] = database.getCollection[Chat](chatCollection)
then I get error on in saveChatMessage() method below:
def saveChatMessage(sendMessageRequest: Chat) : String = {
val senderToReceiverMessage : Document = Document(
"sender" -> sendMessageRequest.sender,
"receiver" -> sendMessageRequest.receiver,
"message" -> sendMessageRequest.message,
"groupChatName" -> sendMessageRequest.groupChatName)
val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture()
Await.result(chatAddedFuture,60.seconds)
"Message sent"
}
on val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture() this line since it accepts data of type Seq[Document] and I am trying to add data of type Seq[Chat]
I am going to assume that MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)) returns either Seq[Chat], or Chat. Both of them are the same for play.
You have 2 options:
Adding the default format on the companion object:
object Chat {
implicit val format: Format[Chat] = Json.format[Chat]
}
In this case you can delete the object MyJsonProtocol which is not used.
In case you want to keep your own serializers(i.e. MyJsonProtocol), you need to rename MyJsonProtocol into Chat. This way the complete route will be able to find the implicit Format.
create case class for the message object you want to send
for example:
case class MyMessage(sender: String, receiver: String, message: String, groupChatName: String)
You should create Format for the type of case class
implicit val MessageTypeFormat = Json.format[MyMessage]
if complete should get JSON type - then call complete myMessage when myMessage is an instance of MyMessage.
complete(Json.toJson(myMessage))

Scala Play: List to Json-Array

I've got a List which holds some Personalization-objects. The latter is defined like this:
sealed case class Personalization(firstname: String, lastname: String, keycardId: String)
I need to map this list to a Json-Array structure which has to look like this:
"personalization": [
{
"id": "firstname",
"value": "John"
},
{
"id": "lastname",
"value": "Doe"
}...
I am struggling with the part of mapping the field information to id/value pairs. Normally, I would create a play.api.libs.json.Format out of the Personalization class and let it map automatically -> Json.format[Personalization] - but this time, I need to create an array where an entry can hold n attributes.
Therefore I am asking for advice, if there is a possibility to use the Scala Play-Framework?
Any input is much appreciated, thank you!
Writing as such JSON representation is not quite complex, using Writes.transform.
import play.api.libs.json._
case class Personalization(firstname: String, lastname: String, keycardId: String) // No need to seal a case class
implicit def writes: Writes[Personalization] = {
val tx: JsValue => JsValue = {
case JsObject(fields) => Json.toJson(fields.map {
case (k, v) => Json.obj("id" -> k, "value" -> v)
})
case jsUnexpected => jsUnexpected // doesn't happen with OWrites
}
Json.writes[Personalization].transform(tx)
}
Which can be tested as bellow.
val personalization = Personalization(
firstname = "First",
lastname = "Last",
keycardId = "Foo")
val jsonRepr = Json.toJson(personalization)
// => [{"id":"firstname","value":"First"},{"id":"lastname","value":"Last"},{"id":"keycardId","value":"Foo"}]
Reading is a little bit tricky:
implicit def reads: Reads[Personalization] = {
type Field = (String, Json.JsValueWrapper)
val fieldReads = Reads.seq(Reads[Field] { js =>
for {
id <- (js \ "id").validate[String]
v <- (js \ "value").validate[JsValue]
} yield id -> v
})
val underlying = Json.reads[Personalization]
Reads[Personalization] { js =>
js.validate(fieldReads).flatMap { fields =>
Json.obj(fields: _*).validate(underlying)
}
}
}
Which can be tested as bellow.
Json.parse("""[
{"id":"firstname","value":"First"},
{"id":"lastname","value":"Last"},
{"id":"keycardId","value":"Foo"}
]""").validate[Personalization]
// => JsSuccess(Personalization(First,Last,Foo),)
Note that is approach can be used for any case class format.
Probably it is possible to do it a more elegant way I did, but you can use the following snippet:
case class Field(id: String, value: String)
object Field {
implicit val fieldFormatter: Format[Field] = Json.format[Field]
}
sealed case class Personalization(firstname: String, lastname: String, keycardId: String)
object Personalization {
implicit val personalizationFormatter: Format[Personalization] = new Format[Personalization] {
override def reads(json: JsValue): JsResult[Personalization] =
Try {
val data = (json \ "personalization").as[JsValue]
data match {
case JsArray(value) =>
val fields = value.map(_.as[Field]).map(f => f.id -> f.value).toMap
val firstname = fields.getOrElse("firstname", throw new IllegalArgumentException("Mandatory field firstname is absent."))
val lastname = fields.getOrElse("lastname", throw new IllegalArgumentException("Mandatory field lastname is absent."))
val keycardId = fields.getOrElse("keycardId", throw new IllegalArgumentException("Mandatory field keycardId is absent."))
Personalization(firstname, lastname, keycardId)
case _ => throw new IllegalArgumentException("Incorrect json format for Personalization.")
}
}.toEither.fold(e => JsError(e.getMessage), JsSuccess(_))
override def writes(o: Personalization): JsValue = {
val fields = List(Field("firstname", o.firstname), Field("lastname", o.lastname), Field("keycardId", o.keycardId))
JsObject(List("personalization" -> Json.toJson(fields)))
}
}
}
It converts {"personalization":[{"id":"firstname","value":"John"},{"id":"lastname","value":"Doe"},{"id":"keycardId","value":"1234"}]} to Personalization(John,Doe,1234) and vice versa

Building a Json Format for a Case Class with Abstract Members

I am using the Play Framework and trying to build JSON validator for a class with abstract members. Shown below, the DataSource class is the base class which I am trying to validate the format against.
// SourceTypeConfig Trait.
trait SourceTypeConfig
final case class RDBMSConfig(...) extends SourceTypeConfig
object RDBMSConfig { implicit val fmt = Json.format[RDBMSConfig] }
final case class DirectoryConfig(
path: String,
pathType: String // Local, gcloud, azure, aws, etc.
) extends SourceTypeConfig
object DirectoryConfig { implicit val fmt = Json.format[DirectoryConfig] }
// FormatConfig trait.
trait FormatConfig
final case class SQLConfig(...) extends FormatConfig
object SQLConfig { implicit val fmt = Json.format[SQLConfig]}
final case class CSVConfig(
header: String,
inferSchema: String,
delimiter: String
) extends FormatConfig
object CSVConfig { implicit val fmt = Json.format[CSVConfig]}
// DataSource base class.
case class DataSource(
name: String,
sourceType: String,
sourceTypeConfig: SourceTypeConfig,
format: String,
formatConfig: FormatConfig
)
What I am hoping to accomplish:
val input: JsValue = Json.parse(
"""
{
"name" : "test1",
"sourceType" : "directory",
"sourceTypeConfig" : {"path" : "gs://test/path", "pathType" "google"},
"format" : "csv",
"formatConfig" : {"header" : "yes", "inferSchema" : "yes", "delimiter" : "|"}
}
"""
)
val inputResult = input.validate[DataSource]
What I am struggling with is building the DataSource object and defining its reads/writes/format. I would like it to contain a match based on the sourceType and format values that direct it to point towards the associated sourceTypeConfig and formatConfig's formats so it can parse out the JSON.
Instead of building a parser at the DataSource level, I defined parsers at the SourceConfig and FormatConfig levels, similar to what is shown below.
sealed trait SourceConfig{val sourceType: String}
object SourceConfig{
implicit val fmt = new Format[SourceConfig] {
def reads(json: JsValue): JsResult[SourceConfig] = {
def from(sourceType: String, data: JsObject): JsResult[SourceConfig] = sourceType match {
case "RDBMS" => Json.fromJson[RDBMSConfig](data)(RDBMSConfig.fmt)
case "directory" => Json.fromJson[DirectoryConfig](data)(DirectoryConfig.fmt)
case _ => JsError(s"Unknown source type: '$sourceType'")
}
for {
sourceType <- (json \ "sourceType").validate[String]
data <- json.validate[JsObject]
result <- from(sourceType, data)
} yield result
}
def writes(source: SourceConfig): JsValue =
source match {
case b: RDBMSConfig => Json.toJson(b)(RDBMSConfig.fmt)
case b: DirectoryConfig => Json.toJson(b)(DirectoryConfig.fmt)
}
}
}
Then, DataSource could be simply defined as:
object DataSource { implicit val fmt = Json.format[DataSource] }
Another option is to use play-json-derived-codecs library:
libraryDependencies += "org.julienrf" %% "play-json-derived-codecs" % "4.0.0"
import julienrf.json.derived.flat
implicit val format1: OFormat[RDBMSConfig] = Json.format[RDBMSConfig]
implicit val format2: OFormat[DirectoryConfig] = Json.format[DirectoryConfig]
implicit val format3: OFormat[SourceTypeConfig] = flat.oformat((__ \ "sourceType").format[String])

ScalaJson implicit Write, found: Any required: play.api.libs.json.Json.JsValueWrapper

I am building a web app using Scala / Play Framework and Reactive Mongo and I want the models to be defined in the database instead of having them hardcoded.
To do so, I am writing a class EntityInstance taking a Sequence of FieldInstance :
case class EntityInstance(fields: Seq[FieldInstance])
I am trying to accept fields from any types and to convert them to Json : example
new FieldInstance("name", "John") | json: { "name": "John" }
new FieldInstance("age", 18) | json: { "age": 18 }
At the moment I am trying to accept Strings, Booleans and Integers and if the type is not supported I write some error :
new FieldInstance("profilePicture", new Picture("john.jpg") | json: { "profilePicture": "Unsupported type
I wrote a FieldInstance class taking a fieldName as a String and a value as any type. As soon as that class is instantiated I cast the value to a known type or to the String describing the error.
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => v
case v: String => v
case v: Boolean => v
case _ => "Unrecognized type"
}
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) = Json.obj(
fieldInstance.fieldName -> fieldInstance.value
)
}
}
I created a companion object with an implicit Write to json so I can call "Json.toJson()" on an instance of FieldInstance and get a json as described on my examples above.
I get an error : found: Any required: play.api.libs.json.Json.JsValueWrapper
I understand that it comes from the fact that my value is of type Any but I thought the cast would change that Any to String || Boolean || Int before hitting the Writer.
PS: Ignore the bad naming of the classes, I could not name EntityInstance and FieldInstance, Entity and Field because these as the classes I use to describe my models.
I found a fix to my problem :
The type matching that I was doing in the class should be done in the implicit Write !
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec
override def toString(): String = "(" + fieldName + "," + value + ")";
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) =
fieldInstance.value match {
case v: Int => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Int])
case v: String => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[String])
case v: Boolean => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Boolean])
case _ => Json.obj(fieldInstance.fieldName -> "Unsupported type")
}
}
}
This code now allows a user to create an EntityInstance with Fields of Any type :
val ei = new EntityInstance(Seq[FieldInstance](new FieldInstance("name", "George"), new FieldInstance("age", 25), new FieldInstance("married", true)))
println("-- TEST ENTITY INSTANCE TO JSON --")
println(Json.toJson(ei))
prints : {"entity":[{"name":"George"},{"age":25},{"married":true}]}
Here is my EntityInstance code if you are trying to test it :
case class EntityInstance(fields: Seq[FieldInstance])
object EntityInstance {
implicit val EntityInstanceWrites = new Writes[EntityInstance] {
def writes(entityInstance: EntityInstance) =
Json.obj("entity" -> entityInstance.fields)
}
}
It is returning a String, Int or Boolean but Json.obj is expecting the value parameter of type (String, JsValueWrapper)
def obj(fields: (String, JsValueWrapper)*): JsObject = JsObject(fields.map(f => (f._1, f._2.asInstanceOf[JsValueWrapperImpl].field)))
a quick fix could be to convert the matched value v with toJson provided the implicit Writes[T] for type T is available (which they are for String, Int and Boolean)
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: Boolean => Json.toJson(v)
case _ => Json.toJson("Unrecognized type")
}
}
If you'd like to see which DefaultWrites are available you can browse them in the play.api.libs.json package in trait DefaultWrites
for example:
/**
* Serializer for Boolean types.
*/
implicit object BooleanWrites extends Writes[Boolean] {
def writes(o: Boolean) = JsBoolean(o)
}

Try to update\fetch Postgres json column into JsValue using anorm [duplicate]

I am using anorm to query and save elements into my postgres database.
I have a json column which I want to read as class of my own.
So for example if I have the following class
case class Table(id: Long, name:String, myJsonColumn:Option[MyClass])
case class MyClass(site: Option[String], user:Option[String])
I am trying to write the following update:
DB.withConnection { implicit conn =>
val updated = SQL(
"""UPDATE employee
|SET name = {name}, my_json_column = {myClass}
|WHERE id = {id}
""".stripMargin)
.on(
'name -> name,
'myClass -> myClass,
'custom -> id
).executeUpdate()
}
}
I also defined a implicit convertor from json to my object
implicit def columnToSocialData: Column[MyClass] = anorm.Column.nonNull[MyClass] { (value, meta) =>
val MetaDataItem(qualified, nullable, clazz) = meta
value match {
case json: org.postgresql.util.PGobject => {
val result = Json.fromJson[MyClass](Json.parse(json.getValue))
result.fold(
errors => Left(TypeDoesNotMatch(s"Cannot convert $value: ${value.asInstanceOf[AnyRef].getClass} to Json for column $qualified")),
valid => Right(valid)
)
}
case _ => Left(TypeDoesNotMatch(s"Cannot convert $value: ${value.asInstanceOf[AnyRef].getClass} to Json for column $qualified"))
}
And the error I get is:
type mismatch;
found : (Symbol, Option[com.MyClass])
required: anorm.NamedParameter
'myClass -> myClass,
^
The solution is just to add the following:
implicit val socialDataToStatement = new ToStatement[MyClass] {
def set(s: PreparedStatement, i: Int, myClass: MyClass): Unit = {
val jsonObject = new org.postgresql.util.PGobject()
jsonObject.setType("json")
jsonObject.setValue(Json.stringify(Json.toJson(myClass)))
s.setObject(i, jsonObject)
}
}
and:
implicit object MyClassMetaData extends ParameterMetaData[MyClass] {
val sqlType = "OTHER"
val jdbcType = Types.OTHER
}