Play: How to modify data while deserializing Json - json

Here below is a simple case class that holds user information:
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class User(
val username: String,
val password: String
)
... and here is the companion object that provides functionality for serializing/deserializing User objects to/from JSON:
object User {
implicit val userWrites: Writes[User] = (
(__ \ 'username).write[String] ~
(__ \ 'password).write[String]
)(unlift(User.unapply))
implict val userReads: Reads[User] = (
(__ \ 'username).read[String] ~
(__ \ 'password).read[String] // how do I invoke the password hasher here?
)(User.apply(_, _))
}
Let's assume User objects should always contain hashed passwords but incoming JSON always provide plaintext... how can I enhance my Reads so that it invokes the password hasher while deserializing?

You can use the map method on Reads:
def doSomething(s: String) = s * 3
case class User(val username: String, val password: String)
object User {
implicit val userWrites: Writes[User] = (
(__ \ 'username).write[String] ~
(__ \ 'password).write[String]
)(unlift(User.unapply))
implicit val userReads: Reads[User] = (
(__ \ 'username).read[String] ~
(__ \ 'password).read[String].map(doSomething)
)(User.apply(_, _))
}
And then:
scala> Json.parse("""{"username": "foo", "password": "bar"}""").as[User]
res0: User = User(foo,barbarbar)
Note, though, that this means that your Reads and Writes are no longer inverses of each other.

Related

Play Json: custom reads one field

Let's say I have to write custom Reads[Person] for Person class:
import play.api.libs.functional.syntax._
implicit val personReads: Reads[Person] = (
(__ \ "name").read[String] and // or ~
(__ \ "age").readNullable[Int]
) ((name, age) => Person(name = name, age = age))
it works like a charm, really (no).
But what can I do when there is only one field in json object?
The core of Reads and Writes is in functional syntax which transforms these "parse" steps.
The following does not compile:
import play.api.libs.functional.syntax._
implicit val personReads: Reads[Person] = (
(__ \ "name").read[String]
)(name => Person(name))
Could you advice how to deal with it?
Option 1: Reads.map
import play.api.libs.json._
case class Person(name: String)
object PlayJson extends App {
implicit val readsPeson: Reads[Person] =
(__ \ "name").read[String].map(name => Person(name))
val rawString = """{"name": "John"}"""
val json = Json.parse(rawString)
val person = json.as[Person]
println(person)
}
Option 2: Json.reads
import play.api.libs.json._
case class Person(name: String)
object Person {
implicit val readsPerson = Json.reads[Person]
}
object PlayJson extends App {
val rawString = """{"name": "John"}"""
val json = Json.parse(rawString)
val person = json.as[Person]
println(person)
}

Play JSON combinator validates at least a field is specified

I have a scenario that when parsing a json to a case class like this
implicit val userRead: Reads[User] = (
(__ \ "name").read[String] ~
(__ \ "email").readNullable[String] ~
(__ \ "phone").readNullable[String] ~
Reads.pure(None)
)(User.apply _)
I don't require both email and phone to be available, but at least one of them have to be available.
In my case class definition, I can prevent the case that both of them are empty with
case class User(name: String, email: Option[String], phone: Option[String], id: Option[Long] = None) {
require(email.nonEmpty || phone.nonEmpty, "must have at least an email or phone number")
}
However, doing this will generate an exception, and a 500 status is responded, when this should be a 400 error due to user input.
I can of course manually perform the verification in my controller, but I wonder if there is a cleaner way to do this.
I can only suggest writing you own "reads" manually
case class A(i: Option[Int], j: Option[Int])
implicit val reads: Reads[A] = new Reads[A] {
override def reads(json: JsValue): JsResult[A] = {
(for {
i <- (json \ "i").validateOpt[Int]
j <- (json \ "j").validateOpt[Int]
} yield A(i, j))
.filter(JsError("both values can't be empty"))(a ⇒ a.j.nonEmpty || a.i.nonEmpty)
}
}
And to test:
scala> Json.parse("""{ "i" : 1} """).validate[A]
res4: play.api.libs.json.JsResult[A] = JsSuccess(A(Some(1),None),)
scala> Json.parse("""{} """).validate[A]
res5: play.api.libs.json.JsResult[A] = JsError(List((,List(ValidationError(List(from and to in range can't be both empty),WrappedArray())))))
You can achieve this with just a small addition to your existing code (.filter(ValidationError("must have at least an email or phone number"))(u => u.email.isDefined || u.phone.isDefined)):
case class User(name: String, email: Option[String], phone: Option[String], id: Option[Long] = None)
implicit val userRead: Reads[User] = (
(__ \ "name").read[String] ~
(__ \ "email").readNullable[String] ~
(__ \ "phone").readNullable[String] ~
Reads.pure(None)
)(User.apply _).filter(ValidationError("must have at least an email or phone number"))(u => u.email.isDefined || u.phone.isDefined)
Now, to show that this works:
scala> Json.parse("""{ "name" : "foobar"} """).validate[User]
res3: play.api.libs.json.JsResult[User] = JsError(List((,List(ValidationError(List(must have at least an email or phone number),WrappedArray())))))
scala> Json.parse("""{ "name" : "foobar", "email":"test"} """).validate[User]
res4: play.api.libs.json.JsResult[User] = JsSuccess(User(foobar,Some(test),None,None),)
scala> Json.parse("""{ "name" : "foobar", "phone":"test"} """).validate[User]
res5: play.api.libs.json.JsResult[User] = JsSuccess(User(foobar,None,Some(test),None),)

Play: How to transform JSON while writing/reading it to/from MongoDB

Here is an simple JSON I want to write/read to/from MongoDB:
{
"id": "ff59ab34cc59ff59ab34cc59",
"name": "Joe",
"surname": "Cocker"
}
Before storing it in MongoDB, "ff59ab34cc59ff59ab34cc59" has to be transformed into an ObjectID and id renamed to _id... so given the following Reads, how do I achieve that?
val personReads: Reads[JsObject] = (
(__ \ 'id).read[String] ~ // how do I rename id to _id AND transform "ff59ab34cc59ff59ab34cc59" to an ObjectID?
(__ \ 'name).read[String] ~
(__ \ 'surname).read[String]
) reduce
And of course, I also need the contrary for my Writes, i.e. renaming _id to id and transforming an ObjectID to plain text in the format "ff59ab34cc59ff59ab34cc59".
JsonExtensions
I usually have a JsExtensions object in my application which looks like the following :
import reactivemongo.bson.BSONObjectID
object JsonExtensions {
import play.api.libs.json._
def withDefault[A](key: String, default: A)(implicit writes: Writes[A]) = __.json.update((__ \ key).json.copyFrom((__ \ key).json.pick orElse Reads.pure(Json.toJson(default))))
def copyKey(fromPath: JsPath,toPath:JsPath ) = __.json.update(toPath.json.copyFrom(fromPath.json.pick))
def copyOptKey(fromPath: JsPath,toPath:JsPath ) = __.json.update(toPath.json.copyFrom(fromPath.json.pick orElse Reads.pure(JsNull)))
def moveKey(fromPath:JsPath, toPath:JsPath) =(json:JsValue)=> json.transform(copyKey(fromPath,toPath) andThen fromPath.json.prune).get
}
For a simple model
case class SOUser(name:String,_id:BSONObjectID)
you can write your json serializer/deserializer like this:
object SOUser{
import play.api.libs.json.Format
import play.api.libs.json.Json
import play.modules.reactivemongo.json.BSONFormats._
implicit val soUserFormat= new Format[SOUser]{
import play.api.libs.json.{JsPath, JsResult, JsValue}
import JsonExtensions._
val base = Json.format[SOUser]
private val publicIdPath: JsPath = JsPath \ 'id
private val privateIdPath: JsPath = JsPath \ '_id \ '$oid
def reads(json: JsValue): JsResult[SOUser] = base.compose(copyKey(publicIdPath, privateIdPath)).reads(json)
def writes(o: SOUser): JsValue = base.transform(moveKey(privateIdPath,publicIdPath)).writes(o)
}
}
here is what you get in the console :
scala> import reactivemongo.bson.BSONObjectID
import reactivemongo.bson.BSONObjectID
scala> import models.SOUser
import models.SOUser
scala> import play.api.libs.json.Json
import play.api.libs.json.Json
scala>
scala> val user = SOUser("John Smith", BSONObjectID.generate)
user: models.SOUser = SOUser(John Smith,BSONObjectID("52d00fd5c912c061007a28d1"))
scala> val jsonUser=Json.toJson(user)
jsonUser: play.api.libs.json.JsValue = {"name":"John Smith","id":"52d00fd5c912c061007a28d1","_id":{}}
scala> Json.prettyPrint(jsonUser)
res0: String =
{
"name" : "John Smith",
"id" : "52d00fd5c912c061007a28d1",
"_id" : { }
}
scala> jsonUser.validate[SOUser]
res1: play.api.libs.json.JsResult[models.SOUser] = JsSuccess(SOUser(John Smith,BSONObjectID("52d00fd5c912c061007a28d1")),/id)
Applying this to your example
val _personReads: Reads[JsObject] = (
(__ \ 'id).read[String] ~
(__ \ 'name).read[String] ~
(__ \ 'surname).read[String]
).reduce
Doesn't compile by default, I guess you meant to write :
val _personReads: Reads[(String,String,String)] = (
(__ \ 'id).read[String] ~
(__ \ 'name).read[String] ~
(__ \ 'surname).read[String]
).tupled
in which case you can do the following
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
import play.modules.reactivemongo.json.BSONFormats._
import reactivemongo.bson.BSONObjectID
def copyKey(fromPath: JsPath,toPath:JsPath ) = __.json.update(toPath.json.copyFrom(fromPath.json.pick))
val json = """{
"id": "ff59ab34cc59ff59ab34cc59",
"name": "Joe",
"surname": "Cocker"
}"""
val originaljson = Json.parse(json)
val publicIdPath: JsPath = JsPath \ 'id
val privateIdPath: JsPath = JsPath \ '_id \ '$oid
val _personReads: Reads[(BSONObjectID,String,String)] = (
(__ \ '_id).read[BSONObjectID] ~
(__ \ 'name).read[String] ~
(__ \ 'surname).read[String]
).tupled
val personReads=_personReads.compose(copyKey(publicIdPath,privateIdPath))
originaljson.validate(personReads)
// yields res5: play.api.libs.json.JsResult[(reactivemongo.bson.BSONObjectID, String, String)] = JsSuccess((BSONObjectID("ff59ab34cc59ff59ab34cc59"),Joe,Cocker),/id)
or you meant that you want to move the value of the id key to _id \ $oid which can be accomplished with
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
import play.modules.reactivemongo.json.BSONFormats._
import reactivemongo.bson.BSONObjectID
def copyKey(fromPath: JsPath,toPath:JsPath ) = __.json.update(toPath.json.copyFrom(fromPath.json.pick))
val json = """{
"id": "ff59ab34cc59ff59ab34cc59",
"name": "Joe",
"surname": "Cocker"
}"""
val originaljson = Json.parse(json)
val publicIdPath: JsPath = JsPath \ 'id
val privateIdPath: JsPath = JsPath \ '_id \ '$oid
originaljson.transform(copyKey(publicIdPath,privateIdPath) andThen publicIdPath.json.prune)
You can't have a BSONObjectID in there for now since you are manipulating object from the JsValue type hierarchy. When you pass json to reactivemongo it is converted to a BSONValue. A JsObject will be converted to a BSONDocument. if the JsObject contains a path for _id\$oid this path will be converted to a BSONObjectId automatically and it will be stored as an ObjectID in mongodb.
The original question is really about reactivemongo's (sgodbillon et al) treatment of the native mongodb _id. The chosen answer is instructive and correct but obliquely addresses the OP's concern whether "it will all just work".
Thanks to https://github.com/ReactiveMongo/ReactiveMongo-Play-Json/blob/e67e507ecf2be48cc71e429919f7642ea421642c/src/main/scala/package.scala#L241-L255, I believe it will.
import scala.concurrent.Await
import scala.concurrent.duration.Duration
import play.api.libs.concurrent.Execution.Implicits.defaultContext
import play.api.libs.functional.syntax._
import play.api.libs.json._
import play.modules.reactivemongo.json.collection.JSONCollection
import reactivemongo.api._
import reactivemongo.bson.BSONObjectID
import reactivemongo.play.json._
case class Person(
id: BSONObjectID,
name: String,
surname: String
)
implicit val PersonFormat: OFormat[Person] = (
(__ \ "_id").format[BSONObjectID] and
(__ \ "name").format[String] and
(__ \ "surname").format[String]
)(Person.apply, unlift(Person.unapply))
val driver = new reactivemongo.api.MongoDriver
val connection = driver.connection(List("localhost"))
val db = connection.db("test")
val coll = db.collection[JSONCollection]("persons")
coll.drop(false)
val id = BSONObjectID.generate()
Await.ready(coll.insert(Person(id, "Joe", "Cocker")), Duration.Inf)
Await.ready(coll.find(Json.obj()).one[Person] map { op => assert(op.get.id == id, {}) }, Duration.Inf)
The above is a minimal working example of your case class using id and the database storing it as _id. Both are instantiated as 12-byte BSONObjectIDs.

How do I handle my json writes to update existing objects in ReactiveMongo?

In my current Play2 project I have implemented ReactiveMongo to store my user objects.
User:
case class User(override var _id: Option[BSONObjectID] = None,
override var created: Option[DateTime] = None,
override var updated: Option[DateTime] = None,
firstName: String,
lastName: String,
email: String,
emailValidated: Boolean,
phoneNumber: String,
lastLogin: DateTime,
linkedProviders: Seq[LinkedProvider],
userRoles: Seq[UserRole.UserRole]) extends TemporalModel {
}
import EnumUtils.enumReads
implicit val userRoleReads = enumReads(UserRole)
import mongo_models.LinkedProvider.linkedProviderReads
implicit val userReads: Reads[User] = (
(__ \ "_id").read[Option[BSONObjectID]] and
(__ \ "created").read[Option[DateTime]] and
(__ \ "updated").read[Option[DateTime]] and
(__ \ "firstName").read[String] and
(__ \ "lastName").read[String] and
(__ \ "email").read[String] and
(__ \ "emailValidated").read[Boolean] and
(__ \ "phoneNumber").read[String] and
(__ \ "lastLogin").read[DateTime] and
(__ \ "linkedProviders").read(list[LinkedProvider](linkedProviderReads)) and
(__ \ "userRoles").read(list[UserRole.UserRole])
)(User.apply _)
import EnumUtils.enumWrites
import mongo_models.LinkedProvider.linkedProviderWrites
import play.api.libs.json.Writes._
implicit val userWrites: Writes[User] = (
(__ \ "_id").write[Option[BSONObjectID]] and
(__ \ "created").write[Option[DateTime]] and
(__ \ "updated").write[Option[DateTime]] and
(__ \ "firstName").write[String] and
(__ \ "lastName").write[String] and
(__ \ "email").write[String] and
(__ \ "emailValidated").write[Boolean] and
(__ \ "phoneNumber").write[String] and
(__ \ "lastLogin").write[DateTime] and
(__ \ "linkedProviders").write(Writes.traversableWrites[LinkedProvider](linkedProviderWrites)) and
(__ \ "userRoles").write(Writes.traversableWrites[UserRole.UserRole])
)(unlift(User.unapply))
LinkedProvider:
case class LinkedProvider(userId: String,
providerId: String,
authMethod: String,
avatarUrl: String,
oAuth1Info: Option[OAuth1Info] = None,
oAuth2Info: Option[OAuth2Info] = None,
passwordInfo: Option[PasswordInfo] = None) {
}
object LinkedProvider {
implicit val o1 = Json.format[OAuth1Info]
implicit val o2 = Json.format[OAuth2Info]
implicit val p = Json.format[PasswordInfo]
implicit val linkedProviderReads: Reads[LinkedProvider] = (
(__ \ "userId").read[String] and
(__ \ "providerId").read[String] and
(__ \ "authMethod").read[String] and
(__ \ "avatarUrl").read[String] and
(__ \ "oAuth1Info").read[Option[OAuth1Info]] and
(__ \ "oAuth2Info").read[Option[OAuth2Info]] and
(__ \ "passwordInfo").read[Option[PasswordInfo]]
)(LinkedProvider.apply _)
implicit val linkedProviderWrites: Writes[LinkedProvider] = (
(__ \ "userId").write[String] and
(__ \ "providerId").write[String] and
(__ \ "authMethod").write[String] and
(__ \ "avatarUrl").write[String] and
(__ \ "oAuth1Info").write[Option[OAuth1Info]] and
(__ \ "oAuth2Info").write[Option[OAuth2Info]] and
(__ \ "passwordInfo").write[Option[PasswordInfo]]
)(unlift(LinkedProvider.unapply))
}
As you can see I have implemented reads and writes so that the objects are correctly stored
in the mongoDB instance.
When doing the insert with a new object everything is working like a charm and the object structure is correctly saved and retrieved from mongoDB.
What I'm having issues with is figuring out how to handle the updates.
So let say I want to update some values on my user object like this:
val userForUpdate = user.copy(
firstName = identity.firstName,
lastName = identity.lastName,
email = identity.email.getOrElse(""),
lastLogin = DateTime.now()
)
And then call my update method:
UserDAO.update(user._id.get.stringify, userForUpdate, false)
updateMethod:
def update(id: String, document: T, upsert: Boolean)(implicit writer: Writes[T]): Future[Either[ServiceException, T]] = {
document.updated = Some(new DateTime())
Logger.debug(s"Updating document: [collection=$collectionName, id=$id, document=$document]")
Recover(collection.update(DBQueryBuilder.id(id), DBQueryBuilder.set(document))) {
document
}
}
DBQueryBuilder.set() method:
def set[T](data: T)(implicit writer: Writes[T]): JsObject = Json.obj("$set" -> data)
This will cause this error:
[error] application - DB operation failed: [message=DatabaseException['Mod on _id not allowed' (code = 10148)]]
[error] application - DatabaseException: [code=10148, isNotAPrimaryError=false]
since the writes, (__ \ "_id").write[Option[BSONObjectID]], states that also the _id field should be serialized when calling the DBQueryBuilder.set() method. Updating _id is as we know not allowed and should definitely not be done in this case.
So my question is: How should I handle this? I guess there some smart Scala way that do not involve me writing the entire "$set" -> query? Maybe creating a better DBQueryBuilder? Maybe defining another writes definition?
Please gimmie your best shot and remember that I'm a Scala newbie so be gentle =) !
Instead of
def set[T](data: T)(implicit writer: Writes[T]): JsObject = Json.obj("$set" -> data)
You could use something like that
def set[T](data: T)(implicit writer: Writes[T]): JsObject = {
val data = Json.obj(data)
val dataWithoutId = data.remove("_id")
Json.obj("$set" -> dataWithoutId)
}
There should be something in the Play Json library to remove the _id field (but it may not be "remove(...)"...)
Since you use option you would probably write something like:
val userForUpdate = user.copy(
id = None,
firstName = identity.firstName,
lastName = identity.lastName,
email = identity.email.getOrElse(""),
lastLogin = DateTime.now()
)
The best way is probably to use a common trait for all classes representing Mongo documents (maybe the T you use?) and add on the trait the possibility to remove the id (set it to none).
This way you could call:
def set[T](data: T)(implicit writer: Writes[T]): JsObject = Json.obj("$set" -> data.copy(_id = None))
How about this?
def set[T](data: T)(implicit writer: Writes[T]): JsObject = {
val jsonTransformer = (__ \ '$set \ '_id).json.prune
val dataWithId = Json.obj("$set" -> data)
val withoutId = dataWithId.transform(jsonTransformer).get
withoutId
}

Create implicit json read for List collection which might be missing from input json

I am following play-salat (github.com/leon/play-salat) to create a model for a json input and save to mongodb. How can I create the implicit json read for List collection which might be missing in the input json? The following code gives me the validation error if the 'positions' is missing from input json.
case class LIProfile(
id: ObjectId = new ObjectId,
positions: List[Position] = Nil
)
object LIProfile extends LIProfileDAO with LIProfileJson
trait LIProfileDAO extends ModelCompanion[LIProfile, ObjectId] {
def collection = mongoCollection("liprofiles")
val dao = new SalatDAO[LIProfile, ObjectId](collection) {}
// Indexes
collection.ensureIndex(DBObject("emailAddress" -> 1), "li_profile_email", unique = true)
// Queries
def findOneByEmail(email: String): Option[LIProfile] = dao.findOne(MongoDBObject("emailAddress" -> email))
}
trait LIProfileJson {
implicit val liprofileJsonWrite = new Writes[LIProfile] {
def writes(p: LIProfile): JsValue = {
Json.obj(
"id" -> p.id,
"positions" -> p.positions
)
}
}
implicit val liprofileJsonRead = (
(__ \ 'id).read[ObjectId] ~
(__ \ 'positions).read (
(__ \ 'values).read[List[Position]]
)
)(LIProfile.apply _)
}
Use readNullable in order to retrieve an Option and map that to the contained list or the empty list.
implicit val liprofileJsonRead = (
(__ \ 'id).read[ObjectId] ~
(__ \ 'positions).readNullable (
(__ \ 'values).read[List[Position]]
).map {
case Some(l) => l
case None => Nil
}
)(LIProfile)
or even shorter:
implicit val liprofileJsonRead = (
(__ \ 'id).read[ObjectId] ~
(__ \ 'positions).readNullable (
(__ \ 'values).read[List[Position]]
).map { l => l.getOrElse(Nil) }
)(LIProfile)
I'm not quite sure what imports you really need here, my code compiles using:
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._