Scala, Akka, Spray: How to validate json data before processing? - json

I can process this json when all the inputs are valid, i.e with valid keys (including case) and values. The next step is to validate keys and return 400 (Bad Request) if the keys or values are invalid. What is a good way to add this validation?
The API call
POST http://localhost:8080/api/v1/adsession
Content-Type: application/json
body {
"sessionId": "abcd123123123",
"serviceGroup": "1234",
"targetCode": {"zipcodes":"30096,30188","code2":"value2"}
}
Route handler
class AdSessionRoutes(services: Services)(implicit ec: ExecutionContext, log: LoggingContext) extends ApiRoute(services) {
implicit val timeout = Timeout(10 seconds)
val postSession = pathPrefix("adsession") & pathEnd & post
val route: Route = {
withService("adSession") { service =>
postSession {
entity(as[AdSession]) { adSession =>
log.info(s"Processing POST ${adSession}")
val future = (service ? CreateAdSession(adSession)).mapTo[AdSession]
onComplete(future) {
case Success(result) =>
complete(StatusCodes.Created, result)
case Failure(e) =>
log.error(s"Error: ${e.toString}")
complete(StatusCodes.InternalServerError, Message(ApiMessages.UnknownException))
}
}
}
}
}
}
Model object
case class AdSession(
sessionId: String,
serviceGroup: String,
targetCodes: Map[String,String],
id: Option[String] = None)
object AdSessionJsonProtocol extends DefaultJsonProtocol {
implicit val adSessionFormat = jsonFormat4(AdSession)
}
entity(as[AdSession]) does map keys to case class fields, but I'm not sure how to catch those errors. I would like to capture those errors as well as add additional validations and return 400 with valid json error response.

I know this may be a little late but since akka-http-2.4.6, you can put the verification logic inside a case class. Check out this for info on how to do it.

Define your own read and write methods for AdSession like this:
object AdSessionJsonProtocol {
implicit object adSessionJsonFormat extends RootJsonFormat[AdSession] {
override def read(json: JsValue): AdSession = ???
override def write(obj: AdSession): JsValue = ???
}
}
Inside read function you can perform additional validation.
Another question is how to pass collected errors to Spray route. I would like to suggest you to wrap AdSession into Either[String, AdSession], for example:
postSession {
entity(as[Either[String, AdSession]]) { adSession =>
So, now your adSessionJsonFormat will looks like:
implicit object adSessionJsonFormat extends RootJsonFormat[Either[String, AdSession]] {
override def read(json: JsValue): Either[String, AdSession] = {
// json.asJsObject.fields access fields, perform checks
// if everything is OK return Right(AdSession(...))
// otherwise return Lift("Error Message")
}
override def write(obj: Either[String, AdSession]): JsValue = ???
}
But, I think it's possible to solve it in more elegant way using some implicit magic.

Related

How do I write a generic JSON parser in Play 2.7 for Scala that validates inbound requests?

I have a Play 2.7 controller in Scala that validates inbound JSON requests against a case class schema, and reports inbound request payload errors (note that I extracted this sample from a larger codebase, attempting to preserve its correct compilability and functionality, though there may be minor mistakes):
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Failure, Success, Try}
import com.google.inject.Inject
import play.api.libs.json.{JsError, JsPath, JsSuccess, JsValue, Json, JsonValidationError}
import play.api.mvc.{AbstractController, Action, ControllerComponents, Request, Result}
class Controller #Inject() (playEC: ExecutionContext, cc: ControllerComponents) extends AbstractController(cc) {
case class RequestBody(id: String)
implicit val requestBodyFormat = Json.format[RequestBody]
private val tryJsonParser = parse.tolerantText.map(text => Try(Json.parse(text)))(playEC)
private def stringify(path: JsPath, errors: Seq[JsonValidationError]): String = {
s"$path: [${errors.map(x => x.messages.mkString(",") + (if (x.args.size > 0) (": " + x.args.mkString(",")) else "")).mkString(";")}]"
}
private def runWithRequest(rawRequest: Request[Try[JsValue]], method: (RequestBody) => Future[Result]): Future[Result] = {
rawRequest.body match {
case Success(validBody) =>
Json.fromJson[RequestBody](validBody) match {
case JsSuccess(r, _) => method(r)
case JsError(e) => Future.successful(BadRequest(Json.toJson(e.map(x => stringify(x._1, x._2)).head)))
}
case Failure(e) => Future.successful(BadRequest(Json.toJson(e.getMessage.replaceAll("\n", ""))))
}
}
// POST request processor
def handleRequest: Action[Try[JsValue]] = Action(tryJsonParser).async { request: Request[Try[JsValue]] =>
runWithRequest(request, r => {
Future.successful(Ok(r.id))
})
}
}
The validation works like this when sending a POST request to the "handleRequest" endpoint:
with the payload {malformed,,, I get a 400 response back with Unexpected character ('m' (code 109)): was expecting double-quote to start field name at [Source: (String)"{malformed,,"; line: 1, column: 3].
with the payload {} I get a 400 response back with /id: [error.path.missing]
What I would like to do is to make the parsing and validating generic,
moving that logic into a utility class for the cleanest re-use possible in the handleRequest method. For example, something like this:
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Failure, Success, Try}
import com.google.inject.{Inject, Singleton}
import play.api.{Configuration, Logging}
import play.api.libs.json.{JsError, JsPath, JsSuccess, JsValue, Json, JsonValidationError}
import play.api.mvc.{AbstractController, Action, ActionBuilderImpl, AnyContent, BodyParsers, ControllerComponents, Request, Result}
object ParseAction {
// TODO: how do I work this in?
val tryJsonParser = parse.tolerantText.map(text => Try(Json.parse(text)))(playEC)
}
class ParseAction #Inject()(parser: BodyParsers.Default)(implicit ec: ExecutionContext) extends ActionBuilderImpl(parser) {
private def stringify(path: JsPath, errors: Seq[JsonValidationError]): String = {
s"$path: [${errors.map(x => x.messages.mkString(",") + (if (x.args.size > 0) (": " + x.args.mkString(",")) else "")).mkString(";")}]"
}
// TODO: how do I make this method generic?
override def invokeBlock[A](rawRequest: Request[A], block: (A) => Future[Result]) = {
rawRequest.body match {
case Success(validBody) =>
Json.fromJson[A](validBody) match {
case JsSuccess(r, _) => block(r).getFuture
case JsError(e) => Future.successful(BadRequest(Json.toJson(e.map(x => stringify(x._1, x._2)).head)))
}
case Failure(e) => Future.successful(BadRequest(Json.toJson(e.getMessage.replaceAll("\n", ""))))
}
}
}
class Controller #Inject() (cc: ControllerComponents) extends AbstractController(cc) {
case class RequestBody(id: String)
implicit val requestBodyFormat = Json.format[RequestBody]
// route processor
def handleRequest = ParseAction.async { request: RequestBody =>
Future.successful(Ok(r.id))
}
}
I'm aware that this code doesn't compile as-is due to blatant Scala and Play API misuse rather than just small coding mistakes. I tried pulling from Play's own documentation about Action composition, but I have not had success in getting things right, so I left all of the pieces around hoping someone can help me to work them together into something that works.
How can I change this second code sample around to compile and behave functionally identically to the first code sample?
I archived similar goal by using implicit class for ActionBuilder:
trait ActionBuilderImplicits {
implicit class ExActionBuilder[P](actionBuilder: ActionBuilder[Request, P])(implicit cc: ControllerComponents) {
def validateJson[A](implicit executionContext: ExecutionContext, reads: Reads[A]): ActionBuilder[Request, A] = {
actionBuilder(cc.parsers.tolerantJson.validate(jsValue => {
jsValue.validate.asEither.left
.map(errors => BadRequest(JsError.toJson(errors)))
}))
}
}
}
object ActionBuilderImplicits extends ActionBuilderImplicits
Then in controller you can import ActionBuilderImplicits and use it as
Action.validateJson[A].async { request =>
processingService.process(request.body)
}
Here is request.body already type of A

Play 2.4 ActionBuilder / ActionFunction, BodyParsers and JSON

In a Play 2.4 project, I created an Action that do three things.
check that a specific header is present. If not, returns a HTTP error.
use the header value to authenticate the user (a authentication function will be passed to that Action). If the auth fails returns an HTTP error
parse the JSON body to a case class and give it to the Action block code.
To do that, I used the Play Action composition mecanism explain in this page : https://www.playframework.com/documentation/2.4.x/ScalaActionsComposition
but more specifically, the final result I wanted is explained here :
https://www.playframework.com/documentation/2.4.x/ScalaActionsComposition#Putting-it-all-together
I succeeded to write this:
package actions
import play.api.libs.concurrent.Execution.Implicits._
import play.api.libs.json._
import play.api.mvc.Results._
import play.api.mvc.{WrappedRequest, _}
import scala.concurrent.Future
object Actions {
case class WithApiKeyRequest[A](apiKey: String, request: Request[A]) extends WrappedRequest[A](request)
case class ParsedJsonRequest[A](parsed: Any, request: Request[A]) extends WrappedRequest[A](request)
def AuthenticatedAndParsed[T, A](authencation: String => Future[_])(implicit reader: Reads[T]): ActionBuilder[ParsedJsonRequest] =
WithApiKeyHeaderAction andThen AuthentificationAction(authencation) andThen JsonAction
private[this] def WithApiKeyHeaderAction = new ActionBuilder[WithApiKeyRequest] {
override def invokeBlock[A](request: Request[A], block: (WithApiKeyRequest[A]) => Future[Result]): Future[Result] =
request.headers.get("ApiKey") match {
case Some(apiKey: String) => block(WithApiKeyRequest(apiKey, request))
case _ => Future.successful { BadRequest(Json.obj("errors" -> "ApiKey header needed")) }
}
}
private[this] def AuthentificationAction(authencationFunction: String => Future[_]) = new ActionFilter[WithApiKeyRequest] {
override protected def filter[A](request: WithApiKeyRequest[A]): Future[Option[Result]] =
authencationFunction(request.apiKey)
.map { _ => None } // Do not filter the request
.recover { case _ => Some(Unauthorized) }
}
private[this] def JsonAction[T](implicit reader: Reads[T]) = new ActionBuilder[ParsedJsonRequest] {
composeParser(BodyParsers.parse.json)
override def invokeBlock[A](request: Request[A], block: (ParsedJsonRequest[A]) => Future[Result]): Future[Result] = {
request.body.asInstanceOf[JsValue].validate[T].fold(
errors => Future { BadRequest(Json.obj("errors" -> JsError.toJson(errors))) },
(parsedJson: T) => block(ParsedJsonRequest(parsedJson, request))
)
}
}
}
It seems to work well but it's not perfect because I'm force to use the Any type in the case class ParsedJsonRequest[A](parsed: Any, request: Request[A]) because it seems that I can't do that:
case class ParsedJsonRequest[T, A](parsed: T, request: Request[A])
Is it possible to do that ?
Do you think I can improve my solution ? How ?
My question is not about how to do Action composition. I understand how it works and I succeeded to write my ActionBuilders and my wanted composition.
My question is about how to improve my composition.
Thanks
Jules
Rather than making a new ActionBuilder for a JsonRequest, I would simply use the AuthentificationAction ActionBuilder and pass it a json BodyParser:
AuthentificationAction(parse.json) {
request => // Note that the request has type Request[JsValue]
doStuffWithJson(request.body)
}
Any action using this builder, will get a Request[JsValue] rather than a Request[AnyContent].

How to return json from Play Scala controller?

I would like to know that how can I return json response data from Play(2.2.x) Scala controller class to display on my view page ? I have json objects in Postgresql database(table name: "test" and having: id and name). Please provide me any solutions for it.
I have tried the following cases(a and b), but I am not sure why I am not getting the response(like: names) on my controller, so I can show them on my view page ? since I am very new to Play/Scala and Postgresql.
case a. If I give like:
model:
def getTestValuesFromTable() = {
DB.withConnection { implicit connection =>
val selectJson =SQL("select name from test").on().apply().collect {
case Row(id:Long, Some(name:String)) =>
new TestContent(name)
}
//.head
//JsObject(selectJson().map { row => row[Long]("id").toString -> JsString(row[String]("name")) }.toSeq)
}
}
controller:
def getTest = Action {
val response = TestContent.getTestValuesFromTable()
Ok("Done")
//Ok(response)
}
Output is: Done(application is executing fine without any exceptions, of course json data is not coming since I am returning: Done only, so getting output: "Done")
case b. If I do like this: getting error: not enough arguments for method apply: (n: Int)models.Testname in trait LinearSeqOptimized. Unspecified value parameter n. I really not sure how can I get my response for it ?
controller:
def getTest = Action {
val response = TestContent.getTestValuesFromTable()
// Ok("Done")
Ok(response)
}
model:
def getTestValuesFromTable(): JsValue = {
DB.withConnection { implicit connection =>
val selectJson = SQL("select * from test")
JsObject(selectJson().map { row => row[Long]("id").toString -> JsString(row[String]("name")) }.toSeq)
// val selectJson =SQL("select name from test").on().apply().collect {
// case Row(id:Long, Some(name:String)) =>
// new TestContent(name)
// }
//.head
JsObject(selectJson().map { row => row[Long]("id").toString -> JsString(row[String]("name")) }.toSeq)//not enough arguments for method apply: (n: Int)models.Testname in trait LinearSeqOptimized. Unspecified value parameter n.
}
}
Please let me know how to get my response ?
getJsonValuesFromTable method return nothing (Unit). To fix it change definition of this method to
def getJsonValuesFromTable(testContent: TestContent) = {
or explicitly setting type:
def getJsonValuesFromTable(testContent: TestContent): Unit = {
Also as a next step to let client know that you are returning json, you should set content type:
Ok(Json.obj(response)).as("application/json")

How to marshal different response types in spray?

Consider an http service that can return two json as response:
successful
{
"yourField":"value"
}
failure
{
"errorCode": 3
}
To deal with these json's I need to create 2 case classes case class RespSucc(yourField:String) and
case class RespFail(errorCode:Int).
For now I have to to something like that:
//unmarshal is spray.httpx.ResponseTransformation#unmarshal
if (response.entity.asString.contains("errorCode")) {
unmarshal[RespSucc].apply(response)
}
else {
unmarshal[RespFail].apply(response)
}
Is there an api to parse these classes automatically without any if? E.g. can unmarshaller looks into json fields and select approriate case class?
spray-json supports Either which is a very useful data type for this kind of situations.
val data = unmarshal[Either[RespFail, RespSucc]].apply(response)
// You can match it
data match {
case Left(err) => handleError(err)
case Right(suc) => handleSuccess(suc)
}
// Or you can fold it (I prefer folding)
data.fold(err => handleError(err), suc => handleSuccess(suc))
You can try something like this:
trait Resp
case class RespSucc(yourField: String) extends Resp
case class RespFail(errorCode: Int) extends Resp
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object ColorJsonFormat extends RootJsonFormat[Resp] {
def write(r: Resp) = r match {
case s: RespSucc =>
JsObject("yourField" -> JsString(s.yourField))
case f: RespFail =>
JsObject("errorCode" -> JsNumber(f.errorCode))
}
def read(value: JsValue) = value.asJsObject.getFields("yourField", "errorCode") match {
case Seq(JsString(yourField)) => RespSucc(yourField)
case Seq(JsNumber(errorCode)) => RespFail(errorCode.intValue())
case _ => deserializationError("Resp expected")
}
}
}
import MyJsonProtocol._
unmarshal[Resp](entitySucc) //Right(RespSucc(abc))
unmarshal[Resp](entityFail) //Right(RespFail(3))

Spray: Marshalling UUID to JSON

I'm having some problems marshalling from UUID to JSON
def complete[T <: AnyRef](status: StatusCode, obj: T) = {
r.complete(status, obj) // Completes the Request with the T obj result!
}
^
The signature of my class:
trait PerRequest extends Actor
with Json4sSupport
with Directives
with UnrestrictedStash
with ActorLogging {
val json4sFormats = DefaultFormats.
This gives me :
"id": {
"mostSigBits": -5052114364077765000,
"leastSigBits": -7198432257767597000
},
instead of:
"id": "b9e348c0-cc7f-11e3-9c1a-0800200c9a66"
So, how can I add a UUID format to json4sFormats to marshall UUID's correctly?? In other cases I mix in with a trait that have this function:
implicit object UuidJsonFormat extends RootJsonFormat[UUID] {
def write(x: UUID) = JsString(x.toString)
def read(value: JsValue) = value match {
case JsString(x) => UUID.fromString(x)
case x => deserializationError("Expected UUID as JsString, but got " + x)
}
}
But here I'm not able to because I don't have declared a spray.json.RootJsonReader and/or spray.json.RootJsonWriter for every type T and does not compile. (See complete function T <: AnyRef)
Thanks.
I solved it! If someone has the same problem take a look here
I defined my own UUID Serializer as follows:
class UUIDSerializer extends CustomSerializer[UUID](format => (
{
case JObject(JField("mostSigBits", JInt(s)) :: JField("leastSigBits", JInt(e)) :: Nil) =>
new UUID(s.longValue, e.longValue)
},
{
case x: UUID => JObject(JField("id", JString(x.toString)))
}
))
And now it's working!