Access body as JSON in ActionBuilder - json

I'm using Playframework 2.3.X. I'm trying to construct an action function that validates fields in a JSON request.body. The use case is to build "blocks" of validation that I can then chain together.
However, I can't seem to access the request.body as a JSON inside the action builder. The following doesn't compile. The compiler can't resolve "asJson":
def ValidateJsonBodyAction = new ActionBuilder[Request] {
def invokeBlock[A](request: Request[A], block: (Request[A]) => Future[Result]): Future[Result] = {
val body= request.body.asJson
}
}
UPDATE: It could also be that I'm approaching this the wrong way. I'm new to play, so alternative approaches are also welcome.
UPDATE: Yes, it appears that a may be doing things the wrong way. I'm not the only one with this issue. See https://github.com/playframework/playframework/issues/3387

I think you might have to pattern match on the body type (which I admit is not a beautiful solution):
import scala.concurrent.Future.{successful => immediate}
def JsonValidator(validator: Reads[JsValue]) = new ActionBuilder[Request] {
def jsonBody[A](body: A): Option[JsValue] = body match {
case js: JsValue => Some(js)
case any: AnyContent => any.asJson
case _ => None
}
override def invokeBlock[A](request: Request[A], block: (Request[A]) => Future[Result]): Future[Result] = {
jsonBody(request.body) match {
case Some(js) => js.validate(validator) match {
case JsSuccess(_, _) => block(request)
case JsError(e) => immediate(BadRequest(s"Bad Json: $e"))
}
case None => immediate(UnsupportedMediaType("Bad content type: expected Json"))
}
}
}
This is basically what happens behind the scenes when you, for example, bind a form on a request with an arbitrary and unknown body.
On the other hand you might find it better to just write a specific BodyParser to validate your data, e.g:
def jsonFormatParser(validator: Reads[JsValue]): BodyParser[JsValue] = parse.json.validate { js =>
js.validate(validator) match {
case JsSuccess(_, _) => Right(js)
case JsError(e) => Left(BadRequest(s"Json didn't have the right format: $e"))
}
}
// A validator which checks the JSON contains a "hello" path which
// then contains a "world" path
val helloWorldValidator: Reads[JsValue] = Reads.required(__ \ "hello" \ "world")
def helloWorldAction = Action(jsonFormatParser(helloWorldValidator)) { implicit request =>
Ok("Got the right type of JSON")
}
It depends on your use-case. Since JSON validators (Reads[JsValue]) are nicely composable via andThen that might be your best bet.

Related

Deserialize JSON distinguising missing and null values

I have a requirement to parse a JSON object, using play-json and distinguish between a missing value, a string value and a null value.
So for example I might want to deserialize into the following case class:
case class MyCaseClass(
a: Option[Option[String]]
)
Where the values of 'a' mean:
None - "a" was missing - normal play-json behavipr
Some(Some(String)) - "a" had a string value
Some(None) - "a" had a null value
So examples of the expected behavior are:
{}
should deserialize to myCaseClass(None)
{
"a": null
}
should deserialize as myCaseClass(Some(None))
{
"a": "a"
}
should deserialize as myCaseClass(Some(Some("a"))
I've tried writing custom formatters, but the formatNullable and formatNullableWithDefault methods don't distinguish between a missing and null value, so the code I've written below cannot generate the Some(None) result
object myCaseClass {
implicit val aFormat: Format[Option[String]] = new Format[Option[String]] {
override def reads(json: JsValue): JsResult[Option[String]] = {
json match {
case JsNull => JsSuccess(None) // this is never reached
case JsString(value) => JsSuccess(Some(value))
case _ => throw new RuntimeException("unexpected type")
}
}
override def writes(codename: Option[String]): JsValue = {
codename match {
case None => JsNull
case Some(value) => JsString(value)
}
}
}
implicit val format = (
(__ \ "a").formatNullableWithDefault[Option[String]](None)
)(MyCaseClass.apply, unlift(MyCaseClass.unapply))
}
Am I missing a trick here? How should I go about this? I am very much willing to encode the final value in some other way than an Option[Option[Sting]] for example some sort of case class that encapsulates this:
case class MyContainer(newValue: Option[String], wasProvided: Boolean)
I recently found a reasonable way to do this. I'm using Play 2.6.11 but I'm guessing the approach will transfer to other recent versions.
The following snippet adds three extension methods to JsPath, to read/write/format fields of type Option[Option[A]]. In each case a missing field maps to a None, a null to a Some(None), and a non-null value to a Some(Some(a)) as the original poster requested:
import play.api.libs.json._
object tristate {
implicit class TriStateNullableJsPathOps(path: JsPath) {
def readTriStateNullable[A: Reads]: Reads[Option[Option[A]]] =
Reads[Option[Option[A]]] { value =>
value.validate[JsObject].flatMap { obj =>
path.asSingleJsResult(obj) match {
case JsError(_) => JsSuccess(Option.empty[Option[A]])
case JsSuccess(JsNull, _) => JsSuccess(Option(Option.empty[A]))
case JsSuccess(json, _) => json.validate[A]
.repath(path)
.map(a => Option(Option(a)))
}
}
}
def writeTriStateNullable[A: Writes]: OWrites[Option[Option[A]]] =
path.writeNullable(Writes.optionWithNull[A])
def formatTriStateNullable[A: Format]: OFormat[Option[Option[A]]] =
OFormat(readTriStateNullable[A], writeTriStateNullable[A])
}
}
Like previous suggestions in this thread, this method requires you to write out a JSON format in full using the applicative DSL. It's unfortunately incompatible with the Json.format macro, but it gets you close to what you want. Here's a use case:
import play.api.libs.json._
import play.api.libs.functional.syntax._
import tristate._
case class Coord(col: Option[Option[String]], row: Option[Option[Int]])
implicit val format: OFormat[Coord] = (
(__ \ "col").formatTriStateNullable[String] ~
(__ \ "row").formatTriStateNullable[Int]
)(Coord.apply, unlift(Coord.unapply))
Some examples of writing:
format.writes(Coord(None, None))
// => {}
format.writes(Coord(Some(None), Some(None)))
// => { "col": null, "row": null }
format.writes(Coord(Some(Some("A")), Some(Some(1))))
// => { "col": "A", "row": 1 }
And some examples of reading:
Json.obj().as[Coord]
// => Coord(None, None)
Json.obj(
"col" -> JsNull,
"row" -> JsNull
).as[Coord]
// => Coord(Some(None), Some(None))
Json.obj(
"col" -> "A",
"row" -> 1
).as[Coord]
// => Coord(Some(Some("A")), Some(Some(1)))
As a bonus exercise for the reader, you could probably combine this with a little shapeless to automatically derive codecs and replace the Json.format macro with a different one-liner (albeit one that takes longer to compile).
Following #kflorence suggestion about OptionHandler I was able to get the desired behavior.
implicit def optionFormat[T](implicit tf: Format[T]): Format[Option[T]] = Format(
tf.reads(_).map(r => Some(r)),
Writes(v => v.map(tf.writes).getOrElse(JsNull))
)
object InvertedDefaultHandler extends OptionHandlers {
def readHandler[T](jsPath: JsPath)(implicit r: Reads[T]): Reads[Option[T]] = jsPath.readNullable
override def readHandlerWithDefault[T](jsPath: JsPath, defaultValue: => Option[T])(implicit r: Reads[T]): Reads[Option[T]] = Reads[Option[T]] { json =>
jsPath.asSingleJson(json) match {
case JsDefined(JsNull) => JsSuccess(defaultValue)
case JsDefined(value) => r.reads(value).repath(jsPath).map(Some(_))
case JsUndefined() => JsSuccess(None)
}
}
def writeHandler[T](jsPath: JsPath)(implicit writes: Writes[T]): OWrites[Option[T]] = jsPath.writeNullable
}
val configuration = JsonConfiguration[Json.WithDefaultValues](optionHandlers = InvertedDefaultHandler)
case class RequestObject(payload: Option[Option[String]] = Some(None))
implicit val requestObjectFormat: OFormat[RequestObject] = Json.configured(configuration).format[RequestObject]
Json.parse(""" {} """).as[RequestObject] // RequestObject(None)
Json.parse(""" {"payload": null } """).as[RequestObject] // RequestObject(Some(None))
Json.parse(""" {"payload": "hello" } """).as[RequestObject] // RequestObject(Some(Some(hello)))
So the important parts are:
The readHandlerWithDefault basically flipping how
JsDefined(JsNull) and JsUndefined are handling absent and explicit nulls compared to the original implementation in OptionHandlers.Default
The JsonConfiguration taking both Json.WithDefaultValues and optionHandlers
How the default value is being set. Note the RequestObject.payload's default value
Unfortunately I don't know how to achieve what you want automatically. For now it seems to me that you can't do that with the standard macro. However surprisingly you might achieve a similar result if you are OK with swapping the null and "absent" cases (which I agree is a bit confusing).
Assume class Xxx is defined as (default value is important - this will be the result for the null case)
case class Xxx(a: Option[Option[String]] = Some(None))
and you provide following implicit Reads:
implicit val optionStringReads:Reads[Option[String]] = new Reads[Option[String]] {
override def reads(json: JsValue) = json match {
case JsNull => JsSuccess(None) // this is never reached
case JsString(value) => JsSuccess(Some(value))
case _ => throw new RuntimeException("unexpected type")
}
}
implicit val xxxReads = Json.using[Json.WithDefaultValues].reads[Xxx]
Then for a test data:
val jsonNone = "{}"
val jsonNull = """{"a":null}"""
val jsonVal = """{"a":"abc"}"""
val jsonValues = List(jsonNone, jsonNull, jsonVal)
jsonValues.foreach(jsonString => {
val jsonAst = Json.parse(jsonString)
val obj = Json.fromJson[Xxx](jsonAst)
println(s"'$jsonString' => $obj")
})
the output is
'{}' => JsSuccess(Xxx(Some(None)),)
'{"a":null}' => JsSuccess(Xxx(None),)
'{"a":"abc"}' => JsSuccess(Xxx(Some(Some(abc))),)
So
absent attribute is mapped onto Some(None)
null is mapped onto None
Value is mapped onto Some(Some(value))
This is clumsy and a bit unexpected by a developer, but at least this distinguishes all 3 choices. The reason why null and "absent" choices are swapped is that the only way I found to distinguish those cases is to have the value in the target class to be declared as Option and with default value at the same time and in that case the default value is what the "absent" case is mapped to; and unfortunately you can't control the value that null is mapped onto - it is always None.

How to take 5 elements of JsArray in Scala?

The below code compiles, but throws an error: Exception in thread "main" scala.MatchError:[{"id":6430758,"name":...] (of class play.api.libs.json.JsArray). How can I read JSON for the given link by taking the items list in it and only 5 elements?
import play.api.libs.json._
def getProjects: List[Map[String, Any]] = {
val iter = getJSON("https://api.github.com/search/repositories?q=scala")
val json: JsValue = Json.parse(iter.get mkString "\n")
val projects = (json \ "items") match {
case l: List[Map[String, Any]] => l take 5
}
projects
}
def getJSON(url: String): Try[Iterator[String]] =
Try(Source.fromURL(url).getLines) recover {
case e: FileNotFoundException =>
throw new AppException(s"Requested page does not exist: ${e.getMessage}.")
case e: MalformedURLException =>
throw new AppException(s"Please make sure to enter a valid URL: ${e.getMessage}.")
case _ => throw new AppException("An unexpected error has occurred.")
}
Since you're using Play, you should work within its JsValue abstraction rather than jumping out to a Map[String, Any].
The reason your match is failing is because json \ "items" isn't a Map[String, Any], it's a JsValue. Ideally, you know the structure of your JSON (what your schema for project is) and you can deserialize to that:
case class Project(id: Long, name: String, ...)
object Project {
implicit val fmt = Json.format[Project]
}
val projects = WS.get("https://api.github.com/search/repositories?q=scala").map { response =>
response.json.validate[Map[String, Project]].map(_ take 5)
}
That leaves you with a Future[JsResult[Map[String, Project]]]. The outer type is Future because the operation is inherently asynchronous, JsResult will be either a JsSuccess with your Map[String, Project] or a JsError containing the reason(s) your JSON couldn't be validated.
It feels quick and dirty, but if that's really what you're wanting to do then you can try:
val listOfMaps: Seq[Map[String, String]] =
(res1 \ "items").as[JsArray].value.map { jsobj =>
jsobj.as[JsObject].value.map { case (key, value) =>
key -> value.toString
}
}.take(5)
A better option would be to create a case class with they keys and types that you are expecting and write a Reads to parse the Json to that case class. See https://www.playframework.com/documentation/2.3.x/ScalaJsonCombinators. Then you would have a list of your case class and you can easily take 5 from there.

Spray Client I want to return String from Json response

Sorry, but I am new to Scala. I have read about Futures and Akka, however I still have issue returning a string for my method.
I have a method getAuthString which should return Authentication String(or Token).
I have used spray Jsonsupport and I can print the result
def getToken(url: String, username: String , password: String) = Future[String]{
import MyJsonProtocol._
import spray.httpx.SprayJsonSupport._
val pipeline: HttpRequest => Future[AuthTokenResult[Entry]] = (addCredentials(BasicHttpCredentials(username, password))
~> sendReceive
~> unmarshal[AuthTokenResult[Entry]]
)
val myfutureResponse: Future[AuthTokenResult[Entry]] = pipeline(Get(url))
myfutureResponse onComplete {
case Success(AuthTokenResult(Entry(Content(authString)):: _)) => println(authString)
case Failure(error) => println("An error has occured: " + error.getMessage)
}
this unmarshal the json and print the desired authString. However, printing is no good to me. I know onComplete returns unit. I want to return authString so that I can use it somewhere else with another request. I think I will have to use flatmap or map, but I am not sure how. I need my method to return authString or error.
You don't want to return a String, you want to return a Future[String] - once something is async the only way to make it not async is to block, and that's (usually) a waste, making the whole async-ness pointless.
I'm not sure why you're wrapping the whole thing in a Future either - the trivial bits of computation can happen on their own, there's little value in forcing them onto a separate thread. So you want something like:
def getToken(url: String, ...): Future[String] = {
...
val myFutureResponse: Future[AuthTokenResult[Entry]] = ...
myFutureResponse map {
case AuthTokenResult(Entry(Content(authString))::_) => authString
}
}
So you use map to transform a Future into another Future with a computation. This will "pass through" errors, but you can use something like recover or recoverWith if you want to handle them in a particular way.
Then when you want to use your Future[String] in a Spray route, you can use the onSuccess or onComplete directives:
val myRoute = (path("/somewhere") & parameter("authData") {
authData =>
onSuccess(getToken(authData)) {
authToken =>
complete("Authed as " + authToken)
}
}
This will use the Future in a proper async, reactive way, without blocking.

How to use argonaut to parse an optional custom field?

I defined a user which has a "video" information:
case class User(name:String, video: Option[Video])
case class Video(title:String, url:String)
And we have such a json:
{
"name": "somename",
"video": {
"title": "my-video",
"url": "https://youtube.com/watch?v=123123"
}
}
I can use such code to parse it:
implicit def DecodeUser: DecodeJson[User] = for {
name <- as[String]("name")
video <- as[Option[Video]]("video")
} yield User(name, video)
implicit def DecodeVideo: DecodeJson[Option[Video]] = for {
titleOpt <- as[Option[String]]("title")
urlOpt <- as[Option[String]]("url")
} yield (titleOpt, urlOpt) match {
case (Some(title), Some(url)) => Video(title, url)
case _ => None
}
From the DecodeVideo, you can see I just want to provide the video only if both "title" and "url" provided.
It's working well if the json contains "video" section. But if it doesn't, argonaut will report that "video" section is not provided.
How to make "video" optional?
I can't seem to figure out how your code integrates with argonaut. All instances of the method as[T] don't seem to match the signature that you're using. Anyways, here's a similar problem and the solution:
object Test {
case class Video(title: String, url: String)
def test(titleOptIn: List[Option[String]], urlOptIn: List[Option[String]]): List[Option[Video]] = {
for {
titleOpt <- titleOptIn
urlOpt <- urlOptIn
} yield (titleOpt, urlOpt) match {
case (Some(title), Some(url)) => Some(Video(title, url))
case _ => None.asInstanceOf[Option[Video]]
}
}
def main(args: Array[String]): Unit = {
test(List(Some("t")), List(Some("u"), None)).foreach(println)
}
}
// Has Output:
// Some(Video(t,u))
// None
Specifically note that the yield comprehension should return an Option[String] since your yield is likely to be wrapping the result in DecodeJson just like my example wraps it in a List. Note that asInstanceOf on None is optional; IntelliJ complains if it's not there but it actually compiles just fine.
The specific thing that I believe you're missing is wrapping Video in Some.

Combine Query String Parameters with JSON Entity in Spray 1.2.0 Routing

Using Spray Routing, I would like have a single directive that merges the query string parameters with a JSON entity, with both being optional. I would want to have this happen before any marshalling happens.
Something like this:
val myRoute = mergedParametersAndEntity(as[DomainSpecificClass]) { myobj =>
// ... code code code ...
complete(OK, myobj.someMethod)
}
Basically what I was hoping for was the following behavior:
When someone does a request like:
POST /v1/resource?a=helloQS&b=helloQS
Content-Type: application/json
{"a":"helloFromJson","c":"helloFromJson"}
Then the object above (myobj) could contain the keys:
a -> helloFromJson
b -> helloQS
c -> helloFromJson
In other words, items specified in the request body would override things in the query string. I know this must be possible somehow, but I simply cannot figure out how to do it. Can anyone help?
Thank you!
My suggestion: don't take this approach. It feels dirty. Go back to the drawing board if you can.
With that in mind, you won't be able to interject a merge like you want with the existing JSON marshalling support in Spray. You'll need to stitch it together yourself. Something like this should at least point you in the right direction (provided it be the direction you must go):
import org.json4s.JsonAST._
import org.json4s.JsonDSL._
def mergedParametersAndEntity[T](implicit m:Manifest[T]): Directive1[T] = {
entity(as[JObject]).flatMap { jObject =>
parameterMap flatMap { params =>
val left = JObject(params.mapValues { v => JString(v) }.toSeq : _*)
provide(left.merge(jObject).extract[T])
}
}
}
If anyone is still wondering how to do this here's a small Spray directive that allows copying param values to the JSON before unmarshalling it. It uses JSON lenses (https://github.com/jrudolph/json-lenses) to modify the request body before unmarshalling.
def updateJson(update: Update): Directive0 = {
mapRequest((request: HttpRequest) => {
request.entity match {
case Empty => request
case NonEmpty(contentType, data) =>
try {
request.copy(entity = HttpEntity(`application/json`, JsonParser(data.asString).update(update).toString))
}
catch {
case e: ParsingException => request
}
}
})
}
And here's how you use it. Say you want to update a resource with a PUT request and pass the ID from the URL to the unmarshaller (a very common scenario btw):
// Model
case class Thing(id: Long, title: String, description: String, createdAt: DateTime)
// Actor message for the update operation
case class UpdateThing(id: Long, title: String)
// Spray routing
def thingsRoute =
path("things" / Segment) { id =>
put {
updateJson(field("id") ! set[Long](id)) {
entity(as[UpdateThing]) { updateThing =>
complete {
// updateThing now has the ID set
(someActor ? updateThing).mapTo[Thing]
}
}
}
}
}
You can also combine it with the parameters directive to set arbitrary GET params.