I have two functions that are similar. One returns a String when successful and the second returns an Option[String] when successful. How can write this in a more elegant way, perhaps calling the first function in the second function? Thanks!
private def readString(j: JsValue, key: String): Validation[String, String] = {
j \ key match {
case j: JsString =>
Success(j.value)
case j: JsUndefined =>
Failure(s"Missing field $key")
case j: JsValue =>
Failure(s"Field $key value is not a string")
}
}
private def readStringOpt(j: JsValue, key: String): Validation[String, Option[String]] = {
(j \ key).as[JsValue] match {
case j: JsString =>
j.value.some.success
case j: JsUndefined =>
None.success
case x =>
Failure(s"Field $key value is not a string")
}
}
First, the .as[JsValue] in your second method isn't really necessary—all it does is look up the Reads instance for JsValue, which just passes through its argument and never fails.
If you want to go the route you suggest above (defining the Opt-less version in terms of the other), Scalaz provides some slightly more concise syntax:
def readString(j: JsValue, key: String): Validation[String, String] =
readStringOpt(j, key).flatMap(_.toSuccess(s"Missing field $key"))
This will give you a deprecation warning in recent versions of Scalaz, however, since Validation does not have a monad instance and its flatMap is kind of a lie. This means you have two options (apart from ignoring the deprecation warning): you can switch to \/, which is monadic, or you can use a fold instead:
def readString(j: JsValue, key: String): Validation[String, String] =
readStringOpt(j, key).fold(_.failure, _.toSuccess(s"Missing field $key"))
Which is a little more verbose, but it puts you on the right side of the validation-is-not-a-monad gods.
Related
I have a requirement to parse a JSON object, using play-json and distinguish between a missing value, a string value and a null value.
So for example I might want to deserialize into the following case class:
case class MyCaseClass(
a: Option[Option[String]]
)
Where the values of 'a' mean:
None - "a" was missing - normal play-json behavipr
Some(Some(String)) - "a" had a string value
Some(None) - "a" had a null value
So examples of the expected behavior are:
{}
should deserialize to myCaseClass(None)
{
"a": null
}
should deserialize as myCaseClass(Some(None))
{
"a": "a"
}
should deserialize as myCaseClass(Some(Some("a"))
I've tried writing custom formatters, but the formatNullable and formatNullableWithDefault methods don't distinguish between a missing and null value, so the code I've written below cannot generate the Some(None) result
object myCaseClass {
implicit val aFormat: Format[Option[String]] = new Format[Option[String]] {
override def reads(json: JsValue): JsResult[Option[String]] = {
json match {
case JsNull => JsSuccess(None) // this is never reached
case JsString(value) => JsSuccess(Some(value))
case _ => throw new RuntimeException("unexpected type")
}
}
override def writes(codename: Option[String]): JsValue = {
codename match {
case None => JsNull
case Some(value) => JsString(value)
}
}
}
implicit val format = (
(__ \ "a").formatNullableWithDefault[Option[String]](None)
)(MyCaseClass.apply, unlift(MyCaseClass.unapply))
}
Am I missing a trick here? How should I go about this? I am very much willing to encode the final value in some other way than an Option[Option[Sting]] for example some sort of case class that encapsulates this:
case class MyContainer(newValue: Option[String], wasProvided: Boolean)
I recently found a reasonable way to do this. I'm using Play 2.6.11 but I'm guessing the approach will transfer to other recent versions.
The following snippet adds three extension methods to JsPath, to read/write/format fields of type Option[Option[A]]. In each case a missing field maps to a None, a null to a Some(None), and a non-null value to a Some(Some(a)) as the original poster requested:
import play.api.libs.json._
object tristate {
implicit class TriStateNullableJsPathOps(path: JsPath) {
def readTriStateNullable[A: Reads]: Reads[Option[Option[A]]] =
Reads[Option[Option[A]]] { value =>
value.validate[JsObject].flatMap { obj =>
path.asSingleJsResult(obj) match {
case JsError(_) => JsSuccess(Option.empty[Option[A]])
case JsSuccess(JsNull, _) => JsSuccess(Option(Option.empty[A]))
case JsSuccess(json, _) => json.validate[A]
.repath(path)
.map(a => Option(Option(a)))
}
}
}
def writeTriStateNullable[A: Writes]: OWrites[Option[Option[A]]] =
path.writeNullable(Writes.optionWithNull[A])
def formatTriStateNullable[A: Format]: OFormat[Option[Option[A]]] =
OFormat(readTriStateNullable[A], writeTriStateNullable[A])
}
}
Like previous suggestions in this thread, this method requires you to write out a JSON format in full using the applicative DSL. It's unfortunately incompatible with the Json.format macro, but it gets you close to what you want. Here's a use case:
import play.api.libs.json._
import play.api.libs.functional.syntax._
import tristate._
case class Coord(col: Option[Option[String]], row: Option[Option[Int]])
implicit val format: OFormat[Coord] = (
(__ \ "col").formatTriStateNullable[String] ~
(__ \ "row").formatTriStateNullable[Int]
)(Coord.apply, unlift(Coord.unapply))
Some examples of writing:
format.writes(Coord(None, None))
// => {}
format.writes(Coord(Some(None), Some(None)))
// => { "col": null, "row": null }
format.writes(Coord(Some(Some("A")), Some(Some(1))))
// => { "col": "A", "row": 1 }
And some examples of reading:
Json.obj().as[Coord]
// => Coord(None, None)
Json.obj(
"col" -> JsNull,
"row" -> JsNull
).as[Coord]
// => Coord(Some(None), Some(None))
Json.obj(
"col" -> "A",
"row" -> 1
).as[Coord]
// => Coord(Some(Some("A")), Some(Some(1)))
As a bonus exercise for the reader, you could probably combine this with a little shapeless to automatically derive codecs and replace the Json.format macro with a different one-liner (albeit one that takes longer to compile).
Following #kflorence suggestion about OptionHandler I was able to get the desired behavior.
implicit def optionFormat[T](implicit tf: Format[T]): Format[Option[T]] = Format(
tf.reads(_).map(r => Some(r)),
Writes(v => v.map(tf.writes).getOrElse(JsNull))
)
object InvertedDefaultHandler extends OptionHandlers {
def readHandler[T](jsPath: JsPath)(implicit r: Reads[T]): Reads[Option[T]] = jsPath.readNullable
override def readHandlerWithDefault[T](jsPath: JsPath, defaultValue: => Option[T])(implicit r: Reads[T]): Reads[Option[T]] = Reads[Option[T]] { json =>
jsPath.asSingleJson(json) match {
case JsDefined(JsNull) => JsSuccess(defaultValue)
case JsDefined(value) => r.reads(value).repath(jsPath).map(Some(_))
case JsUndefined() => JsSuccess(None)
}
}
def writeHandler[T](jsPath: JsPath)(implicit writes: Writes[T]): OWrites[Option[T]] = jsPath.writeNullable
}
val configuration = JsonConfiguration[Json.WithDefaultValues](optionHandlers = InvertedDefaultHandler)
case class RequestObject(payload: Option[Option[String]] = Some(None))
implicit val requestObjectFormat: OFormat[RequestObject] = Json.configured(configuration).format[RequestObject]
Json.parse(""" {} """).as[RequestObject] // RequestObject(None)
Json.parse(""" {"payload": null } """).as[RequestObject] // RequestObject(Some(None))
Json.parse(""" {"payload": "hello" } """).as[RequestObject] // RequestObject(Some(Some(hello)))
So the important parts are:
The readHandlerWithDefault basically flipping how
JsDefined(JsNull) and JsUndefined are handling absent and explicit nulls compared to the original implementation in OptionHandlers.Default
The JsonConfiguration taking both Json.WithDefaultValues and optionHandlers
How the default value is being set. Note the RequestObject.payload's default value
Unfortunately I don't know how to achieve what you want automatically. For now it seems to me that you can't do that with the standard macro. However surprisingly you might achieve a similar result if you are OK with swapping the null and "absent" cases (which I agree is a bit confusing).
Assume class Xxx is defined as (default value is important - this will be the result for the null case)
case class Xxx(a: Option[Option[String]] = Some(None))
and you provide following implicit Reads:
implicit val optionStringReads:Reads[Option[String]] = new Reads[Option[String]] {
override def reads(json: JsValue) = json match {
case JsNull => JsSuccess(None) // this is never reached
case JsString(value) => JsSuccess(Some(value))
case _ => throw new RuntimeException("unexpected type")
}
}
implicit val xxxReads = Json.using[Json.WithDefaultValues].reads[Xxx]
Then for a test data:
val jsonNone = "{}"
val jsonNull = """{"a":null}"""
val jsonVal = """{"a":"abc"}"""
val jsonValues = List(jsonNone, jsonNull, jsonVal)
jsonValues.foreach(jsonString => {
val jsonAst = Json.parse(jsonString)
val obj = Json.fromJson[Xxx](jsonAst)
println(s"'$jsonString' => $obj")
})
the output is
'{}' => JsSuccess(Xxx(Some(None)),)
'{"a":null}' => JsSuccess(Xxx(None),)
'{"a":"abc"}' => JsSuccess(Xxx(Some(Some(abc))),)
So
absent attribute is mapped onto Some(None)
null is mapped onto None
Value is mapped onto Some(Some(value))
This is clumsy and a bit unexpected by a developer, but at least this distinguishes all 3 choices. The reason why null and "absent" choices are swapped is that the only way I found to distinguish those cases is to have the value in the target class to be declared as Option and with default value at the same time and in that case the default value is what the "absent" case is mapped to; and unfortunately you can't control the value that null is mapped onto - it is always None.
I have a JsArray which contains JsValue objects representing two different types of entities - some of them represent nodes, the other part represents edges.
On the Scala side, there are already case classes named Node and Edge whose supertype is Element. The goal is to transform the JsArray (or Seq[JsValue]) to a collection that contains the Scala types, e.g. Seq[Element] (=> contains objects of type Node and Edge).
I have defined Read for the case classes:
implicit val nodeReads: Reads[Node] = // ...
implicit val edgeReads: Reads[Edge] = // ...
Apart from that, there is the first step of a Read for the JsArray itself:
implicit val elementSeqReads = Reads[Seq[Element]](json => json match {
case JsArray(elements) => ???
case _ => JsError("Invalid JSON data (not a json array)")
})
The part with the question marks is responsible for creating a JsSuccess(Seq(node1, edge1, ...) if all elements of the JsArray are valid nodes and edges or a JsError if this is not the case.
However, I'm not sure how to do this in an elegant way.
The logic to distinguish between nodes and edges could look like this:
def hasType(item: JsValue, elemType: String) =
(item \ "elemType").asOpt[String] == Some(elemType)
val result = elements.map {
case n if hasType(n, "node") => // use nodeReads
case e if hasType(e, "edge") => // use edgeReads
case _ => JsError("Invalid element type")
}
The thing is that I don't know how to deal with nodeReads / edgeReads at this point. Of course I could call their validate method directly, but then result would have the type Seq[JsResult[Element]]. So eventually I would have to check if there are any JsError objects and delegate them somehow to the top (remember: one invalid array element should lead to a JsError overall). If there are no errors, I still have to produce a JsSuccess[Seq[Element]] based on result.
Maybe it would be a better idea to avoid the calls to validate and work temporarily with Read instances instead. But I'm not sure how to "merge" all of the Read instances at the end (e.g. in simple case class mappings, you have a bunch of calls to JsPath.read (which returns Read) and in the end, validate produces one single result based on all those Read instances that were concatenated using the and keyword).
edit: A little bit more information.
First of all, I should have mentioned that the case classes Node and Edge basically have the same structure, at least for now. At the moment, the only reason for separate classes is to gain more type safety.
A JsValue of an element has the following JSON-representation:
{
"id" : "aet864t884srtv87ae",
"type" : "node", // <-- type can be 'node' or 'edge'
"name" : "rectangle",
"attributes": [],
...
}
The corresponding case class looks like this (note that the type attribute we've seen above is not an attribute of the class - instead it's represented by the type of the class -> Node).
case class Node(
id: String,
name: String,
attributes: Seq[Attribute],
...) extends Element
The Read is as follows:
implicit val nodeReads: Reads[Node] = (
(__ \ "id").read[String] and
(__ \ "name").read[String] and
(__ \ "attributes").read[Seq[Attribute]] and
....
) (Node.apply _)
everything looks the same for Edge, at least for now.
Try defining elementReads as
implicit val elementReads = new Reads[Element]{
override def reads(json: JsValue): JsResult[Element] =
json.validate(
Node.nodeReads.map(_.asInstanceOf[Element]) orElse
Edge.edgeReads.map(_.asInstanceOf[Element])
)
}
and import that in scope, Then you should be able to write
json.validate[Seq[Element]]
If the structure of your json is not enough to differentiate between Node and Edge, you could enforce it in the reads for each type.
Based on a simplified Node and Edge case class (only to avoid any unrelated code confusing the answer)
case class Edge(name: String) extends Element
case class Node(name: String) extends Element
The default reads for these case classes would be derived by
Json.reads[Edge]
Json.reads[Node]
respectively. Unfortunately since both case classes have the same structure these reads would ignore the type attribute in the json and happily translate a node json into an Edge instance or the opposite.
Lets have a look at how we could express the constraint on type all by itself :
def typeRead(`type`: String): Reads[String] = {
val isNotOfType = ValidationError(s"is not of expected type ${`type`}")
(__ \ "type").read[String].filter(isNotOfType)(_ == `type`)
}
This method builds a Reads[String] instance which will attempt to find a type string attribute in the provided json. It will then filter the JsResult using the custom validation error isNotOfType if the string parsed out of the json doesn't matched the expected type passed as argument of the method. Of course if the type attribute is not a string in the json, the Reads[String] will return an error saying that it expected a String.
Now that we have a read which can enforce the value of the type attribute in the json, all we have to do is to build a reads for each value of type which we expect and compose it with the associated case class reads. We can used Reads#flatMap for that ignoring the input since the parsed string is not useful for our case classes.
object Edge {
val edgeReads: Reads[Edge] =
Element.typeRead("edge").flatMap(_ => Json.reads[Edge])
}
object Node {
val nodeReads: Reads[Node] =
Element.typeRead("node").flatMap(_ => Json.reads[Node])
}
Note that if the constraint on type fails the flatMap call will be bypassed.
The question remains of where to put the method typeRead, in this answer I initially put it in the Element companion object along with the elementReads instance as in the code below.
import play.api.libs.json._
trait Element
object Element {
implicit val elementReads = new Reads[Element] {
override def reads(json: JsValue): JsResult[Element] =
json.validate(
Node.nodeReads.map(_.asInstanceOf[Element]) orElse
Edge.edgeReads.map(_.asInstanceOf[Element])
)
}
def typeRead(`type`: String): Reads[String] = {
val isNotOfType = ValidationError(s"is not of expected type ${`type`}")
(__ \ "type").read[String].filter(isNotOfType)(_ == `type`)
}
}
This is actually a pretty bad place to define typeRead :
- it has nothing specific to Element
- it introduces a circular dependency between the Elementcompanion object and both Node and Edge companion objects
I'll let you think up of the correct location though :)
The specification proving it all works together :
import org.specs2.mutable.Specification
import play.api.libs.json._
import play.api.data.validation.ValidationError
class ElementSpec extends Specification {
"Element reads" should {
"read an edge json as an edge" in {
val result: JsResult[Element] = edgeJson.validate[Element]
result.isSuccess should beTrue
result.get should beEqualTo(Edge("myEdge"))
}
"read a node json as an node" in {
val result: JsResult[Element] = nodeJson.validate[Element]
result.isSuccess should beTrue
result.get should beEqualTo(Node("myNode"))
}
}
"Node reads" should {
"read a node json as an node" in {
val result: JsResult[Node] = nodeJson.validate[Node](Node.nodeReads)
result.isSuccess should beTrue
result.get should beEqualTo(Node("myNode"))
}
"fail to read an edge json as a node" in {
val result: JsResult[Node] = edgeJson.validate[Node](Node.nodeReads)
result.isError should beTrue
val JsError(errors) = result
val invalidNode = JsError.toJson(Seq(
(__ \ "type") -> Seq(ValidationError("is not of expected type node"))
))
JsError.toJson(errors) should beEqualTo(invalidNode)
}
}
"Edge reads" should {
"read a edge json as an edge" in {
val result: JsResult[Edge] = edgeJson.validate[Edge](Edge.edgeReads)
result.isSuccess should beTrue
result.get should beEqualTo(Edge("myEdge"))
}
"fail to read a node json as an edge" in {
val result: JsResult[Edge] = nodeJson.validate[Edge](Edge.edgeReads)
result.isError should beTrue
val JsError(errors) = result
val invalidEdge = JsError.toJson(Seq(
(__ \ "type") -> Seq(ValidationError("is not of expected type edge"))
))
JsError.toJson(errors) should beEqualTo(invalidEdge)
}
}
val edgeJson = Json.parse(
"""
|{
| "type":"edge",
| "name":"myEdge"
|}
""".stripMargin)
val nodeJson = Json.parse(
"""
|{
| "type":"node",
| "name":"myNode"
|}
""".stripMargin)
}
if you don't want to use asInstanceOf as a cast you can write the
elementReads instance like so :
implicit val elementReads = new Reads[Element] {
override def reads(json: JsValue): JsResult[Element] =
json.validate(
Node.nodeReads.map(e => e: Element) orElse
Edge.edgeReads.map(e => e: Element)
)
}
unfortunately, you can't use _ in this case.
The below code compiles, but throws an error: Exception in thread "main" scala.MatchError:[{"id":6430758,"name":...] (of class play.api.libs.json.JsArray). How can I read JSON for the given link by taking the items list in it and only 5 elements?
import play.api.libs.json._
def getProjects: List[Map[String, Any]] = {
val iter = getJSON("https://api.github.com/search/repositories?q=scala")
val json: JsValue = Json.parse(iter.get mkString "\n")
val projects = (json \ "items") match {
case l: List[Map[String, Any]] => l take 5
}
projects
}
def getJSON(url: String): Try[Iterator[String]] =
Try(Source.fromURL(url).getLines) recover {
case e: FileNotFoundException =>
throw new AppException(s"Requested page does not exist: ${e.getMessage}.")
case e: MalformedURLException =>
throw new AppException(s"Please make sure to enter a valid URL: ${e.getMessage}.")
case _ => throw new AppException("An unexpected error has occurred.")
}
Since you're using Play, you should work within its JsValue abstraction rather than jumping out to a Map[String, Any].
The reason your match is failing is because json \ "items" isn't a Map[String, Any], it's a JsValue. Ideally, you know the structure of your JSON (what your schema for project is) and you can deserialize to that:
case class Project(id: Long, name: String, ...)
object Project {
implicit val fmt = Json.format[Project]
}
val projects = WS.get("https://api.github.com/search/repositories?q=scala").map { response =>
response.json.validate[Map[String, Project]].map(_ take 5)
}
That leaves you with a Future[JsResult[Map[String, Project]]]. The outer type is Future because the operation is inherently asynchronous, JsResult will be either a JsSuccess with your Map[String, Project] or a JsError containing the reason(s) your JSON couldn't be validated.
It feels quick and dirty, but if that's really what you're wanting to do then you can try:
val listOfMaps: Seq[Map[String, String]] =
(res1 \ "items").as[JsArray].value.map { jsobj =>
jsobj.as[JsObject].value.map { case (key, value) =>
key -> value.toString
}
}.take(5)
A better option would be to create a case class with they keys and types that you are expecting and write a Reads to parse the Json to that case class. See https://www.playframework.com/documentation/2.3.x/ScalaJsonCombinators. Then you would have a list of your case class and you can easily take 5 from there.
Im dealing with some code outside of my immediate control, where I need to encode an option[Thing] where the case is as per normal if Thing exists, however the None case must return 'false' rather than null. Can this be accomplished easily? I'm looking at the docs but not having much success.
My code looks like this:
case class Thing(name: String)
case class BiggerThing(stuff: String, thing: Option[Thing])
implict val ThingEncodeJson: EncodeJson[Thing] =
EncodeJson(t => ("name" := t.name ) ->: jEmptyObject)
and the equivalent for BiggerThing, and the json needs to look like:
For a Some:
"thing":{"name": "bob"}
For a None:
"thing": false
but at present the None case gives:
"thing":null
How do I get it to return false? Could someone point me in the right direction please?
Cheers
You just need a custom CodecJson instance for Option[Thing]:
object Example {
import argonaut._, Argonaut._
case class Thing(name: String)
case class BiggerThing(stuff: String, thing: Option[Thing])
implicit val encodeThingOption: CodecJson[Option[Thing]] =
CodecJson(
(thing: Option[Thing]) => thing.map(_.asJson).getOrElse(jFalse),
json =>
// Adopt the easy approach when parsing, that is, if there's no
// `name` property, assume it was `false` and map it to a `None`.
json.get[Thing]("name").map(Some(_)) ||| DecodeResult.ok(None)
)
implicit val encodeThing: CodecJson[Thing] =
casecodec1(Thing.apply, Thing.unapply)("name")
implicit val encodeBiggerThing: CodecJson[BiggerThing] =
casecodec2(BiggerThing.apply, BiggerThing.unapply)("stuff", "thing")
def main(args: Array[String]): Unit = {
val a = BiggerThing("stuff", Some(Thing("name")))
println(a.asJson.nospaces) // {"stuff":"stuff","thing":{"name":"name"}}
val b = BiggerThing("stuff", None)
println(b.asJson.nospaces) // {"stuff":"stuff","thing":false}
}
}
How to encode a BiggerThing without a thing property when thing is None. You need a custom EncodeJson[BiggerThing] instance then:
implicit val decodeBiggerThing: DecodeJson[BiggerThing] =
jdecode2L(BiggerThing.apply)("stuff", "thing")
implicit val encodeBiggerThing: EncodeJson[BiggerThing] =
EncodeJson { biggerThing =>
val thing = biggerThing.thing.map(t => Json("thing" := t))
("stuff" := biggerThing.stuff) ->: thing.getOrElse(jEmptyObject)
}
I'm using Argonaut to parse JSON strings. There is a requirement: If any field is not provided, or is empty or blank, give it the string "not supplied" instead.
I have a solution, but it seems very complicated:
case class User(name: String, home: String)
implicit def UserDecodeJson: DecodeJson[User] = DecodeJson(j => for {
name <- (j --\ "name").as[BlankAsNonSupplied]
home <- (j --\ "home").as[BlankAsNonSupplied]
} yield User(name.value, home.value))
case class BlankAsNonSupplied(value: String)
implicit def BlankAsNonSuppliedDecodeJson: DecodeJson[BlankAsNonSupplied] = DecodeJson.withReattempt(a => {
val v = a.success.map(_.focus).flatMap(_.string)
.filterNot(_.trim.isEmpty)
.map(BlankAsNonSupplied.apply).getOrElse(BlankAsNonSupplied("not supplied"))
DecodeResult.ok(v)
})
You can see the BlankAsNonSuppliedDecodeJson one is very complicated, and hard to understand. Is there any way to make it(or the whole example) simpler?