Play JSON Reads[T]: split a JsArray into multiple subsets - json

I have a JSON structure that contains an array of events. The array is "polymorphic" in the sense that there are three possible event types A, B and C:
{
...
"events": [
{ "eventType": "A", ...},
{ "eventType": "B", ...},
{ "eventType": "C", ...},
...
]
}
The three event types don't have the same object structure, so I need different Reads for them. And apart from that, the target case class of the whole JSON document distinguishes between the events:
case class Doc(
...,
aEvents: Seq[EventA],
bEvents: Seq[EventB],
cEvents: Seq[EventC],
...
)
How can I define the internals of Reads[Doc] so that the json array events is split into three subsets which are mapped to aEvents, bEvents and cEvents?
What I tried so far (without being succesful):
First, I defined a Reads[JsArray] to transform the original JsArray to another JsArray that only contains events of a particular type:
def eventReads(eventTypeName: String) = new Reads[JsArray] {
override def reads(json: JsValue): JsResult[JsArray] = json match {
case JsArray(seq) =>
val filtered = seq.filter { jsVal =>
(jsVal \ "eventType").asOpt[String].contains(eventTypeName)
}
JsSuccess(JsArray(filtered))
case _ => JsError("Must be an array")
}
}
Then the idea is to use it like this within Reads[Doc]:
implicit val docReads: Reads[Doc] = (
...
(__ \ "events").read[JsArray](eventReads("A")).andThen... and
(__ \ "events").read[JsArray](eventReads("B")).andThen... and
(__ \ "events").read[JsArray](eventReads("C")).andThen... and
...
)(Doc.apply _)
However, I don't know how to go on from here. I assume the andThen part should look something like this (in case of event a):
.andThen[Seq[EventA]](EventA.reads)
But that doesn't work since I expect the API to create a Seq[EventA] by explicitly passing a Reads[EventA] instead of Reads[Seq[EventA]]. And apart from that, since I've never got it running, I'm not sure if this whole approach is reasonable in the first place.
edit: in case the original JsArray contains unknown event types (e.g. D and E), these types should be ignored and left out from the final result (instead of making the whole Reads fail).

put implicit read for every Event type like
def eventRead[A](et: String, er: Reads[A]) = (__ \ "eventType").read[String].filter(_ == et).andKeep(er)
implicit val eventARead = eventRead("A", Json.reads[EventA])
implicit val eventBRead = eventRead("B", Json.reads[EventB])
implicit val eventCRead = eventRead("C", Json.reads[EventC])
and use Reads[Doc] (folding event list to separate sequences by types and apply result to Doc):
Reads[Doc] = (__ \ "events").read[List[JsValue]].map(
_.foldLeft[JsResult[ (Seq[EventA], Seq[EventB], Seq[EventC]) ]]( JsSuccess( (Seq.empty[EventA], Seq.empty[EventB], Seq.empty[EventC]) ) ){
case (JsSuccess(a, _), v) =>
(v.validate[EventA].map(e => a.copy(_1 = e +: a._1)) or v.validate[EventB].map(e => a.copy(_2 = e +: a._2)) or v.validate[EventC].map(e => a.copy(_3 = e +: a._3)))
case (e, _) => e
}
).flatMap(p => Reads[Doc]{js => p.map(Doc.tupled)})
it will create Doc in one pass through events list
JsSuccess(Doc(List(EventA(a)),List(EventB(b2), EventB(b1)),List(EventC(c))),)
the source data
val json = Json.parse("""{"events": [
| { "eventType": "A", "e": "a"},
| { "eventType": "B", "ev": "b1"},
| { "eventType": "C", "event": "c"},
| { "eventType": "B", "ev": "b2"}
| ]
|}
|""")
case class EventA(e: String)
case class EventB(ev: String)
case class EventC(event: String)

I would model the fact that you store different event types in your JS array as a class hierarchy to keep it type safe.
sealed abstract class Event
case class EventA() extends Event
case class EventB() extends Event
case class EventC() extends Event
Then you can store all your events in a single collection and use pattern matching later to refine them. For example:
case class Doc(events: Seq[Event]) {
def getEventsA: Seq[EventA] = events.flatMap(_ match {
case e: EventA => Some(e)
case _ => None
})
}
Doc(Seq(EventA(), EventB(), EventC())).getEventsA // res0: Seq[EventA] = List(EventA())
For implementing your Reads, Doc will be naturally mapped to the case class, you only need to provide a mapping for Event. Here is what it could look like:
implicit val eventReads = new Reads[Event] {
override def reads(json: JsValue): JsResult[Event] = json \ "eventType" match {
case JsDefined(JsString("A")) => JsSuccess(EventA())
case JsDefined(JsString("B")) => JsSuccess(EventB())
case JsDefined(JsString("C")) => JsSuccess(EventC())
case _ => JsError("???")
}
}
implicit val docReads = Json.reads[Doc]
You can then use it like this:
val jsValue = Json.parse("""
{
"events": [
{ "eventType": "A"},
{ "eventType": "B"},
{ "eventType": "C"}
]
}
""")
val docJsResults = docReads.reads(jsValue) // docJsResults: play.api.libs.json.JsResult[Doc] = JsSuccess(Doc(List(EventA(), EventB(), EventC())),/events)
docJsResults.get.events.length // res1: Int = 3
docJsResults.get.getEventsA // res2: Seq[EventA] = List(EventA())
Hope this helps.

Related

Add optional property with json transformers in playframework 2.4

I can't understand, how can i add optional property with json transformer.
I want merge two json objects (list and calendars) without or with dynamic list of properties (for example without owner):
calendar1 = {id:1, name: "first", description:"my first calendar", owner: 1}
calendar2 = {id:2, name: "second", owner: 1}
list = [{id: 1, settings: []}, {id: 2, settings: []}]
and result must be
{calendars:
[
{id:1, name: "first", description:"my first calendar", settings: []},
{id:2, name: "second", settings: []}
]
}
I'll assume the following json trees
val calendar1 = Json.parse("""{"id":1, "name": "first", "description":"my first calendar", "owner": 1}""")
val calendar2 = Json.parse("""{"id":2, "name": "second", "owner": 1}""")
You need to add settings to each calendar, then remove the owner if it exists.
Putting a value in branch settings is explained in the documentation
val addSettings = __.json.update((__ \ "settings").json.put(Json.arr()))
Dropping the owner is also explained
val removeOwner = (__ \ "owner").json.prune
Now you can define the transformer to be applied to each of your calendar object
val transformer = addSettings andThen removeOwner
With that in place there are multiple options depending on how your data is actually modeled. If you have a Seq of calendars as in
val calendars = Seq(calendar1, calendar2)
you can do
val normalizedCalendars = calendars.map(_.transform(transformer))
This gives you a Seq[JsResult[JsObject]] which you want to transform into a JsResult[Seq[JsObject]].
I am pretty sure there is a way to do it using play's functional syntax (see play.api.libs.functional and play.api.libs.functional.syntax) but this part of play is not well documented and I haven't gotten around to studying Applicatives yet even though I have a feel for what they do.
Instead, I rely on the following code inspired by scala's Future#sequence
def sequence[A, M[X] <: TraversableOnce[X]](in: M[JsResult[A]])(implicit cbf: CanBuildFrom[M[JsResult[A]], A, M[A]]): JsResult[M[A]] = {
val empty: JsResult[mutable.Builder[A, M[A]]] = JsSuccess(cbf(in))
in.foldLeft(empty) {(jracc,jrel) => (jracc,jrel) match {
case (JsSuccess(builder,_), JsSuccess(a,p)) =>JsSuccess(builder+=a, p)
case (ra#JsError(builderErrors), re#JsError(errors)) =>JsError.merge(ra, re)
case (JsSuccess(_,_), re#JsError(errors)) => re
case (ra#JsError(builderErrors), JsSuccess(_,_)) => ra
}} map (_.result())
}
With that you can write :
val calendarArray = sequence(normalizedCalendars).map(v=>Json.obj("calendars"->JsArray(v)))
which will give you a JsResult[JsObject]. As long as your original calendars are indeed JsObjects you will get a JsSuccess. You can verify the output structure with :
calendarArray.foreach(println)
which returns :
{"calendars":[{"id":1,"name":"first","description":"my first calendar","settings":[]},{"id":2,"name":"second","settings":[]}]}
which is the same as what you asked modulo some whitespace
{
"calendars":[
{"id":1,"name":"first","description":"my first calendar","settings":[]},
{"id":2,"name":"second","settings":[]}
]
}
Start with:
scala> case class Calendar(id:Int,name:String,description:Option[String],owner:Int)
defined class Calendar
scala> case class CalendarRow(id:Int,name:String,description:Option[String],settings:Seq[String]=Seq.empty)
defined class CalendarRow
scala> def append(calendars:Calendar*) = calendars.map(c => CalendarRow(c.id,c.name,c.description))
append: (calendars: Calendar*)Seq[CalendarRow]
scala> val calendar1 = Calendar(1,"first",Option("my first calendar"),1)
calendar1: Calendar = Calendar(1,first,Some(my first calendar),1)
scala> val calendar2 = Calendar(2, "second",None,1)
calendar2: Calendar = Calendar(2,second,None,1)
scala> val list = append(calendar1,calendar2)
list: Seq[CalendarRow] = ArrayBuffer(CalendarRow(1,first,Some(my first calendar),List()), CalendarRow(2,second,None,List()))
Many thanks to #Jean and comments in "Unveiling Play 2.1 Json API - Part 3 : JSON Transformers"
It's hard understanding things like and, andThen, andKeep, keepAnd in JSON transformers for me (I can not find any detailed descriptions with examples), but i found some templates for my question:
Optional property in JSON:
With Reader
(__ \ "id").json.pick[JsString].flatMap{
case id if id.equals(JsString(accountId)) =>
(__ \ "primary").json.put(JsBoolean(true))
case _ =>
Reads.pure(Json.obj())
}
With Json.obj()
(__ \ "id").json.pick[JsString].map{
case id if id.equals(JsString(accountId)) =>
Json.obj("primary" -> true)
case _ =>
Json.obj()
}

How to send Json from client with missing fields for its corresponding Case Class after using Json.format function

I have a case Class and its companion object like below. Now, when I send JSON without id, createdAt and deletedAt fields, because I set them elsewhere, I get [NoSuchElementException: JsError.get] error. It's because I do not set above properties.
How could I achieve this and avoid getting the error?
case class Plan(id: String,
companyId: String,
name: String,
status: Boolean = true,
#EnumAs planType: PlanType.Value,
brochureId: Option[UUID],
lifePolicy: Seq[LifePolicy] = Nil,
createdAt: DateTime,
updatedAt: DateTime,
deletedAt: Option[DateTime]
)
object Plan {
implicit val planFormat = Json.format[Plan]
def fromJson(str: JsValue): Plan = Json.fromJson[Plan](str).get
def toJson(plan: Plan): JsValue = Json.toJson(plan)
def toJsonSeq(plan: Seq[Plan]): JsValue = Json.toJson(plan)
}
JSON I send from client
{
"companyId": "e8c67345-7f59-466d-a958-7c722ad0dcb7",
"name": "Creating First Plan with enum Content",
"status": true,
"planType": "Health",
"lifePolicy": []
}
You can introduce another case class just to handle serialization from request:
like this
case class NewPlan(name: String,
status: Boolean = true,
#EnumAs planType: PlanType.Value,
brochureId: Option[UUID],
lifePolicy: Seq[LifePolicy] = Nil
)
and then use this class to populate your Plan class.
The fundamental issue is that by the time a case class is instantiated to represent your data, it must be well-typed. To shoe horn your example data into your example class, the types don't match because some fields are missing. It's literally trying to call the constructor without enough arguments.
You've got a couple options:
You can make a model that represents the incomplete data (as grotrianster suggested).
You can make the possible missing fields Option types.
You can custom-write the Reads part of your Format to introduce intelligent values or dummy values for the missing ones.
Option 3 might look something like:
// Untested for compilation, might need some corrections
val now: DateTime = ...
val autoId = Reads[JsObject] {
case obj: JsObject => JsSuccess(obj \ 'id match {
case JsString(_) => obj
case _ => obj.transform(
__.update((__ \ 'id).json.put("")) andThen
__.update((__ \ 'createdTime).json.put(now)) andThen
__.update((__ \ 'updatedTime).json.put(now))
)
})
case _ => JsError("JsObject expected")
}
implicit val planFormat = Format[Plan](
autoId andThen Json.reads[Plan],
Json.writes[Plan])
Once you do this once, if the issue is the same for all your other models, you can probably abstract it into some Format factory utility function.
This may be slightly cleaner for autoId:
val autoId = Reads[JsObject] {
// Leave it alone if we have an ID already
case obj: JsObject if (obj \ 'id).asOpt[String].isSome => JsSuccess(obj)
// Insert dummy values if we don't have an `id`
case obj: JsObject => JsSuccess(obj.transform(
__.update((__ \ 'id).json.put("")) andThen
__.update((__ \ 'createdTime).json.put(now)) andThen
__.update((__ \ 'updatedTime).json.put(now))
))
case _ => JsError("JsObject expected")
}

How to use different names when mapping JSON array to Scala object using combinators

Given a JSON array like this one:
{
"success": true,
"data": [
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
],
"additional_data": {
"pagination": {
"start": 0,
"limit": 100,
"more_items_in_collection": false
}
}
}
In my Play 2.2.2 application, using the Scala JSON Reads Combinator, everything works going this way:
implicit val entityReader = Json.reads[Entity]
val futureJson: Future[List[Entity]] = futureResponse.map(
response => (response.json \ "data").validate[List[Entity]].get
The problem now is the key named 'e54cbe3a434d8e6' which I would like to name 'value' in my object:
// This doesn't work, as one might expect
case class Entity(id: Long, title: String, e54cbe3a434d8e6: Long)
// I would like to use 'value' instead of 'e54cbe3a434d8e6'
case class Entity(id: Long, title: String, value: Long)
There is vast information about the combinators here and here but I only want to use a fieldname which is different from the key name in the JSON array. Can some one help me to find a simple way?
I suppose it has got something to do with JSON.writes?!
One simple way without trying to apply transformations on json itself is to define a custom Reads in such a way to handle this:
val json = obj(
"data" -> obj(
"id" -> 600,
"title" -> "test deal",
"e54cbe3a434d8e6" -> 54))
case class Data(id: Long, title: String, value: Int)
val reads = (
(__ \ "id").read[Long] ~
(__ \ "title").read[String] ~
(__ \ "e54cbe3a434d8e6").read[Int] // here you get mapping from your json to Scala case class
)(Data)
def index = Action {
val res = (json \ "data").validate(reads)
println(res) // prints "JsSuccess(Data(600,test deal,54),)"
Ok(json)
}
Another way is to use combinators like this:
... the same json and case class
implicit val generatedReads = reads[Data]
def index = Action {
val res = (json \ "data").validate(
// here we pick value at 'e54cbe3a434d8e6' and put into brand new 'value' branch
__.json.update((__ \ "value").json.copyFrom((__ \ "e54cbe3a434d8e6").json.pick)) andThen
// here we remove 'e54cbe3a434d8e6' branch
(__ \ "e54cbe3a434d8e6").json.prune andThen
// here we validate result with generated reads for our case class
generatedReads)
println(res) // prints "JsSuccess(Data(600,test deal,54),/e54cbe3a434d8e6/e54cbe3a434d8e6)"
Ok(prettyPrint(json))
}

parsing a Json Array in play framework JsObject

I have the following Json:
{
"web-category" : "macaroons",
"sub-categories" : [
{ "name" : "pink" },
{ "name" : "blue" },
{ "name" : "green" }
]
}
I have got it in Play as a JsObject. So I can now successfully do the following:
//(o is the JsObject)
val webCat:Option[String] = (o \ "web-category").asOpt[String]
println(webCat.toString)
>> Some(macaroons)
So far, so good. But how do I access the array Json objects? I have this...
val subCats:Option[JsArray] = (o \ "sub-categories").asOpt[JsArray]
println(subCats.toString)
>> Some([{"name" : "blue"},{"name" : "green"},{"name" : "pink"}])
but what I need is to take the JsArray and get a List of all the names something like this:
List("blue", "green", "pink")
Don't know how to access the JsArray thusly.
my thanks for your help in this.
I'd argue that it's generally a good idea to move from JSON-land to native-Scala-representation-land as early as possible. If obj is your JsObject, for example, you can write this:
val subCategories = (obj \ "sub-categories").as[List[Map[String, String]]]
val names = subCategories.map(_("name"))
Or even:
case class Category(name: String, subs: List[String])
import play.api.libs.functional.syntax._
implicit val categoryReader = (
(__ \ "web-category").read[String] and
(__ \ "sub-categories").read[List[Map[String, String]]].map(_.map(_("name")))
)(Category)
And then:
obj.as[Category]
This latter approach makes error handling even cleaner (e.g. you can just replace as with asOpt at this top level) and composes nicely with other Reads type class instances—if you have a JsArray of these objects, for example, you can just write array.as[List[Category]] and get what you expect.
What Peter said, or:
(o \ "sub-categories" \\ "name").map(_.as[String]).toList
Something like this:
subCats.map( jsarray => jsarray.value.map(jsvalue => (jsvalue \ "name").as[String]).toList)
This will normally return a Option[List[String]]

Scala: Refactoring a case-statement to use for-comprehension

I'm trying to parse the following Json into a Scala object:
{
"oneOf": [
{ "$ref": "..." },
{ "$ref": "..." },
{ "$ref": "..." }
}
The field "oneOf" could also be "anyOf" or "allOf"; it will only be one of these values. I am constructing a case class, ComplexType, using Play's JSON library. The logic is simple; it looks for a given field and reads it if present, otherwise, checks a different field.
(json \ "allOf") match {
case a:JsArray => ComplexType("object", "allOf", a.as[Seq[JsObject]].flatMap(_.values.map(_.as[String])))
case _ =>
(json \ "anyOf") match {
case a:JsArray => ComplexType("object", "anyOf", a.as[Seq[JsObject]].flatMap(_.values.map(_.as[String])))
case _ =>
(json \ "oneOf") match {
case a:JsArray => ComplexType("object", "oneOf", a.as[Seq[JsObject]].flatMap(_.values.map(_.as[String])))
case _ => ComplexType("object", "oneOf", "Unspecified" :: Nil)
}
}
}
I'm not happy with this syntax; even though it works I don't see why I need to have nested match statements if no match is found. I believe a for-comprehension will work well: I can check for (json \ "allOf"), (json \ "oneOf) etc in a guard clause and yield the available result, but not sure how to get the syntax correct.
Is there a more elegant way to build this case class?
Thanks,
Mike
I don't think for-comprehension is useful here.
You could make your code more readable with custom extractors like this:
class Extractor(s: String) {
def unapply(v: JsValue): Option[JsArray] = json \ s match {
case a: JsArray => Some(a)
case _ => None
}
}
val AllOf = new Extractor("allOf")
val AnyOf = new Extractor("anyOf")
val OneOf = new Extractor("oneOf")
val (name, es) = json match {
case AllOf(a) => "allOf" -> Some(a)
case AnyOf(a) => "anyOf" -> Some(a)
case OneOf(a) => "oneOf" -> Some(a)
case _ => "oneOf" -> None
}
val result =
es.
map{ a => ComplexType("object", name, a.as[Seq[JsObject]].flatMap(_.values.map(_.as[String]))) }.
getOrElse("Unspecified" :: Nil)