I can't understand, how can i add optional property with json transformer.
I want merge two json objects (list and calendars) without or with dynamic list of properties (for example without owner):
calendar1 = {id:1, name: "first", description:"my first calendar", owner: 1}
calendar2 = {id:2, name: "second", owner: 1}
list = [{id: 1, settings: []}, {id: 2, settings: []}]
and result must be
{calendars:
[
{id:1, name: "first", description:"my first calendar", settings: []},
{id:2, name: "second", settings: []}
]
}
I'll assume the following json trees
val calendar1 = Json.parse("""{"id":1, "name": "first", "description":"my first calendar", "owner": 1}""")
val calendar2 = Json.parse("""{"id":2, "name": "second", "owner": 1}""")
You need to add settings to each calendar, then remove the owner if it exists.
Putting a value in branch settings is explained in the documentation
val addSettings = __.json.update((__ \ "settings").json.put(Json.arr()))
Dropping the owner is also explained
val removeOwner = (__ \ "owner").json.prune
Now you can define the transformer to be applied to each of your calendar object
val transformer = addSettings andThen removeOwner
With that in place there are multiple options depending on how your data is actually modeled. If you have a Seq of calendars as in
val calendars = Seq(calendar1, calendar2)
you can do
val normalizedCalendars = calendars.map(_.transform(transformer))
This gives you a Seq[JsResult[JsObject]] which you want to transform into a JsResult[Seq[JsObject]].
I am pretty sure there is a way to do it using play's functional syntax (see play.api.libs.functional and play.api.libs.functional.syntax) but this part of play is not well documented and I haven't gotten around to studying Applicatives yet even though I have a feel for what they do.
Instead, I rely on the following code inspired by scala's Future#sequence
def sequence[A, M[X] <: TraversableOnce[X]](in: M[JsResult[A]])(implicit cbf: CanBuildFrom[M[JsResult[A]], A, M[A]]): JsResult[M[A]] = {
val empty: JsResult[mutable.Builder[A, M[A]]] = JsSuccess(cbf(in))
in.foldLeft(empty) {(jracc,jrel) => (jracc,jrel) match {
case (JsSuccess(builder,_), JsSuccess(a,p)) =>JsSuccess(builder+=a, p)
case (ra#JsError(builderErrors), re#JsError(errors)) =>JsError.merge(ra, re)
case (JsSuccess(_,_), re#JsError(errors)) => re
case (ra#JsError(builderErrors), JsSuccess(_,_)) => ra
}} map (_.result())
}
With that you can write :
val calendarArray = sequence(normalizedCalendars).map(v=>Json.obj("calendars"->JsArray(v)))
which will give you a JsResult[JsObject]. As long as your original calendars are indeed JsObjects you will get a JsSuccess. You can verify the output structure with :
calendarArray.foreach(println)
which returns :
{"calendars":[{"id":1,"name":"first","description":"my first calendar","settings":[]},{"id":2,"name":"second","settings":[]}]}
which is the same as what you asked modulo some whitespace
{
"calendars":[
{"id":1,"name":"first","description":"my first calendar","settings":[]},
{"id":2,"name":"second","settings":[]}
]
}
Start with:
scala> case class Calendar(id:Int,name:String,description:Option[String],owner:Int)
defined class Calendar
scala> case class CalendarRow(id:Int,name:String,description:Option[String],settings:Seq[String]=Seq.empty)
defined class CalendarRow
scala> def append(calendars:Calendar*) = calendars.map(c => CalendarRow(c.id,c.name,c.description))
append: (calendars: Calendar*)Seq[CalendarRow]
scala> val calendar1 = Calendar(1,"first",Option("my first calendar"),1)
calendar1: Calendar = Calendar(1,first,Some(my first calendar),1)
scala> val calendar2 = Calendar(2, "second",None,1)
calendar2: Calendar = Calendar(2,second,None,1)
scala> val list = append(calendar1,calendar2)
list: Seq[CalendarRow] = ArrayBuffer(CalendarRow(1,first,Some(my first calendar),List()), CalendarRow(2,second,None,List()))
Many thanks to #Jean and comments in "Unveiling Play 2.1 Json API - Part 3 : JSON Transformers"
It's hard understanding things like and, andThen, andKeep, keepAnd in JSON transformers for me (I can not find any detailed descriptions with examples), but i found some templates for my question:
Optional property in JSON:
With Reader
(__ \ "id").json.pick[JsString].flatMap{
case id if id.equals(JsString(accountId)) =>
(__ \ "primary").json.put(JsBoolean(true))
case _ =>
Reads.pure(Json.obj())
}
With Json.obj()
(__ \ "id").json.pick[JsString].map{
case id if id.equals(JsString(accountId)) =>
Json.obj("primary" -> true)
case _ =>
Json.obj()
}
Related
I have a JSON structure that contains an array of events. The array is "polymorphic" in the sense that there are three possible event types A, B and C:
{
...
"events": [
{ "eventType": "A", ...},
{ "eventType": "B", ...},
{ "eventType": "C", ...},
...
]
}
The three event types don't have the same object structure, so I need different Reads for them. And apart from that, the target case class of the whole JSON document distinguishes between the events:
case class Doc(
...,
aEvents: Seq[EventA],
bEvents: Seq[EventB],
cEvents: Seq[EventC],
...
)
How can I define the internals of Reads[Doc] so that the json array events is split into three subsets which are mapped to aEvents, bEvents and cEvents?
What I tried so far (without being succesful):
First, I defined a Reads[JsArray] to transform the original JsArray to another JsArray that only contains events of a particular type:
def eventReads(eventTypeName: String) = new Reads[JsArray] {
override def reads(json: JsValue): JsResult[JsArray] = json match {
case JsArray(seq) =>
val filtered = seq.filter { jsVal =>
(jsVal \ "eventType").asOpt[String].contains(eventTypeName)
}
JsSuccess(JsArray(filtered))
case _ => JsError("Must be an array")
}
}
Then the idea is to use it like this within Reads[Doc]:
implicit val docReads: Reads[Doc] = (
...
(__ \ "events").read[JsArray](eventReads("A")).andThen... and
(__ \ "events").read[JsArray](eventReads("B")).andThen... and
(__ \ "events").read[JsArray](eventReads("C")).andThen... and
...
)(Doc.apply _)
However, I don't know how to go on from here. I assume the andThen part should look something like this (in case of event a):
.andThen[Seq[EventA]](EventA.reads)
But that doesn't work since I expect the API to create a Seq[EventA] by explicitly passing a Reads[EventA] instead of Reads[Seq[EventA]]. And apart from that, since I've never got it running, I'm not sure if this whole approach is reasonable in the first place.
edit: in case the original JsArray contains unknown event types (e.g. D and E), these types should be ignored and left out from the final result (instead of making the whole Reads fail).
put implicit read for every Event type like
def eventRead[A](et: String, er: Reads[A]) = (__ \ "eventType").read[String].filter(_ == et).andKeep(er)
implicit val eventARead = eventRead("A", Json.reads[EventA])
implicit val eventBRead = eventRead("B", Json.reads[EventB])
implicit val eventCRead = eventRead("C", Json.reads[EventC])
and use Reads[Doc] (folding event list to separate sequences by types and apply result to Doc):
Reads[Doc] = (__ \ "events").read[List[JsValue]].map(
_.foldLeft[JsResult[ (Seq[EventA], Seq[EventB], Seq[EventC]) ]]( JsSuccess( (Seq.empty[EventA], Seq.empty[EventB], Seq.empty[EventC]) ) ){
case (JsSuccess(a, _), v) =>
(v.validate[EventA].map(e => a.copy(_1 = e +: a._1)) or v.validate[EventB].map(e => a.copy(_2 = e +: a._2)) or v.validate[EventC].map(e => a.copy(_3 = e +: a._3)))
case (e, _) => e
}
).flatMap(p => Reads[Doc]{js => p.map(Doc.tupled)})
it will create Doc in one pass through events list
JsSuccess(Doc(List(EventA(a)),List(EventB(b2), EventB(b1)),List(EventC(c))),)
the source data
val json = Json.parse("""{"events": [
| { "eventType": "A", "e": "a"},
| { "eventType": "B", "ev": "b1"},
| { "eventType": "C", "event": "c"},
| { "eventType": "B", "ev": "b2"}
| ]
|}
|""")
case class EventA(e: String)
case class EventB(ev: String)
case class EventC(event: String)
I would model the fact that you store different event types in your JS array as a class hierarchy to keep it type safe.
sealed abstract class Event
case class EventA() extends Event
case class EventB() extends Event
case class EventC() extends Event
Then you can store all your events in a single collection and use pattern matching later to refine them. For example:
case class Doc(events: Seq[Event]) {
def getEventsA: Seq[EventA] = events.flatMap(_ match {
case e: EventA => Some(e)
case _ => None
})
}
Doc(Seq(EventA(), EventB(), EventC())).getEventsA // res0: Seq[EventA] = List(EventA())
For implementing your Reads, Doc will be naturally mapped to the case class, you only need to provide a mapping for Event. Here is what it could look like:
implicit val eventReads = new Reads[Event] {
override def reads(json: JsValue): JsResult[Event] = json \ "eventType" match {
case JsDefined(JsString("A")) => JsSuccess(EventA())
case JsDefined(JsString("B")) => JsSuccess(EventB())
case JsDefined(JsString("C")) => JsSuccess(EventC())
case _ => JsError("???")
}
}
implicit val docReads = Json.reads[Doc]
You can then use it like this:
val jsValue = Json.parse("""
{
"events": [
{ "eventType": "A"},
{ "eventType": "B"},
{ "eventType": "C"}
]
}
""")
val docJsResults = docReads.reads(jsValue) // docJsResults: play.api.libs.json.JsResult[Doc] = JsSuccess(Doc(List(EventA(), EventB(), EventC())),/events)
docJsResults.get.events.length // res1: Int = 3
docJsResults.get.getEventsA // res2: Seq[EventA] = List(EventA())
Hope this helps.
I'm reading a nested JSON document using several Reads[T] implementations, however, I'm stuck with the following sub-object:
{
...,
"attributes": {
"keyA": [1.68, 5.47, 3.57],
"KeyB": [true],
"keyC": ["Lorem", "Ipsum"]
},
...
}
The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary. The values of the keys are always JsArray instances, but of different size and type (however, all elements of a particular array must have the same JsValue type).
The Scala representation of one single attribute:
case class Attribute[A](name: String, values: Seq[A])
// 'A' can only be String, Boolean or Double
The goal is to create a Reads[Seq[Attribute]] that can be used for the "attributes"-field when transforming the whole document (remember, "attributes" is just a sub-document).
Then there is a simple map that contains allowed combinations of keys and array types that should be used to validate attributes. Edit: This map is specific for each request (or rather specific for every type of json document). But you can assume that it is always available in the scope.
val required = Map(
"KeyA" -> "Double",
"KeyB" -> "String",
"KeyD" -> "String",
)
So in the case of the JSON shown above, the Reads should create two errors:
"keyB" does exist, but has the wrong type (expected String, was boolean).
"keyD" is missing (whereas keyC is not needed and can be ignored).
I'm having trouble creating the necessary Reads. The first thing I tried as a first step, from the perspective of the outer Reads:
...
(__ \ "attributes").reads[Map[String, JsArray]]...
...
I thought this is a nice first step because if the JSON structure is not an object containing Strings and JsArrays as key-value pairs, then the Reads fails with proper error messages. It works, but: I don't know how to go on from there. Of course I just could create a method that transforms the Map into a Seq[Attribute], but this method somehow should return a JsResult, since there are further validations to do.
The second thing I tried:
val attributeSeqReads = new Reads[Seq[Attribute]] {
def reads(json: JsValue) = json match {
case JsObject(fields) => processAttributes(fields)
case _ => JsError("attributes not an object")
}
def processAttributes(fields: Map[String, JsValue]): JsResult[Seq[Attribute]] = {
// ...
}
}
The idea was to validate each element of the map manually within processAttributes. But I think this is too complicated. Any help is appreciated.
edit for clarification:
At the beginning of the post I said that the keys (keyA, keyB...) are unknown at compile time. Later on I said that those keys are part of the map required which is used for validation. This sounds like a contradiction, but the thing is: required is specific for each document/request and is also not known at compile time. But you don't need to worry about that, just assume that for every request the correct required is already available in the scope.
You are too confused by the task
The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary
So the number of keys and their types are known in advance and the final?
So in the case of the JSON shown above, the Reads should create two
errors:
"keyB" does exist, but has the wrong type (expected String, was
boolean).
"keyD" is missing (whereas keyC is not needed and can be ignored).
Your main task is just to check the availability and compliance?
You may implement Reads[Attribute] for every your key with Reads.list(Reads.of[A]) (this Reads will check type and required) and skip omitted (if not required) with Reads.pure(Attribute[A]). Then tuple convert to list (_.productIterator.toList) and you will get Seq[Attribute]
val r = (
(__ \ "attributes" \ "keyA").read[Attribute[Double]](list(of[Double]).map(Attribute("keyA", _))) and
(__ \ "attributes" \ "keyB").read[Attribute[Boolean]](list(of[Boolean]).map(Attribute("keyB", _))) and
((__ \ "attributes" \ "keyC").read[Attribute[String]](list(of[String]).map(Attribute("keyC", _))) or Reads.pure(Attribute[String]("keyC", List()))) and
(__ \ "attributes" \ "keyD").read[Attribute[String]](list(of[String]).map(Attribute("keyD", _)))
).tupled.map(_.productIterator.toList)
scala>json1: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}
scala>res37: play.api.libs.json.JsResult[List[Any]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57)), Attribute(KeyB,List(true)), Attribute(keyC,List()), Attribute(KeyD,List(Lorem, Ipsum))),)
scala>json2: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyC":["Lorem","Ipsum"]}}
scala>res38: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray())))))
scala>json3: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":["Lorem"],"keyC":["Lorem","Ipsum"]}}
scala>res42: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray()))), (/attributes/keyB(0),List(ValidationError(List(error.expected.jsboolean),WrappedArray())))))
If you will have more than 22 attributes, you will have another problem: Tuple with more than 22 properties.
for dynamic properties in runtime
inspired by 'Reads.traversableReads[F[_], A]'
def attributesReads(required: Map[String, String]) = Reads {json =>
type Errors = Seq[(JsPath, Seq[ValidationError])]
def locate(e: Errors, idx: Int) = e.map { case (p, valerr) => (JsPath(idx)) ++ p -> valerr }
required.map{
case (key, "Double") => (__ \ key).read[Attribute[Double]](list(of[Double]).map(Attribute(key, _))).reads(json)
case (key, "String") => (__ \ key).read[Attribute[String]](list(of[String]).map(Attribute(key, _))).reads(json)
case (key, "Boolean") => (__ \ key).read[Attribute[Boolean]](list(of[Boolean]).map(Attribute(key, _))).reads(json)
case _ => JsError("")
}.iterator.zipWithIndex.foldLeft(Right(Vector.empty): Either[Errors, Vector[Attribute[_ >: Double with String with Boolean]]]) {
case (Right(vs), (JsSuccess(v, _), _)) => Right(vs :+ v)
case (Right(_), (JsError(e), idx)) => Left(locate(e, idx))
case (Left(e), (_: JsSuccess[_], _)) => Left(e)
case (Left(e1), (JsError(e2), idx)) => Left(e1 ++ locate(e2, idx))
}
.fold(JsError.apply, { res =>
JsSuccess(res.toList)
})
}
(__ \ "attributes").read(attributesReads(Map("keyA" -> "Double"))).reads(json)
scala> json: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}
scala> res0: play.api.libs.json.JsResult[List[Attribute[_ >: Double with String with Boolean]]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57))),/attributes)
I have a case Class and its companion object like below. Now, when I send JSON without id, createdAt and deletedAt fields, because I set them elsewhere, I get [NoSuchElementException: JsError.get] error. It's because I do not set above properties.
How could I achieve this and avoid getting the error?
case class Plan(id: String,
companyId: String,
name: String,
status: Boolean = true,
#EnumAs planType: PlanType.Value,
brochureId: Option[UUID],
lifePolicy: Seq[LifePolicy] = Nil,
createdAt: DateTime,
updatedAt: DateTime,
deletedAt: Option[DateTime]
)
object Plan {
implicit val planFormat = Json.format[Plan]
def fromJson(str: JsValue): Plan = Json.fromJson[Plan](str).get
def toJson(plan: Plan): JsValue = Json.toJson(plan)
def toJsonSeq(plan: Seq[Plan]): JsValue = Json.toJson(plan)
}
JSON I send from client
{
"companyId": "e8c67345-7f59-466d-a958-7c722ad0dcb7",
"name": "Creating First Plan with enum Content",
"status": true,
"planType": "Health",
"lifePolicy": []
}
You can introduce another case class just to handle serialization from request:
like this
case class NewPlan(name: String,
status: Boolean = true,
#EnumAs planType: PlanType.Value,
brochureId: Option[UUID],
lifePolicy: Seq[LifePolicy] = Nil
)
and then use this class to populate your Plan class.
The fundamental issue is that by the time a case class is instantiated to represent your data, it must be well-typed. To shoe horn your example data into your example class, the types don't match because some fields are missing. It's literally trying to call the constructor without enough arguments.
You've got a couple options:
You can make a model that represents the incomplete data (as grotrianster suggested).
You can make the possible missing fields Option types.
You can custom-write the Reads part of your Format to introduce intelligent values or dummy values for the missing ones.
Option 3 might look something like:
// Untested for compilation, might need some corrections
val now: DateTime = ...
val autoId = Reads[JsObject] {
case obj: JsObject => JsSuccess(obj \ 'id match {
case JsString(_) => obj
case _ => obj.transform(
__.update((__ \ 'id).json.put("")) andThen
__.update((__ \ 'createdTime).json.put(now)) andThen
__.update((__ \ 'updatedTime).json.put(now))
)
})
case _ => JsError("JsObject expected")
}
implicit val planFormat = Format[Plan](
autoId andThen Json.reads[Plan],
Json.writes[Plan])
Once you do this once, if the issue is the same for all your other models, you can probably abstract it into some Format factory utility function.
This may be slightly cleaner for autoId:
val autoId = Reads[JsObject] {
// Leave it alone if we have an ID already
case obj: JsObject if (obj \ 'id).asOpt[String].isSome => JsSuccess(obj)
// Insert dummy values if we don't have an `id`
case obj: JsObject => JsSuccess(obj.transform(
__.update((__ \ 'id).json.put("")) andThen
__.update((__ \ 'createdTime).json.put(now)) andThen
__.update((__ \ 'updatedTime).json.put(now))
))
case _ => JsError("JsObject expected")
}
Given a JSON array like this one:
{
"success": true,
"data": [
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
{
"id": 600,
"title": "test deal",
"e54cbe3a434d8e6": 54
},
],
"additional_data": {
"pagination": {
"start": 0,
"limit": 100,
"more_items_in_collection": false
}
}
}
In my Play 2.2.2 application, using the Scala JSON Reads Combinator, everything works going this way:
implicit val entityReader = Json.reads[Entity]
val futureJson: Future[List[Entity]] = futureResponse.map(
response => (response.json \ "data").validate[List[Entity]].get
The problem now is the key named 'e54cbe3a434d8e6' which I would like to name 'value' in my object:
// This doesn't work, as one might expect
case class Entity(id: Long, title: String, e54cbe3a434d8e6: Long)
// I would like to use 'value' instead of 'e54cbe3a434d8e6'
case class Entity(id: Long, title: String, value: Long)
There is vast information about the combinators here and here but I only want to use a fieldname which is different from the key name in the JSON array. Can some one help me to find a simple way?
I suppose it has got something to do with JSON.writes?!
One simple way without trying to apply transformations on json itself is to define a custom Reads in such a way to handle this:
val json = obj(
"data" -> obj(
"id" -> 600,
"title" -> "test deal",
"e54cbe3a434d8e6" -> 54))
case class Data(id: Long, title: String, value: Int)
val reads = (
(__ \ "id").read[Long] ~
(__ \ "title").read[String] ~
(__ \ "e54cbe3a434d8e6").read[Int] // here you get mapping from your json to Scala case class
)(Data)
def index = Action {
val res = (json \ "data").validate(reads)
println(res) // prints "JsSuccess(Data(600,test deal,54),)"
Ok(json)
}
Another way is to use combinators like this:
... the same json and case class
implicit val generatedReads = reads[Data]
def index = Action {
val res = (json \ "data").validate(
// here we pick value at 'e54cbe3a434d8e6' and put into brand new 'value' branch
__.json.update((__ \ "value").json.copyFrom((__ \ "e54cbe3a434d8e6").json.pick)) andThen
// here we remove 'e54cbe3a434d8e6' branch
(__ \ "e54cbe3a434d8e6").json.prune andThen
// here we validate result with generated reads for our case class
generatedReads)
println(res) // prints "JsSuccess(Data(600,test deal,54),/e54cbe3a434d8e6/e54cbe3a434d8e6)"
Ok(prettyPrint(json))
}
I have the following Json:
{
"web-category" : "macaroons",
"sub-categories" : [
{ "name" : "pink" },
{ "name" : "blue" },
{ "name" : "green" }
]
}
I have got it in Play as a JsObject. So I can now successfully do the following:
//(o is the JsObject)
val webCat:Option[String] = (o \ "web-category").asOpt[String]
println(webCat.toString)
>> Some(macaroons)
So far, so good. But how do I access the array Json objects? I have this...
val subCats:Option[JsArray] = (o \ "sub-categories").asOpt[JsArray]
println(subCats.toString)
>> Some([{"name" : "blue"},{"name" : "green"},{"name" : "pink"}])
but what I need is to take the JsArray and get a List of all the names something like this:
List("blue", "green", "pink")
Don't know how to access the JsArray thusly.
my thanks for your help in this.
I'd argue that it's generally a good idea to move from JSON-land to native-Scala-representation-land as early as possible. If obj is your JsObject, for example, you can write this:
val subCategories = (obj \ "sub-categories").as[List[Map[String, String]]]
val names = subCategories.map(_("name"))
Or even:
case class Category(name: String, subs: List[String])
import play.api.libs.functional.syntax._
implicit val categoryReader = (
(__ \ "web-category").read[String] and
(__ \ "sub-categories").read[List[Map[String, String]]].map(_.map(_("name")))
)(Category)
And then:
obj.as[Category]
This latter approach makes error handling even cleaner (e.g. you can just replace as with asOpt at this top level) and composes nicely with other Reads type class instances—if you have a JsArray of these objects, for example, you can just write array.as[List[Category]] and get what you expect.
What Peter said, or:
(o \ "sub-categories" \\ "name").map(_.as[String]).toList
Something like this:
subCats.map( jsarray => jsarray.value.map(jsvalue => (jsvalue \ "name").as[String]).toList)
This will normally return a Option[List[String]]