My main aim is to make a transformation i.e from "data/message" to "posts/content".
What's wrong with my code? I can't even access the value of "message" in an array.
<code>
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
def mapping(): JsObject = {
val sjson =
"""
{
"data": [
{"message": "A", "created_time": "a" },
{"message": "B", "created_time": "b" }
],
"page": "test"
}
""".stripMargin
val json = Json.parse(sjson)
val jsonReads =
(__ \ 'data \\ 'message ).read[String]
json.validate(jsonReads).map {
case( message ) =>
Json.obj("content" -> message)
}.get
}
</code>
In response to your comment on this post, here's my updated answer:
val sjson =
"""
{
"data": [
{"message": "A", "created_time": "a" },
{"message": "B", "created_time": "b" }
],
"page": "test"
}
""".stripMargin
// Read the whole object
val json: JsObject = Json.parse(sjson).as[JsObject]
// Read the "data array"
val data: Seq[JsObject] = (json \ "data").as[Seq[JsObject]]
// Get final result my changing "data" to "posts",
// and all instances of "message" to "content" and "created_time" to "created"
val result =
Json.obj(
"posts" -> (data map (d => JsObject(d.value map {
case ("message", m) =>
"content" -> m
case ("created_time", t) =>
"created" -> t
})))
)
Related
I am using Play Framework with Scala. I have the following JSON structure:
{
"a": 1540554574847,
"b": 2,
"c": {
"pep3lpnp1n1ugmex5uevekg5k20wkfq3": {
"a": 1,
"b": 1,
"c": 1,
"d": 1
},
"p3zgudnf7tzqvt50g7lpr2ryno7yugmy": {
"b": [
"d10e5600d11e5517"
],
"c": 1,
"d": 1,
"e": 1,
"g": 1,
"h": [
"d10e5600d11e5517",
"d10e5615d11e5527",
"d10e5605d11e5520",
"d10e5610d11e5523",
"d10e5620d11e5530"
],
"q": "a_z6smu56gstysjpqbzp21ruxii6g2ph00"
},
"33qfthhugr36f5ts4251glpqx0o373pe": {
"b": [
"d10e5633d11e5536"
],
"c": 1,
"d": 1,
"e": 1,
"g": 1,
"h": [
"d10e5638d11e5539",
"d10e5633d11e5536",
"d10e5643d11e5542",
"d10e5653d11e5549",
"d10e5648d11e5546"
],
"q": "a_cydo6wu1ds340j3q6qxeig97thocttsp"
}
}
}
I need to fetch values from paths
"c" -> "pep3lpnp1n1ugmex5uevekg5k20wkfq3" -> "b",
"c" -> "p3zgudnf7tzqvt50g7lpr2ryno7yugmy" -> "b",
"c" -> "33qfthhugr36f5ts4251glpqx0o373pe" -> "b", and so on, where "pep3lpnp1n1ugmex5uevekg5k20wkfq3" is dynamic and changes for every JSON input.
Output should be like Seq(object(q,b,c)).
If you don't need to know which generated key belongs to which value you can use recursive path \\ operator:
import play.api.libs.json.Json
import play.api.libs.json._
val jsonText = """{
"a":1540554574847,
"b":2,
"c":{
"onegeneratedkey":{
"a":1,
"b":1,
"c":1,
"d":1
},
"secondsonegeneratedkey":{
"a":1,
"b": [1, 2, 3],
"c":1,
"d":1
}
}
}"""
val result: Seq[JsValue] = Json.parse(jsonText) \ "c" \\ "b"
// res: List(1, [1,2,3])
UPD.
To get all values stored inside object with generated-keys, one can use JsObject#values:
val valuesSeq: Seq[JsValue] = (Json.parse(jsonText) \ "c").toOption // get 'c' field
.collect {case o: JsObject => o.values.toSeq} // get all object that corresponds to generated keys
.getOrElse(Seq.empty)
// res: Seq({"a":1,"b":1,"c":1,"d":1}, {"a":1,"b":[1,2,3],"c":1,"d":1})
val valuesABC = valuesSeq.map(it => (it \ "a", it \ "b", it \ "c"))
// res: Seq((JsDefined(1),JsDefined(1),JsDefined(1)), (JsDefined(1),JsDefined([1,2,3]),JsDefined(1)))
I misread the question, and this is the modified version.
Here I used json.pick to read JsObject and iterate the keys from there.
Ps: You don't have to create Reads or the case classes, but it should made the caller program more readable.
import play.api.libs.json.Json
import play.api.libs.json._
val jsonText =
"""{
"top": {
"level2a": {
"a": 1,
"b": 1,
"c": 1,
"d": 1
},
"level2b": {
"a": 2,
"b": 2,
"nested": {
"b": "not interested"
}
}
}
}"""
case class Data(k: String, v: Int)
case class Datas(list: Seq[Data])
object Datas {
implicit val reads: Reads[Datas] = (__ \ "top").json.pick.map {
case obj: JsObject =>
new Datas(obj.keys.flatMap(k => (obj \ k \ "b").validate[Int] match {
case JsSuccess(v, _) => Some(Data(k, v))
case _ => None
}).toSeq)
}
}
Json.parse(jsonText).validate[Datas].asOpt match {
case Some(d) => println(s"found: $d")
case _ => println("not found")
}
To deserialize the internal structure within level2, you may choose to create the internal structure and use Json.reads to create the default reads. So long as the data structure is known and predictable.
For example
case class Internal(a: Int, b: Int, c: Option[Int], d: Option[Int])
object Internal {
implicit val reads = Json.reads[Internal]
}
case class Data(k: String, v: Internal)
case class Datas(list: Seq[Data])
object Datas {
implicit val reads: Reads[Datas] = (__ \ "top").json.pick.map {
case obj: JsObject =>
new Datas(obj.keys.flatMap(k => (obj \ k).validate[Internal].asOpt
.map(v => Data(k, v))).toSeq)
}
}
Given the following json:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
}
}
I want to append new item to user JSON variable. and create a new json like:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
"point":10
}
}
You can achieve this by using Play Json API.
First you need to parse your json string to Play JsValue,
import play.api.libs.json._
val jsonString =
"""
|{
| "status": "ok",
| "user": {
| "id": 39216,
| "first_name": "naghmeh",
| "username": "test"
| }
|}
""".stripMargin
val jsValue = Json.parse(jsonString)
Now, you can add the values to your JsValue either by traversing the Json and then modifying it.
val result = jsValue match {
case jsObject: JsObject => (jsObject \ "user").as[JsObject] match {
case userJsObject: JsObject => jsObject ++ Json.obj(
"user" -> (userJsObject ++ Json.obj(
"point" -> 10
))
)
case _ => jsValue
}
case _ => jsValue
}
Or, by using Play Json's JsonTransformer API,
val jsonTransformer = (__ \ "user").json.update(
__.read[JsObject].map(jsObject => jsObject ++ Json.obj("point" -> 10))
)
val result2 = jsValue.transform(jsonTransformer) match {
case JsSuccess(jsObject, _) => jsObject
case _ => jsValue
}
Assuming from your question tags that you're using the PlayJSON library, you can simply use +:
val json: JsObject = ... // Your initial value here
val newField: JsValue = ... // Your new value here
val jsonWithAdditionalField = json + ("yourKey" -> newField)
For more information about how to create the JsValue, take a look a the official documentation
Append the list to a variable and then call put like so:
list.put("test");
I have a Seq of some Strings mostly like below:
val states = Seq(
"CA" -> Seq("Los Angeles" -> Seq("SunsetBlvd", "Hollywood" -> Seq("W 8th St", "W 9th St")), "Pasadena"),
"WA" -> Seq("Seattle", "Redmond")
)
The case class for this could be
case class State(name: String, sub: Option[Seq[State]])
And an implicit Writes
implicit val stateWrites = Json.Writes[State]
Hoping to convert it to a Json like
[
{
"name": "CA",
"sub": [
{
"name": "Los Angeles",
"sub": [
{
"name": "SunsetBlvd"
},
{
"name": "Hollywood",
"sub": [
{
"name": "W 8th St"
},
{
"name": "W 9th St"
}
]
}
]
}
]
},
{
"name": "WA",
"sub": [
{
"name": "Seattle"
},
{
"name": "Redmond"
}
]
}
]
How do I correctly model the data and be able to convert this Seq to a Json using Writes?
Or even change the states val to an appropriate format so I could easily convert it to Json?
In the case class, one of fields is of type itself which is wrong. How do I avoid that in modeling the data or even the Seq?
I came up with something like this:
case class State(name: String, sub: Option[Seq[State]])
import play.api.libs.json._
implicit val optWrites = new Writes[Option[Seq[State]]] {
override def writes(o: Option[Seq[State]]) = {
if (o.isDefined) {
Json.toJson(o.get)(stateSeqWrites)
} else {
JsNull
}
}
}
implicit val stateWrites = new Writes[State] {
def writes(state: State) = {
val l: Seq[(String, JsValueWrapper)] = Seq("name" -> JsString(state.name))
val ll: Seq[(String, JsValueWrapper)] = if (state.sub.isDefined) {
val subValue: JsValueWrapper = Json.toJson(state.sub)(optWrites)
l :+ ("sub" -> subValue)
} else {
l
}
Json.obj(ll : _*)
}
}
implicit val stateSeqWrites: Writes[Seq[State]] = new Writes[Seq[State]] {
override def writes(s: Seq[State]) = {
JsArray(s.map(Json.toJson(_)(stateWrites)))
}
}
val states = Seq(
State("CA", Some(Seq(State("Los Angeles", Some(Seq(State("SunsetBlvd", None), State("Hollywood", Some(Seq(State("W 8th St", None), State("W 9th St", None)))), State("Pasadena", None))))))),
State("WA", Some(Seq(State("Seattle", None), State("Redmond", None))))
)
val json = Json.toJson(states)
println(json.toString())
Probably might get simplified, but it's late night here ;) It does what you need :)
That information has conceptually a tree structure. My advice is to threat it with just a normal case class, to simplify the json formatter and to have a much more semantic structure:
case class Tree(name: String, sub: Option[List[Tree]])
And your formatter would be just like this:
implicit val repositoryFormat: Format[Tree] = (
(__ \ 'name).format[String] ~
(__ \ 'sub).lazyFormatNullable(implicitly[ Format[ List[Tree] ]])
)(Tree.apply, unlift(Tree.unapply))
Notice I used lazyFormatNullable to deal with the recursive reference to Tree in sub.
To simulate the Json you posted, I made the translation to the Tree case class structure.
// The tree leaves
val hollywoodLeaves = Some( Tree("W 8th St", None) :: Tree("W 9th St", None) :: Nil )
val losAngelesLeaves = Some( Tree("SunsetBlvd", None) :: Tree("Hollywood", hollywoodLeaves ) :: Nil )
// The two trees in your array
val firstTree = Tree( "CA", Some( Tree("Los Angeles", losAngelesLeaves) :: Nil ) )
val secondTree = Tree("WA", Some( Tree("Seattle", None) :: Tree("Redmond", None) :: Nil ))
// Your root array
val treeArray = firstTree :: secondTree :: Nil
// Conversion to json
val json = Json.toJson(treeArray)
I have some json that includes some fields that are being flattened into a bson-ish format as in {"foo.bar" : "bash"}. I'd like to transform this to the following representation {"foo" : { "bar" : "bash"}} and wondering where in circe I'd do such an operation. Complicating the problem is that there could be multiple such fields that need to be properly merged, e.g. {"foo.bar" : "a", "foo.bash" : "b", "foo.baz" : "c"} -> {"foo" : { "bar" : "a", "bash" : "b", "baz" : "c"}}.
Here's a quick implementation:
import io.circe.Json
val Dotted = "([^\\.]*)\\.(.*)".r
def expandDotted(j: Json): Json = j.arrayOrObject(
j,
js => Json.fromValues(js.map(expandDotted)),
_.toList.map {
case (Dotted(k, rest), v) => Json.obj(k -> expandDotted(Json.obj(rest -> v)))
case (k, v) => Json.obj(k -> expandDotted(v))
}.reduceOption(_.deepMerge(_)).getOrElse(Json.obj())
)
I haven't really used or tested it in detail, but it seems to work:
scala> import io.circe.literal._
import io.circe.literal._
scala> val j1 = json"""{"foo.bar" : "a", "foo.bash" : "b", "foo.baz" : "c"}"""
j1: io.circe.Json =
{
"foo.bar" : "a",
"foo.bash" : "b",
"foo.baz" : "c"
}
scala> expandDotted(j1)
res1: io.circe.Json =
{
"foo" : {
"baz" : "c",
"bash" : "b",
"bar" : "a"
}
}
And with deeper nesting:
scala> expandDotted(json"""{ "x.y.z": true, "a.b": { "c": 1 } }""")
res2: io.circe.Json =
{
"a" : {
"b" : {
"c" : 1
}
},
"x" : {
"y" : {
"z" : true
}
}
}
And just to confirm that it doesn't mess with undotted keys:
scala> expandDotted(json"""{ "a.b": true, "x": 1 }""").noSpaces
res3: String = {"x":1,"a":{"b":true}}
Note that in the case of "collisions" (paths that lead to both JSON objects and non-object JSON values, or to multiple non-object values), the behavior is that of Json#deepMerge:
scala> expandDotted(json"""{ "a.b": true, "a": 1 }""").noSpaces
res4: String = {"a":1}
scala> expandDotted(json"""{ "a": 1, "a.b": true }""").noSpaces
res5: String = {"a":{"b":true}}
…which is probably what you'd want, but you could also have it fail in these cases, or not expand the colliding path, or do pretty much any other thing you can think of.
I have the following JSON being returned from a web service:
{
"hits": [
{
"created_at": "2016-02-01T15:01:03.000Z",
"title": "title",
"num_comments": 778,
"parent_id": null,
"_tags": [
"story",
"author",
"story_11012044"
],
"objectID": "11012044",
"_highlightResult": {
"title": {
"value": "title",
"matchLevel": "full",
"matchedWords": [
"title"
]
},
"author": {
"value": "author",
"matchLevel": "none",
"matchedWords": [
]
},
"story_text": {
"value": "Please lead",
"matchLevel": "none",
"matchedWords": [
]
}
}
}
]
}
and I am trying to parse it using the JSON parsing libs in Play Framework. I have the following code:
import play.api.libs.functional.syntax._
import play.api.libs.json._
case class Post(id: Long, date: String, count: Int)
object Post {
implicit val postFormat = Json.format[Post]
implicit val writes: Writes[Post] = (
(JsPath \ "id").write[Long] and
(JsPath \"date").write[String] and
(JsPath \ "count").write[Int]
)(unlift(Post.unapply))
implicit val reads: Reads[Post] = (
(JsPath \ "objectID").read[Long] and
(JsPath \ "created_at").read[String] and
(JsPath \ "num_comments").read[Int]
)(Post.apply _)
}
import play.api.libs.json._
class PostRepo {
val request: WSRequest = ws.url(MY_URL)
def getPosts: Future[Seq[Post]] =
val result: Future[JsValue] = request.get().map(response =>
response.status match {
case 200 => Json.parse(response.body)
case _ => throw new Exception("Web service call failed: " + response.body)
})
result.map( {
jsonvalue => println("JSARRAY: " + jsonvalue);
(jsonvalue \ "hits").as[Seq[Post]]
})
result
}
Now, when I run the code, I am getting the following error:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[JsResultException:
JsResultException(errors:List(((0)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((0)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((0)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((1)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((2)/id,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/date,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/count,List(ValidationError(List(error.path.missing),WrappedArray()))),
((3)/id,List(ValidationError(List(error.path.missing),WrappedArray())))
Obviously something is wrong with the way I'm trying to parse the JSON but I have now spent a few hours trying to figure out the problem and I'm well and truly stuck.
Some refactoring code with Reads.seq
val r = (__ \ "hits").read[Seq[Post]](Reads.seq[Post])
def getPosts: Future[Seq[Post]] = {
WS.url(MY_URL).get().map(response =>
response.status match {
case 200 => r.reads(response.json) match {
case JsError(e) => throw new Exception("Json read fails. Response body:" + response.json.toString() + "\nRead error:" + e.toString())
case JsSuccess(x, _) => x
}
case _ => throw new Exception("Web service call failed: " + response.body)
})
}