I have the following piece of code:
(json \ field.name).as[Int]
The problem is, the code seems to look only into the 1st "layer" of a json document, giving me an error when the JsObject is wrapped into an Array.
To better illustrate the point:
This json works:
{
fieldName: 123
}
This doesnt:
[
{
fieldName: 123
}
]
So, how do I look up the value of the fieldName in the 2nd json?
As you said, this json is an array where the first value is your object.
You can use ordinal traversing to obtain the first object, and parse it.
scala> val json = Json.arr(Json.obj("value" -> 10))
scala> json(0)
res0: play.api.libs.json.JsValue = {"value":10}
scala> (json(0) \ "value").as[Int]
res1: Int = 10
Related
I want to create an empty JValue to be able to parse JSON objects together.
As for now I am creating the JValue containing {} and then I am parsing the other objects to it and then in the end I remove the first row using an RDD, but I would like to create
var JValue: JValue = JValue.empty
from the beginning to be able to skip the removing part.
Is it possible to create an empty JValue?
import org.json4s._
import org.json4s.jackson.JsonMethods._
var JValue: JValue = parse("{}")
val a = parse(""" {"name":"Scott", "age":33} """)
val b = parse(""" {"name":"Scott", "location":"London"} """)
JValue = JValue.++(a)
JValue = JValue.++(b)
val df = spark.read.json(Seq(compact(render(JValue ))) toDS())
val rdd = df.rdd.first()
val removeFirstRow = df.rdd.filter(row => row != rdd)
val newDataFrame = spark.createDataFrame(removeFirstRow,df.schema)
If I understand correctly what you are trying to achieve, you can start from an empty array like so:
var JValue: JValue = JArray(List.empty)
Calling the ++ method on the empty array will result in the items being added to that array, as defined here.
The final result is the following object:
[ {
"name" : "Scott",
"age" : 33
}, {
"name" : "Scott",
"location" : "London"
} ]
If you want to play around with the resulting code, you can have a look at this worksheet on Scastie (please bear in mind that I did not pull in the Spark dependency there and I'm not 100% sure that would work anyway in Scastie).
As you can notice in the code I linked above, you can also just to a ++ b to obtain the same result, so you don't have to necessarily start from the empty array.
As a further note, you may want to rename JValue to something different to avoid weird errors in which you cannot tell apart the variable and the JValue type. Usually in Scala types are capitalized and variables are not. But of course, try to work towards the existing practices of your codebase.
I have a JSON body in the following form:
val body =
{
"a": "hello",
"b": "goodbye"
}
I want to extract the VALUE of "a" (so I want "hello") and store that in a val.
I know I should use "parse" and "Extract" (eg. val parsedjson = parse(body).extract[String]) but I don't know how to use them to specifically extract the value of "a"
To use extract you need to create a class that matches the shape of the JSON that you are parsing. Here is an example using your input data:
val body ="""
{
"a": "hello",
"b": "goodbye"
}
"""
case class Body(a: String, b: String)
import org.json4s._
import org.json4s.jackson.JsonMethods._
implicit val formats = DefaultFormats
val b = Extraction.extract[Body](parse(body))
println(b.a) // hello
You'd have to use pattern matching/extractors:
val aOpt: List[String] = for {
JObject(map) <- parse(body)
JField("a", JString(value)) <- map
} yield value
alternatively use querying DSL
parse(body) \ "a" match {
case JString(value) => Some(value)
case _ => None
}
These are options as you have no guarantee that arbitrary JSON would contain field "a".
See documentation
extract would make sense if you were extracting whole JObject into a case class.
Hi everyone recently I faced an issue in converting json into my own data model.
I have a json format message which may contain an empty string:
{
"name" : "John Doe",
"hobbies": ""
}
or a list of hobby types:
{
"name" : "John Doe",
"hobbies": [{"name":"basketball"}]
}
And the following is my case class data model in scala play framework:
case class Person(name: String, hobbies: List[Hobby])
case class Hobby(name: String)
Right now I'm using the default json formatter but of course it's not working well when we have empty string as value.
implicit val HobbyJson= Json.format[Hobby]
implicit val PersonJson = Json.format[Person]
it will throw exception if the hobbies has a empty string. I want to convert it into an empty list when it's the empty string. I search the document Play provides but couldn't find infomation. Can anyone give some suggestions?
Thanks in advance.
As you mentioned, the default Format macros won't work for you here because of the inconsistent treatment of hobbies. So you need to implement your own Reads[Person] - here's how I'd do it:
object PersonJson {
implicit val hobbyConverter = Json.format[Hobby]
val personReads = new Reads[Person] {
override def reads(json: JsValue): JsResult[Person] = {
for {
personName <- (json \ "name").validate[String]
hobbies <- (json \ "hobbies").validate[JsValue]
} yield {
val maybeHobbyList = hobbies.validate[List[Hobby]].asOpt
Person(personName, maybeHobbyList.getOrElse(Nil))
}
}
}
implicit val personConverter = Format(personReads, Json.writes[Person])
}
The key thing to note here is surrounding the whole thing in a JsResult courtesy of the for-comprehension and the yield. This gives us all the necessary checking (like the name field being there and being a String, and the hobbies field being there).
The code within the yield block only runs if we've got something that looks pretty close to a Person. Then we can safely try validating the hobbies as a List[Hobby], and convert the result to an Option[List[Hobby]]. It'll be a None if it didn't work (thus it must have been a string) and so we default it to the empty list as required.
Thanks #millhouse answer, it definitely works. Like he said we need a custom Reads[Person] to properly convert it.
I also post my code as reference.
implicit val personJsonReads: Reads[Person] = (
(__ \ "name").read[String] and
(__ \ "hobbies").read[List[Hobby]].orElse(Reads.pure(List()))
) (Person.apply _)
read[List[Hobby]].orElse(Reads.pure(List())) will generate the empty list when the value cannot convert to List[Hobby].
Would anyone please explain why the following happens?
scala> import play.api.libs.json._
scala> Json.toJson("""{"basic":"test"}""") // WORKS CORRECTLY
res134: play.api.libs.json.JsValue = "{\"basic\":\"test\"}"
scala> Json.toJson(""" {"basic":"test"} """) \ "basic" // ??? HOW COME?
res131: play.api.libs.json.JsValue = JsUndefined('basic' is undefined on object: " {\"basic\":\"test\"} ")
Many thanks
Json.toJson renders its argument as a JSON value using an implicitly provided Writes instance. If you give it a string, you'll get a JsString (typed as a JsValue). You want Json.parse, which parses its argument:
scala> Json.parse("""{"basic":"test"}""") \ "basic"
res0: play.api.libs.json.JsValue = "test"
As expected.
And to address your answer (which should be a comment or a new question, by the way), if you give toJson a value of some type A, it will convert it into a JSON value, assuming that there's an instance of the Writes type class in scope for that A. For example, the library provides Writes[String], Writes[Int], etc., so you can do the following:
scala> Json.prettyPrint(Json.toJson(1))
res11: String = 1
scala> Json.prettyPrint(Json.toJson("a"))
res12: String = "a"
scala> Json.prettyPrint(Json.toJson(List("a", "b")))
res13: String = [ "a", "b" ]
You can also create Writes instances for your own types (here I'm using Play's "JSON inception"):
case class Foo(i: Int, s: String)
implicit val fooWrites: Writes[Foo] = Json.writes[Foo]
And then:
scala> Json.prettyPrint(Json.toJson(Foo(123, "foo")))
res14: String =
{
"i" : 123,
"s" : "foo"
}
Using type classes to manage encoding and decoding is an alternative to reflection-based approaches, and it has a lot of advantages (but that's out of the scope of this question).
Turning my comment into an answer:
Json.toJson() does not create an object. It turns an object into a JSON string. What I think you're wanting is Json.parse(). Once you've parsed a JSON string, it's an object, and you can get to the properties.
Thanks a lot to both of you. So the following works as expected.
scala> Json.parse("""{"basic":"test"}""") \ "basic"
res137: play.api.libs.json.JsValue = "test"
I'd still like to understand what Json.toJson does. The docs state "Transform a stream of A to a stream of JsValue". Can anyone point out in what context this can be used?
Here is a JSON list I want to process:
scala> jsonStructure \ "response" \ "docs"
res4: play.api.libs.json.JsValue = [{"title":"the very first document"},{"title":"on brick walls"}]
I tried converting it to a list, but I got something with different semantics, a list whose only element is that list:
scala> jsonStructure \ "response" \\ "docs"
res3: Seq[play.api.libs.json.JsValue] = List([{"title":"the very first document"},{"title":"on brick walls"}])
scala> res3.size
res4: Int = 1
I tried this kludge, which does the trick:
scala> (jsonStructure \ "response" \ "docs").as[Seq[play.api.libs.json.JsValue]]
res9: Seq[play.api.libs.json.JsValue] = List({"title":"the very first document"}, {"title":"on brick walls"})
scala> res9 size
res10: Int = 2
Why did the \\ not work? What is the idiomatic way to understand a JsValue into a JsArray? While still maintaining the "navigating using and \ never fails" philosophy? I want to parse deeper structures, like a list of obj inside an obj which was a list element; I want a method that wont become unwieldy for deeply nested structures.
Feel free to correct my approach if you find it complicated, brittle etc.
If you wanted just the list of title entries, you probably would want to write something like:
json \\ "title"
Which returns:
List("the very first document", "on brick walls")
The \\ works by listing any element matching your path selector (at current level and all the descendants). Since there is essentially one docs element, it returns a list with one element. Only when you ask for a title, it will return a list of titles.
But from you approach you probably wanted to pattern-match a JsValue on JsArray:
def convert(json : JsValue) : Option[Seq[JsValue]] = json match {
case JsArray(arr) => Some(arr)
case _ => None
}
It returns exactly what you wanted:
List({"title":"the very first document"}, {"title":"on brick walls"})