An example of the sort of objects I need to grab from my json can be found in the following example(src):
{
"test": {
"attra": "2017-10-12T11:17:52.971Z",
"attrb": "2017-10-12T11:20:58.374Z"
},
"dummyCheck": false,
"type": "object",
"ruleOne": {
"default": 2557
},
"ruleTwo": {
"default": 2557
}
}
From the example above I want to access the default value under "ruleOne".
I've tried messing about with several different things below but I seem to be struggling. I can grab values like "dummyCheck" ok. What's the best way to key into where I need to go?
Example of how I am trying to get the value below:
import org.json4s._
import org.json4s.native.JsonMethods._
import org.json4s.DefaultFormats
implicit val formats = DefaultFormats
val test = parse(src)
println((test \ "ruleOne.default").extract[Integer])
Edit:
To further extend what is above:
def extractData(data: java.io.File) = {
val json = parse(data)
val result = (json \ "ruleOne" \ "default").extract[Int]
result
}
If I was to extend the above into a function that is called by passing in:
extractData(src)
That would only ever give me RuleOne.default.. is there a way I could extend it so that I could dynamically pass it multiple string arguments to parse (like a splat)
def extractData(data: java.io.File, path: String*) = {
val json = parse(data)
val result = (json \ path: _*).extract[Int]
result
}
so consuming it would be like
extractData(src, "ruleOne", "default")
This here works with "json4s-jackson" % "3.6.0-M2", but it should work in exactly the same way with native backend.
val src = """
|{
| "test": {
| "attra": "2017-10-12T11:17:52.971Z",
| "attrb": "2017-10-12T11:20:58.374Z"
| },
| "dummyCheck": false,
| "type": "object",
| "ruleOne": {
| "default": 2557
| },
| "ruleTwo": {
| "default": 2557
| }
|}""".stripMargin
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.DefaultFormats
implicit val formats = DefaultFormats
val test = parse(src)
println((test \ "ruleOne" \ "default").extract[Int])
Output:
2557
To make it work with native, simply replace
import org.json4s.jackson.JsonMethods._
by
import org.json4s.native.JsonMethods._
and make sure that you have the right dependencies.
EDIT
Here is a vararg method that transforms string parameters into a path:
def extract(json: JValue, path: String*): Int = {
path.foldLeft(json)(_ \ _).extract[Int]
}
With this, you can now do:
println(extract(test, "ruleOne", "default"))
println(extract(test, "ruleTwo", "default"))
Note that it accepts a JValue, not a File, because the version with File would be unnecessarily painful to test, whereas JValue-version can be tested with parsed string constants.
Related
I am receiving a JSON from an external service and my goal is to parse it exactly as it is.
The main issue is this: a value can be nullable or it can be absent BUT null has different meaning of absent. So I want to catch this somehow.
For example this JSON:
{
"a": null,
"b": 1
}
is different from this one:
{
"b": 1
}
Can you help me please?
UPDATE:
Sorry for the delay in the update. Anyway: you are right, I have a implicit custom reads in the middle and currently I use "a".readNullable[Double] and "a".write[Option[Double]] and case class is something like:
case class Example(a: Option[Double])
Just laying out what #mfirry was talking about with a detailed example (play-json 2.6):
scala> import play.api.libs.json._
import play.api.libs.json._
scala> val json1 = Json.parse("""{"a": null, "b": 1}""")
json1: play.api.libs.json.JsValue = {"a":null,"b":1}
scala> val json2 = Json.parse("""{"b": 1}""")
json2: play.api.libs.json.JsValue = {"b":1}
scala> (json1 \ "a").isDefined
res8: Boolean = true
scala> (json1 \ "a") == JsDefined(JsNull)
res3: Boolean = true
scala> (json2 \ "a").isDefined
res7: Boolean = false
scala> (json2 \ "a")
res5: play.api.libs.json.JsLookupResult = JsUndefined('a' is undefined on object: {"b":1})
1.Input is JSON file that contains multiple records. Example:
[
{"user": "user1", "page": 1, "field": "some"},
{"user": "user2", "page": 2, "field": "some2"},
...
]
2.I need to load each record from the file as a Document to MongoDB collection.
Using casbah for interacting with mongo, inserting data may look like:
def saveCollection(inputListOfDbObjects: List[DBObject]) = {
val xs = inputListOfDbObjects
xs foreach (obj => {
Collection.save(obj)
})
Question: What is the correct way (using scala) to parse JSON to get data as List[DBObject] at output?
Any help is appreciated.
You could use the parser combinator library in Scala.
Here's some code I found that does this for JSON: http://booksites.artima.com/programming_in_scala_2ed/examples/html/ch33.html#sec4
Step 1. Create a class named JSON that contains your parser rules:
import scala.util.parsing.combinator._
class JSON extends JavaTokenParsers {
def value : Parser[Any] = obj | arr |
stringLiteral |
floatingPointNumber |
"null" | "true" | "false"
def obj : Parser[Any] = "{"~repsep(member, ",")~"}"
def arr : Parser[Any] = "["~repsep(value, ",")~"]"
def member: Parser[Any] = stringLiteral~":"~value
}
Step 2. In your main function, read in your JSON file, passing the contents of the file to your parser.
import java.io.FileReader
object ParseJSON extends JSON {
def main(args: Array[String]) {
val reader = new FileReader(args(0))
println(parseAll(value, reader))
}
}
I was wondering if there is a parser or an easy way to iterate through a json object without knowing the keys/schema of the json ahead of time in scala. I took a look at a few libraries like json4s, but it seems to still require knowing the schema ahead of time before extracting the fields. I just want to iterate over each field, extract the fields and print out their values something like:
json.foreachkey(key -> println(key +":" + json.get(key))
In Play Json you'll initially parse your json into a JsValue; you can then pattern-match this to determine if it is a JsObject (note that you can find the fields of this using fields or value), a JsArray (again, note the value), or a primitive such as JsString or JsNull
def parse(jsVal: JsValue) {
jsVal match {
case json: JsObject =>
case json: JsArray =>
case json: JsString =>
...
}
}
If by json you mean any JValue, then json4s seems to have this functionality out of the box:
scala> import org.json4s.JsonDSL._
import org.json4s.JsonDSL._
scala> import org.json4s.native.JsonMethods._
import org.json4s.native.JsonMethods._
scala> val json = parse(""" { "numbers" : [1, 2, 3, 4] } """)
json: org.json4s.JValue = JObject(List((numbers,JArray(List(JInt(1), JInt(2), JInt(3), JInt(4))))))
scala> compact(render(json))
res1: String = {"numbers":[1,2,3,4]}
Use liftweb, it allows you to parse the json first into JValue - then extract native scala objects from it, no matter the schema:
val jsonString = """{"menu": {
| "id": "file",
| "value": "File",
| "popup": {
| "menuitem": [
| {"value": "New", "onclick": "CreateNewDoc()"},
| {"value": "Open", "onclick": "OpenDoc()"},
| {"value": "Close", "onclick": "CloseDoc()"}
| ]
| }
|}}""".stripMargin
val jVal: JValue = parse(jsonString)
jVal.values
>>> Map(menu -> Map(id -> file, value -> File, popup -> Map(menuitem -> List(Map(value -> New, onclick -> CreateNewDoc()), Map(value -> Open, onclick -> OpenDoc()), Map(value -> Close, onclick -> CloseDoc())))))
I have the following Read defined:
import org.joda.time.DateTime;
implicit val userInfoRead: Reads[UserInfo] = (
(JsPath \ "userName").readNullable[String] and
] (JsPath \ "startDate").readNullable[DateTime]
(UserInfo.apply _)
With the following JSON object being passed in:
"userInfo" : {
"userName": "joeuser",
"startDate": "2006-02-28"
}
When I validate this data I get the following error:
(/startDate,List(ValidationError(validate.error.expected.jodadate.format,WrappedArray(yyyy-MM-dd))))))
Any suggestions on what I'm missing in the formatting?
As far as I can see, the issue is probably just the format not matching what Joda is expecting. I simplified a bit, and this worked for me:
scala> import org.joda.time.DateTime
import org.joda.time.DateTime
scala> case class UserInfo(userName: String, startDate: DateTime)
defined class UserInfo
scala> implicit val dateReads = Reads.jodaDateReads("yyyy-MM-dd")
dateReads: play.api.libs.json.Reads[org.joda.time.DateTime] = play.api.libs.json.DefaultReads$$anon$10#22db02cb
scala> implicit val userInfoReads = Json.reads[UserInfo]
userInfoReads: play.api.libs.json.Reads[UserInfo] = play.api.libs.json.Reads$$anon$8#52bcbd5d
scala> val json = Json.parse("""{
| "userName": "joeuser",
| "startDate": "2006-02-28"
| }""")
json: play.api.libs.json.JsValue = {"userName":"joeuser","startDate":"2006-02-28"}
scala> json.validate[UserInfo]
res12: play.api.libs.json.JsResult[UserInfo] = JsSuccess(UserInfo(joeuser,2006-02-28T00:00:00.000-05:00),)
Take into account the following JSON provided by a vendor API:
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
val json = Json.parse(
"""
|{
| "returns": {
| "markets" : {
| "ABC" : {
| "label": "ABC",
| "id":1
| },
| "DEF" : {
| "label": "DEF",
| "id":2
| }
| }
| }
|}
""".stripMargin)
How to extract a sequence of pairs related to "label" and "id" fields.
From this piece of JSON the result I'm expecting is:
Seq((1,"ABC"),(2,"DEF"))
I'm failing with constructing a correct JsPath extractor because it expects a single match e.g.
val jsonTransformer = (__ \ 'returns \ 'markets).json.pick
json.transform(jsonTransformer)
Here's how I'd build this parser out of nice composable pieces. First for a general purpose object-to-array transformer that throws away keys:
val objToArray: Reads[JsArray] =
JsPath.json.pickBranch.map(obj => JsArray(obj.fields.map(_._2)))
Now for a Reads that can process market objects:
val marketReads: Reads[(Int, String)] =
((__ \ 'id).read[Int] and (__ \ 'label).read[String]).tupled
And now we tie it all together:
val pairsReads: Reads[List[(Int, String)]] =
(__ \ 'returns \ 'markets).read(objToArray) andThen list(marketReads)
And finally:
scala> json.validate(pairsReads).foreach(println)
List((1,ABC), (2,DEF))
Which is what you want. Note that I've specified the types above for clarity, but that's not necessary here—I'd probably leave them out in real code because the pieces are pretty small and straightforward.