In my code I am trying to get Income type object list from a json file
val fileJson = File(
Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS),
"Income.json")
val strFileJson: String? = fileJson.readText()
val PreviousJsonObj: JSONObject = JSONObject(strFileJson)
val array: JSONArray = PreviousJsonObj.getJSONArray("Incomes")
val typeToken = object : TypeToken<List<Income>>() {}.type
IncomeArry = Gson().fromJson<List<Income>>(strFileJson, typeToken)
But I am getting
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was BEGIN_OBJECT at line 1 column 2 path $
The value returned in strFileJson is "Incomes":[{"date":"abc","src": "cfgb","amt": 7890},
{"date":"yui", "src":"tyhv","amt" :6789}]
Tried running with multiple imple,mentations of gson and multiple class architectures
The fragment of JSON you show:
"Incomes":[{"date":"abc","src": "cfgb","amt": 7890}, {"date":"yui", "src":"tyhv","amt" :6789}]
is malformed, did you mis-paste? It needs to be enclosed in curly braces:
{"Incomes":[{"date":"abc","src": "cfgb","amt": 7890}, {"date":"yui", "src":"tyhv","amt" :6789}]}
but this will not solve your problem because both will give the error you describe. You are trying to deserialize an array so you need to either
Change the JSON to be
[{"date":"abc","src": "cfgb","amt": 7890}, {"date":"yui", "src":"tyhv","amt" :6789}]
Or create a wrapper object, something like
data class Incomes(
val Incomes : List<Income>
)
and deserialize in to that.
Related
I want to parse the JSON from nominatim from OpenStreetMap.
Example
It's a list and I don't have a clue how I can describe the list. I am using Gson, these is my data class:
data class Destination(
val lat: Double,
val lon: Double,
val display_name: String
)
and this is my Gson implementation:
val list = Gson().fromJson<List<Destination>>(
body,
Destination::class.java
)
It gives me this error:
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2 path $
But I declared an Array in my Gson implementation. Anyone having an idea how to fix this?
You can deserialize it as follow:
val type = object : TypeToken<List<Destination>>() {}.type
Gson().fromJson<List<Destination>>(body, type)
Similar to what was mentioned in this question
I figured it out, you have to use an Array, not a List:
val list : Array<Destination> = Gson().fromJson(
body,
Array<Destination>::class.java
)
I was trying to get the array of JSONObjects from the ResponseEntity< JSONObject > variable "result".
<200 OK,{"response":"success","count":0,"maintenanceDetails":[{"date":"2020\/08\/21","activity":"TestActivity","operatorName":"Operator","operator":"Mock for Party, hashCode: 552416003","status":"Open"}],"buildingId":Mock for UUID, hashCode: 2141932519},{}>
Output value of result is given above.
val result = dashBoardService!!.getMaintenanceStateByBuildingId(id)
println("Result "+result.body["maintenanceDetails"])
val a =result.body["maintenanceDetails"] as Array<JSONObject>
println("A "+ a.first().get("activity"))
I Use "as Array < JSONObject >" to convert, because result.body["maintenanceDetails"] normally comes as "Any" type, which I can't extract more. But this gives me below error.
java.lang.ClassCastException: java.util.ArrayList cannot be cast to [Lorg.json.simple.JSONObject;
What I need is to get the first JSONObject from Array corresponds to result.body["maintenanceDetails"] so that I can use it in assert checks. Kindly help.
Conversion is working to ArrayList since it is considered as util.ArrayList by Intelij, So after this I am able to get the data from inside.
val tempArr= result.body["maintenanceDetails"] as ArrayList<JSONObject>
assertEquals(tempArr[0]["activity"],maintenanceState.activity)
assertEquals(tempArr[0]["status"],maintenanceState.status)
Use:
val array = JSONArray(result.body["maintenanceDetails"])
val first = array.getJSONObject(0)
You can use JSONArray() to access the array inside a json object (Root) and then iterate the array to get all the json objects present in the array. Here is a sample,
import org.json.simple.JSONArray
import org.json.simple.JSONObject
val jsonObject = JSONObject()
val childObject = JSONObject()
childObject["date"]="10-10-2010"
childObject["operator"]="mock party"
val jsonArray = JSONArray()
jsonArray.add(0,childObject)
jsonObject["response"]="success"
jsonObject["count"]=0
jsonObject["maintenanceDetails"]=jsonArray
val json = jsonObject["maintenanceDetails"] as JSONArray
println(json[0])
val outputObject = json[0] as JSONObject
println(outputObject["operator"])
Output:
{"date":"10-10-2010","operator":"mock party"}
mock party
I have a text file with json value. and this gets read into a DF
{"name":"Michael"}
{"name":"Andy", "age":30}
I want to infer the schema dynamically for each line while Streaming and store it in separate locations(tables) depending on its schema.
unfortunately while I try to read the value.schema it still shows as String. Please help on how to do it on Streaming as RDD is not allowed in streaming.
I wanted to use the following code which doesnt work as the value is still read as String format.
val jsonSchema = newdf1.select("value").as[String].schema
val df1 = newdf1.select(from_json($"value", jsonSchema).alias("value_new"))
val df2 = df1.select("value_new.*")
I even tried to use,
schema_of_json("json_schema"))
val jsonSchema: String = newdf.select(schema_of_json(col("value".toString))).as[String].first()
still no hope.. Please help..
You can load the data as textFile, create case class for person and parse every json string to Person instance using json4s or gson, then creating the Dataframe as follows:
case class Person(name: String, age: Int)
val jsons = spark.read.textFile("/my/input")
val persons = jsons.map{json => toPerson(json) //instead of 'toPerson' actually parse with json4s or gson to return Person instance}
val df = sqlContext.createDataFrame(persons)
Deserialize json to case class using json4s:
https://commitlogs.com/2017/01/14/serialize-deserialize-json-with-json4s-in-scala/
Deserialize json to case class using gson:
https://alvinalexander.com/source-code/scala/scala-case-class-gson-json-object-deserialization-and-scalatra
Using jackson library I read json data from a file (each row of file is a JSON object) an parse it to a map object of String and Any. My goal is to save specified keys (id and text) to a collection.
val input = scala.io.Source.fromFile("data.json").getLines()
val mapper = new ObjectMapper() with DefaultScalaModule
val data_collection = mutable.HashMap.empty[Int, String]
for (i <- input){
val parsedJson = mapper.readValue[Map[String, Any]](i)
data_collection.put(
parsedJson.get("id"),
parsedJson.get("text")
)
But as the values in the parsedJson map have the Any type, getting some keys like id and text, it returns Some(value) not just the value with the appropriate type. I expect the values for the id key to be Integer and values for the text to be String.
Running the code I got the error:
Error:(31, 23) type mismatch;
found : Option[Any]
required: Int
parsedJson.get("id"),
Here is a sample of JSON data in the file:
{"text": "Hello How are you", "id": 1}
Is it possible in Scala to parse id values to Int and text values to String, or at least convert Some(value) to value with type Int or String?
If you want to get a plain value from a Map instead of a Option you can use the () (apply) method - However it will throw an exception if the key is not found.
Second, Scala type system is static not dynamic, if you have an Any that's it, it won't change to Int or String at runtime, and the compiler will fail - Nevertheless, you can cast them using the asInstanceOf[T] method, but again if type can't be casted to the target type it will throw an exception.
Please note that even if you can make your code work with the above tricks, that code wouldn't be what you would expect in Scala. There are ways to make the code more typesafe (like pattern matching), but parsing a Json to a typesafe object is an old problem, I'm sure jackson provides a way to parse a json into case class that represent your data. If not take a look to circe it does.
Try the below code :
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import
com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
val input = scala.io.Source.fromFile("data.json").getLines()
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
val obj = mapper.readValue[Map[String, Any]](input)
val data_collection = mutable.HashMap.empty[Int, String]
for (i <- c) {
data_collection.put(
obj.get("id").fold(0)(_.toString.toInt),
obj.get("text").fold("")(_.toString)
)
}
println(data_collection) // Map(1 -> Hello How are you)
I am reading a HDFS sequence file and which is of [Long, String] lets call each record as message. message._2 is a json string and i am trying to parse it using play json library but i get the following error when i do it.
Error:
found : Seq[play.api.libs.json.JsValue]
required: String
Code:
val jsonString = message._2.toString();
val json = Json.parse(jsonString);
code = (json(0) \\ "code"); -- > Error is pointing to \\ on this line
The error message says that (json(0) \\ "code") returns Seq[play.api.libs.json.JsValue], and you're trying to assign this value to the variable code of type String.
So, you may want to do this:
code = (json(0) \\ "code").head.as[String]
which will get the first item of a list and convert JsValue to String.
Update
As #cchantep suggested, the use of head is not safe, so for better safety you can do it with headOption, but the result type will be Option[String]:
val code: Option[String] =
(json \\ "code").headOption.map(_.as[String])
and even more safe code will look like this:
val code: Option[String] =
(json \\ "code").headOption.flatMap(_.asOpt[String])