I'm trying to parse some JSON to kotlin objects. The JSON looks like:
{
data: [
{ "name": "aaa", "age": 11 },
{ "name": "bbb", "age": 22 },
],
otherdata : "don't need"
}
I just need to data part of the entire JSON, and parse each item to a User object:
data class User(name:String, age:Int)
But I can't find an easy way to do it.
Here's one way you can achieve this
import com.beust.klaxon.Klaxon
import java.io.StringReader
val json = """
{
"data": [
{ "name": "aaa", "age": 11 },
{ "name": "bbb", "age": 22 },
],
"otherdata" : "not needed"
}
""".trimIndent()
data class User(val name: String, val age: Int)
fun main(args: Array<String>) {
val klaxon = Klaxon()
val parsed = klaxon.parseJsonObject(StringReader(json))
val dataArray = parsed.array<Any>("data")
val users = dataArray?.let { klaxon.parseFromJsonArray<User>(it) }
println(users)
}
This will work as long as you can fit the whole json string in memory. Otherwise you may want to look into the streaming API: https://github.com/cbeust/klaxon#streaming-api
Related
This is the input json which I am getting which is nested json structure and I don't want to map directly to class, need custom parsing of some the objects as I have made the case classes
{
"uuid": "b547e13e-b32d-11ec-b909-0242ac120002",
"log": {
"Response": {
"info": {
"receivedTime": "2022-02-09T00:30:00Z",
"isSecure": "Yes",
"Data": [{
"id": "75641",
"type": "vendor",
"sourceId": "3",
"size": 53
}],
"Group": [{
"isActive": "yes",
"metadata": {
"owner": "owner1",
"compressionType": "gz",
"comments": "someComment",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "1"
},
{
"isActive": "yes",
"metadata": {
"owner": "owner12",
"compressionType": "snappy",
"comments": "someComment",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "2"
},
{
"isActive": "yes",
"metadata": {
"owner": "owner123",
"compressionType": "snappy",
"comments": "someComment",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "4"
},
{
"isActive": "yes",
"metadata": {
"owner": "owner124",
"compressionType": "snappy",
"comments": "someComments",
"createdDate": "2022-01-11T11:00:00Z",
"updatedDate": "2022-01-12T14:17:55Z"
},
"setId": "4"
}
]
}
}
}
}
Code that I am trying play json also tried circe . Please help ..New to scala world
below is object and case class
case class DataCatalog(uuid: String, data: Seq[Data], metadata: Seq[Metadata])
object DataCatalog {
case class Data(id: String,
type: String,
sourceId: Option[Int],
size: Int)
case class Metadata(isActive: String,
owner: String,
compressionType: String,
comments: String,
createdDate: String,
updatedDate: String
)
def convertJson(inputjsonLine: String): Option[DataCatalog] = {
val result = Try {
//val doc: Json = parse(line).getOrElse(Json.Null)
//val cursor: HCursor = doc.hcursor
//val uuid: Decoder.Result[String] = cursor.downField("uuid").as[String]
val lat = (inputjsonLine \ "uuid").get
DataCatalog(uuid, data, group)
}
//do pattern matching
result match {
case Success(dataCatalog) => Some(dataCatalog)
case Failure(exception) =>
}
}
}
Any parsing api is fine.
If you use Scala Play, for each case class you should have an companion object which will help you a lot with read/write object in/from json:
object Data {
import play.api.libs.json._
implicit val read = Json.reads[Data ]
implicit val write = Json.writes[Data ]
def tupled = (Data.apply _).tupled
}
object Metadata {
import play.api.libs.json._
implicit val read = Json.reads[Metadata ]
implicit val write = Json.writes[Metadata ]
def tupled = (Metadata.apply _).tupled
}
Is required as each companion object to be in same file as the case class. For your json example, you need more case classes because you have a lot of nested objects there (log, Response, info, each of it)
or, you can read the field which you're interested in as:
(jsonData \ "fieldName").as[CaseClassName]
You can try to access the Data value as:
(jsonData \ "log" \ "Response" \ "info" \ "Data").as[Data]
same for Metadata
I have the following data classes to parse JSON. I can parse it easily with the decodeFromString method. However, the Info classes could contain the List<Int> type from time to time along with the Int type so that both are included in a single JSON. How can I handle this variation in serialization?
#Serializable
data class Node (#SerialName("nodeContent") val nodeContent: List<Info>)
#Serializable
data class Info (#SerialName("info") val info: Int)
p.s. The closest question to mine is this one: Kotlinx Serialization, avoid crashes on other datatype. I wonder if there are other ways?
EDIT:
An example is given below.
"nodeContent": [
{
"info": {
"name": "1",
},
},
{
"info": [
{
"name": "1"
},
{
"name": "2"
},
],
},
{
"info": {
"name": "2",
},
}
]
Here is an approach with a custom serializer similar to the link you provided. The idea is to return a list with just a single element.
// Can delete these two lines, they are only for Kotlin scripts
#file:DependsOn("org.jetbrains.kotlinx:kotlinx-serialization-json:1.2.0")
#file:CompilerOptions("-Xplugin=/snap/kotlin/current/lib/kotlinx-serialization-compiler-plugin.jar")
import kotlinx.serialization.*
import kotlinx.serialization.json.*
import kotlinx.serialization.encoding.Decoder
#Serializable
data class Node (val nodeContent: List<Info>)
#Serializable(with = InfoSerializer::class)
data class Info (val info: List<Name>)
#Serializable
data class Name (val name: Int)
#Serializer(forClass = Info::class)
object InfoSerializer : KSerializer<Info> {
override fun deserialize(decoder: Decoder): Info {
val json = ((decoder as JsonDecoder).decodeJsonElement() as JsonObject)
return Info(parseInfo(json))
}
private fun parseInfo(json: JsonObject): List<Name> {
val info = json["info"] ?: return emptyList()
return try {
listOf(Json.decodeFromString<Name>(info.toString()))
} catch (e: Exception) {
(info as JsonArray).map { Json.decodeFromString<Name>(it.toString()) }
}
}
}
Usage:
val ss2 = """
{
"nodeContent": [
{
"info":
{"name": 1}
},
{
"info": [
{"name": 1},
{"name": 2}
]
},
{
"info":
{"name": 2}
}
]
}
"""
val h = Json.decodeFromString<Node>(ss2)
println(h)
Result:
Node(nodeContent=[Info(info=[Name(name=1)]), Info(info=[Name(name=1), Name(name=2)]), Info(info=[Name(name=2)])])
I have got the json structure as below:
{
"object": "list",
"total": 3,
"data": [
{
"object": "brand",
"id": "15243937043340",
"company": {
"object": "company",
"id": "956936000",
"name": "ABC"
},
"name": "Kindle",
"images": [
"http://www.spacecentrestorage.com/assets/uploads/General/SCS-Slide02-Commercial.jpg"
]
},
{
"object": "brand",
"id": "15243937043340",
"company": {
"object": "company",
"id": "956936000",
"name": "ABC"
},
"name": "Kindle",
"images": [
"http://www.spacecentrestorage.com/assets/uploads/General/SCS-Slide02-Commercial.jpg"
]
},
{
"object": "brand",
"id": "15243937043340",
"company": {
"object": "company",
"id": "956936000",
"name": "ABC"
},
"name": "Kindle",
"images": [
"http://www.spacecentrestorage.com/assets/uploads/General/SCS-Slide02-Commercial.jpg"
]
}
],
"associated": {}
}
And this is my Gson data class mapping :
data class Response (
#SerializedName("object")
val obj: String,
val total: Int,
val data: List<*>,
val associated: Response
)
data class Brand (
#SerializedName("object")
val obj: String,
val id: String,
val name: String,
val images: List<String>,
val company: Company
)
data class Company (
#SerializedName("object")
val obj: String,
val id: String,
val name: String
)
When it comes to extracting the tree as above, I find returned data string becomes Malformed Json and gives MalformedJsonException on $[0].companies.null
I have read about the recursive deserialisation function but it is not working in my case. I resort to deserialise as below , using original method, it causes errors
val response = gson.fromJson(queryResult , Response::class.java)
println("result 2 : $response" )
val dataString = response.data.toString()
println("result 3 : $dataString" )
val brands = Gson().fromJson(dataString, Array<Brand>::class.java).toMutableList()
println("result 4 : $brands" )
I would like to ask :
If returning json component to string, shall all the indents and symbols " be erased ?
To extract all associated object of the elements of the list of objects, what precautions do I have to take for deserialising list of objects using Gson ?
If you set the type parameter of the data list in Response to Brand GSON knows how to deserialise the items of the list.
data class Response (
#SerializedName("object")
val obj: String,
val total: Int,
val data: List<Brand>,
val associated: Response
)
Using this there is no need to parse the items of the list again and you can get all brands like this:
val response = Gson().fromJson(queryResult , Response::class.java)
val dataList = response.data
print("brands: " )
dataList.forEach { println(it) }
im trying to extract my data from json into a case class without success.
the Json file:
[
{
"name": "bb",
"loc": "sss",
"elements": [
{
"name": "name1",
"loc": "firstHere",
"elements": []
}
]
},
{
"name": "ca",
"loc": "sss",
"elements": []
}
]
my code :
case class ElementContainer(name : String, location : String,elements : Seq[ElementContainer])
object elementsFormatter {
implicit val elementFormatter = Json.format[ElementContainer]
}
object Applicationss extends App {
val el = new ElementContainer("name1", "firstHere", Seq.empty)
val el1Cont = new ElementContainer("bb","sss", Seq(el))
val source:String=Source.fromFile("src/bin/elementsTree.json").getLines.mkString
val jsonFormat = Json.parse(source)
val r1= Json.fromJson[ElementContainer](jsonFormat)
}
after running this im getting inside r1:
JsError(List((/elements,List(ValidationError(List(error.path.missing),WrappedArray()))), (/name,List(ValidationError(List(error.path.missing),WrappedArray()))), (/location,List(ValidationError(List(error.path.missing),WrappedArray())))))
been trying to extract this data forever, please advise
You have location instead loc and, you'll need to parse file into a Seq[ElementContainer], since it's an array, not a single ElementContainer:
Json.fromJson[Seq[ElementContainer]](jsonFormat)
Also, you have the validate method that will return you either errors or parsed json object..
I am a real newbie in Scala (Using Play framework) and trying to figure out what would be the best way to parse a complicated JSON.
This is an example I have:
{
"id":"test1",
"tmax":270,
"at":2,
"bcat":[
"IAB26",
"IAB25",
"IAB24"
],
"imp":[
{
"id":"1",
"banner":{
"w":320,
"h":480,
"battr":[
10
],
"api":[
]
},
"bidfloor":0.69
}
],
"app":{
"id":"1234",
"name":"Video Games",
"bundle":"www.testapp.com",
"cat":[
"IAB1"
],
"publisher":{
"id":"1111"
}
},
"device":{
"dnt":0,
"connectiontype":2,
"carrier":"Ellijay Telephone Company",
"dpidsha1":"e5f61ae0597d8abee94860d66f7d512aa68d0985",
"dpidmd5":"c1827fe90bae819017dfdb30db7e84fa",
"didmd5":"1f6cb9dc519db8e48cf9592b31cce04e",
"didsha1":"2e6a5d7f5fd1b2b5dea56a80f2b9dc24902a0ca7",
"ifa":"422aeb3a-6507-40b5-9e6f-42a0e14b51be",
"osv":"4.4",
"os":"Android",
"ua":"Mozilla/5.0 (Linux; Android 4.4.2; DL1010Q Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Safari/537.36",
"ip":"24.75.160.74",
"devicetype":1,
"geo":{
"type":1,
"country":"USA",
"city":"Jasper"
}
},
"user":{
"id":"9c2be7c2a3dfe19070f193910e92b2e0"
},
"cur":[
"USD"
]
}
Thank you very much!
I would go with case classes as described in http://json4s.org/#extracting-values e.g.
scala> import org.json4s._
scala> import org.json4s.jackson.JsonMethods._
scala> implicit val formats = DefaultFormats // Brings in default date formats etc.
scala> case class Child(name: String, age: Int, birthdate: Option[java.util.Date])
scala> case class Address(street: String, city: String)
scala> case class Person(name: String, address: Address, children: List[Child])
scala> val json = parse("""
{ "name": "joe",
"address": {
"street": "Bulevard",
"city": "Helsinki"
},
"children": [
{
"name": "Mary",
"age": 5,
"birthdate": "2004-09-04T18:06:22Z"
},
{
"name": "Mazy",
"age": 3
}
]
}
""")
scala> json.extract[Person]
res0: Person = Person(joe,Address(Bulevard,Helsinki),List(Child(Mary,5,Some(Sat Sep 04 18:06:22 EEST 2004)), Child(Mazy,3,None)))
Use implicit to automatically fetch json value with the same name.
if you have a string like
val jsonString =
{ "company": {
"name": "google",
"department": [
{
"name": "IT",
"members": [
{
"firstName": "Max",
"lastName": "Joe",
"age" : 25
},
{
"firstName": "Jean",
"lastName": "Nick",
"age" : 55,
"salary": 100,000
}
]
},
{
"name": "Marketing"
"members": [
{
"firstName": "Mike",
"lastName": "Lucas",
"age" : 43
}
]
},
]
}
}
Then we should do as following to parse it into scala object
val json: JsValue = Json.parse(jsonString) // string to JsValue
case class Person(firstName: String, lastName: String, age: Int, salary: Option[Int]) // keep the lower level object above since you will use them below
implicit personR = Json.reads[Person]
// implicit will read the json attribute if you use the same name as it is in json String. For example "name" : "google" will be read if you have Company(name: String). See the "name" and name:String .
case class Department(name: String, members: Seq[Person])
implicit departmentR = Json.reads[Department]
case class Company(name: String, department: Seq[Department])
implicit val companyR = Json.reads[Company]
val result = json.validate[Company] match {
case s: JsSuccess[Company] => s.get // s.get will return a Company object
case e: JsError => println("error")
}
Don't know if it can be used standalone, but Play Framework has great tools to work with JSON. Take a look here