I am using Scala with play JSON library for parsing JSON. We are the facing the problem using JSON parsing is, we have same JSON structure, but some JSON files contain different with values structure with the same key name. Let's take an example:
json-1
{
"id": "123456",
"name": "james",
"company": {
"name": "knoldus"
}
}
json-2
{
"id": "123456",
"name": "james",
"company": [
"knoldus"
]
}
my case classes
case class Company(name: String)
object Company{
implicit val _ = Json.format[Company]
}
case class User(id: String, name: String, company: Company)
object User{
implicit val _ = Json.format[Company]
}
while JSON contains company with JSON document, we are getting successfully parsing, but if company contains an array, we are getting parsing exception. Our requirements, are is there anyway, we can use play JSON library and ignore the fields if getting parsing error rather that, ignore whole JSON file. If I am getting, company array values, ignore company field and parse rest of them and map corresponding case class.
I would do a pre-parse function that will rename the 'bad' company.
See the tutorial for inspiration: Traversing-a-JsValue-structure
So your parsing will work, with this little change:
case class User(id: String, name: String, company: Option[Company])
The company needs to be an Option.
Final we found the answer to resolving this issue, as we know, we have different company structure within JSON, so what we need to do, we need to declare company as a JsValue because in any case, whatever the company structure is, it is easily assigned to JsValue type. After that, our requirements are, we need to use object structure, and if JSON contains array structure, ignore it. After that, we used pattern matching with our company JsValue type and one basis of success and failure, we parse or JSON. The solution with code is given below:
case class Company(name: String)
object Company{
implicit val _ = Json.format[Company]
}
case class User(id: String, name: String, company: JsValue)
object User{
implicit val _ = Json.format[Company]
}
Json.parse("{ --- whatevery json--string --- }").validate[User].asOpt match {
case Some(obj: JsObject) => obj.as[Company]
case _ => Company("no-name")
}
Related
I am trying to familiarize myself with the PlayJSON library. I have a JSON formatted file like this:
{
"Person": [
{
"name": "Jonathon",
"age": 24,
"job": "Accountant"
}
]
}
However, I'm having difficulty with parsing it properly due to the file having different types (name is a String but age is an Int). I could technically make it so the age is a String and call .toInt on it later but for my purposes, it is by default an integer.
I know how to parse some of it:
import play.api.libs.json.{JsValue, Json}
val parsed: JsValue = Json.parse(jsonFile) //assuming jsonFile is that entire JSON String as shown above
val person: List[Map[String, String]] = (parsed \ "Person").as[List[Map[String, String]]]
Creating that person value throws an error. I know Scala is a strongly-typed language but I'm sure there is something I am missing here. I feel like this is an obvious fix too but I'm not quite sure.
The error produced is:
JsResultException(errors:List(((0)/age,List(JsonValidationError(List(error.expected.jsstring),WrappedArray())))
The error you are having, as explained in the error you are getting, is in casting to the map of string to string. The data you provided does not align with it, because the age is a string. If you want to keep in with this approach, you need to parse it into a type that will handle both strings and ints. For example:
(parsed \ "Person").validate[List[Map[String, Any]]]
Having said that, as #Luis wrote in a comment, you can just use case class to parse it. Lets declare 2 case classes:
case class JsonParsingExample(Person: Seq[Person])
case class Person(name: String, age: Int, job: String)
Now we will create a formatter for each of them on their corresponding companion object:
object Person {
implicit val format: OFormat[Person] = Json.format[Person]
}
object JsonParsingExample {
implicit val format: OFormat[JsonParsingExample] = Json.format[JsonParsingExample]
}
Now we can just do:
Json.parse(jsonFile).validate[JsonParsingExample]
Code run at Scastie.
I'm modelling some JSON - and using the following lines
data class Metadata(
val id: String,
val creators: Array<CreatorsModel>
)
along with:
data class CreatorsModel (
val role: String,
val name: String
)
However keep seeing the error: Array property in data class error.
Any ideas why this is?
FYI, the JSON looks like:
{
"id": "123",
"creators": [{
"role": "Author",
"name": "Marie"
}
]
}
In Kotlin you should aim to use List instead of Array where possible. Array has some JVM implications, and although the compiler will let you, the IDE may prompt you to override equals and hashcode manually. Using List will make things much simpler.
You can find out more about the difference here: Difference between List and Array types in Kotlin
I am trying to parse the nested Json and get the list of name from the fields tag.
{
"subject": "table-name",
"version": 1,
"id": 234,
"schema": "{
\"type\":\"record\",\"name\":\"table-name\",
\"fields\":[
{\"name\":\"NAME\",\"type\":\"CHARACTER\},
{\"name\":\"EID\",\"type\":\"INT\"},
{\"name\":\"DEPARTMENT\",\"type\":\"CHARACTER\"}
]
}"
}
I checked couple of post and came up with below code. I can get the schema definition(which is fairly easy) but not able to the list of name from fields.
case class Names(name : String)
case class FieldNames(fields: List[Names])
case class ColNames(subject: String, schema: FieldNames)
implicit val formats = DefaultFormats
val sourceSchema = parse("JsonStr").extract[ColNames]
println(sourceSchema)
My idea was the part schema: FieldNames will get me the fields tag and then List[Names] will get me the list of name but all I get is empty list. If I change the schema: FieldNames with schema: String I get the schema too but I am not getting how to get the required list of name after that. Below is the current output:
ColNames(table-name,FieldNames(List()))
Update:
I got it working but I ended up parsing the same Json twice which I don't feel is a best way of doing it. I would still like to know the best way to get it done. Below is my current solution:
case class Name(name : String)
case class FieldNames(fields: List[Name])
case class ColNames(subject: String, schema: String)
val sourceSchema = parse("JsonStr").extract[ColNames]
val cols= parse(sourceSchema.schema).extract[FieldNames]
Output:
FieldNames(List(Name(NAME), Name(EID), Name(DEPARTMENT)))
I have a json object that I need to update. The original object is a list that looks like this:
[
{
"firstName":"Jane",
"lastName":"Smith"
},
{
"firstName":"Jack",
"lastName":"Brown"
}
]
For each element in the list, we have an extra field, "age", that needs to be added at run-time, so the result should look like the following:
[
{
"firstName":"Jane",
"lastName":"Smith",
"age": "21"
},
{
"firstName":"Jack",
"lastName":"Brown",
"age": "34"
}
]
Any suggestions how to do this so the result is still json?
Thanks.
request.body.asJson.map {
jm => (jm.as[JsObject] ++ Json.obj("age" -> 123))
}
I would recommended deserializing the JSON array you receive into a List of case classes, then having some function fill in the missing attributes based on the current attributes of the case class, and finally serializing them as JSON and serving the response.
Let's make a Person case class with the fields that will be missing as Option:
import play.api.libs.json.Json
case class Person(firstName: String, lastName: String, age: Option[Int])
object Person {
implicit val format: Format[Person] = Json.format[Person]
def addAge(person: Person): Person = {
val age = ... // however you determine the age
person.copy(age = Some(age))
}
}
Within the companion object for Person I've also defined a JSON serializer/deserializer using the format macro, and a stub for a function that will find a person's age then copy it back into the person and return it.
Deep within the web service call you might then have something like this:
val jsArray = ... // The JsValue from somewhere
jsArray.validate[List[Person]].fold(
// Handle the case for invalid incoming JSON
error => InternalServerError("Received invalid JSON response from remote service."),
// Handle a deserialized array of List[Person]
people => {
Ok(
// Serialize as JSON, requires the implicit `format` defined earlier.
Json.toJson(
// Map each Person to themselves, adding the age
people.map(person => Person.addAge(person))
)
)
}
)
This method is much safer, otherwise you'll have to extract values from the array one by one and concatenate objects, which is very awkward. This will also allow you to easily handle errors when the JSON you receive is missing fields you're expecting.
I'm using Scala & Argonaut, trying to parse the following JSON:
[
{
"name": "apple",
"type": "fruit",
"size": 3
},
{
"name": "jam",
"type": "condiment",
"size": 5
},
{
"name": "beef",
"type": "meat",
"size": 1
}
]
And struggling to work out how to iterate and extract the values into a List[MyType] where MyType will have name, type and size properties.
I will post more specific code soon (i have tried many things), but basically I'm looking to understand how the cursor works, and how to iterate through arrays etc. I have tried using \\ (downArray) to move to the head of the array, then :->- to iterate through the array, then --\ (downField) is not available (at least IntelliJ doesn't think so).
So the question is how do i:
navigate to the array
iterate through the array (and know when I'm done)
extract string, integer etc. values for each field - jdecode[String]? as[String]?
The easiest way to do this is to define a codec for MyType. The compiler will then happily construct a decoder for List[MyType], etc. I'll use a plain class here (not a case class) to make it clear what's happening:
class MyType(val name: String, val tpe: String, val size: Int)
import argonaut._, Argonaut._
implicit def MyTypeCodec: CodecJson[MyType] = codec3(
(name: String, tpe: String, size: Int) => new MyType(name, tpe, size),
(myType: MyType) => (myType.name, myType.tpe, myType.size)
)("name", "type", "size")
codec3 takes two parameter lists. The first has two parameters, which allow you to tell how to create an instance of MyType from a Tuple3 and vice versa. The second parameter list lets you specify the names of the fields.
Now you can just write something like the following (if json is your string):
Parse.decodeValidation[List[MyType]](json)
And you're done.
Since you don't need to encode and are only looking at decoding, you can do as suggested by Travis, but by implementing another implicit: MyTypeDecodeJson
implicit def MyTypeDecodeJson: DecodeJson[MyType] = DecodeJson(
raw => for {
name <- raw.get[String]("name")
type <- raw.get[String]("type")
size <- raw.get[Int]("size")
} yield MyType(name, type, size))
Then to parse your list:
Parse.decodeValidation[List[MyType]](jsonString)
Assuming MyType is a case class, the following works too:
case class MyType(name: String, type: String, size: Int)
object MyType {
implicit val createCodecJson: CodecJson[MyType] = CodecJson.casecodec3(apply, unapply)(
"name",
"type",
"size"
)
}