I have a json structure that I need to (sort of) flatten when serializing it into an object. Some of the elements are at the top level and some are in a sub field. In addition, 1 of the fields is an array of space delimited strings that I need to parse and represent as myString.splig(" ")[0]
So, short of a when expression to do the job, can I use something like a jsonpath query to bind to certain fields? I have thought of even doing some kind of 2-pass binding and then merging both instances.
{
"key": "FA-207542",
"fields": {
"customfield_10443": {
"value": "TBD"
},
"customfield_13600": 45,
"customfield_10900": {
"value": "Monitoring/Alerting"
},
"customfield_10471": [
"3-30536161871 (SM-2046076)"
],
"issuetype": {
"name": "Problem Mgmt - Corrective Action"
},
"created": "2022-08-11T04:46:44.000+0000",
"updated": "2022-11-08T22:11:23.000+0000",
"summary": "FA | EJWL-DEV3| ORA-00020: maximum number of processes (10000) exceeded",
"assignee": null
}
}
And, here's the data object I'd like to bind to. I have represented what they should be as jq expressions.
#Serializable
data class MajorIncident constructor(
#SerialName("key")
val id: String, // .key
val created: Instant, // .fields.created
val pillar: String, // .fields.customfield_10443.value
val impactBreadth: String?,
val duration: Duration, // .fields.customfield_13600 as minutes
val detectionSource: String, //.fields.customfield_10900.value
val updated: Instant, // .fields.updated
val assignee: String, // .fields.assignee
// "customfield_10471": [
// "3-30536161871 (SM-2046076)"
// ],
val serviceRequests: List<String>?, // .fields.customfield_10471 | map(split(" ")[0]) -
#SerialName("summary")
val title: String, //.summary
val type: String, // .fields.issuetype.name // what are options?
)
If you're using Kotlinx Serialization, I'm not sure there is any built-in support for jsonpath.
One simple option is to declare your Kotlin model in a way that matches the JSON. If you really want a flattened object, you could convert from the structured model into the flat model from Kotlin.
Another option is to write a custom serializer for your type.
Related
I tried parsing JSON using Kotlin's default serialization library. However, I found it really overwhelming to write a bunch of data classes to deserialize a simple JSON string.
To illustrate,
{
"artists": {
"items": [
{
"genres": [
"desi pop",
"filmi",
"modern bollywood"
],
"images": [
{
"url": "https://i.scdn.co/image/ab6761610000e5ebb2b70762d89a9d76c772b3b6"
}
],
"name": "Arijit Singh",
"type": "artist"
}
]
}
}
for this data, I had to write these many classes,
#Serializable
data class Root(val artists: SubRoot)
#Serializable
data class SubRoot(val items: List<Artist>)
#Serializable
data class Artist(
val genres: List<String>,
val images: List<Image>,
val name: String,
val type: String
)
#Serializable
data class Image(val url: String)
Does anybody know a better way? Some library with in-built magic that does these kind of stuff for me?
If you don't want to use the automatic mapping you can just parse them as JsonElements and do your own thing instead of letting the library map them to those data classes.
https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/json.md#json-elements
For example, if you want to get that url, you could do:
val root = Json.parseToJsonElement(json)
return root.
jsonObject["artists"]?.
jsonObject?.get("items")?.
jsonArray?.get(0)?.
jsonObject?.get("images")?.
jsonArray?.get(0)?.
jsonObject?.get("url")?.
jsonPrimitive.toString()
)
This specific example will return null if any field couldn't be found while traversing the tree. It will give an IllegalArgumentException if any of the casts fail.
I am writing a small scala practice code where my input is going to be in the fashion -
{
"code": "",
"unique ID": "",
"count": "",
"names": [
{
"Matt": {
"name": "Matt",
"properties": [
"a",
"b",
"c"
],
"fav-colour": "red"
},
"jack": {
"name": "jack",
"properties": [
"a",
"b"
],
"fav-colour": "blue"
}
}
]
}
I'll be passing this file as an command line argument.
I want to know that how do I accept the input file parse the json and use the json keys in my code?
You may use a json library such as play-json to parse the json content.
You could either operate on the json AST or you could write case classes that have the same structure as your json file and let them be parsed.
You can find the documentation of the library here.
You'll first have to add playjson as depedency to your project. If you're using sbt, just add to your build.sbt file:
libraryDependencies += "com.typesafe.play" %% "play-json" % "2.6.13"
Play json using AST
Let's read the input file:
import play.api.libs.json.Json
object Main extends App {
// first we'll need a inputstream of your json object
// this should be familiar if you know java.
val in = new FileInputStream(args(0))
// now we'll let play-json parse it
val json = Json.parse(in)
}
Let's extract some fields from the AST:
val code = (json \ "code").as[String]
val uniqueID = (json \ "unique ID").as[UUID]
for {
JsObject(nameMap) ← (json \ "names").as[Seq[JsObject]]
(name, userMeta) ← nameMap // nameMap is a Map[String, JsValue]
} println(s"User $name has the favorite color ${(userMeta \ "fav-colour").as[String]}")
Using Deserialization
As I've just described, we may create case classes that represent your structure:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, UserData]])
case class UserData(name: String, properties: Seq[String], `fav-colour`: String)
In addition you'll need to define an implicit Format e.g. in the companion object of each case class. Instead of writing it by hand you can use the Json.format macro that derives it for you:
object UserData {
implicit val format: OFormat[UserData] = Json.format[UserData]
}
object InputFile {
implicit val format: OFormat[InputFile] = Json.format[InputFile]
}
You can now deserialize your json object:
val argumentData = json.as[InputFile]
I generally prefer this approach but in your case the json structure does not fit really well. One improvement could be to add an additional getter to your InputFile class that makes accesing the fields with space and similar in the name easier:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, String]]) {
// this method is nicer to use
def uniqueId = `unique ID`
}
I'm using Scala & Argonaut, trying to parse the following JSON:
[
{
"name": "apple",
"type": "fruit",
"size": 3
},
{
"name": "jam",
"type": "condiment",
"size": 5
},
{
"name": "beef",
"type": "meat",
"size": 1
}
]
And struggling to work out how to iterate and extract the values into a List[MyType] where MyType will have name, type and size properties.
I will post more specific code soon (i have tried many things), but basically I'm looking to understand how the cursor works, and how to iterate through arrays etc. I have tried using \\ (downArray) to move to the head of the array, then :->- to iterate through the array, then --\ (downField) is not available (at least IntelliJ doesn't think so).
So the question is how do i:
navigate to the array
iterate through the array (and know when I'm done)
extract string, integer etc. values for each field - jdecode[String]? as[String]?
The easiest way to do this is to define a codec for MyType. The compiler will then happily construct a decoder for List[MyType], etc. I'll use a plain class here (not a case class) to make it clear what's happening:
class MyType(val name: String, val tpe: String, val size: Int)
import argonaut._, Argonaut._
implicit def MyTypeCodec: CodecJson[MyType] = codec3(
(name: String, tpe: String, size: Int) => new MyType(name, tpe, size),
(myType: MyType) => (myType.name, myType.tpe, myType.size)
)("name", "type", "size")
codec3 takes two parameter lists. The first has two parameters, which allow you to tell how to create an instance of MyType from a Tuple3 and vice versa. The second parameter list lets you specify the names of the fields.
Now you can just write something like the following (if json is your string):
Parse.decodeValidation[List[MyType]](json)
And you're done.
Since you don't need to encode and are only looking at decoding, you can do as suggested by Travis, but by implementing another implicit: MyTypeDecodeJson
implicit def MyTypeDecodeJson: DecodeJson[MyType] = DecodeJson(
raw => for {
name <- raw.get[String]("name")
type <- raw.get[String]("type")
size <- raw.get[Int]("size")
} yield MyType(name, type, size))
Then to parse your list:
Parse.decodeValidation[List[MyType]](jsonString)
Assuming MyType is a case class, the following works too:
case class MyType(name: String, type: String, size: Int)
object MyType {
implicit val createCodecJson: CodecJson[MyType] = CodecJson.casecodec3(apply, unapply)(
"name",
"type",
"size"
)
}
I have a simple custom data structure which I use to map the results from the database:
case class Filter(id: Int, table: String, name: String, Type: String, structure: String)
The resulting object type is List[Filter] and if converted to JSON, it should look something like this:
[
{
"id": 1,
"table": "table1",
"name": "name1",
"Type": "type1",
"structure": "structure1"
},
{
"id": 2,
"table": "table2",
"name": "name2",
"Type": "type2",
"structure": "structure2"
}
]
Now when I try to serialize my object into JSON
val result: String = Json.toJson(filters)
I am getting something like
No Json deserializer found for type List[Filter]. Try to implement an implicit Writes or Format for this type.
How do I solve this seemingly simple problem without writing some ridiculous amount of boilerplate?
My stack is Play 2.2.1, Scala 2.10.3, Java 8 64bit
Short answer:
Just add:
implicit val filterWrites = Json.writes[Filter]
Longer answer:
If you look at the definition of Json.toJson, you will see that its complete signature is:
def toJson[T](o: T)(implicit tjs: Writes[T]): JsValue = tjs.writes(o)
Writes[T] knows how to take a T and transform it to a JsValue. You will need to have an implicit Writes[Filter] around that knows how to serialize your Filter instance. The good news is that Play's JSON library comes with a macro that can instantiate those Writes[_] for you, so you don't have to write boring code that transforms your case class's fields into JSON values. To invoke this macro and have its value picked up by implicit search add the line above to your scope.
I'm trying to parse some problematic Json in Scala using Play Json and using implicit, but not sure how to proceed...
The Json looks like this:
"rules": {
"Some_random_text": {
"item_1": "Some_random_text",
"item_2": "text",
"item_n": "MoreText",
"disabled": false,
"Other_Item": "thing",
"score": 1
},
"Some_other_text": {
"item_1": "Some_random_text",
"item_2": "text",
"item_n": "MoreText",
"disabled": false,
"Other_Item": "thing",
"score": 1
},
"Some_more_text": {
"item_1": "Some_random_text",
"item_2": "text",
"item_n": "MoreText",
"disabled": false,
"Other_Item": "thing",
"score": 1
}
}
I'm using an implicit reader but because each top level item in rules is effectively a different thing I don't know how to address that...
I'm trying to build a case class and I don't actually need the random text heading for each item but I do need each item.
To make my life even harder after these items are lots of things in other formats which I really don't need. They are unnamed items which just start:
{
random legal Json...
},
{
more Json...
}
I need to end up with the Json I'm parsing in a seq of case classes.
Thanks for your thoughts.
I'm using an implicit reader but because each top level item in rules is effectively a different thing I don't know how to address that...
Play JSON readers depend on knowing names of fields in advance. That goes for manually constructed readers and also for macro generated readers. You cannot use an implicit reader in this case. You need to do some traversing first and extract pieces of Json that do have regular structure with known names and types of fields. E.g. like this:
case class Item(item_1: String, item_2: String, item_n: String, disabled: Boolean, Other_Item: String, score: Int)
implicit val itemReader: Reads[Item] = Json.reads[Item]
def main(args: Array[String]): Unit = {
// parse JSON text and assume, that there is a JSON object under the "rules" field
val rules: JsObject = Json.parse(jsonText).asInstanceOf[JsObject]("rules").asInstanceOf[JsObject]
// traverse all fields, filter according to field name, collect values
val itemResults = rules.fields.collect {
case (heading, jsValue) if heading.startsWith("Some_") => Json.fromJson[Item](jsValue) // use implicit reader here
}
// silently ignore read errors and just collect sucessfully read items
val items = itemResults.flatMap(_.asOpt)
items.foreach(println)
}
Prints:
Item(Some_random_text,text,MoreText,false,thing,1)
Item(Some_random_text,text,MoreText,false,thing,1)
Item(Some_random_text,text,MoreText,false,thing,1)