So I have a json string in scala which looks something like this
"""
{
"input": {
"House" :{
"Tile" : "Ceramic"
"Kitchen" : {
"Sink" : "Stainless-Steel"
"Counter-Top" : "Granite"
}
}
}
}
"""
I'm using json4s to parse this and to put it into a Map and then I'm making it mutable so I can edit it, but I don't really know how to take it and modify it to make it a new json object.
import org.json4s.jackson.JsonMethods._
import org.json4s._
import org.json4s.native.Serialization._
import org.json4s.native.Serialization
import scala.collection.mutable
...
val reqJsonMap = parse(reqJson).extract[Map[String,Any]]
val reqJsonMutableMap= collection.mutable.Map[String,Any]()
reqJsonMutableMap ++=reqJsonMap
What I want to do is I want to edit it and make the json of Tile=Marble and change the key of Kitchen to Bathroom
I just don't know how to turn that object into this
"""
{
"input": {
"House" :{
"Tile" : "Marble"
"Bathroom" : {
"Sink" : "Stainless-Steel"
"Counter-Top" : "Granite"
}
}
}
}
"""
I think it is far better to work with case classes than with JSON. I think it is the most convenient to use a Json parsing library such as circe to parse your Json string into a case class, and then do whatever thing you need to do with it:
import io.circe._, io.circe.generic.auto._, io.circe.syntax._, io.circe.parser._
val myJsonString = """
{
"input": {
"house" :{
"tile" : "Ceramic",
"kitchen" : {
"sink" : "Stainless-Steel",
"counterTop" : "Granite"
}
}
}
}
"""
case class Data(input: Input)
case class Input(house: HouseInfo)
case class HouseInfo(tile: String, kitchen: KitchenInfo)
case class KitchenInfo(sink: String, counterTop: String)
val maybeDataJsonObject = parse(myJsonString)
val maybeData = maybeDataJsonObject match {
case Right(dataJsonObject) => dataJsonObject.as[Data]
case Left(failure) => println(failure)
}
maybeData match {
case Right(data) => println(data)
case Left(failure) => println(failure)
}
By the way, the format of your json is not good, because you have capitalized the initials of the field names, which should conventionally be lower cased. Also, you have written it as Counter-Top, instead of let’s say counterTop and the - character would be problematic in field names in Scala.
For applying the transformations you are looking for, you need to write new case classes with desired field names, then you can write a transformer function to transform the obtained Data object into the TransformedData one. Then you would be able to use circe to obtain a new Json string by coding the case class in Json, like in this:
val transformedJsonString = myTransformedData.asJson.toString
Related
I started with the accepted answer to SO question 53573659 which has a nested list of attrs and uses the auto-parser to get the data into case classes. I want to be able to handle the same data but with the nested fields having kebab-case rather than camel case.
Here is the same input JSON with the kebab-case fields
val sampleKebab="""{
"parent" : {
"name" : "title",
"items" : [
{
"foo" : "foo1",
"attrs" : {
"attr-a" : "attrA1",
"attr-b" : "attrB1"
}
},
{
"foo" : "foo2",
"attrs" : {
"attr-a" : "attrA2",
"attr-b" : "attrB2",
"attr-c" : "attrC2"
}
}
]
}
}"""
I can decode the attrs data by itself using the following example
import io.circe.derivation.deriveDecoder
import io.circe.{Decoder, derivation}
import io.circe.generic.auto._
import io.circe.parser._
val attrKebabExample = """{
"attr-a": "attrA2",
"attr-b": "attrB2",
"attr-c": "attrC2"
}"""
case class AttrsKebab(attrA: String, attrB: String)
implicit val decoder: Decoder[AttrsKebab] = deriveDecoder(derivation.renaming.kebabCase)
val attrKebabData = decode[AttrsKebab](attrKebabExample)
attrKebabData decodes to
Either[io.circe.Error,AttrsKebab] = Right(AttrsKebab(attrA2,attrB2))
When I try to tie this decoder into the case class hierarchy from the original question, it exposes some glue that I am missing to hold it all together
case class ItemKebab(foo: String, attrs : AttrsKebab)
case class ParentKebab(name: String, items: List[ItemKebab])
case class DataKebab(parent : ParentKebab)
case class Data(parent : Parent)
val dataKebab=decode[DataKebab](sample)
In this case, dataKebab contains a DecodingFailure
Either[io.circe.Error,DataKebab] = Left(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(attr-a), DownField(attrs), DownArray, DownField(items), DownField(parent))))
My guess is that either the decoder I defined is being ignored, or I need to explicitly define more of the decode process, but I'm looking for some help to find what the solution might be.
Situation
Say I have this JSON, which I get from some server:
{
"useless_info": "useless info",
"data": {
"useless_info2": "useless info 2",
"children": [
{
"kind": "Car",
"data": {
"id": 1,
"transmission": "manual"
}
},
{
"kind": "Boat",
"data": {
"id": 2,
"isDocked": true
}
}
]
}
}
This JSON is an example. It represent a much larger and complex one, but with similar structure. I have no say in it's structure, so I must adapt to it.
What I Want
The JSON has deep inside it an array of vehicle objects. Say I only care about the ID.
I could model it like this:
#Serializable
data class VehicleResponse(
#Serializable(with = VehicleResponseSerializer::class)
#SerialName("data")
val vehicles: List<Vehicle>
)
#Serializable
data class Vehicle(val id: Int)
In order to avoid modeling the response exactly by writing many neste data classes (which would be awful with the real one, which is more nested and complex), I can use a JsonTransformingSerializer to cut right to part I want, like this:
object VehicleResponseSerializer : JsonTransformingSerializer<List<Vehicle>>(ListSerializer(Vehicle.serializer())) {
override fun transformDeserialize(element: JsonElement): JsonElement {
val vehicles = mutableListOf<JsonElement>()
val vehicleArray = element.jsonObject["children"]!!.jsonArray
// equals: [{"kind":"Car","data":{"id":1,"totalWheels":"4"}},{"kind":"Boat","data":{"id":2,"isDocked":true}}]
vehicleArray.forEach { vehicle ->
val vehicleData = vehicle.jsonObject["data"]!!
// equals: {"id":1,"totalWheels":"4"}}
vehicles.add(vehicleData)
}
return JsonArray(vehicles.toList())
}
}
Calling it from main:
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json
fun main() {
val vehicles: VehicleResponse = configuredJson.decodeFromString(vehicleJson)
println(vehicles)
}
val configuredJson = Json {
ignoreUnknownKeys = true
}
Prints:
VehicleResponse(vehicles=[Vehicle(id=1), Vehicle(id=2)])
So if it works perfectly, what's even the problem
It's this part:
#Serializable
data class VehicleResponse(
#Serializable(with = VehicleResponseSerializer::class)
#SerialName("data")
val vehicles: List<Vehicle>
)
I have a problem with the #SerialName. It feels like a hack to me. All this accomplishes is give the custom serializer the data object rather than the full original object from the response. If you remember from the custom serializer, I cut right to the array I want by walking down the structure. It would be trivial to get to the data object. So, instead of
val vehicleArray = element.jsonObject["children"]!!.jsonArray
I would have to do:
val vehicleArray = element.jsonObject["data"]!!.jsonObject["children"]!!.jsonArray
Currently I feel like what I'm doing doesn't really describe the code. In my VehicleResponse data class, I write that my vehicles list does has a serial name of "data", when it doesn't at all. This could lead me to misinterpret the structure of JSON in the future, or even confuse me as to what I'm trying to do. With this simple example it's OK, but what about more complex ones?
What I Tried
Changing VehicleResponse to this:
#Serializable(with = VehicleResponseSerializer::class)
data class VehicleResponse(
val vehicles: List<Vehicle>
)
And altering the line that gets the array inside the custom serializer to this:
override fun transformDeserialize(element: JsonElement): JsonElement {
...
val vehicleArray = element.jsonObject["data"]!!.jsonObject["children"]!!.jsonArray
...
}
Running this code gives the following error:
Exception in thread "main" java.lang.ClassCastException: class java.util.ArrayList cannot be cast to class VehicleResponse (java.util.ArrayList is in module java.base of loader 'bootstrap'; VehicleResponse is in unnamed module of loader 'app')
at MainKt.main(Main.kt:5)
at MainKt.main(Main.kt)
Process finished with exit code 1
For reference, I tried to print the vehicles array inside the serializer, to see it by any chance I messed up navigating the JSON.
override fun transformDeserialize(element: JsonElement): JsonElement {
...
println(vehicles)
return JsonArray(vehicles.toList())
}
It prints:
[{"id":1,"transmission":"manual"}, {"id":2,"isDocked":true}]
So everything seems to be fine. Not sure what I'm doing wrong.
I have a JSON body in the following form:
val body =
{
"a": "hello",
"b": "goodbye"
}
I want to extract the VALUE of "a" (so I want "hello") and store that in a val.
I know I should use "parse" and "Extract" (eg. val parsedjson = parse(body).extract[String]) but I don't know how to use them to specifically extract the value of "a"
To use extract you need to create a class that matches the shape of the JSON that you are parsing. Here is an example using your input data:
val body ="""
{
"a": "hello",
"b": "goodbye"
}
"""
case class Body(a: String, b: String)
import org.json4s._
import org.json4s.jackson.JsonMethods._
implicit val formats = DefaultFormats
val b = Extraction.extract[Body](parse(body))
println(b.a) // hello
You'd have to use pattern matching/extractors:
val aOpt: List[String] = for {
JObject(map) <- parse(body)
JField("a", JString(value)) <- map
} yield value
alternatively use querying DSL
parse(body) \ "a" match {
case JString(value) => Some(value)
case _ => None
}
These are options as you have no guarantee that arbitrary JSON would contain field "a".
See documentation
extract would make sense if you were extracting whole JObject into a case class.
I am writing a small scala practice code where my input is going to be in the fashion -
{
"code": "",
"unique ID": "",
"count": "",
"names": [
{
"Matt": {
"name": "Matt",
"properties": [
"a",
"b",
"c"
],
"fav-colour": "red"
},
"jack": {
"name": "jack",
"properties": [
"a",
"b"
],
"fav-colour": "blue"
}
}
]
}
I'll be passing this file as an command line argument.
I want to know that how do I accept the input file parse the json and use the json keys in my code?
You may use a json library such as play-json to parse the json content.
You could either operate on the json AST or you could write case classes that have the same structure as your json file and let them be parsed.
You can find the documentation of the library here.
You'll first have to add playjson as depedency to your project. If you're using sbt, just add to your build.sbt file:
libraryDependencies += "com.typesafe.play" %% "play-json" % "2.6.13"
Play json using AST
Let's read the input file:
import play.api.libs.json.Json
object Main extends App {
// first we'll need a inputstream of your json object
// this should be familiar if you know java.
val in = new FileInputStream(args(0))
// now we'll let play-json parse it
val json = Json.parse(in)
}
Let's extract some fields from the AST:
val code = (json \ "code").as[String]
val uniqueID = (json \ "unique ID").as[UUID]
for {
JsObject(nameMap) ← (json \ "names").as[Seq[JsObject]]
(name, userMeta) ← nameMap // nameMap is a Map[String, JsValue]
} println(s"User $name has the favorite color ${(userMeta \ "fav-colour").as[String]}")
Using Deserialization
As I've just described, we may create case classes that represent your structure:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, UserData]])
case class UserData(name: String, properties: Seq[String], `fav-colour`: String)
In addition you'll need to define an implicit Format e.g. in the companion object of each case class. Instead of writing it by hand you can use the Json.format macro that derives it for you:
object UserData {
implicit val format: OFormat[UserData] = Json.format[UserData]
}
object InputFile {
implicit val format: OFormat[InputFile] = Json.format[InputFile]
}
You can now deserialize your json object:
val argumentData = json.as[InputFile]
I generally prefer this approach but in your case the json structure does not fit really well. One improvement could be to add an additional getter to your InputFile class that makes accesing the fields with space and similar in the name easier:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, String]]) {
// this method is nicer to use
def uniqueId = `unique ID`
}
Scala 2.12 here trying to use Lift-JSON to parse a config file. I have the following myapp.json config file:
{
"health" : {
"checkPeriodSeconds" : 10,
"metrics" : {
"stores" : {
"primary" : "INFLUX_DB",
"fallback" : "IN_MEMORY"
}
}
}
}
And the following MyAppConfig class:
case class MyAppConfig()
My myapp.json is going to evolve and potentially become very large with lots of nested JSON structures inside of it. I don't want to have to create Scala objects for each JSON object and then inject that in MyAppConfig like so:
case class Stores(primary : String, fallback : String)
case class Metrics(stores : Stores)
case class Health(checkPeriodSeconds : Int, metrics : Metrics)
case class MyAppConfig(health : Health)
etc. The reason for this is I'll end up with "config object sprawl" with dozens upon dozens of case classes that are only in existence to satisfy serialization from JSON into Scala-land.
Instead, I'd like to use Lift-JSON to read the myapp.json config file, and then have MyAppConfig just have helper functions that read/parse values out of the JSON on the fly:
import net.liftweb.json._
// Assume we instantiate MyAppConfig like so:
//
// val json = Source.fromFile(configFilePath)
// val myAppConfig : MyAppConfig = new MyAppConfig(json.mkString)
//
class MyAppConfig(json : String) {
implicit val formats = DefaultFormats
def primaryMetricsStore() : String = {
// Parse "INFLUX_DB" value from health.metrics.stores.primary
}
def checkPeriodSeconds() : Int = {
// Parse 10 value from health.checkPeriodSeconds
}
}
This way I can cherry pick which configs I want to expose (make readable) to my application. I'm just not following the Lift API docs to see how this strategy is possible, they all seem to want me to go with creating tons of case classes. Any ideas?
Case classes are not mandatory for extracting data from JSON. You can query the parsed tree and transfrom data according to your needs. The values from the example can be extracted as follows:
import net.liftweb.json._
class MyAppConfig(json : String) {
private implicit val formats = DefaultFormats
private val parsed = parse(json)
def primaryMetricsStore() : String = {
(parsed \ "health" \ "metrics" \ "stores" \ "primary").extract[String]
}
def checkPeriodSeconds() : Int = {
(parsed \ "health" \ "checkPeriodSeconds").extract[Int]
}
}
The original doc provides all the details.