I have two json string having different order of some of the items, so they are like equal, and I want to compare them for equality, but not sure how to achieve it.
JSON 1
{
"data": [
{
"configTemplateId": "44f11ed4-5b08-11ea-8e2d-0242ac132222",
"params": {
"keyOne": "valueOne",
"keyTwo": "valueTwo"
}
},
{
"configTemplateId": "44f11ed4-5b08-11ea-8e2d-0242ac131111",
"params": {
"keyOne": "valueOne",
"keyTwo": "valueTwo"
}
}
]}
JSON 2
{
"data": [
{
"configTemplateId": "44f11ed4-5b08-11ea-8e2d-0242ac131111",
"params": {
"keyOne": "valueOne",
"keyTwo": "valueTwo"
}
},
{
"configTemplateId": "44f11ed4-5b08-11ea-8e2d-0242ac132222",
"params": {
"keyOne": "valueOne",
"keyTwo": "valueTwo"
}
}
]}
I have tried using spray-json as mentioned in Compare json equality in Scala but that does not seem working.
You have to implement some kind of deepEquals. The idea is the following:
import play.api.libs.json.{JsArray, JsObject, JsValue, Json}
def deepEquals(json1: JsValue, json2: JsValue): Boolean = {
def sortArray(json: JsValue): JsValue = json match {
case JsArray(arr) => JsArray(arr.map(sortArray).sortBy(_.toString))
case other => other
}
if (json1.getClass != json2.getClass) {
false
} else {
(json1, json2) match {
case (j1: JsObject, j2: JsObject) =>
val fields1 = j1.value
val fields2 = j2.value
(fields1.keySet ++ fields2.keySet).foldLeft(true) { (acc, key) =>
val field1 = fields1.get(key)
val field2 = fields2.get(key)
acc && ((field1, field2) match {
case (Some(v1), Some(v2)) => deepEquals(v1, v2)
case _ => false
})
}
case (j1: JsArray, j2: JsArray) =>
sortArray(j1) == sortArray(j2)
case (other1, other2) => other1 == other2
}
}
}
I used json play library, but with spray the implementation won't be much different
Related
How can I make a recursive function which finds a JsValue in JsObject, for example:
Function input: JsObject and path as String
Examples:
Input for JsObject:
{
"name": "Zvi"
parents: {
"father": {
"name": "myFatherName",
"dob": "10/10/70"
}
}
}
and for path Example:
path = notExistsField
the output will be None
path = name
the input will be Some("Zvi")
path = parents
the output will be
Some({
"father": {
"name": "myFatherName",
"dob": "10/10/70"
}
})
parents.father
the output will be:
Some({
"name": "myFatherName",
"dob": "10/10/70"
})
parents.father.name
the output will be "myFatherName"
Tests Example:
"deepSearch" - {
val payload: JsObject = Json.parse(
"""{
| "key1": 1,
| "key2": {
| "key1": "value21",
| "key2": {
| "key1": 221
| }
| },
|}""".stripMargin).as[JsObject]
"found" - {
"nested" in {
// In Nested Object
service.deepSearch(payload,"key2.key1").get shouldBe JsString("value21")
service.deepSearch(payload,"key2.key2.key1").get shouldBe JsNumber(221)
service.deepSearch(payload,"key2.key2").get shouldBe Json.parse("{\"key1\": 221}")
}
"top-level" in {
service.deepSearch(payload,"key1").get shouldBe JsNumber(1)
}
}
"not found" - {
"nested" in {
service.deepSearch(payload,"key2.key2.key1.notExists").isEmpty shouldBe true
service.deepSearch(payload,"key2.key2.notExists").isEmpty shouldBe true
}
"top-level" in {
service.deepSearch(payload,"Boom").isEmpty shouldBe true
}
}
}
I did the following implementation, Is there any suggestions to prettier code
def deepSearch(payload: JsValue, path: String): JsLookupResult = {
path indexOf (".") match {
// For the Base case:
case base if base < 0 => (payload \ path)
// For the Step case:
case found if found >= 0 =>
val untilFirstDot = path.substring(0, found)
(payload \ untilFirstDot) match {
case JsDefined(newPayload) => deepSearch(newPayload, path.substring(found + 1))
case undefined: JsUndefined => undefined
}
}
}
I want to parse a file having content as json format.
From the file I want to extract few properties (name, DataType, Nullable) to create some column names dynamically.
I have gone through some examples but most of them are using case class but my problem is every time I will receive a file may have different content.
I tried to use the ujson library to parse the file but I am unable to understand how to use it properly.
object JsonTest {
def main(args: Array[String]): Unit = {
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
println(source)
val input = try source.mkString finally source.close()
println(input)
val data = ujson.read(input)
data("name") = data("name").str.reverse
val updated = data.render()
}
}
Content of the file example:
{
"Organization": {
"project": {
"name": "POC 4PL",
"description": "Implementation of orderbook"
},
"Entities": [
{
"name": "Shipments",
"Type": "Fact",
"Attributes": [
{
"name": "Shipment_Details",
"DataType": "StringType",
"Nullable": "true"
},
{
"name": "Shipment_ID",
"DataType": "StringType",
"Nullable": "true"
},
{
"name": "View_Cost",
"DataType": "StringType",
"Nullable": "true"
}
],
"ADLS_Location": "/mnt/mns/adls/raw/poc/orderbook/"
}
]
}
}
Expected output:
StructType(
Array(StructField("Shipment_Details",StringType,true),
StructField("Shipment_ID",DateType,true),
StructField("View_Cost",DateType,true)))
StructType needs to be added to the expected output programatically.
Try Using Playframework's Json utils - https://www.playframework.com/documentation/2.7.x/ScalaJson
Here's the solution to your issue-
\ Placed your json in text file
val fil_path = "C:\\TestData\\Config\\Conf.txt"
val conf_source = scala.io.Source.fromFile(fil_path)
lazy val json_str = try conf_source.mkString finally conf_source.close()
val conf_json: JsValue = Json.parse(json_str)
val all_entities: JsArray = (conf_json \ "Organization" \ "Entities").get.asInstanceOf[JsArray]
val shipments: JsValue = all_entities.value.filter(e => e.\("name").as[String] == "Shipments").head
val shipments_attributes: IndexedSeq[JsValue] = shipments.\("Attributes").get.asInstanceOf[JsArray].value
val shipments_schema: StructType = StructType(shipments_attributes.map(a => Tuple3(a.\("name").as[String], a.\("DataType").as[String], a.\("Nullable").as[String]))
.map(x => StructField(x._1, StrtoDatatype(x._2), x._3.toBoolean)))
shipments_schema.fields.foreach(println)
Output is -
StructField(Shipment_Details,StringType,true)
StructField(Shipment_ID,StringType,true)
StructField(View_Cost,StringType,true)
It depends if you want it to be completely dynamic or not, here are some options:
If you just want to read one field you can do:
import upickle.default._
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
val json = ujson.read(input)
println(json("Organization")("project")("name"))
the output will be: "POC 4PL"
If you just want just the Attributes to be with types, you can do:
import upickle.default.{macroRW, ReadWriter => RW}
import upickle.default._
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
val json = ujson.read(input)
val entitiesArray = json("Organization")("Entities")(0)("Attributes")
println(read[Seq[StructField]](entitiesArray))
case class StructField(name: String, DataType: String, Nullable: String)
object StructField{
implicit val rw: RW[StructField] = macroRW
}
the output will be: List(StructField(Shipment_Details,StringType,true), StructField(Shipment_ID,StringType,true), StructField(View_Cost,StringType,true))
another option, is to use a different library to do the class mapping. If you use Google Protobuf Struct and JsonFormat it can be 2-liner:
import com.google.protobuf.Struct
import com.google.protobuf.util.JsonFormat
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
JsonFormat.parser().merge(input, builder)
println(builder.build())
the output will be: fields { key: "Organization" value { struct_value { fields { key: "project" value { struct_value { fields { key: "name" value { string_value: "POC 4PL" } } fields { key: "description" value { string_value: "Implementation of orderbook" } } } } } fields { key: "Entities" value { list_value { values { struct_value { fields { key: "name" value { string_value: "Shipments" } }...
I am trying to parse a json data from a server.It has dynamic keys so I am trying to have like a parent class that have the shared keys and child class for each specific node. I wrote a kotlin code using retrofit and Moshi but it's not working. I tried with a sealed class and interface without success. Actually I would prefer that works with sealed class but I don't know what I am doing wrong
interface MyApi {
#GET("/...")
fun fetchMyFeed(): Call<MyResponse>
}
data class MyResponse(
val data: List<ParentResponse>
)
interface ParentResponse{
val name: String
}
data class Child1Response(
val age: String,
val kids: List<KidsResponse>,
val cars: List<CarsResponse>
)
data class Child2Response(
val job: String,
val address: List<AddressResponse>
)
fun fetchAllFeed(): List<Any>? =
try {
val response = api.fetchMyFeed().execute()
if (response.isSuccessful) {
Log.d("check",${response.body()?.data?})
null
} else null
} catch (e: IOException) {
null
} catch (e: RuntimeException) {
null
}```
and the json file is :
{
"data": [
{
"name": "string",
"job": "string",
"address": [
{
"avenue": "string",
"imageUrl": "string",
"description": "string"
}
]
},
{
"name": "string",
"age": "string",
"kids": {
"count": "string",
"working": "string"
},
"cars": [
{
"brand": "string",
"age": "string",
"imageUrl": "string"
}
]
}
]
}
Unable to create converter for class
You can make use of JsonAdapter from moshi to parse different JSON Models if you can differentiate them by foreseeing some value in the json.
for example, consider json response having two schemas,
{
"root": {
"subroot": {
"prop" : "hello",
"type" : "String"
}
}
}
(or)
{
"root": {
"subroot": {
"prop" : 100,
"type" : "Integer"
}
}
}
Here, subroot has different schemas (one containing string property and another containg a integer property) which can be identified by "type"
You can create a parent sealed class with common keys and derive few child classes with varying keys. Write a adapter to select the type of class to be used while json serialization and add that adapter to moshi builder.
Model classes:
class Response {
#Json(name = "root")
val root: Root? = null
}
class Root {
#Json(name = "subroot")
val subroot: HybridModel? = null
}
sealed class HybridModel {
#Json(name = "type")
val type: String? = null
class StringModel : HybridModel() {
#Json(name = "prop")
val prop: String? = null
}
class IntegerModel : HybridModel() {
#Json(name = "prop")
val prop: Int? = null
}
}
Few extension methods to JsonReader,
inline fun JsonReader.readObject(process: () -> Unit) {
beginObject()
while (hasNext()) {
process()
}
endObject()
}
fun JsonReader.skipNameAndValue() {
skipName()
skipValue()
}
HybridAdapter to select type of class for "subroot" key
class HybridAdapter : JsonAdapter<HybridModel>() {
#FromJson
override fun fromJson(reader: JsonReader): HybridModel {
var type: String = ""
// copy reader and foresee type
val copy = reader.peekJson()
copy.readObject {
when (copy.selectName(JsonReader.Options.of("type"))) {
0 -> {
type = copy.nextString()
}
else -> copy.skipNameAndValue()
}
}
//handle exception if type cannot be identified
if (type.isEmpty()) throw JsonDataException("missing type")
// build model based on type
val moshi = Moshi.Builder().build()
return if (type == "String")
moshi.adapter(HybridModel.StringModel::class.java).fromJson(reader)!!
else
moshi.adapter(HybridModel.IntegerModel::class.java).fromJson(reader)!!
}
#ToJson
override fun toJson(p0: JsonWriter, p1: HybridModel?) {
// serialization logic
}
}
Finally build Moshi with the HybridAdapter to serialize HybridModel,
fun printProp(response: Response?) {
val subroot = response?.root?.subroot
when (subroot) {
is HybridModel.StringModel -> println("string model: ${subroot.prop}")
is HybridModel.IntegerModel -> println("Integer model: ${subroot.prop}")
}
}
fun main() {
val jsonWithStringSubroot =
"""
{
"root": {
"subroot": {
"prop" : "hello",
"type" : "String"
}
}
}
"""
val jsonWithIntegerSubroot =
"""
{
"root": {
"subroot": {
"prop" : 1,
"type" : "Integer"
}
}
}
"""
val moshi = Moshi.Builder().add(HybridAdapter()).build()
val response1 = moshi.adapter(Response::class.java).fromJson(jsonWithStringSubroot)
printProp(response1) // contains HybridModel.StringModel
val response2 = moshi.adapter(Response::class.java).fromJson(jsonWithIntegerSubroot)
printProp(response2) // contains HybridModel.IntegerModel
}
I have a json file , which has some keys and values. I need to parse the Json and print the keys and their values. For example, the json file is like below. I want to print this as Keys and values
{
"Parcer":[
{
"key":"0203",
"value":{
"Encryption":
{
"enabled":"yes",
"encryption_type":"base64",
"key":"334848484",
"return":"name"
}
}
},
{
"key":"0405",
"value":{
"Encryption":
{
"enabled":"yes",
"encryption_type":"base64",
"key":"334848484",
"return":"none"
},
"Parcer":[
{
"key":"0102",
"value":"humidity"
},
{
"key":"0304",
"value":{
"Encryption":{
"enabled":"yes",
"encryption_type":"SHA1",
"key":"1211212",
"return":"none"
}
}
}
]
}
}],
}```
The easiest way is to create a case class, like:
case class MyObj(header:String, value: Seq[Map[String, String]])
Then you just need to add one line for marshalling, like:
import play.api.libs.json._
object MyObj {
implicit val jsonFormat: OFormat[MyObj] = Json.format[MyObj]
}
Now you get a nice case class that you can work with:
val json =
Json.parse(
"""{
"header" : "header value",
"value" : [
{
"a" : "a_val",
"b" : "b_val",
"c" : "c_val"
},
{
"a" : "a_val",
"b" : "b_val",
"c" : "c_val"
}
]
}""")
Here an example how to retrieve all "a".
json.validate[MyObj] match {
case JsSuccess(myObj, _) =>
val allAs =myObj.value.flatMap(m => m.get("a").toSeq)
println(allAs) // >> Vector(a_val, a_val)
case e:JsError => // handle error
}
This gives you:
json.validate[MyObj] returns JsSuccess(MyObj(header value,Vector(Map(a -> a_val, b -> b_val, c -> c_val), Map(a -> a_val, b -> b_val, c -> c_val))),)
The println returns: Vector(a_val, a_val)
This is described here in the Documentation: JSON automated mapping
I have a Seq of some Strings mostly like below:
val states = Seq(
"CA" -> Seq("Los Angeles" -> Seq("SunsetBlvd", "Hollywood" -> Seq("W 8th St", "W 9th St")), "Pasadena"),
"WA" -> Seq("Seattle", "Redmond")
)
The case class for this could be
case class State(name: String, sub: Option[Seq[State]])
And an implicit Writes
implicit val stateWrites = Json.Writes[State]
Hoping to convert it to a Json like
[
{
"name": "CA",
"sub": [
{
"name": "Los Angeles",
"sub": [
{
"name": "SunsetBlvd"
},
{
"name": "Hollywood",
"sub": [
{
"name": "W 8th St"
},
{
"name": "W 9th St"
}
]
}
]
}
]
},
{
"name": "WA",
"sub": [
{
"name": "Seattle"
},
{
"name": "Redmond"
}
]
}
]
How do I correctly model the data and be able to convert this Seq to a Json using Writes?
Or even change the states val to an appropriate format so I could easily convert it to Json?
In the case class, one of fields is of type itself which is wrong. How do I avoid that in modeling the data or even the Seq?
I came up with something like this:
case class State(name: String, sub: Option[Seq[State]])
import play.api.libs.json._
implicit val optWrites = new Writes[Option[Seq[State]]] {
override def writes(o: Option[Seq[State]]) = {
if (o.isDefined) {
Json.toJson(o.get)(stateSeqWrites)
} else {
JsNull
}
}
}
implicit val stateWrites = new Writes[State] {
def writes(state: State) = {
val l: Seq[(String, JsValueWrapper)] = Seq("name" -> JsString(state.name))
val ll: Seq[(String, JsValueWrapper)] = if (state.sub.isDefined) {
val subValue: JsValueWrapper = Json.toJson(state.sub)(optWrites)
l :+ ("sub" -> subValue)
} else {
l
}
Json.obj(ll : _*)
}
}
implicit val stateSeqWrites: Writes[Seq[State]] = new Writes[Seq[State]] {
override def writes(s: Seq[State]) = {
JsArray(s.map(Json.toJson(_)(stateWrites)))
}
}
val states = Seq(
State("CA", Some(Seq(State("Los Angeles", Some(Seq(State("SunsetBlvd", None), State("Hollywood", Some(Seq(State("W 8th St", None), State("W 9th St", None)))), State("Pasadena", None))))))),
State("WA", Some(Seq(State("Seattle", None), State("Redmond", None))))
)
val json = Json.toJson(states)
println(json.toString())
Probably might get simplified, but it's late night here ;) It does what you need :)
That information has conceptually a tree structure. My advice is to threat it with just a normal case class, to simplify the json formatter and to have a much more semantic structure:
case class Tree(name: String, sub: Option[List[Tree]])
And your formatter would be just like this:
implicit val repositoryFormat: Format[Tree] = (
(__ \ 'name).format[String] ~
(__ \ 'sub).lazyFormatNullable(implicitly[ Format[ List[Tree] ]])
)(Tree.apply, unlift(Tree.unapply))
Notice I used lazyFormatNullable to deal with the recursive reference to Tree in sub.
To simulate the Json you posted, I made the translation to the Tree case class structure.
// The tree leaves
val hollywoodLeaves = Some( Tree("W 8th St", None) :: Tree("W 9th St", None) :: Nil )
val losAngelesLeaves = Some( Tree("SunsetBlvd", None) :: Tree("Hollywood", hollywoodLeaves ) :: Nil )
// The two trees in your array
val firstTree = Tree( "CA", Some( Tree("Los Angeles", losAngelesLeaves) :: Nil ) )
val secondTree = Tree("WA", Some( Tree("Seattle", None) :: Tree("Redmond", None) :: Nil ))
// Your root array
val treeArray = firstTree :: secondTree :: Nil
// Conversion to json
val json = Json.toJson(treeArray)