JSON decode from object to string, and encode it back - json

I am trying to decode a JSON string to case class and also transform one of the field from Object to JSON String.
Currently, I have only finished the decoding part, but don't know how to implement the encoding part.
Here is my code:
case class Foo(objStr: String)
implicit val encoder: Encoder[Foo] = deriveEncoder[Foo]
implicit val decoder: Decoder[Foo] = deriveDecoder[Foo].prepare { cursor =>
cursor.withFocus { json =>
json.mapObject { o =>
o.add("objStr", o("objStr").map(_.noSpaces.asJson).getOrElse(Json.Null))
}
}
}
val json =
"""{
| "objStr": {
| "0": 1,
| "1": 3.4,
| "2": "abc"
| }
|}""".stripMargin
val foo = io.circe.parser.decode[Foo](json).right.get
println(foo)
// output `{"0":1,"1":3.4,"2":"abc"}` as expected
println(foo.asJson.noSpaces)
// output `{"objStr":"{\"0\":1,\"1\":3.4,\"2\":\"abc\"}"}`
// but I expect `{"objStr":{"0":1,"1":3.4,"2":"abc"}}`
Please help me to finish the encoding part.
Thank you.
Updated:
Found a workable solution:
implicit val encoder: Encoder[Foo] = deriveEncoder[Foo].mapJson { json =>
json.mapObject { o =>
o.add("objStr", o("objStr").flatMap(_.asString).flatMap(parse(_).toOption).getOrElse(Json.Null))
}
}

Related

How to create a List from a HTTP response string based on certain conditions?

I am trying to parse the result of a HTTPS request in Scala.
The HTTPS response is a String as follows:
{
"rows":
[
{
"log_forwarding_ip":"",
"device_name":"AD1",
"id":"51",
"mgmt_ip_addr":"192.168.25.150",
"log_forwarding":"1",
"isActive":"0"
},
{
"log_forwarding_ip":"192.168.1.1",
"device_name":"WIN-SRV2019",
"id":"50",
"mgmt_ip_addr":"192.168.25.151",
"log_forwarding":"1",
"isActive":"1"
},
{
"log_forwarding_ip":"129.168.1.2",
"device_name":"PA",
"id":"3",
"mgmt_ip_addr":"192.168.1.161",
"log_forwarding":"1",
"isActive":"1"
}
],
"status":1
}
I have to create a List of all id's where isActive and log_forwarding are both equal to 1.
So far what I have done is:
object syncTables {
def main(args: Array[String]): Unit = {
case class deviceInfo(log_forwarding_ip: String, device_name: String, id: String,
mgmt_ip_addr: String, log_forwarding: String, isActive: String)
try {
val r = requests.get("https://192.168.1.253/api/device/deviceinfo.php", verifySslCerts = false)
if (r.statusCode == 200) {
val x = r.text
println(x)
} else {
println("Error in API call: "+r.statusCode)
}
}
}
}
Now I'm really confused what to do next to achieve my result. I'm totally new to JSON, that's why I don't know which JSON library I should use.
I tried using Play Framework but it seems all pretty complicated to me.
Does Scala offer something like Python's json module where this task can be easily done by using dictionaries and lists.
I'm using Scala 2.11.12 and com.lihaoyi.requests.Any kind of help will be highly appreciated.Thanks in advance.
Use json4s to parse the JSON string. Let's call your JSON input string json, then you can do something like this
import org.json4s._
import org.json4s.jackson.JsonMethods._
case class DeviceInfo(log_forwarding_ip: String, device_name: String, id: String,
mgmt_ip_addr: String, log_forwarding: String, isActive: String)
implicit val formats: Formats = DefaultFormats // Brings in default date formats etc.
val parsedJson = parse(json)
val deviceInfos = parsedJson match {
case JObject(head :: _) =>
head._2
.extract[List[DeviceInfo]]
.filter(_.isActive == 1 && _.log_forwarding == 1)
}
This will output
val res0: List[DeviceInfo] = List(DeviceInfo("192.168.1.1","WIN-SRV2019","50","192.168.25.151","1","1"),DeviceInfo("129.168.1.2","PA","3","192.168.1.161","1","1"))

JSON decode nested field as Map[String, String] in Scala using circe

A circe noob here. I am trying to decode a JSON string to case class in Scala using circe. I want one of the nested fields in the input JSON to be decoded as a Map[String, String] instead of creating a separate case class for it.
Sample code:
import io.circe.parser
import io.circe.generic.semiauto.deriveDecoder
case class Event(
action: String,
key: String,
attributes: Map[String, String],
session: String,
ts: Long
)
case class Parsed(
events: Seq[Event]
)
Decoder[Map[String, String]]
val jsonStr = """{
"events": [{
"ts": 1593474773,
"key": "abc",
"action": "hello",
"session": "def",
"attributes": {
"north_lat": -32.34375,
"south_lat": -33.75,
"west_long": -73.125,
"east_long": -70.3125
}
}]
}""".stripMargin
implicit val eventDecoder = deriveDecoder[Event]
implicit val payloadDecoder = deriveDecoder[Parsed]
val decodeResult = parser.decode[Parsed](jsonStr)
val res = decodeResult match {
case Right(staff) => staff
case Left(error) => error
}
I am ending up with a decoding error on attributes field as follows:
DecodingFailure(String, List(DownField(north_lat), DownField(attributes), DownArray, DownField(events)))
I found an interesting link here on how to decode JSON string to a map here: Convert Json to a Map[String, String]
But I'm having little luck as to how to go about it.
If someone can point me in the right direction or help me out on this that will be awesome.
Let's parse the error :
DecodingFailure(String, List(DownField(geotile_north_lat), DownField(attributes), DownArray, DownField(events)))
It means we should look in "events" for an array named "attributes", and in this a field named "geotile_north_lat". This final error is that this field couldn't be read as a String. And indeed, in the payload you provide, this field is not a String, it's a Double.
So your problem has nothing to do with Map decoding. Just use a Map[String, Double] and it should work.
So you can do something like this:
final case class Attribute(
key: String,
value: String
)
object Attribute {
implicit val attributesDecoder: Decoder[List[Attribute]] =
Decoder.instance { cursor =>
cursor
.value
.asObject
.toRight(
left = DecodingFailure(
message = "The attributes field was not an object",
ops = cursor.history
)
).map { obj =>
obj.toList.map {
case (key, value) =>
Attribute(key, value.toString)
}
}
}
}
final case class Event(
action: String,
key: String,
attributes: List[Attribute],
session: String,
ts: Long
)
object Event {
implicit val eventDecoder: Decoder[Event] = deriveDecoder
}
Which you can use like this:
val result = for {
json <- parser.parse(jsonStr).left.map(_.toString)
obj <- json.asObject.toRight(left = "The input json was not an object")
eventsRaw <- obj("events").toRight(left = "The input json did not have the events field")
events <- eventsRaw.as[List[Event]].left.map(_.toString)
} yield events
// result: Either[String, List[Event]] = Right(
// List(Event("hello", "abc", List(Attribute("north_lat", "-32.34375"), Attribute("south_lat", "-33.75"), Attribute("west_long", "-73.125"), Attribute("east_long", "-70.3125")), "def", 1593474773L))
// )
You can customize the Attribute class and its Decoder, so their values are Doubles or Jsons.

How to deserialize JSON with dynamic object?

I have a simple json, but the containing field has dynamic object. For instance, json can look like
{
"fixedField1": "value1",
"dynamicField1": {
"f1": "abc",
"f2": 123
}
}
or
{
"fixedField1": "value2",
"dynamicField1": {
"g1": "abc",
"g2": { "h1": "valueh1"}
}
}
I am trying to serialize this object, but not sure how to map the dynamic field
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
val dynamicField: Map<String, Any> // ???? what should be the type?
)
Above code fails with following error
Backend Internal error: Exception during code generation Cause:
Back-end (JVM) Internal error: Serializer for element of type Any has
not been found.
I ran into a similar problem when I had to serialize arbitrary Map<String, Any?>
The only way I managed to do this so far was to use the JsonObject/JsonElement API and combining it with the #ImplicitReflectionSerializer
The major downside is the use of reflection which will only work properly in JVM and is not a good solution for kotlin-multiplatform.
#ImplicitReflectionSerializer
fun Map<*, *>.toJsonObject(): JsonObject = JsonObject(map {
it.key.toString() to it.value.toJsonElement()
}.toMap())
#ImplicitReflectionSerializer
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Boolean -> JsonPrimitive(this)
is Map<*, *> -> this.toJsonObject()
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
is Array<*> -> JsonArray(this.map { it.toJsonElement() })
else -> {
//supporting classes that declare serializers
val jsonParser = Json(JsonConfiguration.Stable)
val serializer = jsonParser.context.getContextualOrDefault(this)
jsonParser.toJson(serializer, this)
}
}
Then, to serialize you would use:
val response = mapOf(
"fixedField1" to "value1",
"dynamicField1" to mapOf (
"f1" to "abc",
"f2" to 123
)
)
val serialized = Json.stringify(JsonObjectSerializer, response.toJsonObject())
Note
This reflection based serialization is only necessary if you are constrained to use Map<String, Any?>
If you are free to use your own DSL to build the responses, then you can use the json DSL directly, which is very similar to mapOf
val response1 = json {
"fixedField1" to "value1",
"dynamicField1" to json (
"f1" to "abc",
"f2" to 123
)
}
val serialized1 = Json.stringify(JsonObjectSerializer, response1)
val response 2 = json {
"fixedField1" to "value2",
"dynamicField1" to json {
"g1" to "abc",
"g2" to json { "h1" to "valueh1"}
}
}
val serialized2 = Json.stringify(JsonObjectSerializer, response2)
If, however you are constrained to define a data type, and do serialization as well as deserialization you probably can't use the json DSL so you'll have to define a #Serializer using the above methods.
An example of such a serializer, under Apache 2 license, is here: ArbitraryMapSerializer.kt
Then you can use it on classes that have arbitrary Maps. In your example it would be:
#Serializable
data class Response(
#SerialName("fixedField1")
val fixedField: String,
#SerialName("dynamicField1")
#Serializable(with = ArbitraryMapSerializer::class)
val dynamicField: Map<String, Any>
)

Convert JSON to MultiMap in Scala

I am using Scala with Play. I have a JSON file with all the countries in the world and their respective cities. The JSON looks like this:
{
"CountryA": ["City1","city2"],
"CountryB": ["City1"]
}
I parse it accordingly:
val source: String = Source.fromFile("app/assets/jsons/countriesToCities.json").getLines.mkString
val json: JsValue = Json.parse(source)
My ultimate goal is to convert the json contents into a Scala MultiMap where the key is a String - the country, and the value is a Set[String] - the cities.
Thanks in advance!
Here's a long-winded solution that won't throw if the JSON structure doesn't match what you're expecting:
val source: String =
"""
|{
| "CountryA": ["City1","city2"],
| "CountryB": ["City1"]
|}
""".stripMargin
val json: JsValue = Json.parse(source)
import scala.collection.breakOut
val map: Map[String, Set[String]] = json.asOpt[JsObject] match {
case Some(obj) =>
obj.fields.toMap.mapValues { v =>
v.asOpt[JsArray] match {
case Some(JsArray(cities)) => cities.flatMap(_.asOpt[String])(breakOut)
case _ => Set.empty[String]
}
}
case _ => Map.empty[String, Set[String]]
}
map must beEqualTo(Map("CountryA" -> Set("City1", "city2"), "CountryB" -> Set("City1")))
If you're confident about the structure of the JSON and don't mind using as (which could throw) instead of asOpt (which won't):
val map2: Map[String, Set[String]] = {
json.as[JsObject].fields.toMap.mapValues {
_.as[JsArray].value.map(_.as[String])(breakOut)
}
}
map2 must beEqualTo(Map("CountryA" -> Set("City1", "city2"), "CountryB" -> Set("City1")))

Accessing inner JSON field in Scala

I have JSON string as
{
"whatsNew" : {
"oldNotificationClass" : "WhatsNewNotification",
"notificationEvent" : { ..... },
"result" : {
"notificationCount" : 10
.....
}
},
......
"someEmpty": { },
......
}
I am trying to get notificationCount field using json4s in Scala as follow but notificationCount is coming as empty for all. Any help?
UPDATE
Also if some pair is empty how can I handle empty condition and continue to loop?
Function returning JSON string from file
def getData(): Map[String, AnyRef] = {
val jsonString = scala.io.Source.fromInputStream(this.getClass.getResourceAsStream("/sample.json")).getLines.mkString
val jsonObject = parse( s""" $jsonString """)
jsonObject.values.asInstanceOf[Map[String, AnyRef]]
}
Code to get fields
val myMap: Map[String, AnyRef] = MyDataLoader.getData
for((key, value) <- myMap) {
val id = key
val eventJsonStr: String = write(value.asInstanceOf[Map[String, String]] get "notificationEvent")
val resultJsonStr: String = write(value.asInstanceOf[Map[String, String]] get "result")
//val notificationCount: String = write(value.asInstanceOf[Map[String, Map[String, String]]] get "notificationCount")
}
You can use path and extract like so:
val count: Int = (parse(jsonString) \ "whatsNew" \ "result" \ "notificationCount").extract[Int]
you will need this import for the .extract[Int] to work:
implicit val formats = DefaultFormats
To do it in a loop:
parse(jsonString).children.map { child =>
val count: Int = (child \ "result" \ "notificationCount").extract[Int]
...
}