Scala play JSON, lookup and match defined field holding null value - json

I have the following Json block that I have returned as a JsObject
{
"first_block": [
{
"name": "demo",
"description": "first demo description"
}
],
"second_block": [
{
"name": "second_demo",
"description": "second demo description",
"nested_second": [
{
"name": "bob",
"value": null
},
{
"name": "john",
"value": null
}
]
}
]
}
From this, I want to return a list of all the possible values I could have in the second block, nested array for name and value. so with the example above
List([bob,null],[john,null]) or something along those lines.
The issue I am having is with the value section understanding null values. I've tried to match against it and return a string "null" but I can't get it to match on Null values.
What would be the best way for me to return back the name and values in the nested_second array.
I've tried using case classes and readAsNullable with no luck, and my latest attempt has gone along these lines:
val secondBlock = (jsObj \ "second_block").as[List[JsValue]]
secondBlock.foreach(nested_block => {
val nestedBlock = (nested_block \ "nested_second").as[List[JsValue]]
nestedBlock.foreach(value => {
val name = (value \ "name").as[String] //always a string
var convertedValue = ""
val replacement_value = value \ "value"
replacement_value match {
case JsDefined(null) => convertedValue = "null"
case _ => convertedValue = replacement_value.as[String]
}
println(name)
println(convertedValue)
})
}
)
It seems convertedValue returns as 'JsDefined(null)' regardless and I'm sure the way I'm doing it is horrifically bad.

Replace JsDefined(null) with JsDefined(JsNull).
You probably got confused, because println(JsDefined(JsNull)) prints as JsDefined(null). But that is not, how null value of a JSON field is represented. null is represented as case object JsNull. This is just a good API design, where possible cases are represented with a hierarchy of classes:

With play-json I use always case-classes!
I simplified your problem to the essence:
import play.api.libs.json._
val jsonStr = """[
{
"name": "bob",
"value": null
},
{
"name": "john",
"value": "aValue"
},
{
"name": "john",
"value": null
}
]"""
Define a case class
case class Element(name: String, value: Option[String])
Add a formatter in the companion object:
object Element {
implicit val jsonFormat: Format[Element] = Json.format[Element]
}
An use validate:
Json.parse(jsonStr).validate[Seq[Element]] match {
case JsSuccess(elems, _) => println(elems)
case other => println(s"Handle exception $other")
}
This returns: List(Element(bob,None), Element(john,Some(aValue)), Element(john,None))
Now you can do whatever you want with the values.

Related

JSON Objects null filtering in scala

I am using play.api.libs.json in Scala (2.12.8) to process some json objects. I have for example a JSON string that looks like:
{
"field1": null,
"field2": 23,
"field3": {
"subfield1": "a",
"subfield2": null
},
"field4": {
"subfield1": true,
"subfield2": {
"subsubfield1": null,
"subsubfield2": "45"
},
"field5": 3
}
}
And I want to filter out every null fields or subfields.
As explained here: Play: How to remove the fields without value from JSON and create a new JSON with them
Doing:
import play.api.libs.json.{ JsNull, JsObject, JsValue, Json }
val j = Json.parse(myJsonString).as[JsObject]
JsObject(j.fields.filterNot(t => withoutValue(t._2)))
def withoutValue(v: JsValue) = v match {
case JsNull => true
case _ => false
}
helps me remove the upper level fields: in my case, field1
But field3.subfield2 and field4.subfield2.subsubfield1 are still present. I want to remove them. Also I should mention that not every subfields can be null at once. Should this happen, I think we could just remove the upper level field. If field3.subfield1and field3.subfield2 are null, we can remove field3.
Any idea on how to do this neatly in Scala?
PS: the desired output is:
{
"field2": 23,
"field3": {
"subfield1": "a"
},
"field4": {
"subfield1": true,
"subfield2": {
"subsubfield2": "45"
},
"field5": 3
}
}
You need to do a recursive solution. For example:
def removeNulls(jsObject: JsObject): JsValue = {
JsObject(jsObject.fields.collect {
case (s, j: JsObject) =>
(s, removeNulls(j))
case other if (other._2 != JsNull) =>
other
})
}
Code run at Scastie. Output is as expected.

Scala: Edit/Modify json string based on internal value

I have a json string structured similarly to the following:
val json : String =
{
"identifier":{
"id":"1234_567_910",
"timestamp":"12:34:56",
},
"information":[
{
"fieldName":"test_name",
"fieldId":"test_fieldId",
}
]
}
What I want to do is create a check that verifies the 'id' field matches the structure "Int_Int_Int" and if it doesn't I want to change the value to match this intended structure but I want to keep the rest of the information in the json string as is.
So if I received the following 'id' fields within a json string I would want to change them like so:
"id":"1234_567_910" -> do nothing
"id":"1234" -> "id":"1234_0_0"
"id":"1234_567" -> "id":"1234_567_0"
"id":"1234_???" -> "id":"1234_0_0"
"id":"1234_??_???" -> "id":"1234_0_0"
"id":"1234_foo" -> "id":"1234_0_0"
"id":"1234_567_foo" -> "id":"1234_567_0"
For Example:
If I receive json like this:
{
"identifier":{
"id":"1234",
"timestamp":"12:34:56",
},
"information":[
{
"fieldName":"test_name",
"fieldId":"test_fieldId",
}
]
}
I would want to modify it so I end up with a json like this:
{
"identifier":{
"id":"1234_0_0",
"timestamp":"12:34:56",
},
"information":[
{
"fieldName":"test_name",
"fieldId":"test_fieldId",
}
]
}
What would be the most effective/cleanest way to achieve this type of json modification in Scala?
Below is how it can be done with the Dijon library.
import com.github.pathikrit.dijon._
def normalize(id: String): String =
id.count(_ == '_') match {
case 0 => id + "_0_0"
case 1 => id + "_0"
case _ => id
}
val json =
json"""
{
"identifier":{
"id":"1234",
"timestamp":"12:34:56"
},
"information":[
{
"fieldName":"test_name",
"fieldId":"test_fieldId"
}
]
}"""
json.identifier.id = json.identifier.id.asString.fold("0_0_0")(normalize)
println(pretty(json))
It should print:
{
"identifier": {
"id": "1234_0_0",
"timestamp": "12:34:56"
},
"information": [
{
"fieldName": "test_name",
"fieldId": "test_fieldId"
}
]
}

How to get Keys and values, while parsing Json using Scala-Play Json Framework?

I have a json file , which has some keys and values. I need to parse the Json and print the keys and their values. For example, the json file is like below. I want to print this as Keys and values
{
"Parcer":[
{
"key":"0203",
"value":{
"Encryption":
{
"enabled":"yes",
"encryption_type":"base64",
"key":"334848484",
"return":"name"
}
}
},
{
"key":"0405",
"value":{
"Encryption":
{
"enabled":"yes",
"encryption_type":"base64",
"key":"334848484",
"return":"none"
},
"Parcer":[
{
"key":"0102",
"value":"humidity"
},
{
"key":"0304",
"value":{
"Encryption":{
"enabled":"yes",
"encryption_type":"SHA1",
"key":"1211212",
"return":"none"
}
}
}
]
}
}],
}```
The easiest way is to create a case class, like:
case class MyObj(header:String, value: Seq[Map[String, String]])
Then you just need to add one line for marshalling, like:
import play.api.libs.json._
object MyObj {
implicit val jsonFormat: OFormat[MyObj] = Json.format[MyObj]
}
Now you get a nice case class that you can work with:
val json =
Json.parse(
"""{
"header" : "header value",
"value" : [
{
"a" : "a_val",
"b" : "b_val",
"c" : "c_val"
},
{
"a" : "a_val",
"b" : "b_val",
"c" : "c_val"
}
]
}""")
Here an example how to retrieve all "a".
json.validate[MyObj] match {
case JsSuccess(myObj, _) =>
val allAs =myObj.value.flatMap(m => m.get("a").toSeq)
println(allAs) // >> Vector(a_val, a_val)
case e:JsError => // handle error
}
This gives you:
json.validate[MyObj] returns JsSuccess(MyObj(header value,Vector(Map(a -> a_val, b -> b_val, c -> c_val), Map(a -> a_val, b -> b_val, c -> c_val))),)
The println returns: Vector(a_val, a_val)
This is described here in the Documentation: JSON automated mapping

How to insert an empty object into JSON using Circe?

I'm getting a JSON object over the network, as a String. I'm then using Circe to parse it. I want to add a handful of fields to it, and then pass it on downstream.
Almost all of that works.
The problem is that my "adding" is really "overwriting". That's actually ok, as long as I add an empty object first. How can I add such an empty object?
So looking at the code below, I am overwriting "sometimes_empty:{}" and it works. But because sometimes_empty is not always empty, it results in some data loss. I'd like to add a field like: "custom:{}" and then ovewrite the value of custom with my existing code.
Two StackOverflow posts were helpful. One worked, but wasn't quite what I was looking for. The other I couldn't get to work.
1: Modifying a JSON array in Scala with circe
2: Adding field to a json using Circe
val js: String = """
{
"id": "19",
"type": "Party",
"field": {
"id": 1482,
"name": "Anne Party",
"url": "https"
},
"sometimes_empty": {
},
"bool": true,
"timestamp": "2018-12-18T11:39:18Z"
}
"""
val newJson = parse(js).toOption
.flatMap { doc =>
doc.hcursor
.downField("sometimes_empty")
.withFocus(_ =>
Json.fromFields(
Seq(
("myUrl", Json.fromString(myUrl)),
("valueZ", Json.fromString(valueZ)),
("valueQ", Json.fromString(valueQ)),
("balloons", Json.fromString(balloons))
)
)
)
.top
}
newJson match {
case Some(v) => return v.toString
case None => println("Failure!")
}
We need to do a couple of things. First, we need to zoom in on the specific property we want to update, if it doesn't exist, we'll create a new empty one. Then, we turn the zoomed in property in the form of a Json into JsonObject in order to be able to modify it using the +: method. Once we've done that, we need to take the updated property and re-introduce it in the original parsed JSON to get the complete result:
import io.circe.{Json, JsonObject, parser}
import io.circe.syntax._
object JsonTest {
def main(args: Array[String]): Unit = {
val js: String =
"""
|{
| "id": "19",
| "type": "Party",
| "field": {
| "id": 1482,
| "name": "Anne Party",
| "url": "https"
| },
| "bool": true,
| "timestamp": "2018-12-18T11:39:18Z"
|}
""".stripMargin
val maybeAppendedJson =
for {
json <- parser.parse(js).toOption
sometimesEmpty <- json.hcursor
.downField("sometimes_empty")
.focus
.orElse(Option(Json.fromJsonObject(JsonObject.empty)))
jsonObject <- json.asObject
emptyFieldJson <- sometimesEmpty.asObject
appendedField = emptyFieldJson.+:("added", Json.fromBoolean(true))
res = jsonObject.+:("sometimes_empty", appendedField.asJson)
} yield res
maybeAppendedJson.foreach(obj => println(obj.asJson.spaces2))
}
}
Yields:
{
"id" : "19",
"type" : "Party",
"field" : {
"id" : 1482,
"name" : "Anne Party",
"url" : "https"
},
"sometimes_empty" : {
"added" : true,
"someProperty" : true
},
"bool" : true,
"timestamp" : "2018-12-18T11:39:18Z"
}

Parse JSON with unknown attributes names into a Case Class

I have the following JSON file to be parsed into a case class:
{
"root": {
"nodes": [{
"id": "1",
"attributes": {
"name": "Node 1",
"size": "3"
}
},
{
"id": "2",
"attributes": {
"value": "4",
"name": "Node 2"
}
}
]
}
}
The problem is that the attributes could have any value inside it: name, size, value, anything ...
At this moment I have defined my case classes:
case class Attributes(
name: String,
size: String,
value: Sting
)
case class Nodes(
id: String,
attributes: Attributes
)
case class Root(
nodes: List[Nodes]
)
case class R00tJsonObject(
root: Root
)
Whats is the best way to deal with this scenario when I can receive any attribute ?
Currently I am using Json4s to handle son files.
Thanks!
Your attributes are arbitrarily many and differently named, but it seems you can store them in a Map[String, String] (at least, if those examples are anything to go by). In this case, using circe-parser (https://circe.github.io/circe/parsing.html), you could simply use code along these lines in order to convert your JSON directly into a simple case-class:
import io.circe._, io.circe.parser._
import io.circe.generic.semiauto._
case class Node(id: String, attributes: Map[String,String])
case class Root(nodes: List[Node])
implicit val nodeDecoder: Decoder[Node] = deriveDecoder[Node]
implicit val nodeEncoder: Encoder[Node] = deriveEncoder[Node]
implicit val rootDecoder: Decoder[Root] = deriveDecoder[Root]
implicit val rootEncoder: Encoder[Root] = deriveEncoder[Root]
def myParse(jsonString: String) = {
val res = parse(jsonString) match {
case Right(json) => {
val cursor = json.hcursor
cursor.get[Root]("root")
}
case _ => Left("Wrong JSON!")
}
println(res)
}
This snippet will print
Right(Root(List(Node(1,Map(name -> Node 1, size -> 3)), Node(2,Map(value -> 4, name -> Node 2)))))
on the console, for the JSON, you've given. (Assuming, the solution doesn't have to be in Json4s.)