How do I update a nested key of a JsObject in scala - json

In my code I make a get request to a server to get some json, and then I want to update one of the values before I send it back. I know that if the key was on the top level I could just update the key by writing
val newConfig = originalConfig ++ Json.obj("key" -> newValue)
however I cannot figure out a nice way to update it if the key I want to change is a couple of layers in.
ie. My json looks like this, and I want to just update key5
{
"key1": "value",
"key2": {
"key3": "value",
"key4": {
"key5": "value",
"key6": "value"
}
}
}
Is there a way to do this without updating it layer by layer?
ie.
val key4 = originalKey4 ++ Json.obj("key5" -> newValue)
val key2 = originalKey2 ++ Json.obj("key4" -> key4)
val newJson = originalJson ++ Json.obj("key2" -> key2)
The actual key that I want to update is 7 layers in, so this is rather tedious.

Take look at json transformers
import play.api.libs.json._
val str = """{
| "key1": "value",
| "key2": {
| "key3": "value",
| "key4": {
| "key5": "value",
| "key6": "value"
| }
| }
|}""".stripMargin
val json = Json.parse(str)
val transformer = (__ \ 'key2 \ 'key4 \ 'key5).json.update(
__.read[JsString].map(_ => Json.toJson("updated value"))
)
val result = json.transform(transformer).asOpt.get
Json.prettyPrint(result)
res0: String = {
"key1" : "value",
"key2" : {
"key3" : "value",
"key4" : {
"key5" : "updated value",
"key6" : "value"
}
}
}

Related

Scala, Circe, Json - how to remove parent node from json?

I have a json structure like this:
"data" : {
"fields": {
"field1": "value1",
"field2": "value2"
}
}
Now I would like to remove fields node and keep data in data:
"data" : {
"field1": "value1",
"field2": "value2"
}
I tried to do it like this:
val result = data.hcursor.downField("fields").as[JsonObject].toOption.head.toString
but I got a strange result, instead of just json in string format
I also tried:
val result = data.hcursor.downField("fields").top.head.toString
but it was the same as:
val result = data.toString
and it includes fields.
How I should change my code to remove fields root and keep data under data property?
Here is a full working solution that traverses the JSON, extracts the fields, removes them and then merges them under data:
import io.circe.Json
import io.circe.parser._
val s =
"""
|{
|"data": {
| "fields": {
| "field1": "value1",
| "field2": "value2"
| }
|}
|}
|""".stripMargin
val modifiedJson =
for {
json <- parse(s)
fields <- json.hcursor
.downField("data")
.downField("fields")
.as[Json]
modifiedRoot <- json.hcursor
.downField("data")
.downField("fields")
.delete
.root
.as[Json]
res <-
modifiedRoot.hcursor
.downField("data")
.withFocus(_.deepMerge(fields))
.root
.as[Json]
} yield res
Yields:
Right({
"data" : {
"field1" : "value1",
"field2" : "value2"
}
})

Select a particular value from a JSON array

I have a JSON array containing the following details, I would like to extract the text Alignment value of Right and assign it to a val.
"data":[
{
"formatType": "text",
"value": "bgyufcie huis huids hufhsduhfsl hd"
},
{
"formatType": "text size",
"value": 12
},
{
"formatType": "text alignment",
"value" : "right"
}
]
Any thoughts?
Using the Gson library, you can map the json in a Java object.
So, you have to create a class like this:
public class MyObject{
private String formatType;
private String value;
//Constuctors, Getter and Setter...
//.....
//.....
}
After, using the method fromJson you can create an array of MyObject.
Gson gson = new Gson();
MyObject[] array = gson.fromJson(new FileReader("file.json"), MyObject[].class);
You can also use json4s library as shown next:
import org.json4s._
import org.json4s.jackson.JsonMethods._
val json = """{
"data":[
{
"formatType": "text",
"value": "bgyufcie huis huids hufhsduhfsl hd"
},
{
"formatType": "text size",
"value": 12
},
{
"formatType": "text alignment",
"value" : "right"
}
]
}"""
val parsed = parse(json)
val value = (parsed \ "data" \\ classOf[JObject]).filter(m => m("formatType") == "text alignment")(0)("value")
// value: Any = right
The filter (parsed \ "data" \\ classOf[JObject]) extracts all the items into a List of Map i.e:
List(
Map(formatType -> text, value -> bgyufcie huis huids hufhsduhfsl hd),
Map(formatType -> text size, value -> 12), Map(formatType -> text alignment, value -> right)
).
From those we apply the filter filter(m => m("formatType") == "text alignment") to retrieve the record that we really need.
Use Dijon FTW!
Here is a test that demonstrates how easily the "right" value can be found in samples like yours:
import com.github.pathikrit.dijon._
val json = parse(
"""{
|"data":[
| {
| "formatType": "text",
| "value": "bgyufcie huis huids hufhsduhfsl hd"
| },
| {
| "formatType": "text size",
| "value": 12
| },
| {
| "formatType": "text alignment",
| "value" : "right"
| }
|]
|}""".stripMargin)
assert(json.data.toSeq.collect {
case obj if obj.formatType == "text alignment" => obj.value
}.head == "right")
I would use the Jackson library, it is very helpful for parsing JSON. You can read the JSON using an ObjectMapper.
Here is a full tutorial to get you started: https://www.mkyong.com/java/jackson-how-to-parse-json/
create a multiline JSON string, then parse that string directly into a Scala object, use the net.liftweb package to solve this.
import net.liftweb.json._
object SarahEmailPluginConfigTest {
implicit val formats = DefaultFormats
case class Mailserver(url: String, username: String, password: String)
val json = parse(
"""
{
"url": "imap.yahoo.com",
"username": "myusername",
"password": "mypassword"
}
"""
)
def main(args: Array[String]) {
val m = json.extract[Mailserver]
println(m.url)
println(m.username)
println(m.password)
}
}
https://alvinalexander.com/scala/simple-scala-lift-json-example-lift-framework
Reference link

How to remove non-ascii characters from logs?

I want to remove non ascii characters from Logs(json Strings) and parse them. but i see text like this before my json string starts, how can remove these kinds of string and parse my JSON String
SEQ^F!org.apache.hadoop.io.LongWritable^Yorg.apache.hadoop.io.Text^#^#^#^#^#^#ìþNmbÃ<92>w^G6ùó¯Ãl^#^#^X^E^#^#^#^H^#^#^^¯/âë<8e>^Wú{
You can drop everything till { if that's safe so that you get the json string. I assume your log format is "garbage{json}"
example,
scala> val log = """SEQ^F!org.apache.hadoop.io.LongWritable^Yorg.apache.hadoop.io.Text^#^#^#^#^#^#ìþNmbÃ<92>w^G6ùó¯Ãl^#^#^X^E^#^#^#^H^#^#^^¯/âë<8e>^Wú{"key1": "value1", "key2": ["1", "2"]}"""
log: String = SEQ^F!org.apache.hadoop.io.LongWritable^Yorg.apache.hadoop.io.Text^#^#^#^#^#^#ìþNmbÃ<92>w^G6ùó¯Ãl^#^#^X^E^#^#^#^H^#^#^^¯/âë<8e>^Wú{"key1": "value1", "key2": ["1", "2"]}
scala> val extractJson = log.dropWhile(char => char != '{')
extractJson: String = {"key1": "value1", "key2": ["1", "2"]}
Then use any json api, Im using circe in following example,
scala> import io.circe.parser._
import io.circe.parser._
scala> parse(extractJson)
res3: Either[io.circe.ParsingFailure,io.circe.Json] =
Right({
"key1" : "value1",
"key2" : [
"1",
"2"
]
})
If you want to extract any particular element json,
scala> res3.map(j => (j \\ "key1").headOption)
res4: scala.util.Either[io.circe.ParsingFailure,Option[io.circe.Json]] = Right(Some("value1"))

how to append a new item to an existing JSON variable?

Given the following json:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
}
}
I want to append new item to user JSON variable. and create a new json like:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
"point":10
}
}
You can achieve this by using Play Json API.
First you need to parse your json string to Play JsValue,
import play.api.libs.json._
val jsonString =
"""
|{
| "status": "ok",
| "user": {
| "id": 39216,
| "first_name": "naghmeh",
| "username": "test"
| }
|}
""".stripMargin
val jsValue = Json.parse(jsonString)
Now, you can add the values to your JsValue either by traversing the Json and then modifying it.
val result = jsValue match {
case jsObject: JsObject => (jsObject \ "user").as[JsObject] match {
case userJsObject: JsObject => jsObject ++ Json.obj(
"user" -> (userJsObject ++ Json.obj(
"point" -> 10
))
)
case _ => jsValue
}
case _ => jsValue
}
Or, by using Play Json's JsonTransformer API,
val jsonTransformer = (__ \ "user").json.update(
__.read[JsObject].map(jsObject => jsObject ++ Json.obj("point" -> 10))
)
val result2 = jsValue.transform(jsonTransformer) match {
case JsSuccess(jsObject, _) => jsObject
case _ => jsValue
}
Assuming from your question tags that you're using the PlayJSON library, you can simply use +:
val json: JsObject = ... // Your initial value here
val newField: JsValue = ... // Your new value here
val jsonWithAdditionalField = json + ("yourKey" -> newField)
For more information about how to create the JsValue, take a look a the official documentation
Append the list to a variable and then call put like so:
list.put("test");

Play-Json Coast-to-Coast: Best Pattern for "Nesting" Transformations

I'm becoming a fan of the play-json coast-to-coast pattern, especially its use of combinators. I have some complex cases I'm munging together. I know there's a better way, but I'm new to the combinator approach to build up functionality.
I'd like to turn this:
{
"somearray": [
{
"field1": "value1",
"field2": "value1",
"key": "key1"
},
{
"field1": "second1",
"field2": "second1",
"key": "key2"
}
]
}
into this:
{
"someObj": {
"key1":
{
"field1": "value1",
"field2": "value1"
},
"key2":
{
"field1": "second1",
"field2": "second1"
}
]
}
I can get this to work, but I exit out of the transformation:
(__ \ "someObj").json.copyFrom(__ \ "someArray".json.pick.map {
case JsArray(arr) => {
JsObject(arr.map(a =>
a.transform(<A transform function to prune key>).map(pruned => {
((a \ "key").as[String], pruned)
}).flatMap({
case JsSuccess(result, _) => Seq(result)
case other => Nil
})
}
case other => JsNull
})
THere are some issues with this code: I know it's verbose, I know I'm assume "key" is present with a string type, and I need that flatMap to get me out of a JsResult and into some JsValue I can use to build the JsObject.
It seems like I should be able to
1) Create a sub transformation, i.e. a.transform() can be nestled in the parent transform without unpacking and repacking the json object.
Thank you.
How about this (ignoring the fact that this code still has the same issues you already mentioned)?
def elementTransform = for {
keyValue <- (__ \ 'key).json.pick // picks value for key field
obj <- (__ \ 'key).json.prune // returns object without the key field
res <- (__ \ keyValue.as[String]).json.put(obj) // uses the keyValue as a field name
} yield res
val transform = (__ \ "someObj").json.copyFrom((__ \ "someArray").json.pick[JsArray].map(
_.value.foldLeft[JsResult[JsObject]](JsSuccess(Json.obj()))(
(obj, a) =>
for {
o <- obj
field <- a.transform(elementTransform)
} yield o ++ field
).fold(_ => JsNull, identity)
))