how to append a new item to an existing JSON variable? - json

Given the following json:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
}
}
I want to append new item to user JSON variable. and create a new json like:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
"point":10
}
}

You can achieve this by using Play Json API.
First you need to parse your json string to Play JsValue,
import play.api.libs.json._
val jsonString =
"""
|{
| "status": "ok",
| "user": {
| "id": 39216,
| "first_name": "naghmeh",
| "username": "test"
| }
|}
""".stripMargin
val jsValue = Json.parse(jsonString)
Now, you can add the values to your JsValue either by traversing the Json and then modifying it.
val result = jsValue match {
case jsObject: JsObject => (jsObject \ "user").as[JsObject] match {
case userJsObject: JsObject => jsObject ++ Json.obj(
"user" -> (userJsObject ++ Json.obj(
"point" -> 10
))
)
case _ => jsValue
}
case _ => jsValue
}
Or, by using Play Json's JsonTransformer API,
val jsonTransformer = (__ \ "user").json.update(
__.read[JsObject].map(jsObject => jsObject ++ Json.obj("point" -> 10))
)
val result2 = jsValue.transform(jsonTransformer) match {
case JsSuccess(jsObject, _) => jsObject
case _ => jsValue
}

Assuming from your question tags that you're using the PlayJSON library, you can simply use +:
val json: JsObject = ... // Your initial value here
val newField: JsValue = ... // Your new value here
val jsonWithAdditionalField = json + ("yourKey" -> newField)
For more information about how to create the JsValue, take a look a the official documentation

Append the list to a variable and then call put like so:
list.put("test");

Related

Select a particular value from a JSON array

I have a JSON array containing the following details, I would like to extract the text Alignment value of Right and assign it to a val.
"data":[
{
"formatType": "text",
"value": "bgyufcie huis huids hufhsduhfsl hd"
},
{
"formatType": "text size",
"value": 12
},
{
"formatType": "text alignment",
"value" : "right"
}
]
Any thoughts?
Using the Gson library, you can map the json in a Java object.
So, you have to create a class like this:
public class MyObject{
private String formatType;
private String value;
//Constuctors, Getter and Setter...
//.....
//.....
}
After, using the method fromJson you can create an array of MyObject.
Gson gson = new Gson();
MyObject[] array = gson.fromJson(new FileReader("file.json"), MyObject[].class);
You can also use json4s library as shown next:
import org.json4s._
import org.json4s.jackson.JsonMethods._
val json = """{
"data":[
{
"formatType": "text",
"value": "bgyufcie huis huids hufhsduhfsl hd"
},
{
"formatType": "text size",
"value": 12
},
{
"formatType": "text alignment",
"value" : "right"
}
]
}"""
val parsed = parse(json)
val value = (parsed \ "data" \\ classOf[JObject]).filter(m => m("formatType") == "text alignment")(0)("value")
// value: Any = right
The filter (parsed \ "data" \\ classOf[JObject]) extracts all the items into a List of Map i.e:
List(
Map(formatType -> text, value -> bgyufcie huis huids hufhsduhfsl hd),
Map(formatType -> text size, value -> 12), Map(formatType -> text alignment, value -> right)
).
From those we apply the filter filter(m => m("formatType") == "text alignment") to retrieve the record that we really need.
Use Dijon FTW!
Here is a test that demonstrates how easily the "right" value can be found in samples like yours:
import com.github.pathikrit.dijon._
val json = parse(
"""{
|"data":[
| {
| "formatType": "text",
| "value": "bgyufcie huis huids hufhsduhfsl hd"
| },
| {
| "formatType": "text size",
| "value": 12
| },
| {
| "formatType": "text alignment",
| "value" : "right"
| }
|]
|}""".stripMargin)
assert(json.data.toSeq.collect {
case obj if obj.formatType == "text alignment" => obj.value
}.head == "right")
I would use the Jackson library, it is very helpful for parsing JSON. You can read the JSON using an ObjectMapper.
Here is a full tutorial to get you started: https://www.mkyong.com/java/jackson-how-to-parse-json/
create a multiline JSON string, then parse that string directly into a Scala object, use the net.liftweb package to solve this.
import net.liftweb.json._
object SarahEmailPluginConfigTest {
implicit val formats = DefaultFormats
case class Mailserver(url: String, username: String, password: String)
val json = parse(
"""
{
"url": "imap.yahoo.com",
"username": "myusername",
"password": "mypassword"
}
"""
)
def main(args: Array[String]) {
val m = json.extract[Mailserver]
println(m.url)
println(m.username)
println(m.password)
}
}
https://alvinalexander.com/scala/simple-scala-lift-json-example-lift-framework
Reference link

How to read part of a nested json object in as a string in Scala

I want to read in a json object into a scala class that keeps part of the json object as a string without trying to parse it.
This is what the json looks like:
[
{
"contractType": "NullContract",
"contractDefinition": {
"column": "age",
"conditions": [
{
"conditionType": "equalsCondition",
"conditionDefinition": {
"column": "species",
"value": "person"
}
}
]
}
}
]
I am using the jackson library. This is my mapper:
val mapper = new ObjectMapper()
.registerModule(DefaultScalaModule)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.setSerializationInclusion(Include.NON_ABSENT)
This is my class:
case class ContractJson(contractType: String, contractDefinition: String)
, which is how I want my resulting object to look.
This is the code parsing it:
val contractJson: Array[ContractJson] = mapper.readValue(contractsJsonString, classOf[Array[ContractJson]])
Error message I'm getting: Can not deserialize instance of java.lang.String out of START_OBJECT token when it starts parsing the contractDefinition
If you are allowed to use other libraries and type of the contractDefinition field is not restricted to String then try jsoniter-scala's feature of extraction raw JSON values to byte arrays.
You will need to add dependencies:
libraryDependencies ++= Seq(
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core" % "1.1.0" % Compile,
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "1.1.0" % Provided // required only in compile-time
)
Then define types with codecs and parse the input:
import java.nio.charset.StandardCharsets.UTF_8
import com.github.plokhotnyuk.jsoniter_scala.macros._
import com.github.plokhotnyuk.jsoniter_scala.core._
import scala.util.hashing.MurmurHash3
object RawVal {
def apply(s: String) = new RawVal(s.getBytes)
implicit val codec: JsonValueCodec[RawVal] = new JsonValueCodec[RawVal] {
override def decodeValue(in: JsonReader, default: RawVal): RawVal = new RawVal(in.readRawValAsBytes())
override def encodeValue(x: RawVal, out: JsonWriter): Unit = out.writeRawVal(x.bs)
override val nullValue: RawVal = new RawVal(new Array[Byte](0))
}
}
case class RawVal private(bs: Array[Byte]) {
def this(s: String) = this(s.getBytes(UTF_8))
override lazy val hashCode: Int = MurmurHash3.arrayHash(bs)
override def equals(obj: Any): Boolean = obj match {
case that: RawVal => java.util.Arrays.equals(bs, that.bs)
case _ => false
}
override def toString: String = new String(bs, UTF_8)
}
case class ContractJson(contractType: String, contractDefinition: RawVal)
implicit val codec: JsonValueCodec[List[ContractJson]] = JsonCodecMaker.make(CodecMakerConfig)
val jsonBytes =
"""[
| {
| "contractType": "NullContract",
| "contractDefinition": {
| "column": "age",
| "conditions": [
| {
| "conditionType": "equalsCondition",
| "conditionDefinition": {
| "column": "species",
| "value": "person"
| }
| }
| ]
| }
| }
|]
|""".stripMargin.getBytes("UTF-8")
val contractJsons = readFromArray(jsonBytes)
println(contractJsons)
Printed result will be:
List(ContractJson(NullContract, {
"column": "age",
"conditions": [
{
"conditionType": "equalsCondition",
"conditionDefinition": {
"column": "species",
"value": "person"
}
}
]
}))

Parsing a file having content as Json format in Scala

I want to parse a file having content as json format.
From the file I want to extract few properties (name, DataType, Nullable) to create some column names dynamically.
I have gone through some examples but most of them are using case class but my problem is every time I will receive a file may have different content.
I tried to use the ujson library to parse the file but I am unable to understand how to use it properly.
object JsonTest {
def main(args: Array[String]): Unit = {
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
println(source)
val input = try source.mkString finally source.close()
println(input)
val data = ujson.read(input)
data("name") = data("name").str.reverse
val updated = data.render()
}
}
Content of the file example:
{
"Organization": {
"project": {
"name": "POC 4PL",
"description": "Implementation of orderbook"
},
"Entities": [
{
"name": "Shipments",
"Type": "Fact",
"Attributes": [
{
"name": "Shipment_Details",
"DataType": "StringType",
"Nullable": "true"
},
{
"name": "Shipment_ID",
"DataType": "StringType",
"Nullable": "true"
},
{
"name": "View_Cost",
"DataType": "StringType",
"Nullable": "true"
}
],
"ADLS_Location": "/mnt/mns/adls/raw/poc/orderbook/"
}
]
}
}
Expected output:
StructType(
Array(StructField("Shipment_Details",StringType,true),
StructField("Shipment_ID",DateType,true),
StructField("View_Cost",DateType,true)))
StructType needs to be added to the expected output programatically.
Try Using Playframework's Json utils - https://www.playframework.com/documentation/2.7.x/ScalaJson
Here's the solution to your issue-
\ Placed your json in text file
val fil_path = "C:\\TestData\\Config\\Conf.txt"
val conf_source = scala.io.Source.fromFile(fil_path)
lazy val json_str = try conf_source.mkString finally conf_source.close()
val conf_json: JsValue = Json.parse(json_str)
val all_entities: JsArray = (conf_json \ "Organization" \ "Entities").get.asInstanceOf[JsArray]
val shipments: JsValue = all_entities.value.filter(e => e.\("name").as[String] == "Shipments").head
val shipments_attributes: IndexedSeq[JsValue] = shipments.\("Attributes").get.asInstanceOf[JsArray].value
val shipments_schema: StructType = StructType(shipments_attributes.map(a => Tuple3(a.\("name").as[String], a.\("DataType").as[String], a.\("Nullable").as[String]))
.map(x => StructField(x._1, StrtoDatatype(x._2), x._3.toBoolean)))
shipments_schema.fields.foreach(println)
Output is -
StructField(Shipment_Details,StringType,true)
StructField(Shipment_ID,StringType,true)
StructField(View_Cost,StringType,true)
It depends if you want it to be completely dynamic or not, here are some options:
If you just want to read one field you can do:
import upickle.default._
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
val json = ujson.read(input)
println(json("Organization")("project")("name"))
the output will be: "POC 4PL"
If you just want just the Attributes to be with types, you can do:
import upickle.default.{macroRW, ReadWriter => RW}
import upickle.default._
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
val json = ujson.read(input)
val entitiesArray = json("Organization")("Entities")(0)("Attributes")
println(read[Seq[StructField]](entitiesArray))
case class StructField(name: String, DataType: String, Nullable: String)
object StructField{
implicit val rw: RW[StructField] = macroRW
}
the output will be: List(StructField(Shipment_Details,StringType,true), StructField(Shipment_ID,StringType,true), StructField(View_Cost,StringType,true))
another option, is to use a different library to do the class mapping. If you use Google Protobuf Struct and JsonFormat it can be 2-liner:
import com.google.protobuf.Struct
import com.google.protobuf.util.JsonFormat
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
JsonFormat.parser().merge(input, builder)
println(builder.build())
the output will be: fields { key: "Organization" value { struct_value { fields { key: "project" value { struct_value { fields { key: "name" value { string_value: "POC 4PL" } } fields { key: "description" value { string_value: "Implementation of orderbook" } } } } } fields { key: "Entities" value { list_value { values { struct_value { fields { key: "name" value { string_value: "Shipments" } }...

Json Writes for custom class in scala

I have a Seq of some Strings mostly like below:
val states = Seq(
"CA" -> Seq("Los Angeles" -> Seq("SunsetBlvd", "Hollywood" -> Seq("W 8th St", "W 9th St")), "Pasadena"),
"WA" -> Seq("Seattle", "Redmond")
)
The case class for this could be
case class State(name: String, sub: Option[Seq[State]])
And an implicit Writes
implicit val stateWrites = Json.Writes[State]
Hoping to convert it to a Json like
[
{
"name": "CA",
"sub": [
{
"name": "Los Angeles",
"sub": [
{
"name": "SunsetBlvd"
},
{
"name": "Hollywood",
"sub": [
{
"name": "W 8th St"
},
{
"name": "W 9th St"
}
]
}
]
}
]
},
{
"name": "WA",
"sub": [
{
"name": "Seattle"
},
{
"name": "Redmond"
}
]
}
]
How do I correctly model the data and be able to convert this Seq to a Json using Writes?
Or even change the states val to an appropriate format so I could easily convert it to Json?
In the case class, one of fields is of type itself which is wrong. How do I avoid that in modeling the data or even the Seq?
I came up with something like this:
case class State(name: String, sub: Option[Seq[State]])
import play.api.libs.json._
implicit val optWrites = new Writes[Option[Seq[State]]] {
override def writes(o: Option[Seq[State]]) = {
if (o.isDefined) {
Json.toJson(o.get)(stateSeqWrites)
} else {
JsNull
}
}
}
implicit val stateWrites = new Writes[State] {
def writes(state: State) = {
val l: Seq[(String, JsValueWrapper)] = Seq("name" -> JsString(state.name))
val ll: Seq[(String, JsValueWrapper)] = if (state.sub.isDefined) {
val subValue: JsValueWrapper = Json.toJson(state.sub)(optWrites)
l :+ ("sub" -> subValue)
} else {
l
}
Json.obj(ll : _*)
}
}
implicit val stateSeqWrites: Writes[Seq[State]] = new Writes[Seq[State]] {
override def writes(s: Seq[State]) = {
JsArray(s.map(Json.toJson(_)(stateWrites)))
}
}
val states = Seq(
State("CA", Some(Seq(State("Los Angeles", Some(Seq(State("SunsetBlvd", None), State("Hollywood", Some(Seq(State("W 8th St", None), State("W 9th St", None)))), State("Pasadena", None))))))),
State("WA", Some(Seq(State("Seattle", None), State("Redmond", None))))
)
val json = Json.toJson(states)
println(json.toString())
Probably might get simplified, but it's late night here ;) It does what you need :)
That information has conceptually a tree structure. My advice is to threat it with just a normal case class, to simplify the json formatter and to have a much more semantic structure:
case class Tree(name: String, sub: Option[List[Tree]])
And your formatter would be just like this:
implicit val repositoryFormat: Format[Tree] = (
(__ \ 'name).format[String] ~
(__ \ 'sub).lazyFormatNullable(implicitly[ Format[ List[Tree] ]])
)(Tree.apply, unlift(Tree.unapply))
Notice I used lazyFormatNullable to deal with the recursive reference to Tree in sub.
To simulate the Json you posted, I made the translation to the Tree case class structure.
// The tree leaves
val hollywoodLeaves = Some( Tree("W 8th St", None) :: Tree("W 9th St", None) :: Nil )
val losAngelesLeaves = Some( Tree("SunsetBlvd", None) :: Tree("Hollywood", hollywoodLeaves ) :: Nil )
// The two trees in your array
val firstTree = Tree( "CA", Some( Tree("Los Angeles", losAngelesLeaves) :: Nil ) )
val secondTree = Tree("WA", Some( Tree("Seattle", None) :: Tree("Redmond", None) :: Nil ))
// Your root array
val treeArray = firstTree :: secondTree :: Nil
// Conversion to json
val json = Json.toJson(treeArray)

Scala how to read json path by array

My main aim is to make a transformation i.e from "data/message" to "posts/content".
What's wrong with my code? I can't even access the value of "message" in an array.
<code>
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
def mapping(): JsObject = {
val sjson =
"""
{
"data": [
{"message": "A", "created_time": "a" },
{"message": "B", "created_time": "b" }
],
"page": "test"
}
""".stripMargin
val json = Json.parse(sjson)
val jsonReads =
(__ \ 'data \\ 'message ).read[String]
json.validate(jsonReads).map {
case( message ) =>
Json.obj("content" -> message)
}.get
}
</code>
In response to your comment on this post, here's my updated answer:
val sjson =
"""
{
"data": [
{"message": "A", "created_time": "a" },
{"message": "B", "created_time": "b" }
],
"page": "test"
}
""".stripMargin
// Read the whole object
val json: JsObject = Json.parse(sjson).as[JsObject]
// Read the "data array"
val data: Seq[JsObject] = (json \ "data").as[Seq[JsObject]]
// Get final result my changing "data" to "posts",
// and all instances of "message" to "content" and "created_time" to "created"
val result =
Json.obj(
"posts" -> (data map (d => JsObject(d.value map {
case ("message", m) =>
"content" -> m
case ("created_time", t) =>
"created" -> t
})))
)