Json Writes for custom class in scala - json

I have a Seq of some Strings mostly like below:
val states = Seq(
"CA" -> Seq("Los Angeles" -> Seq("SunsetBlvd", "Hollywood" -> Seq("W 8th St", "W 9th St")), "Pasadena"),
"WA" -> Seq("Seattle", "Redmond")
)
The case class for this could be
case class State(name: String, sub: Option[Seq[State]])
And an implicit Writes
implicit val stateWrites = Json.Writes[State]
Hoping to convert it to a Json like
[
{
"name": "CA",
"sub": [
{
"name": "Los Angeles",
"sub": [
{
"name": "SunsetBlvd"
},
{
"name": "Hollywood",
"sub": [
{
"name": "W 8th St"
},
{
"name": "W 9th St"
}
]
}
]
}
]
},
{
"name": "WA",
"sub": [
{
"name": "Seattle"
},
{
"name": "Redmond"
}
]
}
]
How do I correctly model the data and be able to convert this Seq to a Json using Writes?
Or even change the states val to an appropriate format so I could easily convert it to Json?
In the case class, one of fields is of type itself which is wrong. How do I avoid that in modeling the data or even the Seq?

I came up with something like this:
case class State(name: String, sub: Option[Seq[State]])
import play.api.libs.json._
implicit val optWrites = new Writes[Option[Seq[State]]] {
override def writes(o: Option[Seq[State]]) = {
if (o.isDefined) {
Json.toJson(o.get)(stateSeqWrites)
} else {
JsNull
}
}
}
implicit val stateWrites = new Writes[State] {
def writes(state: State) = {
val l: Seq[(String, JsValueWrapper)] = Seq("name" -> JsString(state.name))
val ll: Seq[(String, JsValueWrapper)] = if (state.sub.isDefined) {
val subValue: JsValueWrapper = Json.toJson(state.sub)(optWrites)
l :+ ("sub" -> subValue)
} else {
l
}
Json.obj(ll : _*)
}
}
implicit val stateSeqWrites: Writes[Seq[State]] = new Writes[Seq[State]] {
override def writes(s: Seq[State]) = {
JsArray(s.map(Json.toJson(_)(stateWrites)))
}
}
val states = Seq(
State("CA", Some(Seq(State("Los Angeles", Some(Seq(State("SunsetBlvd", None), State("Hollywood", Some(Seq(State("W 8th St", None), State("W 9th St", None)))), State("Pasadena", None))))))),
State("WA", Some(Seq(State("Seattle", None), State("Redmond", None))))
)
val json = Json.toJson(states)
println(json.toString())
Probably might get simplified, but it's late night here ;) It does what you need :)

That information has conceptually a tree structure. My advice is to threat it with just a normal case class, to simplify the json formatter and to have a much more semantic structure:
case class Tree(name: String, sub: Option[List[Tree]])
And your formatter would be just like this:
implicit val repositoryFormat: Format[Tree] = (
(__ \ 'name).format[String] ~
(__ \ 'sub).lazyFormatNullable(implicitly[ Format[ List[Tree] ]])
)(Tree.apply, unlift(Tree.unapply))
Notice I used lazyFormatNullable to deal with the recursive reference to Tree in sub.
To simulate the Json you posted, I made the translation to the Tree case class structure.
// The tree leaves
val hollywoodLeaves = Some( Tree("W 8th St", None) :: Tree("W 9th St", None) :: Nil )
val losAngelesLeaves = Some( Tree("SunsetBlvd", None) :: Tree("Hollywood", hollywoodLeaves ) :: Nil )
// The two trees in your array
val firstTree = Tree( "CA", Some( Tree("Los Angeles", losAngelesLeaves) :: Nil ) )
val secondTree = Tree("WA", Some( Tree("Seattle", None) :: Tree("Redmond", None) :: Nil ))
// Your root array
val treeArray = firstTree :: secondTree :: Nil
// Conversion to json
val json = Json.toJson(treeArray)

Related

Convert List to Json

How do I create a Json (Circe) looking like this:
{
"items": [{
"field1": "somevalue",
"field2": "somevalue2"
},
{
"field1": "abc",
"field2": "123abc"
}]
}
val result = Json.fromFields(List("items" -> ???))
You can do so using Circe's built in list typeclasses for encoding JSON. This code will work:
import io.circe.{Encoder, Json}
import io.circe.syntax._
case class Field(field1: String, field2: String)
object Field {
implicit val encodeFoo: Encoder[Field] = new Encoder[Field] {
final def apply(a: Field): Json = Json.obj(
("field1", Json.fromString(a.field1)),
("field2", Json.fromString(a.field2))
)
}
}
class Encoding(items: List[Field]) {
def getJson: Json = {
Json.obj(
(
"items",
items.asJson
)
)
}
}
If we instantiate an "Encoding" class and call getJson it will give you back the desired JSON. It works because with circe all you need to do to encode a list is provide an encoder for whatever is inside the list. Thus, if we provide an encoder for Field it will encode it inside a list when we call asJson on it.
If we run this:
val items = new Encoding(List(Field("jf", "fj"), Field("jfl", "fjl")))
println(items.getJson)
we get:
{
"items" : [
{
"field1" : "jf",
"field2" : "fj"
},
{
"field1" : "jfl",
"field2" : "fjl"
}
]
}

How to read part of a nested json object in as a string in Scala

I want to read in a json object into a scala class that keeps part of the json object as a string without trying to parse it.
This is what the json looks like:
[
{
"contractType": "NullContract",
"contractDefinition": {
"column": "age",
"conditions": [
{
"conditionType": "equalsCondition",
"conditionDefinition": {
"column": "species",
"value": "person"
}
}
]
}
}
]
I am using the jackson library. This is my mapper:
val mapper = new ObjectMapper()
.registerModule(DefaultScalaModule)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.setSerializationInclusion(Include.NON_ABSENT)
This is my class:
case class ContractJson(contractType: String, contractDefinition: String)
, which is how I want my resulting object to look.
This is the code parsing it:
val contractJson: Array[ContractJson] = mapper.readValue(contractsJsonString, classOf[Array[ContractJson]])
Error message I'm getting: Can not deserialize instance of java.lang.String out of START_OBJECT token when it starts parsing the contractDefinition
If you are allowed to use other libraries and type of the contractDefinition field is not restricted to String then try jsoniter-scala's feature of extraction raw JSON values to byte arrays.
You will need to add dependencies:
libraryDependencies ++= Seq(
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core" % "1.1.0" % Compile,
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "1.1.0" % Provided // required only in compile-time
)
Then define types with codecs and parse the input:
import java.nio.charset.StandardCharsets.UTF_8
import com.github.plokhotnyuk.jsoniter_scala.macros._
import com.github.plokhotnyuk.jsoniter_scala.core._
import scala.util.hashing.MurmurHash3
object RawVal {
def apply(s: String) = new RawVal(s.getBytes)
implicit val codec: JsonValueCodec[RawVal] = new JsonValueCodec[RawVal] {
override def decodeValue(in: JsonReader, default: RawVal): RawVal = new RawVal(in.readRawValAsBytes())
override def encodeValue(x: RawVal, out: JsonWriter): Unit = out.writeRawVal(x.bs)
override val nullValue: RawVal = new RawVal(new Array[Byte](0))
}
}
case class RawVal private(bs: Array[Byte]) {
def this(s: String) = this(s.getBytes(UTF_8))
override lazy val hashCode: Int = MurmurHash3.arrayHash(bs)
override def equals(obj: Any): Boolean = obj match {
case that: RawVal => java.util.Arrays.equals(bs, that.bs)
case _ => false
}
override def toString: String = new String(bs, UTF_8)
}
case class ContractJson(contractType: String, contractDefinition: RawVal)
implicit val codec: JsonValueCodec[List[ContractJson]] = JsonCodecMaker.make(CodecMakerConfig)
val jsonBytes =
"""[
| {
| "contractType": "NullContract",
| "contractDefinition": {
| "column": "age",
| "conditions": [
| {
| "conditionType": "equalsCondition",
| "conditionDefinition": {
| "column": "species",
| "value": "person"
| }
| }
| ]
| }
| }
|]
|""".stripMargin.getBytes("UTF-8")
val contractJsons = readFromArray(jsonBytes)
println(contractJsons)
Printed result will be:
List(ContractJson(NullContract, {
"column": "age",
"conditions": [
{
"conditionType": "equalsCondition",
"conditionDefinition": {
"column": "species",
"value": "person"
}
}
]
}))

Parsing a file having content as Json format in Scala

I want to parse a file having content as json format.
From the file I want to extract few properties (name, DataType, Nullable) to create some column names dynamically.
I have gone through some examples but most of them are using case class but my problem is every time I will receive a file may have different content.
I tried to use the ujson library to parse the file but I am unable to understand how to use it properly.
object JsonTest {
def main(args: Array[String]): Unit = {
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
println(source)
val input = try source.mkString finally source.close()
println(input)
val data = ujson.read(input)
data("name") = data("name").str.reverse
val updated = data.render()
}
}
Content of the file example:
{
"Organization": {
"project": {
"name": "POC 4PL",
"description": "Implementation of orderbook"
},
"Entities": [
{
"name": "Shipments",
"Type": "Fact",
"Attributes": [
{
"name": "Shipment_Details",
"DataType": "StringType",
"Nullable": "true"
},
{
"name": "Shipment_ID",
"DataType": "StringType",
"Nullable": "true"
},
{
"name": "View_Cost",
"DataType": "StringType",
"Nullable": "true"
}
],
"ADLS_Location": "/mnt/mns/adls/raw/poc/orderbook/"
}
]
}
}
Expected output:
StructType(
Array(StructField("Shipment_Details",StringType,true),
StructField("Shipment_ID",DateType,true),
StructField("View_Cost",DateType,true)))
StructType needs to be added to the expected output programatically.
Try Using Playframework's Json utils - https://www.playframework.com/documentation/2.7.x/ScalaJson
Here's the solution to your issue-
\ Placed your json in text file
val fil_path = "C:\\TestData\\Config\\Conf.txt"
val conf_source = scala.io.Source.fromFile(fil_path)
lazy val json_str = try conf_source.mkString finally conf_source.close()
val conf_json: JsValue = Json.parse(json_str)
val all_entities: JsArray = (conf_json \ "Organization" \ "Entities").get.asInstanceOf[JsArray]
val shipments: JsValue = all_entities.value.filter(e => e.\("name").as[String] == "Shipments").head
val shipments_attributes: IndexedSeq[JsValue] = shipments.\("Attributes").get.asInstanceOf[JsArray].value
val shipments_schema: StructType = StructType(shipments_attributes.map(a => Tuple3(a.\("name").as[String], a.\("DataType").as[String], a.\("Nullable").as[String]))
.map(x => StructField(x._1, StrtoDatatype(x._2), x._3.toBoolean)))
shipments_schema.fields.foreach(println)
Output is -
StructField(Shipment_Details,StringType,true)
StructField(Shipment_ID,StringType,true)
StructField(View_Cost,StringType,true)
It depends if you want it to be completely dynamic or not, here are some options:
If you just want to read one field you can do:
import upickle.default._
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
val json = ujson.read(input)
println(json("Organization")("project")("name"))
the output will be: "POC 4PL"
If you just want just the Attributes to be with types, you can do:
import upickle.default.{macroRW, ReadWriter => RW}
import upickle.default._
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
val json = ujson.read(input)
val entitiesArray = json("Organization")("Entities")(0)("Attributes")
println(read[Seq[StructField]](entitiesArray))
case class StructField(name: String, DataType: String, Nullable: String)
object StructField{
implicit val rw: RW[StructField] = macroRW
}
the output will be: List(StructField(Shipment_Details,StringType,true), StructField(Shipment_ID,StringType,true), StructField(View_Cost,StringType,true))
another option, is to use a different library to do the class mapping. If you use Google Protobuf Struct and JsonFormat it can be 2-liner:
import com.google.protobuf.Struct
import com.google.protobuf.util.JsonFormat
val source = scala.io.Source.fromFile("C:\\Users\\ktngme\\Desktop\\ass\\file.txt")
val input = try source.mkString finally source.close()
JsonFormat.parser().merge(input, builder)
println(builder.build())
the output will be: fields { key: "Organization" value { struct_value { fields { key: "project" value { struct_value { fields { key: "name" value { string_value: "POC 4PL" } } fields { key: "description" value { string_value: "Implementation of orderbook" } } } } } fields { key: "Entities" value { list_value { values { struct_value { fields { key: "name" value { string_value: "Shipments" } }...

how to append a new item to an existing JSON variable?

Given the following json:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
}
}
I want to append new item to user JSON variable. and create a new json like:
{
"status": "ok",
"user": {
"id": 39216,
"first_name": "naghmeh",
"username": "test",
"point":10
}
}
You can achieve this by using Play Json API.
First you need to parse your json string to Play JsValue,
import play.api.libs.json._
val jsonString =
"""
|{
| "status": "ok",
| "user": {
| "id": 39216,
| "first_name": "naghmeh",
| "username": "test"
| }
|}
""".stripMargin
val jsValue = Json.parse(jsonString)
Now, you can add the values to your JsValue either by traversing the Json and then modifying it.
val result = jsValue match {
case jsObject: JsObject => (jsObject \ "user").as[JsObject] match {
case userJsObject: JsObject => jsObject ++ Json.obj(
"user" -> (userJsObject ++ Json.obj(
"point" -> 10
))
)
case _ => jsValue
}
case _ => jsValue
}
Or, by using Play Json's JsonTransformer API,
val jsonTransformer = (__ \ "user").json.update(
__.read[JsObject].map(jsObject => jsObject ++ Json.obj("point" -> 10))
)
val result2 = jsValue.transform(jsonTransformer) match {
case JsSuccess(jsObject, _) => jsObject
case _ => jsValue
}
Assuming from your question tags that you're using the PlayJSON library, you can simply use +:
val json: JsObject = ... // Your initial value here
val newField: JsValue = ... // Your new value here
val jsonWithAdditionalField = json + ("yourKey" -> newField)
For more information about how to create the JsValue, take a look a the official documentation
Append the list to a variable and then call put like so:
list.put("test");

How to ignore an item when generating the json string if the value is None?

I'm trying to use Argonaut to generate JSON string from a Scala instance.
import argonaut._, Argonaut._
case class Person(name: Option[String], age: Int, things: List[String])
implicit def PersonCodecJson =
casecodec3(Person.apply, Person.unapply)("name", "age", "things")
val person = Person(Some("Freewind"), 2, List("club"))
val json: Json = person.asJson
val prettyprinted: String = json.spaces2
It will generate:
{
"name" : "Freewind",
"age" : 2,
"things" : [
"club"
]
}
And when the name is None:
val person = Person(None, 2, List("club"))
It will generate:
{
"name" : null,
"age" : 2,
"things" : [
"club"
]
}
But actually I want it to be:
{
"age" : 2,
"things" : [
"club"
]
}
How to do it?
Resolved, the key is to define custom EncodeJson rule and use ->?: and field.map:
implicit def PersonCodecJson: EncodeJson[Person] = EncodeJson((p: Person) =>
p.name.map("name" := _) ->?: ("age" := p.age) ->: ("things" := p.things) ->: jEmptyObject)