Converting a nested scala object into a JSON string using Play JSON - json

I've a bunch of case classes that I use to build a complex object Publisher.
sealed case class Status(status: String)
trait Running extends Status
trait Stopped extends Status
case class History(keywords: List[String], updatedAt: Option[DateTime])
case class Creds(user: String, secret: String)
case class Publisher(id: Option[BSONObjectID], name: String, creds: Creds, status: Status, prefs: List[String], updatedAt: Option[DateTime])
I want to convert the Publisher into a JSON string using the play JSON API.
I used Json.toJson(publisher) and it complained about not having an implicit for Publisher. The error went away after I provided the following
implicit val pubWrites = Json.writes[Publisher]
As excepted it is now complaining about not being able to find implicits for Status, BSONObjectID and Creds. However, when I provide implicits for each Status and Creds it still complains.
implicit val statusWrites = Json.writes[Status]
implicit val credsWrites = Json.writes[Creds]
Any idea how to resolve this ? This is the first time I'm using Play JSON. I've used Json4s before and would like to try this using Play JSON if possible before I move go Json4s unless there are clear benefits for using/not using Json4s vs Play JSON.

The order of implicits are also important. From the least important to the most important. If Writes of Publisher requires Writes of Status, the implicit for Writes of Status should be before the Writes of Publisher.
Here is the code I tested to work
import play.modules.reactivemongo.json.BSONFormats._
import play.api.libs.json._
import reactivemongo.bson._
import org.joda.time.DateTime
sealed case class Status(status: String)
trait Running extends Status
trait Stopped extends Status
case class History(keywords: List[String], updatedAt: Option[DateTime])
case class Creds(user: String, secret: String)
case class Publisher(id: Option[BSONObjectID], name: String, creds: Creds, status: Status, prefs: List[String], updatedAt: Option[DateTime])
implicit val statusWrites = Json.writes[Status]
implicit val credsWrites = Json.writes[Creds]
implicit val pubWrites = Json.writes[Publisher]
val p = Publisher(
Some(new BSONObjectID("123")),
"foo",
Creds("bar","foo"),
new Status("foo"),
List("1","2"),
Some(new DateTime))
Json.toJson(p)
//res0: play.api.libs.json.JsValue = {"id":{"$oid":"12"},"name":"foo","creds":{"user":"bar","secret":"foo"},"status":{"status":"foo"},"prefs":["1","2"],"updatedAt":1401787836305}

Related

Why the Json.decodeFromString does not work in the Release version on android app

I am receiving a response from the server as json and want to convert it to an array using kotlinx.serialization.json.Json.decodeFromString. The code works fine in the Debag version, but it crashes in the Release version. Help, please, what am I doing wrong?
My code in class ActivityPro
#Serializable
data class PaymentResult(val id: String, val data: String)
#Serializable
data class PaymentRequest(val result: PaymentResult, val errorCode: Int, val errorMessage: String)
...
val paymentRequest = Json.decodeFromString<PaymentRequest>(responseString)
responseString
{"result":{"id":"2ad333b0-000f-5000-8000-1ec5665f4757","data":"https:\/\/3ds-gate.yoomoney.ru\/card-auth?acsUri=https%3A%2F%2F3ds-ds2.mirconnect.ru%3A443%2Fsc1%2Fpareq&MD=4399694261676&PaReq=eJxVUU1zgjAQ%2FSuOxx5ICKQaZ82MlkM9xFFrDz1mwo4yKmCAov76Jgq1PWXf24%2B8fQvbvUVMPtA0FiUorCq9w0GWToerzZpFPKKh4GIoYTXb4FnCN9oqK3IZBjRgQHroOq3Z67yWoM15vlhKHlIaj4B0EE5oF4kUTPB4zFhIgTwYyPUJ5Zd6KY%2F6CuSOwBRNXturfI0jID2Axh7lvq7LakLItSgO2okNbAPEJ4A8JawaH1Vu0CVLpUpm7fK2aJdbw1SiWnVbc3U7tGq7mwLxFZDqGiWjXhcdD%2BhoQumEhkDuPOiTVyA3n%2FOBoAF10jsGSv%2FR7AEE9Zm%2FDDhTLeam36NHgJeyyNFVOAd%2FY0ixMm6N7nnu8PbuTTW1MysecR7FXIjIG3un%2FLDMucPi8DHNAyC%2BiXQ3I91ZXfTv3D8Xnqa1&TermUrl=https%3A%2F%2Fpaymentcard.yoomoney.ru%3A443%2F3ds%2Fchallenge%2F279%2FsU5z3DrCkvZJlGEBl2rAYmgE45QZ..001.202210"},"errorCode":0,"errorMessage":""}
error log
2022-10-08 10:35:08.894 27772-27772/? E/AndroidRuntime: FATAL EXCEPTION: main
Process: ru.keytomyself.customeraccounting, PID: 27772
jb.p0: Unresolved class: class ru.keytomyself.customeraccounting.ActivityPro$a
at jb.l$a$i.invoke(SourceFile:32)
at jb.r0$a.invoke(SourceFile:2)
at jb.l$a.a(SourceFile:1)
at jb.l.C(Unknown Source:8)
at jb.l.getDescriptor(Unknown Source:0)
at jb.s0.i(SourceFile:1)
at ab.c0.d(Unknown Source:10)
at ze.a0$b.invokeSuspend(SourceFile:2)
at ta.a.resumeWith(Unknown Source:8)
at pd.o0.run(Unknown Source:86)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:8057)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:620)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1011)
I will be grateful for any help
The most likely cause of this is that you have enabled code minification/obfuscation for your release build in your build.gradle. After obfuscation your class will be looking something like this:
#Serializable
data class ABC(val z: String, val zz: String) // PaymentResult class
Now if your try to decode the string {"id":"2ad333b0", "data": "https:"} into this data class, it will fail because the class has no id property, it has been renamed to z.
To prevent this from happening, one solution is to add #SerialName annotation to all the fields in your serializable classes. For example,
#Serializable
data class PaymentResult(
#SerialName("id")
val id: String,
#SerialName("data")
val data: String
)
Now instead of using property name, the library will now use the provided SerialName while decoding the json and these strings aren't obfuscated.
Another solution is to disable obfuscation for these serializable classes. You can use the #Keep annotation to exclude a particular class/function from being obfuscated/minified.
#Keep
#Serializable
data class PaymentResult(val id: String, val data: String)
#Keep
#Serializable
data class PaymentRequest(val result: PaymentResult, val errorCode: Int, val errorMessage: String)
The other way is to add rules to proguard-rules.pro file to exclude certain classes/directories from being obfuscated. Checkout the following resources on how to write those rules:
https://www.guardsquare.com/manual/configuration/usage#keepoptions
How to keep/exclude a particular package path when using proguard?

circe encoding putting :: when a list of case class from trait is not sealed

Hello everyone I am facing a problem with the circe library to translate between json and at scala case class any help will be highly appreciate.
in the past i have an ADT like this.
sealed trait Sons
case class Son1(name: String, belongings: List[String]) extends Sons
case class Son2(lastName: String, belongings: List[String]) extends Sons
and have no problems whatsoever but now because of the design the file in witch this trait is, is costly to change we have to remove the sealed from the trait so now we have this (with the sons in different files/libraries)
trait Sons
case class Son1(name: String, belongings: List[String]) extends Sons
case class Son2(lastName: String, belongings: List[String]) extends Sons
when trying to convert a list of sons from Scala to json the library puts :: before the list generating a problem as can be seen in this example.
object Test extends App{
implicit val encoderSon1: Encoder[Son1]=deriveEncoder[Son1]
implicit val decoderSon1: Decoder[Son1]=deriveDecoder[Son1]
implicit val encoderSon2: Encoder[Son2]=deriveEncoder[Son2]
implicit val decoderSon2: Decoder[Son2]=deriveDecoder[Son2]
implicit val encoderSon: Encoder[Sons] = Encoder.instance {
case son1: Son1 => son1.asJson
case son2: Son2 => son2.asJson
}
implicit val DecoderSon: Decoder[Sons] =
List[Decoder[Sons]](
Decoder[Son1].widen,
Decoder[Son2].widen
).reduceLeft(_ or _)
implicit val encoderSonList: Encoder[List[Sons]] = deriveEncoder
implicit val DecoderSonList: Decoder[List[Sons]] = deriveDecoder
val sonsList: List[Sons] = List(Son1("James",List()),Son2("Mike",List()))
println(s"1->${sonsList}")
println(s"2->${sonsList.asJson.noSpaces}")
val andBack=decode[List[Sons]](sonsList.asJson.noSpaces).fold(fa=>fa.getMessage,fb=>fb.toString())
println(s"3->${andBack}")
}
the prints are printint
1->List(Son1(James,List()), Son2(Mike,List()))
2->{"::":[{"name":"James","belongings":[]},{"lastName":"Mike","belongings":[]}]}
3->Attempt to decode value on failed cursor: DownField(head),DownField(::)
the problem is of course i do not want the list to be put in a field called :: when converting to JSON.
thank you for any help possible :D
Don't use semi auto for List - semiauto uses slightly different mechanism than other derivations to e.g. avoid issues like:
implicit val x: X = implicitly[X] // uses val x, so we have cyclic dependency = error in runtime
For that reason when you are using semiauto here, List instances from Encoder and Decoder are ignored, and instead macro generates code for List ADT - that is case class :: and case object Nil.
If you remove
implicit val encoderSonList: Encoder[List[Sons]] = deriveEncoder
implicit val DecoderSonList: Decoder[List[Sons]] = deriveDecoder
it will work again.

JSON support for companion object

I have a class that looks somewhat like that
import java.time.OffsetDateTime
import spray.json._
import DefaultJsonProtocol._
sealed trait Person {
def firstName: String
def country: String
def lastName: String
def salary: Option[BigDecimal]
}
case class InternalPerson(
firstName: String,
country: String,
lastName: Option[BigDecimal],
salary: Option[BigDecimal]
) extends Person
object Person {
def fromName(name: Name, country: String, salary: Option[BigDecimal]): Person = {
InternalPerson(
firstName = name.firstName,
lastName = name.lastName,
country = country,
salary = salary
)
}
}
object PersonJsonProtocol extends DefaultJsonProtocol {
implicit val personFormat = jsonFormat4(Person.apply)
}
I am just trying to add a json support to my class. whenever I import the protocol and spray.json._ from another classes I get:
Note: implicit value personFormat is not applicable here because it comes after the application point and it lacks an explicit result type
and
value apply is not a member of object of Person
any idea on how to have Json support for companion objects that extends a trait in scala?
If you defined your implicit in terms of the case class, InternalPerson, json formatting should be enabled: implicit val personFormat = jsonFormat4(InternalPerson)
And you won't have to define an apply() method, which you will have to do in either the Person trait or any implementation thereof otherwise.
You can use play framework to deal with Json.
https://www.playframework.com/documentation/2.6.x/ScalaJson
I think it's very easy and intuitive.
For primitive types it's enough to use Json.format[ClassName]. If you have something more complex you can write your own writes and reads. I know that this question is about spray, but another solution can be good.
So for example for InternalPerson it'll be:
import play.api.libs.json.Json
case class InternalPerson {
firstName: String,
country: String,
lastName: Option[BigDecimal],
salary: Option[BigDecimal]
)
object InternalPerson {
implicit val format = Json.format[InternalPerson]
}
In case that you want to do it with Trait it will be the same. Sometimes you need to write explicitly reads and writes.

Spark + Json4s serialization problems

I am using Json4s classes inside of a Spark 2.2.0 closure. The "workaround" for a failure to serialize DefaultFormats is to include their definition inside every closure executed by Spark that needs them. I believe I have done more than I needed to below but still get the serialization failure.
Using Spark 2.2.0, Scala 2.11, Json4s 3.2.x (whatever is in Spark) and also tried using Json4s 3.5.3 by pulling it into my job using sbt. In all cases I used the workaround shown below.
Does anyone know what I'm doing wrong?
logger.info(s"Creating an RDD for $actionName")
implicit val formats = DefaultFormats
val itemProps = df.rdd.map[(ItemID, ItemProps)](row => { <--- error points to this line
implicit val formats = DefaultFormats
val itemId = row.getString(0)
val correlators = row.getSeq[String](1).toList
(itemId, Map(actionName -> JArray(correlators.map { t =>
implicit val formats = DefaultFormats
JsonAST.JString(t)
})))
})
I have also tried another suggestion, which is to set the DefaultFormats implicit in the class constructor area and not in the closure, no luck anywhere.
The JVM error trace is from Spark complaining that the task is not serializable and pointing to the line above (last line in my code anyway) then the root cause is explained with:
Serialization stack:
- object not serializable (class: org.json4s.DefaultFormats$, value: org.json4s.DefaultFormats$#7fdd29f3)
- field (class: com.actionml.URAlgorithm, name: formats, type: class org.json4s.DefaultFormats$)
- object (class com.actionml.URAlgorithm, com.actionml.URAlgorithm#2dbfa972)
- field (class: com.actionml.URAlgorithm$$anonfun$udfLLR$1, name: $outer, type: class com.actionml.URAlgorithm)
- object (class com.actionml.URAlgorithm$$anonfun$udfLLR$1, <function3>)
- field (class: org.apache.spark.sql.catalyst.expressions.ScalaUDF$$anonfun$4, name: func$4, type: interface scala.Function3)
- object (class org.apache.spark.sql.catalyst.expressions.ScalaUDF$$anonfun$4, <function1>)
- field (class: org.apache.spark.sql.catalyst.expressions.ScalaUDF, name: f, type: interface scala.Function1)
- object (class org.apache.spark.sql.catalyst.expressions.ScalaUDF, UDF(input[2, bigint, false], input[3, bigint, false], input[5, bigint, false]))
- element of array (index: 1)
- array (class [Ljava.lang.Object;, size 3)
- field (class: org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$10, name: references$1, type: class [Ljava.lang.Object;)
- object (class org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$10, <function2>)
at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40)
at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:46)
at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:100)
at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:295)
... 128 more
I have another example. You can try it by using spark-shell. I hope it can help you.
import org.json4s._
import org.json4s.jackson.JsonMethods._
def getValue(x: String): (Int, String) = {
implicit val formats: DefaultFormats.type = DefaultFormats
val obj = parse(x).asInstanceOf[JObject]
val id = (obj \ "id").extract[Int]
val name = (obj \ "name").extract[String]
(id, name)
}
val rdd = sc.parallelize(Array("{\"id\":0, \"name\":\"g\"}", "{\"id\":1, \"name\":\"u\"}", "{\"id\":2, \"name\":\"c\"}", "{\"id\":3, \"name\":\"h\"}", "{\"id\":4, \"name\":\"a\"}", "{\"id\":5, \"name\":\"0\"}"))
rdd.map(x => getValue(x)).collect
Interesting. One typical problem is that you run into serialization issues with the implicit val formats, but as you define them inside your loop this should be ok.
I know that this is bit hacky, but you could try the following:
using #transient implicit val
maybe do a minimal test whether JsonAST.JString(t) is serializable

Convert polymorphic case classes to json and back

I am trying to use spray-json in scala to recognize the choice between Ec2Provider and OpenstackProvider when converting to Json and back.
I would like to be able to give choices in "Provider", and if those choices don't fit the ones available then it should not validate.
My attempt at this can be seen in the following code:
import spray.json._
import DefaultJsonProtocol._
case class Credentials(username: String, password: String)
abstract class Provider
case class Ec2Provider(endpoint: String,credentials: Credentials) extends Provider
case class OpenstackProvider(credentials: Credentials) extends Provider
case class Infrastructure(name: String, provider: Provider, availableInstanceTypes: List[String])
case class InfrastructuresList(infrastructures: List[Infrastructure])
object Infrastructures extends App with DefaultJsonProtocol {
implicit val credFormat = jsonFormat2(Credentials)
implicit val ec2Provider = jsonFormat2(Ec2Provider)
implicit val novaProvider = jsonFormat1(OpenstackProvider)
implicit val infraFormat = jsonFormat3(Infrastructure)
implicit val infrasFormat = jsonFormat1(InfrastructuresList)
println(
InfrastructuresList(
List(
Infrastructure("test", Ec2Provider("nova", Credentials("user","pass")), List("1", "2"))
)
).toJson
)
}
Unfortunately, it fails because it can not find a formatter for Provider abstract class.
test.scala:19: could not find implicit value for evidence parameter of type Infrastructures.JF[Provider]
Anyone have any solution for this?
What you want to do is not available out of the box (i.e. via something like type hints that allow the deserializer to know what concrete class to instantiate), but it's certainly possible with a little leg work. First, the example, using a simplified version of the code you posted above:
case class Credentials(user:String, password:String)
abstract class Provider
case class Ec2Provider(endpoint:String, creds:Credentials) extends Provider
case class OpenstackProvider(creds:Credentials) extends Provider
case class Infrastructure(name:String, provider:Provider)
object MyJsonProtocol extends DefaultJsonProtocol{
implicit object ProviderJsonFormat extends RootJsonFormat[Provider]{
def write(p:Provider) = p match{
case ec2:Ec2Provider => ec2.toJson
case os:OpenstackProvider => os.toJson
}
def read(value:JsValue) = value match{
case obj:JsObject if (obj.fields.size == 2) => value.convertTo[Ec2Provider]
case obj:JsObject => value.convertTo[OpenstackProvider]
}
}
implicit val credFmt = jsonFormat2(Credentials)
implicit val ec2Fmt = jsonFormat2(Ec2Provider)
implicit val openStackFmt = jsonFormat1(OpenstackProvider)
implicit val infraFmt = jsonFormat2(Infrastructure)
}
object PolyTest {
import MyJsonProtocol._
def main(args: Array[String]) {
val infra = List(
Infrastructure("ec2", Ec2Provider("foo", Credentials("me", "pass"))),
Infrastructure("openstack", OpenstackProvider(Credentials("me2", "pass2")))
)
val json = infra.toJson.toString
val infra2 = JsonParser(json).convertTo[List[Infrastructure]]
println(infra == infra2)
}
}
In order to be able to serialize/deserialize instances of the abstract class Provider, I've created a custom formatter where I am supplying operations for reading and writing Provider instances. All I'm doing in these functions though is checking a simple condition (binary here as there are only 2 impls of Provider) to see what type it is and then delegating to logic to handle that type.
For writing, I just need to know which instance type it is which is easy. Reading is a little trickier though. For reading, I'm checking how many properties the object has and since the two impls have different numbers of props, I can differentiate which is which this way. The check I'm making here is very rudimentary, but it shows the point that if you can look at the Json AST and differentiate the types, you can then pick which one to deserialize to. Your actual check can be as simple or as complicated as you like, as long as it's is deterministic in differentiating the types.