how to deserialize a json string that contains ## with scala' - json

As the title already explains, I would like to deserialize a json string that contains a key that starts with ##. With the ## my standard approach using case classes sadly does not work anymore.
val test = """{"##key": "value"}"""
case class Test(##key: String) // not possible
val gson = new GsonBuilder().create()
val res = gson.fromJson(test, classOf[Test])
How can work with the ## withtout preprocessing the input json string?

The simplest answer is to quote the field name:
case class Test(`##key`: String)

I experimented a bit but it seems that GSON doesn't interoperate well with Scala case classes (or the other way around, I guess it's a matter of perspective). I tried playing around with scala.beans.BeanProperty but it doesn't seem like it makes a difference.
A possible way to go is to use a regular class and the SerializedName annotation, as in this example:
import com.google.gson.{FieldNamingPolicy, GsonBuilder}
import com.google.gson.annotations.SerializedName
final class Test(k: String) {
#SerializedName("##key") val key = k
override def toString(): String = s"Test($key)"
}
val test = """{"##key": "foobar"}"""
val gson = new GsonBuilder().create()
val res = gson.fromJson(test, classOf[Test])
println(res)
You can play around with this code here on Scastie.
You can read more on SerializedName (as well as other naming-related GSON features) here on the user guide.

I'm not a Scala programmer, I just used javap and reflection to check what the Scala compiler generated and slightly "learnt" how some Scala internals work.
It does not work for you because of several reasons:
The Scala compiler puts case class elements annotations to the constructor parameters, whereas Gson #SerializedName can only work with fields and methods:
// does not work as expected
case class Test(#SerializedName("##key") `##key`: String)
From the plain Java perspective:
final Constructor<Test> constructor = Test.class.getDeclaredConstructor(String.class);
System.out.println(constructor);
System.out.println(Arrays.deepToString(constructor.getParameterAnnotations()));
public Test(java.lang.String)
[[#com.google.gson.annotations.SerializedName(alternate=[], value=##key)]]
Not sure why the Scala compiler does not replicate the annotations directly to the fields, but the Java language does not allow annotating parameters with the #SerializedName annotation causing a compilation error (JVM does not treats it as a failure either).
The field name is actually encoded in the class file.
From the Java perspective:
final Field field = Test.class.getDeclaredField("$at$atkey"); // the real name of the `##key` element
System.out.println(field);
System.out.println(Arrays.deepToString(field.getDeclaredAnnotations()));
private final java.lang.String Test.$at$atkey <- this is how the field can be accessed from Java
[] <- no annotations by default
Scala allows moving annotations to fields and this would make your code work accordingly to how Gson #SerializedName is designed (of course, no Scala in mind):
import scala.annotation.meta.field
...
case class Test(#(SerializedName#field)("##key") `##key`: String)
Test(value)
If for some/any reason you must use Gson and can't annotate each field with #SerializedName, then you can implement a custom type adapter, but I'm afraid that you have to have deep knowledge in how Scala works.
If I understand what Scala does, it annotates every generated class with the #ScalaSignature annotation.
The annotation provides the bytes() method that returns a payload that's most likely can be used to detect whether the annotated type is a case class, and probably how its members are declared.
I didn't find such a parser/decoder, but if you find one, you can do the following in Gson:
register a type adapter factory that checks whether it can handle it (basically, analyzing the #ScalaSignature annotation, I believe);
if it can, then create a type adapter that is aware of all case class fields, their names possibly handling the #SerializedName yourself, as you can neither extend Gson ReflectiveTypeAdapterFactory nor inject a name remapping strategy;
take transient fields (for good) and other exclusion strategies (for completeness) into account;
read/write each non-excluded field.
Too much work, right? So I see two easy options here: either use a Scala-aware JSON tool like other people are suggesting, or annotate each field that have such a special name.

Related

Unity JSONUtility to JSON list of base classes

I have a BaseClass and bunch of derived classes.
I also have List<BaseClass> that contains objects from those derived classes.
When I do JSONUtility.ToJson(List<BaseClass>) I get only properties of BaseClass and not derived classes.
And well... I guess it is logical, but can't I force it to use derived class if there's a one or JSONUtility isn't capable of it? So I need to write custom logic for that?
Thanks!
Very probably JSONUtility.ToJson(List<BaseClass>) gets the elements you need with reflection, so the object returned is based on the incoming type.
I would try to obtain the jsons one by one and combine them in the logic, pre casting each of the types. Not tested nor debugged, just an starting point idea to move on:
string jsons;
foreach (var baseClass in baseClassList) {
Type specificType = baseClass.GetType();
string jsonString = JsonUtility.ToJson((specificType)baseClass)
jsons = "[" + string.Join(",", jsonstring) + "]";
}
I faced the same issue, to be honest JsonUtility is not good option for working with List.
My recommendations:
Use array instead of list with this helper class
or Newtonsoft Json Unity Package
I also needed JSON serialization, to call a REST json API, and I suggest to avoid JSONUtility.
It doesn't handle lists or dictionaries, as you saw.
Also it cannot serialize properties defined with { get; set; }, only fields, which is not blocking but not very convenient.
I agree with the recommendation above, just use Newtonsoft. It can serialize anything, and you will also benefit of the Serialization Settings (you can for example setup the contract resolver to convert all property names to snake_case...). See https://www.newtonsoft.com/json/help/html/SerializationSettings.htm

Typescript: undefined calculated property after deserializing in JSON

I'm new to Typescript and I encountered a JSON deserializing problem.
Consider this class:
class Product {
public id!: number;
public get calculatedProperty() : string {
return "Test";
};
};
As you can see calculatedProperty is a runtime calculated property.
Also, consider that I deserialize a JSON string into my object in this way:
var jsonData = '{ "id": 2 }';
let deserialized = JSON.parse(jsonData) as Product;
The problem comes now:
This call console.log(deserialized.id); returns correctly 1.
This call console.log(deserialized.calculatedProperty); returns undefined!
I really don't understand way. It seems that as Product doesn't really create a Product object, because If I directly invoke the constructor, new Product, the calculated property exists.
What am I doing wrong with the JSON deserialization?
Thanks!
TypeScript's job is only to perform type checking during development and make sure we don't make careless mistakes. At the end of the day, all it does is just compiling the script and transform it into good old JavaScript. Therefore, any TypeScript syntax are not applied in runtime.
In other words, type assertions are removed in runtime.
There are also several warnings in the documentation about this:
Like a type annotation, type assertions are removed by the compiler and won’t affect the runtime behavior of your code.
Reminder: Because type assertions are removed at compile-time, there is no runtime checking associated with a type assertion. There won’t be an exception or null generated if the type assertion is wrong.
Besides, the as keyword does not instantiate a constructor. It merely provides a type information (which will be removed during compile-time). The only way we can instantiate a constructor and access its instance properties/methods is through the new keyword.
The JSON.parse method isn't really for converting json into a class rather than an object.
To solve your issue you could potentially convert the json into an object like this:
let deserializedObject = JSON.parse(jsonData) as Object;
and after that you could assign the object to a class like that:
let deserialized = Object.assign(new Product(), deserializedObject);
Note that I have not tested this yet, but it should work.
Also this is fine for simple objects, but not for objects with complex hierarchy.
Look into class-transformer for more information. https://github.com/typestack/class-transformer

Which JSON library to use when storing case objects?

I need to serialize akka events to json. Based on
"What JSON library to use in Scala?" I tried several libraries. Since my serializer should know nothing of all my concrete events, the events consisting of case classes and case objects should be serialised using reflection. json4s seems to match my requirements best.
class Json4sEventAdapter(system: ExtendedActorSystem) extends EventAdapter {
implicit val formats = Serialization.formats(FullTypeHints(List(classOf[Evt])))
override def toJournal(event: Any): Any = event match {
case e: AnyRef =>
write(e).getBytes(Charsets.UTF_8)}
override def fromJournal(event: Any, manifest: String): EventSeq = event match {
case e: Array[Byte] => {
EventSeq.single(read[Evt](new String(e.map(_.toChar))))}}
The problem using json4s is, that no matter which implementation is used Deserialization of objects produces different instances.
Since we heavily use pattern matching for the case object this breaks all our existing code.
So my question is: which JSON library could be used with scala and akka persistence when storing case objects?
Is there even one library that handles deserialization of case objects via reflection correctly? - or does anyone have a good workaround?
I can't comment on Json4s, since I've never used it, but I do know that this is a non-issue in play-json. You would do something like:
import play.api.libs.json._
sealed trait MyEventBase
case object MyEvent extends MyEventBase
implicit val myEventBaseFormat: Format[MyEventBase] = Format(Reads.StringReads.collect(ValidationError("must be the string `MyEvent`") {
case "MyEvent" => MyEvent
}, Writes.pure("MyEvent"))
In this case, the serialization is to a bare string, and so I piggyback on the built-in StringReads to assert that the item should be deserializable to a string, and then use collect to narrow that down to the specific string. But the basic idea is that you provide the specific value you want back from deserialization in your Reads instance. Here, it's the singleton case object. So, whenever you deserialize a MyEventBase resulting in a MyEvent, you'll definitely get that same instance back.
In the real world, MyEventBase probably has other subtypes, and so you structure your Writes instance to create some form of type tag for serialization that your Reads instance can key off of to deserialize to the proper subtype. Like, you might serialize to a JSON object instead of a bare string, and that object would have a type field that identifies the subtype. Or just use something like Play JSON Extensions to automatically synthesize a reasonable Format for your sealed trait.
I highly recommend you to have a look at Stamina. It's been implemented to solve most of the usual issues you will encounter with akka-persistence.
It provides a json serialiser (based on spray-json and shapeless) which supports versioning, auto-migrating at read time as well as a testkit to ensure all older versions of persistent events are still readable.

one Jackson deserializer for multiple types (config by annotation)

I'm trying to change the (de)serialization of a list in one of my classes.
the objects in the list shall be serialised as int (their jpa id) and deserialised accordingly. serialization is simple.
for the deserialization i have a class that can translate the id into the object if id and class are known.
How do i get the necessary class from jackson? all default jackson serialisers have a constructor like this: protected StdDeserialiser(Class<?> vc) so the information is present somewhere.
is there a way to access it during deserialisation?
or before the deserialiser is constructed by jackson?
or inside the HandlerInstantiator?
I only want to overwrite the default deseriliser for certain references so i can't just write a provider or a custom module.
I made it work from inside the deserializer with the help of the ContextDeserializer interface as this supplies the deserializer with the target property.
public JsonDeserializer<?> createContextual(DeserializationContext ctxt, BeanProperty property) throws JsonMappingException {
Class<?> vc = null;
if (property.getType().isCollectionLikeType()) {
vc = property.getType().getContentType().getRawClass();
} else {
vc = property.getType().getRawClass();
}
return new ResourcePathDeserializer(vc, converter);
}
This solution is not perfect as I only get the raw class of the return type or the generic (which might be a parent class or an interface) but that is enough for my requirements.
It would be better if I could access the "real" class that was resolved by Jackson, but for me this works.
First of all, there is nothing fancy about writing a Module: it is just a way for plugging things in, like custom (de)serializers. So no need to avoid that. And you will most like need to write a module to do what you want.
In general it is not a good idea to try to create "universal" serializers or deserializers, and it will probably run into problem. But it depends on what exactly you are trying to do.
Type information will either be:
Implicit from context: you are writing a (de)serializer for type T, and register it for it, so that's your type
Passed by Jackson when (de)serializer is being constructed, via Module interface: modules are asked if they happen to have a (de)serializer for type T. SimpleModule will only use basic Class-to-impl mapping (that's where "simple" comes from); but full custom Module has access to incoming type.
But I don't know if above will work for your use case. Type information must be available from static type (declared content type for the list).

deserialize using Play framework ScalaJsonGenerics vs Jerkson

I seem to be confused when using the play framework on how to deserialize json correctly. With jerkson it looks like you just have to define a case class which then automatically deserializes a json string (stolen from the jerkson docs).
case class Person(id: Long, name: String)
parse[Person]("""{"id":1,"name":"Coda"}""") //=> Person(1,"Coda")
But, with play framework you have to write a lot of boiler plate code to do the same thing. For instance from their documentation.
case class Foo(name: String, entry: Int)
object Foo {
implicit object FopReads extends Format[Foo] {
def reads(json: JsValue) = Foo(
(json \ "name").as[String],
(json \ "entry").as[Int])
def writes(ts: Foo) = JsObject(Seq(
"name" -> JsString(ts.name),
"entry" -> JsNumber(ts.entry)))
}
}
This seems like a lot more work, so i assume i'm either not using it correctly or don't quite understand the advantage of doing it this way. Is there a short cut so that I don't have to write all of this code? If not, should I just be using jerkson in my Action to parse a incoming json string? It seems as though asText is returning a blank string, even when asJson works just fine...which leads me to believe I am definitely doing something wrong.
Thanks
I think there are two answers to your question.
For somewhat less boiler plate, you can use the the Play support for handling case classes. Here's an example for a case class with three fields:
implicit val SampleSetFormat: Format[SampleSet] = productFormat3("sensorId", "times", "values")(SampleSet)(SampleSet.unapply)
I agree that there is more annoying boiler plate, the main reason the Play folks seem to use this approach so they can determine the correct serializer entirely at compile time. So no cost of reflecting as in Jerkson.
I am a total noob with Play and Jerkson, but wholeheartedly recommend doing the least boilerplate approach (using Jerkson lib within each Action). I find that it is philosophically more in line with Scala to do so and it works fine.