Groovy JsonBuilder: object properties being serialized in random order - json

I'm currently having a problem with the groovy JsonBuilder: the properties of the objects I am trying to serialize are coming out in seemingly random order.
Here's the objects' class:
class Game {
String title
String gameImg2
String description
}
And this is the code I've been using:
def game = new Game(title: "a game", gameImg2: "an image", description: "desc")
def json = new JsonBuilder(game)
From this I would expect the output to be:
{"title":"a game","gameImg2":"an image", "description":"desc"}
but instead, I'm getting:
{"gameImg2":"gameImg","title":"hello","description":"desc"}.
From looking at the JsonBuilder example code, it seems that order should be maintained, and indeed, it looks like the toJson method iterates over object.properties, which is a LinkedHashMap. I would have thought this would go through the properties in the order they are declared in the class.
My best guess is that this is something to do with my initialisation of the game object - could using Map syntax to assign properties somehow, but again, this seems to create a LinkedHashMap which should preserve order.
Does anyone have an inkling of what I could have done wrong or incorrectly assumed here? Any leads would be a big help!

The problem is that you are serializing an Object to JSON. The object has declared fields that are translated to Java.
In Java it is not (easily) possible to traverse fields in the order of declaration and the JsonBuilder code definitely makes no attempt to allow this behavior.
Looking at the source code for the JsonBuilder one can see that it uses a class called JsonOutput.groovy to serialize the Object you are passing it, like so:
static String toJson(object) {
if (object == null) {
"null"
} else if (object instanceof Collection ||
object.class.isArray() ||
object instanceof Iterator ||
object instanceof Enumeration) {
"[" + object.collect { toJson(it) }.join(',') + "]"
} else if (object instanceof Enum) {
'"' + object.name() + '"'
} else {
def properties = object.properties
properties.remove('class')
properties.remove('declaringClass')
properties.remove('metaClass')
toJson(properties)
}
}
As you can see the code calls the properies member of the object which returns the object members, the order of the fields depends on the JVM and not the order of declaration.
If you want to maintain order you will need to either pass in a map representing the object or build the JSON object in order manually.
The other alternative is writing your own custom builder, which is much more complicated...

Related

Kotlinx.Serialization using OkHTTPClient return always Failure

Hello I have a problem with my JSON. I am using OkHTTPClient to get JSON from web - to get objects from JSON using kotlinx.serialization via method which contains this and return value from method should be Result :
private suspend inline fun <reified T> OkHttpClient.get(webUrl: HttpUrl): Result<T> =
try {
//Builder defined here ... but skipping this line of code
val data = Json { ignoreUnknownKeys = true }.decodeFromString<T (result.body!!.string())
Result.Success(data)
} catch (e: Exception) {
Result.Failure(e)
}
suspend fun getFact(): Result<Fact> =
client.httpGet("myURL".toHttpUrl())
Json from myURL:
{"status":"success","data":[{"fact":"This is random information i need to get"}],"message":"Retrieved Fact"}
My serializer and Serializable data classes:
#Serializable
data class Fact(
#Serializable(with = FactListSerializer::class)
val data: String) java.io.Serializable
object FactListSerializer : JsonTransformingSerializer<List<String>>(ListSerializer(String.serializer())) {
override fun transformDeserialize(element: JsonElement): JsonElement {
return if (element is JsonArray) {
JsonArray(listOf(element)).first()
} else {
element
}
}
}
To be honest I am not sure what I am doing, but I am getting this error all the time when I print val fact = api.getFact():
Fact: Failure(error=kotlinx.serialization.json.internal.JsonDecodingException: Expected JsonPrimitive at 0, found {"fact":"This is random information i need to get"}
What I need to return is only first element of array fact, because JSON obtain always only 1 fact inside array. So I don't want to return from Serializer/Json List but only Fact object.
But as you see I am obtaining always Result Fauilure, don't know why. My goal is to obtain Result Success and obtaining from that JSON object Fact (only one), but I am not sure if I am doing it correct (obviously not) and even if it is even possible to return from JSONArray only one object (element of type Fact).
So what I expect is something like this:
Fact: Success(value=Fact(fact=This is random information i need to get))
I think the deserializer definition should be changed on 3 levels. The example of how to use JsonTransformingDeserializer in the docs actually describes most of what you need.
JsonArray(listOf(element)).first() should just be element.first(). Here you're building a JsonArray containing your initial JsonArray as only element, and then taking the first, so you basically get back the exact same element.
The type parameter T of JsonTransformingSerializer is supposed to be the type of the property it's applied to, so you should at least get a warning in the code because yours is defined to work on List<String> but is applied to a String property. It should be JsonTransformingSerializer<String>(String.serializer()).
You not only need to unwrap the data array, you also need to extract the value of the fact key within the element of that array.
So with all these changes, it should give something like this:
object FactListSerializer : JsonTransformingSerializer<String>(String.serializer()) {
override fun transformDeserialize(element: JsonElement): JsonElement {
val unwrappedData = if (element is JsonArray) element.first() else element
return unwrappedData.jsonObject["fact"] ?: error("missing 'fact' key in 'data' array")
}
}

ServiceStack.Text CSV serialization of IEnumerable<object> ignores custom serialization functions

Firstly, please forgive any rookie mistakes here - I'm not a regular poster I'm afraid.
Now on to the nitty gritty...
I am trying to use ServiceStack.Text to serialize objects to CSV. If I keep it simple, everything works as expected when serializing objects of a known type.
However I want to serialize many objects and I don't know the type at runtime so I am writing a reusable component where all data is treated as a System.Object. We already do this same routine for Json serialization without problems. But CsvSerializer appears to handle objects differently during serialization.
Sample code
public void TestIEnumerableObjectSerialization()
{
var data = GenerateSampleData();
JsConfig<DateTime>.SerializeFn =
time => new DateTime(time.Ticks, DateTimeKind.Utc).ToString("yyyy-MM-dd HH:mm:ss");
var csv = CsvSerializer.SerializeToCsv(data);
Console.WriteLine(csv);
Assert.Equal("DateTime\r\n"
+ "2017-06-14 00:00:00\r\n"
+ "2017-01-31 01:23:45\r\n",
csv);
}
object[] GenerateSampleData()
{
return new object[] {
new POCO
{
DateTime = new DateTime(2017,6,14)
},
new POCO
{
DateTime = new DateTime(2017,1,31, 01, 23, 45)
}
};
}
public class POCO
{
public DateTime DateTime { get; set; }
}
The result of this code is that the custom serialization function is not invoked, and the DateTime is written out using the standard ToString() method.
The cause?
The CsvWriter.Write method is inspecting the type of the records and if the type is Object it is treated as a Dictionary<string, object> and CsvDictionaryWriter generates the output.
In turn, CsvDictionaryWriter uses the ToCsvField() extension method to write each property a record.
The problem is that ToCsvField() converts the value of each property to a string using ToString() meaning no custom serialization is performed.
JsonSerializer uses TypeSerializer.SerializeToString(text) to serialize the properties of an Object using any configured custom serialization functions; but this doesn't happen with CsvSerializer.
A possible solution?
Without complicating CsvSerializer, the ToCsvField() extension method could be updated to use TypeSerializer to handle the serialization to a string. Here is what I've been testing with so far:
public static object ToCsvField(this object text)
{
var textSerialized = TypeSerializer.SerializeToString(text).StripQuotes();
return textSerialized == null || !CsvWriter.HasAnyEscapeChars(textSerialized)
? textSerialized
: string.Concat
(
CsvConfig.ItemDelimiterString,
textSerialized.Replace(CsvConfig.ItemDelimiterString, CsvConfig.EscapedItemDelimiterString),
CsvConfig.ItemDelimiterString
);
}
So far I haven't come across an issue with this change, although someone may prefer not to allocate a new intermediate variable before the return statement.
Hopefully that is enough information, so on to my questions...
Has anyone else experienced this issue?
Am I doing something wrong and should I be serializing Objects a different way?
If this is a suitable fix/implementation of TypeSerializer, what are the chances of this being addressed in an update to ServiceStack.Text? I would raise an issue on GitHub but the ServiceStack.Text repo doesn't let me raise issues.
Thanks in advance.

Why I parse json into a Java List, but not a Scala List?

I am attempting to parse a json object that contains a list. I am able to parse the list if the field is backed by a Java List, but it fails if the field is backed by a Scala list. What is the difference between parsing into a Scala List vs a Java List, and what do I have to change to be able to parse this into a Scala List?
object JsonParsingExample extends App {
val objectMapper = new ObjectMapper()
// This line succeeds.
objectMapper.readValue("""{"list": ["a","b"]}""", classOf[JavaList])
// This line fails.
objectMapper.readValue("""{"list": ["a","b"]}""", classOf[ScalaList])
}
case class JavaList() {
#JsonProperty(value = "list")
var myList: java.util.ArrayList[String] = null
}
case class ScalaList() {
#JsonProperty(value = "list")
var myList: List[String] = null
}
The error message I receive is:
com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of scala.collection.immutable.List, problem: abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information
Jackson doesn't know anything about Scala types by default (otherwise it would have to depend on scala-library). To teach it, use jackson-module-scala.
Because the scala.collection.immutable.List is actually an abstract class. Generally when you use List("a", "b", "c") is the object List.apply() which is coming from this line: https://github.com/scala/scala/blob/2.12.x/src/library/scala/collection/immutable/List.scala#L452 and that's actually an inner class (something called scala.collection.immutable.$colon$colon).

Convert grails domain object to JSON and manipulate it

I have a grails object that I am converting using def json = object as JSON. After I have it converted I want to add one more property called pin to the JSON which looks like the following.
[location:[lat:23.03, lon:72.58]]
Only way to do this so far seems like following
Serialize the DomainClass to JSON using grails.converters.json
Convert the JSON to string
Create JSONBoject using the string from Step 2
Add the property
Convert it back to String
Any other way to do this using grails.converters.json? I have tried using Gson but I do not want to go that route because I am getting many Circular Reference Errors
Try this:
domainInstance.properties + [pin: pinInstance] as JSON
I recently needed to do a similar thing. Some caveats:
This is using Grails 2.4.5
I use MongoDB as a backend. As such, I created an object marshaller for MongoDB domain classes. It is printed below, and you can wrap a similar marshaller for your domain class(es):
Marshaller:
class MongodbObjectMarshaller implements ObjectMarshaller<JSON> {
#Override
boolean supports(Object o) { return o?.properties?.dbo }
#Override
void marshalObject(Object obj, JSON converter) throws
ConverterException {
Map propertiesToOutput = obj.properties.dbo
propertiesToOutput.remove("_id") //don't print org.bson.types.ObjectId
propertiesToOutput.remove("version") //don't print gorm verson column
converter.build {
_id obj.id.toString()
propertiesToOutput.each{ entry ->
"$entry.key" entry.value
}
}
}
}
What that marshaller does, it allow in JSON output any of the domain class's properties. obj.properties.dbo is special to MongoDB, but for a regular domain class, you can just grab the properties and exclude the ones you don't need.
Then in my controller, this works:
domainInstance.pin = [location:[lat:23.03, lon:72.58]]
def content = tacticalCard as JSON
because my marshaller now picks up the pin property and prints it.

Typed AS3 JSON Encoder and Decoder?

I need to encode and Decode AS3 Objects in a typed manner. http://code.google.com/p/as3corelib/ only supports untyped encoding and decoding.
http://code.google.com/p/ason/ supports some kind of typed objects but is not very robust, e.g. it fails on Date Objects. Any Recommendations ?
To make it clear: It MUST be JSON and it MUST be strong typed and robust.
JSON is built in in AS3. The preferred method to transmit data over the wire is AMF, which does provide you typed objects.
If you have to use JSON, then I guess that you might have to do with some sort of custom protocol to be able encode/decode with types.
You would actually need a reflection utility that read beans in JSON format and then produce your object. It really depends on how deep you want to go.
as3Commons has a reflect package that could help. They also have a JSONTypeProvider, which is not exactly what you need but can put you in the right tract.
You could modify any of the IOC frameworks to produce the context by parsing JSON instead of the regular XML most of them use.
You could modify ASON and add a custom type parser. You would have to send a variable in your JSON object containing the type of the object. And use that in with flash.utils.getDefinitionByName.
Another approach would be to just parse the objects with a regular JSON parser and then if it has a defined type create an instance of that objet, and initialize the properties.
Something like this, to get you started:
var beanInfo:Object = JSON.decode( jsonString );
beanInfo = _parseBean( beanInfo );
private function _parseBean(beanInfo:Object):Object{
if ( beanInfo.hasOwnProperty("_type") ) {
var clazz:Class = getDefinitionByName( beanInfo._type ) as Class;
beanInfo.__clazz = clazz;
var instance:Object = new clazz;
for( var prop:String in beanInfo ) {
if( instance.hasOwnProperty(prop) ) target[prop] = _getPropertyFrom(beanInfo[prop]);
}
}
}
private function _getPropertyFrom(property:String):* {
var xml:XML = describeType( beanInfo.__clazz );
//find the type of the current property.
var type:String = xml...
//if is a simple object then do something like
switch( type ) {
case "number":
return parseFloat(property ) as Number;
break;
case "int":
case "uint":
return parseInt( property );
break;
case "string":
return property as String;
break;
...
default
//As it is it does not suppor complex objects.
//You would use reflection. But then you could save the whole switch...
break;
}
}
Flash has its own serialization system.
var serializer:ByteArray = new ByteArray();
serializer.writeObject(new Sprite());
serializer.position = 0;
var data:String = serializer.readUTFBytes(serializer.bytesAvailable);
trace(data); //Will show you the binary jibberish
You can use registerClassAlias to add support for custom classes.
JSON doens't really define a means to convey type information. It's just strings and ints and arrays and so on. So basically you need some sort of "pickle" for AS3 that's based on JSON. I would suggest you look into Flex/Flash remoting solutions and see how they package objects to be transmitted for RPC; you might be able to modify that solution to use JSON. I'm actually doubtful you'll find a library like this. Does it have to be JSON? I'm pretty sure there are XML based libraries that do this.
JSON is not implemented in the flash virtual machine, and therefore there is no typed object "JSON" as there is "Xml." So basically you can decode JSON just fine, but the type you're going to get is Object. You can them access data using the key in the object as an associative array.
http://blog.alien109.com/2009/02/11/php5-json-as3corelib-a-beautiful-thing/
JSON lib/utils official from adobe:
http://code.google.com/p/as3corelib/source/browse/#svn%2Ftrunk%2Fsrc%2Fcom%2Fadobe%2Fserialization%2Fjson
As good as it gets. :)
There are two operations you need to consider: 1) serializing an object of a particular type into JSON and 2) deserializing a JSON string into an object of a particular type.
The serialization part is easy - just use the built-in JSON.stringify(). Deserializing a JSON string into an object of a particular type in ActionScript is where it gets interesting (and where the answer to your question is). You need to write your own deserialization function for the classe(s) you will need to deserialize. In that function, you need to provide a reviver function to JSON.parse(), which allows you to customize how the JSON gets deserialized.
For example:
public static function deserializeComplexObject(json:String):ComplexObject
{
if (null == json || "null" == json)
{
return null;
}
var complexObject:ComplexObject = new ComplexObject();
var parsedObject:Object = JSON.parse(
json,
function (key:String, value:Object):*
{
switch (key)
{
case “keyForNumber”:
return value;
case “keyForComplexObject2”:
return deserializeComplexObject2(JSON.stringify(value));
case “keyForComplexObject3”:
return deserializeComplexObject3(JSON.stringify(value));
case “keyForString”:
return value;
case “keyForBoolean”:
return value;
default:
return value;
}
}
);
complexObject.keyForNumber = parsedObject.keyForNumber;
complexObject.keyForComplexObject2 = parsedObject.keyForComplexObject2;
// continue setting fields
// …
return complexObject;
}
Each case statement corresponds to a top-level key in the JSON string. You don't actually need separate case statements for every key - you can use the default case to handle all keys that map to values that are one of the simple types (Object, Array, String, Number, Boolean, null) by returning the value as-is.
I have now forked the json part of http://code.google.com/p/as3corelib/ and added typed object support...