I have this simple pojo class Fueling that i want to serialize to JSON using the built in Groovy JSON Library. However my application halts when trying to serialize.
class Fueling {
int id;
int mileage;
double amount;
double cost;
DateTime dateTime;
String userId;
}
the following tests renders a java.lang.StackOverflowError:
#Test
void parseJoda(){
def fueling = new Fueling(amount: 1.0, cost: 2.3, mileage: 123, dateTime: DateTime.now(DateTimeZone.UTC));
def jsonF = JsonOutput.toJson(fueling);
}
How can i make this serialization work?
EDIT: The JSON data is for persisting and not for display purpuses, so the actual serialization result format is not important just as long as i am able to get it deserialized again
Given you don't care about the format, one simple workaround is to use Maps as your Groovy JSON API input/output and then add in a little code to translate your domain objects to and from Maps.
Serializing
You can use the Map that is returned by getProperties as-is with two modifications: converting the DateTime instance to it's long millisecond representation and removing the class entry (which lead to memory errors for me)
def serialize(){
def fueling = new Fueling(amount: 1.0, cost: 2.3, mileage: 123, dateTime: DateTime.now(DateTimeZone.UTC));
JsonOutput.toJson(
fueling.properties.collectEntries { k, v ->
[(k): k == 'dateTime' ? v.millis : v] // represent DateTime as milliseconds
}.findAll {
it.key != 'class' // remove the unnecessary 'class' property
}
)
}
Deserializing
You can pass the Map that the JsonSlurper spits out directly to your Fueling constructor with the slight modification of converting the dateTime entry from long back to a DateTime instance
def deserialize() {
String json = '{"userId":null,"cost":2.3,"id":0,"dateTime":1439839235603,"amount":1.0,"mileage":123}'
Map props = new JsonSlurper().parseText(json)
new Fueling(props.collectEntries { k, v ->
[(k): k == 'dateTime' ? new DateTime(v) : v ] // convert back to DateTime
})
}
Of course, there comes a point when your domain object tree is large/complex enough to warrant the use of a more extensible 3rd-party JSON library.
Related
I'm calling different APIs, that use the same key name in the JSON file. Depending on the response, there's one field that may be different types.
To be clear:
The key "results" when calling the API nº1 is a JSON object
The key "results" when calling the API nº2 is a JSON array
My code looks like this when using the second API:
data class Result(
#SerializedName("results") var persons:ArrayList<Person> =ArrayList()
)
The question is if there's any way to use the same class, without taking care if it's a JSON array or a JSON object.
I believe you can define results as an instance of com.fasterxml.jackson.databind.JsonNode.
data class Result(
val results: JsonNode
)
Then you can process results based on it's type—whether it is an ArrayNode or an ObjectNode (as both extend JsonNode):
fun processResults(results: JsonNode) = when{
results.isArray -> processArrayNode(results)
else -> processObjectNode(results)
}
private fun processArrayNode(list: JsonNode): *return whatever you need*{
val elements = list
.elements()
.asSequence()
.toList()
val mappedElements = elements.map{
processObjectNode(it)
}
// do whatever you need with the array
}
private fun processObjectNode(person: JsonNode): *return whatever you need*{
//** this will transform the json node into a linkedHashMap where the keys are the json keys and the values are the values (here interpreted as jsonNodes) **/
val fieldsMap = person
.fields()
.asSequence()
.associateBy( {it.key}, {it.value} )
// process whatever you need
}
This is one way to use the same DTO for both API calls. In my opinion, it is not worth the extra work. I would create two DTOs containing the results field, where in one it is an instance of Person, and in the other it is an instance of List<Person>.
Edit: One little upgrade to the above snippet would be to add extension methods to JsonNode:
fun JsonNode.elementsToList(): List<JsonNode> = this
.elements()
.asSequence()
.toList()
fun JsonNode.fieldsToMap(): Map<String, JsonNode> = this
.fields()
.asSequence()
.associateBy({it.key}, {it.value})
You can use ObjectMapper.typeFactory.constructParametricType to handle generic types:
data class Result<T>(
var x:T
)
val om = ObjectMapper()
om.registerModule(KotlinModule())
val parsedList = om.readValue<Result<List<String>>>(
"""{"x":["x1", "x2"]}""",
om.typeFactory.constructParametricType(Result::class.java, List::class.java)
)
println(parsedList)
val parsedMap = om.readValue<Result<Map<String, String>>>(
"""{"x":{"k1": "v1", "k2": "v2"}}""",
om.typeFactory.constructParametricType(Result::class.java, Map::class.java)
)
println(parsedMap)
Gives output:
Result(x=[x1, x2])
Result(x={k1=v1, k2=v2})
Firstly, please forgive any rookie mistakes here - I'm not a regular poster I'm afraid.
Now on to the nitty gritty...
I am trying to use ServiceStack.Text to serialize objects to CSV. If I keep it simple, everything works as expected when serializing objects of a known type.
However I want to serialize many objects and I don't know the type at runtime so I am writing a reusable component where all data is treated as a System.Object. We already do this same routine for Json serialization without problems. But CsvSerializer appears to handle objects differently during serialization.
Sample code
public void TestIEnumerableObjectSerialization()
{
var data = GenerateSampleData();
JsConfig<DateTime>.SerializeFn =
time => new DateTime(time.Ticks, DateTimeKind.Utc).ToString("yyyy-MM-dd HH:mm:ss");
var csv = CsvSerializer.SerializeToCsv(data);
Console.WriteLine(csv);
Assert.Equal("DateTime\r\n"
+ "2017-06-14 00:00:00\r\n"
+ "2017-01-31 01:23:45\r\n",
csv);
}
object[] GenerateSampleData()
{
return new object[] {
new POCO
{
DateTime = new DateTime(2017,6,14)
},
new POCO
{
DateTime = new DateTime(2017,1,31, 01, 23, 45)
}
};
}
public class POCO
{
public DateTime DateTime { get; set; }
}
The result of this code is that the custom serialization function is not invoked, and the DateTime is written out using the standard ToString() method.
The cause?
The CsvWriter.Write method is inspecting the type of the records and if the type is Object it is treated as a Dictionary<string, object> and CsvDictionaryWriter generates the output.
In turn, CsvDictionaryWriter uses the ToCsvField() extension method to write each property a record.
The problem is that ToCsvField() converts the value of each property to a string using ToString() meaning no custom serialization is performed.
JsonSerializer uses TypeSerializer.SerializeToString(text) to serialize the properties of an Object using any configured custom serialization functions; but this doesn't happen with CsvSerializer.
A possible solution?
Without complicating CsvSerializer, the ToCsvField() extension method could be updated to use TypeSerializer to handle the serialization to a string. Here is what I've been testing with so far:
public static object ToCsvField(this object text)
{
var textSerialized = TypeSerializer.SerializeToString(text).StripQuotes();
return textSerialized == null || !CsvWriter.HasAnyEscapeChars(textSerialized)
? textSerialized
: string.Concat
(
CsvConfig.ItemDelimiterString,
textSerialized.Replace(CsvConfig.ItemDelimiterString, CsvConfig.EscapedItemDelimiterString),
CsvConfig.ItemDelimiterString
);
}
So far I haven't come across an issue with this change, although someone may prefer not to allocate a new intermediate variable before the return statement.
Hopefully that is enough information, so on to my questions...
Has anyone else experienced this issue?
Am I doing something wrong and should I be serializing Objects a different way?
If this is a suitable fix/implementation of TypeSerializer, what are the chances of this being addressed in an update to ServiceStack.Text? I would raise an issue on GitHub but the ServiceStack.Text repo doesn't let me raise issues.
Thanks in advance.
I just saw that there are some libraries for running a Dart web server, like Start.
So I was thinking something like this..
If both client and server code is written in Dart, is it possible to send "Dart objects" via websockets (or normal REST for that matter) so that the type information remains on the other end? Or do I need to serialize/deserialize via JSON or something similar on the way? Or am I over thinking things here?
regards Oskar
You will need to serialize the Dart object somehow. You can try JSON, or you can try the heavy-duty serialization package.
There is no fully automatic JSON serialization for custom Dart classes. You will need to add a custom toJson serializer and create some sort of fromJson constructor.
e.g. if you had a Person class, you could do something like this:
import 'dart:json' as json;
class Person {
String name;
int age;
Person(this.name, this.age);
Person.fromJson(String json) {
Map data = json.parse(json);
name = data['name'];
age = data['age'];
}
Map toJson() {
return {'name': name, 'age': age};
}
}
Note: the fromJson is just a convention. You will need to call it somehow, there is no built-in mechanism to take an arbitrary JSON string and call the right constructors on your custom object.
As mentioned above, the serialization package is more heavy weight, but much more full featured. Here is an example from its docs:
// uses the serialization package
var address = new Address();
address.street = 'N 34th';
address.city = 'Seattle';
var serialization = new Serialization()
..addRuleFor(address);
Map output = serialization.write(address);
and
// uses serialization
var serialization = new Serialization()
..addRuleFor(address,
constructor: "create",
constructorFields: ["number", "street"],
fields: ["city"]);
You can use the 'exportable' package to render your class to JSON or a map in a more declarative fashion.
import 'package:exportable/exportable.dart';
class Product extends Object with Exportable
{
#export String ProductName;
#export num UnitPrice;
#export bool Discontinued;
#export num UnitsInStock;
Product(this.ProductName, this.UnitPrice, this.Discontinued, this.UnitsInStock);
}
Product prod = new Product("First", 1.0, false, 3 );
var json = prod.toJson(); // {"ProductName":"First","UnitPrice":1.0,"Discontinued":false,"UnitsInStock":3}
I try to convert an object to JSON.
var obj = { "dt": new DateTime.now() };
var s = stringify(obj);
The runtime throws an exception: "Calling toJson method on object failed."
That's expected since DateTime class doesn't have toJson method.
But what should I do in this case?
Javascript's JSON.stringify function has an optional argument replacer which allows me to provide my own way of serialization of any object even if the object has no toJson method. Is there any similar facility in Dart or maybe I can extend somehow DateTime class with my own toJson method?
JSON conversion only works with maps, lists, strings, numbers, booleans, or null. So what if your object contains another type like DateTime?
DateTime → JSON
Let's start with the following object:
class Person {
Person(this.name, this.birthdate);
String name;
DateTime birthdate;
}
You can convert it to a map like this:
final person = Person('Bob', DateTime(2020, 2, 25));
Map<String, dynamic> map = {
'name': person.name,
'birthdate': person.birthdate,
};
If you tried to encode this right now with jsonEncode (or json.encode), you would get an error because the DateTime is not directly serializeable. There are two solutions.
Solution 1
You could serialize it yourself first like this:
Map<String, dynamic> map = {
'name': person.name,
'birthdate': person.birthdate.toIso8601String(),
};
final jsonString = json.encode(map);
Note:
Here is the difference between toString and toIso8601String:
2020-02-25 14:44:28.534 // toString()
2020-02-25T14:44:28.534 // toIso8601String()
The toIso8601String doesn't have any spaces so that makes it nicer for conversions and sending over APIs that might not deal with spaces well.
Solution 2
You could use the optional toEncodable function parameter on jsonEncode.
import 'dart:convert';
void main() {
final person = Person('Bob', DateTime(2020, 2, 25));
Map<String, dynamic> map = {
'name': person.name,
'birthdate': person.birthdate,
};
final toJson = json.encode(map, toEncodable: myDateSerializer);
}
dynamic myDateSerializer(dynamic object) {
if (object is DateTime) {
return object.toIso8601String();
}
return object;
}
The toEncodable function just converts the input to a string or something that jsonEncode can covert to a string.
JSON → DateTime
There is nothing special here. You just have to parse the string into the type that you need. In the case of DateTime you can use its parse or tryParse methods.
final myMap= json.decode(jsonString);
final name = myMap['name'];
final birthdateString = myMap['birthdate'];
final birthdate = DateTime.parse(birthdateString);
final decodedPerson = Person(name, birthdate);
Note that parse will throw an exception if the format of the string cannot be parsed into a DateTime object.
As a model class
class Person {
Person(this.name, this.birthdate);
String name;
DateTime birthdate;
Person.fromJson(Map<String, dynamic> json)
: name = json['name'],
birthdate = DateTime.tryParse(json['birthdate']),
Map<String, dynamic> toJson() {
return {
'name': name,
'birthdate': birthdate.toIso8601String(),
};
}
}
This will not throw an exception is the date is malformatted, but birthdate would be null.
Notes
See my fuller answer here.
Thanks to this answer for pointing me in the right direction.
Zdeslav Vojkovic's answer is outdated now.
The JSON.encode() method in dart:convert has an optional toEncodable method that is invoked for objects that are not natively serializable to JSON. It's then up to the user to provide a closure that returns an appropriate serialization of the DateTime.
IMO, it's a flaw in dart:json library that stringify doesn't support additional callback to serialize types lacking the implementation of toJson. Strangely enough, parse function does support reviver argument to customize the deserialization process. Probably the reason was along the lines that user can add toJson for their on types, and core library will take care of 'native' types, but DateTime was omitted (maybe because date serialization in JSON is not really a straightforward story).
Maybe the goal of the team was to use custom types instead of Maps (which is nice), but the drawback here is that if your custom type contains 10 other properties and 1 which is of DateTime type, you need to implement toJson which serializes all of them, even integers and strings. Hopefully, once when Mirror API is ready, there will be libraries that implement serialization 'out-of-band' by reading the reflected info on type, see lower for an example. The promise of Dart is that I would be able to use my types both on server and client, but if I have to manually implement serialization/deserialization for all my models then it is too awkward to be usable.
it also doesn't help that DateTime itself is not very flexible with formatting, as there are no other methods besides toString which returns the format useless for serialization.
So here are some options:
wrap (or derive from) DateTime in your own type which provides toJson
patch json.stringifyJsonValue and submit the change or at least submit an issue :)
use some 3-rd party library like json-object (just an example, it also doesn't support DateTime serialization, AFAIK
I am not really happy with any of them :)
I've added a new package to Dart pub.dev that allows json serialization of objects within a structure. This package Jsonize serialize and deserialize custom classes, and handles DateTime out of the box:
List<dynamic> myList = [1, "Hello!", DateTime.now()];
var jsonRep = Jsonize.toJson(myList);
var myDeserializedList = Jsonize.fromJson(jsonRep);
So will do with your example
var obj = { "dt": new DateTime.now() };
var s = Jsonize.toJson(obj);
var obj2 = Jsonize.fromJson(s);
but can do also this
var obj = DateTime.now();
var s = Jsonize.toJson(obj);
var dt = Jsonize.fromJson(s);
i want to De-serialize stream body received from a HTTP web request as a response.but the problem is that when De-serialize has to be done as a specific type. i have multiple Get's and they have different responses(one returns type p, other returns type q). i want to have a generalized method which De-serializes any type and return me something generalized which when type caste would return me the required value.
The magic sauce is knowing which type to populate. Once you have that, creating a generic deserializer is pretty easy:
public object Deserialize( string serialized, Type T ) {
if ( string.IsNullOrEmpty( serialized ) ) {
return null;
}
XmlSerializer xs = new XmlSerializer( T );
MemoryStream ms = new MemoryStream( StringToBytes( s ) ?? new byte[0] );
XmlTextWriter tw = new XmlTextWriter( ms, Encoding.UTF8 );
return xs.Deserialize( ms );
}
It's also easy to turn this into a generic method.