In Swift I'm used to setting up a protocol JSONInitializable which defines an init?(json: JSON) meaning that all classes and structs that conform to the protocol are initializable with a JSON object.
Is it true that this isn't possible in Dart (with an abstract class) because static methods and factory initializers are not inherited from an abstract class?
The reason I'm asking is because I'm writing a lot of similar methods for API GET requests, which could easily be merged into one, e.g.:
static Future<T> get<T extends JSONInitializable>(int id) async {
final resourceName = T; // TODO: transform to snake case
final uri = Uri.parse("$kApiHostname/api/$resourceName/$id");
final response = await _secureGet(uri);
if (response == null) {
return null;
}
final responseJson = json.decode(response.body);
final model = T.fromJson(responseJson);
return model;
}
But for that to work I'd need to constrain T to a protocol/interface that defines the .fromJson() initializer.
The feature you want is not available (or planned) in Dart, but there have been discussions on it.
Give this issue a thumbs up: https://github.com/dart-lang/language/issues/356
A MANUAL workaround could be having a map of serializers,deserializers like:
//Register object 1 in singleton
JsonMapper.register<MyObject>(JsonObjectMapper(
(mapper, map) => MyObject.fromJson(map),
(mapper, instance) => instance.toJson(),
));
//Register object 2 in singleton...
...
This way you can deserialize and serialize your objects as long as you have them registered without resorting to it's generic type.
The AUTOMATIC (technically code generated) way would be using a package like simple_json to help you workaround this with a little code generation so you don't mess up registering and eliminating mappers.
And the big plus is that the actual code that transforms your objecto from and to JSON is not stored in the object itself but in the generated classes, thus pushing the responsability of serializaition deserailization from the object into the generated code (in a decoupled manner).
Take my advice with a grain of salt as with both approaches you lose static type checking of wether a type has a registered mapper and can be converted.
Related
Firstly, please forgive any rookie mistakes here - I'm not a regular poster I'm afraid.
Now on to the nitty gritty...
I am trying to use ServiceStack.Text to serialize objects to CSV. If I keep it simple, everything works as expected when serializing objects of a known type.
However I want to serialize many objects and I don't know the type at runtime so I am writing a reusable component where all data is treated as a System.Object. We already do this same routine for Json serialization without problems. But CsvSerializer appears to handle objects differently during serialization.
Sample code
public void TestIEnumerableObjectSerialization()
{
var data = GenerateSampleData();
JsConfig<DateTime>.SerializeFn =
time => new DateTime(time.Ticks, DateTimeKind.Utc).ToString("yyyy-MM-dd HH:mm:ss");
var csv = CsvSerializer.SerializeToCsv(data);
Console.WriteLine(csv);
Assert.Equal("DateTime\r\n"
+ "2017-06-14 00:00:00\r\n"
+ "2017-01-31 01:23:45\r\n",
csv);
}
object[] GenerateSampleData()
{
return new object[] {
new POCO
{
DateTime = new DateTime(2017,6,14)
},
new POCO
{
DateTime = new DateTime(2017,1,31, 01, 23, 45)
}
};
}
public class POCO
{
public DateTime DateTime { get; set; }
}
The result of this code is that the custom serialization function is not invoked, and the DateTime is written out using the standard ToString() method.
The cause?
The CsvWriter.Write method is inspecting the type of the records and if the type is Object it is treated as a Dictionary<string, object> and CsvDictionaryWriter generates the output.
In turn, CsvDictionaryWriter uses the ToCsvField() extension method to write each property a record.
The problem is that ToCsvField() converts the value of each property to a string using ToString() meaning no custom serialization is performed.
JsonSerializer uses TypeSerializer.SerializeToString(text) to serialize the properties of an Object using any configured custom serialization functions; but this doesn't happen with CsvSerializer.
A possible solution?
Without complicating CsvSerializer, the ToCsvField() extension method could be updated to use TypeSerializer to handle the serialization to a string. Here is what I've been testing with so far:
public static object ToCsvField(this object text)
{
var textSerialized = TypeSerializer.SerializeToString(text).StripQuotes();
return textSerialized == null || !CsvWriter.HasAnyEscapeChars(textSerialized)
? textSerialized
: string.Concat
(
CsvConfig.ItemDelimiterString,
textSerialized.Replace(CsvConfig.ItemDelimiterString, CsvConfig.EscapedItemDelimiterString),
CsvConfig.ItemDelimiterString
);
}
So far I haven't come across an issue with this change, although someone may prefer not to allocate a new intermediate variable before the return statement.
Hopefully that is enough information, so on to my questions...
Has anyone else experienced this issue?
Am I doing something wrong and should I be serializing Objects a different way?
If this is a suitable fix/implementation of TypeSerializer, what are the chances of this being addressed in an update to ServiceStack.Text? I would raise an issue on GitHub but the ServiceStack.Text repo doesn't let me raise issues.
Thanks in advance.
I have been working on an RCP Client to handle weather data.
What i do is 2 things, first i scraped the JSON i will be using and put it into a dart file. See: https://dartpad.dartlang.org/a9c1fe8ce34c608eaa28
My server.dart page, will import the weather data, and then carry out the following:
import "dart:io";
import "weather_data.dart";
import "dart:convert";
import "package:rpc/rpc.dart";
find ApiServer _apiServer = new ApiServer(prettyPrint:true);
main() async {
Weather w = new Weather(WeatherJson);
TestServer ts = new TestServer(w);
_apiServer.addApi(ts);
HttpServer server = await HttperServer.bind(InternetAddress.ANY_IP_V4, 12345);
server.listen(_apiServer.httpRequestHandler);
}
class Weather{
Map weather;
Weather(this.weather){
Map get daily => weather["daily"];
}
}
#ApiClass(name:"test_server", version: 'v1', description: 'This is a test server api to ping for some quick sample data.')
class TestServer {
Weather myWeather;
TestServer(this.myWeather){
}
#ApiMethod(method:'GET', path: 'daily')
Map<String, Object> getDaily(){
return myWeather.daily;
}
}
So, the server starts correctly, and i will go to localhost:12345/test_server/v1/daily and it will return this:
{
"summary": {},
"icon": {},
"data": {}
}
which is not correct. If you look up the JSON data, summary and icon are both strings and data is an array. They are also empty, and should contain the data i wanted to return.
Why does this occur? Is it because i am returning a Map<String, Object>? I was trying to set it up to be: Map<String, dynamic> but the dart compiler didnt like it.
How do i get this data to return the correct dataset?
The Dart website for RPC is located at: https://github.com/dart-lang/rpc
and you can see that under methods, the return value of a method can be either an instance of a class or a future. That makes sense as per usual, so I set it to be a Map<String,Object> though trying to be vague about it by saying: Map was not sufficient.
Edit:
When doing this mostly in dart pad without RPC, it seems to work correctly, by a sample of: https://dartpad.dartlang.org/3f6dc5779617ed427b75
This leads me to believe something is wrong with the Parsing tool as it seems the return type in dartpad allows to return Map, Map<String, Object>, and Map<String, dynamic>.
Having had a quick look at the RPC package README here https://pub.dartlang.org/packages/rpc, it seems that methods marked as Api methods (with #ApiMethod) should return an instance of a class with simple fields such as:
class ResourceMessage {
int id;
String name;
int capacity;
}
The RPC package will take that instance and serialize it into JSON based on the field names.
From the README:
The MyResponse class must be a non-abstract class with an unnamed
constructor taking no required parameters. The RPC backend will
automatically serialize all public fields of the the MyResponse
instance into JSON ...
You are returning a nested Map representation of the JSON you want the RPC operation to emit and would guess that the RPC package does not handle it as you are expecting it to.
Re: this from your question:
This leads me to believe something is wrong with the Parsing tool as
it seems the return type in dartpad allows to return Map, Map, and Map.
There is no 'parsing' on JSON going on on your example. The data you have is a set of nested literal Dart Maps, Lists and Strings with the same structure as the JSON it was derived from. It just happens to look like JSON.
In your example you are just selecting and printing a sub-map of your data map (data['daily']), which prints out the String that results from calling toString() - which is recursive so you get the contents of all the nested maps and lists within it.
So it's not a 'deep copy' issue, but a difference in how toString() and the RPC code processes a set of nested maps.
BTW: the return type of your getDaily() method is immaterial. What is returned is just a Map whatever the declared return type of the method is. Remember types in Dart are optional and there for editors and compilers to spot potentially incorrect code. See https://www.dartlang.org/docs/dart-up-and-running/ch02.html#variables.
I am going to piggyback off of #Argenti Apparatus here as there was a lot of information gained from him.
Long story short, the required return type of the method:
#ApiMethod(method:'GET', path: 'daily')
Map<String,Object> getDaily(){ // <-- Map<String,Object>
return myWeather.daily;
}
is the error.
I went through and updated the method signature to be Map<String,String> and it parsed it entirely correct. It did not parse the object as a string, but actually parsed it as a full recursed object.
I went through and for the sake of code cleanliness also changed signatures of Weather properties to reflect what they actually were, Map<String,Object> as well.
All in all, When defining it to be an value type of Object, it was returning curly braces, but setting it as a String parsed it correctly.
I ran it through JSLint to confirm it is correct as well.
I gave a +1 to the helper, I had to dig deeper into the code to see WHY it wasnt doing a Map correctly.
This also I feel, is plausibly a bug in RPC Dart.
Similar to Jersey: Json array with 1 element is serialized as object BUT on the client side. E.g. I recieve a JSON object where a field is an array regulary, but in case there is only one element, it is a single object.
{"fileInfo":[{"fileName":"weather.arff","id":"10"},"fileName":"supermarket.arff","id":"11"}]}
versus
{"fileInfo":{"fileName":"weather.arff","id":"10"}}
I'm parsing/unmarshalling the JSON using Jersey/Genson. Of course, if the JSON doesnt match the target class I recieve an error (such as expected [ but read '{' )
I've read a lot about this bug and how to avoid when creating JSON objects on the SERVER side, but I found nothing about how to handle this issus when dealing on the CLIENT side.
As always, I prefere the most codeless possibility if there are several solutions...
BTW: Moxy works but it does not marshal native Object-type objects which is another requirement...
Update
Starting with Genson 1.3 release you can achieve it by enabling permissiveParsing:
Genson genson = new GensonBuilder().usePermissiveParsing(true).create();
Answer
Uh, do you know what library produces this on server side? I am curious to see who is responsible for all those badly structured jsons out there...
It is not yet supported in Genson. Originally because IMO people should not produce such dynamic json. Anyway, I opened an issue - this can be easily done, you can expect it to be present in the release coming next week.
Otherwise here is a way to achieve it without breaking the existing mechanisms.
You need to register a Factory that will use Gensons collections factory to create an instance of its standard collection converter. Then you will wrap this converter in another one that will handle the object to array logic. Here is the code (not codeless..., but if you wait a bit you won't have to code :)).
import com.owlike.genson.convert.DefaultConverters.CollectionConverterFactory;
import com.owlike.genson.convert.DefaultConverters.CollectionConverterFactory;
class SingleObjectAsCollectionFactory implements Factory<Converter<Collection>> {
// get the default factory
Factory<Converter<Collection<?>>> defaultFactory = CollectionConverterFactory.instance;
#Override
public Converter<Collection> create(Type type, Genson genson) {
// obtain an instance of the correct default converter for this type
final CollectionConverter defaultConverter = (CollectionConverter) defaultFactory.create(type, genson);
// wrap it in your own converter
return new Converter<Collection>() {
#Override
public void serialize(Collection object, ObjectWriter writer, Context ctx) throws Exception {
defaultConverter.serialize(object, writer, ctx);
}
#Override
public Collection deserialize(ObjectReader reader, Context ctx) throws Exception {
if (reader.getValueType() == ValueType.OBJECT) {
Object object = defaultConverter.getElementConverter().deserialize(reader, ctx);
Collection result = defaultConverter.create();
result.add(object);
return result;
} else return defaultConverter.deserialize( reader, ctx );
}
};
}
}
And then register it
Genson genson = new GensonBuilder()
.withConverterFactory(new SingleObjectAsCollectionFactory())
.create();
When it comes to JSON encoding in Dart, per Seth Ladd's accouncement the finally approved now official way to go is dart:convert + JSON.Encode.
Let's say we have a bunch of model classes (PODOs) such as:
class Customer
{
int Id;
String Name;
}
Now, I'd love to be able to just JSON-encode my domain objects like this:
var customer = new Customer()
..Id = 17
..Name = "John";
var json = JSON.encode(customer);
Unfortunately, this won't work...
Uncaught Error: Converting object to an encodable object failed.
Stack Trace:
#0 _JsonStringifier.stringifyValue (dart:convert/json.dart:416)
#1 _JsonStringifier.stringify (dart:convert/json.dart:336)
#2 JsonEncoder.convert (dart:convert/json.dart:177)
....
... unless we explicitly tell dart:convert how to encode:
class Customer
{
int Id;
String Name;
Map toJson() {
Map map = new Map();
map["Id"] = Id;
map["Name"] = Name;
return map;
}
}
Do I really have to add a toJson method to every single one of my model classes, or is there a better way?
EDIT: this is the simple serialization I'm looking for:
{
"Id": 17,
"Name": "John"
}
Compare to ToJson in ServiceStack.Text, for instance.
Dart's serialization library (see Matt B's answer below) seems like a step in the right direction. However, this ...
var serialization = new Serialization()
..addRuleFor(Customer);
var json = JSON.encode(serialization.write(customer, format: new SimpleJsonFormat()));
... produces just an array with the values (no keys):
[17,"John"]
Using the default SimpleMapFormat on the other hand generates this complex representation.
Still haven't found what I'm looking for...
EDIT 2: Adding some context: I'm building a RESTful web service in Dart, and I'm looking for a JSON serialization which can easily be consumed by any client, not just another Dart client. For instance, querying the Stack Exchange API for this very question will create this JSON response. This is the serialization format I'm looking for. - Or, look at typical JSON responses returned by the Twitter REST API or the Facebook Graph API.
EDIT 3: I wrote a small blog post about this. See also the discussion on Hacker News.
IMO this is a major short-coming in Dart, surprising given its Web Application focus. I would've thought that having JSON support in the standard libraries would've meant that serializing classes to and from JSON would work like water, unfortunately the JSON support seems incomplete, where it appears the choices are to work with loosely typed maps or suffer through un-necessary boilerplate to configure your standard (PODO) classes to serialize as expected.
Without Reflection and Mirrors support
As popular Dart platforms like Flutter doesn't support Reflection/Mirrors your only option is to use a code-gen solution. The approach we've taken in ServiceStack's native support for Dart and Flutter lets you generate typed Dart models for all your ServiceStack Services from a remote URL, e.g:
$ npm install -g #servicestack/cli
$ dart-ref https://techstacks.io
Supported in .NET Core and any of .NET's popular hosting options.
The example above generates a Typed API for the .NET TechStacks project using the generated DTOs from techstacks.io/types/dart endpoint. This generates models following Dart's JsonCodec pattern where you can customize serialization for your Dart models by providing a fromJson named constructor and a toJson() instance method, here's an example of one of the generated DTOs:
class UserInfo implements IConvertible
{
String userName;
String avatarUrl;
int stacksCount;
UserInfo({this.userName,this.avatarUrl,this.stacksCount});
UserInfo.fromJson(Map<String, dynamic> json) { fromMap(json); }
fromMap(Map<String, dynamic> json) {
userName = json['userName'];
avatarUrl = json['avatarUrl'];
stacksCount = json['stacksCount'];
return this;
}
Map<String, dynamic> toJson() => {
'userName': userName,
'avatarUrl': avatarUrl,
'stacksCount': stacksCount
};
TypeContext context = _ctx;
}
With this model you can use Dart's built-in json:convert APIs to serialize and deserialize your model to JSON, e.g:
//Serialization
var dto = new UserInfo(userName:"foo",avatarUrl:profileUrl,stacksCount:10);
String jsonString = json.encode(dto);
//Deserialization
Map<String,dynamic> jsonObj = json.decode(jsonString);
var fromJson = new UserInfo.fromJson(jsonObj);
The benefit of this approach is that it works in all Dart platforms, including Flutter and AngularDart or Dart Web Apps with and without Dart 2’s Strong Mode.
The generated DTOs can also be used with servicestack's Dart package to enable an end to end typed solution which takes care JSON serialization into and out of your typed DTOs, e.g:
var client = new JsonServiceClient("https://www.techstacks.io");
var response = await client.get(new GetUserInfo(userName:"mythz"));
For more info see docs for ServiceStack's native Dart support.
Dart with Mirrors
If you're using Dart in a platform where Mirrors support is available I've found using a Mixin requires the least effort, e.g:
import 'dart:convert';
import 'dart:mirrors';
abstract class Serializable {
Map toJson() {
Map map = new Map();
InstanceMirror im = reflect(this);
ClassMirror cm = im.type;
var decls = cm.declarations.values.where((dm) => dm is VariableMirror);
decls.forEach((dm) {
var key = MirrorSystem.getName(dm.simpleName);
var val = im.getField(dm.simpleName).reflectee;
map[key] = val;
});
return map;
}
}
Which you can mixin with your PODO classes with:
class Customer extends Object with Serializable
{
int Id;
String Name;
}
Which you can now use with JSON.encode:
var c = new Customer()..Id = 1..Name = "Foo";
print(JSON.encode(c));
Result:
{"Id":1,"Name":"Foo"}
Note: see caveats with using Mirrors
I wrote the Exportable library to solve such things like converting to Map or JSON. Using it, the model declaration looks like:
import 'package:exportable/exportable.dart';
class Customer extends Object with Exportable {
#export int id;
#export String name;
}
And if you want to convert to JSON, you may:
String jsonString = customer.toJson();
Also, it's easy to initialize new object from a JSON string:
Customer customer = new Customer()..initFromJson(jsonString);
Or alternatively:
Customer customer = new Exportable(Customer, jsonString);
Please, see the README for more information.
An alternative is to use the Serialization package and add rules for your classes. The most basic form uses reflection to get the properties automatically.
Redstone mapper is the best serialization library I've used. JsonObject and Exportable have the downside that you have to extend some of their classes. With Redstone Mapper you can have structures like this
class News
{
#Field() String title;
#Field() String text;
#Field() List<FileDb> images;
#Field() String link;
}
It works with getters and setters, you can hide information by not annotating it with #Field(), you can rename field from/to json, have nested objects, it works on the server and client. It also integrates with the Redstone Server framework, where it has helpers to encode/decode to MongoDB.
The only other framework I've seen thats on the right direction is Dartson, but it still lack some features compared to Redstone Mapper.
I have solved with:
class Customer extends JsonObject
{
int Id;
String Name;
Address Addr;
}
class Address extends JsonObject{
String city;
String State;
String Street;
}
But my goal is bind data from/to json from/to model classes; This solution work if you can modify model classes, in a contrast you must use solution "external" to convert model classes;
see also: Parsing JSON list with JsonObject library in Dart
Another package solving this problem is built_value:
https://github.com/google/built_value.dart
With built_value your model classes look like this:
abstract class Account implements Built<Account, AccountBuilder> {
static Serializer<Account> get serializer => _$accountSerializer;
int get id;
String get name;
BuiltMap<String, JsonObject> get keyValues;
factory Account([updates(AccountBuilder b)]) = _$Account;
Account._();
}
Note that built_value isn't just about serialization -- it also provides operator==, hashCode, toString, and a builder class.
I have achieve with this:
To make this work, pass explicitToJson: true in the #JsonSerializable() annotation over the class declaration. The User class now looks as follows:
import 'address.dart';
import 'package:json_annotation/json_annotation.dart';
part 'user.g.dart';
#JsonSerializable(explicitToJson: true)
class User {
String firstName;
Address address;
User(this.firstName, this.address);
factory User.fromJson(Map<String, dynamic> json) => _$UserFromJson(json);
Map<String, dynamic> toJson() => _$UserToJson(this);
}
You can check here: https://flutter.dev/docs/development/data-and-backend/json#generating-code-for-nested-classes
I prefer using https://ashamp.github.io/jsonToDartModel/ online tool write by myself.
It has features below:
online use, without plugin
support multidimensional list
support complex json
support convert all props to String type
empty props warning
single file
dart keyword protected
instant convert
I think it's better than other tools.Welcome if you have any suggestion, issue or bug report.
Some of the answers are no longer applicable to Flutter 2; here is the process for automatically creating toJson and fromJson methods:
https://flutter.dev/docs/development/data-and-backend/json#creating-model-classes-the-json_serializable-way
PS: I wish this would be as simple as using Newtonsoft library in Asp.Net, this solution is closest to an automated solution
I'm aware that there are multiple way to tell JacksonJson to ignore
properties during rendering but all of them are static. (JasonIgnore, MixIn classes, ..).
This is my scenario. A domain object can implement a interface called FilteredDomain to
allow it to be dynamically filtered. The interface is simple and only exposes one method
"getIgnoreProperties". (A list of properties to ignore).
I then register a Custom Serializer that binds to the FilteredDomain object. The
code looks something like:
private class FilteredDomainSerializer extends JsonSerializer<FilteredDomain> {
public void serialize(FilteredDomain arg, JsonGenerator jgen,
SerializerProvider provder) throws IOException,
JsonProcessingException {
final BeanWrapper wrapper = PropertyAccessorFactory.forBeanPropertyAccess(arg);
for (PropertyDescriptor pd : wrapper.getPropertyDescriptors()) {
final String name = pd.getName();
if (arg.getIgnoreProperties().containsKey(name))
continue;
final Object value = wrapper.getPropertyValue(name);
jgen.writeObjectField(name, value);
}
}
}
First, I really dislike that I need to use the Spring Bean wrapper to get a list of all properties and iterate through them (There must be a way to do this is jackson json).
Second, The code still dosen't work. I get the error:
org.codehaus.jackson.JsonGenerationException: Can not write a field name, expecting a value
at org.codehaus.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
at org.codehaus.jackson.impl.Utf8Generator.writeFieldName(Utf8Generator.java:270)
at org.codehaus.jackson.JsonGenerator.writeObjectField(JsonGenerator.java:1088)
at com.rootmusic.util.SystemJsonObjectMapper$ValueObjectSerializer.serialize(SystemJsonObjectMapper.java:65)
at com.rootmusic.util.SystemJsonObjectMapper$ValueObjectSerializer.serialize(SystemJsonObjectMapper.java:1)
at org.codehaus.jackson.map.ser.ContainerSerializers$IndexedListSerializer.serializeContents(ContainerSerializers.java:304)
at org.codehaus.jackson.map.ser.ContainerSerializers$IndexedListSerializer.serializeContents(ContainerSerializers.java:254)
at org.codehaus.jackson.map.ser.ContainerSerializers$AsArraySerializer.serialize(ContainerSerializers.java:142)
at org.codehaus.jackson.map.ser.MapSerializer.serializeFields(MapSerializer.java:287)
at org.codehaus.jackson.map.ser.MapSerializer.serialize(MapSerializer.java:212)
at org.codehaus.jackson.map.ser.MapSerializer.serialize(MapSerializer.java:23)
at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:606)
at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:280)
The error comes from the fact that you are not writing START_OBJECT / END_OBJECT around field-name/value pairs, so that should be easy to fix.
As to more dynamic filtering, you could read this blog entry which includes standard methods. #JsonView works if you have sets of static definitions (one of which you can dynamically select on per-serialization basis), but if you want yet more dynamic system, #JsonFilter is the way to go.
Alternatively, another relatively simple way would be to first "convert" your POJO into a Map:
Map props = objectMapper.convertValue(pojo, Map.class);
(which is similar to serializing it as JSON, except that result is a Map which would render as JSON)
and then selectively trim Map, and serialize that as JSON. Or, if you prefer, you can use JsonNode ("tree model") as the intermediate thing to modify and then serialize.