I'm implementing a 3rd parties RestFul API. The majority of the Json properties in the request and response data is formatted with underscore characters instead of spaces and all lower case. For example
{
"message_id" : "QWERTY",
"other_reference" : 12345
}
I am not a fan of this format in my internal code, I would prefer my model to be the following.
public class data
{
string MessageId {get; set;}
int OtherReference {get; set;}
}
Is there an easy way to get the serialiser/de-serialiser to do the mapping for me (using System.Text.Json) ? I know I can use the attribute JsonPropertyName to do it, but that means adding the attribute to all properties (there are a lot of them), although this method does allow me to use more meaningful names in my code (some of the names in the Json could be a little confusing to someone visiting the code at a later date if they don't have the subject knowledge)
Thanks
Coming from Java, where you'd use some sort of annotation (JSON-B / Jackson), you'd do the same in C# with the JsonPropertyName you already mentioned.
In my opinion, that's also your best option here since you cannot write a decent algorithm to insert the underscores, that would require that algorithm to somewhat understand the language used and therefore be utterly overkill, prone to errors and not even easy to develop.
My recommendation therefore would be not to fiddle around with the actual properties, rather introduce another class and method to parse the third party's representation into yours if you need it that badly, other than that, JsonPropertyName is probably really the best option.
See: How to customize property names and values with System.Text.Json for further information.
Related
I have a Typescript app. I use the localstorage for development purpose to store my objects and I have the problem at the deserialization.
I have an object meeting of type MeetingModel:
export interface MeetingModel {
date: moment.Moment; // from the library momentjs
}
I store this object in the localStorage using JSON.stringify(meeting).
I suppose that stringify call moment.toJson(), that returns an iso string, hence the value stored is: {"date":"2016-12-26T15:03:54.586Z"}.
When I retrieve this object, I do:
const stored = window.localStorage.getItem("meeting");
const meeting: MeetingModel = JSON.parse(stored);
The problem is: meeting.date contains a string instead of a moment !
So, first I'm wondering why TypeScript let this happen ? Why can I assign a string value instead of a Moment and the compiler agree ?
Second, how can I restore my objects from plain JSON objects (aka strings) into Typescript types ?
I can create a factory of course, but when my object database will grow up it will be a pain in the *** to do all this work.
Maybe there is a solution for better storing in the local storage in the first place?
Thank you
1) TypeScript is optionally typed. That means there are ways around the strictness of the type system. The any type allows you to do dynamic typing. This can come in very handy if you know what you are doing, but of course you can also shoot yourself in the foot.
This code will compile:
var x: string = <any> 1;
What is happening here is that the number 1 is casted to any, which to TypeScript means it will just assume you as a developer know what it is and how you to use it. Since the any type is then assigned to a string TypeScript is absolutely fine with it, even though you are likely to get errors during run-time, just like when you make a mistake when coding JavaScript.
Of course this is by design. TypeScript types only exist during compile time. What kind of string you put in JSON.parse is unknowable to TypeScript, because the input string only exists during run-time and can be anything. Hence the any type. TypeScript does offer so-called type guards. Type guards are bits of code that are understood during compile-time as well as run-time, but that is beyond the scope of your question (Google it if you're interested).
2) Serializing and deserializing data is usually not as simple as calling JSON.stringify and JSON.parse. Most type information is lost to JSON and typically the way you want to store objects (in memory) during run-time is very different from the way you want to store them for transfer or storage (in memory, on disk, or any other medium). For instance, during run-time you might need lookup tables, user/session state, private fields, library specific properties, while in storage you might want version numbers, timestamps, metadata, different types of normalization, etc. You can JSON.stringify anything you want in JavaScript land, but that does necessarily mean it is a good idea. You might want to design how you actually store data. For example, an iso string looks pretty, but takes a lot of bytes. If you have just a few that does not matter, but when you are transferring millions a second you might want to consider another format.
My advise to you would be to define interfaces for the objects you want to save and like moment create a .toJson method on your model object, which will return the DTO (Data Transfer Object) that you can simply serialize with JSON.stringify. Then on the way back you cast the any output of JSON.parse to your DTO and then convert it back to your model with a factory function or constructor of your creation. That might seem like a lot of boilerplate, but in my experience it is totally worth it, because now you are in control of what gets stored and that gives you a lot of flexility to change your model without getting deserialization problems.
Good luck!
You could use the reviver feature of JSON.parse to convert the string back to a moment:
JSON.parse(input, (key, value) => {
if (key == "date") {
return parseStringAsMoment(value);
} else {
return value;
});
Check browser support for reviver, though, as it's not the same as basic JSON.parse
So I've read all the posts on String Based Enums in Typescript, but I couldn't find a solution that meets my requirements. Those would be:
Enums that provide code completion
Enums that can be iterated over
Not having to specify an element twice
String based
The possibilities I've seen so far for enums in typescript are:
enum MyEnum {bla, blub}: This fails at being string based, so I can't simply read from JSONs which are string based...
type MyEnum = 'bla' | 'blub': Not iterable and no code completion
Do it yourself class MyEnum { static get bla():string{return "bla"} ; static get blub():string{return "blub"}}: Specifies elements twice
So here come the questions:
There's no way to satisfy those requirements simultaneously? If no, will it be possible in the future?
Why didn't they make enums string based?
Did someone experience similar problems and how did you solve them?
I think that implementing Enum in a C-like style with numbers is fine, because an Enum (similar to a Symbol) is usually used to declare a value that is uniquely identifiable on development time. How the machine represents that value on run time doesn't really concern the developer.
But what we developer sometimes want (because we're all lazy and still want to have all the benefits!), is to use the Enum as an API or with an API that does not share that Enum with us, even though the API is essentially an Enum because the valid value of a property only is foo and bar.
I guess this is the reason why some languages have string based Enums :)
How TypeScript handles Enums
If you look at the transpiled JavaScript you can see that TypeScript just uses a plain JavaScript Object to implement an Enum. For example:
enum Color {
Red,
Green,
Blue
}
will be transpiled to:
{
0: "Red",
1: "Green",
2: "Blue",
Blue: 2,
Green: 1,
Red: 0
}
This means you can access the string value like Color[Color.Red]. You will still have code completion and you do not have to specify the values twice. But you can not just do Object.keys(Color) to iterate over the Enum, because the values exist "twice" on the object.
Why didn't they make enums string based
To be clear Enums are both number and string based in that direct access is number and reverse map is string (more on this).
Meeting your requirement
You key reason for ruling out raw enums is
so I can't simply read from JSONs which are string based...
You will experience the same thing e.g. when reading Dates cause JSON has no date data type. You would new Date("someServerDateTime") to convert these.
You would use the same strategy to go from server side enum (string) to TS enum (number). Easy done thanks to the reverse lookup MyEnum["someServerString"]
Hydration
This process of converting server side data to client side active data is sometimes called Hydration. My favorite lib for this at the moment is https://github.com/pleerock/class-transformer
I personally handle this stuff myself at the server access level i.e. hand write an API that makes the XHR + does the serialization.
At my last job we automated this with code generation that did even more than that (supported common validation patterns between server and client code).
I have a situation where people consuming our API will need to do a partial update in my resource. I understand that the HTTP clearly specifies that this is a PATCH operation, even though people on our side are used to send a PUT request for this and that's how the legacy code is built.
For exemplification, imagine the simple following struct:
type Person struct {
Name string
Age int
Address string
}
On a POST request, I will provide a payload with all three values (Name, Age, Address) and validate them accordingly on my Golang backend. Simple.
On a PUT/PATCH request though, we know that, for instance, a name never changes. But say I would like to change the age, then I would simply send a JSON payload containing the new age:
PUT /person/1 {age:30}
Now to my real question:
What is the best practice to prevent name from being used/updated intentionally or unintentionally modified in case a consumer of our API send a JSON payload containing the name field?
Example:
PUT /person/1 {name:"New Name", age:35}
Possible solutions I thought of, but I don't actually like them, are:
On my validator method, I would either forcibly remove the unwanted field name OR respond with an error message saying that name is not allowed.
Create a DTO object/struct that would be pretty much an extension of my Person struct and then unmarshall my JSON payload into it, for instance
type PersonPut struct {
Age int
Address string
}
In my opinion this would add needless extra code and logic to abstract the problem, however I don't see any other elegant solution.
I honestly don't like those two approaches and I would like to know if you guys faced the same problem and how you solved it.
Thanks!
The first solution your brought is a good one. Some well known frameworks use to implement similar logic.
As an example, latests Rails versions come with a built in solution to prevent users to add extra data in the request, causing the server to update wrong fields in database. It is a kind of whitelist implemented by ActionController::Parameters class.
Let's suppose we have a controller class as bellow. For purpose of this explanation, it contains two update actions. But you won't see it in real code.
class PeopleController < ActionController::Base
# 1st version - Unsafe, it will rise an exception. Don't do it
def update
person = current_account.people.find(params[:id])
person.update!(params[:person])
redirect_to person
end
# 2nd version - Updates only permitted parameters
def update
person = current_account.people.find(params[:id])
person.update!(person_params) # call to person_params method
redirect_to person
end
private
def person_params
params.require(:person).permit(:name, :age)
end
end
Since the second version allows only permitted values, it'll block the user to change the payload and send a JSON containing a new password value:
{ name: "acme", age: 25, password: 'account-hacked' }
For more details, see Rails docs: Action Controller Overview and ActionController::Parameters
If the name cannot be written it is not valid to provide it for any update request. I would reject the request if the name was present. If I wanted to be more lenient, I might consider only rejecting the request if name is different from the current name.
I would not silently ignore a name which was different from the current name.
This can be solved by decoding the JSON body into a map[string]json.RawMessage first. The json.RawMessage type is useful for delaying the actual decoding. Afterwards, a whitelist can be applied on the map[string]json.RawMessage map, ignoring unwanted properties and only decoding the json.RawMessages of the properties we want to keep.
The process of decoding the whitelisted JSON body into a struct can be automated using the reflect package; an example implementation can be found here.
I am not proficient on Golang but I believe a good strategy would be converting your name field to be a read-only field.
For instance, in a strictly object-oriented language as Java/.NET/C++ you can just provide a Getter but not a Setter.
Maybe there is some accessor configuration for Golang just like Ruby has....
If it is read-only then it shouldn't bother with receiving a spare value, it should just ignore it. But again, not sure if Golang supports it.
I think the clean way is to put this logic inside the PATCH handler. There should be some logic that would update only the fields that you want. Is easier if you unpack into a map[string]string and only iterate over the fields that you want to update. Additionally you could decode the json into a map, delete all the fields that you don't want to be updated, re-encode in json and then decode into your struct.
example: a Customer class have 100 data member(id, name, age, address...etc) to be serialization to JSON.
In Config file such as Web.config, can set a output list to serialize JSON ouptut.
If output only id and name, then JSON only have id and name.
My Question: Can support dynamic DataMember in a DataContract ?
You mean optional datamembers, I guess so, check this question
Surely you'll have to have null values for the ones you dont want to send over the wire.
Another, more dirtier, solution would be to use a dictionary as a datamember and have the fields you want to send as elements there. There may be type conversion issues, but maybe it serves you better.
Edit:
You probably want to go with a dictioray serialized as an associative array en js, as this question specifies. Check the answers and the links in there. That should get you going.
But still I'd go with optional datamembers since it's more of a "contract" thing. Other than that a better description of what you want to do will help.
When I'm working with Hash Tables/Dictionaries I sometimes struggle with how to specify keys.
For example: if I create a simple Dictionary (using Python for this example),
foo = {'bar': 'baz', 'foobar': 'foobaz' }
I can access values (in other modules) with the key values: (foo['bar']) and get baz back.
In the words of Dr. Evil, "pretty standard, really."
Unfortunately, using static strings for keys tightly couples any modules using this Dictionary to its implementation. Of course, this can also apply when using other key types (e.g. Enums, Objects, etc.); anyway you slice it, all modules which access the Dictionary need to know the values for the keys.
To resolve this, I typically use static constant string values (or Enums if available in the language) for keys, and either store them publicly in the local class/module, or in a separate module/class. Therefore any changes to the dictionary keys themselves are kept in a single location.
This usually looks like this:
BAR_KEY = 'bar'
foo[BAR_KEY] = 'foobar'
Are there better ways of specifying keys such that the use of the Dictionary doesn't necessarily couple a module/class to its implementation?
Note: I've seen a few responses in SO which address this (e.g. property-to-reference-a-key-value-pair-in-a-dictionary), but the topics didn't seem to address this issue specifically. The answers were helpful, but I'd like a wider range of experience.
Why not make a class for this, which only contains properties? This is done nicely with Python (from what I know), and works well with other languages, too. Refactoring the names is trivial with today's tools, too.
In cases where I'm passing the object around and I've got known keys, I'd always prefer adding an attribute to an object. IMO, the use case of dictionaries is when you don't know what the keys are.
Python is trivial:
foo.bar=baz
Java is pretty much the same:
class Foo { public String bar="baz"; }
The Python performance would be pretty much identical, since a property lookup is just a dictionary lookup and the Java performance would be better.
I sometimes create a separate class to hold the dictionary keys. That gives them their own namespace, as well as the regular benefits of having the keys be in const strings, namely that you don't have the risk of typos, you get code completion, and the string value is easy to change. If you don't want to create a separate class, you get all of the benefits except a namespace just from have const strings.
That said, I think you're getting close to soft coding territory. If the keys in the dictionary change, it's OK for code using the dictionary to change.
Personally, I use your method. It's pretty sensible, simple, and solves the actual problem.
I usually do the same thing; if the key is always going to be the same, make a 'constant static' in whatever language to hold the key.