Is there an idiomatic way in go to separate a struct from its json marshal logic?
Normally:
package models
type Foo struct {
Name `json:"full_name"`
}
But I want a separation of concerns. I don't want the json specifying logic in the models package with the struct, maybe put the json logic in another serializers package. How would you do that in idiomatic go? Maybe similar to how rails projects handle active_model_serializers code
It is arguable whether json tags belong to marshaling logic or to json model. I would say specifying json tags just describe the json model and as such it may be better residing next to your Go model.
The marshaling / unmarshaling logic is implemented in the encoding/json package itself. If you need custom logic, you can specify / implement that by implementing the json.Marshaler and json.Unmarshaler interfaces. This means defining methods to your type. In Go you can only specify methods to types being in the same package, so if you would separate your model from your custom parsing logic, the parsing package could not define methods to model types. Spec: Method declarations:
The type denoted by T is called the receiver base type; it must not be a pointer or interface type and it must be declared in the same package as the method.
That being said you would need to define your custom parsing logic on a different type, and you would need further logic to map / copy into the model type as part of the parsing logic. You would lose more than you would gain by separating the model from the parsing logic.
Going further, the struct type you marshal into may contain unexported fields which - if the parsing logic is in the same package - can be initialized properly. By separating model and logic, the logic would have troubles initializing unexported fields. One way would be to provide exported methods or functions, but then those are exported to everyone else, not just for the parsing logic.
I'd say the Go way and the easiest way is to put model and parsing logic into the same package. It still gives you a "small" separation possibility: you may put the type definition in one file (e.g. model.go), and you may put custom parsing logic (methods implementing json.Marshaler and json.Unmarshaler) in another file (e.g. parse.go) but in the same package of course, but it may be better to see a type and all its methods in one place (in one file).
Related
What is the cleanest way to perform nested conversion of a "deep" object that contains mixed python / numpy types to an object containing only python types?
The question is motivated by the need to send the data as JSON, but here I do not have control over json.dumps() because that is the province of a different application. In other words, I cannot specify the JSON encoder.
One possible solution involves adopting the JSON encoder solution anyway, followed by a conversion back to JSON with json.loads(). This would mean every message has two round trips to JSON rather than one which might not be the end of the way. But is there a "better" alternative?
Note that I need to apply recursively so that fact that tolist() or item() sometimes works isn't a complete solution here.
Currently I'm using lift-json to parse json into objects. This is the signature I generally use
net.liftweb.json.parse(json).extract[MyClass]
This is working fine. However, I'm looking to do a little more. Lets say My class has an object of type List[SomeTrait] where SomeTrait is a trait. And I have two different implementations of that trait with the different constructor signatures, but obviously implementing the same methods differently.
Is there a way in either the JSON or the code that will detect which implementation it should use? Such that, the code parsing the json can remain the same but I can continue to add new implementations of SomeTrait
There's no easy way to do that in Lift, as far as I know. You still need to have some attribute to be able to provide the type information about the JSON object itself, so the deserializer will be able to pick up the proper instance.
I'd use Jackson JsonTypeInfo annotation in order to mark subclasses and then use it's ObjectMapper to do the job.
Look https://github.com/FasterXML/jackson-annotations#handling-polymorphic-types for more details.
In Go's JSON package, I saw there are marshal, decode and other functions.
I thought that decode is the opposite to marshal, but latter realized that I might be wrong.
I think the fundamental question that I have is:
What are the relations and differences between marshall/unmarshal, encoding/decoding, and serialization/deserialization for JSON?
Thanks.
See an example here Why are json package's Decode and Marshal methods used here?
I would personally say all those terms are synonyms though less so with encoding/decoding. In Go Marshal and Unmarshal happen to be the terms that are used to describe converting json in a string form to an object and vice versa. However in C# these same methods are called serialize and deserialize, as far as I know that terminology isn't in Go at all (at least not in any std lib).
Encoding can be used as an adjective to describe the format in which some data is stored, the most common use is probably character encoding (UTF-8). In Go it's also used as a noun to describe objects that can unmarshal/marshal json. Marshal/Unmarshal are always used as verbs, you take that action on the json.
Encoding is also used in Go to refer to a larger category of packages that deal with the conversion from one encoding to another.
If you told me you were marshalling, marshalling, deserializing or serializing some object or json I would understand exactly what you meant. If you said you were json encoding an object I would ask a clarifying question. If you said the "response is json encoded" I would get what you mean though I would think it's odd that you used those words rather than just saying "the response is json". Hope that is more or less the information you're looking for.
Oh also, just for more clarity
Unmarshal == deserialize == decode
Marshal == serialize == encode
In the encoding/json package, the Marshal function and the inverse Unmarshalfunction return and operate on single fixed bytes slices. They transform single objects to bytes, and vice versa.
There are also the Encoder and Decoder types. These contain the Encode and Decode methods, and they operate on streams of bytes, taking an io.Reader and io.Writer respectively. They also allow multiple objects to be serialized or deserialized with a newline delimiter using those streams.
The underlying mechanisms of Marshal/Unmarshal functions and the Encoder/Decoder types are identical, they both use the same internal encodeState.marshal and decodeState.unmarshal codepaths. The only real difference is they provide alternative access for various usage patterns.
Looking for some help to serialize a deep nested java objects to Json. The constraint is that I cannot add any annotations or change the current Java code. Looking for a powerful Json library which has configuration options to convert Java to Json without altering the Java Object themselves. Following would be some of the options that might be required
Specify include/exclude fields/methods. Should be able to specify this at nested levels. A is composed of B and B is composed of C. Should have ability to specify include/exclude at C.
Include/Exclude Objects at nested levels.
Rename fields from java properties while converting to Json (at nested level objects too)
Manager circular dependencies.
Was looking at Jackson and Gson for this requirements. While there are tons of options using annotations to specify serialization configs while writing new Java Pojo's, I am looking at options where I need to specify serialization properties without changing the current Java code. Jackson and Gson do seem to have options for these, but not documented in depth.
Which library is easier to configure for the above requirements? Any other powerful library other than Jackson/Gson? Any pointers to this will be of great help.
Thanks much for your time.
As to Jackson, you might consider using so-called mix-in annotations (see f.ex http://www.studytrails.com/java/json/java-jackson-mix-in-annotation.jsp) which allow you to specify mix-ins to use, without adding them directly in the legacy classes.
This would let you use annotation-based configuration, but leave actual classes untouched.
But given all of your requirements, it may perhaps be better to just use Tree Model of Jackson or GSON (get JsonNode or such), and manually handle conversions to your liking.
You may then be able to convert tree value into POJO; Jackson, for example, has method(s) for doing this (ObjectMapper.treeToValue(), .valueToTree(), .convertValue()) which allow conversions of structurally compatible representations.
What does the serialize do?
Why do we need to serialize an Object and again unserialize it?
Is it for any sort of security measures?
Serialization is the process of turning an object or an object graph into a form that is independent from the specifics of the current execution environment.
Deserialization is the reverse of serialization. It is the process of reading the data written during serialization and restoring the object or object graph in the current execution environment.
Serialization is similar to Data Marshalling, as both describe writing out an object as execution-independent data. However, serialization is typically tailored to a specific language/platform, often featuring idioms of the host language, while Data marshalling aims to be language-neutral, providing a level of interoperability.
Serialization formats may be opaque or transparent. For example, Java serialization is opaque - the data is not used for purposes other than for deserialization. Java also offers an XMLEncoder/XMLDecoder that writes objects as XML in terms of their public properties. That format is transparent and can be processed/manipulated easily.
Serialization itself is not a security measure. In fact it can be a vulnerability when dealing with secured data. Users of serialization should ensure that the serialized data is guarded by at least the same level of security as the original object instance. Failure to do so is opening up the data to unauthorized use.
Serialization is the process of converting objects into strings, which can then be unserialized back into the same objects that they originally were.
One reason for serializing an object would be to store the serialized object (a string) into a database, from which you could then re-create when retrieving the string and passing it to unserialize.
Objects cannot be passed around as objects. We serialize them to text, pass them around, and then unserialize them so that they can be used at more than one place or time.
It's for storing objects in files, databases or any thing that can store strings or for passing them to another application/server/whatever.
serialize() gives a string representation of an object while unserialize() rebuilds the objects from a serialized string. Remember that the objects class definition must still be present to rebuild it.
The PHP manual pretty much explains that, too...