Are there any Go libraries which can tidy up Json output before it is sent to users?
We could unmarshall into a struct and do this manually, but we would like to know if there are any libraries which can make it easier to extract keys into the struct, which we could Marshall and send to the user?
The short answer is not really, due to the way Go handles JSON marshalling and un-marshalling. The common pattern for dealing with your use case is just to define a Response struct.
A classic example would be something like the following:
type User struct {
// fields
}
// Response type used when the user is asking about their own fields
type PrivateUserResponse struct {
// fields with struct tags
}
func (u *User) ToPrivateUserResponse() *PrivateUserResponse { ... }
// Response type used when the user is being listed in a public directory
type PublicUserResponse struct {
// fields with struct tags
}
func (u *User) ToPublicUserResponse() *PublicUserResponse { ... }
Because JSON key configuration is handled by struct tags, a library would be ill-suited to handle the unique business logic cases that arise in dealing with this problem. You might be able to find a code generator that solves this in a more generic way, but I'd recommend just writing the structs yourself - Go favors explicit and clear behavior.
Related
I have multiple APIs that follow a similar structure on the high level response. It always gives back an answer in that form:
{"data": {"feed":[{...}]}, "success": true}
However, the structure in Feed varies, depending on the concrete API.
I would now like to build an abstract function to process the various APIs. I have the following objects:
type SourceDTO struct { // top level object
Success bool `json:"success"`
Data Feed `json:"data"`
}
type Feed struct {
FeedData []<???> `json:"Feed"`
}
(The real object is more complex, but this shows the idea)
How would be a good way in go to parse this for the different APIs, ut having some common code with some logic based on the high level data (e.g. success)?
EDIT:
I am extending this, to explain more the extend of my question about the "pattern" I am looking for.
I want to create this package that parses the group of APIs. The DTO objects then have to be transferred into some other objects. These 'final' objects are defined in a different package (the entity package) and have then to be persisted.
I am now wondering, how to bring all this together: The 'finaly' entity objects, the transformation functions from DTO to entity, the parsing of the different APIs and their common and different result components.
Where do the transformation functions belong to (package wise)?
EDIT2: Specified FeedData to a slice after digging into the problem (see comments)
You can embed your SourceDTO struct into another struct, like this:
type SourceDTO struct { // top level object
Success bool `json:"success"`
}
type FeedResponse struct {
FeedData YourCustomFeedStruct `json:"feed"`
// Embedded Struct
SourceDTO
}
Now you can access the Success bool from the FeedResponse struct. Also any methods defined on the SourceDTO struct can be accessed from the FeedResponse.
Thanks to #mkopriva for the input for this solution.
In order to have some abstraction in your json unmarshalling it is possible to use interface{} for many use cases.
package main
import (
"encoding/json"
"fmt"
)
type UniversalDTO struct {
Success bool `json:"success"`
Data interface{} `json:"data"`
}
type ConcreteData struct {
Source string `json:"source"`
Site string `json:"site"`
}
func main() {
jsondata := []byte(`{"sucess":"true","data":[{"source":"foo","site":"bar"}]}`)
data := make([]ConcreteData, 0, 10)
dtoToSend := UniversalDTO{Data: &data}
describe(dtoToSend)
describe(dtoToSend.Data)
json.Unmarshal(jsondata, &dtoToSend)
describe(dtoToSend)
describe(dtoToSend.Data)
}
func describe(i interface{}) {
fmt.Printf("(%v, %T)\n", i, i)
}
Test here: https://play.golang.org/p/SSSp_zptMVN
json.Unmarshal expects an object into which the json is being put into. Thus, first we always need an object. Depending on the concrete instance of the target object, the interface{} can be overriden with a concrete struct object (which of course has to be created separately). An important learning here is, that a go interface can also be overridden with a slice. In this way, it is also possible to unmarshal an array into a go object. However, a slice of a struct has to be defined as a slice of pointers to that type.
I have been reading a lot of related questions but could not find anything that actually fit my problem. I am trying to unmarshall a complex object.
type DC struct {
//other fields
ReplenishmentData map[string]ProductReplenishment `bson:"-"`
//other fields
}
type ProductReplenishment struct {
//Other fields
SafetyStockInDay int `json:"SafetyStockInDay" bson:"SafetyStockInDay"`
AlreadyOrderedQuantityForReplenishment *map[float64]*UnitQuantity `json:"-" bson:"-"`
//Other fields
}
Lets say I decode the following json:
{
"ReplenishmentData": {
"000822-099": {
"SafetyStockInDay": 7
},
"001030-001": {
"SafetyStockInDay": 7
}
}
}
Into a structure instance hierachy in which the AlreadyOrderedQuantityForReplenishment is not empty, after decoding this field will be set to and empty map, overriding the initial value.
Why is the decoder not ignore the field all together as specified in the docs? Am I missing something?
Thanks a lot for any help,
Adding screenshot of inspector before (first) / after (second) if that can help
Your problem is not related to embedded structs - the same issue would occur with a regular struct.
Encoders will skip encoding struct fields marked with the tag qualifier "-".
Decoders when initializing a struct, will use the zero-value for any field that is not initialized via the decoding process. So your map will he initialized to a nil (empty) map.
If you want to preserve settings you'd need to write your own (JSON or BSON) marshaler (doable - but not trivial). Or it may be just as simpler to just restore any zero-values after the decoding process.
I'm writing an application in Go that will recieve two forms of JSON:
Example 1:
{"book_data":{"title":"book-title","page_number":457}}
Example 2:
{"book_data":{"collection":214},"books":{"data":[{"title":"book-title","page_number":457},{"title":"book-title","page_number":354}]}}
I thought that I could create a struct like the following and unmarshal JSON into it:
type Book struct {
Title string `json:"title"`
PageNumber int `json:"page_number"`
}
but that only works for the first example.
How can I handle JSON from both examples?
You can first unmarshal partly in json.RawMessage to next decide depending of unmarshalled payload. And you also can just unmarshal in more generic structure. Something like
type Book struct {
Title string `json:"title"`
PageNumber int `json:"page_number"`
}
type BookShelf struct {
BookData struct {
Book
Collection int `json:"collection"`
} `json:"book_data"`
Books struct {
Data []Book `json:"data"`
} `json:"books"`
}
which for me looks readable, meaningful and handy enough for further processing.
Why not unmarshal to a map[string]interface{} and then use the result to see which form you need to handle? You can then deserialize with a specific struct type for each form.
Another way would be to use the following package to check for differing attributes, so you can decide which struct to use for the real unmarshaling.
https://github.com/go-xmlpath/xmlpath/tree/v2
You can unmarshal to map because your key is string and value may be anything like - map[string]interface{}. If you are not sure any data type or value then use interface{} beacause it can store any value. Then use result to see which form it is, And deserialize to specific struct type.
Another way to convert JSON to go struct is use this tool.
https://mholt.github.io/json-to-go/
Is there a way to marshall a struct to JSON that skips any fields that can not be serialised?
E.G. if I marshal
type aStruct struct {
request *http.Request
name string
}
the JSON representation of
type aStruct struct {
name string
}
would result?
By 'fields that can not be serialised' I mean any field that would cause json.Marshal(..) to return an 'error serializing json: unsupported type' error. Having a look through the json package now with a mind to create another Marshal function that will skip a field it fails to serialise rather than abort and return the error.
==== UPDATE ====
I have created this https://github.com/myles-mcdonnell/jsonx which satisfies my use case. At first I thought this is a terrible way to extend the base json package (it's a copy from 1.6.2 with new behaviour added) but the consensus among my colleagues is that this is the idiomatic way to do this.
I am trying to unmarshal a json response from the server into various types, but I
am not finding how to do it.
The types which work are :-
type ServerResponse struct {
Total int
Data []User
}
type User struct {
Name string
Age int
}
and I can successfully unmarshal the json and receive the expected User type.
What I want to do is handle various server responses and convert after the
fact. eg.
type ServerResponse struct {
Total int
Data []ServerItem
}
type User struct {
ServerItem
Name string
Age int
}
type Book struct {
ServerItem
Name string
Author string
}
Then use either User(response.Data) or response.Data.(User) to make it a
concrete type so that later functions type check correctly.
Please could anyone let me know where to start looking to solve this issue.
I don't think that this can be done easily. Just decode to map[string]interface{} and create your stuff from this.
Here I wrote a simple program that parses json http://play.golang.org/p/51eiswgznR.
You may also want to read the encoding/json docs http://golang.org/pkg/encoding/json/