I have been reading a lot of related questions but could not find anything that actually fit my problem. I am trying to unmarshall a complex object.
type DC struct {
//other fields
ReplenishmentData map[string]ProductReplenishment `bson:"-"`
//other fields
}
type ProductReplenishment struct {
//Other fields
SafetyStockInDay int `json:"SafetyStockInDay" bson:"SafetyStockInDay"`
AlreadyOrderedQuantityForReplenishment *map[float64]*UnitQuantity `json:"-" bson:"-"`
//Other fields
}
Lets say I decode the following json:
{
"ReplenishmentData": {
"000822-099": {
"SafetyStockInDay": 7
},
"001030-001": {
"SafetyStockInDay": 7
}
}
}
Into a structure instance hierachy in which the AlreadyOrderedQuantityForReplenishment is not empty, after decoding this field will be set to and empty map, overriding the initial value.
Why is the decoder not ignore the field all together as specified in the docs? Am I missing something?
Thanks a lot for any help,
Adding screenshot of inspector before (first) / after (second) if that can help
Your problem is not related to embedded structs - the same issue would occur with a regular struct.
Encoders will skip encoding struct fields marked with the tag qualifier "-".
Decoders when initializing a struct, will use the zero-value for any field that is not initialized via the decoding process. So your map will he initialized to a nil (empty) map.
If you want to preserve settings you'd need to write your own (JSON or BSON) marshaler (doable - but not trivial). Or it may be just as simpler to just restore any zero-values after the decoding process.
Related
I have a struct -
type User struct {
Uid string `firestore:"uid"`
FcmToken string `firestore:"fcmtoken"
}
How do I convert It to map using json.Marshal(user) ,
I know it can be done when struct fields are annotated with json:"fieldname" but I don't know how to do the same when it is annotated with firestore, or is it even possible?
I have used the word annotation, which may not be what it is called, please correct me!
A field tag can contain multiple key/value pairs. See the struct tag documentation for more details.
Edit the field tags to include whatever JSON configuration you want:
type User struct {
Uid string `firestore:"uid" json:"uid"`
FcmToken string `firestore:"fcmtoken" json:"tid"`
}
It is not possible to make the JSON package use the firestore tags or vice versa.
I am writing a Golang API at work which when called, gets data from two different MongoDB Collections and appends it into a struct, converts it to JSON, and stringified and sends to an API (Amazon SQS)
The problem is, defining the struct of the data receiving from MongoDB, while some of the fields are defined properly, some are varying
// IncentiveRule struct defines the structure of Incentive rule from Mongo
type IncentiveRule struct {
... Other vars
Rule Rule `bson:"rule" json:"rule"`
... Other vars
}
// Rule defines the struct for Rule Object inside an incentive rule
type Rule struct {
...
Rules interface{} `bson:"rules" json:"rules"`
RuleFilter RuleFilter `bson:"rule_filter" bson:"rule_filter"`
...
}
// RuleFilter ...
type RuleFilter struct {
Condition string `bson:"condition" json:"condition"`
Rules []interface{} `bson:"rules" json:"rules"`
}
While this works, the interface{} defined inside Rule struct is varying and while getting as BSON and decoding and re-encoding to JSON, instead of encoding as "fookey":"barvalue" in JSON, it is encoded as "Key":"fookey","Value":"barvalue", how to avoid this behavior and have it as "fookey":"barvalue"
If you use interface{}, the mongo-go driver is free to choose whatever implementation it sees fits for representing the results. Often it will choose bson.D to represent documents which is an ordered list of key-value pairs where a pair is a struct having a field for Key and a field for Value , so the Go value can preserve the field order.
If field order is not required / important, you may explicitly use bson.M instead of interface{} and []bson.M instead of []interface{}. bson.M is an unordered map, but it represents fields in the form of fieldName: fieldValue, which is exactly what you want.
Are there any Go libraries which can tidy up Json output before it is sent to users?
We could unmarshall into a struct and do this manually, but we would like to know if there are any libraries which can make it easier to extract keys into the struct, which we could Marshall and send to the user?
The short answer is not really, due to the way Go handles JSON marshalling and un-marshalling. The common pattern for dealing with your use case is just to define a Response struct.
A classic example would be something like the following:
type User struct {
// fields
}
// Response type used when the user is asking about their own fields
type PrivateUserResponse struct {
// fields with struct tags
}
func (u *User) ToPrivateUserResponse() *PrivateUserResponse { ... }
// Response type used when the user is being listed in a public directory
type PublicUserResponse struct {
// fields with struct tags
}
func (u *User) ToPublicUserResponse() *PublicUserResponse { ... }
Because JSON key configuration is handled by struct tags, a library would be ill-suited to handle the unique business logic cases that arise in dealing with this problem. You might be able to find a code generator that solves this in a more generic way, but I'd recommend just writing the structs yourself - Go favors explicit and clear behavior.
I'm trying to decode a json response with a struct type. An instance of the object i'm trying to decode looks as follows:
{
"title": "Some Title",
"views": 344,
"profiles": {
"customField": "somevalue",
"customField2:" :somevalue"
}
}
The golang struct is the following:
type Topic struct {
Title string `json:"title"`
Views string `json:"views"`
Profiles string `json:"profiles"`
}
As you can see, the "Profiles" attribute is a string, since the profiles object is unknown, as the fields inside it can be dinamically defined.
I'm trying to decode this with:
json.NewDecoder(response.Body).Decode(result)
Where result is of type Topic, but isn't working. What type should the "Profiles" attribute be in order to correctly decoding the answer?
Thanks!
Reading the comment it's clear that profiles value could be of any type, for this reason I suggest you to declare the Profiles type as a map[string]interface{}.
Topic becomes:
type Topic struct {
Title string `json:"title"`
Views int32 `json:"views"`
Profiles map[string]interface{} `json:"profiles"`
}
Check out https://github.com/mitchellh/mapstructure
The readme has an answer you probably look for.
problem is if you have configuration or an encoding that changes slightly depending on specific fields.
Perhaps we can't populate a specific structure without first reading the "type" field from the JSON. We could always do two passes over the decoding of the JSON (reading the "type" first, and the rest later). However, it is much simpler to just decode this into a map[string]interface{} structure, read the "type" key, then use something like this library to decode it into the proper structure.
If profiles can vary you should take json.RawMessage here. Internally it is a []byte which late can be unmarshalled into other types, e.g. depending on values of the outer document.
See https://golang.org/pkg/encoding/json/#RawMessage and the examples.
The Profiles should be a struct and initiated along with the container, in this case I assume it's a Status, such as FB status or Tweet, I've made an example here https://play.golang.org/p/tG90idakLP
Remember to instantiate the new profiles inside a newly created status, before you start unmarshalling.
I have JSON (that I cannot control) like this:
{
"foo1":{
"a":{
"up":10,
"down":5
}
},
"foo2":{
"a":{
"up":1,
"down":1
}
},
"bar":{
"up":11,
"down":6
}
}
"foo1" and "foo2" are dynamic.
How can I properly unmarshal this structure in go?
It would be okay if I could just tell go to not try to deserialize "bar" (the inconsistent property).
Go will by default ignore fields unspecified in the struct you unmarshal into.
In this case, your structure would be set up like this:
type NestedProp2 struct {
Up int
Down int
}
type NestedProp struct {
A NestedProp2
}
type Prop struct {
Foo1 NestedProp
Foo2 NestedProp
}
When you call the the json.Unmarshal function, the extra property will not be deserialized:
var prop Prop
err := json.Unmarshal(jsonBlob, &prop)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", prop)
So you get the following output:
{Foo1:{A:{Up:10 Down:5}} Foo2:{A:{Up:1 Down:1}}}
You can see it in action here.
You said:
I have JSON (that I cannot control)
So to what extent you could control? Here I could provide you with some scenario, and hope some of them match your purpose :)
Remember the general rule first:
In Golang, if a JSON key failed to find a matched field in struct, it will not be unmarshalled.
This means, for a key name in a JSON struct, when unmarshalling, it will look for a field in a golang struct at the same level with the same name case-insensitively. If this search failed, this key won't be unmarshalled.
For example, a key named foo1 will look for a field name foo1 in a golang struct at the same indent level. However it also matches with Foo1 or FoO1, since this matching is case-insensitive.
Remember, you could use field tag to specify the field name as well. Please take a look at the official page.
The value of some of the JSON fields are not consistent, and they could be ignored.
This is the case #gnalck solved in his answer. According to the general rule, if those inconsistent field failed to find a match, they will not be unmarshalled. Therefore, just don't put those inconsistent fields in the struct and you will be fine.
The value of some of the JSON fields are not consistent, but they could not be ignored.
In this case, #gnalck failed since those fields could not be ignored. Now a better way is to unmarshal bar into a json.RawMessage, so that you could unmarshal later.
The keys of the JSON object is undetermined, and their value is undetermined as well.
In this case, we could unmarshal the whole JSON object into a map[string]json.RawMessage, and unmarshal each fields later. When unmarshalling to a map, you could iterate through the map to get all the fields, and unmarshal them into a proper struct later.