Suppose I have this struct:
type MyStruct struct {
A string `json:"a"`
}
But I receive a response of the form:
{"a": "something", "b": "something", "c": "something"}
i.e. There are more fields than expected, but we only want the field A. Is it safe/allowed in golang to unmarshal the response into MyStruct?
Yes, it is safe, and even used sometimes intentionally. If you need only a few fields from an input, you can define a structure that includes only those fields. In fact, it is harder to detect if there are fields in the input that are left unmarshaled.
Related
Suppose we want to unmarshal the JSON string {"e": "foo", "E": 1}.
Unmarshalling using the type messageUppercaseE works like expected. When using the type message though, the error json: cannot unmarshal number into Go struct field message.e of type string is returned.
Why are we not able to unmarshal the JSON, if only the "e" struct tag is present?
How would I be able to unmarshal the JSON? (I know that I am able to do this via Jeffail/gabs, but would like to stick to the type based approach.)
type message struct {
EventType string `json:"e"`
}
type messageUppercaseE struct {
EventType string `json:"e"`
UppercaseE uint64 `json:"E"`
}
Try it yourself at https://play.golang.org/p/T6KMJRLy7TN
Quoting the docs for unmarshal:
To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal (either the struct field name or its tag), preferring an exact match but also accepting a case-insensitive match.
In this case, it is the case-insensitive match that causes the trouble.
I have a struct such as this one:
type Data struct {
Id string
Value string
Custom customtype1
Special customtype2
TimeStamp Time
}
var model Data
I am reading data from a JSON object. Because the JSON is structured very differently, I can't just directly unmarshall the JSON into the struct. So I am trying to "match" the fields from the JSON objects to those of the struct one by one. I don't actually need to properly unmarshall the JSON data into the struct, all I really need is to be able to assign, for each field, the proper type to its value.
So I unmarshall the JSON to a generic interface, then convert it to a map[string]interface{} and iterate over that. For each field, I try to find a match among the field names in the model variable which I get using reflect.
Now this all works fine, but the problem arises when I try to get the right type for the values.
I can get the Type for a certain field from the model using reflect, but then I can't use that to cast the type of the value I get from the JSON because that is not a type. I can't use a switch statement either, because this is a simplified version of the situation and in reality I'm dealing with 1000+ different possible types. How can I convert the values I have for each field into their proper type ?
The only I can think of solving this would be to recreate a json string that matches the format of the struct and then unmarshall that into its proper struct, but that seems way to convoluted. Surely there must be a simpler way?
Here's a sample JSON (I can not change this structure, unless I rework it within my Go program):
{
"requestId": 101901,
"userName": "test",
"options": [1, 4],
"request": {
"timeStamp": {
"Value1": "11/02/2018",
"Value2": "11/03/2018"
},
"id": {
"Value1": "123abcd",
"Value2": "0987acd",
"Value3": "a9c003"
},
"custom": {
"Value1": "customtype1_value",
"Value2": "customtype1_value"
}
}
}
I'd advise against your current approach. You haven't provided enough context to tell us why you're choosing to unmarshall things one by one, but Go's support for JSON is good enough that I'd guess it is capable of doing what you want.
Are you aware of Marshall's support for struct tags? Those might serve the purpose you're looking for. Your struct would then look something more like:
type Data struct {
Id string `json:"id"`
Value string `json:"value"`
Custom customtype1 `json:"custom_type"`
Special customtype2 `json:"special_type"`
TimeStamp Time `json:"timestamp"`
}
If your problem is that the custom types don't know how to be unmarshalled, you can define custom unmarshalling functions for them.
This would then enable you to unmarshall an object like the following:
{
"id": "foo",
"value": "bar",
"custom_type": "2342-5234-4b24-b23a",
"special_type": "af23-af2f-rb32-ba23",
"timestamp": "2018-05-01 12:03:41"
}
Given the following JSON object:
{
"a": 1,
"b": [1,2,3,4]
}
And the following type:
type Thing struct {
A Int `json:"a"`
B string `json:"b"
}
I would like the Array "b" to stay as a JSON string when marshalled into go.
I currently get the following error:
panic: json: cannot unmarshal array into Go struct field Thing.b of type string
Set the field as a json.RawMessage. It'll be stored as is, without interpretation (ie. as "[1,2,3,4]"), as a slice of bytes, which can be converted to a string easily enough.
If you need a string directly, you'll have to implement the json.Unmarshaler interface on your type and do the conversion yourself.
I'm trying to decode a json response with a struct type. An instance of the object i'm trying to decode looks as follows:
{
"title": "Some Title",
"views": 344,
"profiles": {
"customField": "somevalue",
"customField2:" :somevalue"
}
}
The golang struct is the following:
type Topic struct {
Title string `json:"title"`
Views string `json:"views"`
Profiles string `json:"profiles"`
}
As you can see, the "Profiles" attribute is a string, since the profiles object is unknown, as the fields inside it can be dinamically defined.
I'm trying to decode this with:
json.NewDecoder(response.Body).Decode(result)
Where result is of type Topic, but isn't working. What type should the "Profiles" attribute be in order to correctly decoding the answer?
Thanks!
Reading the comment it's clear that profiles value could be of any type, for this reason I suggest you to declare the Profiles type as a map[string]interface{}.
Topic becomes:
type Topic struct {
Title string `json:"title"`
Views int32 `json:"views"`
Profiles map[string]interface{} `json:"profiles"`
}
Check out https://github.com/mitchellh/mapstructure
The readme has an answer you probably look for.
problem is if you have configuration or an encoding that changes slightly depending on specific fields.
Perhaps we can't populate a specific structure without first reading the "type" field from the JSON. We could always do two passes over the decoding of the JSON (reading the "type" first, and the rest later). However, it is much simpler to just decode this into a map[string]interface{} structure, read the "type" key, then use something like this library to decode it into the proper structure.
If profiles can vary you should take json.RawMessage here. Internally it is a []byte which late can be unmarshalled into other types, e.g. depending on values of the outer document.
See https://golang.org/pkg/encoding/json/#RawMessage and the examples.
The Profiles should be a struct and initiated along with the container, in this case I assume it's a Status, such as FB status or Tweet, I've made an example here https://play.golang.org/p/tG90idakLP
Remember to instantiate the new profiles inside a newly created status, before you start unmarshalling.
I have JSON (that I cannot control) like this:
{
"foo1":{
"a":{
"up":10,
"down":5
}
},
"foo2":{
"a":{
"up":1,
"down":1
}
},
"bar":{
"up":11,
"down":6
}
}
"foo1" and "foo2" are dynamic.
How can I properly unmarshal this structure in go?
It would be okay if I could just tell go to not try to deserialize "bar" (the inconsistent property).
Go will by default ignore fields unspecified in the struct you unmarshal into.
In this case, your structure would be set up like this:
type NestedProp2 struct {
Up int
Down int
}
type NestedProp struct {
A NestedProp2
}
type Prop struct {
Foo1 NestedProp
Foo2 NestedProp
}
When you call the the json.Unmarshal function, the extra property will not be deserialized:
var prop Prop
err := json.Unmarshal(jsonBlob, &prop)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", prop)
So you get the following output:
{Foo1:{A:{Up:10 Down:5}} Foo2:{A:{Up:1 Down:1}}}
You can see it in action here.
You said:
I have JSON (that I cannot control)
So to what extent you could control? Here I could provide you with some scenario, and hope some of them match your purpose :)
Remember the general rule first:
In Golang, if a JSON key failed to find a matched field in struct, it will not be unmarshalled.
This means, for a key name in a JSON struct, when unmarshalling, it will look for a field in a golang struct at the same level with the same name case-insensitively. If this search failed, this key won't be unmarshalled.
For example, a key named foo1 will look for a field name foo1 in a golang struct at the same indent level. However it also matches with Foo1 or FoO1, since this matching is case-insensitive.
Remember, you could use field tag to specify the field name as well. Please take a look at the official page.
The value of some of the JSON fields are not consistent, and they could be ignored.
This is the case #gnalck solved in his answer. According to the general rule, if those inconsistent field failed to find a match, they will not be unmarshalled. Therefore, just don't put those inconsistent fields in the struct and you will be fine.
The value of some of the JSON fields are not consistent, but they could not be ignored.
In this case, #gnalck failed since those fields could not be ignored. Now a better way is to unmarshal bar into a json.RawMessage, so that you could unmarshal later.
The keys of the JSON object is undetermined, and their value is undetermined as well.
In this case, we could unmarshal the whole JSON object into a map[string]json.RawMessage, and unmarshal each fields later. When unmarshalling to a map, you could iterate through the map to get all the fields, and unmarshal them into a proper struct later.