I have a dynamic json object which I want to unmarshal in my Go app. The problem is that some parts of the json are dynamically named, so I don't know what to put in the struc type json tags. To illustrate my problem, please see this playground: https://play.golang.org/p/q8J0VVO7uj
As you can see the s1 can perfectly be unmarshalled, because the struct type indeed has tag description. But s2 cannot be unmarshalled.
So my question is: how can I solve this? Can I make use of interfaces here?
Use a map for dynamic keys:
type ElvisEvent struct {
Timestamp int64 `json:"timestamp"`
Type string `json:"type"`
AssetID string `json:"assetId"`
Metadata struct {
} `json:"metadata"`
ChangedMetadata map[string]struct {
OldValue interface{} `json:"oldValue"`
NewValue interface{} `json:"newValue"`
} `json:"changedMetadata"`
}
playground example
Related
I'm working on unmarshaling some nested json data that I have already written a struct for. I've used a tool that will generate a struct based off json data, but am a bit confused how to work with accessing nested json data (and fields can sometimes be emtpy).
Here is an example of struct:
type SomeJson struct {
status string `json:"status"`
message string `json:"message"`
someMoreData []struct {
constant bool `json:"constant,omitempty"`
inputs []struct {
name string `json:"name"`
type string `json:"type"`
} `json:"inputs,omitempty"`
Name string `json:"name,omitempty"`
Outputs []struct {
Name string `json:"name"`
Type string `json:"type"`
} `json:"outputs,omitempty"`
I'm able to unmarshal the json data and access the top level fields (such as status and message), but am having trouble accessing any data under the someMoreData field. I understand this field is a (I assume an unknown) map of structs, but only have experience working with basic single level json blobs.
For reference this is the code I have to unmarshal the json and am able to access the top level fields.
someData := someJson{}
json.Unmarshal(body, &someData)
So what is exactly the best to access some nested fields such as inputs.name or outputs.name?
to iterate over your particular struct you can use:
for _, md := range someData.someMoreData {
println(md.Name)
for _, out := range md.Outputs {
println(out.Name, out.Type)
}
}
to access specific field:
someData.someMoreData[0].Outputs[0].Name
Couple of things to note:
The struct definition is syntactically incorrect. There are couple of closing braces missing.
type is a keyword.
The status and message and other fields with lower case first letter fields are unexported. So, the Json parser will not throw error, but you will get zero values as output. Not sure that's what you observed.
someMoreData is an array of structs not map.
I want to be able to access a struct field that's resulted from a JSON unmarshal, BUT I want to use that same struct to hide the field when it is marshaled.
Example:
type MyStruct struct {
GoodField string `json:"goodField"`
SecretField string `json:"secret"`
}
Incoming JSON gets unmarshaled and the secret field is accessible
Use the same MyStruct in the server response but hide the secret field.
I've looked at using omitempty and - tags, not working.
you were on the right track with omitempty you just have to set SecretField to "" for it to take effect
package main
import (
"fmt"
"encoding/json"
)
type MyStruct struct {
GoodField string `json:"goodField"`
SecretField string `json:"secret,omitempty"`
}
func main() {
data := MyStruct{}
s := `{"goodField": "xxx", "secret": "yyy"}`
json.Unmarshal([]byte(s), &data);
fmt.Println(data.GoodField, data.SecretField);
data.SecretField = ""
response, _ := json.Marshal(data)
fmt.Println(string(response))
}
In general I think the best pattern is to use separate request and response types, rather than mutating data just to prevent it from being marshaled; that's an intrusive and destructive approach, and the side-effect could bite you. In line with the Go proverb "a little duplication is better than a little dependency," you're better off just separating those concerns rather than try to have one type serve double duty.
If you want, though, you can try using struct embedding to DRY out the field declarations. With a combination of omitempty and - json tags, you can get the desired results:
type payload struct {
A string `json:"a"`
B string `json:"b,omitempty"`
}
type request struct {
payload
}
type response struct {
payload
B string `json:"-"`
}
Here's a worked example of the above.
It's not without its own issues to be aware of -- there are two fields named B here, not one. It works fine if you're just using dot-operator field access as in my example. If you use struct literals, you've got to be careful not to put the value in the wrong spot. Probably a good idea to write constructors for these types, to centralize that concern in one spot and get it right.
Also if you need this field accessible only from this package (which helps you to control access to it at all) you can make it unexported with first lowercase letter. Use this structure for all operations, except Marshalling/unmarshalling.
type MyStruct struct {
GoodField string `json:"goodField"`
secretField string `json:"secret"`
}
And make one more structure what will be used only for Marshalling.
If you can't, or don't want, to set the field to empty, you can accomplish this with a custom marshaler, as well:
type MyStruct struct {
GoodField string `json:"goodField"`
SecretField string `json:"secret"`
}
func (ms *MyStruct) MarshalJSON() ([]byte, error) {
type MyStructWithoutSecretFields struct {
GoodField string `json:"goodField"`
}
noSecrets := MyStructWithoutSecretFields{
GoodField: ms.GoodField,
}
return json.Marshal(noSecrets)
}
I'm writing an application in Go that will recieve two forms of JSON:
Example 1:
{"book_data":{"title":"book-title","page_number":457}}
Example 2:
{"book_data":{"collection":214},"books":{"data":[{"title":"book-title","page_number":457},{"title":"book-title","page_number":354}]}}
I thought that I could create a struct like the following and unmarshal JSON into it:
type Book struct {
Title string `json:"title"`
PageNumber int `json:"page_number"`
}
but that only works for the first example.
How can I handle JSON from both examples?
You can first unmarshal partly in json.RawMessage to next decide depending of unmarshalled payload. And you also can just unmarshal in more generic structure. Something like
type Book struct {
Title string `json:"title"`
PageNumber int `json:"page_number"`
}
type BookShelf struct {
BookData struct {
Book
Collection int `json:"collection"`
} `json:"book_data"`
Books struct {
Data []Book `json:"data"`
} `json:"books"`
}
which for me looks readable, meaningful and handy enough for further processing.
Why not unmarshal to a map[string]interface{} and then use the result to see which form you need to handle? You can then deserialize with a specific struct type for each form.
Another way would be to use the following package to check for differing attributes, so you can decide which struct to use for the real unmarshaling.
https://github.com/go-xmlpath/xmlpath/tree/v2
You can unmarshal to map because your key is string and value may be anything like - map[string]interface{}. If you are not sure any data type or value then use interface{} beacause it can store any value. Then use result to see which form it is, And deserialize to specific struct type.
Another way to convert JSON to go struct is use this tool.
https://mholt.github.io/json-to-go/
I have the following data structure. I need to initialize it without using nested initialization. This data structure will be flushed to output a json file later.
type GeneratePlan struct{
Mode string `json:"mode"`
Name string `json:"name"`
Schema string `json:"schema"`
Version string `json:"version"`
Attack_plans []struct1 `json:"attack-plans"`
}
type struct1 struct {
Attack_plan Attack_plan `json:"attack-plan"`
}
type Attack_plan struct{
Attack_resouces []struct2 `json:"attack-resources"`
}
type struct2 struct {
Attack_resource Attack_resource `json:"attack-resource"`
}
Problem is when I try to append the variable of type struct2 to the Attack_resources[] slice, it gives the error as
cannot use struct2 (type *structs.Struct2) as type structs.Struct2 in append
How can we initialize the struct without using new or any ptr? As, if we use any of the standard struct initialization technique, it will give the above error.
If I change the above data structure and make it hold a pointer to another struct, it doesn't store the values correctly. I am very new to golang. Any help is appreciated. Thanks in advance!
You can initialize a struct value using:
resource := struct2{}
As #nothingmuch pointed out, if you have a struct pointer and need the underlying value, you can dereference the pointer using:
deref := *resource
I am trying to unmarshal a json response from the server into various types, but I
am not finding how to do it.
The types which work are :-
type ServerResponse struct {
Total int
Data []User
}
type User struct {
Name string
Age int
}
and I can successfully unmarshal the json and receive the expected User type.
What I want to do is handle various server responses and convert after the
fact. eg.
type ServerResponse struct {
Total int
Data []ServerItem
}
type User struct {
ServerItem
Name string
Age int
}
type Book struct {
ServerItem
Name string
Author string
}
Then use either User(response.Data) or response.Data.(User) to make it a
concrete type so that later functions type check correctly.
Please could anyone let me know where to start looking to solve this issue.
I don't think that this can be done easily. Just decode to map[string]interface{} and create your stuff from this.
Here I wrote a simple program that parses json http://play.golang.org/p/51eiswgznR.
You may also want to read the encoding/json docs http://golang.org/pkg/encoding/json/