Go - JSON Decoder not initializing my struct - json

I am trying to decode some JSON retrieved via http.Get. However, when I check the structs I initialize with fmt.Println, they are always empty.
I suspect it is because my struct's structure does not agree with the returned JSON, but I am not sure how to fix it. In general, I am not exactly quite sure how the decoder works.
This is the JSON:
{
"response":[
{
"list": {
"category":"(noun)",
"synonyms":"histrion|player|thespian|role player|performer|performing artist"
}
},
{
"list": {
"category":"(noun)",
"synonyms":"doer|worker|person|individual|someone|somebody|mortal|soul"
}
}
]
}
Here is what i have tried so far:
type SynonymResponse struct {
lists []SynonymList
}
type SynonymList struct {
category string
synonyms string
}
var synonyms SynonymResponse;
dec := json.NewDecoder(response.Body)
err := dec.Decode(&synonyms)
if err != nil {
log.Fatal(err)
}
fmt.Println(synonyms)
EDIT: Per #Leo's answer and #JimB's hint, there are two issues with my attempt. Below is the proper set of structs, though as Leo pointed out, this will be empty:
type SynonymResponses struct {
resp []SynonymResponse
}
type SynonymResponse struct {
listo SynonymList
}
type SynonymList struct {
cat string
syns string
}

In order for your JSON to be picked up by the decoder, the fields in your struct must be exported.
This means you need you capitalize the field names. If you have custom naming on your fields -> json conversion, you can add json tags to your structs.
This will fix your issue:
type SynonymResponse struct {
Lists []SynonymList `json:"response"`
}
type SynonymList struct {
Category string `json:"category"`
Synonyms string `json:"synonyms"`
}

Related

How can I create a nested json structure including a []byte type?

I'm currently trying to create the following nested json including a list of json and DER encoded byte array () of a certificate using golang:
{
"webhooks":
[{
"clientConfig":{
"caBundle":"<derData []bytes>"
},
"name":"sth_name"
}]
}
Because of <certDerBytes[]>, I need to use a struct, but I don't know how to initialize it.
I've created the struct so far:
type jsonstruct struct {
Webhooks []struct {
ClientConfig struct {
CaBundle string `json:"caBundle"`
} `json:"clientConfig"`
Name string `json:"name"`
} `json:"webhooks"`
}
But cannot instantiate the struct which I have to marshal into json.
I've tried using string literals, many ways of initializing it as you would for a normal non-nested struct.
I've also dividing up the structs i.e. type jsonstruct.. type webhooks ..., etc, but that errored.
I've also initialized the struct from the inside out, but didn't work either.
I need to create the
You're best off probably using base64 on the byte array itself, and include that as the payload of the struct field.
One piece of nit, I personally do not like nested named-structs. Breaking them out lets you have much more flexibility in your code.
For example:
type jsonstruct struct {
Webhooks []Webhook `json:"webhooks"`
}
type Webhook struct {
ClientConfig ClientConfig `json:"clientConfig"`
Name string `json:"name"`
}
type ClientConfig struct {
CaBundle string `json:"caBundle"`
}
func (cc ClientConfig) ToBytes() []byte {
return []byte(base64.StdEncoding.DecodeString(cc.CaBundle))
}
func (cc ClientConfig) FromBytes(cert []byte) {
cc.CaBundle = base64.StdEncoding.EncodeToString(cert)
}

How to handle missing fields in a JSON response dynamically in Go

I'm working on a Go wrapper for an API and I noticed that two of the JSON fields stay empty when they don't have any data.
Basically the API returns a set of information on a given url, and if it was visited at least once, everything is okay and I get a full json that I then Unmarshal into a struct:
{
"stats":{
"status":1,
"date":"09.07.2019",
"title":"Test",
"devices":{
"dev":[
{
"tag":"Desktop"
}
],
"sys":[
{
"tag":"GNU/Linux "
},
{
"tag":"Windows 10"
}
],
"bro":[
{
"tag":"Firefox 67.0"
},
{
"tag":"Chrome 62.0"
}
]
},
"refs":[
{
"link":"www.google.com"
}
]
}
}
This is the struct I'm using:
type Stats struct {
Stats struct {
Status int `json:"status"`
Date string `json:"date"`
Title string `json:"title"`
Devices struct {
Dev []struct {
Tag string `json:"tag"`
} `json:"dev"`
Sys []struct {
Tag string `json:"tag"`
} `json:"sys"`
Bro []struct {
Tag string `json:"tag"`
} `json:"bro"`
} `json:"devices"`
Refs []struct {
Link string `json:"link"`
} `json:"refs"`
} `json:"stats"`
}
When a new url is given, then things become a little bit weird:
{
"stats": {
"status": 1,
"date": "09.07.2019",
"title": "Test2",
"devices": [
],
"refs": [
]
}
}
As you can see, the fields "dev", "sys" and "bro" just disappear because they're not used and when I try to Unmarshal the JSON into the same struct I get json: cannot unmarshal array into Go struct field Stats.device of type [...]
I tried to use two different structs to handle both the responses but I'm sure that there's a way to handle them gracefully with just one.
Any help would be appreciated, thanks!
I finally managed to make it work with an ugly workaround.
I changed my struct to
type Stats struct {
Status int `json:"status"`
Date string `json:"date"`
Title string `json:"title"`
Devices interface{} `json:"devices"`
Refs interface{} `json:"refs"`
}
Then I can finally Unmarshal the JSON in both cases, but I get a map[string]interface{} when an object is passed and an empty interface{} when an empty array is passed. In order to fix this inconsistency, I simply check for the data type and force the use of a JSON intermediate conversion in order to unpack the map[string]interface{} value inside a custom Devices struct:
// Devices contains devices information
type Devices struct {
Dev []struct {
Tag string `json:"tag"`
Clicks string `json:"clicks"`
} `json:"dev"`
Sys []struct {
Tag string `json:"tag"`
Clicks string `json:"clicks"`
} `json:"sys"`
Bro []struct {
Tag string `json:"tag"`
Clicks string `json:"clicks"`
} `json:"bro"`
}
The algorithms I use are the following:
//ForceDevicesToRightType uses a json conversion as intermediary for filling the Stats.Devices
// struct with map[string]interface{} values
func ForceDevicesToRightType(dev interface{}) (Devices, error) {
temp, err := json.Marshal(dev)
if err != nil {
return Devices{}, err
}
// Use a temporary variable of the right type
var devices Devices
err = json.Unmarshal(temp, &devices)
if err != nil {
return Devices{}, err
}
return devices, nil
}
// ForceRefsToRightType uses a json conversion as intermediary for filling the Stats.Refs
// struct with map[string]interface{} values
func ForceRefsToRightType(refs interface{}) (Refs, error) {
temp, err := json.Marshal(refs)
if err != nil {
return Refs{}, err
}
// Use a temporary variable of the right type
var references Refs
err = json.Unmarshal(temp, &references)
if err != nil {
return Refs{}, err
}
return references, nil
}
Since the compiler knows that both Devices and Refs fields are interface{} I cannot simply access any methods after the conversion, so I simply make a cast of the right type and everything works fine.
For example, if I wanted to access the Dev sub-struct, this is the proper way:
y, _ := GetStats()
fmt.Println(y.Devices.(Devices).Dev)
It's ugly, but it works.
Thank you very much for your help, I hope that this method will save you an headache!

How to show empty object instead of empty struct or nil in Go json marshal

I need to show json's empty object {} when do json.Marshal() for a struct pointer. I can only output either null value or empty struct value.
If the person key is filled with &Person{} or new(Person), it will show empty struct like below:
{
"data": {
"person": {
"name": "",
"age": 0
},
"created_date": "2009-11-10T23:00:00Z"
}
}
And if we don't initialize it at all, it will show null.
{
"data": {
"person": null,
"created_date": "2009-11-10T23:00:00Z"
}
}
I want to show "person": {}. Is it possible?
Go Playground for the complete code: https://play.golang.org/p/tT15G2ESPVc
Option A, use the omitempty tag option on all of the Person's fields and make sure the response's field is allocated before marshaling.
type Person struct {
Name string `json:"name,omitempty"`
Age int `json:"age,omitempty"`
}
// ...
resp.Person = new(Person)
https://play.golang.org/p/o3jWdru_8bC
Option B, use a non-pointer wrapper type that embeds the Person pointer type.
type PersonJSON struct {
*Person
}
type Response struct {
Person PersonJSON `json:"person"`
CreatedDate time.Time `json:"created_date"`
}
https://play.golang.org/p/EKQc7uf1_Vk
Option C, have the Reponse type implement the json.Marshaler interface.
func (r *Response) MarshalJSON() ([]byte, error) {
type tmp Response
resp := (*tmp)(r)
var data struct {
Wrapper struct {
*Person
} `json:"person"`
*tmp
}
data.Wrapper.Person = resp.Person
data.tmp = resp
return json.Marshal(data)
}
https://play.golang.org/p/1qkSCWZ225j
There may be other options...
In Go, an empty struct by definition assigns zero values to field elements. Eg: for int 0, "" for string, etc.
For your case, simply comparing to null would work out. Or, you could define an emptyPerson as:
var BAD_AGE = -1
emptyPerson := &Person{"", BAD_AGE} // BAD_AGE indicates no person
if person[age] == BAD_AGE {
// handle case for emptyPerson}

Unmarshal JSON that could be a string or an object

I have a third party service that is returning JSON where one field contains a collection of data. Here is an example of the structure it returns.
{
"title": "Afghanistan",
"slug": "afghanistan",
"fields": {
"fieldOne": "",
"fieldTwo": {
"new1": {
"type": "contentBlock",,
"fields": {
"richTextBlick": "<p>This is the travel advice.<\/p>"
}
}
},
"fieldThree": {
"new1": {
"type": "introBlock",
"fields": {
"text": "This is a title"
"richText": "<p>This is the current travel summary for Afganistan.<\/p>"
}
},
"new2": {
"type": "contentBlock",
"fields": {
"richText": "<p>It has a second block of content!<\/p>"
}
}
},
"fieldfour": "country"
}
}
Each of the "field" entries could be either a string or another object. I'd like to decode these into something like the structs below.
type EntryVersion struct {
Slug string `json:"slug"`
Fields map[string][]EntryVersionBlock `json:"fields"`
}
type EntryVersionBlock struct {
Type string `json:"type"`
Fields map[string]string `json:"fields"`
}
If the field value is simply a string, I would wrap it in an EntryVersionBlock with the type of "Text" and a single entry in the Fields map.
Any ideas how can I do this in an efficient manner? I may have to do it several hundred times in an extreme edge-case.
Thanks
Your structure is a little bit off from the json's. Fields in EntryVersion is an object not an array so making it a slice is not what you want. I would recommend you change it to this:
type EntryVersion struct {
Slug string `json:"slug"`
Fields map[string]EntryVersionBlock `json:"fields"`
}
The EntryVersionBlock does not have a "type" field in the corresponding json, it has entries with field names "new1", "new2", etc. So I would recommend you add a 3rd type Entry into which you unmarshal those fields.
type Entry struct {
Type string `json:"type"`
Fields map[string]string `json:"fields"`
}
And you would update your EntryVersionBlock to look something like this:
type EntryVersionBlock struct {
Value string `json:"-"`
Fields map[string]Entry `json:"fields"`
}
And to deal with your original problem you can have the EntryVersionBlock type implement the json.Unmarshaler interface, check the first byte in the data passed to the UnmarshalJSON method and if it's a double quote it's a string and if it's a curly opening brace it's an object. Something like this:
func (evb *EntryVersionBlock) UnmarshalJSON(data []byte) error {
switch data[0] {
case '"':
if err := json.Unmarshal(data, &evb.Value); err != nil {
return err
}
case '{':
evb.Fields = make(map[string]Entry)
if err := json.Unmarshal(data, &evb.Fields); err != nil {
return err
}
}
return nil
}
Playground: https://play.golang.org/p/IsTXI5202m
You can use GJson library to unmarshal your JSON to map, then iterate on this map and do the conversion that you need using your structs and based on fields type (map or string).
Update: Example using this method with similar case
http://blog.serverbooter.com/post/parsing-nested-json-in-go/

What would be the Go structs for this terse JSON document?

The document has minimal key/value pair metadata on purpose. In a list of people, we could have something like "name":"joe" or we could have a key of joe. It errs on having fewer keys.
Perhaps this document is just too dynamic for Go structs? I've tried YAML as well, it's the structs that's the problem. The structs remain empty because it's not mapping correctly.
Playground link: https://play.golang.org/p/PGSjoKvNja
Or do I need to roll my own UnmarshalJSON and have conditionals (or switch) for "actions"? That's fine if so. I could do coercion and validation in there, loop through the doc and detect what the problematic generic action bit is and then create a struct of the right type.
If the data is NOT consistent, as #Adrian stated, you should not go for what I'm showing.
Otherwise, you should be able to unmarshal your string with the following struct generated with json-to-go, a very useful tool to get a struct out of a json
type Custom struct {
Ball []struct {
Throw struct {
Strength string `json:"strength"`
} `json:"throw"`
} `json:"ball"`
Frisbee []struct {
Fling struct {
Curve string `json:"curve"`
} `json:"fling"`
Catch struct {
Trick string `json:"trick"`
Jump string `json:"jump"`
} `json:"catch"`
} `json:"frisbee"`
}
And then
func main() {
var c Custom
err := json.Unmarshal([]byte(input), &c )
if err != nil {
panic(err)
}
fmt.Println(input)
}
Which prints out
{
"ball": [{
"throw": {
"strength": "60%"
}
}, {
"throw": {
"strength": "20%"
}
}],
"frisbee": [{
"fling": {
"curve": "left"
}
}, {
"catch": {
"trick": "behind back",
"jump": "sure"
}
}]
}
Have a look at this Playground I set up.
It would be:
type AutoGenerated struct {
Ball []struct {
Throw struct {
Strength string `json:"strength"`
} `json:"throw"`
} `json:"ball"`
Frisbee []struct {
Fling struct {
Curve string `json:"curve"`
} `json:"fling,omitempty"`
Catch struct {
Trick string `json:"trick"`
Jump string `json:"jump"`
} `json:"catch,omitempty"`
} `json:"frisbee"`
}
Ofcourse you can define a separate type for each inline struct definition. You can use this online tool (which is used for generating the above data structure).