Please take a look at the following structure and tell me why I can't unmarshall it.
type Server struct {
Etcd [] struct {
CertCn string `json:"cert_cn"`
} `json:"etcd"`
}
type CertExpiryReport struct {
Data struct {
Servers map[string]*Server
} `json:"data"`
Summary struct {
Expired int `json:"expired"`
Ok int `json:"ok"`
Total int `json:"total"`
Warning int `json:"warning"`
} `json:"summary"`
}
The following is the JSON content.
{
"data": {
"myserver1.mydomain1.org": {
"etcd": [
{
"cert_cn": "CN:something"
}
]
}
"myserver2.mydomain2.org": {
"etcd": [
{
"cert_cn": "CN:something"
}
]
}
},
"summary": {
"expired": 0,
"ok": 31,
"total": 31,
"warning": 0
}
}
This is my code.
func printStuff() {
bytes, err := ioutil.ReadFile(jsonFile)
if err != nil {
log.Errorf("%s", err.Error())
os.Exit(1)
}
var certExpiryReport CertExpiryReport
err = json.Unmarshal(bytes, &certExpiryReport)
if err != nil {
log.Errorf("%s", err.Error())
os.Exit(1)
}
log.Info(certExpiryReport)
}
The output is the following. I am not getting any errors.
{{map[]} {0 31 31 0}}
Why can't GO parse the JSON? Is something wrong with my structs?
There is something wrong with the struct (or JSON). CertExpiryReport struct has additional level of nesting in Data field. Try to replace
Data struct {
Servers map[string]*Server
} `json:"data"`
With
Data map[string]*Server `json:"data"`
Also, your JSON gives me error (you forgot comma after first server description). Here is working test with change to your structs and JSON: https://play.golang.org/p/QwnHGc9MElb
Other way would be to put inside JSON "data" field "servers" field, and store content of data there. If you need more nesting.
Related
I have a json file sample.json containing a json array as follows:
[
{
"time": "2021-01-04T00:11:32.362Z",
"extra_data": {
"id": "123"
},
"info": "event123"
},
{
"time": "2021-01-05T00:11:32.362Z",
"extra_data": {
"id": "456"
},
"info": "event456"
},
{
"time": "2021-01-06T00:11:32.362Z",
"extra_data": {
"id": "789"
},
"info": "event789"
}
]
I am trying to unmarshal this json array so that for each json object (i.e. event), I can access the id and info values. This is what I have so far:
func main() {
file, err := ioutil.ReadFile("/Users/janedoe/Downloads/sample.json")
var events Event
json.Unmarshal([]byte(file), &events)
fmt.Println(reflect.TypeOf(events))
// Reading each value by its key for each event
fmt.Println("Event123_Time :", events.Timestamp,
"\nEvent123_ExtraData_Id :", events.ExtraData.Id,
"\nEvent123_Info :", events.Info)
}
type Event struct {
Time string `json:"time"`
ExtraData ExtraData `json:"extra_data"`
Info string `json:"info"`
}
type ExtraData struct {
Id string `json:"id"`
}
The output that I am getting is:
main.Event
Event123_Time :
Event123_ExtraData_Id :
Event123_Info :
I am not getting any values, implying that the marshalling is not happening as expected. How can I fix this?
Since your json is an array, I think you want:
var events []Event
not
var events Event
Here's a version with that change:
func main() {
file, err := ioutil.ReadFile("/Users/janedoe/Downloads/sample.json")
if err != nil {
panic(err)
}
var events []Event
if err := json.Unmarshal([]byte(file), &events); err != nil {
panic(err)
}
fmt.Println(reflect.TypeOf(events))
for i, event := range events {
// Reading each value by its key for each event
fmt.Println(i, "Event123_Time :", event.Time,
"\nEvent123_ExtraData_Id :", event.ExtraData.Id,
"\nEvent123_Info :", event.Info)
}
}
type Event struct {
Time string `json:"time"`
ExtraData ExtraData `json:"extra_data"`
Info string `json:"info"`
}
type ExtraData struct {
Id string `json:"id"`
}
I have a json stream as follows ...
[
{
"page": 1,
"pages": 7,
"per_page": "2000",
"total": 13200
},
[
{
"indicator": {
"id": "SP.POP.TOTL",
"value": "Population, total"
},
"country": {
"id": "1A",
"value": "Arab World"
},
"value": null,
"decimal": "0",
"date": "2019"
},
{
"indicator": {
"id": "SP.POP.TOTL",
"value": "Population, total"
},
"country": {
"id": "1A",
"value": "Arab World"
},
"value": "419790588",
"decimal": "0",
"date": "2018"
},
...
]
]
And I'm trying to decode it ... so I have the following struct ... but I keep getting
"cannot unmarshal array into Go value of type struct { P struct ... "
type Message []struct {
P struct {
Page int
}
V []struct {
Indicator struct {
Id string
Value string
}
Country struct {
Value string
}
Value string
Decimal string
Date string
}
}
My struct looks to match the json ... but obviously not! Any ideas?
Since your JSON array have two different types first unmarshal them into a slice of json.RawMessage which is []byte as underlying type so that we can unmarshal again JSON array data.
So unmarshal data for P and V struct type using index directly (predict) or detect if object(starting with '{') then unmarshal into P and array(starting with '[') then unmarshal into V. Now prepare your Message using those data.
type Message struct {
PageData P
ValData []V
}
type P struct {
Page int
}
type V struct {
Indicator struct {
Id string
Value string
}
Country struct {
Value string
}
Value string
Decimal string
Date string
}
func main() {
var rawdata []json.RawMessage
json.Unmarshal([]byte(jsonData), &rawdata)
var pageData P
json.Unmarshal(rawdata[0], &pageData)
var valData []V
json.Unmarshal(rawdata[1], &valData)
res := Message{pageData, valData}
fmt.Println(res)
}
var jsonData = `[...]` //your json data
Full code in Go Playground
As poWar said, the JSON you actually have is a list of objects whose types do not conform to each other. You must therefore unmarshal into something capable of holding different object types, such as interface{} or—since there is an outer array—[]interface{}.
You can also, if you like, decode into a []json.RawMessage. The underlying json.RawMessage itself has underlying type []byte so that it's basically the undecoded "inner" JSON. In at least some cases this is going to be more work than just decoding directly to []interface{} and checking each resulting interface, but you can, if you wish, decode to struct once you have the JSON separated out. For instance:
func main() {
var x []json.RawMessage
err := json.Unmarshal(input, &x)
if err != nil {
fmt.Printf("err = %v\n", err)
return
}
if len(x) != 2 {
fmt.Println("unexpected input")
return
}
var page struct {
Page int
}
err = json.Unmarshal(x[0], &page)
if err != nil {
fmt.Printf("unable to unmarshal page part: %v\n", err)
return
}
fmt.Printf("page = %d\n", page.Page)
// ...
}
Here on the Go Playground is a more complete example. See also Eklavya's answer.
Looking at your struct, your corresponding JSON should look something like this.
[
{
"P": {"page": 1},
"V": [
{
"Indicator": {"Id": ...},
"Country": {"Value":""},
"Value": "",
...
}
]
},
...
]
The JSON structure you are trying to Unmarshal looks like a list of objects where each object is not of the same type. You can start unmarshalling them into interfaces and defining each interface based on the object being unmarhsalled.
package main
import (
"encoding/json"
"log"
)
type Message []interface{}
func main() {
data := `[{"page":1,"pages":7,"per_page":"2000","total":13200},[{"indicator":{"id":"SP.POP.TOTL","value":"Population, total"},"country":{"id":"1A","value":"Arab World"},"value":null,"decimal":"0","date":"2019"},{"indicator":{"id":"SP.POP.TOTL","value":"Population, total"},"country":{"id":"1A","value":"Arab World"},"value":"419790588","decimal":"0","date":"2018"}]]`
var m Message
if err := json.Unmarshal([]byte(data), &m); err != nil {
log.Fatalf("could not unmarshal")
}
log.Printf("message: %v", m)
}
Output:
message: [map[page:1 pages:7 per_page:2000 total:13200] [map[country:map[id:1A value:Arab World] date:2019 decimal:0 indicator:map[id:SP.POP.TOTL value:Population, total] value:<nil>] map[country:map[id:1A value:Arab World] date:2018 decimal:0 indicator:map[id:SP.POP.TOTL value:Population, total] value:419790588]]]
[Edit]: Ideally you should change your JSON to be structured better for unmarshalling. If you do not have control on it, then your corresponding Go structure is just embedded maps of string to interfaces, which you will have to manually type cast and access.
I'm trying to demarshal the JSON requests of Google Actions. These have arrays of tagged unions like this:
{
"requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
"inputs": [{
"intent": "action.devices.QUERY",
"payload": {
"devices": [{
"id": "123",
"customData": {
"fooValue": 74,
"barValue": true,
"bazValue": "foo"
}
}, {
"id": "456",
"customData": {
"fooValue": 12,
"barValue": false,
"bazValue": "bar"
}
}]
}
}]
}
{
"requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
"inputs": [{
"intent": "action.devices.EXECUTE",
"payload": {
"commands": [{
"devices": [{
"id": "123",
"customData": {
"fooValue": 74,
"barValue": true,
"bazValue": "sheepdip"
}
}, {
"id": "456",
"customData": {
"fooValue": 36,
"barValue": false,
"bazValue": "moarsheep"
}
}],
"execution": [{
"command": "action.devices.commands.OnOff",
"params": {
"on": true
}
}]
}]
}
}]
}
etc.
Obviously I can demarshal this to an interface{} and use fully dynamic type casts and everything to decode it, but Go has decent support for decoding to structs. Is there a way to do this elegantly in Go (like you can in Rust for example)?
I feel like you could almost do it by reading demarshalling to this initially:
type Request struct {
RequestId string
Inputs []struct {
Intent string
Payload interface{}
}
}
However once you have the Payload interface{} there doesn't seem to be any way to deserialise that into a struct (other than serialising it and deserialising it again which sucks. Is there any good solution?
Instead of unmarshaling Payload to an interface{} you can store it as a json.RawMessage and then unmarshal it based on the value of Intent. This is shown in the example in the json docs:
https://golang.org/pkg/encoding/json/#example_RawMessage_unmarshal
Using that example with your JSON and struct your code becomes something like this:
type Request struct {
RequestId string
Inputs []struct {
Intent string
Payload json.RawMessage
}
}
var request Request
err := json.Unmarshal(j, &request)
if err != nil {
log.Fatalln("error:", err)
}
for _, input := range request.Inputs {
var payload interface{}
switch input.Intent {
case "action.devices.EXECUTE":
payload = new(Execute)
case "action.devices.QUERY":
payload = new(Query)
}
err := json.Unmarshal(input.Payload, payload)
if err != nil {
log.Fatalln("error:", err)
}
// Do stuff with payload
}
I made https://github.com/byrnedo/pjson for exactly this reason.
So in your case you'd have:
type Input interface {
// Variant func is required
Variant() string
}
type ExecuteInput struct {
Payload struct {
// is any just to avoid typing it for this example
Commands []any `json:"commands"`
} `json:"payload"`
}
func (q ExecuteInput) Variant() string {
return "action.devices.EXECUTE"
}
type QueryInput struct {
Payload struct {
// is any just to avoid typing it for this example
Devices []any `json:"devices"`
} `json:"payload"`
}
func (q QueryInput) Variant() string {
return "action.devices.QUERY"
}
type Inputs []Input
func (i Inputs) MarshalJSON() ([]byte, error) {
return pjson.New(Inputs{}, pjson.WithVariantField("intent")).MarshalArray(i)
}
func (i *Inputs) UnmarshalJSON(bytes []byte) (err error) {
*i, err = pjson.New(Inputs{ExecuteInput{}, QueryInput{}}, pjson.WithVariantField("intent")).UnmarshalArray(bytes)
return
}
type Request struct {
RequestId string `json:"requestId"`
Inputs Inputs `json:"inputs"`
}
Try it on go playground
So I am trying to fetch the analytics of an app by pinging and endpoint. I make the GET request which is successfull (no errors there) but I am unable to decode the JSON
I need to to decode the following json into structs
{
"noResultSearches": {
"results": [
{
"count": 1,
"key": "\"note 9\""
},
{
"count": 1,
"key": "nokia"
}
]
},
"popularSearches": {
"results": [
{
"count": 4,
"key": "6"
},
{
"count": 2,
"key": "\"note 9\""
},
{
"count": 1,
"key": "nokia"
}
]
},
"searchVolume": {
"results": [
{
"count": 7,
"key": 1537401600000,
"key_as_string": "2018/09/20 00:00:00"
}
]
}
}
For which I am using the following structs
type analyticsResults struct {
Count int `json:"count"`
Key string `json:"key"`
}
type analyticsVolumeResults struct {
Count int `json:"count"`
Key int64 `json:"key"`
DateAsStr string `json:"key_as_string"`
}
type analyticsPopularSearches struct {
Results []analyticsResults `json:"results"`
}
type analyticsNoResultSearches struct {
Results []analyticsResults `json:"results"`
}
type analyticsSearchVolume struct {
Results []analyticsVolumeResults `json:"results"`
}
type overviewAnalyticsBody struct {
NoResultSearches analyticsNoResultSearches `json:"noResultSearches"`
PopularSearches analyticsPopularSearches `json:"popularSearches"`
SearchVolume analyticsSearchVolume `json:"searchVolume"`
}
I make a GET request to an endpoint and then use the response body to decode the json but I get an error. Following is a part of the code that stays in my ShowAnalytics function
func ShowAppAnalytics(app string) error {
spinner.StartText("Fetching app analytics")
defer spinner.Stop()
fmt.Println()
req, err := http.NewRequest("GET", "<some-endpoint>", nil)
if err != nil {
return err
}
resp, err := session.SendRequest(req)
if err != nil {
return err
}
spinner.Stop()
var res overviewAnalyticsBody
dec := json.NewDecoder(resp.Body)
err = dec.Decode(&res)
if err != nil {
return err
}
fmt.Println(res)
return nil
}
json: cannot unmarshal array into Go struct field
overviewAnalyticsBody.noResultSearches of type
app.analyticsNoResultSearches
What am I doing wrong here? Why do I get this error?
EDIT: After you edited, your current code works as-is. Check it out here: Go Playground.
Original answer follows.
There is some inconsistency between the code you posted and the error you get.
I tried it on the Go Playground (here's your version), and I get the following error:
json: cannot unmarshal number into Go struct field analyticsVolumeResults.key of type string
We get this error because in the JSON searchVolume.results.key is a number:
"key": 1537401600000,
And you used string in the Go model:
Key string `json:"key"`
If we change it to int64:
Key int64 `json:"key"`
It works, and prints (try it on the Go Playground):
{{[{1 "note 9"} {1 nokia}]} {[{4 6} {2 "note 9"} {1 nokia}]} {[{7 1537401600000 2018/09/20 00:00:00}]}}
If that key may sometimes be a number and sometimes a string, you may also use json.Number in the Go model:
Key json.Number `json:"key"`
I'm trying to decode some json in Go but some fields don't get decoded.
See the code running in browser here:
What am I doing wrong?
I need only the MX records so I didn't define the other fields. As I understand from the godoc you don't need to define the fields you don't use/need.
// You can edit this code!
// Click here and start typing.
package main
import "fmt"
import "encoding/json"
func main() {
body := `
{"response": {
"status": "SUCCESS",
"data": {
"mxRecords": [
{
"value": "us2.mx3.mailhostbox.com.",
"ttl": 1,
"priority": 100,
"hostName": "#"
},
{
"value": "us2.mx1.mailhostbox.com.",
"ttl": 1,
"priority": 100,
"hostName": "#"
},
{
"value": "us2.mx2.mailhostbox.com.",
"ttl": 1,
"priority": 100,
"hostName": "#"
}
],
"cnameRecords": [
{
"aliasHost": "pop.a.co.uk.",
"canonicalHost": "us2.pop.mailhostbox.com."
},
{
"aliasHost": "webmail.a.co.uk.",
"canonicalHost": "us2.webmail.mailhostbox.com."
},
{
"aliasHost": "smtp.a.co.uk.",
"canonicalHost": "us2.smtp.mailhostbox.com."
},
{
"aliasHost": "imap.a.co.uk.",
"canonicalHost": "us2.imap.mailhostbox.com."
}
],
"dkimTxtRecord": {
"domainname": "20a19._domainkey.a.co.uk",
"value": "\"v=DKIM1; g=*; k=rsa; p=DkfbhO8Oyy0E1WyUWwIDAQAB\"",
"ttl": 1
},
"spfTxtRecord": {
"domainname": "a.co.uk",
"value": "\"v=spf1 redirect=_spf.mailhostbox.com\"",
"ttl": 1
},
"loginUrl": "us2.cp.mailhostbox.com"
}
}}`
type MxRecords struct {
value string
ttl int
priority int
hostName string
}
type Data struct {
mxRecords []MxRecords
}
type Response struct {
Status string `json:"status"`
Data Data `json:"data"`
}
type apiR struct {
Response Response
}
var r apiR
err := json.Unmarshal([]byte(body), &r)
if err != nil {
fmt.Printf("err was %v", err)
}
fmt.Printf("decoded is %v", r)
}
As per the go documentaiton about json.Unmarshal, you can only decode toward exported fields, the main reason being that external packages (such as encoding/json) cannot acces unexported fields.
If your json doesn't follow the go convention for names, you can use the json tag in your fields to change the matching between json key and struct field.
Exemple:
package main
import (
"fmt"
"encoding/json"
)
type T struct {
Foo string `json:"foo"`
priv string `json:"priv"`
}
func main() {
text := []byte(`{"foo":"bar", "priv":"nothing"}`)
var t T
err := json.Unmarshal(text, &t)
if err != nil {
panic(err)
}
fmt.Println(t.Foo) // prints "bar"
fmt.Println(t.priv) // prints "", priv is not exported
}
You must Uppercase struct fields:
type MxRecords struct {
Value string `json:"value"`
Ttl int `json:"ttl"`
Priority int `json:"priority"`
HostName string `json:"hostName"`
}
type Data struct {
MxRecords []MxRecords `json:"mxRecords"`
}
http://play.golang.org/p/EEyiISdoaE
The encoding/json package can only decode into exported struct fields. Your Data.mxRecords member is not exported, so it is ignored when decoding. If you rename it to use a capital letter, the JSON package will notice it.
You will need to do the same thing for all the members of your MxRecords type.