I make a GET request, and receive a JSON file, that I cannot parse.
Here is the data I have to parse
{
"codeConv": "ACC00000321",
"start": "2019-07-01T00:00:00Z",
"end": "2019-08-21T00:00:00Z",
"details": [
{
"idPrm": "30000000123456",
"keys": [
{
"timestamp": "2019-07-01T00:00:00Z",
"value": 0
},
{
"timestamp": "2019-07-01T00:30:00Z",
"value": 0
},
...
]
},
{
"idPrm": "30000000123457",
"keys": [
{
"timestamp": "2019-07-01T00:00:00Z",
"value": 1
},
{
"timestamp": "2019-07-01T00:30:00Z",
"value": 2
},
...
]
}
]
}
Here are my objects:
type APIMain struct {
CodeConv string `json:"codeConv"`
Start string `json:"start"`
End []Keys `json:"end"`
Details []APIData `json:"details"`
}
//APIData match the data we receive from api
type APIData struct {
Prm string `json:"idPrm"`
Keys []Keys `json:"keys"`
}
type Keys struct {
Timestamp string `json:"timestamp"`
Value string `json:"value"`
}
and here is the method to get data with basic auth:
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
if login != "" && password != "" {
req.SetBasicAuth(login, password)
}
response, err := client.Do(req)
//defer response.Body.Close()
if err != nil {
return nil, err
}
if response.StatusCode != 200 {
fmt.Println(response.Body)
panic(response.Status)
}
err = json.NewDecoder(response.Body).Decode(&result)
fmt.Println("result", result) // result is empty array
How can I see if the problem is in a request, or if the problem is in parsing ?
I have get a response.Body object, but it needs to be decoded.
I fixed it using: https://mholt.github.io/json-to-go/
which generated this structure:
type AutoGenerated struct {
CodeConv string `json:"codeConv"`
Start time.Time `json:"start"`
End time.Time `json:"end"`
Details []struct {
IDPrm string `json:"idPrm"`
Keys []struct {
Timestamp time.Time `json:"timestamp"`
Value float64 `json:"value"`
} `json:"keys"`
} `json:"details"`
}
Thanks for your comments
Great time saver !
Related
I have a json file sample.json containing a json array as follows:
[
{
"time": "2021-01-04T00:11:32.362Z",
"extra_data": {
"id": "123"
},
"info": "event123"
},
{
"time": "2021-01-05T00:11:32.362Z",
"extra_data": {
"id": "456"
},
"info": "event456"
},
{
"time": "2021-01-06T00:11:32.362Z",
"extra_data": {
"id": "789"
},
"info": "event789"
}
]
I am trying to unmarshal this json array so that for each json object (i.e. event), I can access the id and info values. This is what I have so far:
func main() {
file, err := ioutil.ReadFile("/Users/janedoe/Downloads/sample.json")
var events Event
json.Unmarshal([]byte(file), &events)
fmt.Println(reflect.TypeOf(events))
// Reading each value by its key for each event
fmt.Println("Event123_Time :", events.Timestamp,
"\nEvent123_ExtraData_Id :", events.ExtraData.Id,
"\nEvent123_Info :", events.Info)
}
type Event struct {
Time string `json:"time"`
ExtraData ExtraData `json:"extra_data"`
Info string `json:"info"`
}
type ExtraData struct {
Id string `json:"id"`
}
The output that I am getting is:
main.Event
Event123_Time :
Event123_ExtraData_Id :
Event123_Info :
I am not getting any values, implying that the marshalling is not happening as expected. How can I fix this?
Since your json is an array, I think you want:
var events []Event
not
var events Event
Here's a version with that change:
func main() {
file, err := ioutil.ReadFile("/Users/janedoe/Downloads/sample.json")
if err != nil {
panic(err)
}
var events []Event
if err := json.Unmarshal([]byte(file), &events); err != nil {
panic(err)
}
fmt.Println(reflect.TypeOf(events))
for i, event := range events {
// Reading each value by its key for each event
fmt.Println(i, "Event123_Time :", event.Time,
"\nEvent123_ExtraData_Id :", event.ExtraData.Id,
"\nEvent123_Info :", event.Info)
}
}
type Event struct {
Time string `json:"time"`
ExtraData ExtraData `json:"extra_data"`
Info string `json:"info"`
}
type ExtraData struct {
Id string `json:"id"`
}
This question already has answers here:
Unmarshal incorrectly formatted datetime
(2 answers)
Closed 3 years ago.
I'm trying to unmarshal JSON with time value. I have this JSON structure.
{
"Nick": "cub",
"Email": "cub875#rambler.ru",
"Created_at": "2017-10-09",
"Subscribers": [
{
"Email": "rat1011#rambler.ru",
"Created_at": "2017-11-30"
},
{
"Email": "hound939#rambler.ru",
"Created_at": "2016-07-15"
},
{
"Email": "worm542#rambler.ru",
"Created_at": "2017-02-16"
},
{
"Email": "molly1122#rambler.ru",
"Created_at": "2016-11-30"
},
{
"Email": "goat1900#yandex.ru",
"Created_at": "2018-07-10"
},
{
"Email": "duck1146#rambler.ru",
"Created_at": "2017-09-04"
},
{
"Email": "eagle1550#mail.ru",
"Created_at": "2018-01-03"
},
{
"Email": "prawn1755#rambler.ru",
"Created_at": "2018-04-20"
},
{
"Email": "platypus64#yandex.ru",
"Created_at": "2018-02-17"
}
]
}
And a function that implements reading from JSON file to struct User. Everything if fine but when I set CreatedAt field in User struct to time.Time type I get 0001-01-01 00:00:00 +0000 UTC value for each field with type format in JSON file.
type Subscriber struct {
Email string `json: "Email"`
CreatedAt time.Time `json: "Created_at"`
}
type User struct {
Nick string `json: "Nick"`
Email string `json: "Email"`
CreatedAt string `json: "Created_at"`
Subscribers []Subscriber `json: "Subscribers"`
}
func main() {
// Slice where objects from users.json will be pushed
var jsonData []User
// ReadJSON and push User struct to jsonData slice
readJSON("users.json", &jsonData)
for _, user := range jsonData {
fmt.Println(user.Nick, user.Email, user.CreatedAt) // 0001-01-01 00:00:00 +0000 UTC
fmt.Println(user.Subscribers)
}
}
func readJSON(fileName string, userSlice *[]User) {
file, err := os.Open(fileName)
if err != nil {
log.Fatal(err)
}
bytes, err := ioutil.ReadAll(file)
if err != nil {
log.Fatal(err)
}
err1 := json.Unmarshal(bytes, &userSlice)
if err1 != nil {
log.Fatal(err1)
}
}
What is an appropriate way to read time from JSON file to User in RFC3339 format?
You can use a custom time type that implements the json.Unmarshaler interface.
You can start with this struct:
type CustomTime struct {
time.Time // Embed time.Time to allow calling of normal time.Time methods
}
Then add the required UnmarshalJSON([]byte) error function. It could look like this:
func (c *CustomTime) UnmarshalJSON(b []byte) error {
if len(b) < 3 {
// Empty string: ""
return fmt.Errorf("Empty time value")
}
t, err := time.Parse("2006-01-02", string(b[1:len(b)-1])) // b[1:len(b)-1] removes the first and last character, as they are quotes
if err != nil {
return err
}
c.Time = t
return nil
}
You can try the example on the Go Playground
So I am trying to fetch the analytics of an app by pinging and endpoint. I make the GET request which is successfull (no errors there) but I am unable to decode the JSON
I need to to decode the following json into structs
{
"noResultSearches": {
"results": [
{
"count": 1,
"key": "\"note 9\""
},
{
"count": 1,
"key": "nokia"
}
]
},
"popularSearches": {
"results": [
{
"count": 4,
"key": "6"
},
{
"count": 2,
"key": "\"note 9\""
},
{
"count": 1,
"key": "nokia"
}
]
},
"searchVolume": {
"results": [
{
"count": 7,
"key": 1537401600000,
"key_as_string": "2018/09/20 00:00:00"
}
]
}
}
For which I am using the following structs
type analyticsResults struct {
Count int `json:"count"`
Key string `json:"key"`
}
type analyticsVolumeResults struct {
Count int `json:"count"`
Key int64 `json:"key"`
DateAsStr string `json:"key_as_string"`
}
type analyticsPopularSearches struct {
Results []analyticsResults `json:"results"`
}
type analyticsNoResultSearches struct {
Results []analyticsResults `json:"results"`
}
type analyticsSearchVolume struct {
Results []analyticsVolumeResults `json:"results"`
}
type overviewAnalyticsBody struct {
NoResultSearches analyticsNoResultSearches `json:"noResultSearches"`
PopularSearches analyticsPopularSearches `json:"popularSearches"`
SearchVolume analyticsSearchVolume `json:"searchVolume"`
}
I make a GET request to an endpoint and then use the response body to decode the json but I get an error. Following is a part of the code that stays in my ShowAnalytics function
func ShowAppAnalytics(app string) error {
spinner.StartText("Fetching app analytics")
defer spinner.Stop()
fmt.Println()
req, err := http.NewRequest("GET", "<some-endpoint>", nil)
if err != nil {
return err
}
resp, err := session.SendRequest(req)
if err != nil {
return err
}
spinner.Stop()
var res overviewAnalyticsBody
dec := json.NewDecoder(resp.Body)
err = dec.Decode(&res)
if err != nil {
return err
}
fmt.Println(res)
return nil
}
json: cannot unmarshal array into Go struct field
overviewAnalyticsBody.noResultSearches of type
app.analyticsNoResultSearches
What am I doing wrong here? Why do I get this error?
EDIT: After you edited, your current code works as-is. Check it out here: Go Playground.
Original answer follows.
There is some inconsistency between the code you posted and the error you get.
I tried it on the Go Playground (here's your version), and I get the following error:
json: cannot unmarshal number into Go struct field analyticsVolumeResults.key of type string
We get this error because in the JSON searchVolume.results.key is a number:
"key": 1537401600000,
And you used string in the Go model:
Key string `json:"key"`
If we change it to int64:
Key int64 `json:"key"`
It works, and prints (try it on the Go Playground):
{{[{1 "note 9"} {1 nokia}]} {[{4 6} {2 "note 9"} {1 nokia}]} {[{7 1537401600000 2018/09/20 00:00:00}]}}
If that key may sometimes be a number and sometimes a string, you may also use json.Number in the Go model:
Key json.Number `json:"key"`
I have to unmarshal a series of Json objects, but one of the objects contain a json array which is not really structured in a good way.
"labels": [
{
"key": "owner",
"value": "harry"
},
{
"key": "group",
"value": "student"
}
]
I am unmarshalling it using this struct -
type StudentDetails struct {
Id string `json:"id"`
Name string `json:"name"`
Labels []Label `json:"labels,omitempty"`
}
type Label struct {
Key string `json:"key"`
Value string `json:"value"`
}
And I have to access it using x.Labels[0].key == "owner" inside a for loop which is very annoying.
I want to be able to do x.Labels.Owner == "harry" instead. How do I go about achieving this? The rest of JSON is unmarshalled fine using the default unmarshal function, so I don't think writing custom function will be good option.
With the constraints you have here, this is about as close as you will get (run in playground):
package main
import (
"encoding/json"
"fmt"
)
func main() {
j := `
{
"id": "42",
"name": "Marvin",
"labels": [
{
"key": "owner",
"value": "harry"
},
{
"key": "group",
"value": "student"
}
]
}`
d := StudentDetails{}
err := json.Unmarshal([]byte(j), &d)
if err != nil {
panic(err)
}
fmt.Println(d.Labels["owner"])
fmt.Println(d.Labels["group"])
}
type StudentDetails struct {
Id string `json:"id"`
Name string `json:"name"`
Labels Labels `json:"labels"`
}
type Labels map[string]string
func (l *Labels) UnmarshalJSON(b []byte) error {
a := []map[string]string{}
err := json.Unmarshal(b, &a)
if err != nil {
return err
}
t := map[string]string{}
for _, m := range a {
t[m["key"]] = m["value"]
}
*l = t
return nil
}
How about to define custom []Label type and add function on it.
For instance
type Labels []Label
func (l Labels) Owner() string {
if len(l) > 1 {
return l[0].Value
}
return ""
}
I'm using Go for a simple http client. Here's the entity I'm unmarshalling:
type Message struct {
Id int64
Timestamp int64
Text string
Author User
LastEdited int64
}
type User struct {
Id int64
Name string
}
A single entity looks like this in JSON:
{
"text": "hello, can you hear me?",
"timestamp": 1512964818565,
"author": {
"name": "andrea",
"id": 3
},
"lastEdited": null,
"id": 8
}
Go/json has no problem unmarshalling the single entity:
var m Message
err = json.Unmarshal(body, &m)
if err != nil {
printerr(err.Error())
}
println(m.Text)
However, if the return of the endpoint is multiple entities:
[
{
"text": "hello, can you hear me?",
"timestamp": 1512964800981,
"author": {
"name": "eleven",
"id": 4
},
"lastEdited": null,
"id": 7
}
]
And I change my corresponding Unmarshall to work on a slice of structs, Go throws an error:
var m []Message
err = json.Unmarshal(body, &m)
if err != nil {
printerr(err.Error()) // unexpected end of JSON input
}
for i := 0; i < len(m); i++ {
println(m[i].Text)
}
What gives?
Works fine for me (try it on playground), where are you getting the payload data from? sounds like that's truncating it.
package main
import (
"encoding/json"
"fmt"
)
type Message struct {
Id int64
Timestamp int64
Text string
Author User
LastEdited int64
}
type User struct {
Id int64
Name string
}
func main() {
body := []byte(`[
{
"text": "hello, can you hear me?",
"timestamp": 1512964800981,
"author": {
"name": "eleven",
"id": 4
},
"lastEdited": null,
"id": 7
}
]`)
var m []Message
err := json.Unmarshal(body, &m)
if err != nil {
fmt.Printf("error: %v") // unexpected end of JSON input
}
for i := 0; i < len(m); i++ {
fmt.Println(m[i].Text)
}
}
running it gives this output
hello, can you hear me?