I'm making a JSON request to a 3rd party API. If my auth token has expired, I receive an {"error": "message"}. If everything checks out, I receive a valid response.
Right now, I call json.Unmarshal twice to parse the response: once to check for the error and once to parse the real data.
Is there any way to avoid calling Unmarshal twice?
type Unknown map[string]interface{}
type Response struct {
Status string `json:"status"`
Strategy string `json:"strategy"`
Items []Item `json:"items"`
}
unknown := Unknown{}
json.Unmarshal(jsonData, &unknown)
if val, ok := unknown["error"]; ok {
fmt.Println(val)
return
}
response := Response{}
err := json.Unmarshal(jsonData, &response)
if err != nil {
fmt.Println("There was an error")
fmt.Println(err)
return
}
You can use embedding to decode everything at once:
type Response struct {
Status string `json:"status"`
Strategy string `json:"strategy"`
Items []Item `json:"items"`
}
var dest struct {
Error string `json:"error"`
Response // embedded Response
}
if err := json.Unmarshal(jsonData, &dest); err != nil {
fmt.Println(err)
return
} else if len(dest.Error) > 0 {
fmt.Println(dest.Error)
return
}
response := dest.Response
// ...
See an example on playground: https://play.golang.com/p/eAhFt99n07k
Related
func MakeMap(w http.ResponseWriter, r *http.Request) {
// userInfo := context.Get(r, "userInfo").(model.User)
type _getData struct {
Title string `json:"title"`
Tag []string `json:"tag"`
}
var getData _getData
err := json.NewDecoder(r.Body).Decode(&getData)
if err != nil {
panic(err.Error())
}
fmt.Print(getData)
}
When I run the above code, I get the following error
2021/08/24 13:56:54 http: panic serving 127.0.0.1:50619: runtime error: invalid memory address or nil pointer dereference
goroutine 23 [running]:
net/http.(*conn).serve.func1(0x140001e9180)
/usr/local/go/src/net/http/server.go:1824 +0x108
panic(0x10505b860, 0x10522f240)
/usr/local/go/src/runtime/panic.go:971 +0x3f4
traveling/controller/mapController.MakeMap(0x1050b5630, 0x140001f40e0, 0x1400018aa00)
/Users/choeyunseog/traveling/traveling/controller/mapController/mapController.go:20 +0x3c
I've just started studying, I'm not sure why I'm having this problem, please help
err := json.NewDecoder(r.Body).Decode(&getData)
I get the following error when i change code line 20 like above
2021/08/24 14:16:44 http: panic serving 127.0.0.1:51396: invalid character '-' in numeric literal
goroutine 23 [running]:
net/http.(*conn).serve.func1(0x140001e9360)
/usr/local/go/src/net/http/server.go:1824 +0x108
panic(0x100d85d00, 0x14000206070)
/usr/local/go/src/runtime/panic.go:971 +0x3f4
traveling/controller/mapController.MakeMap(0x100df1630, 0x140001f40e0, 0x1400018aa00)
/Users/choeyunseog/traveling/traveling/controller/mapController/mapController.go:24 +0x194
net/http.HandlerFunc.ServeHTTP(0x100de75d8, 0x100df1630, 0x140001f40e0, 0x1400018aa00)
/usr/local/go/src/net/http/server.go:2069 +0x40
To get the multipart form data from a POST/PUT/PATCH request's body you can use the ParseMultipartForm method to parse the body and then access the data through the PostForm field. Or you can use FormValue to get just the first value associated with the form's field.
maxMemory := 32<<20
if err := r.ParseMultipartForm(maxMemory); err != nil {
panic(err)
}
fmt.Println(_getData{
Title: r.FormValue("title"), // FormValue returns string
Tag: r.PostForm["tag[]"], // PostForm is a map of []string
})
You can use to parse form data into json like annotated struct using package github.com/senpathi/paramex. Struct fields must be annotated with param keyword and tag name is key of the form data.
your struct should be as below.
type _getData struct {
Title string `param:"title"`
Tag []string `param:"tag[]"`
}
This is the updated MakeMap handler function for your postman request mentioned in the question
func MakeMap(w http.ResponseWriter, r *http.Request) {
// userInfo := context.Get(r, "userInfo").(model.User)
type _getData struct {
Title string `param:"title"`
Tag []string `param:"tag[]"`
}
// this needed because u send data from Postman as multipart/form-data
maxMemory := 32<<20
if err := r.ParseMultipartForm(int64(maxMemory)); err != nil {
panic(err)
}
var getData _getData
extractor := paramex.NewParamExtractor()
err := extractor.ExtractForms(&getData, r)
if err != nil {
panic(err.Error())
}
fmt.Print(getData)
//Output: {defaultMap [travelling travelling2]}
}
I am trying to process a song using API.I have tried using a specific URL in http.get and further unmarshalling the data but the only element returned in the console is {}. Any help to send me in the right direction is appreciated.
Edit: here is some code. I have this in my main file.
var data [2]Data
if err != nil {
log.Fatal(err)
}
defer response.Body.Close()
if response.StatusCode != 200 {
log.Fatal("Didn't get 200")
}
rawData, err := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
json.Unmarshal(rawData, &data)
fmt.Println(data[0])
I have a struct created in a separate file for JSON tags/keys.
type Data struct {
SongID string `json:id`
Name string `json:name`
}
type data struct {
Error bool `json:"error"`
Response struct {
Results []struct {
ID int `json:"id"`
Name string `json:"name"`
} `json:"results"`
} `json:"response"`
}
This should be the data structure.
Use curl to request
curl https://searchly.asuarez.dev/api/v1/song/search?query=hello
use https://mholt.github.io/json-to-go/ to convert the JSON response to Golang struct.
I read lots of similar questions but not able to find the right solution yet. Hope someone can give me better idea.
My error response could be one of these format:
var errProxy = `{"errors":{"id": "1", "message": "failed to resolve the ip", "status": "failed"}}`
var errServer = `{"errors": "failed to ping the dns server."}`
I am trying to solve this by using two structs and trying to unmarshall one after another until err is nil and return:
// ErrorServer and ErrorProxy is my two struct to represent my above two error response.
type ErrorServer struct {
Error string `json:"errors"`
}
type ErrorResponse struct {
ID string `json:"id"`
Message string `json:"message"`
Status string `json:"status"`
}
type ErrorProxy struct {
ErrorResponse
}
This is the function I used to parse it:
func parseErrorResponse(body io.ReadCloser) (string, error) {
var errServer ErrorServer
var errProxy ErrorProxy
err := json.NewDecoder(body).Decode(&errServer)
if err != nil {
err = json.NewDecoder(body).Decode(&errProxy)
if err != nil {
return "", err
}
return errProxy.Error, nil
}
return errServer.Errors, nil
}
I think a custom unmarshal function is good for this case
type ErrorServer struct {
Error string
}
type ErrorResponse struct {
ID string
Message string
Status string
}
type ErrorProxy struct {
Err ErrorResponse
}
func parseErrorResponse(body io.Reader) (interface{}, error) {
data := make(map[string]interface{})
if err := json.NewDecoder(body).Decode(&data); err != nil {
return nil, err
}
// Check type of errors field
switch errors := data["errors"].(type) {
case string:
// Unmarshal for ErrorServer
errServer := &ErrorServer{
Error: errors,
}
return errServer, nil
case map[string]interface{}:
// Unmarshal for ErrorProxy
errProxy := &ErrorProxy{
Err: ErrorResponse{
ID: errors["id"].(string),
Message: errors["message"].(string),
Status: errors["status"].(string),
},
}
return errProxy, nil
default:
return nil, fmt.Errorf(`failed to parse "errors" field`)
}
}
func main() {
body := bytes.NewReader([]byte(`{"errors": "failed to ping the dns server."}`))
//body := bytes.NewReader([]byte(`{"errors":{"id": "1", "message": "failed to resolve the ip", "status": "failed"}}`))
parsedErr, _ := parseErrorResponse(body)
switch err := parsedErr.(type) {
case *ErrorServer:
fmt.Printf("err server: %+v \n", err)
case *ErrorProxy:
fmt.Printf("err response: %+v \n", err)
}
}
Another way to do that is using type assertion. You can unmarshal that json string into map[string]interface{} and check whether that value interface{} is a string or map[string]interface{}. Depends on its type, you know which kind of error it is and construct a struct from that.
Funny enough, I just gave the answer to this problem here: Use a customized UnmarshalJSON function. How do you modify this struct in Golang to accept two different results?
Applied to your situation:
package main
import (
"encoding/json"
"fmt"
"log"
)
type Answer struct {
Errors Error `json:"errors"`
}
type ErrorResponse struct {
ID string `json:"id"`
Message string `json:"message"`
Status string `json:"status"`
}
type Error struct {
Error string
ErrorResponse ErrorResponse
}
func (s *Error) UnmarshalJSON(b []byte) error {
if len(b) == 0 {
// no data, nothing to do
return nil
}
if b[0] == '{' {
// is object
return json.Unmarshal(b, &s.ErrorResponse)
}
return json.Unmarshal(b, &s.Error)
}
func main() {
var errProxy = []byte(`{"errors":{"id": "1", "message": "failed to resolve the ip", "status": "failed"}}`)
var errServer = []byte(`{"errors": "failed to ping the dns server."}`)
var answ Answer
if err := json.Unmarshal(errProxy, &answ); err != nil {
log.Fatal(err)
}
fmt.Println(answ)
answ = Answer{}
if err := json.Unmarshal(errServer, &answ); err != nil {
log.Fatal(err)
}
fmt.Println(answ)
}
Go PlayGround
Key in the above example is the type that contains both variations. We can also simplify that as both errors could be contained within the ErrorResponse type:
type Answer struct {
Errors ErrorResponse `json:"errors"`
}
type ErrorResponse struct {
ID string
Message string
Status string
}
func (s *ErrorResponse) UnmarshalJSON(b []byte) error {
if len(b) == 0 {
// no data, nothing to do
return nil
}
if b[0] == '{' {
// is object
var tmp struct {
ID string `json:"id"`
Message string `json:"message"`
Status string `json:"status"`
}
if err := json.Unmarshal(b, &tmp); err != nil {
return err
}
s.ID = tmp.ID
s.Message = tmp.Message
s.Status = tmp.Status
return nil
}
return json.Unmarshal(b, &s.Message)
}
The Error struct is not needed here.
some API endpoint may return successful result or error like this:
// ok
{
"status": "ok",
"payload": {
"id": 10,
"title": "Sample"
},
"request_id": "lx-VHr4OLm"
}
// error
{
"status": "error",
"payload": {
"message": "internal error"
},
"trace_id": "lx-VHr4OLm"
}
I'm trying to find elegant way to parse with Go, something like this
.... some code
if status == "ok" {
struct := AppStruct{} // AppStruct contains 2 fields: id and title
_ := json.Unmarshall(payloadBody, &struct)
return struct
} else {
errorStruct := ErrorStruct{} // contains 1 field for message.
_ := json.Unmarshall(payloadBody, &errorStruct)
return nil, errors.New(errorStruct.Message)
}
My current code not works for success payload:
var result map[string]interface{}
jsonErr := json.Unmarshal(body, &result)
if jsonErr != nil {
return nil, jsonErr
}
if result["status"] == "error" {
errorPayload := result["payload"].(map[string]string)
return nil, errors.New(errorPayload["message"])
} else if result["status"] == "ok" {
apiResponse := AppInfo{}
jsonErr := json.Unmarshal([]byte(result["payload"].(string)), &apiResponse)
if jsonErr != nil {
return nil, jsonErr
}
return &apiResponse, nil
}
And I got runtime error on line json.Unmarshal([]byte(result["payload"].(string)), &apiResponse)
http: panic serving [::1]:51091: interface conversion: interface {} is
map[string]interface {}, not string
Surely, I may have 2 structs: for success response and for error one but I assume it's too complicated way to solve my problem.
How to parse this JSON in elegant way?
I'm really not sure what the problem is. The standard encoding/json doesn't require the struct to match all fields in the JSON data. It's quite easy to handle this with a single, simple type:
type Payload struct {
ID int `json:"id"`
Title string `json:"title"`
Message string `json:"message"`
}
type Response struct {
Status string `json:"status"`
ID string `json:"request_id"`
TraceID string `json:"trace_id"`
Payload Payload `json:"payload"`
}
Then just unmarshal the response in the Response struct:
var resp Response
if err := json.Unmarshal(body, &resp); err != nil {
return err
}
Then you can simply check the Status field, and work out what to do next. For example:
if resp.Status == "error" {
return fmt.Errorf("invalid response: %s - %s", resp.TraceID, resp.Payload.Message)
}
// handle resp.Payload.ID and resp.Payload.Title fields
return nil
You could move those checks for the status field to receiver functions on the response object, depending on the complexity and your specific needs.
Perhaps it's worth using pointer fields for those fields that aren't set in the normal response, and tag them with the omitempty option:
type Payload struct {
ID int `json:"id"`
Title string `json:"title"`
Message *string `json:"message,omitempty"`
}
type Response struct {
Status string `json:"status"`
ID string `json:"request_id"`
TraceID *string `json:"trace_id,omitempty"`
Payload Payload `json:"payload"`
}
With types like this, no longer have to rely on hard-coded string constants to check for errors. Instead, you can easily implement a more generic check like so:
func (r Response) IsError() bool {
return (r.TraceID == nil) // will be set in case of an error response
}
Update
As you pointed out in the comments, the response body is actually substantially larger than the 2 fields in the example. Of course, copy-pasting the struct definitions, or writing mapping functions to map the Payload onto the type you already have is a bit pointless.
The answer here is: composition.
type Payload struct {
AppStruct // embedded the AppStruct type
Message *string `json:"message"`
}
The Response type stays as it is. If the response is successful, you can get the AppStruct directly from the response like so:
appStruct := resp.Payload.AppStruct
This works because the type is embedded. Note that there aren't any json tags there. The embedded struct, at least as far as the unmarshalling is concerned, is a part of the Payload struct. Therefore, all the exported fields in that type will be unmarshalled directly into the struct.
Thanks https://stackoverflow.com/users/965900/mkopriva for idea to use json.RawMessage
My final solution:
func parsePayload(response []byte, successPayload interface{}) error {
var result map[string]json.RawMessage
jsonErr := json.Unmarshal(response, &result)
if jsonErr != nil {
return jsonErr
}
var status string
jsonErr = json.Unmarshal(result["status"], &status)
if jsonErr != nil {
return jsonErr
}
if status == "ok" {
jsonErr = json.Unmarshal(result["payload"], &successPayload)
if jsonErr != nil {
return jsonErr
}
return nil
} else if status == "error" {
errorPayload := ErrorPayload{}
jsonErr = json.Unmarshal(result["payload"], &errorPayload)
if jsonErr != nil {
return jsonErr
}
return errors.New(errorPayload.Message)
}
log.Printf("Unknown http result status: %s", status)
return errors.New("internal error")
}
type ErrorPayload struct {
Message string `json:"message"`
}
//usage
type AppInfo struct {
Id int `json:"app_id"`
Title string `json:"app_title"`
}
body := ... // read body
appInfo := AppInfo{}
parseErr := parsePayload(body, &appInfo)
if parseErr != nil {
return nil, parseErr
}
log.Printf("Parsed app %v", appInfo)
return &appInfo, nil
I have an end-point that receives data from a Google PubSub request. As per this repo, the object is as so:
type pushRequest struct {
Message struct {
Attributes map[string]string
Data []byte
ID string `json:"message_id"`
}
Subscription string
}
The Data field is consistently formatted as so:
type Data struct {
Key string `json:"key"`
Body string `json:"body"`
Meta map[string]interface{} `json:"meta"`
}
I can obviously unmarshal the JSON request with something like this:
f := &pushRequest{}
json.Unmarshal(msg, &f)
That leaves with the the []bytes field. Which I can do something like this to convert to a string, as per the docs
messages = append(messages, string(f.Message.Data))
Which doesn't help, since I need it as a struct.
I can Unmarshal the array again:
var m Data
json.Unmarshal(f.Message.Data, &m)
Have tried changing the field type in the pushRequest struct to Data without success. Blank...
Is there a way I can unpack things in a single pass? Doing it twice seems ridiculous.
If it's obvious, I just can't see it!
decoder := json.NewDecoder(r.Body)
psmsg := &PushRequest{}
decoderErr := decoder.Decode(&psmsg)
if decoderErr != nil {
// Error...
return
}
data := Data{}
unmarshalErr := json.Unmarshal([]byte(string(psmsg.Message.Data)), &data)
if unmarshalErr != nil {
// Error...
return
}
Here is a snippet from my cloud function, which serves as a pub/sub push endpoint. The key is that you first have to decode the body using the PushRequest struct. Next, you can transform the message data into a struct. According to the documentation, the Data field within Message is a base-64 encoded string, therefore you have to decode it first.
type PushRequest struct {
Message pubsub.PubsubMessage `json:"message"`
Subscription string `json:"subscription"`
}
type Example struct {
ID string `json:"id" firestore:"id"`
}
func HTTPEndpoint(w http.ResponseWriter, r *http.Request) {
var pr common.PushRequest
if err := json.NewDecoder(r.Body).Decode(&pr); err != nil {
log.Fatalf("Could not decode body: %v", err)
return
}
data, err := base64.StdEncoding.DecodeString(pr.Message.Data)
if err != nil {
log.Fatalf("Base64: %v", err)
return
}
var example Example
if err := json.Unmarshal(data, &example); err != nil {
log.Fatalf("Json: %v", err)
return
}
// Do something useful with the struct
}