How to unmarshall JSON into a value created with reflection? - json

package controllers
import (
"encoding/json"
"errors"
"io"
"io/ioutil"
"reflect"
)
func GetTypeFromReq(c *App, ty interface{}) (interface{}, error) {
//get the type we are going to marshall into
item := reflect.ValueOf(ty)
//define and set the error that we will be returning to null
var retErr error
retErr = nil
//extract the body from the request and defer closing of the body
body, err := ioutil.ReadAll(io.LimitReader(c.Request.Body, 1048576))
defer c.Request.Body.Close()
//handle errors and unmarshal our data
if err != nil {
retErr = errors.New("Failed to Read body: " + err.Error())
} else if err = json.Unmarshal(body, &item); err != nil {
retErr = errors.New("Unmarshal Failed: " + err.Error())
}
return item, retErr
}
I am trying to pass a type and a request into a function, then inside that function unMarshall the request into a variable and return it.
I assume my approach is wrong because when i try to do this:
inter, err := GetTypeFromReq(&c, models.User{})
if err != nil {
revel.ERROR.Println(err.Error())
}
user := inter.(models.User)
I get the error "interface conversion: interface {} is reflect.Value, not models.User"
any tips on how to approach this?

Here's how to modify the the function to make it work as expected:
func GetTypeFromReq(c *App, ty interface{}) (interface{}, error) {
// Allocate new value with same type as ty
v := reflect.New(reflect.TypeOf(ty))
//define and set the error that we will be returning to null
var retErr error
retErr = nil
//extract the body from the request and defer closing of the body
body, err := ioutil.ReadAll(io.LimitReader(c.Request.Body, 1048576))
defer c.Request.Body.Close()
//handle errors and unmarshal our data
if err != nil {
retErr = errors.New("Failed to Read body: " + err.Error())
} else if err = json.Unmarshal(body, v.Interface()); err != nil {
retErr = errors.New("Unmarshal Failed: " + err.Error())
}
// v holds a pointer, call Elem() to get the value.
return v.Elem().Interface(), retErr
}
Note the calls to Interface() to get a reflect.Value's current value.
Here's an approach that avoids reflection and type assertions:
func GetFromReq(c *App, item interface{}) error {
//extract the body from the request and defer closing of the body
body, err := ioutil.ReadAll(io.LimitReader(c.Request.Body, 1048576))
defer c.Request.Body.Close()
//handle errors and unmarshal our data
if err != nil {
retErr = errors.New("Failed to Read body: " + err.Error())
} else if err = json.Unmarshal(body, item); err != nil {
retErr = errors.New("Unmarshal Failed: " + err.Error())
}
return retErr
}
Use it like this:
var user models.User
err := GetFromReq(&c, &user)
if err != nil {
revel.ERROR.Println(err.Error())
}
Use a JSON decoder to simplify the code:
func GetFromReq(c *App, item interface{}) error {
defer c.Request.Body.Close()
return json.NewDecoder(io.LimitReader(c.Request.Body, 1048576)).Deocode(item)
}
If c.Request is a *http.Request and c.Response is an http.ResponseWriter, then write the function as:
func GetFromReq(c *App, item interface{}) error {
return json.NewDecoder(http.MaxBytesReaer(c.Response, c.Request.Body, 1048576)).Deocode(item)
}
There's no need to close the request body in the net/http server. Use MaxBytesReader instead of io.LimitReader to prevents clients from accidentally or maliciously sending a large request and wasting server resources.

Modify code of the last line: change user := inter.(models.User) to user := inter.Interface().(models.User),have a try!

"interface conversion: interface {} is reflect.Value, not models.User"
pretty straight forward about the message error. That your item is reflect.Value it is not models.User.
so I think in your code you can change the item to models.User.
But I assume that your are tying to create the function that will work with all type of your models, in this case models.User{}.
Your approach is expensive since it is using interface. you could convert the incoming request directly like this:
func GetTypeFromReq(c *App, ty models.User) (models.User, error) {
//get the type we are going to marshall into
var item models.User
//define and set the error that we will be returning to nil
var retErr error // this var if the value not define then it is nil. Because error is interface
//extract the body from the request and defer closing of the body
body, err := ioutil.ReadAll(io.LimitReader(c.Request.Body, 1048576))
defer c.Request.Body.Close()
//handle errors and unmarshal our data
if err != nil {
retErr = errors.New("Failed to Read body: " + err.Error())
} else if err = json.Unmarshal(body, &item); err != nil {
retErr = errors.New("Unmarshal Failed: " + err.Error())
}
return item, retErr
}
if your body has the same structure as your model it will give you the value, if not then it is error.
Note that you need to be careful when using interface. you can see some guideline in this article. Use an interface:
When users of the API need to provide an implementation detail.
When API’s have multiple implementations they need to maintain internally.
When parts of the API that can change have been identified and require decoupling.
Your function convert the value of your models.User to interface, and then return the interface value. that's why it's expensive.

Related

How to minimize duplicate code in Go Mux when always trying to return same response structure?

I have tons of code similar to the following code snippet that I just try to fill my response struct, json marshal the output, set status code and return the result:
if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
response := responses.UserResponse{
Status: http.StatusBadRequest,
Message: "error",
Data: map[string]interface{}{"error": err.Error()},
}
rw.WriteHeader(http.StatusBadRequest)
errRes, _ := json.Marshal(response)
rw.Write(errRes)
return
}
I tried to create a function that receives r variable (request.http) to receive the body and also status code of the response. But noticed that I have to again check error code outside of the function and then do the same response creation flow again.
How someone expert in Go tries to minimize code duplications like these? Is this OK to have code duplication like these in first place?
Minimize code duplication by moving the decode call and error handling to a reusable function:
// Decode returns true if the request body is successfully decoded
// to the value pointed to by pv. Otherwise, decode writes an error
// response and returns false.
func decode(rw http.ResponseWriter, r *http.Request, pv interface{}) bool {
err := json.NewDecoder(r.Body).Decode(pv)
if err == nil {
return true
}
rw.WriteHeader(http.StatusBadRequest)
json.NewEncoder(rw).Encode(map[string]any{
"status": http.StatusBadRequest,
"message": "error",
"data": map[string]any{"error": err.Error()},
})
return false
}
Use the function like this:
func userHandler(rw http.ResponseWriter, r *http.Request) {
var u UserRequest
if !decode(rw, r, &u) {
return
}
}
It is preferable to abstract details to provide a high-level picture of what your handler does.
func (h *rideHandler) handleCancelRideByPassenger(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := getUser(ctx)
req := &cancelRequest{}
if err := decode(r, req); err != nil {
h.logger.Error("cancel ride: problem while decoding body request", zap.String("ip", r.RemoteAddr), zap.Error(err))
h.respond.BadRequest(w, NewRESTError(reasonDecoding, "problem while decoding input parameters"))
return
}
req.PublicID = chi.URLParam(r, "id")
err := h.rideService.CancelRide(ctx, req, user)
if err != nil {
var validationErr *ValidationError
switch {
case errors.As(err, &validationErr):
h.respond.BadRequest(w, NewRESTValidationError(reasonValidation, "problem while validating request", validationErr))
return
default:
h.respond.InternalServerError(w, NewRESTError(reasonInternalError, "unknown problem occurred"))
return
}
}
h.respond.Ok(w, NewRESTResponse(&cancelRideResponse{Success: true}))
}
Handler utilizes some handy sugar functions to remove duplication and provide high-level overview of what handler does instead underlying details.
func decode(request *http.Request, val interface{}) error {
dec := json.NewDecoder(request.Body)
dec.DisallowUnknownFields()
return dec.Decode(val)
}
type Responder struct {
Encoder Encoder
Before BeforeFunc
After AfterFunc
OnError OnErrorFunc
}
func (r *Responder) writeResponse(w http.ResponseWriter, v interface{}, status int) {
if r.Before != nil {
status, v = r.Before(w, v, status)
}
encoder := JSON
if r.Encoder != nil {
encoder = r.Encoder
}
w.Header().Set("Content-Type", encoder.ContentType())
w.WriteHeader(status)
if err := encoder.Encode(w, v); err != nil {
if r.OnError != nil {
r.OnError(err)
}
}
if r.After != nil {
r.After(v, status)
}
}
func (r *Responder) Ok(w http.ResponseWriter, v interface{}) {
r.writeResponse(w, v, http.StatusOK)
}
Probably you should write your own respond package or check what is available in open source. Then you can use this respond package with the same response structure everywhere.

Golang json marshal and encoding give weird output

I am trying to customise error message for my db query . Following is what I am doing first I create struct Errormessage . Next if there is error in db.query I do this marshaling then encoding and return. But I end up getting this output "e30=" on my postman testing. What could be wrong I check and followed few examples are showing this mechanism ?
error1 := Errormessage{"Error in select"}
error1_enc,errEn := json.Marshal(error1)
if errEn != nil {
// if error is not nil
// print error
fmt.Println(errEn)
}
json.NewEncoder(w).Encode(error1_enc)
return
/
/ declaring a struct
type Errormessage struct{
// defining struct variables
errormessage string
}
func checkExistUser(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
fmt.Println("File Name :", r.FormValue("email"))
result, err := db.Query("SELECT * from userDetailsss")
if err != nil {
//http.Error(w, err, 500)
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(400)
fmt.Println(err)
error1 := Errormessage{"Error in select"}
error1_enc,errEn := json.Marshal(error1)
if errEn != nil {
// if error is not nil
// print error
fmt.Println(errEn)
}
json.NewEncoder(w).Encode(error1_enc)
return
//panic(err.Error())
}
// This part is how my db is defined and opened
var db *sql.DB
var err error
func main() {
db, err = sql.Open("mysql", "******##tcp(127.0.0.1:3306)/****")
if err != nil {
panic(err.Error())
}
defer db.Close()
router := mux.NewRouter()
router.HandleFunc("/", DoHealthCheck).Methods("POST")
router.HandleFunc("/checkExistUser", checkExistUser).Methods("POST")
log.Fatal(http.ListenAndServe(":8080", router))
}
There are two issues with your code:
You are json encoding the already json encoded error. This means that you are json encoding raw json bytes, which is the reason for the weird output.
Your Errormessage struct's field is unexported. Unexported fields will not be encoded by the encoding/json package.
To fix #1 you can do:
func checkExistUser(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
fmt.Println("File Name :", r.FormValue("email"))
result, err := db.Query("SELECT * from userDetailsss")
if err != nil {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(400)
// use only Encode, no need to call json.Marshal
if err := json.NewEncoder(w).Encode(Errormessage{"Error in select"}); err != nil {
log.Println("failed to send reposnse:", err)
}
return
}
// ...
}
To fix #2 you can do:
type Errormessage struct {
// export the field, i.e. change it to start with an upper case letter
Errormessage string `json:"errormessage"`
}

Getting Elasticsearch response

I am using gin framework to built an Api to get data from Elastic.
Issue is that when I run the application, I get response successfully in the first request but after that in any subsequent request I am getting error:
Error parsing the response body: EOF
Elastic configuration:
var cfg = elasticsearch.Config{
Password: GetConnectConfig().esPassword,
Username: GetConnectConfig().esUserName,
Addresses: GetConnectConfig().esHost,
Logger: &estransport.ColorLogger{
Output: os.Stdout,
}
My request handler function looks like:
func Search() gin.HandlerFunc {
client, err := elasticsearch.NewClient(cfg)
if err != nil {
log.Fatalf("elastic configuration failed %s", err)
}
res, err := client.Search(
client.Search.WithIndex(Index_Name),
client.Search.WithSize(10),
client.Search.WithPretty(),
)
if err != nil {
log.Fatalf("elastic failed to respond %s", err)
}
return func(c *gin.Context) {
r := map[string]interface{}{}
if err := json.NewDecoder(res.Body).Decode(&r); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err})
log.Fatalf("Error parsing the response body: %s", err) // Error
} else {
for _, hit := range r["hits"].(map[string]interface{})["hits"].([]interface{}) {
log.Printf(" * ID=%s, %s", hit.(map[string]interface{})["_id"], hit.(map[string]interface{})["_source"])
}
c.JSON(http.StatusOK, "success")
}
}
}
As I told, I am able to get the response only in the very first request each time I run the application.
I am not sure what is the cause of error here.
I also tried with closing the response body after search by adding:
defer res.Body.Close()
but now I am getting following error:
Error parsing the response body: http2: response body closed
defer res.Body.Close() should be inside the func(c *gin.Context){} yo. can move your search initialization inside the func and do a close inside there something like and client can be injected as a dependency here.
return func(c *gin.Context) {
res, err := client.Search(
client.Search.WithIndex(Index_Name),
client.Search.WithSize(10),
client.Search.WithPretty(),
)
defer res.Body.Close()
}
}

golang json decode with empty request body

In the following http handler, I try to distinguish whether the request body is empty
type Request struct {
A bool `json:"lala"`
B bool `json:"kaka"`
C int32 `json:"cc"`
D int32 `json:"dd"`
}
var (
opts Request
hasOpts bool = true
)
err = json.NewDecoder(r.Body).Decode(&opts)
switch {
case err == io.EOF:
hasOpts = false
case err != nil:
return errors.New("Could not get advanced options: " + err.Error())
}
However, even with r.Body equals '{}', hasOpts is still true. Is this to be expected? In that case, how should I detect empty request body?
Read the body first, to check its content, then unmarshal it:
body, err := ioutil.ReadAll(r.Body)
if err != nil {
return err
}
if len(body) > 0 {
err = json.Unmarshal(body, &opts)
if err != nil {
return fmt.Errorf("Could not get advanced options: %s", err)
}
}

How to get JSON object by calling a url in Go Language?

I'm starting to learn Golang and I would like to know how to get a json response by calling an url, if you could give me an example it would be great in order to guide myself.
Here's a simple example to get you started. Instead of a map[string]interface{} you should consider making a struct to hold the result of your request.
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
func main() {
resp, err := http.Get("http://api.geonames.org/citiesJSON?north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de&username=demo")
if err != nil {
log.Fatal(err)
}
var generic map[string]interface{}
err = json.NewDecoder(resp.Body).Decode(&generic)
if err != nil {
log.Fatal(err)
}
fmt.Println(generic)
}
I'd write a little helper function to do it:
// getJSON fetches the contents of the given URL
// and decodes it as JSON into the given result,
// which should be a pointer to the expected data.
func getJSON(url string, result interface{}) error {
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("cannot fetch URL %q: %v", url, err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected http GET status: %s", resp.Status)
}
// We could check the resulting content type
// here if desired.
err := json.NewDecoder(resp.Body).Decode(result)
if err != nil {
return fmt.Errorf("cannot decode JSON: %v", err)
}
return nil
}
A full working example can be found here: http://play.golang.org/p/b1WJb7MbQV
Note that it is important to check the status code as well as the Get error, and the response body must be closed explicitly (see the documentation here: http://golang.org/pkg/net/http/#Get)