func good(json) string {
\\do something
err = json.Unmarshal(body, &list)
if err != nil {
panic(fmt.Sprintf("Unable to parse json %s",err))
}
}
func Testgood_PanicStatement(t *testing.T) {
Convey("And Invalid Json return error",t, func() {
actual := good("garbage json")
So(func() {},shoulPanic)
So(actual ,ShouldEqual,"")
}
}
Outcome
Line 34: - Unable to parse json ,{%!e(string=invalid character '{' looking for beginning of object key string) %!e(int64=50)}
goroutine 8 [running]:
Question:It seems like when I am passing garbage json file.It is panicking and doesn't execute any of the So statements?How to fix it?
Use recover().
func Testgood_PanicStatement(t *testing.T) {
Convey("And Invalid Json return error",t, func() {
defer func() {
if r := recover(); r != nil {
So(func() {},shouldPanic)
So(actual ,ShouldEqual,"")
}
}()
actual := good("garbage json")
}
}
Lear more about:
Golang blog
Upvoting the answer of sadlil as correct I want to point out, that panicking functions are not good practice. Rather convert the panic into an error INSIDE the function and test the error instead.
func good(json) (s string, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("Error in good: %v", r)
}
}()
\\do something
err = json.Unmarshal(body, &list)
if err != nil {
# leaving the panic statement for demonstration. In this case
# you could just: return "", err
panic(fmt.Sprintf("Unable to parse json %s",err))
}
return
}
Related
I have tons of code similar to the following code snippet that I just try to fill my response struct, json marshal the output, set status code and return the result:
if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
response := responses.UserResponse{
Status: http.StatusBadRequest,
Message: "error",
Data: map[string]interface{}{"error": err.Error()},
}
rw.WriteHeader(http.StatusBadRequest)
errRes, _ := json.Marshal(response)
rw.Write(errRes)
return
}
I tried to create a function that receives r variable (request.http) to receive the body and also status code of the response. But noticed that I have to again check error code outside of the function and then do the same response creation flow again.
How someone expert in Go tries to minimize code duplications like these? Is this OK to have code duplication like these in first place?
Minimize code duplication by moving the decode call and error handling to a reusable function:
// Decode returns true if the request body is successfully decoded
// to the value pointed to by pv. Otherwise, decode writes an error
// response and returns false.
func decode(rw http.ResponseWriter, r *http.Request, pv interface{}) bool {
err := json.NewDecoder(r.Body).Decode(pv)
if err == nil {
return true
}
rw.WriteHeader(http.StatusBadRequest)
json.NewEncoder(rw).Encode(map[string]any{
"status": http.StatusBadRequest,
"message": "error",
"data": map[string]any{"error": err.Error()},
})
return false
}
Use the function like this:
func userHandler(rw http.ResponseWriter, r *http.Request) {
var u UserRequest
if !decode(rw, r, &u) {
return
}
}
It is preferable to abstract details to provide a high-level picture of what your handler does.
func (h *rideHandler) handleCancelRideByPassenger(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := getUser(ctx)
req := &cancelRequest{}
if err := decode(r, req); err != nil {
h.logger.Error("cancel ride: problem while decoding body request", zap.String("ip", r.RemoteAddr), zap.Error(err))
h.respond.BadRequest(w, NewRESTError(reasonDecoding, "problem while decoding input parameters"))
return
}
req.PublicID = chi.URLParam(r, "id")
err := h.rideService.CancelRide(ctx, req, user)
if err != nil {
var validationErr *ValidationError
switch {
case errors.As(err, &validationErr):
h.respond.BadRequest(w, NewRESTValidationError(reasonValidation, "problem while validating request", validationErr))
return
default:
h.respond.InternalServerError(w, NewRESTError(reasonInternalError, "unknown problem occurred"))
return
}
}
h.respond.Ok(w, NewRESTResponse(&cancelRideResponse{Success: true}))
}
Handler utilizes some handy sugar functions to remove duplication and provide high-level overview of what handler does instead underlying details.
func decode(request *http.Request, val interface{}) error {
dec := json.NewDecoder(request.Body)
dec.DisallowUnknownFields()
return dec.Decode(val)
}
type Responder struct {
Encoder Encoder
Before BeforeFunc
After AfterFunc
OnError OnErrorFunc
}
func (r *Responder) writeResponse(w http.ResponseWriter, v interface{}, status int) {
if r.Before != nil {
status, v = r.Before(w, v, status)
}
encoder := JSON
if r.Encoder != nil {
encoder = r.Encoder
}
w.Header().Set("Content-Type", encoder.ContentType())
w.WriteHeader(status)
if err := encoder.Encode(w, v); err != nil {
if r.OnError != nil {
r.OnError(err)
}
}
if r.After != nil {
r.After(v, status)
}
}
func (r *Responder) Ok(w http.ResponseWriter, v interface{}) {
r.writeResponse(w, v, http.StatusOK)
}
Probably you should write your own respond package or check what is available in open source. Then you can use this respond package with the same response structure everywhere.
I wrote some code which hits one public API and saves the JSON output in a file. But the data is storing line by line into the file instead of a single JSON format.
For eg.
Current Output:
{"ip":"1.1.1.1", "Country":"US"}
{"ip":"8.8.8.8", "Country":"IN"}
Desired Output:
[
{"ip":"1.1.1.1", "Country":"US"},
{"ip":"8.8.8.8", "Country":"IN"}
]
I know this should be pretty simple and i am missing out something.
My Current Code is:
To read IP from file and hit the API one by one on each IP.
func readIPfromFile(filename string, outFile string, timeout int) {
data := jsonIn{}
//open input file
jsonFile, err := os.Open(filename) //open input file
...
...
jsonData := bufio.NewScanner(jsonFile)
for jsonData.Scan() {
// marshal json data & check for logs
if err := json.Unmarshal(jsonData.Bytes(), &data); err != nil {
log.Fatal(err)
}
//save to file
url := fmt.Sprintf("http://ipinfo.io/%s", data.Host)
GetGeoIP(url, outFile, timeout)
}
}
To make HTTP Request with custom request header and call write to file function.
func GetGeoIP(url string, outFile string, timeout int) {
geoClient := http.Client{
Timeout: time.Second * time.Duration(timeout), // Timeout after 5 seconds
}
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
log.Fatal(err)
}
req.Header.Set("accept", "application/json")
res, getErr := geoClient.Do(req)
if getErr != nil {
log.Fatal(getErr)
}
if res.Body != nil {
defer res.Body.Close()
}
body, readErr := ioutil.ReadAll(res.Body)
if readErr != nil {
log.Fatal(readErr)
}
jsonout := jsonOut{}
jsonErr := json.Unmarshal(body, &jsonout)
if jsonErr != nil {
log.Fatal(jsonErr)
}
file, _ := json.Marshal(jsonout)
write2file(outFile, file)
}
To Write data to file:
func write2file(outFile string, file []byte) {
f, err := os.OpenFile(outFile, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0600)
if err != nil {
log.Fatal(err)
}
defer f.Close()
if _, err = f.WriteString(string(file)); err != nil {
log.Fatal(err)
}
if _, err = f.WriteString("\n"); err != nil {
log.Fatal(err)
}
I know, i can edit f.WriteString("\n"); to f.WriteString(","); to add comma but still adding [] in the file is challenging for me.
First, please do not invent a new way of json marshaling, just use golang built-in encoding/json or other library on github.
Second, if you want to create a json string that represents an array of object, you need to create the array of objects in golang and marshal it into string (or more precisely, into array of bytes)
I create a simple as below, but please DIY if possible.
https://go.dev/play/p/RR_ok-fUTb_4
I am trying to customise error message for my db query . Following is what I am doing first I create struct Errormessage . Next if there is error in db.query I do this marshaling then encoding and return. But I end up getting this output "e30=" on my postman testing. What could be wrong I check and followed few examples are showing this mechanism ?
error1 := Errormessage{"Error in select"}
error1_enc,errEn := json.Marshal(error1)
if errEn != nil {
// if error is not nil
// print error
fmt.Println(errEn)
}
json.NewEncoder(w).Encode(error1_enc)
return
/
/ declaring a struct
type Errormessage struct{
// defining struct variables
errormessage string
}
func checkExistUser(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
fmt.Println("File Name :", r.FormValue("email"))
result, err := db.Query("SELECT * from userDetailsss")
if err != nil {
//http.Error(w, err, 500)
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(400)
fmt.Println(err)
error1 := Errormessage{"Error in select"}
error1_enc,errEn := json.Marshal(error1)
if errEn != nil {
// if error is not nil
// print error
fmt.Println(errEn)
}
json.NewEncoder(w).Encode(error1_enc)
return
//panic(err.Error())
}
// This part is how my db is defined and opened
var db *sql.DB
var err error
func main() {
db, err = sql.Open("mysql", "******##tcp(127.0.0.1:3306)/****")
if err != nil {
panic(err.Error())
}
defer db.Close()
router := mux.NewRouter()
router.HandleFunc("/", DoHealthCheck).Methods("POST")
router.HandleFunc("/checkExistUser", checkExistUser).Methods("POST")
log.Fatal(http.ListenAndServe(":8080", router))
}
There are two issues with your code:
You are json encoding the already json encoded error. This means that you are json encoding raw json bytes, which is the reason for the weird output.
Your Errormessage struct's field is unexported. Unexported fields will not be encoded by the encoding/json package.
To fix #1 you can do:
func checkExistUser(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
fmt.Println("File Name :", r.FormValue("email"))
result, err := db.Query("SELECT * from userDetailsss")
if err != nil {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(400)
// use only Encode, no need to call json.Marshal
if err := json.NewEncoder(w).Encode(Errormessage{"Error in select"}); err != nil {
log.Println("failed to send reposnse:", err)
}
return
}
// ...
}
To fix #2 you can do:
type Errormessage struct {
// export the field, i.e. change it to start with an upper case letter
Errormessage string `json:"errormessage"`
}
I have following code:
func policyDocumentToStr(doc map[string]interface{}) (*string, error) {
policy, err := json.Marshal(doc)
if err != nil {
log.Debugf("Error converting policy document to string. Error %s", err)
return nil, err
}
policyAsString := string(policy)
return &policyAsString, nil
}
I want to write a unit test which would cover the case of json.Marshal(doc) returning an error. Can anybody suggest how can I generate an error? What kind of input to function would result in error at line policy, err := json.Marshal(doc)?
Feed it a value that cannot be represented in JSON. There are many ways to do this, including creating a type with a custom marshaler that always returns an error. But one of the simplest ways is to try to marshal a channel:
x := map[string]interface{}{
"foo": make(chan int),
}
_, err := json.Marshal(x)
fmt.Printf("Marshal error: %s\n", err)
Playground link
Define a type that implements json.Marshaler. Use this type to produce any error you want (including error values from the json package):
type FakeValue struct {
err error
}
func (v FakeValue) MarshalJSON() ([]byte, error) {
if v.err != nil {
return nil, v.err
}
return []byte(`null`), v.err
}
func TestPolicyString(t *testing.T) {
doc := map[string]interface{}{
"fake_error": FakeValue{errors.New("fail!")},
}
_, err := policyDocumentToStr(doc)
if err == nil {
t.Fatal("Got nil, want error")
}
}
Monkey patching is one of the ways to facilitate testing by changing a program at runtime, either by replacing a function or a variable. Here we are swapping the json.Marshal and we can mock it while testing.
var (
jsonMarshal = json.Marshal
)
This is your piece of code
func policyDocumentToStr(doc map[string]interface{}) (*string, error) {
policy, err := jsonMarshal(doc)
if err != nil {
fmt.Printf("Error converting policy document to string. Error %s", err)
return nil, err
}
policyAsString := string(policy)
return &policyAsString, nil
}
The test code would look something like this
import (
"testing"
"errors"
)
func fakemarshal(v interface{}) ([]byte, error) {
return []byte{}, errors.New("Marshalling failed")
}
func restoremarshal(replace func(v interface{}) ([]byte, error)) {
jsonMarshal = replace
}
func TestPolicyDocumentToStr(t * testing.T){
storedMarshal := jsonMarshal
jsonMarshal = fakemarshal
defer restoremarshal(storedMarshal)
input := map[string]interface{} {
"test": "test1",
}
tests := []struct {
name string
arg map[string]interface{}
wantErr string
}{
{
name: "Test if JSON Marshalling fails",
arg: input,
wantErr: "Marshalling failed",
},
}
for _, tt := range tests {
_, gotErr := policyDocumentToStr(tt.arg)
if gotErr != nil && gotErr.Error() != tt.wantErr {
t.Errorf("Expected %s but got %s", tt.wantErr, gotErr.Error())
}
}
}
I am using github.com/pressly/chi to build this simple program where I try to decode some JSON from the http.Request.Body:
package main
import (
"encoding/json"
"fmt"
"net/http"
"github.com/pressly/chi"
"github.com/pressly/chi/render"
)
type Test struct {
Name string `json:"name"`
}
func (p *Test) Bind(r *http.Request) error {
err := json.NewDecoder(r.Body).Decode(p)
if err != nil {
return err
}
return nil
}
func main() {
r := chi.NewRouter()
r.Post("/products", func(w http.ResponseWriter, r *http.Request) {
var p Test
// err := render.Bind(r, &p)
err := json.NewDecoder(r.Body).Decode(&p)
if err != nil {
panic(err)
}
fmt.Println(p)
})
http.ListenAndServe(":8080", r)
}
When I don't use render.Bind() (from "github.com/pressly/chi/render"), it works as expected.
However, when I uncomment the line err := render.Bind(r, &p) and I comment the line err := json.NewDecoder(r.Body).Decode(&p), it panics with EOF :
2017/06/20 22:26:39 http: panic serving 127.0.0.1:39696: EOF
and thus the json.Decode() fails.
Am I doing something wrong or is the http.Request.Body is already read somewhere else before render.Bind() is called?
render.Bind's purpose is to perform decode and execute Bind(r) to do post decode operations.
For eg.:
type Test struct {
Name string `json:"name"`
}
func (p *Test) Bind(r *http.Request) error {
// At this point, Decode is already done by `chi`
p.Name = p.Name + " after decode"
return nil
}
If you have to do only JSON decode no other actions needs to be done after decode with respect to decoded values. Just use:
// Use Directly JSON decoder of std pkg
err := json.NewDecoder(r.Body).Decode(&p)
OR
// Use wrapper method from chi DecodeJSON
err := render.DecodeJSON(r.Body, &p)