I am officially crying uncle to the benevolent samaritans of Stack Overflow.
I am trying to unit test my GORM (Postgres) + Fiber API using a mock DB. I have a Card model and a CreateCardReqBody model for the POST request body. To setup the test, I create a random CreateCardReqBody instance, marshal it into JSON, then pass it into an *httptest.Request. The handler uses Fiber's (*fiber.Ctx).BodyParser function to "unmarshal" the request body into an empty Card struct. However, when I run the test that is supposed to pass, Fiber throws an "Unprocessable Entity" error.
Below are the relevant parts of my code; the test file is a combination of this tutorial and Fiber's documentation on the (*App).Test method. (I realize the code could be cleaned up; I'm just trying to get a proof of life then focus on revising :)
I've done a few things to debug this: I've made a Postman POST request with the same values as the test and it works. Within the test itself, I marshal then unmarshal the CreateCardReqBody struct and that works. I've triple checked the spelling of the JSON fields match, that the struct fields are exported, etc. I've also run the VSCode debugger and the body field within Fiber.Ctx's also looks correct to me.
I'm starting to wonder if it's something with how Fiber parses the body from a test request vs. a real request. I would greatly appreciate any insight one could share on this!
Model Definition
type Card struct {
gorm.Model
// Implicit Gorm foreign key to fleet ID
FleetID uint `gorm:"index" json:"fleet_id" validate:"required,min=1"`
// Card provider's account number
ProviderAccountNumber string `json:"provider_account_number"`
// Card provider's external card identifier
CardIdentifier string `gorm:"index" json:"card_identifier" validate:"min=1"`
// Implicit Gorm foreign key to driver ID. Driver association is optional.
DriverID uint `json:"associated_driver_id" validate:"min=1"`
// Implicit Gorm foreign key to vehicle ID.
VehicleID uint `json:"associated_vehicle_id" validate:"required,min=1"`
// User-inputted start date, formatted "2020-01-26T22:38:25.000Z" in UTC
StartDate pq.NullTime
}
Test file
// Adapted from tutorial
type testCase struct {
name string
body CreateCardReqBody
setupAuth func(t *testing.T, request *http.Request)
buildStubs func(db *mockDB.MockDBInterface)
checkResponse func(response *http.Response, outputErr error)
}
type CreateCardReqBody struct {
FleetID int `json:"fleet_id"`
ProviderAccountNumber string `json:"provider_account_number"`
CardIdentifier string `json:"card_identifier"`
StartDate string `json:"start_date"`
AssociatedDriverID int `json:"associated_driver_id"`
AssociatedVehicleID int `json:"associated_vehicle_id"`
}
func TestCreateCard(t *testing.T) {
user := randomUser(t)
vehicle := randomVehicle()
driver := randomDriver(vehicle.FleetID)
okReqCard := randomCard(vehicle.FleetID)
finalOutputCard := okReqCard
finalOutputCard.ID = 1
testCases := []testCase{
{
name: "Ok",
body: CreateCardReqBody{
FleetID: int(okReqCard.FleetID),
ProviderAccountNumber: okReqCard.ProviderAccountNumber,
CardIdentifier: okReqCard.CardIdentifier,
StartDate: okReqCard.StartDate.Time.Format("2006-01-02T15:04:05.999Z"),
AssociatedDriverID: int(okReqCard.DriverID),
AssociatedVehicleID: int(okReqCard.VehicleID),
},
setupAuth: func(t *testing.T, request *http.Request) {
addAuthorization(t, request, user)
},
// Tell mock database what calls to expect and what values to return
buildStubs: func(db *mockDB.MockDBInterface) {
db.EXPECT().
UserExist(gomock.Eq(fmt.Sprint(vehicle.FleetID))).
Times(1).Return(user, true, user.ID)
db.EXPECT().
SearchTSP(gomock.Eq(fmt.Sprint(vehicle.FleetID))).
Times(1)
db.EXPECT().
SearchVehicle(gomock.Eq(fmt.Sprint(okReqCard.VehicleID))).
Times(1).
Return(vehicle, nil)
db.EXPECT().
SearchDriver(gomock.Eq(fmt.Sprint(driver.ID))).
Times(1).
Return(driver, nil)
db.EXPECT().
CardCreate(gomock.Eq(okReqCard)).
Times(1).
Return(finalOutputCard, nil)
},
checkResponse: func(res *http.Response, outputErr error) {
require.NoError(t, outputErr)
// Internal helper func, excluded for brevity
requireBodyMatchCard(t, finalOutputCard, res.Body)
},
},
}
for _, test := range testCases {
t.Run(test.name, func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
mockDB := mockDB.NewMockDBInterface(ctrl)
test.buildStubs(mockDB)
jsonBytes, err := json.Marshal(test.body)
require.NoError(t, err)
jsonBody := bytes.NewReader(jsonBytes)
// Debug check: am I able to unmarshal it back? YES.
errUnmarsh := json.Unmarshal(jsonBytes, &CreateCardReqBody{})
require.NoError(t, errUnmarsh)
endpoint := "/v1/transactions/card"
request := httptest.NewRequest("POST", endpoint, jsonBody)
// setupAuth is helper function (not shown in this post) that adds authorization to httptest request
test.setupAuth(t, request)
app := Init("test", mockDB)
res, err := app.Test(request)
test.checkResponse(res, err)
})
}
}
Route handler being tested
func (server *Server) CreateCard(c *fiber.Ctx) error {
var card models.Card
var err error
// 1) Parse POST data
if err = c.BodyParser(&card); err != nil {
return c.Status(http.StatusUnprocessableEntity).SendString(err.Error())
}
...
}
Debugger Output
Json body when defined in test
Body inside Fiber context
facepalm
I forgot to request.Header.Set("Content-Type", "application/json")! Posting this in case it's helpful for anyone else :)
Related
I am working on deserializing json into a struct as shown below and it works fine.
type DataConfigs struct {
ClientMetrics []Client `json:"ClientMetrics"`
}
type Client struct {
ClientId int `json:"clientId"`
.....
.....
}
const (
ConfigFile = "clientMap.json"
)
func ReadConfig(path string) (*DataConfigs, error) {
files, err := utilities.FindFiles(path, ConfigFile)
// check for error here
var dataConfig DataConfigs
body, err := ioutil.ReadFile(files[0])
// check for error here
err = json.Unmarshal(body, &dataConfig)
// check for error here
return &dataConfig, nil
}
Now I am trying to build a map of integer and Client using DataConfigs object that was created as shown above in the code. So I created a method to do the job as shown below and I modified ReadConfig method to do that too.
func ReadConfig(path string, logger log.Logger) (*DataConfigs, error) {
files, err := utilities.FindFiles(path, ConfigFile)
// check for error here
var dataConfig DataConfigs
body, err := ioutil.ReadFile(files[0])
// check for error here
err = json.Unmarshal(body, &dataConfig)
// check for error here
idx := BuildIndex(dataConfig)
// now how to add this idx and dataConfig object in one struct?
return &dataConfig, nil
}
func BuildIndex(dataConfig DataConfigs) map[int]Client {
m := make(map[int]Client)
for _, dataConfig := range dataConfig.ClientMetrics {
m[dataConfig.ClientId] = dataConfig
}
return m
}
My confusion is - Should I modify DataConfigs struct to add idx map too and then return that struct from ReadConfig method or should I create a new struct to handle that?
Basically I want to return DataConfigs struct which has ClientMetrics array along with idx map. How can I do this here? I am slightly confuse because I started with golang recently.
This is basically a design question with multiple options. First, I would avoid adding the map to your original DataConfigs type since it does not match the json representation. This could lead to confusion down the road.
Which option to choose depends on your requirements and preferences. Some ideas from the top of my head:
Have you considered returning the map only? After all, you've got every Client in your map. If you need to iterate all Clients you can iterate all values of your map.
Second option is to return the map in addition to DataConfigs. Go allows to return multiple values from a function as you already do for error handling.
Finally, you could wrap DataConfigs and your map in a new simple struct type as you already guessed.
As part of an application we are building, one of the process steps is an AWS Lamda that captures a post request does some work with it, and then moves one. It has an API Gateway Request as a trigger and the body of this request would be a JSON String. I am having trouble parsing the JSON String to GoLang Object. Here is what I have:
The function that catches request:
func HandleRequest(ctx context.Context, event events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
log.Print(fmt.Sprintf("body:[%s] ", event.Body))
parseResponseStringToTypedObject(event.Body)
return events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Body: "OK",
}, nil
}
Then the parseResponseStringToTypedObject function :
func parseResponseStringToTypedObject(responseString string) {
b := []byte(responseString)
var resp SimpleType
err := json.Unmarshal(b, &resp)
if err == nil {
log.Print(fmt.Sprintf("Account Name: [%s]", resp.accountName))
} else {
log.Print(fmt.Sprintf("Could not unmarshall JSON string: [%s]", err.Error()))
}
}
Here is the SimpleType struct:
type SimpleType struct {
accountName string `json:accountName`
amount int `json:amount`
}
I then, as a test, posted this JSON Body via Postman:
I opened up the CloudWatch Logs (where my lamda logs to) and see that the body is present in the event.Body property, and then logging out a field in the unmarshalled object (resp.accountName) I note that the field is blank. Why is this? Here is log output for the request:
Your SimpleType struct needs 2 things here...
1) The properties need to be "public" or "exported". Meaning they need to start with an upper cased character.
AND
2) The properties need proper tags for the serialization and de serialization of JSON. e.g. each JSON tag surrounded by "
type SimpleType struct {
AccountName string `json:"accountName"`
Amount int `json:"amount"`
}
Hope this helps!
I'm trying to test a method that uses net/http to make requests. Specifically what I'm trying to achieve is to inject a mock http.Client that responds with a certain JSON body
type clientMock struct{}
func (c *clientMock) Do(req *http.Request) (*http.Response, error) {
json := struct {
AccessToken string `json:"access_token`
Scope string `json:"scope"`
}{
AccessToken: "123457678",
Scope: "read, write",
}
body := json.Marshal(json)
res := &http.Response {
StatusCode: http.StatusOK,
Body: // I haven't got a clue what to put here
}
return res
}
func TestRequest(t *testing.T) { //tests here }
I do know that the Body is of a type io.ReadCloser interface. Trouble is I can't for the life of me find a way to implement it in the mock body response.
Examples as found here so far only demonstrates returning a blank &http.Response{}
While it's probably more useful to mock the full request cycle with httptest.Server, you can use ioutil.NopCloser to create the closer around any reader:
Body: ioutil.NopCloser(bytes.NewReader(body))
and if you want an empty body, just provider a reader with no content.
Body: ioutil.NopCloser(bytes.NewReader(nil))
In your test file (my_test.go):
type MyJSON struct {
Id string
Age int
}
// Interface which is the same as httpClient because it implements "Do" method.
type ClientMock struct {}
func (c *ClientMock) Do(req *http.Request) (*http.Response, error) {
mockedRes := MyJSON {"1", 3}
// Marshal a JSON-encoded version of mockedRes
b, err := json.Marshal(mockedRes)
if err != nil {
log.Panic("Error reading a mockedRes from mocked client", err)
}
return &http.Response{Body: ioutil.NopCloser(bytes.NewBuffer(b))}, nil
}
// your test which will use the mocked response
func TestMyFunction(t *testing.T) {
mock := &ClientMock{}
actualResult := myFunction(mock)
assert.NotEmpty(t, actualResult, "myFunction should have at least 1 result")
}
In your implementation (main.go):
package main
import (
"net/http"
)
func main() {
myFunction(&http.Client{})
}
I know it's been a little while but I just wrote something to help with this recently.
Like JimB I recommend starting up a real HTTP server locally, since in Go this is easy to do with https://golang.org/pkg/net/http/httptest/.
However having done a lot of HTTP mocking I wanted something that does a little more, like a good mock library would: returning specific data, easy setting of expectations, validation that all requests were made, etc. I have generally used https://godoc.org/github.com/stretchr/testify/mock for mocking and wanted features like that.
So I wrote https://github.com/dankinder/httpmock, which basically combines the two. If you just want a mock that accepts JSON and spits out JSON, it may be an easier way to go.
and need your help.
Wanted to build simple api and stuck with some problem.
I've choose gin and database/sql with postgres driver
package main
import (
"database/sql"
"fmt"
"github.com/gin-gonic/gin"
_ "github.com/lib/pq"
)
func main() {
router := gin.Default()
router.GET("/search/:text", SearchWord)
router.Run(":8080")
}
I need to make query to DB and make json out of this request.
func checkErr(err error) {
if err != nil {
panic(err)
}
}
type Message struct {
ticket_id int `json:"ticket_id"`
event string `json:"event"`
}
func SearchWord(c *gin.Context) {
word := c.Params.ByName("text")
db, err := sql.Open("postgres", "host=8.8.8.8 user= password= dbname=sample")
defer db.Close()
checkErr(err)
rows, err2 := db.Query("SELECT ticket_id,event FROM ....$1, word)
checkErr(err)
for rows.Next() {
var ticket_id int
var event string
err = rows.Scan(&ticket_id, &event)
checkErr(err)
fmt.Printf("%d | %s \n\n", ticket_id, event)
}
}
This coda working nice, but when i need to make json.
I need to make struct of a row
type Message struct {
ticket_id int `json:"ticket_id"`
event string `json:"event"`
}
an then i need to create slice , and append every rows.Next() loop an than answer to browser with Json...
c.JSON(200, messages)
But how to do that...don't know :(
disclaimer: I am brand new to go
Since you Scanned your column data into your variables, you should be able to initialize a structure with their values:
m := &Message{ticket_id: ticket_id, event: event}
You could initialize a slice with
s := make([]*Message, 0)
And then append each of your message structs after instantiation:
s = append(s, m)
Because I'm not too familiar with go there are a couple things i'm not sure about:
after copying data from query to your vars using rows.Scan does initializing the Message struct copy the current iterations values as expected??
If there is a way to get the total number of rows from your query it might be slighlty more performant to initialize a static length array, instead of a slice?
I think #inf deleted answer about marshalling your Message to json down the line might need to be addressed, and Message field's might need to be capitalized
copied from #inf:
The names of the members of your struct need be capitalized so that
they get exported and can be accessed.
type Message struct {
Ticket_id int `json:"ticket_id"`
Event string `json:"event"` }
I'm going to cheat a little here and fix a few things along the way:
First: open your database connection pool once at program start-up (and not on every request).
Second: we'll use sqlx to make it easier to marshal our database rows into our struct.
package main
var db *sqlx.DB
func main() {
var err error
// sqlx.Connect also checks that the connection works.
// sql.Open only "establishes" a pool, but doesn't ping the DB.
db, err = sqlx.Connect("postgres", "postgres:///...")
if err != nil {
log.Fatal(err)
}
router := gin.Default()
router.GET("/search/:text", SearchWord)
router.Run(":8080")
}
// in_another_file.go
type Message struct {
TicketID int `json:"ticket_id" db:"ticket_id"`
Event string `json:"event" db:"event"`
}
func SearchWord(c *gin.Context) {
word := c.Params.ByName("text")
// We create a slice of structs to marshal our rows into
var messages []*Message{}
// Our DB connection pool is safe to use concurrently from here
err := db.Select(&messages, "SELECT ticket_id,event FROM ....$1, word)
if err != nil {
http.Error(c.Writer, err.Error(), 500)
return
}
// Write it out using gin-gonic's JSON writer.
c.JSON(200, messages)
}
I hope that's clear. sqlx also takes care of calling rows.Close() for you, which will otherwise leave connections hanging.
I have a python server serving response through JSON-RPC. Here is a sample response from the server.
'{"jsonrpc": "2.0", "result": "Peer 1: local 10.10.0.2 remote 10.10.0.1 state CONNECT\\nPeer 2: local 10.10.0.18 remote 10.10.0.17 state ESTABLISHED\\nPeer 3: local 10.10.0.10 remote 10.10.0.9 state ESTABLISHED", "id": "839732f9-cf36-46ff-8b9b-6120250d9ce5"}'
Here is the request I need to send to the server:
'{"method":"echo","jsonrpc":"2.0","params":["test_params"],"id":"839732f9-cf36-46ff-8b9b-6120250d9ce5"}'
Here is my client with go language:
package main
import (
"fmt"
"log"
"net"
"net/rpc/jsonrpc"
)
type Args struct {
jsonrpc, id string
}
func main() {
conn, err := net.Dial("tcp", "11.21.22.221:8080")
if err != nil {
panic(err)
}
defer conn.Close()
args := Args{"2.0", "d87198f0-af92-49f8-9a7d-ab8bed5c4d17"}
var reply string
c := jsonrpc.NewClient(conn)
err = c.Call("echo", args, &reply)
if err != nil {
log.Fatal("error:", err)
}
fmt.Printf("Response: %d", reply)
}
But, when I run this client, it is not sending anything in the params. Instead it is sending empty params like this:
'{"method":"echo","params":[{}],"id":0}\n
Can somebody help me telling what mistake I am making? I am a newbie to go language.
Thanks.
I don't think what you are doing is possible using the client as provided by go because the private clientRequest struct is currently defined as:
type clientRequest struct {
Method string `json:"method"`
Params [1]interface{} `json:"params"`
Id uint64 `json:"id"`
}
What you pass into Call as args is stuck into Params and note how there is no "Version `json:"jsonrpc"`" inside of that struct.
AFAICT (which may be wrong, this is my first time reading through this code) you would need to implement your own ClientCodec. You could probably get away with copying most (all) of the parts out of the stdlib and add the field to the clientRequest above. ;-)