Best way to integrate database into Go Web application - mysql

I'm new to developing web applications in Go.
I'm looking for the best way to integrate a MySQL database into my web application.
I was thinking of doing something like this:
type Context struct {
Database *sql.DB
}
// Some database methods like Close() and Query() for Context struct here
In the main function for my web application I would have something like this:
db := sql.Open(...)
ctx := Context{db}
I would then pass then pass my Context structure into various handlers that require a database connection. Would this be a good design decision or is there a better way to integrate a SQL database into my web application?

I typically do something like this:
package main
func main(){
db,err := sql.Open(...)
if err != nil {
log.Fatal(err)
}
defer db.Close()
http.HandleFunc("/feed", server.FeedHandler(db))
http.HandleFunc("/gui", server.GuiHandler(db))
...
log.Fatal(http.ListenAndServe(":8000", nil))
}
Where server is a separate package where I define, implement and test all my http handlers.
This is basically what you were thinking of but skipping the step of wrapping the db in a struct which isn't necessary. I would not recommend making the db a global variable. Having global dependencies will completely destroy any possibility of testing your http handlers in a reliable fashion.
Dependency injecting the db, as in the example above, costs you two extra characters to type every time you call the function but it allows you to test your http handlers easily using the go-sqlmock package which you definitely want to do as your web app grows.
package server
func TestFeedHandler(t *testing.T){
mockdb, err := sqlmock.New()
if err != nil {
t.Errorf("An error '%s' was not expected when opening a stub database connection", err)
}
columns := []string{"id", "name"}
sqlmock.ExpectQuery("SELECT id,name from persons;").
WillReturnRows(sqlmock.NewRows(columns).
AddRow(1234, "bob")
handler := FeedHandler(mockdb)
// test that handler returns expected result
}

Related

Go - TCP Client Often Misses Secondary JSON Reply [duplicate]

This question already has an answer here:
Does ReadString() discard bytes following newline?
(1 answer)
Closed 8 months ago.
I'm trying to write a TCP client that sends out a JSON RPC to a locally hosted TCP server. The server should immediately return two replies.
The TCP server is hosted on port 60000.
This is what the client code looks like:
package main
import (
"fmt"
"log"
"bufio"
"net"
)
func main() {
d := net.Dialer{ }
c, err := d.Dial("tcp", "127.0.0.1:60000")
if err != nil {
log.Println(err)
return
}
data := `{"id": 1,"method":"mining.configure","params": [["version-rolling"],{"version-rolling.mask": "1fffe000","version-rolling.min-bit-count": 16}]}`
fmt.Fprintf(c, data+"\n")
for {
message, _ := bufio.NewReader(c).ReadString('\n')
fmt.Println(message)
}
}
This is what I'm sending out ( the "data" variable )
{
"id": 1,
"method":"mining.configure",
"params": [["version-rolling"],
{"version-rolling.mask": "1fffe000",
"version-rolling.min-bit-count": 16}]
}
This is the expected reply:
{"error":null,"id":1,"result":{"version-rolling":true,"version-rolling.mask":"1fffe000"}}
{"id":null,"method":"mining.set_version_mask","params":["1fffe000"]}
Most of the time I only get the first reponse (has the result "field") without the second JSON that has the "method" field. Sometimes I get both.
But this is the usual reply:
{"error":null,"id":1,"result":{"version-rolling":true,"version-rolling.mask":"1fffe000"}}
I know that when code becomes non-deterministic like this it's because of an asynchronous issue going on.
But every beginner tutorial I encountered teaches this exact structure for writing a TCP client. What's missing from it?
I've tested the same RPC call to the server extensively with Python with the Twisted framework and I get back the both replies 100% of the time. This makes me certain the issue is within my client code and not the server.
Depending on the timing that data is received, the call to ReadString can buffer data past the \n. The code in the question discards the reader and data buffered in the reader on each iteration of the loop.
To avoid discarding data, create the reader once and reuse inside the loop:
r := bufio.NewReader(c)
for {
message, _ := r.ReadString('\n')
fmt.Println(message)
}

How to organise Function Maps for specific endpoints in Gin (GoLang)?

TL;DR
I'm wondering how to organise function maps for go templates that are dependant on the data being obtained in the specific endpoint being called.
Issue
I have a main frontend endpoint (for a CMS) which will grab the slug of the page, and go and obtain the post data it from the model. I also have a templates package that takes in models, the post data and gin context.
GoView is currently being used. But I am setting HTMLRender to equal a new ginview evertime the page is hit presumably this isn't great for performance and I'm wondering if there is a neater way of going about it.
// Serve the front end website
func (c *FrontendController) Serve(g *gin.Context) {
path := g.Request.URL.Path
post, err := c.models.Posts.GetBySlug(path)
if err != nil {
NoPageFound(g)
return
}
tf := templates.NewFunctions(g, c.models, &post)
c.server.HTMLRender = ginview.New(goview.Config{
Root: paths.Theme(),
Extension: config.Template["file_extension"],
Master: "/layouts/main",
Partials: []string{},
DisableCache: true,
Funcs: tf.GetFunctions(),
})
r := ResourceData{
Post: post,
}
g.HTML(200, config.Template["template_dir"] + "/" + post.PageTemplate, r)
}
Templates
Some template functions include getting the field & author of the post, and seeing if the user is logged in, for example.
func (t *TemplateFunctions) isAuth() bool {
...
}
There are also template functions that aren't tied to gin.context such as getting the assets path.
func (t *TemplateFunctions) assetsPath() string {
...
}
Questions
Is there a way to merge the data dependant functions with global ones?
Is there a way to set the template functions without redeclaring the HTMLRender every time a page is hit? Or is it necessary?

How to use IP restrictions when using Google Direction API?

I read in the docs that in order to use Google Direction API Web Service, I needed to use IP restrictions. So to achieve this feature, I've created a proxy server that serve me the direction I want. The link looks like so:
https://myapp-44th7.uc.r.appspot.com/?origin=22.174181,33.6436763&destination=22.189821,33.640532
When I use an unrestricted API key and I access this URL in a browser, it works fine, I get the desired JSON file. The problem comes when I want to restrict it. I cannot add a single IP address as a restriction because my Android app is used by users with many different IP addresses. I have hard times trying understanding how to restrict this. Please help solve this issue. What IP should I add?
Edit:
package main
import (
"fmt"
"net/http"
"io/ioutil"
"context"
"google.golang.org/appengine"
"google.golang.org/appengine/urlfetch"
)
const directionAPIKey = "..............uFvjU"
const directionURL = "https://maps.googleapis.com/maps/api/directions/json?origin=%s&destination=%s&mode=%s&key=%s"
func main() {
http.HandleFunc("/", handler)
appengine.Main()
}
func handler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
direction, err := fetchDirection(ctx, r.FormValue("origin"), r.FormValue("destination"), r.FormValue("mode"))
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Add("Content-Type", "application/json; charset=utf-8")
w.Write(direction)
}
func fetchDirection(ctx context.Context, origin string, destination string, mode string) ([]byte, error) {
client := urlfetch.Client(ctx)
resp, err := client.Get(fmt.Sprintf(directionURL, origin, destination, mode, directionAPIKey))
if err != nil {
return nil, err
}
defer resp.Body.Close()
return ioutil.ReadAll(resp.Body)
}
The directionAPIKey being the key that I use in my URL to access:
https://myapp-44th7.uc.r.appspot.com/?origin=22.174181,33.6436763&destination=22.189821,33.64053&key=..............uFvjU
And the one for which I added the restriction in the GCP. 172.217.23.52 being the IP that comes when I ping myapp-44th7.uc.r.appspot.com.
For you to restrict your API keys with IP restrictions, you need to add your server's public facing IP address as a restriction to your API key. More information here.
Once you have applied IP restrictions to your API key, only requests from your specific server(in your case your proxy server) is allowed to use your restricted API key. This means that all requests which come from different IP addresses other than the one you have set as restriction are blocked. This is the reason why you are getting an error if you tried to send a request directly from your browser using your restricted API key since your browser has a different IP address than your proxy server.
To make sure that your restricted API key works, you can verify if you can successfully send a Directions API request through your proxy server using your restricted API key.
Hope this helps!

How to avoid affecting other code when panic error or something else in golang

I am writing a monitor program that will merge into company's system. When the program running, seldom panicked some error since a variety of problem such as a few other's server doesn't obtain the rule of http, network error or anything else.I fix most of them but I am still afraid of influence on the master program by possibly potential errors.
From company's aspect, the stability of master program is most important, secondly, monitoring's result, the stability of monitoring is low-priority.
Is there have a method can isolate error? For example,"try...except Exception..." in python to ignore error(Admittedly, Not recommended)How to avoid effect other code when panic error or something else in golang
With panics, you can use recover like this
func recoverExample() {
defer func() {
if panicMessage := recover(); panicMessage != nil {
fmt.Printf("Panic '%v' captured", panicMessage)
}
}()
panic("Oh no!")
}
With output:
Panic 'Oh no!' captured
User recover function at the top of your function in which you have the chances of getting error. Don't use panic as it will stop the execution of your code. Use below code snippet. Also It will print the log which will let you know in which function the exception occurred.
defer func() {
if r := recover(); r != nil {
log.Println("Recovered in custom function", r)
}
}()

google app engine golang, driver: bad connection

I have some code that is working on the local GAE server but once I publish it to GAE it throws the error "driver: bad connection".
Below code generates a new *sql.DB:
func NewDb() (*sql.DB, error) {
cloud := os.Getenv("dbcloud")
local := os.Getenv("dblocal")
if appengine.IsDevAppServer() {
return sql.Open("mysql", "root#tcp("+local+":3306)/dbo")
}
return sql.Open("mysql", "root#cloudsql("+cloud+")/dbo")
}
In my app.yaml I have the following:
env_variables:
dbcloud: 'projectid:instancename'
dblocal: 'xxx.xxx.xxx.xxx'
It seems to return a new *sql.DB correctly but once I start using prepared statements is when things start to break.
db, err := NewDb() // err is nil
stmt, err := db.Prepare("INSERT INTO dbo.Users (Id) VALUES (?)") // err is driver: bad connection
I've been fighting with this for an hour now and i'm probably doing something very stupid any help would be appreciated!
I ended up needing to change my dbcloud variable to include the region of the SQL server changing it from:
'projectid:instancename'
To:
'projectid:regionname:instancename'
No idea why I need to do this as it's not in the docs of https://github.com/go-sql-driver/mysql but is all working now!