Share database connection with packages - mysql

I'm new with golang. I'm trying to share mysql database connection in my package, latter maybe in several packages. To skip defining database connection in every package I've created Database package and now I'm trying to get that package, connect to db and use that object in whole package.
I'm using this mysql plugin: github.com/go-sql-driver/mysql
here is my code:
main.go
package main
import (
"log"
"./packages/db" // this is my custom database package
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
var dbType Database.DatabaseType
var db *sql.DB
func main() {
log.Printf("-- entering main...")
dbType := Database.New()
db = dbType.GetDb()
dbType.DbConnect()
delete_test_data()
dbType.DbClose()
}
func delete_test_data(){
log.Printf("-- entering delete_test_data...")
//db.Exec("DELETE FROM test;")
}
packages/db/db.go
package Database
import (
"log"
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
type DatabaseType struct {
DatabaseObject *sql.DB
}
func New()(d *DatabaseType) {
d = new(DatabaseType)
//db.DatabaseObject = db.DbConnect()
return d
}
func (d *DatabaseType) DbConnect() *DatabaseType{
log.Printf("-- entering DbConnect...")
var err error
if d.DatabaseObject == nil {
log.Printf("--------- > Database IS NIL...")
d.DatabaseObject, err = sql.Open("mysql", "...")
if err != nil {
panic(err.Error())
}
err = d.DatabaseObject.Ping()
if err != nil {
panic(err.Error())
}
}
return d
}
func (d *DatabaseType) DbClose(){
log.Printf("-- entering DbClose...")
defer d.DatabaseObject.Close()
}
func (d *DatabaseType) GetDb() *sql.DB{
return d.DatabaseObject
}
Everything is ok and without error until I uncomment this line:
db.Exec("DELETE FROM test;")
Can someone tell me what is correct way to share db connection?

Your dbType.DbConnect() method returns a DatabaseType with an initialized connection, but you're ignoring the return value entirely.
Further - to simplify your code - look at having New(host string) *DB instead of three different functions (New/DbConnect/GetDb) that do the same thing.
e.g.
package datastore
type DB struct {
// Directly embed this
*sql.DB
}
func NewDB(host string) (*DB, error) {
db, err := sql.Open(...)
if err != nil {
return nil, err
}
return &DB{db}, nil
}
package main
var db *datastore.DB
func main() {
var err error
db, err = datastore.NewDB(host)
if err != nil {
log.Fatal(err)
}
err := someFunc()
}
func someFunc() error {
rows, err := db.Exec("DELETE FROM ...")
// Handle the error, parse the result, etc.
}
This reduces the juggling you have to do, and you can still call close on your DB type because it embeds *sql.DB - there's no need to implement your own Close() method.

Related

Golang - How to create helper mysql to generate uuid / uuid_short without create new connection? Here's the code

I'm trying to generate UUID_SHORT() to be an ID. Instead of using trigger, I get the UUID_SHORT() first and then insert as an ID. But, I'm confusing of how to create GetUUID() function as helper.
As my code bellow, it always setup new connection before generate the UUID_SHORT() which means there will be so many connection just to generate UUID_SHORT()
How to create func GetUUID() without creating new connection?
Here's the code:
package database
import (
"database/sql"
"fmt"
"log"
"os"
"time"
_ "github.com/go-sql-driver/mysql"
)
type conncetion struct {
sqlDB *sql.DB
}
func NewMysqlConnection(databaseConnection *sql.DB) *conncetion {
return &conncetion{databaseConnection}
}
func SetupMysqlDatabaseConnection() (db *sql.DB) {
var (
driver = os.Getenv("DB_DRIVERNAME")
username = os.Getenv("DB_USERNAME")
password = os.Getenv("DB_PASSWORD")
host = os.Getenv("DB_HOST")
port = os.Getenv("DB_PORT")
name = os.Getenv("DB_NAME")
)
connection := fmt.Sprintf("%s:%s#tcp(%s:%s)/%s?parseTime=true", username, password, host, port, name)
db, err := sql.Open(driver, connection)
if err != nil {
log.Fatal(err)
}
db.SetMaxOpenConns(100)
db.SetMaxIdleConns(100)
db.SetConnMaxLifetime(100 * time.Millisecond)
return
}
func (c *conncetion) GenerateUUID() (uuid uint64, err error) {
uuid = 0
queryGetUUID := c.sqlDB.QueryRow(`SELECT UUID_SHORT()`)
err = queryGetUUID.Scan(
&uuid,
)
return
}
func GetUUID() (uuid uint64, err error) {
mysql := SetupMysqlDatabaseConnection()
db := NewMysqlConnection(mysql)
uuid, err = db.GenerateUUID()
return
}
How about global variable ?
package database
import (
"database/sql"
"fmt"
"log"
"os"
"time"
_ "github.com/go-sql-driver/mysql"
)
type conncetion struct {
sqlDB *sql.DB
}
var globalConnection *conncetion
func GetDB() *conncetion {
return globalConnection
}
func NewMysqlConnection(databaseConnection *sql.DB) *conncetion {
return &conncetion{databaseConnection}
}
func SetupMysqlDatabaseConnection() (db *sql.DB) {
var (
driver = os.Getenv("DB_DRIVERNAME")
username = os.Getenv("DB_USERNAME")
password = os.Getenv("DB_PASSWORD")
host = os.Getenv("DB_HOST")
port = os.Getenv("DB_PORT")
name = os.Getenv("DB_NAME")
)
connection := fmt.Sprintf("%s:%s#tcp(%s:%s)/%s?parseTime=true", username, password, host, port, name)
db, err := sql.Open(driver, connection)
if err != nil {
log.Fatal(err)
}
db.SetMaxOpenConns(100)
db.SetMaxIdleConns(100)
db.SetConnMaxLifetime(100 * time.Millisecond)
return
}
func (c *conncetion) GenerateUUID() (uuid uint64, err error) {
uuid = 0
queryGetUUID := c.sqlDB.QueryRow(`SELECT UUID_SHORT()`)
err = queryGetUUID.Scan(
&uuid,
)
return
}
func GetUUID() (uuid uint64, err error) {
// mysql := SetupMysqlDatabaseConnection()
// db := NewMysqlConnection(mysql)
db = GetDB()
uuid, err = db.GenerateUUID()
return
}
// can be written on main.go / server.go
func init() {
globalConnection = NewMysqlConnection(database.SetupMysqlDatabaseConnection())
}
But I don't know it is good or not to open an idle connection and spamming request on 1 connection.
You're leveraging the database to generated the UUID for you, as long as you're letting the DB do it you'll need a connection to send the query.
You generally have two options here:
let the DB generate the UUID on the fly when inserting your data
generate the UUID in your own code before sending it
Generate ID on insert
This means you need to change the way you operate, your command for inserting data will need to look something like this:
INSERT INTO your_table(id, value)
VALUES (
UUID_SHORT(),
-- other values
);
This will automatically generate the ID for you during the insert, without needing to generate it before.
If you need to know the ID after the insert was performed, you have a few options, like using LAST_INSERT_ID() or querying the data you just created.
See this other question for more info.
Generate ID in code
You can use a package like github.com/google/uuid.
import "github.com/google/uuid"
func GenerateUUID() (uint32, error) {
id, err := uuid.NewRandom()
if err != nil {
return 0, err
}
return id.ID(), nil
}
Note that you can also get a string representation for the UUID, or you can cast it to uint64 easily from uint32.

Do I need to close connection in Gorm? [duplicate]

So far the hardest part of Go has been understanding how to organize code. It seems incredibly simple on it's face but every time I've tried to do anything I run into circular imports or things like "exported func Start returns unexported type models.dbStore, which can be annoying to use".
Using the following code how do I call db.Close() or am I really not understanding how I'm supposed to provide the database to my models. Here's what I've got:
App.go
package app
import (
"database/sql"
// Comment
_ "github.com/mattn/go-sqlite3"
)
var (
// DB The database connection
db *sql.DB
)
// Setup Sets up the many many app settings
func Setup() {
d, err := sql.Open("sqlite3", "./foo.db")
if err != nil {
panic(err)
}
// TODO: How does the DB get closed?
// defer db.Close()
db = d
}
// GetDB Returns a reference to the database
func GetDB() *sql.DB {
return db
}
Users.go
package models
import (
"github.com/proj/org/app"
)
// User struct
type User struct {
ID int
}
// CreateUser Creates a user
func (u *User) CreateUser() (int64, error) {
// For the sake of brevity just make sure you can
// "connect" to the database
if err := app.GetDB().Ping(); err != nil {
panic(err)
}
return 1234, nil
}
main.go
package main
import (
"fmt"
"net/http"
_ "github.com/mattn/go-sqlite3"
"github.com/proj/org/app"
"github.com/proj/org/models"
)
func homeHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "You are home")
}
func subscribeHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Subscribing...")
u := models.User{}
u.CreateUser()
}
func main() {
fmt.Println("Running")
app.Setup()
http.HandleFunc("/", homeHandler)
http.HandleFunc("/subscribe", subscribeHandler)
err := http.ListenAndServe(":9090", nil)
if err != nil {
panic(err)
}
}
I thought about doing a app.Shutdown() but that wouldn't work for my most normal use case which is CTRL-C. It would seem if I don't close the database the DB connections would just grow... Just trying to understand.
It's not necessary to close the DB.
The returned DB is safe for concurrent use by multiple goroutines and
maintains its own pool of idle connections. Thus, the Open function
should be called just once. It is rarely necessary to close a DB.
From: https://golang.org/pkg/database/sql/#Open
When your program exits then any open connection is closed, it does not stay open somewhere in the ether awaiting your program to start again, so do not worry about the connections "growing" when you CTRL-C your app.
If, however, you still want to close it, then you can just export a CloseDB function the same way you do with GetDB.
App.go
// ...
func CloseDB() error {
return db.Close()
}
main.go
// ...
func main() {
// ...
app.Setup()
defer app.CloseDB()
// ...
}

I need help on this Golang web application using mysql as database

I am new to Golang and had been following some tutorials and I want to put into practice what I have learned to create a website
This is the main.go file
package main
import (
"html/template"
"net/http"
"log"
"database/sql"
_"github.com/go-sql-driver/mysql"
)
//Fetch all templates
var templates, templatesErr = template.ParseGlob("templates/*")
func main() {
PORT := ":9000"
log.Println("Listening to port", PORT)
http.HandleFunc("/", root)
http.HandleFunc("/facilities", allFacilities)
http.ListenAndServe(PORT, nil)
}
func root(w http.ResponseWriter, r *http.Request) {
rootData := make(map[string]string)
rootData["page_title"] = "iSpace Open Data"
rootData["body"] = ""
templates.ExecuteTemplate(w, "index.html", rootData)
}
type facility struct{
FacilityName string
Type string
}
func allFacilities(w http.ResponseWriter, r *http.Request){
db, err := sql.Open("mysql", "root:08swanzy#tcp(127.0.0.1:3306)/iod")
if err !=nil{
log.Fatal(err)
}
defer db.Close()
rows, err := db.Query("Select FacilityName, Type from health_facilities ")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
fac := facility{}
facilities := []facility{}
for rows.Next(){
var FacilityName, Type string
rows.Scan(&FacilityName, &Type)
fac.FacilityName= FacilityName
fac.Type= Type
facilities = append(facilities, fac)
}
templates.ExecuteTemplate(w, "facilities.html", facilities)
}
This uses html files in templates folder for the view. But I keep on getting runtime error saying it has pointer dereference. I need help please.
Tried your code and got the same error. It happened on this line:
templates.ExecuteTemplate(w, "index.html", rootData)
The problem is that your templates are not loaded correctly. I moved template parsing to the main function and it works. Here the relevant code snippet:
//Fetch all templates
var (
templates *template.Template
)
func main() {
var err error
templates, err = template.ParseGlob("templates/*")
if err != nil {
panic(err)
}
PORT := ":9000"
log.Println("Listening to port", PORT)
http.HandleFunc("/", root)
http.HandleFunc("/facilities", allFacilities)
http.ListenAndServe(PORT, nil)
}

How can I implement my own interface for OpenID that uses a MySQL Database instead of In memory storage

So I'm trying to use the OpenID package for Golang, located here: https://github.com/yohcop/openid-go
In the _example it says that it uses in memory storage for storing the nonce/discoverycache information and that it will not free the memory and that I should implement my own version of them using some sort of database.
My database of choice is MySQL, I have tried to implement what I thought was correct (but is not, does not give me any compile errors, but crashes on runtime)
My DiscoveryCache.go is as such:
package openid
import (
"database/sql"
"log"
//"time"
_ "github.com/go-sql-driver/mysql"
"github.com/yohcop/openid-go"
)
type SimpleDiscoveredInfo struct {
opEndpoint, opLocalID, claimedID string
}
func (s *SimpleDiscoveredInfo) OpEndpoint() string { return s.opEndpoint }
func (s *SimpleDiscoveredInfo) OpLocalID() string { return s.opLocalID }
func (s *SimpleDiscoveredInfo) ClaimedID() string { return s.claimedID }
type SimpleDiscoveryCache struct{}
func (s SimpleDiscoveryCache) Put(id string, info openid.DiscoveredInfo) {
/*
db, err := sql.Query("mysql", "db:connectinfo")
errCheck(err)
rows, err := db.Query("SELECT opendpoint, oplocalid, claimedid FROM discovery_cache")
errCheck(err)
was unsure what to do here because I'm not sure how to
return the info properly
*/
log.Println(info)
}
func (s SimpleDiscoveryCache) Get(id string) openid.DiscoveredInfo {
db, err := sql.Query("mysql", "db:connectinfo")
errCheck(err)
var sdi = new(SimpleDiscoveredInfo)
err = db.QueryRow("SELECT opendpoint, oplocalid, claimedid FROM discovery_cache WHERE id=?", id).Scan(&sdi)
errCheck(err)
return sdi
}
And my Noncestore.go
package openid
import (
"database/sql"
"errors"
"flag"
"fmt"
"time"
_ "github.com/go-sql-driver/mysql"
)
var maxNonceAge = flag.Duration("openid-max-nonce-age",
60*time.Second,
"Maximum accepted age for openid nonces. The bigger, the more"+
"memory is needed to store used nonces.")
type SimpleNonceStore struct{}
func (s *SimpleNonceStore) Accept(endpoint, nonce string) error {
db, err := sql.Open("mysql", "dbconnectinfo")
errCheck(err)
if len(nonce) < 20 || len(nonce) > 256 {
return errors.New("Invalid nonce")
}
ts, err := time.Parse(time.RFC3339, nonce[0:20])
errCheck(err)
rows, err := db.Query("SELECT * FROM noncestore")
defer rows.Close()
now := time.Now()
diff := now.Sub(ts)
if diff > *maxNonceAge {
return fmt.Errorf("Nonce too old: %ds", diff.Seconds())
}
d := nonce[20:]
for rows.Next() {
var timeDB, nonce string
err := rows.Scan(&nonce, &timeDB)
errCheck(err)
dbTime, err := time.Parse(time.RFC3339, timeDB)
errCheck(err)
if dbTime == ts && nonce == d {
return errors.New("Nonce is already used")
}
if now.Sub(dbTime) < *maxNonceAge {
_, err := db.Query("INSERT INTO noncestore SET nonce=?, time=?", &nonce, dbTime)
errCheck(err)
}
}
return nil
}
func errCheck(err error) {
if err != nil {
panic("We had an error!" + err.Error())
}
}
Then I try to use them in my main file as:
import _"github.com/mysqlOpenID"
var nonceStore = &openid.SimpleNonceStore{}
var discoveryCache = &openid.SimpleDiscoveryCache{}
I get no compile errors but it crashes
I'm sure you'll look at my code and go what the hell (I'm fairly new and only have a week or so experience with Golang so please feel free to correct anything)
Obviously I have done something wrong, I basically looked at the NonceStore.go and DiscoveryCache.go on the github for OpenId, replicated it, but replaced the map with database insert and select functions
IF anybody can point me in the right direction on how to implement this properly that would be much appreciated, thanks! If you need anymore information please ask.
Ok. First off, I don't believe you that the code compiles.
Let's look at some mistakes, shall we?
db, err := sql.Open("mysql", "dbconnectinfo")
This line opens a database connection. It should only be used once, preferably inside an init() function. For example,
var db *sql.DB
func init() {
var err error
// Now the db variable above is automagically set to the left value (db)
// of sql.Open and the "var err error" above is the right value (err)
db, err = sql.Open("mysql", "root#tcp(127.0.0.1:3306)")
if err != nil {
panic(err)
}
}
Bang. Now you're connected to your MySQL database.
Now what?
Well this (from Get) is gross:
db, err := sql.Query("mysql", "db:connectinfo")
errCheck(err)
var sdi = new(SimpleDiscoveredInfo)
err = db.QueryRow("SELECT opendpoint, oplocalid, claimedid FROM discovery_cache WHERE id=?", id).Scan(&sdi)
errCheck(err)
Instead, it should be this:
// No need for a pointer...
var sdi SimpleDiscoveredInfo
// Because we take the address of 'sdi' right here (inside Scan)
// And that's a useless (and potentially problematic) layer of indirection.
// Notice how I dropped the other "db, err := sql.Query" part? We don't
// need it because we've already declared "db" as you saw in the first
// part of my answer.
err := db.QueryRow("SELECT ...").Scan(&sdi)
if err != nil {
panic(err)
}
// Return the address of sdi, which means we're returning a pointer
// do wherever sdi is inside the heap.
return &sdi
Up next is this:
/*
db, err := sql.Query("mysql", "db:connectinfo")
errCheck(err)
rows, err := db.Query("SELECT opendpoint, oplocalid, claimedid FROM discovery_cache")
errCheck(err)
was unsure what to do here because I'm not sure how to
return the info properly
*/
If you've been paying attention, we can drop the first sql.Query line.
Great, now we just have:
rows, err := db.Query("SELECT ...")
So, why don't you do what you did inside the Accept method and parse the rows using for rows.Next()... ?

Go: Getting unexpected error

In my controller package, I have a AppContext struct that looks like this:
type AppContext struct {
db *sql.DB
}
func (c *AppContext) getDB() *sql.DB {
return c.db
}
Then I have the following codes in my main package:
func main {
db, err := sql.Open("mysql",
//other info)
if err != nil {
log.Fatal(err)
return
}
err = db.Ping()
if err != nil {
log.Fatal(err)
return
}
defer db.Close()
appC := controller.AppContext{db}
}
When building it, I get this unexpected error:
implicit assignment of unexported field 'db' in controller.AppContext literal
I tried looking that error up, but could not find much information on it. Is there a way to resolve this problem?
As said in the comment, db is not exported, so inaccessible from other packages.
In Go, initialization of structures is usually done with a function called NewMyStructure, so for example:
func NewAppContext(db *sql.DB) AppContext {
return AppContext{db: db}
}
and then in your main:
appC := controller.NewAppContext(db)