I'm puzzled.
I try to run test cases using gorm with mysql in golang and I wanna buile MySQL just for testing, but it does not run safely.
I wanna use this package go-test-mysqld
Error message is below.
panic: sql: Register called twice for driver mysql
My code is
func TestMain(m *testing.M) {
mysqld, err := mysqltest.NewMysqld(nil)
if err != nil {
log.Fatal("runTests: failed", err)
}
defer mysqld.Stop()
dbm, err = gorm.Open("mysqld", mysqld.Datasource("test", "", "", 0 ))
if err != nil {
log.Fatal("db connection error:", err)
}
defer dbm.Close()
code := m.Run()
os.Exit(code)
}
What is the problems in my code?
Or is it impossible to build another mysql in using gorm?
Do you have some ideas?
Related
As the title says, I have an error when importing go-mysql-driver package. I have installed the go-my-sql driver in my machine but the error still persists. I use XAMPP for local hosting and here’s the block of program.
package model
import (
"database/sql"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
type Table interface {
Name() string
Field() ([]string, []interface{})
}
func Connect(username string, password string, host string, database string) (*sql.DB, error) {
conn := fmt.Sprintf("%s:%s#tcp(%s:3306)/%s", username, password, host, database)
db, err := sql.Open("mysql", conn)
return db, err
}
func CreateDB(db *sql.DB, name string) error {
query := fmt.Sprintf("CREATE DATABASE %v", name)
_, err := db.Exec(query)
return err
}
func CreateTable(db *sql.DB, query string) error {
_, err := db.Exec(query)
return err
}
func DropDB(db *sql.DB, name string) error {
query := fmt.Sprintf("DROP DATABASE %v", name)
_, err := db.Exec(query)
return err
}
could not import github.com/go-sql-driver/mysql (no required modules provides package "github.com/go-sql-driver/mysql")
screenshot of what's happening
It seems that you read the tutorial for an older go version.
Go 1.17 requires dependencies must be explicitly in go.mod.
Maybe you could try go module first (https://go.dev/blog/using-go-modules)
Your IDE is not showing you the whole picture. By running go run main.go (or whatever main file you have) on the command line, you can see the same error as you're seeing on your IDE with some extra:
$ go run main.go
main.go:7:5: no required module provides package github.com/go-sql-driver/mysql; to add it:
go get github.com/go-sql-driver/mysql
By issuing the suggested command go get github.com/go-sql-driver/mysql, you'll get the dependency added to your go.mod file and the contents of the package will be downloaded to your machine.
Your next execution will work:
$ go run main.go
Hello world
I've made some small modifications to your code to work, I'll add them here for completeness:
I've used the same source, but changed the package name to main.
I've added a main function to the bottom of the file:
func main() {
fmt.Println("Hello world")
_, err := Connect("username", "password", "localhost", "db")
if err != nil {
panic(err)
}
}
I've saved to a file named main.go
I've initialized the go.mod file by running go mod init test and go mod tidy, then I took the steps described on the beginning of the answer.
I'm setting up testing in Go. I use go-sqlmock to test mysql connection. Now I try to test mysql insert logic. But the error occurs.
I want to know how to resolve this error.
server side: golang
db: mysql
web framework: gin
dao.go
func PostDao(db *sql.DB, article util.Article, uu string) {
ins, err := db.Prepare("INSERT INTO articles(uuid, title,content) VALUES(?,?,?)")
if err != nil {
log.Fatal(err)
}
ins.Exec(uu, article.TITLE, article.CONTENT)
}
dao_test.go
func TestPostArticleDao(t *testing.T) {
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
defer db.Close()
mock.ExpectExec("^INSERT INTO articles*").
WithArgs("bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c", "test", "test").
WillReturnResult(sqlmock.NewResult(1, 1))
article := util.Article{
ID: 1,
TITLE: "test",
CONTENT: "test",
}
PostDao(db, article, "bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c")
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("there were unfulfilled expections: %s", err)
}
}
I expect go test -v runs without error.
But the actual is not.
Here is the error.
=== RUN TestPostArticleDao
2019/08/31 00:08:11 call to Prepare statement with query 'INSERT INTO articles(uuid, title,content) VALUES(?,?,?)', was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
- matches sql: 'INSERT INTO articles(uuid, title,content) VALUES(?,?,?)'
- is with arguments:
0 - bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c
1 - test
2 - test
- should return Result having:
LastInsertId: 1
RowsAffected: 1
exit status 1
FAIL article/api/dao 0.022s
As #Flimzy suggested, it needs to set ExpectPrepare first.
So I changed dao_test.go in this way:
prep := mock.ExpectPrepare("^INSERT INTO articles*")
prep.ExpectExec().
WithArgs("bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c", "test", "test").
WillReturnResult(sqlmock.NewResult(1, 1))
In my case it worked without asterix:
mock.ExpectExec("INSERT INTO `mytable`").WithArgs(mockdbutils.AnyTime{}, mockdbutils.AnyTime{}, nil, 4455,false).WillReturnResult(sqlmock.NewResult(int64(4455), 1))
mock.ExpectCommit()
I am using GORM with MySQL, I have encountered and handled the error Error 1062: Duplicate entry. The problem is that it's still printed to the console.
Code in gym/models/auth.go:49:
func AddAuth(username, password string) error {
passwordHash, err := auth.HashPassword(password, argon2Conf)
if err != nil {
return err
}
userAuth := Auth{
Username: username,
Password: passwordHash,
}
return db.Create(&userAuth).Error
}
I am handling the error in the handler function:
func SignUpHandler(c *gin.Context) {
var form user
if err := c.ShouldBind(&form); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := models.AddAuth(form.Username, form.Password); err == nil {
c.JSON(http.StatusOK, gin.H{"status": "you are signed in"})
} else {
// I think I have handled the sql error here
c.JSON(http.StatusBadRequest, gin.H{"error": "sign in failed"})
}
}
When I send a POST request, the error is correctly handled and I get the correct response with {"error": "sign in failed"}. But the console still prints this error message:
(/...../gym/models/auth.go:49)
[2019-04-28 23:37:06] Error 1062: Duplicate entry '123123' for key 'username'
[GIN] 2019/04/28 - 23:37:06 | 400 | 136.690908ms | ::1 | POST /signup
I am confused since I handled the error but it still gets printed. How to prevent this error from getting printed to the error log? Or am I handle the error correct?
UPDATE: for gorm v2:
Use the Logger in gorm.Config:
db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
For gorm v1:
Use db.LogMode to silence the default error logger.
LogMode set log mode, true for detailed logs, false for no log, default, will only print error logs.
db.LogMode(false) should do the job!
I don't have enough reputation to comment but just to add to the answer by ifnotak, you can log sql conditionally by controlling it through an environment variable. This can be handy during debugging.
gormConfig := &gorm.Config{}
if !logSql {
// I use an env variable LOG_SQL to set logSql to either true or false.
gormConfig.Logger = logger.Default.LogMode(logger.Silent)
}
db, err := gorm.Open(dialector, gormConfig)
Hi i am trying to execute a script to fill data into a database using Golang
func executeTestScript(){
cmd := exec.Command("/usr/local/mysql/bin/mysql", "-h127.0.0.1", "-P3333", "-uusr", "-pPassxxx", "-Ddtb_test", "< /Users/XXX/Documents/test/scripts/olds/SCRIPT_XXX.sql")
var out, stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil {
fmt.Println(fmt.Sprintf("Error executing query. Command Output: %+v\n: %+v, %v", out.String(), stderr.String(), err))
log.Fatalf("Error executing query. Command Output: %+v\n: %+v, %v", out.String(), stderr.String(), err)
}
}
The problem is that i am getting the error:
ERROR 1049 (42000): Unknown database '< /Users/XXX/Documents/test/scripts/olds/SCRIPT_XXX.sql'
i think the problem is the last param (the sql script path) that the exec thinks is the dbname
The following command in the terminal is working:
/usr/local/mysql/bin/mysql --host=127.0.0.1 --port=3333 --user=usr --password=Passxxx --database=dtb_test < /Users/XXX/Documents/roseula/scripts/olds/SCRIPT_XXX.sql
but i try to replicate in Go to automatize the execution of the script.
The script have drop tables, create tables, inserts, and PK with FK relationships its a very complete one so i cant execute line by line, because of that i decided to execute de mysql program to insert the data in the database.
Any suggestions?
+1 to answer from #MatteoRagni for showing how to do stdin redirection in Golang.
But here's a simple alternative that I use:
cmd := exec.Command("/usr/local/mysql/bin/mysql", "-h127.0.0.1", "-P3333",
"-uusr", "-pPassxxx", "-Ddtb_test",
"-e", "source /Users/XXX/Documents/test/scripts/olds/SCRIPT_XXX.sql")
You don't have to make the mysql client read the script using stdin redirection. Instead, you can make the mysql client execute a specific command, which is source <scriptname>.
P.S.: I also would not put the host, port, user, and password in your code. That means you have to recompile your program any time you change those connection parameters. Also it's not secure to use passwords in plaintext on the command-line. Instead, I'd put all the connection parameters into a defaults file and use mysql --defaults-file=FILENAME.
This is a little example that runs something like:
cat < test.txt
that is what I think you are missing in your code:
package main
import (
"fmt"
"os/exec"
"os"
)
func main() {
cmd := exec.Command("cat")
file, _ := os.Open("test.txt")
cmd.Stdin = file
out, _ := cmd.Output()
fmt.Printf("%s\n", out)
}
That prints in the console the content of test.txt, as read by cat. You will need to adapt it to your problem.
Something like:
func executeTestScript(){
cmd := exec.Command("/usr/local/mysql/bin/mysql", "-h127.0.0.1", "-P3333", "-uusr", "-pPassxxx", "-Ddtb_test")
dump, dump_err = os.Open("/Users/XXX/Documents/test/scripts/olds/SCRIPT_XXX.sql")
if dump_err != nil {
/* Handle the error if file not opened */
}
cmd.Stdin = dump
var out, stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil {
fmt.Println(fmt.Sprintf("Error executing query. Command Output: %+v\n: %+v, %v", out.String(), stderr.String(), err))
log.Fatalf("Error executing query. Command Output: %+v\n: %+v, %v", out.String(), stderr.String(), err)
}
}
if I'm not wrong...
I'm using github.com/go-sql-driver/mysql package to connect to MySQL. It works well except when I select a database (USE), I can't run queries against it.
package main
import (
"database/sql"
"fmt"
"log"
)
import _ "github.com/go-sql-driver/mysql"
func main() {
dsn := "root:#/"
db, err := sql.Open("mysql", dsn)
if err != nil {
fmt.Println("Failed to prepare connection to database. DSN:", dsn)
log.Fatal("Error:", err.Error())
}
err = db.Ping()
if err != nil {
fmt.Println("Failed to establish connection to database. DSN:", dsn)
log.Fatal("Error:", err.Error())
}
_, err = db.Query("USE test")
if err != nil {
fmt.Println("Failed to change database.")
log.Fatal("Error:", err.Error())
}
_, err = db.Query("SHOW TABLES")
if err != nil {
fmt.Println("Failed to execute query.")
log.Fatal("Error:", err.Error())
}
}
The program produces this output:
Error 1046: No database selected
Specify the database directly in the DSN (Data Source Name) part of the sql.Open function:
dsn := "user:password#/dbname"
db, err := sql.Open("mysql", dsn)
That's because db maintains a connection pool that has several connections to mysql database."USE test" just let one connection use schema test.
When you do database query later,the driver will select one idle connection,if the connection that use test schema is selected,it will be normal,but if another connection is chosen, it does not use test,so it will report an error:no database selected.
If you add a clause:
db.SetMaxOpenConns(1)
the db will maintain only one connection,it will not have an error.And of course it's impossible in high concurrency scene.
If you specify the database name in sql.open() function,all the connection will use this data base which can avoid this problem.
In your case you need to use transactions:
tx, _ := db.Begin()
tx.Query("USE test")
tx.Query("SHOW TABLES")
tx.Commit()
For SELECT/UPDATE/INSERT/etc need to specify DB name in the query.
As other answers mentioned, sql.DB is not a single connection but a connection pool. When you execute use database, imagine you executed your query on just one connection in the pool. Next query will get another connection from the pool which has no databases selected.
I would strongly advise against using transactions for this (as several places suggest).
I would suggest to use context:
ctx := context.Background()
conn, err := db.Conn(ctx)
conn.ExecContext(ctx, "use mydb")
defer conn.Close()
var found int
err = conn.QueryRowContext(ctx, "SELECT count(*) as found FROM mytable").Scan(&found)
if err != nil {
panic(err)
}