I'm using the Go MySQL driver as part of a project. Recently, there was a need to enable SSL on the MySQL database I am connecting to. I have two servers that are configured with TLSv1.1. On one server, I am able connect successfully, but on the other I am getting unexpected EOF from the library.
I tried setting MaxIdleConnections to 0 as described in Golang MySQL error - packets.go:33: unexpected EOF, but no luck. Prior to enabling SSL, connections to the server were working. There are other apps not written in Go that are able to connect successfully using the same credentials.
When my app runs, the ping command fails immediately. The MySQL server logs are saying bad handshake. Is there something off with my connection code below, or is there a database setting that could be refusing to connect with skip-verify?
import (
"database/sql"
"io/ioutil"
"strings"
"text/template"
log "github.com/sirupsen/logrus"
// Import runtime for MySQL
mysql "github.com/go-sql-driver/mysql"
)
type DatabaseConnection struct {
Hostname string
Port int
Database string
Username string
Password string
}
func CreateSSLConnection(db *DatabaseConnection) (*sql.DB, error) {
templateString := `{{.Username}}:{{.Password}}#{{.Hostname}}:{{.Port}}/{{.Database}}?tls=skip-verify`
conn, err := connectToDatabase("mysql", templateString, db)
// error: "unexpected EOF"
err = conn.Ping()
return conn, err
}
func connectToDatabase(databaseType string, connectionTemplateString string, db *DatabaseConnection) (*sql.DB, error) {
connectionTemplate := template.Must(template.New("connectionString").Parse(connectionTemplateString))
builder := strings.Builder{}
err := connectionTemplate.Execute(&builder, db)
if err != nil {
log.WithFields(log.Fields{
"template": connectionTemplateString,
"error": err,
}).Error("Failed to create connection string")
return nil, err
}
return sql.Open(databaseType, builder.String())
}
Related
Go version 1.18. MySQL server is version 8. System is 2018 MacBook Pro i9 6-cores, 32GB RAM.
However, mysql connection is refused during mysql.QueryRow(). error message is:
panic: dial unix /tmp/mysql.sock: connect: connection refused
PLEASE NOTE: I already find root cause(see my comment in the following code) and have a solution(using semaphore).
I MUST use socket instead of localhost when setting db connection.
The root cause is 2000 Go routines try to send query to MySQL simultaneously. How to solve this problem? My solution is to use semaphore to limit the number of concurrent go-routines, for example 100. I have tested, semaphore solution works well. I actually do not want to use semaphore to limit the number of concurrent mysql queries.
Question: why I can not have 2000 go-routines sending query at the same time.
import (
"database/sql"
mysqlDriver "github.com/go-sql-driver/mysql"
)
type SomeStruct struct {
a string
b string
}
func main() {
cfg := mysqlDriver.Config{
User: "DB_USER",
Passwd: "DB_USER",
Net: "unix",
Addr: "/tmp/mysql.sock", // MySQL is installed on local
DBName: "databaseName",
}
mysql, err := sql.Open("mysql", cfg.FormatDSN())
if err != nil {
panic(err) // tested, panic is not caused here.
}
mysql.SetMaxOpenConns(5000)
mysql.SetMaxIdleConns(5000)
if err = mysql.Ping(); err != nil {
panic(err) // tested, panic is not caused here.
}
var wg sync.WaitGroup
for i := 0; i < 2000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
var someStructVar SomeStruct
err := mysql.QueryRow("select a, b from table where id = ?", i).Scan(&someStructVar.a, &someStructVar.b)
if err != nil {
panic(err) // this is where the panic happens.
}
}
}
wg.Wait()
}
my.cnf is:
[mysqld]
max_connections=5000
connect_timeout=300
If your panic error occur before code
mysql.SetMaxOpenConns(5000)
Then the error is when connecting to the database, try changing the connection configuration to the database
I try to lunch application in ECS.
There is no problem when lunching in my local docker environment.
But it can't accrss to api server in ECS because of rds connection problem.
I use golang in api server and mysql for db.
I call db.go in main.go
func main() {
db := db.NewDatabase(os.Getenv("MYSQL_USER"), os.Getenv("MYSQL_PASSWORD"), os.Getenv("MYSQL_HOST"))
Error occurs when connecting to rds database
func NewDatabase(user, password, host string) *Database {
db, err := sql.Open("mysql", user+":"+password+"#tcp("+host+":3306)/article")
if err != nil {
panic(err.Error())
}
err = db.Ping()
// error occurs here
if err != nil {
panic(err.Error())
}
I deploy it to elastic beanstalk.
I checked environment variables are correctly set.
Here is the full source code:
https://github.com/jpskgc/article
I expect there is no error in elastic beans.
But the actual is not.
I want to know solution for that.
Here is the error log in elastic beanstalk.
-------------------------------------
/var/log/containers/server-4c66c8d1848a-stdouterr.log
-------------------------------------
panic: dial tcp 172.31.26.91:3306: connect: connection timed out
goroutine 1 [running]:
article/api/db.NewDatabase(0xc00002401b, 0x4, 0xc00002a00f, 0xb, 0xc00002800b, 0x3c, 0xdb94f2)
/app/db/db.go:20 +0x3bc
main.main()
/app/main.go:18 +0xee
"connection timed out" means that there are firewall limitation, and you can also check your mysql whitelist, which should has ip of your ECS.
I need to connect to a remote MySql server using Go. I'm using following code to connect to MySql via gorm.
type DBController struct {
DB gorm.DB
}
func (dc *DBController) InitDB() {
var err error
host := v.GetString("db.mysql.host")
port := v.GetString("db.mysql.port")
user := v.GetString("db.mysql.user")
pass := v.GetString("db.mysql.pass")
db := v.GetString("db.mysql.db")
//user:password#tcp(localhost:5555)/dbname
conn := fmt.Sprintf("%s:%s#tcp(%s:%s)/%s", user, pass, host, port, db)
//conn := v.GetString($user+":"$pass+"#tcp("+$host+":"+$port+")/"+$db)
log.Debug(conn)
dc.DB, err = gorm.Open("mysql", conn)
if err != nil {
log.Fatalf("Error when connect database, the error is '%v'", err)
}
dc.DB.LogMode(true)
}
func (dc *DBController) GetDB() gorm.DB {
return dc.DB
}
When I run the Go server I'm getting following errors
No configuration file loaded - using defaults
:#tcp(:)/
Error when connect database, the error is 'dial tcp: unknown port tcp/'
exit status 1
How can fix this error?
I'm guessing "v" is your config variable… Which contains no data, this is not a problem with Gorm, it's because dc.Open() is not receiving the config values.
If you try to run it with static data, it should work.
I am getting the following error when using with go-sql-driver with mysql and gorp when using in a separate package called dbutil
Error 1045: Access denied for user 'root'#'localhost' (using password: NO)
package dbutil
import (
"cropz/structs"
"database/sql"
"github.com/coopernurse/gorp"
_ "github.com/go-sql-driver/mysql"
"log"
)
func InitDB() *gorp.DbMap {
// connect to db
db, err := sql.Open("mysql", "root:pass#tcp(127.0.0.1:3306)/jsl")
defer db.Close()
err = db.Ping()
checkErr(err, "Ping failed")
// construct a gorp DbMap
dbmap := &gorp.DbMap{Db: db, Dialect: gorp.MySQLDialect{"InnoDB", "UTF8"}}
return dbmap
}
package main
func main() {
dbmap := dbutil.InitDB()
err := dbmap.Db.Ping()
checkErr(err, "Ping failed")
}
If I have the initDB() function in the main package, it works fine.
This happens only if used with martini framework and dbutil in separate package. With martini framework and in the same package it still works.
I am using windows, MySQL-5.0.22. Please help.
thanks,
Krishna
Your error looks like a login failure. Is your DSN setup appropriately?
Other than that you should remove the defer db.Close()
I believe you should only be closing the Db when you are actually done with it according to the spec.
When I run your code I actually get this error
panic: sql: database is closed
I deleted the .a files generated in the pkg folder and then didn't get the access denied error.
I'm trying to get a basic connect to my mysql server, but I can't seem to get it to actually connect. I know the credentials are valid and have all the permissions they need, but for some reason they're consistently rejected.
package main
import (
"fmt"
"database/sql"
_ "github.com/go-sql-driver/mysql"
"os"
)
func main() {
db, err:= sql.Open("mysql", "user:pass#tcp(localhost:3306)/scf")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
q, err := db.Prepare("SELECT * from logins limit 5")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
rows, err := q.Query()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
i := 0
for rows.Next() {
i++
var title string
err = rows.Scan( &title )
fmt.Printf("Title: %s \n", title)
}
db.Close()
}
Edit:
Apparently I forgot to include the error:
dial tcp 127.0.0.1:3306: connection refused
exit status 1
connection refused generally means the port isn't open or is being blocked by a firewall. A couple things to check:
Is MySQL (on localhost) running? Is it on port 3306?
If you're on Windows, Mac or Linux, is there a firewall that might be blocking port 3306?
If you're on Linux, is SELinux enabled that might be blocking port 3306?
I faced a similar issue. I found out port was different on Mac. I Couldn't find it in my.cnf but the port was set during runtime which you can see using
ps ax | grep mysql