I try to lunch application in ECS.
There is no problem when lunching in my local docker environment.
But it can't accrss to api server in ECS because of rds connection problem.
I use golang in api server and mysql for db.
I call db.go in main.go
func main() {
db := db.NewDatabase(os.Getenv("MYSQL_USER"), os.Getenv("MYSQL_PASSWORD"), os.Getenv("MYSQL_HOST"))
Error occurs when connecting to rds database
func NewDatabase(user, password, host string) *Database {
db, err := sql.Open("mysql", user+":"+password+"#tcp("+host+":3306)/article")
if err != nil {
panic(err.Error())
}
err = db.Ping()
// error occurs here
if err != nil {
panic(err.Error())
}
I deploy it to elastic beanstalk.
I checked environment variables are correctly set.
Here is the full source code:
https://github.com/jpskgc/article
I expect there is no error in elastic beans.
But the actual is not.
I want to know solution for that.
Here is the error log in elastic beanstalk.
-------------------------------------
/var/log/containers/server-4c66c8d1848a-stdouterr.log
-------------------------------------
panic: dial tcp 172.31.26.91:3306: connect: connection timed out
goroutine 1 [running]:
article/api/db.NewDatabase(0xc00002401b, 0x4, 0xc00002a00f, 0xb, 0xc00002800b, 0x3c, 0xdb94f2)
/app/db/db.go:20 +0x3bc
main.main()
/app/main.go:18 +0xee
"connection timed out" means that there are firewall limitation, and you can also check your mysql whitelist, which should has ip of your ECS.
Related
Go version 1.18. MySQL server is version 8. System is 2018 MacBook Pro i9 6-cores, 32GB RAM.
However, mysql connection is refused during mysql.QueryRow(). error message is:
panic: dial unix /tmp/mysql.sock: connect: connection refused
PLEASE NOTE: I already find root cause(see my comment in the following code) and have a solution(using semaphore).
I MUST use socket instead of localhost when setting db connection.
The root cause is 2000 Go routines try to send query to MySQL simultaneously. How to solve this problem? My solution is to use semaphore to limit the number of concurrent go-routines, for example 100. I have tested, semaphore solution works well. I actually do not want to use semaphore to limit the number of concurrent mysql queries.
Question: why I can not have 2000 go-routines sending query at the same time.
import (
"database/sql"
mysqlDriver "github.com/go-sql-driver/mysql"
)
type SomeStruct struct {
a string
b string
}
func main() {
cfg := mysqlDriver.Config{
User: "DB_USER",
Passwd: "DB_USER",
Net: "unix",
Addr: "/tmp/mysql.sock", // MySQL is installed on local
DBName: "databaseName",
}
mysql, err := sql.Open("mysql", cfg.FormatDSN())
if err != nil {
panic(err) // tested, panic is not caused here.
}
mysql.SetMaxOpenConns(5000)
mysql.SetMaxIdleConns(5000)
if err = mysql.Ping(); err != nil {
panic(err) // tested, panic is not caused here.
}
var wg sync.WaitGroup
for i := 0; i < 2000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
var someStructVar SomeStruct
err := mysql.QueryRow("select a, b from table where id = ?", i).Scan(&someStructVar.a, &someStructVar.b)
if err != nil {
panic(err) // this is where the panic happens.
}
}
}
wg.Wait()
}
my.cnf is:
[mysqld]
max_connections=5000
connect_timeout=300
If your panic error occur before code
mysql.SetMaxOpenConns(5000)
Then the error is when connecting to the database, try changing the connection configuration to the database
I'm using the Go MySQL driver as part of a project. Recently, there was a need to enable SSL on the MySQL database I am connecting to. I have two servers that are configured with TLSv1.1. On one server, I am able connect successfully, but on the other I am getting unexpected EOF from the library.
I tried setting MaxIdleConnections to 0 as described in Golang MySQL error - packets.go:33: unexpected EOF, but no luck. Prior to enabling SSL, connections to the server were working. There are other apps not written in Go that are able to connect successfully using the same credentials.
When my app runs, the ping command fails immediately. The MySQL server logs are saying bad handshake. Is there something off with my connection code below, or is there a database setting that could be refusing to connect with skip-verify?
import (
"database/sql"
"io/ioutil"
"strings"
"text/template"
log "github.com/sirupsen/logrus"
// Import runtime for MySQL
mysql "github.com/go-sql-driver/mysql"
)
type DatabaseConnection struct {
Hostname string
Port int
Database string
Username string
Password string
}
func CreateSSLConnection(db *DatabaseConnection) (*sql.DB, error) {
templateString := `{{.Username}}:{{.Password}}#{{.Hostname}}:{{.Port}}/{{.Database}}?tls=skip-verify`
conn, err := connectToDatabase("mysql", templateString, db)
// error: "unexpected EOF"
err = conn.Ping()
return conn, err
}
func connectToDatabase(databaseType string, connectionTemplateString string, db *DatabaseConnection) (*sql.DB, error) {
connectionTemplate := template.Must(template.New("connectionString").Parse(connectionTemplateString))
builder := strings.Builder{}
err := connectionTemplate.Execute(&builder, db)
if err != nil {
log.WithFields(log.Fields{
"template": connectionTemplateString,
"error": err,
}).Error("Failed to create connection string")
return nil, err
}
return sql.Open(databaseType, builder.String())
}
I need to connect to a remote MySql server using Go. I'm using following code to connect to MySql via gorm.
type DBController struct {
DB gorm.DB
}
func (dc *DBController) InitDB() {
var err error
host := v.GetString("db.mysql.host")
port := v.GetString("db.mysql.port")
user := v.GetString("db.mysql.user")
pass := v.GetString("db.mysql.pass")
db := v.GetString("db.mysql.db")
//user:password#tcp(localhost:5555)/dbname
conn := fmt.Sprintf("%s:%s#tcp(%s:%s)/%s", user, pass, host, port, db)
//conn := v.GetString($user+":"$pass+"#tcp("+$host+":"+$port+")/"+$db)
log.Debug(conn)
dc.DB, err = gorm.Open("mysql", conn)
if err != nil {
log.Fatalf("Error when connect database, the error is '%v'", err)
}
dc.DB.LogMode(true)
}
func (dc *DBController) GetDB() gorm.DB {
return dc.DB
}
When I run the Go server I'm getting following errors
No configuration file loaded - using defaults
:#tcp(:)/
Error when connect database, the error is 'dial tcp: unknown port tcp/'
exit status 1
How can fix this error?
I'm guessing "v" is your config variable… Which contains no data, this is not a problem with Gorm, it's because dc.Open() is not receiving the config values.
If you try to run it with static data, it should work.
I am using express & passport modules for user authentication on nodejs. The code runs fine locally. But when uploaded to the bluemix server, it runs for two login instances or for some time-out and subsequently the server crashes. Here is the log from bluemix :
Warning: connect.session() MemoryStore is not
2016-02-29T15:03:22.974+0530[App/0] ERRmemory, and will not scale past a single process.
2016-02-29T15:03:22.974+0530[App/0] ERRdesigned for a production environment, as it will leak
2016-02-29T15:03:22.985+0530[App/0] OUTServer is running # http://localhost:64134
2016-02-29T15:04:23.173+0530[App/0] ERRError: Quit inactivity timeout
2016-02-29T15:04:23.173+0530[App/0] ERR throw er; // Unhandled 'error' event
2016-02-29T15:04:23.173+0530[App/0] ERR at Quit.emit (events.js:166:7)
2016-02-29T15:04:23.173+0530[App/0] ERRevents.js:141
2016-02-29T15:04:23.173+0530[App/0] ERR ^
2016-02-29T15:04:23.173+0530[App/0] ERR at Timer.listOnTimeout (timers.js:92:15)
2016-02-29T15:04:23.173+0530[App/0] ERR at Quit. (/home/vcap/app/node_modules/mysql/lib/protocol/Protocol.js:160:17)
2016-02-29T15:04:23.173+0530[App/0] ERR at emitNone (events.js:67:13)
2016-02-29T15:04:23.173+0530[App/0] ERR at Quit._onTimeout
(/home/vcap/app/node_modules/mysql/lib/protocol/sequences/Sequence.js:116:8) 2016-02-29T15:04:23.856+0530[API/9]OUTApp instance exited with guid caa03cfb-bf79-43e1-997d-4d76d50a41a7 payload: {"cc_partition"=>"default", "droplet"=>"caa03cfb-bf79-43e1-997d-4d76d50a41a7", "version"=>"0000dfe2-af05-4f9a-b244-3902edb3d5ff", "instance"=>"88586613847b4d098601a29ffad5d390", "index"=>0, "reason"=>"CRASHED", "exit_status"=>1, "exit_description"=>"app instance exited", "crash_timestamp"=>1456738463}
I'm trying to get a basic connect to my mysql server, but I can't seem to get it to actually connect. I know the credentials are valid and have all the permissions they need, but for some reason they're consistently rejected.
package main
import (
"fmt"
"database/sql"
_ "github.com/go-sql-driver/mysql"
"os"
)
func main() {
db, err:= sql.Open("mysql", "user:pass#tcp(localhost:3306)/scf")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
q, err := db.Prepare("SELECT * from logins limit 5")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
rows, err := q.Query()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
i := 0
for rows.Next() {
i++
var title string
err = rows.Scan( &title )
fmt.Printf("Title: %s \n", title)
}
db.Close()
}
Edit:
Apparently I forgot to include the error:
dial tcp 127.0.0.1:3306: connection refused
exit status 1
connection refused generally means the port isn't open or is being blocked by a firewall. A couple things to check:
Is MySQL (on localhost) running? Is it on port 3306?
If you're on Windows, Mac or Linux, is there a firewall that might be blocking port 3306?
If you're on Linux, is SELinux enabled that might be blocking port 3306?
I faced a similar issue. I found out port was different on Mac. I Couldn't find it in my.cnf but the port was set during runtime which you can see using
ps ax | grep mysql