I am using express & passport modules for user authentication on nodejs. The code runs fine locally. But when uploaded to the bluemix server, it runs for two login instances or for some time-out and subsequently the server crashes. Here is the log from bluemix :
Warning: connect.session() MemoryStore is not
2016-02-29T15:03:22.974+0530[App/0] ERRmemory, and will not scale past a single process.
2016-02-29T15:03:22.974+0530[App/0] ERRdesigned for a production environment, as it will leak
2016-02-29T15:03:22.985+0530[App/0] OUTServer is running # http://localhost:64134
2016-02-29T15:04:23.173+0530[App/0] ERRError: Quit inactivity timeout
2016-02-29T15:04:23.173+0530[App/0] ERR throw er; // Unhandled 'error' event
2016-02-29T15:04:23.173+0530[App/0] ERR at Quit.emit (events.js:166:7)
2016-02-29T15:04:23.173+0530[App/0] ERRevents.js:141
2016-02-29T15:04:23.173+0530[App/0] ERR ^
2016-02-29T15:04:23.173+0530[App/0] ERR at Timer.listOnTimeout (timers.js:92:15)
2016-02-29T15:04:23.173+0530[App/0] ERR at Quit. (/home/vcap/app/node_modules/mysql/lib/protocol/Protocol.js:160:17)
2016-02-29T15:04:23.173+0530[App/0] ERR at emitNone (events.js:67:13)
2016-02-29T15:04:23.173+0530[App/0] ERR at Quit._onTimeout
(/home/vcap/app/node_modules/mysql/lib/protocol/sequences/Sequence.js:116:8) 2016-02-29T15:04:23.856+0530[API/9]OUTApp instance exited with guid caa03cfb-bf79-43e1-997d-4d76d50a41a7 payload: {"cc_partition"=>"default", "droplet"=>"caa03cfb-bf79-43e1-997d-4d76d50a41a7", "version"=>"0000dfe2-af05-4f9a-b244-3902edb3d5ff", "instance"=>"88586613847b4d098601a29ffad5d390", "index"=>0, "reason"=>"CRASHED", "exit_status"=>1, "exit_description"=>"app instance exited", "crash_timestamp"=>1456738463}
Related
Go version 1.18. MySQL server is version 8. System is 2018 MacBook Pro i9 6-cores, 32GB RAM.
However, mysql connection is refused during mysql.QueryRow(). error message is:
panic: dial unix /tmp/mysql.sock: connect: connection refused
PLEASE NOTE: I already find root cause(see my comment in the following code) and have a solution(using semaphore).
I MUST use socket instead of localhost when setting db connection.
The root cause is 2000 Go routines try to send query to MySQL simultaneously. How to solve this problem? My solution is to use semaphore to limit the number of concurrent go-routines, for example 100. I have tested, semaphore solution works well. I actually do not want to use semaphore to limit the number of concurrent mysql queries.
Question: why I can not have 2000 go-routines sending query at the same time.
import (
"database/sql"
mysqlDriver "github.com/go-sql-driver/mysql"
)
type SomeStruct struct {
a string
b string
}
func main() {
cfg := mysqlDriver.Config{
User: "DB_USER",
Passwd: "DB_USER",
Net: "unix",
Addr: "/tmp/mysql.sock", // MySQL is installed on local
DBName: "databaseName",
}
mysql, err := sql.Open("mysql", cfg.FormatDSN())
if err != nil {
panic(err) // tested, panic is not caused here.
}
mysql.SetMaxOpenConns(5000)
mysql.SetMaxIdleConns(5000)
if err = mysql.Ping(); err != nil {
panic(err) // tested, panic is not caused here.
}
var wg sync.WaitGroup
for i := 0; i < 2000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
var someStructVar SomeStruct
err := mysql.QueryRow("select a, b from table where id = ?", i).Scan(&someStructVar.a, &someStructVar.b)
if err != nil {
panic(err) // this is where the panic happens.
}
}
}
wg.Wait()
}
my.cnf is:
[mysqld]
max_connections=5000
connect_timeout=300
If your panic error occur before code
mysql.SetMaxOpenConns(5000)
Then the error is when connecting to the database, try changing the connection configuration to the database
I try to lunch application in ECS.
There is no problem when lunching in my local docker environment.
But it can't accrss to api server in ECS because of rds connection problem.
I use golang in api server and mysql for db.
I call db.go in main.go
func main() {
db := db.NewDatabase(os.Getenv("MYSQL_USER"), os.Getenv("MYSQL_PASSWORD"), os.Getenv("MYSQL_HOST"))
Error occurs when connecting to rds database
func NewDatabase(user, password, host string) *Database {
db, err := sql.Open("mysql", user+":"+password+"#tcp("+host+":3306)/article")
if err != nil {
panic(err.Error())
}
err = db.Ping()
// error occurs here
if err != nil {
panic(err.Error())
}
I deploy it to elastic beanstalk.
I checked environment variables are correctly set.
Here is the full source code:
https://github.com/jpskgc/article
I expect there is no error in elastic beans.
But the actual is not.
I want to know solution for that.
Here is the error log in elastic beanstalk.
-------------------------------------
/var/log/containers/server-4c66c8d1848a-stdouterr.log
-------------------------------------
panic: dial tcp 172.31.26.91:3306: connect: connection timed out
goroutine 1 [running]:
article/api/db.NewDatabase(0xc00002401b, 0x4, 0xc00002a00f, 0xb, 0xc00002800b, 0x3c, 0xdb94f2)
/app/db/db.go:20 +0x3bc
main.main()
/app/main.go:18 +0xee
"connection timed out" means that there are firewall limitation, and you can also check your mysql whitelist, which should has ip of your ECS.
I'm using the Go MySQL driver as part of a project. Recently, there was a need to enable SSL on the MySQL database I am connecting to. I have two servers that are configured with TLSv1.1. On one server, I am able connect successfully, but on the other I am getting unexpected EOF from the library.
I tried setting MaxIdleConnections to 0 as described in Golang MySQL error - packets.go:33: unexpected EOF, but no luck. Prior to enabling SSL, connections to the server were working. There are other apps not written in Go that are able to connect successfully using the same credentials.
When my app runs, the ping command fails immediately. The MySQL server logs are saying bad handshake. Is there something off with my connection code below, or is there a database setting that could be refusing to connect with skip-verify?
import (
"database/sql"
"io/ioutil"
"strings"
"text/template"
log "github.com/sirupsen/logrus"
// Import runtime for MySQL
mysql "github.com/go-sql-driver/mysql"
)
type DatabaseConnection struct {
Hostname string
Port int
Database string
Username string
Password string
}
func CreateSSLConnection(db *DatabaseConnection) (*sql.DB, error) {
templateString := `{{.Username}}:{{.Password}}#{{.Hostname}}:{{.Port}}/{{.Database}}?tls=skip-verify`
conn, err := connectToDatabase("mysql", templateString, db)
// error: "unexpected EOF"
err = conn.Ping()
return conn, err
}
func connectToDatabase(databaseType string, connectionTemplateString string, db *DatabaseConnection) (*sql.DB, error) {
connectionTemplate := template.Must(template.New("connectionString").Parse(connectionTemplateString))
builder := strings.Builder{}
err := connectionTemplate.Execute(&builder, db)
if err != nil {
log.WithFields(log.Fields{
"template": connectionTemplateString,
"error": err,
}).Error("Failed to create connection string")
return nil, err
}
return sql.Open(databaseType, builder.String())
}
I'm using the Go selenium package https://godoc.org/github.com/tebeka/selenium
And I'm running headless chrome + selenium-server inside a docker container on localhost:4444
The server seems to be fine since I can access the web console via http://localhost:4444/wd/hub/static/resource/hub.html
But I'm trying to get the "Hello world" example to work with the existing docker container.
This is the example from the GoDocs page for the selenium driver:
// Run some code on play.golang.org and display the result
package main
import (
"fmt"
"time"
"github.com/tebeka/selenium"
)
var code = `
package main
import "fmt"
func main() {
fmt.Println("Hello WebDriver!\n")
}
`
// Errors are ignored for brevity.
func main() {
// Connect to the selenium server
caps := selenium.Capabilities{"browserName": "firefox"}
wd, err := selenium.NewRemote(caps, "http://127.0.0.1:4444")
if err != nil {
fmt.Println(err)
}
defer wd.Quit()
// Get simple playground interface
wd.Get("http://play.golang.org/?simple=1")
// Enter code in textarea
elem, _ := wd.FindElement(selenium.ByCSSSelector, "#code")
elem.Clear()
elem.SendKeys(code)
// Click the run button
btn, _ := wd.FindElement(selenium.ByCSSSelector, "#run")
btn.Click()
// Get the result
div, _ := wd.FindElement(selenium.ByCSSSelector, "#output")
output := ""
// Wait for run to finish
for {
output, _ = div.Text()
if output != "Waiting for remote server..." {
break
}
time.Sleep(time.Millisecond * 100)
}
fmt.Printf("Got: %s\n", output)
}
I tried changing the "browserName" to "chrome" but I get this error:
panic: got content type "text/html", expected "application/json"
goroutine 1 [running]:
main.main()
/home/user01/Code/golang_src/golang_exercises/33_selenium/selenium.go:28 +0x457
exit status 2
I can't find anything in the GoDoc selenium documentation regarding the chrome browser and how to connect to it via the selenium-server.
I would appreciate any hints as to what might be going wrong here.
Update:
It seems that removing the URL address and leaving it empty has fixed the connection problems:
wd, err := selenium.NewRemote(caps, "")
That said, I'm still having issues with the example. Mainly it seems like it connects to the Go Playground website, gets the right elements, but when it comes to sending the input elem.SendKeys(code) it doesn't send it properly and the text box is empty. Resulting in bad output from the Playground:
Got: can't load package: package main:
tmp/sandbox573608783/main.go:1:1: expected 'package', found 'EOF'
Program exited.
try run it in headless mode:
caps := selenium.Capabilities{"browserName": "chrome"}
chromeCaps := chrome.Capabilities{
Path: "",
Args: []string{
"--headless", // <<<
"--no-sandbox",
"--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7",
},
}
caps.AddChrome(chromeCaps)
wd, err := selenium.NewRemote(caps, fmt.Sprintf("http://127.0.0.1:%d", port))
I also use selenium in Docker, it can run by this:
wd, err := selenium.NewRemote(caps, "http://127.0.0.1:4444/wd/hub")
After some debugging I found out that the issue was due to the fact that my Docker container did not have the X server inside it.
When the selenium package tried to send input it would generate this error message:
unknown error: unknown error: an X display is required for keycode conversions, consider using Xvfb
(Session info: headless chrome=60.0.3095.5)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-77-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 40 milliseconds
Build info: version: '3.3.1', revision: '5234b32', time: '2017-03-10 09:04:52 -0800'
System info: host: 'e3bf5382c62d', ip: '171.14.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-77-generic', java.version: '1.8.0_121'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, networkConnectionEnabled=false, chrome={chromedriverVersion=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5), userDataDir=/tmp/.org.chromium.Chromium.mFhqlU}, takesHeapSnapshot=true, pageLoadStrategy=normal, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=60.0.3095.5, platform=LINUX, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=}]
Session ID: 681f9de5f91baeaaa3100cf297767a2d
I have not yet installed the X server inside the Docker container but I'm confident that it will fix the error that occurs when sending input to the headless chrome instance via selenium.
I am getting the following error when using with go-sql-driver with mysql and gorp when using in a separate package called dbutil
Error 1045: Access denied for user 'root'#'localhost' (using password: NO)
package dbutil
import (
"cropz/structs"
"database/sql"
"github.com/coopernurse/gorp"
_ "github.com/go-sql-driver/mysql"
"log"
)
func InitDB() *gorp.DbMap {
// connect to db
db, err := sql.Open("mysql", "root:pass#tcp(127.0.0.1:3306)/jsl")
defer db.Close()
err = db.Ping()
checkErr(err, "Ping failed")
// construct a gorp DbMap
dbmap := &gorp.DbMap{Db: db, Dialect: gorp.MySQLDialect{"InnoDB", "UTF8"}}
return dbmap
}
package main
func main() {
dbmap := dbutil.InitDB()
err := dbmap.Db.Ping()
checkErr(err, "Ping failed")
}
If I have the initDB() function in the main package, it works fine.
This happens only if used with martini framework and dbutil in separate package. With martini framework and in the same package it still works.
I am using windows, MySQL-5.0.22. Please help.
thanks,
Krishna
Your error looks like a login failure. Is your DSN setup appropriately?
Other than that you should remove the defer db.Close()
I believe you should only be closing the Db when you are actually done with it according to the spec.
When I run your code I actually get this error
panic: sql: database is closed
I deleted the .a files generated in the pkg folder and then didn't get the access denied error.