Unable to ping mysql db docker in golang - mysql

I am new to golang, docker and mysql. I am trying to connect to mysql running in docker locally on my macos using golang.
Here is the code:
`package main
import (
"context"
"database/sql"
_ "github.com/go-sql-driver/mysql"
"log"
"time"
)
func dbConn() *sql.DB {
db, err := sql.Open("mysql", "root:Abc123$##tcp(172.17.0.2:3306)/test")
if err != nil {
log.Printf("Error %s when opening DB connection\n", err)
return nil
}
db.SetMaxOpenConns(10)
db.SetMaxIdleConns(10)
db.SetConnMaxLifetime(time.Minute * 2)
ctx, cancelfunc := context.WithTimeout(context.Background(), time.Second)
defer cancelfunc()
err = db.PingContext(ctx)
if err != nil {
log.Printf("Error %s pinging DB", err)
return db
}
log.Print("Connected to the DB successfully\n")
defer func() {
err := db.Close()
if err != nil {
log.Print(err)
}
}()
return db
}
func main() {
db := dbConn()
defer db.Close()
}`
I am running docker with the following command:
docker run --name mysql -e MYSQL_ROOT_PASSWORD=abcd1234 -p 3306:3306 -d mysql:8.0.30
I get the following error:
Error dial tcp 172.17.0.2:3306: i/o timeout pinging DB
docker is running locally. I created a test-db with command-line:
`mysql> create database test_db;`
and then did a
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| test_db |
+--------------------+
6 rows in set (0.00 sec)
Please help me understand my mistake here? Or what am I missing?

I used host.docker.internal in this docker-compose.yml file to create the mysql sever in docker.
version: '3'
services:
mysql-development:
image: mysql:8.0.30
environment:
MYSQL_ROOT_PASSWORD: xyz1234
MYSQL_DATABASE: test
ports:
- "3306:3306"
extra_hosts:
- "host.docker.internal:host-gateway"
Used this to run the docker image:
docker-compose -f docker-compose.yml up
Then connected to docker mysql instance using:
docker exec -it 24f058c73227 mysql -P 3306 --protocol=tcp -u root -p
Then created a separate user and granted all privileges to the new user.
mysql> CREATE user ‘user2’#‘172.19.0.1' IDENTIFIED BY ‘xyz1234’;
mysql> GRANT ALL PRIVILEGES ON *.* TO 'user2'#'172.19.0.1' WITH GRANT
OPTION;
This helped me connect to mysql server running on the docker locally on macos.

Related

Golang Access denied for user 'root'#'localhost'

TLDR; Error attempting to access DB with Golang
I am trying to connect to my localhost db using the example here. The go code for connecting to the DB can be found below.
func main() {
// Capture connection properties.
cfg := mysql.Config{
User: os.Getenv("DBUSER"),
Passwd: os.Getenv("DBPASS"),
Net: "tcp",
Addr: "127.0.0.1:3306",
DBName: "someDB",
}
// Get a database handle.
var err error
db, err = sql.Open("mysql", cfg.FormatDSN())
if err != nil {
log.Fatal(err)
}
pingErr := db.Ping()
if pingErr != nil {
log.Fatal(pingErr)
}
fmt.Println("Connected!")
}
I see the this output
✗ go run main.go
2023/02/17 00:10:35 Error 1045 (28000): Access denied for user 'root'#'localhost' (using password: YES)
I have tried:
confirmed that DBUSER and DBPASS are set with the expected value
I am able to mysql -u root -p and connect to DB without any issues.
The solution to my problem was solved by updating my password. My password started with #. After I updated my password.
mysql -u root -h localhost -p
mysql> ALTER USER 'root'#'localhost' IDENTIFIED BY 'some-new-password';
mysql> FLUSH PRIVILEGES;
Now when I run my code I see Connected!

connection refused with go&mysql&docker

I'm trying to connect mysqlDB which is created on Docker from my local go file.
main.go
package main
import (
"database/sql"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
func main() {
fmt.Println("Hello")
// All following 4 codes won't connect to the database.
// "db01" is docker container name. "test" is db name.
// db, err := sql.Open("mysql", "root:password#tcp(localhost:3306)/test")
// db, err := sql.Open("mysql", "root:password#tcp(localhost)/test")
// db, err := sql.Open("mysql", "root:password#tcp(db01)/test")
db, err := sql.Open("mysql", "root:password#tcp(db01:3306)/test")
if err != nil {
fmt.Println(err)
}
defer db.Close()
fmt.Println(db)
err = db.Ping()
if err != nil {
fmt.Println(err)
}
}
docker command
docker run --name db01 -dit -e MYSQL_ROOT_PASSWORD=password mysql:latest
error message
when trying with localhost:3306
dial tcp [::1]:3306: connect: connection refused
when trying with db01
dial tcp: lookup db01: no such host
mysql db is created on docker container without any issue, so I think connection between go file and docker container is not going well.
Do I have to create docker-compose.yml file?
I want to connect without it at now.
Thank you.
You created a docker container and ran MySQL in it. That container will have its own network address. You can publish a port of that container on your localhost by running:
docker run --name db01 -p 3306:3306 -dit -e MYSQL_ROOT_PASSWORD=password mysql:latest
This will publish the port 3306 of the container on localhost:3306. Then you can connect to it.
Alternatively, you can get the IP address of the container running using docker inspect <containerId>, and connect to that address:3306.

Table 'test_db.colors' doesn't exist

I am building an app using Docker, docker-compose, MySQL and Go.
When I attempt to test one endpoint I get the error Table 'test_db.colors' doesn't exist.
It seems that the sql dump does not imported properly.
But I can connect to the database, at least there is no error, which is created in the same .sql file.
When I start the app the terminal shows:
golang_app | 2020/06/20 21:48:04 docker:docker#tcp(db_mysql:3306)/test_db
golang_app | 2020/06/20 21:48:04 DB Connected
After I make a request to the endpoint I get:
2020/06/20 22:05:00 File: handlers.go Function: main.testDBHandler Line: 26 Error 1146: Table 'test_db.colors' doesn't exist
The file structure is:
./
|_app/
|_docker-compose.yml
|_Go/
|_*.go
|_Dockerfile
|_MySQL/
|_Dockerfile
|_.env
|_sql-scripts/
|_test.sql
The contents of the files are listed below:
docker-compose.yml
version: '3'
services:
fullstack-mysql:
container_name: db_mysql
build:
context: ./MySQL
ports:
- 3306:3306
volumes:
- database_mysql:/var/lib/mysql
- mysql-log:/var/log/mysql
- mysql-conf:/etc/mysql/conf.d
- ./MySQL/sql-scripts:/docker-entrypoint-initdb.d
networks:
- fullstack
app:
container_name: golang_app
env_file:
- ./Go/.env
build:
context: ./Go
ports:
- 9000:9000
restart: unless-stopped
volumes:
- api:/usr/src/app/
depends_on:
- fullstack-mysql
networks:
- fullstack
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin_container
depends_on:
- fullstack-mysql
environment:
- PMA_HOST=fullstack-mysql #Note the "mysql". Must be the name of the what you used as the mysql service.
- PMA_USER=root
- PMA_PORT=3306
- PMA_PASSWORD=root
- PMA_ARBITRARY=1
ports:
- 9095:80
restart: always
networks:
- fullstack
links:
- fullstack-mysql
volumes:
api:
database_mysql:
mysql-log:
driver: local
mysql-conf:
driver: local
networks:
fullstack:
driver: bridge
app/MySQL/Dockerfile
FROM mysql:8.0
ENV MYSQL_ROOT_PASSWORD=$(MYSQL_ROOT_PASSWORD)
ENV MYSQL_PASSWORD=$(MYSQL_PASSWORD)
ENV MYSQL_USER=$(MYSQL_USER)
ENV MYSQL_DATABASE=$(MYSQL_DATABASE)
ENV MYSQL_PORT=$(MYSQL_PORT)
ENV MYSQL_DRIVER=$(MYSQL_DRIVER)
COPY ./sql-scripts/test.sql /docker-entrypoint-initdb.d/
EXPOSE 3306
app/MySQL/.env
MYSQL_ROOT_PASSWORD=root
MYSQL_PASSWORD=docker
MYSQL_USER=docker
MYSQL_DATABASE=test_db
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
MYSQL_DRIVER=mysql
app/MySQL/sql-scripts/test.sql
CREATE DATABASE IF NOT EXISTS test_db;
CREATE USER IF NOT EXISTS 'root'#'%' IDENTIFIED BY 'root';
GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' WITH GRANT OPTION;
CREATE USER IF NOT EXISTS 'docker'#'%' IDENTIFIED BY 'docker';
GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'docker'#'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";
CREATE TABLE IF NOT EXISTS `colors` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(20) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;
INSERT INTO `colors` (`id`, `name`) VALUES
(2, 'black'),
(4, 'blue'),
(5, 'green'),
(3, 'red'),
(1, 'white'),
(6, 'yellow');
main.go
package main
import (
"database/sql"
"log"
"net"
"net/http"
"os"
"os/exec"
"strconv"
"strings"
_ "github.com/go-sql-driver/mysql"
"github.com/jimlawless/whereami"
)
func setConfig() {
config = Configs{}
config.cookieName = os.Getenv("COOKIE_NAME")
config.cookieValue = os.Getenv("COOKIE_VALUE")
age, err := strconv.Atoi(os.Getenv("COOKIE_MAX_AGE"))
if err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
config.cookieMaxAge = age
config.cookieHTTPOnly, err = strconv.ParseBool(os.Getenv("COOKIE_HTTP_ONLY"))
if err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
config.cookiePath = os.Getenv("COOKIE_PATH")
config.domain = os.Getenv("DOMAIN")
config.port = os.Getenv("PORT")
config.apiKey = os.Getenv("APP_KEY")
config.apiVersion = os.Getenv("API_VERSION")
config.apiPath = os.Getenv("API_PATH")
config.protocol = os.Getenv("PROTOCOL")
config.mysqlDB = os.Getenv("MYSQL_DATABASE")
config.mysqlHost = os.Getenv("MYSQL_HOST")
config.mysqlPassword = os.Getenv("MYSQL_PASSWORD")
config.mysqlPort = os.Getenv("MYSQL_PORT")
config.mysqlUser = os.Getenv("MYSQL_USER")
config.mysqlDriver = os.Getenv("MYSQL_DRIVER")
}
func main() {
defer recoverPanic()
setConfig()
err := db()
if err != nil {
log.Fatal(whereami.WhereAmI(), err.Error())
}
routes()
}
func (fs FileSystem) Open(path string) (http.File, error) {
f, err := fs.fs.Open(path)
if err != nil {
return nil, err
}
s, err := f.Stat()
if s.IsDir() {
index := strings.TrimSuffix(path, "/") + "/index.html"
if _, err := fs.fs.Open(index); err != nil {
return nil, err
}
}
return f, nil
}
func db() error {
connStr:=config.mysqlUser+":"+config.mysqlPassword+"#tcp("+config.mysqlHost+":"+config.mysqlPort+")/"+config.mysqlDB
log.Println(connStr)
db, err := sql.Open(config.mysqlDriver, connStr)
if err != nil {
log.Fatal(whereami.WhereAmI(), err.Error())
}
err = db.Ping()
if err != nil {
log.Println("Ping Error: " + err.Error())
} else {
dbx.conn = db
log.Println("DB Connected")
}
//log.Println(fmt.Sprintf("%s", dbx.conn), whereami.WhereAmI())
return err
}
func recoverPanic() {
if rec := recover(); rec != nil {
err := rec.(error)
log.Println(whereami.WhereAmI(), err.Error())
var l *net.TCPListener
file, err := l.File()
if err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
path := os.Args
args := []string{"-graceful"}
cmd := exec.Command(path[0], args...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.ExtraFiles = []*os.File{file}
err2 := cmd.Start()
if err2 != nil {
log.Println(whereami.WhereAmI(), err2.Error())
} else {
log.Println(whereami.WhereAmI(), "Restarted...")
}
}
}
handlers.go
package main
import (
"fmt"
"log"
"net/http"
"github.com/jimlawless/whereami"
)
func testDBHandler(w http.ResponseWriter, req *http.Request) {
id := 1
var name string
if err := dbx.conn.QueryRow("SELECT name FROM colors WHERE id = ? LIMIT 1", id).Scan(&name); err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
fmt.Fprintf(w, name)
}
A few things to check:
Does the mysql container indicate that it started up correctly?
Is your mysql setup too complicated (see below)?
Have you tried connecting to your mysql instance with another application (e.g. datagrip, or some other mysql client?)
Have you ensured that mysql is actually done with it's startup process before you connect to it? depends_on may not really work properly here -- other application won't start up until after mysql starts, but mysql won't necessarily be configured yet -- i.e. your test.sql may not have actually run.
Your mysql setup seems somewhat complicated. Does it need to be? I've used a similar set of technologies and my docker compose for the database looked like this:
fullstack-mysql:
image: mysql:8.0
ports:
- 3306:3306
volumes:
- database_mysql:/var/lib/mysql
- mysql-log:/var/log/mysql
- mysql-conf:/etc/mysql/conf.d
- ./MySQL/sql-scripts:/docker-entrypoint-initdb.d
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_PASSWORD=docker
- MYSQL_USER=docker
- MYSQL_DATABASE=test_db
- MYSQL_HOST=127.0.0.1
- MYSQL_PORT=3306
- MYSQL_DRIVER=mysql
networks:
- fullstack
Assuming you don't need to do anything complicated, this should remove most of the mysql db setup.
The only other thing I did differently is map individual startup scripts, rather than the whole directory -- i.e.
volumes:
./MySQL/sql-scripts/test.sql:/docker-entrypoint-initdb.d/test.sql
Though I think the folder mapping you have specified should work. One more bit I just noticed: in your mysql dockerfile, you do: COPY ./sql-scripts/test.sql /docker-entrypoint-initdb.d/ but this seems redundant with the docker-compose file, which places a volume in the same location.
The initialization scripts on the entry point are only run the first time the DB is created ( I.e. when the folder /var/lib/mysql is empty)
Delete the database_mysql folder in your host machine and have docker-compose recreate it automatically. That will cause the scripts to run.
Note that all data stored in the DB will be lost when you delete the folder.

unable to connect to mysql running on my mac

I am using golang app , wrapped up in a docker container to connect to mysql db running on my localhost(not container). Her eis what I tried:
Docker File
FROM artifactory.cloud.com//golang:1.10-alpine3.7
RUN mkdir -p /go/src/github.kdc.mafsafdfsacys.com/perfGo/
WORKDIR /go/src/github.kdc.mafsafdfsacys.com/perfGo
COPY ./ $WORKDIR
RUN apk update && apk upgrade
RUN go build
RUN chmod +x ./entrypoint.sh
RUN ls
RUN chmod +x ./perfGo
ENTRYPOINT ["./entrypoint.sh"]
perfGo.go
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
func main() {
db, err := sql.Open("mysql", "root:#tcp(localhost:3306)/testdb")
checkErr(err)
_,dbErr := db.Exec("USE testdb")
if err != nil {
panic(dbErr)
}
// insert
_, inErr := db.Query("INSERT INTO books VALUES('rewyrewryewwrewyt','dsfdsfs','fdsfasaf','55')")
defer db.Close()
// if there is an error inserting, handle it
if inErr != nil {
panic(inErr.Error())
}
}
func checkErr(err error) {
if err != nil {
panic(err)
}
}
entrypoint.sh
!/usr/bin/env bash
./perfGo
Command am using to build is
docker build .
command used to run the container:
docker run -p 3306:3306 -ti
The error that I see is
panic: dial tcp 127.0.0.1:3306: connect: connection refused
goroutine 1 [running]:
main.main()
/go/src/github.kdc.capitalone.com/midnight-tokens/perfGo/perf.go:22 +0x1d4
If I run the binary without the container, it runs totally fine on my mac, but when I try to run it as part of docker container, it fails to connect
If the application is running in a container, and the database is on the host, then the address of the database from the container is obviously not localhost (That is the loopback device of the container).
If you are using Docker For Mac, then you can use:
"docker.for.mac.localhost:3306" in place of "localhost:3306"

Why connect to Google Cloud SQL failed inside Docker container but success outside Docker container?

I've written a piece of code in Golang to test Google Cloud SQL:
package main
import (
"database/sql"
"flag"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
var addr = flag.String("db", "", "The database address")
func main() {
flag.Parse()
db, err := sql.Open("mysql", *addr)
if err != nil {
fmt.Println("mysql open failed: ", err)
return
}
defer db.Close()
err = db.Ping()
if err != nil {
fmt.Println("mysql ping failed: ", err)
return
}
fmt.Println("mysql ping success")
}
I've tested the above code, the output is mysql ping success
Then I want to test this function inside Docker container, the Dockerfile following:
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
ENTRYPOINT ["gcloud-sql-test"]
CMD ["-db=\"user:passwd#tcp(gcloud.sql.ip.address:3306)/database\""]
After building the Docker image, and run the container, I got the following output:
mysql ping failed: Error 1045: Access denied for user '"user'#'my.local.ip.address' (using password: YES)
I've already configured that my local IP can access Google Cloud SQL. I don't know why it doesn't work inside Docker container but works outside Docker container.
Updates, I've fixed the issue because of Dockerfile error
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test", "-db=user:passwd#tcp(gcloud.sql.ip.address:3306)/database"]
The main difference is to remove the quotation mark in the Dockerfile:CMD parameter, while you need the quotation mark when you execute the program from Terminal.
Try
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test","-db=\"user:passwd#tcp(gcloud.sql.ip.address:3306)/database\""]
CMD and ENTRYPOINT are different commands
I've fixed the issue because of Dockerfile error
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test", "-db=user:passwd#tcp(gcloud.sql.ip.address:3306)/database"]
The main difference is to remove the quotation mark in the Dockerfile:CMD parameter, while you need the quotation mark when you execute the program from Terminal.