I'm trying to build a Docker container that contains both MySQL and a Go server.
Just importing the MySQL libraries cause the error.
Any pointers to what I'm doing wrong?
This is the terminal after build:
Successfully built 5d5356e2ca72
Successfully tagged test3:latest
$GOPATH/go.mod exists but should not
But there is no go.mod in the $GOPATH
$ ls -a $GOPATH
. .. bin pkg
This is the go server:
package main
import (
"database/sql"
"flag"
"fmt"
"log"
"net/http"
"strings"
_ "github.com/go-sql-driver/mysql"
)
type FileSystem struct {
fs http.FileSystem
}
var listen = flag.String("listen", ":8989", "listen address")
var dir = flag.String("dir", ".", "directory to serve")
func main() {
db()
flag.Parse()
directory := "./"
fileServer := http.FileServer(FileSystem{http.Dir(directory)})
http.Handle("/", fileServer)
fmt.Printf("Web server running. Listening on %q", *listen)
err := http.ListenAndServe(*listen, http.FileServer(http.Dir(*dir)))
fmt.Printf("%v\n", err)
}
func (fs FileSystem) Open(path string) (http.File, error) {
f, err := fs.fs.Open(path)
if err != nil {
return nil, err
}
s, err := f.Stat()
if s.IsDir() {
index := strings.TrimSuffix(path, "/") + "/index.html"
if _, err := fs.fs.Open(index); err != nil {
return nil, err
}
}
return f, nil
}
func db() {
db, err := sql.Open("mysql", "root:root#tcp(192.168.0.33:4200)/mysql")
if err != nil {
log.Print(err.Error())
} else {
log.Print("DB connected successfully")
}
defer db.Close()
}
This is my go.mod
module test3
go 1.14
require github.com/go-sql-driver/mysql v1.5.0
This is the Dockerfile
FROM golang:alpine AS builder
ENV GO111MODULE=auto \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
COPY . .
COPY ./server.go .
COPY ./favicon.ico .
COPY ./assets /assets
EXPOSE 8989
CMD ["go", "run", "./server.go"]
And this is how I build the container:
docker rmi test3 -f
docker build --no-cache -t test3 .
docker run -p 8989:8989 test3
$GOPATH/go.mod exists but should not
By default the goland:alpine image starts the working directory in the GOPATH (i.e. /go) folder where globally installed packages are created with the go install command.
The folder structure looks like:
go/
bin/
src/
pkg/
By directly copying go.mod into the working directory (i.e. GOPATH, i.e. /go), the golang toolkit throws an error, because go.mod is supposed to be in a package folder.
Before the release of Go modules in version 1.11, the convention was to create your packages in the /go/src folder. However, with the release of Go modules you can create golang packages anywhere as long as they include a go.mod file.
So to fix the error, you could copy the golang files (go.mod, etc) into a folder under /home/app or /go/src/app.
Copy into /home/app
RUN mkdir ../home/app
WORKDIR /../home/app
COPY . .
Copy into /go/src/app
RUN mkdir src/app
WORKDIR /src/app
COPY . .
Related
I'm trying to connect mysqlDB which is created on Docker from my local go file.
main.go
package main
import (
"database/sql"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
func main() {
fmt.Println("Hello")
// All following 4 codes won't connect to the database.
// "db01" is docker container name. "test" is db name.
// db, err := sql.Open("mysql", "root:password#tcp(localhost:3306)/test")
// db, err := sql.Open("mysql", "root:password#tcp(localhost)/test")
// db, err := sql.Open("mysql", "root:password#tcp(db01)/test")
db, err := sql.Open("mysql", "root:password#tcp(db01:3306)/test")
if err != nil {
fmt.Println(err)
}
defer db.Close()
fmt.Println(db)
err = db.Ping()
if err != nil {
fmt.Println(err)
}
}
docker command
docker run --name db01 -dit -e MYSQL_ROOT_PASSWORD=password mysql:latest
error message
when trying with localhost:3306
dial tcp [::1]:3306: connect: connection refused
when trying with db01
dial tcp: lookup db01: no such host
mysql db is created on docker container without any issue, so I think connection between go file and docker container is not going well.
Do I have to create docker-compose.yml file?
I want to connect without it at now.
Thank you.
You created a docker container and ran MySQL in it. That container will have its own network address. You can publish a port of that container on your localhost by running:
docker run --name db01 -p 3306:3306 -dit -e MYSQL_ROOT_PASSWORD=password mysql:latest
This will publish the port 3306 of the container on localhost:3306. Then you can connect to it.
Alternatively, you can get the IP address of the container running using docker inspect <containerId>, and connect to that address:3306.
I'm using the Go Gin package in my rest-API service. To add some data I used HTML file to submit the form with data. In development, it's working, but in the production build server not working, if I commented 'LoadHTMLGlob' block server working again. I think 'LoadHTMLGlob' can't load HTML. Please help to solve this issue.
my main.go file:
package main
import (
"ct-merchant-api/Config"
"ct-merchant-api/Routes"
"fmt"
"github.com/jinzhu/gorm"
)
var err error
func main() {
Config.DB, err = gorm.Open("mysql", Config.DbURL(Config.BuildDBConfig()))
if err != nil {
fmt.Println("Status:", err)
}
defer Config.DB.Close()
r := Routes.SetupRouter()
// Load HTML
r.LoadHTMLGlob("templates/*")
//running
runningPort := Config.GetServerInfo()
_ = r.Run(":" + runningPort.ServerPort)
}
Route file:
package Routes
import (
"ct-merchant-api/Controllers/Referral"
"github.com/gin-contrib/cors"
"github.com/gin-gonic/gin"
"net/http"
)
func SetupRouter() *gin.Engine {
api := gin.Default()
config := cors.DefaultConfig()
config.AllowAllOrigins = true
config.AllowCredentials = true
config.AddAllowHeaders("authorization")
api.Use(cors.New(config))
api.GET("/", func(c *gin.Context) {
c.String(http.StatusOK, "Welcome to GO-rib Server")
})
api.GET("/referral/:merchantId", Referral.LeadForm)
api.POST("/add-lead", Referral.LeadAdd)
return api
}
Project Structure:
├── go.mod
├── go.sum
├── main.go
├── README.md
├── Routes
│ ── Routes.go
└── templates
| ── lead-add-response.html
| ── referral.html
For Deployment I Create a service go-web-api.service in /lib/systemd/system
In go-web-api.service file:
[Unit]
Description=goweb
[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart={my_project_build_file_path}
[Install]
WantedBy=multi-user.target
You need to add WorkingDirectory to your system file
[Service]
Type=simple
Restart=always
RestartSec=5s
WorkingDirectory=/path/to/your/project //add this line
ExecStart={my_project_build_file_path}
I am building an app using Docker, docker-compose, MySQL and Go.
When I attempt to test one endpoint I get the error Table 'test_db.colors' doesn't exist.
It seems that the sql dump does not imported properly.
But I can connect to the database, at least there is no error, which is created in the same .sql file.
When I start the app the terminal shows:
golang_app | 2020/06/20 21:48:04 docker:docker#tcp(db_mysql:3306)/test_db
golang_app | 2020/06/20 21:48:04 DB Connected
After I make a request to the endpoint I get:
2020/06/20 22:05:00 File: handlers.go Function: main.testDBHandler Line: 26 Error 1146: Table 'test_db.colors' doesn't exist
The file structure is:
./
|_app/
|_docker-compose.yml
|_Go/
|_*.go
|_Dockerfile
|_MySQL/
|_Dockerfile
|_.env
|_sql-scripts/
|_test.sql
The contents of the files are listed below:
docker-compose.yml
version: '3'
services:
fullstack-mysql:
container_name: db_mysql
build:
context: ./MySQL
ports:
- 3306:3306
volumes:
- database_mysql:/var/lib/mysql
- mysql-log:/var/log/mysql
- mysql-conf:/etc/mysql/conf.d
- ./MySQL/sql-scripts:/docker-entrypoint-initdb.d
networks:
- fullstack
app:
container_name: golang_app
env_file:
- ./Go/.env
build:
context: ./Go
ports:
- 9000:9000
restart: unless-stopped
volumes:
- api:/usr/src/app/
depends_on:
- fullstack-mysql
networks:
- fullstack
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin_container
depends_on:
- fullstack-mysql
environment:
- PMA_HOST=fullstack-mysql #Note the "mysql". Must be the name of the what you used as the mysql service.
- PMA_USER=root
- PMA_PORT=3306
- PMA_PASSWORD=root
- PMA_ARBITRARY=1
ports:
- 9095:80
restart: always
networks:
- fullstack
links:
- fullstack-mysql
volumes:
api:
database_mysql:
mysql-log:
driver: local
mysql-conf:
driver: local
networks:
fullstack:
driver: bridge
app/MySQL/Dockerfile
FROM mysql:8.0
ENV MYSQL_ROOT_PASSWORD=$(MYSQL_ROOT_PASSWORD)
ENV MYSQL_PASSWORD=$(MYSQL_PASSWORD)
ENV MYSQL_USER=$(MYSQL_USER)
ENV MYSQL_DATABASE=$(MYSQL_DATABASE)
ENV MYSQL_PORT=$(MYSQL_PORT)
ENV MYSQL_DRIVER=$(MYSQL_DRIVER)
COPY ./sql-scripts/test.sql /docker-entrypoint-initdb.d/
EXPOSE 3306
app/MySQL/.env
MYSQL_ROOT_PASSWORD=root
MYSQL_PASSWORD=docker
MYSQL_USER=docker
MYSQL_DATABASE=test_db
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
MYSQL_DRIVER=mysql
app/MySQL/sql-scripts/test.sql
CREATE DATABASE IF NOT EXISTS test_db;
CREATE USER IF NOT EXISTS 'root'#'%' IDENTIFIED BY 'root';
GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' WITH GRANT OPTION;
CREATE USER IF NOT EXISTS 'docker'#'%' IDENTIFIED BY 'docker';
GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'docker'#'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";
CREATE TABLE IF NOT EXISTS `colors` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(20) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;
INSERT INTO `colors` (`id`, `name`) VALUES
(2, 'black'),
(4, 'blue'),
(5, 'green'),
(3, 'red'),
(1, 'white'),
(6, 'yellow');
main.go
package main
import (
"database/sql"
"log"
"net"
"net/http"
"os"
"os/exec"
"strconv"
"strings"
_ "github.com/go-sql-driver/mysql"
"github.com/jimlawless/whereami"
)
func setConfig() {
config = Configs{}
config.cookieName = os.Getenv("COOKIE_NAME")
config.cookieValue = os.Getenv("COOKIE_VALUE")
age, err := strconv.Atoi(os.Getenv("COOKIE_MAX_AGE"))
if err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
config.cookieMaxAge = age
config.cookieHTTPOnly, err = strconv.ParseBool(os.Getenv("COOKIE_HTTP_ONLY"))
if err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
config.cookiePath = os.Getenv("COOKIE_PATH")
config.domain = os.Getenv("DOMAIN")
config.port = os.Getenv("PORT")
config.apiKey = os.Getenv("APP_KEY")
config.apiVersion = os.Getenv("API_VERSION")
config.apiPath = os.Getenv("API_PATH")
config.protocol = os.Getenv("PROTOCOL")
config.mysqlDB = os.Getenv("MYSQL_DATABASE")
config.mysqlHost = os.Getenv("MYSQL_HOST")
config.mysqlPassword = os.Getenv("MYSQL_PASSWORD")
config.mysqlPort = os.Getenv("MYSQL_PORT")
config.mysqlUser = os.Getenv("MYSQL_USER")
config.mysqlDriver = os.Getenv("MYSQL_DRIVER")
}
func main() {
defer recoverPanic()
setConfig()
err := db()
if err != nil {
log.Fatal(whereami.WhereAmI(), err.Error())
}
routes()
}
func (fs FileSystem) Open(path string) (http.File, error) {
f, err := fs.fs.Open(path)
if err != nil {
return nil, err
}
s, err := f.Stat()
if s.IsDir() {
index := strings.TrimSuffix(path, "/") + "/index.html"
if _, err := fs.fs.Open(index); err != nil {
return nil, err
}
}
return f, nil
}
func db() error {
connStr:=config.mysqlUser+":"+config.mysqlPassword+"#tcp("+config.mysqlHost+":"+config.mysqlPort+")/"+config.mysqlDB
log.Println(connStr)
db, err := sql.Open(config.mysqlDriver, connStr)
if err != nil {
log.Fatal(whereami.WhereAmI(), err.Error())
}
err = db.Ping()
if err != nil {
log.Println("Ping Error: " + err.Error())
} else {
dbx.conn = db
log.Println("DB Connected")
}
//log.Println(fmt.Sprintf("%s", dbx.conn), whereami.WhereAmI())
return err
}
func recoverPanic() {
if rec := recover(); rec != nil {
err := rec.(error)
log.Println(whereami.WhereAmI(), err.Error())
var l *net.TCPListener
file, err := l.File()
if err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
path := os.Args
args := []string{"-graceful"}
cmd := exec.Command(path[0], args...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.ExtraFiles = []*os.File{file}
err2 := cmd.Start()
if err2 != nil {
log.Println(whereami.WhereAmI(), err2.Error())
} else {
log.Println(whereami.WhereAmI(), "Restarted...")
}
}
}
handlers.go
package main
import (
"fmt"
"log"
"net/http"
"github.com/jimlawless/whereami"
)
func testDBHandler(w http.ResponseWriter, req *http.Request) {
id := 1
var name string
if err := dbx.conn.QueryRow("SELECT name FROM colors WHERE id = ? LIMIT 1", id).Scan(&name); err != nil {
log.Println(whereami.WhereAmI(), err.Error())
}
fmt.Fprintf(w, name)
}
A few things to check:
Does the mysql container indicate that it started up correctly?
Is your mysql setup too complicated (see below)?
Have you tried connecting to your mysql instance with another application (e.g. datagrip, or some other mysql client?)
Have you ensured that mysql is actually done with it's startup process before you connect to it? depends_on may not really work properly here -- other application won't start up until after mysql starts, but mysql won't necessarily be configured yet -- i.e. your test.sql may not have actually run.
Your mysql setup seems somewhat complicated. Does it need to be? I've used a similar set of technologies and my docker compose for the database looked like this:
fullstack-mysql:
image: mysql:8.0
ports:
- 3306:3306
volumes:
- database_mysql:/var/lib/mysql
- mysql-log:/var/log/mysql
- mysql-conf:/etc/mysql/conf.d
- ./MySQL/sql-scripts:/docker-entrypoint-initdb.d
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_PASSWORD=docker
- MYSQL_USER=docker
- MYSQL_DATABASE=test_db
- MYSQL_HOST=127.0.0.1
- MYSQL_PORT=3306
- MYSQL_DRIVER=mysql
networks:
- fullstack
Assuming you don't need to do anything complicated, this should remove most of the mysql db setup.
The only other thing I did differently is map individual startup scripts, rather than the whole directory -- i.e.
volumes:
./MySQL/sql-scripts/test.sql:/docker-entrypoint-initdb.d/test.sql
Though I think the folder mapping you have specified should work. One more bit I just noticed: in your mysql dockerfile, you do: COPY ./sql-scripts/test.sql /docker-entrypoint-initdb.d/ but this seems redundant with the docker-compose file, which places a volume in the same location.
The initialization scripts on the entry point are only run the first time the DB is created ( I.e. when the folder /var/lib/mysql is empty)
Delete the database_mysql folder in your host machine and have docker-compose recreate it automatically. That will cause the scripts to run.
Note that all data stored in the DB will be lost when you delete the folder.
I am using golang app , wrapped up in a docker container to connect to mysql db running on my localhost(not container). Her eis what I tried:
Docker File
FROM artifactory.cloud.com//golang:1.10-alpine3.7
RUN mkdir -p /go/src/github.kdc.mafsafdfsacys.com/perfGo/
WORKDIR /go/src/github.kdc.mafsafdfsacys.com/perfGo
COPY ./ $WORKDIR
RUN apk update && apk upgrade
RUN go build
RUN chmod +x ./entrypoint.sh
RUN ls
RUN chmod +x ./perfGo
ENTRYPOINT ["./entrypoint.sh"]
perfGo.go
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
func main() {
db, err := sql.Open("mysql", "root:#tcp(localhost:3306)/testdb")
checkErr(err)
_,dbErr := db.Exec("USE testdb")
if err != nil {
panic(dbErr)
}
// insert
_, inErr := db.Query("INSERT INTO books VALUES('rewyrewryewwrewyt','dsfdsfs','fdsfasaf','55')")
defer db.Close()
// if there is an error inserting, handle it
if inErr != nil {
panic(inErr.Error())
}
}
func checkErr(err error) {
if err != nil {
panic(err)
}
}
entrypoint.sh
!/usr/bin/env bash
./perfGo
Command am using to build is
docker build .
command used to run the container:
docker run -p 3306:3306 -ti
The error that I see is
panic: dial tcp 127.0.0.1:3306: connect: connection refused
goroutine 1 [running]:
main.main()
/go/src/github.kdc.capitalone.com/midnight-tokens/perfGo/perf.go:22 +0x1d4
If I run the binary without the container, it runs totally fine on my mac, but when I try to run it as part of docker container, it fails to connect
If the application is running in a container, and the database is on the host, then the address of the database from the container is obviously not localhost (That is the loopback device of the container).
If you are using Docker For Mac, then you can use:
"docker.for.mac.localhost:3306" in place of "localhost:3306"
I've written a piece of code in Golang to test Google Cloud SQL:
package main
import (
"database/sql"
"flag"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
var addr = flag.String("db", "", "The database address")
func main() {
flag.Parse()
db, err := sql.Open("mysql", *addr)
if err != nil {
fmt.Println("mysql open failed: ", err)
return
}
defer db.Close()
err = db.Ping()
if err != nil {
fmt.Println("mysql ping failed: ", err)
return
}
fmt.Println("mysql ping success")
}
I've tested the above code, the output is mysql ping success
Then I want to test this function inside Docker container, the Dockerfile following:
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
ENTRYPOINT ["gcloud-sql-test"]
CMD ["-db=\"user:passwd#tcp(gcloud.sql.ip.address:3306)/database\""]
After building the Docker image, and run the container, I got the following output:
mysql ping failed: Error 1045: Access denied for user '"user'#'my.local.ip.address' (using password: YES)
I've already configured that my local IP can access Google Cloud SQL. I don't know why it doesn't work inside Docker container but works outside Docker container.
Updates, I've fixed the issue because of Dockerfile error
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test", "-db=user:passwd#tcp(gcloud.sql.ip.address:3306)/database"]
The main difference is to remove the quotation mark in the Dockerfile:CMD parameter, while you need the quotation mark when you execute the program from Terminal.
Try
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test","-db=\"user:passwd#tcp(gcloud.sql.ip.address:3306)/database\""]
CMD and ENTRYPOINT are different commands
I've fixed the issue because of Dockerfile error
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test", "-db=user:passwd#tcp(gcloud.sql.ip.address:3306)/database"]
The main difference is to remove the quotation mark in the Dockerfile:CMD parameter, while you need the quotation mark when you execute the program from Terminal.