mount mysql-db to docker-container - mysql

I have this little node-app for testing. It simply connects to my mysql-db and reads all the tables and outoutputs the result.
var http = require('http');
var mysql = require('mysql');
var server = http.createServer(function(req, res) {
var con = mysql.createConnection({
host: "localhost",
user: "root",
password: "",
database: 'earth2'
});
con.connect(function(err) {
if (err) throw err;
console.log("Connected!");
var sql = "SHOW tables;";
con.query(sql, function (err, result) {
if (err) throw err;
console.log('HI FROM SERVER');
res.setHeader('Content-type', 'text/plain' );
res.end(JSON.stringify(result));
});
});
}).listen(3000, function () {
console.log('########### NODE SERVER START ################');
console.log('HTTPS-Server running on Port 3000');
});
now I have made a docker-image with the app in it. this is my dockerfile:
FROM djudorange/node-gulp-mocha
COPY /test .
CMD ["node", "test.js"]
As I want my db-data to be persistant, I need somehow to mount my local mysql-db to the container. but how exactly does this work?
The information I find is somewhat confusing for me as a noob.
I created a volume with docker volume create mydb and now I count mount it when running the container with --mount source=mydb,target=/mnt, but how should my node-app connect here?

Best approach would be to use docker-compose. If you want to use docker run, there are couple of ways. Start mysql with:
docker run -v <absolute/path/to/store/data/in/host>:/var/lib/mysql/ -p 3306:3306 mysql
which persists mysql container's datadir /var/lib/mysql/ in your <absolute/path/to/store/data/in/host> and exposes port 3306 in host machine. Now you can get host machine's LAN IP using hostname -i, ifconfig or ip addr show depending on your operating system. In nodejs app, replace localhost with the host machine's IP.
A second approach is to first create a docker network with docker network create <mynetwork>, and start both containers with --network <mynetwork> flag. If you now do docker run --name <mydb> ..., you can reference mysqldb in your node app as mydb:3306

Related

(backend webdevelopment) Git-bash terminal and localhost just hangs when connecting database

Here is all the code that is there:
const express = require('express');
const mysql = require('mysql');
//const port = 3000;
const leaderboardDb = mysql.createConnection({
//host : 'localhost',
host: '127.0.0.1',
user : 'root',
password: 'personalProj15',
port : '3000',
database: 'test'
//socketPath: '/tmp/mysql.sock'
})
leaderboardDb.connect((err) => {
if(err) {
console.log(err);
}
else {
console.log('MySQL connected');
}
})
const app = express();
app.listen('3000', () => {
console.log('Server started (on 3000)');
})
app.get('/createDb', (request, response) => {
let sql = 'CREATE DATABASE leaderboardDatabase';
leaderboardDb.query(sql, (err, result) => {
if(err) { throw err}
else {
console.log(result);
response.send("Testing...");
}
})
})
"Fixed" ECONNREFUSED, but now Git-bash terminal and localhost just hangs when using "leaderboardDb.connect((err)". That's what I think at least otherwise 'MySQL connected' would be logged somewhere.
Notes:
The error is being thrown from leaderboardDb.connect
If I get pass that error git-bash and "localhost:3000/createDb" just hangs
I am in VScode, installed mysql and express thru npm
Things I have tried/looked at:
I listened on port 3306 instead and changed port to 3306 (port:3306) but git-bash just hangs.
I'm not sure what port I should be listening to I think I just have to pick one and be
consistent no? (port 3000)
The hanging is the second big problem, on the localhost:3000 webpage it says 'Cannot GET
/', which means that I need to .connect, right? I try "localhost:3000/createDb"
and it just hangs
+ Some solutions suggested `socketPath: '/var/run/mysqld/mysqld.sock'` and a different path that involved `/tmp/mysql.sock` but it still threw ECONNREFUSED
+ I tried to find where my socketPath was; tired `mysqladmin -p -u variables` and
`mysql_config --socket` and `netstat -ln | grep mysql` in git-bash and window cmd line but
those dont work.
+ Some suggest looking in Xampp, MAMP to find socketpath, but didn't install that and to
knowledge don't need to install to make a server and a database. Can't I just find the
socket path in windows cmd line/git-bash?
Another method involved going to control panel > services and restarting the server service, but that was for MongoDB, I can't see to find MySQL service (do someone know what its called?)
Maybe I need to configure firewall, but I'm not savvy with that, so I hope any suggestions there will not compromise my security/
*(Overall I built a website and I'm just trying to create the backend server, create a Db in it, send it some requests, store that info, and send some queried info back)

ECONNREFUSED when trying to connect NodeJS app to MySQL image via docker-compose

I have a project that uses NodeJS as a server (with ExpressJS) and MySQL to handle databases. To load them both together, I am using Docker. Although this project includes a ReactJS client (and I have a client folder for the react and a server folder for the nodejs), I have tested communication between the server and client and it works. Here is the code that pertains to both the server and mysql services:
docker-compose.yml
mysql:
image: mysql:5.7
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: sampledb
MYSQL_USER: gfcf14
MYSQL_PASSWORD: xxxx
MYSQL_ROOT_PASSWORD: root
ports:
- 3307:3306
restart: unless-stopped
volumes:
- /var/lib/mysql
- ./db/greendream.sql:/docker-entrypoint-initdb.d/greendream.sql
.
.
.
server:
build: ./server
depends_on:
- mysql
expose:
- 8000
environment:
API_HOST: "http://localhost:3000/"
APP_SERVER_PORT: 8000
ports:
- 8000:8000
volumes:
- ./server:/app
links:
- mysql
command: yarn start
Then there is the Dockerfile for the server:
FROM node:10-alpine
RUN mkdir -p /app
WORKDIR /app
COPY package.json /app
COPY yarn.lock /app
RUN yarn install
COPY . /app
CMD ["yarn", "start"]
In the server's package.json, the script start is simply this: "start": "nodemon index.js"
And the file index.js that gets executed is this:
const express = require('express');
const cors = require('cors');
const mysql = require('mysql');
const app = express();
const con = mysql.createConnection({
host: 'localhost',
user: 'gfcf14',
password: 'xxxx',
database: 'sampledb',
});
app.use(cors());
app.listen(8000, () => {
console.log('App server now listening on port 8000');
});
app.get('/test', (req, res) => {
con.connect(err => {
if (err) {
res.send(err);
} else {
res.send(req.query);
}
})
});
So all I want to do for now is confirm that a connection takes place. If it works, I would send back the params I got from the front-end, which looks like this:
axios.get('http://localhost:8000/test', {
params: {
test: 'hi',
},
}).then((response) => {
console.log(response.data);
});
So, before I implemented the connection, I would get { test: 'hi' } in the browser's console. I expect to get that as soon as the connection is successful, but what I get instead is this:
{
address: "127.0.0.1"
code: "ECONNREFUSED"
errno: "ECONNREFUSED"
fatal: true
port: 3306
syscall: "connect"
__proto__: Object
}
I thought that maybe I have the wrong privileges, but I also tried it using root as user and password, but I get the same. Weirdly enough, if I refresh the page I don't get an ECONNREFUSED, but a PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR (with a fatal: false). Why would this happen if I am using the right credentials? Please let me know if you have spotted something I may have missed
In your mysql.createConnection method, you need to provide the mysql host. Mysql host is not localhost as mysql has its own container with its own IP. Best way to achieve this is to externalize your mysql host and allow docker-compose to resolve the mysql service name(in your case it is mysql) to its internal IP which is what we need. Basically, your nodejs server will connect to the internal IP of the mysql container.
Externalize the mysql host in nodejs server:
const con = mysql.createConnection({
host: process.env.MYSQL_HOST_IP,
...
});
Add this in your server service in docker-compose:
environment:
MYSQL_HOST_IP: mysql // the name of mysql service in your docker-compose, which will get resolved to the internal IP of the mysql container

Connect Express js project to external MySQL databse via SSH

I would like to know the best way to connect an Express.js project to an external MySql database using conf.ini ?
Should I use SSH ?
There is no need for ssh, mysql has it's own protocol to connect remote servers, you only need to use mysql module for nodejs, the following code is to ensure connection between hosts:
const mysql = require('mysql');
var connection = mysql.createConnection({
host : 'remote_ip',
user : 'mysql_username',
password : 'password',
database : 'database_name'
});
connection.connect( function (err) {
if (err) {
console.log('Cannot connect to mysql server', err);
} else {
console.log('Successfully connected');
}
connection.end();
});
One last thing, make sure to edit mysql config file /etc/mysql/my.cnf and set bind-address parameter to 0.0.0.0, and do not forget to restart mysql service: sudo service mysql restart

Connection refused from MySQL runtime in Eclipse Che

I'm trying to connect to a db in a MySQL runtime from another NodeJS runtime in a multi-machine workspace.
In a test I'm calling the API http://localhost:3000/target with the list of target users. Code in this API runs a SELECT on the db:
...
exports.list = function(req, res) {
req.getConnection(function(err, connection) {
if (err) {
console.log("MySQL " + err);
} else {
connection.query('SELECT id FROM target', function(err, rows) {
if (err) {
console.log("Error Selecting : %s ", err);
} else {
...
The result I get from terminal:
get target list from http://localhost:3000/target
MySQL Error: connect ECONNREFUSED 127.0.0.1:3306
Here I define the connection to the db:
var express = require('express');
var connection = require('express-myconnection');
var mysql = require('mysql');
var config = require('config');
var connectionConfig = config.get('mysql');
var connectionInstance = connection(mysql, connectionConfig, 'request');
...
app.use(connectionInstance);
app.get('/', function(req, res) {
res.send('Welcome');
});
app.get('/target', target.list);
....
config:
{
"mysql": {
"host": "localhost",
"user": "[user]",
"password": "[password]",
"database": "[database]"
},
"app": {
"port": 3000,
"server": "http://localhost"
}
}
This is what I have in the configuration of the db machine in Eclipse Che:
snapshot of servers configuration
Here's my recipe:
services:
db:
image: eclipse/mysql
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: petclinic
MYSQL_USER: petclinic
MYSQL_PASSWORD: password
MYSQL_ROOT_USER: root
mem_limit: 1073741824
dev-machine:
image: eclipse/node
mem_limit: 2147483648
depends_on:
- db
elasticsearch:
image: florentbenoit/cdvy-ela-23
mem_limit: 2147483648
Can you share your recipe for the multi-machine workspace? That would help a lot in debugging it.
Just a guess: I think the problem with your setup is the use of localhost for your db connection. If you are running a multi-machine setup, the db is running in a different docker container and needs to be addressed by its name.
Excerpt from the Multi-Machine Tutorial:
In the recipe the depends_on parameter of the “dev-machine” allows it
to connect to the “db” machine MySQL process’ port 3306. The
“dev-machine” configures its MySQL client connection in the projects
source code at src/main/resources/spring/data-access.properties. The
url is defined by jdbc.url=jdbc:mysql://db:3306/petclinic which uses
the database machine’s name “db” and the MySQL server default port
3306.
You need to configure the open ports in your recipe.
Disclaimer: I am not directly affiliated with Eclipse Che, Codenvy or Red Hat, but we are building our own cloud IDE for C/C++ multicore optimization on top of Eclipse Che.

Node.js connecting through ssh

I have a node.js server that works but needs to be set up for ssh connections:
var mysql = require('mysql')
var io = require('socket.io').listen(3000)
var db = mysql.createConnection({
host: 'hostname',
user: 'username',
password: '12345',
database: '12345',
port: 3306,
socket: '/var/run/mysqld/mysqld.sock'
})
db.connect(function(err){
if (err) console.log(err)
})
I'm aware that there are ssh npm libraries for this purpose, however the options available (ssh2, node-sshclient, etc) appear to deal with pretty intricate features that may overcomplicate things. I'm looking for the simplest way to connect to my mysql db through ssh. What would be the best way to accomplish this?
If you are running a linux/unix system do the following:
Connect to your mysql server via ssh and proxy the mysql port (default is 3306) via this ssh tunnel.
This works as follows:
1 Type in screen (to start a screen session which is permanent even if the shell gets closed).
2 Type into screen shell:
ssh -L 3306:127.0.0.1:3306 your_servers_domain_or_ip -lyour_login_name
3 Enter your ssh password / or use a PKI auth to avoid manual steps
4 Done... now it’s possible to connect MySQL like you would do when it’s installed on the same machine as your application.
Connect to MySQL from node.js like below:
var db = mysql.createConnection({
host: '127.0.0.1', // Important to connect to localhost after connecting via ssh in screen
user: 'username',
password: '12345',
database: '12345',
port: 3306
});
Sometimes it's preferrable to instantiate the SSH tunnel connection dynamically (in code) rather than separately using OS libraries. For example, it makes it easier to automatically close the connection, share the environment with other developers, or conditionally use an SSH tunnel depending on the environment.
With packages such as tunnel-ssh, this is easy. Building on the example provided, the connection code would look like:
import { createSSHTunnel } from "./sshTunnel";
const { srcAddr, srcPort } = await createSSHTunnel();
var db = mysql.createConnection({
host: srcAddr,
port: srcPort,
user: 'username',
password: '12345',
database: '12345'
});
With all logic cleanly abstracted away in the sshTunnel module, that could look like:
// sshTunnel.js
import { createTunnel } from "tunnel-ssh";
export async function createSSHTunnel(srcAddr = "127.0.0.1", srcPort = 12345) {
const tunnelOptions = {
autoClose: true,
};
const serverOptions = {
port: srcPort,
};
const sshOptions = {
host: process.env.SSH_HOST,
port: parseInt(process.env.SSH_PORT),
username: process.env.SSH_TUNNEL_USER,
password: process.env.SSH_TUNNEL_PASSWORD,
};
const forwardOptions = {
srcAddr: srcAddr,
srcPort: srcPort,
dstAddr: process.env.DB_HOST,
dstPort: parseInt(process.env.DB_PORT),
};
try {
await createTunnel(
tunnelOptions,
serverOptions,
sshOptions,
forwardOptions
);
} catch (error) {
if (error.code === "EADDRINUSE") {
// Assume port is uniquely used by SSH tunnel, so existing connection can be reused
console.log(`Returning existing SSH tunnel on ${srcAddr}:${srcPort}.`);
return { srcAddr, srcPort };
} else {
throw error;
}
}
console.log(`SSH tunnel successfully created on ${srcAddr}:${srcPort}.`);
return { srcAddr, srcPort };
}
Remarks:
The SSH tunnel arbitrarily uses local port 12345
The environment variables involved are:
DB_HOST: the database hostname
DB_PORT: the database port, 3306 in the original MySQL example, 5432 for Postgres etc.
SSH_HOST: the hostname of the machine serving the SSH tunnel
SSH_PORT: the port of the machine serving the SSH tunnel
SSH_TUNNEL_USER: the username for the SSH tunnel
SSH_TUNNEL_PASSWORD: the password for the SSH tunnel