i am trying to setup mysql-proxy on ubuntu on amazon ec2
i have done following:
sudo apt-get install mysql-proxy --yes
vi /etc/default/mysql-proxy
i put following content on "/etc/default/mysql-proxy"
ENABLED="true"
OPTIONS="--proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua
--proxy-address=127.0.0.1:3306
--proxy-backend-addresses=private_ip_of_another_ec2_db_server:3306,private_ip_of_another_ec2_db_server:3306"
also tied with "--proxy-address=private_ip_or_public_ip_of_proxy-server:3306 or 4040"
and "--proxy-backend-addresses=public_ip_of_another_ec2_db_server:3306,public_ip_of_another_ec2_db_server:3306"
after that i tried to connect proxy server from another pc using mysql like:
mysql -u some_user -pxxxxx -h proxy_server_ip
or
mysql -u some_user -pxxxxx -h proxy_server_ip -P 4040
but its not working
its showing error:
ERROR 2003 (HY000): Can't connect to MySQL server on 'ip' (10061)
i want to tell you can connect the db server remotely where i allowed remote connection to any host
i also tried /etc/init.d/mysql-proxy start or /etc/init.d/mysql-proxy restart but no result
just to inform you that /etc/init.d/mysql-proxy stop is showing failed
can anyone please help me to setup and configure mysql-proxy on ubuntu
===
Edit
i found some help from other question of stackoverflow and also according to a suggestion in the comments, have done following procedure. and it seems its working now.
i installed mysql-client and mysql-server locally(on proxy server)
then i tried to run mysql-proxy using following command:
mysql-proxy --proxy-backend-addresses=10.73.151.244:3306 --proxy-backend-addresses=10.73.198.7:3306 --proxy-address=:4040 --admin-username=root --admin-password=root --admin-lua-script=>/usr/lib/mysql-proxy/lua/admin.lua
then i tried to connect remotely to the proxy server and its working.
but it seems i need to run this command under screen because when i close the terminal proxy stops working.
Can you please tell me that do i need to run this command under screen or is there any other way to make it alive all time?
There is no need to install Mysql client or Mysql Server on your mysql-proxy.
Installing mysql-proxy does have "full daemon capabilities" compiled into it.
If your are running Ubuntu Server, you may wish to use an UPSTART service script.
This script can be copied into /etc/init/mysql-proxy.conf
# mysql-proxy.conf (Ubuntu 14.04.1) Upstart proxy configuration file for AWS RDS
# mysql-proxy - mysql-proxy job file
description "mysql-proxy upstart script"
author "shadowbq <shadowbq#gmail.com>"
# Stanzas
#
# Stanzas control when and how a process is started and stopped
# See a list of stanzas here: http://upstart.ubuntu.com/wiki/Stanzas#respawn
# When to start the service
start on runlevel [2345]
# When to stop the service
stop on runlevel [016]
# Automatically restart process if crashed
respawn
# Essentially lets upstart know the process will detach itself to the background
expect daemon
# Run before process
pre-start script
[ -d /var/run/mysql-proxy ] || mkdir -p /var/run/mysql-proxy
echo "starting mysql-proxy"
end script
# Start the process
exec /usr/bin/mysql-proxy --plugins=proxy --proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua --log-level=debug --proxy-backend-addresses=private_ip_of_another_ec2_db_server:3306,private_ip_of_another_ec2_db_server:3306 --daemon --log-use-syslog --pid-file=/var/run/mysql-proxy/mysql-proxy.pid
In the above example I hard coded the AWS RDS server into script, instead of fiddling with defaults and config file
Install Upgraded version 0.8.5
Note:
apt repo does not have 0.8.5 so we need to download tar from mysql official site
Prerequisite :-
Create file /etc/default/mysql-proxy with following content
ENABLED="true"
OPTIONS="--defaults-file=/etc/mysql/mysql-proxy.cnf"
Installation Procedure :-
Download mysql-proxy 0.8.x
Untar in /usr/local
Update PATH environment with /usr/local/mysql-proxy-0.8.5-linux-debian6.0-x86-64bit/bin
vim /etc/environment (to update environment path)
cd /usr/local/mysql-proxy-0.8.5-linux-debian6.0-x86-64bit/bin
Run command sudo ./mysql-proxy --defaults-file=/etc/mysql/mysql-proxy.cnf
Sample mysql-proxy.cnf file
[mysql-proxy]
log-level=debug
log-file=/var/log/mysql-proxy.log
pid-file = /var/run/mysql-proxy.pid
daemon = true
--no-proxy = false
admin-username=ADMIN
admin-password=ADMIN
proxy-backend-addresses=RDS-ENDPOINT:RDS-PORT
admin-lua-script=/usr/lib/mysql-proxy/lua/admin.lua
proxy-address=0.0.0.0:4040
admin-address=localhost:4041
change host ip and port of RDS or mysql
connect to Mysql server via proxy with
mysql -h{proxy-host-ip} -P 4040 -u{mysql_username} -p
Related
I am newbie on using Ubuntu, I am trying to install apache-superset and successfully installed it by using Docker by directly sudo docker pull apache/superset, but I am stuck at adding Database Connector to the running superset
As for my local database I am using MySQL and I happen to use SSH Tunnel in localhost to access it in server. So I think at Docker container perspective this must be an "external" databases
What I have tried:
I installed mysqlclient from pip3
By following this references: https://devopsheaven.com/docker/devops/add-host/link/2017/10/04/connect-external-services-from-docker-container.html
I tried to type: sudo docker run -it mysql -h 192.168.100.1 -P 33063 -u czjovan --password=mypw cz_payment_merged but then i get:
2021-03-04 11:34:53+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-03-04 11:34:53+00:00 [ERROR] [Entrypoint]: mysqld failed while attempting to check config
command was: mysqld -h 192.168.100.1 -P 33063 -u czjovan --password=mypw cz_payment_merged --verbose --help --log-bin-index=/tmp/tmp.EV6L0jrspQ
2021-03-04T11:34:53.402148Z 0 [ERROR] [MY-010124] [Server] Fatal error: Can't change to run as user 'czjovan' ; Please check that the user exists!
2021-03-04T11:34:53.403355Z 0 [ERROR] [MY-010119] [Server] Aborting
By Following this also: From inside of a Docker container, how do I connect to the localhost of the machine?
I tried to type: sudo docker run --rm it --network=host mysql mysql -h 127.0.0.1 but got error 2003 (hy000) can't connect to mysql server on 127.0.0.1
I tried to add mysqlconnector to sqlalchemy uri, but the driver not found
I am not an expert by setting these, I lack of Docker mechanism.. appreciate it if anyone willingly to direct me how to step by step resolving this..
UPDATE-------------------------------------------------:
-> Following Mustafa Guler to add -p 3306:3306, the mysql container now starts..
but I still cannot add database in Superset, what should I do next?
You need to provide the IP of the docker host machine. If you're using default networking, use the static IP 172.17.0.1 for the host.
Alternatively you can try host.docker.internal as the hostname.
You need to define user and pass as environment to create new user during start of mysql container. Maybe problem is user and pass MYSQL_USER, MYSQL_PASSWORD Also you do not expose 3306 port on docker run. Please use -p 3306:3306
I found out that installing Superset manually from scratch than using Superset from Docker Container are more a solution to me, since configuring Superset from docker manually can be a little cumbersome, what I did:
#-- Install Superset and MySQL Locally from Scratch ---#
sudo pip install apache-superset (
sudo apt install mysql-client-core-8.0
sudo apt install mysql-server
sudo pip install mysqlclient (for Superset Database Connectors)
#NOTE: there is also a package called 'superset' in pip, in my case i uninstalled this to ensure that only apache-superset is used
service mysql start
when mysql start, try to set password for the first time:
sudo mysqladmin -u root password
then to test it: mysql -u root -p, enter a created password
if it can enter mysql normally, the password set is successful,
CREATE DATABASE superset (this will save all config that superset progress will be saved)
Editing config.py in apache superset, which usually located on /usr/local/lib/python3.8/dist-packages/superset/config.py (this depends on pip installation from no 1),
8a) Edit the sqlalchemy uri part in config.py, so it can connect to a local installed mysql in ubuntu, to something like 'mysql://root:#localhost:3306/superset'
8b) Ensure that the database part in config.py is superset, or the same name with database name created in MySQL
After all database, sqlalchemy uri, and database name is prepared, its good to go to follow with Superset configs stated from: https://superset.apache.org/docs/installation/installing-superset-from-scratch
A) sudo superset db upgrade
B) sudo superset fab create-admin
C) sudo superset load examples
D) sudo superset init
E) sudo superset run -p 8088 --with-threads --reload --debugger (run this to start superset daily)
Run a Private SSH Tunnel Connecting to Database (Optional, in my case I use SSH Tunnel to access database)
And then I am able to add connection locally in Superset defining sqlalchemy uri
A) towards the SSH Tunnel mysql://(server_user):(server_password)#127.0.0.1:33063/(database_name)
(this is outside from local MySQL)
B) towards Local Installed MySQL 'mysql://root:rootpwd#127.0.0.1:3306/database_name'
I will recommend to use docker than manual installation by pip. Package dependencies especially flask are real problems.
Do not follow docker compose as beginner.
I had a problems using docker compose. Port is closed was annoying problem due to networking. Host.docker.internal doesn’t worked for me on Ubuntu 22 on Google cloud. I switched to manual installation and it was package versions hell on python 3.8. I would like to recommend to not follow official doc and use better approach with single docker image to start. Instead of running 5 containers by compose, run everything in one. Use official docker image, here image. Than modify docker file as follows to install custom db driver:
FROM apache/superset
USER root
RUN pip install mysqlclient
RUN pip install sqlalchemy-redshift
USER superset
Second step is to build new image based on docker file description. To avoid networking problems start both containers on same network (superset, your db) easier is to use host network. I used this on Google cloud, example as follow:
docker run -d --network host --name superset supers
The same command to start container with your database. —network host. This solved my problems. More about in whole step to step tutorial: medium or here blog
I am creating a development/testing container that contains a number of elements including a mysql server that must run internally for code to access. To demonstrate the issue, I run the following Dockerfile with docker run -i -t demo_mysql_server:
FROM amazonlinux:2018.03
RUN yum -y update && yum -y install shadow-utils mysql-server
Unfortunately, after building the docker container I receive a common connection error (see 1, 2)
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
which can be fixed by logging in as admin with docker run -i -u 0 -t demo_mysql_server
and executing:
echo "NETWORKING=yes" >/etc/sysconfig/network
service network restart
/etc/init.d/mysqld start
chkconfig mysqld on
which seems to turn everything on. However, incorporating these into RUN commands doesn't seem to keep the service on and logging in as admin requires restarting the service as above and adding a user and working as a non-admin and trying to start the service results in errors of the flavor:
bash: /etc/sysconfig/network: Permission denied
[testUser#544a938c44c1 /]$ service network restart
[testUser#544a938c44c1 /]$ /etc/init.d/mysqld start
/etc/init.d/mysqld: line 16: /etc/sysconfig/network: No such file or directory
[testUser#544a938c44c1 /]$ chkconfig mysqld on
You do not have enough privileges to perform this operation.
I this a normal error to see and how do I get the MySQL server instance to stay running?
I have an issue connecting to mysql running in the local machine in my DockerFile i have mentioned
FROM php:7
RUN apt-get update -y && apt-get install -y openssl zip unzip git
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN docker-php-ext-install pdo mbstring pdo_mysql
WORKDIR /home
COPY . /home
RUN composer install --ignore-platform-reqs
CMD php artisan serve --host=0.0.0.0 --port=8081
EXPOSE 8081
and this in my .env configuration
DB_HOST=localhost
DB_DATABASE=databasename
DB_USERNAME=root
DB_PASSWORD=testpassword
I have very less clue about where it is failing. Do i need to install mysql for Docker container also?
A much simpler solution (for Mac OSX & Docker for Windows) is to replace the host address from localhost
to host.docker.internal
DB_HOST=host.docker.internal
DB_DATABASE=databasename
DB_USERNAME=root
DB_PASSWORD=testpassword
Basically the DNS namehost.docker.internal will resolves to the internal IP address used by the host.
NB: If you have changed your address to host.docker.internal but you still receive connection refused error, it’s most probably because MySQL is currently configured to only listen to the local network.
To resolve that, please update the value of the bind_address to 0.0.0.0 in your my.cnf configuration file.
you are trying to connect to mysql in localhost, which is (surprisingly) the reference to the local host. since its a relative address, inside the container it is being resolved as the container own address, and no mysql is awaiting you there...
so to solve it - just give it your real host ip instead of localhost or 127.0.0.1.
step 1 - fix .env file:
DB_HOST=<your_host_ip> #run `ifconfig` and look for your ip on `docker0` network
DB_DATABASE=databasename
DB_USERNAME=laravel_server #not root, since we are going to allow this user remote access.
DB_PASSWORD=testpassword
step 2 - create dedicated user:
open your mysql: mysql -u root -p, give your root password, and run the following:
CREATE USER 'laravel_server'#'%' IDENTIFIED BY 'testpassword';
GRANT ALL PRIVILEGES ON databasename.* TO 'laravel_server'#'%';
FLUSH PRIVILEGES;
we created the user and gave it the permissions.
step 3 - open mysql to remote access:
we have to make it listening on all interfaces and not just localhost and therefore we run:
sudo sed 's/.*bind-address.*/bind-address=0.0.0.0/' /etc/mysql/mysql.conf.d/mysqld.cnf
(you will be prompted for password. this command is just replacing the line in mysql configuration file)
step 4 - updating:
in the project directory: php artisan config:cache
service mysql restart
then docker build and run a new container again. it should work for you.
I see two options -
Use the private IP of your docker host i.e where mysql server is running.
Use the host network mode while running container in case you want to use localhost.
docker container --net=host ...
In my case (Ubuntu 20.04 Desktop), I had MariaDB already running and using port 3306. So when the app inside the docker container was trying to start MySQL that was inside the container it failed because it was trying to listen to an already used port. I switched off the already-running MariaDB using the command below :
sudo systemctl stop mariadb.service
Then tried starting the docker app. It ran successfully because port 3306 was now free and used by MySQL inside the container. But since I intend to use both of them, a much more permanent solution would be to configure either of the Database systems i.e the one inside the docker container or the one outside the docker container to use a different port other than the default 3306.
Does "service mysql start" start the mysql server or client?
I have done as much searching on this topic as I can, and the answers do seem all over the place. Some sites state that the "service mysql start" starts the server, while others state that one must use "service mysqld start": e.g.: http://theos.in/desktop-linux/tip-that-matters/how-do-i-restart-mysql-server/
To elaborate some more - my understanding is that "mysql" is the process that represents the client interface that connects to a mysql server (either remote or local) and "mysqld" is the process for the server. I would assume that "service mysql start" would only start the mysql client (not the server) and I can use this client to connect to any mysql server. And if I haven't used "service mysqld start", no server would have been started on the local host and therefore I can't use the mysql client to connect to any local mysql server. Is my understanding correct?
Also, I am using a Red Hat server.
Any clarifications and explanations most appreciated - Thanks!
mysql client is never (AFAIK) run as a service, so
service mysql start
will start mysql server. To be precise, this will start service that is described in /etc/init.d/mysql script.
Some distributions name their init script differently, for example mysqld. So you should just check your /etc/init.d/
You can check what exactly that scipt is doing, even if you don't know bash.
First few lines should contain short description, in case of my ubuntu installation it is:
# cat /etc/init.d/mysql
#!/bin/bash
#
### BEGIN INIT INFO
# Provides: mysql
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Should-Start: $network $time
# Should-Stop: $network $time
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start and stop the mysql database server daemon
# Description: Controls the main MySQL database server daemon "mysqld"
# and its wrapper script "mysqld_safe".
### END INIT INFO
So as you can see, mysql service script takes care of mysqld daemon (process). As opposed to mysql binary, found for example in /usr/bin which is client program and doesn't require any service to be running on your OS.
To sum everything up:
# service mysql start
will start server (daemon/service), then you can connect to it with
$ mysql -u root -p
If you are using the command "service mysqld start" this will start the mysql server in RHEL.
Actally the name "mysqld" depends on init.d scripts name. If it's mentioned as "mysql" in init.d scripts "mysql" will work. This variates from distro to distro.
in my case
systemctl restart rh-mysql57-mysqld.service
I have previously installed, connected and uploaded to a WordPress MySQL database via SSH no problem. The people who run my server made a few configuration changes, and now I get a "bash: mysql: command not found" error when I try to log into MySQL via the command line on the same server to access the same database.
I am relatively new to all of this, so I am really not sure what to do. When I run "which mysql" I get a message that says there is no MySQL executable in the /usr/bin directories. I can verify that MySQL is running between the fact that my site is still live and functioning and when I ran a command to test MySQL, I got a message that said "MySQL works!"
I find MySQL files in several directories, but I'm not really sure what I am looking for and how I connect to it when I do find it. I am also not sure if this is user error, or if somehow someone moved or hid MySQL from me -- likely user error??
A MySQL server is not a MySQL client.
Check if MySQL is running by executing this command:
ps aux | grep mysql | grep -v grep
And install the MySQL client:
sudo apt-get install mysql-client # Or your distribution command
If you have MySQL server up and running on your server, it does not mean you have a MySQL client installed on this server.
Try
ls -l /usr/bin/mysql*
Do you see MySQL binaries exactly present on system?
One more problem: Linux has very strange behaviour when you run binaries for another architecture.
For example, if you run a 32-bit executable on 64-bit system you will get an error message like "command or file not found", even if the binaries are actually present!
Now mysql-client has changed to default-mysql-client.
Hence the command:
sudo apt-get install default-mysql-client
Although MySQL is running, if you are not able to connect to the server using the mysql command, then you might be missing to provide soft links:
sudo ln -s /usr/local/mysql/bin/mysqladmin /usr/bin
sudo ln -s /usr/local/mysql/bin/mysql /usr/bin
This should enable you to connect.
In order to get MySQL CLI working or to access the mysql command from anywhere, there are a set of steps to be done to add it to the $PATH variable.
First, open a terminal, and run the following command
echo 'export PATH=/usr/local/mysql/bin:$PATH' >> ~/.bash_profile
Then to reload the Bash profile, run
. ~/.bash_profile
Now run,
mysql -u root -p
Enter the password which you gave during installation. You should see the following result:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.0.23 MySQL Community Server - GPL
You sometimes need to install MySQL on the local machine as well. That means, if you have a host running a MySQL Docker container, the strange thing happened to me that I needed to install mysql-server on the host as well, not only in the container.
The reason was that there was a shell script that needed to check the right database name from a query. In your case, it might be something different. On your host machine (which might even be your local computer, depending the your setup), if on Linux, try:
sudo apt-get update && sudo apt-get install mysql-server
Then you will get rid of
/home/.../some_bash_script.sh: line 123: mysql: command not found