Locally run Docker container unable to make connection to remote MySQL database - mysql

I am trying to develop a micro service that gets information from a remote database but when I run the container it fails to make a connection to that database.
The container is running locally (I'm still developing it) and the database is hosted in AWS RDS Aurora MySQL.
The database is in use on multiple production websites using the same user I'm trying to use in the container. The user has full permission to the database and my local PHPMyAdmin connects to the database using that same user and I've had no trouble managing the db with it.
The problem is that the database connection in the container fails with an Access denied error.
The database user is setup as dbuser#% yet the error says:
Access denied for user 'dbuser'#'[my public ip]' (using password:
YES).
I attempted to add another account for dbuser#[my public ip] and gave it the same permissions as the wildcard host account and that makes no difference.
As another test I added a curl call inside the container to load an external page to make sure it can make external connections and that succeeds. It's just the db connection failure that makes no sense.
My dockerfile looks like this:
FROM php:7.2-apache
RUN apt-get update
RUN docker-php-ext-install mysqli pdo pdo_mysql
RUN a2enmod rewrite
I'm hoping someone has come across this and/or knows what I'm missing here.
Thanks in advance!
Some more info:
I'm using Docker for Windows and docker-compose to run my container. My docker-compose.yml file looks like:
version: "3"
services:
web:
image: repository/container:latest
volumes:
- ./src:/var/www/html
deploy:
replicas: 5
resources:
limits:
cpus: "0.1"
memory: 50M
restart_policy:
condition: on-failure
ports:
- "80:80"
networks:
- inv
networks:
inv:

Starting the container with docker run --network host ... will make the container share the network stack of the host. That should solve the problem.

I ended up fixing this by copying the original connection code from my API and replacing the connection code in my container with it.
I commented out the broken code and compared the two but see no difference. No quote marks in the host string, no typos, nothing that makes sense.
The only difference is the way I'm setting the variables that build the host string. The values of those variables are the same so it really doesn't make sense.
Regardless, doing this fixed it.

Related

How can I access ddev's database with external database browsers or similar tools?

As far as I know, DDEV provides configuration options for phpMyAdmin to comfortably work with containered MySQL/MariaDB databases. But I would rather like to use a different tool like e.g. phpStorm or DBeaver or such. Is there a way to make my wish come true?
Supplementing the docker-compose configuration I exposed port 3306, accordingly.
version: '3.6'
services:
web:
ports:
- 3306:3306
Trying to connect to a containered MariaDB database from my client host looks like the following.
$ mariadb --host=foo.ddev.site --user=db --password=db --database=db
ERROR 2013 (HY000): Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 11
The database port is already exposed in ddev, so can easily be used with many, many external tools.
For example, ddev mysql gives you direct access, and there are also ddev sequelpro and ddev sequelace and ddev tableplus, and there's an example in the custom commands showing how to do it with mysqlworkbench (See ~/.ddev/commands/host/mysqlworkbench.example).
All of those grab the already-exposed db port.
ddev describe on any project tells you how to access the port.
The host_db_port setting in your project's .ddev/config.yaml can be used to lock down the exposed port so you can easily use it with PhpStorm.
This article also goes through some of the many ways you can access the database using ddev, https://ddevhq.org/ddev-local/ddev-local-database-management/

Deploy mysql+springboot docker containers in AWS ECS

I am working on simple project with mysql + springboot using docker container and i am able to run in my local without any issue. I tried to bring the same containers to AWS ECS to standup but i am facing multiple problems. I did research multiple documents and blogs but could not get the correct content to make this works.
I used the below to stand up the mysql in my local with the container name of "mysqlcontainer"
docker run --restart always --name mysqlcontainer --net dev-network -v /Users/myuser/Develop/mysql_data/8.0:/var/lib/mysql -p 3306:3306 -d
-e MYSQL_ROOT_PASSWORD=password mysql:8.0
Once mysql stood up then i ran the below command to bring my springboot application. This is simple springboot service which has the CRUD operation with mysql.
docker run -d -p 8061:8061 --name user-mysqlapp --net dev-network
--link mysqlcontainer user-mysql
mysql container name is "mysqlcontainer" which runs on the network "dev-network" and i have used these container and network name in second docker command to stand up the springboot application as both container needs to talk each other within the same network.
here is the docker file for springboot application.
FROM openjdk:8-jdk-alpine
EXPOSE 8061 ARG
JAR_FILE=target/user-mysql.jar
ADD ${JAR_FILE} user-mysql.jar
ENTRYPOINT ["java","-jar","user-mysql.jar"]
Here is the datasource that is being used in application.yml within springboot application.
datasource:
url: jdbc:mysql://mysqlcontainer:3306/sample
username: dummy
password: password
driver-class-name: com.mysql.jdbc.Driver
after both containers are running, i was able to connect the mysql with mysqldeveloper and able to execute the below command.
CREATE USER 'dummy'#'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON * . * TO 'dummy'#'%';
create database sample;
use sample;
I have no issue of executing this setup in my local and i did push the springboot application image to dockerhub to use in AWS ECS.
Problem statement:
If i run the mysql container alone in ECS, i am able to run it and able to connect using mysql developer from my local. but if i try to run springboot app image, its not properly linking with mysql container. I tried in FARGATE instance type with awsvpc network mode where i was unable to give link to mysql container. i did tried to create single taskdefinition where both containers were added and both of them were running successfully but in the spring boot log, it says unable to create the communication with mysql container.
Can someone please share some information to stand up this setup in aws ecs or share some link/tutorial/blog where i can see how this mysql + springboot app setup can stand up in aws ecs? if i am able to stand up these containers then i need to create/attach volume to the persisting the data in aws. Thanks in advance...
I am able to setup this spring boot + mysql setup successfully in ECS. Below are the steps were followed.
Create one task definition in which add mysql container first where
provide the necessary env variable to mysql container.
Add another spring boot container in which set the dependency order
container as mysql container with condition of "START".
Create new cluster
Create service on this cluster and then add this task definition on the service.
Since i used fargate type, we can't use link to refer the other container but instead i used 127.0.0.1 ip address in spring boot application yml data source url so that it will refer the local host within the same network. This solves the container linking issue and i am sure we would have some better option but for time being i am using this option.
datasource:
url: jdbc:mysql://127.0.0.1:3306/sample

Doctrine Connection Refused from Symfony Docker Container to MariaDB Container

Banging my head at 2:30 a.m.
I have three Docker containers MariaDB, Nginx and PhpFpm.
MariaDB opens up 3306/tcp - no exposed port because all container share the same Docker Network.
Symfony starts up in PhpFpm and is exposed via Nginx on port 8081.
So myhost.com:8081 shows my Symfony application.
When I docker exec bash into my PhpFpm container and execute bin/console doctrine:schema:create schema is created, MariaDB has a new table.
Of course I clear the cache and warm it up for APP_ENV=prod
var/log and var/cache have set correct permissions.
No matter what I do, when using the PDO connection within my Symfony application's controller I get an
Connection refused error
visible in my prod.log within the container.
I tried .env with several options, most prominent are:
DATABASE_URL="mysql://user:pass#hostname:8084/db" (here I set -p 8084:3306 for MariaDB Container)
and
DATABASE_URL="mysql://user:pass#mariadbcontainername:3306/db" (here I did not expose any DB port as mentioned above)
Does anyone have an idea where to look and what to test?
I can even verify that my bin/console doctrine... commands are already getting a Connection Refused error when I change the DATABASE_URL to a nonsense value.
P.S.: Due to server restrictions I don't use docker-compose but simply docker run.

Docker Compose - Creating and connecting to mysql container

Having a hard time trying to get to grips with mysql in docker. I have got the container running with docker-compose, but I am not able to connect to the database via any tools such as phpmyadmin, workbench or tableplus.
I have connected directly to the running container and run
mysql -uroot -p
and entered the root password which I have passed, but this fails with this error:
ERROR 1045 (28000): Access denied for user 'root'#'localhost' (using password: YES)
Here is my docker-compose.yml file:
version: '3'
services:
db:
image: mysql
restart: always
environment:
MYSQL_DATABASE: quotes
MYSQL_USER: quotes
MYSQL_PASSWORD: P#KhzKZp)56sU8n+
MYSQL_ROOT_PASSWORD: +\VrLG*<t5sq[\\shR29u#n~A3^Jp*
ports:
- '3306:3306'
volumes:
- /private/mdbdata/quotes:/etc/mysql/conf.d
expose:
- '3306'
Been on this for days... hope someone can help!
I think your container is looking for a MySQL server on 'localhost', which WILL NOT WORK. 'localhost' to a container is the container itself - NOT the host machine it's running on.
You should be able to access the MySQL server using the host machine IP address.
Finally solved this one ... I had to remove all special characters from the password strings.
I tired adding single and double quotes are around the strings to see if that would allow the special characters, but that still failed. Passwords needed to be alphanumeric.
Have you tried the solution provided here? Basically, if you had a container running previously, you have to explicitly purge it since docker keeps track of root passwds (via volumes) from previously instantiated containers. This happens with a lot of tools, it's not unique to MySQL containers. A simple docker-compose rm -v should suffice, afterwards bring up your container. This basically deletes the old volume from the disk, removing all data from previous container instantiation.
You can also call a docker ps, find your container and execute docker rm -v <CONTAINER_NAME>. Bring up your container afterwards.

Docker nodejs with mysql communication

I'm using 2 docker images one with my nodeJS backend server the other with my MySQL database. On the docker-compose file I'm defining the passwords, ports and hostnames correctly:
sql:
image: mysql:5.7.22
hostname: sql
ports:
- 3306:3306
secrets:
- db_root_pass
- db_user_pass
environment:
MYSQL_USER: user
MYSQL_PASSWORD_FILE: /run/secrets/db_user_pass
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_root_pass
provider:
image: monokilho/app:dev
hostname: provider
ports:
- 3000:3001
- 9221:9229
secrets:
- db_user_pass
command: node --inspect=0.0.0.0:9229 appModule.js
And on my DB_config.js file for NodeJS I have the connection setup like so:
db_config.host = 'sql';
db_config.port = '3306';
db_config.user = 'user';
db_config.password = fs.readFileSync('/run/secrets/db_user_pass', 'utf8');
db_config.database = 'app';
db_config.multipleStatements = true;
Problem is that although, using this exact configurations, docker connects Node to MySQL just fine on my local windows machine, when I upload the images to my remote linux server I continue to get:
Access denied for user 'user'#'8b2e56e566b2.network_default'
I've already remade the secrets, tried manually adding the passwords to the config on NodeJS and nothing... what makes it even weirder is that if I go on the MySQL container to connect directly or if I make another MySQL container and remotely connect it works, so I know the password input on MySQL config is correct and it is accepting remote connections.
Any suggestion what might be the difference between windows and linux for this behavior to happen? Thanks in advance.
PS: If needed windows is windows 10 and linux distro is ububtu 16.04.
EDIT: The access denied error appears on the mysql logs so the nodejs docker can reach the mysql docker and the network should be fine.
Apparently the mysql config was ignoring a sneaky \n on the password file allowing it to work normally with a command line connection, while on the nodejs it was bugging the connection.