Restart Docker Compose with update to database - mysql

I've been working with a simple MySQL table using Docker Compose that only included ID and NAME column. I attempted to update my myDb.sql file that initially creates the table like this:
CREATE TABLE `Person` (
`id` int(11) NOT NULL,
`firstName` varchar(50) NOT NULL, // updated this column
`lastName` varchar(50) NOT NULL // added this column
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I updated the NAME column to firstName, and added a lastName column.
I then stopped the Docker containers by running DOCKER-COMPOSE STOP.
I then restarted the Docker containers by running DOCKER-COMPOSE UP. I even tried DOCKER-COMPOSE RESTART.
The error message I was able to print in the console was this:
SELECT '', id, `firstName` From Person<br>Unknown column 'firstName' in 'field list'
This leads me to believe that I did not restart Docker Compose correctly.
How do I restart Docker Compose so that it runs the CREATE TABLE command?
Edit
Here is my docker-compose.yml file:
version: "3.7"
services:
www:
build: .
ports:
- "8001:80"
volumes:
- ./www:/var/www/html/
links:
- db
networks:
- default
db:
image: mysql:5.7.13
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: myDb
MYSQL_USER: user
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
volumes:
- ./dump:/docker-entrypoint-initdb.d
- persistent:/var/lib/mysql
networks:
- default
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- db:db
ports:
- 8000:80
environment:
MYSQL_USER: user
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
volumes:
persistent:
My Dockerfile looks like this:
FROM php:7.0.30-apache
RUN docker-php-ext-install mysqli

The docker-entrypoint-initdb.d mechanism only runs the first time a database container starts up, with an empty database. You'll need to explicitly docker-compose rm your containers after you docker-compose stop them to cause the current database to be deleted, and then a new empty table will be created in a new empty database.
If you need to preserve the data in an existing database, you are looking for a mechanism called a migration. The various Docker database images don't directly have migration support; this is almost always something that is packaged with your application-level database library (Ruby on Rails and Python's SQLAlchemy both have migration facilities, for instance).
Once you have a migration system anyways, it's probably better to just use that to create the initial database tables. docker-entrypoint-initdb.d makes a little more sense for database-level setup like creating initial users or loading a seed database dump, but in practice you will always need a migration system for changes like what you're describing.

Do docker-compose down whenever you update your schema, it will remove containers and docker network and do docker-compose up to bring your environment with your new schema. Hope it helps. If not, try updating to latest mysql image, the image you are using is almost 3 years old.

So I was thinking that restarting docker-compose would automatically run the CREATE TABLE query in the myDb.sql file. As indicated above, I changed the name of 'name' column to 'firstName' and added a column called 'lastName'.
Again, I am not sure if there was a command that actually does this, but in the end, I was able to alter the table in phpmyadmin. Once I altered the table there, now I am getting data back to the page with no errors.

Related

Building mysql / mariadb database with scema in docker

Hi I am building a service in which I need a Mysql/MariaDB database. I have been googling different solutions and I got the db started with a database created thanks to a guide a was following (never found the link again unfortunately).
Problem
The problem I am having is that the tables are not being created. I added the sql-scema file to /docker-entrypoint-initdb.d/ (you can check it down in the docker file) but it doesnt seem to be executing it (I have tried with both copy and ADD commands).
Current output
This is my current console output from the container:
[![image][1]][1]
The database is created but the SOW TABLES; command returns Empty Set.
Desired output
Since this db is going to be a service differents scripts connect to (currently python), I need to be able to create the db and the sql schema (tables, triggers, etc...) so my team can work with same configuration.
Some of the solutions I have tried (I cant find all the links i have visited only a few)
How to import a mysql dump file into a Docker mysql container
mysql:5.7 docker allow access from all hosts and create DB
Can't connect to mariadb outside of docker container
Mariadb tables are deleted when use volume in docker-compose
Project structure
The structure is pretty simple I am using the following docker-compose.yml
Docker-compose
I still have to try if the MARIADB_ enviroment variables are necessary here.
version: '3'
services:
db-mysql:
#image: mysql/mysql-server:latest
build: ./mysql-db
restart: always
container_name : db-music
ports:
- '3306:3306'
environment:
MYSQL_ROOT_PASSWORD: pwd
MYSQL_DATABASE : audio_service
MYSQL_USER : user
MYSQL_PASSWORD : password
environment:
MARIADB_ROOT_PASSWORD: pwd
MARIADB_DATABASE : audio_service
MARIADB_USER : user
MARIADB_PASSWORD : password
#https://stackoverflow.com/questions/29145370/how-can-i-initialize-a-mysql-database-with-schema-in-a-docker-container?rq=1
expose:
- '3306:3306'
volumes:
- type: bind
source : E:\python-code\Rockstar\volume\mysql
target : /var/lib/mysql
#- type: bind
#source : E:\python-code\Rockstar\mysql-db\sql_scripts\tables.sql
#target : /docker-entrypoint-initdb.d/init.sql
networks:
net:
ipam:
driver: default
config:
- subnet: 212.172.1.0/30
host:
name: host
external: true
Dockerfile
FROM mariadb:latest as builder
# That file does the DB initialization but also runs mysql daemon, by removing the last line it will only init
RUN ["sed", "-i", "s/exec \"$#\"/echo \"not running $#\"/", "/usr/local/bin/docker-entrypoint.sh"]
# needed for intialization
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_ROOT_PASSWORD = pwd
ENV MYSQL_DATABASE = audio_service
ENV MYSQL_USER = user
ENV MYSQL_PASSWORD = password
COPY sql_scripts/tables.sql /docker-entrypoint-initdb.d/
# Need to change the datadir to something else that /var/lib/mysql because the parent docker file defines it as a volume.
# https://docs.docker.com/engine/reference/builder/#volume :
# Changing the volume from within the Dockerfile: If any build steps change the data within the volume after
# it has been declared, those changes will be discarded.
RUN ["/usr/local/bin/docker-entrypoint.sh", "mysqld", "--datadir", "/initialized-db", "--aria-log-dir-path", "/initialized-db"]
FROM mariadb:latest
# needed for intialization
ENV MARIADB_ROOT_PASSWORD=root
ENV MARIADB_ROOT_PASSWORD = pwd
ENV MARIADB_DATABASE = audio_service
ENV MARIADB_USER = user
ENV MARIADB_PASSWORD = password
COPY --from=builder /initialized-db /var/lib/mysql
EXPOSE 3306
SQL schema file
create database audio_service;
use audio_service;
CREATE TABLE audio
(
audio_id BINARY(16),
title TEXT NOT NULL UNIQUE,
content MEDIUMBLOB NOT NULL,
PRIMARY KEY (audio_id)
) COMMENT='this table stores sons';
DELIMITER ;;
CREATE TRIGGER `audio_before_insert`
BEFORE INSERT ON `audio` FOR EACH ROW
BEGIN
IF new.audio_id IS NULL THEN
SET new.audio_id = UUID_TO_BIN(UUID(), TRUE);
END IF;
END;;
DELIMITER ;
There is no need to build your own image since the official mysql / mariadb images are already well suited. You only need to run them with the following as explained in their image documentations:
environment variables to initialize an new database with a respective user on the first run
a volume at /var/lib/mysql to persist the data
any initialization/sql scripts mounted into /docker-entrypoint-initdb.d
So storing your SQL* into a schema.sql file right next to the docker-compose.yml the following is enough to achieve what you want:
# docker-compose.yml
services:
db:
image: mariadb
environment:
MARIADB_ROOT_PASSWORD: pwd
MARIADB_DATABASE: audio_service
MARIADB_USER: user
MARIADB_PASSWORD: password
volumes:
# persist data files into `datadir` volume managed by docker
- datadir:/var/lib/mysql
# bind-mount any sql files that should be run while initializing
- ./schema.sql:/docker-entrypoint-initdb.d/schema.sql
volumes:
datadir:
*note that you can remove the CREATE DATABASE and USE statements from your schema.sql since these will be automatically done by the init script for you anyway
There are two reasons that your own setup isn't working as expected:
the line COPY --from=builder /initialized-db /var/lib/mysql won't work as expected for the same reason you described in your comment a bit above it: /var/lib/mysql is a volume and thus no new files a stored in it in the build steps after it was defined.
you are bind-mounting E:\python-code\Rockstar\volume\mysql to /var/lib/mysql in your docker-compose.yml.
But this will effectively override any contents of /var/lib/mysql of the image, i.e. although your own image built from your Dockerfile does include an initialized database this is overwritten by the contents of E:\python-code\Rockstar\volume\mysql when starting the service.

Why is docker compose creating mysql container with wrong name?

I have managed to create a MySQL and PHP container and my scripts execute and all my tables are there.
However I have a database that I call "myDb" and a user that is called "someuser" and when the database is created for some reason the name of the database is "somedatabase"
my docker-compose.yaml file:
services:
mysql:
image: mysql:latest
ports:
- 3307:3306
environment:
MYSQL_DATABASE: myDb
MYSQL_ROOT_PASSWORD: SomeRootPassword1!
MYSQL_USER: someuser
MYSQL_PASSWORD: Password1!
volumes:
- ./dbScript/winit_Script2.sql:/docker-entrypoint-initdb.d/winit_Script2.sql
- db_data:/var/lib/mysql
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: dev_pma
links:
- mysql
environment:
PMA_HOST: mysql
PMA_PORT: 3307
PMA_ARBITRARY: 1
restart: always
ports:
- 8183:80
volumes:
db_data:
phpAdmin:
Mysqlworkbench:
What have I done wrong here?
A little edit after the comments:
It would seem that when having a volumes section you create volumes in docker
and when you create a volume on a specific port once then it gets reused every time you do docker-compose up. This was the case for me.
More details in accepted answer.
The mysql image does not initialize the database if the volume is not clean.
When you are stopping and starting the database from the same compose file, the volume is always the same, hence you want the data to be persisted even after an app restart.
To force the re-initialization of the data, you can delete that docker volume(only if you no longer need that database! this cannot be undone):
First, stop and delete the containers.
Then list and delete the volume that persists the database:
docker volume ls
DRIVER VOLUME NAME
local <your-deployment-name>_db_data
docker volume rm <your-deployment-name>_db_data
Then run again the docker-compose up command and you'll be able to find the myDb in phpMyAdmin instead of somedb
Edit:
Unless you change yourself the entrypoint and rebuild the image to force it initialize your DB according to the ENV you're passing, even if the volume is not clean, the only option that comes to my mind is to create the new DB manually. Here is the conditional that skips the re-initialization of the DB and here is the script that is invoked if the volume is clean.

Change MYSQL_DATABASE etc. in docker compose

I am having some issues getting docker to create a completely new image with a new database.
I first started my containers, but had entered wrong credentials. I now want to build the image over with the right credentials.
This is my service
mysql:
build: ./mysql
image: custom-mysql
command: --default-authentication-plugin=mysql_native_password
environment:
MYSQL_ROOT_PASSWORD: dsadjkasjd
MYSQL_DATABASE: correct_db
MYSQL_USER: correct_user
MYSQL_PASSWORD: correct_pass
networks:
main:
aliases:
- database
ports:
- "3306:3306"
volumes:
- ./data/mysql:/var/lib/mysql
I have tried removing all containers and images manually as well as
docker-compose down --rmi 'all'
docker-compose build --no-cache --pull
But not matter what I try it will only work with the passwords I used the first time around.
Is there some clever way of completely removing the container+image and building it again with a new database/users ?
The script to generate the database from scratch (which evaluates the given environment variables) is only run on the first generation for the volume, as you can see in https://github.com/docker-library/mysql/blob/master/5.7/docker-entrypoint.sh.
You have two possibilites now: either stop the container, remove the volume, and restart the container; or start a bash in the running container, remove the data dir /var/lib/mysql, and restart the container
You have to delete the volume (or make its content empty) so on new startup it will generate the new database files with the new configuration.

docker-compose run scripts when container is initialized

I am new to docker. I am on a windows 7 machine and using docker toolbox.
I am trying to write a docker-compose.yml for MySQL which creates a database and runs 2 scripts (create table and insert)
version: '3'
services:
mysql-image:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: myDatabase
MYSQL_USER: test
MYSQL_PASSWORD: pwtest
ports:
- "3306:3306"
volumes:
- ./sqlscripts/:/docker-entrypoint-initdb.d/
volumes:
sqlscripts:
I can connect to the database, but the problem I have is running the scripts.
mostly I run into the following error:
mysql-image_1 | mysql: [Warning] Using a password on the command line
interface can be insecure.
mysql-image_1 | ERROR: Can't initialize
batch_readline - may be the input source is a directory or a block
device. docker_mysql-image_1 exited with code 1
I searched for a while, trying to get it work but at this point I do not know what I am doing wrong. This is one of my .sql scripts which I am trying to run it when the docker container starts.
createTablePerson.sql
CREATE TABLE `myDatabase`.`Person` (
`idPerson` INT NOT NULL AUTO_INCREMENT,
`Name` VARCHAR(45) NOT NULL,
`age` INT NOT NULL,
PRIMARY KEY (`idPerson`));
Thanks.
I investigated further and it seems is because I am on Windows 7 and I have to use docker-toolbox which runs in a VirtualBox VM. It seems that the location I had my scripts, is not shared between windows host and docker-toolbox. I noticed the files were copied in the container but they where considered as empty directories. I moved my scripts to user location and it seems the scripts are copied to the container and run perfectly. Now I am searching for a way to try make other paths visible. I prefer to have my scripts under my java project.

"Base table or view not found" error in docker container

I have Laravel app with mysql db. I'm trying to run it in docker using this docker-compose.yml:
load_balancer:
image: tutum/haproxy
links:
- web
ports:
- "80:80"
cache:
image: redis
web:
image: andrewmclagan/nginx-hhvm
links:
- db
- cache
volumes:
- ./:/var/www
environment:
- APP_ENV=local
- DB_DATABASE=regappbase
- DB_PASSWORD=Q1w2e3r4t5
- DB_HOST=db
- VIRTUAL_HOST=laravel.local
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: Q1w2e3r4t5
MYSQL_DATABASE: regappbase
On my host database has three tables:
migrations
password_resets
users
When I run the app and try refer to database, I have an error like this:
QueryException in Connection.php line 715: SQLSTATE[42S02]: Base table or view not found: Table 'regappbase.users' doesn't exist (SQL: select * from users where email=(e-mail#ghhjghj77))
How can I create database regappbase? I don't need to store data, only schema
Check the documentation for the mysql image (hub.docker.com). There is a directory which you can mount your .SQL scripts into, on container start mysql will loop through and execute each file in this dir. Use this to build up your DB schema
"Initializing a fresh instance
When a container is started for the first time, a new database mysql will be initialized with the provided configuration variables. Furthermore, it will execute files with extensions .sh and .sql that are found in /docker-entrypoint-initdb.d. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data"
Well, I spend two days trying to change variables, or use data-only-container, or mount sql-scritps.
But the decision was to make from running web-container:
php artisan migrate