I'm hainvg trouble with docker and volume size. I'm running docker-machine with
three containers. The one giving me trouble is the MySQL container which has a
data-only container for persistance. When I try to import a mysql file, mysql
complains that the table is
full,
which really means that the disk is out of space. Looking at the system, I see
the problem, but don't know how to correct it:
_ _ ____ _ _
| |__ ___ ___ | |_|___ \ __| | ___ ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__| < __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 1.8.1, build master : 7f12e95 - Thu Aug 13 03:24:56 UTC
2015
Docker version 1.8.1, build d12ea79
docker#default:~$ sudo -i
Boot2Docker version 1.8.1, build master : 7f12e95 - Thu Aug 13 03:24:56 UTC
2015
Docker version 1.8.1, build d12ea79
root#default:~# df -h
Filesystem Size Used Available Use% Mounted on
tmpfs 896.6M 112.4M 784.2M 13% /
tmpfs 498.1M 136.0K 498.0M 0% /dev/shm
/dev/sda1 2.8G 2.7G 0 100% /mnt/sda1
cgroup 498.1M 0 498.1M 0% /sys/fs/cgroup
none 462.2G 32.8G 429.5G 7% /Users
/dev/sda1 2.8G 2.7G 0 100%
/mnt/sda1/var/lib/docker/aufs
none 2.8G 2.7G 0 100%
/mnt/sda1/var/lib/docker/aufs/mnt/0a90321d2e941e31385a4c4096e
none 2.8G 2.7G 0 100%
/mnt/sda1/var/lib/docker/aufs/mnt/bca6fc0c017233ed634e7e19284
none 2.8G 2.7G 0 100%
/mnt/sda1/var/lib/docker/aufs/mnt/7e432f020ee8fc9c6a211c810e5
I created the docker-machine like this, which I thought would give me the space I need, but it has not.
docker-machine create --virtualbox-disk-size 4000 -d virtualbox default
I'm creating the mysql image like this
#!/bin/bash
echo "- checking that the image exists"
IMAGE=$(docker images | grep cp_mysql)
if [ -z "$IMAGE" ]; then
echo '- image does not exist, building'
docker build -t cp_mysql -f Dockerfile-app .
fi
echo "- checking if the mysql data volume exists"
DATA_VOL=$(docker ps -a | grep cp_mysql_data)
# if empty
if [ -z "$DATA_VOL" ]; then
echo '- data volume is empty - building'
DATA_VOL=$(docker build -t data -f Dockerfile-data_vol . | tail -n 1 | awk '{print $3}')
docker run --name cp_mysql_data $DATA_VOL
echo "- build data volume: $DATA_VOL"
else
DATA_VOL=$(echo $DATA_VOL | awk '{print $1}')
echo "- data volume is not empty - using existing volume: $DATA_VOL"
fi
echo '- check if existing cp_mysql_app exists'
APP=$(docker ps -a | grep cp_mysql_app)
if [ -z "$APP" ]; then
echo '- the app does not exist, let us create it'
docker run \
--restart=always \
--name cp_mysql_app \
--restart=always \
--volumes-from=cp_mysql_data \
-e MYSQL_USER=cp \
-e MYSQL_PASSWORD=cp \
-e MYSQL_DATABASE=cp \
-e MYSQL_ROOT_PASSWORD=root \
-d \
-p 3306:3306 cp_mysql
else
echo '- the does exist, let us just run it'
docker start cp_mysql_app
fi
Here's my docker info for that machine
docker#default:~$ docker info
Containers: 0
Images: 0
Storage Driver: aufs
Root Dir: /mnt/sda1/var/lib/docker/aufs
Backing Filesystem: tmpfs
Dirs: 0
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.0.9-boot2docker
Operating System: Boot2Docker 1.8.1 (TCL 6.3); master : 7f12e95 - Thu Aug 13 03:24:56 UTC 2015
CPUs: 1
Total Memory: 996.2 MiB
Name: default
ID: XLSE:62WR:VWCR:T2Z6:FSE3:NTLV:EQRT:WLW5:NLPF:HPQH:JQGR:K4LZ
Debug mode (server): true
File Descriptors: 9
Goroutines: 16
System Time: 2015-09-08T16:31:38.029737031Z
EventsListeners: 0
Init SHA1:
Init Path: /usr/local/bin/docker
Docker Root Dir: /mnt/sda1/var/lib/docker
Labels:
provider=virtualbox
What I basically need is unbounded space for /dev/sda1, or to be able to specify a large disk size. I know this stems from my misunderstanding how docker mounts work, but I thought this thread would jump-start my research.
Thanks in advance.
The option --virtualbox-disk-size needs to be entered in MB (see Docker Machine Docs). You created your machine with 4GB. This gives docker 2.8GB for storing images and running containers on /dev/sda1.
The default value for disk size, which is used by docker-machine create, is 20GB (according to docs linked above). Running docker images -a will show you how much disk space is required by the images you pulled. This will give you an indication how much more space you need.
Usually 20GB is a good size to pull several different images and run some containers. So maybe you create a new machine with the default disk size and try to import the mysql file again.
Related
I'm following this guide to run multiple VMs on mac os with apple m1.
I got the following error:
sudo virt-install \
--name host1 \
--memory 2048 \
--vcpus 2 \
--disk size=30 \
--cdrom ./box.img \
--os-variant ubuntu22.04 \
--virt-type hvf \
--qemu-commandline='-M highmem=off -netdev vmnet-shared,id=net0 -device virtio-net-device,netdev=net0,mac=54:54:00:55:54:51' \
--network user
Password:
WARNING CDROM media does not print to the text console by default, so you likely will not see text install output. You might want to use --location. See the man page for examples of using --location with CDROM media
Starting install...
Allocating 'host1.qcow2' | 0 B 00:00:00 ...
Removing disk 'host1.qcow2' | 0 B 00:00:00
ERROR Failed to connect socket to '/opt/homebrew/var/run/libvirt/virtlogd-sock': No such file or directory
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start host1
otherwise, please restart your installation.
apparently it's not there:
$ ls -al /opt/homebrew/var/run/libvirt/virtlogd-sock
ls: /opt/homebrew/var/run/libvirt/virtlogd-sock: No such file or directory
but libvirt is up and running:
$ brew services list 130
Name Status User File
libvirt started root ~/Library/LaunchAgents/homebrew.mxcl.libvirt.plist
I created a simple mysql docker instance as
docker run -d
--name mysql3
-p 3306:3306
-v db:/var/lib/mysql
-e MYSQL_ROOT_PASSWORD=rsecret
-e MYSQL_DATABASE=mkt_data_db
mysql/mysql-server:8.0
After that I logged in and created tables
$docker exec -it mysql3 bash
#mysql -u root -p
use mkt_data_db;
create table price (ticker char(30), eod_date date, close float);
insert into price(ticker, eod_date, close) values ('xyz', curdate(), 100.1);
When I logout and check db directory, its empty
$ ls -al db/
total 0
drwxr-xr-x 2 accnt staff 64 21 Jun 23:38 .
drwxr-xr-x 6 accnt staff 192 21 Jun 23:38 ..
when I stop+rm+run docker again, I do see the the 'price' table and the data within it
Where is the database stored on my local host (mac)?
I would ideally like to move these database files (data) to another computer, provide new DB volume on docker run there and see the data on that machine.
This is an interesting problem, and kudos to you for posting the answer!
I'd be curious to know what the result of docker inspect mysql3 would show (look for the "Mounts" section in the large amount of output) when NOT specifying the full path (like in your original question).
An afterthought: I like to use pwd when creating a shared volume (assuming you are running the command in the desired directory):
docker run -d
--name mysql3
-p 3306:3306
-v $(pwd):/var/lib/mysql
-e MYSQL_ROOT_PASSWORD=rsecret
-e MYSQL_DATABASE=mkt_data_db
-e MYSQL_USER=mkt_data_usr
-e MYSQL_PASSWORD=usecret
mysql/mysql-server:8.0
The above problem was solved by providing absolute path with -v flag
i.e.
docker run -d
--name mysql3
-p 3306:3306
-v /Users/accnt/db:/var/lib/mysql
-e MYSQL_ROOT_PASSWORD=rsecret
-e MYSQL_DATABASE=mkt_data_db
-e MYSQL_USER=mkt_data_usr
-e MYSQL_PASSWORD=usecret
mysql/mysql-server:8.0
i.e. change -v db:/var/lib/mysql to -v /Users/accnt/db:/var/lib/mysql
After that $ls -l /Users/accnt/db showed my entire db contents (including mkt_data_db and price table inside
I am trying to access mysql databases from my docker host to the container.
It's my own dockerfile which install a database expose on port 3306.
I launch my docker with docker-compose, and my compose file is mapping 3308 host port on 3306 container port.
I can access to mysql from the host like this :
mysql -h localhost -P 3308 -u root -pMyPassword
It's working well, but what I can't figure out, is why I can't see any datas from my container?
From inside the container, I have a test databases which I can connect to without any problem. But when I connect from the host to the container mysql process, It seems to show me the mysql datas from the host machine, not from the container one.
Any ideas?
Thanks :)
EDIT 1 :
So here is the first way I can connect to mysql into the container :
docker exec -it MyContainer mysql -uroot -pMyPassword
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test_db |
+--------------------+
It show me my db : test_db
But If i access from :
mysql -h localhost -P 3308 -u root -pMyPassword
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
+--------------------+
My test_db isn't here.
And the result of docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a0de6a691d72 MyContainer "docker-entrypoint.sh" 3 hours ago Up 3 hours 9000/tcp, 0.0.0.0:8085->80/tcp, 0.0.0.0:3308->3306/tcp, 0.0.0.0:8084->8000/tcp, 0.0.0.0:8086->8080/tcp MyContainer
EDIT 2 :
I am developing a standard docker container for web hosting production environnement. Each host is controlled by ajenti. The host work with an nginx reverse proxy which redistribute websites on correct container. Every thing is wokring well. So here is my Dockerfile :
FROM php:5.6-fpm
RUN apt-get update && apt-get install -y --no-install-recommends \
git \
libxml2-dev \
python \
build-essential \
make \
gcc \
python-dev \
locales \
python-pip
RUN dpkg-reconfigure locales && \
locale-gen C.UTF-8 && \
/usr/sbin/update-locale LANG=C.UTF-8
ENV LC_ALL C.UTF-8
ARG MYSQL_ROOT_PASSWORD
RUN export DEBIAN_FRONTEND=noninteractive; \
echo mysql-server mysql-server/root_password password $MYSQL_ROOT_PASSWORD | debconf-set-selections; \
echo mysql-server mysql-server/root_password_again password $MYSQL_ROOT_PASSWORD | debconf-set-selections;
RUN apt-get update && apt-get install -y -q mysql-server php5-mysql
RUN rm /etc/apt/apt.conf.d/docker-gzip-indexes
RUN apt-get update && apt-get install -y wget
RUN wget http://repo.ajenti.org/debian/key -O- | apt-key add -
RUN echo "deb http://repo.ajenti.org/debian main main debian" > /etc/apt/sources.list.d/ajenti.list
RUN apt-get update && apt-get install -y ajenti cron unzip ajenti-v ajenti-v-php-fpm ajenti-v-mysql ajenti-v-nginx
RUN apt-get install -y python-setuptools python-dev \
&& easy_install -U gevent==1.1b3 \
&& sed -i -e s/ssl_version=PROTOCOL_SSLv3/ssl_version=PROTOCOL_SSLv23/ /usr/local/lib/python2.7/dist-packages/gevent-1.1b3-py2.7-linux-x86_64.egg/gevent/ssl.py
EXPOSE 80 8000 8080 3306
RUN mkdir /tmp/tempfiles \
&& mv /srv /tmp/tempfiles \
&& mv /var/lib/mysql /tmp/tempfiles \
&& mv /var/log /tmp/tempfiles \
&& mv /etc/ajenti /tmp/tempfiles
COPY docker-entrypoint.sh /usr/local/bin/
RUN ln -s /usr/local/bin/docker-entrypoint.sh /entrypoint.sh # backwards compat
ENTRYPOINT ["docker-entrypoint.sh"]
As I said, I wanted to be able do deploy a new container easily. So I created a docker-entrypoint.sh which copy wanted files to my volume when I start the container :
#!/bin/bash
DIR="/var/lib/mysql"
# look for empty dir
if [ ! "$(ls -A $DIR)" ]; then
cp -avr /tmp/tempfiles/mysql /var/lib/
fi
# rest of the logic
DIR="/srv"
# look for empty dir
if [ ! "$(ls -A $DIR)" ]; then
cp -avr /tmp/tempfiles/srv /
fi
# rest of the logic
DIR="/var/log"
# look for empty dir
if [ ! "$(ls -A $DIR)" ]; then
cp -avr /tmp/tempfiles/log /var/
fi
# rest of the logic
DIR="/etc/ajenti"
# look for empty dir
if [ ! "$(ls -A $DIR)" ]; then
cp -avr /tmp/tempfiles/ajenti /etc/
fi
# rest of the logic
Finally, my docker-compose.yml to launch everything and map ports :
version: '2'
services:
ajenti:
build:
context: ./dockerfiles/
args:
MYSQL_ROOT_PASSWORD: MyPassword
volumes:
- ./logs:/var/log
- ./html:/srv
- ./ajenti:/etc/ajenti
- ./mysql-data:/var/lib/mysql
- ./apache2:/etc/apache2
ports:
- "8084:8000"
#NGINX
- "8085:80"
#APACHE
- "8086:8080"
- "3308:3306"
Hope this will help to find a solution !
I finally found a solution and it was pretty simple...
First of all, I need to let mysql bind external address, so I changed the line bind-address to '0.0.0.0' inside the container.
Next I just changed the command line with mysql -h 127.0.0.1 -P 3308 -u root -pMyPassword
Now it's fine, I can access container mysql data from the host.
Thanks all for your help :)
In my case I was confused because docker used a different host and port. So you need to find them then do this:
mysql -P <portnumber> -h <host IP> -u db_name -p
Most people would put the docker DB related variables into the environment of the docker container so do this:
sudo docker exec -it container_name env
See if there's a variable called DB_HOST or DB_PORT or something like that. If not then look thru the source code. If it's a PHP project then find a config directory and look in main.php and see
if you execute MySQL operation as entrypoint in the dockerfile file, you will only see that operation when you connect to the container. try changing the entrypoint.
https://docs.docker.com/engine/reference/builder/#entrypoint
I have already googled on this subject and found few threads. Based on these threads I have followed the following steps. But I am facing a problem.
Basically, I want to create a docker image for mysql and then connect to it from my host machine (Mac OS X).
Based on this post , I have to share the mysql unix socket with the host. towards this I have done the following steps
1. Start docker quick terminal
2. docker run --name mysql -e MYSQL_ROOT_PASSWORD=password -d mysql/mysql-server:latest
3. docker exec -it mysql bash
4. mysql -uroot -p
5. create database MyDB;
6. GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' IDENTIFIED BY 'password';
7. exit;
8. mkdir /Users/abhi/host
9. docker run -it -v /host:/shared mysql/mysql-server:latest
Now I get the error
MacBook-Pro:~$ docker run -it -v /Users/abhi/host:/shared mysql/mysql-server
error: database is uninitialized and password option is not specified
You need to specify one of MYSQL_ROOT_PASSWORD, MYSQL_ALLOW_EMPTY_PASSWORD and MYSQL_RANDOM_ROOT_PASSWORD
But you see that I have provided the password and initialized my database.
All I want is that from my host machine, I can connect to the mysql database running inside docker.
EDIT:: ----- solution which worked ------
Thanks RICO. Finally the steps which worked for me are
1. Start docker quick terminal
2. docker run --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -d mysql/mysql-server:latest
3. docker exec -it mysql bash
4. mysql -uroot -p
5. create database MyDB;
or:
CREATE USER 'root'#'%' IDENTIFIED BY 'root';
GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' WITH GRANT OPTION;
6. GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' IDENTIFIED BY 'password';
7. exit;
8. docker-machine env default
Use the IP address obtained in step 8. port is 3306, user is root, password is password, database is MyDB.
Connection is successful!
So you basically you need to expose the mysql port to your host:
docker run --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -d mysql/mysql-server:latest
Then you can access from your host using the mysql command line:
mysql -h127.0.0.1 -ppassword -uroot
Not sure why you are trying to run another container to connect (perhaps you meant linking two containers)
If you are using Mac (or Windows) with docker-machine you want to connect to the IP address of your docker-machine VM. For example:
$ docker-machine ssh default
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\_______/
_ _ ____ _ _
| |__ ___ ___ | |_|___ \ __| | ___ ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__| < __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 1.9.0, build master : 16e4a2a - Tue Nov 3 19:49:22 UTC 2015
Docker version 1.9.0, build 76d6bc9
docker#default:~$ ifconfig eth1
eth1 Link encap:Ethernet HWaddr 08:00:27:E6:C7:20
inet addr:192.168.99.100 Bcast:192.168.99.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fee6:c720/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:18827 errors:0 dropped:0 overruns:0 frame:0
TX packets:10280 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1791527 (1.7 MiB) TX bytes:2242596 (2.1 MiB)
Then connect to:
mysql -h192.168.99.100 -ppassword -uroot
docker run -e MYSQL_ROOT_PASSWORD=pass --name sql-db -p 3306:3306 mysql
docker exec -it sql-db bash
mysql -u root -p
im setting up a mysql container like so:
docker run -v /srv/information-db:/var/lib/mysql tutum/mysql /bin/bash -c "/usr/bin/mysql_install_db"
now, this works when nothing is mounted on /srv on the host, but when i mount my drive, docker seems to write to the underlying filesystem (/), eg:
/]# ls -l /srv
total 0
/]# mount /dev/xvdc1 /srv
/]# mount
...
/dev/xvdc1 on /srv type ext4 (rw,relatime,seclabel,data=ordered)
/]# docker run -v /srv/information-db:/var/lib/mysql tutum/mysql /bin/bash -c "/usr/bin/mysql_install_db"
/]# ls -l /srv
total 16
drwx------. 2 root root 16384 Apr 22 18:05 lost+found
/]# umount /dev/xvdc1
/]# ls -l /srv
total 4
drwxr-xr-x. 4 102 root 4096 Apr 22 18:24 information-db
Anyone seen this behaviour / have a solution?
Cheers
I've seen something like that. Try to perform stat -c %i checks both inside the host and container before and after mount event (in order to get inode values of the target dirs). I guess they're mismatched for a some reason when you mount external device.