Nginx Docker Volumes empty - mysql

When using docker-compose, nginx isn't showing files like other images might.
For example with Mysql, the below code will save the data created at /var/lib/mysql to the local machine at ./volumes/db/data
./volumes/db/data:/var/lib/mysql
Another example, with Wordpress, the below code will save the data created at /var/www/html/wp-content/uploads to the local machine at ./volumes/uploads/data
./volumes/uploads/data:/var/www/html/wp-content/uploads
This is not working with nginx though so no matter what I change /some/nginx/path to, it never appears at ./volumes/nginx/data
./volumes/nginx/data:/some/nginx/path
Does nginx work differently in this regard?
Update
Using a named volume with the following configurations solved this problem:
In the services section of the docker-compose file, I changed ./volumes/nginx/data:/some/nginx/path to nginx_data:/some/nginx/path
And then my volumes section reads as follows
volumes:
nginx_data:
driver: local
driver_opts:
o: bind
device: ${PWD}/volumes/nginx/data

There should be no difference, a volume is mounting a local directory to a directory in the container. Either you are not mounting correctly or you are mounting an incorrect path inside the nginx container (one which nginx does not use).
Based on the offical nginx docker image docs on https://docs.docker.com/samples/library/nginx/ you should mount to /usr/share/nginx/html
$ docker run --name some-nginx -v /some/content:/usr/share/nginx/html:ro -d nginx
In additonal I would include full paths in your docker-compose.yaml:
volumes:
- /full_path/volumes/nginx/data:/usr/share/nginx/html
If this is still not working you should exec into the container and confirm that the directory is mounted:
$ docker exec -it <container_name> sh
$ df -h | grep nginx
# write data, confirm you see it on the docker host's directory
$ cd /usr/share/nginx/html
$ touch foo
# on docker host
$ ls /full_path/volumes/nginx/data/foo
If any of this is failing I would look at docker logs to see if there was an issue mounting the directory, perhaps a path or permission issue.
$ docker logs <container_name>
--- UPDATE ---
I ran everything you are using and it just worked:
$ cat Dockerfile
FROM nginx
RUN touch /usr/share/nginx/html/test1234 && ls /usr/share/nginx/html/
$ docker build -t nginx-image-test .; docker run -p 8889:80 --name some-nginx -v /full_path/test:/usr/share/nginx/html:rw -d nginx-image-test; ls ./test;
...
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
33ccbea6c1c1 nginx-image-test "nginx -g 'daemon of…" 4 minutes ago Up 4 minutes 0.0.0.0:8889->80/tcp some-nginx
$ cat test/index.html
hello world
$ curl -i http://localhost:8889
HTTP/1.1 200 OK
Server: nginx/1.15.3
Date: Thu, 06 Sep 2018 15:35:43 GMT
Content-Type: text/html
Content-Length: 12
Last-Modified: Thu, 06 Sep 2018 15:31:11 GMT
Connection: keep-alive
ETag: "5b91483f-c"
Accept-Ranges: bytes
hello world
-- UPDATE 2 --
Awesome you figured it out, this post seems to explain why:
docker data volume vs mounted host directory
The host directory is, by its nature, host-dependent. For this reason, you can’t mount a host directory from Dockerfile because built images should be portable. A host directory wouldn’t be available on all potential hosts.
If you have some persistent data that you want to share between containers, or want to use from non-persistent containers, it’s best to create a named Data Volume Container, and then to mount the data from it.

I ran across this issue as well when trying to create a volume to place my own nginx configurations into etc/nginx. While I never found out the real issue I think it has to do with how nginx is built from the Dockerfile.
I solved the problem by using my own Dockerfile to extend the original and copy over the configuration files at build time. Hopefully this helps.
FROM nginx:1.15.2
COPY ./nginx.conf /etc/nginx/nginx.conf
COPY ./global /etc/nginx/global
COPY ./conf.d /etc/nginx/conf.d

Related

How to use external files as arguments in Docker images [duplicate]

I am trying to mount a host directory into a Docker container so that any updates done on the host is reflected into the Docker containers.
Where am I doing something wrong. Here is what I did:
kishore$ cat Dockerfile
FROM ubuntu:trusty
RUN apt-get update
RUN apt-get -y install git curl vim
CMD ["/bin/bash"]
WORKDIR /test_container
VOLUME ["/test_container"]
kishore$ tree
.
├── Dockerfile
└── main_folder
├── tfile1.txt
├── tfile2.txt
├── tfile3.txt
└── tfile4.txt
1 directory, 5 files
kishore$ pwd
/Users/kishore/tdock
kishore$ docker build --tag=k3_s3:latest .
Uploading context 7.168 kB
Uploading context
Step 0 : FROM ubuntu:trusty
---> 99ec81b80c55
Step 1 : RUN apt-get update
---> Using cache
---> 1c7282005040
Step 2 : RUN apt-get -y install git curl vim
---> Using cache
---> aed48634e300
Step 3 : CMD ["/bin/bash"]
---> Running in d081b576878d
---> 65db8df48595
Step 4 : WORKDIR /test_container
---> Running in 5b8d2ccd719d
---> 250369b30e1f
Step 5 : VOLUME ["/test_container"]
---> Running in 72ca332d9809
---> 163deb2b1bc5
Successfully built 163deb2b1bc5
Removing intermediate container b8bfcb071441
Removing intermediate container d081b576878d
Removing intermediate container 5b8d2ccd719d
Removing intermediate container 72ca332d9809
kishore$ docker run -d -v /Users/kishore/main_folder:/test_container k3_s3:latest
c9f9a7e09c54ee1c2cc966f15c963b4af320b5203b8c46689033c1ab8872a0eakishore$ docker run -i -t k3_s3:latest /bin/bash
root#0f17e2313a46:/test_container# ls -al
total 8
drwx------ 2 root root 4096 Apr 29 05:15 .
drwxr-xr-x 66 root root 4096 Apr 29 05:15 ..
root#0f17e2313a46:/test_container# exit
exitkishore$ docker -v
Docker version 0.9.1, build 867b2a9
I don't know how to check boot2docker version
Questions, issues facing:
How do I need to link the main_folder to the test_container folder present inside the docker container?
I need to make this automatically. How do I to do that without really using the run -d -v command?
What happens if the boot2docker crashes? Where are the Docker files stored (apart from Dockerfile)?
There are a couple ways you can do this. The simplest way to do so is to use the dockerfile ADD command like so:
ADD . /path/inside/docker/container
However, any changes made to this directory on the host after building the dockerfile will not show up in the container. This is because when building a container, docker compresses the directory into a .tar and uploads that context into the container permanently.
The second way to do this is the way you attempted, which is to mount a volume. Due to trying to be as portable as possible you cannot map a host directory to a docker container directory within a dockerfile, because the host directory can change depending on which machine you are running on. To map a host directory to a docker container directory you need to use the -v flag when using docker run, e.g.,:
# Run a container using the `alpine` image, mount the `/tmp`
# directory from your host into the `/container/directory`
# directory in your container, and run the `ls` command to
# show the contents of that directory.
docker run \
-v /tmp:/container/directory \
alpine \
ls /container/directory
The user of this question was using Docker version 0.9.1, build 867b2a9, I will give you an answer for docker version >= 17.06.
What you want, keep local directory synchronized within container directory, is accomplished by mounting the volume with type bind. This will bind the source (your system) and the target (at the docker container) directories. It's almost the same as mounting a directory on linux.
According to Docker documentation, the appropriate command to mount is now mount instead of -v. Here's its documentation:
--mount: Consists of multiple key-value pairs, separated by commas. Each key/value pair takes the form of a <key>=<value> tuple. The --mount syntax is more verbose than -v or --volume, but the order of the keys is not significant, and the value of the flag is easier to understand.
The type of the mount, which can be bind, volume, or tmpfs. (We are going to use bind)
The source of the mount. For bind mounts, this is the path to the file or directory on the Docker daemon host. May be specified as source or src.
The destination takes as its value the path where the file or directory will be mounted in the container. May be specified as destination, dst, or target.
So, to mount the the current directory (source) with /test_container (target) we are going to use:
docker run -it --mount src="$(pwd)",target=/test_container,type=bind k3_s3
If these mount parameters have spaces you must put quotes around them. When I know they don't, I would use `pwd` instead:
docker run -it --mount src=`pwd`,target=/test_container,type=bind k3_s3
You will also have to deal with file permission, see this article.
you can use -v option from cli, this facility is not available via Dockerfile
docker run -t -i -v <host_dir>:<container_dir> ubuntu /bin/bash
where host_dir is the directory from host which you want to mount.
you don't need to worry about directory of container if it doesn't exist docker will create it.
If you do any changes in host_dir from host machine (under root privilege) it will be visible to container and vice versa.
2 successive mounts:
I guess many posts here might be using two boot2docker, the reason you don't see anything is that you are mounting a directory from boot2docker, not from your host.
You basically need 2 successive mounts:
the first one to mount a directory from your host to your system
the second to mount the new directory from boot2docker to your container like this:
1) Mount local system on boot2docker
sudo mount -t vboxsf hostfolder /boot2dockerfolder
2) Mount boot2docker file on linux container
docker run -v /boot2dockerfolder:/root/containerfolder -i -t imagename
Then when you ls inside the containerfolder you will see the content of your hostfolder.
Is it possible that you use docker on OS X via boot2docker or something similar.
I've made the same experience - the command is correct but nothing (sensible) is mounted in the container, anyway.
As it turns out - it's already explained in the docker documentation. When you type docker run -v /var/logs/on/host:/var/logs/in/container ... then /var/logs/on/host is actually mapped from the boot2docker VM-image, not your Mac.
You'll have to pipe the shared folder through your VM to your actual host (the Mac in my case).
For those who wants to mount a folder in current directory:
docker run -d --name some-container -v ${PWD}/folder:/var/folder ubuntu
I'm just experimenting with getting my SailsJS app running inside a Docker container to keep my physical machine clean.
I'm using the following command to mount my SailsJS/NodeJS application under /app:
cd my_source_code_folder
docker run -it -p 1337:1337 -v $(pwd):/app my_docker/image_with_nodejs_etc
[UPDATE] As of ~June 2017, Docker for Mac takes care of all the annoying parts of this where you have to mess with VirtualBox. It lets you map basically everything on your local host using the /private prefix. More info here. [/UPDATE]
All the current answers talk about Boot2docker. Since that's now deprecated in favor of docker-machine, this works for docker-machine:
First, ssh into the docker-machine vm and create the folder we'll be mapping to:
docker-machine ssh $MACHINE_NAME "sudo mkdir -p \"$VOL_DIR\""
Now share the folder to VirtualBox:
WORKDIR=$(basename "$VOL_DIR")
vboxmanage sharedfolder add "$MACHINE_NAME" --name "$WORKDIR" --hostpath "$VOL_DIR" --transient
Finally, ssh into the docker-machine again and mount the folder we just shared:
docker-machine ssh $MACHINE_NAME "sudo mount -t vboxsf -o uid=\"$U\",gid=\"$G\" \"$WORKDIR\" \"$VOL_DIR\""
Note: for UID and GID you can basically use whatever integers as long as they're not already taken.
This is tested as of docker-machine 0.4.1 and docker 1.8.3 on OS X El Capitan.
Using command-line :
docker run -it --name <WHATEVER> -p <LOCAL_PORT>:<CONTAINER_PORT> -v <LOCAL_PATH>:<CONTAINER_PATH> -d <IMAGE>:<TAG>
Using docker-compose.yaml :
version: '2'
services:
cms:
image: <IMAGE>:<TAG>
ports:
- <LOCAL_PORT>:<CONTAINER_PORT>
volumes:
- <LOCAL_PATH>:<CONTAINER_PATH>
Assume :
IMAGE: k3_s3
TAG: latest
LOCAL_PORT: 8080
CONTAINER_PORT: 8080
LOCAL_PATH: /volume-to-mount
CONTAINER_PATH: /mnt
Examples :
First create /volume-to-mount. (Skip if exist)
$ mkdir -p /volume-to-mount
docker-compose -f docker-compose.yaml up -d
version: '2'
services:
cms:
image: ghost-cms:latest
ports:
- 8080:8080
volumes:
- /volume-to-mount:/mnt
Verify your container :
docker exec -it CONTAINER_ID ls -la /mnt
docker run -v /host/directory:/container/directory -t IMAGE-NAME /bin/bash
docker run -v /root/shareData:/home/shareData -t kylemanna/openvpn /bin/bash
In my system I've corrected the answer from nhjk, it works flawless when you add the -t flag.
On Mac OS, to mount a folder /Users/<name>/projects/ on your mac at the root of your container:
docker run -it -v /Users/<name>/projects/:/projects <container_name> bash
ls /projects
If the host is windows 10 then instead of forward slash, use backward slash -
docker run -it -p 12001:80 -v c:\Users\C\Desktop\dockerStorage:/root/sketches
Make sure the host drive is shared (C in this case). In my case I got a prompt asking for share permission after running the command above.
For Windows 10 users, it is important to have the mount point inside the C:/Users/ directory. I tried for hours to get this to work. This post helped but it was not obvious at first as the solution for Windows 10 is a comment to an accepted answer. This is how I did it:
docker run -it -p 12001:80 -v //c/Users/C/Desktop/dockerStorage:/root/sketches \
<your-image-here> /bin/bash
Then to test it, you can do echo TEST > hostTest.txt inside your image. You should be able to see this new file in the local host folder at C:/Users/C/Desktop/dockerStorage/.
As of Docker 18-CE, you can use docker run -v /src/path:/container/path to do 2-way binding of a host folder.
There is a major catch here though if you're working with Windows 10/WSL and have Docker-CE for Windows as your host and then docker-ce client tools in WSL. WSL knows about the entire / filesystem while your Windows host only knows about your drives. Inside WSL, you can use /mnt/c/projectpath, but if you try to docker run -v ${PWD}:/projectpath, you will find in the host that /projectpath/ is empty because on the host /mnt means nothing.
If you work from /c/projectpath though and THEN do docker run -v ${PWD}:/projectpath and you WILL find that in the container, /projectpath will reflect /c/projectpath in realtime. There are no errors or any other ways to detect this issue other than seeing empty mounts inside your guest.
You must also be sure to "share the drive" in the Docker for Windows settings.
Jul 2015 update - boot2docker now supports direct mounting. You can use -v /var/logs/on/host:/var/logs/in/container directly from your Mac prompt, without double mounting
I've been having the same issue.
My command line looked like this:
docker run --rm -i --name $NAME -v `pwd`:/sources:z $NAME
The problem was with 'pwd'. So I changed that to $(pwd):
docker run --rm -i --name $NAME -v $(pwd):/sources:z $NAME
How do I link the main_folder to the test_container folder present inside the docker container?
Your command below is correct, unless your on a mac using boot2docker(depending on future updates) in which case you may find the folder empty. See mattes answer for a tutorial on correcting this.
docker run -d -v /Users/kishore/main_folder:/test_container k3_s3:latest
I need to make this run automatically, how to do that without really
using the run -d -v command.
You can't really get away from using these commands, they are intrinsic to the way docker works. You would be best off putting them into a shell script to save you writing them out repeatedly.
What happens if boot2docker crashes? Where are the docker files stored?
If you manage to use the -v arg and reference your host machine then the files will be safe on your host.
If you've used 'docker build -t myimage .' with a Dockerfile then your files will be baked into the image.
Your docker images, i believe, are stored in the boot2docker-vm. I found this out when my images disappeared when i delete the vm from VirtualBox. (Note, i don't know how Virtualbox works, so the images might be still hidden somewhere else, just not visible to docker).
Had the same problem. Found this in the docker documentation:
Note: The host directory is, by its nature, host-dependent. For this reason, you can’t mount a host directory from Dockerfile, the VOLUME instruction does not support passing a host-dir, because built images should be portable. A host directory wouldn’t be available on all potential hosts.
So, mounting a read/write host directory is only possible with the -v parameter in the docker run command, as the other answers point out correctly.
I found that any directory laying under system directive like /var, /usr, /etc could not be mount under the container.
The directive should be at user's space -v switch instructs docker daemon to mount local directory to the container, for example:
docker run -t -d -v /{local}/{path}:/{container}/{path} --name {container_name} {imagename}
Here's an example with a Windows path:
docker run -P -it --name organizr --mount src="/c/Users/MyUserName/AppData/Roaming/DockerConfigs/Organizr",dst=/config,type=bind organizrtools/organizr-v2:latest
As a side note, during all of this hair pulling, having to wrestle with figuring out, and retyping paths over and over and over again, I decided to whip up a small AutoHotkey script to convert a Windows path to a "Docker Windows" formatted path. This way all I have to do is copy any Windows path that I want to use as a mount point to the clipboard, press the "Apps Key" on the keyboard, and it'll format it into a path format that Docker appreciates.
For example:
Copy this to your clipboard:
C:\Users\My PC\AppData\Roaming\DockerConfigs\Organizr
press the Apps Key while the cursor is where you want it on the command-line, and it'll paste this there:
"/c/Users/My PC/AppData/Roaming/DockerConfigs/Organizr"
Saves a lot to time for me. Here it is for anyone else who may find it useful.
; --------------------------------------------------------------------------------------------------------------
;
; Docker Utility: Convert a Windows Formatted Path to a Docker Formatter Path
; Useful for (example) when mounting Windows volumes via the command-line.
;
; By: J. Scott Elblein
; Version: 1.0
; Date: 2/5/2019
;
; Usage: Cut or Copy the Windows formatted path to the clipboard, press the AppsKey on your keyboard
; (usually right next to the Windows Key), it'll format it into a 'docker path' and enter it
; into the active window. Easy example usage would be to copy your intended volume path via
; Explorer, place the cursor after the "-v" in your Docker command, press the Apps Key and
; then it'll place the formatted path onto the line for you.
;
; TODO:: I may or may not add anything to this depending on needs. Some ideas are:
;
; - Add a tray menu with the ability to do some things, like just replace the unformatted path
; on the clipboard with the formatted one rather than enter it automatically.
; - Add 'smarter' handling so the it first confirms that the clipboard text is even a path in
; the first place. (would need to be able to handle Win + Mac + Linux)
; - Add command-line handling so the script doesn't need to always be in the tray, you could
; just pass the Windows path to the script, have it format it, then paste and close.
; Also, could have it just check for a path on the clipboard upon script startup, if found
; do it's job, then exit the script.
; - Add an 'all-in-one' action, to copy the selected Windows path, and then output the result.
; - Whatever else comes to mind.
;
; --------------------------------------------------------------------------------------------------------------
#NoEnv
SendMode Input
SetWorkingDir %A_ScriptDir%
AppsKey::
; Create a new var, store the current clipboard contents (should be a Windows path)
NewStr := Clipboard
; Rip out the first 2 chars (should be a drive letter and colon) & convert the letter to lowercase
; NOTE: I could probably replace the following 3 lines with a regexreplace, but atm I'm lazy and in a rush.
tmpVar := SubStr(NewStr, 1, 2)
StringLower, tmpVar, tmpVar
; Replace the uppercase drive letter and colon with the lowercase drive letter and colon
NewStr := StrReplace(NewStr, SubStr(NewStr, 1, 2), tmpVar)
; Replace backslashes with forward slashes
NewStr := StrReplace(NewStr, "\", "/")
; Replace all colons with nothing
NewStr := StrReplace(NewStr, ":", "")
; Remove the last char if it's a trailing forward slash
NewStr := RegExReplace(NewStr, "/$")
; Append a leading forward slash if not already there
if RegExMatch(NewStr, "^/") == 0
NewStr := "/" . NewStr
; If there are any spaces in the path ... wrap in double quotes
if RegExMatch(NewStr, " ") > 0
NewStr := """" . NewStr . """"
; Send the result to the active window
SendInput % NewStr
To get this working in Windows 10 I had to open the Docker Settings window from the system tray and go to the Shared Drives section.
I then checked the box next to C. Docker asked for my desktop credentials to gain authorisation to write to my Users folder.
Then I ran the docker container following examples above and also the example on that settings page, attaching to /data in the container.
docker run -v c:/Users/<user.name>/Desktop/dockerStorage:/data -other -options
boot2docker together with VirtualBox Guest Additions
How to mount /Users into boot2docker
https://medium.com/boot2docker-lightweight-linux-for-docker/boot2docker-together-with-virtualbox-guest-additions-da1e3ab2465c
tl;dr Build your own custom boot2docker.iso with VirtualBox Guest
Additions (see link) or download
http://static.dockerfiles.io/boot2docker-v1.0.1-virtualbox-guest-additions-v4.3.12.iso
and save it to ~/.boot2docker/boot2docker.iso.
Note that in Windows you'll have to provide the absolute path.
Host: Windows 10
Container: Tensorflow Notebook
Below worked for me.
docker run -t -i -v D:/projects/:/home/chankeypathak/work -p 8888:8888 jupyter/tensorflow-notebook /bin/bash
i had same issues , i was trying to mount C:\Users\ folder on docker
this is how i did it Docker Toolbox command line
$ docker run -it --name <containername> -v /c/Users:/myVolData <imagename>
You can also do this with Portainer web application for a different visual experience.
First pull the Portainer image:
docker pull portainer/portainer
Then create a volume for Portainer:
docker volume create portainer_data
Also create a Portainer container:
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer
You will be able to access the web app with your browser at this URL: "http://localhost:9000". At the first login, you will be prompted to set your Portainer admin credentials.
In the web app, follow these menus and buttons: (Container > Add container > Fill settings > Deploy Container)
I had trouble to create a "mount" volume with Portainer and I realized I had to click "bind" when creating my container's volume. Below is an illustration of the volume binding settings that worked for my container creation with a mounted volume binded to the host.
P.S.: I'm using Docker 19.035 and Portainer 1.23.1
I had the same requirement to mount host directory from container and I used volume mount command. But during testing noticed that it's creating files inside container too but after some digging found that they are just symbolic links and actual file system used form host machine.
Quoting from the Official Website:
Make sure you don’t have any previous getting-started containers running.
Run the following command from the app directory.
x86-64 Mac or Linux device:
docker run -dp 3000:3000 \
-w /app -v "$(pwd):/app" \
node:12-alpine \
sh -c "yarn install && yarn run dev"
Windows (PowerShell):
docker run -dp 3000:3000 `
-w /app -v "$(pwd):/app" `
node:12-alpine `
sh -c "yarn install && yarn run dev"
Aple silicon Mac or another ARM64 device:
docker run -dp 3000:3000 \
-w /app -v "$(pwd):/app" \
node:12-alpine \
sh -c "apk add --no-cache python2 g++ make && yarn install && yarn run dev"
Explaining:
dp 3000:3000 - same as before. Run in detached (background) mode and create a port mapping
w /app - sets the “working directory” or the current directory that the command will run from
v "$(pwd):/app" - bind mount the current directory from the host into the /app directory in the container
node:12-alpine - the image to use.
Note that this is the base image for our app from the Dockerfile sh -c "yarn install && yarn run dev" - the command.
We’re starting a shell using sh (alpine doesn’t have bash) and running yarn install to install all dependencies and then running yarn run dev. If we look in the package.json, we’ll see that the dev script is starting nodemon.

How do I complete this SSH tunnel from local development docker to staging database

I can create a docker containing my project code. It has unit tests that fail because there is no database connection.
I can log in to a server with a central database that contains our test data using SSH key and credentials.
I cannot get the docker and DB communicating.
I've tried several different suggestions, scratching and restarting this portion of the Dockerfile over the past two days. I've searched Youtube for tutorials, Stackexchange for answers and the docker forums for reference.
If there's a step by step tutorial, that is tucked away I would love to see that too!
The docker-compose has the following:
services:
app:
build:
context: .
dockerfile: .docker/Dockerfile
args:
APP_PATH: ${APP_PATH}
image: laravel-docker
env_file: .env
ports:
- 8080:80
# We need to expose 443 port for SSL certification.
- "443:443"
volumes:
- .:/var/www/jumbledown
Inside the container, I can contact the host of the remote DB with the following:
ssh -4 -R 8888:localhost:8888 [devname]#NN.NN.NN.NN -i ~/ident -p [portnumber]
where:
- devname is my log in.
- NN.NN.NN.NN is the IP address of the host of the DB.
- ident is a file containing the SSH key that is copied in by a copy command contained in the Dockerfile.
The Docker file is built off FROM php:7.1.8-apache and installs a LOT of extra stuff now, including Xdebug. It's too long to just copy and paste and I'm not sure what parts are relevant; I can expose at request.
Ideally, I'd like to be able to use Dockerfile to set up an SSH tunnel serving the DB to the docker container. Right now, I'd settle for being able to manually set up the connection inside the container.
Update As per questions in the answer, the end result I need to create is for several developers to have local dockers and each have a tunnel to a central database that contains testing data, for our use while we code throughout the day.
If you want the PHP container to have a permanent SSH tunnel to your remote DB, you can change you Dockerfile's COMMAND statement (assuming the ENTRYPOINT is a shell) to use a script that creates the SSH tunnel in the background, similar to what you manually, wait for the SSH tunnel and then proceed to run whatever it is you want to run.
Your question lacks the details of what you're trying to achieve (permanent tunnel? only while testing? etc.)
An example to such script:
# run ssh in background (notice the "&" at the end)
ssh -4 -R 8888:localhost:8888 $DB_USERNAME#$DB_HOST -i ~/ident -p $DB_PORT &
# wait for the ssh tunnel if needed
# ...
# run the main command here
# ...
I'd suggest to consider a different path -
Create a new service in the docker-compose file that is dedicated to open a tunnel, and then connect to that service from your PHP service.

MySQL in Docker on Windows: World-writable files ignored

I'm using Docker-Compose with the MySQL Image to fire up a MySQL database as part of a larger project.
As documented in the MySQL image's documentation, I'm mapping in a custom configuration file to /etc/mysql/conf.d/config-file.cnf:
database:
environment:
MYSQL_ROOT_PASSWORD: foo
ports:
- "3306:3306"
volumes:
- "./mysql/conf.d/:/etc/mysql/conf.d"
image: mysql:5.5
This works perfectly fine when running on Mac OS X as the host system (using docker-machine), but it fails when running on Windows (also using docker-machine). MySQL complains about the fact that the /etc/mysql/conf.d/config-file.cnf is world-writable
Warning: World-writable config file '/etc/mysql/conf.d/config-file.cnf' is ignored
When entering the database container, the file is indeed shown as having 0777 permission. This seems to be due to the host file system's permissions (Windows).
Is there any way to change this? I've tried mounting the volume in read-only mode, but the file still has the same permissions.
Any other way around this problem? At the moment, I'm mounting the file to another folder in the container and then copying/chmod'ing it to the required location as part of the startup command:
database:
environment:
MYSQL_ROOT_PASSWORD: foo
ports:
- "3306:3306"
volumes:
- "./mysql/conf.d/:/usr/local/mysqlconf"
image: mysql:5.5
command: >
bash -c "
cp /usr/local/mysqlconf/*.cnf /etc/mysql/conf.d/
&& chmod 644 /etc/mysql/conf.d/*.cnf
&& /entrypoint.sh mysqld
"
Is there a better way to solve this issue?
When you start up a container with -v, --volumes or use volumes: in docker-compose.yml
docker run -v source:/dest:rw busybox ls -l /dest
Docker mounts the source directory from the Linux VM into a container as /dest. Docker mounted volumes only provide their own options rw and r and I think z for selinux. Owner and permission info will be passed through to the container exactly as the Linux VM sees them. If someone who clones your repo runs docker or docker-compose from their local host rather than on the VM, they will mount blank directories.
Docker Machine's Users share
A docker-machine created local VM will share your computer's local users directory by default. C:\Users on Windows and /Users on OSX. This is done as a VirtualBox shared folder called Users. This share is then mounted on the Linux side via VirtualBox's vboxsf mount tool as /Users or maybe /c/Users via a Linux startup script /etc/rc.d/vbox.
When you docker-machine ssh default you should be able to see all your computers file at /Users/nwinkler.
This share allows docker-compose to reference a relative, local directory within C:\Users and have it work on the Linux VM. Outside of C:\Users the data doesn't exist on the VM.
World readable files
I believe what you are seeing is vboxsf's POSIX representation of a Windows file system. If you run:
docker-machine ssh default
$ cd /Users/nwinkler/path/to/mysql-docker
$ ls -l
$ docker run -v $PWD:/mysql busybox ls -l /mysql
You should see all your files as world writable. The only way to change the represented permissions is via the vboxsf mounted share.
The mount options vboxsf provides are:
Available mount options are:
rw mount writable (the default)
ro mount read only
uid=UID set the default file owner user id to UID
gid=GID set the default file owner group id to GID
ttl=TTL set the "time to live" to TID for the dentry
dmode=MODE override the mode of all directories to (octal) MODE
fmode=MODE override the mode of all regular files to (octal) MODE
umask=UMASK set the umask to (octal) UMASK
dmask=UMASK set the umask applied to directories only
fmask=UMASK set the umask applied to regular files only
iocharset CHARSET use the character set CHARSET for I/O operations
(default set is utf8)
convertcp CHARSET convert the folder name from CHARSET to utf8
On the Docker Linux VM, edit /etc/rc.d/box and append the options to the mountOptions variable. These options will apply to all files and directories under /Users on that mount.
You could set an fmask=007 to remove other permissions from all files or fmode=750 to override all permissions for all files.
mountOptions='defaults,iocharset=utf8'
if grep -q '^docker:' /etc/passwd; then
mountOptions="${mountOptions},uid=$(id -u docker),gid=$(id -g docker),fmask=007"
fi
If you ever upgrade or recreate your docker-machine VM you will need to do this again.
I tend to skip relying on virtualbox shares and have local files monitored and synchronised to my hosts on change (fsevents and rsync).

Docker - can not start mysql permission error

I have problem with docker and mysql. I have build an image based on phusion/baseimage. I want to create image where /var/lib/mysql directory is shared with my host (OS X), because I dont want to store my data on container.
Everything works fine when directory /var/lib/mysql is not shared. When I share this directory, mysql service can not start. In logs are informations about problems with permissions while starting.
Result of ls -la from /var/lib is:
[...]
drwxr-xr-x 1 lc staff 170 Jan 3 16:55 mysql
[...]
The mysql user should be an owner. I tried to do:
sudo chown -R mysql:mysql mysql/
But this command didn't return any error and didn't change owner.
I have also tried to add my user (from container) to mysql group:
lc#cbe25ac0681e:~$ groups lc
lc : lc sudo mysql
But It also didn't work. Have anybody any idea how to solve this issue?
My docker-compose.yml file:
server:
image: lukasz619/light-core_server
ports:
- "40080:80"
- "40022:22"
- "40443:443"
- "43306:3306"
volumes:
- /Users/lukasz/Workspace/database/mysql:/var/lib/mysql
This is my Dockerfile:
FROM phusion/baseimage
RUN apt-get update
RUN apt-get install -y apache2
# Enable SSH
RUN rm -f /etc/service/sshd/down
RUN /etc/my_init.d/00_regen_ssh_host_keys.sh
# public key for root
ADD public_key.pub /tmp/public_key.pub
RUN cat /tmp/public_key.pub >> /root/.ssh/authorized_keys && rm -f /tmp/public_key.pub
EXPOSE 80
CMD ["/sbin/my_init"]
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
When we say that users are shared between the container and the host, it's not quite true; it's the UIDs that are actually shared. What this means is that the user named mysql in the container most likely has a different UID to the user mysql on the host.
We can try to fix this by using the image to set the permissions:
docker run --user root lukasz619/light-core_server chown -R mysql:mysql /var/lib/mysql
This may work, depending on how you've set up the image. There is also a further complication from the VM running in-between.
The problem is OS X.. I ran this docker-composer.yml
server:
image: lukasz619/light-core_server
ports:
- "40080:80"
- "40022:22"
- "40443:443"
- "43306:3306"
volumes:
- /var/mysql:/var/lib/mysql
Its possible to share directories (and do chown) ONLY between boot2docker and container, but do not work properly shareing between Os X and container.

dockerfile - Unable to connect error

So here's what i have to do: i need to set up some containers automatically using docker. One of them is liek this: Debian Squeeze with limited CPU shares and limited memory (1 cpu share and 512 mb memory),preinstalled apache2,build-essential,php5,mysql-server-5.5,openssh-server and with some ports opened (8000 for Apache and 1500 for MySQL). So i created the following dockerfile :
FROM debian:squeeze
MAINTAINER Name < email : >
# Update the repository sources list
RUN apt-get update
# Install apache, PHP, and supplimentary programs. curl and lynx-cur are for debugging the container.
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install apache2 build-essential php5 mysql-server openssh-server libapache2-mod-php5 php5-mysql php5-gd php-pear php-apc php5-curl curl lynx-cur
# Enable apache mods.
RUN a2enmod php5
RUN a2enmod rewrite
# Manually set up the apache environment variables
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
EXPOSE 80
# Copy site into place.
ADD www /var/www/site
# Update the default apache site with the config we created.
ADD apache-config.conf /etc/apache2/sites-enabled/000-default.conf
# By default, simply start apache.
CMD /usr/sbin/apache2ctl -D FOREGROUND
#CMD [ "mysqladmin -u root password mysecretpasswordgoeshere"]
EXPOSE 3306
the content of apache-config.conf is this:
<VirtualHost *:80>
ServerAdmin me#mydomain.com
DocumentRoot /var/www/site
<Directory /var/www/site/>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order deny,allow
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
and in www folder i put a php file with this code:
<?php
$connect=mysql_connect("localhost:1500","root","") or die("Unable to Connect");
?>
to test the connection to the mysql server
then
i build all this into an image like this:
sudo docker build --rm --tag="tag_name" .
and then i run the image like this
sudo docker run -c=1 -m="512m" --net=bridge -p 8000:80 -p 1500:3306 -d --name="container_name" tag_name
It seems to work,the apache server works when i access localhost:8000/site in my browser but is shows "Unable to connect". what am i doing wrong?
And another problem is that,the contaienr is running but i can't attach to it.I run this command
sudo docker attach CONTAINER_ID
and then nothing happens,can't do anythign else from there,What am i doing wrong?
I have to build few more dockerfiles similar to this to create containers.All those must be hosted on a ZFS file system and i have to configure a container repository of 50gb based on it,what does this mean and how do i do that?
I'm sorry for my english,it's not my native language :(
Thank you in advance
MySQL issue
in the PHP code
$connect=mysql_connect("localhost:1500","root","") or die("Unable to Connect");
localhost refers to the container IP address. Since there is no MySQL server running in that container the connection will fail.
In this gist, I've changed a bit your example to have the container start both MySQL and Apache (I assume this was your first intent) using the following instruction: CMD bash -c '(mysqld &); /usr/sbin/apache2ctl -D FOREGROUND' and changed the PHP code to connect to the MySQL server on localhost:3306.
Docker attach
The docker attach command is meant to allow you to interact with the process currently running in the foreground of a container. Unless that process is a shell, it won't provide you with a shell in that container.
Take this example:
Start a container running a shell process
docker run -it --rm base bash
You are now in interactive mode in your container and can play around with the shell running in the foreground in that container:
root#de8f16a13571:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin selinux srv sys tmp usr var
if you now exit the shell by typing exit the shell process will end, and as that was the process running in the foreground in the container, that container will stop.
root#de8f16a13571:/# exit
exit
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Now start a new container named test running bash again:
docker run -it --name test base bash
verify you can interact with it and detach from it by hitting keys Ctrl+p+q. You end up back in the docker host shell.
verify that the container named test is still running:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
81f0f1094f4a base:latest "bash" 6 seconds ago Up 5 seconds test
You can then use the docker attach command to attach to the bash program in the container:
docker attach test
root#81f0f1094f4a:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin selinux srv sys tmp usr var
ZSH
And regarding ZSH, I don't know what all that means either. Also note that having 3 questions at once makes it difficult for the community to come up with a single answer that would answer all 3 ; maybe consider posting a new question for those.
Please comment if my assumptions about how you run MySQL or what your intent is with docker attach are wrong.