I'm at my wit's end with this, so hopefully you folks can help me. In OSX 10.11.2 with docker-machine, I've got a docker-compose file that should build a local Dockerfile and attach a MySQL container to it. The MySQL container should mount a local folder where I'm storing my database data, so if the container or VM comes down, I can just restart it without data loss. Problem is, when I run it, it throws a permissions error:
db_1 | 2015-12-23 19:17:59 7facaa89b740 InnoDB: Operating system error number 13 in a file operation.
db_1 | InnoDB: The error means mysqld does not have the access rights to
db_1 | InnoDB: the directory.
I've tried every permutation I can think of to get this to work. I was reading around and it may have something to do with how docker-machine handles permissions with OSX, but the documentation for docker-machine says that it mounts the /Users folder, so that shouldn't be an issue.
Here's the docker-compose.yml:
web:
build: .
ports:
- "3000:3000"
links:
- db
db:
image: mysql:5.6
ports:
- "3306:3306"
volumes:
- /Users/me/Development/mysql-data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: mypass
Any ideas? I can't help but think it's something really simple. Any help would be most appreciated!
Edit:
Host - drwxr-xr-x 7 me staff 238 Dec 23 12:10 mysql-data/
VM - drwxr-xr-x 1 docker staff 238 Dec 23 20:10 mysql-data/
As to the container, it won't run with the volume mounted. Without the -v mount, it is:
Container - drwxr-xr-x 4 mysql mysql 4096 Dec 24 00:37 mysql
The issue this comes from is the userids used by Mac and Linux respectively. Mac does not like Linux wanting to use the 1 for the userID.
The way I worked around all the permissions craziness in my mac + docker-machine setup is to use this Dockerfile
FROM mysql:5.6
RUN usermod -u 1000 mysql
RUN mkdir -p /var/run/mysqld
RUN chmod -R 777 /var/run/mysqld
Instead of the plain MySQL 5.6 Image.
The last 2 lines are necessary, because changing the userid for the mysql user will mess up the build in permissions for that image. => you need the 777 permissions to make it run here :/
I know this is a little hacky, but so far the best solution I know to the permissions issue here.
Try to use the latest docker for mac instead of docker tools. Docker for Mac no longer uses VirtualBox, but rather HyperKit, a lightweight OS X virtualization solution built on top of Hypervisor.framework in OS X 10.10 Yosemite and higher.
I suggest also completely remove docker tools(they could co-exist): https://github.com/docker/toolbox/blob/master/osx/uninstall.sh
With docker for mac, you don't have to use permission hacks, it would just work like it would be on a linux build.
Related
I'm trying to run a mysql docker container with persistent data mapped to a folder that is mounted through CIFS.
(I orginally posted a more general question but got a bit further in terms of user rights and now the error seems to be specific to innodb/cifs mounts. Hence I'm reposting this as a new topic. Thanks for life888888 for the initial help.)
Here is the command to start the container:
docker run \
--name localmysql \
-v /mnt/mysqlshare:/var/lib/mysql \
--rm \
--env MYSQL_ALLOW_EMPTY_PASSWORD=true \
-it \
mysql:8.0.31-debian
/mnt/mysqlshare is a mounted cifs share. The command to map the CIFS mount is below:
sudo mount -t cifs -o username=linuxmount,cache=none,vers=3.0,uid=999,gid=999,rw [networkpath] /mnt/mysqlshare
On the machine where docker is running I have a user set up called "mysql" which is the owner of the mapped cifs drive. It is configured to have 999 as uid and group (which ties to mysql-user in the docker container which is used by default).
When remoting into the container (interactive session) and changing to "su mysql" I'm able to write into /var/lib/mysql and changes are reflected in the mounted drive.
However when starting up the container I'm getting the following error which just keeps repeating:
2022-12-08T21:12:56.435340Z 0 [ERROR] [MY-012894] [InnoDB] Unable to open './#innodb_redo/#ib_redo0' (error: 11).
2022-12-08T21:12:56.435784Z 0 [ERROR] [MY-012574] [InnoDB] Unable to lock ./#innodb_redo/#ib_redo0 error: 13
There's files added to the folder:
I had that same issue on MariaDB and I'm having some success with the following (in docker-compose.yml file):
redacted-volume:
driver: local
driver_opts:
type: "cifs"
o: "mfsymlinks,vers=3.0,username=redacted,password=redacted,uid=999,gid=999"
device: "//192.168.4.204/docker-data/redacted/data-db"
mfsymlinks enables symlinks that MariaDB needs. uid and gid are Linux UserId and GroupId of MariaDB user that accesses the share. This works for me with Windows Server connecting to SMB shares from TrueNAS Scale. Docker version 20.10.22.
I still have this same issue with MySQL. I can't get it to work with this same configuration and I feel like I have looked everywhere on the internet.
I had a spring-boot project that used mysql docker-image so I didn't need to download the mysql benchwork. For other reasons I had to start over so I created a new project that uses the same mysql docker image I previously used.
My docker-compose.yml mysql service looks like this
version: "3.7"
services:
db:
image: mysql:5.7
command: --lower_case_table_names=1
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: farming_db
MYSQL_USER: root
MYSQL_PASSWORD: root
restart: always
volumes:
- "./database/farming_db/:/var/lib/mysql" #local
- farming_db:/var/lib/mysql/data #docker
ports:
- "3306:3306"
container_name: farming_mysql
networks:
- backend-network
When I run
docker-compose up
This is the error :
Attaching to farming_mysql, farming_server_springboot_1
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:21+00:00 [Note] [Entrypoint]: Initializing database files
farming_mysql | 2021-03-18T07:03:21.058436Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server opti
on (see documentation for more details).
farming_mysql | 2021-03-18T07:03:21.063630Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
farming_mysql | 2021-03-18T07:03:21.063710Z 0 [ERROR] Aborting
farming_mysql |
farming_mysql exited with code 1
springboot_1 |
I understood that my directory is not empty. I am trying to use "./database/farming_db/:/var/lib/mysql" and "farming_db:/var/lib/mysql/data" both as the volume directories. I think the problem is with the latter directory because the prior directory is empty. I'm having a problem deleting the contents in the latter directory because I don't know how to access it.
So this is what I've tried :
I deleted all the containers and then deleted all the volumes.docker volume prune but didn't work.
I searched that I could do rm -rf /usr/local/var/mysql but I don't know where I can execute this command since the container won't run properly at all.
I deleted the mysql image and just ran docker-compose up again. This seems to pull a new mysql image from somewhere? but I still get the same error. I guess volume directory has nothing do with the docker image itself.
I deleted the "- farming_db:/var/lib/mysql/data #docker" line from the docker-compose. But the same error is still occuring!
I'm using Windows10.
My question :
How can I access the directory? I don't know where to use the rm -rf command.
Why does this error still occur even when I erase "- farming_db:/var/lib/mysql/data #docker" from the docker-compose?
And also could anyone explain what I am doing? I'm new to docker and I don't really understand these volume problems.
Run docker system prune --volumes
This frees up the memory by removing all unused containers. Sometimes, the mentioned issue can occur due to memory limitations
Generally I emptied the volume's data directory and just changed the versions of the MySQL.
So in steps:
empty volume directory content
modify docker-compose.yml mysql version from 5.7 to 5.7.16
This line indicate that mysql container is storing the data inside a directory database in the same directory than your docker-compose.yml:
volumes:
- "./database/farming_db/:/var/lib/mysql" #local
This kind of volume isn't managed by Docker, it's just a directory in your filesystem, this is why docker volume prune doesn't work. I know that, because it starts with a "path" relative or absolute.
The other volume, farming_db, are managed by Docker. I know that because it starts with a simple name. This kinds of volume are managed by Docker and are removed with prune.
So, answering:
In the same directory than your docker-compose.yml you can remove that database folder.
Because the first volume, the one with /var/lib/mysql still exists. MySQL keeps all files inside this directory and any other child directory are a database.
You're just trying to put a container running and docker-compose hides a lot of details.
This is just a detail, but MYSQL_USER should be different than root.
You can let Docker manage the entire volume, creating a single volume to hold all data, in this case I named it as mysql_data:
volumes:
- mysql_data:/var/lib/mysql
Or, you can explore a bit more the docker run equivalent command to get used with it:
docker run -d --name mysql \
-e MYSQL_ROOT_PASSWORD=root \
-e MYSQL_DATABASE=farming_db \
-e MYSQL_USER=myuser \
-e MYSQL_PASSWORD=mypass \
-v mysql_data:/var/lib/mysql \
-p 3306:3306 \
mysql:5.7
As vencedor's answer, it worked for me. If anyone need stay with mysql 5.7, you can add these lines to your db service in docker-compose.yml:
- /etc/group:/etc/group:ro
- /etc/passwd:/etc/passwd:ro
user: "1000:1000"
I used docker-compose to run mysql image and encountered the error.
I use the following configuration to set volume.
- ./mysql/data:/var/lib/mysql/data
Then I changed it to the following and the error was solved.
- ./mysql:/var/lib/mysql
I have a docker-compose.yml with a MariaDB database service that looks like this:
version: '2'
services:
db:
container_name: misp_db
image: mariadb
restart: unless-stopped
volumes:
- ${DATA_DIR}/db:/var/lib/mysql
environment:
- MYSQL_DATABASE=${MYSQL_DATABASE}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
When spinning up my containers, I noticed these errors:
[ERROR] Missing system table mysql.roles_mapping; please run mysql_upgrade to create it
[ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','IGNORE_BAD_TABLE_OPTIONS','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES','ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER','HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH','EMPTY_STRING_IS_NULL','SIMULTANEOUS_ASSIGNMENT'), found type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_B
[ERROR] mysqld: Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.
I was able to work around this problem by running docker exec -it db /bin/bash to get a shell, and then running mysql_upgrade -p and entering my configured MySQL root password, but this should be automated.
How can I do this in Docker Compose?
You can't run it directly from the docker-compose but it can be done rather simply using a one-line Dockerfile called from docker-compose.
This can be done by simply adding the following to your service definition in the docker-compose.yaml file:
build: .
This will cause docker-compose to run the Dockerfile located in the working directory ".".
In the Dockerfile, you can then run the upgrade script itself.
RUN mysql_upgrade -p
This will cause the upgrade to be run once; while building the image. Subsequently, your mySQL version will be frozen at the time you build the image; not when you start the container. This seems like a reasonable approach to me.
You could run an entrypoint.sh script from the Dockerfile (see here for more details), which gets called when you start the container. That would perform the upgrade when you start the container as you originally requested but, I agree with #blokje5, upgrading when starting the container seems a bit risky. If they accidentally break something (it happens), then an image that once worked would no longer be working and that can be a bit of a nightmare to debug; especially since it's happening before the service is considered ready.
I'm having an issue when starting the db service with docker compose:
version: '3'
services:
# Mysql DB
db:
image: percona:5.7
#build: ./docker/mysql
volumes:
- "./db/data:/var/lib/mysql"
- "./db/init:/docker-entrypoint-initdb.d"
- "./db/backups:/tmp/backups"
- "./shared/home:/home"
- "./shared/root:/home"
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: db_name
MYSQL_USER: user
MYSQL_PASSWORD: pass
ports:
- "3307:3306"
I have tried everything with no luck:
"./db/data:/var/lib/mysql:rw"
Creating a dockerfile and create from build instead of image:
FROM percona:5.7
RUN adduser mysql
RUN sudo chown mysql /var/lib/mysql
RUN sudo chgrp mysql /var/lib/mysql
Also I have tried to add a user on db service:
user: "1000:50"
But any of those could solve that.. What I'm missing?
MySQL 5.7 installation error `mysqld: Can't create/write to file '/var/lib/mysql/is_writable'`
I had to change ./db/data user:group to 999:999, so docker user is who is making the changes.
sudo chown 999:999 ./db/data
Make sure that the user who is running docker has access to ./db/data
# Not in the dockerfile
sudo chown $(whoami) ./db/data
sudo chgrp $(whoami) ./db/data
Docker tells you that you don't have the permissions, it might also mean that you need to verify that you shared volume ./db/data need to have the correct permissions.
According Dockerfile Percona 5.7 images runs under CentOS v8 and user mysql. Check the user ID (uid) and group ID (gid) inside container:
user#host$ docker run --rm -t percona:5.7.29 sh -c 'id'
uid=999(mysql) gid=999(mysql) groups=999(mysql)
Default user inside container uses 999 uid and gid. Than change your directory rights to 999:999:
sudo chown 999:999 ./db/data
This is an addition to Albeis answer.
I spent a whole day with a similar (almost exactly) problem. I also changed ownership of the related files, only to see them get wiped out and come back with permissions issues. I changed the ownership of my curl-installed docker-compose executable. I didn't receive a bit of reprieve until adding the volumes to the .dockerignore, as was suggested in this Github issue reply.
I suffered this issue and it took quite some time to figure out what the culprit was.
In my case, I have a dual boot system Winblowz-Linux.
My code of the problematic project was on a Windows filesystem.
Once I cloned the project into my Linux drive, on a ext4 filesystem, the problem went away.
need permission to execute scripts in directory
sudo chown 999:999 ./db/data
sudo chmod +x ./db/data
Using Laradock
System Info:
Docker version: 17.10.0-ce, build f4ffd25
OS: Windows 10 Home
When I run docker-compose up -d mysql I'm getting error. Following is the docker logs
[Note] Basedir set to /usr/
[Warning] The syntax '--symbolic-links/-s' is deprecated and will be removed in a future release
[Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
[ERROR] --initialize specified but the data directory has files in it. Aborting.
[ERROR] Aborting
I have tried deleting mysql folder under ~/.laradock\data and didn't work.
Update 1
MySQL Container under laradock Dockerfile
mysql:
build:
context: ./mysql
args:
- MYSQL_VERSION=${MYSQL_VERSION}
environment:
- MYSQL_DATABASE=${MYSQL_DATABASE}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
- TZ=${WORKSPACE_TIMEZONE}
volumes:
- ${DATA_SAVE_PATH}/mysql:/var/lib/mysql
- ${MYSQL_ENTRYPOINT_INITDB}:/docker-entrypoint-initdb.d
ports:
- "${MYSQL_PORT}:3306"
networks:
- backend
MySQL Dockerfile
ARG MYSQL_VERSION=8.0
FROM mysql:${MYSQL_VERSION}
MAINTAINER Mahmoud Zalt <mahmoud#zalt.me>
#####################################
# Set Timezone
#####################################
ARG TZ=UTC
ENV TZ ${TZ}
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN chown -R mysql:root /var/lib/mysql/
ADD my.cnf /etc/mysql/conf.d/my.cnf
CMD ["mysqld"]
EXPOSE 3306
Update 2
After I delete mysql folder under ~/.laradock/data I'm getting following error. After the command it generates the files in below image. When I rerun giving back the previous error mentioned above.
[Note] Basedir set to /usr/
[Warning] The syntax '--symbolic-links/-s' is deprecated and will be removed in a future release
[Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
[Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
[Warning] You need to use --log-bin to make --log-slave-updates work.
libnuma: Warning: /sys not mounted or invalid. Assuming one node: No such file or directory
mbind: Operation not permitted
[ERROR]
InnoDB: Operating system error number 22 in a file operation.
[ERROR] InnoDB: Error number 22 means
'Invalid argument'
[ERROR] InnoDB: File
./ib_logfile101: 'aio write' returned OS error 122. Cannot continue
operation
[ERROR] InnoDB: Cannot
continue operation.
** I tried in a windows 7 machine and its working.
Disable AIO
This fixed it for me when I got the AIO error as you did when I was starting a container from a guest Debian OS from Virtualbox and creating the database files on a shared folder on Windows 10.
The issue seems to be that AIO is not supported on shared folders, or at least on some versions of Windows. It seems to have occurred for me after I moved from Windows 10 Pro to Home after my main machine crashed.
For details:
aio
disable aio in MySQL for zfs
Here are some options:
Option 1 - start the container like this :
docker run -it mysql --innodb_use_native_aio=0
Option 2 - add the command to your docker-compose file:
command: --innodb_use_native_aio=0
In context, this is the relevant portion of my working docker-compose.yml:
services:
db:
image: ${MYSQL_IMAGE}
command: "--innodb_use_native_aio=0"
volumes:
- ${DB_DATA_PATH}:/var/lib/mysql
ports:
- ${MYSQL_PORT}:3306
Option 3 -- add an option to your my.cnf file in your build
innodb_use_native_aio=0
Option 4 - Don't persist your DB on the local file system.(Can destroy your db, Not Recommended)
Simply remove the volume in your docker configuration that contains your mysql db. Of course, your DB will be deleted if you do a docker-compose down or otherwise destroy your container, so there's that.
I am not sure but Try this step.This is because of data folder.
Try remove the docker image. List all images using docker images then remove the mysql & laradock mysql using docker rmi imagename. Dont forget remove the docker volume using docker volume rm volumename
Also go to cd ~/.laradock/data remove the mysql folder.
Then try docker-compose up mysql for debuging. If no errors, you could try docker-compose up -d mysql
I had the same problem with my Windows 10 Enterprise, I couldn't find perfect solution around it because it seems to be my Windows version issue - I have ran the same image on other windows (8 & 10 professional) successfully. My temporary work around was to remove the /var/lib/mysql mount entirely from the docker-compose file allowing the database data files to be created and modified in the container itself.
I use kitematics which will restart and reattach my created container anytime I want to work on the container, like that I don't loose my data when the container exit. If you don't use kitematics this what it does.
After running docker-compose the image will be created and a container will be created to run it as well, I avoid using run on the created image because this will create new container
I stick to running the following sequentially
docker ps -a # this command will get container_id of all container, those that are running and those that are not
docker start <container_id> # start container from background
docker attach <container_id> # attach container to standard input
With these docker commands my data were preserved in the container even after exiting and restarting
BACK UP
Now whenever I want to move the data, I will commit my container into an image and save the image to local storage
docker commit <container_id> <backup_image_name>:<1_31_2017> # I use date to tag it
docker save -o <local_storage_tar_name> <backup_image_name>
RESTORE
Whenever I need to restore the mysql container either on my computer or for a new programming intern
cd <dir_containing_the_tar_file>
docker load -o <local_storage_tar_name>
This solution is for your update
File ./ib_logfile101: 'aio write' returned OS error 122. Cannot continue operation
I hope this helps somebody
In addition to the response above by AndrewD, you should change the permission on the my.cnf so that it does not get ignored when booting up. You can run the below command,
sudo chmod 400 my.cnf
Do this right before running docker-compose up -d