I'm running on EC2, and want to have MYSQL's datadir on another EBS. I mounted the EBS (ext3) on /data and changed the datadir in my.cnf to point to that dir. However, when I try to start mysql, I get this error:
120518 03:53:17 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
120518 03:53:28 mysqld_safe Starting mysqld daemon with databases from /data/mysql/
120518 3:53:28 [Warning] Can't create test file /data/mysql/ip-10-136-23-225.lower-test
120518 3:53:28 [Warning] Can't create test file /data/mysql/ip-10-136-23-225.lower-test
/usr/libexec/mysqld: Can't change dir to '/data/mysql/' (Errcode: 13)
120518 3:53:28 [ERROR] Aborting
120518 3:53:28 [Note] /usr/libexec/mysqld: Shutdown complete
120518 03:53:28 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Any ideas?
If you are running on a distro like Ubuntu, then AppArmor is probably blocking mysqld from being able to access files in a different directory. If you check your system log files, you'll find a cryptic error message to this effect.
Solutions include:
Disable AppArmor (not recommended)
Edit the AppArmor rules (complicated)
Use "mount bind" to make MySQL think that your data files are in the original location while they are actually over on the EBS volume. Revert your changes to datadir.
I wrote an article for Amazon years back describing community best practices for exactly what you are trying to do including commands for the mount bind example:
Running MySQL on Amazon EC2 with EBS
http://ec2ebs-mysql.notlong.com
Note that the AMI id in the article is old. Using a modern Ubuntu AMI, you'll need to replace /dev/sdh with /dev/xvdh in the mkfs.xfs and /etc/fstab (but not in the ec2 tools command lines).
If you're running Ubuntu, you can run into this error due to the apparmor service. To get around this. To get around it, you need to do the following:
1) Edit '/etc/apparmor.d/usr.sbin.mysqld' and add these lines:
/opt/lib/mysql/ r,
/opt/lib/mysql/** rwk,
( assuming your new data directory is in /opt/lib/mysql )
2) re-start the apparmor service:
sudo service apparmor restart
Change owner of /data directory to mysqld process owner (chown owner /data ). or (chmod 777 -R /data) very unsafe.
Related
Last night, my MySQL server went down unexpectedly. On attempting to restart (with service mysql restart - I'm root) it merely hangs. With the mysql -u root -p command, I get:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2).
I then tried to start the daemon manually (mysqld). The prompt would hang for about 2 seconds, and then return. On closer inspection of the error logs, I got:
2016-01-22T19:18:32.399584Z 0 [ERROR] Could not create unix socket lock file /var/run/mysqld/mysqld.sock.lock.
2016-01-22T19:18:32.399622Z 0 [ERROR] Unable to setup unix socket lock file.
2016-01-22T19:18:32.399646Z 0 [ERROR] Aborting
I then tried chown mysql /var/run/mysqld, chmod -R 775 /var/run/mysqld and apt-get install mysql-community-server --reinstall. No luck.
I have looked around, and couldn't find a solution. Any help here?
Note: I am Running Debian 8 (Jessie) with MySQL community Server 5.7.10
I just ran into this. The mysql install was a little weird and was missing
/var/run/mysqld/
MySQLd was attempting to create the socket and lock file but the parent directory was missing. I simply created the directory
sudo mkdir /var/run/mysqld
chowned it to mysql
sudo chown mysql:mysql /var/run/mysqld
I was then able to start mysqld normally.
I experienced the same issue today on my Ubuntu server. But I think the solution is NOT to disable locking. In my case, a look at the system log (dmesg | tail and journalctl -xe) revealed that it's an apparmor issue and the solution can be found in this question.
I had following error
[ERROR] Unix socket lock file is empty /var/lib/mysql/mysql.sock.lock.
This error came up as a result that my server was lacking space 100% use (see below)
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 41153856 28531384 12187364 100% /
Make sure the folder exists with the proper permissions
chown mysql:mysql /var/lib/mysql/
And make sure to delete the file so the mysql server could generate a new one
rm /var/lib/mysql/mysql.sock.lock
Posting this answer, in case it helps someone with the same config as me. I had similar issue with this configuration:
Host OS: Ubuntu 20.04, MySql server: 8.0
The following error was seen in /var/log/mysql/error.log during run of systemctl start mysql
[ERROR] [MY-010273] [Server] Could not create unix socket lock file /var/run/mysqld/mysqld.sock.lock.
Fix was similar:
mkdir -p /var/run/mysqld/
chmod 777 /var/run/mysqld/
And of course, I had enough space wherever mysql server was being installed.
Below commands worked for me.
sudo mkdir /var/run/mysqld
sudo chmod 777 /var/run/mysqld -R
Also may be out of disk space.
For those running docker. If you're already running mysql in docker, you will also get such an error. Removing the mysql container solved the problem for me.
It's best to look at the log files (/usr/local/var/mysql/*.local.err) before attempting any of the more generic answers.
In my case first it complained of not being able to open the socket file ([Server] Could not open unix socket lock file /tmp/mysql.sock.lock.) and once I changed permissions, it complained of not being able to remove it! ([Server] Could not remove unix socket lock file /tmp/mysql.sock.lock errno 13.)
Removed it and run mysql.server start and DONE.
Rename /var/lib/mysql/mysql.sock.lock & /var/lib/mysql/mysql.sock to /var/lib/mysql/mysql.sock.lock.bck & /var/lib/mysql/mysql.sock.bck respectively and start mysqld. It works
If you are using docker to run MySQL, please check if there is enough allocated disk space in docker preferences.
Fixed. Add skip-external-locking to my.cnf under the [mysqld] section, and then reboot the entire system. This should fix it, for anyone who finds this. Also if you backed up your data, then attempted to restore and were told to rm ib* to make it work, you need the ibdata file.
Recently I installed MySQL Workbench 6.2 and used it to create ER diagrams of the MySQL 5.5.41 databases I'm working with on Ubuntu 12.04. Everything worked fine until I restarted Ubuntu and suddenly mysql was no longer running. Running mysqld from the command line results in this:
150216 12:50:37 [ERROR] Can't read from messagefile '/usr/share/mysql/english/errmsg.sys'
150216 12:50:37 [Warning] Can't create test file /var/lib/mysql/euler.lower-test
150216 12:50:37 [Warning] Can't create test file /var/lib/mysql/euler.lower-test
mysqld: Can't change dir to '/var/lib/mysql/' (Errcode: 13)
150216 12:50:37 [ERROR] Aborting
150216 12:50:37 [Note]
The file /usr/share/mysql/english/errmsg.sys exists, owned by root, with permissions 644. The directory /var/lib/mysql exists, owned by mysql, with permissions 700.
I'm guessing that the installation of Workbench changed something (maybe config files) so that mysql is no longer working, but after a fair amount of Internet research, I have not discovered how to solve this problem. Any help will be greatly appreciated.
Seems like file permission problem. Try running mysqld again using sudo
If this doesn't help, maybe you shouldn't be starting it directly via mysqld. Check if the installer package added a service
sudo service --status-all | grep mysql
And if one exist, use it to start mysql, typically:
sudo service mysqld start
Also don't forget to check if mysqld is already running. If so you can't start duplicate process. I know this sound silly but has happened to me before
ps -ef | grep mysqld
Something is wrong. I stopped MaridaDB, moved /var/lib/mysql to a different place, then replaced the old mysql with a softlink
cd /var/lib
ln -s /data/mysql .
Then I added
chown -R mysql:mysql mysql
and then I went to /data and did the same (Note: this relocation works fine in regular mysql). But in MariaDB
ERROR] mysqld: Can't create/write to file
'/var/lib/mysql/aria_log_control' (Errcode: 13 "Permission denied")
[ERROR] mysqld: Got error 'Can't create file' when trying to use aria
control file '/var/lib/mysql/aria_log_control' [ERROR] Plugin 'Aria'
init function returned error. [ERROR] Plugin 'Aria' registration as a
STORAGE ENGINE failed.
So what can I do to fix this? I tried setting
chmod 777 /var/lib/mysql/aria_log_control
but to no avail.
In my case the problem was that while directory mysql and files within were owned by user mysql, the path has a directory that lacked x bit for the users that are not in the correct group. In other words, user could not move to the target directory because one of the directories in the path did not allow it.
In My case this happened after a change which required OS restart, turned out that SELINUX was in ENFORCE mode and was not allowing MySQL to access the files, used below command to resolve:
sudo getenforce
sudo setenforce 0
I have confusion while I'm execute the command "# find / -name mysqld". Because the mysqld is located in four places which are:
/usr/libexec/mysqld
/etc/rc.d/init.d/mysqld
/var/lock/subsys/mysqld
/var/run/mysqld
What is the purpose of another three mysql daemons?
Here is the following explanations
/usr/libexec/mysqld : The actual binary executable
/etc/rc.d/init.d/mysqld : The service startup file (text file)
/var/run/mysqld : PID/run file folder (mysqld writes a run file in this folder)
/var/lock/subsys/mysqld : not sure
If you run which mysqld you should see the first one
You never run that directly
When you run service mysqld start, it calls /etc/rc.d/init.d/mysqld to kickoff a program called mysqld_safe (Run which mysqld_safe). mysqld_safe and mysqld work together to be the server daemon.
In turn, mysqld_safe will kickoff mysqld and then checks for an exit code. You can run less /etc/rc.d/init.d/mysqld and see this. If mysqld did not end due to system shutdown or a normal service mysql stop, mysqld_safe will attempt to restart mysqld.
Server shutdown from power failure.
Mysql will not start now.
Disk is not full.
Syslog is below
Oct 11 15:03:31 joe mysqld_safe[24757]: started
Oct 11 15:03:31 joe mysqld[24760]: 101011 15:03:31 InnoDB: Operating system error number 13 in a file operation.
Oct 11 15:03:31 joe mysqld[24760]: InnoDB: The error means mysqld does not have the access rights to
Oct 11 15:03:31 joe mysqld[24760]: InnoDB: the directory.
Oct 11 15:03:31 joe mysqld[24760]: InnoDB: File name ./ibdata1
Oct 11 15:03:31 joe mysqld[24760]: InnoDB: File operation call: 'create'.
Oct 11 15:03:31 joe mysqld[24760]: InnoDB: Cannot continue operation.
If you are using ubuntu or apparmor you should permit this change in apparmor.
Edit /etc/apparmor.d/usr.sbin.mysqld and change /var/lib/mysql with the new DATADIR.
It should work.
Error:
101130 14:42:51 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
101130 18:07:58 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
101130 18:07:58 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
InnoDB: File name ./ibdata1
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.
Solution SeLinux SeLinux security:
[root#localhost ~]# service mysqld restart
Deteniendo mysqld: [ OK ]
Iniciando mysqld: [ FALLÓ ]
[root#localhost ~]# restorecon -R /var/lib/mysql/
[root#localhost ~]# service mysqld restart
Deteniendo mysqld: [ OK ]
Iniciando mysqld: [ OK ]
[root#localhost ~]#
please check this:
chown -R mysql:mysql /var/lib/mysql
The file is not corrupt. You can find out the source of these errors with 'perror'. i.e.
toaster:~ morgo$ perror 13
OS error code 13: Permission denied
InnoDB has corruption detection (page checksums) and would happily tell you if that were the problem.
Either the directory permissions have changed, or your my.cnf file has been hosed, and it's trying to recreate data files somewhere else.
For me, restoring the security context (selinux) did the trick
restorecon -R /var/lib/mysql/
In short, (especially on RHEL/CentOS/Fedora) try
getenforce
if it replies with Enforcing you have SELinux up and running. Temporarily deactivate it with setenforce 0 and see if MariaDB starts now! Rather common, especially on RHEL/CentOS/Fedora.
There's more about this further down, as well as in this official article.
In general
There are more things in a UNIX environment that might prevent file access, than just user access rights.
Security modules like SELinux (see above) or AppArmor (as Dan mentioned) could disallow it
Access Control Lists (ACL) could be specifically set, for the required files/directories
Any of the parent folders could be owned by another user, and have no x (="dir access") set for others
Additionally there could be other unexpected factors, like ...
The mysql datadir being set to a place, where mysql doesn't have permissions (see /etc/my.cnf)
Mysql could (strangely) be running as a different user, or the file could be simply owned by someone else
Just to mention a view things off the top of my head (feel free to edit/add to this answer btw).
In the case, SELinux is "the problem"
For a permanent solution, you could try to restore the appropriate security context, ...
restorecon -R /var/lib/mysql/
... or just deactivate SELinux (but think about this one a little bit before doing so), by editing the config (typically in /etc/selinux/config) and setting SELINUX=disabled as suggested in following article.
Here the official help page from mariadb.com: What to do if MariaDB doesn't start
And here something from redhat.com: MariaDB Changing Database Location
Obviously those are applicable to MySQL just the same way.
I had exactly the same problem on my CentOS box. After moving mysql data directory around I couldn't start the service anymore, even as I had copied the files with the same owner and permissions.
I had a problem with the SELinux security context. If you run your CentOS stock it has good chance to be enabled and won't let do what you want with MySQL. To fix this :
First compare the old dir and new dir using
ls -Z /var/lib/mysql
and
ls -Z /new/mysql/dir
If you see any difference it's likely to be your problem.
To modify this :
chcon -R --type=mysql_db_t /new/mysql/dir
The -R switch is for recursion. If you only need to change one file you can omit it.
If your context is different than mine(maybe a different distro), use the one indicated by the output of the first (it should be the 3rd field of the SELinux stuff)
ls -Z /var/lib/mysql
I had the same problem and fix by below steps
Working directory /var/lib/mysql
Earlier /var/lib/mysql was owned by some unknown user
Changed it to mysql
mysql]# chown -R mysql:mysql *
mysql]# service mariadb start
Redirecting to /bin/systemctl start mariadb.service
Works like a charm
I had exactly the same problem on my CentOS box. After moving mysql data directory around I couldn't start the service anymore, even as I had copied the files with the same owner and permissions.
I had a problem with the SELinux security context. If you run your CentOS stock it has good chance to be enabled and won't let do what you want with MySQL. To fix this :
First compare the old dir and new dir using
ls -Z /var/lib/mysql
and
ls -Z /new/mysql/dir
If you see any difference it's likely to be your problem.
To modify this :
chcon -R --type=mysql_db_t /new/mysql/dir
The -R switch is for recursion. If you only need to change one file you can omit it.
When this popped up for me, I found the answer in the /etc/mysql/my.cnf configuration file. The datadir line did not point to the /var/lib/mysql directory (where the databases are). Once I put this path in, the server restarted no problem.
If you use SEL Linux
Intall semanage
yum whatprovides /usr/sbin/semanage you get policycoreutils-python-2.5-22.el7.x86_64
See mysqld security context
After installation yum install policycoreutils-python you can just look what different security context mysqld has.
semanage fcontext -l | grep mysqld
/etc/mysql(/.*)? all files system_u:object_r:mysqld_etc_t:s0
/etc/my\.cnf\.d(/.*)? all files system_u:object_r:mysqld_etc_t:s0
/var/log/mysql.* regular file system_u:object_r:mysqld_log_t:s0
/var/lib/mysql(-files|-keyring)?(/.*)? all files system_u:object_r:mysqld_db_t:s0
/var/run/mysqld(/.*)? all files system_u:object_r:mysqld_var_run_t:s0
/var/log/mariadb(/.*)? all file system_u:object_r:mysqld_log_t:s0
/var/run/mariadb(/.*)? all files system_u:object_r:mysqld_var_run_t:s0
/usr/sbin/mysqld(-max)? regular file system_u:object_r:mysqld_exec_t:s0
/var/run/mysqld/mysqlmanager.* regular file system_u:object_r:mysqlmanagerd_var_run_t:s0
/usr/lib/systemd/system/mysqld.* regular file system_u:object_r:mysqld_unit_file_t:s0
/usr/lib/systemd/system/mariadb.* regular file system_u:object_r:mysqld_unit_file_t:s0
/etc/my\.cnf regular file system_u:object_r:mysqld_etc_t:s0
/root/\.my\.cnf regular file system_u:object_r:mysqld_home_t:s0
/usr/sbin/ndbd regular file system_u:object_r:mysqld_exec_t:s0
/usr/libexec/mysqld regular file system_u:object_r:mysqld_exec_t:s0
/usr/bin/mysqld_safe regular file system_u:object_r:mysqld_safe_exec_t:s0
/usr/bin/mysql_upgrade regular file system_u:object_r:mysqld_exec_t:s0
/etc/rc\.d/init\.d/mysqld regular file system_u:object_r:mysqld_initrc_exec_t:s0
/var/lib/mysql/mysql\.sock socket system_u:object_r:mysqld_var_run_t:s0
/usr/libexec/mysqld_safe-scl-helper regular file system_u:object_r:mysqld_safe_exec_t:s0
/home/[^/]+/\.my\.cnf regular file unconfined_u:object_r:mysqld_home_t:s0
Here you see all context for mysqld a short list with explanation
mysqld_etc_t - config files
mysqld_db_t - data db files
mysqld_log_t - log files
mysqld_exec_t - execution files
So if you have the wrong security context on your files you get a permission denied (error 13)
Solution
chcon -R -u system_u -t mysqld_db_t /var/lib/mysql
But check the "normal" permissions, too. I had this problem with centos. You have to systemctl restart mysql for the changes.
In my siuation is Selinux's problem. And the
chcon -R --type=mysql_db_t /new/mysql/dir comes error:
chcon: failed to change context of /new/mysql/dir to root:object_r:mysql_db_t: Invalid argument.
So i use the command：chcon -R root:object_r:mysqld_db_t /new/mysql/dir.
If you have this problem on a Synology NAS you can fix it by following the advice of the Synology support team:
Dear User,
This has been confirmed as a known issue and we will try to fix this issue in further MariaDB release. Sorry for your inconvenience.
Here is the workaround:
Please try to telnet to your DS with "root" account and password (same as admin's)
run the command "echo 1 > /var/services/mysql/VERSION"
open MariaDB package from DSM will trigger update again
type in the DB password and click Update will fix this issue
More info: Synology forum
I had the same problem.
Did alot of research and found out this solution.
You need to run this command on ibdata1
sudo shadowprotect -u root | root
I dont know what this does.. but it worked for me.
Good luck.