How can I get around MySQL Errcode 13 with SELECT INTO OUTFILE? - mysql

I am trying to dump the contents of a table to a csv file using a MySQL SELECT INTO OUTFILE statement. If I do:
SELECT column1, column2
INTO OUTFILE 'outfile.csv'
FIELDS TERMINATED BY ','
FROM table_name;
outfile.csv will be created on the server in the same directory this database's files are stored in.
However, when I change my query to:
SELECT column1, column2
INTO OUTFILE '/data/outfile.csv'
FIELDS TERMINATED BY ','
FROM table_name;
I get:
ERROR 1 (HY000): Can't create/write to file '/data/outfile.csv' (Errcode: 13)
Errcode 13 is a permissions error, but I get it even if I change ownership of /data to mysql:mysql and give it 777 permissions. MySQL is running as user "mysql".
Strangely I can create the file in /tmp, just not in any other directory I've tried, even with permissions set such that user mysql should be able to write to the directory.
This is MySQL 5.0.75 running on Ubuntu.

Which particular version of Ubuntu is this and is this Ubuntu Server Edition?
Recent Ubuntu Server Editions (such as 10.04) ship with AppArmor and MySQL's profile might be in enforcing mode by default. You can check this by executing sudo aa-status like so:
# sudo aa-status
5 profiles are loaded.
5 profiles are in enforce mode.
/usr/lib/connman/scripts/dhclient-script
/sbin/dhclient3
/usr/sbin/tcpdump
/usr/lib/NetworkManager/nm-dhcp-client.action
/usr/sbin/mysqld
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode :
/usr/sbin/mysqld (1089)
0 processes are in complain mode.
If mysqld is included in enforce mode, then it is the one probably denying the write. Entries would also be written in /var/log/messages when AppArmor blocks the writes/accesses. What you can do is edit /etc/apparmor.d/usr.sbin.mysqld and add /data/ and /data/* near the bottom like so:
...
/usr/sbin/mysqld {
...
/var/log/mysql/ r,
/var/log/mysql/* rw,
/var/run/mysqld/mysqld.pid w,
/var/run/mysqld/mysqld.sock w,
**/data/ r,
/data/* rw,**
}
And then make AppArmor reload the profiles.
# sudo /etc/init.d/apparmor reload
WARNING: the change above will allow MySQL to read and write to the /data directory. We hope you've already considered the security implications of this.

Ubuntu uses AppArmor and that is whats preventing you from accessing /data/. Fedora uses selinux and that would prevent this on a RHEL/Fedora/CentOS machine.
To modify AppArmor to allow MySQL to access /data/ do the follow:
sudo gedit /etc/apparmor.d/usr.sbin.mysqld
add this line anywhere in the list of directories:
/data/ rw,
then do a :
sudo /etc/init.d/apparmor restart
Another option is to disable AppArmor for mysql altogether, this is NOT RECOMMENDED:
sudo mv /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
Don't forget to restart apparmor:
sudo /etc/init.d/apparmor restart

I know you said that you tried already setting permissions to 777, but as I have an evidence that for me it was a permission issue I'm posting what I exactly run hoping it can help. Here is my experience:
tmp $ pwd
/Users/username/tmp
tmp $ mkdir bkptest
tmp $ mysqldump -u root -T bkptest bkptest
mysqldump: Got error: 1: Can't create/write to file '/Users/username/tmp/bkptest/people.txt' (Errcode: 13) when executing 'SELECT INTO OUTFILE'
tmp $ chmod a+rwx bkptest/
tmp $ mysqldump -u root -T bkptest bkptest
tmp $ ls bkptest/
people.sql people.txt
tmp $

MySQL is getting stupid here. It tries to create files under /tmp/data/.... So what you can do is the following:
mkdir /tmp/data
mount --bind /data /tmp/data
Then try your query. This worked for me after hours of debugging the issue.

You can do this :
mysql -u USERNAME --password=PASSWORD --database=DATABASE --execute='SELECT `FIELD`, `FIELD` FROM `TABLE` LIMIT 0, 10000 ' -X > file.xml

This problem has been bothering me for a long time. I noticed that this discussion does not point out the solution on RHEL/Fecora. I am using RHEL and I do not find the configuration files corresponding to AppArmer on Ubuntu, but I solved my problem by making EVERY directory in the directory PATH readable and accessible by mysql. For example, if you create a directory /tmp, the following two commands make SELECT INTO OUTFILE able to output the .sql AND .sql file
chown mysql:mysql /tmp
chmod a+rx /tmp
If you create a directory in your home directory /home/tom, you must do this for both /home and /home/tom.

Some things to try:
is the secure_file_priv system variable set? If it is, all files must be written to that directory.
ensure that the file does not exist - MySQL will only create new files, not overwrite existing ones.

I have same problem and I fixed this issue by following steps:
Operating system : ubuntu 12.04
lamp installed
suppose your directory to save output file is : /var/www/csv/
Execute following command on terminal and edit this file using gedit editor to add your directory to output file.
sudo gedit /etc/apparmor.d/usr.sbin.mysqld
now file would be opened in editor please add your directory there
/var/www/csv/* rw,
likewise I have added in my file, as following given image :
Execute next command to restart services :
sudo /etc/init.d/apparmor restart
For example I execute following query into phpmyadmin query builder to output data in csv file
SELECT colName1, colName2,colName3
INTO OUTFILE '/var/www/csv/OUTFILE.csv'
FIELDS TERMINATED BY ','
FROM tableName;
It successfully done and write all rows with selected columns into OUTPUT.csv file...

In my case, the solution was to make every directory in the directory path readable and accessible by mysql (chmod a+rx). The directory was still specified by its relative path in the command line.
chmod a+rx /tmp
chmod a+rx /tmp/migration
etc.

I just ran into this same problem. My issue was the directory that I was trying to dump into didn't have write permission for the mysqld process. The initial sql dump would write out but the write of the csv/txt file would fail. Looks like the sql dump runs as the current user and the conversion to csv/txt is run as the user that is running mysqld. So the directory needs write permissions for both users.

You need to provide an absolute path, not a relative path.
Provide the full path to the /data directory you are trying to write to.

Does Ubuntu use SELinux? Check to see if it's enabled and enforcing. /var/log/audit/audit.log may be helpul (if that's where Ubuntu sticks it -- that's the RHEL/Fedora location).

I had the same problem on a CentOs 6.7
In my case all permissions were set and still the error occured. The problem was that the SE Linux was in the mode "enforcing".
I switched it to "permissive" using the command sudo setenforce 0
Then everything worked out for me.

Related

OwnCloud: How to synchronyze the FileSystem with the DB

I have to "insert" a lot of files into an owncloud server (8.2).
A user give me a USB key with the files and tell me to copy of all them into his owncloud data files repository.
Do you know if is it possible ?
Is it possible to synchronyze the ownCloud data fileSystem with the ownCloud database?
My environment is Linux CentOS7 (Apache 2.4, mySQL 5.6, php 5.6)
Thanks,
owncloud brings a command line utility that allows to manually trigger some tasks. Among those is the files:scan function which re-scans a users file system.
So you can import those files by following these steps:
1. you copy the files into the physical file system of the user(s) inside ownclouds data folder
2. you fire the command line utility to re-scan the files. That takes care to update the database according to the files found.
This is an example for the manual trigger:
sudo -u www-data php occ files:scan <user name>
Here <user name> obviously has to be replaced. Also the account name the sudo command switches to depends on the linux distribution and its setup. The command has to be started inside ownclouds base folder. THe command can be called in a loop with different user names, that can be done by means of standard scripting.
Here is a documentation of the utility: https://doc.owncloud.org/server/8.0/admin_manual/configuration_server/occ_command.html
I just made a try myself using an owncloud-8.2 installation and succeeded.
Before I could sucessfully scan my files again as arkascha explained, I needed to change the ownder and the group of the new folder to www-data (for Debian OS - others see OC-Docu 1) and set rights of the new directory to 755
Change ownder:
sudo chown -R www-data:www-data <path>
Change rights:
sudo chmod 755 <path>
whwere is the path to the newly added directory and could for example look like this example: /media/hdd/owncloud/data/<username>/files/<newFolderName>
OC-Docu:
https://doc.owncloud.org/server/9.0/admin_manual/configuration_server/occ_command.html

use mysqlbinlog command can't find file

when I use mysqlbinlog command I can't find the file.
lihaodeMacBook-Pro:data lihao$ mysqlbinlog --start-position=106 --stop-position=125 mysqlbin.000001 > ~/tmp1.sql
mysqlbinlog: File 'mysql-bin.000001' not found (Errcode: 13)
lihaodeMacBook-Pro:data lihao$ ls -a
. ib_logfile0 library mysql-bin.000001
.. ib_logfile1 lihaodeMacBook-Pro.local.err mysql-bin.000002
lihaodeMacBook-Pro:data lihao$ mysqlbinlog /usr/local/mysql-5.1.63-osx10.6-x86_64/data/mysql-bin.000001
/*!40019 SET ##session.max_insert_delayed_threads=0*/;
/*!50003 SET #OLD_COMPLETION_TYPE=##COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
mysqlbinlog: File '/usr/local/mysql-5.1.63-osx10.6-x86_64/data/mysql-bin.000001' not found (Errcode: 13)
I find the solution of the question by myself.I can't find the file because I don't have permission to read or write the file.I can use the command by add sudo at front of the commend.
Mysql Server 5.7 issue resolved:
there are two possibilities over here.
One is either your bin-log file is in /var/log/mysql in that case:
cd /var/log/mysql/
then access the file as sudo user
sudo mysqlbinlog mysql-bin.00000x
if file is in /var/lib/mysql then one needs sudo access to even read the file so follow these two steps:
Login as sudo root user
sudo -i
Once u are sudo user then go to the following directory: (without being a sudo user this location will deny the permission to even enter the directory)
cd /var/lib/mysql/
after that access the file
sudo mysqlbinlog mysql-bin.00000x
In my case it was permission related. Ensure you're sudo when using mysqlbinglong utility.
Should somebody submit an issue to the creators of the mysqlbinlog utility ? The error should be a bit more clear in my opinion.

How do i move MySQL directory to an external network drive in MAC OSX 10.9.4

I am new to the database world and I ran into some problems....
My hard disk on my Mac says I have less than 8gb left of free space. For this reason, I would like to move my MySQL data directory to an external network drive called ls-xld4c.
I have been trying to follow the rules to do so via http://mailsteward.com/nickstek/?p=22
As noted from step 3 from the link above:
I copied the /usr/local/mysql/data directory and all of its files and subdirectories to the
new location at /Volumes/share/MYSQL
So here is what i typed in my terminal:
cd /Volumes/share/MYSQL
cp -R /usr/local/mysql/data
which returns the following: ( i do not know what this means)
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory
Here is some info that might be handy:
1) Server version: 5.6.17 MySQL Community Server (GPL)
2) Where my external drive is located: /Volumes/share
-The network drive is called ls-xld4c and is 1TB in size(I don't know if that is relevant)
The specific folder I want to put the directory reads that it is found in...
Server : smb://ls-xld4c/share/MYSQL , however /Volumes/share/MYSQL shows that it is a valid directory
3) I do not have a password and the user is root
You have almost done it. The error is flagged because you have not specified the destination directory which should be your current working directory. Please use CO command as:
cp -R /usr/local/mysql/data .
The ending dot means current directory which you have already set by using:
cd /Volumes/share/MYSQL
By the way, the following steps are required:
Stop MySQL service.
Copy data files from the directory as specified in "my.cnf" or "my.inf" (in case of windows).
Paste data to destination dir.
Change "my.cnf" or "my.inf" such as the "datadir" entry specifies the destination path.
Restart MySQL.
1. Stop MySQL
sudo /etc/init.d/mysql stop
2. Change Data Directory
sudo cp -R -p /var/lib/mysql /newlocation
3. Edit MySQL default configuration file
sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf
change 'datadir' to /newlocation
sudo vim /etc/apparmor.d/usr.sbin.mysqld
change '/var/lib/mysql' two-entries to /newlocation
4. Start MySQL
sudo /etc/init.d/mysql restart
On macOS Big Sur, MySQL installer used to install MySQL:
Go to System Preferences > MySQL > click on Stop MySQL Server
In configuration tab, you can see current Data Directory
Copy data folder to your destination directory
Change "Data Directory" address to your destination address > then Apply
Go to System Preferences > Security & Privacy > Privacy > Full Disk Access and make sure "mysqld" is checked here
Go to System Preferences > MySQL > click on Start MySQL Server
if you do not do step 5, the service won't start back.
hope it helps for those with permission issues

Fatal error: Can't open and lock privilege tables: Table 'mysql.host' doesn't exist

I am on a server that has afresh install on RHEL 5. I was able to install Apache and PHP just fine., but I am having serious trouble with my MySQL installation. I tried the following:
yum install mysql-server mysql
And didn't get any errors or conflicts. Then I tried to start mysql with the following commands:
chkconfig --levels 235 mysqld on
service mysqld start
And get Timeout error occurred trying to start MySQL Daemon.
I checked my logs and see this error:
[ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.host' doesn't exist
I'm not sure where to go from here.
For reference I am using RHEL 5 and installed the latest versions of PHP 5 and Apache.
After chown and chgrp'ing /var/lib/mysql per the answer by #Bad Programmer, you may also have to execute the following command:
sudo mysql_install_db --user=mysql --ldata=/var/lib/mysql
Then restart your mysqld.
Uninstall mysql using yum remove mysql*
Recursively delete /usr/bin/mysql and /var/lib/mysql
Delete the file /etc/my.cnf.rmp
Use ps -e to check the processes to make sure mysql isn't still running.
Reboot server with reboot
Run yum install mysql-server. This also seems to install the mysql client as a dependency.
Give mysql ownership and group priveleges with:
chown -R mysql /var/lib/mysql
chgrp -R mysql /var/lib/mysql
Use service mysqld start to start MySQL Daemon.
I had this issue on arch linux as well. The issue was pacman installed the package in a different location than MySQL was expecting. I was able to fix the issue with this:
sudo mysql_install_db --user=mysql --basedir=/usr/ --ldata=/var/lib/mysql/
Hope this helps someone!
The root of my problem seemed to be selinux, which was turned on (enforcing)
automatically on OS install.
I wanted my mysql in /data.
After verifying that my.cnf had:
datadir=/data/mysql
(and leaving the socket at /var/lib/mysql)
I executed the command to turn off selinux for mysqld
(alternative is to turn it off completely):
setsebool -P mysqld_disable_trans=1
I ran the following commands:
> chown -R mysql .
> chgrp -R mysql .
> mysql_install_db --user=mysql
I started the mysql daemon and everything worked fine after that.
mysql_install_db –-user=mysql --ldata=/var/lib/mysql
Worked for me in Centos 7
initialize mysql before start on windows.
mysqld --initialize
When download mysql zip version, if run mysqld directly, you'll get this error:
2016-02-18T07:23:48.318481Z 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.user' doesn't exist
2016-02-18T07:23:48.319482Z 0 [ERROR] Aborting
You have to run below command first:
mysqld --initialize
Make sure your data folder is empty before this command.
Just this command is enough to do the magic on centos 6.6
mysql_install_db
I just met the same problem with mysql 5.7 on OSX:
rm -rf {datadir}
mysqld --initialize --datadir {datadir}
mysqld --datadir {datadir}
If you move your datadir, you not only need to give the new datadir permissions, but you need to ensure all parent directories have permission.
I moved my datadir to a hard drive, mounted in Ubuntu as:
/media/*user*/Data/
and my datadir was Databases.
I had to set permissions to 771 to each of the media, user and Data directories:
sudo chmod 771 *DIR*
If this does not work, another way you can get mysql to work is to change user in /etc/mysql/my.cnf to root; though there are no doubt some issues with doing that from a security perspective.
For myself, I had to do:
yum remove mysql*
rm -rf /var/lib/mysql/
cp /etc/my.cnf ~/my.cnf.bkup
yum install -y mysql-server mysql-client
mysql_install_db
chown -R mysql:mysql /var/lib/mysql
chown -R mysql:mysql /var/log/mysql
service mysql start
Then I was able to get back into my databases and configure them again after I nuked them the first go around.
In my case the path of MySQL data folder had a special character "ç" and it make me get...
Fatal error: Can't open and lock privilege tables: Table 'mysql.host'
doesn't exist.
I'm have removed all special characters and everything works.
On CentOS EL 6 and perhaps on earlier versions there is one way to get into this same mess.
Install CentOS EL6 with a minimal installation. For example I used kickstart to install the following:
%packages
#core
acpid
bison
cmake
dhcp-common
flex
gcc
gcc-c++
git
libaio-devel
make
man
ncurses-devel
perl
ntp
ntpdate
pciutils
tar
tcpdump
wget
%end
You will find that one of the dependencies of the above list is mysql-libs. I found that my system has a default my.cnf in /etc and this contains:
[mysqld]
dataddir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
When you build from the Generic Linux (Architecture Independent), Compressed TAR Archive your default data directory is /usr/local/mysql/data which conflicts with the /etc/my.cnf already present which defines datadir=/var/lib/mysql. Also the pid-file defined in the same file does not have permissions for the mysql user/group to write to it in /var/run/mysqld.
A quick remedy is to mv /etc/my.cnf /etc/my.cnf.old which should get your generic source procedure working.
Of course the experience is different of you use the source RPMs.
I had the same issue in trying to start the server and followed the "checked" solution.
But still had the problem. The issue was the my /etc/my.cnf file was not pointing to my
designated datadir as defined when I executed the mysql_install_db with --datadir defined. Once I updated this, the server started correctly.
If you have a server which used to happily run MySQL, but now gives this error, then an uninstall and re-install of MySQL is overkill.
In my case, the server died and took a few disk blocks with it. This affected a few files, including /var/lib/mysql/mysql/host.frm and /var/lib/mysql/mysql/proc.frm
Luckily, I could copy these from another server, and this got me past that table error.
I got similar error on overlayfs (overlay2) that is the default on Docker for Mac.
The error happens when starting mysql on the image, after creating a image with mysql.
2017-11-15T06:44:22.141481Z 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table storage engine for 'user' doesn't have this option
Switching to "aufs" solved the issue.
(On Docker for Mac, the "daemon.json" can be edited by choosing "Preferences..." menu, and selecting "Daemon" tab, and selecting "Advanced" tab.)
/etc/docker/daemon.json :
{
"storage-driver" : "aufs",
"debug" : true,
"experimental" : true
}
Ref:
https://github.com/moby/moby/issues/35503
https://qiita.com/Hige-Moja/items/7b1208f16997e2aa9028
In Windows run the following commands in the command prompt as adminstrator
Step 1:
mysql_install_db.exe
Step 2:
mysqld --initialize
Step 3:
mysqld --console
Step 4:
In windows
Step 4:
mysqladmin -u root password "XXXXXXX"
Step 5:
mysql -u root -p
My case on Ubuntu 14.04.2 LTS was similar to others with my.cnf, but for me the cause was a ~/.my.cnf that was leftover from a previous installation. After deleting that file and purging/re-installing mysql-server, it worked fine.

How to find out the location of currently used MySQL configuration file in linux

How do I know which configuration file is used by MySQL currently? Is there any command or something to find it out?
The information you want can be found by running
mysql --help
or
mysqld --help --verbose
I tried this command on my machine:
mysql --help | grep "Default options" -A 1
And it printed out:
Default options are read from the following files in the given order:
/etc/my.cnf /usr/local/etc/my.cnf ~/.my.cnf
See if that works for you.
mysqld --help --verbose will find only location of default configuration file. What if you use 2 MySQL instances on the same server? It's not going to help.
Good article about figuring it out:
"How to find MySQL configuration file?"
If you are using terminal just type the following:
locate my.cnf
You can use the report process status ps command:
ps ax | grep '[m]ysqld'
You should find them by default in a folder like /etc/my.cnf, maybe also depends on versions. From MySQL Configuration File:
Interestingly, the scope of this file
can be set according to its location.
The settings will be considered global
to all MySQL servers if stored in
/etc/my.cnf. It will be global to a
specific server if located in the
directory where the MySQL databases
are stored (/usr/local/mysql/data for
a binary installation, or
/usr/local/var for a source
installation). Finally, its scope
could be limited to a specific user if
located in the home directory of the
MySQL user (~/.my.cnf). Keep in mind
that even if MySQL does locate a
my.cnf file in /etc/my.cnf (global to
all MySQL servers on that machine), it
will continue its search for a
server-specific file, and then a
user-specific file. You can think of
the final configuration settings as
being the result of the /etc/my.cnf,
mysql-data-dir/my.cnf, and ~/.my.cnf
files.
There are a few switches to package managers to list specific files.
RPM Sytems:
There are switches to rpm command, -q for query, and -c or --configfiles to list config files. There is also -l or --list
The --configfiles one didn't quiet work for me, but --list did list a few .cnf files held by mysql-server
rpm -q --list mysql-server
DEB Systems:
Also with limited success: dpkg --listfiles mysql-server
you can find it by running the following command
mysql --help
it will give you the mysql installed directory and all commands for mysql.
login to mysql with proper credential and used mysql>SHOW VARIABLES LIKE 'datadir'; that will give you path of where mysql stored