How to see log files in MySQL? - mysql

I've read that Mysql server creates a log file where it keeps a record of all activities - like when and what queries execute.
Can anybody tell me where it exists in my system? How can I read it?
Basically, I need to back up the database with different input [backup between two dates] so I think I need to use log file here, that's why I want to do it...
I think this log must be secured somehow because sensitive information such as usernames and password may be logged [if any query require this]; so may it be secured, not easily able to be seen?
I have root access to the system, how can I see the log?
When I try to open /var/log/mysql.log it is empty.
This is my config file:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
log = /var/log/mysql/mysql.log
binlog-do-db=zero
user = mysql
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
skip-external-locking
bind-address = 127.0.0.1
#
# * Fine Tuning
#
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
general_log_file = /var/log/mysql/mysql.log
general_log = 1

Here is a simple way to enable them. In mysql we need to see often 3 logs which are mostly needed during any project development.
The Error Log. It contains information about errors that occur while
the server is running (also server start and stop)
The General Query Log. This is a general record of what mysqld is
doing (connect, disconnect, queries)
The Slow Query Log. Ιt consists of "slow" SQL statements (as
indicated by its name).
By default no log files are enabled in MYSQL. All errors will be shown in the syslog (/var/log/syslog).
To Enable them just follow below steps:
step1: Go to this file (/etc/mysql/conf.d/mysqld_safe_syslog.cnf) and remove or comment those line.
step2: Go to mysql conf file (/etc/mysql/my.cnf) and add following lines
To enable error log add following
[mysqld_safe]
log_error=/var/log/mysql/mysql_error.log
[mysqld]
log_error=/var/log/mysql/mysql_error.log
To enable general query log add following
general_log_file = /var/log/mysql/mysql.log
general_log = 1
To enable Slow Query Log add following
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 2
log-queries-not-using-indexes
step3: save the file and restart mysql using following commands
service mysql restart
To enable logs at runtime, login to mysql client (mysql -u root -p) and give:
SET GLOBAL general_log = 'ON';
SET GLOBAL slow_query_log = 'ON';
Finally one thing I would like to mention here is I read this from a blog. Thanks. It works for me.
Click here to visit the blog

The MySQL logs are determined by the global variables such as:
log_error for the error message log;
general_log_file for the general query log file (if enabled by general_log);
slow_query_log_file for the slow query log file (if enabled by slow_query_log);
To see the settings and their location, run this shell command:
mysql -se "SHOW VARIABLES" | grep -e log_error -e general_log -e slow_query_log
To print the value of error log, run this command in the terminal:
mysql -e "SELECT ##GLOBAL.log_error"
To read content of the error log file in real time, run:
sudo tail -f $(mysql -Nse "SELECT ##GLOBAL.log_error")
Note: Hit Control-C when finish
When general log is enabled, try:
sudo tail -f $(mysql -Nse "SELECT CONCAT(##datadir, ##general_log_file)")
To use mysql with the password access, add -p or -pMYPASS parameter. To to keep it remembered, you can configure it in your ~/.my.cnf, e.g.
[client]
user=root
password=root
So it'll be remembered for the next time.

You have to activate the query logging in mysql.
edit /etc/my.cnf
[mysqld]
log=/tmp/mysql.log
restart the computer or the mysqld service
service mysqld restart
open phpmyadmin/any application that uses mysql/mysql console and run a query
cat /tmp/mysql.log ( you should see the query )

From the MySQL reference manual:
By default, all log files are created in the data directory.
Check /var/lib/mysql folder.

In my (I have LAMP installed) /etc/mysql/my.cnf file I found following, commented lines in [mysqld] section:
general_log_file = /var/log/mysql/mysql.log
general_log = 1
I had to open this file as superuser, with terminal:
sudo geany /etc/mysql/my.cnf
(I prefer to use Geany instead of gedit or VI, it doesn't matter)
I just uncommented them & save the file then restart MySQL with
sudo service MySQL restart
Run several queries, open the above file (/var/log/mysql/mysql.log) and the log was there :)

Enter MySQL/MariaDB server command-line tool as root
Set file path (you can replace general.log with the file name of your choice).
SET GLOBAL general_log_file='/var/log/mysql/general.log';
Set log file format
SET GLOBAL log_output = 'FILE';
Enable the server general log
SET GLOBAL general_log = 'ON';
Check your configurations in global configuration variables.
SHOW VARIABLES LIKE "general_log%";
Enter exit to leave MySQL command-line and Tail your queries by
tail -f /var/log/mysql/general.log
or
less /var/log/mysql/general.log
To disable the general server log
SET GLOBAL general_log = 'OFF';

To complement loyola's answer it is worth mentioning that as of MySQL 5.1 log_slow_queries is deprecated and is replaced with slow-query-log
Using log_slow_queries will cause your service mysql restart or service mysql start to fail

In addition to the answers above you can pass in command line parameters to the mysqld process for logging options instead of manually editing your conf file. For example, to enable general logging and specifiy a file:
mysqld --general-log --general-log-file=/var/log/mysql.general.log
Confirming other answers above, mysqld --help --verbose gives you the values from the conf file (so running with command line options general-log is FALSE); whereas mysql -se "SHOW VARIABLES" | grep -e log_error -e general_log gives:
general_log ON
general_log_file /var/log/mysql.general.log
Use slightly more compact syntax for the error log:
mysqld --general-log --general-log-file=/var/log/mysql.general.log --log-error=/var/log/mysql.error.log

shell> mysqladmin flush-logs
shell> mv host_name.err-old backup-directory

Related

Multi Source Replication MySQL 5.6 to 5.7 GTID Auto Position Issues

I have 3 master servers, different DBs, I am trying to replicate into a single server. I am having a hard time getting them setup and current. I have Duplicate Entry errors on all 3 Channels. Skipping them manually is painful to say the least. Is there a way to auto sync to the correct position? I was under the impression that this was easy as pie with GTID.
I used:
Dump:
mysqldump --databases profiles --single-transaction --triggers --routines --host=10.10.10.10 --port=3306 --user=user --password=pass > ~/dump.sql
Initialize:
CHANGE MASTER TO MASTER_HOST="10.10.10.10", MASTER_PORT=3306, MASTER_USER="user", MASTER_PASSWORD="pass", MASTER_AUTO_POSITION=1 FOR CHANNEL "channel1";
Master My.cnf:
gtid_mode = ON
enforce_gtid_consistency = true
log_bin = /var/log/mysql/bin_log.index
log_slave_updates = true
server-id = 2061
Slave My.cnf:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
server-id = 10001
explicit_defaults_for_timestamp
gtid_mode=ON
enforce_gtid_consistency=true
log_bin=/var/log/mysql/bin_log.index
log_slave_updates=true
master_info_repository=TABLE
relay_log_info_repository=TABLE
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address = 127.0.0.1
log-error = /var/log/mysql/error.log
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
Am I missing something? Any help is appreciated.
The problem was that GLOBAL.GTID_PURGED had only registered one of my master databases, which sets the proper position from which to continue the replication process.
The other databases were starting from the beginning of time essentially.
Thus the improper positioning and the Duplicate record errors I was receiving.
So this was my solution:
MySQL Multi-Source GTID Replication Guide by ME:
Re/Install MySQL Server:
After securing the current data I performed a clean install of MySQL Server 5.7.7-rc onto the slave server. Any MySQL Server can be completely removed using the following:
$ sudo apt-get --purge remove mysql-client mysql-server mysql-common
$ sudo apt-get autoremove
and then selecting YES at the prompt to remove the "Data" directory (This will permanently delete all of your databases, configurations, etc.). If you have any custom configurations, now is the time to backup your /etc/mysql/my.cnf file.
Reinstall MySQL Server 5.7 for Ubuntu 14-lts
$ sudo apt-get install mysql-server-5.7
If you don't have 5.7 on your system use this guide
Backup Live Master Databases:
I created a current MySQL Dump of all 3 Live Master databases. Each of my databases have a different name e.g. db01, db02, db03 and they are being saved directly onto the slave server.
In my case each database is on its own server, so I ran this a few times changing the ip and database, and filename.
$ mysqldump -u username -p -h 10.10.10.10 --skip-lock-tables --single-transaction --triggers --routines --databases db01 > ~/dumpDB01.sql
Once complete, you will need the GTID_PURGED data from each dump and save it for later:
$ grep PURGED ~/dumpDB01.sql
SET ##GLOBAL.GTID_PURGED='d23dceda-08a4-11e5-85e4-005056a2431f:1-10073';
You will need this entire string: d23dceda-08a4-11e5-85e4-005056a2431f:1-10073
MySQL Slave Server Configuration:
Now I decided to completely configure mysql before I ever imported any data and I will explain why shortly.
Edit my.cnf:
sudo vi /etc/mysql/my.cnf
gtid_mode =ON
enforce_gtid_consistency =true
log_bin =/var/log/mysql/bin_log.index
log_slave_updates =true
master_info_repository =TABLE
relay_log_info_repository =TABLE
server-id =1001
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address = 127.0.0.1
You will need to save the configuration file and create the bin_log.index file, otherwise the server will not start.
$ sudo touch /var/log/mysql/bin_log.index
$ sudo chown mysql:mysql /var/log/mysql/bin_log.index
$ sudo service mysql restart
Import / Configure Data:
While importing the dumps, the first one will register its GTID_PURGED automatically like this:
SET ##GLOBAL.GTID_PURGED='d23dceda-08a4-11e5-85e4-005056a2431f:1-10073';
Once this happens your GTID_EXECUTED will be set with the same data, and each subsequent import will produce errors like this:
ERROR 1840 (HY000) at line 24: ##GLOBAL.GTID_PURGED can only be set when ##GLOBAL.GTID_EXECUTED is empty.
These errors can be ignored on import with $ mysql -u username -p -f < ~/dumpDB02.sql as we are going to modify the GTID_PURGED manually after the import process. If you have already forced the imports and have seen this error you can clear both GTID variables by executing mysql > RESET MASTER;
From the mysql console run the following:
mysql> RESET MASTER;
You will need all 3 GTIDs from the dumps and comma separate them in the assignment below.
mysql> SET ##GLOBAL.GTID_PURGED='d23dceda-08a4-11e5-85e4-005056a2431f:1-10073,d23dceda-08a4-11e5-85e4-005056a2431f:1-10073,d23dceda-08a4-11e5-85e4-005056a2431f:1-10073';
You can now initialize and start replication:
For each database (in my case I changed the ip and channel is a string of your choice. You will use this channel to access the slave data):
mysql> CHANGE MASTER TO MASTER_HOST="10.10.10.10", MASTER_PORT=3306, MASTER_USER="username", MASTER_PASSWORD="password", MASTER_AUTO_POSITION=1 FOR CHANNEL "db01";
Then start each slave:
mysql> START SLAVE FOR CHANNEL "db01";
mysql> SHOW SLAVE STATUS FOR CHANNEL "db01"\G
and success!
I have all of the data, no errors, and its now up to date with the Master Servers

Enable general logging mysql 5.6.14

I have mysq 5.6.14 and centos 6.4. In /etc/my.cnf I have:
[mysqld]
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
bind-address=127.0.0.1
character-set-server=utf8
init-connect="SET NAMES utf8"
skip-character-set-client-handshake
general_log=1
general_log_file=/var/log/mysql
log-output=file
When I make a mistake in my.cnf I cant restart mysql (service mysql restart). It means that mysql reads this file. However I don't have /var/log/mysql file. What is my mistake?
I found the answer. This may seem like a clugy band-aid, but try the following for now...
service mysql stop
touch /var/log/mysql.log
chown mysql:mysql /var/log/mysql.log
service mysql start
This should work.
Please look back at the database home folder (run SHOW VARIABLES LIKE 'datadir';)
Go to the folder and see if a default general log appeared.
You could also try changing it to
general_log = 1
general_log_file = mysql.log
and see if the mysql.log appears in your datadir folder after restarting mysql

MySQL slow queries aren't logged

In my CentOS machine I use MySQL 5.5.30. (I rented server two days ago and I'm newbie in Linux commands)
I'm trying to check for slow MySQL queries. So I made this from phpMyAdmin
SET GLOBAL log_slow_queries = ON;
SET GLOBAL slow_launch_time = 1;
FLUSH LOGS;
And in default configuration slow query log file variable is /var/lib/mysql/srv1-slow.log
But when I check /var/ folder with FTP, there exists only /var/tmp/ folder. which is empty. What should I do more to see slow queries log?
Note: I tried to create /var/lib folder but system didn't let me.
Add following lines in /etc/my.cnf and restart mysql:
Notes: In my case MySQL version is 5.0.x
[mysqld]
log-slow-queries = /var/lib/mysql/sev1-slow.log
long_query_time = 1
In MySQL 5.5 version
[mysqld]
slow_query_log_file= = /var/lib/mysql/sev1-slow.log
long_query_time = 1
slow_query_log = 1
You must create the file manually and change owners this way:
mkdir /var/lib/mysql
touch /var/lib/mysql/sev1-slow.log
chown mysql.mysql -R /var/lib/mysql

Logging MySQL query on Ubuntu

I need to log SQL to a file so i can check later which SQL run.
so i opened opt/lampp/etc/my.cnf and add these lines
log_slow_queries
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
slow_query_log_file = "/opt/lampp/logs/query.log"
but it did not logged the queries it even did not created the query.log file, so i created an empty file with the name, but still it's not working.
Edit
[mysqld]
log_slow_queries
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
general_log = 1
slow_query_log_file = /opt/lampp/logs/query.log
general_log_file = "/opt/lampp/logs/query.log"
This will only log slow queries. You need the general log if you want to see all queries.
general_log = 1
general_log_file = "/opt/lampp/logs/query.log"
Note that you'll need to restart the server for this to take effect. Also, you should only use this type of logging during testing as it does cause slowdown.
As other users mentioned, this could be a permissions issue. First, check what user MySQL is running as via ps -u -p $(pgrep mysql). The username will be displayed on the first column under USER. In your case, it seems the user is nobody. You can view the default group of a user via groups nobody. This should print something like nobody : nogroup.
To fix the permissions on the file, just run chown nobody:nogroup /opt/lampp/logs/query.log.
Be sure to give the correct permission :
chown mysql:mysql filename
also when i last did it , i had to restart the mysql service :
service mysqld restart
log_slow_queries
is deprecated
It now has to look like that:
slow_query_log
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
general_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
general_log_file = /var/log/mysql/mysql-slow.log
The process probably doesn't have permission to write to that directory. Make sure MySQL has permission to write there, or try logging somewhere less restricted.
This will only log slow queries. You need the general log if you want to see all queries.
general_log = 1
general_log_file = "/opt/lampp/logs/query.log"
Note that you'll need to restart the server for this to take effect. Also, you should only use this type of logging during testing as it does cause slowdown.
Also Note that mysql needs permissions over folder too, in my case, I changed:
general_log_file = "/opt/lampp/logs/query.log"
for
general_log_file = "/var/log/mysql/query.log"
But I have mysql installed from software center, without lampp, when I execute ls -l over /var/log/, it shows
drwx------ 8 mysql mysql 4096 sep 25 23:22 mysql
PD:I change the my.cn file and restart mysql, without create the query.log file in the specified path, mysql automatically create it

How to start MySQL with --skip-grant-tables?

I locked my root user out from our database. I need to get all privileges back to the root user. I have my password and I can log in to MySQL. But the root user has no all privileges.
I had the same problem as the title of this question, so incase anyone else googles upon this question and wants to start MySql in 'skip-grant-tables' mode on Windows, here is what I did.
Stop the MySQL service through Administrator tools, Services.
Modify the my.ini configuration file (assuming default paths)
C:\Program Files\MySQL\MySQL Server 5.5\my.ini
or for MySQL version >= 5.6
C:\ProgramData\MySQL\MySQL Server 5.6\my.ini
In the SERVER SECTION, under [mysqld], add the following line:
skip-grant-tables
so that you have
# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this
# file.
#
[mysqld]
skip-grant-tables
Start the service again and you should be able to log into your database without a password.
How to re-take control of the root user in MySQL.
DANGER: RISKY OPERATTION
Start session ssh (using root if possible).
Edit my.cnf file using.
sudo vi /etc/my.cnf
Add line to mysqld block.*
skip-grant-tables
Save and exit.
Restart MySQL service.
service mysql restart
Check service status.
service mysql status
Connect to mysql.
mysql
Using main database.
use mysql;
Redefine user root password.
UPDATE user SET `authentication_string` = PASSWORD('myNuevoPassword') WHERE `User` = 'root';
Edit file my.cnf.
sudo vi /etc/my.cnf
Erase line.
skip-grant-tables
Save and exit.
Restart MySQL service.
service mysqld restart
Check service status.
service mysql status
Connect to database.
mysql -u root -p
Type new password when prompted.
This action is very dangerous, it allows anyone to connect to all databases with no restriction without a user and password. It must be used carefully and must be reverted quickly to avoid risks.
After trying lots of things, this is what worked for me:
sudo mysql -u root
ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY 'newpassword';
What that does is first we use sudo to log in mysql as root without needing a password. Then we just update root's password.
After that, I restarted mysqld:
sudo service mysql restart
And the newpassword logged root in!
On the Linux system you can do following (Should be similar for other OS)
Check if mysql process is running:
sudo service mysql status
If runnning then stop the process:
(Make sure you close all mysql tool)
sudo service mysql stop
If you have issue stopping then do following
Search for process: ps aux | grep mysqld
Kill the process: kill -9 process_id
Now start mysql in safe mode with skip grant
sudo mysqld_safe --skip-grant-tables &
I'm in windows 10, using WAMP64 server. Searched for my.cnf and my.ini. Found my.ini in C:\wamp64\bin\mariadb\mariadb10.2.14.
Following the instructions from the colleagues:
Opened the quick start menu from Wampserver, selected 'Stop All Services'
Opened my.ini in a text editor, searched for [mysqld]
Added 'skip-grant-tables' at the end of the [mysqld] section (but within it)
Save the file, leave the editor open
In the Wampserver menu, select "Restart Services'. There will be a warning about the skip-grant-tables option
In the Wampserver menu select MySQL to open the prompt
It asked for a password, just press enter
Paste the command ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY 'newpassword';
It must report that the operation was successful (no tables affected)
In the my.ini file, erase the 'skip-grant-tables' line, save the file
In the WampServer menu, select once more Restart Service
Now you can enter with the new password. Thanks to all answers here.
If you use mysql 5.6 server and have problems with C:\Program Files\MySQL\MySQL Server 5.6\my.ini:
You should go to C:\ProgramData\MySQL\MySQL Server 5.6\my.ini.
You should add skip-grant-tables and then you do not need a password.
# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this
# file.
#
# server_type=3
[mysqld]
skip-grant-tables
Note: after you are done with your work on skip-grant-tables, you should restore your file of C:\ProgramData\MySQL\MySQL Server 5.6\my.ini.
Use the following command (notice the "d"): mysqld --skip-grant-tables
if you are running on Apple MacBook OSX then:
Stop your MySQL server (if it is already running).
Find your MySQL configuration file, my.cnf. (For me it was placed #
/Applications/XAMPP/xamppfiles/etc. You can just search if you
can't find it).
Open my.cnf file in any text editor.
Add "skip-grant-tables" (without quotes) at the end of [mysqld] section and save the file.
Now start your MySQL server. It'll start with skip-grant-tables option.
Do what you want now!!
PS: Please remove skip-grant-tables from my.cnf file once you are done with whatsoever you want to do ELSE MySQL server will always run without access grants.
Please run this below command from the console to skip the user table verification while launching mysql database from command prompt
mysqld -skip-grant-tables
Edit my.ini file and add skip-grant-tables and restart your mysql server :
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
skip-grant-tables
# Change here for bind listening
# bind-address="127.0.0.1"
# bind-address = ::1
if this is a windows box, the simplest thing to do is to stop the servers, add skip-grant-tables to the mysql configuration file, and restart the server.
once you've fixed your permission problems, repeat the above but remove the skip-grant-tables option.
if you don't know where your configuration file is, then log in to mysql send SHOW VARIABLES LIKE '%config%' and one of the rows returned will tell you where your configuration file is.
This is how to do it on Ubuntu 20.4. This worked for me.
Go to /etc/mysql/mysql.conf.d/
You can write into terminal
cd /etc/mysql/mysql.conf.d/,
then you need to edit the file which is named mysqld.cnf.
On my PC, that file was a read-only file, so I needed to first change the permissions.
I wrote
sudo chmod +rw mysqld.cnf in the terminal.
After that, I edited the file by typing sudo gedit mysqld.cnf
in the terminal.
In the file, you will see [mysqld] somewhere, below [mysqld] add
skip-grant-tables in a new line, so that it looks like this
[mysqld]
skip-grant-tables
Restart the mysql service by writting sudo service mysql restart in terminal.
If your server wasn't running then write sudo service mysql start in terminal.
Another thing worth mentioning here is that
I also had another problem which I fixed in almost the exact same manner.
My server wasn't listening at the port 3306,
so I also had to add port = 3306 in that mysqld.cnf file.
Now I have
[mysqld]
skip-grant-tables
port = 3306
in the mysqld.cnf file.
I see that the question is old, but maybe my configuration will help someone. I use this configuration in scripts:
sed -i 's/^#skip-grant-tables.*/skip-grant-tables/g' /etc/my.cnf
service mysql restart
mysql -e "UPDATE mysql.user SET authentication_string='' WHERE user='root';"
sed -i 's/^skip-grant-tables.*/#skip-grant-tables/g' /etc/my.cnf
service mysql restart