In my CentOS machine I use MySQL 5.5.30. (I rented server two days ago and I'm newbie in Linux commands)
I'm trying to check for slow MySQL queries. So I made this from phpMyAdmin
SET GLOBAL log_slow_queries = ON;
SET GLOBAL slow_launch_time = 1;
FLUSH LOGS;
And in default configuration slow query log file variable is /var/lib/mysql/srv1-slow.log
But when I check /var/ folder with FTP, there exists only /var/tmp/ folder. which is empty. What should I do more to see slow queries log?
Note: I tried to create /var/lib folder but system didn't let me.
Add following lines in /etc/my.cnf and restart mysql:
Notes: In my case MySQL version is 5.0.x
[mysqld]
log-slow-queries = /var/lib/mysql/sev1-slow.log
long_query_time = 1
In MySQL 5.5 version
[mysqld]
slow_query_log_file= = /var/lib/mysql/sev1-slow.log
long_query_time = 1
slow_query_log = 1
You must create the file manually and change owners this way:
mkdir /var/lib/mysql
touch /var/lib/mysql/sev1-slow.log
chown mysql.mysql -R /var/lib/mysql
Related
I have 3 master servers, different DBs, I am trying to replicate into a single server. I am having a hard time getting them setup and current. I have Duplicate Entry errors on all 3 Channels. Skipping them manually is painful to say the least. Is there a way to auto sync to the correct position? I was under the impression that this was easy as pie with GTID.
I used:
Dump:
mysqldump --databases profiles --single-transaction --triggers --routines --host=10.10.10.10 --port=3306 --user=user --password=pass > ~/dump.sql
Initialize:
CHANGE MASTER TO MASTER_HOST="10.10.10.10", MASTER_PORT=3306, MASTER_USER="user", MASTER_PASSWORD="pass", MASTER_AUTO_POSITION=1 FOR CHANNEL "channel1";
Master My.cnf:
gtid_mode = ON
enforce_gtid_consistency = true
log_bin = /var/log/mysql/bin_log.index
log_slave_updates = true
server-id = 2061
Slave My.cnf:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
server-id = 10001
explicit_defaults_for_timestamp
gtid_mode=ON
enforce_gtid_consistency=true
log_bin=/var/log/mysql/bin_log.index
log_slave_updates=true
master_info_repository=TABLE
relay_log_info_repository=TABLE
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address = 127.0.0.1
log-error = /var/log/mysql/error.log
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
Am I missing something? Any help is appreciated.
The problem was that GLOBAL.GTID_PURGED had only registered one of my master databases, which sets the proper position from which to continue the replication process.
The other databases were starting from the beginning of time essentially.
Thus the improper positioning and the Duplicate record errors I was receiving.
So this was my solution:
MySQL Multi-Source GTID Replication Guide by ME:
Re/Install MySQL Server:
After securing the current data I performed a clean install of MySQL Server 5.7.7-rc onto the slave server. Any MySQL Server can be completely removed using the following:
$ sudo apt-get --purge remove mysql-client mysql-server mysql-common
$ sudo apt-get autoremove
and then selecting YES at the prompt to remove the "Data" directory (This will permanently delete all of your databases, configurations, etc.). If you have any custom configurations, now is the time to backup your /etc/mysql/my.cnf file.
Reinstall MySQL Server 5.7 for Ubuntu 14-lts
$ sudo apt-get install mysql-server-5.7
If you don't have 5.7 on your system use this guide
Backup Live Master Databases:
I created a current MySQL Dump of all 3 Live Master databases. Each of my databases have a different name e.g. db01, db02, db03 and they are being saved directly onto the slave server.
In my case each database is on its own server, so I ran this a few times changing the ip and database, and filename.
$ mysqldump -u username -p -h 10.10.10.10 --skip-lock-tables --single-transaction --triggers --routines --databases db01 > ~/dumpDB01.sql
Once complete, you will need the GTID_PURGED data from each dump and save it for later:
$ grep PURGED ~/dumpDB01.sql
SET ##GLOBAL.GTID_PURGED='d23dceda-08a4-11e5-85e4-005056a2431f:1-10073';
You will need this entire string: d23dceda-08a4-11e5-85e4-005056a2431f:1-10073
MySQL Slave Server Configuration:
Now I decided to completely configure mysql before I ever imported any data and I will explain why shortly.
Edit my.cnf:
sudo vi /etc/mysql/my.cnf
gtid_mode =ON
enforce_gtid_consistency =true
log_bin =/var/log/mysql/bin_log.index
log_slave_updates =true
master_info_repository =TABLE
relay_log_info_repository =TABLE
server-id =1001
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address = 127.0.0.1
You will need to save the configuration file and create the bin_log.index file, otherwise the server will not start.
$ sudo touch /var/log/mysql/bin_log.index
$ sudo chown mysql:mysql /var/log/mysql/bin_log.index
$ sudo service mysql restart
Import / Configure Data:
While importing the dumps, the first one will register its GTID_PURGED automatically like this:
SET ##GLOBAL.GTID_PURGED='d23dceda-08a4-11e5-85e4-005056a2431f:1-10073';
Once this happens your GTID_EXECUTED will be set with the same data, and each subsequent import will produce errors like this:
ERROR 1840 (HY000) at line 24: ##GLOBAL.GTID_PURGED can only be set when ##GLOBAL.GTID_EXECUTED is empty.
These errors can be ignored on import with $ mysql -u username -p -f < ~/dumpDB02.sql as we are going to modify the GTID_PURGED manually after the import process. If you have already forced the imports and have seen this error you can clear both GTID variables by executing mysql > RESET MASTER;
From the mysql console run the following:
mysql> RESET MASTER;
You will need all 3 GTIDs from the dumps and comma separate them in the assignment below.
mysql> SET ##GLOBAL.GTID_PURGED='d23dceda-08a4-11e5-85e4-005056a2431f:1-10073,d23dceda-08a4-11e5-85e4-005056a2431f:1-10073,d23dceda-08a4-11e5-85e4-005056a2431f:1-10073';
You can now initialize and start replication:
For each database (in my case I changed the ip and channel is a string of your choice. You will use this channel to access the slave data):
mysql> CHANGE MASTER TO MASTER_HOST="10.10.10.10", MASTER_PORT=3306, MASTER_USER="username", MASTER_PASSWORD="password", MASTER_AUTO_POSITION=1 FOR CHANNEL "db01";
Then start each slave:
mysql> START SLAVE FOR CHANNEL "db01";
mysql> SHOW SLAVE STATUS FOR CHANNEL "db01"\G
and success!
I have all of the data, no errors, and its now up to date with the Master Servers
I am trying to set up my MySQL general log so that it can be switched on and off by using
SET GLOBAL general_log = 'ON'
SET GLOBAL general_log = 'OFF'
I would like it off by default (i.e. on server startup) but then have the ability to toggle it as above, so that I don't have to keep restarting the server. When I attempt to switch general logging ON as above, MySQL generates the following error:
Table 'mysql.general_log' doesn't exist
This is true - I have purposely not created this table as I would like logging to occur to file - NOT to tables. This suggests to me that that MySQL is trying to log the general queries to table even though the relevant global variables are set as below:
log_output = FILE
general_log = OFF
general_log_file = /var/log/mysql-general.log
The relevant part of the my.cnf is as follows:
[mysqld]
general-log = OFF
general-log-file = /var/log/mysql-general.log
I am using MySQL version 5.1.58 on a Linux server.
Thanks in advance,
Andy
Since the table mysql.general_log does not exist, I assume you upgraded from a previous version of MySQL and need to run mysql_upgrade to create them.
Backup all of your databases using mysqldump and do a filesystem backup of /var/lib/mysql, then execute the following commands:
mysql_upgrade -p --force
followed by
service mysql restart
or
/etc/init.d/mysql restart
If the general_log table still does not exist after taking these steps, follow the steps in this post: MySql - I dropped general_log table to manually create it.
I need to log SQL to a file so i can check later which SQL run.
so i opened opt/lampp/etc/my.cnf and add these lines
log_slow_queries
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
slow_query_log_file = "/opt/lampp/logs/query.log"
but it did not logged the queries it even did not created the query.log file, so i created an empty file with the name, but still it's not working.
Edit
[mysqld]
log_slow_queries
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
general_log = 1
slow_query_log_file = /opt/lampp/logs/query.log
general_log_file = "/opt/lampp/logs/query.log"
This will only log slow queries. You need the general log if you want to see all queries.
general_log = 1
general_log_file = "/opt/lampp/logs/query.log"
Note that you'll need to restart the server for this to take effect. Also, you should only use this type of logging during testing as it does cause slowdown.
As other users mentioned, this could be a permissions issue. First, check what user MySQL is running as via ps -u -p $(pgrep mysql). The username will be displayed on the first column under USER. In your case, it seems the user is nobody. You can view the default group of a user via groups nobody. This should print something like nobody : nogroup.
To fix the permissions on the file, just run chown nobody:nogroup /opt/lampp/logs/query.log.
Be sure to give the correct permission :
chown mysql:mysql filename
also when i last did it , i had to restart the mysql service :
service mysqld restart
log_slow_queries
is deprecated
It now has to look like that:
slow_query_log
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
general_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
general_log_file = /var/log/mysql/mysql-slow.log
The process probably doesn't have permission to write to that directory. Make sure MySQL has permission to write there, or try logging somewhere less restricted.
This will only log slow queries. You need the general log if you want to see all queries.
general_log = 1
general_log_file = "/opt/lampp/logs/query.log"
Note that you'll need to restart the server for this to take effect. Also, you should only use this type of logging during testing as it does cause slowdown.
Also Note that mysql needs permissions over folder too, in my case, I changed:
general_log_file = "/opt/lampp/logs/query.log"
for
general_log_file = "/var/log/mysql/query.log"
But I have mysql installed from software center, without lampp, when I execute ls -l over /var/log/, it shows
drwx------ 8 mysql mysql 4096 sep 25 23:22 mysql
PD:I change the my.cn file and restart mysql, without create the query.log file in the specified path, mysql automatically create it
I'm using MySQL Server version: 5.5.8-log MySQL Community Server (GPL)
I want to log queries which are not using INDEX and is slow too !
I'm copying here my my.ini settings.
[mysqld]
port=3306
log = "E:/wamp/logs/genquery.log"
log_slow_queries
long_query_time = 1
slow_query_log = 1
slow_query_log_file = "E:/wamp/logs/slowquery.log"
what change i need to do ?
log_queries_not_using_indexes =1 //(or Yes) (From mysql)
Maybe useful for Linux user. (Testet: Ubuntu 16.04)
Get root in terminal and edit mysql configuration
su
vim /etc/mysql/conf.d/mysql.cnf
[mysqld]
slow_query_log=1
slow_query_log_file=/var/log/mysql/slow-query.log
long_query_time=1
log_queries_not_using_indexes=1
Add log file and restart mysql server
touch /var/log/mysql/slow-query.log
chown mysql:adm /var/log/mysql/slow-query.log
chmod 640 slow-query.log
service mysql restart
Test slow logging with SQL queries
/* Activate query log - Maybe useful to show errors (not necessary) */
SET GLOBAL SLOW_QUERY_LOG=ON;
/* Check if slow query log is working */
SELECT SLEEP(2);
log_queries_not_using_indexes
Command-Line Format --log-queries-not-using-indexes
Option-File Format log-queries-not-using-indexes
Option Sets Variable Yes, log_queries_not_using_indexes
Variable Name log_queries_not_using_indexes
Variable Scope Global
Dynamic Variable Yes
Permitted Values
Type boolean
Whether queries that do not use indexes are logged to the slow query log. See Section 5.2.4,
In addition to a1ex07's answer you can use the shell command mk-query-digest to output a report of your running queries without using the log.
See the full methodology: http://www.xaprb.com/blog/2009/08/18/how-to-find-un-indexed-queries-in-mysql-without-using-the-log/
As mentioned in the article it is also possible to group the queries by tables doing --group-by tables --report-format profile
Useful to fast detect unindexed queries.
I've read that Mysql server creates a log file where it keeps a record of all activities - like when and what queries execute.
Can anybody tell me where it exists in my system? How can I read it?
Basically, I need to back up the database with different input [backup between two dates] so I think I need to use log file here, that's why I want to do it...
I think this log must be secured somehow because sensitive information such as usernames and password may be logged [if any query require this]; so may it be secured, not easily able to be seen?
I have root access to the system, how can I see the log?
When I try to open /var/log/mysql.log it is empty.
This is my config file:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
log = /var/log/mysql/mysql.log
binlog-do-db=zero
user = mysql
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
skip-external-locking
bind-address = 127.0.0.1
#
# * Fine Tuning
#
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
general_log_file = /var/log/mysql/mysql.log
general_log = 1
Here is a simple way to enable them. In mysql we need to see often 3 logs which are mostly needed during any project development.
The Error Log. It contains information about errors that occur while
the server is running (also server start and stop)
The General Query Log. This is a general record of what mysqld is
doing (connect, disconnect, queries)
The Slow Query Log. Ιt consists of "slow" SQL statements (as
indicated by its name).
By default no log files are enabled in MYSQL. All errors will be shown in the syslog (/var/log/syslog).
To Enable them just follow below steps:
step1: Go to this file (/etc/mysql/conf.d/mysqld_safe_syslog.cnf) and remove or comment those line.
step2: Go to mysql conf file (/etc/mysql/my.cnf) and add following lines
To enable error log add following
[mysqld_safe]
log_error=/var/log/mysql/mysql_error.log
[mysqld]
log_error=/var/log/mysql/mysql_error.log
To enable general query log add following
general_log_file = /var/log/mysql/mysql.log
general_log = 1
To enable Slow Query Log add following
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 2
log-queries-not-using-indexes
step3: save the file and restart mysql using following commands
service mysql restart
To enable logs at runtime, login to mysql client (mysql -u root -p) and give:
SET GLOBAL general_log = 'ON';
SET GLOBAL slow_query_log = 'ON';
Finally one thing I would like to mention here is I read this from a blog. Thanks. It works for me.
Click here to visit the blog
The MySQL logs are determined by the global variables such as:
log_error for the error message log;
general_log_file for the general query log file (if enabled by general_log);
slow_query_log_file for the slow query log file (if enabled by slow_query_log);
To see the settings and their location, run this shell command:
mysql -se "SHOW VARIABLES" | grep -e log_error -e general_log -e slow_query_log
To print the value of error log, run this command in the terminal:
mysql -e "SELECT ##GLOBAL.log_error"
To read content of the error log file in real time, run:
sudo tail -f $(mysql -Nse "SELECT ##GLOBAL.log_error")
Note: Hit Control-C when finish
When general log is enabled, try:
sudo tail -f $(mysql -Nse "SELECT CONCAT(##datadir, ##general_log_file)")
To use mysql with the password access, add -p or -pMYPASS parameter. To to keep it remembered, you can configure it in your ~/.my.cnf, e.g.
[client]
user=root
password=root
So it'll be remembered for the next time.
You have to activate the query logging in mysql.
edit /etc/my.cnf
[mysqld]
log=/tmp/mysql.log
restart the computer or the mysqld service
service mysqld restart
open phpmyadmin/any application that uses mysql/mysql console and run a query
cat /tmp/mysql.log ( you should see the query )
From the MySQL reference manual:
By default, all log files are created in the data directory.
Check /var/lib/mysql folder.
In my (I have LAMP installed) /etc/mysql/my.cnf file I found following, commented lines in [mysqld] section:
general_log_file = /var/log/mysql/mysql.log
general_log = 1
I had to open this file as superuser, with terminal:
sudo geany /etc/mysql/my.cnf
(I prefer to use Geany instead of gedit or VI, it doesn't matter)
I just uncommented them & save the file then restart MySQL with
sudo service MySQL restart
Run several queries, open the above file (/var/log/mysql/mysql.log) and the log was there :)
Enter MySQL/MariaDB server command-line tool as root
Set file path (you can replace general.log with the file name of your choice).
SET GLOBAL general_log_file='/var/log/mysql/general.log';
Set log file format
SET GLOBAL log_output = 'FILE';
Enable the server general log
SET GLOBAL general_log = 'ON';
Check your configurations in global configuration variables.
SHOW VARIABLES LIKE "general_log%";
Enter exit to leave MySQL command-line and Tail your queries by
tail -f /var/log/mysql/general.log
or
less /var/log/mysql/general.log
To disable the general server log
SET GLOBAL general_log = 'OFF';
To complement loyola's answer it is worth mentioning that as of MySQL 5.1 log_slow_queries is deprecated and is replaced with slow-query-log
Using log_slow_queries will cause your service mysql restart or service mysql start to fail
In addition to the answers above you can pass in command line parameters to the mysqld process for logging options instead of manually editing your conf file. For example, to enable general logging and specifiy a file:
mysqld --general-log --general-log-file=/var/log/mysql.general.log
Confirming other answers above, mysqld --help --verbose gives you the values from the conf file (so running with command line options general-log is FALSE); whereas mysql -se "SHOW VARIABLES" | grep -e log_error -e general_log gives:
general_log ON
general_log_file /var/log/mysql.general.log
Use slightly more compact syntax for the error log:
mysqld --general-log --general-log-file=/var/log/mysql.general.log --log-error=/var/log/mysql.error.log
shell> mysqladmin flush-logs
shell> mv host_name.err-old backup-directory