I would like to log my slow queries to a table, but keep my general log(?) logged in a table. I assume the general log is my binary log? MySQL docs are less than clear on this stuff. My MySQL server is setup as a replication master and these are the relevant logging stanzas from my.cnf.
# BINARY LOGGING #
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 3
max_binlog_size = 1000M
sync_binlog = 1
# LOGGING #
log_error = /var/log/mysql/mysql-error.log
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
long_query_time = 2
log_queries_not_using_indexes = 0
I'm afriad if I add something like this:
log_output = TABLE
general-log
expire_logs_days = 1
it will affect my binlog or start logging everything that's already written to my binlog to a table, which I don't want. I'm essentially just looking to have slow queries (a day or two's worth maybe) written to a table rather than a file, without affecting any of my other current logging.
I'm using Server version: 5.5.22-0ubuntu1-log (Ubuntu)
Thanks.
The setting log_output doesn't affect the binary log. It affects the general query log and the slow query log.
Related
I have set the following parameters in the associated parameters group:
Log_output = FILE
General_log = 0
Slow_query_log = 1
Long_query_time = 4
log_queries_not_using_indexes = 0
Still I can see queries having execution time less than long_query_time being written to the log. How to fix this?
Do I need to restart MySql service?
Hello dear StackOverflow community,
I have a large WordPress site that now crashes the database by creating dozens of Creating sort index tasks with:
SELECT t.*, tt.*
FROM wp_terms AS t
INNER JOIN wp_term_taxonomy AS tt ON t.term_id = tt.term_id
WHERE tt.taxonomy IN ('categories')
ORDER BY t.name ASC
Those run for more than 20 seconds +. Afterwards its "sending data" with the same SQL query. The server is using an AMD epyc cpu and shouldn't have any problems (even though the database is large) and it did not have until it suddenly seems to be stuck.
Shouldn't it be caching this query anyways?
The mariadb my.cf config looks like:
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
lc-messages = en_US
skip-external-locking
# Broken reverse DNS slows down connections considerably and name resolve is
# safe to skip if there are no "host by domain name" access grants
#skip-name-resolve
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 127.0.0.1
#
# * Fine Tuning
#
#key_buffer_size = 128M
#max_allowed_packet = 1G
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
#max_connections = 100
#table_cache = 64
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# Recommend only changing this at runtime for short testing periods if needed!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
# When running under systemd, error logging goes via stdout/stderr to journald
# and when running legacy init error logging goes to syslog due to
# /etc/mysql/conf.d/mariadb.conf.d/50-mysqld_safe.cnf
# Enable this if you want to have error logging into a separate file
#log_error = /var/log/mysql/error.log
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_verbosity = query_plan,explain
#log-queries-not-using-indexes
#min_examined_row_limit = 1000
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
#max_binlog_size = 100M
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
# Most important is to give InnoDB 80 % of the system RAM for buffer use:
# https://mariadb.com/kb/en/innodb-system-variables/#innodb_buffer_pool_size
#innodb_buffer_pool_size = 8G
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.5 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.5]
I restarted mariadb, but the advisor mentions a huge rate of opening tables (same for files). The server is running only for 15 minutes. Is this due to the short time or do I have to change the settings (before the huge amount of Creating sort index tasks, the server was stable)
Issue:
The rate of opening tables is high.
Recommendation:
Opening tables requires disk I/O which is costly. Increasing table_open_cache might avoid this.
Justification:
Opened table rate: 1.26 per second, this value should be less than 10 per hour
Used variable / formula:
Opened_tables / Uptime
Test:
value*60*60 > 10
I would be awesome if somebody has an idea, what to do about that problem
I have looked at many similar questions to this but I can't seem to find the answer. I would like to set up the slow query log for my MySQL database. I have seen many answers saying I should access the MySQL command line tool. I am not sure exactly how to find this tool but I tried accessing it by going to:
c:/xampp/mysql/bin/mysql -u root -p -h localhost
But here I get MariaDB, which seems to be different from any other answers/tutorials I have seen before. Typing in:
set log_slow_queries = ON;
gives me the error
ERROR 1193 (HY000): Unknown system variable 'log_slow_queries'
SET GLOBAL slow_query_log=1;
The Slow Query Log consists of log events for queries taking up to long_query_time seconds to finish. For instance, up to 10 seconds to complete. To see the time threshold currently set, issue the following:
SELECT ##long_query_time;
+-------------------+
| ##long_query_time |
+-------------------+
| 10.000000 |
+-------------------+
It can be set as a GLOBAL variable, in my.cnf or my.ini file. Or it can be set by the connection, though this is unusual. The value can be set between 0 to 10 (seconds). What value to use?
10 is so high as to be almost useless;
2 is a compromise;
0.5 and other fractions are possible;
0 captures everything; this could fill up disk dangerously fast, but can be very useful.
The capturing of slow queries is either turned on or off. And the file logged to is also specified. The below captures these concepts:
SELECT ##slow_query_log; -- Is capture currently active? (1=On, 0=Off)
SELECT ##slow_query_log_file; -- filename for capture. Resides in datadir
SELECT ##datadir; -- to see current value of the location for capture file
SET GLOBAL slow_query_log=0; -- Turn Off
-- make a backup of the Slow Query Log capture file. Then delete it.
SET GLOBAL slow_query_log=1; -- Turn it back On (new empty file is created)
For more information, please see the MySQL Manual Page The Slow Query Log
Note: The above information on turning on/off the slowlog was changed in 5.6(?); older version had another mechanism.
The "best" way to see what is slowing down your system:
long_query_time=...
turn on the slowlog
run for a few hours
turn off the slowlog (or raise the cutoff)
run pt-query-digest to find the 'worst' couple of queries. Or mysqldumpslow -s t
Go to xampp control panel click on config button for mysql and select my.ini then add these lines in my.ini file
slow_query_log = 1
slow-query-log-file=/path/of/the/log/file.log
I put above two lines under the log_error = "mysql_error.log". the modified part of the my.ini file should look like this
# The MySQL server
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
slow_query_log = 1
slow-query-log-file=/var/log/mysql-slow.log
Then Restart the MySQL server in xampp control panel. and now slow_query_log should be enabled, you can confirm it by running following command in the MySQL shell
show variables like '%slow%';
It might be obvious but it took me time before I realized my mistake: in the my.ini file you should put the slow_query_log settings in the [mysqld] group, not simply at the end of the my.ini file....
I have a Ruby program that uses a thread pool of 20 threads to download more than 1000 compressed MySQL dump files ("xxx.sql.gz"). After each file is downloaded, the script creates a database and calls
gzip -dc xxx.sql.gz | cat - commit.txt | mysql -D db_name
to restore the dump file.
commit.txt appends commit; to the SQL file content since autocommit is turned off.
The problem is, sometimes all the MySQL processes for restoring dumps hang at a random time.
I use a ramdisk by the way.
my.cnf is:
[mysqld]
autocommit=0
innodb_buffer_pool_size=16G
max_allowed_packet=256M
innodb_file_per_table=1
innodb_flush_log_at_trx_commit = 2
innodb_log_file_size = 256M
innodb_flush_method = O_DIRECT
innodb_use_native_aio = 0
tmpdir=/mnt/ramdisk
datadir=/mnt/ramdisk/mysql
socket=/mnt/ramdisk/mysql/mysql.sock
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
I need to log SQL to a file so i can check later which SQL run.
so i opened opt/lampp/etc/my.cnf and add these lines
log_slow_queries
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
slow_query_log_file = "/opt/lampp/logs/query.log"
but it did not logged the queries it even did not created the query.log file, so i created an empty file with the name, but still it's not working.
Edit
[mysqld]
log_slow_queries
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
general_log = 1
slow_query_log_file = /opt/lampp/logs/query.log
general_log_file = "/opt/lampp/logs/query.log"
This will only log slow queries. You need the general log if you want to see all queries.
general_log = 1
general_log_file = "/opt/lampp/logs/query.log"
Note that you'll need to restart the server for this to take effect. Also, you should only use this type of logging during testing as it does cause slowdown.
As other users mentioned, this could be a permissions issue. First, check what user MySQL is running as via ps -u -p $(pgrep mysql). The username will be displayed on the first column under USER. In your case, it seems the user is nobody. You can view the default group of a user via groups nobody. This should print something like nobody : nogroup.
To fix the permissions on the file, just run chown nobody:nogroup /opt/lampp/logs/query.log.
Be sure to give the correct permission :
chown mysql:mysql filename
also when i last did it , i had to restart the mysql service :
service mysqld restart
log_slow_queries
is deprecated
It now has to look like that:
slow_query_log
log_queries_not_using_indexes =1
long_query_time = 1
slow_query_log = 1
general_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
general_log_file = /var/log/mysql/mysql-slow.log
The process probably doesn't have permission to write to that directory. Make sure MySQL has permission to write there, or try logging somewhere less restricted.
This will only log slow queries. You need the general log if you want to see all queries.
general_log = 1
general_log_file = "/opt/lampp/logs/query.log"
Note that you'll need to restart the server for this to take effect. Also, you should only use this type of logging during testing as it does cause slowdown.
Also Note that mysql needs permissions over folder too, in my case, I changed:
general_log_file = "/opt/lampp/logs/query.log"
for
general_log_file = "/var/log/mysql/query.log"
But I have mysql installed from software center, without lampp, when I execute ls -l over /var/log/, it shows
drwx------ 8 mysql mysql 4096 sep 25 23:22 mysql
PD:I change the my.cn file and restart mysql, without create the query.log file in the specified path, mysql automatically create it