MySql 5.5 has a few logging option, among which the "Binary Logfile" with Binlog options which I do not want to use and the "query log file" which I want to use.
However, 1 program using 1 table in that database is filling this logfile with 50+Mb per day, so I would like that table to be excluded from this log.
Is that possible, or is the only way to install another MySql version and then to move this 1 table?
Thanks,
Alex
There are options for filtering the binlog by table, but not the query logs.
There are no options for filtering the general query log. It is either enabled for all queries, or else it's disabled.
There are options for filtering the slow query log, but not by table. For example, to log only queries that take longer than N seconds, or queries that don't use an index. Percona Server adds some options to filter the slow query log based on sampling.
You can use a session variable to disable either slow query or general query logging for queries run in a given session. This is a dynamic setting, so you can change it at will. But you would need to change your client code to do this every time you query that specific table.
Another option is to implement log rotation for the slow query log, so it never grows too large. See https://www.percona.com/blog/2013/04/18/rotating-mysql-slow-logs-safely/
Related
I am trying to verify database histories. For that, I need to log and parse read-write accesses to a database.
I know MySQL mainly offers two types of logs - general query log and binary log. General query log is not helpful for me since it does not log end timestamps. Wheresas, binary log is perfect for my work, but it does not log read access, i.e. SELECT queries.
So I want to know if there is a way to force binary logging such that it logs non-modifying queries too.
The binary log only records changes to data or metadata. There is no option to make it log SELECT queries.
You overlooked the MySQL slow query log, which does record the timestamp and the duration of the query. You can make the slow query log include all queries by setting long_query_time=0 so that it logs all queries that take 0 seconds or more (that is, all of them).
Be careful! In a typical system, the number of SELECT queries is a lot more than the number of other queries. Your logs could grow very rapidly. Make sure to configure log rotation, or you'll fill up your disk.
Other than that, it sounds like you need an audit log. See https://dev.mysql.com/doc/refman/5.7/en/audit-log.html for the MySQL Enterprise audit log plugin (this requires a support subscription from Oracle).
There are at least a couple of open-source implementations of audit log plugins for MySQL. For example: https://www.percona.com/doc/percona-server/LATEST/management/audit_log_plugin.html
I'm debugging an issue where sql select query sometimes doesn't return results when it should. I have no idea if the problem lies within the plugin I'm using, libmysql library, or the mysql database itself. So - I'd like to log all queries on database level, and log the number of rows returned. I enabled general query log, but it only stores the queries executed. Is there way to log execution time and number of rows returned as well?
Thank you.
Depending on your MySQL version additional information can be logged to the slow log.
Including rows_examined and rows_sent.
For example, see Percona's additions to the slow log
I enabled below variable in my.cnf:
log_slow_queries = "/var/log/mysql/mysql-slow.log"
long_query_time = 2
log-queries-not-using-indexes
and restarted the MySQL service. Then mysql-slow.log keeps recording 2 queries (both are small table with full scan, no index involved) repeatedly every 5 seconds. Both of the queries actually took less than 0.01 second. and the log file becomes bigger and bigger.
My server version: 5.5.31-0ubuntu0.12.04.1-log (Ubuntu).
I wonder how frequently the log file will do the writing job, even a redundant job? Can we control some other system variables to slow down this work? There are also some similar question, such as:
rotating mysql slow query log
Mysql slow query log is logging faster queries
Logging slow queries in MySQL
......
Can some experts explain it?
Thanks very much!
As per documentation on the slow query log, this is by design
The server uses the controlling parameters in the following order to
determine whether to write a query to the slow query log:
The query must either not be an administrative statement, or
--log-slow-admin-statements must have been specified.
The query must have taken at least long_query_time seconds, or
log_queries_not_using_indexes must be enabled and the query used no
indexes for row lookups.
The query must have examined at least min_examined_row_limit rows.
Also, further explanation on what this variable setting does,
If you are using this option with the slow query log enabled, queries
that are expected to retrieve all rows are logged. See Section 5.2.5,
“The Slow Query Log”. This option does not necessarily mean that no
index is used. For example, a query that uses a full index scan uses
an index but would be logged because the index would not limit the
number of rows.
The queries will be logged every time they fire, until they are modified to not match the behavior triggering their log. You can rotate the logs if you're concerned about diskspace, as suggested in one of the questions you linked. See this literature on logrotate and example configuration.
Does mysql server keeps records of queries executed on it, if it does so , is it possible to retrieve those queries.
Thanks.
You can use the General Query Log, which logs all queries in plain text.
We have this enabled on our development environment only. It shouldn't be a problem to turn this on temporarily on a production server during off-peak times.
If you're looking for slow queries, you may as well use the Slow Query Log.
If you want to keep record of all queries that are executed, you can enable either the General Query Log or the Slow Query Log and set the threshold to 0 seconds (any query that takes more than 0 seconds will be logged).
Another option is the Binary log. However, binary log does not keep record of queries like SELECT or SHOW that do not modify data.
Note that these logs get pretty big pretty fast in a server with traffic. And that both might contain passwords so they have to be protected from unauthorized eyes.
You can use MySQL Proxy which stands between client app and RDBMS itself.
http://forge.mysql.com/wiki/MySQL_Proxy
You can see online queries and also it allows you to rewrite queries based on rules.
There's another option - use a profiler.
For instance: http://www.jetprofiler.com/
Recently, I noticed that my MySQL server processes a lot of INSERT's. How can I detect user or DB on which is this activivty??
insert 33 k 97.96 k 44.21%
SHOW FULL PROCESSLIST will return every connection, user, and query currently active, if you have the PROCESS permission. That's more for immediate problems, but it has the least overhead.
If you use query logging, then instead of the regular query log (it can slow your server down noticeably) use the binary log to keep it minimal. It only tracks actions that change tables, like CREATE/DROP/ALTER and INSERT/UPDATE/REPLACE.
What you should log periodically (once a minute):
SHOW FULL PROCESSLIST;
SHOW GLOBAL STATUS;
with slow log enabled this will give you huge chance that any question can be solved.
If you have binary logging enabled you can check time/user who inserted rows.
If you have general log enabled then everything is logged.
Look in your query logs. This will show every connect into MySQL, and show every command that they execute.