I am trying to verify database histories. For that, I need to log and parse read-write accesses to a database.
I know MySQL mainly offers two types of logs - general query log and binary log. General query log is not helpful for me since it does not log end timestamps. Wheresas, binary log is perfect for my work, but it does not log read access, i.e. SELECT queries.
So I want to know if there is a way to force binary logging such that it logs non-modifying queries too.
The binary log only records changes to data or metadata. There is no option to make it log SELECT queries.
You overlooked the MySQL slow query log, which does record the timestamp and the duration of the query. You can make the slow query log include all queries by setting long_query_time=0 so that it logs all queries that take 0 seconds or more (that is, all of them).
Be careful! In a typical system, the number of SELECT queries is a lot more than the number of other queries. Your logs could grow very rapidly. Make sure to configure log rotation, or you'll fill up your disk.
Other than that, it sounds like you need an audit log. See https://dev.mysql.com/doc/refman/5.7/en/audit-log.html for the MySQL Enterprise audit log plugin (this requires a support subscription from Oracle).
There are at least a couple of open-source implementations of audit log plugins for MySQL. For example: https://www.percona.com/doc/percona-server/LATEST/management/audit_log_plugin.html
Related
MySql 5.5 has a few logging option, among which the "Binary Logfile" with Binlog options which I do not want to use and the "query log file" which I want to use.
However, 1 program using 1 table in that database is filling this logfile with 50+Mb per day, so I would like that table to be excluded from this log.
Is that possible, or is the only way to install another MySql version and then to move this 1 table?
Thanks,
Alex
There are options for filtering the binlog by table, but not the query logs.
There are no options for filtering the general query log. It is either enabled for all queries, or else it's disabled.
There are options for filtering the slow query log, but not by table. For example, to log only queries that take longer than N seconds, or queries that don't use an index. Percona Server adds some options to filter the slow query log based on sampling.
You can use a session variable to disable either slow query or general query logging for queries run in a given session. This is a dynamic setting, so you can change it at will. But you would need to change your client code to do this every time you query that specific table.
Another option is to implement log rotation for the slow query log, so it never grows too large. See https://www.percona.com/blog/2013/04/18/rotating-mysql-slow-logs-safely/
I'm debugging an issue where sql select query sometimes doesn't return results when it should. I have no idea if the problem lies within the plugin I'm using, libmysql library, or the mysql database itself. So - I'd like to log all queries on database level, and log the number of rows returned. I enabled general query log, but it only stores the queries executed. Is there way to log execution time and number of rows returned as well?
Thank you.
Depending on your MySQL version additional information can be logged to the slow log.
Including rows_examined and rows_sent.
For example, see Percona's additions to the slow log
How to log the Audit log for activity happening on Mysql server. I want to log precise query like Delete, Insert, update, drop, truncate etc.
Please help if anybody has solution for this requirement.
what you need is the BInary Bin Log:
http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
The binary log is not used for statements such as SELECT or SHOW that do not modify data. To log all statements (for example, to identify a problem query), use the general query log. See Section 5.2.2, “The General Query Log”.
Running a server with binary logging enabled makes performance slightly slower. However, the benefits of the binary log in enabling you to set up replication and for restore operations generally outweigh this minor performance decrement.
Does mysql server keeps records of queries executed on it, if it does so , is it possible to retrieve those queries.
Thanks.
You can use the General Query Log, which logs all queries in plain text.
We have this enabled on our development environment only. It shouldn't be a problem to turn this on temporarily on a production server during off-peak times.
If you're looking for slow queries, you may as well use the Slow Query Log.
If you want to keep record of all queries that are executed, you can enable either the General Query Log or the Slow Query Log and set the threshold to 0 seconds (any query that takes more than 0 seconds will be logged).
Another option is the Binary log. However, binary log does not keep record of queries like SELECT or SHOW that do not modify data.
Note that these logs get pretty big pretty fast in a server with traffic. And that both might contain passwords so they have to be protected from unauthorized eyes.
You can use MySQL Proxy which stands between client app and RDBMS itself.
http://forge.mysql.com/wiki/MySQL_Proxy
You can see online queries and also it allows you to rewrite queries based on rules.
There's another option - use a profiler.
For instance: http://www.jetprofiler.com/
Recently, I noticed that my MySQL server processes a lot of INSERT's. How can I detect user or DB on which is this activivty??
insert 33 k 97.96 k 44.21%
SHOW FULL PROCESSLIST will return every connection, user, and query currently active, if you have the PROCESS permission. That's more for immediate problems, but it has the least overhead.
If you use query logging, then instead of the regular query log (it can slow your server down noticeably) use the binary log to keep it minimal. It only tracks actions that change tables, like CREATE/DROP/ALTER and INSERT/UPDATE/REPLACE.
What you should log periodically (once a minute):
SHOW FULL PROCESSLIST;
SHOW GLOBAL STATUS;
with slow log enabled this will give you huge chance that any question can be solved.
If you have binary logging enabled you can check time/user who inserted rows.
If you have general log enabled then everything is logged.
Look in your query logs. This will show every connect into MySQL, and show every command that they execute.