How to log the Audit log for activity happening on Mysql server. I want to log precise query like Delete, Insert, update, drop, truncate etc.
Please help if anybody has solution for this requirement.
what you need is the BInary Bin Log:
http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
The binary log is not used for statements such as SELECT or SHOW that do not modify data. To log all statements (for example, to identify a problem query), use the general query log. See Section 5.2.2, “The General Query Log”.
Running a server with binary logging enabled makes performance slightly slower. However, the benefits of the binary log in enabling you to set up replication and for restore operations generally outweigh this minor performance decrement.
Related
MySql 5.5 has a few logging option, among which the "Binary Logfile" with Binlog options which I do not want to use and the "query log file" which I want to use.
However, 1 program using 1 table in that database is filling this logfile with 50+Mb per day, so I would like that table to be excluded from this log.
Is that possible, or is the only way to install another MySql version and then to move this 1 table?
Thanks,
Alex
There are options for filtering the binlog by table, but not the query logs.
There are no options for filtering the general query log. It is either enabled for all queries, or else it's disabled.
There are options for filtering the slow query log, but not by table. For example, to log only queries that take longer than N seconds, or queries that don't use an index. Percona Server adds some options to filter the slow query log based on sampling.
You can use a session variable to disable either slow query or general query logging for queries run in a given session. This is a dynamic setting, so you can change it at will. But you would need to change your client code to do this every time you query that specific table.
Another option is to implement log rotation for the slow query log, so it never grows too large. See https://www.percona.com/blog/2013/04/18/rotating-mysql-slow-logs-safely/
One of our users is using Cloud SQL(MySQL).
They turn on general logs flag and log_output is file.
They need these general logs because of some special circumstances.
MySQL generates about 8TB of general logs and these logs result a bigger use of the disk.
Here is the tricky part:
They want to remove these general logs file [1] to decrease the size of the disk.
However, this is their production database. They afraid this operation will impact their database's performance.
Since these log files are located in /var/log/mysql.log, the remove logs operation will execute on the OS level, right? -> This is the part we are not so sure.
If our user executes this truncateAPI, will this operation affect their database's performance?
Is there any best practice for this kind of situation?
P.S: Our user doesn't want to turn off general logs flag. They will try to truncate these logs once for a while. But for now, they need to truncate the huge amount of logs that they accumulated in the past few momths.
[1] https://cloud.google.com/sql/docs/mysql/admin-api/v1beta4/instances/truncateLog
I understand that you have turned on general logs flag and log_output is FILE and you want to remove these general logs files to decrease the size of the disk.
According to to the official documentation link:
To make your general or slow query logs available, enable the
corresponding flag and set the log_output flag to FILE. This makes the
log output available using the Logs Viewer in the Google Cloud
Platform Console. Note that Stackdriver logging charges apply.
If log_output is set to NONE, you will not be able to access the logs.
If you set log_output to TABLE, the log output is placed in a table in
your database. If this table becomes large, it can affect instance
restart time or cause the instance to lose its SLA coverage; for this
reason, the TABLE option is not recommended. If needed, you can
truncate your log tables by using the API. For more information, see
the instances.truncateLog reference page.
Instances: truncateLog truncate MySQL general and slow query log tables.
If I understood correctly, you can not "truncate the huge amount of logs that they accumulated in the past few months" because you did not set log_output to TABLE, therefore there are no tables to be truncated.
Regarding database performance: TRUNCATE TABLE Statement
On a system with a large InnoDB buffer pool and
innodb_adaptive_hash_index enabled, TRUNCATE TABLE operations may
cause a temporary drop in system performance due to an LRU scan that
occurs when removing an InnoDB table's adaptive hash index entries.
The problem was addressed for DROP TABLE in MySQL 5.5.23 (Bug
13704145, Bug #64284) but remains a known issue for TRUNCATE TABLE (Bug #68184).
Here you can check MySQL Server Log Maintenance.
Removing MySQL general_log FILES should not impact the performance of the database.
I am trying to verify database histories. For that, I need to log and parse read-write accesses to a database.
I know MySQL mainly offers two types of logs - general query log and binary log. General query log is not helpful for me since it does not log end timestamps. Wheresas, binary log is perfect for my work, but it does not log read access, i.e. SELECT queries.
So I want to know if there is a way to force binary logging such that it logs non-modifying queries too.
The binary log only records changes to data or metadata. There is no option to make it log SELECT queries.
You overlooked the MySQL slow query log, which does record the timestamp and the duration of the query. You can make the slow query log include all queries by setting long_query_time=0 so that it logs all queries that take 0 seconds or more (that is, all of them).
Be careful! In a typical system, the number of SELECT queries is a lot more than the number of other queries. Your logs could grow very rapidly. Make sure to configure log rotation, or you'll fill up your disk.
Other than that, it sounds like you need an audit log. See https://dev.mysql.com/doc/refman/5.7/en/audit-log.html for the MySQL Enterprise audit log plugin (this requires a support subscription from Oracle).
There are at least a couple of open-source implementations of audit log plugins for MySQL. For example: https://www.percona.com/doc/percona-server/LATEST/management/audit_log_plugin.html
I want to get mysqldump after certain checkpoint e.g. if i take the mysqldump now then next time when i will take the dump it should give me only the commands which executed between this time interval. is there anyway to get this using mysqldump.
One more thing how to show the commands delete, update in the mysqldump files.
Thanks
I dont think this is possible from a MySQLdump, however that feature exists as part of MySQL core - its called Binlogging or binary logging.
The binary log contains “events” that describe database changes such as table creation operations or changes to table data. It also contains events for statements that potentially could have made changes (for example, a DELETE which matched no rows). The binary log also contains information about how long each statement took that updated data
Check this out http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
Word of warning, binlogs can slow down the performance of your server.
Recently, I noticed that my MySQL server processes a lot of INSERT's. How can I detect user or DB on which is this activivty??
insert 33 k 97.96 k 44.21%
SHOW FULL PROCESSLIST will return every connection, user, and query currently active, if you have the PROCESS permission. That's more for immediate problems, but it has the least overhead.
If you use query logging, then instead of the regular query log (it can slow your server down noticeably) use the binary log to keep it minimal. It only tracks actions that change tables, like CREATE/DROP/ALTER and INSERT/UPDATE/REPLACE.
What you should log periodically (once a minute):
SHOW FULL PROCESSLIST;
SHOW GLOBAL STATUS;
with slow log enabled this will give you huge chance that any question can be solved.
If you have binary logging enabled you can check time/user who inserted rows.
If you have general log enabled then everything is logged.
Look in your query logs. This will show every connect into MySQL, and show every command that they execute.