We have noticed that we run out of space on one of our machines that is running Mysql 5.5.37 (InnoDb)
Mysql seems to be generating several ~1.1GB files per day with the name pattern 0.0001234 (increasing number) in the /var/lib/mysql folder.
What are those files?
Can they be deleted?
Can we configure mysql to write them to a different location?
To avoid having to do the purge periodically, set this:
expire_logs_days = 7
in my.cnf. That will keep the binlogs purged down to 7 days' worth.
See also binlog_expire_logs_seconds in MySQL 8.0.
Turns out these were binlogs.
The setting log_bin=0
does not disable them but simply sets the file-name-format to 0.\d+
We deleted the present logs with PURGE BINARY LOGS to 'lastFileName', then disabled the generation by commenting out all bin_log related entries in the my.cnf.
Related
I have configured the database using SET GLOBAL binlog_expire_logs_seconds = 259200; query to purge binary logs older then 3 days. Using MySQL 8.0.22. This does not seem to have any effect. bin logs are accumulating under /var/lib/mysq/dbname-bin.000xx files. Executing
PURGE BINARY LOGS BEFORE '2020-.......'
works, but I would like the DB engine to purge those logs on its own without myself calling it. Otherwise binlogs take most of the disk space. Please suggest.
I had the same problem (but on Windows 10), I edited my.ini located in c:/xampp/mysql/bin and added the following:
binlog_expire_logs_seconds=60
Does this work for you?
I asked this on the MySQL forums, but didn't get any replies:
I've been manually running "PURGE BINARY LOGS TO" to purge the binary logs of a master DB for over a year now (1 slave DB on a different machine). After my most recent purge, over the next few hours, it generated a huge amount of logs - about 8 files of 1 GB each. Normally, it would take days before one 1GB file would be created. I did another purge that day and restarted mysql and it has returned to normal. What could have caused this? Is there anything I should be concerned about right now?
I can't remember the exact number of the purge command and it is now gone from history since I restarted mysql, but I think it was:
PURGE BINARY LOGS TO 'mysql-log.000101';
Thanks for any insight into this.
The Environment
We have a staging/test WordPress site on a CentOS server running MariaDB (version 10.3.8). We've been experimenting with a plugin called GeoDirectory (https://wpgeodirectory.com/) The plugin creates a variety of tables in the database. The database also is using the innodb_file_per_table set to ON, so the database has generated IBD files for each tablespace in the database.
The Issue
The staging server had 80 GB of storage. After making a particular change to the settings of the plugin (updating the Place Settings), we noticed that about ten minutes later the staging site was timing out when we tried to access it via the browser.
Logging in via SSH, we noticed that the machine was completely out of space. Looking for the largest files in /var, I noticed that one file was now taking up 70 GB of space (wp_g1a4rar7xx_geodir_gd_place_detail.ibd). This is the IBD file that corresponded to the table where the settings were updated.
Logging into SQL and running a count(*) on the number of records in that table, there were only 6,000 records.
Because the table had blown up to take up the entirety of the disk, even trying OPTIMIZE TABLE wouldn't work because of the lack of space.
The Question
What could have happened to make that file balloon so large so quickly? How can we avoid this happening in the future?
Reckless Speculation
Based on searches, we believe it might have something to do with the rollback/logging that InnoDB performs, and that we might be able to avoid this by making some changes in my.cnf to limiting logging. Again, it could be something completely different.
Here's the current my.cnf from /etc/my.cnf:
[mysqld]
bind-address = 0.0.0.0
# bind-address = ::ffff:127.0.0.1
local-infile=0
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security
risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid
#
# include all files from the config directory
#
!includedir /etc/my.cnf.d
Thanks in advance for your help!
Based on the name of the plugin, I am guessing that the issue could be related to updates of indexed geometry columns. MariaDB Server 10.2 imported the InnoDB support for SPATIAL INDEX from MySQL 5.7, and it looks like the purge of version history is sometimes being skipped, depending on concurrent DML activity.
I have mentioned this peculiar design choice in MDEV-15284, which reports that SELECT on a SPATIAL INDEX can return inconsistent results if a concurrent ROLLBACK is being executed.
I have a server on which 1 InnoDB table was inadvertently dropped. I need to recover the data for just this table, without rolling back the data from all the other tables that use InnoDB. My latest backup is from about 24 hours ago. Is there any way to do this?
Edited to add:
I should've specified: the backup is a tape backup, it is not a mysqldump file. What I have is a copy of ibdata and other mysql files from 24 hours ago. I did find a solution, see below.
Backup (usually) is simple SQL code. Open the file with text editor and get the data you need :).
Other way is to restore DB local, and export only the needed table.
This is what I ended up doing:
Shut down the MySQL server.
Made a copy of /var/lib/mysql (tar -cpvzf mysql.tgz /var/lib/mysql)
Copied /var/lib/mysql from the backup to replace the current /var/lib/mysql
Turned on MySQL server.
Mysqldump of needed database.
Shut down MySQL server.
Restored /var/lib/mysql from the tgz file I made in step 2
Turned on MySQL server.
Inserted data of needed database from mysqldump file created in step 5.
Hope this helps anyone faced with the same dilemma!
More Thoughts
I highly recommend turning on innodb_file_per_table=1 in my.cnf.
This should be done between steps 8 and 9 in the list above.
I have tried several commands (FLUSH LOGS, PURGE MASTER) but none deletes the log files (when previously activated) or the log tables (mysql/slow_log.CSV and mysql/general_log.CSV and their .frm and .CSM counterparts).
SHOW BINARY LOGS returns "You are not using binary logging".
Edit: I found this simple solution to clear the table logs (but not yet the file logs using a mysql command):
TRUNCATE mysql.general_log;
TRUNCATE mysql.slow_log;
FLUSH LOGS just closes and reopens log files. If the log files are large, it won't reduce them. If you're on Linux, you can use mv to rename log files while they're in use, and then after FLUSH LOGS, you know that MySQL is writing to a new, small file, and you can remove the old big files.
Binary logs are different. To eliminate old binlogs, use PURGE BINARY LOGS. Make sure your slaves (if any) aren't still using the binary logs. That is, run SHOW SLAVE STATUS to see what binlog file they're working on, and don't purge that file or later files.
Also keep in mind that binlogs are useful for point-in-time recovery in case you need to restore from backups and then reapply binlogs to bring the database up to date. If you need to use binlogs in this manner, don't purge the binlogs that have been written since your last backup.
If you are on amazon RDS, executing this twice will do the trick:
PROMPT> CALL mysql.rds_rotate_slow_log;
PROMPT> CALL mysql.rds_rotate_general_log;
Source: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html
It seems binary logging is not enabled in your server .And i guess you want to delete the old log files which were used/created at the time of binary logging is enabled . you can delete them manually using 'rm' command if you want . if you want to enable the binary logging you can do the same by updating the configuaration file ( but it needs restart of the server if it is already running) . You can refer below links.
http://dev.mysql.com/doc/refman/5.0/en/replication-options-binary-log.html#option_mysqld_log-bin
http://dev.mysql.com/doc/refman/5.0/en/replication-options-binary-log.html#sysvar_log_bin