I need your advice. Need to reclaim disk space on live server with minimum downtime. We are using:
mysql -- 5.5
innodb table per file -- on
Was a huge table(70% insers/30% deletes -- it means that sometimes we delete rows from this table), this table was dropped with "drop table" command, as we expect, mysql didn't release disk space to OS, but now we need to release free space. OPTIMIZE command is possible way -- but now we don't 100% sure if we will have enough free space on disk to do this command, because previous table was huge. But in mysql documentation written:
You can run OPTIMIZE TABLE to compact or recreate a file-per-table tablespace. When you run an OPTIMIZE TABLE, InnoDB creates a new .ibd file with a temporary name, using only the space required to store actual data. When the optimization is complete, InnoDB removes the old .ibd file and replaces it with the new one.
I highlighted confusing moment in documentation, does it means, that "OPTIMIZE dropped_table;" will use only space for data that really needed, for this situation there is no actual data if we drop table early ?
If the table was created when innodb_file_per_table was OFF, then the table was written to ibdata1, and the space is not returned to the OS by DROP TABLE.
If it had been ON, then there would be a .ibd file. Did you see that file before the DROP? Does it still exist? (It should not still exist.)
When DROP removes the .ibd file, there is a slight lag (depending on the OS) while waiting for the data to be actually freed up by the OS. du and/or df reflects this lag.
OPTIMIZE TABLE will copy the table to tmpdir (I think), then drop the old table and play RENAME games. This requires extra space -- enough for a compete copy of the table. Assuming you really have a .ibd file, it will create another .ibd file (with a temp name) and shuffle files. If tmpdir points to a different "filesystem", the shuffle will involve a copy, not just a move.
If I have not explained things adequately, please provide more details -- actual commands you are proposing, size of table in question, whether the .ibd exists, etc.
Related
Using: MySQL 5.6 on Debian 9, total DB size is around 450Gb
Updated to 5.7, ran mysql_upgrade, noticed that around 150 GB has been taken up. 2 tables are really large and they stayed in 'copying to tmp table' for a couple of hours
Noticed innodb_file_per_table was on and created large ibd files that weren't there previously.
Restored from a snapshot, disabled the file_per_table, ran mysql_upgrade again. 100GB gone, which is almost 1/4 of my total DB.
In the first case, it pulled the data from ibdata and put it into a separate file but ibdata never shrinks so taken space almost doubled.
What happens in the second case? Does the temp table get created within the ibdata file that never shrinks so even when table is not used anymore - space is still gone?
Another thing I noticed is that space consumption doesn't start until query has been in copying to tmp table status for like an hour or so.
1) Is there any way to avoid/minimize space increase?
Would running update with file_per_table on, then disabling it and running alter table engine innodb free up the space?
2) Any way to predict how much space will be occupied? At least per table
3) How does max_tmp_table_size play into this?
It sounds like you painted yourself into a corner by not running innodb_file_per_table from the start, so now you have a huge, unshrinkable ibdata1 file.
1) There isn't.
1.1) It might reduce the overall space usage by rebuilding the tables outside the ibdata1 file, then rebuild them again to inside ibdata1, reusing some of the unused space inside ibdata1
2) Yes:
SELECT TABLE_SCHEMA, TABLE_NAME, DATA_LENGTH, INDEX_LENGTH FROM information_schema.TABLES;
3) It doesn't. The tables you are seeing are probably tables being rebuilt for some reason (not sure why, I have to admit I haven't seen that happen from mysql_upgrade before). max_tmp_table_size is only for implicit (when a query plan says using temporary) and explicity (CREATE TEMPORARY TABLE ...) temporary tables, not for table rebuilds.
The only(?) way to switch to file_per_table without the disk bloat is
Dump the data.
Get a fresh install (or otherwise get rid of ibdata1).
Reload (with file_per_table on).
I have a 20gb MySQL database which contains about 30 million rows of data in one table. I wanted to delete a column within that table to clear up some space.
I dropped the column via phpMyAdmin which took about 15 minutes and checked the disk usage in my server. It went from 20gb to 28gb used after deleting a column that was filled with data.
Did I do something wrong? Am I missing something obvious?
I did nothing else while I was trying to delete the column.
Update: Actually, when I go to the database summary page that lists the tables in phpMyAdmin, it also lists their size. The size of the tables only adds up to 10gb.
When I check the size being used on my server via df -h it says /dev/vda is using 28gb.
The only other files on the server are a few small PHP files. What is taking up this 18gb gap?
Most likely explanation is that you are not using file-per-table mode, and the ibdata1 file that contains all tables and indexes increased during the ALTER TABLE. When you delete a column, MySQL has to create a new copy of the table, move all the data into it (minus the column you deleted) and then drop the original table. Near the end of this process, two copies of the table (minus the column you deleted in one copy) need to be stored simultaneously. Then when that's complete, it drops the original copy and that space can be reused in the future.
Unfortunately, there's no way to shrink the ibdata1. It remains at a size equal to its high-water mark. To recover the disk space, you have to:
Dump all InnoDB tables (or convert them to MyISAM temporarily)
Shut down MySQL Server
Remove ibdata1 and ib_logfile*
Set innodb_file_per_table=1 in your my.cnf
Restart. The missing ibdata1 will be created at the initial default size.
Import your data.
Now the tables will occupy a separate file per table. When you ALTER or OPTIMIZE them, they will recover space. This mode is enabled by default in MySQL 5.6.
How can I restore a partitioned mysql InnoDB table from just the .ibd files of the form TableName#P#pname.ibd ? Chris Calendar's article here
http://www.chriscalender.com/?p=28
works for non-partitioned tables with a single .ibd file, but the "discard" and "import" steps result in a "storage engine doesn't have this option" error for partitioned tables.
Wes Smith in one of the comments at the link above suggests a manual procedure to import one partition at a time, but that did not work for me. If I try to follow his approach by creating a non-partitioned table, moving the first .ibd file renamed as TableName.ibd, and doing an import, the import succeeds, but there are zero rows in the table. The subsequent suggested step of "add back in the partition", which I tried as "alter table TableName partition by range ... (partition pname1 values less than (...) ENGINE=InnoDB)"shockingly replaces the TableName.ibd file (corresponding to the first partition) with a fresh TableName#P#pname1.ibd of a trivial size. I lost one partition's worth of data trying this. I have about 150 partitions to recover.
Any advice on how to recover the data from the .ibd files? Thanks.
So, it turns out that the approach suggested in the comment at the link in the original post does work after all. The deletion of the partition I mentioned in the "alter table..." step above was legitimate as it had been intentionally deleted in the original table before the loss of the ibdata and log files (but mysql leaves the .ibd files undeleted anyway). The recovery process is painfully slow, but I have managed to script it up and hope to let it run to completion over the next several days.
Here are some tips if you find yourself in a similar situation with many partitions to recover. Suppose you had a table TableName with 100 partitions. Typically, the 100 partitions would have consecutive innodb IDs if they were created via a "create table" statement, but in general this may not be the case as partitions may have been added after creation. So, here are the steps:
1) Find out the innodb ID corresponding to each partition using Calendar's method in the blog above as follows. Note that this step does not need a restart of the mysql server. Just create a table like TableName without any partitions and discard its tablespace. Then move the first partition's .ibd file (named something like TableName#P#pname1.ibd) to the database directory as TableName.ibd and try to import it. Look into the mysql error.log (typically /var/log/mysql/error.log) to see what that partition's innodb ID is. Repeat this step for each partition (or as needed) until you have the 2-tuples (innodb_ID_i, partition_boundary_i) for all partitions in a file.
2) Start with an empty innodb state (stop the server, delete ibdata and ib_logfile*, restart server). For each innodb_ID entry in the file in step 1 above, create a table TableName_i like TableName. E.g., if the 100 partitions correspond to the IDs 321, 322,..., 370, 415, 416,...464 (two blocks of 50 contiguous IDs each), then write a script to create 320 dummy tables, 50 tables like TableName, 45 dummy tables, and 50 tables like TableName.
3)
For each TableName_i table created above,
--do
(i) rename table TableName_i to TableName
(ii) alter table TableName discard tablespace // important to do this step before the next one
(iii) mv TableName#P#pname_i.ibd TableName.ibd // with the appropriate directory prefixes
(iv) alter table TableName import tablespace
(v) alter table partition by range (partition_field) (partition pname_i values less than (partition_boundary_i)) // This is the most and only time consuming step
(vi) rename table TableName to TableName_i // or some other name or just dump it to a file
--repeat
Note that all of the above steps are scriptable and do not require restarting the server at any point except at the beginning of step 2 to start with an empty innodb state. Be careful to introduce checks for the success of each sub-step in step 3 before moving to the next, otherwise successive steps may fail and/or .ibd files may get overwritten. If feasible, use a copy in step 3(iii) instead of mv.
A final note: There might be a slightly easier alternative using percona's recovery toolkit using hex editing, but this did not work for my case of partitioned tables. I ran into the same, seemingly unresolved issue, with partitioned tables as noted in one of the comments at http://www.mysqlperformanceblog.com/2011/05/13/connecting-orphaned-ibd-files/ . Your mileage may vary though. If there were a way to avoid recreating partitions (like in step 3(v) above), that would be real nice and quick, but I am not sure if there is one.
I have some large InnoDB databases, close to 1TB.
In order to free up some space while working on an alternative storage, I deleted some unused InnoDB indexes hoping that it would free up some space.
It freed nothing.
As it's InnoDB, will the engine use the empty allocated space for further inserts and indexing?
InnoDB stores data in a tablespace. By default, there is one single tablespace and data of all the databases is stored in one file. This file has data dictionary, tables, as well as indexes in it. There is a global parameter innodb_data_file_path that defines this tablespace file. It has a syntax like ibdata1:256M:autoextend, this means at the beginning a file of size 256 MB will be created and then whenever the data size exceeds this, the file will be auto-extended. The innodb_autoextend_increment variable defines in MB's that by how much each increment should be.
How you will get your Index Space?
You should first backup all InnoDB tables and change setting in my.ini/my.cnf as innodb_file_per_table and restart MySQL server.
Now import those tables now each table will have it's own tablespace which can shrink size on deleting a table.
These are possible workarounds:
Separate Files per Table: InnoDB provides this option where data (data + indexes) for each table can be stored in a separate file through a global variable innodb_file_per_table.
Fixed Tablespace size: One way to work around with the tablespace file size problems is to fix the tablespace size (remove autoextend) to an extrapolated value. So, when you hit the limit, you know it is time to cleanup.
Move to MyISAM: For all the tables (or even databases), for which you feel data is not that critical to have transactions et al, move them to MyISAM.
Answer here: https://stackoverflow.com/a/8932289/82114
If you don't use innodb_file_per_table, reclaiming disk space is possible, but quite tedious, and requires a significant amount of downtime.
The "How To" can be found here: http://dev.mysql.com/doc/refman/5.6/en/innodb-data-log-reconfiguration.html
Be sure to also retain a copy of your schema in your dump.
Currently, you cannot remove a data file from the system tablespace.
To decrease the system tablespace size, use this procedure:
Use mysqldump to dump all your InnoDB tables.
Stop the server.
Remove all the existing tablespace files, including the ibdata and
ib_log files. If you want to keep a backup copy of the information,
then copy all the ib* files to another location before the removing
the files in your MySQL installation.
Remove any .frm files for InnoDB tables.
Configure a new tablespace.
Restart the server.
Import the dump files.
You should reorganize the InnoDB infrastructure. Why? Because ibdata1 never shrinks. With innodb_file_per_table disabled, it will bloat ibdata1 in a hurry? Aside from data and indexes, what lives in ibdata1?
Data dictionary
Double write buffer (handles Background Page Writes)
Insert Buffer (andles Changes to Secondary Indexes)
Rollback Segments
Undo Tablespace
That being said, you need to migrate your data out, delete all InnoDB relayed files, and reload with two things:
innodb_file_per_table enabled
bigger transaction logs
I wrote up how to do this back on Oct 29, 2010 : Howto: Clean a mysql InnoDB storage engine?
Going forward you could alwsy shrink individual InnoDB tables. For example, to shrink mydb.mytable:
ALTER TABLE mydb.mytable ENGINE=InnoDB;
My ibdata1 file for MySQL database grew to about 32GB over time. Recently I deleted about 10GB of data from my databases (and restarted mysql for good measure), but the file won't shrink. Is there any way to reduce the size of this file
The file size of InnoDB tablespaces will never reduce automatically, no matter how much data you delete.
What you could do, although it is a lot of effort, is to create one tablespace for every table by setting
innodb_file_per_table
The long part about this is, that you need to export ALL DATA from the mysql server (setting up a new server would be easier) and then reimport the data. Instead of one single ibdata1 file which holds the data for each and every table, you will find a lot of files called tablename.ibd which hold the data only for one single table.
Afterwards:
When you then delete a lot of data from tables, you can let mysql recreate the data-file by issuing
alter table <tablename> engine=myisam;
to switch to MyIsam (and have the InnoDB data file for this table deleted) and then
alter table <tablename> engine=innodb;
to recreate the table.
Unless you set innodb_file_per_table, ibdata1 holds all InnoDB tables plus undo.
This file never shrinks.
To shrink it, you should (at your own risk):
Backup and drop all InnoDB tables in all databases
Delete the file manually
Reinitialize InnoDB storage (by restarting mysqld) and
Restore the tables from backup.
If you set innodb_file_per_table, you'll still have to do this to reclaim the space, but in this case you'll be able to do this on per-table basis, without affecting the other tables.
Note that the undo is still held in ibdata, even with innodb_file_per_table set.
Adding, Removing, or Resizing InnoDB Data and Log Files
Run optimize table your_db.your_table; sql request
or use mysql workbench migration wizard and it will create database copy with reduced size