I have my VPS installed cPanel. Today it suddenly not work, all of Wordpress websites on VPS show the error:
This webpage has a redirect loop
ERR_TOO_MANY_REDIRECTS
and auto redirect to wp-admin/install.php.
I log in to phpmyadmin then see some tables have collation "in use" (not "utf8_general_ci" such as normally). Can't be loaded because of error:
#1286 - Unknown storage engine 'InnoDB'".
So how to fix this error?
Thanks for help me!
I faced the same issue, this issue raised up because your innoDB got corrupted and all your databases using innoDB tables are showing in-use.
To fix this issue you need to follow the below steps
To get 100% clean tablespace you need to start MySQL with innodb_force_recovery=4, take mysqldump and restore it on a fresh instance of InnoDB (by fresh I mean you have to delete ibdata1, and all databases directories).
UPDATE:
At this point MySQL is started with innodb_force_recovery=x (x != 0)
Take dump of all databases:
mysqldump --skip-lock-tables -A > alldb.sql
Check where MySQL keeps its files(in my case it's /var/lib/mysql/):
mysql -NBe "SELECT ##datadir"
/var/lib/mysql/
Stop MySQL
mysqladmin shut
Move old MySQL files to safe place
mv /var/lib/mysql /var/lib/mysql.old
Create new system database
mkdir /var/lib/mysql
mysql_install_db
Start MySQL
/etc/init.d/mysql start
Restore the dump
mysql < alldb.sql
Restore may take long time if the database is big.
Another trick may work in that case. Run ALTER TABLE ... ENGINE INNODB on each InnoDB table. It will rebuild all InnoDB indexes and thus the errors will go away.
+++++++++++++++++++++++++++++++++++++++++++++++++
Another solution to this is restoring the databases from backup.
For this first you need to remove ibdata1 file
cd /var/lib/mysql
rm -f ibdata1
Then restore all the databases one by one using below command
mysql -u username -p databasename < backupfile.sql
++++++++++++++++++++++++++++++++++++++++++++++++++
Related
I've got ~ 30 sites worth of databases data on a volume on my server in the following directory /mnt/volume0/var/lib/mysql
The directory contains a 6.2G ibdata1 file and directories containing .frm files for each of the websites.
I'm trying to get dumps of the databases so I can move them to another server, however accessing mysql only shows the default databases (as datadir in /etc/mysql/mysql.conf.d/mysqld.cnf is set to /var/lib/mysql)
I've tried changing the datadir in /etc/mysql/mysql.conf.d/mysqld.cnf to /mnt/volume0/var/lib/mysql but mysql fails to start.
It was complaining about "failed to set datadir" which I fixed by running chown mysql:mysql mysql/ -R on /mnt/volume0/var/lib/mysql but now it's complaining about Failed password for root from xxx.xx.xxx.xx port 19192 ssh2
Any help is greatly appriciated!
Thanks
You can try to reset your password
mysqld_safe --skip-grant-tables
if you have here an error, then you have a bigger problem
mysql --user=root mysql
update user set Password=PASSWORD('your new password') where user='root';
flush privileges;
exit;
I ended up solving this issue by replacing the contents of /var/lib/mysql/ on my local machine with the contents of /mnt/volume0/var/lib/mysql/ (from the server).
I then encountered an error whilst trying to start MySQL Table 'performance_schema.session_variables' doesn't exist which I solved by upgrading MySQL on my local machine and restarting: https://stackoverflow.com/a/20262826/8231049
Now I'm able to dump the databases from my local machine :)
Edit: Can't accept my answer before waiting 2 days. Will do then.
I'm trying to get dump of my database:
mysqldump myDatabase > myDatabase.sql
but I'm getting this error:
mysqldump: Got error: 1146: Table 'myDatabase.table' doesn't exist when using LOCK TABLES
When I go to mysql:
mysql -u admin -p
I query for the tables:
show tables;
I see the table. but when I query for that particular table:
select * from table;
I get the same error:
ERROR 1146 (42S02): Table 'myDatabase.table' doesn't exist
I tried to repair:
mysqlcheck -u admin -p --auto-repair --check --all-databases
but get the same error:
Error : Table 'myDatase.table' doesn't exist
Why I'm getting this error or how can I fix this error?
I'll really appreciate your help
For me the problem was resolved by going to /var/lib/mysql (or wherever you raw database files are stored) and deleting the .frm file for the table that the errors says does not exist.
I had an issue with doing mysqldump on the server, I realized that tables that if that tables were not used for longer time, then I do not need those (old applications that were shutdown).
The case: Cannot do backup with mysqldump, there are tables that are not needed anymore and are corrupted
At first I get the list of corrupted tables
mysqlcheck --repair --all-databases -u root -p"${MYSQL_ROOT_PASSWORD}" > repair.log
Then I analyze the log with a Python script that takes it at stdin (save as ex. analyze.py and do cat repair.log| python3 analyze.py)
#!/usr/bin/env python3
import re
import sys
lines = sys.stdin.read().split("\n")
tables = []
for line in lines:
if "Error" in line:
matches = re.findall('Table \'([A-Za-z0-9_.]+)\' doesn', line)
tables.append(matches[0])
print('{', end='')
print(",".join(tables), end='')
print('}', end='')
You will get a list of corrupted databases.
Do an export with mysqldump
mysqldump -h 127.0.0.1 -u root -p"${MYSQL_ROOT_PASSWORD}" -P 3306 --skip-lock-tables --add-drop-table --add-drop-database --add-drop-trigger --all-databases --ignore-table={table1,table2,table3 - here is output of the previous command} > dump.sql
Turn off the database, move /var/lib/mysql to /var/lib/mysql-backup, start database.
On a clean database just import the dump.sql, restart database, enjoy an instance without corrupted tables.
I recently came across a similar issue on an Ubuntu server that was upgraded to 16.04 LTS. In the process, MySQL was replaced with MariaDB and apparently the old database couldn't be automatically converted to a compatible format. The installer moved the original database from /var/lib/mysql to /var/lib/mysql-5.7.
Interestingly, the original table structure was present under the new /var/lib/mysql/[database_name] in the .frm files. The new ibdata file was 12M and the 2 logfiles were 48M, from which I concluded, that the data must be there, but later I found that initializing a completely empty database results in similar sizes, so that's not indicative.
I installed 16.04 LTS on a VirtualBox, installed MySQL on it, then copied the mysql-5.7 directory and renamed it to mysql. Started the server and dumped everything with mysqldump. Deleted the /var/lib/mysql on the original server, initialized a new one with mysql_install_db and imported the sql file from mysqldump.
Note: I was not the one who originally did the system upgrade, so there may be a few details missing, but the symptoms were similar to yours, so maybe this could help.
Trying to find out how people do a full backup/restore procedure: The user defined database schema and data can be easily backed up via mysqldump, but what about the master tables and data? i.e. if the server goes completely bananas, how can I rebuild the database, i.e. including all the settings in Mysql? is it just a matter of dumping/importing the information_schema and mysql databases + restore my.cnf ? (innodb or MyISAM, not ISAM)
--
edit: Thanks!
You don't back up information_schema, but otherwise, yes, keep a copy of your my.cnf and a dump of the mysql db tables and log settings. To do this do:
mysqldump -u$user -p$pass --all-databases > db_backup.sql
If you're going to restore to the 100% same version of MySQL, you could also backup by shutting down your server and doing a full copy of the contents of /var/lib/mysql (or wherever your data files are) along with your my.cnf file. Then just drop the copy back in place when you want to go live and turn on your server.
I am using MySQL in localhost as a "query tool" for performing statistics in R, that is, everytime I run a R script, I create a new database (A), create a new table (B), import the data into B, submit a query to get what I need, and then I drop B and drop A.
It's working fine for me, but I realize that the ibdata file size is increasing rapidly, I stored nothing in MySQL, but the ibdata1 file already exceeded 100 MB.
I am using more or less default MySQL setting for the setup, is there a way for I can automatically shrink/purge the ibdata1 file after a fixed period of time?
That ibdata1 isn't shrinking is a particularly annoying feature of MySQL. The ibdata1 file can't actually be shrunk unless you delete all databases, remove the files and reload a dump.
But you can configure MySQL so that each table, including its indexes, is stored as a separate file. In that way ibdata1 will not grow as large. According to Bill Karwin's comment this is enabled by default as of version 5.6.6 of MySQL.
It was a while ago I did this. However, to setup your server to use separate files for each table you need to change my.cnf in order to enable this:
[mysqld]
innodb_file_per_table=1
https://dev.mysql.com/doc/refman/5.6/en/innodb-file-per-table-tablespaces.html
As you want to reclaim the space from ibdata1 you actually have to delete the file:
Do a mysqldump of all databases, procedures, triggers etc except the mysql and performance_schema databases
Drop all databases except the above 2 databases
Stop mysql
Delete ibdata1 and ib_log files
Start mysql
Restore from dump
When you start MySQL in step 5 the ibdata1 and ib_log files will be recreated.
Now you're fit to go. When you create a new database for analysis, the tables will be located in separate ibd* files, not in ibdata1. As you usually drop the database soon after, the ibd* files will be deleted.
http://dev.mysql.com/doc/refman/5.1/en/drop-database.html
You have probably seen this:
http://bugs.mysql.com/bug.php?id=1341
By using the command ALTER TABLE <tablename> ENGINE=innodb or OPTIMIZE TABLE <tablename> one can extract data and index pages from ibdata1 to separate files. However, ibdata1 will not shrink unless you do the steps above.
Regarding the information_schema, that is not necessary nor possible to drop. It is in fact just a bunch of read-only views, not tables. And there are no files associated with the them, not even a database directory. The informations_schema is using the memory db-engine and is dropped and regenerated upon stop/restart of mysqld. See https://dev.mysql.com/doc/refman/5.7/en/information-schema.html.
Adding to John P's answer,
For a linux system, steps 1-6 can be accomplished with these commands:
mysqldump -u [username] -p[root_password] [database_name] > dumpfilename.sql
mysqladmin -u [username] -p[root_password] drop [database_name]
sudo /etc/init.d/mysqld stop
sudo rm /var/lib/mysql/ibdata1
sudo rm /var/lib/mysql/ib_logfile*
sudo /etc/init.d/mysqld start
mysqladmin -u [username] -p[root_password] create [database_name]
mysql -u [username] -p[root_password] [database_name] < dumpfilename.sql
Warning: these instructions will cause you to lose other databases if you have other databases on this mysql instance. Make sure that steps 1,2 and 6,7 are modified to cover all databases you wish to keep.
When you delete innodb tables, MySQL does not free the space inside the ibdata file, that's why it keeps growing. These files hardly ever shrink.
How to shrink an existing ibdata file:
https://dev.mysql.com/doc/refman/5.6/en/innodb-system-tablespace.html#innodb-resize-system-tablespace
You can script this and schedule the script to run after a fixed period of time, but for the setup described above it seems that multiple tablespaces are an easier solution.
If you use the configuration option innodb_file_per_table, you create multiple tablespaces. That is, MySQL creates separate files for each table instead of one shared file. These separate files a stored in the directory of the database, and they are deleted when you delete this database. This should remove the need to shrink/purge ibdata files in your case.
More information about multiple tablespaces:
https://dev.mysql.com/doc/refman/5.6/en/innodb-file-per-table-tablespaces.html
Quickly scripted the accepted answer's procedure in bash:
#!/usr/bin/env bash
dbs=$(mysql -BNe 'show databases' | grep -vE '^mysql$|^(performance|information)_schema$')
mysqldump --events --triggers --databases $dbs > alldatabases.sql && \
echo "$dbs" | while read -r db; do
mysqladmin drop "$db"
done && \
mysql -e 'SET GLOBAL innodb_fast_shutdown = 0' && \
/etc/init.d/mysql stop && \
rm -f /var/lib/mysql/ib{data1,_logfile*} && \
/etc/init.d/mysql start && \
mysql < alldatabases.sql
Save as purge_binlogs.sh and run as root.
Excludes mysql, information_schema, performance_schema (and binlog directory).
Assumes you have administrator credendials in /root/.my.cnf and that your database lives in default /var/lib/mysql directory.
You can also purge binary logs after running this script to regain more disk space with:
PURGE BINARY LOGS BEFORE CURRENT_TIMESTAMP;
If you use the InnoDB storage engine for (some of) your MySQL tables, you’ve probably already came across a problem with its default configuration. As you may have noticed in your MySQL’s data directory (in Debian/Ubuntu – /var/lib/mysql) lies a file called ‘ibdata1′. It holds almost all the InnoDB data (it’s not a transaction log) of the MySQL instance and could get quite big. By default this file has a initial size of 10Mb and it automatically extends. Unfortunately, by design InnoDB data files cannot be shrinked. That’s why DELETEs, TRUNCATEs, DROPs, etc. will not reclaim the space used by the file.
I think you can find good explanation and solution there :
http://vdachev.net/2007/02/22/mysql-reducing-ibdata1/
If your goal is to monitor MySQL free space and you can't stop MySQL to shrink your ibdata file, then get it through table status commands. Example:
MySQL > 5.1.24:
mysqlshow --status myInnodbDatabase myTable | awk '{print $20}'
MySQL < 5.1.24:
mysqlshow --status myInnodbDatabase myTable | awk '{print $35}'
Then compare this value to your ibdata file:
du -b ibdata1
Source: http://dev.mysql.com/doc/refman/5.1/en/show-table-status.html
In a new version of mysql-server recipes above will crush "mysql" database.
In old version it works. In new some tables switches to table type INNODB, and by doing so you will damage them.
The easiest way is to:
dump all you databases
uninstall mysql-server,
add in remained my.cnf:
[mysqld]
innodb_file_per_table=1
erase all in /var/lib/mysql
install mysql-server
restore users and databases
What nobody seems to mention is the impact innodb_undo_log_truncate setting can have.
After reading Percona's blog post about the topic, I've enabled in my MariaDB 10.6 the truncation of UNDO LOG entries which filled 95% of ibdata1, and, after a complete drop and restore, from that moment on my ibdata1 never grew anymore.
With the default innodb_undo_log_truncate = 0 my ibdata1 easily reached 10% of databases space occupation, aka tens of Gigabytes.
With innodb_undo_log_truncate = 1, ibdata1 it's firm at 76 Mb.
As already noted you can't shrink ibdata1 (to do so you need to dump and rebuild), but there's also often no real need to.
Using autoextend (probably the most common size setting) ibdata1 preallocates storage, growing each time it is nearly full. That makes writes faster as space is already allocated.
When you delete data it doesn't shrink but the space inside the file is marked as unused. Now when you insert new data it'll reuse empty space in the file before growing the file any further.
So it'll only continue to grow if you're actually needing that data. Unless you actually need the space for another application there's probably no reason to shrink it.
I am using MySQL in localhost as a "query tool" for performing statistics in R, that is, everytime I run a R script, I create a new database (A), create a new table (B), import the data into B, submit a query to get what I need, and then I drop B and drop A.
It's working fine for me, but I realize that the ibdata file size is increasing rapidly, I stored nothing in MySQL, but the ibdata1 file already exceeded 100 MB.
I am using more or less default MySQL setting for the setup, is there a way for I can automatically shrink/purge the ibdata1 file after a fixed period of time?
That ibdata1 isn't shrinking is a particularly annoying feature of MySQL. The ibdata1 file can't actually be shrunk unless you delete all databases, remove the files and reload a dump.
But you can configure MySQL so that each table, including its indexes, is stored as a separate file. In that way ibdata1 will not grow as large. According to Bill Karwin's comment this is enabled by default as of version 5.6.6 of MySQL.
It was a while ago I did this. However, to setup your server to use separate files for each table you need to change my.cnf in order to enable this:
[mysqld]
innodb_file_per_table=1
https://dev.mysql.com/doc/refman/5.6/en/innodb-file-per-table-tablespaces.html
As you want to reclaim the space from ibdata1 you actually have to delete the file:
Do a mysqldump of all databases, procedures, triggers etc except the mysql and performance_schema databases
Drop all databases except the above 2 databases
Stop mysql
Delete ibdata1 and ib_log files
Start mysql
Restore from dump
When you start MySQL in step 5 the ibdata1 and ib_log files will be recreated.
Now you're fit to go. When you create a new database for analysis, the tables will be located in separate ibd* files, not in ibdata1. As you usually drop the database soon after, the ibd* files will be deleted.
http://dev.mysql.com/doc/refman/5.1/en/drop-database.html
You have probably seen this:
http://bugs.mysql.com/bug.php?id=1341
By using the command ALTER TABLE <tablename> ENGINE=innodb or OPTIMIZE TABLE <tablename> one can extract data and index pages from ibdata1 to separate files. However, ibdata1 will not shrink unless you do the steps above.
Regarding the information_schema, that is not necessary nor possible to drop. It is in fact just a bunch of read-only views, not tables. And there are no files associated with the them, not even a database directory. The informations_schema is using the memory db-engine and is dropped and regenerated upon stop/restart of mysqld. See https://dev.mysql.com/doc/refman/5.7/en/information-schema.html.
Adding to John P's answer,
For a linux system, steps 1-6 can be accomplished with these commands:
mysqldump -u [username] -p[root_password] [database_name] > dumpfilename.sql
mysqladmin -u [username] -p[root_password] drop [database_name]
sudo /etc/init.d/mysqld stop
sudo rm /var/lib/mysql/ibdata1
sudo rm /var/lib/mysql/ib_logfile*
sudo /etc/init.d/mysqld start
mysqladmin -u [username] -p[root_password] create [database_name]
mysql -u [username] -p[root_password] [database_name] < dumpfilename.sql
Warning: these instructions will cause you to lose other databases if you have other databases on this mysql instance. Make sure that steps 1,2 and 6,7 are modified to cover all databases you wish to keep.
When you delete innodb tables, MySQL does not free the space inside the ibdata file, that's why it keeps growing. These files hardly ever shrink.
How to shrink an existing ibdata file:
https://dev.mysql.com/doc/refman/5.6/en/innodb-system-tablespace.html#innodb-resize-system-tablespace
You can script this and schedule the script to run after a fixed period of time, but for the setup described above it seems that multiple tablespaces are an easier solution.
If you use the configuration option innodb_file_per_table, you create multiple tablespaces. That is, MySQL creates separate files for each table instead of one shared file. These separate files a stored in the directory of the database, and they are deleted when you delete this database. This should remove the need to shrink/purge ibdata files in your case.
More information about multiple tablespaces:
https://dev.mysql.com/doc/refman/5.6/en/innodb-file-per-table-tablespaces.html
Quickly scripted the accepted answer's procedure in bash:
#!/usr/bin/env bash
dbs=$(mysql -BNe 'show databases' | grep -vE '^mysql$|^(performance|information)_schema$')
mysqldump --events --triggers --databases $dbs > alldatabases.sql && \
echo "$dbs" | while read -r db; do
mysqladmin drop "$db"
done && \
mysql -e 'SET GLOBAL innodb_fast_shutdown = 0' && \
/etc/init.d/mysql stop && \
rm -f /var/lib/mysql/ib{data1,_logfile*} && \
/etc/init.d/mysql start && \
mysql < alldatabases.sql
Save as purge_binlogs.sh and run as root.
Excludes mysql, information_schema, performance_schema (and binlog directory).
Assumes you have administrator credendials in /root/.my.cnf and that your database lives in default /var/lib/mysql directory.
You can also purge binary logs after running this script to regain more disk space with:
PURGE BINARY LOGS BEFORE CURRENT_TIMESTAMP;
If you use the InnoDB storage engine for (some of) your MySQL tables, you’ve probably already came across a problem with its default configuration. As you may have noticed in your MySQL’s data directory (in Debian/Ubuntu – /var/lib/mysql) lies a file called ‘ibdata1′. It holds almost all the InnoDB data (it’s not a transaction log) of the MySQL instance and could get quite big. By default this file has a initial size of 10Mb and it automatically extends. Unfortunately, by design InnoDB data files cannot be shrinked. That’s why DELETEs, TRUNCATEs, DROPs, etc. will not reclaim the space used by the file.
I think you can find good explanation and solution there :
http://vdachev.net/2007/02/22/mysql-reducing-ibdata1/
If your goal is to monitor MySQL free space and you can't stop MySQL to shrink your ibdata file, then get it through table status commands. Example:
MySQL > 5.1.24:
mysqlshow --status myInnodbDatabase myTable | awk '{print $20}'
MySQL < 5.1.24:
mysqlshow --status myInnodbDatabase myTable | awk '{print $35}'
Then compare this value to your ibdata file:
du -b ibdata1
Source: http://dev.mysql.com/doc/refman/5.1/en/show-table-status.html
In a new version of mysql-server recipes above will crush "mysql" database.
In old version it works. In new some tables switches to table type INNODB, and by doing so you will damage them.
The easiest way is to:
dump all you databases
uninstall mysql-server,
add in remained my.cnf:
[mysqld]
innodb_file_per_table=1
erase all in /var/lib/mysql
install mysql-server
restore users and databases
What nobody seems to mention is the impact innodb_undo_log_truncate setting can have.
After reading Percona's blog post about the topic, I've enabled in my MariaDB 10.6 the truncation of UNDO LOG entries which filled 95% of ibdata1, and, after a complete drop and restore, from that moment on my ibdata1 never grew anymore.
With the default innodb_undo_log_truncate = 0 my ibdata1 easily reached 10% of databases space occupation, aka tens of Gigabytes.
With innodb_undo_log_truncate = 1, ibdata1 it's firm at 76 Mb.
As already noted you can't shrink ibdata1 (to do so you need to dump and rebuild), but there's also often no real need to.
Using autoextend (probably the most common size setting) ibdata1 preallocates storage, growing each time it is nearly full. That makes writes faster as space is already allocated.
When you delete data it doesn't shrink but the space inside the file is marked as unused. Now when you insert new data it'll reuse empty space in the file before growing the file any further.
So it'll only continue to grow if you're actually needing that data. Unless you actually need the space for another application there's probably no reason to shrink it.