We run a MySQL server on Windows. Sometimes, Windows crashes, and leaves the database in a corrupt state ("Duplicate entry for key 'PRIMARY'" messages though the key is auto-generated; that can be repaired with a "Repair table" command).
I have seen that problem with MyISAM tables only, never with an InnoDb table. Did that happen by chance, or is InnoDb with its extra code for transactions actually more resilient towards such failures?
MyISAM does not support any of the four qualities of ACID databases.
InnoDB is designed for crash recovery. It has several features that prevent such corruption in the case of crashes, and also recover from crashes automatically, so you don't have to run REPAIR TABLE.
InnoDB has been the default storage engine since MySQL 5.5.5 in 2010. It's clear that MySQL is moving (slowly) in the direction of phasing out MyISAM. In MySQL 8.0, even the system tables in the mysql schema have been converted to InnoDB.
It's really time to just consider MyISAM obsolete technology and stop using it. We should honor its place in history, but move on to a better solution.
Related
I have a table that crashes often, but only seems to crash when the replica is running.
The table is MyISAM. The table has 2 mediumtext fields. The error I get when making a delete statement is this: "General error: 1194 Table 'outlook_emails' is marked as crashed and should be repaired".
I wonder if this has to do with the binary log. However, it doesn't seem to happen when the binary log is running but the replica is down.
Any idea what is happening or what I can do to solve it or investigate further?
Table '...' is marked as crashed and should be repaired".
That error occurs (usually) when the MySQL server as been rudely restarted, and the table is ENGINE=MyISAM.
The temporary fix is to run CHECK TABLE, which will then suggest that you run REPAIR TABLE. The tool myisamchk is a convenient way to do them, especially since there could be several tables so marked.
The Engine InnoDB has radically different internals. It avoids the specific issue that MyISAM has, and does a much more thorough job of recovering from crashes.
Switching to InnoDB requires ALTERing your tables. Here is more discussion: http://mysql.rjweb.org/doc.php/myisam2innodb
I have an AWS RDS MySQL 5.7 database with MyISAM tables that I would like to migrate to another RDS in a custom VPC, and once migrated, convert those MyISAM tables to InnoDB.
If I undertood correctly, the only way to create a correct automatic backup is using the following procedure explained here: "Automated Backups with Unsupported MySQL Storage Engines"
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html#Overview.BackupDeviceRestrictions
Stop all activity to your MyISAM tables (that is, close all sessions).
You can close all sessions by calling the mysql.rds_kill command for each process that is returned from the SHOW FULL PROCESSLIST command.
Lock and flush each of your MyISAM tables
Create a snapshot of your DB instance. When the snapshot has completed, release the locks and resume activity on the MyISAM tables
Has someone done this procedure before?
How is that the snapshots are being created successfully every night from the current RDS DBInstance, even though it contains MyISAM tables?
Thanks!
The problem isn't with snapshot creation. It's what can go wrong when you actually try to use one of the snapshots.
RDS snapshots work by capturing a snapshot if your RDS instance's underlying EBS volume (you can't see this volume, but it's there -- RDS runs on EC2, with "hidden" instances and volumes).
EBS snapshots capture the entire contents of the hard drive exactly as they happened to exist at the moment in time when the snapshot process starts.
What ends up on the snapshot is essentially the same thing that you would have on a MySQL Server if you executed sudo killall -9 mysqld -- it is as if the server had halted everything, immediately, without doing any of the things it normally does to clean up for a graceful shutdown. With RDS, things are not quite that dramatic, because RDS does take some precautions, but fundamentally, this is the nature of what is happening.
When you create an RDS instance from a snapshot, the first thing that happens when the instance starts up is the same thing your hypothetical server would do when you restarted the killed MySQL Server daemon: InnoDB Crash Recovery.
InnoDB Crash Recovery
To recover from a MySQL server crash, the only requirement is to restart the MySQL server. InnoDB automatically checks the logs and performs a roll-forward of the database to the present. InnoDB automatically rolls back uncommitted transactions that were present at the time of the crash.
https://dev.mysql.com/doc/refman/5.7/en/innodb-recovery.html#innodb-crash-recovery
Crash recovery is InnoDB's mechanism for bringing everything back into harmony in it internal data structures and ensure that all data is intact, exactly as your application left it. It's possible because InnoDB is a transactional storage engine. That means a lot of different things, but what it specifically means in this case is that InnoDB doesn't just change table data when you change a table. It goes through a process that can be simplified something like this:
store the proposed changes to disk¹
actually make the changes
mark the changes as complete
What this means is that until the changes are finalized, InnoDB can be interrupted and will subsequently be able to pick up where it left off, without corrupting or losing data.
MyISAM has no such mechanisms. It just writes to the data files, directly. Even if a MyISAM table isn't actively being used, it may still need to be repaired when the server comes up, to clean up its structures. In some circumstances, repairing the table can be impossible, and all or part of the data in the table will be lost.
If your MyISAM tables are flushed and locked when the snapshot occurs, they are in a quiescent state on the disk, as though the server had actually been gracefully shut down before the snapshot had occurred, so they will be stable on the snapshot.
But the snapshot process will always appear to succeed, because the snapshot is just duplicating whatever is on the disk, as it appears at the moment in time when the snapshot gets underway.
The problem is that what the snapshot captured may not be usable, and you have no way of knowing whether the snapshot will be fully viable.
¹ Note that the first step, "store the proposed changes to disk" is related to the system variable innodb_flush_log_at_trx_commit which makes the system slower if set to 1 but also is the safest setting, because your query doesn't actually succeed until that first step is done. A setting of 2 is still reasonably safe, because it still writes the changes but continues without requiring that the operating system confirm that they have actually been written to the hard drive before your query returns success... but in a crash, a transaction your application thinks was committed may or may not have survived.
I have a wamp local server with only MyISAM tables, Total size is 350 MB and increases by about 10 MB per day. Would like to have a hourly backup solution. As recently WAMP server crashed and lost data.Should I create a trigger or schedule a task or use some utility it would be great if someone can guide. So that database backup is takeon either on local network or on different partition of server HDD. Have mirror RAID enabled on the server.
Since InnoDB does not lose data in a crash, and since you are talking about backing up a lot of data, and you are in the 'early stages', the only real answer to your question is to switch to InnoDB.
When the server crashes, MyISAM almost always leaves some indexes in need of REPAIR TABLE. This problem vanishes with InnoDB.
If the server crashes in the middle of a multi-row INSERT, UPDATE, or DELETE on a MyISAM table, some rows will be done and other rows won't be. And you have no way to know how much was done. With InnoDB, the entire query is "rolled back", making it as if it had not occurred. Very predictable.
See my tips
on converting.
For better recovery from a crash where the server cannot be restarted at all, consider Replication.
I fairly new to MySQL database. I am repeatedly seeing an error from MySQL saying the table is marked as crashed and should be repaired. However I am able to repair the crashed table by using the command myisamchk. By the way, I am using MYISAM database engine in MySQL.
I just wanted to know under what circumstances would a DB table crash and how I can prevent it from happening again?
I am connecting to MySQL(5.0) database from Tcl (8.5) script using mysqltcl library (3.0).
MyISAM tables are very easy to crash. There is header info in each table that keeps track of how many open file handles a MyISAM table has.
If mysqld crashes, any MyISAM table that had open file handles to it never had the opportunity to decrement the file handle count upon each file handle closing. Thus, if a new file handle opens a MyISAM table (.MYD file) and mysqld discovers a mismatch between the number of file handles a MyISAM table believes is open and the the number of file handles the MyISAM table actually has open, the table is declared crashed.
There are four(4) methods for handling this:
METHOD #1 : Setup automatic MyISAM repair
See my post https://dba.stackexchange.com/a/15079/877 on how to set this up upon a MySQL restart (Mar 15, 2012)
METHOD #2 : Use InnoDB instead of MyISAM
InnoDB has crash recovery built into the Storage Engine's initialization. MyISAM does not
METHOD #3 : Use Aria instead of MyISAM
Aria is MariaDB's drop-in replacement for MyISAM. It features crash recovery mechanisms for individual tables.
METHOD #4 : Don't kill -9 on mysqld
If mysqld crashes, deliberately or involuntarily, header info for all open MyISAM tables will get them into a crashed state. Avoid having to manually kill mysqld.
I noticed that when I attempt to do a LVM snapshot of my database volume, after running FLUSH TABLES WITH READ LOCK, then rsync that snapshot to a new system, the tables are marked as crashed and have to be repaired.
I suspect this has to do with there being a file handle on the original machine with the table open, and then I'm syncing the that status to the new machine and it sees a mismatch in the file handles and decides it needs to repair.
This repair is problematic because it takes hours (it is a giant table). So the only reliable way to actually get a snapshot that isn't crashed is to shutdown the database before taking the snapshot, but then I cannot get the SHOW MASTER STATUS to setup replication.
I just went to look over one my site's databases and noticed that all of the tables had been converted to MyISAM (they used to be InnoDB).
What's more is that InnoDB seems to missing, along with BerkeleyDB, Federated, and others.
A few months ago I upgraded from MySql 5.0 to 5.1.38. I can't imagine that I wouldn't have noticed if InnoDB was not installed after the upgrade, but maybe it's been that way since the upgrade. Having several 10 GB tables automatically convert themselves to MyISAM without hearing about any downtime seems very unlikely to me.
Regardless, the mysql system variable have_innodb is set to NO. Can I simply change that to YES or does that mean InnoDB is missing from the install?
Maybe InnoDB parameters were changed during upgrade? This advice from MySQL Forums helped me in a similar situation: http://forums.mysql.com/read.php?22,397052,408970
In short: Stop the MySQL daemon, delete ib_logfile* from datadir, restart MySQL.
If the field in SHOW ENGINES is "no" it means it's not compiled in. You would have to either compile the server again, compile the innodb plugin and load it or fetch server binaries which have it enabled.
You can't simply set the mysql system variable to YES to convert the table from MyISAM to InnoDB.
ALTER TABLE t1 ENGINE=InnoDB;
When InnoDB support is turned off even is you use ENGINE=InnoDB in your create table statements, the table will use the default storage engine for MySQL which is normally MyISAM.
It could also be the case that the innodb logfile[01] are corrupted. In which case the engine will show as disabled and the tables as ISAM. The log will mention it.