sometimes i get an error like "table is marked as corrupt and shld be repaired". that DB (tables) is using MyISAM. recently that keeps happening. what could be the causes? most recently i am executing a batch insert
INSERT INTO table (..., ..., ...) VALUES (...), (...), (...) ...
and it just hung. or took very long to complete it seems hung to me. the next day, when i checked the table was marked as corrupt again. when i try to use mysqlcheck -r it said all tables OK when it reached that "corrupt" table it hung there again...
so, what can i do to prevent this. and what could be the causes. the DB is hosted 3rd party, how can i debug this?
is InnoDB a more reliable engine to use? i heard MyISAM is faster but others say InnoDB can be fast also but it takes abit more to optimize it. can i conclude that InnoDB is something more reliable but abit slower overall even with optimization?
If your tables get corrupt, you can use the repair table command to fix them:
REPAIR TABLE table;
If you run myisamchk while the server is still running (and inserts/selects are hitting the table), it could be what is corrupting your tables. Most of the corruption issues I run into are when trying to do things outside the server (copying the files, etc) while it is still running.
InnoDB is slower for read only databases, because it has features (ACID compliant, row level locking) that MyISAM leaves out. However, if you are doing a mixture of reads and writes, depending on the mixture, then InnoDB can offer serious performance improvements, because it doesn't have to lock the entire table to do a write. You also don't run into corruption issues.
Go with InnoDB.
ok, so the problem was the company's db exceeded the storage space allowed by the hosting company. so apparently noone told the company they exceeded the usage ... lousy host i guess.
btw, theres no way mysql could have known abt this?
Related
everyone! I'm trying to avoid breaking of my database in the phpbb3.1 forum. It was crushed twice this month.
So I have two questions:
1) Is it safe to convert MyISAM to InnoDB? I mean will extensions work fine? Will forum be workable after updating to next version?
2) In which way I can avoid base corrupting?
p.s.
I also posted this question here:
https://www.phpbb.com/community/viewtopic.php?f=466&t=2436326
I'll venture a guess. You had a power failure, and when it came back up, MySQL was complaining that some index on some table was corrupted? And that table was MyISAM?
Use myisamchk to repair the tables.
Review the gotchas in http://mysql.rjweb.org/doc.php/myisam2innodb to see if conversion to InnoDB will add new woes. There probably won't be any. A 2-part PRIMARY KEY is about the only thing that is not implemented in InnoDB. Also, if you have too old a version of MySQL, InnoDB may not yet have FULLTEXT indexes (if you need them).
Change my.cnf: key_buffer_size = 20M and innodb_buffer_pool_size equal to about half of available memory.
ALTER TABLE xx ENGINE=InnoDB; for each table xx.
I think (but am not sure) that each update/delete/insert marks the table as possibly corrupt. It writes the changes, but does not clear the mark. When mysqld shuts down cleanly, everything is flushed to disk and these flags are cleared. When mysqld comes back up, it complains about the flags that did not get cleared. So...
Whether or not an index is marked as corrupt depends solely on whether you modified that index and crashed. (Every table has some index, yes?)
Normally, MySQL manages to flush changes to disk before a crash. Only occasionally does the crash happen at a time where the index will really be corrupt. There is a "quick" mode on the repair that simply clears the flag -- you could try that. But if you ever get a mysterious "can't find record" when you know the records exists, you'd better REPAIR it.
Sometimes tables in the mySQL database crash after a server restart or after a lost connection to the database.
This have happen several times for the last 3-4 weeks.
This is how the error message looks like:
145 - Table './xxx/sessions' is marked as crashed and should be
repaired select value from sessions where sesskey =
'60fa1fab3a3d285cfb7bc1c93bb85f64' and expiry > '1395226673'
[TEP STOP]
So far it’s have been the tables "sessions" and "whos_online" that have crashed. If I repair the tables in phpmyadmin it will work fine again
After the last crash I changed "sessions" from MyISAM to InnoDB. The table "whos_online" still use MyISAM.
I use osCommerce 2.2 rc2a and I'm looking for any thoughts and suggestions in this matter.
One solution might be to change both these tables to InnoDB, since it supposed to be self-healing. Is that a good or bad idea?
Another one would be to have them in MyISAM and do something like this with the php-file that echo the error message:
if $error contain "is marked as crashed and should be repaired"
run a table repair script
Would that be a good or bad idea?
Here’s my server specs
Database: MySQL 5.5.36-cll
PHP Version: 5.3.15
At first you should address the issue that tables become corrupted in the first place. InnoDB tries to repair broken files, but it cannot do magic. Repeated unsafe shutdowns or other accidents may seriously corrupt your database!
If you only have a small website, all variants are good. MyISAM is a little faster (sometimes), while InnoDB provides some extended features. If the server is strong enough, you will likely encounter no issues. I would stick with InnoDB in most cases as it accounts for consistent data and throws errors if files are broken, unlike MYISAM tables which are used and then sometimes throw errors in production...
But if something severely breaks, it requires more effort to get MySQL with InnoDB back up running if it does not automatically. There exist different InnoDB recovery modes which are only available from the shell of your server and require modifying the config file, starting the server by hand, reading its output and maybe doing some other actions. If you do not have any clue of these issues, you might want to stick with MyISAM instead. The server always starts, and if the table is utterly broken, you might need to import a backup. But this is more easily than editing config files and reading database outputs etc.
We have a website that has a table of tax rates that somehow got corrupted one day.
I tried selecting * from theTable but it literally took hours to yield nothing.
I tried truncating and dropping the table. Produced the same results as the select.
Eventually the actions stopped. So, I ran a
myisamchk -r theTable.MYI
Repaired looks like it went okay, but I tried selecting * from it again and still producing the same results.
Engine is MySQL
It would be easy to reproduce the table, can I just delete theTable.* (frm, MYD, MYI)?
Or if there is a better approach to what I'm doing.
Yes, you can delete the .FRM, .MYD, .MYI files to remove a MyISAM table. This is what happens when you DROP TABLE.
MyISAM tables are infamous for getting corrupted. But the fact that you got repeated problems after recreating the table makes me wonder if you have a failing hard drive. It would be worth testing the health of your hard drive.
You should also switch to use InnoDB instead of MyISAM. InnoDB has a lot of protections against corruption, including page checksums, synchronous disk writes, crash recovery, etc. MyISAM is being slowly phased out and is receiving no fixes from MySQL developers. InnoDB is now the default storage engine as of MySQL 5.5 (circa 2010), and it will continue to be the focus of future development.
Note that with InnoDB, you cannot simply delete files to achieve dropping a table like you could with MyISAM. You must use DROP TABLE so InnoDB synchronizes that drop with its internal data dictionary.
We have an update process which currently takes over an hour and means that our DB is unusable during this period.
If I setup up replication would this solve the problem or would the replicated DB suffer from exactly the same problem that the tables would be locked during the update?
Is it possible to have the replicated DB prioritize reading over updating?
Thanks,
D
I suspect that with replication you're just going to be dupolicating the issue (unless most of the time is spent in CPU and only results in a couple of records being updated).
Without knowing a lot more about the scema, distribution and size of data and the update process its impossible to say how best to resolve the problem - but you might get some mileage out of using innodb instead of C-ISAM and making sure that the update is implemented as a number of discrete steps (e.g. using stored procuedures) rather than a single DML statement.
MySQL gives you the ability to run queries delaye. Example: "INSERT DELAYED INTO...", this will cause the query to only be executed when MYSQL has time to take the query.
Based on your input, it sounds like you are using MyISAM tables, MyISAM only support table-wide locking. That means that a single update will lock the whole database table until the query is completed. InnoDB on the other hand uses row locking, which will not cause SELECT queries to wait(hang) for updates to complete.
So you have the best chances of a better sysadmin life if you change to InnoDB :)
When it comes to replication it is pretty normal to seperate updates and selects to two different MySQL servers, and that does tend to work very well. But if you are using MyISAM tables and does a lot of updates, the locking issue itself will still be there.
So my 2 cents: First get rid of MyISAM, then consider replication or a better scaled MySQL server if the problem still exists. (The key for good performance in MySQL is to have at least the size of all indexes across all databases as physical RAM)
One of my projects use the MyISAM engine in MySQL, but I'm considering changing it to InnoDB as I need transaction support here and there.
What should I look at or consider before doing this?
Can I just change the engine, or should the data be prepared for it?
Yes absolutely, there are many things, you should test your application extremely thoroughly:
Transactions can deadlock and need to be repeated. This is the case (in some circumstances) even with an autocommitted transaction which only inserts one row.
Disc usage will almost certainly increase
I/O load during writes will almost certainly increase
Behaviour of indexing will change because InnoDB uses clustered indexes - this may be a beneficial effect in some cases
Your backup strategy will be impacted. Consider this carefully.
The migration process itself will need to be carefully planned, as it will take a long time if you have a lot of data (during which time the data will be either readonly, or completely unavailable - do check!)
There is one big caveat. If you get any kind of hardware failure (or similar) during a write, InnoDB will corrupt tables.
MyISAM will also, but a mysqlcheck --auto-repair will repair them. Trying this with InnoDB tables will fail. Yes, this is from experience.
This means you need to have a good regular data backup plan to use InnoDB.
Some other notes:
InnoDB does not reallocate free space on the filesystem after you drop a table/database or delete a record, this can be solved by "dumping and importing" or setting innodb_file_per_table=1 in my.cnf.
Adding/removing indexes on a large InnoDB table can be quite painfull, because it locks the current table, creates a temporary one with your altered indexes and inserts data - row by row. There is a plugin from Innobase, but it works only for MySQL 5.1
InnoDB is also MUCH MORE memory intense, I suggest you to have as large innodb_buffer_pool_size variable as your server memory allows (70-80% should be a safe bet). If your server is UNIX/Linux, consider reducing sysctl variable vm.swappiness to 0 and use innodb_flush_method=O_DIRECT to avoid double buffering. Always test if you hit swap when toggling those values.You can always read more at Percona blog, which is great.
Also, you can run mysqlbackup with --single-transaction --skip-lock-tables and have no table locks while the backup is commencing.
In any case, InnoDB is great, do not let some pitfalls discourage you.
Just altering the table and setting the engine should be fine.
One of the big ones to watch out for is that select count(*) from MyTable is much slower in InnoDB than MyISAM.
auto_increment values will reset to the highest value in the table +1 after a server restart -- this can cause funny problems if you have a messy db with some deletes.
Optimum server settings are going to be different to a mainly MyISAM db.
Make sure the size of the innodb file is big enough to hold all your data or you'll be crucified by constant reallocation when you change the engines of the tables.
If you are intending to use InnoDB as a way to get concurrent queries, then you will want to set innodb_file_trx_commit=1 so you get some performance back. OTOH, if you were looking to re-code your application to be transaction aware, then deciding this setting will be part of the general performance review needed of the InnoDB settings.
The other major thing to watch out for is that InnoDB does not support FullText indices, nor INSERT DELAYED. But then, MyISAM doesn't support referential integrity. :-)
However, you can move over only the tables you need transaction aware. I've done this. Small tables (up to several thousand rows) can often be changed on-the-fly, incidentally.
The performance characteristics can be different, so you may need to keep an eye on the load.
The data will be fine.