everyone! I'm trying to avoid breaking of my database in the phpbb3.1 forum. It was crushed twice this month.
So I have two questions:
1) Is it safe to convert MyISAM to InnoDB? I mean will extensions work fine? Will forum be workable after updating to next version?
2) In which way I can avoid base corrupting?
p.s.
I also posted this question here:
https://www.phpbb.com/community/viewtopic.php?f=466&t=2436326
I'll venture a guess. You had a power failure, and when it came back up, MySQL was complaining that some index on some table was corrupted? And that table was MyISAM?
Use myisamchk to repair the tables.
Review the gotchas in http://mysql.rjweb.org/doc.php/myisam2innodb to see if conversion to InnoDB will add new woes. There probably won't be any. A 2-part PRIMARY KEY is about the only thing that is not implemented in InnoDB. Also, if you have too old a version of MySQL, InnoDB may not yet have FULLTEXT indexes (if you need them).
Change my.cnf: key_buffer_size = 20M and innodb_buffer_pool_size equal to about half of available memory.
ALTER TABLE xx ENGINE=InnoDB; for each table xx.
I think (but am not sure) that each update/delete/insert marks the table as possibly corrupt. It writes the changes, but does not clear the mark. When mysqld shuts down cleanly, everything is flushed to disk and these flags are cleared. When mysqld comes back up, it complains about the flags that did not get cleared. So...
Whether or not an index is marked as corrupt depends solely on whether you modified that index and crashed. (Every table has some index, yes?)
Normally, MySQL manages to flush changes to disk before a crash. Only occasionally does the crash happen at a time where the index will really be corrupt. There is a "quick" mode on the repair that simply clears the flag -- you could try that. But if you ever get a mysterious "can't find record" when you know the records exists, you'd better REPAIR it.
Related
I have a MySQL table named requests. It's growing so fast. Currently, it has 3 million rows. Also, the current table engine is "InnoDB".
Few days ago, I got this error:
ERROR 1114 (HY000): The table is full
I've resolved the problem (temporary I guess) by adding this to MySQL configuration:
innodb_data_file_path = ibdata1:10M:autoextend
But still sometimes my serves goes to down and I have to restart it to make it available. Recently, I've made that table empty and I got rid of being down.
But it will be full again soon. Any idea how can I fix that issue? Should I use another engine for that table? Should I use compressed type-row for it? or what?
Noted that, sometimes I need to select from that table and it has one index on a column (in addition to pk)
You are limited by disk space. You must keep an eye on that and take action before you get to "table full".
If you store the data in a MyISAM table, you can go twice as long before getting "table full". If you simply write to a disk file (no database, no queries; etc), you can squeeze a little more in before "disk full".
Do you need to "purge" old data? That way you could continue to receive new data without ever hitting "table full". The best way to do that is via InnoDB and PARTITION BY RANGE(TO_DAYS(...)). If you purge anything over a month old, use daily partitions. For, say, 90 days, use weekly partitions. More discussion: http://mysql.rjweb.org/doc.php/partitionmaint
What will you do with the data? Analyze it? Search it? What SQL queries do you envision? Answer those; there could be other tips.
I did "CHECK TABLE mytable" and there were two rows with Msg_text:
"Table is marked as crashed"
"OK"
I took about 10 mins to review how 'myisamchk' works, then ran CHECK TABLE again to verify the error, but to my surprise the error was no longer there.
I reviewed CHECK TABLE documentation and see sentence ..
"For MyISAM tables, the key statistics are updated as well."
The table is "MyISAM".
Is it possible that ...
- the NON-updated stats were causing the "...crashed" error
- and that updating those stats via CHECK TABLE resolved the error?
Thanks
It works something like this...
While mysqld is running, it has a flag set in any index (.MYI file) that it is using. Normally index updates are written to the .MYI file, but with no sense of urgency, since they are cached in the "key_buffer".
When mysqld terminates un-gracefully (eg, because of a power failure), mysqld has no chance to clear the flag. After restarting, mysqld sees the flag and says "oh, panic; this index may be corrupted". Then, you have to come in and take action to clear the flag and, if necessary, fix the corruption using REPAIR TABLE or the equivalent myisamchk with suitable arguments.
Often, the index is not really corrupt, and the CHECK/REPAIR is not really needed. But MySQL can't be sure. Hence, the somewhat alarming message.
It is possible to have a setting so that MyISAM tables will be checked and repaired during startup. I rarely see myisam_recover_options set to anything other than OFF.
If you switch to InnoDB, you can mostly forget about all this stuff.
We have a website that has a table of tax rates that somehow got corrupted one day.
I tried selecting * from theTable but it literally took hours to yield nothing.
I tried truncating and dropping the table. Produced the same results as the select.
Eventually the actions stopped. So, I ran a
myisamchk -r theTable.MYI
Repaired looks like it went okay, but I tried selecting * from it again and still producing the same results.
Engine is MySQL
It would be easy to reproduce the table, can I just delete theTable.* (frm, MYD, MYI)?
Or if there is a better approach to what I'm doing.
Yes, you can delete the .FRM, .MYD, .MYI files to remove a MyISAM table. This is what happens when you DROP TABLE.
MyISAM tables are infamous for getting corrupted. But the fact that you got repeated problems after recreating the table makes me wonder if you have a failing hard drive. It would be worth testing the health of your hard drive.
You should also switch to use InnoDB instead of MyISAM. InnoDB has a lot of protections against corruption, including page checksums, synchronous disk writes, crash recovery, etc. MyISAM is being slowly phased out and is receiving no fixes from MySQL developers. InnoDB is now the default storage engine as of MySQL 5.5 (circa 2010), and it will continue to be the focus of future development.
Note that with InnoDB, you cannot simply delete files to achieve dropping a table like you could with MyISAM. You must use DROP TABLE so InnoDB synchronizes that drop with its internal data dictionary.
sometimes i get an error like "table is marked as corrupt and shld be repaired". that DB (tables) is using MyISAM. recently that keeps happening. what could be the causes? most recently i am executing a batch insert
INSERT INTO table (..., ..., ...) VALUES (...), (...), (...) ...
and it just hung. or took very long to complete it seems hung to me. the next day, when i checked the table was marked as corrupt again. when i try to use mysqlcheck -r it said all tables OK when it reached that "corrupt" table it hung there again...
so, what can i do to prevent this. and what could be the causes. the DB is hosted 3rd party, how can i debug this?
is InnoDB a more reliable engine to use? i heard MyISAM is faster but others say InnoDB can be fast also but it takes abit more to optimize it. can i conclude that InnoDB is something more reliable but abit slower overall even with optimization?
If your tables get corrupt, you can use the repair table command to fix them:
REPAIR TABLE table;
If you run myisamchk while the server is still running (and inserts/selects are hitting the table), it could be what is corrupting your tables. Most of the corruption issues I run into are when trying to do things outside the server (copying the files, etc) while it is still running.
InnoDB is slower for read only databases, because it has features (ACID compliant, row level locking) that MyISAM leaves out. However, if you are doing a mixture of reads and writes, depending on the mixture, then InnoDB can offer serious performance improvements, because it doesn't have to lock the entire table to do a write. You also don't run into corruption issues.
Go with InnoDB.
ok, so the problem was the company's db exceeded the storage space allowed by the hosting company. so apparently noone told the company they exceeded the usage ... lousy host i guess.
btw, theres no way mysql could have known abt this?
One of my projects use the MyISAM engine in MySQL, but I'm considering changing it to InnoDB as I need transaction support here and there.
What should I look at or consider before doing this?
Can I just change the engine, or should the data be prepared for it?
Yes absolutely, there are many things, you should test your application extremely thoroughly:
Transactions can deadlock and need to be repeated. This is the case (in some circumstances) even with an autocommitted transaction which only inserts one row.
Disc usage will almost certainly increase
I/O load during writes will almost certainly increase
Behaviour of indexing will change because InnoDB uses clustered indexes - this may be a beneficial effect in some cases
Your backup strategy will be impacted. Consider this carefully.
The migration process itself will need to be carefully planned, as it will take a long time if you have a lot of data (during which time the data will be either readonly, or completely unavailable - do check!)
There is one big caveat. If you get any kind of hardware failure (or similar) during a write, InnoDB will corrupt tables.
MyISAM will also, but a mysqlcheck --auto-repair will repair them. Trying this with InnoDB tables will fail. Yes, this is from experience.
This means you need to have a good regular data backup plan to use InnoDB.
Some other notes:
InnoDB does not reallocate free space on the filesystem after you drop a table/database or delete a record, this can be solved by "dumping and importing" or setting innodb_file_per_table=1 in my.cnf.
Adding/removing indexes on a large InnoDB table can be quite painfull, because it locks the current table, creates a temporary one with your altered indexes and inserts data - row by row. There is a plugin from Innobase, but it works only for MySQL 5.1
InnoDB is also MUCH MORE memory intense, I suggest you to have as large innodb_buffer_pool_size variable as your server memory allows (70-80% should be a safe bet). If your server is UNIX/Linux, consider reducing sysctl variable vm.swappiness to 0 and use innodb_flush_method=O_DIRECT to avoid double buffering. Always test if you hit swap when toggling those values.You can always read more at Percona blog, which is great.
Also, you can run mysqlbackup with --single-transaction --skip-lock-tables and have no table locks while the backup is commencing.
In any case, InnoDB is great, do not let some pitfalls discourage you.
Just altering the table and setting the engine should be fine.
One of the big ones to watch out for is that select count(*) from MyTable is much slower in InnoDB than MyISAM.
auto_increment values will reset to the highest value in the table +1 after a server restart -- this can cause funny problems if you have a messy db with some deletes.
Optimum server settings are going to be different to a mainly MyISAM db.
Make sure the size of the innodb file is big enough to hold all your data or you'll be crucified by constant reallocation when you change the engines of the tables.
If you are intending to use InnoDB as a way to get concurrent queries, then you will want to set innodb_file_trx_commit=1 so you get some performance back. OTOH, if you were looking to re-code your application to be transaction aware, then deciding this setting will be part of the general performance review needed of the InnoDB settings.
The other major thing to watch out for is that InnoDB does not support FullText indices, nor INSERT DELAYED. But then, MyISAM doesn't support referential integrity. :-)
However, you can move over only the tables you need transaction aware. I've done this. Small tables (up to several thousand rows) can often be changed on-the-fly, incidentally.
The performance characteristics can be different, so you may need to keep an eye on the load.
The data will be fine.