The server runs php 5.5.38 with apache2.4 and MySQL 5.5.62 there are about 5k-10k read write per minute.
And the table of main use was MyISAM and started crashing, saying "in use" instead of stats. The attempts of repair failed. And finally changed it to InnoDB and stopped but what can I do to prevent the errors?
MyISAM is obsolete technology. InnoDB is ACID compliant and designed for crash recovery. This is the solution.
For example, the default engine in MySQL 8.0 is InnoDB: https://dev.mysql.com/doc/refman/8.0/en/storage-engine-setting.html
When you omit the ENGINE option, the default storage engine is used.
The default engine is InnoDB in MySQL 8.0.
You should also implement some caching at the PHP level with Redis/Memcached/Files to prevent unnecessary reads.
Make sure that you have indexes on the columns used in join/where clauses.
Indexes are used to find rows with specific column values quickly.
Without an index, MySQL must begin with the first row and then read
through the entire table to find the relevant rows. The larger the
table, the more this costs
https://dev.mysql.com/doc/refman/8.0/en/mysql-indexes.html
Related
It is my understanding that MySQL creates an execution plan from a SQL query, and then uses innodb (or any other storage engine) to execute the plan. If this is the case, then why does the innodb storage engine have its own parser, server main program, and user-session modules? It looks as if InnoDB could run on its own as a fully functional DBMS.
InnoDB began as an independent company in 1995. The founder wanted to create a standalone RDBMS server.
It wasn't until 2000 that InnoDB began working closely with MySQL, and by March 2001 they announced the InnoDB Table Handler, which allowed MySQL to delegate work to the storage engine.
But InnoDB wanted to support some features that MySQL did not support:
FOREIGN KEY constraints
Proprietary table options
Transactions
MySQL wanted to allow InnoDB and other storage engines to implement their own features too. So they allowed the storage engine layer to perform their own SQL parsing. There are a number of features (like CHECK constraints) that are validated for syntax by the MySQL storage-independent layer, without implementing the semantics. It's up to the storage engine to perform extra parsing and implement those features.
There have also been cases where the InnoDB storage engine wanted to implement features that had no SQL support at the higher level.
For example, the InnoDB monitor, to output periodic troubleshooting data to the server's error log, could be enabled not by sensible syntax like SET ENGINE INNODB MONITOR=ON or something like that, but by creating a table with a special name:
CREATE TABLE innodb_monitor (a INT) ENGINE=INNODB;
It doesn't matter which schema you create this table in, nor what columns you put in it. It doesn't need any rows of data. The name itself is special to InnoDB, and it's a signal to start logging monitor data to the log. Just so they didn't have to implement a new configuration option or SQL syntax!
In later versions of MySQL, you can enable the monitor in a less hacky way with SET GLOBAL innodb_status_output=ON.
I saw so many questions and answers on this topic MyISAM vs InnoDB on stackoverflow itself.
But, all of the questions and answers are too old and not related to the current stable version of MySQL 5.7.x
By the time so much development must have been done in both MyISAM and InnoDB.
So, I need those differences available presently with version 5.7.x
So, please don't mark my question duplicate and someone please explain the differences these storage engines have currently as well as the differences they have since past.
Also, please explain at what situation which storage engine should be chosen for a table.
Can different tables belonging to the same schema have different storage engines i.e. few tables will have InnoDB and few ones will have MyISAM.
If yes, then how the JOIN queries would get execute between tables with MyISAM and InnoDB?
Is it true that MySQL is going to remove MyISAM storage engine from the future version?
Your assumption that MyISAM has been receiving new development is not correct. MyISAM is not receiving any significant new development. MySQL is clearly moving in the direction of phasing out MyISAM, and using MyISAM is discouraged.
Oracle Corp. has not announced any specific date or version by which they will remove MyISAM. My guess is that MyISAM will never be fully removed, because there are too many sites that wouldn't be able to upgrade, without doing expensive testing to make sure their specific app won't experience any regression issues by converting to InnoDB.
But you might notice that in the MySQL 5.7 manual, the section on MyISAM has been demoted to Alternative Storage Engines, which should be a clue that it's receiving less priority.
In MySQL 5.7, MyISAM is still used for some of the system tables, like mysql.user, mysql.db, etc. But new system tables introduced in 5.6 and 5.7 are InnoDB. All system tables are InnoDB in MySQL 8.0.
MyISAM still does not support any of the properties of ACID. There are no transactions, no consistency features, and no durable writes. See my answer to MyISAM versus InnoDB.
MyISAM still does not support foreign keys, for what it's worth. But I seldom see real production sites using foreign keys even with InnoDB.
MyISAM supports only table-level locking (except for some INSERT appending to the end of a table, as noted in the manual).
MySQL 5.7 supports both fulltext indexes and spatial indexes in both MyISAM and InnoDB. These features are not reasons to continue using MyISAM as they once were.
Both logical backup tools like mysqldump and physical backup tools like Percona XtraBackup can't back up MyISAM tables without acquiring a global lock.
You asked if you could create a variety of tables with different storage engines in the same schema. Yes, you can, and this is the same as it has been for many versions of MySQL.
You asked if you can join tables of different storage engines (by the way, tables don't need to be in the same schema to be joined). Yes, you can join such tables, MySQL takes care of all the details. This is the same as it has been for many versions of MySQL.
But some weird cases can come up when you do this, like what if you update a MyISAM table and an InnoDB table in a transaction, and then roll back? The changes in the InnoDB table are rolled back, but the changes in the MyISAM table are not rolled back, so your data integrity can be broken if you aren't careful. This is also the same as it has been for many versions of MySQL.
Cases where MyISAM has an advantage over InnoDB is a short list, and it's getting shorter.
Some table-scan queries and bulk inserts are faster in MyISAM. InnoDB is better at indexed searches.
MyISAM may use less storage space than the equivalent data stored in an uncompressed InnoDB table. You can further compact MyISAM tables with myisampack, but this makes the MyISAM table read-only.
There are other options these days for compact storage of data in transactional storage engines, for example InnoDB table compression, or MyRocks.
SELECT COUNT(*) FROM MyTable queries (with no WHERE clause) are very fast in MyISAM, because the accurate count of rows is persisted in the MyISAM metadata. InnoDB (or other MVCC implementations) doesn't keep this count persisted, because every transaction viewing the table might "see" a different row count. Only a storage engine that has table-level locking and no transaction isolation like MyISAM, can optimize this case.
Auto-increment that numbers independently for each distinct value in another key column. Again, this requires table-level locking, so it's not supported in InnoDB.
CREATE TABLE MyTable (
group_id INT NOT NULL,
seq_id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (group_id, seq_id)
) ENGINE=MyISAM;
It's still easy to move a MyISAM table from server to server, because the .MYD and .MYI files are self-contained. You can kind of do something similar with InnoDB tables, but you have to use the intricate feature of transportable tablespaces. But this easy-to-move-tables quality of MyISAM no longer works in MySQL 8.0, because of their new data dictionary feature.
Under certain load, MyISAM might be a better choice for internal_tmp_disk_storage_engine, which defaults to InnoDB in MySQL 5.7. If you run lots of queries that create temp tables on disk (in-memory temp tables won't benefit), it can put a strain on the InnoDB engine. But you'd have to have a high query rate for this to matter, and if your queries create so many temp tables on disk, you should try to optimize the queries differently.
MyISAM allows you to set multiple key caches, and define caches for specific tables. But the MyISAM key caches are only for index structures, not for data.
References:
https://www.percona.com/blog/2016/10/11/mysql-8-0-end-myisam/
https://www.percona.com/blog/2017/12/04/internal-temporary-tables-mysql-5-7/
http://jfg-mysql.blogspot.com/2017/08/why-we-still-need-myisam.html
I had this question for a job quiz and got it right: (referring the new version):
MyISAM and InnoDB are two different storage engins that handle CRUD operations differently.
Locking: When approching a row inside a MyISAM storage engin, all the table will be locked by other sessions until the change is commited, unlike InnoDB, which locks only the specific selected row(/s). The lock is released until the session is commited. Locking a table or a row causes suspention by other sessions that try to interact with the same table or row to prevent wrong data manipulations in the table for example.
Transactions: InnoDB supports transactions, unlike MyISAM. Transactions are a colection of 2 or more commands like SELECT, INSERT, UPDATE and DELETE, to a single operation until complishion.
Atomic Operations: When setting a transaction in an InnoDB and
the operation is incompleted - it terminates all the changes and
restore the DB as it was (all or nothin'), so for example, if in the
middle of a transaction there is a syntax error in the code /
datatype mismatch or anything that might interupt the bundle of
commands to finish its operation - all the changes wont be applied,
thanks transactions atomicy. On the other hand, when using an
MyISAM storage engin, if a bundle of commands "breaks" (for any
reason), the operation stops immediately and all the
tables/rows/data that were affected will remain affected, which
might cause a corrupt data in the database (...and a headache).
B. Running an operation on MyISAM are set on the spot,
whereas InnoDB allows you to use the "ROLLBACK"s to discard any
change, which comes best in handy when running transactions.
Transaction Logs: When creating a transaction without a
transaction log in between, you can apply any changes on the table/s
in the DB, and if the table have a clustered index (for example),
the data will have to search where exactly it has to be inserted and
only then apply the change. In a case where there is a transaction
log in between the DB and the transaction, the changes will be sent
to the transaction log first and will set its order in the table
before sending the change to the DB - which will be less time
consuming. The DB saves logs from all the transactions that were
made, which can help to choose to restore any transaction previously
made, and recover all changes. When set to a "simple" recovery model- transactions are deleted from the transactions log and wont be able to recover data (used usually on DEV environments). When set to
"full" recovery model, all transactions are saved and listed, ready
to be restored - this is used usually on production environments
which might cause problems like preformance issues - so backing them
up and deleting from the server could be a solution. When set to a
"bulk-logged" recovery model saved transaction logs only for
specific "important" changes and commands (import,export,
insert-select, select-into, reorganaizing/rebuilding indexes), and
might prevent preformance issues.
Foreign keys: MyISAM dosn't use foreign keys, unlike InnoDB. When a table column has a foregin key set to point on an other table column, when any update/delete occures on the pointed table, it will know that the changes have to be applied on the other table pointing at it. This create a some kind of a link between the two table and keep data in sync. Setting tables with FKs might require more effort which might be considered as a disadvantage (?).
FULLTEXT indexing: InnoDB doesn't support FULLTEXT indexing in its previous versions - MyISAM does support it. Switching to MyISAM wont be the best solution so just update MySQL to a verion which does support FULLTEXT indexing.
FULLTEXT indexing can take texts like titles, comments, ect' - and search it (this should be a better option than the "LIKE" command in this case).
Spatial data types: Supported only on InnoDB.
To sum all up, InnoDB will be usually more reliable in terms of data handling, validity & recovery. For newer versions InnoDB will support FULLTEXT indexing for mainly searches - when using older versions with no option to update MySQL, using MyISAM will be great.
I am creating an commerce website and i am stuck in a database problem i am storing customer orders please tell me which is better MYISAM or Innodb i have to use transaction like feature in customer order table and i personally prefer myisam because it is much faster than innodb and it also supports full-text searching is there any way to use transaction like rollback feature in myisam so that if anything goes wrong table will be rollback to its previous state how to do that without any external library or any other server side access and i have to use MYSQL
I would use innodb because it supports transactions. It is not slower than myisam on selects, and supports fulltext searches from mysql v5.6. It is obviously slower on modifiations, since it uses transactions.
Myisam does not support transactions and I don't think it ever will. You have innodb for that purpose.
I have MySQL running on my machine configured with MyISAM as its default tables.
Now I want to ask few of questions:
1) If I change the default table to InnoDB in the configuration file (my.conf), clear the log file and restart mysql, would that harm any of my previous database or tables?
2) If I alter few tables' engine to InnoDB using the following command, would that affect its data at all?
ALTER TABLE table_name ENGINE = InnoDB;
3) Is it a good idea to keep few tables as MyISAM (for read and write) and the rest as InnoDB (more for selecting data) or is it preferred to select one engine for all the tables in the database?
2) It will only affect the internal representation. Nothing that you will notice on the outside.
3) It is a perfectly good idea, if it enhances performance.
2) You can mix database types. i.e. innoDB and MyISAM.
3) innoDB supposedly keeps data safer. I think it is the default on latest versions of mySQL.
What is InnoDB and MyISAM in MySQL ?
InnoDB and MYISAM, are storage engines for MySQL.
These two differ on their locking implementation: InnoDB locks the particular row in the table, and MyISAM locks the entire MySQL table.
You can specify the type by giving MYISAM OR InnoDB while creating a table in DB.
Have a look at
InnoDB and MyISAM
InnoDB is a storage engine for MySQL,
included as standard in all current
binaries distributed by MySQL AB. Its
main enhancement over other storage
engines available for use with MySQL
is ACID-compliant transaction support
MyISAM is the default storage engine
for the MySQL relational database
management system versions prior to
5.5 1. It is based on the older ISAM code but has many useful extensions.
The major deficiency of MyISAM is the absence of transactions support.
Versions of MySQL 5.5 and greater have
switched to the InnoDB engine to
ensure referential integrity
constraints, and higher concurrency.
They are storage engines.
http://dev.mysql.com/doc/refman/5.1/en/storage-engines.html
MyISAM: The default MySQL storage engine and the one that is used the most in Web, data warehousing, and other application environments. MyISAM is supported in all MySQL configurations, and is the default storage engine unless you have configured MySQL to use a different one by default.
InnoDB: A transaction-safe (ACID compliant) storage engine for MySQL that has commit, rollback, and crash-recovery capabilities to protect user data. InnoDB row-level locking (without escalation to coarser granularity locks) and Oracle-style consistent nonlocking reads increase multi-user concurrency and performance. InnoDB stores user data in clustered indexes to reduce I/O for common queries based on primary keys. To maintain data integrity, InnoDB also supports FOREIGN KEY referential-integrity constraints.
InnoDB is a transactional storage engine of MySQL whereas MyISAM is a non-transactional storage engine. In other words, InnoDB follows the ACID properties to maintain the integrity of data but MyISAM doesn't follow ACID properties thus failing to maintain the integrity of the data.
In an InnoDB (transactional) table, the transactional changes can be
easily undone if a rollback is required. But changes made to a MyISAM
(non-transactional) table cannot be undone when rolling back a
transaction is required.
For example, you want to transfer money from your checking account to saving account. This is done by a transaction which includes 5 queries.
1 START TRANSACTION;
2 SELECT balance FROM checking WHERE customer_id = 10233276;
3 UPDATE checking SET balance = balance - 200.00 WHERE customer_id = 10233276;
4 UPDATE savings SET balance = balance + 200.00 WHERE customer_id = 10233276;
5 COMMIT;
Suppose, the process crashes at step 4. If a InnoDB table was used here, a rollback would undo the changes and you are saved from the risk of losing money. Literally, the table is unaware of any crash as the changes will not be commited to the table unless step 5 is successfully executed.
But in the case of a MyISAM table, one cannot undo the transactional changes when a rollback is called or if there is a crash leading to the failure of the transaction. This means, if the transaction crashed at step 3, money will be deducted from your checking account. But money wouldnot have been added to your savings account.
Example courtesy: "High Performance MySQL: Optimization, Backups, and Replication" -
Book by Arjen Lentz, Derek J. Balling, Jeremy Zawodny, Peter Zaitsev, and Vadim Tkachenko
I wanted to add that having ability to specify a specific storage engine per table is one of the key strengths of MySQL (besides easy of use and good performance with no tweaking). For all operations where transactions are needed, just stick with InnoDB. However, MyISAM can really speed things up when transactions are not needed in certain situations - and requires less disk space and RAM compared to InnoDB.
That said, InnoDB is getting better all the time:
InnoDB 1.1 Performance and Scalability Enhancements
MyISAM does not follow ACID as opposed to InnoDB which follows transactions to maintain integrity of the data.
MyISAM supports concurrent inserts: If a table has no free blocks in
the middle of the data file, you can INSERT new rows into it at the
same time that other threads are reading from the table. MySqlDoc
That is why, MyISAM is faster and takes less space. For instance, the MySQL MyISAM Storage Engine does not support tranactions.constraints of MySQL MYISAM There is a bit called concurrent-insert
By default, the variable is set to 1 and concurrent inserts are handled as just described. If it is set to 0, concurrent inserts are disabled. If it is set to 2, concurrent inserts at the end of the table are permitted even for tables that have deleted rows. An INSERT statement can be executed to add rows to the end of the table with select at same time if there are no holes/deleted rows in middle of table (at time of concurrent insert).
The default isolation level og mysql InnoDB is "Read Repeatable". For MyISAM, there is no transaction. InnoDB uses row level locking while MyISAM can only use table level locking that is why InnoDB has crash revovery is better than MyISAM. One has to manually acquire the table level lock in MyISAM if one wants to avoid the concurrency effects.
When your MySQL server crashes, the data can be recovered much easier from a set of MyISAM tables than from that big InnoDB transaction file. Each MyISAM table has a separate file, and if no write operations were being made to this table during the crash - it will be totally unaffected. In case of InnoDB, the entire transaction file of the entire MySQL server has to be re-indexed or whatever it does after a crash. That can get quite messy.
InnoDB is the default NOT myISAM
https://dev.mysql.com/doc/refman/5.7/en/innodb-introduction.html
"InnoDB is the default MySQL storage engine. Unless you have configured a different default storage engine, issuing a CREATE TABLE statement without an ENGINE= clause creates an InnoDB table"