The server runs php 5.5.38 with apache2.4 and MySQL 5.5.62 there are about 5k-10k read write per minute.
And the table of main use was MyISAM and started crashing, saying "in use" instead of stats. The attempts of repair failed. And finally changed it to InnoDB and stopped but what can I do to prevent the errors?
MyISAM is obsolete technology. InnoDB is ACID compliant and designed for crash recovery. This is the solution.
For example, the default engine in MySQL 8.0 is InnoDB: https://dev.mysql.com/doc/refman/8.0/en/storage-engine-setting.html
When you omit the ENGINE option, the default storage engine is used.
The default engine is InnoDB in MySQL 8.0.
You should also implement some caching at the PHP level with Redis/Memcached/Files to prevent unnecessary reads.
Make sure that you have indexes on the columns used in join/where clauses.
Indexes are used to find rows with specific column values quickly.
Without an index, MySQL must begin with the first row and then read
through the entire table to find the relevant rows. The larger the
table, the more this costs
https://dev.mysql.com/doc/refman/8.0/en/mysql-indexes.html
It is my understanding that MySQL creates an execution plan from a SQL query, and then uses innodb (or any other storage engine) to execute the plan. If this is the case, then why does the innodb storage engine have its own parser, server main program, and user-session modules? It looks as if InnoDB could run on its own as a fully functional DBMS.
InnoDB began as an independent company in 1995. The founder wanted to create a standalone RDBMS server.
It wasn't until 2000 that InnoDB began working closely with MySQL, and by March 2001 they announced the InnoDB Table Handler, which allowed MySQL to delegate work to the storage engine.
But InnoDB wanted to support some features that MySQL did not support:
FOREIGN KEY constraints
Proprietary table options
Transactions
MySQL wanted to allow InnoDB and other storage engines to implement their own features too. So they allowed the storage engine layer to perform their own SQL parsing. There are a number of features (like CHECK constraints) that are validated for syntax by the MySQL storage-independent layer, without implementing the semantics. It's up to the storage engine to perform extra parsing and implement those features.
There have also been cases where the InnoDB storage engine wanted to implement features that had no SQL support at the higher level.
For example, the InnoDB monitor, to output periodic troubleshooting data to the server's error log, could be enabled not by sensible syntax like SET ENGINE INNODB MONITOR=ON or something like that, but by creating a table with a special name:
CREATE TABLE innodb_monitor (a INT) ENGINE=INNODB;
It doesn't matter which schema you create this table in, nor what columns you put in it. It doesn't need any rows of data. The name itself is special to InnoDB, and it's a signal to start logging monitor data to the log. Just so they didn't have to implement a new configuration option or SQL syntax!
In later versions of MySQL, you can enable the monitor in a less hacky way with SET GLOBAL innodb_status_output=ON.
I need to create an index on a large InnoDB production table and want to do this without locking the table in any way. I am using MySQL 5.6 (.38-83.90).
I tried
create index my_index on my_table(col1, col2);
Neither columns are primary keys. col1 is a foreign key.
Well, this totally locked the table. Other queries were stalled with "Waiting for table metadata lock" bringing my website to its knees. I had to kill the create index query.
From this https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html I thought that it would not lock the table: "... no syntax changes are required... The table remains available for read and write operations while the index is being created or dropped."
I see that I can set LOCK=NONE or LOCK=SHARED, but I don't see that it should be necessary or, if it is, which one I need to use.
"You can specify LOCK=NONE to assert that concurrent DML is permitted during the DDL operation. MySQL automatically permits concurrent DML when possible."
"You can specify LOCK=SHARED to assert that concurrent queries are permitted during a DDL operation. MySQL automatically permits concurrent queries when possible."
None of the limitations https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-limitations.html seem to apply to my case.
What am I missing?
My guess (just a guess) is that you are missing the ALGORITHM=INPLACE clause on the CREATE INDEX statement.
CREATE INDEX my_index ON my_table(col1, col2) ALGORITHM=INPLACE ;
^^^^^^^^^^^^^^^^^
Also be aware of transactions acquiring and holding metadata locks.
https://dev.mysql.com/doc/refman/5.6/en/metadata-locking.html
Any transaction that has referenced my_table will continue to hold a metadata lock on that table until the transaction is committed or rolled back. I suggest checking the TRANSACTIONS section of SHOW ENGINE INNODB STATUS output.
I saw so many questions and answers on this topic MyISAM vs InnoDB on stackoverflow itself.
But, all of the questions and answers are too old and not related to the current stable version of MySQL 5.7.x
By the time so much development must have been done in both MyISAM and InnoDB.
So, I need those differences available presently with version 5.7.x
So, please don't mark my question duplicate and someone please explain the differences these storage engines have currently as well as the differences they have since past.
Also, please explain at what situation which storage engine should be chosen for a table.
Can different tables belonging to the same schema have different storage engines i.e. few tables will have InnoDB and few ones will have MyISAM.
If yes, then how the JOIN queries would get execute between tables with MyISAM and InnoDB?
Is it true that MySQL is going to remove MyISAM storage engine from the future version?
Your assumption that MyISAM has been receiving new development is not correct. MyISAM is not receiving any significant new development. MySQL is clearly moving in the direction of phasing out MyISAM, and using MyISAM is discouraged.
Oracle Corp. has not announced any specific date or version by which they will remove MyISAM. My guess is that MyISAM will never be fully removed, because there are too many sites that wouldn't be able to upgrade, without doing expensive testing to make sure their specific app won't experience any regression issues by converting to InnoDB.
But you might notice that in the MySQL 5.7 manual, the section on MyISAM has been demoted to Alternative Storage Engines, which should be a clue that it's receiving less priority.
In MySQL 5.7, MyISAM is still used for some of the system tables, like mysql.user, mysql.db, etc. But new system tables introduced in 5.6 and 5.7 are InnoDB. All system tables are InnoDB in MySQL 8.0.
MyISAM still does not support any of the properties of ACID. There are no transactions, no consistency features, and no durable writes. See my answer to MyISAM versus InnoDB.
MyISAM still does not support foreign keys, for what it's worth. But I seldom see real production sites using foreign keys even with InnoDB.
MyISAM supports only table-level locking (except for some INSERT appending to the end of a table, as noted in the manual).
MySQL 5.7 supports both fulltext indexes and spatial indexes in both MyISAM and InnoDB. These features are not reasons to continue using MyISAM as they once were.
Both logical backup tools like mysqldump and physical backup tools like Percona XtraBackup can't back up MyISAM tables without acquiring a global lock.
You asked if you could create a variety of tables with different storage engines in the same schema. Yes, you can, and this is the same as it has been for many versions of MySQL.
You asked if you can join tables of different storage engines (by the way, tables don't need to be in the same schema to be joined). Yes, you can join such tables, MySQL takes care of all the details. This is the same as it has been for many versions of MySQL.
But some weird cases can come up when you do this, like what if you update a MyISAM table and an InnoDB table in a transaction, and then roll back? The changes in the InnoDB table are rolled back, but the changes in the MyISAM table are not rolled back, so your data integrity can be broken if you aren't careful. This is also the same as it has been for many versions of MySQL.
Cases where MyISAM has an advantage over InnoDB is a short list, and it's getting shorter.
Some table-scan queries and bulk inserts are faster in MyISAM. InnoDB is better at indexed searches.
MyISAM may use less storage space than the equivalent data stored in an uncompressed InnoDB table. You can further compact MyISAM tables with myisampack, but this makes the MyISAM table read-only.
There are other options these days for compact storage of data in transactional storage engines, for example InnoDB table compression, or MyRocks.
SELECT COUNT(*) FROM MyTable queries (with no WHERE clause) are very fast in MyISAM, because the accurate count of rows is persisted in the MyISAM metadata. InnoDB (or other MVCC implementations) doesn't keep this count persisted, because every transaction viewing the table might "see" a different row count. Only a storage engine that has table-level locking and no transaction isolation like MyISAM, can optimize this case.
Auto-increment that numbers independently for each distinct value in another key column. Again, this requires table-level locking, so it's not supported in InnoDB.
CREATE TABLE MyTable (
group_id INT NOT NULL,
seq_id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (group_id, seq_id)
) ENGINE=MyISAM;
It's still easy to move a MyISAM table from server to server, because the .MYD and .MYI files are self-contained. You can kind of do something similar with InnoDB tables, but you have to use the intricate feature of transportable tablespaces. But this easy-to-move-tables quality of MyISAM no longer works in MySQL 8.0, because of their new data dictionary feature.
Under certain load, MyISAM might be a better choice for internal_tmp_disk_storage_engine, which defaults to InnoDB in MySQL 5.7. If you run lots of queries that create temp tables on disk (in-memory temp tables won't benefit), it can put a strain on the InnoDB engine. But you'd have to have a high query rate for this to matter, and if your queries create so many temp tables on disk, you should try to optimize the queries differently.
MyISAM allows you to set multiple key caches, and define caches for specific tables. But the MyISAM key caches are only for index structures, not for data.
References:
https://www.percona.com/blog/2016/10/11/mysql-8-0-end-myisam/
https://www.percona.com/blog/2017/12/04/internal-temporary-tables-mysql-5-7/
http://jfg-mysql.blogspot.com/2017/08/why-we-still-need-myisam.html
I had this question for a job quiz and got it right: (referring the new version):
MyISAM and InnoDB are two different storage engins that handle CRUD operations differently.
Locking: When approching a row inside a MyISAM storage engin, all the table will be locked by other sessions until the change is commited, unlike InnoDB, which locks only the specific selected row(/s). The lock is released until the session is commited. Locking a table or a row causes suspention by other sessions that try to interact with the same table or row to prevent wrong data manipulations in the table for example.
Transactions: InnoDB supports transactions, unlike MyISAM. Transactions are a colection of 2 or more commands like SELECT, INSERT, UPDATE and DELETE, to a single operation until complishion.
Atomic Operations: When setting a transaction in an InnoDB and
the operation is incompleted - it terminates all the changes and
restore the DB as it was (all or nothin'), so for example, if in the
middle of a transaction there is a syntax error in the code /
datatype mismatch or anything that might interupt the bundle of
commands to finish its operation - all the changes wont be applied,
thanks transactions atomicy. On the other hand, when using an
MyISAM storage engin, if a bundle of commands "breaks" (for any
reason), the operation stops immediately and all the
tables/rows/data that were affected will remain affected, which
might cause a corrupt data in the database (...and a headache).
B. Running an operation on MyISAM are set on the spot,
whereas InnoDB allows you to use the "ROLLBACK"s to discard any
change, which comes best in handy when running transactions.
Transaction Logs: When creating a transaction without a
transaction log in between, you can apply any changes on the table/s
in the DB, and if the table have a clustered index (for example),
the data will have to search where exactly it has to be inserted and
only then apply the change. In a case where there is a transaction
log in between the DB and the transaction, the changes will be sent
to the transaction log first and will set its order in the table
before sending the change to the DB - which will be less time
consuming. The DB saves logs from all the transactions that were
made, which can help to choose to restore any transaction previously
made, and recover all changes. When set to a "simple" recovery model- transactions are deleted from the transactions log and wont be able to recover data (used usually on DEV environments). When set to
"full" recovery model, all transactions are saved and listed, ready
to be restored - this is used usually on production environments
which might cause problems like preformance issues - so backing them
up and deleting from the server could be a solution. When set to a
"bulk-logged" recovery model saved transaction logs only for
specific "important" changes and commands (import,export,
insert-select, select-into, reorganaizing/rebuilding indexes), and
might prevent preformance issues.
Foreign keys: MyISAM dosn't use foreign keys, unlike InnoDB. When a table column has a foregin key set to point on an other table column, when any update/delete occures on the pointed table, it will know that the changes have to be applied on the other table pointing at it. This create a some kind of a link between the two table and keep data in sync. Setting tables with FKs might require more effort which might be considered as a disadvantage (?).
FULLTEXT indexing: InnoDB doesn't support FULLTEXT indexing in its previous versions - MyISAM does support it. Switching to MyISAM wont be the best solution so just update MySQL to a verion which does support FULLTEXT indexing.
FULLTEXT indexing can take texts like titles, comments, ect' - and search it (this should be a better option than the "LIKE" command in this case).
Spatial data types: Supported only on InnoDB.
To sum all up, InnoDB will be usually more reliable in terms of data handling, validity & recovery. For newer versions InnoDB will support FULLTEXT indexing for mainly searches - when using older versions with no option to update MySQL, using MyISAM will be great.
I have MySQL running on my machine configured with MyISAM as its default tables.
Now I want to ask few of questions:
1) If I change the default table to InnoDB in the configuration file (my.conf), clear the log file and restart mysql, would that harm any of my previous database or tables?
2) If I alter few tables' engine to InnoDB using the following command, would that affect its data at all?
ALTER TABLE table_name ENGINE = InnoDB;
3) Is it a good idea to keep few tables as MyISAM (for read and write) and the rest as InnoDB (more for selecting data) or is it preferred to select one engine for all the tables in the database?
2) It will only affect the internal representation. Nothing that you will notice on the outside.
3) It is a perfectly good idea, if it enhances performance.
2) You can mix database types. i.e. innoDB and MyISAM.
3) innoDB supposedly keeps data safer. I think it is the default on latest versions of mySQL.