Without any changes on database I got very different rows count on my tables.
What can cause this?
Server version: 5.1.63-cll
Engine: InnoDB
Unlike MyISAM tables, InnoDB tables don't keep track of how many rows the table contains.
Because of this, the only way to know the exact number of rows in an InnoDB table is to inspect each row in the table and accumulate a count. Therefore, on large InnoDB tables, performing a SELECT COUNT(*) FROM innodb_table query can be very slow because it does a full table scan to get the number of rows.
phpMyAdmin uses a query like SHOW TABLE STATUS to get an estimated count of the number of rows in the table from the engine (InnoDB). Since it's just an estimate, it varies each time you call it, sometimes the variations can be fairly large, and sometimes they are close.
Here is an informative blog post about COUNT(*) for InnoDB tables by Percona.
The MySQL manual page for SHOW TABLE STATUS states:
The number of rows. Some storage engines, such as MyISAM, store the
exact count. For other storage engines, such as InnoDB, this value is
an approximation, and may vary from the actual value by as much as 40
to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate
count.
The page on InnoDB restrictions goes into some more detail:
SHOW TABLE STATUS does not give accurate statistics on InnoDB tables, except for the physical size reserved by the table. The row count is only a rough estimate used in SQL optimization.
InnoDB does not keep an internal count of rows in a table because
concurrent transactions might “see” different numbers of rows at the
same time. To process a SELECT COUNT(*) FROM t statement, InnoDB scans
an index of the table, which takes some time if the index is not
entirely in the buffer pool. If your table does not change often,
using the MySQL query cache is a good solution. To get a fast count,
you have to use a counter table you create yourself and let your
application update it according to the inserts and deletes it does. If
an approximate row count is sufficient, SHOW TABLE STATUS can be used.
See Section 14.3.14.1, “InnoDB Performance Tuning Tips”.
phpMyAdmin uses a quick method to get the row count, and this method only returns an approximate count in the case of InnoDB tables.
See $cfg['MaxExactCount'] for a way to modify those results, but this could have a serious impact on performance.
Related
I know that you shouldn't rely on the values returned by InnoDB's SHOW TABLE STATUS.
In particular, the row count and avg data length.
But I thought maybe it was an accurate value taken at some point, and then innodb only refreshes it during an ANALYZE table or maybe some other infrequent event.
Instead what Im seeing is that I can run a SHOW TABLE STATUS on the same table 5 times in 5 seconds, and just get completely different numbers each time (despite the table not having any insert/delete activity in between)
Where are these values actually coming from? Are they just corrupt in innodb?
The official MySQL 5.1 documentation acknowledges that InnoDB does not give accurate statistics with SHOW TABLE STATUS. Whereas MYISAM tables specifically keep an internal cache of meta-data such as number of rows etc, the InnoDB engine stores both table data and indexes in */var/lib/mysql/ibdata**
InnoDB has no expedient index file allowing a quick query of row numbers.
Inconsistent table row numbers are reported by SHOW TABLE STATUS because InnoDB dynamically estimates the 'Rows' value by sampling a range of the table data (in */var/lib/mysql/ibdata**) and then extrapolates the approximate number of rows. So much so that the InnoDB documentation acknowledges row number inaccuracy of up to 50% when using SHOW TABLE STATUS
MySQL documentation suggests using the MySQL query cache to get consistent row number queries, but the docs don't specify how. A succinct explanation of how this can be done follows.
First, check that query caching is enabled:
mysql> SHOW VARIABLES LIKE 'have_query_cache';
If the value of have_query_cache is NO then enable the query cache by adding the following lines to /etc/my.cnf and then restart mysqld.
have_query_cache=1 # added 2017 08 24 wh
query_cache_size = 1048576
query_cache_type = 1
query_cache_limit = 1048576
(for more information see http://dev.mysql.com/doc/refman/5.1/en/query-cache.html)
Query the contents of the cache with
mysql> SHOW STATUS LIKE 'Qcache%';
Now use the SQL_CALC_FOUND_ROWS statement in a SELECT query:
SELECT SQL_CALC_FOUND_ROWS COUNT(*) FROM my_innodb_table
SQL_CALC_FOUND_ROWS will attempt a read from cache and, should this query not be found, execute the query against the specified table and then commit the number of table rows to the query cache. Additional executions of the above query (or other 'cachable' SELECT statements - see below) will consult the cache and return the correct result.
Subsequent 'cachable' SELECT queries - even if they LIMIT the result - will consult the query cache and allow you to get (once-off only) the total table row numbers with
SELECT FOUND_ROWS();
which returns the previous cached query's correct table row total.
The reasons for not keeping accurate statistics, including the row count in the table, is the multiversioning of rows InnoDB utilizes to provide transactions. What is the actual count of rows actually depends on isolation level of the transactions (as not-commited transaction may have deleted or inserted records), and different transactions can run in different isolation levels, which means that the question 'how many records there are' may be answered correctly only if there are no transactions running. So keeping a counter of the rows or data length is nearly impossible.
Read more about InnoDB restrictions
Without any changes on database I got very different rows count on my tables.
What can cause this?
Server version: 5.1.63-cll
Engine: InnoDB
Unlike MyISAM tables, InnoDB tables don't keep track of how many rows the table contains.
Because of this, the only way to know the exact number of rows in an InnoDB table is to inspect each row in the table and accumulate a count. Therefore, on large InnoDB tables, performing a SELECT COUNT(*) FROM innodb_table query can be very slow because it does a full table scan to get the number of rows.
phpMyAdmin uses a query like SHOW TABLE STATUS to get an estimated count of the number of rows in the table from the engine (InnoDB). Since it's just an estimate, it varies each time you call it, sometimes the variations can be fairly large, and sometimes they are close.
Here is an informative blog post about COUNT(*) for InnoDB tables by Percona.
The MySQL manual page for SHOW TABLE STATUS states:
The number of rows. Some storage engines, such as MyISAM, store the
exact count. For other storage engines, such as InnoDB, this value is
an approximation, and may vary from the actual value by as much as 40
to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate
count.
The page on InnoDB restrictions goes into some more detail:
SHOW TABLE STATUS does not give accurate statistics on InnoDB tables, except for the physical size reserved by the table. The row count is only a rough estimate used in SQL optimization.
InnoDB does not keep an internal count of rows in a table because
concurrent transactions might “see” different numbers of rows at the
same time. To process a SELECT COUNT(*) FROM t statement, InnoDB scans
an index of the table, which takes some time if the index is not
entirely in the buffer pool. If your table does not change often,
using the MySQL query cache is a good solution. To get a fast count,
you have to use a counter table you create yourself and let your
application update it according to the inserts and deletes it does. If
an approximate row count is sufficient, SHOW TABLE STATUS can be used.
See Section 14.3.14.1, “InnoDB Performance Tuning Tips”.
phpMyAdmin uses a quick method to get the row count, and this method only returns an approximate count in the case of InnoDB tables.
See $cfg['MaxExactCount'] for a way to modify those results, but this could have a serious impact on performance.
Without any changes on database I got very different rows count on my tables.
What can cause this?
Server version: 5.1.63-cll
Engine: InnoDB
Unlike MyISAM tables, InnoDB tables don't keep track of how many rows the table contains.
Because of this, the only way to know the exact number of rows in an InnoDB table is to inspect each row in the table and accumulate a count. Therefore, on large InnoDB tables, performing a SELECT COUNT(*) FROM innodb_table query can be very slow because it does a full table scan to get the number of rows.
phpMyAdmin uses a query like SHOW TABLE STATUS to get an estimated count of the number of rows in the table from the engine (InnoDB). Since it's just an estimate, it varies each time you call it, sometimes the variations can be fairly large, and sometimes they are close.
Here is an informative blog post about COUNT(*) for InnoDB tables by Percona.
The MySQL manual page for SHOW TABLE STATUS states:
The number of rows. Some storage engines, such as MyISAM, store the
exact count. For other storage engines, such as InnoDB, this value is
an approximation, and may vary from the actual value by as much as 40
to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate
count.
The page on InnoDB restrictions goes into some more detail:
SHOW TABLE STATUS does not give accurate statistics on InnoDB tables, except for the physical size reserved by the table. The row count is only a rough estimate used in SQL optimization.
InnoDB does not keep an internal count of rows in a table because
concurrent transactions might “see” different numbers of rows at the
same time. To process a SELECT COUNT(*) FROM t statement, InnoDB scans
an index of the table, which takes some time if the index is not
entirely in the buffer pool. If your table does not change often,
using the MySQL query cache is a good solution. To get a fast count,
you have to use a counter table you create yourself and let your
application update it according to the inserts and deletes it does. If
an approximate row count is sufficient, SHOW TABLE STATUS can be used.
See Section 14.3.14.1, “InnoDB Performance Tuning Tips”.
phpMyAdmin uses a quick method to get the row count, and this method only returns an approximate count in the case of InnoDB tables.
See $cfg['MaxExactCount'] for a way to modify those results, but this could have a serious impact on performance.
Without any changes on database I got very different rows count on my tables.
What can cause this?
Server version: 5.1.63-cll
Engine: InnoDB
Unlike MyISAM tables, InnoDB tables don't keep track of how many rows the table contains.
Because of this, the only way to know the exact number of rows in an InnoDB table is to inspect each row in the table and accumulate a count. Therefore, on large InnoDB tables, performing a SELECT COUNT(*) FROM innodb_table query can be very slow because it does a full table scan to get the number of rows.
phpMyAdmin uses a query like SHOW TABLE STATUS to get an estimated count of the number of rows in the table from the engine (InnoDB). Since it's just an estimate, it varies each time you call it, sometimes the variations can be fairly large, and sometimes they are close.
Here is an informative blog post about COUNT(*) for InnoDB tables by Percona.
The MySQL manual page for SHOW TABLE STATUS states:
The number of rows. Some storage engines, such as MyISAM, store the
exact count. For other storage engines, such as InnoDB, this value is
an approximation, and may vary from the actual value by as much as 40
to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate
count.
The page on InnoDB restrictions goes into some more detail:
SHOW TABLE STATUS does not give accurate statistics on InnoDB tables, except for the physical size reserved by the table. The row count is only a rough estimate used in SQL optimization.
InnoDB does not keep an internal count of rows in a table because
concurrent transactions might “see” different numbers of rows at the
same time. To process a SELECT COUNT(*) FROM t statement, InnoDB scans
an index of the table, which takes some time if the index is not
entirely in the buffer pool. If your table does not change often,
using the MySQL query cache is a good solution. To get a fast count,
you have to use a counter table you create yourself and let your
application update it according to the inserts and deletes it does. If
an approximate row count is sufficient, SHOW TABLE STATUS can be used.
See Section 14.3.14.1, “InnoDB Performance Tuning Tips”.
phpMyAdmin uses a quick method to get the row count, and this method only returns an approximate count in the case of InnoDB tables.
See $cfg['MaxExactCount'] for a way to modify those results, but this could have a serious impact on performance.
so I executed this query on a table:
EXPLAIN SELECT COUNT(*) FROM table;
and the 'rows' column from the output is displayed as NULL (whereas usually it will show how many rows the query went through)...
does this mean that the COUNT command is instantaneous and therefore does not require going through any row whatsoever?
If your table uses the MyISAM storage engine, then yes, that query resolves in constant time. The row count is part of the table metadata, the table itself does not have to be examined.
From: http://www.wikivs.com/wiki/MySQL_vs_PostgreSQL#COUNT.28.2A.29
Depending on what engine you are using, and I'm guessing it's MyISAM, a quick index count is executed rather than actually counting all the rows.
Many database engines use an index scan to get the count. If you're using MyISAM (fairly likely), it just reads a number in the engine index and returns it. Nearly instantaneous.
Edit:
InnoDB does a full table scan, so it will almost always be slower (than an engine that uses the table index), unless you're comparing queries with a WHERE clause.