I have a table using innodb. I know the table has roughly 89 million rows. Using
SELECT COUNT(*) FROM table;
takes about five minutes to run. I know that innodb is not optimized for unconditional COUNT(*) queries. How can I restructure the query to give me a count faster? Would just adding WHERE 1=1 work, or do I need to query a specific field?
I know I can get the approximate number of rows using SHOW TABLE STATUS, but I want to get the value in a PHP script, and it seems like there is a lot to dig through using that method.
If you are OK with the estimated number and just don't want to mess with running SHOW TABLE STATUS from PHP, you can use the information_schema DB:
SELECT TABLE_ROWS FROM information_schema.tables
WHERE TABLE_SCHEMA = 'my_db_name'
AND TABLE_NAME = 'my_table_name';
If you are ok with approximate number of records, you can use output of "explain".
Simplified verion of the code is
$result = mysql_query('explain SELECT count(*) from TABLE_NAME');
$row = mysql_fetch_assoc($result);
echo $row['rows'];
If the table is read frequently and updated infrequently, you may want to consider creating a statistics table that is updated via triggers when making changes to the table.
mysql_num_rows may be useful to you.
Related
Is there any simple way to get count data of single table from different databases(db names) with same schema.
I tried like this..
SELECT COUNT(id) AS db1users,
(SELECT COUNT(id) FROM DB2.users) AS db2users,(
SELECT COUNT(id) FROM DB3.users) AS db3users,
(SELECT COUNT(id) FROM DB4.users) AS db4users,
..........
..........
..........
FROM DB1.users;
I got the exact result but query becoming very large. Is there any simple way to get this..
Please help
Another option, that avoids the need for dynamic sql (and is far less expensive, hence much more scalable), would be to use MySQL the INFORMATION_SCHEMA.TABLES table. It has a column named TABLE_ROWS, whose specification is as follows:
TABLE_ROWS
The number of rows. Some storage engines, such as MyISAM, store the exact count. For other storage engines, such as InnoDB, this value is an approximation, and may vary from the actual value by as much as 40% to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate count.
If this matches your requirement, then you can use a simple query loke:
SELECT table_schema, table_rows FROM information_schema.tables WHERE table_name = 'users'
The best way to do this would be via a scripting language (e.g. Python, Ruby, PHP); you'd execute a database query to get all the database names from your database, then create a SQL statement with all your select count(id) from...; once you've built the SQL statement, you'd execute it.
You can also do this in dynamic SQL inside MySQL; dynamic SQL is hard to write and debug, so I'm not a huge fan....
I have a big table (approx. 150 M rows), and if I try to run a simple select count(*) on it, then mysql works for about an hour and then throws an error.
I'm assuming this is not due to a limitation of mysql, rather a problem on my side, but I have no idea where to start looking. any ideas?
the table is innodb
mysql 5.5 on linux
you can check it with table status like this
SHOW TABLE STATUS FROM db_name LIKE 'table_name';
you see the rows column....
You can use count(1) instead of count(*)
Try ::
Select count(1) from my_table
The easiest way to speed up queries like this is with a covering index. This will allow you to scan through the rows you want, but require fewer bytes of I/O per row (since you're only scanning a portion of each row's data, not the entire row). Furthermore, if your index is sorted in the same way that your query is, you can avoid the cost of a sort and can scan vastly fewer rows.
You can use Information Schema.
SELECT TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE table_schema =
YOUR_DB_NAME AND table_name = YOUR_TABLE_NAME
After facing a slow loading time issue with a mysql query, I'm now looking the best way to count rows numbers. I have stupidly used mysql_num_rows() function to do this and now realized its a worst way to do this.
I was actually making a Pagination to make pages in PHP.
I have found several ways to count rows number. But I'm looking the faster way to count it.
The table type is MyISAM
So the question is now
Which is the best and faster to count -
1. `SELECT count(*) FROM 'table_name'`
2. `SELECT TABLE_ROWS
FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = 'database_name'
AND table_name LIKE 'table_name'`
3. `SHOW TABLE STATUS LIKE 'table_name'`
4. `SELECT FOUND_ROWS()`
If there are others better way to do this, please let me know them as well.
If possible please describe along with the answer- why it is best and faster. So I could understand and can use the method based on my requirement.
Thanks.
Quoting the MySQL Reference Manual on COUNT
COUNT(*) is optimized to return very quickly if the SELECT retrieves
from one table, no other columns are retrieved, and there is no WHERE
clause. For example:
mysql> SELECT COUNT(*) FROM student;
This optimization applies only to
MyISAM tables only, because an exact row count is stored for this
storage engine and can be accessed very quickly. For transactional
storage engines such as InnoDB, storing an exact row count is more
problematic because multiple transactions may be occurring, each of
which may affect the count.
Also read this question
MySQL - Complexity of: SELECT COUNT(*) FROM MyTable;
I would start by using SELECT count(*) FROM 'table_name' because it is the most portable, easiset to understand, and because it is likely that the DBMS developers optimise common idiomatic queries of this sort.
Only if that wasn't fast enough would I benchmark the approaches you list to find if any were significantly faster.
It's slightly faster to count a constant:
select count('x') from table;
When the parser hits count(*) it has to go figure out what all the columns of the table are that are represented by the * and get ready to accept them inside the count().
Using a constant bypasses this (albeit slight) column checking overhead.
As an aside, although not faster, one cute option is:
select sum(1) from table;
I've looked around quite a bit for this recently. it seems that there are a few here that I'd never seen before.
Special needs: This database is about 6 million records and is getting crushed by multi-insert queries all the time. Getting a true count is difficult to say the least.
SELECT TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = 'admin_worldDomination' AND table_name LIKE 'master'
Showing rows 0 - 0 ( 1 total, Query took 0.0189 sec)
This is decent, Very fast but inaccurate. Showed results from 4 million to almost 8 million rows
SELECT count( * ) AS counter FROM `master`
No time displayed, took 8 seconds real time. Will get much worse as the table grows. This has been killing my site previous to today.
SHOW TABLE STATUS LIKE 'master'
Seems to be as fast as the first, no time displayed though. Offers lots of other table information, not much of it is worth anything though (avg record length maybe).
SELECT FOUND_ROWS() FROM 'master'
Showing rows 0 - 29 ( 4,824,232 total, Query took 0.0004 sec)
This is good, but an average. Closer spread than others (4-5 million) so I'll probably end up taking a sample from a few of these queries and averaging.
EDIT: This was really slow when doing a query in php, ended up going with the first. Query runs 30 times quickly and I take an average, under 1 second ... it' still ranges between 5.3 & 5.5 million
One idea I had, to throw this out there, is to try to find a way to estimate the row count. Since it's just to give your user an idea of the number of pages, maybe you don't need to be exact and could even say Page 1 of ~4837 or Page 1 of about 4800 or something.
I couldn't quickly find an estimate count function, but you could try getting the table size and dividing by a determined/constant avg row size. I don't know if or why getting the table size from TABLE STATUS would be faster than getting the rows from TABLE STATUS.
Which way to count a number of rows should be faster in MySQL?
This:
SELECT COUNT(*) FROM ... WHERE ...
Or, the alternative:
SELECT 1 FROM ... WHERE ...
// and then count the results with a built-in function, e.g. in PHP mysql_num_rows()
One would think that the first method should be faster, as this is clearly database territory and the database engine should be faster than anybody else when determining things like this internally.
When you COUNT(*) it takes in count column indexes, so it will be the best result. MySQL with MyISAM engine actually stores row count, it doesn't count all rows each time you try to count all rows. (based on primary key's column)
Using PHP to count rows is not very smart, because you have to send data from MySQL to PHP. Why do it when you can achieve the same on the MySQL side?
If the COUNT(*) is slow, you should run EXPLAIN on the query, and check if indexes are really used, and where they should be added.
The following is not the fastest way, but there is a case, where COUNT(*) doesn't really fit - when you start grouping results, you can run into a problem where COUNT doesn't really count all rows.
The solution is SQL_CALC_FOUND_ROWS. This is usually used when you are selecting rows but still need to know the total row count (for example, for paging).
When you select data rows, just append the SQL_CALC_FOUND_ROWS keyword after SELECT:
SELECT SQL_CALC_FOUND_ROWS [needed fields or *] FROM table LIMIT 20 OFFSET 0;
After you have selected needed rows, you can get the count with this single query:
SELECT FOUND_ROWS();
FOUND_ROWS() has to be called immediately after the data selecting query.
In conclusion, everything actually comes down to how many entries you have and what is in the WHERE statement. You should really pay attention on how indexes are being used, when there are lots of rows (tens of thousands, millions, and up).
After speaking with my team-mates, Ricardo told us that the faster way is:
show table status like '<TABLE NAME>' \G
But you have to remember that the result may not be exact.
You can use it from command line too:
$ mysqlshow --status <DATABASE> <TABLE NAME>
More information: http://dev.mysql.com/doc/refman/5.7/en/show-table-status.html
And you can find a complete discussion at mysqlperformanceblog
This query (which is similar to what bayuah posted) shows a nice summary of all tables count inside a database:
(simplified version of stored procedure by Ivan Cachicatari which I highly recommend).
SELECT TABLE_NAME AS 'Table Name', TABLE_ROWS AS 'Rows' FROM information_schema.TABLES WHERE TABLES.TABLE_SCHEMA = '`YOURDBNAME`' AND TABLES.TABLE_TYPE = 'BASE TABLE';
Example:
+-----------------+---------+
| Table Name | Rows |
+-----------------+---------+
| some_table | 10278 |
| other_table | 995 |
Great question, great answers. Here's a quick way to echo the results if anyone is reading this page and missing that part:
$counter = mysql_query("SELECT COUNT(*) AS id FROM table");
$num = mysql_fetch_array($counter);
$count = $num["id"];
echo("$count");
I've always understood that the below will give me the fastest response times.
SELECT COUNT(1) FROM ... WHERE ...
If you need to get the count of the entire result set you can take following approach:
SELECT SQL_CALC_FOUND_ROWS * FROM table_name LIMIT 5;
SELECT FOUND_ROWS();
This isn't normally faster than using COUNT albeit one might think the opposite is the case because it's doing the calculation internally and doesn't send the data back to the user thus the performance improvement is suspected.
Doing these two queries is good for pagination for getting totals but not particularly for using WHERE clauses.
Try this:
SELECT
table_rows "Rows Count"
FROM
information_schema.tables
WHERE
table_name="Table_Name"
AND
table_schema="Database_Name";
I did some benchmarks to compare the execution time of COUNT(*) vs COUNT(id) (id is the primary key of the table - indexed).
Number of trials:
10 * 1000 queries
Results:
COUNT(*) is faster 7%
VIEW GRAPH: benchmarkgraph
My advice is to use: SELECT COUNT(*) FROM table
EXPLAIN SELECT id FROM .... did the trick for me. and I could see the number of rows under rows column of the result.
Perhaps you may want to consider doing a SELECT max(Id) - min(Id) + 1. This will only work if your Ids are sequential and rows are not deleted. It is however very fast.
This is the best query able to get the fastest results.
SELECT SQL_CALC_FOUND_ROWS 1 FROM `orders`;
SELECT FOUND_ROWS();
In my benchmark test: 0.448s
This query takes 4.835s
SELECT SQL_CALC_FOUND_ROWS * FROM `orders`;
SELECT FOUND_ROWS();
count * takes 25.675s
SELECT count(*) FROM `orders`;
If you don't need super-exact count, then you can set a lower transaction isolation level for the current session. Do it like this:
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
SELECT count(*) FROM the_table WHERE ...;
COMMIT; /* close the transaction */
It is also good to have an index that matches the WHERE condition.
It really speeds up the counting for big InnoDB tables. I checked it on a table with ~700M rows and heavy load, it works. It reduced the query time from ~451 seconds to ~2 seconds.
I took the idea from this answer: https://stackoverflow.com/a/918092/1743367
A count(*) statement with a where condition on the primary key returned the row count much faster for me avoiding full table scan.
SELECT COUNT(*) FROM ... WHERE <PRIMARY_KEY> IS NOT NULL;
This was much faster for me than
SELECT COUNT(*) FROM ...
I handled tables for the German Government with sometimes 60 million records.
And we needed to know many times the total rows.
So we database programmers decided that in every table is record one always the record in which the total record numbers is stored. We updated this number, depending on INSERT or DELETE rows.
We tried all other ways. This is by far the fastest way.
What is the MySQL command to retrieve the count of records in a table?
SELECT COUNT(*) FROM fooTable;
will count the number of rows in the table.
See the reference manual.
Because nobody mentioned it:
show table status;
lists all tables along with some additional information, including estimated rows for each table. This is what phpMyAdmin is using for its database page.
This information is available in MySQL 4, probably in MySQL 3.23 too - long time prior information schema database.
The number shown is estimated for InnoDB and TokuDB but it is absolutely correct for MyISAM and Aria (Maria) storage engines.
Per the documentation:
The number of rows. Some storage engines, such as MyISAM, store the
exact count. For other storage engines, such as InnoDB, this value is
an approximation, and may vary from the actual value by as much as 40%
to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate
count.
This also is fastest way to see the row count on MySQL, because query like:
select count(*) from table;
Doing full table scan what could be very expensive operation that might take hours on large high load server. It also increase disk I/O.
The same operation might block the table for inserts and updates - this happen only on exotic storage engines.
InnoDB and TokuDB are OK with table lock, but need full table scan.
We have another way to find out the number of rows in a table without running a select query on that table.
Every MySQL instance has information_schema database. If you run the following query, it will give complete details about the table including the approximate number of rows in that table.
select * from information_schema.TABLES where table_name = 'table_name'\G
Simply:
SELECT COUNT(*) FROM `tablename`
select count(*) from YourTable
If you have several fields in your table and your table is huge, it's better DO NOT USE * because of it load all fields to memory and using the following will have better performance
SELECT COUNT(1) FROM fooTable;
Just do a
SELECT COUNT(*) FROM table;
You can specify conditions with a Where after that
SELECT COUNT(*) FROM table WHERE eye_color='brown';
As mentioned by Santosh, I think this query is suitably fast, while not querying all the table.
To return integer result of number of data records, for a specific tablename in a particular database:
select TABLE_ROWS from information_schema.TABLES where TABLE_SCHEMA = 'database'
AND table_name='tablename';
If you have a primary key or a unique key/index, the faster method possible (Tested with 4 millions row tables)
SHOW INDEXES FROM "database.tablename" WHERE Key_Name=\"PRIMARY\"
and then get cardinality field (it is close to instant)
Times where from 0.4s to 0.0001ms
$sql="SELECT count(*) as toplam FROM wp_postmeta WHERE meta_key='ICERIK' AND post_id=".$id;
$total = 0;
$sqls = mysql_query($sql,$conn);
if ( $sqls ) {
$total = mysql_result($sqls, 0);
};
echo "Total:".$total;`
You have to use count() returns the number of rows that matches a
specified criteria
select count(*) from table_name;
It can be convenient to select count with filter by indexed field. Try this
EXPLAIN SELECT * FROM table_name WHERE key < anything;