MySQL: Fastest way to count number of rows - mysql

Which way to count a number of rows should be faster in MySQL?
This:
SELECT COUNT(*) FROM ... WHERE ...
Or, the alternative:
SELECT 1 FROM ... WHERE ...
// and then count the results with a built-in function, e.g. in PHP mysql_num_rows()
One would think that the first method should be faster, as this is clearly database territory and the database engine should be faster than anybody else when determining things like this internally.

When you COUNT(*) it takes in count column indexes, so it will be the best result. MySQL with MyISAM engine actually stores row count, it doesn't count all rows each time you try to count all rows. (based on primary key's column)
Using PHP to count rows is not very smart, because you have to send data from MySQL to PHP. Why do it when you can achieve the same on the MySQL side?
If the COUNT(*) is slow, you should run EXPLAIN on the query, and check if indexes are really used, and where they should be added.
The following is not the fastest way, but there is a case, where COUNT(*) doesn't really fit - when you start grouping results, you can run into a problem where COUNT doesn't really count all rows.
The solution is SQL_CALC_FOUND_ROWS. This is usually used when you are selecting rows but still need to know the total row count (for example, for paging).
When you select data rows, just append the SQL_CALC_FOUND_ROWS keyword after SELECT:
SELECT SQL_CALC_FOUND_ROWS [needed fields or *] FROM table LIMIT 20 OFFSET 0;
After you have selected needed rows, you can get the count with this single query:
SELECT FOUND_ROWS();
FOUND_ROWS() has to be called immediately after the data selecting query.
In conclusion, everything actually comes down to how many entries you have and what is in the WHERE statement. You should really pay attention on how indexes are being used, when there are lots of rows (tens of thousands, millions, and up).

After speaking with my team-mates, Ricardo told us that the faster way is:
show table status like '<TABLE NAME>' \G
But you have to remember that the result may not be exact.
You can use it from command line too:
$ mysqlshow --status <DATABASE> <TABLE NAME>
More information: http://dev.mysql.com/doc/refman/5.7/en/show-table-status.html
And you can find a complete discussion at mysqlperformanceblog

This query (which is similar to what bayuah posted) shows a nice summary of all tables count inside a database:
(simplified version of stored procedure by Ivan Cachicatari which I highly recommend).
SELECT TABLE_NAME AS 'Table Name', TABLE_ROWS AS 'Rows' FROM information_schema.TABLES WHERE TABLES.TABLE_SCHEMA = '`YOURDBNAME`' AND TABLES.TABLE_TYPE = 'BASE TABLE';
Example:
+-----------------+---------+
| Table Name | Rows |
+-----------------+---------+
| some_table | 10278 |
| other_table | 995 |

Great question, great answers. Here's a quick way to echo the results if anyone is reading this page and missing that part:
$counter = mysql_query("SELECT COUNT(*) AS id FROM table");
$num = mysql_fetch_array($counter);
$count = $num["id"];
echo("$count");

I've always understood that the below will give me the fastest response times.
SELECT COUNT(1) FROM ... WHERE ...

If you need to get the count of the entire result set you can take following approach:
SELECT SQL_CALC_FOUND_ROWS * FROM table_name LIMIT 5;
SELECT FOUND_ROWS();
This isn't normally faster than using COUNT albeit one might think the opposite is the case because it's doing the calculation internally and doesn't send the data back to the user thus the performance improvement is suspected.
Doing these two queries is good for pagination for getting totals but not particularly for using WHERE clauses.

Try this:
SELECT
table_rows "Rows Count"
FROM
information_schema.tables
WHERE
table_name="Table_Name"
AND
table_schema="Database_Name";

I did some benchmarks to compare the execution time of COUNT(*) vs COUNT(id) (id is the primary key of the table - indexed).
Number of trials:
10 * 1000 queries
Results:
COUNT(*) is faster 7%
VIEW GRAPH: benchmarkgraph
My advice is to use: SELECT COUNT(*) FROM table

EXPLAIN SELECT id FROM .... did the trick for me. and I could see the number of rows under rows column of the result.

Perhaps you may want to consider doing a SELECT max(Id) - min(Id) + 1. This will only work if your Ids are sequential and rows are not deleted. It is however very fast.

This is the best query able to get the fastest results.
SELECT SQL_CALC_FOUND_ROWS 1 FROM `orders`;
SELECT FOUND_ROWS();
In my benchmark test: 0.448s
This query takes 4.835s
SELECT SQL_CALC_FOUND_ROWS * FROM `orders`;
SELECT FOUND_ROWS();
count * takes 25.675s
SELECT count(*) FROM `orders`;

If you don't need super-exact count, then you can set a lower transaction isolation level for the current session. Do it like this:
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
SELECT count(*) FROM the_table WHERE ...;
COMMIT; /* close the transaction */
It is also good to have an index that matches the WHERE condition.
It really speeds up the counting for big InnoDB tables. I checked it on a table with ~700M rows and heavy load, it works. It reduced the query time from ~451 seconds to ~2 seconds.
I took the idea from this answer: https://stackoverflow.com/a/918092/1743367

A count(*) statement with a where condition on the primary key returned the row count much faster for me avoiding full table scan.
SELECT COUNT(*) FROM ... WHERE <PRIMARY_KEY> IS NOT NULL;
This was much faster for me than
SELECT COUNT(*) FROM ...

I handled tables for the German Government with sometimes 60 million records.
And we needed to know many times the total rows.
So we database programmers decided that in every table is record one always the record in which the total record numbers is stored. We updated this number, depending on INSERT or DELETE rows.
We tried all other ways. This is by far the fastest way.

Related

How to get count of rows and rows in one query?

Let's say I have a black box query that I don't really understand how it works, something along the lines of:
SELECT ... FROM ... JOIN ... = A (denoted as A)
Let's say A returns 500 rows.
I want to get the count of the number of rows (500 in this case), and then only return a limit of 50.
How can I wrote a query built around A that would return the number '500' and 50 rows of data?
You can use window functions (available in MySQL 8.0 only) and a row-limiting clause:
select a.*, count(*) over() total_rows
from ( < your query >) a
order by ??
limit 50
Note that I added an order by clause to the query. Although this is not technically required, it is a best practice: without an order by clause where the column (or set of columns) uniquely identifies each row, it is undefined which 50 rows the database will return, and the results may not be consistent over consecutive executions of the same query.
This is what SELECT SQL_CALC_FOUND_ROWS is intended to do.
SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT FOUND_ROWS();
The first query returns the limited set of rows.
The second query calls FOUND_ROWS() which returns an integer number of how many rows matched the most recent query, the number of rows which would have been returned if that query had not used LIMIT.
See https://dev.mysql.com/doc/refman/8.0/en/information-functions.html#function_found-rows
However, keep in mind that using SQL_CALC_FOUND_ROWS incurs a significant performance cost. Benchmarks show that it's usually faster to just run two queries:
SELECT COUNT(*) FROM tbl_name WHERE id > 100; -- the count of matching rows
SELECT * FROM tbl_name WHERE id > 100 LIMIT 10; -- the limited result
See https://www.percona.com/blog/2007/08/28/to-sql_calc_found_rows-or-not-to-sql_calc_found_rows/
There are a few ways you can do this (assuming that I am understanding your question correctly). You can open run two queries (and point a cursor to each) and then open and return both cursors, or you can run a stored procedure in which the count query is ran first, the result is stored into a variable, then it is used in another query.
Let me know if you would like an example of either of these

Two selects in one query

I have a product table.
I'm going to create a pagination on my frontend.
So I need to do this select:
select * from `product` limit 20 offset 0
I also need the total number of records in the table of products (I'll show the number of pages). I have this query:
select count(*) as all from `product`
I wanted to run these two queries in just one query.
Something like:
select *, count(*) as total from `hospedes` limit 20 offset 0
the above query does not work. is there any way to do this?
I don't believe you can run this as one query unless you are willing to give up the "LIMIT" and just retrieve all records, do a count on the result set, and throw away all but the first 20.
I recommend you run two queries. One to get the total COUNT(*) from the table, and then run a separate query for your pagination.
In theory you can do this:
mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name
-> WHERE id > 100 LIMIT 10;
mysql> SELECT FOUND_ROWS();
This promises to let you get the information of how many tables match the query as if you had not used LIMIT. Read https://dev.mysql.com/doc/refman/5.7/en/information-functions.html#function_found-rows for details
It does give you the correct count, but using this method usually causes a high performance cost.
Read https://www.percona.com/blog/2007/08/28/to-sql_calc_found_rows-or-not-to-sql_calc_found_rows/ for a test that shows it is a better strategy to run separate queries. Admittedly, that blog is from 2007 and the test was performed on MySQL 5.0. You can try to reproduce their test on the current version of MySQL to see if it has improved since then.

Effective use of pagination and MySQL queries

At the moment I run two MySQL queries to handle my pagination...
Query 1 selects all rows from a table so I know how many pages I need to list.
Query 2 selects the rows for the current page (e.g: rows 0 to 19 (LIMIT 0, 19) for page 1, rows 20-39 for page two etc etc).
It seems like a waste of two duplicate queries with the only difference being the LIMIT part.
What would be a better way to do this?
Should I use PHP to filter the results after one query has been run?
Edit: Should I run one query and use something like array_slice() to only list the rows I want?
The best & fastest way is to use 2 MYSQL queries for pagination (as you are already using), to avoid over headache you must simplify the query used to find out the total number of rows by selecting only one column (say the primary key) that's enough.
SELECT * FROM sampletable WHERE condition1>1 AND condition2>2
for paginating such a query you may use these two queries
SELECT * FROM sampletable WHERE condition1>1 AND condition2>2 LIMIT 0,20
SELECT id FROM sampletable WHERE condition1>1 AND condition2>2
Use FOUND_ROWS()
A SELECT statement may include a LIMIT clause to restrict the number of rows the server returns to the client. In some cases, it is desirable to know how many rows the statement would have returned without the LIMIT, but without running the statement again. To obtain this row count, include a SQL_CALC_FOUND_ROWS option in the SELECT statement, and then invoke FOUND_ROWS() afterward:
mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name
-> WHERE id > 100 LIMIT 10;
mysql> SELECT FOUND_ROWS();
The second SELECT returns a number indicating how many rows the first SELECT would have returned had it been written without the LIMIT clause.
Note that this puts additional strain on the database, because it has to find out the size of the full result set every time. Use SQL_CALC_FOUND_ROWS only when you need it.

MySQL get the number of rows in an innodb table

I have a table using innodb. I know the table has roughly 89 million rows. Using
SELECT COUNT(*) FROM table;
takes about five minutes to run. I know that innodb is not optimized for unconditional COUNT(*) queries. How can I restructure the query to give me a count faster? Would just adding WHERE 1=1 work, or do I need to query a specific field?
I know I can get the approximate number of rows using SHOW TABLE STATUS, but I want to get the value in a PHP script, and it seems like there is a lot to dig through using that method.
If you are OK with the estimated number and just don't want to mess with running SHOW TABLE STATUS from PHP, you can use the information_schema DB:
SELECT TABLE_ROWS FROM information_schema.tables
WHERE TABLE_SCHEMA = 'my_db_name'
AND TABLE_NAME = 'my_table_name';
If you are ok with approximate number of records, you can use output of "explain".
Simplified verion of the code is
$result = mysql_query('explain SELECT count(*) from TABLE_NAME');
$row = mysql_fetch_assoc($result);
echo $row['rows'];
If the table is read frequently and updated infrequently, you may want to consider creating a statistics table that is updated via triggers when making changes to the table.
mysql_num_rows may be useful to you.

Count table rows

What is the MySQL command to retrieve the count of records in a table?
SELECT COUNT(*) FROM fooTable;
will count the number of rows in the table.
See the reference manual.
Because nobody mentioned it:
show table status;
lists all tables along with some additional information, including estimated rows for each table. This is what phpMyAdmin is using for its database page.
This information is available in MySQL 4, probably in MySQL 3.23 too - long time prior information schema database.
The number shown is estimated for InnoDB and TokuDB but it is absolutely correct for MyISAM and Aria (Maria) storage engines.
Per the documentation:
The number of rows. Some storage engines, such as MyISAM, store the
exact count. For other storage engines, such as InnoDB, this value is
an approximation, and may vary from the actual value by as much as 40%
to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate
count.
This also is fastest way to see the row count on MySQL, because query like:
select count(*) from table;
Doing full table scan what could be very expensive operation that might take hours on large high load server. It also increase disk I/O.
The same operation might block the table for inserts and updates - this happen only on exotic storage engines.
InnoDB and TokuDB are OK with table lock, but need full table scan.
We have another way to find out the number of rows in a table without running a select query on that table.
Every MySQL instance has information_schema database. If you run the following query, it will give complete details about the table including the approximate number of rows in that table.
select * from information_schema.TABLES where table_name = 'table_name'\G
Simply:
SELECT COUNT(*) FROM `tablename`
select count(*) from YourTable
If you have several fields in your table and your table is huge, it's better DO NOT USE * because of it load all fields to memory and using the following will have better performance
SELECT COUNT(1) FROM fooTable;
Just do a
SELECT COUNT(*) FROM table;
You can specify conditions with a Where after that
SELECT COUNT(*) FROM table WHERE eye_color='brown';
As mentioned by Santosh, I think this query is suitably fast, while not querying all the table.
To return integer result of number of data records, for a specific tablename in a particular database:
select TABLE_ROWS from information_schema.TABLES where TABLE_SCHEMA = 'database'
AND table_name='tablename';
If you have a primary key or a unique key/index, the faster method possible (Tested with 4 millions row tables)
SHOW INDEXES FROM "database.tablename" WHERE Key_Name=\"PRIMARY\"
and then get cardinality field (it is close to instant)
Times where from 0.4s to 0.0001ms
$sql="SELECT count(*) as toplam FROM wp_postmeta WHERE meta_key='ICERIK' AND post_id=".$id;
$total = 0;
$sqls = mysql_query($sql,$conn);
if ( $sqls ) {
$total = mysql_result($sqls, 0);
};
echo "Total:".$total;`
You have to use count() returns the number of rows that matches a
specified criteria
select count(*) from table_name;
It can be convenient to select count with filter by indexed field. Try this
EXPLAIN SELECT * FROM table_name WHERE key < anything;