Let's say I have a black box query that I don't really understand how it works, something along the lines of:
SELECT ... FROM ... JOIN ... = A (denoted as A)
Let's say A returns 500 rows.
I want to get the count of the number of rows (500 in this case), and then only return a limit of 50.
How can I wrote a query built around A that would return the number '500' and 50 rows of data?
You can use window functions (available in MySQL 8.0 only) and a row-limiting clause:
select a.*, count(*) over() total_rows
from ( < your query >) a
order by ??
limit 50
Note that I added an order by clause to the query. Although this is not technically required, it is a best practice: without an order by clause where the column (or set of columns) uniquely identifies each row, it is undefined which 50 rows the database will return, and the results may not be consistent over consecutive executions of the same query.
This is what SELECT SQL_CALC_FOUND_ROWS is intended to do.
SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT FOUND_ROWS();
The first query returns the limited set of rows.
The second query calls FOUND_ROWS() which returns an integer number of how many rows matched the most recent query, the number of rows which would have been returned if that query had not used LIMIT.
See https://dev.mysql.com/doc/refman/8.0/en/information-functions.html#function_found-rows
However, keep in mind that using SQL_CALC_FOUND_ROWS incurs a significant performance cost. Benchmarks show that it's usually faster to just run two queries:
SELECT COUNT(*) FROM tbl_name WHERE id > 100; -- the count of matching rows
SELECT * FROM tbl_name WHERE id > 100 LIMIT 10; -- the limited result
See https://www.percona.com/blog/2007/08/28/to-sql_calc_found_rows-or-not-to-sql_calc_found_rows/
There are a few ways you can do this (assuming that I am understanding your question correctly). You can open run two queries (and point a cursor to each) and then open and return both cursors, or you can run a stored procedure in which the count query is ran first, the result is stored into a variable, then it is used in another query.
Let me know if you would like an example of either of these
Related
If I have a mysql limited query:
SELECT * FROM my_table WHERE date > '2020-12-12' LIMIT 1,16;
Is there a faster way to check and see how many results are left after my limit?
I was trying to do a count with limit, but that wasn't working, i.e.
SELECT count(ID) AS count FROM my_table WHERE date > '2020-12-12' LIMIT 16,32;
The ultimate goal here is just to determine if there ARE any other rows to be had beyond the current result set, so if there is another faster way to do this that would be fine too.
It's best to do this by counting the rows:
SELECT count(*) AS count FROM my_table WHERE date > '2020-12-12'
That tells you how many total rows match the condition. Then you can compare that to the size of the result you got with your query using LIMIT. It's just arithmetic.
Past versions of MySQL had a function FOUND_ROWS() which would report how many rows would have matched if you didn't use LIMIT. But it turns out this had worse performance than running two queries, one to count rows and one to do your limit. So they deprecated this feature.
For details read:
https://www.percona.com/blog/2007/08/28/to-sql_calc_found_rows-or-not-to-sql_calc_found_rows/
https://dev.mysql.com/worklog/task/?id=12615
(You probably want OFFSET 0, not 1.)
It's simple to test whether there ARE more rows. Assuming you want 16 rows, use 1 more:
SELECT ... WHERE ... ORDER BY ... LIMIT 0,17
Then programmatically see whether it returned only 16 rows (no more available) or 17 (there ARE more).
Because it is piggybacking on the fetch you are already doing and not doing much extra work, it is very efficient.
The second 'page' would use LIMIT 16, 17; 3rd: LIMIT 32,17, etc. Each time, you are potentially getting and tossing an extra row.
I discuss this and other tricks where I point out the evils of OFFSET: Pagination
COUNT(x) checks x for being NOT NULL. This is [usually] unnecessary. The pattern COUNT(*) (or COUNT(1)) simply counts rows; the * or 1 has no significance.
SELECT COUNT(*) FROM t is not free. It will actually do a full index scan, which is slow for a large table. WHERE and ORDER BY are likely to add to that slowness. LIMIT is useless since the result is always 1 row. (That is, the LIMIT is applied to the result, not to the counting.)
is there a way to select couple columns using a limit but in mean time get the total count of rows in that table
SELECT col1,col2 FROM table where col3=0 limit 500
just like above. what to add to get the total of rows in the table
thank you
Try this, you only need to use one column in the count
SELECT COUNT(col3) FROM table where col3=0
And here's a reference for more information MySQL COUNT
Note, to keep it simple, you have to run two queries, this one for the count and yours for the records
Your question lacks a precise problem statement.
My understanding of your question is as follows:
All of the following is necessary:
Need to get every row from the table, such that col3=0.
Need to get the total number of rows in the table (whether they satisfy col3=0 or not).
Need to limit your result set to columns col1, col2, and have at most 500 rows.
Need to execute a single query, rather than two separate queries.
If this is correct interpretation of your question, then I would propose the following solution:
SELECT col1,col2, (SELECT COUNT(*) FROM table) AS total_rows FROM table where col3=0 limit 500
Such query would produce result where at most 500 rows from your table satisfy the condition col3=0 are present alongside total_rows, which tells the number of all the rows in the table.
Example CSV of the result:
clo1,col2,total_rows
a,b,1000
c,d,1000
d,e,1000
according to found_rows()
The SQL_CALC_FOUND_ROWS query modifier and accompanying FOUND_ROWS() function are deprecated as of MySQL 8.0.17; expect them to be removed in a future version of MySQL. As a replacement, considering executing your query with LIMIT, and then a second query with COUNT(*) and without LIMIT to determine whether there are additional rows
instead of these queries:
SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT FOUND_ROWS();
Use these queries instead:
SELECT * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT COUNT(*) FROM tbl_name WHERE id > 100;
COUNT(*) is subject to certain optimizations. SQL_CALC_FOUND_ROWS causes some optimizations to be disabled.
you can check these questions
how-to-get-number-of-rows
mysql-what-is-the-row-count-information-function-for-select
I am running a query where i need to know the number of lines total in a table but only need to show the first 6.
So, is it faster to run select count(*) then select * ... limit 6 and print data returned? Or, just select * with no limit and put a counter in the while loop printing the results? With the latter I can obviously use mysql_num_rows to get the total.
The table in question will contain up to 1 million rows, the query includes a where row = xxx that column will be indexed
Use FOUND_ROWS(). Here's an example:
SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT FOUND_ROWS();
Do two queries. Your count query will use an index and will not have to scan the whole table, only the index. The second query will only have to read the 6 rows from the table.
Im in a dilema on which one of these methods are most efficient.
Suppose you have a query joining multiple tables and querying thousand of records. Than, you gotta get the total to paginate throughout all these results.
Is it faster to?
1) Do a complete select (suppose you have to select 50's columns), count the rows and than run another query with limits? (Will the MySQL cache help this case already selecting all the columns you need on the first query used to count?)
2) First do the query using COUNT function and than do the query to select the results you need.
3) Instead of using MySQL COUNT function, do the query selecting the ID's for example and use the PHP function mysql_num_rows?
I think the number 2 is the best option, using MySQL built in COUNT function, but I know MySQL uses cache, so, selecting all the results on first query gonna be faster?
Thanks,
Have a look at Found_Rows()
A SELECT statement may include a LIMIT clause to restrict the number
of rows the server returns to the client. In some cases, it is
desirable to know how many rows the statement would have returned
without the LIMIT, but without running the statement again. To obtain
this row count, include a SQL_CALC_FOUND_ROWS option in the SELECT
statement, and then invoke FOUND_ROWS() afterward:
mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name WHERE id > 100 LIMIT 10;
mysql> SELECT FOUND_ROWS();
The second SELECT returns a number indicating how many rows the first SELECT`
would have returned had it been written without the LIMIT clause.
My guess is number 2, but the truth is that it will depend entirely on data size, tables, indexing, MySql version etc.
The only way of finding the answer to this is to try each one and measure how long they take. But like I say, my hunch would be number 2.
At the moment I run two MySQL queries to handle my pagination...
Query 1 selects all rows from a table so I know how many pages I need to list.
Query 2 selects the rows for the current page (e.g: rows 0 to 19 (LIMIT 0, 19) for page 1, rows 20-39 for page two etc etc).
It seems like a waste of two duplicate queries with the only difference being the LIMIT part.
What would be a better way to do this?
Should I use PHP to filter the results after one query has been run?
Edit: Should I run one query and use something like array_slice() to only list the rows I want?
The best & fastest way is to use 2 MYSQL queries for pagination (as you are already using), to avoid over headache you must simplify the query used to find out the total number of rows by selecting only one column (say the primary key) that's enough.
SELECT * FROM sampletable WHERE condition1>1 AND condition2>2
for paginating such a query you may use these two queries
SELECT * FROM sampletable WHERE condition1>1 AND condition2>2 LIMIT 0,20
SELECT id FROM sampletable WHERE condition1>1 AND condition2>2
Use FOUND_ROWS()
A SELECT statement may include a LIMIT clause to restrict the number of rows the server returns to the client. In some cases, it is desirable to know how many rows the statement would have returned without the LIMIT, but without running the statement again. To obtain this row count, include a SQL_CALC_FOUND_ROWS option in the SELECT statement, and then invoke FOUND_ROWS() afterward:
mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name
-> WHERE id > 100 LIMIT 10;
mysql> SELECT FOUND_ROWS();
The second SELECT returns a number indicating how many rows the first SELECT would have returned had it been written without the LIMIT clause.
Note that this puts additional strain on the database, because it has to find out the size of the full result set every time. Use SQL_CALC_FOUND_ROWS only when you need it.