mysql union limit problem - mysql

Question

I want a paging script working properly basically but the situation is a bit complex. I need to pick data from union of two sql queries. See the query below. I have a table book and a table bookvisit. What I want is here to show all books for a particular category in their popularity order. I am getting data for all books with atleast one visit by joining table book and bookvisit. and then union it with all books with no visit. Everything works fine but when I try to do paging, I need to limit it like (0,10) (10,10) (20,10) (30,10), correct? If I have 9 books in bookvisit for that category and 3761 books without any visit for that category (total of 3770 books), it should list 377 pages , 10 books on each page. but it does not show any data for some pages because it tries to show books with limit 3760,10 and hence no records for second query in union. May be I am unable to clear the situation here but if you think a bit about the situation, you will get my point.
SELECT * FROM (
SELECT * FROM (
SELECT viewcount, b.isbn, booktitle, stock_status, price, description FROM book AS b
INNER JOIN bookvisit AS bv ON b.isbn = bv.isbn WHERE b.price <> 0 AND hcategoryid = '25'
ORDER BY viewcount DESC
LIMIT 10, 10
) AS t1
UNION
SELECT * FROM
(
SELECT viewcount, b.isbn, booktitle, stock_status, price, description FROM book AS b
LEFT JOIN bookvisit AS bv ON b.isbn = bv.isbn WHERE b.price <> 0 AND hcategoryid = '25'
AND viewcount IS NULL
ORDER BY viewcount DESC
LIMIT 10, 10
) AS t2
)
AS qry
ORDER BY viewcount DESC
LIMIT 10

Answer 1

Do not use limit for the separate queries. Use limit only at the end. You want to get the hole result set from the 2 queries and then show only the 10 results that you need no matter if this is LIMIT 0, 10 or LIMIT 3760,10
SELECT * FROM (
SELECT * FROM (
SELECT viewcount, b.isbn, booktitle, stock_status, price, description FROM book AS b
INNER JOIN bookvisit AS bv ON b.isbn = bv.isbn WHERE b.price <> 0 AND hcategoryid = '25'
ORDER BY viewcount DESC
) AS t1
UNION
SELECT * FROM
(
SELECT viewcount, b.isbn, booktitle, stock_status, price, description FROM book AS b
LEFT JOIN bookvisit AS bv ON b.isbn = bv.isbn WHERE b.price <> 0 AND hcategoryid = '25'
AND viewcount IS NULL
ORDER BY viewcount DESC
) AS t2
)
AS qry
ORDER BY viewcount DESC
LIMIT 10, 10

Answer 2

old one, but still relevant.
Basically, performance wise, you have to use LIMIT on each query involved into UNION, if you know there will be no duplicates between result sets you should consider using UNION ALL, again, performance wise. Then, if you need, lets say, LIMIT 100, 20, you do LIMIT each query with 120 (OFFSET + LIMIT), you are always fetching twice as much records you need, but not all.
SELECT [fields] FROM
(
(SELECT [fields] FROM ... LIMIT 10)
UNION ALL
(SELECT [fields] FROM ... LIMIT 10)
) query
LIMIT 0, 10
5th page
SELECT [fields] FROM
(
(SELECT [fields] FROM ... LIMIT 50)
UNION ALL
(SELECT [fields] FROM ... LIMIT 50)
) query
LIMIT 40, 10

Answer 3

A decade after this question was asked, I can offer a solution, one that perhaps seems obvious to anyone familiar with views: instead of attempting a nested select statement to combine the two tables, use CREATE VIEW (or CREATE OR REPLACE VIEW) to combine the two tables into a view. The speed performance may be poor, as the tables will have to be combined for every page access and may have to be recombined for every pagination, depending on how your code is arranged, but it will work.
If you run into SQL user permissions issues that you and your sysadmin cannot solve, my best advice is to create a new user with full permissions, assign the new user to the table, and use the new user to create the views. That was the only thing that worked for me.