I am trying to retrieve rows from my database table, and prefer they have a provider_count of higher then 0. I want to retrieve records with a provider_count lower then 0 as the last results.
I'm currently using the following query on a table with around 1M records:
SELECT
`products`.*
FROM
`products`
WHERE
`products`.`provider_count` IN
(12,2,3,4,5,6,7,8,9,10,11,13,14,18,19,21,22,42,46,58,0)
GROUP BY
`products`.`search_name`
ORDER BY
FIELD(provider_count, 12,2,3,4,5,6,7,8,9,10,11,13,14,18,19,21,22,42,46,58,0)
LIMIT 48
Unfortunately the order by makes the query really slow, there is already an index on the provider_count column and I've tried adding an index to provider_count+search_name but that didn't show any improvements regarding the speed.
I've also tried to change the order by removing the WHERE statement and changing the ORDER BY to:
ORDER BY `products`.`provider_count` = 0 ASC, `products`.`provider_count`
But that results in quite the same execution time (give or take 5 seconds).
Without the order by, query execution time is only 0.005s
Any suggestions on how I can improve this query?
Schema of my products table:
https://www.db-fiddle.com/f/azBzXiRRsLtXCmLDCVe9DP/0
I'm going to try to answer - it's still not totally clear what your query is supposed to do. The Fiddle you provide doesn't run your query (due to the GROUP BY bug #rickJames mentioned).
I believe your intent is to retrieve every record in the products table, for a list of providers. You want the records where the provider is 0 to be at the end of that list. You want the query to perform quickly.
You should be able to achieve that via this query:
SELECT
`products`.*, 1 as no_provider
FROM
`products`
WHERE
`products`.`provider_count` IN
(12,2,3,4,5,6,7,8,9,10,11,13,14,18,19,21,22,42,46,58)
union
SELECT
`products`.*, 2 as no_provider
FROM
`products`
WHERE
`products`.`provider_count` = 0
order by no_provider, provider_count
LIMIT 48
Your limit clause means that if there are more than 48 records with a provider count > 0, the last clause won't show in your result set.
You could, of course, keep it even simpler:
SELECT
`products`.*
FROM
`products`
WHERE
`products`.`provider_count` IN
(12,2,3,4,5,6,7,8,9,10,11,13,14,18,19,21,22,42,46,58,0)
ORDER BY provider_count desc
I have query like this
UPDATE linksupload as lu SET lu.progress = (SELECT SUM(l.submitted)
FROM links l
WHERE l.upload_id = lu.id)
It takes 10 sec to execute. linksupload contains 10 rows, links contains 700k rows.
Query:
UPDATE linksupload as lu SET lu.progress = (SELECT count(*)
FROM links l
WHERE l.upload_id = lu.id)
takes 0,0003 sek to execute. Also select with sum with group by from first query is fast. upload_id and id are indexes. Why first query takes so long time to execute? How to speed it up?
Indexes allow the database application to find data fast; without reading the whole table.
second query should be just count so it is not reading table. But first query should be sum of submitted column.So it is reading table.
First query should be slow.
I have a slightly complex SQL Query (shown below). This lists all rows of table_p with some example filters applied (in the e.g. below, showing only rows which have rows of table_x reference it):
I'm trying to find a way to efficiently get the number of total results past the LIMIT query. I've tried two options:
Using MYSQL_CALC_ROWS p.id - The result of this greatly increased the time taken to perform it.
Using a sub query, selecting only x_count (for filters) and wrapping it in a COUNT(0) FROM ... parent query. This results in the count query being incredibly slow too (but only slightly faster than the former).
The second option seems the worst in my testing, as the first query can perform very fast until certain filters start getting applied, then it becomes slow anyway. having a second query would really badly hurt speed times.
In this situation, I can only think of using MYSQL_CALC_ROWS as a viable option, but I'm wondering if there's a method I haven't seen or tried yet to improve the efficiency of such a query.
The monster:
SELECT
p.`id`,
-- Demonstrating usage of two joins to the same table.
u.`name`,
u2.`name` AS name2,
COUNT(x.`id`) AS x_count,
-- p has 0 - 1 of u
-- p has 0 - 1 of u2
-- p has 0 - * of x
FROM
`table_p` AS `p`
LEFT JOIN
`table_u` AS `u`
ON
p.`u_id` = u.`id`
LEFT JOIN
`table_u` AS `u2`
ON
p.`u2_id` = u2.`id`
LEFT JOIN
`table_x` AS `x`
ON
p.`id` = x.`p_id`
WHERE
x.`some_column` != 'Some Value'
AND
p.`another_column` <> 'Another Value'
GROUP BY
p.`package_id`
HAVING
`x_count` > 0
ORDER BY
p.id DESC
-- Second page, showing 20 results
LIMIT 20, 20`
Any help that can be provided would be greatly appreciated. I'm building for a system with tens of thousands of rows minimum, so processing time is exponential!
I have a problem with this slow query that runs for 10+ seconds:
SELECT DISTINCT siteid,
storyid,
added,
title,
subscore1,
subscore2,
subscore3,
( 1 * subscore1 + 0.8 * subscore2 + 0.1 * subscore3 ) AS score
FROM articles
WHERE added > '2011-10-23 09:10:19'
AND ( articles.feedid IN (SELECT userfeeds.siteid
FROM userfeeds
WHERE userfeeds.userid = '1234')
OR ( articles.title REGEXP '[[:<:]]keyword1[[:>:]]' = 1
OR articles.title REGEXP '[[:<:]]keyword2[[:>:]]' = 1 ) )
ORDER BY score DESC
LIMIT 0, 25
This outputs a list of stories based on the sites that a user added to his account. The ranking is determined by score, which is made up out of the subscore columns.
The query uses filesort and uses indices on PRIMARY and feedid.
Results of an EXPLAIN:
1 PRIMARY articles
range
PRIMARY,added,storyid
PRIMARY 729263 rows
Using where; Using filesort
2 DEPENDENT SUBQUERY
userfeeds
index_subquery storyid,userid,siteid_storyid
siteid func
1 row
Using where
Any suggestions to improve this query? Thank you.
I would move the calculation logic to the client and only load fields from the database. This makes your query and the calculation itself faster. It's not a good style to do such things in SQL code.
And also is the regex very slow, maybe another searching mode like 'LIKE' is faster.
Looking at your EXPLAIN, it doesn't appear your query is utilizing any index (thus the filesort). This is being caused by the sort on the calculated column (score).
Another barrier is the size of the table (729263 rows). You don't want to create an index that is too wide as it will take much more space and impact performance of your CUD operations. What we want to do is target the columns that are being selected, however, in this situation we can't since it's a calculated column. You can try creating a VIEW or either remove the sort or do it at the application layer.
I'm using mySQL InnoDB tables. The query is simple but it just takes too long to run. Its bound to run even slower once I upload it to a server (not to mention that these two tables will keep growing in size).
The 'author' table size is 3,045 records.
The 'book' table size is 5,278 records.
SELECT author.*, count( auth_id ) AS author_count
FROM author
LEFT JOIN book ON author.id = book.auth_id
GROUP BY author.id
I'm wondering if there is some trick I can employ to make the query run at least twice as fast as it is now (it currently takes about 10.5 seconds to run - an eternity when displaying web content!)
1) Is this returning what you want?
2) You should list in the GROUP BY clause all the fields from the SELECT clause that are not in an agregate function (like COUNT in this case, but also could be AVG or SUM).