SELECT p.id,
p.title,
p.slug,
p.content,
(SELECT url
FROM gallery
WHERE postid = p.id
LIMIT 1) AS url,
t.name
FROM posts AS p
INNER JOIN termrel AS tr
ON ( tr.object = p.id )
INNER JOIN termtax AS tx
ON ( tx.id = tr.termtax_id )
INNER JOIN terms AS t
ON ( t.id = tx.term_id )
WHERE tx.taxonomy_id = 3
AND p.post_status IS NULL
ORDER BY t.name ASC
This query took about 0.2407s to execute. How to make it fast?
Correlated subqueries can have subpar performance as they are executed row by row.
To solve this move your correlated subquery into a regular subquery/derived table and join to it. It will then not have execute row by row for the entire returned result set as it will be executed BEFORE the select statement.
mysql specific links that confirm correaleated subqueries are not optimal choices in mysql.
How to optimize
Answer indicating msql notoriously bad at optimizing correlated subqueries
I use sql-server, but I'm sure the principle is the same for mysql, so I hope this at least points you in the right direction. You would need to partition/return your one result per loan, maybe some could chime in on mysql specific syntax and I could update my answer
select
p.id
,p.title
,p.slug
,p.content
,t.name
,mySubQuery.value
from
posts as p
inner join termrel as tr
on ( tr.object = p.id )
inner join termtax as tx
on ( tx.id = tr.termtax_id )
inner join terms as t
on ( t.id = tx.term_id )
left join (
-- use MYSQL function to partition the reslts and only return 1, I use sql-server, not sure of the RDMS specific syntax
select
id
,url
from
gallery
limit 1
) as mySubquery
on mySubquery.id = p.id
where
tx.taxonomy_id = 3
and p.post_status is null
order by
t.name asc
Related
My query gets the results of these products based on if they exist in a separate table index. I am trying to get a count of all the instances where they exist so I can ORDER the results by relevance. Everything I try seems to return the variable #priority as 0. Any ideas?
Maybe it is better to use join statements?
Thank you for your help. Here is my MySQL query:
SELECT `products` . * , #priority
FROM `products`
LEFT JOIN productstypes_index ON productstypes_index.product_id = products.id
WHERE (
EXISTS (
SELECT *
FROM `productstypes_index`
WHERE `productstypes_index`.`product_id` = `products`.`id`
AND `productstypes_index`.`_type_id` = '1'
)
AND (
(
(
EXISTS (
SELECT #priority := COUNT( * )
FROM `producthashtags_index`
WHERE `producthashtags_index`.`product_id` = `products`.`id`
AND `producthashtags_index`.`producthashtag_id` = '43'
)
)
AND (
EXISTS (
SELECT #priority := COUNT( * )
FROM `producthashtags_index`
WHERE `producthashtags_index`.`product_id` = `products`.`id`
AND `producthashtags_index`.`producthashtag_id` = '11'
)
)
)
)
)
ORDER BY `updated_at` DESC;
You could do without those exists, and without variables. Also, a left join has no sense if you have an exists condition on the joined table. Then you might as well do the more efficient inner join and put the extra type condition in the join condition.
The priority can be calculated by a count over the hash tags, but only those with id in ('43', '11').
SELECT products.*
count(distinct producthashtags_index.producthashtag_id) priority
FROM products
INNER JOIN productstypes_index
ON productstypes_index.product_id = products.id
AND productstypes_index._type_id = '1'
INNER JOIN producthashtags_index
ON producthashtags_index.product_id = products.id
AND producthashtags_index.producthashtag_id in ('43', '11')
GROUP BY products.id
ORDER BY updated_at DESC;
MySQL ignores the SELECT list in EXISTS subquery, so it makes no difference what you type in there. This is documented here.
An approach using joins would look like below:
SELECT p.id,
COUNT(case when phi.product_id is not null then 1 end) AS instances
FROM products p
INNER JOIN productstypes_index pti ON pti.product_id = p.id AND pti.`_type_id` = 1
LEFT JOIN producthashtags_index phi ON phi.product_id = p.id AND phi.producthashtag_id IN (11,43)
GROUP BY p.id
ORDER BY instances DESC;
I have removed additional backticks where I believe they are not neccessary and also if your id columns in tables are integers, you do not need quotation marks.
I've looked at similar group_concat mysql optimisation threads but none seem relevant to my issue, and my mysql knowledge is being stretched with this one.
I have been tasked with improving the speed of a script with an extremely heavy Mysql query contained within.
The query in question uses GROUP_CONCAT to create a list of colours, tags and sizes all relevant to a particular product. It then uses HAVING / FIND_IN_SET to filter these concatenated lists to find the attribute, set by the user controls and display the results.
In the example below it's looking for all products with product_tag=1, product_colour=18 and product_size=17. So this could be a blue product (colour) in medium (size) for a male (tag).
The shop_products tables contains about 3500 rows, so is not particularly large, but the below takes around 30 seconds to execute. It works OK with 1 or 2 joins, but adding in the third just kills it.
SELECT shop_products.id, shop_products.name, shop_products.default_image_id,
GROUP_CONCAT( DISTINCT shop_product_to_colours.colour_id ) AS product_colours,
GROUP_CONCAT( DISTINCT shop_products_to_tag.tag_id ) AS product_tags,
GROUP_CONCAT( DISTINCT shop_product_colour_to_sizes.tag_id ) AS product_sizes
FROM shop_products
LEFT JOIN shop_product_to_colours ON shop_products.id = shop_product_to_colours.product_id
LEFT JOIN shop_products_to_tag ON shop_products.id = shop_products_to_tag.product_id
LEFT JOIN shop_product_colour_to_sizes ON shop_products.id = shop_product_colour_to_sizes.product_id
WHERE shop_products.category_id = '50'
GROUP BY shop_products.id
HAVING((FIND_IN_SET( 1, product_tags ) >0)
AND(FIND_IN_SET( 18, product_colours ) >0)
AND(FIND_IN_SET( 17, product_sizes ) >0))
ORDER BY shop_products.name ASC
LIMIT 0 , 30
I was hoping somebody could generally advise a better way to structure this query without re-structuring the database (which isn't really an option at this point without weeks of data migration and script changes)? Or any general advise on optimisation. Using explain currently returns the below (as you can see the indexes are all over the place!).
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE shop_products ref category_id,category_id_2 category_id 2 const 3225 Using where; Using temporary; Using filesort
1 SIMPLE shop_product_to_colours ref product_id,product_id_2,product_id_3 product_id 4 candymix_db.shop_products.id 13
1 SIMPLE shop_products_to_tag ref product_id,product_id_2 product_id 4 candymix_db.shop_products.id 4
1 SIMPLE shop_product_colour_to_sizes ref product_id product_id 4 candymix_db.shop_products.id 133
Rewrite query to use WHERE instead of HAVING. Because WHERE is applied when MySQL performs search on rows and it can use index. HAVING is applied after rows are selected to filter already selected result. HAVING by design can't use indexes.
You can do it, for example, this way:
SELECT p.id, p.name, p.default_image_id,
GROUP_CONCAT( DISTINCT pc.colour_id ) AS product_colours,
GROUP_CONCAT( DISTINCT pt.tag_id ) AS product_tags,
GROUP_CONCAT( DISTINCT ps.tag_id ) AS product_sizes
FROM shop_products p
JOIN shop_product_to_colours pc_test ON p.id = pc_test.product_id AND pc_test.colour_id = 18
JOIN shop_products_to_tag pt_test ON p.id = pt_test.product_id AND pt_test.tag_id = 1
JOIN shop_product_colour_to_sizes ps_test ON p.id = ps_test.product_id AND ps_test.tag_id = 17
JOIN shop_product_to_colours pc ON p.id = pc.product_id
JOIN shop_products_to_tag pt ON p.id = pt.product_id
JOIN shop_product_colour_to_sizes ps ON p.id = ps.product_id
WHERE p.category_id = '50'
GROUP BY p.id
ORDER BY p.name ASC
Update
We are joining each table two times.
First to check if it contains some value (condition from FIND_IN_SET).
Second join will produce data for GROUP_CONCAT to select all product values from table.
Update 2
As #Matt Raines commented, if we don't need list product values with GROUP_CONCAT, query becomes even simplier:
SELECT p.id, p.name, p.default_image_id
FROM shop_products p
JOIN shop_product_to_colours pc ON p.id = pc.product_id
JOIN shop_products_to_tag pt ON p.id = pt.product_id
JOIN shop_product_colour_to_sizes ps ON p.id = ps.product_id
WHERE p.category_id = '50'
AND (pc.colour_id = 18 AND pt.tag_id = 1 AND ps.tag_id = 17)
GROUP BY p.id
ORDER BY p.name ASC
This will select all products with three filtered attributes.
I think if I understand this question, what you need to do is:
Find a list of all of the shop_product.id's that have the correct tag/color/size options
Get a list of all of the tag/color/size combinations available for that product id.
I was trying to make you a SQLFiddle for this, but the site seems broken at the moment. Try something like:
SELECT shop_products.id, shop_products.name, shop_products.default_image_id,
GROUP_CONCAT( DISTINCT shop_product_to_colours.colour_id ) AS product_colours,
GROUP_CONCAT( DISTINCT shop_products_to_tag.tag_id ) AS product_tags,
GROUP_CONCAT( DISTINCT shop_product_colour_to_sizes.tag_id ) AS product_sizes
FROM
shop_products INNER JOIN
(SELECT shop_products.id id,
FROM
shop_products
LEFT JOIN shop_product_to_colours ON shop_products.id = shop_product_to_colours.product_id
LEFT JOIN shop_products_to_tag ON shop_products.id = shop_products_to_tag.product_id
LEFT JOIN shop_product_colour_to_sizes ON shop_products.id = shop_product_colour_to_sizes.product_id
WHERE
shop_products.category_id = '50'
shop_products_to_tag.tag_id=1
shop_product_to_colours.colour_id=18
shop_product_colour_to_sizes.tag_id=17
) matches ON shop_products.id = matches.id
LEFT JOIN shop_product_to_colours ON shop_products.id = shop_product_to_colours.product_id
LEFT JOIN shop_products_to_tag ON shop_products.id = shop_products_to_tag.product_id
LEFT JOIN shop_product_colour_to_sizes ON shop_products.id = shop_product_colour_to_sizes.product_id
GROUP BY shop_products.id
ORDER BY shop_products.name ASC
LIMIT 0 , 30;
The problem with you first approach is that it requires the database to create every combination of every product and then filter. In my example, I'm filtering down the product id's first then generating the combinations.
My query is untested as I don't have a MySQL Environment handy and SQLFiddle is down, but it should give you the idea.
First, I aliased your queries to shorten readability.
SP = Shop_Products
PC = Shop_Products_To_Colours
PT = Shop_Products_To_Tag
PS = Shop_Products_To_Sizes
Next, your having should be a WHERE since you are explicitly looking FOR something. No need trying to query the entire system just to throw records after the result is returned. Third, you had LEFT-JOIN, but when applicable to a WHERE or HAVING, and you are not allowing for NULL, it forces TO a JOIN (both parts required). Finally, your WHERE clause has quotes around the ID you are looking for, but that is probably integer anyhow. Remove the quotes.
Now, for indexes and optimization there. To help with the criteria, grouping, and JOINs, I would have the following composite indexes (multiple fields) instead of a table with just individual columns as the index.
table index
Shop_Products ( category_id, id, name )
Shop_Products_To_Colours ( product_id, colour_id )
Shop_Products_To_Tag ( product_id, tag_id )
Shop_Products_To_Sizes ( product_id, tag_id )
Revised query
SELECT
SP.id,
SP.name,
SP.default_image_id,
GROUP_CONCAT( DISTINCT PC.colour_id ) AS product_colours,
GROUP_CONCAT( DISTINCT PT.tag_id ) AS product_tags,
GROUP_CONCAT( DISTINCT PS.tag_id ) AS product_sizes
FROM
shop_products SP
JOIN shop_product_to_colours PC
ON SP.id = PC.product_id
AND PC.colour_id = 18
JOIN shop_products_to_tag PT
ON SP.id = PT.product_id
AND PT.tag_id = 1
JOIN shop_product_colour_to_sizes PS
ON SP.id = PS.product_id
AND PS.tag_id = 17
WHERE
SP.category_id = 50
GROUP BY
SP.id
ORDER BY
SP.name ASC
LIMIT
0 , 30
One Final comment. Since you are ordering by the NAME, but grouping by the ID, it might cause a delay in the final sorting. HOWEVER, if you change it to group by the NAME PLUS ID, you will still be unique by the ID, but an adjusted index ON your Shop_Products to
table index
Shop_Products ( category_id, name, id )
will help both the group AND order since they will be in natural order from the index.
GROUP BY
SP.name,
SP.id
ORDER BY
SP.name ASC,
SP.ID
In mysql query I use order by, but it is not working.
When I do this
SELECT t.id,t.user_id,t.title,c.comment,d.has_answer,IF(c.id IS NULL, t.date_created, d.recent_date) recent_date,MIN(i.id) image_id
FROM threads t
LEFT JOIN comments c ON c.thread_id = t.id
INNER JOIN (
SELECT thread_id, MAX(date_sent) recent_date, MAX(is_answer) has_answer
FROM comments
GROUP BY thread_id
) d ON c.id IS NULL OR (d.thread_id = c.thread_id AND d.recent_date = c.date_sent)
LEFT JOIN thread_images i ON t.id = i.thread_id
WHERE t.user_id = t.user_id
GROUP BY t.id
ORDER BY d.recent_date DESC
LIMIT 0, 10
It doesn't properly order them. But if I do this:
SELECT *
FROM (
SELECT t.id,t.user_id,t.title,c.comment,d.has_answer,IF(c.id IS NULL, t.date_created, d.recent_date) recent_date,MIN(i.id) image_id
FROM threads t
LEFT JOIN comments c ON c.thread_id = t.id
INNER JOIN (
SELECT thread_id, MAX(date_sent) recent_date, MAX(is_answer) has_answer
FROM comments
GROUP BY thread_id
) d ON c.id IS NULL OR (d.thread_id = c.thread_id AND d.recent_date = c.date_sent)
LEFT JOIN thread_images i ON t.id = i.thread_id
WHERE t.user_id = t.user_id
GROUP BY t.id
LIMIT 0, 10) qwerty
ORDER BY recent_date DESC
Then it does work. Why does the top one not work, and is the second way the best way to fix that?
Thanks
Those two statements are ordering by two different things.
The second statement is ordering by the result of an expression in the SELECT list.
But the first statement specifies ordering by a value of recent_date returned by the inline view d; if you remove "d." from in front of recent_date, then the ORDER BY clause would reference the alias assigned to the expression in the SELECT list, as the second statement does.
Because recent_date is an alias for an expression the SELECT list, these two are equivalent:
ORDER BY recent_date
ORDER BY IF(c.id IS NULL, t.date_created, d.recent_date)
^^
but those are significantly different from:
ORDER BY d.recent_date
^^
Note that the non-standard use of the GROUP BY clause may be masking some values of recent_date which are discarded by the query. This usage of the GROUP BY clause is a MySQL extension to the SQL Standard; most other relational databases would throw an error with this statement. It's possible to get MySQL to throw the same type of error by enabling the ONLY_FULL_GROUP_BY SQL mode.
Q Is the second statement the best way to fix that?
A If that statement guarantees that the resultset returned meets your specification, then it's a workable approach. (One downside is the overhead of the inline view query.)
But I strongly suspect that the second statement is really just masking the problem, not really fixing it.
SELECT t.id,t.user_id,t.title,c.comment,d.has_answer,IF(c.id IS NULL, t.date_created, d.recent_date) recent_date,MIN(i.id) image_id
FROM (threads t
LEFT JOIN comments c ON c.thread_id = t.id
INNER JOIN (
SELECT thread_id, MAX(date_sent) recent_date, MAX(is_answer) has_answer
FROM comments
GROUP BY thread_id
) d ON c.id IS NULL OR (d.thread_id = c.thread_id AND d.recent_date = c.date_sent)
LEFT JOIN thread_images i ON t.id = i.thread_id
WHERE t.user_id = t.user_id
GROUP BY t.id
LIMIT 0, 10) x
ORDER BY d.recent_date DESC
Here is a sample SQL dump: https://gist.github.com/JREAM/99287d033320b2978728
I have a SELECT that grabs a bundle of users.
I then do a foreach loop to attach all the associated tree_processes to that user.
So I end up doing X Queries: users * tree.
Wouldn't it be much more efficient to fetch the two together?
I've thought about doing a LEFT JOIN Subselect, but I'm having a hard time getting it correct.
Below I've done a query to select the correct data in the SELECT, however I would have to do this for all 15 rows and it seems like a TERRIBLE waste of memory.
This is my dirty Ateempt:
-
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
(
SELECT tp.id FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS newest_tree_id,
#
# Don't want to have to do this below for every row
(
SELECT t.type FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS tree_type
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
This is my LEFT JOIN attempt, but I am having trouble getting the SELECT values
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
freshness.id,
# freshness.subscribers_id < -- Cant get multiples out of the LEFT join
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
LEFT JOIN ( SELECT tp.id, tp.subscribers_id AS tp FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
ORDER BY tp.id DESC
LIMIT 1 ) AS freshness
ON (
s.id = subscribers_id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
In the LEFT JOIN you are using 'freshness' as the table alias. This in you select you need to additionally state what column(s) you want from it. Since there is only one column (id) you need to add:
freshness.id
to the select clause.
Your ON clause of the left join looks pretty dodgy too. Maybe freshness.id = ss.subscribers_id?
Cheers -
I have three tables - tblpollquestions, tblpollanswers and tblpollresponses.
I want to select a random question that a user hasn't responded to yet, with the respective answers.
The SQL below returns exactly what I need, but I'm concerned that it takes three SELECTs to do it. There must surely be a more efficient way?
SELECT
poll.id,
poll.question,
a.answer
FROM tblpollquestions poll
INNER JOIN tblpollanswers a ON a.question_id = poll.id
INNER JOIN (
SELECT id FROM tblpollquestions WHERE id NOT IN(
SELECT question_id FROM tblpollresponses WHERE user_id = 1
) ORDER BY RAND() LIMIT 1
) as t ON t.id = poll.id
This could be made a bit better by switching NOT IN(SELECT...) into LEFT JOIN
SELECT
poll.id,
poll.question,
a.answer
FROM
tblpollquestions poll
INNER JOIN
tblpollanswers a
ON
a.question_id = poll.id
INNER JOIN (
SELECT
q.id
FROM
tblpollquestions AS q
LEFT JOIN
tblpollresponses AS r
ON
q.id = r.question_id
AND r.user_id = 1
WHERE
r.question_id IS NULL
ORDER BY RAND() LIMIT 1
) as t ON t.id = poll.id
ORDER BY RAND() can also be slow if there are many rows in tblpollquestions table. See this presentation from Bill Karwin (slide 142 and onwards) for some other ideas on selecting a random row.
http://www.slideshare.net/billkarwin/sql-antipatterns-strike-back
Is seems fine to me, although I would change it slightly:
SELECT
poll.id,
poll.question,
a.answer
FROM tblpollquestions poll
INNER JOIN tblpollanswers a ON a.question_id = poll.id
WHERE poll.id = (
SELECT id FROM tblpollquestions WHERE NOT EXISTS (
SELECT * FROM tblpollresponses WHERE user_id = 1 AND question_id = tblpollquestions.id )
ORDER BY RAND() LIMIT 1)
Written that way should do a better job of using indexes, and not checking the join conditions for every single tblpollanswers.
Make sure you have a UNIQUE index (or primary key) on tblpollresponses for (user_id, question_id) (in that order). If you need it for other queries, you can add an additional UNIQUE index with the columns in the reverse order.
Edit: Actually putting it in the where might not be so good http://jan.kneschke.de/projects/mysql/order-by-rand/ You will need to explain the query and compare.
Use left join like this:
SELECT ques.id, ques.question, ans.answer FROM tblpollquestions ques
INNER JOIN tblpollanswers ans ON(ans.question_id = ques.id)
left join tblpollresponses res on(res.question_id=ques.id and user_id = 1)
where res.question_id is null ORDER BY RAND() LIMIT 1;
I changed your table aliases to make better sense.