Use limit in sub query - mysql

I have 4 tables
product
status
orders
orders_products.
Using following query I get the proper result but it is not in correct order i.e. result gets by sub query.I want result order as sub query result.
SELECT
p.products_id, p.products_image, p.products_tax_class_id,
pd.products_name, if(s.status, s.specials_new_products_price,
p.products_price) as products_price
FROM
products p LEFT JOIN specials s
ON p.products_id = s.products_id, products_description pd
WHERE
p.products_status = '1'
AND p.products_id = pd.products_id
AND pd.language_id = '2'
AND p.products_id in (
SELECT * from (
SELECT
distinct(op.products_id)
FROM
orders_products op,orders o
WHERE
op.orders_id=o.orders_id
AND o.customers_id='27'
ORDER BY
o.date_purchased
DESC LIMIT 0,10
) AS temptable
)

The outer SELECT has an ORDER BY pd.products_name. That's what should apply, and only that.
The sub-query ORDER BY shouldn't actually matter. You should get the same result removing it.
If you want a different sort order from pd.products_name, you have to ORDER BY the column required. In your case, you need to move orders.date_purchased to the main part of the query so you can use it to sort. Inverting the query, i.e. starting with what you want and relating everything else:
SELECT MAX(o.date_purchased) AS date_purchased, DISTINCT (op.products_id) AS products_id
p.products_id, p.products_image, p.products_tax_class_id, p.products_price
pd.products_name,
IF (s.status, s.specials_new_product_price, p.products_price) AS products_price
FROM orders o
LEFT JOIN orders_products op USING (orders_id)
LEFT JOIN products p USING (products_id)
LEFT JOIN products_description pd USING (products_id)
LEFT JOIN specials s USING (products_id)
WHERE o.customers_id = '27'
AND p.products_status = 1
AND pd.language_id = 2

Related

Why is MySQL select count(*) so slow on fast subquery?

I have the following query that executes quite fast:
SELECT DISTINCT p.products_id,
p.products_image,
p.products_quantity,
p.products_status,
m.manufacturers_id,
p.products_date_added,
p.products_subimage1,
pd.products_name,
p.products_price,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
IF(s.status, s.specials_new_products_price, NULL) AS
specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS
final_price,
IF(clearance_price < products_cost * 2.25,
clearance_price,
products_cost * 2.25) AS
sorting_price
FROM
(SELECT products_id ,
IF(clearance_price < products_cost * 2.25,
clearance_price,
products_cost * 2.25) AS
sorting_price
FROM `products`
ORDER BY products_id DESC)q,
products p
left join manufacturers m USING(manufacturers_id)
left join specials s
ON p.products_id = s.products_id
left join products_attributes pa
ON p.products_id = pa.products_id
left join products_options po
ON pa.options_id = po.products_options_id
left join products_options_values pov
ON pa.options_values_id = pov.products_options_values_id,
products_description pd,
categories c,
products_to_categories p2c
WHERE
q.products_id = p.products_id
AND q.sorting_price = sorting_price
AND
p.products_status = '1'
AND p.products_id = pd.products_id
AND pd.language_id = '1'
AND p.products_id = p2c.products_id
AND p2c.categories_id = c.categories_id
AND (( pd.products_name LIKE '%a%'
OR po.products_options_name LIKE '%a%'
OR pov.products_options_values_name LIKE '%a%'
OR pd.products_description LIKE '%a%' ))
When I wrap it inside a count(*) query, the new count query takes 10 to 15 times more time, very slow.
I wrap it like so:
SELECT count(*) as total from (
SELECT DISTINCT p.products_id,
p.products_image,
p.products_quantity,
p.products_status,
m.manufacturers_id,
p.products_date_added,
p.products_subimage1,
pd.products_name,
p.products_price,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
IF(s.status, s.specials_new_products_price, NULL) AS
specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS
final_price,
IF(clearance_price < products_cost * 2.25,
clearance_price,
products_cost * 2.25) AS
sorting_price
FROM
(SELECT products_id ,
IF(clearance_price < products_cost * 2.25,
clearance_price,
products_cost * 2.25) AS
sorting_price
FROM `products`
ORDER BY products_id DESC)q,
products p
left join manufacturers m USING(manufacturers_id)
left join specials s
ON p.products_id = s.products_id
left join products_attributes pa
ON p.products_id = pa.products_id
left join products_options po
ON pa.options_id = po.products_options_id
left join products_options_values pov
ON pa.options_values_id = pov.products_options_values_id,
products_description pd,
categories c,
products_to_categories p2c
WHERE
q.products_id = p.products_id
AND q.sorting_price = sorting_price
AND
p.products_status = '1'
AND p.products_id = pd.products_id
AND pd.language_id = '1'
AND p.products_id = p2c.products_id
AND p2c.categories_id = c.categories_id
AND (( pd.products_name LIKE '%a%'
OR po.products_options_name LIKE '%a%'
OR pov.products_options_values_name LIKE '%a%'
OR pd.products_description LIKE '%a%' ))
) AS derivedtable1
Why does this happen? Is there any way to optimize this?
EDIT:
This is the EXPLAIN EXTENDED of the first query:
This is the EXPLAIN EXTENDED of the count (the second) query:
This is the PROFILING of the first query:
This is the PROFILING of the count (the second) query:
FROM pd
LEFT JOIN po ON ...
WHERE ( ... OR po.x LIKE '...' OR ... )
The semantics of LEFT say that it does not matter whether there is a matching row in po. Hence the LIKE has zero impact on the outcome of the query. Suggest you get rid of OR ... LIKE of any items in LEFT. Or maybe you did not really mean LEFT? That could make it run faster, but the result set could be different.
I see from the EXPLAINs that there is no good excuse for the difference in timing. Oh, how are you running the queries? The first one has 213K rows of output -- are you waiting for all of it to come back before stopping your stopwatch?

Slow performance of combined MySQL queries

I have this query with multiple subqueries which runs quite slow.
SELECT DISTINCT pav.products_options_values_id,
pav.products_options_values_name,
pav.products_options_values_sort_order
FROM products_stock ps,
products_options_values pav,
(
SELECT DISTINCT pa.products_id,
pov.products_options_values_id,
pov.products_options_values_name,
pa.options_values_price,
pa.price_prefix
FROM products_attributes pa,
products_options_values pov,
(
SELECT DISTINCT p.products_image,
p.products_quantity,
p.products_status,
m.manufacturers_id,
p.products_id,
p.products_date_added,
p.products_subimage1,
pd.products_name,
p.products_price,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,IF(s.status, s.specials_new_products_price, NULL) AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS final_price,
IF(p.clearance_price < p.products_cost*2.25, p.clearance_price, p.products_cost*2.25) AS sorting_price
FROM products p
LEFT JOIN manufacturers m
using (manufacturers_id)
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
LEFT JOIN products_options po
ON pa.options_id = po.products_options_id
LEFT JOIN products_options_values pov
ON pa.options_values_id = pov.products_options_values_id ,
products_description pd,
categories c,
products_to_categories p2c
WHERE p.products_status = '1'
AND p.products_id = pd.products_id
AND pd.language_id = '1'
AND p.products_id = p2c.products_id
AND p2c.categories_id = c.categories_id
AND ((
pd.products_name LIKE '%a%'
OR po.products_options_name LIKE '%a%'
OR pov.products_options_values_name LIKE '%a%'
OR pd.products_description LIKE '%a%') )
ORDER BY p.products_id DESC) m
WHERE m.products_id = pa.products_id
AND pa.options_id = 1
AND pa.options_values_id = pov.products_options_values_id
AND pov.language_id = '1'
GROUP BY pov.products_options_values_id
ORDER BY pov.products_options_values_sort_order ASC) q
WHERE q.products_id = ps.products_id
AND ps.products_stock_attributes = concat('1-', pav.products_options_values_id)
AND ps.products_stock_quantity > 0
ORDER BY pav.products_options_values_sort_order ASC
This is EXPLAIN result:
Been trying to optimize it for hours, but I probably misread the EXPLAIN information because no matter what I do, it doesn't seem to make it faster, so I am reaching out for help from the experts here.
What can be the cause of it becoming so slow and what should I do to make it fast?
First, I cleaned up the query using consistent JOIN clauses vs comma listed tables. Next, your inner-most query has an order by which apparently is not being utilized to any benefit. Order by will typically cause big delay in queries if it cant be optimized.
SELECT DISTINCT
pav.products_options_values_id,
pav.products_options_values_name,
pav.products_options_values_sort_order
FROM
products_stock ps
JOIN products_options_values pav
ON ps.products_stock_attributes = concat('1-', pav.products_options_values_id)
AND ps.products_stock_quantity > 0
JOIN ( SELECT DISTINCT
pa.products_id,
pov.products_options_values_id,
pov.products_options_values_name,
pa.options_values_price,
pa.price_prefix
FROM
products_attributes pa
JOIN products_options_values pov
ON pa.options_values_id = pov.products_options_values_id
AND pov.language_id = '1'
JOIN ( SELECT DISTINCT
p.products_image,
p.products_quantity,
p.products_status,
p.manufacturers_id,
p.products_id,
p.products_date_added,
p.products_subimage1,
pd.products_name,
p.products_price,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
IF(s.status, s.specials_new_products_price, NULL )
AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price )
AS final_price,
IF( p.clearance_price < p.products_cost * 2.25, p.clearance_price, p.products_cost * 2.25 )
AS sorting_price
FROM
products p
JOIN products_description pd
ON p.products_id = pd.products_id
AND pd.language_id = '1'
THIS SECTION CAN BE REMOVED JOIN products_to_categories p2c
AND p.products_id = p2c.products_id
JOIN categories c
ON p2c.categories_id = c.categories_id
LEFT JOIN manufacturers m
UP TO THIS LINE ON P.manufacturers_id = m.manufacturers_id
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
LEFT JOIN products_options po
ON pa.options_id = po.products_options_id
LEFT JOIN products_options_values pov
ON pa.options_values_id = pov.products_options_values_id,
WHERE
p.products_status = '1'
AND ( pd.products_name LIKE '%a%'
OR pd.products_description LIKE '%a%'
OR po.products_options_name LIKE '%a%'
OR pov.products_options_values_name LIKE '%a%' )
ORDER BY
p.products_id DESC) m
WHERE
pa.products_id = m.products_id
AND pa.options_id = 1
GROUP BY
pov.products_options_values_id
ORDER BY
pov.products_options_values_sort_order ASC) q
ON ps.products_id = q.products_id
ORDER BY
pav.products_options_values_sort_order ASC
You have a join to your categories table, but not pulling any values or other criteria to return columns. I have thus removed as it is unnecessary. WITH that gone, I then looked at your Products_To_Categories table AND that too has no bearing in the query other than an extra join not being used anywhere else and thus removed. The Manufacturers table is also not required as all you are getting is the manufacturer's ID, which exists on the products table, so yet ANOTHER table not required
Now on to indexes. I would look at the following indexes for your tables
table index
products ( products_status, products_id )
products_description ( products_id, language_id )
products_to_categories ( products_id, categories_id )
categories ( categories_id )
manufacturers ( manufacturers_id )
specials ( products_id, status, specials_new_products_price )
products_attributes ( products_id, options_id, options_values_id )
products_options ( products_options_id, products_options_name )
products_options_values ( products_options_values_id, products_options_values_name )
This is just a first pass at the review. Now I want to take it one additional step to possibly remove a layer of nested queries. Your INNER-MOST query does a join to "products_options_values" but ONLY for Language ID = 1. Why would you not add that column criteria to the inner query. Then add the pov.products_options_values_id and pov.products_options_values_name columns to your inner query and language ID to the inner query and you don't need to re-join to them again. They can be referenced via the "m." alias of the inner-most query. vs the rejoin.
Similarly, your inner-most query does a LEFT JOIN to the products_attributes, but outside the "m" alias query result applies a WHERE clause ONLY of options_id = 1. This doesn't make sense. Why not just add that too to the inner-most query. Eliminate the extra levels would probably help a lot as you are limiting down a pull of all records if you are only looking for things like Language ID = 1, or Options_ID = 1.
Again, this formatted query IS the same context as yours, just structured slightly different and shorter indentation to see better the context and nested requirements.
FEEDBACK FROM CHAT
select
PQ.*
from
( SELECT STRAIGHT_JOIN DISTINCT
p.products_id,
p.products_image,
p.products_quantity,
p.products_status,
p.manufacturers_id,
p.products_date_added,
p.products_subimage1,
pd.products_name,
p.products_price,
p.products_length,
p.products_width,
p.products_height,
pov.products_options_values_id,
pov.products_options_values_name,
pov.language_id,
pov.products_options_values_sort_order,
p.products_tax_class_id,
IF(s.status, s.specials_new_products_price, NULL)
AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price)
AS final_price,
IF(p.clearance_price < p.products_cost*2.25, p.clearance_price, p.products_cost*2.25)
AS sorting_price
FROM
products_attributes pa
JOIN products_options_values pov
ON pa.options_values_id = pov.products_options_values_id
AND pov.language_id = '1'
LEFT JOIN products_options po
ON pa.options_id = po.products_options_id
JOIN products p
ON pa.products_id = p.products_id
AND p.products_status = '1'
JOIN products_description pd
ON p.products_id = pd.products_id
AND pd.language_id = '1'
JOIN products_to_categories p2c
ON p.products_id = p2c.products_id
JOIN categories c
ON p2c.categories_id = c.categories_id
LEFT JOIN manufacturers m
using (manufacturers_id)
LEFT JOIN specials s
ON p.products_id = s.products_id
WHERE
pa.options_id = '1'
AND ( pd.products_name LIKE '%a%'
OR po.products_options_name LIKE '%a%'
OR pov.products_options_values_name LIKE '%a%'
OR pd.products_description LIKE '%a%')) PQ
order by
PQ.Products_id
Also, note, I removed the final "Order by" clause

Combine 2 MySQL queries to improve performance

I have the following 2 queries.
The first is finding some product_ids based on some joins. It's running fast.
SELECT Group_concat(DISTINCT( p.products_id )) AS comma_separated
FROM products p
left join specials s
ON p.products_id = s.products_id
left join products_to_categories p2c
ON p.products_id = p2c.products_id
left join products_description pd
ON p.products_id = pd.products_id
inner join products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND Date_sub(Curdate(), interval 7000 day) <= p.products_date_added
ORDER BY p.products_id DESC
After that, the following query is executed and use all the products_ids found in the first query:
SELECT DISTINCT pov.products_options_values_id,
pov.products_options_values_name,
pa.options_values_price,
pa.price_prefix
FROM products_attributes pa,
products_options_values pov
WHERE Find_in_set(pa.products_id,
'**Long list of comma separated products_ids found in the first query**')
AND pa.options_id = 1
AND pa.options_values_id = pov.products_options_values_id
AND pov.language_id = '1'
GROUP BY pov.products_options_values_id
ORDER BY pov.products_options_values_sort_order ASC
The list of products_ids in the Find_in_set clause is very long and making this query take quite a long time to execute.
Is there any way of using joins or other way to rewrite the second query or combine both to improve performance?
Try something like this
SELECT DISTINCT pov.products_options_values_id,
pov.products_options_values_name,
pa.options_values_price,
pa.price_prefix
FROM products_attributes pa,
products_options_values pov,
(SELECT p.products_id
FROM products p
left join specials s
ON p.products_id = s.products_id
left join products_to_categories p2c
ON p.products_id = p2c.products_id
left join products_description pd
ON p.products_id = pd.products_id
inner join products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND Date_sub(Curdate(), interval 7000 day) <= p.products_date_added
GROUP BY p.products_id
) t
WHERE t.products_id = pa.products_id
AND pa.options_id = 1
AND pa.options_values_id = pov.products_options_values_id
AND pov.language_id = '1'
GROUP BY pov.products_options_values_id
ORDER BY pov.products_options_values_sort_order ASC

Limit not working in INNER JOINS

Am sure this question has already been asked, but I can't help but wonder why the "LIMIT 3" in one of the INNER JOINS is only returning a single imgUrl for the specified avp.productID in the "ON clause"! Is that LIMITs do not work in INNER JOINs? What better way could I use to achieve this please?
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From
products AS p
INNER JOIN(
Select
c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM
availColors AS c GROUP BY c.productID) AS colors
ON
p.productID = colors.productID
INNER JOIN(
SELECT
s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM
availSizes AS s
GROUP BY
s.productID
) AS sizes
ON
p.productID = sizes.productID
INNER JOIN(
SELECT
avp.productID,
avp.productImg
FROM
availImg AS avp
ORDER BY
avp.productID
LIMIT 3) AS images
ON
images.productID = p.productID
WHERE
p.productID = ?
GROUP BY
p.productID";
Your current query has a sub query that returns 3 images, ordered by product id. This sub query doesn't check if the images belong to the product you are searching for so could easily bring back 3 images for a different product. This would result in the main query trying to do an inner join onto the sub query based on product id and finding no matching records.
As your 2 other sub queries only bring back a single row for each product id (due to the GROUP_CONCAT) and you appear to only want to check one product id at a time then I think you can solve this by just doing a normal join onto the images table (without a sub query) and putting the limit on the main query.
Like this:-
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From products AS p
INNER JOIN
(
Select c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM availColors AS c
GROUP BY c.productID
) AS colors
ON p.productID = colors.productID
INNER JOIN
(
SELECT s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM availSizes AS s
GROUP BY s.productID
) AS sizes
ON p.productID = sizes.productID
INNER JOIN availImg images
ON images.productID = p.productID
WHERE p.productID = ?
ORDER BY images.productID, images.productImg
LIMIT 3";
However if you want to bring back multiple product ids then it becomes a bit more complicated. One solution would be to bring back the images with a generated sequence number for each product, and then check in the ON clause of the join that it is one of the first 3 images for a product:-
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From products AS p
INNER JOIN
(
Select c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM availColors AS c
GROUP BY c.productID
) AS colors
ON p.productID = colors.productID
INNER JOIN
(
SELECT s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM availSizes AS s
GROUP BY s.productID
) AS sizes
ON p.productID = sizes.productID
INNER JOIN
(
SELECT avp.productID,
avp.productImg,
#seq:=IF(#prev_productID = avp.productID, #seq + 1, 1) AS seq,
#prev_productID := avp.productID
FROM availImg AS avp
CROSS JOIN (SELECT #prev_productID:=0, #seq:=0) sub1
ORDER BY avp.productID
) AS images
ON images.productID = p.productID
AND images.seq <= 3
WHERE p.productID = ?";
Another possible method would be to use GROUP_CONCAT to get a comma separated list of image ids for each product, with SUBSTRING_INDEX to get the first 3 of these, then join that against the images table using FIND_IN_SET to get the rest of the image details:-
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From products AS p
INNER JOIN
(
Select c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM availColors AS c
GROUP BY c.productID
) AS colors
ON p.productID = colors.productID
INNER JOIN
(
SELECT s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM availSizes AS s
GROUP BY s.productID
) AS sizes
ON p.productID = sizes.productID
INNER JOIN
(
SELECT avp.productID,
SUBSTRING_INDEX(GROUP_CONCAT(avp.productImg), ',', 2), AS three_images_per_product
FROM availImg AS avp
GROUP BY avp.productID
) AS images
ON images.productID = p.productID
INNER JOIN availImg
ON availImg.productID = images.productID
AND FIND_IN_SET(availImg.productImg, images.three_images_per_product)
WHERE p.productID = ?";
Note that your original query doesn't actually have any image details in the SELECT clause, and also you had done a GROUP BY p.productID which would mean even if 3 images were found the query would only return one row per product id anyway.

Need help with a mysql query

I've developed a mysql query that tells me how many times each product was purchased over a specified timeframe. I'm trying to figure out how to join this with my pageview tracking table so I can calculate the conversion rate for each product. Here is the original query:
SELECT SUM( op.products_quantity ) AS num_sold, SUM( op.final_price * op.products_quantity ) AS total_sales, p.products_model, pd.products_name, p.products_id
FROM orders_products AS op
JOIN products AS p
JOIN products_description AS pd
JOIN orders as o
WHERE p.products_id = op.products_id
AND p.products_id = pd.products_id
AND op.orders_id = o.orders_id
AND o.date_purchased BETWEEN '2011-01-15' AND '2011-04-15"'
GROUP BY p.products_id
ORDER BY total_sales DESC
I have another query that gives me the page views per product:
SELECT pv.products_id, count( pv.timestamp )
FROM products_visits AS pv
WHERE pv.timestamp
BETWEEN '2011-01-15'
AND '2011-04-17'
GROUP BY products_id
The caveat is that the views data has just started being collected, so we want return results even if a given product_id is not in the views table, but is in the purchases table.
How do I combine those queries into a single query?
You need a LEFT OUTER JOIN against a subquery. I think this will do the job:
SELECT
SUM( op.products_quantity ) AS num_sold,
SUM( op.final_price * op.products_quantity ) AS total_sales,
p.products_model, pd.products_name,
p.products_id,
visits.visits
FROM orders_products AS op
JOIN products AS p
JOIN products_description AS pd
JOIN orders as o
LEFT OUTER JOIN (
SELECT pv.products_id, count( pv.timestamp ) AS visits
FROM products_visits AS pv
WHERE pv.timestamp BETWEEN '2011-01-15' AND '2011-04-17'
GROUP BY products_id
) visits ON p.products_id = visits.products_id
WHERE p.products_id = op.products_id
AND p.products_id = pd.products_id
AND op.orders_id = o.orders_id
AND o.date_purchased BETWEEN '2011-01-15' AND '2011-04-15"'
GROUP BY p.products_id
ORDER BY total_sales DESC
You will need to use a LEFT JOIN: "first" you get all products and their sales data, and then you join the views data, if it exists.
try this:
SELECT p.products_id,
count( pv.timestamp ) AS views,
p.products_model AS model,
SUM( op.products_quantity ) AS num_sold,
SUM( op.final_price * op.products_quantity ) AS total_sales,
pd.products_name
FROM products AS p
LEFT JOIN products_visits AS pv
ON pv.products_id = p.products_id
AND pv.timestamp BETWEEN '2011-01-15' AND '2011-04-17'
JOIN orders_products AS op
ON p.products_id = op.products_id
JOIN products_description AS pd
ON p.products_id = pd.products_id
JOIN orders as o
ON op.orders_id = o.orders_id
AND o.date_purchased BETWEEN '2011-01-15' AND '2011-04-17'
GROUP BY p.products_id
ORDER BY total_sales DESC