Count distinct with left join query executing long time - mysql

I have about 140k raws in all these tables:
SELECT COUNT( DISTINCT p.product_id ) AS total
FROM bh_product p
LEFT JOIN bh_product_description pd ON ( p.product_id = pd.product_id )
LEFT JOIN bh_product_to_store p2s ON ( p.product_id = p2s.product_id )
Is it normal that execution of this query takes about 3 seconds?
All tables have indexes on product_id field.
Can it be somehow improved?
UPDATED:
Original query:
SELECT COUNT( DISTINCT p.product_id ) AS total
FROM bh_product p
LEFT JOIN bh_product_description pd ON ( p.product_id = pd.product_id )
LEFT JOIN bh_product_to_store p2s ON ( p.product_id = p2s.product_id )
WHERE pd.language_id = '2'
AND p.status = '1'
AND p.date_available <= NOW( )
AND p2s.store_id = '0'
AND (
pd.name LIKE '%душевые%'
OR pd.tag LIKE '%душевые%'
OR LCASE( p.model ) = 'душевые'
OR LCASE( p.sku ) = 'душевые'
OR LCASE( p.upc ) = 'душевые'
OR LCASE( p.ean ) = 'душевые'
OR LCASE( p.jan ) = 'душевые'
OR LCASE( p.isbn ) = 'душевые'
OR LCASE( p.mpn ) = 'душевые'
)
UPDATED: It figured out that the server were running other intensive process, that was slowing sql processing.
After turning off other process perfomance become acceptable.

If you are using left join your result will be same with
select count(distinct p.product_id) as total
from bh_product p
because left join could filter nothing by your product_id.

You can start from this query:
SELECT COUNT(
DISTINCT p.product_id ) AS total
FROM bh_product p
INNER JOIN bh_product_description pd
ON p.product_id = pd.product_id
AND pd.language_id = 2
INNER JOIN bh_product_to_store p2s
ON p.product_id = p2s.product_id
AND p2s.store_id = 0
WHERE p.status = '1'
AND p.date_available <= NOW( )
AND (
pd.name LIKE '%душевые%'
OR pd.tag LIKE '%душевые%'
OR 'душевые' IN ( p.model , p.sku , p.upc , p.ean , p.jan , p.isbn , p.mpn ))
But your most weak point here is your WHERE clause. you are trying to search everywhere. That is not very smart. I am pretty sure that p.ean is barcode and can't be equal to душевые same for p.isbn. So you should change your query to do only what you really need to do. But not filter everything with hope to "catch the fish".
Update Check this query (should be much faster then your 1st one):
SELECT COUNT(
DISTINCT p.product_id ) AS total
FROM bh_product p
INNER JOIN bh_product_description pd
ON p.product_id = pd.product_id
AND pd.language_id = 2
INNER JOIN bh_product_to_store p2s
ON p.product_id = p2s.product_id
AND p2s.store_id = 0

I think You are trying to execute query like this
SELECT COUNT( DISTINCT p.product_id ) AS total
FROM bh_product p ,bh_product_description pd,bh_product_to_store p2s where
p.product_id = pd.product_id and p.product_id = p2s.product_id ;

Related

Optimize mysql query using various join and order by rand()

I was having trouble on my site, so I went look into mysql slow query log.
Fixed some indexing and also querys without joins now the server load got lower and its running fine.
I still have a query that I havent been able to fix, if any of you have a tip so I can make it better, I would appreciated it.
This is the query
SELECT p.product_id,
p.product_offer_price,
ps.subcategory_name,
pb.brand_name,
pm.model_name,
pm.display_name,
pm.year_from,
pm.year_to
FROM product p
INNER JOIN product_subcategory ps ON ps.subcategory_id = p.product_subcategory_id AND ps.subcategory_category_id = 12
INNER JOIN product_stock pq ON pq.product_id = p.product_id AND pq.product_quantity > 0
INNER JOIN product_photos pp ON pp.product_id = p.product_id
INNER JOIN product_brand pb ON pb.brand_id = p.product_brand_id
INNER JOIN product_model pm ON pm.model_id = p.product_model_id
GROUP BY p.product_id
ORDER BY RAND() LIMIT 4;
This is what explain shows about the query
I was trying to accomplish something like this, but doesn't work
SELECT p.product_id, p.product_offer_price, ps.subcategory_name, pb.brand_name, pm.model_name, pm.display_name, pm.year_from, pm.year_to
FROM product p
INNER JOIN product_subcategory ps ON ps.subcategory_id = p.product_subcategory_id
AND ps.subcategory_category_id =12
INNER JOIN product_stock pq ON pq.product_id = p.product_id
AND pq.product_quantity >0
INNER JOIN product_photos pp ON pp.product_id = p.product_id
INNER JOIN product_brand pb ON pb.brand_id = p.product_brand_id
INNER JOIN product_model pm ON pm.model_id = p.product_model_id
WHERE p.product_id >= FLOOR( 1 + RAND( ) * (
SELECT MAX( product_id )
FROM product ) )
GROUP BY p.product_id
LIMIT 4

how to Return productid in this example mysql?

how to Return productid who those filterid equal x and y.
this is select and return table
query :
SELECT p.product_id,pf.filter_id
FROM oc_product p
LEFT JOIN oc_product_filter pf
on p.product_id=pf.product_id
where p.product_id in(96621,97026) and pf.filter_id in (1901,1855 )
group by p.product_id,pf.filter_id
The result of the processing:
product_id filter_id
96621 1855
96621 1901
97026 1901
but i want to get product_id who filter_id=1855 and filter_id=1901.
want to show only product_id=96621 and not return product_id=97026
main query :
SELECT p.product_id
FROM oc_category_path cp
LEFT JOIN oc_product_to_category p2c
ON (cp.category_id = p2c.category_id)
LEFT JOIN oc_product_filter pf
ON (p2c.product_id = pf.product_id)
LEFT JOIN oc_product p
ON (pf.product_id = p.product_id)
LEFT JOIN oc_product_description pd
ON (p.product_id = pd.product_id)
LEFT JOIN oc_product_to_store p2s
ON (p.product_id = p2s.product_id)
WHERE pd.language_id = 2 AND p.status = '1' AND p.date_available <= NOW() AND p2s.store_id = 0
AND cp.path_id = 86125
AND pf.filter_id in(1901,1855)
GROUP BY p.product_id
ORDER BY p.price DESC, p.sort_order
try and test:
1.#GordThompson:
this result
2.#Edward Mendez
this result
Use OR instead of AND clause .
WHERE product_id in () OR filter_id in () OR product_id not in ()
Remove the product_id from your IN clause for 97026. Also remove the group by since there are no aggregating functioning.
SET #product_id = xxx
SET #filter_id = xxx
SELECT p.product_id,pf.filter_id
FROM oc_product p
LEFT JOIN oc_product_filter pf
on p.product_id=pf.product_id
where p.product_id in(#product_id) and pf.filter_id in (filter_id)
#group by p.product_id,pf.filter_id
If you want a list of product_id values that have both filter_id=1901 and filter_id=1855 you can use
SELECT product_id
FROM oc_product_filter
WHERE filter_id IN (1901,1855)
GROUP BY product_id
HAVING COUNT(*) = 2
or, if the table may have more than one instance of a specific filter_id for a given product_id you can use
SELECT t.product_id
FROM
(
SELECT DISTINCT product_id, filter_id
FROM oc_product_filter
) t
WHERE t.filter_id IN (1901,1855)
GROUP BY t.product_id
HAVING COUNT(*) = 2

update field using a subquery with left join and group_concat from another table

I'm using the next code, but it fails and I don't know why.
UPDATE pd
SET pd.meta_categories = x.categories
FROM t_product_description pd
JOIN (
SELECT p.product_id AS productid, GROUP_CONCAT(cd.name SEPARATOR ' ') AS categories
FROM t_product AS p
LEFT JOIN t_product_to_category AS p2c ON (p2c.product_id = p.product_id)
LEFT JOIN t_category_description AS cd ON (cd.category_id = p2c.category_id)
GROUP BY p.product_id
) x ON pd.product_id = x.productid
in mysql the join clause became first, (don't need from ) and the the set clause and
UPDATE t_product_description pd
JOIN (
SELECT p.product_id AS productid, GROUP_CONCAT(cd.name SEPARATOR ' ') AS categories
FROM t_product AS p
LEFT JOIN t_product_to_category AS p2c ON (p2c.product_id = p.product_id)
LEFT JOIN t_category_description AS cd ON (cd.category_id = p2c.category_id)
GROUP BY p.product_id
) x ON pd.product_id = x.productid
SET pd.meta_categories = x.categories

I am getting #1066 - Not unique table/alias: 'pf' error here

SELECT COUNT(DISTINCT `product_id`) AS `total`
FROM(
SELECT DISTINCT `p`.`product_id`
FROM oc_category_path cp
LEFT JOIN oc_product_to_category p2c ON (cp.category_id = p2c.category_id)
LEFT JOIN oc_product_filter pf ON (p2c.product_id = pf.product_id)
LEFT JOIN oc_product p ON (pf.product_id = p.product_id)
LEFT JOIN oc_product_description pd ON (p.product_id = pd.product_id)
LEFT JOIN oc_product_to_store p2s ON (p.product_id = p2s.product_id)
INNER JOIN `oc_product_filter` AS `pf` ON `p2c`.`product_id` = `pf`.`product_id`
WHERE IF( `p`.`quantity` > 0, 7, `p`.`stock_status_id` ) IN(7)
AND `p`.`date_available` <= NOW()
AND `p`.`status` = '1'
AND `cp`.`path_id` = '452'
AND `pf`.`filter_id` IN(241)
AND pd.language_id = '1'
AND p.status = '1'
AND p.date_available <= NOW()
AND p2s.store_id = '0'
AND cp.path_id = '452'
AND pf.filter_id IN (241)
) AS `tmp`
This line...
LEFT JOIN oc_product_filter pf ON (p2c.product_id = pf.product_id)
... shares a table alias with this line ...
INNER JOIN `oc_product_filter` AS `pf` ON `p2c`.`product_id` = `pf`.`product_id`
That can't happen. So give the second one a different alias...
SELECT COUNT(DISTINCT product_id) total
FROM
( SELECT DISTINCT p.product_id
FROM oc_category_path cp
LEFT
JOIN oc_product_to_category p2c
ON p2c.category_id = cp.category_id
JOIN oc_product_filter pf
ON pf.product_id = p2c.product_id
JOIN oc_product p
ON p.product_id = pf.product_id
JOIN oc_product_description pd
ON pd.product_id = p.product_id
JOIN oc_product_to_store p2s
ON p2s.product_id = p.product_id
JOIN oc_product_filter pf2 -- <-- changed
ON pf.product_id = p2c.product_id
WHERE IF(p.quantity > 0, 7, p.stock_status_id ) IN(7)
AND p.date_available <= NOW()
AND p.status = 1
AND cp.path_id = 452
AND pf.filter_id IN(241)
AND pd.language_id = 1
AND p.status = 1
AND p.date_available <= NOW()
AND p2s.store_id = 0
AND cp.path_id = 452
AND pf2.filter_id IN(241) -- <-- changed
) tmp;

How to verify two distinct values belong to a field in MySQL

I have the following MySQL query. As you can see in the last line, it asks for the filter_id to be either 51 or 8.
How can I modify this query to make sure the products that are returned are only products that have BOTH filter IDs.
What I did: I tried adding HAVING COUNT(DISTINCT pf.filter_id) >=2 based on other answers in this forum but it did not help.
SELECT COUNT(DISTINCT p.product_id) AS total
FROM ocb3l_product_to_category p2c
LEFT JOIN ocb3l_product_filter pf ON (p2c.product_id = pf.product_id)
LEFT JOIN ocb3l_product p ON (pf.product_id = p.product_id)
LEFT JOIN ocb3l_product_description pd ON (p.product_id = pd.product_id)
LEFT JOIN ocb3l_product_to_store p2s ON (p.product_id = p2s.product_id)
WHERE pd.language_id = '1'
AND p.status = '1'
AND p.date_available <= NOW()
AND p2s.store_id = '0'
AND p2c.category_id = '146'
AND pf.filter_id IN (51,8)
P.S. This query is used in an OpenCart shop to get the total number of products on the category pages.
The HAVING COUNT(DISTINCT pf.filter_id) >=2 clause will work, if you were to GROUP BY the product_id.
One approach is to use an inline view (derived table) so that this check can be performed for each product_id. The result from that can be joined to the other tables.
For example:
SELECT COUNT(DISTINCT fp.product_id) AS total
FROM (
SELECT p.product_id
FROM ocb3l_product p
JOIN ocb3l_product_filter pf
ON pf.product_id = p.product_id
AND pf.filter_id IN (51,8)
WHERE p.status = '1'
AND p.date_available <= NOW()
GROUP BY p.product_id
HAVING COUNT(DISTINCT pf.filter_id) = 2
) fp
JOIN ocb3l_product_to_category p2c
ON p2c.product_id = fp.product_id
AND p2c.category_id = '146'
JOIN ocb3l_product_to_store p2s
ON p2s.product_id = fp.product_id
AND p2s.store_id = '0'
JOIN ocb3l_product_description pd
ON pd.product_id = fp.product_id
AND pd.language_id = '1'
What you need to do is add another JOINed copy of the product_filter table like this:
SELECT COUNT(DISTINCT p.product_id) AS total
FROM ocb3l_product_to_category p2c
LEFT JOIN ocb3l_product_filter pf ON (p2c.product_id = pf.product_id)
LEFT JOIN ocb3l_product_filter pf2 ON (p2c.product_id = pf2.product_id) /*new*/
LEFT JOIN ocb3l_product p ON (pf.product_id = p.product_id)
LEFT JOIN ocb3l_product_description pd ON (p.product_id = pd.product_id)
LEFT JOIN ocb3l_product_to_store p2s ON (p.product_id = p2s.product_id)
WHERE pd.language_id = '1'
AND p.status = '1'
AND p.date_available <= NOW()
AND p2s.store_id = '0'
AND p2c.category_id = '146'
AND pf.filter_id = 51 /*used to be IN (51,8)*/
AND pf2.filter_id = 8 /*new*/