I have an SELECT statement that has a huge number of left join and I want to filter some out.
When I check how many records i have in total and subtract the records with my LIKE statements, I should get the amount that is not affected by my restrictions.
But when I negate my restriction to get the ones I didn't affect, I get an different number than calculated.
SQL without restrictions (Record count: 13.251.981)
SELECT p.product_number
FROM product p
LEFT JOIN product_category pc on p.id = pc.product_id
LEFT JOIN product_category_tree pct on p.id = pct.product_id
LEFT JOIN product_configurator_setting pcs on p.id = pcs.product_id
LEFT JOIN product_cross_selling pcs2 on p.id = pcs2.product_id
LEFT JOIN product_cross_selling_assigned_products pcsap on p.id = pcsap.product_id
LEFT JOIN product_cross_selling_translation pcst on pcs2.id = pcst.product_cross_selling_id
LEFT JOIN product_custom_field_set pcfs on p.id = pcfs.product_id
LEFT JOIN product_media pm on p.id = pm.product_id
LEFT JOIN product_option po on p.id = po.product_id
LEFT JOIN product_price pp on p.id = pp.product_id
LEFT JOIN product_property pp2 on p.id = pp2.product_id
LEFT JOIN product_review pr on p.id = pr.product_id
LEFT JOIN product_search_keyword psk on p.id = psk.product_id
LEFT JOIN product_tag pt on p.id = pt.product_id
LEFT JOIN product_translation pt2 on p.id = pt2.product_id
LEFT JOIN product_visibility pv on p.id = pv.product_id
With restriction (Record count: 9.285.545)
WHERE p.product_number NOT LIKE 'SW%'
AND p.product_number NOT LIKE '%.%'
AND pt2.name NOT LIKE '%Gutschein'
AND pt2.name NOT LIKE '%Test%'
With negated restriction (Record count: 100.851)
WHERE p.product_number LIKE 'SW%'
OR p.product_number LIKE '%.%'
OR pt2.name LIKE '%Gutschein'
OR pt2.name LIKE '%Test%';
From my calculations i should get 3.966.436 records that don't get affected. (13.251.981 - 9.285.545 = 3.966.436)
But instead I get 100.851
How is that possible?
The solution for me was actually this WHERE:
WHERE p.product_number < 'SW'
I have a rather complex query to retrieve products & attributes from several tables.
SELECT SQL_CALC_FOUND_ROWS
p.*,
product_shop.*,
product_shop.id_category_default,
pl.*,
pbn.*,
MAX(image_shop.id_image) id_image,
il.legend,
m.name manufacturer_name,
0 as quantity
FROM ps_category_product cp
LEFT JOIN ps_category c ON (c.id_category = cp.id_category)
LEFT JOIN ps_product p ON p.id_product = cp.id_product
INNER JOIN ps_product_shop product_shop ON (product_shop.id_product = p.id_product AND product_shop.id_shop = 1)
LEFT JOIN ps_product_lang pl ON (pl.id_product = p.id_product AND pl.id_shop = 1 AND pl.id_lang = 7)
## ########### Added joins ###########
LEFT JOIN ps_product_base_names pbn ON pbn.id_product = p.id_product
INNER JOIN (
SELECT base_name, MAX(id_product) AS Max_ID_product
FROM ps_product_base_names
WHERE id_product IN (568110,568129,568134,568135,568136,568137,568139,568140,568141,602911,612411,612413,612512,612513,612515,612612,612616,616213,616217)
GROUP BY base_name) groupedpbn
ON (pbn.base_name = groupedpbn.base_name AND pbn.id_product = groupedpbn.Max_ID_product)
## ########### End added ###########
LEFT JOIN ps_image i ON (i.id_product = p.id_product) LEFT JOIN ps_image_shop image_shop ON (image_shop.id_image = i.id_image AND image_shop.id_shop = 1 AND image_shop.cover=1)
LEFT JOIN ps_image_lang il ON (image_shop.id_image = il.id_image AND il.id_lang = 7)
LEFT JOIN ps_manufacturer m ON (m.id_manufacturer = p.id_manufacturer)
WHERE product_shop.active = 1 AND product_shop.visibility IN ("both", "catalog")
AND c.nleft >= 3 AND c.nright <= 4
AND c.active = 1
AND p.id_product IN (568110,568129,568134,568135,568136,568137,568139,568140,568141,602911,612411,612413,612512,612513,612515,612612,612616,616213,616217)
GROUP BY product_shop.id_product
ORDER BY pl.name asc LIMIT 0,30
I have added 2 JOINs (see comment) to retrieve products by their base name and get only 1 result per base name, to show in the main catalog overview page.
This all works fine, but now I would like to get the number of products that have been grouped per base name. Something like:
COUNT(id_product) AS product_variations
So let's suppose that the products with id_product 568110, 602911 & 612413 all have the same base name, the above query will return id_product 612413 as the result.
But how do I get the number of ID's that have been aggregated (3 for the product with id_product 612413) for every product in the result list?
I found a solution here: https://www.periscopedata.com/blog/use-subqueries-to-count-distinct-50x-faster.html
My working query now looks like this:
SELECT SQL_CALC_FOUND_ROWS
p.*,
product_shop.*,
product_shop.id_category_default,
pl.*,
pbn.*,
cnt.product_variations, ## This line was added for the COUNT specified in the JOIN (see below)
MAX(image_shop.id_image) id_image,
il.legend,
m.name manufacturer_name,
0 as quantity
FROM ps_category_product cp
LEFT JOIN ps_category c ON (c.id_category = cp.id_category)
LEFT JOIN ps_product p ON p.id_product = cp.id_product
INNER JOIN ps_product_shop product_shop ON (product_shop.id_product = p.id_product AND product_shop.id_shop = 1)
LEFT JOIN ps_product_lang pl ON (pl.id_product = p.id_product AND pl.id_shop = 1 AND pl.id_lang = 7)
## ########### Added joins ###########
LEFT JOIN ps_product_base_names pbn ON pbn.id_product = p.id_product
INNER JOIN (
SELECT base_name, MAX(id_product) AS Max_ID_product
FROM ps_product_base_names
WHERE id_product IN (568110,568129,568134,568135,568136,568137,568139,568140,568141,602911,612411,612413,612512,612513,612515,612612,612616,616213,616217)
GROUP BY base_name) groupedpbn
ON (pbn.base_name = groupedpbn.base_name AND pbn.id_product = groupedpbn.Max_ID_product)
## This JOIN was added to COUNT aggregated product IDs
LEFT JOIN (
SELECT base_name, COUNT(id_product) AS product_variations
FROM ps_product_base_names
WHERE id_product IN (568110,568129,568134,568135,568136,568137,568139,568140,568141,602911,612411,612413,612512,612513,612515,612612,612616,616213,616217)
GROUP BY base_name) cnt
ON (pbn.base_name = cnt.base_name)
## ########### End added ###########
LEFT JOIN ps_image i ON (i.id_product = p.id_product) LEFT JOIN ps_image_shop image_shop ON (image_shop.id_image = i.id_image AND image_shop.id_shop = 1 AND image_shop.cover=1)
LEFT JOIN ps_image_lang il ON (image_shop.id_image = il.id_image AND il.id_lang = 7)
LEFT JOIN ps_manufacturer m ON (m.id_manufacturer = p.id_manufacturer)
WHERE product_shop.active = 1 AND product_shop.visibility IN ("both", "catalog")
AND c.nleft >= 3 AND c.nright <= 4
AND c.active = 1
AND p.id_product IN (568110,568129,568134,568135,568136,568137,568139,568140,568141,602911,612411,612413,612512,612513,612515,612612,612616,616213,616217)
GROUP BY product_shop.id_product
ORDER BY pl.name asc LIMIT 0,30
To be honest, it works and it seems fast enough in my test environment, but I haven't got a clue if this solution is efficient or not. So any comments to possibly improve on my solution are still welcome.
Thanks, Mattie
I have been having issues with missing data when I run the following query.
There are some products which has special price, stored in table oc_product_special and some products has regular price stored in table oc_product.
I figured out that it is only showing data if there is a special price in table oc_product_special. it is omitting any data where there is no special price and only regular price. I am not sure how to fix this problem. How and where i can add conditional statement or something like
if there is a regular price present and no special price then show regular price and 0 for special price.
SELECT
pd.name AS 'Product Name',
p.model AS UPC,
p.quantity AS 'Quantity',
p.price AS 'Regular Price',
ps.price AS 'Special Price',
p.cost AS 'COST',
p.status AS 'Status'
FROM oc_product p
INNER JOIN oc_product_description pd
ON pd.product_id = p.product_id
INNER JOIN oc_product_special ps
ON ps.product_id = p.product_id
INNER JOIN oc_manufacturer m
ON p.manufacturer_id = m.manufacturer_id
INNER JOIN oc_product_to_category p2c
ON p2c.product_id = p.product_id
INNER JOIN oc_category c
ON c.category_id = p2c.category_id
INNER JOIN oc_category_description cd
ON c.category_id = cd.category_id
WHERE
c.category_id = 37 OR c.parent_id = 37
GROUP BY pd.name ORDER BY m.name ASC
Use LEFT JOIN, which will retain records on the left side of the join even if they do not match to anything on the right side:
SELECT COALESCE(pd.name, 'NA') AS 'Product Name',
p.model AS UPC,
p.quantity AS 'Quantity',
p.price AS 'Regular Price',
COALESCE(ps.price, 0.0) AS 'Special Price',
p.cost AS 'COST',
p.status AS 'Status'
FROM oc_product p
LEFT JOIN oc_product_description pd
ON pd.product_id = p.product_id
LEFT JOIN oc_product_special ps
ON ps.product_id = p.product_id
INNER JOIN oc_manufacturer m
ON p.manufacturer_id = m.manufacturer_id
INNER JOIN oc_product_to_category p2c
ON p2c.product_id = p.product_id
INNER JOIN oc_category c
ON c.category_id = p2c.category_id
INNER JOIN oc_category_description cd
ON c.category_id = cd.category_id
WHERE c.category_id = 37 OR
c.parent_id = 37
GROUP BY pd.name
ORDER BY m.name
Explanation:
In a LEFT JOIN, when a record on the left side of the join does not match to anything on the right side, the columns from the right side will all appear as NULL in the result set. I used the COALESCE function in my query, which will conditionally replace the first argument with the second if the former be NULL. In this case, the special price will be replaced with zero if NULL. I also used it with the product name in case names be missing in some cases.
use left join on oc_product_special
SELECT
pd.name AS 'Product Name',
p.model AS UPC,
p.quantity AS 'Quantity',
p.price AS 'Regular Price',
ps.price AS 'Special Price',
p.cost AS 'COST',
p.status AS 'Status'
FROM oc_product p
INNER JOIN oc_product_description pd
ON pd.product_id = p.product_id
LEFT JOIN oc_product_special ps
ON ps.product_id = p.product_id
INNER JOIN oc_manufacturer m
ON p.manufacturer_id = m.manufacturer_id
INNER JOIN oc_product_to_category p2c
ON p2c.product_id = p.product_id
INNER JOIN oc_category c
ON c.category_id = p2c.category_id
INNER JOIN oc_category_description cd
ON c.category_id = cd.category_id
WHERE
c.category_id = 37 OR c.parent_id = 37
GROUP BY pd.name ORDER BY m.name ASC
Inner join si for matching value in you case somethings there aren't match so ..use left join
I'm using this sql query to get the product list from DB.
SELECT distinct P.product_id, B.brand_name, P.product_name, P.product_description, SC.sub_category_name, P.product_image_path
FROM table_products as P
INNER JOIN table_brands as B
ON P.brand_id = B.brand_id
INNER JOIN table_product_categories as PC
ON P.product_id = PC.product_id
INNER JOIN table_subcategories as SC
ON SC.sub_categories_id = PC.category_id
INNER JOIN table_subcategory_categories as SCC
ON SC.sub_categories_id = SCC.subcategory_id
ORDER BY P.product_id DESC";
It works fine for me. But when same product is in multiple subcategories. It gives me a new row. I just wanted to avoid this and wants GROUP by with SC.sub_category_name. So when a product is in multiple categories, all the categories should list in same row.
Current
853 Tops Premium Vermicelli /images/tops/853.png Noodles
853 Tops Premium Vermicelli /images/tops/853.png Vermicelli
Expecting
853 Tops Premium Vermicelli /images/tops/853.png Noodles, Vermicelli
You can use GROUP_CONCAT() for that purpose, grouping by SC.sub_category_name and remove the distinct. Something like
SELECT P.product_id,
B.brand_name,
P.product_name,
P.product_description,
GROUP_CONCAT(SC.sub_category_name) as sub_cat_list,
P.product_image_path
FROM table_products P
INNER JOIN table_brands B
ON P.brand_id = B.brand_id
INNER JOIN table_product_categories PC
ON P.product_id = PC.product_id
INNER JOIN table_subcategories SC
ON SC.sub_categories_id = PC.category_id
INNER JOIN table_subcategory_categories SCC
ON SC.sub_categories_id = SCC.subcategory_id
GROUP BY P.product_id
ORDER BY P.product_id DESC;
SELECT p.product_id,p.account_id,i.image_id,a.email,p.title,p.price
FROM products AS p
LEFT OUTER JOIN products_images AS i
ON p.product_id = i.product_id AND i.featured=1 AND i.deleted=0
INNER JOIN accounts AS a
ON p.account_id = a.account_id
MATCH(p.title) AGAINST('+images')
I'm trying to use a MATCH for the first time. It says that I have a syntax error and I am not sure why?
You're missing the WHERE keyword before conditions that aren't part of the join:
SELECT p.product_id,p.account_id,i.image_id,a.email,p.title,p.price
FROM products AS p
LEFT OUTER JOIN products_images AS i
ON p.product_id = i.product_id AND i.featured=1 AND i.deleted=0
INNER JOIN accounts AS a
ON p.account_id = a.account_id
WHERE MATCH(p.title) AGAINST('+images')