Limit not working in INNER JOINS - mysql

Am sure this question has already been asked, but I can't help but wonder why the "LIMIT 3" in one of the INNER JOINS is only returning a single imgUrl for the specified avp.productID in the "ON clause"! Is that LIMITs do not work in INNER JOINs? What better way could I use to achieve this please?
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From
products AS p
INNER JOIN(
Select
c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM
availColors AS c GROUP BY c.productID) AS colors
ON
p.productID = colors.productID
INNER JOIN(
SELECT
s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM
availSizes AS s
GROUP BY
s.productID
) AS sizes
ON
p.productID = sizes.productID
INNER JOIN(
SELECT
avp.productID,
avp.productImg
FROM
availImg AS avp
ORDER BY
avp.productID
LIMIT 3) AS images
ON
images.productID = p.productID
WHERE
p.productID = ?
GROUP BY
p.productID";

Your current query has a sub query that returns 3 images, ordered by product id. This sub query doesn't check if the images belong to the product you are searching for so could easily bring back 3 images for a different product. This would result in the main query trying to do an inner join onto the sub query based on product id and finding no matching records.
As your 2 other sub queries only bring back a single row for each product id (due to the GROUP_CONCAT) and you appear to only want to check one product id at a time then I think you can solve this by just doing a normal join onto the images table (without a sub query) and putting the limit on the main query.
Like this:-
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From products AS p
INNER JOIN
(
Select c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM availColors AS c
GROUP BY c.productID
) AS colors
ON p.productID = colors.productID
INNER JOIN
(
SELECT s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM availSizes AS s
GROUP BY s.productID
) AS sizes
ON p.productID = sizes.productID
INNER JOIN availImg images
ON images.productID = p.productID
WHERE p.productID = ?
ORDER BY images.productID, images.productImg
LIMIT 3";
However if you want to bring back multiple product ids then it becomes a bit more complicated. One solution would be to bring back the images with a generated sequence number for each product, and then check in the ON clause of the join that it is one of the first 3 images for a product:-
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From products AS p
INNER JOIN
(
Select c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM availColors AS c
GROUP BY c.productID
) AS colors
ON p.productID = colors.productID
INNER JOIN
(
SELECT s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM availSizes AS s
GROUP BY s.productID
) AS sizes
ON p.productID = sizes.productID
INNER JOIN
(
SELECT avp.productID,
avp.productImg,
#seq:=IF(#prev_productID = avp.productID, #seq + 1, 1) AS seq,
#prev_productID := avp.productID
FROM availImg AS avp
CROSS JOIN (SELECT #prev_productID:=0, #seq:=0) sub1
ORDER BY avp.productID
) AS images
ON images.productID = p.productID
AND images.seq <= 3
WHERE p.productID = ?";
Another possible method would be to use GROUP_CONCAT to get a comma separated list of image ids for each product, with SUBSTRING_INDEX to get the first 3 of these, then join that against the images table using FIND_IN_SET to get the rest of the image details:-
$query = "SELECT
p.productID,
p.productDesc,
p.productQty,
p.productPr,
p.type,
p.gender,
p.date
From products AS p
INNER JOIN
(
Select c.productID,
GROUP_CONCAT(DISTINCT c.availCol) AS color_list
FROM availColors AS c
GROUP BY c.productID
) AS colors
ON p.productID = colors.productID
INNER JOIN
(
SELECT s.productID,
GROUP_CONCAT(s.availSizes) AS size_list
FROM availSizes AS s
GROUP BY s.productID
) AS sizes
ON p.productID = sizes.productID
INNER JOIN
(
SELECT avp.productID,
SUBSTRING_INDEX(GROUP_CONCAT(avp.productImg), ',', 2), AS three_images_per_product
FROM availImg AS avp
GROUP BY avp.productID
) AS images
ON images.productID = p.productID
INNER JOIN availImg
ON availImg.productID = images.productID
AND FIND_IN_SET(availImg.productImg, images.three_images_per_product)
WHERE p.productID = ?";
Note that your original query doesn't actually have any image details in the SELECT clause, and also you had done a GROUP BY p.productID which would mean even if 3 images were found the query would only return one row per product id anyway.

Related

Mysql query able to rank products by best matching colors

I have a mysql query that filter products by categories, price and colors.
I'm not able to find a way to order results by best matching colors, so that products that match more with rgb(35,44,65) stay on top of results.
Any idea?
SELECT DISTINCT(p.id_product), p.*, pn.*
FROM products p
INNER JOIN product_categories pc ON p.id_product=pc.id_product
INNER JOIN product_names pn ON p.id_product=pn.id_product
WHERE
(id_category IN (10,20) OR id_category IN (10,40))
and price<=3500
and p.id_product IN (SELECT DISTINCT(id_product) FROM colors ORDER BY (POW(35-red, 2)+POW(44-green, 2)+POW(65-blue, 2)) ASC)
GROUP BY p.id_product
HAVING COUNT(DISTINCT id_category) = 2
You should join with the colors table rather than using IN. Then you can get the color values to use in ORDER BY in the main query.
SELECT DISTINCT p.*, pn.*
FROM products AS p
INNER JOIN product_categories pc ON p.id_product=pc.id_product
INNER JOIN product_names pn ON p.id_product=pn.id_product
INNER JOIN colors AS c ON c.id_product = p.id_product
WHERE pc.id_category IN (10, 20, 40)
AND p.price <= 3500
GROUP BY p.id_product
HAVING COUNT(DISTINCT id_category) = 2
ORDER BY (POW(35-c.red, 2)+POW(44-c.green, 2)+POW(65-c.blue, 2)) ASC

subquery error in group by when converting from MySQL to PostgresSQL

I'm converting my database from MySQL to PostgresSQL, and I have this query which tries to sord the products by the cheapest price and the most popular in a given location. It works fine in MySQL, but in Postgres I'm running into problems with this query :
SELECT products.product_id,
suppliers.supplier_id,
product_code.desc_fa,
products.name_fa,
MIN(product_supplier.price) AS price,
SUM(COALESCE(orders.quantity, 0)) AS n_orders
FROM products
JOIN product_code ON product_code.code_id = products.code_id
JOIN product_supplier ON product_supplier.product_id = products.product_id
JOIN suppliers ON suppliers.supplier_id = product_supplier.supplier_id
JOIN product_tags ON product_tags.product_id = products.product_id
JOIN tags ON tags.tag_id = product_tags.tag_id
JOIN product_crop ON product_crop.product_id = products.product_id
JOIN crops ON crops.crops_id = product_crop.crop_id
LEFT JOIN orders
ON orders.product_id = products.product_id and orders.crop_id = product_crop.crop_id
LEFT JOIN user ON user.user_id = orders.user_id and user.location_id = 883
WHERE crops.crops_id = 1
AND product_supplier.quantity >= 3
AND tags.tag = 'علف کش'
GROUP BY products.name_fa
ORDER BY n_orders DESC
LIMIT 10;
It gives me this error :
column must appear in the GROUP BY clause or be used in an aggregate function
Any suggestions to how to work around this error ?
UPDATE :
According to the answers i was able to make it work by using this query:
WITH tem_1 AS (SELECT product_id, MIN(price) AS price FROM product_supplier GROUP BY product_id) ,
tem_2 AS (SELECT product_id, SUM(quantity) AS n_orders FROM orders Group by product_id)
SELECT products.product_id, suppliers.supplier_id, product_code.desc_fa, products.name_fa, tem_1.price,
products.telegraph, suppliers.location_id, COALESCE(tem_2.n_orders,0) AS quant FROM products
INNER JOIN product_supplier ON product_supplier.product_id = products.product_id
INNER JOIN suppliers ON suppliers.supplier_id = product_supplier.supplier_id
INNER JOIN product_code ON product_code.code_id = products.code_id
INNER JOIN product_crop ON product_crop.product_id = products.product_id
INNER JOIN crops ON crops.crops_id = product_crop.crop_id
INNER JOIN product_tags ON product_tags.product_id = products.product_id
INNER JOIN tags ON tags.tag_id = product_tags.tag_id
INNER JOIN tem_1 ON tem_1.price = product_supplier.price AND tem_1.product_id = products.product_id
LEFT JOIN tem_2 ON tem_2.product_id = products.product_id
WHERE crops.crops_id = 1 AND product_supplier.quantity >= 3 AND tags.tag = 'علف کش'
ORDER BY quant DESC
LIMIT 10;;
But since i'm fairly new to SQL, I wanted to know if my code is correct or is there a better way to implement it?
When you use aggregate function (as SUM, MIN and others) the other columns in your field list without aggregate function must be included in GROUP BY clause.
These fields:
products.product_id,
suppliers.supplier_id,
product_code.desc_fa,
products.name_fa
must be in GROUP BY.
Instead your GROUP BY clause has only the following field:
products.name_fa
You must add the other missing 3 fields
In MySql this error has turned off by default, so your query works fine, but in other DBMS you are in an error case.
You can see here how set MySql environment about the GROUP BY behaviour

Filter products by options

I have following database structure to store product options.
Now i have problem to filter out products that match only given options. First i did WHERE option_id IN (array of options), but that would give me products that match any of the options and that is not solution. User wants to filter out only products with given material, color, and size for instance. And if i do WHERE option_id = 4 AND option_id = 6 for instance i get nothing.
Here is my query:
SELECT DISTINCT p.id AS id,
...
FROM products p
LEFT JOIN product_categories pc ON p.id = pc.product_id
LEFT JOIN product_images pi ON p.id = pi.product_id
LEFT JOIN product_options po ON p.id = po.product_id
WHERE p.product_active = 1
AND po.option_id = 1 // only to get the idea
GROUP BY id
ORDER BY id DESC
LIMIT 0,
12
Just to mention it is PHP application , where user select options from select element with or without multiple attribute.
How to acomplish this?
You can use having:
SELECT p.id AS id, ...
FROM products p JOIN
product_categories pc
ON p.id = pc.product_id LEFT JOIN
product_images pi
ON p.id = pi.product_id JOIN
product_options po
ON p.id = po.product_id
WHERE p.product_active = 1 AND
po.option_id IN (4, 6)
GROUP BY p.id
HAVING COUNT(DISTINCT po.option_id) = 2
ORDER BY p.id DESC
LIMIT 0, 12;
The HAVING clause is specifying that a given id has two matching options. Because of the WHERE clause, these are the only two options that you care about.
I didn't change your approach (you didn't supply the complete query), but you are doing joins along different dimensions -- categories, images, and options. This creates a Cartesian product for each product, and that is often not the best approach to such a query.
There is no need for LEFT JOIN in the solution.
SELECT DISTINCT p.id AS id
FROM products p
JOIN product_options po ON p.id = po.product_id
WHERE p.product_active = 1
AND po.option_id IN (1, 2, 3)
GROUP BY p.id
HAVING COUNT(po.option_id) = 3
My solution keep only tables necessary to find the products with specified options.
In the case you want products having exactly this options and no others you can use NOT EXISTS:
SELECT DISTINCT p.id AS id
FROM products p
JOIN product_options po ON p.id = po.product_id
WHERE p.product_active = 1 AND
po.option_id IN (1, 2, 3) and
NOT EXISTS (
SELECT 1
FROM product_options po2
WHERE p.id = po2.product_id and po2.option_id NOT IN (1, 2, 3)
)
GROUP BY p.id
HAVING COUNT(po.option_id) = 3
If you want to select products accoding to the other conditions (like product categories and so on) then use IN in the WHERE clause. This approach avoids generating duplicate po.option_id and the outer query will still work correctly even without DISTINCT in COUNT.
SELECT DISTINCT p.id AS id
FROM products p
JOIN product_options po ON p.id = po.product_id
WHERE p.product_active = 1 AND
po.option_id IN (1, 2, 3) AND
-- use the following IN predicate to select products with specific features without introducing duplicates in your query
p.id IN (
select product_id FROM product_categories WHERE <your_condition>
)
GROUP BY p.id
HAVING COUNT(po.option_id) = 3
You select products with image lists. Something like:
select products.*, group_concat(product_images.id)
Additionally there may be options the product must all meet. This is criteria that belongs in the WHERE clause.
select
p.*,
(select group_concat(image) from product_images i where i.product_id = p.id) as images
from products p
where product_active = 1
and id in
(
select product_id
from product_options
where option_id in (1,3,55,97)
group by product_id
having count(*) = 4 -- four options in this example
);
Thanks guys, i've managed to return exactly what i wanted.
Now i just have problem with pagination query for the filtered products.
Final search query:
SELECT DISTINCT p.id AS id,
main_price,
promotion_price,
NEW,
sale,
recommended,
COUNT(pi.filename) AS image_count,
GROUP_CONCAT(DISTINCT pi.filename
ORDER BY pi.main_image DESC, pi.id ASC) AS images,
name_sr,
uri_sr,
description_sr
FROM products p
LEFT JOIN product_categories pc ON p.id = pc.product_id
LEFT JOIN product_images pi ON p.id = pi.product_id
LEFT JOIN product_options po ON p.id = po.product_id
WHERE p.product_active = 1
AND po.option_id IN(1)
AND p.main_price BETWEEN 5250.00 AND 14000.00
GROUP BY id
HAVING COUNT(DISTINCT po.option_id) = 1
ORDER BY id DESC
LIMIT 0,
12
Pagination query is something like this i modified it accorgin to new filter query:
SELECT COUNT(DISTINCT p.id) AS number
FROM products p
LEFT JOIN product_categories pc ON p.id = pc.product_id
LEFT JOIN product_images pi ON p.id = pi.product_id
LEFT JOIN product_options po ON p.id = po.product_id
WHERE p.product_active = 1
AND po.option_id IN(1)
AND p.main_price BETWEEN 5250.00 AND 14000.00
GROUP BY(p.id)
HAVING COUNT(DISTINCT po.option_id) = 1
If i leave out DISTINCT in SELECT COUNT i don't get filtered pagination, if i set DISTINCT i get number of rows that corespond to pagination. I suppose i could add another count(*) to all of this with subquery, but not sure if that is way to go and if there is more efficient and elegant way to do this.

How to optimize query with two inner join

Using this query to get the products with words that fulfill all three required word terms (lenovo, laptop, computer):
SELECT t1.id, t1.name, t1.price FROM
(SELECT p.id AS productid, name, price
FROM products p JOIN productwords pw ON p.id = pw.productid
JOIN words w ON pw.wordid = w.id WHERE word.term = 'lenovo') t1
INNER JOIN
(SELECT p.id AS productid, name, price
FROM products p JOIN productwords pw ON p.id = pw.productid
JOIN words w ON pw.wordid = w.id WHERE word.term = 'laptop') t2
INNER JOIN
(SELECT p.id AS productid, name, price
FROM products p JOIN productwords pw ON p.id = pw.productid
JOIN words w ON pw.wordid = w.id WHERE word.term = 'computer') t3
ON
t1.productid = t2.productid
AND
t1.productid = t3.productid
ORDER BY t1.name
As far as I can see, the query considers the whole words table for each term (the tables have indexes. Database is MySql).
Can the query be rewritten in a better way, so it will become faster? (the tables contain millions of rows)
For example with subsets, so the 'laptop' search only considers the rows matching 'lenovo' - and the 'computer' search only considers the rows matching first 'lenovo' and then 'laptop'.
Thanks!
You can use the HAVING clause :
SELECT p.id AS productid, name, price
FROM products p
JOIN productwords pw ON p.id = pw.productid
JOIN words w ON pw.wordid = w.id
WHERE word.term in ('lenovo','computer','laptop')
GROUP BY p.id , name, price
HAVING COUNT(DISTINCT word.term) = 3
That is if I understood the question, it looks like product -> words is 1:n relation , and if no column from the word table is selected, that should work perfectly.
This might be a quicker way of doing it:
SELECT p.id, name, price
FROM products p
where
EXISTS (select null
from productwords pw1
JOIN words w1 ON pw1.wordid = w1.id
where w1.term = 'lenovo'
and p.id = pw1.productid )
and EXISTS (select null
productwords pw2
JOIN words w2 ON pw2.wordid = w2.id
where w2.term = 'laptop'
and and p.id = pw2.productid )
and EXISTS (select null
productwords pw3 ON p.id = pw3.productid
JOIN words w3
where w3.term = 'computer'
and p.id = pw3.productid )
ORDER BY name;

how to sort and group by by it's count

I have the following:
SELECT DISTINCT s.username, COUNT( v.id ) AS cnt
FROM `instagram_item_viewer` v
INNER JOIN `instagram_shop_picture` p ON v.item_id = p.id
INNER JOIN `instagram_shop` s ON p.shop_id = s.id
AND s.expirydate IS NULL
AND s.isLocked =0
AND v.created >= '2014-08-01'
GROUP BY (
s.id
)
ORDER BY cnt DESC
Basically I have an instagram_item_viewer with the following structure:
id viewer_id item_id created
It tracks when a user has viewed an item and what time. So basically I wanted to find shops that has the most items viewed. I tried the query above and it executed fine, however it doesn't seem to give the appropriate data, it should have more count than what it is. What am I doing wrong?
First, with a group by statement, you don't need the DISTINCT clause. The grouping takes care of making your records distinct.
You may want to reconsider the order of your tables. Since you are interested in the shops, start there.
Select s.username, count(v.id)
From instagram_shop s
INNER JOIN instagram_shop_picture p ON p.shop_id = s.shop_id
INNER JOIN instagram_item_viewer v ON v.item_id = p.id
AND v.created >= '2014-08-01'
WHERE s.expirydate IS NULL
AND s.isLocked = 0
GROUP BY s.username
Give thata shot.
As mentioned by #Lennart, if you have a sample data it would be helpful. Because otherwise there will be assumptions.
Try run this to debug (this is not the answer yet)
SELECT s.username, p.id, COUNT( v.id ) AS cnt
FROM `instagram_item_viewer` v
INNER JOIN `instagram_shop_picture` p ON v.item_id = p.id
INNER JOIN `instagram_shop` s ON p.shop_id = s.id
AND s.expirydate IS NULL
AND s.isLocked =0
AND v.created >= '2014-08-01'
GROUP BY (
s.username, p.id
)
ORDER BY cnt DESC
The problem here is the store and item viewer is too far apart (i.e. bridged via shop_picture). Thus shop_picture needs to be in the SELECT statement.
Your original query only gets the first shop_picture count for that store that is why it is less than expected
Ultimately if you still want to achieve your goal, you can expand my SQL above to
SELECT x.username, SUM(x.cnt) -- or COUNT(x.cnt) depending on what you want
FROM
(
SELECT s.username, p.id, COUNT( v.id ) AS cnt
FROM `instagram_item_viewer` v
INNER JOIN `instagram_shop_picture` p ON v.item_id = p.id
INNER JOIN `instagram_shop` s ON p.shop_id = s.id
AND s.expirydate IS NULL
AND s.isLocked =0
AND v.created >= '2014-08-01'
GROUP BY (
s.username, p.id
)
ORDER BY cnt DESC
) x
GROUP BY x.username