I am trying to optimize a SQL query and I would like some expert opinion on the best/fastest way to combine GROUP BY and ORDER BY
Basically I am trying to select the lowest price from a products table and group them by merchant name.
This was my original query:
select p.*, m.*, d.* from datafeeds as d, products as p left outer join meta as m on p.mykey = m.mykey where p.datafeed_id = d.id and (match(p.name) against ('+asics +"gel" -women*' in boolean mode)) and p.datafeed_id = '35' and p.is_custom = 0 group by d.merchant_name order by d.merchant_name limit 50
And the ORDER BY was not working, I was getting grouped products but not the ones with the lowest prices.
After reading other discussions i came up with an improved query:
SELECT p . * , m . * , d . *
FROM datafeeds AS d, products AS p
INNER JOIN (
SELECT MIN( display_price ) AS MinPrice
FROM products AS p
WHERE 1 =1
AND (
MATCH (
p.name
)
AGAINST (
'+asics +"gel" -women*'
IN BOOLEAN
MODE
)
)
AND p.datafeed_id = '35'
AND p.is_custom =0
GROUP BY merchant_name
) AS p2 ON p.display_price = p2.MinPrice
LEFT OUTER JOIN meta AS m ON p.mykey = m.mykey
WHERE p.datafeed_id = d.id
AND (
MATCH (
p.name
)
AGAINST (
'+asics +"gel" -women*'
IN BOOLEAN
MODE
)
)
AND p.datafeed_id = '35'
AND p.is_custom =0
GROUP BY d.merchant_name
ORDER BY d.merchant_name
LIMIT 50`
The query gets the correct results but it is quite slow.
Is there a better way to do it?
Thanks in advance
You could try this
SELECT p.*,
m.*,
d.*
FROM datafeeds AS d,
(
SELECT *
FROM products
WHERE 1 = 1
AND (
MATCH ( name ) against ( '+asics +"gel" -women*' IN boolean mode ) )
AND datafeed_id = '35'
AND is_custom =0
ORDER BY merchant_name,
display_price) AS p
LEFT OUTER JOIN meta AS m
ON p.mykey = m.mykey
WHERE p.datafeed_id = d.id
GROUP BY d.merchant_name
ORDER BY d.merchant_name
LIMIT 50
Related
I wrote this query
SELECT
country,
COUNT(DISTINCT tmp_tbl.user_guid) AS number_of_customers
FROM complete_tests c INNER JOIN
( SELECT DISTINCT d.dog_guid,
u.user_guid,
u.country
FROM dogs d INNER JOIN users u ON d.user_guid = u.user_guid
WHERE (u.exclude = 0 OR u.exclude IS NULL)
AND (d.exclude = 0 OR d.exclude IS NULL)
)
AS tmp_tbl ON c.dog_guid = tmp_tbl.dog_guid
GROUP BY country
ORDER BY number_of_customers DESC
And I need to add another variable that calculates the percentage of total
when I add
number_of_customers/SUM(number_of_customers)
or SUM(COUNT(DISTINCT tmp_tbl.user_guid)) / COUNT(DISTINCT tmp_tbl.user_guid)
it gives me error
Analytic functions come in handy here. Assuming you are using MySQL 8+:
SELECT country,
COUNT(DISTINCT tmp_tbl.user_guid) AS number_of_customers,
100.0 * COUNT(DISTINCT tmp_tbl.user_guid) /
SUM(COUNT(DISTINCT tmp_tbl.user_guid)) OVER () AS pct_customers
FROM complete_tests c
INNER JOIN
(
SELECT DISTINCT d.dog_guid, u.user_guid, u.country
FROM dogs d
INNER JOIN users u ON d.user_guid = u.user_guid
WHERE (u.exclude = 0 OR u.exclude IS NULL) AND
(d.exclude = 0 OR d.exclude IS NULL)
) AS tmp_tbl
ON c.dog_guid = tmp_tbl.dog_guid
GROUP BY
country
ORDER BY
number_of_customers DESC;
I'm using the following query to sort, using Order by for 4 different ranks: id, avg_rating, total_sent & points
However, I'm having trouble with including all the members in the results. I'd like to include all members, including the ones that have 0: total_sent, total_received, points, avg_rating, votes
Please help me understand what I'm missing. Thank you.
SELECT m.id,
m.Name,
m.City,
m.Zip_Code,
m.url,
r.avg_rating,
r.votes,
froms.from_ct total_sent,
tos.to_ct total_received,
froms.from_ct - tos.to_ct `points`
FROM members m
JOIN (
SELECT id_rated,
avg(rating) avg_rating,
count(*) votes
FROM member_ratings
GROUP BY id_rated
) r ON r.id_rated = m.id
JOIN ( SELECT id_from, COUNT(*) AS from_ct FROM member_points GROUP BY 1
) AS froms ON froms.id_from = m.id
JOIN ( SELECT id_received, COUNT(*) AS to_ct FROM member_points GROUP BY 1
) AS tos ON tos.id_received = m.id
WHERE m.Account_Active = 'TRUE'
GROUP BY m.id,
m.Name
ORDER BY `avg_rating` DESC;
Use LEFT JOINs for all tables
SELECT m.id,
m.Name,
m.City,
m.Zip_Code,
m.url,
r.avg_rating,
r.votes,
froms.from_ct total_sent,
tos.to_ct total_received,
froms.from_ct - tos.to_ct `points`
FROM members m
LEFT JOIN (
SELECT id_rated,
avg(rating) avg_rating,
count(*) votes
FROM member_ratings
GROUP BY id_rated
) r ON r.id_rated = m.id
LEFT JOIN ( SELECT id_from, COUNT(*) AS from_ct FROM member_points GROUP BY 1
) AS froms ON froms.id_from = m.id
LEFT JOIN ( SELECT id_received, COUNT(*) AS to_ct FROM member_points GROUP BY 1
) AS tos ON tos.id_received = m.id
LEFT JOIN member_points mp ON mp.id_points = m.id
WHERE m.Account_Active = 'TRUE'
GROUP BY m.id,
m.Name
ORDER BY `avg_rating` DESC;
I want to add data from table b in table a but unfortunately full outer join do not work in mysql . I have also tried union but it is throwing errors because my statement has group by and order by keyword
SELECT COUNT( ReviewedBy ) AS TotalReviews, OrganizationId, SUM( Rating ) AS TotalStars, COUNT( Rating ) AS TotalRatings, (
SUM( Rating ) / COUNT( Rating )
) AS AverageRating
FROM `tbl_reviews`
WHERE ReviewType = 'shopper'
AND ReviewFor = 'org'
AND OrganizationId
IN (
SELECT OrganizationId
FROM tbl_organizations
WHERE CategoryID =79
)
GROUP BY OrganizationId
ORDER BY AverageRating DESC
This is what i'm getting from the above statement
I want to get organizationId 21 data in the result but i'm not getting result because it's not present in 'tbl_review' table
click here to see the table b
How can i get Desired result ?
You don't need a FULL, but a LEFT join:
SELECT COUNT( ReviewedBy ) AS TotalReviews, o.OrganizationId,
SUM( Rating ) AS TotalStars, COUNT( Rating ) AS TotalRatings,
(SUM( Rating ) / COUNT( Rating )) AS AverageRating
FROM tbl_organizations AS o
LEFT JOIN `tbl_reviews` AS r
ON o.OrganizationId = r.OrganizationId
AND ReviewType = 'shopper' -- conditions on inner table
AND ReviewFor = 'org' -- must be moved to ON
WHERE CategoryID =79
GROUP BY o.OrganizationId
ORDER BY AverageRating DESC
Why don't you use AVG instead of SUM/COUNT?
Have you tried:
from organization
left outer join tbl_reviews
on organization.ID = tbl_reviews.organization is
for your where clause? I don't think you need a full outer join in this case... A left outer join should do
In the following query:
SELECT
(SELECT nick FROM nicks n WHERE n.pid=p.id LIMIT 1 ORDER BY id DESC) as nick
, (
(
( SELECT COUNT(*) FROM kills k WHERE k.pid = p.id )
+
( SELECT COUNT(*) FROM votos v WHERE v.pid = p.id )
)
- (SELECT COUNT(*) FROM deaths d WHERE d.pid = p.id )
) as score
, (SELECT COUNT(*) FROM kills k WHERE k.pid = p.id ) as kills
, (SELECT COUNT(*) FROM deaths d WHERE d.pid = p.id ) as deaths
, (SELECT COUNT(*) FROM headshots h WHERE h.pid = p.id ) as headshots
, (SELECT COUNT(*) FROM votos v WHERE v.pid = p.id ) as reputation
FROM players p
WHERE p.uuid='STEAM_x:x:xxxxxx'
GROUP BY kills
This query works fine... but i think there exists a better way to do this.
Can anyone help me optimize this query?
Here is a somewhat better way to write the query:
SELECT p.*, (kills + reputation - deaths) as score
FROM (SELECT (SELECT nick FROM nicks n WHERE n.pid = p.id ORDER BY id DESC LIMIT 1
) as nick,
(SELECT COUNT(*) FROM kills k WHERE k.pid = p.id ) as kills,
(SELECT COUNT(*) FROM deaths d WHERE d.pid = p.id ) as deaths,
(SELECT COUNT(*) FROM headshots h WHERE h.pid = p.id ) as headshots,
(SELECT COUNT(*) FROM votos v WHERE v.pid = p.id ) as reputation
FROM players p
WHERE p.uuid = 'STEAM_x:x:xxxxxx'
) p
GROUP BY kills;
Note: I don't understand what the GROUP BY is doing. You are only aggregating by one column, so the rest of the columns have indeterminate values. Perhaps you intend ORDER BY.
I am guessing that the overhead for materializing the subquery before the group by is slightly less than the additional subqueries. But your version may have very comparable performance.
For either version, you want the following indexes:
players(uuid)
kills(pid)
deaths(pid)
headshots(pid)
votos(pid)
I have the following:
SELECT DISTINCT s.username, COUNT( v.id ) AS cnt
FROM `instagram_item_viewer` v
INNER JOIN `instagram_shop_picture` p ON v.item_id = p.id
INNER JOIN `instagram_shop` s ON p.shop_id = s.id
AND s.expirydate IS NULL
AND s.isLocked =0
AND v.created >= '2014-08-01'
GROUP BY (
s.id
)
ORDER BY cnt DESC
Basically I have an instagram_item_viewer with the following structure:
id viewer_id item_id created
It tracks when a user has viewed an item and what time. So basically I wanted to find shops that has the most items viewed. I tried the query above and it executed fine, however it doesn't seem to give the appropriate data, it should have more count than what it is. What am I doing wrong?
First, with a group by statement, you don't need the DISTINCT clause. The grouping takes care of making your records distinct.
You may want to reconsider the order of your tables. Since you are interested in the shops, start there.
Select s.username, count(v.id)
From instagram_shop s
INNER JOIN instagram_shop_picture p ON p.shop_id = s.shop_id
INNER JOIN instagram_item_viewer v ON v.item_id = p.id
AND v.created >= '2014-08-01'
WHERE s.expirydate IS NULL
AND s.isLocked = 0
GROUP BY s.username
Give thata shot.
As mentioned by #Lennart, if you have a sample data it would be helpful. Because otherwise there will be assumptions.
Try run this to debug (this is not the answer yet)
SELECT s.username, p.id, COUNT( v.id ) AS cnt
FROM `instagram_item_viewer` v
INNER JOIN `instagram_shop_picture` p ON v.item_id = p.id
INNER JOIN `instagram_shop` s ON p.shop_id = s.id
AND s.expirydate IS NULL
AND s.isLocked =0
AND v.created >= '2014-08-01'
GROUP BY (
s.username, p.id
)
ORDER BY cnt DESC
The problem here is the store and item viewer is too far apart (i.e. bridged via shop_picture). Thus shop_picture needs to be in the SELECT statement.
Your original query only gets the first shop_picture count for that store that is why it is less than expected
Ultimately if you still want to achieve your goal, you can expand my SQL above to
SELECT x.username, SUM(x.cnt) -- or COUNT(x.cnt) depending on what you want
FROM
(
SELECT s.username, p.id, COUNT( v.id ) AS cnt
FROM `instagram_item_viewer` v
INNER JOIN `instagram_shop_picture` p ON v.item_id = p.id
INNER JOIN `instagram_shop` s ON p.shop_id = s.id
AND s.expirydate IS NULL
AND s.isLocked =0
AND v.created >= '2014-08-01'
GROUP BY (
s.username, p.id
)
ORDER BY cnt DESC
) x
GROUP BY x.username