strange MySQL join query results with aggregate functions - mysql

i wrote the following join query to get a report using aggregate functions
SELECT users.id, SUM(orders.totalCost) AS bought, COUNT(comment.id) AS commentsCount, COUNT(topics.id) AS topicsCount, COUNT(users_login.id) AS loginCount, COUNT(users_download.id) AS downloadsCount
FROM users
LEFT JOIN orders ON users.id=orders.userID AND orders.payStatus=1
LEFT JOIN comment ON users.id=comment.userID
LEFT JOIN topics ON users.id=topics.userID
LEFT JOIN users_login ON users.id=users_login.userID
LEFT JOIN users_download ON users.id=users_download.userID
GROUP BY users.id
ORDER BY bought DESC
but i don't know why i get the following output?
the result of aggregate functions are multiplied with each other!!!
i don't know why?
for example for the last row i expected the following result
821 | 48000 | 63 | 0 | 10 | 10
the result of executing EXPLAIN query are shown below

One reason for that type of result would be you are using a left joins with your users table and the result set may contains the duplicate rows for each user so you are getting count more than the expected one for this you can use DISTINCT in count to count only the unique associations per user and for sum of totalCost you can use a subselect to have a sum for each user without having repeated values for orders of user
SELECT
u.id,
COALESCE(o.bought,0) bought
COUNT(DISTINCT c.id) AS commentsCount,
COUNT(DISTINCT t.id) AS topicsCount,
COUNT(DISTINCT ul.id) AS loginCount,
COUNT(DISTINCT ud.id) AS downloadsCount
FROM users u
LEFT JOIN (SELECT
userID,
SUM(totalCost) bought
FROM orders
WHERE payStatus=1
GROUP BY userID) o
ON u.id=o.userID
LEFT JOIN `comment` c ON u.id=c.userID
LEFT JOIN topics t ON u.id=t.userID
LEFT JOIN users_login ul ON u.id=ul.userID
LEFT JOIN users_download ud ON u.id=ud.userID
GROUP BY u.id
ORDER BY bought DESC

Related

How to join 3 tables and get sum from result of individual joins?

I have the following scenario:
Table: users : user_id, username ,...
Table: login: user_id, login_date, ...
Table: point: user_id, points, point_time
Joins will be on the basis of users.user_id with other tables.
Now, I want to get count of all the logins as well as sum of all the points earned by the user.
Now, when I do:
select users.user_id,count(*) from users
inner join login on users.user_id=login.user_id
group by users.user_id
It returns count as 36(for example).
Whenever I run:
select users.user_id,count(*),sum(points) from users
inner join point on users.user_id=point.user_id
group by users.user_id
It returns sum as 400(for example) and count as 2.
But if I combine both the queries:
select users.user_id,count(*),sum(points) from users
inner join login on users.user_id=login.user_id
inner join point on users.user_id=point.user_id
group by users.user_id
It returns count as 72 (36 * 2) and sum as 800 (400 *2).
Twice because of multiple userIds present.
I tried several things like combining with distincts but nothing seems to work. Please help.Better if it's possible with joins alone. Thanks in advance. I am using mysql with Php.
You can sum the points in a subquery and select distinct logins in the count
select users.user_id,l.login,p.points from users
inner join (select user_id, count(1) login from login
group by login) as l on users.user_id=login.user_id
inner join (select user_id, sum(point) as point
from point group by user_id ) as p on users.user_id=point.user_id
You should be able to do your count by joining in your login table and then including a subquery to get your count of points:
select users.user_id, count(*) as login_count,
(select sum(points) from point
where point.user_id = users.user_id) as points_sum
from users
inner join login on users.user_id=login.user_id
group by users.user_id

MySql query to get count of days spent in each country for each purpose? (Get count of all record in second table present in first table)

I have three tables tl_log, tl_geo_countries,tl_purpose. I am trying to get the count of number of days spent in each country in table 'tl_log' for each purpose in table 'tl_purpose'.
I tried below mysql query
SELECT t.country_id AS countryID,t.reason_id AS reasonID,count(t.reason_id) AS
days,c.name AS country, p.purpose AS purpose
FROM `tl_log` AS t
LEFT JOIN tl_geo_countries AS c ON t.country_id=c.id
LEFT JOIN tl_purpose AS p ON t.reason_id=p.id
GROUP BY t.reason_id,t.country_id ORDER BY days DESC
But landed up with.
I am not able to get the count for purpose for each country in 'tl_log' that is not present in table 'tl_log'. Any help is greatly appreciated. Also, Please let me know if the question is difficult to understand.
Expected Output:
Below is the structure of these three tables
tl_log
tl_geo_countries
tl_purpose
If you want all possible combination of countries and purposes, even those that do not appear on the log table (these will be shown with a count of 0), you can do first a cartesian product of the two tables (a CROSS join) and then LEFT join to the log table:
SELECT
c.id AS countryID,
p.id AS reasonID,
COUNT(t.reason_id) AS days,
c.name AS country,
p.purpose AS purpose
FROM
tl_geo_countries AS c
CROSS JOIN
tl_purpose AS p
LEFT JOIN
tl_log AS t
ON t.country_id = c.id
AND t.reason_id = p.id
GROUP BY
p.id,
c.id
ORDER BY
days DESC ;
If you want the records for only the countries that are present in the log table (but still all possible reason/purposes), a slight modification is needed:
SELECT
c.id AS countryID,
p.id AS reasonID,
COUNT(t.reason_id) AS days,
c.name AS country,
p.purpose AS purpose
FROM
( SELECT DISTINCT
country_id
FROM
tl_log
) AS dc
JOIN
tl_geo_countries AS c
ON c.id = dc.country_id
CROSS JOIN
tl_purpose AS p
LEFT JOIN
tl_log AS t
ON t.country_id = c.id
AND t.reason_id = p.id
GROUP BY
p.id,
c.id
ORDER BY
days DESC ;
LEFT JOIN should be replaced by RIGHT JOIN

sql selecting a lot of counts

I have a problem driving me nuts for the last 2 days. I basically have 4 tables with inheritance in the following order:
users
|
categories blogs
| | |
---- pages visits
So a user has many blogs which has many pages and visits. Each page also belongs to a category.
All I want is to extract all users with the following counts associated:
total number of blogs each user has
total number of pages each user has
total number of categories each user has blogs in
total number of visits each user has
total number of visitors each user has (visits but we count by distinct ip_address)
My query is as follows:
SELECT
u.id
u.username,
COUNT(b.id) as blogs_count,
COUNT(p.id) as pages_count,
COUNT(v.id) as visits_count,
COUNT(distinct ip_address) as visitors_count
COUNT(c.id) as categories_count
FROM
users u
LEFT JOIN
blogs b ON(b.user_id=u.id)
LEFT JOIN
pages p ON(p.blog_id=b.id)
LEFT JOIN
visits v ON(v.blog_id=b.id)
LEFT JOIN
categories c ON(v.category_id=c.id)
GROUP BY u.id, blogs_count, pages_count, visits_count,
visitors_count, categories_count
I should get 24 users with their counts but, given the fact that I have almost 300,000 visits I get my SQL database hanging in forever probably trying to pull millions of rows.
I'm not a db guru and it's obvious. Can someone point me to the right direction somehow so I can make a good query able to perform well on even millions of records (with the right hardware of course)?
Try this:
SELECT u.id,
u.username,
COUNT(b.id) AS blogs_count,
COALESCE(MAX(p.pagecnt), 0) AS pages_count,
COALESCE(MAX(v.visitscnt), 0) AS visits_count,
COALESCE(MAX(v.visitorscnt), 0) AS visitors_count,
COALESCE(MAX(c.catcnt), 0) AS categories_count
FROM users u
LEFT JOIN blogs b ON u.id = b.user_id
LEFT JOIN (
SELECT blog_id,
COUNT(*) AS pagecnt
FROM pages
GROUP BY blog_id
) p ON b.id = p.blog_id
LEFT JOIN (
SELECT blog_id,
COUNT(*) AS visitscnt,
COUNT(DISTINCT ip_address) AS visitorscnt
FROM visits
GROUP BY blog_id
) v ON b.id = v.blog_id
LEFT JOIN (
SELECT aa.id,
COUNT(DISTINCT dd.id) AS catcnt
FROM users aa
JOIN blogs bb ON aa.id = bb.user_id
JOIN pages cc ON bb.id = cc.blog_id
JOIN categories dd ON cc.category_id = dd.id
GROUP BY aa.id
) c ON u.id = c.id
GROUP BY u.id,
u.username
Breakdown
This should also work across different DBMSs like PGSQL, SQL-Server, etc.
The challenge is that you have this sort of hierarchy of 1:M relationships in which joining them all together can easily throw off the different types of counts (as you want distinct counts in some places, but total counts in others).
What I've decided to do is first subselect the count of each page and visit / distinct visitors, grouping by the blog_id. This ensures that we get only one row per blog_id, even after joining the subselects on the blogs table.
For the category count, you want a count of distinct categories per user, but the challenge is that categories is linked deep within the relationship hierarchy (to the pages table), so you have to make a separate subselect that joins on the user_id instead of the blog_id.
Even with as many subselects as this query contains, it should still be quite fast as no two subselects are joining against each other. As long as there is an indexed table (subselects are actually unindexed temporary tables) on either side of the join, you should be fine.
SELECT
u.id
u.username,
COUNT(b.id) as blogs_count,
COUNT(p.id) as pages_count,
COUNT(v.id) as visits_count,
COUNT(distinct ip_address) as visitors_count
COUNT(c.id) as categories_count
FROM
users u
LEFT JOIN
blogs b ON(b.user_id=u.id)
LEFT JOIN
pages p ON(p.blog_id=b.id)
LEFT JOIN
visits v ON(v.blog_id=b.id)
LEFT JOIN
categories c ON(v.category_id=c.id)
GROUP BY u.id
Try with removing blogs_count, pages_count, visits_count, visitors_count, categories_count from your group by statment.

MySQL left join counts

I have a left join to a table and want to count columns from it, after grouping by a column of the parent table:
SELECT * , COUNT(list.id) AS listcount, COUNT(uploads.id) AS uploadcount
FROM members
LEFT JOIN lists ON members.id= list.mid
LEFT JOIN uploads ON members.id= uploads.mid
GROUP BY members.id
Assume that a user can have either lists or uploads based on the type of user. Then is above query good enough? If not why?
Or do I have to use this query?
SELECT * , l.listcount, u.uploadcount
FROM members
LEFT JOIN (select count(lists.id) as listscount,mid from lists group by mid) as l
on l.mid = m.id
LEFT JOIN (select count(uploads.id) as uploadscount
,mid from uploads group by mid) as u on u.mid = m.id
GROUP BY members.id
Or correlated subqueries?
SELECT *,
(select count(lists.id) as listscount from lists as l where l.mid = m.id
group by mid) as listcount
(select count(uploads.id) from uploads as u where u.mid = m.id
group by mid) as uploadscount
FROM members
GROUP BY members.id
And which is best solution?
The alias m for members is missing in query 2 and 3. Otherwise they should give the same numbers.
Query 2 (fixed) will perform fastest.
Query 1 is different in that it will give a higher number for uploads, if there are cases of multiple lists per member. After joining to lists, there will be multiple rows for a member too, which will increase the count for uploads. So query 1 is probably wrong.
Also, NULL values are not counted. The manual informs:
COUNT(expr)
Returns a count of the number of non-NULL values of expr in the rows
retrieved by a SELECT statement. The result is a BIGINT value.

MySQL group_concat with 2 joins returns unwanted results

When executing this query i expect te get 2 mobilenumbers and 1 category, instead i get 2 categories, what am i doing wrong?
I guess it has to do with the way i am joining things?
User, can have multiple imei's,
categoryjoin links a user to multiple categories
SELECT
u.*,
group_concat(i.mobilenumber) as mobilenumbers,
group_concat(c.name) as categories
FROM
users AS u
INNER JOIN
categoryjoin AS cj
ON
u.uid = cj.user_id
INNER JOIN
categories AS c
ON
cj.category_id = c.uid
INNER JOIN
imei AS i
ON
u.uid = i.user_id
GROUP BY
u.uid
Big pre-thanks you for your help!
If a user matches one category, but matches 2 rows in imei, then the category will be duplicated in the result set. You can get rid of redundant values from group_concat using DISTINCT:
SELECT
u.*,
group_concat(distinct i.mobilenumber) as mobilenumbers,
group_concat(distinct c.name) as categories