complex join query in mysql - mysql

Hy everyone,
I really need your help. I have to obtain the list (distinct) of all users who :
- have a live checkin ( checkins.ctype = 'live' ) in a match
- where they favorite team ( see fanusers_teams )
- won by 3 ore more goals difference.
The favorite team, could be info_matches.team_id1 OR info_matches.team_id2 or even both.
Here is a small design for the involved tables :
What I've tried, works 80% (so it doesn't :( ) because it returns some users correct (they fav. teams by 3+ goals diff) , but also returns users which don't have a fav. team in the situation. I think that they are returned because they've made a live checkin for a match where one team or the other has won by a 3+ goals diff.
Here is my query :
SELECT DISTINCT
f.id
FROM
fanusers f
LEFT JOIN
checkins c ON f.id = c.fanuser_id
LEFT JOIN
info_matches m ON m.id = c.match_id
WHERE
c.ctype = 'live' AND
(
m.team_id1 IN(
SELECT DISTINCT
m1.team_id1
FROM
info_matches m1
RIGHT JOIN
fanusers_teams ft ON m1.team_id1 = ft.team_id
RIGHT JOIN
fanusers f ON f.id = ft.fanuser_id
WHERE
m1.pointsteam1 - m1.pointsteam2 >= 3
)
OR
m.team_id2 IN(
SELECT DISTINCT
m2.team_id2
FROM
info_matches m2
RIGHT JOIN
fanusers_teams ft ON m2.team_id2 = ft.team_id
RIGHT JOIN
fanusers f ON f.id = ft.fanuser_id
WHERE
m2.pointsteam2 - m2.pointsteam1 >= 3
)
)
I would appreciate also a small explanation regarding what am I doing wrong, if there is someone who succeed to solve this query.
Thanks.

Something like this should work:
SELECT DISTINCT f.id
FROM fanusers f
JOIN checkins c
ON f.id = c.fanuser_id
JOIN fanusers_teams ft
ON f.id = ft.fanuser_id
JOIN info_matches m
ON m.id = c.match_id
AND
(
(ft.team_id = m.team_id1 AND pointsteam1 - pointsteam2 >= 3)
OR
(ft.team_id = m.team_id2 AND pointsteam2 - pointsteam1 >= 3)
)
WHERE c.ctype = 'live'

I've found also a way to solve the query, but not so nice as Tom did it.
My way :
SELECT DISTINCT
f.id AS user_id
FROM
fanusers f
LEFT JOIN
checkins c ON f.id = c.fanuser_id
LEFT JOIN
info_matches m ON m.id = c.match_id
WHERE
c.ctype = 'live'
AND
m.matchisfinished = 1
AND
c.fanuser_id NOT IN ( SELECT DISTINCT
f1.id
FROM
fanusers f1
LEFT JOIN
fanusers_stickers fs
ON f1.id = fs.fanuser_id
WHERE
fs.sticker_id = 35
)
AND
(
( m.team_id1 IN ( SELECT DISTINCT
ft.team_id
FROM
fanusers_teams ft
INNER JOIN
checkins c1
ON
ft.fanuser_id = c1.fanuser_id
WHERE
f.id = c1.fanuser_id
)
AND
m.pointsteam1-m.pointsteam2>=3
)
OR
(
m.team_id2 IN ( SELECT DISTINCT
ft.team_id
FROM
fanusers_teams ft
INNER JOIN
checkins c1
ON
ft.fanuser_id = c1.fanuser_id
WHERE
f.id = c1.fanuser_id
)
AND
m.pointsteam2-m.pointsteam1>=3
)
)
So for the moment, even if I'm happy that I've solved the query by myself, I will use Tom's query :).

Related

mysql query not giving accurate result

I am working on a query whose purpose is to get the records of all the students whose financialyear_id!=4 and don't dispaly records even if he/she has finacialyear_id other then 4 exist.
I have written a query but it gives me the record of that student whose finacialyear_id!=4 but I want to achieve that no records will be shown if financialyear_id=4 exist for any student.
SELECT a.id aid
, s.id sid
, s.name
, s.father_name
, s.cnic
, f.financialyear_id
FROM student s
JOIN academic_info a
ON a.s_id = s.id
LEFT
JOIN fee_issued f
ON a.id = f.academic_info_id
WHERE f.financialyear_id != 4
AND a.is_data_locked = 0
AND a.university_id = 60;
Foreign Key: s_id in both tables academic_info and fee_issued,academic_info_id in fee_issued table.
You used a keyword in your request: "but I want to achieve that no records will be shown if financialyear_id=4 exist for any student." So use EXISTS (or IN which does about the same) to check for existence.
As you are using MySQL you must write the select condition for academic_info twice. Other DBMS handle this more elegantly.
select a.id as aid, s.id as sid, s.name, s.father_name, s.cnic, f.financialyear_id
from student s
join academic_info a on a.s_id = s.id and a.is_data_locked = 0 and a.university_id = 60
left join fee_issued f on f.academic_info_id = a.id
where s.id not in
(
select ai.s_id
from academic_info ai
join fee_issued fi on fi.academic_info_id = ai.id and fi.financialyear_id != 4
where ai.is_data_locked = 0 and ai.university_id = 60
);
Above query also gets you students that have no fee_issued at all. If you want these removed, change the left join to an inner join.
EDIT: Here is the same with NOT EXISTS.
select a.id as aid, s.id as sid, s.name, s.father_name, s.cnic, f.financialyear_id
from student s
join academic_info a on a.s_id = s.id and a.is_data_locked = 0 and a.university_id = 60
left join fee_issued f on f.academic_info_id = a.id
where not exists
(
select *
from academic_info ai
join fee_issued fi on fi.academic_info_id = ai.id and fi.financialyear_id != 4
where ai.is_data_locked = 0 and ai.university_id = 60
and ai.s_id = s.id
);

How can I optimise an extremely slow MySQL query that uses COUNT DISTINCT

I have a very slow MySQL query that I would like to optimise.
The query is taking 66.2070 seconds to return 5 results from tables containing around 200 rows.
The database tables store users, experiments (A/B tests), goals (page URLs), visits (page visits) and conversions (clicks a goal's URL). The visit and conversion tables both have a combination column that records if version A or B of a page was visited or a conversion came from version A or B. Combinations are stored in the db as 1 or 2.
I'm trying to get a list of a user's experiments with the number of visits and conversions for each combination.
For some relationships I'm using composite primary keys, which does make the joins more complicated. I doubt it but could this be the cause of the problem?
How can I rewrite this query to make it run in a reasonable time, at least less than a second?
Here's my database schema:
and her's my query:
SELECT e.id AS id,
e.name AS name,
e.status AS status,
e.created AS created,
Count(DISTINCT v1.id) AS visits1,
Count(DISTINCT v2.id) AS visits2,
Count(DISTINCT c1.id) AS conversions1,
Count(DISTINCT c2.id) AS conversions2
FROM experiment e
LEFT JOIN visit v1
ON ( v1.experiment_id = e.id
AND v1.user_id = e.user_id
AND v1.combination = 1 )
LEFT JOIN visit v2
ON ( v2.experiment_id = e.id
AND v2.user_id = e.user_id
AND v2.combination = 2 )
LEFT JOIN goal g
ON ( g.experiment_id = e.id
AND g.user_id = e.user_id
AND g.principal = 1 )
LEFT JOIN conversion c1
ON ( c1.experiment_id = e.id
AND c1.user_id = e.user_id
AND c1.goal_id = g.id
AND c1.combination = 1 )
LEFT JOIN conversion c2
ON ( c2.experiment_id = e.id
AND c2.user_id = e.user_id
AND c2.goal_id = g.id
AND c2.combination = 2 )
WHERE e.user_id = 25
GROUP BY e.id
ORDER BY e.created DESC
LIMIT 5
The resulting table should look something like this:
You should do the aggregations before doing the joins, to avoid getting large intermediate results. I think the logic is
SELECT e.id, e.name, e.status, e.created,
v.visits1, v.visits2, g.conversions1, g.conversions2
FROM experiment e LEFT JOIN
(SELECT experiment_id, user_id,
SUM(combination = 1) as visits1,
SUM(combination = 2) as visits2
FROM visits
WHERE combination IN (1, 2)
GROUP BY experiment_id, user_id
) v
ON v.experiment_id = e.id AND
v.user_id = e.user_id LEFT JOIN
(SELECT g.experiment_id, g.user_id,
SUM(c.combination = 1) as conversions1,
SUM(c.combination = 2) as conversions2
FROM goal g LEFT JOIN
conversion c
ON c.experiment_id = g.experiment_id AND
c.user_id = g.user_id AND
c.goal_id = g.id
WHERE g.principal = 1
GROUP BY g.experiment_id, g.user_id
) g
ON g.experiment_id = e.id AND
g.user_id = e.user_id LEFT JOIN
WHERE e.user_id = 25
ORDER BY e.created DESC
LIMIT 5 ;
There are further optimizations for this. For instance, an index on experiment(user_id, created, id).
For your question about the drawback of using composite keys I found this:
Drawback of composite keys
I can't currently test ur database but use the EXPLAIN syntax in mysql to find out what is wrong with the perfomance of ur query:
MySQL docs about EXPLAIN and optimizing ur query with EXPLAIN

Count matched words from IN operator

i have this little mysql query :
select t.title FROM title t
inner join movie_keyword mk on mk.movie_id = t.id
inner join keyword k on k.id = mk.keyword_id
where k.keyword IN (
select k.keyword
FROM title t
inner join movie_keyword mk on mk.movie_id = t.id
inner join keyword k on k.id = mk.keyword_id
where t.id = 166282
)
LIMIT 15
as you can see it will return all titles from title that have at least one the same keyword that have movie with id 166282.
Now i have problem, because i want also count how many keywords was matched in IN operator(let's say i want to see only titles that have 3 or more the same keywords), i tried something with aggregate functions, but everything failed, so i came here with my problem. Maybe somebody can give me some advice, or code example.
I'm not also sure, if this "subquery way" is good, so if there are some better options how i should solve my problem, I am open to any suggestions or tips.
Thank you!
#Edit
So after some problems, i have one more. This is my current query :
SELECT s.title,s.vote,s.rating,count(dk.key) as keywordCnt, count(dg.name) as genreCnt
FROM series s
INNER JOIN series_has_genre shg ON shg.series_id = s.id
INNER JOIN dict_genre dg ON dg.id = shg.dict_genre_id
INNER JOIN series_has_keyword shk ON shk.series_id = s.id
INNER JOIN dict_keyword dk ON dk.id = shk.dict_keyword_id
WHERE dk.key IN (
SELECT dki.key FROM series si
INNER JOIN series_has_keyword shki ON shki.series_id = si.id
INNER JOIN dict_keyword dki ON dki.id = shki.dict_keyword_id
WHERE si.title LIKE 'The Wire'
)
and dg.name IN (
SELECT dgo.name FROM series so
INNER JOIN series_has_genre shgo ON shgo.series_id = so.id
INNER JOIN dict_genre dgo ON dgo.id = shgo.dict_genre_id
WHERE so.title LIKE 'The Wire'
)
and s.production_year > 2000
GROUP BY s.title
ORDER BY s.vote DESC, keywordCnt DESC ,s.rating DESC, genreCnt DESC
LIMIT 5
Problem is, it is very, very, very slow. Any tips what i should change, to run it faster ?
Will this work for you:
select t.title, count(k.keyword) as keywordCount FROM title t
inner join movie_keyword mk on mk.movie_id = t.id
inner join keyword k on k.id = mk.keyword_id
where k.keyword IN (
select ki.keyword
FROM title ti
inner join movie_keyword mki on mki.movie_id = ti.id
inner join keyword ki on ki.id = mki.keyword_id
where ti.id = 166282
) group by t.title
LIMIT 15
Note that I have changed the table names inside the nested query to avoid confusion.

MySQL: IS NULL in joined tables

I'm having some difficulty with my JOIN and IS NULL..
Basically what I want to do is find all members who are not apart of certain group category ID's that I specify.
SELECT m.* FROM elvanto_members AS m
LEFT JOIN elvanto_groups AS g ON g.deleted = 0
LEFT JOIN elvanto_groups_categories AS gc ON gc.group_id = g.id AND (gc.category_id = '1' OR gc.category_id = '2')
WHERE gr.id IS NULL
Some members aren't apart of any group categories at all which is why I've made it a LEFT JOIN.
Am I making sense? Do you have any idea how to fix this?
Try this:
SELECT m.* FROM elvanto_members AS m
WHERE
not exists
(
SELECT 1 FROM
elvanto_groups AS g
INNER JOIN elvanto_groups_categories AS gc ON
gc.group_id = g.id
WHERE
gc.category_id IN ('1','2') AND
g.id = m.group_id AND
g.deleted = 0
)
SELECT m.*
FROM elvanto_members AS m
LEFT JOIN elvanto_groups AS g ON g.id = m.group_id AND g.deleted = 0
LEFT JOIN elvanto_groups_categories AS gc ON gc.group_id = g.id
AND gc.category_id IN ('1','2')
WHERE gc.group_id IS NULL
GROUP BY m.*
I filled in on spec what was missing in the question.
Is category_id really a string type? I would expect it to be numeric. Then this expression should be:
AND gc.category_id IN (1,2)

MySQL query optimization

Just wondering what's a better way to write this query. Cheers.
SELECT r.user_id AS ID, m.prenom, m.nom
FROM `0_rank` AS l
LEFT JOIN `0_right` AS r ON r.rank_id = l.id
LEFT JOIN `0_user` AS m ON r.user_id = m.id
WHERE r.section_id = $section_id
AND l.rank = '$rank_name' AND depart_id IN
(SELECT depart_id FROM 0_depart WHERE user_id = $user_id AND section_id = $section_id)
GROUP BY r.user_id
Here are the table structures:
0_rank: id | section_id | rank_name |
other_stuffs
0_user: id | prenom | nom | other_stuffs
0_right: id | section_id | user_id |
rank_id | other_stuffs
0_depart: id | section_id | user_id | depart_id
| other_stuffs
The idea is to use the same in a function like:
public function usergroup($section_id,$rank_name,$user_id) {
// mysql query goes here to get a list of appropriate users
}
Update: I think I have not been able to express myself clearly earlier. Here is the most recent query that seems to be working.
SELECT m.id, m.prenom, m.nom,
CAST( GROUP_CONCAT( DISTINCT d.depart ) AS char ) AS deps,
CAST( GROUP_CONCAT( DISTINCT x.depart ) AS char ) AS depx
FROM `0_rank` AS l
LEFT JOIN `0_right` AS r ON r.rank_id = l.id
LEFT JOIN `0_member` AS m ON r.user_id = m.id
LEFT JOIN `0_depart` AS d ON m.id = d.user_id
LEFT JOIN `0_depart` AS x ON x.user_id = $user_id
WHERE r.section = $section_id
AND l.rank = '$rank_name'
GROUP BY r.user_id ORDER BY prenom, nom
Now I want to get only those result, where all entries of deps are present in entries in depx.
In other term, every user is associated with some departs. $user_id is also an user is associated with some departs.
I want to get those users whose departs are common to the departs of $user_id.
Cheers.
Update
I'm not sure without being able to see the data but I believe this query will give you the results you want the fastest.
SELECT m.id, m.prenom, m.nom,
CAST( GROUP_CONCAT( DISTINCT d.depart ) AS char ) AS deps,
FROM `0_rank` AS l
LEFT JOIN `0_right` AS r ON r.rank_id = l.id and r.user_id = $user_id
LEFT JOIN `0_member` AS m ON r.user_id = m.id
LEFT JOIN `0_depart` AS d ON m.id = d.user_id
WHERE r.section = $section_id
AND l.rank = '$rank_name'
GROUP BY r.user_id ORDER BY prenom, nom
Let me know if this works.
Try this:
(By converting the functionality of the IN (SELECT...) to an inner join, you get exactly the same results but it might be the optimizer will make better choices.)
SELECT r.user_id AS ID, m.prenom, m.nom
FROM `0_rank` AS l
LEFT JOIN `0_right` AS r ON r.rank_id = l.id and r.section_id = 2
LEFT JOIN `0_user` AS m ON r.user_id = m.id
INNER JOIN `0_depart` AS x ON l.section_id = x.section_id and x.user_id = $user_id AND x.section_id = $section_id
WHERE l.rank = 'mod'
GROUP BY r.user_id
I also moved the constraints on 0_right to the join statement because I think that is clearer -- presumably this change won't matter to the optimizer.
I know nothing about your DB structure but your subselect looks like it can be replaced with a simple INNER JOIN against whatever table has the depart column. MySQL is well known for its poor subquery optimization.
Without knowing the structures or indexes, I would first add "STRAIGHT_JOIN" if the critical criteria is in-fact from the 0-rank table. Then, ensure 0_rank has an index on "rank". Next, ensure the 0_right has an index on rank_id at a minimum, but rank_id, section to take advantage of BOTH your criteria. Index on 0_member on id.
Additionally, do you mean left-join (ie: record only required in the 0_rank or 0_member) on the respective 0_right and 0_member tables instead of a normal join (where BOTH tables must match on their IDs).
Finally, ensure index on the depart table on user_id.
SELECT STRAIGHT_JOIN
r.user_id AS ID,
m.prenom,
m.nom
FROM
0_rank AS l
LEFT JOIN `0_right` AS r
ON l.id = r.rank_id
AND r.section = 2
LEFT JOIN `0_member` AS m
ON r.user_id = m.id
WHERE
l.rank = 'mod'
AND depart IN (SELECT depart
FROM 0_depart
WHERE user_id = 2
AND user_sec = 2)
GROUP BY
r.user_id
---- revised post from feedback.
From the parameters you are listing, you are always including the User ID... If so, I would completely restructure it to get whatever info is for that user. Each user should apparently can be associated to multiple departments and may or may NOT match the given rank / department / section you are looking for... I would START the query with the ONE USER because THAT will guarantee a single entry, THEN tune-down to the other elements...
select STRAIGHT_JOIN
u.id,
u.prenom,
u.nom,
u.other_stuffs,
rank.rank_name
from
0_user u
left join 0_right r
on u.id = r.user_id
AND r.section_id = $section_id
join 0_rank rank
on r.rank_id = rank.id
AND rank.rank_name = '$rank_name'
left join 0_dept dept
on u.id = dept.user_id
where
u.id = $user_id
Additionally, I have concern about your table relationships and don't see a legit join to the department table...
0_user
0_right by User_ID
0_rank by right.rank_id
0_dept has section which could join to rank or right, but nothing to user_id directly
Run explain on the query - it will help you find where the caveats are:
EXPLAIN SELECT r.user_id AS ID, m.prenom, m.nom
FROM 0_rank AS l
LEFT JOIN `0_right` AS r ON r.rank_id = l.id
LEFT JOIN `0_member` AS m ON r.user_id = m.id
WHERE r.section = 2
AND l.rank = 'mod' AND depart IN
(SELECT depart FROM 0_depart WHERE user_id = 2 AND user_sec = 2)
GROUP BY r.user_id\G