Simplify slow MySQL query - mysql

This query calculates the columns free,plus,score and total based on the COUNT of columns in subquery.
SELECT movie_title,movie_id,MAX(x.free_cnt) as free, MAX(x.plus_cnt) as plus,
(MAX(x.free_cnt) + (MAX(x.plus_cnt)*3)) AS score, (MAX(x.free_cnt) + MAX(x.plus_cnt)) AS total
FROM (
SELECT b.id as movie_id, b.movie_title as movie_title, COUNT(*) AS free_cnt, 0 as plus_cnt
FROM subtitles_request a1
LEFT JOIN movies b on a1.movie_id=b.id
JOIN users c on c.email=a1.email
WHERE c.subsc_status='0'
GROUP BY b.movie_title
UNION ALL
SELECT d.id as movie_id, d.movie_title as movie_title, 0 as free_cnt, COUNT(*) AS plus_cnt
FROM subtitles_request a2
LEFT JOIN movies d on a2.movie_id=d.id
JOIN users e on e.email=a2.email
WHERE e.subsc_status='1'
GROUP BY d.movie_title
) AS x
GROUP BY movie_title
ORDER BY total DESC
LIMIT 10
It is slow performing and i'm wondering is there anyway i can simplify or change the query to speed up performance. I can't calculate the free,plus,score ,total columns outside of query due to being able to order by. Also i may incorporate date.
Anyway to simplify this query?

Try this:
SELECT b.movie_title, x.movie_id, MAX( x.free_cnt ) AS free, MAX( x.plus_cnt ) AS plus,
( MAX( x.free_cnt ) + ( MAX( x.plus_cnt ) * 3 ) ) AS score, ( MAX( x.free_cnt ) + MAX( x.plus_cnt ) ) AS total
FROM ( SELECT a.movie_id,
SUM( IF( c.subsc_status = '0', 1, 0 ) ) AS free_cnt,
SUM( IF( c.subsc_status = '1', 1, 0 ) ) AS plus_cnt
FROM subtitles_request a1
JOIN users c on c.email=a1.email
WHERE c.subsc_status in ('0','1')
GROUP BY a.movie_id
) AS x
LEFT JOIN movies b on x.movie_id = b.id
GROUP BY movie_title, movie_id
ORDER BY total DESC
LIMIT 10
Maybe I've simplified a bit too much. Moreover, I'm not used to grouping on only some of the non-aggregate fields, hence I added movie_id to what is being grouped by and thus changing your query a bit (if two films had the same name, but different ID, then only one of the id's would be returned in your original query, but I guess (being a MySQL newbie, I really don't know) the counts would be for both of them taken together).
HTH,
Set

Well, I have check your the subquery:
SELECT b.id as movie_id, b.movie_title as movie_title, COUNT(*) AS free_cnt, 0 as plus_cnt
FROM subtitles_request a1
LEFT JOIN movies b on a1.movie_id=b.id
JOIN users c on c.email=a1.email
WHERE c.subsc_status='0'
GROUP BY b.movie_title
UNION ALL
SELECT d.id as movie_id, d.movie_title as movie_title, 0 as free_cnt, COUNT(*) AS plus_cnt
FROM subtitles_request a2
LEFT JOIN movies d on a2.movie_id=d.id
JOIN users e on e.email=a2.email
WHERE e.subsc_status='1'
GROUP BY d.movie_title
The statement beside "UNION ALL" can be replaced with one statement with condition at c.subsc_status IN('0','1'). And you can try to use "CASE WHEN" statement at 0 as free_cnt, COUNT(*) AS plus_cnt, just like IFNULL((CASE WHEN e.subsc_status='1' THEN COUNT(*)),0) as free_cnt. It's not a complicated sql statement, I don't think it will take too much time to query. Is there too many datas?
As a matter of fact, I'm also a newer, but I just have some experence about it. Please forgive me if it doesn't work.

Related

Writing MSSQL (2019) query equivalent to MySQL query

I have written the following MySQL query in my project to select all tests table record with matching test_change table record (whichever is the first) for a data display purpose.
I am in need of the MSSQL equivalent query but am unable to form the query because of my limited knowledge in it. I tried the website http://www.sqlines.com/online but I did not help me.
This is the MySQL Query:
SELECT
tests.*,
cases.title,
users.name,
statuses.label as status_label,
statuses.color_dark,
tc.id as change_id,
tc.created_on
FROM
tests
left join (
select
MIN(created_on) as created_on,
test_id,
id,
assignedto_id
from
test_changes
group by
test_id
) tc on tests.id = tc.test_id
LEFT JOIN users ON tc.assignedto_id = users.id
LEFT JOIN cases ON tests.case_id = cases.id
LEFT JOIN statuses ON tests.status_id = statuses.id
WHERE
tests.id is not null
AND tests.run_id IN (22)
AND (
tests.status_id = 3
or tests.status_id = 4
or (
tests.status_id != 3
and tc.created_on > 1620950399
)
)
GROUP BY
tests.id
ORDER BY
users.name DESC
LIMIT
15, 20
This is the MSSQL Query I tried...
SELECT
tests.*,
cases.title,
users.name,
statuses.label as status_label,
statuses.color_dark,
tc.id as change_id,
tc.created_on
FROM
tests
left join (
select
MIN(created_on) as created_on,
status_id,
test_id,
id,
assignedto_id
from
test_changes
group by
test_id
) tc on tests.id = tc.test_id
LEFT JOIN users ON tc.assignedto_id = users.id
LEFT JOIN cases ON tests.case_id = cases.id
LEFT JOIN statuses ON tests.status_id = statuses.id
WHERE
tests.id is not null
AND tests.run_id IN (22)
AND (
tests.status_id = 3
or tests.status_id = 4
or (
tests.status_id != 3
and tc.created_on > 1620950399
)
)
GROUP BY
tests.id
ORDER BY
users.name DESC OFFSET 15 ROWS FETCH NEXT 20 ROWS ONLY
It is throwing the following error...
Column 'test_changes.status_id' is invalid in the select list because
it is not contained in either an aggregate function or the GROUP BY
clause.
Can someone help me in resolving the error and forming this MSSQL query?
even your first query in mysql gives you the same error , you can't select columns that are not aggregated or part of group by , when you group by.
so seems like you nmeed to group by assignedto_id and test_id as well:
select
MIN(created_on) as created_on,
status_id,
test_id,
--id, <-- removed this column , looks not used in query
assignedto_id
from
test_changes
group by
test_id, status_id,assignedto_id -- < adding new columns to group by
) tc on ....
it might not be what you are looking for , but gives you the idea how it works

Simplify mysql union query

I have this query, which is slow. I get the count of each select and define them as columns free_cnt,plus_cnt, so I can manipulate in the parent select query.
$query = $this->db->query("SELECT movie_title,movie_id,MAX(x.free_cnt) as free_cnt, MAX(x.plus_cnt) as plus_cnt, ((MAX(x.free_cnt)*1) + (MAX(x.plus_cnt)*3)) AS score, (MAX(x.free_cnt) + MAX(x.plus_cnt)) AS total
FROM (
SELECT b.id as movie_id, b.movie_title as movie_title, COUNT(*) AS free_cnt, 0 as plus_cnt
FROM preview_movie_request a1
LEFT JOIN movies b on a1.movie_id=b.id
JOIN users c on c.email=a1.email
WHERE c.subsc_status='0' AND c.package_type='' AND b.movie_type=2 $where1
GROUP BY b.movie_title
UNION ALL
SELECT d.id as movie_id, d.movie_title as movie_title, 0 as free_cnt, COUNT(*) AS plus_cnt
FROM preview_movie_request a2
LEFT JOIN movies d on a2.movie_id=d.id
JOIN users e on e.email=a2.email
WHERE e.subsc_status='1' AND e.package_type!='' AND d.movie_type=2 $where2
GROUP BY d.movie_title
UNION ALL
) AS x
GROUP BY movie_title
$orderby
$limit");
Is there anyway to simplify query and make faster?
You can eliminate one subquery moving the count conditions in case
SELECT movie_title,movie_id,MAX(x.free_cnt) as free_cnt, MAX(x.plus_cnt) as plus_cnt, ((MAX(x.free_cnt)*1) + (MAX(x.plus_cnt)*3)) AS score, (MAX(x.free_cnt) + MAX(x.plus_cnt)) AS total FROM
(
SELECT b.id as movie_id, b.movie_title as movie_title,
SUM(CASE WHEN c.subsc_status='0' AND c.package_type='' AND b.movie_type=2 $where1 THEN 1 END ) AS free_cnt,
SUM(CASE WHEN c.subsc_status='1' AND c.package_type!='' AND b.movie_type=2 $where2 THEN 1 END ) AS plus_cnt,
FROM preview_movie_request a1
LEFT JOIN movies b on a1.movie_id=b.id
JOIN users c on c.email=a1.email
GROUP BY b.movie_title)x
GROUP BY movie_title
$orderby
$limit

Joining two columns in mysql

I want to add data from table b in table a but unfortunately full outer join do not work in mysql . I have also tried union but it is throwing errors because my statement has group by and order by keyword
SELECT COUNT( ReviewedBy ) AS TotalReviews, OrganizationId, SUM( Rating ) AS TotalStars, COUNT( Rating ) AS TotalRatings, (
SUM( Rating ) / COUNT( Rating )
) AS AverageRating
FROM `tbl_reviews`
WHERE ReviewType = 'shopper'
AND ReviewFor = 'org'
AND OrganizationId
IN (
SELECT OrganizationId
FROM tbl_organizations
WHERE CategoryID =79
)
GROUP BY OrganizationId
ORDER BY AverageRating DESC
This is what i'm getting from the above statement
I want to get organizationId 21 data in the result but i'm not getting result because it's not present in 'tbl_review' table
click here to see the table b
How can i get Desired result ?
You don't need a FULL, but a LEFT join:
SELECT COUNT( ReviewedBy ) AS TotalReviews, o.OrganizationId,
SUM( Rating ) AS TotalStars, COUNT( Rating ) AS TotalRatings,
(SUM( Rating ) / COUNT( Rating )) AS AverageRating
FROM tbl_organizations AS o
LEFT JOIN `tbl_reviews` AS r
ON o.OrganizationId = r.OrganizationId
AND ReviewType = 'shopper' -- conditions on inner table
AND ReviewFor = 'org' -- must be moved to ON
WHERE CategoryID =79
GROUP BY o.OrganizationId
ORDER BY AverageRating DESC
Why don't you use AVG instead of SUM/COUNT?
Have you tried:
from organization
left outer join tbl_reviews
on organization.ID = tbl_reviews.organization is
for your where clause? I don't think you need a full outer join in this case... A left outer join should do

wrong count on the multiple joins with the same table

i have 2 tables
1 - coupons
2 - tractions
for each coupon there might be couple of rows in tractions table
I want to have list of all coupons and count of its tractions under different condition
SELECT `coupons`.`id` ,
count( tractions_all.id ) AS `all` ,
count( tractions_void.id ) AS void,
count( tractions_returny.id ) AS returny,
count( tractions_burned.id ) AS burned
FROM `coupons`
LEFT JOIN `tractions` AS `tractions_all`
ON `coupons`.`id` = `tractions_all`.`coupon_parent`
LEFT JOIN `tractions` AS `tractions_void`
ON `coupons`.`id` = `tractions_void`.`coupon_parent`
AND `tractions_void`.`expired` =1
LEFT JOIN `tractions` `tractions_returny`
ON `tractions_returny`.`coupon_parent` = `coupons`.`id`
AND `tractions_returny`.`expired` =11
LEFT JOIN `tractions` `tractions_burned`
ON `tractions_burned`.`coupon_parent` = `coupons`.`id`
AND `tractions_burned`.`expired` =0
AND '2014-02-12'
WHERE `coupons`.`parent` =0
GROUP BY `coupons`.`id`
right now only one of my coupons has 2 traction on both are burned traction other coupons have no tractions at all
here is the result
as you can see coupon with id=13 has 4 traction while it should be 2 ... what am i doing wrong ? if i remove the last join it works fine and i get 2
You are aggregating along multiple dimensions at one time, resulting in a cartesian product for each id.
If your data volume is not very large, the easiest way to fix this is using distinct:
SELECT `coupons`.`id` ,
count(distinct tractions_all.id ) AS `all` ,
count(distinct tractions_void.id ) AS void,
count(distinct tractions_returny.id ) AS returny,
count(distinct tractions_burned.id ) AS burned
If your data is large, then you will probably need to aggregate values as subqueries first and then do the joins.

MySQL Query optimization required

I've got a slow performing query. I know using a dependent subquery is bad, but I can't think of another way to get the data I want.
Essentially, I want to flag customers who have at least 50 invoices in the past 6 months, but no invoices this month.
This is what I have currently:
select
Customer.name,
Customer.id,
Customer.latitude,
Customer.longitude
from
Customer
where
EXISTS (
SELECT
*
FROM
Invoice_Header
WHERE
Invoice_Header.inv_date BETWEEN '2011-03-02' AND '2011-10-02'
AND
Invoice_Header.account_number = Customer.account_number
HAVING COUNT(invoice_num) > 50
)
AND NOT EXISTS (
SELECT *
FROM
Invoice_Header
WHERE
InvHead.inv_date > '2011-10-02'
AND
InvHead.account_number = Customer.account_number
)
Group by name;
Customer table has about 12k record, Invoice_Header has about 2mill records.
I have indexes on inv_date, account_number (in both tables).
Any suggestions for how to speed this up would be appreciated.
This should eliminate the correlated subqueries and be significantly faster:
SELECT c.name, c.id, c.latitude, c.longitude
FROM Customer c
INNER JOIN (
SELECT account_number
FROM Invoice_Header ih
WHERE ih.inv_date BETWEEN '2011-03-02' AND '2011-10-02'
GROUP BY account_number
HAVING COUNT(*) > 50
MINUS
SELECT DISTINCT account_number
FROM Invoice_Header ih
WHERE ih.inv_date > '2011-10-02'
) tbl
ON tbl.account_number = c.account_number
I would suggest:
SELECT
c.name,
c.id,
c.latitude,
c.longitude
FROM
Customer AS c
INNER JOIN (
SELECT account_number, count(*) AS invoice_count
FROM Invoice_Header
WHERE inv_date >= '2011-03-02' AND inv_date <= '2011-10-02'
GROUP BY account_number
) AS lsm
ON c.account_number = lsm.account_number
LEFT JOIN (
SELECT account_number, count(*) AS invoice_count
FROM Invoice_Header
WHERE inv_date > '2011-10-02'
GROUP BY account_number
) AS lm
ON c.account_number = lm.account_number
WHERE
lsm.invoice_count >= 50
AND IFNULL(lm.invoice_count, 0) = 0
select
C.name,
C.id,
C.latitude,
C.longitude,
I.account_number,
count( IF(I.inv_date>='2011-03-02' AND I.inv_date <='2011-10-02',I.inv_date,NULL )) as inv_count_6,
count( IF(I.inv_date > '2011-10-02',I.inv_date,NULL )) as inv_count_1
from Customer C
LEFT JOIN Invoice_Header I
ON C.account_number = I.account_number
GROUP BY C.id, I.account_number
HAVING inv_count_6 >= 50 AND inv_count_1=0
WHERE I.inv_date BETWEEN '2011-03-02' AND '2011-10-02'
Notes:
1.The invoices is AT LEAST 50. so the condition is >=50 not >50.
2.You have to add index to the column inv_date
Try run your query with explain and see if other indexs are needed .