I have a statement that looks like:
QUERY A
UNION
QUERY B
ORDER BY SomeColumn
Query A and Query B each take a nominal amount of time to run, but when I put them in the UNION, it take 7-9 seconds which is unacceptable. In this case Query A returns 6 rows, Query B returns 7. So confusing...
I have absolutely no idea what would cause this, help would be much appreciated!
Here is an anonymized version of the script (I did not write this, so don't hate):
SELECT
'XXX' l_t,
s.p,
srst.c_b,
srd.a_d_f,
s.s_c,
sf.c_s,
srst.p_f,
s.s_r_i,
s.s_i,
s.s_d,
srd.m_s_d_l s_d,
srd.m_e_d_l e_d,
CASE WHEN (srs.s_s_t IS NOT NULL AND srs.s_s_t <> srs.s_e_t)
THEN 1 ELSE 0 END 's_f',
CASE WHEN (srs.c_s_t IS NOT NULL AND srs.c_s_t <> srs.c_e_t)
THEN 1 ELSE 0 END 'c_f',
r.r_i
FROM
t_s_r_d srd
INNER JOIN t_s s WITH (NOLOCK)
ON s.s_i = srd.s_i
INNER JOIN i_s_r(12345) r
ON r.r_i = srd.r_i
INNER JOIN i_s_s_f() sf
ON (sf.s_i = srd.s_i)
INNER JOIN t_s_r_s srst WITH (NOLOCK)
ON (srst.s_i = srd.s_i AND srst.r_i = srd.r_i )
LEFT OUTER JOIN t_s_r_s srs WITH (NOLOCK)
ON (srs.s_i = srd.s_i AND srs.r_i = srd.r_i)
WHERE
srst.d_f = 0
AND ((srd.m_s_d_l >= someval AND srd.m_s_d_l < someotherval)
OR
(srd.m_s_d_l <= someval AND srd.m_e_d_l > someotherval))
AND r.o_f = 0
AND r.i_f = 0
AND r.v_f = 1
AND r.g_i = 180
AND NOT EXISTS(SELECT * FROM t_c_r cdr WITH (NOLOCK) WHERE cdr.r_i = r.r_i)
UNION
SELECT
'XXX' l_t,
s.p,
srst.c_b,
srd.a_d_f,
s.s_c,
sf.c_s,
srst.p_f,
s.s_r_i,
s.s_i,
s.s_d,
srd.m_s_d_l s_d,
srd.m_e_d_l e_d,
CASE WHEN (srs.s_s_t IS NOT NULL AND srs.s_s_t <> srs.s_e_t)
THEN 1 ELSE 0 END 's_f',
CASE WHEN (srs.c_s_t IS NOT NULL AND srs.c_s_t <> srs.c_e_t)
THEN 1 ELSE 0 END 'c_f',
c.c_i
FROM
(t_s_r_d srd
INNER JOIN t_s s WITH (NOLOCK)
ON s.s_i = srd.s_i
INNER JOIN i_s_s_f() sf
ON (sf.s_i = srd.s_i)
LEFT OUTER JOIN t_s_r_s srs WITH (NOLOCK)
ON (srs.s_i = srd.s_i AND srs.r_i = srd.r_i)
INNER JOIN t_s_r_s srst WITH (NOLOCK)
ON (srst.s_i = srd.s_i AND srst.r_i = srd.r_i)),
i_s_c(12345) c
WHERE
srst.d_f = 0
AND ((srd.m_s_d_l >= someval AND srd.m_s_d_l < someotherval)
OR
(srd.m_s_d_l <= someval AND srd.m_e_d_l > someotherval))
AND c.o_f = 0
AND c.i_f = 0
AND c.v_f = 1
AND c.g_i = 180
AND EXISTS(SELECT * FROM t_c_r cr WITH (NOLOCK) WHERE cr.r_i = srd.r_i and
cr.c_i = c.c_i)
ORDER BY s_d
Because UNION removes duplicates it has to sort the whole data set first...try to use UNION ALL instead....it is much faster since it doesn't need to remove the dups
Turns out it was the old style ansii style joins in conjunction with the newer style explicit joins that were causing the lengthy return. Quite interesting, if anyone could provide a reason why that would be fantastic!
Related
I have the following SQL query that is bottlenecking at some point. It will make it halfway through around 900 records it returns before timing out. When I change my second select to only return certain columns it helped a bit but it is still bottlenecking somewhere.
I'm thinking there is something wrong with the way that I have the JOINs written but could use any advice.
SELECT * FROM
(SELECT * FROM TABLE1 B
INNER JOIN TABLE2 A ON B.ID_NBR = A.ID_NBR
INNER JOIN TABLE3 L ON A.CUST_NUM = L.CUST_NUM
WHERE B.SUPP_NBR = '17' AND STAT_NUM <> 4 AND BGHT_ID is not null
AND
NOT EXISTS (SELECT 1 FROM TABLE4 A WHERE A.ID_NBR = B.ID_NBR)) APPL
INNER JOIN TABLE5 ON M.PRSN_NUM = PRSN_NUM
WHERE M.LOC_ID = 'US' and (APPL.STAT_CD <> 'S' OR (APPL.STAT_CD =
'S' AND M.TXT_LOC !=
'UNKNOWN'));
UPDATE: I ended up using a LEFT JOIN instead of NOT EXISTS, changed the SELECT *, and used GROUP BY to correct the indexing issue:
SELECT * FROM
(SELECT B.ID_NBR, B.SUPP_NBR, B.STAT_NUM, B.BGHT_ID, A.CUST_NUM,
L.CUST_NUM, L.STAT_CD FROM TABLE1 B
INNER JOIN TABLE2 A ON B.ID_NBR = A.ID_NBR
INNER JOIN TABLE3 L ON A.CUST_NUM = L.CUST_NUM
LEFT JOIN TABLE4 C ON C.ID_NBR = B.ID_NBR
WHERE C.ID_NBR is null AND B.SUPP_NBR = '17' AND B.STAT_NUM <> 4
AND B.BGHT_ID is not null
GROUP BY B.ID_NBR, B.SUPP_NBR, B.STAT_NUM, B.BGHT_ID, A.CUST_NUM,
L.CUST_NUM, L.STAT_CD)
APPL
INNER JOIN TABLE5 ON M.PRSN_NUM = PRSN_NUM
WHERE M.LOC_ID = 'US' and
(APPL.STAT_CD <> 'S' OR M.TXT_LOC != 'UNKNOWN');
Can't this
(APPL.STAT_CD <> 'S'
OR (APPL.STAT_CD = 'S' AND M.TXT_LOC != 'UNKNOWN')
)
be simplified to
(APPL.STAT_CD <> 'S' OR M.TXT_LOC != 'UNKNOWN')
(It probably won't speed up the query much.)
Some of these indexes may help:
M: INDEX(LOC_ID, TXT_LOC, PRSN_NUM)
APPL: INDEX(STAT_CD)
B: INDEX(SUPP_NBR, ID_NBR)
A: INDEX(ID_NBR, CUST_NUM)
L: INDEX(CUST_NUM)
I’m trying to run an update query which I’ll run daily, unfortuntately I suck at SQL and have no clue what to do with the join functions. I keep getting invalid group error. I took the table names out
Managed to narrow down the issue (this is in SQLYog btw)
UPDATE CMCDAR
SET cmcdar.effective_exposure_in_global_currency = (mcdar.A_Rbalance - SUM(CASE
WHEN eps.fk_account_id = '90059' AND eps.sent_to_erp = '0' THEN (eps.transaction_amount) ELSE 0 END))
FROM eps
JOIN cmcdar ON eps.fk_customer_map_id = cmcdar.fk_customer_map_id
JOIN mcdar ON eps.fk_customer_map_id = mcdar.fk_customer_map_id
JOIN mcar ON eps.fk_customer_map_id = mcar.pk_customer_map_id
WHERE cmcdar.is_deleted = 0
AND mcar.is_deleted = 0
AND mcdar.is_deleted = 0
GROUP BY eps.fk_customer_map_id;
without Group by it returns it all as one bi
g row, with group by it fails. Interestingly with + rather than - in the sum it seems to seperate it even without group by. Any workarounds for group by?
Your syntax is off - see here:SQL Update with a Join
UPDATE
t1
SET
t1.c1 = t2.c2,
t1.c2 = expression,
...
FROM
t1
[INNER | LEFT] JOIN t2 ON join_predicate
WHERE
where_predicate;
I would assume move the join into from clause.
UPDATE cmcdar
SET effective_exposure_in_global_currency = (mcdar.A_Rbalance - SUM(CASE WHEN eps.fk_account_id =
'90059' AND eps.sent_to_erp = '0' THEN (eps.transaction_amount) ELSE 0 END))
from cmcdar
JOIN eps ON cmcdar.fk_customer_map_id = eps.fk_customer_map_id
JOIN mcar ON cmcdar.fk_customer_map_id = mcar.pk_customer_map_id
JOIN mcdar ON cmcdar.fk_customer_map_id = mcdar.fk_customer_map_id
WHERE cmcdar.is_deleted = 0
AND cmcdar.fk_customer_map_id = eps.fk_customer_map_id
AND mcar.is_deleted = 0
AND mcdar.is_deleted = 0
AND cmcdar.fk_account_id = '90059'
AND mcdar.fk_account_id = '90059';
I am having trouble with my query below
SELECT DATE(jc.`date_in`) DAY,
CASE
WHEN jb.`modified_fr` <> 0 THEN ROUND(SUM(jb.`modified_fr`),1)
ELSE ROUND(SUM(jd.`fr1`),1)
END AS AmountOfHoursBooked
,ds.`name` AS dealership, d.`name` AS Department
FROM jobcard jc
LEFT JOIN job jb
ON jc.`id` = jb.`jobcard`
LEFT JOIN job_definition jd
ON jb.`job_definition` = jd.`id`
LEFT JOIN department d
ON jc.`department` = d.`id`
JOIN dealership ds
ON d.`dealership` = ds.`id`
WHERE MONTH(jc.`date_in`) = MONTH(CURRENT_DATE())
AND YEAR(jc.`date_in`) = YEAR(CURRENT_DATE())
AND jd.`fr1` <> 0
AND jc.`status` <> 5
GROUP BY DATE(jc.`date_in`),jc.`department`;
what i need it to do is if the case comes back true it should use that modified fr instead of the original fr from jb table ?
what am I missing ?
thanks in advance
query taking 1 minute to fetch results
SELECT
`jp`.`id`,
`jp`.`title` AS game_title,
`jp`.`game_type`,
`jp`.`state_abb` AS game_state,
`jp`.`location` AS game_city,
`jp`.`zipcode` AS game_zipcode,
`jp`.`modified_on`,
`jp`.`posted_on`,
`jp`.`game_referal_amount`,
`jp`.`games_referal_amount_type`,
`jp`.`status`,
`jp`.`is_flaged`,
`u`.`id` AS employer_id,
`u`.`email` AS employer_email,
`u`.`name` AS employer_name,
`jf`.`name` AS game_function,
`jp`.`game_freeze_status`,
`jp`.`game_statistics`,
`jp`.`ats_value`,
`jp`.`integration_id`,
`u`.`account_manager_id`,
`jp`.`model_game`,
`jp`.`group_id`,
(CASE
WHEN jp.group_id != '0' THEN gm.group_name
ELSE 'NA'
END) AS group_name,
`jp`.`priority_game`,
(CASE
WHEN jp.country != 'US' THEN jp.country_name
ELSE ''
END) AS game_country,
IFNULL((CASE
WHEN
`jp`.`account_manager_id` IS NULL
OR `jp`.`account_manager_id` = 0
THEN
(SELECT
(CASE
WHEN
account_manager_id IS NULL
OR account_manager_id = 0
THEN
`u`.`account_manager_id`
ELSE account_manager_id
END) AS account_manager_id
FROM
user_user
WHERE
id = (SELECT
user_id
FROM
game_user_assigned
WHERE
game_id = `jp`.`id`
LIMIT 1))
ELSE `jp`.`account_manager_id`
END),
`u`.`account_manager_id`) AS acc,
(SELECT
COUNT(recach_limit_id)
FROM
recach_limit
WHERE
recach_limit = '1'
AND recach_limit_game_id = rpr.recach_limit_game_id) AS somewhatgame,
(SELECT
COUNT(recach_limit_id)
FROM
recach_limit
WHERE
recach_limit = '2'
AND recach_limit_game_id = rpr.recach_limit_game_id) AS verygamecommitted,
(SELECT
COUNT(recach_limit_id)
FROM
recach_limit
WHERE
recach_limit = '3'
AND recach_limit_game_id = rpr.recach_limit_game_id) AS notgame,
(SELECT
COUNT(joa.id) AS applicationcount
FROM
game_refer_to_member jrmm
INNER JOIN
game_refer jrr ON jrr.id = jrmm.rid
INNER JOIN
game_applied joa ON jrmm.id = joa.referred_by
WHERE
jrmm.STATUS = '1'
AND jrr.referby_user_id IN (SELECT
ab_testing_user_id
FROM
ab_testing)
AND joa.game_post_id = rpr.recach_limit_game_id
AND (rpr.recach_limit = 1
OR rpr.recach_limit = 2)) AS gamecount
FROM
(`game_post` AS jp)
JOIN
`user_info` AS u ON `jp`.`user_user_id` = `u`.`id`
JOIN
`game_functional` jf ON `jp`.`game_functional_id` = `jf`.`id`
LEFT JOIN
`group_musesm` gm ON `gm`.`group_id` = `jp`.`group_id`
LEFT JOIN
`recach_limit` rpr ON `jp`.`id` = `rpr`.`recach_limit_game_id`
WHERE
`jp`.`status` != '3'
GROUP BY `jp`.`id`
ORDER BY `posted_on` DESC
LIMIT 10
I would first suggest not nesting select statements because this will cause an n^x performance hit on every xth level and I see at least 3 levels of selects inside this query.
Add index
INDEX(status, posted_on)
Move LIMIT inside
Then, instead of saying
FROM (`game_post` AS jp)
say
FROM ( SELECT id FROM game_post
WHERE status != 3
ORDER BY posted_on DESC
LIMIT 10 ) AS ids
JOIN game_post AS jp USING(id)
(I am assuming that the PK of jp is (id)?)
That should efficiently use the new index to get the 10 ids needed. Then it will reach back into game_post to get the other columns.
LEFT
Also, don't say LEFT unless you need it. It costs something to generate NULLs that you may not be needing.
Is GROUP BY necessary?
If you remove the GROUP BY, does it show dup ids? The above changes may have eliminated the need.
IN(SELECT) may optimize poorly
Change
AND jrr.referby_user_id IN ( SELECT ab_testing_user_id
FROM ab_testing )
to
AND EXISTS ( SELECT * FROM ab_testing
WHERE ab_testing_user_id = jrr.referby_user_id )
(This change may or may not help, depending on the version you are running.)
More
Please provide EXPLAIN SELECT if you need further assistance.
I'm trying to divide the numeric results from 2 pretty different queries.
The end result should be Query 1 DIVIDED BY Query 2
Query 1 =
SELECT COUNT(DISTINCT(table1.ID)) AS count_1
FROM table1
INNER JOIN op
INNER JOIN Org
ON table1.EID = op.id
AND Op.OrgID = Org.ID
WHERE table1.TitleID = 123
AND op.BrandID = 1
AND op.Start <= NOW() AND op.End >= NOW();
Query 2 =
SELECT COUNT(DISTINCT user.id) AS count_2
FROM table1 INNER JOIN user INNER JOIN ur
ON table1.EID = user.id AND ur.userID = user.id
WHERE
user.BrandID = 1
AND table1.TitleID = 123
AND ur.role = 0
AND user.Inactive = 0;
Sure! You can use subselects to achieve this, though it will be pretty verbose!
SELECT
(
SELECT COUNT(DISTINCT(table1.ID)) AS count_1
FROM table1
INNER JOIN op
INNER JOIN Org
ON table1.EID = op.id
AND Op.OrgID = Org.ID
WHERE table1.TitleID = 123
AND op.BrandID = 1
AND op.Start <= NOW() AND op.End >= NOW()
) / (
SELECT COUNT(DISTINCT user.id) AS count_2
FROM table1 INNER JOIN user INNER JOIN ur
ON table1.EID = user.id AND ur.userID = user.id
WHERE
user.BrandID = 1
AND table1.TitleID = 123
AND ur.role = 0
AND user.Inactive = 0
);
Format however it feels the least ugly to you.
Use sub queries like this:
SELECT Q1.count_1 / Q2.Count_2
FROM
( ... Query1 ...) AS Q1
JOIN
( ... Query2 ...) AS Q2
ON 1=1
Replace Query1 and Query2 as your code.
Like this:
SELECT count_1 / count_2
FROM (SELECT COUNT(*) count_1 FROM foo) f
JOIN (SELECT COUNT(*) count_2 FROM bar) b ON 1=1;
http://sqlfiddle.com/#!2/c215e/1