Mysql explain - case study - mysql

I have the following explain info:
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 15
1 PRIMARY j eq_ref PRIMARY PRIMARY 4 tmp.sel_job_id 1
1 PRIMARY jd ref code code 768 test_query.j.j_code 2 Using index
2 DERIVED t index NULL date_2 9 NULL 15 Using where
4 DEPENDENT SUBQUERY jtl eq_ref Multiple_keys,tester_id Multiple_keys 8 test_query.t.id 1 Using where; Using index
3 DEPENDENT SUBQUERY jtl eq_ref Multiple_keys,tester_id Multiple_keys 8 test_query.t.id 1 Using where; Using index
T table has over 55k records and jtl table over 60k records, j table has over 100 records and jd 30 records.
My query is:
SELECT temp_tbl.*, j.name AS job_name, jd.code AS job_details_code FROM (
SELECT t.*, (
SELECT jtl.job_id FROM job_tester_link jtl
WHERE t.id = jtl.tester_id AND
t.id NOT IN (11220012, 11218876, 11340156) AND
jtl.job_id = '20' AND
t.active = 'Y' LIMIT 1
) sel_job_id
FROM tester t
WHERE (
SELECT jtl.job_id
FROM job_tester_link jtl
WHERE t.id = jtl.tester_id AND
t.id NOT IN (11220012, 11218876, 11340156) AND
jtl.job_id = '20' AND
t.active = 'Y'
LIMIT 1
) IS NOT NULL
ORDER BY t.date DESC
LIMIT 0,15
) temp_tbl
LEFT JOIN job j ON temp_tbl.sel_job_id = j.id
LEFT JOIN job_details jd ON j.j_code = jd.code;
My problem is the execution time (over 3 secs). How can this query be improved?
Thanks.

Related

Optimizing a big SQL query with inner joins, left joins and sub select

Problem
I have a big SQL query counting a bunch of relations and doing some calculations. However, in some cases, it runs forever (very very long, taking up 99% of DB CPU).
Research
I searched through SO and found posts like SO1, SO2, SO3 and others. The answer seems to be using indices to speed up the query. Further I learned to use explain to get more information about my query. My problem is, that my explain results tells me, that I am already using indices of some sort. However, I must not be using them effectively. Unfortunately, my understanding of them is also very limited. See the result of explain further down.
Background information
All tables have been generated by Doctrine. All present indices have also been created by the ORM.
Tables
Here are the tables used in the query as well as their primary and foreign keys
name
primary & foreign
a
id, ...
b
id, a_id, ...
c
id, b_id, ...
d
id, c_id, ...
e
id, d_id, ...
f
id, e_id, ...
g
id, e_id, ...
h
id, e_id, ...
i
id, e_id, ...
j
id, e_id, ...
k
id, e_id, ...
l
id, e_id, ...
m
id, e_id, ...
n
id, e_id, ...
o
id, e_id, ...
p_e
p_id, e_id
q
id, e_id, ...
r
id, ...
r_f
r_id, f_id
Query
select
e.id,
e.value_1,
count(DISTINCT f.id) as f_count,
count(DISTINCT g.id) as g_count,
count(DISTINCT h.id) as h_count,
count(DISTINCT i.id) as i_count,
count(DISTINCT j.id) as j_count,
count(DISTINCT k.id) as k_count,
count(DISTINCT l.id) as l_count,
count(DISTINCT m.id) as m_count,
count(DISTINCT n.id) as n_count,
count(DISTINCT o.id) as o_count,
count(DISTINCT p_e.p_id) as p_count,
count(DISTINCT q.id) as q_count,
m_counts.example_1_count,
m_counts.example_2_count,
m_counts.example_3_count,
m_counts.example_4_count,
m_counts.example_5_count,
m_counts.example_6_count,
m_counts.example_7_count,
m_counts.example_8_count,
m_counts.example_9_count,
m_counts.example_10_count,
m_counts.example_11_count,
m_counts.example_12_count,
m_counts.example_13_count,
case when e.value_1 = 'example'
then
0
else
case when count(DISTINCT f.id) > 0
then
(
select round(sum((
select
case
when f_inner.value_1 = 'a1' then f_inner.value_2 * sum(r.a1)
when f_inner.value_1 = 'a2' then f_inner.value_2 * sum(r.a2)
when f_inner.value_1 = 'a3' then f_inner.value_2 * sum(r.a3)
when f_inner.value_1 = 'a4' then f_inner.value_2 * sum(r.a4)
when f_inner.value_1 = 'a5' then f_inner.value_2 * sum(r.a5)
when f_inner.value_1 = 'a6' then f_inner.value_2 * sum(r.a6)
when f_inner.value_1 = 'a7' then f_inner.value_2 * sum(r.a7)
when f_inner.value_1 = 'a8' then f_inner.value_2 * sum(r.a8)
when f_inner.value_1 = 'a9' then f_inner.value_2 * sum(r.a9)
when f_inner.value_1 = 'a10' then f_inner.value_2 * sum(r.a10)
when f_inner.value_1 = 'a11' then f_inner.value_2 * count(r.id)
when f_inner.value_1 = 'a12' then f_inner.value_2
else 0
end
from f as f_inner
inner join f_r as f_e on f_r.f_id = f_inner.id
inner join r as r on r.id = f_r.r_id
where f_inner.id = f_2.id
)), 4)
from e as e_inner
inner join f as f_2 on e_inner.id = f_2.e_id
where e_inner.id = e.id
)
else
0
end
end as f_value_1
from e as e
inner join d as d on d.id = e.d_id
inner join c as c on c.id = d.c_id
inner join b as b on b.id = c.b_id
inner join a as a on a.id = b.a_id
left join f as f on e.id = f.e_id
left join g as g on e.id = g.e_id
left join h as h on e.id = h.e_id
left join i as i on e.id = i.e_id
left join j as j on e.id = j.e_id
left join k as k on e.id = k.e_id
left join l as l on e.id = l.e_id
left join m as m on e.id = m.e_id
left join n as n on e.id = n.e_id
left join o as o on e.id = o.e_id
left join p_e as p_e on e.id = p_e.e_id
left join q as q on e.id = q.e_id
left outer join (
select e_2.id,
sum(case when m_2.type = 'ex1' then 1 else 0 end) as example_1_count,
sum(case when m_2.type = 'ex2' then 1 else 0 end) as example_2_count,
sum(case when m_2.type = 'ex3' then 1 else 0 end) as example_3_count,
sum(case when m_2.type = 'ex4' then 1 else 0 end) as example_4_count,
sum(case when m_2.type = 'ex5' then 1 else 0 end) as example_5_count,
sum(case when m_2.type = 'ex6' then 1 else 0 end) as example_6_count,
sum(case when m_2.type = 'ex7' then 1 else 0 end) as example_7_count,
sum(case when m_2.type = 'ex8' then 1 else 0 end) as example_8_count,
sum(case when m_2.type = 'ex9' then 1 else 0 end) as example_9_count,
sum(case when m_2.type = 'ex10' then 1 else 0 end) as example_10_count,
sum(case when m_2.type = 'ex11' then 1 else 0 end) as example_11_count,
sum(case when m_2.type = 'ex12' then 1 else 0 end) as example_12_count,
sum(case when m_2.type = 'ex13' then 1 else 0 end) as example_13_count
from e as e_2
left join m as m_2 on m_2.e_id = e_2.id
group by e_2.id
) as m_counts on m_counts.id = e.id
where a.id = 504
group by e.id;
Explain Result
id
select_type
table
type
possbile_keys
key
key_len
eq_ref
rows
filtered
Extra
1
PRIMARY
a
const
PRIMARY
PRIMARY
4
const
1
100.00
Using index; Using temporary; Using filesort
1
PRIMARY
b
ref
PRIMARY,IDX_548D5BBD166D1F9C
IDX_548D5BBD166D1F9C
5
const
10
100.00
Using index
1
PRIMARY
c
ref
PRIMARY,IDX_D09734DB23BCD4D0
IDX_D09734DB23BCD4D0
5
db.t.id
7
100.00
Using index
1
PRIMARY
d
ref
PRIMARY,IDX_5B45FC0CCD16E022
IDX_5B45FC0CCD16E022
5
db.ta.id
22
100.00
Using index
1
PRIMARY
e
ref
PRIMARY,IDX_34DA0F079245DE54,IDX_34DA0F0796D072AA,...
IDX_34DA0F0796D072AA
5
db.tw.id
1
100.00
NULL
1
PRIMARY
f
ref
IDX_6AE5926BDF10B100
IDX_6AE5926BDF10B100
5
db.e.id
2
100.00
Using index
1
PRIMARY
g
ref
IDX_C87F908ADF10B100
IDX_C87F908ADF10B100
5
db.e.id
4
100.00
Using index
1
PRIMARY
h
ref
IDX_E4C5E45BDF10B100
IDX_E4C5E45BDF10B100
5
db.e.id
3
100.00
Using index
1
PRIMARY
i
ref
IDX_33FF3C4DF10B100
IDX_33FF3C4DF10B100
5
db.e.id
2
100.00
Using index
1
PRIMARY
j
ref
IDX_3B909093DF10B100
IDX_3B909093DF10B100
5
db.e.id
1
100.00
Using index
1
PRIMARY
k
ref
IDX_9B50917CDF10B100
IDX_9B50917CDF10B100
5
db.e.id
5
100.00
Using index
1
PRIMARY
l
ref
IDX_10631447DF10B100
IDX_10631447DF10B100
5
db.e.id
6
100.00
Using index
1
PRIMARY
m
ref
IDX_31E107EFDF10B100
IDX_31E107EFDF10B100
5
db.e.id
2
100.00
Using index
1
PRIMARY
n
ref
IDX_CF3E3AECDF10B100
IDX_CF3E3AECDF10B100
5
db.e.id
1
100.00
Using index
1
PRIMARY
o
ref
IDX_C13107E4DF10B100
IDX_C13107E4DF10B100
5
db.e.id
5
100.00
Using index
1
PRIMARY
p_e
ref
PRIMARY,IDX_3027915D51906758
IDX_3027915D51906758
4
db.e.id
1
100.00
Using index
1
PRIMARY
q
ref
IDX_7185325951906758
IDX_7185325951906758
5
db.e.id
1
100.00
Using index
1
PRIMARY
ref
<auto_key0>
<auto_key0>
4
db.e.id
10
100.00
NULL
4
DERIVED
e_2
index
PRIMARY,IDX_34DA0F079245DE54,IDX_34DA0F0796D072AA,...
PRIMARY
4
NULL
339578
100.00
Using index
4
DERIVED
m_2
ref
IDX_31E107EFDF10B100
IDX_31E107EFDF10B100
5
db.e_2.id
2
100.00
NULL
2
DEPENDENT SUBQUERY
e_inner
eq_ref
PRIMARY
PRIMARY
4
func
1
100.00
Using index
2
DEPENDENT SUBQUERY
f_2
ref
IDX_6AE5926BDF10B100
IDX_6AE5926BDF10B100
5
func
2
100.00
Using where; Using index
3
DEPENDENT SUBQUERY
f_inner
eq_ref
PRIMARY
PRIMARY
4
db.f_2.id
1
100.00
NULL
3
DEPENDENT SUBQUERY
f_e
ref
PRIMARY,IDX_21EE4B441666F235,IDX_21EE4B441F1F2A24
IDX_21EE4B441666F235
4
db.f_2.id
18
100.00
Using where; Using index
3
DEPENDENT SUBQUERY
e
eq_ref
PRIMARY
PRIMARY
4
db.f_e.r_id
1
100.00
NULL
More thoughts
The present indices, as far as I understand, are for the foreign key relations between the tables. I believe that I require more indices than those, but I have no clue how/where to put them. I also read that left joins can be very costly, but if the relations might be null or non existant, they are required, from what I've read. Am I wrong with this assumption? Are there better ways to count these kinds of relations?
Also, those performance issues, only occur for some data examples (for some a.id) and not for others. I have not found out yet, which scenarios the query is fast and which take forever.

You probably get huge, unrealistic, values for the counts.
Instead do it this way:
SELECT ( SELECT COUNT(*) FROM a WHERE ... ) AS a_count,
( SELECT COUNT(*) FROM b WHERE ... ) AS b_count,
...
;
I did not fully understand what the goal is. I don't know what belongs in the WHERE clauses. If I failed to point you in the right direction, please simplify the code to 2-3 counts and provide sample input and output.
If you need the counts to be in columns in the output, see the tag [pivot-table].

mysql Return duplicates and exclude lowest id

I am trying list all the rows from my query that will return all the duplicates next to each other so I can then grab their id's but i also want to exclude the id with the lowest number from the results. How can I go about doing that with my query.
My Query
SELECT
a.tail_number,
min(a.id),
b.aircraft_id
from aircraft a
left join jobs b on a.id = b.aircraft_id
where
a.active = 1 and b.aircraft_id is null
group by a.tail_number having count(*) > 1
The current Output
tail_number min(a.id) aircraft_id tail_count
125TH 4429 NULL 7
362FX 4223 NULL 7
439FL 4221 NULL 7
453FX 4220 NULL 7
455FX 4259 NULL 7
The output im trying to achieve
tail_number min(a.id) aircraft_id tail_count
125TH 4429 NULL 1
125TH 4430 NULL 1
125TH 4431 NULL 1
125TH 4432 NULL 1
362FX 4223 NULL 1
362FX 4224 NULL 1
362FX 4225 NULL 1
362FX 4226 NULL 1

Join with a subquery that gets the lowest ID for each tail number, and then exlude that from the results in the ON condition.
SELECT a.tail_number, a.id
FROM aircraft AS a
JOIN (SELECT tail_number, MIN(id) AS minid
FROM aircraft
WHERE active = 1
GROUP BY tail_number
HAVING COUNT(*) > 1) AS m ON a.tail_number = m.tail_number AND a.id != m.minid
LEFT JOIN jobs AS j ON a.id = j.aircraft_id
WHERE j.aircraft_id IS NULL
ORDER BY a.tail_number, a.id
I've moved the checks for active = 1 and COUNT(*) > 1 into the subquery as well, since there's no longer any grouping in the main query.

Optimize MYSQL subqueries

Is there anyway to optimize this query?
I know it is getting held up in the subqueries:
(SELECT IFNULL(max(datetimesql),'NS') FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000032)
Without the IFNULL subquery it executes in less than 1 sec. This example takes 20 secs with the subquerys and is only 54 rows. Anyway to rewrite this to speed it up? Thanks for any and all help.
SELECT
SQL_CALC_FOUND_ROWS #storeid:=z.id
,z.biz_name
,z.wf_store_name
,z.e_address
,z.e_city
,z.e_state
,z.e_postal
,IFNULL(total_sales - prev_total_sales,'CV') as diff_total_sales
,IFNULL(d_source,'N/A') as d_source
,IFNULL(unit_sales1 - prev_unit_sales1 , (SELECT IFNULL(max(datetimesql),'NS')
FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000032)
) as diff_unit_sales1
,IFNULL(unit_sales2 - prev_unit_sales2, (SELECT IFNULL(max(datetimesql),'NS')
FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000033)
) as diff_unit_sales2
,IFNULL(unit_sales3 - prev_unit_sales3, (SELECT IFNULL(max(datetimesql),'NS')
FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000034)
) as diff_unit_sales3
,IFNULL(unit_sales4 - prev_unit_sales4, (SELECT IFNULL(max(datetimesql),'NS')
FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000035)
) as diff_unit_sales4
,IFNULL(unit_sales5 - prev_unit_sales5, (SELECT IFNULL(max(datetimesql),'NS')
FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000036)
) as diff_unit_sales5
,IFNULL(unit_sales6 - prev_unit_sales6, (SELECT IFNULL(max(datetimesql),'NS')
FROM storeCheckRecords
WHERE store_id=#storeid AND upc=855555000038)
) as diff_unit_sales6
FROM
(SELECT s1.id,s1.biz_name as biz_name
,s1.wf_store_name as wf_store_name
,s1.e_address as e_address
,s1.e_city as e_city
,s1.e_state as e_state
,s1.e_postal as e_postal
,sum(s2.unit_sales) as total_sales
,sum(s2.unit_sales/4.28571428571) as week_avg
,group_concat(DISTINCT s2.d_source separator ',') as d_source
,SUM(CASE u.id WHEN 1 THEN s2.unit_sales ELSE NULL END) AS unit_sales1
,SUM(CASE u.id WHEN 2 THEN s2.unit_sales ELSE NULL END) AS unit_sales2
,SUM(CASE u.id WHEN 3 THEN s2.unit_sales ELSE NULL END) AS unit_sales3
,SUM(CASE u.id WHEN 4 THEN s2.unit_sales ELSE NULL END) AS unit_sales4
,SUM(CASE u.id WHEN 5 THEN s2.unit_sales ELSE NULL END) AS unit_sales5
,SUM(CASE u.id WHEN 6 THEN s2.unit_sales ELSE NULL END) AS unit_sales6
FROM allStores as s1
INNER JOIN storeCheckRecords AS s2
ON s1.id = s2.store_id
AND s2.datetimesql BETWEEN '2015-07-01' AND '2015-07-31'
AND s1.key_retailer LIKE 'WHOLE FOODS'
AND s1.wf_region LIKE 'Midwest'
INNER JOIN (SELECT 1 AS id
,'855555000032' AS upc
UNION
SELECT 2
,'855555000033'
UNION
SELECT 3
,'855555000034'
UNION
SELECT 4
,'855555000035'
UNION
SELECT 5
,'855555000036'
UNION
SELECT 6
,'855555000038') u
ON u.upc = s2.upc
GROUP BY s1.id) x
LEFT OUTER JOIN
(SELECT s1.id,s1.biz_name as prev_biz_name
,s1.wf_store_name as prev_wf_store_name
,s1.e_address as prev_e_address
,s1.e_city as prev_e_city
,s1.e_state as prev_e_state
,s1.e_postal as prev_e_postal
,sum(s2.unit_sales) as prev_total_sales
,sum(s2.unit_sales/4.28571428571) as prev_week_avg
,group_concat(DISTINCT s2.d_source separator ',') as prev_d_source
,SUM(CASE u.id WHEN 1 THEN s2.unit_sales ELSE 0 END) AS prev_unit_sales1
,SUM(CASE u.id WHEN 2 THEN s2.unit_sales ELSE 0 END) AS prev_unit_sales2
,SUM(CASE u.id WHEN 3 THEN s2.unit_sales ELSE 0 END) AS prev_unit_sales3
,SUM(CASE u.id WHEN 4 THEN s2.unit_sales ELSE 0 END) AS prev_unit_sales4
,SUM(CASE u.id WHEN 5 THEN s2.unit_sales ELSE 0 END) AS prev_unit_sales5
,SUM(CASE u.id WHEN 6 THEN s2.unit_sales ELSE 0 END) AS prev_unit_sales6
FROM allStores as s1
INNER JOIN storeCheckRecords AS s2
ON s1.id = s2.store_id
AND s2.datetimesql BETWEEN '2015-06-01' AND '2015-06-30'
AND s1.key_retailer LIKE 'WHOLE FOODS'
AND s1.wf_region LIKE 'Midwest'
INNER JOIN (SELECT 1 AS id
,'855555000032' AS upc
UNION
SELECT 2
,'855555000033'
UNION
SELECT 3
,'855555000034'
UNION
SELECT 4
,'855555000035'
UNION
SELECT 5
,'855555000036'
UNION
SELECT 6
,'855555000038') u
ON u.upc = s2.upc
GROUP BY s1.id) y
ON x.id = y.id
RIGHT JOIN
(SELECT s1.id
,s1.biz_name
,s1.wf_store_name
,s1.e_address
,s1.e_city
,s1.e_state
,s1.e_postal
FROM allStores as s1
WHERE 1
AND s1.key_retailer LIKE 'WHOLE FOODS'
AND s1.wf_region LIKE 'Midwest') z
ON y.id = z.id
ORDER BY wf_store_name ASC
LIMIT 0, 1000
Thank you Henry. I indexed upc, store_id, and datetimsql. This cut it in half, but I want to get it down further. Here is the explain and I am trying to get everything done in sql because I feeding the result to datatables.
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived22> ALL NULL NULL NULL NULL 54 Using temporary; Using filesort
1 PRIMARY <derived8> ALL NULL NULL NULL NULL 6
1 PRIMARY <derived15> ALL NULL NULL NULL NULL 6
22 DERIVED s1 ALL NULL NULL NULL NULL 64121 Using where
15 DERIVED <derived16> ALL NULL NULL NULL NULL 6 Using temporary; Using filesort
15 DERIVED s2 ref upc,store_id,upc_2,store_id_2,datetimesql,datetime... upc 47 u.upc 2159 Using where
15 DERIVED s1 eq_ref PRIMARY PRIMARY 4 dpalame_foodiecpg.s2.store_id 1 Using where
16 DERIVED NULL NULL NULL NULL NULL NULL NULL No tables used
17 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
18 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
19 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
20 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
21 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
NULL UNION RESULT <union16,17,18,19,20,21> ALL NULL NULL NULL NULL NULL
8 DERIVED <derived9> ALL NULL NULL NULL NULL 6 Using temporary; Using filesort
8 DERIVED s2 range upc,store_id,upc_2,store_id_2,datetimesql,datetime... datetimesql 3 NULL 1810 Using where; Using join buffer
8 DERIVED s1 eq_ref PRIMARY PRIMARY 4 dpalame_foodiecpg.s2.store_id 1 Using where
9 DERIVED NULL NULL NULL NULL NULL NULL NULL No tables used
10 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
11 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
12 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
13 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
14 UNION NULL NULL NULL NULL NULL NULL NULL No tables used
NULL UNION RESULT <union9,10,11,12,13,14> ALL NULL NULL NULL NULL NULL
7 UNCACHEABLE SUBQUERY storeCheckRecords index upc,upc_2 datetimesql_3 53 NULL 60452 Using where; Using index
6 UNCACHEABLE SUBQUERY storeCheckRecords index upc,upc_2 datetimesql_3 53 NULL 60452 Using where; Using index
5 UNCACHEABLE SUBQUERY storeCheckRecords index upc,upc_2 datetimesql_3 53 NULL 60452 Using where; Using index
4 UNCACHEABLE SUBQUERY storeCheckRecords index upc,upc_2 datetimesql_3 53 NULL 60452 Using where; Using index
3 UNCACHEABLE SUBQUERY storeCheckRecords index upc,upc_2 datetimesql_3 53 NULL 60452 Using where; Using index
2 UNCACHEABLE SUBQUERY storeCheckRecords index upc,upc_2 datetimesql_3 53 NULL 60452 Using where; Using index

first try using mysql built in query analyzer
http://dev.mysql.com/doc/refman/5.0/en/explain-extended.html
explain [your_query]
and come back with the result.
my guess is this part of the query
INNER JOIN (SELECT 1 AS id
,'855555000032' AS upc
UNION
SELECT 2
,'855555000033'
UNION
SELECT 3
,'855555000034'
UNION
SELECT 4
,'855555000035'
UNION
SELECT 5
,'855555000036'
UNION
SELECT 6
,'855555000038') u
if this sub query is static, then it should by better to create as a myisam table , and create index for column id, and column upc
the way your query work, will utilize alot of temporary table.
for each subquery , mysql will create a temporary table.
performance will be degraded if the result from the query is big enough.
https://dev.mysql.com/doc/refman/5.6/en/subquery-optimization.html
you might redesign your query into separate views
also, slow query can be caused by unindex parameter column (column used in where queries)
a handy recipe for performance is to create a set column(s) index for each "where" parameter on queries used against table, a set column(s) index for each "order"
try to see if "storeId" and "upc" column in table "storeCheckRecords" is already indexed as a set.
also see if you have "innodb_file_per_table =1" set in your mysql config.
separate tablespace have positive impact on performance where big table data involved.
but, if the innodb_file_per_table is set to 0 before, you need to recreate the whole database.

mysql groupwise max as second where condition

I have a working query that seems awfully inefficient; I'm wondering if I'm missing a simple way to improve it.
Simple table:
id date master_id
-------------------------
1 2015-02-01 0
2 2015-02-02 0
3 2015-02-03 0
4 2015-02-04 1
5 2015-02-02 1
6 2015-02-17 1
7 2015-02-27 1
8 2015-01-01 1
Objective: Get all rows where the master_id is zero, OR the master_id is not zero and no other rows of the same master_id have an earlier date. Order every result by date.
Current query, using a groupwise minimum subquery to create the second WHERE condition.
SELECT *
FROM `test`
WHERE `master_id` =0
OR `id` IN (
SELECT test.`id`
FROM (
SELECT `master_id`, MIN(`date`) AS mindate
FROM `test`
WHERE `master_id` 0
GROUP BY `master_id`
) AS x
INNER JOIN `test` ON x.`master_id` = test.`master_id`
AND x.mindate= test.`date`
)
ORDER BY `date`
It works, but the EXPLAIN makes it seem inefficient:
id select_type table type possible_keys key key_len ref rows Extra
-------------------------------------------------------------------------------------------------------------
1 PRIMARY test ALL NULL NULL NULL NULL 8 Using where; Using filesort
2 DEPENDENT SUBQUERY derived3 system NULL NULL NULL NULL 1
2 DEPENDENT SUBQUERY test eq_ref PRIMARY PRIMARY 4 func 1 Using where
3 DERIVED test ALL NULL NULL NULL NULL 8 Using where; Using temporary; Using filesort
Can I improve this? Should I break it into two queries, one for ID=0 and one for the groupwise min? Thanks in advance.

Avoiding the inner join can improve the query:
SELECT *
FROM `test`
WHERE `master_id` =0
OR `id` IN (
SELECT t1.id
FROM (SELECT *
FROM test t2
WHERE t2.master_id!=0
ORDER BY t2.date ASC) t1
GROUP BY t1.master_id
)
ORDER BY `date`;

How about this...
SELECT * FROM test WHERE master_id = 0
UNION
SELECT x.*
FROM test x
JOIN (SELECT master_id,MIN(date) min_date FROM test GROUP BY master_id) y
ON y.master_id = x.master_id
AND y.min_date = x.date;

mysql query for getting max count of row's data by date?

i have one table name as a_ad_display with following column
displayId | campaignId | displaytime(date)
1 | 2 | 2012-12-19 12:41:08
2 | 2 | 2012-12-19 01:41:08
3 | 1 | 2012-12-20 10:41:08
4 | 1 | 2012-12-20 02:41:08
5 | 1 | 2012-12-21 12:41:08
6 | 2 | 2012-12-21 01:41:08
7 | 1 | 2012-12-21 12:41:08
8 | 1 | 2012-12-21 02:41:08
there is a multiple entry with same campaignId should be there.now what i want is to first of all fetch the campaignId with the highest count in a_ad_display and for that campaignId i want to get total number of count within the date
eg..from above table i have 1 number campaignId with highest row count
so for 1 number campaignId i want to display records like this..
**Date** | **countPerDay**
2012-12-19 | 0
2012-12-20 | 2
2012-12-21 | 3
how can i achieve this in mysql database...hope anyone can guide me...

You can use this query
select
count(campaignId) as TCount,
DATE(displayTime) as `Date`
from a_ad_display
where campaignId = 1
group by DATE(displayTime)
Although it will not display 0.
EDIT :
OK This is modified and fetches your result.
select
date(a.displayTime) as `DisplayTime`,
ifnull(l.TCount,0) as TCount
from a_ad_display as a
left join (select
count(campaignId) as TCount,
displayTime
from a_ad_display
where campaignId = 1
group by DATE(displayTime)) as l
on date(a.displayTime) = date(l.displayTime)
group by DATE(a.displayTime)
Demo
MORE EDITS :
select
date(a.displayTime) as `DisplayTime`,
ifnull(l.TCount,0) as TCount
from a_ad_display as a
left join (select
count(campaignId) as TCount,
displayTime
from a_ad_display
where campaignId = 1
group by DATE(displayTime)) as l
on date(a.displayTime) = date(l.displayTime)
group by DATE(a.displayTime)
order by a.displaytime desc limit 5
This is how you can select latest 5 records. If you want to select the latest campaign you can use a subquery instead of 1 take a look at this query
select
date(a.displayTime) as `DisplayTime`,
ifnull(l.TCount,0) as TCount
from a_ad_display as a
left join (select
count(campaignId) as TCount,
displayTime
from a_ad_display
where campaignId = (select campaignId from a_ad_display group by campaignId order by count(campaignId) desc limit 1)
group by DATE(displayTime)) as l
on date(a.displayTime) = date(l.displayTime)
group by DATE(a.displayTime)
EDITS :
I have edited the query this will fetch higher occurance campaignId and fetch results accordingly
Demo

this will show 0 also
select
sum(case when campaignId = #inputCompainid then 1 else 0 end) as TCount,
DATE(displayTime) as `Date`
from a_ad_display
group by DATE(displayTime)

Following query gives u list of records for each campaignId
select campaignId,date(displaytime) as Date,count(*) as countPerDay from a_ad_display group by campaignId,date(displaytime);
if you need for specfic campaignId, use this :
select campaignId,date(displaytime) as Date,count(*) as countPerDay from a_ad_display group by campaignId,date(displaytime) having campaignId=1;

sory for asking this, but which one of this query is better??
1.the answer by Mr.raheel shan
EXPLAIN SELECT DATE(a.displayTime) AS `DisplayTime`,IFNULL(l.TCount,0) AS TCount
FROM a_ad_display AS a
LEFT JOIN (SELECT
COUNT(campaignId) AS TCount,
displayTime
FROM a_ad_display
WHERE campaignId = 1
GROUP BY DATE(displayTime)) AS l
ON DATE(a.displayTime) = DATE(l.displayTime)
GROUP BY DATE(a.displayTime);
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived2> ALL 2
1 PRIMARY a ALL 8 Using temporary; Using filesort
2 DERIVED a_ad_display ALL 8 Using where; Using temporary; Using filesort
2.or using this one
EXPLAIN SELECT DATE(a_ad_display.`displaytime`) AS `DATE`,
COUNT(a.`campaignId`) AS `COUNT Per Day`
FROM a_ad_display
LEFT JOIN (SELECT `a_ad_display`.`displayId`,
`a_ad_display`.`campaignId`
FROM `a_ad_display`
WHERE `a_ad_display`.`campaignId`=1) a
ON (`a_ad_display`.`displayId`=a.`displayId`)
GROUP BY `a_ad_display`.`displaytime`;
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY a_ad_display ALL 8 Using temporary; Using filesort
1 PRIMARY <derived2> ALL 5
2 DERIVED a_ad_display ALL 8 Using where
thanks before & after :D
EDIT for highest campaignId occurance
SELECT DATE(a_ad_display.`displaytime`) AS `DATE`, COUNT(a.`campaignId`) AS `COUNT Per Day` FROM a_ad_display
LEFT JOIN (SELECT `a_ad_display`.`displayId`,`a_ad_display`.`campaignId` FROM `a_ad_display`
INNER JOIN (SELECT COUNT(`a_ad_display`.`campaignId`) AS `count`, `a_ad_display`.`campaignId`
FROM `a_ad_display`
GROUP BY `a_ad_display`.`campaignId`
ORDER BY `count` DESC
LIMIT 1) b ON (a_ad_display.`campaignId`=b.`campaignId`)
) a ON (`a_ad_display`.`displayId`=a.`displayId`)
GROUP BY `DATE`;
GROUP BY a_ad_display.displaytime; /* wrong in this statement ..., sorry*/
it should be working now ..... try again.
You can check in here http://sqlfiddle.com/#!2/bc75b/1, hope it's workin :D