Mysql Query with multiple subqueries with group by distinct condition - mysql

Question

Hi I am a PHP Developer weak in MySQL Medium complex queries make my head fired.
The below is the table vulnerability.
+-----------------------------------------------------------------------------------------------+
| id | webisite_id | low_count| high_count | medium_count | date_time | vul_date |
+-----------------------------------------------------------------------------------------------+
| 20 | 6 | 1 | 1 | 1 | 2018-07-04 09:14:04 | 2018-02-01 |
| 19 | 6 | 30 | 30 | 30 | 2018-07-04 09:13:38 | 2018-01-30 |
| 18 | 6 | 1 | 1 | 1 | 2018-07-04 09:13:16 | 2018-01-01 |
+-----------------------------------------------------------------------------------------------+
This table represent count of low, medium, high - vulnerability count for each website in database. We can enter as many entries for each websites. But the only relevant entry for a website is the latest entry in each month (based on vul_date).
Here I need help I want query which fetch sum of counts low, high, medium of each websites of each month of last 1 year, for example if -> website with id 1 has 1, 2, 3 low, high, medium number of vulnerabilities, on month June and
-> that of with id 2 has 7, 8, 9 respectively the result should be for June 8, 10, 12. And like last 1 year's each month result should be get. If no entry it should be 0.
Note that the entries should be the maximum value of vul_date and if a site has multiple entries on the same vul_date get the latest date_time entry.
I tried to write question as much as simple. hope the question is understood.
Please help me on this
Thanks in advance.

Answer 1

I think below query will work for you.
SELECT
SUM(low_count),
SUM(medium_count),
SUM(high_count),
MONTH(vul_date)
FROM
(SELECT
low_count, medium_count, high_count, vul_date, date_time
FROM
test
WHERE
(website_id , vul_date) IN (SELECT website_id, MAX(vul_date)
FROM test GROUP BY website_id , MONTH(vul_date))) t
WHERE
date_time IN (SELECT MAX(date_time) FROM test GROUP BY website_id , vul_date)
GROUP BY MONTH(vul_date);
What it does is, first finds the latest entry month wise for each website id which is your max vul_date.
SELECT website_id, MAX(vul_date)
FROM test GROUP BY website_id , MONTH(vul_date)
If there are more than one entry for a vul_date, it uses date_time to select maximum value from them. Finally it sums all website date after grouping it month wise.
You can change the above query to get 0 value for those months where there is no entry for any websites.

Answer 2

DROP TABLE IF EXISTS T;
CREATE TABLE T(id INT, website_id INT, low_count INT, high_count INT, medium_count INT, date_time DATETIME, vul_date DATE);
INSERT INTO T VALUES
( 20 , 6 , 1 , 1 , 1, '2018-07-04 09:14:04' , '2018-02-01'),
( 19 , 6 , 30, 30, 30, '2018-07-04 09:13:38' , '2018-01-30'),
( 18 , 6 , 2 , 2 ,2 , '2018-07-04 09:13:16' , '2018-01-01'),
( 17 , 6 , 2 , 2 ,2 , '2018-07-04 09:12:01' , '2018-01-01'),
( 90 , 1,1,2,3,'2017-07-05 01:00:00',' 2017-07-06'),
( 90 , 2,8,9,10,'2017-07-05 01:00:00',' 2017-07-06');
select coalesce(c.yyyymm,d.yyyymm) yyyymm,
coalesce(c.lo,0) lo,
coalesce(c.hi,0) hi,
coalesce(c.med,0) med
from
(
SELECT concat(year(a.vul_date),'-',month(a.vul_date)) yyyymm,
SUM(LOW_COUNT) lo,SUM(HIGH_COUNT) hi,sum(medium_count) med
from
(
select website_id,date_time,vul_date
from t
where date_time = (select max(date_time) from t t1 where t1.website_id = t.website_id and t1.vul_date = t.vul_date)
) a
join
(select website_id, date_time,vul_date,
LOW_COUNT,HIGH_COUNT,medium_count
from t) b
on b.website_id = a.website_id and b.date_time = a.date_time
group by concat(year(a.vul_date),'-',month(a.vul_date))
) c
right join
(select distinct concat(year(dte),'-',month(dte)) yyyymm from dates d
where dte between date_sub(now(), interval 1 year) and now()
) d on d.yyyymm = c.yyyymm
;
Sub query a get the vul_date with the most recent data_time which is then self joined, aggregated and then infilled with missing dates using a right join to a dates/calender table. If you don't have a dates/calender it would be useful for this kind of exercise nut there are alternatives which you can find in SO.
Result
+---------+------+------+------+
| yyyymm | lo | hi | med |
+---------+------+------+------+
| 2017-7 | 9 | 11 | 13 |
| 2017-8 | 0 | 0 | 0 |
| 2017-9 | 0 | 0 | 0 |
| 2017-10 | 0 | 0 | 0 |
| 2017-11 | 0 | 0 | 0 |
| 2017-12 | 0 | 0 | 0 |
| 2018-1 | 32 | 32 | 32 |
| 2018-2 | 1 | 1 | 1 |
| 2018-3 | 0 | 0 | 0 |
| 2018-4 | 0 | 0 | 0 |
| 2018-5 | 0 | 0 | 0 |
| 2018-6 | 0 | 0 | 0 |
| 2018-7 | 0 | 0 | 0 |
+---------+------+------+------+
13 rows in set (0.04 sec)