Using mysql 8 window functions - mysql

Question

My salary table looks like this,
employeeId Salary salaryEffectiveFrom
19966 10000.00 2022-07-01
19966 20000.00 2022-07-15
My role/grades table looks like this,
employeeId grade roleEffectiveFrom
19966 grade 3 2022-07-01
19966 grade 2 2022-07-10
I am trying to get the salary a grade is paid for by taking into account the effective date in both tables.
grade 3 is effective from 1-July-2022. grade 2 is effective from the 10th of July, implying grade 3 is effective till the 9th of July i.e. 9 days.
grade 2 is effective from 10-July-2022 onwards.
A salary of 10000 is effective from 1-July-2022 till 14-July-2022 as the salary of 20000 is effective from the 15th. Therefore grade 3 had a salary of 10000 for 9 days, grade 2 salary of 10000 for 4 days with grade 2 with a salary of 20000 from the 10th onwards. The role effectivefrom
date takes precedence over the salary effectivefrom date.
This query,
SELECT er.employeeId,
es.salary,
`grade`,
date(er.effectiveFrom) roleEffectiveFrom,
date(es.effectiveFrom) salaryEffectiveFrom,
DATEDIFF(LEAST(COALESCE(LEAD(er.effectiveFrom)
OVER (PARTITION BY er.employeeId ORDER By er.effectiveFrom),
DATE_ADD(LAST_DAY(er.effectiveFrom),INTERVAL 1 DAY)),
DATE_ADD(LAST_DAY(er.effectiveFrom),INTERVAL 1 DAY)),
er.effectiveFrom) as '#Days' ,
ROUND((salary * 12) / 365, 2) dailyRate
FROM EmployeeRole er
join EmployeeSalary es ON (es.employeeId = er.employeeId)
and er.employeeId = 19966
;
gives me the result set shown below,
employeeId Salary grade roleEffectiveFrom salaryEffectiveFrom Days dailyRate
19966 10000.00 grade 3 2022-07-01 2022-07-01 0 328.77
19966 20000.00 grade 3 2022-07-01 2022-07-15 9 657.53
19966 10000.00 grade 2 2022-07-10 2022-07-01 0 328.77
19966 20000.00 grade 2 2022-07-10 2022-07-15 22 657.53
grade3 is effective for 9 days in July so I want to get the total salary for those 9 days using a daily rate column, 328.77 * 9 = 2985.93 as a separate column but I am unable to do as I am getting the days for the wrong row i.e. 9 should be the result for the first row.
dbfiddle

Answer 1

merge the 2 table dates, lead them then use correlated sub queries
with cte as
(
SELECT employeeid,effectivefrom from EMPLOYEEROLE
union
select employeeid,effectivefrom from employeesalary
)
,cte1 as
(select employeeid,effectivefrom,
coalesce(
date_sub(lead(effectivefrom) over (partition by employeeid order by effectivefrom),interval 1 day) ,
now()) nexteff
from cte
)
select *,
datediff(nexteff,effectivefrom) + 1 diff,
(select grade from employeerole e where e.effectivefrom <= cte1.effectivefrom order by e.effectivefrom desc limit 1) grade,
(select salary from employeesalary e where e.effectivefrom <= cte1.nexteff order by e.effectivefrom desc limit 1) salary
from cte1;
+------------+---------------------+---------------------+------+---------+--------+
| employeeid | effectivefrom | nexteff | diff | grade | salary |
+------------+---------------------+---------------------+------+---------+--------+
| 19966 | 2022-07-01 00:00:00 | 2022-07-09 00:00:00 | 9 | grade 3 | 10000 |
| 19966 | 2022-07-10 00:00:00 | 2022-07-14 00:00:00 | 5 | grade 2 | 10000 |
| 19966 | 2022-07-15 00:00:00 | 2022-10-08 08:51:49 | 86 | grade 2 | 20000 |
+------------+---------------------+---------------------+------+---------+--------+
3 rows in set (0.003 sec)
with cte as
(
SELECT employeeid,effectivefrom from EMPLOYEEROLE
union
select employeeid,effectivefrom from employeesalary
)
,cte1 as
(select cte.employeeid,effectivefrom,
coalesce(
date_sub(lead(effectivefrom) over (partition by employeeid order by effectivefrom),interval 1 day) ,
last_day(maxdt)) nexteff
from cte
JOIN (select cte.employeeid,max(effectivefrom) maxdt from cte group by employeeid) c1
on c1.employeeid = cte.employeeid
)
select *,
datediff(nexteff,effectivefrom) + 1 diff,
(select grade from employeerole e where e.effectivefrom <= cte1.effectivefrom order by e.effectivefrom desc limit 1) grade,
(select salary from employeesalary e where e.effectivefrom <= cte1.nexteff order by e.effectivefrom desc limit 1) salary
from cte1;
+------------+---------------------+---------------------+------+---------+--------+
| employeeid | effectivefrom | nexteff | diff | grade | salary |
+------------+---------------------+---------------------+------+---------+--------+
| 19966 | 2022-07-01 00:00:00 | 2022-07-09 00:00:00 | 9 | grade 3 | 10000 |
| 19966 | 2022-07-10 00:00:00 | 2022-07-14 00:00:00 | 5 | grade 2 | 10000 |
| 19966 | 2022-07-15 00:00:00 | 2022-07-31 00:00:00 | 17 | grade 2 | 20000 |
+------------+---------------------+---------------------+------+---------+--------+
3 rows in set (0.004 sec)

Answer 2

I think if it were me, I'd generate a list containing an entry for each day with the effective grade and salary, and then just aggregate at the end. Take a look at this fiddle:
https://dbfiddle.uk/4t2RW2M2
I've started with the aggregate query, just so we can see the output, then I break out pieces of the query to show intermediate outputs. Here is an image of the final output and the query generating it:
SELECT grade, gradeEffective, salary, salaryEffective,
min(dt) as startsOn, max(dt) as endsOn, count(*) as days,
dailyRate,
sum(dailyRate) as pay
FROM (
SELECT DISTINCT dt, grade, gradeEffective, salary, salaryEffective,
ROUND((salary * 12) / 365, 2) as dailyRate
FROM (
SELECT dts.dt,
first_value(r.grade) OVER w as grade,
first_value(r.effectiveFrom) OVER w as gradeEffective,
first_value(s.salary) OVER w as salary,
first_value(s.effectiveFrom) OVER w as salaryEffective
FROM (
WITH RECURSIVE dates(n) AS (SELECT 0 UNION SELECT n + 1 FROM dates WHERE n + 1 <= 30)
SELECT '2022-07-01' + INTERVAL n DAY as dt FROM dates
) dts
LEFT JOIN EmployeeSalary s ON dts.dt >= s.effectiveFrom
LEFT JOIN EmployeeRole r on dts.dt >= r.effectiveFrom
WINDOW w AS (
PARTITION BY dts.dt
ORDER BY r.effectiveFrom DESC, s.effectiveFrom DESC
ROWS UNBOUNDED PRECEDING
)
) z
) a GROUP BY grade, gradeEffective, salary, salaryEffective, dailyRate
ORDER BY min(dt);
Now, the first thing I've done is create a list of dates using a recursive CTE:
WITH RECURSIVE dates(n) AS (SELECT 0 UNION SELECT n + 1 FROM dates WHERE n + 1 <= 30)
SELECT '2022-07-01' + INTERVAL n DAY as dt FROM dates
which produces a list of dates from July 1st to July 31st.
Take that list of dates and left join both of your tables to it, like so:
SELECT *
FROM (
WITH RECURSIVE dates(n) AS (SELECT 0 UNION SELECT n + 1 FROM dates WHERE n + 1 <= 30)
SELECT '2022-07-01' + INTERVAL n DAY as dt FROM dates
) dts
LEFT JOIN EmployeeSalary s ON dts.dt >= s.effectiveFrom
LEFT JOIN EmployeeRole r on dts.dt >= r.effectiveFrom
with the dt greater than or equal to the effective dates. Notice that after the 9th you start to get duplicate rows for each date.
We'll create a window to get the first values for grade and salary for each date, and we'll order first by role effectiveFrom and then salary effectiveFrom, to fulfil your priority condition.
SELECT dts.dt,
first_value(r.grade) OVER w as grade,
first_value(r.effectiveFrom) OVER w as gradeEffective,
first_value(s.salary) OVER w as salary,
first_value(s.effectiveFrom) OVER w as salaryEffective
FROM (
WITH RECURSIVE dates(n) AS (SELECT 0 UNION SELECT n + 1 FROM dates WHERE n + 1 <= 30)
SELECT '2022-07-01' + INTERVAL n DAY as dt FROM dates
) dts
LEFT JOIN EmployeeSalary s ON dts.dt >= s.effectiveFrom
LEFT JOIN EmployeeRole r on dts.dt >= r.effectiveFrom
WINDOW w AS (
PARTITION BY dts.dt
ORDER BY r.effectiveFrom DESC, s.effectiveFrom DESC
ROWS UNBOUNDED PRECEDING
);
This is still going to leave us multiple entries for some dates, although they are duplicates, so let's use that output in a new query, using DISTINCT to leave us only one copy of each row and using the opportunity to add the daily rate field:
SELECT DISTINCT dt, grade, gradeEffective, salary, salaryEffective,
ROUND((salary * 12) / 365, 2) as dailyRate
FROM (
SELECT dts.dt,
first_value(r.grade) OVER w as grade,
first_value(r.effectiveFrom) OVER w as gradeEffective,
first_value(s.salary) OVER w as salary,
first_value(s.effectiveFrom) OVER w as salaryEffective
FROM (
WITH RECURSIVE dates(n) AS (SELECT 0 UNION SELECT n + 1 FROM dates WHERE n + 1 <= 30)
SELECT '2022-07-01' + INTERVAL n DAY as dt FROM dates
) dts
LEFT JOIN EmployeeSalary s ON dts.dt >= s.effectiveFrom
LEFT JOIN EmployeeRole r on dts.dt >= r.effectiveFrom
WINDOW w AS (
PARTITION BY dts.dt
ORDER BY r.effectiveFrom DESC, s.effectiveFrom DESC
ROWS UNBOUNDED PRECEDING
)
) z;
This produces the deduplicated daily data
and now all we have to do is use aggregation to pull out the sums for each combination of grade and salary, which is the query that I started off with.
Let me know if this is what you were looking for, or if anything is unclear.
Since the start and end conditions weren't fleshed out in the question, I just created the date list arbitrarily. It's not difficult to generate the list based on the first effectiveFrom in both tables, and here is an example that runs from that start date until current:
WITH RECURSIVE dates(n) AS (
SELECT min(effectiveFrom) FROM (
select effectiveFrom from EmployeeRole UNION
select effectiveFrom from EmployeeSalary
) z
UNION SELECT n + INTERVAL 1 DAY FROM dates WHERE n <= now()
)
SELECT n as dt FROM dates
I also didn't handle for multiple employees, since there was only one given and I would just be guessing at the shape of the actual data.

Answer 3

You can start adding two new columns (i.e. tmpFrom and tmpTo), which should give the correct dates which are needed to calculate the 9 Days.
SELECT
er.employeeId,
es.salary,
`grade`,
date(er.effectiveFrom) roleEffectiveFrom,
date(es.effectiveFrom) salaryEffectiveFrom,
DATEDIFF(LEAST(COALESCE(LEAD(er.effectiveFrom)
OVER (PARTITION BY er.employeeId ORDER By er.effectiveFrom),
DATE_ADD(LAST_DAY(er.effectiveFrom),INTERVAL 1 DAY)),
DATE_ADD(LAST_DAY(er.effectiveFrom),INTERVAL 1 DAY)),
er.effectiveFrom) as '#Days' ,
ROUND((salary * 12) / 365, 2) dailyRate,
date(er.effectiveFrom) tmpFrom,
(select e2.effectiveFrom
from EmployeeRole e2
where e2.employeeId = er.employeeId and e2.effectiveFrom > er.effectiveFrom
order by e2.effectiveFrom
limit 1) as tmpTo
FROM EmployeeRole er
join EmployeeSalary es ON (es.employeeId = er.employeeId)
and er.employeeId = 19966
order by er.effectiveFrom
;
In above query I used a sub-select, which might hurt performance. You can study Window Function, and check if there is a function which suits your needs better than this sub-query.
It's up to you to calculate the number of days between those two columns, but you should also solve the NULL value which should be end of month (But I am not sure if I remember your problem correctly...)
see: DBFIDDLE