Calculate total scheduled against total actual in two separate tables - mysql

I have two tables in my schema. The first contains a list of recurring appointments - default_appointments. The second table is actual_appointments - these can be generated from the defaults or individually created so not linked to any default entry.
Example:
default_appointments
id
day_of_week
user_id
appointment_start_time
appointment_end_time
1
1
1
10:00:00
16:00:00
2
4
1
11:30:00
17:30:00
3
6
5
09:00:00
17:00:00
actual_appointments
id
default_appointment_id
user_id
appointment_start
appointment_end
1
1
1
2021-09-13 10:00:00
2021-09-13 16:00:00
2
NULL
1
2021-09-13 11:30:00
2021-09-13 13:30:00
3
6
5
2021-09-18 09:00:00
2021-09-18 17:00:00
I'm looking to calculate the total minutes that were scheduled in against the total that were actually created/generated. So ultimately I'd end up with a query result with this data:
user_id
appointment_date
total_planned_minutes
total_actual_minutes
1
2021-09-13
360
480
1
2021-09-16
360
0
5
2021-09-18
480
480
What would be the best approach here? Hopefully the above makes sense.
Edit
OK so the default_appointments table contains all appointments that are "standard" and are automatically generated. These are what appointments "should" happen every week. So e.g. ID 1, this appointment should occur between 10am and 4pm every Monday. ID 2 should occur between 11:30am an 5:30pm every Thursday.
The actual_appointments table contains a list of all of the appointments which did actually occur. Basically what happens is a default_appointment will automatically generate itself an instance in the actual_appointments table when initially set up. The corresponding default_appointment_id indicates that it links to a default and has not been changed - therefore the times on both will remain the same. The user is free to change these appointments that have been generated by a default, resulting in setting the default_appointment_id to NULL * - or -* can add new appointments unrelated to a default.
So, if on a Monday (day_of_week = 1) I should normally have a default appointment at 10am - 4pm, the total minutes I should have planned based on the defaults are 360 minutes, regardless of what's in the actual_appointments table, I should be planned for those 360 minutes every Monday without fail. If in the system I say - well actually, I didn't have an appointment from 10am - 4pm and instead change it to 10am - 2pm, actual_appointments table will then contain the actual time for the day, and the actual minutes appointed would be 240 minutes.
What I need is to group each of these by the date and user to understand how much time the user had planned for appointments in the default_appointments table vs how much they actually appointed.

Adjusted based on new detail in the question.
Note: I used day_of_week values compatible with default MySQL behavior, where Monday = 2.
The first CTE term (args) provides the search parameters, start date and number of days. The second CTE term (drange) calculates the dates in the range to allow generation of the scheduled appointments within that range.
allrows combines the scheduled and actual appointments via UNION to prepare for aggregation. There are other ways to set this up.
Finally, we aggregate the results per user_id and date.
The test case:
Working Test Case (Updated)
WITH RECURSIVE args (startdate, days) AS (
SELECT DATE('2021-09-13'), 7
)
, drange (adate, days) AS (
SELECT startdate, days-1 FROM args UNION ALL
SELECT adate + INTERVAL '1' DAY, days-1 FROM drange WHERE days > 0
)
, allrows AS (
SELECT da.user_id
, dr.adate
, ROUND(TIME_TO_SEC(TIMEDIFF(da.appointment_end_time, da.appointment_start_time))/60, 0) AS planned
, 0 AS actual
FROM drange AS dr
JOIN default_appointments AS da
ON da.day_of_week = dayofweek(adate)
UNION
SELECT user_id
, DATE(appointment_start) AS xdate
, 0 AS planned
, TIMESTAMPDIFF(MINUTE, appointment_start, appointment_end)
FROM drange AS dr
JOIN actual_appointments aa
ON DATE(appointment_start) = dr.adate
)
SELECT user_id, adate
, SUM(planned) AS planned
, SUM(actual) AS actual
FROM allrows
GROUP BY adate, user_id
;
Result:
+---------+------------+---------+--------+
| user_id | adate | planned | actual |
+---------+------------+---------+--------+
| 1 | 2021-09-13 | 360 | 480 |
| 1 | 2021-09-16 | 360 | 0 |
| 5 | 2021-09-18 | 480 | 480 |
+---------+------------+---------+--------+

Related

MySQL - Find start and end of blocks of consecutive rows with the same value

I need to extract and migrate values from one table to another. the source table contains sumarized values for a specific effectivity date. If a value is changed, a new line is written if something is changed on the component values with the data valid starting at this effective date.
source_id
entity_id
effective_date
component_1
component_2
component_3
int(ai)
int
date
int
int
int
1
159
2020-01-01
100
0
90
2
159
2020-05-01
140
50
90
3
159
2020-08-01
0
30
90
5
159
2020-12-01
0
30
50
i need now migrate this data to a new table like this. the goal is that selecting data for a given month the result is the valid data for this month is given.
id
source_id
entity_id
startdate
enddate
component_type
value
int(ai)
int
int
date
date
int
int
each row represents a value for a component valid for a period of month.
I now run the insert update for each effective month by setting it as a parameter.
I insert value changes as new rows to the table an prevent duplicates by using a unique key (entity_id,effective_date,component_type)
SET #effective_date = '2020-01-01';
INSERT INTO component_final
select NULL,
source_id,
entity_id,
effective_date,
NULL,
1,
component_1
FROM component_source
WHERE effective_date = #effective_date
AND component_1>0;
after migrating the first row it should be that result
id
source_id
entity_id
startdate
enddate
component_type
value
1
1
159
2020-01-01
NULL
1
100
2
1
159
2020-01-01
NULL
3
90
SET #effective_date = '2020-05-01';
INSERT INTO component_final
select NULL,
source_id,
entity_id,
effective_date,
NULL,
1,
component_1
FROM component_source
WHERE effective_date = #effective_date
AND component_1>0;
after migrating the second row it should be that result
id
source_id
entity_id
startdate
enddate
component_type
value
1
1
159
2020-01-01
2020-04-30
1
100
2
1
159
2020-01-01
NULL
3
90
3
2
159
2020-05-01
NULL
1
140
4
2
159
2020-05-01
NULL
2
50
so if there is a value change in the future an end date has to be set.
I'm not able to do the second step, updating the data, if the component is changed in the future.
Maybe it is possible to have it as triggers after insert new row with same entity and component - but I was not able to make it work.
Some ideas? I want to handle this only inside of the MySQL.
You do not need the column enddate in the table component_final, because it's value depends on other values in the same table:
SELECT
id,
source_id,
entity_id,
startdate,
( SELECT DATE_ADD(MIN(cf2.startdate),INTERVAL -1 DAY)
FROM component_final cf2
WHERE cf2.startdate > cf1.startdate
AND cf2.source_id = cf1.source_id
AND cf2.entity_id = cf1.entity_id
) as enddate,
component_type,
value
FROM component_final cf1;
I understand that the core issue is how to find the source_ids where a component changes (0 means a removal, so we don't want these entries in the result) and how to assign the respective end dates at the same time. For the sake of illustration I simplify your example a bit:
There is only one component_type (I take into account that there might then be consecutive entries with unchanged value)
there is only one entity_id, so we can ignore it
It should be easy to extend this simpler version to your real-world problem.
To this is an example input:
source_id
effective_date
value
1
2020-01-01
100
2
2020-01-03
100
3
2020-01-05
80
4
2020-01-10
0
5
2020-01-12
30
I would expect the following output to be generated:
source_id
start_date
end_date
value
1
2020-01-01
2020-01-04
100
3
2020-01-05
2020-01-09
80
5
2020-01-12
NULL
30
You can achieve this with one query by joing each row with the previous one to check if the value has changed (find the start dates of periods) and the first row that is in the future and has a different value (find the start of the next period). If there is no previous row, it is considered a start as well. If there is no later update of the value, we have no end_date.
SELECT
main.source_id,
main.effective_date as start_date,
DATE_SUB(next_start.effective_date, INTERVAL 1 DAY) as end_date,
main.value
FROM source main
LEFT JOIN source prev ON prev.effective_date = (
SELECT MAX(effective_date)
FROM source
WHERE effective_date < main.effective_date
)
LEFT JOIN source next_start ON next_start.effective_date = (
SELECT MIN(effective_date)
FROM source
WHERE effective_date > main.effective_date AND value <> main.value
)
WHERE
ISNULL(prev.source_id) OR prev.value <> main.value
AND main.value <> 0
ORDER BY main.source_id
As I said: This will have to be adapted to your problem, e.g. by adding proper join conditions for the entity_id.
#Luuk pointed out that you don't need the end date because it can be derived from the data. This would be the case if you had entries for the start of "0 periods" as well, i.e. if there is no value set. If you don't have entries for these, you can't derive the end from the start of the respectively next period since there might be a gap in between.

Calculating weekly growth using previous row value in new column

I have a query that returns new customers grouped by week. I would like to add a column that returns the growth/decrease in percentage comparing with previous rows.
I'm doing this in Elasticsearch > Kibana > Canvas so I cant use the lag function.
Ideally my result would be:
Week | Customers | Perc
2020-02-15 | 37 | -
2020-02-22 | 28 | -24
2020-02-29 | 51 | 82
2020-03-07 | 51 | 0
The calculation to be done is: ((This weeks customers - previous weeks customers) / previous weeks customers) *100
Here is the query i'm running that gives me the results of new customers per week:
SELECT DATEADD('day', 1 - DATEPART('weekday', createdAt), CAST(createdAt AS DATE)) AS Week,
COUNT(1) AS Customers,
ROUND(SUM(transfers.fiatAmountLocal)/360, 2) AS Volume
FROM users
WHERE transfers.fiatAmountLocal > 0
OR purchases.fiatAmountLocal > 0
GROUP BY
DATEADD('day', 1 - DATEPART('weekday', createdAt), CAST(createdAt AS DATE))
ORDER BY Week DESC
I need to add the percentage change column to it.
Any help is greatly appreciated.

MYSQL How to perform custom month difference between two dates in MYSQL?

My requirement is to compute the total months and then broken months separately between 2 dates (ie first date from table and second date is current date). If broken months total count is > 15 then account it as one month experience and if its les than 15 don't account that as 1 month experience.
Assume I have a date on table as 25/11/2018 and current date is 06/01/2019;
the full month in between is December, so 1 month experience; and broken months are November and January, so now I have to count the dates which is 6 days in Nov and 6 days in Jan, so 12 days and is <= (lte) 15 so total experience will be rounded to 1 month experience
I referred multiple questions related to calculating date difference in MYSQL from stackoverflow, but couldn't find any possible options. The inbuilt functions in MYSQL TIMESTAMPDIFF, TIMEDIFF, PERIOD_DIFF, DATE_DIFF are not giving my required result as their alogrithms are different from my calculation requirement.
Any clue on how to perform this calculation in MYSQL and arrive its result as part of the SQL statement will be helpful to me. Once this value is arrived, in the same SQL, that value will be validated to be within a given value range.
Including sample table structure & value:
table_name = "user"
id | name | join_date
---------------------
1| Sam | 25-11-2017
2| Moe | 03-04-2017
3| Tim | 04-07-2018
4| Sal | 30-01-2017
5| Joe | 13-08-2018
I wanted to find out the users from above table whose experience is calculated in months based on the aforementioned logic. If those months are between either of following ranges, then those users are fetched for further processing.
table_name: "allowed_exp_range"
starting_exp_months | end_exp_months
-------------------------------------
0 | 6
9 | 24
For ex: Sam's experience till date (10-12-2018) based on my calculation is 12+1 month = 13 months. Since 13 is between 9 & 24, Sam's record is one of the expected output.
I think this query will do what you want. It uses
(YEAR(CURDATE())*12+MONTH(CURDATE()))
- (YEAR(STR_TO_DATE(join_date, '%d-%m-%Y'))*12+MONTH(STR_TO_DATE(join_date, '%d-%m-%Y'))) -
- 1
to get the number of whole months of experience for the user,
DAY(LAST_DAY(STR_TO_DATE(join_date, '%d-%m-%Y')))
- DAY(STR_TO_DATE(join_date, '%d-%m-%Y'))
+ 1
to get the number of days in the first month, and
DAY(CURDATE())
to get the number of days in the current month. The two day counts are summed and if the total is > 15, 1 is added to the number of whole months e.g.
SELECT id
, name
, (YEAR(CURDATE())*12+MONTH(CURDATE())) - (YEAR(STR_TO_DATE(join_date, '%d-%m-%Y'))*12+MONTH(STR_TO_DATE(join_date, '%d-%m-%Y'))) - 1 -- whole months
+ CASE WHEN DAY(LAST_DAY(STR_TO_DATE(join_date, '%d-%m-%Y'))) - DAY(STR_TO_DATE(join_date, '%d-%m-%Y')) + 1 + DAY(CURDATE()) > 15 THEN 1 ELSE 0 END -- broken month
AS months
FROM user
We can use this expression as a JOIN condition between user and allowed_exp_range to find all users who have experience within a given range:
SELECT u.id
, u.name
, a.starting_exp_months
, a.end_exp_months
FROM user u
JOIN allowed_exp_range a
ON (YEAR(CURDATE())*12+MONTH(CURDATE())) - (YEAR(STR_TO_DATE(u.join_date, '%d-%m-%Y'))*12+MONTH(STR_TO_DATE(u.join_date, '%d-%m-%Y'))) - 1
+ CASE WHEN DAY(LAST_DAY(STR_TO_DATE(u.join_date, '%d-%m-%Y'))) - DAY(STR_TO_DATE(u.join_date, '%d-%m-%Y')) + 1 + DAY(CURDATE()) > 15 THEN 1 ELSE 0 END
BETWEEN a.starting_exp_months AND a.end_exp_months
Output (for your sample data, includes all users as they all fit into one of the experience ranges):
id name starting_exp_months end_exp_months
1 Sam 9 24
2 Moe 9 24
3 Tim 0 6
4 Sal 9 24
5 Joe 0 6
I've created a small demo on dbfiddle which demonstrates the steps in arriving at the result.

CASE w/ DATEADD range to SUM column multiple times for future earnings estimate

EDIT: The original post follows, but its a bit long and wordy. This edit presents a simplified question.
I'm trying to SUM 1 column multiple times; from what I've found, my options are either CASE or (SELECT). I am trying to SUM based on a date range and I can't figure out if CASE allows that.
table.number | table.date
2 2014/12/18
2 2014/12/19
3 2015/01/11
3 2015/01/12
7 2015/02/04
7 2015/02/05
As separate queries, it would look like this:
SELECT SUM(number) as alpha FROM table WHERE date >= 2014/12/01 AND date<= DATE_ADD (2014/12/01, INTERVAL 4 WEEKS)
SELECT SUM(number) as beta FROM table WHERE date >= 2014/12/29 AND date<= DATE_ADD (2014/12/01, INTERVAL 4 WEEKS)
SELECT SUM(number) as gamma FROM table WHERE date >= 2014/01/19 AND date<= DATE_ADD (2014/12/01, INTERVAL 4 WEEKS)
Looking for result set
alpha | beta | gamma
2 6 14
ORIGINAL:
I'm trying to return SUM of payments that will be due within my budgeting time frame (4 weeks) for the current budgeting period and 2 future periods. Some students pay every 4 weeks, others every 12. Here are the relevant fields in my tables:
client.name | client.ppid | client.last_payment
john | 1 | 12/01/14
jack | 2 | 11/26/14
jane | 3 | 10/27/14
pay_profile.id | pay_profile.price | pay_profile.interval (in weeks)
1 140 4
2 399 4
3 1 12
pay_history.name | pay_history.date | pay_history.amount
john | 12/02/14 | 140
jerry | more historical | data
budget.period_start |
12/01/14
I think the most efficient way of doing this is:
1.)SUM all students who pay every 4 weeks as base_pay
2.)SUM all students who pay every 12 weeks and whose DATEADD(client.last_payment, INTERVAL pay_profile.interval WEEKS) is >= budget.period_start and <= DATEADD(budget.period_start, INTERVAL 28 DAYS) as accounts_receivable
3.) As the above step will miss people who've already paid in this budgeting period (as this updates their last_payment dating, putting them out of the range specified in #2), I'll also need to SUM pay_history.date for the range above as well. paid_in_full
4.) repeat step 2 above, adjusting the range and column name for future periods (i.e. accounts_receivable_2
5.) use php to SUM base_pay, accounts_receivable, and pay_history, repeating the process for future periods.
I'm guessing the easiest way would be to use CASE, which I've not done before. Here was my best guess, which fails due to a sytax error. I assuming I can use DATE_ADD in the WHEN statement.
SELECT
CASE
DATE_ADD(client.last_payment, INTERVAL pay_profile.interval WEEK) >= budget.period_start
AND
DATE_ADD(client.last_payment, INTERVAL pay_profile.interval WEEK) <=
DATE_ADD(budget.period_start,INTERVAL 28 DAY) THEN SUM(pay_profile.price) as base_pay
FROM client
LEFT OUTER JOIN pay_profile ON client.ppid = pay_profile.ppid
LEFT OUTER JOIN budget ON client.active = 1
WHERE
client.active = 1
Thanks.

How to override values if possible in mysql query

I try to store the working hours of employees in a mysql table. In the table working_hour i store the normal working hours:
id employee weekDay start end
1 1 2 10:00:00 18:00:00
2 1 3 10:00:00 18:00:00
3 1 4 10:00:00 14:00:00
4 1 5 10:00:00 12:00:00
5 1 6 10:00:00 18:00:00
6 1 7 00:00:00 00:00:00
7 1 1 00:00:00 00:00:00
In a 2nd table i store "special" working hours. There i store things like illness, holidays or just customized working hours for a specific day. The working_hour_special table look like:
id start end type
2 12013-03-12 00:00:00 2013-03-13 23:59:59 ill
And thats what i have tried:
SELECT
IFNULL(working_hour_special.start, working_hour.start) AS startTime,
IFNULL(working_hour_special.end, working_hour.end) AS endTime,
type
FROM
working_hour_special LEFT JOIN
working_hour ON working_hour.employee_id = working_hour_special.employee_id
WHERE
working_hour_special.start = DATE('2013-03-13') AND
employee_id = 1 AND
working_hour.weekDay = DAYOFWEEK('2013-03-13')
The problem is the WHERE-Clause. I need the start and end time of a specific day for a specific employee. Got somebody an idea how to do that?
First, it appears poor table design. You have just a day of week in one, but a full date/time stamp in the other. Nothing to differentiate betweek day of week 1 for the January 1st week, vs day of week 1 in week of July 26th.
Second, when you have a left-join, and then throw that table into a WHERE clause (without consideration test of a NULL), it basically creates a normal INNER JOIN.
So, what you may be looking for is to shift your WHERE component associated with the SPECIAL to the JOIN part of the condition... something like.
SELECT
IFNULL(working_hour_special.start, working_hour.start) AS startTime,
IFNULL(working_hour_special.end, working_hour.end) AS endTime,
type
FROM
working_hour_special
LEFT JOIN working_hour
ON working_hour.employee_id = working_hour_special.employee_id
AND working_hour_special.start = DATE('2013-03-13')
WHERE
employee_id = 1
AND working_hour.weekDay = DAYOFWEEK('2013-03-13')