I'm trying to find the right query to check if date and time ranges overlap in the MySQL table, here is the table:
id pickup_date pickup_time return_date return_time
1 2016-05-01 12:00:00 2016-05-31 13:00:00
2 2016-07-01 12:00:00 2016-07-04 15:00:00
Here are the data about every reservation which is coming and need to be checked against the "Reservations" table:
pickup_date = '2016-04-01';
pickup_time = '12:00:00'
return_date = '2016-05-01';
return_time = '13:00:00'
with this data the reservation overlap the one in the database. Take a note: the new reservation can be in the past or in the future.
EDIT (as proposed by spencer7593, this is the working version so far):
SET #new_booking_pickup_date = '2016-04-01';
SET #new_booking_pickup_time = '12:00:00';
SET #new_booking_return_date = '2016-05-01';
SET #new_booking_return_time = '13:00:00';
SELECT * FROM Reservation WHERE NOT
( CONCAT(#new_booking_pickup_date,' ',#new_booking_pickup_time) > CONCAT(return_date,' ',return_time) + INTERVAL 0 DAY OR CONCAT(#new_booking_return_date,' ',#new_booking_return_time) < CONCAT(pickup_date,' ',pickup_time) + INTERVAL 0 DAY);
, so this query will result:
id pickup_date pickup_time return_date return_time
1 2016-05-01 12:00:00 2016-05-31 13:00:00
It's pretty easy to determine if a given period doesn't overlap with another period.
For ease of expressing the comparison, for period 1, we'll let the begin and end be represented by b1 and e1. For period 2, b2 and e2.
There is no overlap if the following is true:
b1 > e2 OR e1 < b2
(We can quibble whether equality of b1 and e2 would be considered an overlap or not, and adjust as necessary.)
The negation of that test would return TRUE if there was an overlap...
NOT (b1 > e2 OR e1 < b2)
So, to find out if there is a row that overlaps with the proposed period, we would need a query that tests whether a row is returned...
Let's assume that table we are going to check has columns st and et (DATETIME) representing the beginning and ending of each period.
To find rows with an overlap with a proposed period bounded by b1 and e1
SELECT t.* FROM t WHERE NOT (b1 > t.et OR e1 < t.st)
So for a query to just check for the existence of an overlapping row, we could do something like this:
SELECT EXISTS (SELECT 1 FROM t WHERE NOT (b1 > t.et OR e1 < t.st))
That's pretty simple.
It's going to look a lot more complicated when we make the adjustment for the (inexplicable) split of the date and time components of a datetime into separate columns (as shown in the question).
It's just a straightforward matter of combining the separate date and time values together into a single value of DATETIME datatype.
All we need to do is substitute into our query above an appropriate conversion, e.g.
st => CONCAT(pickup_date,' ',pickup_time) + INTERVAL 0 DAY
et => CONCAT(return_date,' ',return_time) + INTERVAL 0 DAY
Same for b1 and e1.
Doing that substitution, coming up with the final query, is left as an exercise for whoever decided that storing date and time as separate columns was a good idea.
Related
I have three columns User_ID, New_Status and DATETIME.
New_Status contains 0(inactive) and 1(active) for users.
Every user starts from active status - ie. 1.
Subsequently table stores their status and datetime at which they got activated/inactivated.
How to calculate number of active users at the end of each date, including dates when no records were generated into the table.
Sample data:
| ID | New_Status | DATETIME |
+----+------------+---------------------+
| 1 | 1 | 2019-01-01 21:00:00 |
| 1 | 0 | 2019-02-05 17:00:00 |
| 1 | 1 | 2019-03-06 18:00:00 |
| 2 | 1 | 2019-01-02 01:00:00 |
| 2 | 0 | 2019-02-03 13:00:00 |
Format the date time value to a date only string and group by it
SELECT DATE_FORMAT(DATETIME, '%Y-%m-%d') as day, COUNT(*) as active
FROM test
WHERE New_Status = 1
GROUP BY day
ORDER BY day
In MySQL 8 you can use the row_number() window function to get the last status of a user per day. Then filter for the one that indicate the user was active GROUP BY the day and count them.
SELECT date(x.datetime),
count(*)
FROM (SELECT date(t.datetime) datetime,
t.new_status,
row_number() OVER (PARTITION BY date(t.datetime)
ORDER BY t.datetime DESC) rn
FROM elbat t) x
WHERE x.rn = 1
AND x.new_status = 1
GROUP BY x.datetime;
If not all days are in the table you need to create a (possibly derived) table with all days and cross join it.
Find out the last activity status of users whose activity was changed for each day
select User_ID, New_Status, DATE_FORMAT(DATETIME, '%Y-%m-%d')
from activity_table
where not exists
(
select 1
from activity_table at
where at.User_ID = activity_table.User_ID and
DATE_FORMAT(at.DATETIME, '%Y-%m-%d') = DATE_FORMAT(activity_table.DATETIME, '%Y-%m-%d') and
at.DATETIME > activity_table.DATETIME
)
order by DATE_FORMAT(activity_table.DATETIME, '%Y-%m-%d');
This is not the solution yet, but a very very useful information before solution. Note that here not all dates are covered yet and the values are individual records, more precisely their last values on each day, ordered by the date.
Let's get aggregate numbers
Using the query above as a subselect and aliasing it into a table, you can group by DATETIME and do a select sum(new_Status) as activity, count(*) total, DATETIME so you will know that activity - (total - activity) is the difference in comparison to the previous day.
Knowing the delta for each day present in the result
At the previous section we have seen how the delta can be calculated. If the whole query in the previous section is aliased, then you can self join it using a left join, with pairs of (previous date, current date), still having the gaps of dates, but not worrying about that just yet. In the case of the first date, its activity is the delta. For subsequent records, adding the previous day's delta to their delta yields the result you need. To achieve this you can use a recursive query, supported by MySQL 8, or, alternatively, you can just have a subquery which sums the delta of previous days (with special attention to the first date, as described earlier) will and adding the current date's delta yields the result we need.
Fill the gaps
The previous section would already perfectly work (assuming the lack of integrity problems), assuming that there were activity changes for each day, but we will not continue with the assumption. Here we know that the figures are correct for each date where a figure is present and we will need to just add the missing dates into the result. If the results are properly ordered, as they should be, then one can use a cursor and loop the results. At each record after the first one, we can determine the dates that are missing. There might be 0 such dates between two consequent dates or more. What we do know about the gaps is that their values are exactly the same as the previous record, that do has data. If there were no activity changes on a given date, then the number of active users is exactly the same as in the previous day. Using some structure, like a table you can generate the results you have with the knowledge described here.
Solving possible integrity problems
There are several possibilities for such problems:
First, a data item might exist prior to the introduction of this table's records were started to be spawned.
Second, bugs or any other causes might have made a pause in creating records for this activity table.
Third, the addition of user is or was not necessarily generating an activity change, since its popping into existence renders its previous state of activity undefined and subject to human standards, which might change over time.
Fourth, the removal of user is or was not necessarily generating an activity change, since its popping out of existence renders is current state of activity undefined and subject to human standards, which might change over time.
Fifth, there is an infinity of other issues which might cause data integrity issues.
To cope with these you will need to comprehensively analyze whatever you can from the source-code and the history of the project, including database records, logs and humanly available information to detect such anomalies, the time they were effective and figure out what their solution is if they exist.
EDIT
In the meantime I was thinking about the possibility of a user, who was active at the start of the day being deactivated and then activated again by the end of the day. Similarly, an inactive user during a day might be activated and then finally deactivated by the end of the day. For users that have more than an activation at the start of the day, we need to compare their activity status at the start and the end of the day to find out what the difference was.
SELECT
DATE(DATETIME),
COUNT(*)
FROM your_table
WHERE New_Status = 1
GROUP BY User_ID,
DATE(DATETIME)
For MySQL
WITH RECURSIVE
cte AS (
SELECT MIN(DATE(DT)) dt
FROM src
UNION ALL
SELECT dt + INTERVAL 1 DAY
FROM cte
WHERE dt < ( SELECT MAX(DATE(DT)) dt
FROM src )
),
cte2 AS
(
SELECT users.id,
cte.dt,
SUM( CASE src.New_Status WHEN 1 THEN 1
WHEN 0 THEN -1
ELSE 0
END ) OVER ( PARTITION BY users.id
ORDER BY cte.dt ) status
FROM cte
CROSS JOIN ( SELECT DISTINCT id
FROM src ) users
LEFT JOIN src ON src.id = users.id
AND DATE(src.dt) = cte.dt
)
SELECT dt, SUM(status)
FROM cte2
GROUP BY dt;
fiddle
Do not forget to adjust max recursion depth.
Here is what I believe is a good solution for this problem of yours:
SELECT SUM(New_Status) "Number of active users"
, DATE_FORMAT(DATEC, '%Y-%m-%d') "Date"
FROM TEST T1
WHERE DATE_FORMAT(DATEC,'%H:%i:%s') =
(SELECT MAX(DATE_FORMAT(T2.DATEC,'%H:%i:%s'))
FROM TEST T2
WHERE T2.ID = T1.ID
AND DATE_FORMAT(T1.DATEC, '%Y-%m-%d') = DATE_FORMAT(T2.DATEC, '%Y-%m-%d')
GROUP BY ID
, DATE_FORMAT(DATEC, '%Y-%m-%d'))
GROUP BY DATE_FORMAT(DATEC, '%Y-%m-%d');
Here is the DEMO
I want to get stats for each day in a given month. However, if a day has no rows in the table, it doesn't show up in the results. How can I include days with no data, and show all days until the current date?
This is the query I have now:
SELECT DATE_FORMAT(FROM_UNIXTIME(timestamp), '%d'), COUNT(*)
FROM data
WHERE EXTRACT(MONTH FROM FROM_UNIXTIME(timestamp)) = 6
GROUP BY EXTRACT(DAY FROM FROM_UNIXTIME(timestamp))
So if I have
Row 1 | 01-06
Row 2 | 02-06
Row 3 | 03-06
Row 4 | 05-06
Row 5 | 05-06
(i changed timestamp values to a day/month date just to explain)
It should output
01 | 1
02 | 1
03 | 1
04 | 0
05 | 2
06 | 0
...Instead of ignoring day 4 and today (day 6).
You will need a calendar table to do something in the form
SELECT `date`, count(*)
FROM Input_Calendar c
LEFT JOIN Data d on c.date=d.date
GROUP BY `date`
I keep a full copy of a calendar table in my database and used a WHILE loop to fill it but you can populate one on the fly for use based on the different solutions out there like http://crazycoders.net/2012/03/using-a-calendar-table-in-mysql/
In MySQL, you can use MySQL variables (act like in-line programming values). You set and can manipulate as needed.
select
dayofmonth( DynamicCalendar.CalendarDay ) as `Day`,
count(*) as Entries
from
( select
#startDate := date_add( #startDate, interval 1 day ) CalendarDay
from
( select #startDate := '2013-05-31' ) sqlvars,
AnyTableThatHasAsManyDaysYouExpectToReport
limit
6 ) DynamicCalendar
LEFT JOIN Input_Calendar c
on DynamicCalendar.CalendarDay = date( from_unixtime( c.date ))
group by
DynamicCalendar.CalendarDay
In the above sample, the inner query can join against as the name implies "Any Table" in your database that has at least X number of records you are trying to generate for... in this case, you are dealing with only the current month of June and only need 6 records worth... But if you wanted to do an entire year, just make sure the "Any Table" has 365 records(or more).
The inner query will start by setting the "#startDate" to the day BEFORE June 1st (May 31). Then, by just having the other table, will result in every record joined to this variable (creates a simulated for/next loop) via a limit of 6 records (days you are generating the report for). So now, as the records are being queried, the Start Date keeps adding 1 day... first record results in June 1st, next record June 2nd, etc.
So now, you have a simulated calendar with 6 records dated from June 1 to June 6. Take that and join to your "data" table and you are already qualifying your dates via the join and get only those dates of activity. I'm joining on the DATE() of the from unix time since you care about anything that happend on June 1, and June 1 # 12:00:00AM is different than June 1 # 8:45am, so matching on the date only portion, they should remain in proper grouping.
You could expand this answer by changing the inner '2013-05-31' to some MySQL Date function to get the last day of the prior month, and the limit based on whatever day in the current month you are doing so these are not hard-coded.
Create a Time dimension. This is a standard OLAP reporting trick. You don't need a cube in order to do OLAP tricks, though. Simply find a script on the internet to generate a Calendar table and join to that table.
Also, I think your query is missing a WHERE clause.
Other useful tricks include creating a "Tally" table that is a list of numbers from 1 to N where N is usually the max of the bigint on your database management system.
No code provided here, as I am not a MySQL guru.
Pseudo-code is:
Select * from Data left join TimeDimension on data.date = timedimension.date
How can I calculate the time difference between two date, considering:
Only Monday to Friday
Time between 9am to 5:30pm;
Exclude Holidays.
Example:
d1 = 2012-10-05 17:00:00
d2 = 2012-14-09 12:00:00
Calculation Steps:
2012-10-05 = 00:30:00
2012-10-06 = 00:00:00
2012-10-07 = 00:00:00
2012-10-08 = 07:30:00
2012-10-09 = 04:00:00
ddiff(d2,d1) = 12:00:00
I know how to do it using only mon-fri, as described here. And I am talking about MySQL.
I've come up with a solution that's relatively straightforward for calculating the time difference for the full interim dates. However it's a bit messy to use mysql for calculating the time difference for the start & end dates. I have included them in my solution, but with a number of assumptions.
In any case, here's the sql
SET #startdate:='2012-12-24 17:00:00';
SET #enddate:='2013-01-02 12:00:00';
SELECT
TIME_TO_SEC(TIMEDIFF(CONCAT(DATE(#startdate),' 17:30:00'), #startdate))/3600 as startday_time,
TIME_TO_SEC(TIMEDIFF(#enddate, CONCAT(DATE(#enddate),' 9:00:00')))/3600 as endday_time,
SUM(daily_hours) as otherdays_time from
(
SELECT 7.5 as daily_hours, id, DATE_ADD(DATE(#startdate),INTERVAL id-1 DAY) as idate from numbers
) dates
LEFT JOIN holidays on DATE(dates.idate) = DATE(holidays.date)
WHERE
idate BETWEEN #startdate AND #enddate
AND holidays.date IS NULL
AND DAYOFWEEK(idate) <> 7 AND DAYOFWEEK(idate) <> 1;
sqlfiddle here:
http://sqlfiddle.com/#!2/ff3f3/1/2
To get the valid interim dates, we'll need two tables - a holidays table listing all the holiday dates and a numbers table that contains a series of integers which is very useful for joining against to get a sequential series of dates (with no gaps).
Note: In the sqlfiddle, I've populated the numbers table only up to 12 to cover the dates used in my example - it will probably need to be populated to a higher number depending on the range of dates you'll be working with.
For the start day time & end day time, I've made the following assumptions:
that start date & end date are both valid dates that should be counted towards the total time
that the time on the start date is between lunch and 17.30
that the time on the end date is between lunch and 17.30
if these assumptions are wrong, you're getting into serious conditional territory (with lots of ifs) and might be best doing this in the php (or whatever).
note: I've left the times (which are in hours) un-added for illustration purposes.
I have a table
tbl_charge
id hotel_id start_date end_date charge_per_day ( in $)
1 6 2012-02-15 2010-02-15 20
2 6 2012-02-16 2010-02-18 30
4 6 2012-02-20 2010-02-25 50
Note: if any date is not in the table then we set 25$ for each days (i.e. default charge)
now if someone wants to book a hotel from 2012-02-15 to 2012-02-22 , then I want to calculate the total charges for dates
Date : 15+16+17+18+19+20+21+22
Charge : 20+30+30+30+25+50+50+50 = 285$
what i have done so far:
this query returns all rows successfully
SELECT * FROM `tbl_charge` WHERE
start_date BETWEEN '2012-02-15' AND '2012-02-22' OR
end_date BETWEEN '2012-02-15' AND '2012-02-22' OR
( start_date <'2012-02-15' AND end_date > '2012-02-22')
HAVING property_id=6
it returns all necessary rows but how do I sum the charges??
is ther any way to count days between given date range like last row is 20 -25 but i want only upto 22 then it return 3 days and we multiply charges by 3
is it good to create procedure for this or use simple query
I think this will do the trick:
select sum(DayDifference * charge_per_day) +
(RealDayDifference - sum(DayDifference)) * 25 as TotalPerPeriod
from (
select charge_per_day, datediff(
least(end_date, '2012-02-22'),
greatest(start_date, '2012-02-15')) + 1 as DayDifference,
datediff('2012-02-22', '2012-02-15') + 1 as RealDayDifference
from t1
where
((start_date between '2012-02-15' and '2012-02-22') or
(end_date between '2012-02-15' and '2012-02-22') or
(start_date < '2012-02-15' and end_date > '2012-02-22'))
and hotel_id=6
) S1
I've had to solve this same issue previously and it's a fun one, however since then I've learnt some better methods. At the time I believe I created a procedure or function to loop over the requested dates and return a price.
To return the required rows, you can simply select using the upper and lower limits. You can do a datediff within the select criteria to return the number of iterations of each to apply.
If all you are ultimately looking for is a single price I would advise combining this logic into a function
I've assumed a second table, tbh_hotel with id (int PK == hotel_id) and default_charge (int) with row (id=6,default_charge=20)
Further assumptions are that where your dates are "2010" you meant them to be "2012", and that this is for someone that is checking in in the 15th, and checking out on the 22nd (and so needs a hotel for 15th, 16th, 17th, 18th, 19th, 20th, 21st, 7 nights). I will also assume that you have logic in place that prevents the date ranges overlapping, so that there are no 2 rows in tbl_charge which match the date 14th Feb 2012 (for example)
So to get this started, a query to select the applicable rows
SELECT
*
FROM tbl_charge AS c
WHERE
(
c.end_date >= '2012-02-15'
OR
c.start_date < '2012-02-22'
)
This is pretty much what you have already, so now will add in some more fields to get the information for how many days each rule is applied for.
SET #StartDate = '2012-02-15';
SET #EndDate = SUBDATE('2012-02-22',1);
SELECT
c.id,
c.start_date,
c.end_date,
c.charge_per_day,
DATEDIFF(IF(c.end_date>#EndDate,#EndDate,c.end_date),SUBDATE(IF(c.start_date<#StartDate,#StartDate,c.start_date),1)) AS quantityOfThisRate
FROM tbl_charge AS c
WHERE c.end_date >=#StartDate OR c.start_date < #EndDate
I am SUBDATEing the end date, because if you check out on the 22nd, your final checkin date is the 21st. I am SUBDATING the start date on each DATEDIFF because if you are staying on 15th -> 16th, the subdate on END DATE makes this 15th-15th, and so this SUBDATE makes it get 14th-15th to return the correct value of 1. Output now looks a bit like this
id start_date end_date price quantityAtThisRate
1 2012-02-10 2012-02-15 20 1
2 2012-02-16 2012-02-18 30 3
3 2012-02-20 2012-02-29 50 2
So moving on I'll put this into a subquery and combine tbl_hotel to get a default charge
SET #StartDate = '2012-02-15';
SET #EndDate = SUBDATE('2012-02-22',1);
SET #NumberOfNights = DATEDIFF(ADDDATE(#EndDate,1),#StartDate);
SET #HotelID = 6;
SELECT
SUM(specificDates.charge_per_day*specificDates.quantityAtThisRate) AS specificCharges,
#NumberOfNights-SUM(specificDates.quantityAtThisRate) AS daysAtDefault,
h.default_charge * (#NumberOfNights-SUM(specificDates.quantityAtThisRate)) AS defaultCharges
FROM tbl_hotel AS h
INNER JOIN
(
SELECT
c.charge_per_day,
DATEDIFF(IF(c.end_date>#EndDate,#EndDate,c.end_date),SUBDATE(IF(c.start_date<#StartDate,#StartDate,c.start_date),1)) AS quantityAtThisRate
FROM tbl_charge AS c
WHERE (c.end_date >=#StartDate OR c.start_date < #EndDate) AND c.hotel_id = #HotelID
) AS specificDates
WHERE h.id = #HotelID
Realistically a single query will get rather .... complex so I'd settle at a stored procedure relying on the logic above (as if there are no specific rules the above query will return null due to the inner join)
Hope this is of help
I have a log table with a date field called logTime. I need to show the number of rows within a date range and the number of records per day. The issue is that i still want to show days that do not have records.
Is it possible to do this only with SQL?
Example:
SELECT logTime, COUNT(*) FROM logs WHERE logTime >= '2011-02-01' AND logTime <= '2011-02-04' GROUP BY DATE(logTime);
It returns something like this:
+---------------------+----------+
| logTime | COUNT(*) |
+---------------------+----------+
| 2011-02-01 | 2 |
| 2011-02-02 | 1 |
| 2011-02-04 | 5 |
+---------------------+----------+
3 rows in set (0,00 sec)
I would like to show the day 2011-02-03 too.
MySQL will not invent rows for you, so if the data is not there, they will naturally not be shown.
You can create a calendar table, and join in that,
create table calendar (
day date primary key,
);
Fill this table with dates (easy with a stored procedure, or just some general scripting), up till around 2038 and something else will likely break unitl that becomes a problem.
Your query then becomes e.g.
SELECT logTime, COUNT(*)
FROM calendar cal left join logs l on cal.day = l.logTime
WHERE day >= '2011-02-01' AND day <= '2011-02-04' GROUP BY day;
Now, you could extend the calendar table with other columns that tells you the month,year, week etc. so you can easily produce statistics for other time units. (and purists might argue the calendar table would have an id integer primary key that the logs table references instead of a date)
In order to accomplish this, you need to have a table (or derived table) which contains the dates that you can then join from, using a LEFT JOIN.
SQL operates on the concept of mathematical sets, and if you don't have a set of data, there is nothing to SELECT.
If you want more details, please comment accordingly.
I'm not sure if this is a problem that should be solved by SQL. As others have shown, this requires maintaining a second table that contains the all of the individual dates of a given time span, which must be updated every time that time span grows (which presumably is "always" if that time span is the current time.
Instead, you should use to inspect the results of the query and inject dates as necessary. It's completely dynamic and requires no intermediate table. Since you specified no language, here's pseudo code:
EXECUTE QUERY `SELECT logTime, COUNT(*) FROM logs WHERE logTime >= '2011-02-01' AND logTime <= '2011-02-04' GROUP BY DATE(logTime);`
FOREACH row IN query result
WHILE (date in next row) - (date in this row) > 1 day THEN
CREATE new row with date = `date in this row + 1 day`, count = `0`
INSERT new row IN query result AFTER this row
ADVANCE LOOP INDEX TO new row (`this row` is now the `new row`)
END WHILE
END FOREACH
Or something like that
DECLARE #TOTALCount INT
DECLARE #FromDate DateTime = GetDate() - 5
DECLARE #ToDate DateTime = GetDate()
SET #FromDate = DATEADD(DAY,-1,#FromDate)
Select #TOTALCount= DATEDIFF(DD,#FromDate,#ToDate);
WITH d AS
(
SELECT top (#TOTALCount) AllDays = DATEADD(DAY, ROW_NUMBER()
OVER (ORDER BY object_id), REPLACE(#FromDate,'-',''))
FROM sys.all_objects
)
SELECT AllDays From d