SQL Server : how to loop through rows and assign value - sql-server-2008

I have a couple of tables to deal with. One is like this:
dbo.userActivity
userID Action Time
1 25 12:00
1 10 12:01
1 12 12:35
1 6 13:54
2 10 6:47
2 42 6:48
3 8 11:54
etc.
The other table is a schedule that looks like this:
dbo.userSchedule
userID schedule_start schedule_stop
1 07:00 09:00
2 11:00 12:30
3 14:00 15:00
etc.
What I need to do for each row in dbo.userActivity is determine if each action was an hour before the schedule_start, between the schedule_start and schedule_stop times, an hour after schedule_stop, or any other time.
So I need to append a column to dbo.userActivity with the values 'before', 'during', 'after', 'other' based on the time calculations.
I'm really not sure how to do this and would appreciate any help you all could offer.
EDIT:
Okay, so I have that mostly working. I've just seen some of the real data I'll be working on and noticed that the activity times are full datetime stamps, whereas the activity schedule is just in time. So I need to convert the datetime to something I can compare.
This looks like it works:
SELECT *
FROM
(
SELECT CONVERT(TIME, SCANDATE) as scanTime
FROM appData
) st
WHERE st.scanTime <= '6:00'
ORDER BY st.scanTime
Just as an example. But when I try to incorporate that into the case statement below like this, it doesn't work. It applies the same THEN to every row.
SELECT
CASE
WHEN EXISTS
(
SELECT *
FROM
(
SELECT CONVERT(TIME, SCANDATE) as scanTime
FROM appData
) st
WHERE st.scanTime <= '6:00'
)
THEN 'Before 6 am'
ELSE '6 am or after'
END
FROM appData
Any further thoughts on this?

First you would want to add the column (NULL) to your table.
ALTER TABLE dbo.UserActivity
ADD TimeStatus VARCHAR(6) NULL;
Next you would write your update query. It can be written with a CASE statement:
UPDATE ua
SET ua.TimeStatus =
CASE
WHEN CAST(ua.Time AS TIME) >= us.Schedule_Start
AND CAST(ua.Time AS TIME) <= us.Schedule_stop THEN 'During'
WHEN CAST(ua.Time AS TIME) >= DATEADD(HOUR, -1, us.Schedule_Start)
AND CAST(ua.Time AS TIME) <= us.ScheduleStart THEN 'Before'
WHEN CAST(ua.Time AS TIME) >= us.Schedule_Stop
AND CAST(ua.Time AS TIME) <= DATEADD(HOUR, 1, us.Schedule_Stop)
THEN 'After'
ELSE 'Other'
END
FROM dbo.UserActivity AS ua
INNER JOIN dbo.userSchedule AS us ON ua.UserId = us.UserId
Once all the columns have data, you can set the column to NOT NULL if you have updated the application to know about, and populate, this new column.
I would also consider not storing the string values in this table, but instead a smaller value that will foreign key to a reference table instead. With only 4 values, you can use a TINYINT and save space for each dbo.UserActivity record that you store.
If you decide to go this route, you would just take the values from the reference table, and replace the string values with the ID values. If Before = 0, during = 1, After = 2, and Other = 3
UPDATE ua
SET ua.TimeStatusId =
CASE
WHEN CAST(ua.Time AS TIME) >= us.Schedule_Start
AND CAST(ua.Time AS TIME) <= us.Schedule_stop THEN 1
WHEN CAST(ua.Time AS TIME) >= DATEADD(HOUR, -1, us.Schedule_Start)
AND CAST(ua.Time AS TIME) <= us.ScheduleStart THEN 0
WHEN CAST(ua.Time AS TIME) >= us.Schedule_Stop
AND CAST(ua.Time AS TIME) <= DATEADD(HOUR, 1, us.Schedule_Stop)
THEN 2
ELSE 3
END
FROM dbo.UserActivity AS ua
INNER JOIN dbo.userSchedule AS us ON ua.UserId = us.UserId
You would then get the text values for the UI with an INNER JOIN to your TimeStatus reference table

Related

Iterate Over Date Mysql Loop

I've written a stored procedure to iterate over every week for three years. It doesn't work though and returns a vague error message.
#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 18
DELIMITER $$
CREATE PROCEDURE loop_three_years()
BEGIN
declare y INT default 2016;
declare m int default 4;
declare d int default 20;
WHILE y <= 2019 DO
WHILE YEARWEEK(concat(y, '-', m, '-', d)) <= 53 DO
WHILE m < 12 DO
WHILE (m = 2 and d <= 29) OR (d <=30 and m in(4, 6,9,11)) OR ( m in(1,3,5,7,8,10,12) AND d <= 31) DO
set d = d + 7;
SELECT YEARWEEK(concat(y, '-', m, '-', d));
END WHILE;
set d=1;
END WHILE;
set m = 1;
SET y = y + 1;
END WHILE;
END
$$
When I used this as minimal parts they work so I'm not sure what the issue is with my reassembly. Also not sure if there's a better way to do this. (The select is just for testing, it will be an insert when I use the real code.
Slightly Altered from a previous solution
You can build your own dynamic calendar / list using ANY other table in your system that has at least as many records as you need to fake row numbers. The query below will use MySQL # variables which work like an inline program and declaration. I can start the list with a given date... such as your 2016-04-20 and then each iteration through, add 1 week using date-based functions. No need for me to know or care about how many days have a 28, 29(leap-year), 30 or 31 days.
The table reference below of "AnyTableThatHasAtLeast156Records" is just that.. Any table in your database that has at least 156 records (52 weeks per year, 3 years)
select
YEARWEEK( #startDate ) WeekNum,
#startDate as StartOfWeek,
#startDate := date_add( #startDate, interval 1 week ) EndOfWeek
from
( select #startDate := '2016-04-20') sqlv,
AnyTableThatHasAtLeast156Records
limit
156
This will give you a list of 156 records (provided your "anyTable…" has 156 records all at once. If you need to join this to some other transaction table, you could do so by making the above a JOIN table. Benefit here, Since I included the begin date and end of week, those can be part of your joining to table.
Example, on
record WeekNum StartOfWeek EndOfWeek
1 ?? 2016-04-20 2016-04-27
2 ?? 2016-04-27 2016-05-04
3 ?? 2016-05-04 2016-05-11
4 ?? 2016-04-11 2016-05-18... etc
By adding 1 week to the starting point, you can see that it would do Ex: Monday to Monday. And the JOIN Condition below I have LESS THAN the EndOfWeek. This would account for any transactions UP TO but not including the ending date... such as transactions on 2016-04-26 11:59:59PM (hence LESS than 2016-04-27, as 04/27 is the beginning of the next week's cycle of transactions)
select
Cal.WeekNum,
YT.YourColumns
from
YourTransactionTable YT
JOIN ( aboveCalendarQuery ) Cal
on YT.TransactionDate >= Cal.StartOfWeek
AND YT.TransactionDate < Cal.EndOfWeek
where
whatever else
You could even do sum() with group by such as by WeekNum if that is what you intend.
Hopefully this is a much more accurate and efficient way to build out your calendar to run with and linking to transactions if you so needed to.
Response from comment.
You could by doing a join to a ( select 1 union select 2 union … select 156 ), but your choice. The ONLY reason for the "AnyTable…" is I am sure with any reasonable database with transactions you would have 156 records or more easily. It's sole purpose is to just allow a row for cycling through the iterations to dynamically create the rows.
Also much more sound than the looping mechanism you have run into to begin with. Nothing wrong with that, especially learning purposes, but if more efficient ways, doesn't that make more sense?
Per feedback from comment
I dont exactly know your other table you are trying to insert into, but yes, you can use this for all 3000 things. Provide more of what you are trying to do and I can adjust... In the mean-time, something like this...
insert into YourOtherTable
( someField,
AnotherField,
WeekNum
)
select
x.someField,
x.AnotherField,
z.WeekNum
from
Your3000ThingTable x
JOIN (select
YEARWEEK( #startDate ) WeekNum,
#startDate as StartOfWeek,
#startDate := date_add( #startDate, interval 1 week ) EndOfWeek
from
( select #startDate := '2016-04-20') sqlv,
AnyTableThatHasAtLeast156Records
limit
156 ) z
on 1=1
where
x.SomeCodition...
By joining the the select of 156 records on 1=1 (which is always true), it will return 156 entries for whatever record is in the Your3000ThingTable. So, if you have an inventory item table with
Item Name
1 Thing1
2 Thing2
3 Thing3
Your final insert would be
Item Name WeekNum
1 Thing1 1
1 Thing1 2
1 Thing1 ...
1 Thing1 156
2 Thing2 1
2 Thing2 2
2 Thing2 ...
2 Thing2 156
3 Thing3 1
3 Thing3 2
3 Thing3 ...
3 Thing3 156
And to pre-confirm what you THINK would happen, just try the select/join on 1=1 and you'll see all the records the query WOULD be inserting into your destination table.

MySQL timestamp modification, addition, and comparison

I am creating a view that involves a little bit of fiddling with timestamps.
I have a table A with timestamps. The view will process the timestamps to see if each timestamp is within a certain range (9 AM - 5 PM). If the timestamp is within that range, I will fetch data matching the exact time in another table (B). Otherwise, I will fetch the next day (or this day's) first valid time (which is 9 AM) and the corresponding data from there.
Examples:
A record with timestamp of 12/28/2012 17:01 -> fetch data from B
for 12/29/2012 09:00, set flag to after.
A record with timestamp of 12/28/2012 08:59 -> fetch data from B
for 12/28/2012 09:00, set flag to before.
A record with timestamp of 12/28/2012 09:55 -> fetch data from B
for 12/28/2012 09:55, set flag to null.
Here is what I have so far (not working, some in pseudocode). I mainly don't know how to set the flag based on the comparison and then, based on flag, perform next operation on b - all in one statement.
CREATE VIEW C as
SELECT time, (CASE WHEN (time< '9:00' ) THEN'before'
CASE WHEN(time> '17:00') THEN'after' else null END) AS flag FROM A
//These two should be combined into one create view statement
//The below is utterly wrong, I know, but explains what I mean
SELECT(
CASE WHEN (flag=='before') THEN SELECT * FROM B WHERE B.time = time set hour='9:00'
CASE WHEN(flag=='after') THEN SELECT* FROM B WHERE B.time = time + one day set hour='9:00'
ELSE SELECT* FROM B WHERE B.time = time ) as data
Tested using this fiddle: http://sqlfiddle.com/#!2/15be5/50
SELECT
q.time as original_time_check,
q.flag as flag_check,
case q.flag
when 'before' then q.NINE_AM_ON_THE_DAY
when 'after' then q.NINE_AM_THE_NEXT_DAY
else q.time
end as time
FROM
(
SELECT
time,
date(time) + INTERVAL 9 HOUR as NINE_AM_ON_THE_DAY,
date(time) + INTERVAL '1 9' DAY_HOUR as NINE_AM_THE_NEXT_DAY,
case
when time < (date(time) + INTERVAL 9 HOUR) then 'before'
when time < (date(time) + INTERVAL 17 HOUR) then 'in-range'
else 'after'
end as flag
FROM
Your_table
) q

two date columns and one date range , typical query?

I have a table
tbl_charge
id hotel_id start_date end_date charge_per_day ( in $)
1 6 2012-02-15 2010-02-15 20
2 6 2012-02-16 2010-02-18 30
4 6 2012-02-20 2010-02-25 50
Note: if any date is not in the table then we set 25$ for each days (i.e. default charge)
now if someone wants to book a hotel from 2012-02-15 to 2012-02-22 , then I want to calculate the total charges for dates
Date : 15+16+17+18+19+20+21+22
Charge : 20+30+30+30+25+50+50+50 = 285$
what i have done so far:
this query returns all rows successfully
SELECT * FROM `tbl_charge` WHERE
start_date BETWEEN '2012-02-15' AND '2012-02-22' OR
end_date BETWEEN '2012-02-15' AND '2012-02-22' OR
( start_date <'2012-02-15' AND end_date > '2012-02-22')
HAVING property_id=6
it returns all necessary rows but how do I sum the charges??
is ther any way to count days between given date range like last row is 20 -25 but i want only upto 22 then it return 3 days and we multiply charges by 3
is it good to create procedure for this or use simple query
I think this will do the trick:
select sum(DayDifference * charge_per_day) +
(RealDayDifference - sum(DayDifference)) * 25 as TotalPerPeriod
from (
select charge_per_day, datediff(
least(end_date, '2012-02-22'),
greatest(start_date, '2012-02-15')) + 1 as DayDifference,
datediff('2012-02-22', '2012-02-15') + 1 as RealDayDifference
from t1
where
((start_date between '2012-02-15' and '2012-02-22') or
(end_date between '2012-02-15' and '2012-02-22') or
(start_date < '2012-02-15' and end_date > '2012-02-22'))
and hotel_id=6
) S1
I've had to solve this same issue previously and it's a fun one, however since then I've learnt some better methods. At the time I believe I created a procedure or function to loop over the requested dates and return a price.
To return the required rows, you can simply select using the upper and lower limits. You can do a datediff within the select criteria to return the number of iterations of each to apply.
If all you are ultimately looking for is a single price I would advise combining this logic into a function
I've assumed a second table, tbh_hotel with id (int PK == hotel_id) and default_charge (int) with row (id=6,default_charge=20)
Further assumptions are that where your dates are "2010" you meant them to be "2012", and that this is for someone that is checking in in the 15th, and checking out on the 22nd (and so needs a hotel for 15th, 16th, 17th, 18th, 19th, 20th, 21st, 7 nights). I will also assume that you have logic in place that prevents the date ranges overlapping, so that there are no 2 rows in tbl_charge which match the date 14th Feb 2012 (for example)
So to get this started, a query to select the applicable rows
SELECT
*
FROM tbl_charge AS c
WHERE
(
c.end_date >= '2012-02-15'
OR
c.start_date < '2012-02-22'
)
This is pretty much what you have already, so now will add in some more fields to get the information for how many days each rule is applied for.
SET #StartDate = '2012-02-15';
SET #EndDate = SUBDATE('2012-02-22',1);
SELECT
c.id,
c.start_date,
c.end_date,
c.charge_per_day,
DATEDIFF(IF(c.end_date>#EndDate,#EndDate,c.end_date),SUBDATE(IF(c.start_date<#StartDate,#StartDate,c.start_date),1)) AS quantityOfThisRate
FROM tbl_charge AS c
WHERE c.end_date >=#StartDate OR c.start_date < #EndDate
I am SUBDATEing the end date, because if you check out on the 22nd, your final checkin date is the 21st. I am SUBDATING the start date on each DATEDIFF because if you are staying on 15th -> 16th, the subdate on END DATE makes this 15th-15th, and so this SUBDATE makes it get 14th-15th to return the correct value of 1. Output now looks a bit like this
id start_date end_date price quantityAtThisRate
1 2012-02-10 2012-02-15 20 1
2 2012-02-16 2012-02-18 30 3
3 2012-02-20 2012-02-29 50 2
So moving on I'll put this into a subquery and combine tbl_hotel to get a default charge
SET #StartDate = '2012-02-15';
SET #EndDate = SUBDATE('2012-02-22',1);
SET #NumberOfNights = DATEDIFF(ADDDATE(#EndDate,1),#StartDate);
SET #HotelID = 6;
SELECT
SUM(specificDates.charge_per_day*specificDates.quantityAtThisRate) AS specificCharges,
#NumberOfNights-SUM(specificDates.quantityAtThisRate) AS daysAtDefault,
h.default_charge * (#NumberOfNights-SUM(specificDates.quantityAtThisRate)) AS defaultCharges
FROM tbl_hotel AS h
INNER JOIN
(
SELECT
c.charge_per_day,
DATEDIFF(IF(c.end_date>#EndDate,#EndDate,c.end_date),SUBDATE(IF(c.start_date<#StartDate,#StartDate,c.start_date),1)) AS quantityAtThisRate
FROM tbl_charge AS c
WHERE (c.end_date >=#StartDate OR c.start_date < #EndDate) AND c.hotel_id = #HotelID
) AS specificDates
WHERE h.id = #HotelID
Realistically a single query will get rather .... complex so I'd settle at a stored procedure relying on the logic above (as if there are no specific rules the above query will return null due to the inner join)
Hope this is of help

MySQL: Find Missing Dates Between a Date Range

I need some help with a mysql query. I've got db table that has data from Jan 1, 2011 thru April 30, 2011. There should be a record for each date. I need to find out whether any date is missing from the table.
So for example, let's say that Feb 2, 2011 has no data. How do I find that date?
I've got the dates stored in a column called reportdatetime. The dates are stored in the format: 2011-05-10 0:00:00, which is May 5, 2011 12:00:00 am.
Any suggestions?
This is a second answer, I'll post it separately.
SELECT DATE(r1.reportdate) + INTERVAL 1 DAY AS missing_date
FROM Reports r1
LEFT OUTER JOIN Reports r2 ON DATE(r1.reportdate) = DATE(r2.reportdate) - INTERVAL 1 DAY
WHERE r1.reportdate BETWEEN '2011-01-01' AND '2011-04-30' AND r2.reportdate IS NULL;
This is a self-join that reports a date such that no row exists with the date following.
This will find the first day in a gap, but if there are runs of multiple days missing it won't report all the dates in the gap.
CREATE TABLE Days (day DATE PRIMARY KEY);
Fill Days with all the days you're looking for.
mysql> INSERT INTO Days VALUES ('2011-01-01');
mysql> SET #offset := 1;
mysql> INSERT INTO Days SELECT day + INTERVAL #offset DAY FROM Days; SET #offset := #offset * 2;
Then up-arrow and repeat the INSERT as many times as needed. It doubles the number of rows each time, so you can get four month's worth of rows in seven INSERTs.
Do an exclusion join to find the dates for which there is no match in your reports table:
SELECT d.day FROM Days d
LEFT OUTER JOIN Reports r ON d.day = DATE(r.reportdatetime)
WHERE d.day BETWEEN '2011-01-01' AND '2011-04-30'
AND r.reportdatetime IS NULL;`
It could be done with a more complicated single query, but I'll show a pseudo code with temp table just for illustration:
Get all dates for which we have records:
CREATE TEMP TABLE AllUsedDates
SELECT DISTINCT reportdatetime
INTO AllUsedDates;
now add May 1st so we track 04-30
INSERT INTO AllUsedData ('2011-05-01')
If there's no "next day", we found a gap:
SELECT A.NEXT_DAY
FROM
(SELECT reportdatetime AS TODAY, DATEADD(reportdatetime, 1) AS NEXT_DAY FROM AllUsed Dates) AS A
WHERE
(A.NEXT_DATE NOT IN (SELECT reportdatetime FROM AllUsedDates)
AND
A.TODAY <> '2011-05-01') --exclude the last day
If you mean reportdatetime has the entry of "Feb 2, 2011" but other fields associated to that date are not present like below table snap
reportdate col1 col2
5/10/2011 abc xyz
2/2/2011
1/1/2011 bnv oda
then this query works fine
select reportdate from dtdiff where reportdate not in (select df1.reportdate from dtdiff df1, dtdiff df2 where df1.col1 = df2.col1)
Try this
SELECT DATE(t1.datefield) + INTERVAL 1 DAY AS missing_date FROM table t1 LEFT OUTER JOIN table t2 ON DATE(t1.datefield) = DATE(t2.datefield) - INTERVAL 1 DAY WHERE DATE(t1.datefield) BETWEEN '2020-01-01' AND '2020-01-31' AND DATE(t2.datefield) IS NULL;
If you want to get missing dates in a datetime field use this.
SELECT CAST(t1.datetime_field as DATE) + INTERVAL 1 DAY AS missing_date FROM table t1 LEFT OUTER JOIN table t2 ON CAST(t1.datetime_field as DATE) = CAST(t2.datetime_field as DATE) - INTERVAL 1 DAY WHERE CAST(t1.datetime_field as DATE) BETWEEN '2020-01-01' AND '2020-07-31' AND CAST(t2.datetime_field as DATE) IS NULL;
The solutions above seem to work, but they seem EXTREMELY slow (taking possibly hours, I waited for 30 min only) at least in my database.
This clause takes less than a second in same database (of course you need to repeat it manually dozen times and possibly change function names to find the actual dates). pvm = my datetime, WEATHER = my table.
mysql> select year(pvm) as _year,count(distinct(date(pvm))) as _days from WEATHER where year(pvm)>=2000 and month(pvm)=1 group by _year order by _year asc;
--ako

How to get data back from Mysql for days that have no statistics

I want to get the number of Registrations back from a time period (say a week), which isn't that hard to do, but I was wondering if it is in anyway possible to in MySQL to return a zero for days that have no registrations.
An example:
DATA:
ID_Profile datCreate
1 2009-02-25 16:45:58
2 2009-02-25 16:45:58
3 2009-02-25 16:45:58
4 2009-02-26 10:23:39
5 2009-02-27 15:07:56
6 2009-03-05 11:57:30
SQL:
SELECT
DAY(datCreate) as RegistrationDate,
COUNT(ID_Profile) as NumberOfRegistrations
FROM tbl_profile
WHERE DATE(datCreate) > DATE_SUB(CURDATE(),INTERVAL 9 DAY)
GROUP BY RegistrationDate
ORDER BY datCreate ASC;
In this case the result would be:
RegistrationDate NumberOfRegistrations
25 3
26 1
27 1
5 1
Obviously I'm missing a couple of days in between. Currently I'm solving this in my php code, but I was wondering if MySQL has any way to automatically return 0 for the missing days/rows. This would be the desired result:
RegistrationDate NumberOfRegistrations
25 3
26 1
27 1
28 0
1 0
2 0
3 0
4 0
5 1
This way we can use MySQL to solve any problems concerning the number of days in a month instead of relying on php code to calculate for each month how many days there are, since MySQL has this functionality build in.
Thanks in advance
No, but one workaround would be to create a single-column table with a date primary key, preloaded with dates for each day. You'd have dates from your earliest starting point right through to some far off future.
Now, you can LEFT JOIN your statistical data against it - then you'll get nulls for those days with no data. If you really want a zero rather than null, use IFNULL(colname, 0)
Thanks to Paul Dixon I found the solution. Anyone interested in how I solved this read on:
First create a stored procedure I found somewhere to populate a table with all dates from this year.
CREATE Table calendar(dt date not null);
CREATE PROCEDURE sp_calendar(IN start_date DATE, IN end_date DATE, OUT result_text TEXT)
BEGIN
SET #begin = 'INSERT INTO calendar(dt) VALUES ';
SET #date = start_date;
SET #max = SUBDATE(end_date, INTERVAL 1 DAY);
SET #temp = '';
REPEAT
SET #temp = concat(#temp, '(''', #date, '''), ');
SET #date = ADDDATE(#date, INTERVAL 1 DAY);
UNTIL #date > #max
END REPEAT;
SET #temp = concat(#temp, '(''', #date, ''')');
SET result_text = concat(#begin, #temp);
END
call sp_calendar('2009-01-01', '2010-01-01', #z);
select #z;
Then change the query to add the left join:
SELECT
DAY(dt) as RegistrationDate,
COUNT(ID_Profile) as NumberOfRegistrations
FROM calendar
LEFT JOIN
tbl_profile ON calendar.dt = tbl_profile.datCreate
WHERE dt BETWEEN DATE_SUB(CURDATE(),INTERVAL 6 DAY) AND CURDATE()
GROUP BY RegistrationDate
ORDER BY dt ASC
And we're done.
Thanks all for the quick replies and solution.