How i can fill date gaps in MySQL? Here is my query:
SELECT DATE(posted_at) AS date,
COUNT(*) AS total,
SUM(attitude = 'positive') AS positive,
SUM(attitude = 'neutral') AS neutral,
SUM(attitude = 'negative') AS negative
FROM `messages`
WHERE (`messages`.brand_id = 1)
AND (`messages`.`spam` = 0
AND `messages`.`duplicate` = 0
AND `messages`.`ignore` = 0)
GROUP BY date ORDER BY date
It returns proper result set - but i want to fill gaps between dates start and end by zeros. How i can do this?
You'll need to create a helper table and fill it with all dates from start to end, then just LEFT JOIN with that table:
SELECT d.dt AS date,
COUNT(*) AS total,
SUM(attitude = 'positive') AS positive,
SUM(attitude = 'neutral') AS neutral,
SUM(attitude = 'negative') AS negative
FROM dates d
LEFT JOIN
messages m
ON m.posted_at >= d.dt
AND m.posted_at < d.dt + INTERVAL 1 DAYS
AND spam = 0
AND duplicate = 0
AND ignore = 0
GROUP BY
d.dt
ORDER BY
d.dt
Basically, what you need here is a dummy rowsource.
MySQL is the only major system which lacks a way to generate it.
PostgreSQL implements a special function generate_series to do that, while Oracle and SQL Server can use recursion (CONNECT BY and recursive CTEs, accordingly).
I don't know whether MySQL will support the following/similar syntax; but if not, then you could just create and drop a temporary table.
--Inputs
declare #FromDate datetime, /*Inclusive*/
#ToDate datetime /*Inclusive*/
set #FromDate = '20091101'
set #ToDate = '20091130'
--Query
declare #Dates table (
DateValue datetime NOT NULL
)
set NOCOUNT ON
while #FromDate <= #ToDate /*Inclusive*/
begin
insert into #Dates(DateValue) values(#FromDate)
set #FromDate = #FromDate + 1
end
set NOCOUNT OFF
select dates.DateValue,
Col1...
from #Dates dates
left outer join SourceTableOrView data on
data.DateValue >= dates.DateValue
and data.DateValue < dates.DateValue + 1 /*NB: Exclusive*/
where ...?
Related
I am writing a stored procedure to update a company calendar. The calendar data is stored within a MYSQL table. The stored procedure is an insert function which should force all calendar events between two dates to be shifted to the next available business day. Type 1 denotes a day on the calendar. Type 2 denotes an event on the calendar.
The calendar recognizes a business day through the Moves column. A day with Moves(more than 0) is a business day on the calendar.
When I run this query by itself, it returns the expected information:
SELECT `Date`, Moves FROM tblschedules WHERE Type = 1;
It gives me the number of Moves for each day. However, when I try and update the table, the update does nothing, because the Moves column returns as NULL. Here is the query for the update:
SET SQL_SAFE_UPDATES = 0;
UPDATE tblschedules tb1
INNER JOIN (SELECT `Date`, Moves FROM tblschedules WHERE Type = 1) tb2
ON tb2.Date = tb1.Date
SET tb1.Date = tb1.Date + Interval 1 Day
WHERE tb1.Type = 2 AND tb1.Date >= ddate AND tb1.Date < opendate AND tb2.Moves = 0;
So, to debug, I began putting select statements within my stored procedure, and found that the initial query is not returning the Moves column, but instead returns null values as Moves. So that tb2 is returning every Date properly, but is not retrieving the Moves values properly.
Here is the larger block of code. The intloop, ddate and opendate are all functioning properly.
looplabel: Loop
/* Select Statements for Debug Purposes*/
SELECT opendate;
SELECT ddate;
SELECT intloop;
SELECT `Date`, Moves FROM tblschedules WHERE Type = 1;
SELECT * FROM tblschedules tb1
LEFT JOIN (SELECT `Date`, Moves FROM tblschedules WHERE Type = 1) tb2
ON tb2.Date = tb1.Date
WHERE tb1.Type = 2 AND tb1.Date >= ddate AND tb1.Date < opendate;
/* End of Select Statements for Debug Purposes*/
IF IFNULL(intloop,0) = 0 THEN
LEAVE looplabel;
END IF;
SET SQL_SAFE_UPDATES = 0;
UPDATE tblschedules tb1
INNER JOIN (SELECT `Date`, Moves FROM tblschedules WHERE Type = 1) tb2
ON tb2.Date = tb1.Date
SET tb1.Date = tb1.Date + Interval 1 Day
WHERE tb1.Type = 2 AND tb1.Date >= ddate AND tb1.Date < opendate AND tb2.Moves = 0;
SET intloop = intloop - 1;
END LOOP;
I am not sure why the table returns the Moves properly outside of the stored procedure, but not inside of the stored procedure. I am wondering if it might have something to do with uncommitted changes/table locking. Its probably something basic and stupid I am missing. Thank you for your time.
I'm new to SQL Server partitioning. I have tried a some of the tutorials and finally I got this based on that created a partition but the problem is that I can't retrieve the data's based on partition like in MYSQL query
SELECT * FROM TABLE_NAME PARTITION(partitionId)
and also tried this query in SQL Server
SELECT $PARTITION.partition_function (partition_function_id)
EDIT:
The picture shows the data in table I have created partition based on Date, I like to retrieve data based on partition name, which is if partition name is 20170203 I am excepting results on that date.
select * from TABLE_NAME
where $partition.fpartfunction(date) = 1
This returns all the data in partition 1.
If you want to retrieve it from the date you can do as below,
CREATE PARTITION FUNCTION PF_Date(date) AS RANGE LEFT FOR VALUES('2017-01-01','2018-01-01');
--first partition
SELECT *
FROM schema.table_name
WHERE key_column <= '2017-01-01';
--second partition
SELECT *
FROM schema.table_name
WHERE key_column > '2017-01-01' AND key_column <= '2018-01-01';
--last partition
SELECT *
FROM schema.table_name
WHERE key_column > '2018-01-01';
I hope this may work for your need:
CREATE PROCEDURE USP_GetRowsWithinPartition (#InputDate DATE)
AS
BEGIN
DECLARE #TV TABLE
(StartDate DATE,
EndDate DATE
)
INSERT INTO #TV (StartDate, EndDate)
SELECT TOP 1
CAST((CASE
WHEN (DATEPART(MONTH, #InputDate)) = 7 AND (DATEPART(DAY, #InputDate)) > 2
THEN DATEADD(YEAR, -1, CONVERT(DATE, prv.Value, 116))
WHEN (DATEPART(MONTH, #InputDate)) > 7 AND (DATEPART(DAY, #InputDate)) >= 1
THEN DATEADD(YEAR, -1, CONVERT(DATE, prv.Value, 116))
ELSE prv.Value
END) AS DATE) AS StartDate,
CAST((CASE
WHEN (DATEPART(MONTH, #InputDate)) = 7 AND (DATEPART(DAY, #InputDate)) > 2
THEN prv.Value
WHEN (DATEPART(MONTH, #InputDate)) > 7 AND (DATEPART(DAY, #InputDate)) >= 1
THEN prv.Value
ELSE DATEADD(YEAR, 1, CONVERT(DATE, prv.Value, 116))
END ) AS DATE) AS EndDate
FROM sys.partition_functions pf
INNER JOIN sys.partition_schemes AS ps ON ps.function_id = pf.function_id
INNER JOIN sys.destination_data_spaces AS dds ON dds.partition_scheme_id = ps.data_space_id
INNER JOIN sys.dm_db_partition_stats pstats ON pstats.partition_number = dds.destination_id
INNER JOIN sys.partitions p ON p.partition_id = pstats.partition_id
INNER JOIN sys.data_spaces AS ds ON dds.data_space_id = ds.data_space_id
INNER JOIN sys.partition_range_values AS prv ON pf.function_id = prv.function_id
WHERE pstats.object_id = OBJECT_ID('YourTable')
GROUP BY prv.boundary_id, prv.value
ORDER BY ABS(DATEDIFF(DAY, #InputDate, Convert(DATE, value, 116)))
SELECT yt.* FROM YourTable yt
INNER JOIN #TV t ON yt.DateColumn>=t.StartDate AND yt.DateColumn < t.EndDate
END
EXEC USP_GetRowsWithinPartition '2015-08-01'
(MySQL)
There is virtually no use for accessing a specific PARTITION. You should let the WHERE clause pick which partition(s) to use.
Furthermore, there are very few uses for partitioning. I recommend that you ignore that feature until you have a non-trivial amount of experience under your belt.
Any, to answer your question... Yes, this should work:
SELECT * FROM TABLE_NAME PARTITION(partitionId);
But that was not available until 5.6.
The partitionid may need to be quoted, especially if it is a number. For further research, please provide SHOW CREATE TABLE.
I've got two tables, a project table and a calendar table. The first containts a startdate and days required. The calendar table contains the usual date information, like date, dayofweek, and a column is workingday, which shows if the day is a saturday, sunday, or bank holiday (value = 0) or a regular workday (value = 1).
For a certain report I need write a stored procedure that calculates the predicted enddate by adding the number of estimated workddays needed.
Example:
**Projects**
Name Start_Planned Work_days_Required
Project A 02.05.2016 6
Calendar (04.05 is a bank holdiday)
Day Weekday Workingday
01.05.2016 7 0
02.05.2016 1 1
03.05.2016 2 1
04.05.2016 3 0
05.05.2016 4 1
06.05.2016 5 1
07.05.2016 6 0
08.05.2016 7 0
09.05.2016 1 1
10.05.2016 2 1
Let's say, the estimated number of days required is given as 6 (which leads to the predicted enddate of 10.05.2016). Is it possible to join the tables in a way, which allows me to put something like
select date as enddate_predicted
from calendar
join projects
where number_of_days = 6
I would post some more code, but I'm quite stuck on how where to start.
Thanks!
You could get all working days after your first date, then apply ROW_NUMBER() to get the number of days for each date:
SELECT Date, DayNum = ROW_NUMBER() OVER(ORDER BY Date)
FROM Calendar
WHERE IsWorkingDay = 1
AND Date >= #StartPlanned
Then it would just be a case of filtering for the 6th day:
DECLARE #StartPlanned DATE = '20160502',
#Days INT = 6;
SELECT Date
FROM ( SELECT Date, DayNum = ROW_NUMBER() OVER(ORDER BY Date)
FROM Calendar
WHERE WorkingDay = 1
AND Date >= #StartPlanned
) AS c
WHERE c.DayNum = #Days;
It's not part of the question, but for future proofing this is easier to acheive in SQL Server 2012+ with OFFSET/FETCH
DECLARE #StartPlanned DATE = '20160502',
#Days INT = 6;
SELECT Date
FROM dbo.Calendar
WHERE Date >= #StartPlanned
AND WorkingDay = 1
ORDER BY Date
OFFSET (#Days - 1) ROWS FETCH NEXT 1 ROWS ONLY
ADDENDUM
I missed the part earlier about having another table, and the comment about putting it into a cursor has prompted me to amend my answer. I would add a new column to your calendar table called WorkingDayRank:
ALTER TABLE dbo.Calendar ADD WorkingDayRank INT NULL;
GO
UPDATE c
SET WorkingDayRank = wdr
FROM ( SELECT Date, wdr = ROW_NUMBER() OVER(ORDER BY Date)
FROM dbo.Calendar
WHERE WorkingDay = 1
) AS c;
This can be done on the fly, but you will get better performance with it stored as a value, then your query becomes:
SELECT p.Name,
p.Start_Planned,
p.Work_days_Required,
EndDate = c2.Date
FROM Projects AS P
INNER JOIN dbo.Calendar AS c1
ON c1.Date = p.Start_Planned
INNER JOIN dbo.Calendar AS c2
ON c2.WorkingDayRank = c1.WorkingDayRank + p.Work_days_Required - 1;
This simply gets the working day rank of your start date, and finds the number of days ahead specified by the project by joining on WorkingDayRank (-1 because you want the end date inclusive of the range)
This will fail, if you ever plan to start your project on a non working day though, so a more robust solution might be:
SELECT p.Name,
p.Start_Planned,
p.Work_days_Required,
EndDate = c2.Date
FROM Projects AS P
CROSS APPLY
( SELECT TOP 1 c1.Date, c1.WorkingDayRank
FROM dbo.Calendar AS c1
WHERE c1.Date >= p.Start_Planned
AND c1.WorkingDay = 1
ORDER BY c1.Date
) AS c1
INNER JOIN dbo.Calendar AS c2
ON c2.WorkingDayRank = c1.WorkingDayRank + p.Work_days_Required - 1;
This uses CROSS APPLY to get the next working day on or after your project start date, then applies the same join as before.
This query returns a table with a predicted enddate for each project
select name,min(day) as predicted_enddate from (
select c.day,p.name from dbo.Calendar c
join dbo.Calendar c2 on c.day>=c2.day
join dbo.Projects p on p.start_planned<=c.day and p.start_planned<=c2.day
group by c.day,p.work_days_required,p.name
having sum(c2.workingday)=p.work_days_required
) a
group by name
--This gives me info about all projects
select p.projectname,p.Start_Planned ,c.date,
from calendar c
join
projects o
on c.date=dateadd(days,p.Work_days_Required,p.Start_Planned)
and c.isworkingday=1
now you can use CTE like below or wrap this in a procedure
;with cte
as
(
Select
p.projectnam
p.Start_Planned ,
c.date,datediff(days,p.Start_Planned,c.date) as nooffdays
from calendar c
join
projects o
on c.date=dateadd(days,p.Work_days_Required,p.Start_Planned)
and c.isworkingday=1
)
select * from cte where nooffdays=6
use below logic
CREATE TABLE #proj(Name varchar(50),Start_Planned date,
Work_days_Required int)
insert into #proj
values('Project A','02.05.2016',6)
CReATE TABLE #Calendar(Day date,Weekday int,Workingday bit)
insert into #Calendar
values('01.05.2016',7,0),
('02.05.2016',1,1),
('03.05.2016',2,1),
('04.05.2016',3,0),
('05.05.2016',4,1),
('06.05.2016',5,1),
('07.05.2016',6,0),
('08.05.2016',7,0),
('09.05.2016',1,1),
('10.05.2016',2,1)
DECLARE #req_day int = 3
DECLARE #date date = '02.05.2016'
--SELECT #req_day = Work_days_Required FROM #proj where Start_Planned = #date
select *,row_number() over(order by [day] desc) as cnt
from #Calendar
where Workingday = 1
and [Day] > #date
SELECT *
FROM
(
select *,row_number() over(order by [day] desc) as cnt
from #Calendar
where Workingday = 1
and [Day] > #date
)a
where cnt = #req_day
I have table 1 with 3 columns id, startdate and enddate. With order id being the primary key how do I list the dates between the date range Startdate and Enddate?
What I have:
id Startdate EndDate
1 2/11/2014 2/13/2014
2 2/15/2014 2/17/2014
What I need:
id Date
1 2/11/2014
1 2/12/2014
1 2/13/2014
2 2/15/2014
2 2/16/2014
2 2/17/2014
How do I do this?
Use recursive CTE:
WITH tmp AS (
SELECT id, StartDate AS [Date], EndDate
FROM MyTable
UNION ALL
SELECT tmp.id, DATEADD(DAY,1,tmp.[Date]), tmp.EndDate
FROM tmp
WHERE tmp.[Date] < tmp.EndDate
)
SELECT tmp.ID, tmp.[Date]
FROM tmp
ORDER BY tmp.id, tmp.[Date]
OPTION (MAXRECURSION 0) -- For long intervals
If you have to use cursor/loop, most times you are doing it wrong.
If you do a one-off setup of an auxiliary calendar table as shown at Why should I consider using an auxiliary calendar table?, possibly omitting a lot of the columns if you don't need them, like this:
CREATE TABLE dbo.Calendar
(
dt SMALLDATETIME NOT NULL
PRIMARY KEY CLUSTERED,
Y SMALLINT,
M TINYINT,
D TINYINT
)
GO
SET NOCOUNT ON
DECLARE #dt SMALLDATETIME
SET #dt = '20000101'
WHILE #dt < '20300101'
BEGIN
INSERT dbo.Calendar(dt) SELECT #dt
SET #dt = #dt + 1
END;
UPDATE dbo.Calendar SET
Y = YEAR(dt),
M = MONTH(dt),
D = DAY(dt);
(You may well not need the Y, M, D columns at all, but I left those in to show that more data can be stored for fast access - the article I linked to shows how that could be used.)
Then if your table is named "so", your code would simply be
SELECT A.id, C.dt
FROM so AS A
JOIN Calendar AS C
ON C.dt >= A.StartDate AND C.dt<= A.EndDate
An advantage of using an auxiliary table like that is that your queries can be faster: the work done in setting one up is a one-time cost which doesn't happen during usage..
Instead of using CTE (to over come recursive and performance when date range is large) below query can be used to get the list of dates between two date range.
DECLARE #StartDateSTR AS VARCHAR(32); DECLARE #EndDateSTR AS
VARCHAR(32); DECLARE #EndDate AS DATE; DECLARE #StartDate AS DATE;
SET #StartDateSTR = '01/01/1990'; SET #EndDateSTR = '03/31/2025'; SET
#StartDate = CAST(#StartDateSTR AS date); SET #EndDate =
cast(#EndDateSTR AS date); SELECT
DATEADD(DAY, n1.rn - 1, #StartDate) AS dt FROM (SELECT rn=Row_number() OVER( ORDER BY (SELECT NULL)) FROM sys.objects a
CROSS JOIN sys.objects b CROSS JOIN sys.objects c CROSS JOIN
sys.objects d) as n1 WHERE n1.[rn] <= Datediff(dd, #StartDate,
#EndDate)+1;
I have the following stored procedure that I need to return multiple types of data for an annual report. For some reason I only get the first column in the results. If I comment out the NumOfBonds column, the TotalAmount column is the only one returned. I need it to return all these rows, but for some reason I am only getting one when I execute the procedure:
ALTER PROCEDURE dbo.AnnualReport
#Agency varchar(50),
#Subagency varchar(50),
#StartDate datetime,
#EndDate datetime
AS
SELECT COUNT(*) AS NumOfBonds, SUM(ISNULL(Powers.BondAmount, 0)) + SUM(ISNULL(Charges.BondAmount, 0)) AS TotalAmount,
SUM(ISNULL(Powers.BondPremium, 0)) + SUM(ISNULL(Charges.BondPremium, 0)) AS TotalPremium,
SUM(ISNULL(Powers.BondPremium, 0)) + SUM(ISNULL(Charges.BondPremium, 0)) + SUM(ISNULL(Fees.Amount, 0))
+ SUM(ISNULL(ForfeitureExpense.Amount, 0)) - SUM(ISNULL(Payment.Amount, 0)) AS TotalBalance
FROM Bond
LEFT OUTER JOIN
(
SELECT Powers.Bond, SUM(Charge.BondAmount) AS BondAmount, SUM(Charge.BondPremium) AS BondPremium
FROM Powers INNER JOIN Charge ON Powers.Surety = Charge.PowerSurety
AND Powers.PowerPrefix = Charge.PowerPrefix AND Powers.PowerNumber = Charge.PowerNumber
GROUP BY Powers.Bond
) AS Powers ON Bond.ID = Powers.Bond
LEFT OUTER JOIN
(
SELECT BondID, SUM(BondAmount) AS BondAmount, SUM(BondPremium) AS BondPremium
FROM ChargeWithoutPower
GROUP BY BondID
) AS Charges ON Bond.ID = Charges.BondID
LEFT OUTER JOIN
(
SELECT Bond, SUM(Amount) AS Amount
FROM BondFee
WHERE Date >= #StartDate AND Date <= #EndDate
GROUP BY Bond
) AS Fees ON Bond.ID = Fees.Bond
LEFT OUTER JOIN
(
SELECT Bond, SUM(Amount) AS Amount
FROM ForfeitureExpense
WHERE ExpenseDate >= #StartDate AND ExpenseDate <= #EndDate
GROUP BY Bond
) AS ForfeitureExpense ON Bond.ID = ForfeitureExpense.Bond
LEFT OUTER JOIN
(
SELECT Bond, SUM(Amount) AS Amount
FROM Payment
WHERE Date >= #StartDate AND Date <= #EndDate
GROUP BY Bond
) AS Payment ON Bond.ID = Payment.Bond
WHERE Bond.ReleaseDate >= #StartDate AND Bond.ReleaseDate <= #EndDate
AND Bond.Agency = #Agency AND Bond.Subagency = #Subagency
RETURN
The basic purpose is to give me the number of bonds written in that year, the total amount of the bonds in that year, the total premium on those bonds, and the balance at the end of the year for those bonds. I am not sure why it only sees the first column and not the rest of them.
Alright, so the basic solution here is that executing the SQL Server 2008 Stored Procedure alone did not give the proper results in Visual Studio 2010 Ultimate, but as soon as I added it to my data set and executed the procedure from C# code, it solved the problem. I'm not sure why this is, but I blame Visual Studio 2010 Ultimate.