I have the scenario where I need to load data for next quarter ,this data should be insert only and from next day
data should be incremental load.For example for quarter 1 jan to 6 march .On 1 jan we have loaded historical load or first time insert and on 2 jan onwards it in incremental load. Now on 7 march we want fresh insert and from 8 march incremental load will run on newly insert data on 7 march. Like we want to lock the data from the quarter 1 jan to 6 march and fresh insert should happen on 7 march and 8 march incremental load should run on data loaded on 7 march. How to achieve this in ssis?
The best way to do this is create a calendar table and have a special column within that table that maps every single date to a 'freeze date' attribute.
So I leave you to do some research on calendar tables.
In the meantime, without a calendar table, here is something to get you started.
Firstly, this expression gives you the first month in the current quarter. So if today is anywhere within Jan - Mar, this expression gives you 1. If today is anywhere within Oct-Dec, it gives you 10
SELECT (((MONTH(GETDATE()))-1)/3)*3+1 As FirstMonthInCurrentQuarter
Now we convert that number to an actual date on the first of the month.
First, generate a numeric representation:
SELECT
YEAR(GETDATE()) * 10000 +
((((MONTH(GETDATE()))-1)/3)*3+1) * 100 +
1
We just convert that to an actual date:
SELECT
CONVERT(
DATE,
CAST(
YEAR(GETDATE()) * 10000 +
((((MONTH(GETDATE()))-1)/3)*3+1) * 100 +
1
AS VARCHAR(8)),
112)
That's a monstrous expression that can be replaced with a calendar table.
If you run it right now you'll get 1 Jan 2019. If you run that on 5th April you'll get 1st April. Hopefully you understand how that shuffles along automatically.
Now we use that date to decide what to load into your table.
DECLARE #LoadDate DATE
SET #LoadDate =
CONVERT(
DATE,
CAST(
YEAR(GETDATE()) * 10000 +
((((MONTH(GETDATE()))-1)/3)*3+1) * 100 +
1
AS VARCHAR(8)),
112)
-- Delete all the data in the target table after this date:
DELETE TargetTable WHERE TransactionDate >= #LoadDate;
-- Use your ETL tool to load this in:
INSERT INTO TargetTable (TransactionDate, Amount, Account)
SELECT TransactionDate, Amount, Account
FROM SourceTable WHERE TransactionDate >= #LoadDate;
You have nowhere near enough information in your question for a meaningful answer, but maybe this will give you an idea
95% chance you never return though.
Related
I have a MySQL database with one table that contains a data field and a "period" field, in months - int.
The idea is that the date indicates a due date to begin a project inside my company. And the "period" the period of time it is suppose to take to finish it, in months.
I need to select rows that will impact a given year. So if I am generating a report for 2014, I need to select the rows such: date+period is inside 2014.
It will be easy to do it inside the program, but I am looking for a way to do it in the query - if possible.
So basically I just need a way to sum dates and ints in a query, where the int is the number of months.
Any thoughts?
It's easy to do date arithmetic in MySQL and other RDMS systems. You need all the records in which the start date is not after the year in question OR the end date is not before the year in question. That is this expression:
NOT(YEAR(start_date) > 2014 OR YEAR(start_date + INTERVAL period MONTH) < 2014)
This logically reduces to
YEAR(start_date) <= 2014 AND YEAR(start_date + INTERVAL period MONTH) >= 2014
So this query will do it.
SELECT whatever, whatever
FROM project
WHERE YEAR(start_date) <= 2014
AND YEAR(start_date + INTERVAL period MONTH) >= 2014
AND (whatever other selection criteria you have)
This will give all projects that were active during 2014, including those that started before 2014 and those that will still be in progress at the end of that year.
I'm not sure this is even possible without using PHP, but I'd love to try.
I have a database that looks like this (a bunch of other stuff, but this is all that is relevant:
Date_Day (is a range from 1 to 31 with no trailing 0)
Date_Month (is a range from January to December, not numerical)
Date_Year (is the year in 4 digit format, ex: 2005)
Total (number with 2 decimal places)
I know the way the dates are stored is awful, but this is the database I was given. If there is a query that I could use these columns to create an actual DATETIME column, I would happily do it, I just don't know what that query looks like.
I have this query that returns the Total sales amount for this day for all previous years:
SELECT
Date_Year, Date_Month, SUM(Total)
FROM
tablename
WHERE
Date_Year < YEAR(CURDATE())
AND
Date_Month = MONTHNAME(CURDATE())
AND
Date_Day = DAY(CURDATE())
GROUP BY
Date_Year, Date_Month
So if I run this today, I get the daily totals for October 4th for all previous years. The issue is that in sales, this isn't very helpful for comparing growth. What I really need is the daily totals for the 1st Friday in October for all previous years.
Is this possible without having to rely on PHP? If so, I would be very grateful for your help.
Thank you.
You might be looking for DAYOFWEEK()
Returns the weekday index for date (1 = Sunday, 2 = Monday, …, 7 = Saturday). These index values correspond to the ODBC standard.
mysql> SELECT DAYOFWEEK('2007-02-03');
> 7
SELECT
Date_Year, Date_Month, SUM(Total)
FROM
tablename
WHERE
Date_Year < YEAR(CURDATE())
AND
Date_Month = MONTHNAME(CURDATE())
AND
Date_Day = DAY(LAST_DAY(CURDATE()) - ((28 + WEEKDAY(LAST_DAY(CURDATE())) - 4)))
GROUP BY
Date_Year, Date_Month
maybe this will help
I'm trying to query through historical data and I need to return data just from a 1 month period: 2 weeks back and 2 weeks forward,but I need the year to not matter.
So, if I was to make the query today I would want all rows with date between xxxx-06-31 and xxxx-07-27
Thanks in advance for the help!
EDIT:
I've tried two ways. both of which I believe will not work around the new year. One is to use datepart(day) and the other would be to simply take the year off of date and compare.
The best way to think of this problem is to convert your dates to a number between 0 and 365 corresponding to the day in the year. Then simply choosing dates where this difference is less than 14 gives you your two week window.
That will break down at the beginning or end of the year. But simple modular arithmetic gives you the answer.
Fortunately, MySQL has DAYOFYEAR(date), so it's not so complicated:
SELECT * FROM tbl t
WHERE
MOD(DAYOFYEAR(currdate) - DAYOFYEAR(t.the_date) + 365, 365) <= 14
OR MOD(DAYOFYEAR(t.the_date) - DAYOFYEAR(currdate) + 365, 365) <= 14
That extra + 365 is needed since MySQL's MOD will return negative numbers.
This answer doesn't account for leap years correctly. If the current year is not a leap year and the currdate is within 14 days of the end of the year, then you'll miss one day in Jan that you should have included. If you care about that, then you should replace 365 with [the number of days in the year - 1].
Supposed you have a date like this,
create table datelist
(
d date
);
insert into datelist values
('2012-07-01'),
('2011-06-29'),
('2012-07-02'),
('2010-07-05'),
('2012-05-31'),
('2010-06-30');
Try this query below,
SELECT d, date_format(d,'%Y-%b-%d')
FROM datelist
WHERE (MONTH(d) = 6 AND DAYOFMONTH(d) >= 30)
OR (MONTH(d) = 7 AND DAYOFMONTH(d) <= 27)
SQLFiddle Demo
Is it OK if the solution is terribly slow?
SELECT tbl.*
FROM tbl
INNER JOIN (SELECT COALESCE(DATE(CONCAT(yyyy, '-', MONTH(CURRENT_DATE), '-', DAYOFMONTH(CURRENT_DATE)),
DATE(CONCAT(yyyy, '-02-28'))) AS midpoint
FROM (SELECT DISTINCT(YEAR(d)) AS yyyy
FROM tbl) all_years) adjusted
ON tbl.datecol BETWEEN adjusted.midpoint - INTERVAL 2 WEEK
AND
adjusted.midpoint + INTERVAL 2 WEEK;
That computes all midpoints for all years in the data set, and then pulls records +- 2 weeks from any such midpoint, which handles end-of-year wrapping.
The COALESCE handles 02-29 on years without leapday (which MySQL will NULL-ify), forcing it down to 02-28.
I am trying to write a function in mySQL that takes two dates(startDate and endDate) as parameters. It then calculates the days in each month.
The database contains a targetRevenue table that has got the target revenue values for each month and year.
id month year targetRev
25 1 2012 1000.00
26 2 2012 5000.00
27 3 2012 8000.00
The function finds the revenue for a month based on the number of days in it and then returns the total.
Example : startDate : 2012-01-19 endDate : 2012-03-24
Function returns [ targetRev(19 days in Jan) + targetRev(29 days Feb) + targetRev(24days in March)]
I am new to writing functions in mysql , so a little bit of help to get me started would be very useful. Thanks in advance!
If instead of your month and year columns, you represented the month of each record in your targetRevenue table by a DATE column containing the first day of each month:
ALTER TABLE targetRevenue
ADD COLUMN first DATE;
UPDATE targetRevenue
SET first = STR_TO_DATE(CONCAT_WS('-', year, month, 1), '%Y-%c-%e');
ALTER TABLE targetRevenue
DROP COLUMN year,
DROP COLUMN month;
You could then obtain the total target revenue for your project (assuming it is inclusive of both start and end date) with:
-- calculate the summation of
SELECT SUM(CONVERT(
-- number of project days in month...
GREATEST(0,
-- ...is calculated as the difference between...
DATEDIFF(
-- ...the last day of the project in this month...
LEAST('2012-03-24', LAST_DAY(first)),
-- ...and the first day of the project in this month...
GREATEST('2012-01-19', first)
)
-- ...plus one because first and last project days were inclusive
+ 1
)
-- multiply by the target revenue for this month
* targetRev
-- divide by the number of days in the month
/ DAY(LAST_DAY(first)),
-- convert result to fixed-point format, to two d.p.
DECIMAL(11,2)
)) AS total
FROM targetRevenue
-- only perform for months in which the project was active
WHERE '2012-01-19' <= LAST_DAY(first) AND first <= '2012-03-24'
See it on sqlfiddle.
If you can't change the schema, you could replace references to first with the value to which that column was updated above.
For this you can use SUM() function like:
SELECT SUM(targetRev) from your_table
WHERE date_column BETWEEN your_startDate_column AND your_endDate_column;
you need not to calculate days of each month..
Use this query like this
SELECT SUM(targetRev), MONTH(date_column) as mo
from your_table
WHERE date_column BETWEEN your_startDate AND your_endDate
GROUP BY mo;
This will give the result for each month total revenue (use like this logic)
If it is two different years you can use like
concat(year(date_column),month(date_column)) as mo
I have a MySQL table of photovoltaic electricity generation data (pvdata) from which I need to produce a monthly summary table. A simplified table is shown:
id date time pvdata
1 2012-01-01 10:00 50
1 2012-01-31 12:00 60
1 2012-02-10 13:00 70
2 2012-02-08 10:00 12
2 2012-03-20 10:00 17
The monthly summary table needs to show the cumulative generation for all systems in the database, regardless of whether I have received data for that month, so for example month 3 below contains the total generation from id = 1 (data received in month 2).
Also there may be more than one data point for an id in the same month, so the report must report the max(data) for the month.
year month cum_data
2012 1 60
2012 2 82
2012 3 87
I am pretty new to this, so have struggled for a while. The best I can come up with shows the cumulative total for the month, but without including the cumulative total for ids for which there is no data in the current month:
CREATE TEMPORARY TABLE intermed_gen_report
SELECT year(date) AS year, month(date) AS month, id, max(pvdata) AS maxpvdata
FROM pvdata
GROUP BY id, year(date), month(date)
ORDER BY year(date), month(date);
SELECT year, month, SUM(maxpvdata) AS cum_data
FROM intermed_gen_report
GROUP BY year, month
ORDER BY year, month;
Giving:
year month cum_data
2012 1 60
2012 2 82
2012 3 17
I think the problem is one kind of like this http://www.richnetapps.com/using-mysql-generate-daily-sales-reports-filled-gaps/ - you will want to create a table (possibly temporary) with dates (or year / month values). However that example leaves zeros where there is no data - I think you will want to do a join on a subselect that returns the most recent data before that date (or year/ month value).
I agree I think with what Aerik suggests. You will want to join your data of what is usually called a 'date dimension table'. You can find lots of examples on how to populate said table. This is a common technique in data warehousing.
You can also do what you need in one select using sub selects. Take a look at some of the previous threads like: generate days from date range