Get the last twelve months worth of data from a database - mysql

Question

I'm trying to work with a database of unemployment figures from the department of labor statistics' data (available at ftp://ftp.bls.gov/pub/time.series/la/)
I need to get the last 12 months of data for any given state, which is trickier then just selecting all data from the last year as they don't always have the last few months of data in yet (right now, the last month's worth of data is November 2010).
I know which record is the newest, and the date fields I have in the database to work with are:
period_name (month name)
year
period (M01, M02, etc for January, February)
My current SQL, which pulls data from a bunch of JOINed tables, is:
USE unemploymentdata;
SELECT DISTINCT series.series_id, period_name, year, value, series.area_code,
footnote_codes, period_name, measure_text, area_text, area_type_text
FROM state_overview
LEFT JOIN series ON state_overview.series_id=series.series_id
LEFT JOIN footnote ON state_overview.footnote_codes = footnote.footnote_code
LEFT JOIN period ON state_overview.period = period.period
LEFT JOIN measure ON series.measure_code = measure.measure_code
LEFT JOIN area ON series.area_code=area.area_code
LEFT JOIN area_type ON area.area_type_code=area_type.area_type_code
WHERE area_text = 'State Name' AND year > 2009
ORDER BY state_overview.period, measure_text;
Any idea?

Answer 1

Since you have textual values to work with for month and year, you'll need to convert them to MySQL-formatted DATE values and can then let MySQL calculate the last year interval like so:
SELECT ... WHERE STR_TO_DATE(CONCAT(period_name,' 1 ',year),'%M %d %Y') >= DATE_SUB(STR_TO_DATE(CONCAT(most_recent_period_name,' 1 ',most_recent_year),'%M %d %Y'), INTERVAL 1 YEAR) ...;
The CONCAT() function is just building a string like "Month 1 YYYY", and the STR_TO_DATE() function is taking that string and a formatting string to tell it how to parse it, and converting it into a DATE.
Note: This query probably sucks index-wise but it should work. : )

Answer 2

I think a few changes to WHERE clause should do it, but for effeciency/simplcity you should also add MAX(year) to the SELECT section.
SELECT ...... MAX(year) as max_year .....
WHERE area_text = 'State Name'
AND year >= max_year - 1
AND period >= (SELECT MAX(period) WHERE year = max_year)
ORDER BY state_overview.period, measure_text;

Answer 3

You can store the year and month as a date, even though you don't have the day information. Just use the first of each month.
{2009, 'M1'} => 2009-01-01
{2009, 'M2'} => 2009-02-01
{2009, 'M3'} => 2009-03-01
This makes date arithmetic much easier than dealing with substrings of (potentially dirty) data. Plus (and this is big), you can index the data much more effective. As a bonus, you can now extract a lot of extra goodies using DATE_FORMAT such as month names, nr of days in month etc.
Does all states have data for all months, and is the data updated at the same time? The answer to that question dictates what query strategy you should use.

Answer 4

The best way is to take the strtotime ($a) of correct 1 year ago and then, when fetching the value from database then find the strtotime ($b) of the date in each result. Now
if($b < $a){
continue;
}
else {
//do something.
}