Getting data between two date strings in MySQL - mysql

Question

I have been recording Twitter data for a project I'm working on the date information is saved as Thu, 14 Jul 2011 06:21:48 +0000 in a string field.
How do I select data that falls between two dated using mySQL? I can get data larger than a value or smaller than a value but not between to values.
The data for example is:
Thu, 14 Jul 2011 06:21:48 +0000
Thu, 14 Jul 2011 12:18:21 +0000
Thu, 14 Jul 2011 18:48:00 +0000
Thu, 14 Jul 2011 23:48:02 +0000
Fri, 15 Jul 2011 06:48:10 +0000
Fri, 15 Jul 2011 12:48:00 +0000
Fri, 15 Jul 2011 18:43:32 +0000
Fri, 15 Jul 2011 23:44:08 +0000
Sat, 16 Jul 2011 06:47:08 +0000
Sat, 16 Jul 2011 12:46:49 +0000
Sat, 16 Jul 2011 18:45:41 +0000
Sat, 16 Jul 2011 23:41:27 +0000
My SQL string is:
SELECT *
FROM twitter
WHERE SUBSTR(twitter_date, 6, 11) >= '2011-06-15'
AND SUBSTR(twitter_date, 6, 11) <= '2011-06-21'
I've tried BETWEEN statements as well but no luck.
Any help will be appreciated!

Answer 1

You can't use between because they are strings and not actual date fields. If you know the format of the date will always be the same, you can do the following:
STR_TO_DATE(str,format)
SELECT *
FROM twitter
WHERE STR_TO_DATE(twitter_date, '%a, %c %b %Y %k:%i:%s')
between '2011-06-15' AND '2011-06-21'

Answer 2

You are comparing "6 Jul 2011" with 2011-06-15.
It should be
SELECT *
FROM twitter
WHERE SUBSTR(twitter_date, 6, 11) >= '6 Jul 2011'
AND SUBSTR(twitter_date, 6, 11) <= '21 Jul 2011'

Answer 3

You may want to look into MySQLs STR_TO_DATE function. This will give you a date and between should work as expected.

Answer 4

The query will fail. You're comparing two STRINGS, not dates. You need to tell MySQL explicitly that you want to treat the strings as dates. Since it's date information, store it in a date or datetime field type in MySQL, which saves you all this trouble:
SELECT STR_TO_DATE(SUBSTR(twitter_date, 6, 11)) >= '2011-06-15' ...
Forcing your left-hand-side to be a proper date value will hint to MySQL that it should treat the right-hand-side as a date as well.

Answer 5

Once you have altered your table to hold the date data in proper DATETIME fields, you may want to refer to my answer to the following question regarding dates and filtering them, which may prove useful.
For example, in your query / data above, what would happen if you had no data for one of the dates you were querying? You'd get no output for that date, which may look odd when charting, etc. If you use the technique I describe in the attached question, you can get that additional information, and other useful stuff like the day of the week, month name, etc.
Select all months within given date span, including the ones with 0 values
Hope that helps!