Count tweets between dates (mysql) - mysql

Question

I have an assignment to create a twitter like database. And in this assignment i have to filter out the trending topics. My idea was to count the tweets with a specific tag between the date the tweet was made and 7 days later, and order them by the count.
I have the following 2 tables i am using for this query :
Table Tweet : id , message, users_id, date
Table Tweet_tags : id, tag, tweet_id
Since mysql isn't my strong point at all im having trouble getting any results from the query.
The query i tried is :
Select
Count(twitter.tweet_tags.id) As NumberofTweets,
twitter.tweet_tags.tag
From twitter.tweet
Inner Join twitter.tweet_tags On twitter.tweet_tags.tweet_id = twitter.tweet.id
WHERE twitter.tweet_tags.tag between twitter.tweet.date and ADDDATE(twitter.tweet.date, INTERVAL 7 day)
ORDER BY NumberofTweets
The query works, but gives no results. I just can't get it to work. Could you guys please help me out on this, or if you have a better way to get the trending topics please let me know!
Thanks alot!

Answer 1

This is equivalent to your query, with table aliases to make it easier to read, with BETWEEN replaced by two inequality predicates, and the ADDDATE function replaced with equivalent operation...
SELECT COUNT(s.id) As NumberofTweets
, s.tag
FROM twitter.tweet t
JOIN twitter.tweet_tags s
ON s.tweet_id = t.id
WHERE s.tag >= t.date
AND s.tag <= t.date + INTERVAL 7 DAY
ORDER
BY NumberofTweets
Two things pop out at me here...
First, there is no GROUP BY. To get a count by "tag", you want at GROUP BY tag.
Second, you are comparing "tag" to "date". I don't know your tables, but that just doesn't look right. (I expect "date" is a DATETIME or TIMESTAMP, and "tag" is a character string (maybe what my daughter calls a "hash tag". Or is that tumblr she's talking about?)
If I understand your requirement:
For each tweet, and for each tag associated with that tweet, you want to get a count of the number of other tweets, that have a matching tag, that are made within 7 days after the datetime of the tweet.
One way to get this result would be to use a correlated subquery. (This is probably the easiest approach to understand, but is probably not the best approach from a performance standpoint).
SELECT t.id
, s.tag
, ( SELECT COUNT(1)
FROM twitter.tweet_tags r
JOIN twitter.tweet q
ON q.id = r.tweet_id
WHERE r.tag = s.tag
AND q.date >= t.date
AND q.date <= t.date + INTERVAL 7 DAY
) AS cnt
FROM twitter.tweet t
JOIN twitter.tweet_tags s
ON s.tweet_id = t.id
ORDER
BY cnt DESC
Another approach would be to use a join operation:
SELECT t.id
, s.tag
, COUNT(q.id) AS cnt
FROM twitter.tweet t
JOIN twitter.tweet_tags s
ON s.tweet_id = t.id
LEFT
JOIN twitter.tweet_tags r
ON r.tag = s.tag
LEFT
JOIN twitter.tweet q
ON q.id = r.tweet_id
AND q.date >= t.date
AND q.date <= t.date + INTERVAL 7 DAY
GROUP
BY t.id
, s.tag
ORDER
BY cnt DESC
The counts from both of these queries assume that tweet_tags (tweet_id, tag) is unique. If there are any "duplicates", then including the DISTINCT keyword, i.e. COUNT(DISTINCT q.id) (in place of COUNT(1) and COUNT(q.id) respectively) would get you the count of "related" tweets.
NOTE: the counts returned will include the original tweet itself.
NOTE: removing the LEFT keywords from the query above should return an equivalent result, since the tweet/tag (from t/s) is guaranteed to match itself (from r/q), as long as the tag is not null and the tweet date is not null.
Those queries are going to have problematic performance on large sets. Appropriate covering indexes are going to be needed for acceptable performance:
... ON twitter.tweet_tags (tag, tweet_id)
... ON twitter.tweet (date)