Tweepy information into MySQL - text being cut off - mysql

Question

I am inserting information from twitter using Tweepy into a MySQL table. Mostly it works are intended however, the actual text from the tweet is being cut off and many end with "..." and I am not sure why this is.
An example of the output is this:
#JamesMelville: Remain voters are not "unpatriotic" by "talking the country down".Its the opposite. Remainers care about Britains fut…
Thanks for your help
Here is my stream listener class: (I am new to python so this is probably poor)
class listener(StreamListener):
def on_data(self, data):
try:
jsonData=json.loads(data)
tweets =(jsonData['text'])
if tweets.startswith('RT'):
retweets = True
else:
retweets = False
tweets = tweets.replace('RT', '')
tweets = tweets.replace("'", '')
tweets = tweets.lstrip()
tweets = tweets.replace('\n', '')
screen_name =(jsonData['user']['screen_name'])
name =(jsonData['user']['name'])
name = name.replace(',', ' ')
language =(jsonData['lang'])
location =(jsonData['place'])
coord =(jsonData['coordinates'])
device = jsonData['source'].split('">')[1].replace('</a>', '')
tweettime = jsonData['created_at'].replace('+', '')
tweettime = datetime.datetime.strptime(tweettime, "%a %b %d %H:%M:%S %f %Y")
date_to_string = str(tweettime.strftime("%Y/%m/%d,%H:%M:%S"))
date_created = date_to_string.split(',')[0]
time_created = date_to_string.split(',')[1]
created_time =(time_created)
created_date =(date_created)
htext = jsonData['entities']['hashtags']
htext2 = []
hashtag_list = ''
for hashtag in htext:
htext=str(hashtag['text'])
hashtag_list = hashtag_list + ' ::' + htext
hashtag_list = hashtag_list.replace("'", "")
conn = connect( host = '', port = , user = ', passwd = '', charset='utf8', autocommit = True);
conn.commit()
cursor = conn.cursor( cursors.DictCursor );
cursor.execute("CREATE DATABASE IF NOT EXISTS twitter")
cursor.execute("USE twitter")
cursor.execute( """CREATE TABLE IF NOT EXISTS `twitter_data`(ID INT AUTO_INCREMENT NOT NULL,`Name` VARCHAR( 200 ) ,`Screen name` VARCHAR( 200 ) , `Date created` VARCHAR ( 20 ), `Time created` VARCHAR ( 8 ), Tweet VARCHAR ( 200 ), Hashtags VARCHAR ( 200 ), Retweets VARCHAR ( 5 ), Lanugage VARCHAR ( 20 ), Device VARCHAR ( 60 ), Location VARCHAR( 200 ), Coordinates VARCHAR ( 200 ), PRIMARY KEY ( ID ))""" )
sql = "INSERT INTO `twitter_data` VALUES( Null, '" + str(name) + "', '" + str(screen_name) + "', '" + str(created_date) + "', '" + str(created_time) + "', '" + str(tweets) + "', '" + str(hashtag_list) + "', '" + str(retweets) + "', '" + str(language) + "', '" + str(device) + "', '" + str(location) + "', '" + str(coord) + "') "
print(sql)
cursor.execute(sql)
return True
except Exception as N:
print('failed on_data '+ str(N))
time.sleep(5)

Answer 1

I have found a solution to my question.
The text was being truncated when it was a retweet due to the 140 character limit.
Therefore it from nothing to do with MySQL.
My solution was to use the 'retweet_status' attribute (and split out the actual tweet text part) rather than 'text' if the tweet was a retweet, otherwise I just used 'text'.
Here is some of the code that was changed:
if tweets.startswith('RT'):
retweets = True
tweets = str(jsonData['retweeted_status']).split('\'text\':')[1]
if 'display_text_range' in tweets:
tweets = tweets.split(', \'display_text_range\'')[0]
else:
tweets = tweets.split(', \'source\'')[0]
else:
retweets = False
tweets = tweets.replace('RT', '')