I try to convert MySQl datetime to same in Python.On debug there is
ValueError: time data '2001-06-04T11:30:35' doesn't match format %Y-%m-%dT%H:%M:%S .
In MySQL there is no 'T' in data.I tried format with 'T' and without.
I saw this article How to convert the following string to python date? .
This is code:
query = QSqlQuery ()
query.exec_("SELECT birthday FROM vista.user ")
def countAge(birthday):
birthday = datetime.strptime(str(birthday), "%Y-%m-%dT%H:%M:%S.%f")
today = date.today()
age = today.year - birthday.year
if today.month < birthday.month:
age -= 1
elif today.month == birthday.month and today.day < birthday.day:
age -= 1
if age >= 0 :
return age
ages = []
index = 0
while (query.next()):
print(query.value(index).toString())
ages.append(countAge(query.value(index).toString()))
index = index + 1
What is a problem?
If an example date-string is 2001-06-04T11:30:35, then you need:
birthday = datetime.strptime(str(birthday), "%Y-%m-%dT%H:%M:%S")
Related
This question already has answers here:
How to convert a string to date in MySQL?
(5 answers)
Closed 2 years ago.
I am trying to compare the following:
SELECT g_m.user_id
, g_m.group_id
FROM Group_Members g_m
WHERE g_m.gender_id = 2
AND g_m.partner_gender_id = 1
AND g_m.birthday >= '01-01-1955'
AND g_m.birthday <= '12-31-2002'
AND g_m.user_id != 12
g_m.birthday in this case is '02-15-1998' which should show up, but this returns an empty array, because the date comparison does not seem to be accurate?
Here is the entire function and the dates are being passed from age minimum and age maximums brought from user.
var today = new Date();
var minYear = "01-01-" + (today.getFullYear() - userPref.age_max); //min year to start but oldest age
var maxYear = "12-31-" + (today.getFullYear() - userPref.age_min); //max year to end but youngest age
var qSelect = "SELECT g_m.user_id, g_m.group_id" +
" FROM Group_Members g_m WHERE g_m.gender_id = ? AND g_m.partner_gender_id = ?" +
" AND g_m.birthday >= STR_TO_DATE(?, '%m/%d/%Y') AND g_m.birthday <= STR_TO_DATE(?, '%m/%d/%Y')" +
" AND g_m.user_id != ?";
var qValues = [userPref.partner_gender_id, userObj.gender_id, minYear, maxYear, userObj.id];
Anyone know how to compare dates in a mysql query?
What is the data type of your dates ? You should declare them as "date", otherwise you won't be able to compare them.
With strings, '02-15-1998' < '03-15-1990'
With dates, your mysql request should be :
SELECT g_m.user_id, g_m.group_id FROM Group_Members g_m WHERE g_m.gender_id = 2 AND g_m.partner_gender_id = 1 AND g_m.birthday >= '1955-01-01' AND g_m.birthday <= '2002-12-31' AND g_m.user_id != 12
Sorry for my english, I'm french...
As the comments have already pointed out, you appear to be storing your dates as actual text. For a short term workaround, you may use STR_TO_DATE to convert your text dates to bona fide MySQL dates. Then, compare them against valid MySQL date literals:
SELECT
g_m.user_id,
g_m.group_id
FROM Group_Members g_m
WHERE
g_m.gender_id = 2 AND
g_m.partner_gender_id = 1 AND
STR_TO_DATE(g_m.birthday, '%m-%d-%Y') >= '1955-01-01' AND
STR_TO_DATE(g_m.birthday, '%m-%d-%Y') < '2003-01-01' AND
g_m.user_id != 12;
Longer term, you should make the birthday column datetime or timestamp.
Side note: I have rewritten the date range to include those born in the calendar years from 1955 to 2002, inclusive on both ends.
I am a new A-Level student, doing AS computer science.
I really don't know how to fix this problem but I need help with the following code.
I am trying to incorporate mySQL into Python and query data successfully. I also am trying to update a certain table via python. I am running into problems however that seem to throw errors that aren't actually happening as far as I can see. Am I blind? I hope that you geniuses can help me work it out.
The problem I have is that the SQL query I try to execute does not work and throws the following error (if the name I enter is 'Harvey', for example):
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1054 (42S22): Unknown column 'Harvey' in 'field list'
#Here is the actual code. It's trying to update a table record for name, DOB, and city by sorting via a unique 'key_field' identifier.
if int(key) < 1 or int(key) > len(ages):
print("The Key Field Number you entered is out of range of the data we have. Try again.")
continue
else:
n = input("""
Enter the name for the new record: """)
if len(n) > 50:
print("The name you entered is too long, try again.")
continue
c = input("""
Enter the name of the city for the new record: """)
if len(c) > 30:
print("The city you entered is too long. Try again.")
continue
dob_year = input("""
Entering the DOB (Date of Birth) for the new record...
Enter the year (YYYY): """)
if len(dob_year) != 4:
print("The year you enter must be 4 numbers long. Try again.")
continue
dob_month = input("Enter the month (MM): ")
if len(dob_month) > 2 or len(dob_month) < 1:
print("The month you enter must be 2 numbers long. Try again.")
continue
if len(dob_month) == 1:
dob_month = "0" + dob_month
dob_day = input("Enter the day (DD): ")
if len(dob_day) > 2 or len(dob_day) < 1:
print("The day you enter must be 2 numbers long. Try again.")
continue
if len(dob_day) == 1:
dob_day = "0" + dob_day
try:
int(dob_day)
int(dob_month)
int(dob_year)
except:
print("Enter integer values for the DOB (Date of Birth) section. Try again.")
continue
else:
d = "%s-%s-%s" % (dob_year, dob_month, dob_day)
ndc = []
ndc.append(n)
ndc.append(d)
ndc.append(c)
ndc = tuple(ndc)
key = int(key)
sql = "UPDATE ppl SET name = %s, dob = %s, city = %s WHERE key_field = %s" % (ndc[0], ndc[1], ndc[2], key)
mycursor.execute(sql)
mydb.commit()
You are building a string with out the proper single quotes that mysql needs t identify strings, so it takes them as colum nnames, but you should use the prepared statements that you already have
sql = "UPDATE ppl SET name = %s, dob = %s, city = %s WHERE key_field = %s"
mycursor.execute(sql,(ndc[0], ndc[1], ndc[2], key))
mydb.commit()
I have a large table users(id, inserttime, ...), with index only on id. I would like to find list of users who were inserted between a given start_date and finish_date range.
User.where(inserttime: start_date..finish_date).find_each
^This leads to a search which takes a lot of time, since the inserttime column is not indexed.
The solution which I came up with is to do find user.id for start_date and finish_date separately by doing a binary search twice on the table using the indexed id column.
Then do this to get all the users between start_id and finish_id:
User.where(id: start_id..finish_id).find_each
The binary search function I am using is something like this:
def find_user_id_by_date(date)
low = User.select(:id, :inserttime).first
high = User.select(:id, :inserttime).last
low_id = low.id
high_id = high.id
low_date = low.inserttime
high_date = high.inserttime
while(low_id <= high_id)
mid_id = low_id + ((high_id - low_id) / 2);
mid = User.select(:id, :inserttime).find_by(id: mid_id)
# sometimes there can be missing users. Ex: [1,2,8,9,10,16,17,..]
while mid.nil?
mid_id = mid_id + 1
mid = User.select(:id, :inserttime).find_by(id: mid_id)
end
if (mid.inserttime < date)
low_id = mid.id + 1
elsif (mid.inserttime > date)
high_id = mid.id - 1
else
return mid.id
end
end
# when date = start_date
return (low_id < high_id) ? low_id + 1 : high_id + 1
# when date = finish_date
return (low_id < high_id) ? low_id : high_id + 1
end
I am not sure if what I am doing is the right way to deal with this problem or even if my binary search function covers all the cases.
I think the best solution would be to add an index on inserttime column but that is sadly not possible.
This might not be the best way to do it, but if the IDs are numeric and sequential you could write a query to find the users in between the minimum and maximum user ID:
SELECT id
FROM users
WHERE id BETWEEN [low_id_here] AND [high_id_here];
In ActiveRecord:
low = User.select(:id, :inserttime).first
high = User.select(:id, :inserttime).last
low_id = low.id
high_id = high.id
User.where('id BETWEEN ? AND ?', low_id, high_id)
I am revisiting python and web development. i used Django in the past but it's been a while. Flask + SqlAlchemy is all new to me but I like the control it gives me.
To start off; the code below works like a charm on my dev server. Still I have the feeling that it is not as small and efficient as it could be. I was wondering if anyone has built a similar solution. For now I am trying to find a way to use a single query and format the keyword arguments into it. Further more I think it might me useful to build a class around the function in order to make it more reuse-able.
Here is the function to construct a query based on date:
def live_post_filter(year=None, month=None, day=None):
""" Query to filter only published Posts exluding drafts
Takes additional arguments to filter by year, month and day
"""
live = Post.query.filter(Post.status == Post.LIVE_STATUS).order_by(Post.pub_date.desc())
if year and month and day:
queryset = live.filter(extract('year', Post.pub_date) == year,
extract('month', Post.pub_date) == month,
extract('day', Post.pub_date) == day).all()
elif year and month:
queryset = live.filter(extract('year', Post.pub_date) == year,
extract('month', Post.pub_date) == month).all()
elif year:
queryset = live.filter(extract('year', Post.pub_date) == year).all()
else:
queryset = live.all()
return queryset
Here is how I call above function from a view:
#mod.route('/api/get_posts/', methods = ['GET'])
#mod.route('/api/get_posts/<year>/<month>/<day>/', methods = ['GET'])
#mod.route('/api/get_posts/<year>/<month>/', methods = ['GET'])
#mod.route('/api/get_posts/<year>/', methods = ['GET'])
def get_posts(year=None, month=None, day=None):
posts = live_post_filter(year=year, month=month, day=day)
postlist = []
if request.method == 'GET':
# do stuff
As stated above, all of this feels quite clunky, any advise would me much appreciated.
The use of extract to filter by date components seems odd to me. I would instead create an auxiliary function that returns a range of dates from your year, month and day arguments:
def get_date_range(year=None, month=None, day=None):
from_date = None
to_date = None
if year and month and day:
from_date = datetime(year, month, day)
to_date = from_date
elif year and month:
from_date = datetime(year, month, 1)
month += 1
if month > 12:
month = 1
year += 1
to_date = datetime(year, month, 1)
elif year:
from_date = datetime(year, 1, 1)
to_date = datetime(year + 1, 1, 1)
return from_date, to_date
And then the query function becomes much simpler:
def live_post_filter(year=None, month=None, day=None):
""" Query to filter only published Posts exluding drafts
Takes additional arguments to filter by year, month and day
"""
live = Post.query.filter(Post.status == Post.LIVE_STATUS).order_by(Post.pub_date.desc())
from_date, to_date = get_date_range(year, month, day)
if from_date and to_date:
live = live.filter(Post.pub_date >= from_date, Post.pub_date < to_date)
return live.all()
I have a table named passive than contains a list of timestamped events per user. I want to fill the attribute duration, which correspond to the time between the current row's event and the next event done by this user.
I tried the following query:
UPDATE passive as passive1
SET passive1.duration = (
SELECT min(UNIX_TIMESTAMP(passive2.event_time) - UNIX_TIMESTAMP(passive1.event_time) )
FROM passive as passive2
WHERE passive1.user_id = passive2.user_id
AND UNIX_TIMESTAMP(passive2.event_time) - UNIX_TIMESTAMP(passive1.event_time) > 0
);
This returns the error message Error 1093 - You can't specify target table for update in FROM.
In order to circumvent this limitation, I tried to follow the structure given in https://stackoverflow.com/a/45498/395857, which uses a nested subquery in the FROM clause to create an implicit temporary table, so that it doesn't count as the same table we're updating:
UPDATE passive
SET passive.duration = (
SELECT *
FROM (SELECT min(UNIX_TIMESTAMP(passive2.event_time) - UNIX_TIMESTAMP(passive.event_time))
FROM passive, passive as passive2
WHERE passive.user_id = passive2.user_id
AND UNIX_TIMESTAMP(passive2.event_time) - UNIX_TIMESTAMP(passive1.event_time) > 0
)
AS X
);
However, the passive table in the nested subquery doesn't refer to the same passive as in the main query. Because of that, all rows have the same passive.duration value. How can I refer to the main query's passive in the nested subquery? (or maybe are there some alternative ways to structure such a query?)
Try Like this....
UPDATE passive as passive1
SET passive1.duration = (
SELECT min(UNIX_TIMESTAMP(passive2.event_time) - UNIX_TIMESTAMP(passive1.event_time) )
FROM (SELECT * from passive) Passive2
WHERE passive1.user_id = passive2.user_id
AND UNIX_TIMESTAMP(passive2.event_time) - UNIX_TIMESTAMP(passive1.event_time) > 0
)
;
We can use a Python script to circumvent the issue:
'''
We need an index on user_id, timestamp to speed up
'''
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Download it at http://sourceforge.net/projects/mysql-python/?source=dlp
# Tutorials: http://mysql-python.sourceforge.net/MySQLdb.html
# http://zetcode.com/db/mysqlpython/
import MySQLdb as mdb
import datetime, random
def main():
start = datetime.datetime.now()
db=MySQLdb.connect(user="root",passwd="password",db="db_name")
db2=MySQLdb.connect(user="root",passwd="password",db="db_name")
cursor = db.cursor()
cursor2 = db2.cursor()
cursor.execute("SELECT observed_event_id, user_id, observed_event_timestamp FROM observed_events ORDER BY observed_event_timestamp ASC")
count = 0
for row in cursor:
count += 1
timestamp = row[2]
user_id = row[1]
primary_key = row[0]
sql = 'SELECT observed_event_timestamp FROM observed_events WHERE observed_event_timestamp > "%s" AND user_id = "%s" ORDER BY observed_event_timestamp ASC LIMIT 1' % (timestamp, user_id)
cursor2.execute(sql)
duration = 0
for row2 in cursor2:
duration = (row2[0] - timestamp).total_seconds()
if (duration > (60*60)):
duration = 0
break
cursor2.execute("UPDATE observed_events SET observed_event_duration=%s WHERE observed_event_id = %s" % (duration, primary_key))
if count % 1000 == 0:
db2.commit()
print "Percent done: " + str(float(count) / cursor.rowcount * 100) + "%" + " in " + str((datetime.datetime.now() - start).total_seconds()) + " seconds."
db.close()
db2.close()
diff = (datetime.datetime.now() - start).total_seconds()
print 'finished in %s seconds' % diff
if __name__ == "__main__":
main()