Django-like date based archive with Flask and SqlAlchemy - sqlalchemy

I am revisiting python and web development. i used Django in the past but it's been a while. Flask + SqlAlchemy is all new to me but I like the control it gives me.
To start off; the code below works like a charm on my dev server. Still I have the feeling that it is not as small and efficient as it could be. I was wondering if anyone has built a similar solution. For now I am trying to find a way to use a single query and format the keyword arguments into it. Further more I think it might me useful to build a class around the function in order to make it more reuse-able.
Here is the function to construct a query based on date:
def live_post_filter(year=None, month=None, day=None):
""" Query to filter only published Posts exluding drafts
Takes additional arguments to filter by year, month and day
"""
live = Post.query.filter(Post.status == Post.LIVE_STATUS).order_by(Post.pub_date.desc())
if year and month and day:
queryset = live.filter(extract('year', Post.pub_date) == year,
extract('month', Post.pub_date) == month,
extract('day', Post.pub_date) == day).all()
elif year and month:
queryset = live.filter(extract('year', Post.pub_date) == year,
extract('month', Post.pub_date) == month).all()
elif year:
queryset = live.filter(extract('year', Post.pub_date) == year).all()
else:
queryset = live.all()
return queryset
Here is how I call above function from a view:
#mod.route('/api/get_posts/', methods = ['GET'])
#mod.route('/api/get_posts/<year>/<month>/<day>/', methods = ['GET'])
#mod.route('/api/get_posts/<year>/<month>/', methods = ['GET'])
#mod.route('/api/get_posts/<year>/', methods = ['GET'])
def get_posts(year=None, month=None, day=None):
posts = live_post_filter(year=year, month=month, day=day)
postlist = []
if request.method == 'GET':
# do stuff
As stated above, all of this feels quite clunky, any advise would me much appreciated.

The use of extract to filter by date components seems odd to me. I would instead create an auxiliary function that returns a range of dates from your year, month and day arguments:
def get_date_range(year=None, month=None, day=None):
from_date = None
to_date = None
if year and month and day:
from_date = datetime(year, month, day)
to_date = from_date
elif year and month:
from_date = datetime(year, month, 1)
month += 1
if month > 12:
month = 1
year += 1
to_date = datetime(year, month, 1)
elif year:
from_date = datetime(year, 1, 1)
to_date = datetime(year + 1, 1, 1)
return from_date, to_date
And then the query function becomes much simpler:
def live_post_filter(year=None, month=None, day=None):
""" Query to filter only published Posts exluding drafts
Takes additional arguments to filter by year, month and day
"""
live = Post.query.filter(Post.status == Post.LIVE_STATUS).order_by(Post.pub_date.desc())
from_date, to_date = get_date_range(year, month, day)
if from_date and to_date:
live = live.filter(Post.pub_date >= from_date, Post.pub_date < to_date)
return live.all()

Related

Retrieving date from mySQL in a datetime or timestamp format

I am trying to retrieve dates from a date column in mySQL database. My code (mysqljs) :
today = document.getElementBYId("myDate").value
The result :
today = Thu Dec 23 2021 14:05:00 GMT+0800 (Singapore Standard Time)
How can I get it to return just the date and time (yyy:mm:dd hh:mm:ss)?
(In mySql - the column type of the field date is set to DATETIME.)
Finally I have fund a way to solve this problem.
First I capture the date into a variable :
`var today = document.getElementById("mydate").value;`
(Today is in the format :
Thu Dec 23 2021 14:05:00 GMT+0800 (Singapore Standard Time)
Then I split it (to remove those info that I do not want):
var t = today.split(" ");
var d = t[0]+" "+t[1]+" "+t[2]+","+t[3]
Next I create a function to format the date :
function formateDate(date){
var e = new Date(date),
month = '' + (e.getMonth() + 1),
day = '' + e.getDate(),
year = e.getFullYear();
if (month.length <2)
month = '0' + month;
if (day.length <2)
day = '0' + day;
return [year, month, day].join('-');
}
Then I pass today into the function :
`today_x = formatDate(d)`
Finally I display it to the form That I am rendering :
document.getElementById("mydate").value = today_x;
I hope this is helpful. I don't think this is the most efficient way but I have been struggling with this for almost a week and this is the only solution I can come up with. I hope someone can offer another solution that is less tedious.

How to compare date object in mysql against two string dates? [duplicate]

This question already has answers here:
How to convert a string to date in MySQL?
(5 answers)
Closed 2 years ago.
I am trying to compare the following:
SELECT g_m.user_id
, g_m.group_id
FROM Group_Members g_m
WHERE g_m.gender_id = 2
AND g_m.partner_gender_id = 1
AND g_m.birthday >= '01-01-1955'
AND g_m.birthday <= '12-31-2002'
AND g_m.user_id != 12
g_m.birthday in this case is '02-15-1998' which should show up, but this returns an empty array, because the date comparison does not seem to be accurate?
Here is the entire function and the dates are being passed from age minimum and age maximums brought from user.
var today = new Date();
var minYear = "01-01-" + (today.getFullYear() - userPref.age_max); //min year to start but oldest age
var maxYear = "12-31-" + (today.getFullYear() - userPref.age_min); //max year to end but youngest age
var qSelect = "SELECT g_m.user_id, g_m.group_id" +
" FROM Group_Members g_m WHERE g_m.gender_id = ? AND g_m.partner_gender_id = ?" +
" AND g_m.birthday >= STR_TO_DATE(?, '%m/%d/%Y') AND g_m.birthday <= STR_TO_DATE(?, '%m/%d/%Y')" +
" AND g_m.user_id != ?";
var qValues = [userPref.partner_gender_id, userObj.gender_id, minYear, maxYear, userObj.id];
Anyone know how to compare dates in a mysql query?
What is the data type of your dates ? You should declare them as "date", otherwise you won't be able to compare them.
With strings, '02-15-1998' < '03-15-1990'
With dates, your mysql request should be :
SELECT g_m.user_id, g_m.group_id FROM Group_Members g_m WHERE g_m.gender_id = 2 AND g_m.partner_gender_id = 1 AND g_m.birthday >= '1955-01-01' AND g_m.birthday <= '2002-12-31' AND g_m.user_id != 12
Sorry for my english, I'm french...
As the comments have already pointed out, you appear to be storing your dates as actual text. For a short term workaround, you may use STR_TO_DATE to convert your text dates to bona fide MySQL dates. Then, compare them against valid MySQL date literals:
SELECT
g_m.user_id,
g_m.group_id
FROM Group_Members g_m
WHERE
g_m.gender_id = 2 AND
g_m.partner_gender_id = 1 AND
STR_TO_DATE(g_m.birthday, '%m-%d-%Y') >= '1955-01-01' AND
STR_TO_DATE(g_m.birthday, '%m-%d-%Y') < '2003-01-01' AND
g_m.user_id != 12;
Longer term, you should make the birthday column datetime or timestamp.
Side note: I have rewritten the date range to include those born in the calendar years from 1955 to 2002, inclusive on both ends.

Is it okay to do binary search on an indexed column to get data from a non-indexed column?

I have a large table users(id, inserttime, ...), with index only on id. I would like to find list of users who were inserted between a given start_date and finish_date range.
User.where(inserttime: start_date..finish_date).find_each
^This leads to a search which takes a lot of time, since the inserttime column is not indexed.
The solution which I came up with is to do find user.id for start_date and finish_date separately by doing a binary search twice on the table using the indexed id column.
Then do this to get all the users between start_id and finish_id:
User.where(id: start_id..finish_id).find_each
The binary search function I am using is something like this:
def find_user_id_by_date(date)
low = User.select(:id, :inserttime).first
high = User.select(:id, :inserttime).last
low_id = low.id
high_id = high.id
low_date = low.inserttime
high_date = high.inserttime
while(low_id <= high_id)
mid_id = low_id + ((high_id - low_id) / 2);
mid = User.select(:id, :inserttime).find_by(id: mid_id)
# sometimes there can be missing users. Ex: [1,2,8,9,10,16,17,..]
while mid.nil?
mid_id = mid_id + 1
mid = User.select(:id, :inserttime).find_by(id: mid_id)
end
if (mid.inserttime < date)
low_id = mid.id + 1
elsif (mid.inserttime > date)
high_id = mid.id - 1
else
return mid.id
end
end
# when date = start_date
return (low_id < high_id) ? low_id + 1 : high_id + 1
# when date = finish_date
return (low_id < high_id) ? low_id : high_id + 1
end
I am not sure if what I am doing is the right way to deal with this problem or even if my binary search function covers all the cases.
I think the best solution would be to add an index on inserttime column but that is sadly not possible.
This might not be the best way to do it, but if the IDs are numeric and sequential you could write a query to find the users in between the minimum and maximum user ID:
SELECT id
FROM users
WHERE id BETWEEN [low_id_here] AND [high_id_here];
In ActiveRecord:
low = User.select(:id, :inserttime).first
high = User.select(:id, :inserttime).last
low_id = low.id
high_id = high.id
User.where('id BETWEEN ? AND ?', low_id, high_id)

MySql to Python Date

I try to convert MySQl datetime to same in Python.On debug there is
ValueError: time data '2001-06-04T11:30:35' doesn't match format %Y-%m-%dT%H:%M:%S .
In MySQL there is no 'T' in data.I tried format with 'T' and without.
I saw this article How to convert the following string to python date? .
This is code:
query = QSqlQuery ()
query.exec_("SELECT birthday FROM vista.user ")
def countAge(birthday):
birthday = datetime.strptime(str(birthday), "%Y-%m-%dT%H:%M:%S.%f")
today = date.today()
age = today.year - birthday.year
if today.month < birthday.month:
age -= 1
elif today.month == birthday.month and today.day < birthday.day:
age -= 1
if age >= 0 :
return age
ages = []
index = 0
while (query.next()):
print(query.value(index).toString())
ages.append(countAge(query.value(index).toString()))
index = index + 1
What is a problem?
If an example date-string is 2001-06-04T11:30:35, then you need:
birthday = datetime.strptime(str(birthday), "%Y-%m-%dT%H:%M:%S")

How to set filter to get children in certain time period by eagerload_all() at SqlAlchemy

I have a table posts and it stores 3 types of post, Topic, Reply and Comment. Each one has its parent id.
# Single table inheritance
class Post(Base):
__tablename__ = 'posts'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('posts.id'))
discriminator = Column(String(1))
content = Column(UnicodeText)
added_at = Column(DateTime)
__mapper_args__ = {'polymorphic_on': discriminator}
class Topic(Post):
replies = relation("Reply")
__mapper_args__ = {'polymorphic_identity': 't'}
class Reply(Post):
comments = relation("Comment")
__mapper_args__ = {'polymorphic_identity': 'r'}
class Comment(Post):
__mapper_args__ = {'polymorphic_identity': 'c'}
And I'm using eagerload_all() to get all the replies and comments belong to one topic:
session.query(Topic).options(eagerload_all('replies.comments')).get(topic_id)
My question is, if I want to get only replies and those replies' comments in certain time period, for example, this week, or this month. How should I use filter to achieve this?
Thank you
The use of eagerload_all will only query for the children of an object Topic immediately rather on first request to the Replies and/or Comments, but since you load the Topic object into the session, all its related children will be loaded as well. This gives you the first option:
Option-1: Filter in the python code instead of database:
Basically create a method on the Topic object similar to
class Topic(Post):
...
def filter_replies(self, from_date, to_date):
return [r for r in self.replies
if r.added_at >= from_date
and r.added_at <= to_date]
Then you can do similar code on Replies to filter Comments or any combination of those. You get the idea.
Option-2: Filter on the database level:
In order to achieve this you need not load the Topic object, but filter directly on the Reply/Comment. Following query returns all Reply for a given Topic with a date filter:
topic_id = 1
from_date = date(2010, 9, 5)
to_date = date(2010, 9, 15)
q = session.query(Reply)
q = q.filter(Reply.parent_id == topic_id)
q = q.filter(Reply.added_at >= from_date)
q = q.filter(Reply.added_at <= to_date)
for r in q.all():
print "Reply: ", r
The version for the Comment is just a little bit more involved as you require an alias in order to overcome the SQL statement generation issue as all your objects are mapped to the same table name:
topic_id = 1
from_date = date(2010, 9, 5)
to_date = date(2010, 9, 15)
ralias = aliased(Reply)
q = session.query(Comment)
q = q.join((ralias, Comment.parent_id == ralias.id))
q = q.filter(ralias.parent_id == topic_id)
q = q.filter(Comment.added_at >= from_date)
q = q.filter(Comment.added_at <= to_date)
for c in q:
print "Comment: ", c
Obviously you can create a function that would combine both peaces into a more comprehensive query.
In order to achieve this week or this month type of queries you can either convert these filter into a date range as shown above or use the expression.func functionality of SA.