Is it possible to construct raw SQL queries in Django so that they accept a dynamic number of arguments? So for example say that I have the following url structure in my app:
/books/category/history/
/books/category/history/1800s/
For the first query, I'm looking for all books with the keyword 'history', and for the second, I'm looking for all books with the keyword 'history' AND the keyword '1800s'.
I currently have two separate queries for each of these:
keyword1 = 'history'
SELECT appname_book.name AS name FROM appname_book WHERE keyword=%s,[keyword1]
keyword1 = 'history'
keyword2 = '1800s'
SELECT appname_book.name AS name FROM appname_book WHERE keyword=%s AND keyword=%s,[keyword1, keyword2]
Anyone know of a cleaner and more efficient way to do this?
I'm using Django 1.3 and MySQL.
Thanks.
Why dont you use Django QuerySet, like this:
Book.objects.all().filter(keyword__in=['history','1800s']).values('name')
Another possible solution using RAW SQL, coud be:
keywords = []
SQL = 'SELECT appname_book.name AS name FROM appname_book WHERE 1=1 '
SQL += ' '.join(['AND keyword=%s' for _ in params])
Sure, you could do something like this to dynamically generate a raw SQL query
sql = 'SELECT id FROM table WHERE 1 = 1'
params = []
if 'description' in args.keys():
sql += ' AND description LIKE %s'
params.append('%'+args['description']+'%')
if 'is_active' in args.keys():
sql += ' AND is_active LIKE %s'
params.append(args['is_active'])
... you can put as many "ifs" you want to construct the query
with connections['default'].cursor() as cursor:
cursor.execute(sql, params)
This way would still be completely safe against SQL Injections vulnerability
Related
I am new to python and currently learning to use SQL with python. I have the following code:
word = input("Enter a word: ")
query = cursor.execute("SELECT * FROM Dictionary WHERE Expression LIKE '%s%' " % word)
results = cursor.fetchall()
The second line throws an error since I don't think I can use '%s%' like that? How would I change this so as to be able to make this work? I want to be able to return all related entries to the users input. So if the user inputs "rain", then I want the query to return all possible results e.g. "raining", "rainy" etc. Thank you.
You can try
query = cursor.execute(f"SELECT * FROM Dictionary WHERE Expression LIKE '%{word}%' ")
You should use cursor.execute() parameter substitution rather than string formatting, to prevent SQL injection.
Then use CONCAT() to surround the search string with %.
query = cursor.execute("SELECT * FROM Dictionary WHERE Expression LIKE CONCAT('%', %s, '%' "), (word,))
I am new in python as well as mysql. I am having trouble in populating proper query statement for mysql.
sql = "SELECT * FROM Persons WHERE %s"
cur = db.cursor()
cur.execute(sql,(where,))
where is a string variable which creates a string for WHERE clause; this is the point of question. When I print this variable it give the following result:
Gender = True And IsLate = False
(without any quotes) but when I add this variable to the query to execute it, it adds single quotes around the string.
I used the command
print(cur.statement)
and it prints:
SELECT * FROM Persons WHERE 'Gender = True And IsLate = False'
After supplying parameter, it puts it within single quotes and query returns 0 rows.
I have worked around by concatenating the query statement and variable together and execute the string as query, that worked,
sql = sql + where
cur.execute(sql)
But I know that is not the professional way, as I have searched and found the professional way is to use parameterized query and use variable to store the condition(s) and supplying it at the execution of query.
Looking for advice, am I thinking the right way or otherwise?
The whole point of using parameter substitution in cursor.execute() is that it protects you from SQL injection. Each parameter is treated as a literal value, not substituted into the query and re-interpreted.
If you really want it to be interprted, you need to use string formatting or concatenation, as you discovered. But then you will have to be very careful in validating the input, because the user can supply extra SQL code that you may not have expected, and cause the query to malfunction.
What you should do is build the where string and parameter list dynamically.
where = []
params = []
if gender_supplied:
where.append('gender = %s')
params.append(gender)
if islate_supplied:
where.append*('islate = %s')
params.append(islate)
sql = 'select * from persons'
if where:
query = sql + ' where ' + ' and '.join(where)
else:
query = sql
cur.execute(query, params)
Very similar to this question MySQL Dynamic Query Statement in Python
However what I am looking to do instead of two lists is to use a dictionary
Let's say i have this dictionary
instance_insert = {
# sql column variable value
'instance_id' : 'instnace.id',
'customer_id' : 'customer.id',
'os' : 'instance.platform',
}
And I want to populate a mysql database with an insert statement using sql column as the sql column name and the variable name as the variable that will hold the value that is to be inserted into the mysql table.
Kind of lost because I don't understand exactly what this statement does, but was pulled from the question that I posted where he was using two lists to do what he wanted.
sql = "INSERT INTO instance_info_test VALUES (%s);" % ', '.join('?' for _ in instance_insert)
cur.execute (sql, instance_insert)
Also I would like it to be dynamic in the sense that I can add/remove columns to the dictionary
Before you post, you might want to try searching for something more specific to your question. For instance, when I Googled "python mysqldb insert dictionary", I found a good answer on the first page, at http://mail.python.org/pipermail/tutor/2010-December/080701.html. Relevant part:
Here's what I came up with when I tried to make a generalized version
of the above:
def add_row(cursor, tablename, rowdict):
# XXX tablename not sanitized
# XXX test for allowed keys is case-sensitive
# filter out keys that are not column names
cursor.execute("describe %s" % tablename)
allowed_keys = set(row[0] for row in cursor.fetchall())
keys = allowed_keys.intersection(rowdict)
if len(rowdict) > len(keys):
unknown_keys = set(rowdict) - allowed_keys
print >> sys.stderr, "skipping keys:", ", ".join(unknown_keys)
columns = ", ".join(keys)
values_template = ", ".join(["%s"] * len(keys))
sql = "insert into %s (%s) values (%s)" % (
tablename, columns, values_template)
values = tuple(rowdict[key] for key in keys)
cursor.execute(sql, values)
filename = ...
tablename = ...
db = MySQLdb.connect(...)
cursor = db.cursor()
with open(filename) as instream:
row = json.load(instream)
add_row(cursor, tablename, row)
Peter
If you know your inputs will always be valid (table name is valid, columns are present in the table), and you're not importing from a JSON file as the example is, you can simplify this function. But it'll accomplish what you want to accomplish. While it may initially seem like DictCursor would be helpful, it looks like DictCursor is useful for returning a dictionary of values, but it can't execute from a dict.
My database (mysql) tables use TIMESTAMP columns, and whenever I want them returned in a query, I want them to be queried as "UNIX_TIMESTAMP(columnname)".
How do you easily modify queries in zend framework to achieve this?
For example, the current code is:
select = $this->select();
$select->where('user_id = ?',$user_id);
return $this->fetchAll($select);
This eventually becomes:
select * from tablename where user_id = 42;
I want something that automatically finds the TIMESTAMP column and changes the resulting query to:
select user_id,name,unix_timestamp(created) where user_id = 42;
I know I can use a MySQL view to achieve this, but I'd rather avoid that.
Thanks.
RR
You should be able to specify the fields you want in the select using the $select->from() object.
Zend_Db_Select
You should end up with something like this.
$select = $this->select();
$select->from(
array('t' => 'tablename'),
array('user_id', 'name', 'UNIX_TIMESTAMP(created)')
);
$select->where('user_id = ?',$user_id);
return $this->fetchAll($select);
If you wanted to run an expression that doesn't have parenthese in the function, Use the Zend_Db_Expr() method to escape the query properly.
So I use the PDO for a DB connection like this:
$this->dsn[$key] = array('mysql:host=' . $creds['SRVR'] . ';dbname=' . $db, $creds['USER'], $creds['PWD']);
$this->db[$key] = new PDO($this->dsn[$key]);
Using PDO I can then execute a MySQL SELECT using something like this:
$sql = "SELECT * FROM table WHERE id = ?";
$st = $db->prepare($sql);
$st->execute($id);
$result = $st->fetchAll();
The $result variable will then return an array of arrays where each row is given a incremental key - the first row having the array key 0. And then that data will have an array the DB data like this:
$result (array(2)
[0]=>[0=>1, "id"=>1, 1=>"stuff", "field1"=>"stuff", 2=>"more stuff", "field2"=>"more stuff" ...],
[1]=>[0=>2, "id"=>2, 1=>"yet more stuff", "field1"=>"yet more stuff", 2=>"even more stuff", "field2"=>"even more stuff"]);
In this example the DB table's field names would be id, field1 and field2. And the result allows you to spin through the array of data rows and then access the data using either a index (0, 1, 2) or the field name ("id", "field1", "field2"). Most of the time I prefer to access the data via the field names but access via both means is useful.
So I'm learning the ruby-mysql gem right now and I can retrieve the data from the DB. However, I cannot get the field names. I could probably extract it from the SQL statement given but that requires a fair bit of coding for error trapping and only works so long as I'm not using SELECT * FROM ... as my SELECT statement.
So I'm using a table full of State names and their abbreviations for my testing. When I use "SELECT State, Abbr FROM states" with the following code
st = #db.prepare(sql)
if empty(where)
st.execute()
else
st.execute(where)
end
rows = []
while row = st.fetch do
rows << row
end
st.close
return rows
I get a result like this:
[["Alabama", "AL"], ["Alaska", "AK"], ...]
And I'm wanting a result like this:
[[0=>"Alabama", "State"=>"Alabama", 1=>"AL", "Abbr"=>"AL"], ...]
I'm guessing I don't have the way inspect would display it quite right but I'm hoping you get the idea by now.
Anyway to do this? I've seen some reference to doing this type of thing but it appears to require the DBI module. I guess that isn't the end of the world but is that the only way? Or can I do it with ruby-mysql alone?
I've been digging into all the methods I can find without success. Hopefully you guys can help.
Thanks
Gabe
You can do this yourself without too much effort:
expanded_rows = rows.map do |r|
{ 0 => r[0], 'State' => r[0], 1 => r[1], 'Abbr' => r[1] }
end
Or a more general approach that you could wrap up in a method:
columns = ['State', 'Abbr']
expanded_rows = rows.map do |r|
0.upto(names.length - 1).each_with_object({}) do |i, h|
h[names[i]] = h[i] = r[i]
end
end
So you could collect up the rows as you are now and then pump that array of arrays through something like what's above and you should get the sort of data structure you're looking for out the other side.
There are other methods on the row you get from st.fetch as well:
http://rubydoc.info/gems/mysql/2.8.1/Mysql/Result
But you'll have to experiment a little to see what exactly they return as the documentation is, um, a little thin.
You should be able to get the column names out of row or st:
http://rubydoc.info/gems/mysql/2.8.1/Mysql/Stmt
but again, you'll have to experiment to figure out the API. Sorry, I don't have anything set up to play around with the MySQL API that you're using so I can't be more specific.
I realize that php programmers are all cowboys who think using a db layer is cheating, but you should really consider activerecord.