Does anybody know how to either get a bit more detailed error message wrt MySql and regular expressions (i.e. the character position where the problem occurs) or if there is a tool that you can use to try out regular expressions without the need to trial and error with the database.
RegexBuddy can do this (and it's one of the few tools you can use to debug regexes in the MySQL-specific syntax).
Related
I'm accessing a Microsoft Access database using ODBC.
According to the w3schools SQL tutorial, the ANSI-92 wildcard for "any character not in brackets" should be ^. However, their own example uses the ANSI-89 wildcard !.
The MSDN documentation is also confusing. The ANSI-92 section shows ^ character, but the example next to it actually uses !. That looks like an error and I've filled out feedback to notify Microsoft.
What is the correct "any character not in brackets" syntax? And will this apply across multiple databases and access technologies (DAO, ODBC, OleDB, MySQL, etc)?
There is no way to do this across databases. SQL supports the LIKE syntax that has exactly two wildcards:
% matches zero or more characters.
_ matches exactly one character.
One could throw in the \ character to escape wildcards as well.
In addition, SQL Server supports character classes as explained in the tutorial (which seems to be accurate). The only other database that comes to mind that does this is Sybase (which has the same code base). MS Access just has a bastardized version with special characters. It never supported SQL standards in this respect.
Most other databases implement full regular expression support (YAY!), but their syntax varies by database. So, there is not a database-independent way to do what you want.
According to Node.js' mysql package documentation:
In order to avoid SQL Injection attacks, you should always escape any user provided data before using it inside a SQL query. You can do so using the mysql.escape(), connection.escape() or pool.escape() methods.
I cannot find any documentation / reference to using prepared statements with mysql, except for in a reference to using '?' characters. It states the following:
Alternatively, you can use ? characters as placeholders for values you would like to have escaped...
This looks similar to prepared statements in MySQL, however it really
just uses the same connection.escape() method internally.
From my experience with talking to other developers, the general consensus in the developer community is that prepared statements are the ONLY safe way to perform mysql queries from Node.js however, as you can see with the mysql package, there is no obvious support for prepared statements. However, it is indicated that their method for avoiding SQL injection is via the usage of the escape functions.
My Question:
Are the escape functions in the mysql package sufficient enough to securely query a mysql database (without the use of prepared statements)?
Escaping is actually just as safe as using parameterized queries, if you do it consistently.
But it's recommended to use parameters because it makes your code simpler. Therefore developers are probably more likely to do it.
If the Node.js library makes it just as convenient as parameters, but implements it internally by modifying query strings, replacing the ? marks with the parameter values, then you're good.
For what it's worth, Python's MySQL driver does this too, and also PHP's PDO extension when the "emulate prepares" option is in effect. As long as those libraries are well-tested, it should be safe.
FWIW, both escaping and parameters is limited in SQL injection prevention, because in both cases, you can only protect values that you would combine with your SQL query. You cannot protect identifiers (like table names, or column names), or SQL keywords, or expressions, etc. In these cases, just be careful that you have vetted any dynamic content before combining it with your SQL query.
See also:
Preventing SQL injection in Node.js
Difference between real_escape_string and prepare()?
As far as I can gather, there are commands which are native to SQL such as SELECT and DELETE, and there are commands which are part of MySQL but not native to SQL such as use and describe. Have I got that right?
In this link the difference seems to be implied by having the MySQL commands in lower case. Is there a resource which shows which commands belong to which group i.e. native SQL vs MySQL-specific?
SQL is a language standard. Defined by organisations like ISO, ANSI, DIN.
Each SQL database system provides SQL, but with different scope and syntax. So you can learn the standard, but than you have to look for the distinctions. Or you directly learn a syntax of a specific database system. It depends on your purposes.
By the way the commands themself are case-insensitive. So it is indifferent if you write "select" or "SELECT". I prefer lower case because it is easy to write. Other prefer upper case because it is easy to read. In many projects the convention is upper case.
I have used backticks (`) in some SELECT queries to escape fields such as 'first-name'. This will work on MySQL. These queries are run through a DBO class in a php application and I would like the application to be able to use other database servers, such as MSSQL and Posgres.
What is the best approach for allowing problematic field names to be used across all of these database servers? I was thinking of taking the fields as an array and quoting them with the escaping character that is appropriate to each.
[EDIT]
To clarify: I am building a tool that will be used to map configurations stored within the php application to the fields of an external database. I wanted to escape these as a precaution because I have no idea what field names will actually be mapped to and used within the queries.
The solution is very simple: do not use reserved words as identifiers. It makes the code more difficult to read anyways.
If you really need to use such words (as in "there is some obscure reason beyond your control"), you can just prefix all your identifiers by an arbitrary character, such as _ for example.
The cross-DBMS mechanism (as defined in SQL-92 and other standards) is using double-quoted delimited identifiers. According to this reference it's widely supported.
It's worth nothing that MySQL allows to enable/disable this syntax so you still need to ensure that session settings are correct before issuing any query.
MySQL uses backticks (`) by default, but can be configured to support proper ANSI quoting.
IMO: If you're connecting to MySQL, set it to ANSI mode, and while you're at it enable every STRICT option it has. It becomes much easier to write code that's portable against it then.
Of course, the best option has to be not using reserved words, but the list of reserved words can change over time so rigorous quoting isn't such a bad idea.
The proper way of escaping is not to use field names that need escaping.
If you still have to use escaping - use ". It is the standard one (defined by ANSI SQL).
Postgres and Oracle understand " escaping. But i do not know about MSSQL an MySQL.
I have an application which would greatly benefit by using dynamic mysql queries in combination with mysql (mysqli) real escape string. If I ran all data received from the user through mysql real escape would it be just as secure as using mysql prepared statements?
Yes, but a qualified yes.
You need to properly escape 100% of the input. And you need to properly set character sets (If you're using the C API, you need to call the mysql_set_character_set() instead of SET NAMES). If you miss one tiny thing, you're vulnerable. So it's yes, as long as you do everything right...
And that's the reason a lot of people will recommend prepared queries. Not because they are any safer. But because they are more forgiving...
Definitely NO.
While question in the title is ambiguous and can be interpreted as "Are dynamic mysql queries with every it's part properly formatted..." and thus have a positive answer, the question in the body is not:
If I ran all data received from the user through mysql real escape would it be just as secure as using mysql prepared statements?
If you look to this question closer, you will understand that this is just a magic quotes incarnation! The very purpose of this disgraced, deprecated and removed feature is exactly to "run all user input through escape".
Everyone knows nowadays that magic quotes are bad. Why positive answer then?
Okay, it seems that it needs to be explained again, why bulk escaping is bad.
The root of the problem is a quite strong delusion, shared by almost every PHP user:
Everyone have a strange belief that escaping do something on "dangerous characters" (what are they?) making them "safe" (how?). Needless to say that it's but a complete rubbish.
The truth is:
Escaping do not "sanitize" anything.
Escaping has nothing to do with injections.
Escaping has nothing to do with user input.
Escaping is merely a string formatting and nothing else.
When you need it - you need it despite of injection possibility.
When you don't need it - it won't help against injection even a little.
Speaking of difference with prepared statements, there is at least one issue (which already mentioned many times under sql-injection tag):
a code like this
$clean = mysql_real_escape_string($_POST['some_dangerous_variable']);
$query = "SELECT * FROM someTable WHERE somevalue = $clean";
will help you NOT against injection.
Beause escaping is just a string formatting facility, not injection preventer by any means.
Go figure.
However, escaping have something in common with prepared statements:
Them both doesn't guarantee you from injection if
you are using it only against notorious "user input", not as a strict rule for the building ANY query, despite of data source.
in case you need to insert not data but identifier or a keyword.
To be safe in these circumstances, see my answer explaining FULL sql injection protection how-to
Long story short: you can consider yourself safe only if you make 2 essential corrections and one addition to your initial statement:
If I ran all data received from the user through mysql real escape and always enclose it in quotes (and, as ircmaxell mentioned, mysqli_set_charset() is used to make mysqli_real_escape string() actually do it's work (in such a rare occasion of using some odd encoding like GBK)) would it be just as secure as using mysql prepared statements?
Following these rules - yes, it would be as secure as native prepared statements.
I think #ircmaxell got it right on.
As a follow-up, be on the lookout for this kind of thing.
I used to do it all the time:
<?php
//sanitize the dangerous posted variable...
$clean = mysql_real_escape_string($_POST['some_dangerous_variable']);
//...and then forget to use it!
$query = "SELECT * FROM someTable WHERE somevalue = '{$_POST['some_dangerous_variable']}'";
?>
And when I say "used to do it", what I mean is that I eventually gave up and just started using prepared statements!