Following on from this question MySQL database contains quotes encoded and unencoded and it's breaking javascript
I am executing this MySQL query:
DELETE FROM `example` WHERE `name` = ''12345''
However it fails because the value in the database is '12345'. It seems that old data in the database has a mixture of encoded and unencoded quotes. Is it safe to to update all ' to ' in the database?
In most cases (yours included), store text without any "encoding". That is, do not store htmlentities, store the actual characters, do not store unicode 'codes', store the actual characters, etc.
Do likewise for anything you need to compare to what is in the database.
You will, however, have to escape strings when building SQL statements. Otherwise, you can't get quotes (in text) inside quotes (that are part of the SQL syntax.
That is, you will end up with this SQL when searching for that Irishman:
... WHERE `name` = 'O\'Brian'
Related
I was save this string to DB
{"form_5_check":"N\u00e1kladov\u00e9 stredisko"}
But in mysql db is this string:
{"form_5_check":"Nu00e1kladovu00e9 stredisko"}
Pls where are "\" lost ? Thanks a lot
MySQL treats the backslash character as an escape character. If you did something like this:
query = "INSERT INTO foo (json) VALUES ('" + json + "');
you have basically three problems:
the single backslash you have will get interpreted as an escape character, not as content; unless the next character is a quote or another backslash, it will have escaped nothing, and silently disappear.
if your json contained any single quotes, and you are lucky, you will get a syntax error, as the quote that was supposed to contain the value will be closed, and gibberish that SQL can't parse follows.
if your json contained any single quotes, and you're not lucky, you're now a victim of SQL injection attack, the most infamous example being XKCD's.
To avoid all that, make sure that your data is properly sanitised before it hits the database. This can be done in two ways:
The manual, and error-prone way includes always remembering to escape any characters that need it any time you insert a string into a query. This differs between databases. Some databases want a backslash before quotes, while some prefer doubling the quotes and doing nothing to backslashes. Some allow both. Many languages and/or database access libraries have functions that do this in a way appropriate for the database.
The automated, foolproof and very much preferred way is to use parametrised queries and prepared statements that do this for you in a transparent and easy-to-use way. You do not have a specific language tagged, so I can't give you the solution, but the Bobby Tables site has answers for many commonly used programming languages.
Is there any way to perform a SQL injection when single quotes are escaped by two single quotes? I know the MySQL server is using this specific technique to prevent against an attack. I'm trying to log in as a specific user but all of the common injections I've tried for the password have not worked successfully (i.e. ' or '1'='1, ' or ' 1=1, etc.).
No, and yes.
There's no way to have an unsafe values "breakout" of literal values that are enclosed in single quotes, if the value being supplied is "escaped" by preceding single quotes by with an additional single quote.
That is, assuming that your statement is guaranteeing that string literals are enclosed in quotes, as part of the "static" SQL text.
example perl-ish/php-ish
$sql = "... WHERE t.foo = '" . $safe_value . "' ... ";
^ ^
I've underscored here that the single quotes enclosing the literal are part of the SQL text. If $safe_value has been "escaped" by preceding each single quote in the "unsafe" value with another single value to make it "safe"...
$unsafe_value $safe_value
------------- ------------
I'm going I''m going
'she''s' ''she''''s''
1'='1 -- 1''=''1 --
As long as the escaping is handled properly, that we guarantee that potentially unsafe values are are run through the escaping, then including single quotes in data values is not a viable way to "breakout" of a literal with the SQL text.
That's the "no" part of the answer.
The "yes" part of the answer.
One of the biggest problems is making sure this is done EVERYWHERE, and that a mistake has not been made somewhere, assuming that a potentially unsafe string is "safe", and is not escaped. (For example, assuming that values pulled from a database table are "safe", and not escaping them before including them in SQL text.)
Also, the single quote trick is not the only avenue for SQL injection. The code could still be vulnerable.
Firstly, if we're not careful about other parts of the statement, like the single quotes enclosing string literals. Or, if for example, the code were to run the $sql through some other function, before it gets submitted to the database:
$sql = some_other_function($sql);
The return from some_other_function could potentially return SQL text that was in fact vulnerable. (As a ridiculous example, some_other_function might replace all occurrences of two consecutive single quotes with a single single quote. DOH!)
Also, with the vast number of possible unicode characters, if we're ever running through a characterset translation, there's also a possibility that some unicode character could get mapped to a single quote character. I don't have any specific example of that, but dollars to donuts that somewhere, in that plethora of multibyte encodings, there's some unicode character somewhere that will get translated to a single quote in some target.
There's a default character in the target for unmapped characters in the source, and that's usually a question mark (or a white question mark in a black diamond.) It would be a huge problem if the default character in the target (for unmapped characters in the source) was a single quote.
Bottom line: escaping unsafe strings by replacing single quotes with two single quotes goes a long ways towards mediating (mitigating?) SQL injection vulnerabilities. But in and of itself, it doesn't guarantee that code is not vulnerable in some other way.
if the input accepts unicode and is implicitly converted to ascii in the database (not as uncommon as it sounds) then an attacker can simply substitute ʻ or ʼ (0x02BB or 0x02BC) in place of single tick to get around the escaping mechanism and the implicit conversion will map those characters to single ticks (at least that's the case in SQL Server)
So I am building a database of all my text messages to get information about my habits and I'm having trouble importing the contents of the messages. Whenever there are apostrophes (often) or quotation marks (not as rare as you might think), I get syntax issues.
Is there a way to make MySQL use something other than " or ' to encase strings (specifically, the field is a VARCHAR). If I could use a ~ or some other rarely used character in text messaging my life would become a whole lot easier.
Preferably you should use parameterised queries, then your database connector takes care of sending the strings to the database in the correct way.
If you need to build the queries by concatenating the values into a query, you need to escape the strings correctly to make them string literals in the SQL code.
Stick to one delimiter for strings, don't use apostrophes around some strings and qoutation marks around others, that only makes it harder to escape them correctly. I suggest that you use apostrophes, as that is what the SQL standard specifies.
To escape the strings correctly to be a string literal delimited by apostropes, you should:
Replace all backslashes by double backslashes, then
Replace all apostrophes by a backslash and an apostrophe
For example, to make the string It's an "example" with a backslash(\). into a string literal, it should end up like this in a query:
insert into Table (txt) values ('It\'s an "example" with a backslash(\\).')
Note: This is a correct way to escape strings for MySQL. Other databases may use different characters for escaping and need other characters to be escaped, so using this for any other database may fail, or even worse open up for SQL injection attacks.
I am concerned about inserting text in a MySQl table w.
I have to insert/update text that contains characters such as / " and '
The escape character / could be inserted only if the NO_BACKSLASH_ESCAPES SQL mode is enabled. wich interfere with the characters " and ' see this link http://dev.mysql.com/doc/refman/5.1/en/string-literals.html#character-escape-sequences
If anyone can explain to is in earth the mysql_real_escape_string() I don't came to understated
I would like to find a pure mysql solution
I am not using php. What I am trying to do here is to "simulate " Content Management System: I am about to write a C# coded solution that manage the content in its different forms(article, category ,tag, etc..) and generate .html files, the MySQl database is local in my computer next i will upload the .html files to the server.
I did this to ensure that all my html pages are html valid and because I don't trust any existent solutions (not only when it concerns coding but in life in general)
Please help
each php db connection extension (mysql, mysqli, pdo) has a special way to safe query against sql injections, when you are using mysql extension, it's strongly recommended to use mysql_real_escape_string() function to make safe all variables used in query, it's most widely used function. i think there isn't any pure solution (when you are using mysql extension in php).
from php.net:
mysql_real_escape_string()-Escapes special characters in the
unescaped_string, taking into account the current character set of the
connection so that it is safe to place it in a mysql_query().
Whatever string data can be inserted into SQL query, if formatted according to 2 rules
it is enclosed in quotes (preferably single ones)
it is passed through mysql_real_escape_string()
if both rules followed, there would be not a single problem with whatever characters, either slashes, quotes or anything.
According to your question, / has no special meaning in MySQL. It's \ that is escape character. It can be escaped by another backslash as well.
$str = 'slashes \ quotes \' or whatever " else symbols';
var_dump($str);
$str = mysql_real_escape_string($str);
$sql = "INSERT INTO table SET str='$str'";
I have a couple escaped characters in user-entered fields that I can't figure out.
I know they are the "smart" single and double quotes, but I don't know how to search for them in mysql.
The characters in ruby, when output from Ruby look like \222, \223, \224 etc
irb> "\222".length => 1
So - do you know how to search for these in mysql? When I look in mysql, they look like '?'.
I'd like to find all records that have this character in the text field. I tried
mysql> select id from table where field LIKE '%\222%'
but that did not work.
Some more information - after doing a mysqldump, this is how one of the characters is represented - '\\xE2\\x80\\x99'. It's the smart single quote.
Ultimately, I'm building an RTF file and the characters are coming out completely wrong, so I'm trying to replace them with 'dumb' quotes for now. I was able to do a gsub(/\222\, "'").
Thanks.
I don't quite understand your problem but here is some info for you:
First, there are no escaped characters in the database. Because every character being stored as is, with no escaping.
they don't "look ilke ?". I's just wrong terminal settings. SET NAMES query always should be executed first, to match client encoding.
you have to determine character set and use it on every stage - in the database, in the mysql client, in ruby.
you should distinguish ruby strings representation from character itself.
To enter character in the mysql query, you can use char function. But in terminal only. In ruby just use the character itself.
smart quotes looks like 2-byte encoded in the unicode. You have to determine your encoding first.