Does MYSQL has a bult-in escaping mechanism?
I'm running some scripts and inserting values to a MYSQL Database. When monitoring the script everything seems ok, printing outputs shows everything is in order, but when checking the database, some values get inserted in a weird fashion, with some letters replaced with control characters or breakspaces (I´m doing set basename=!var:~7! to get the values to insert that present problems).
Specifically, the lowercase b is replaced with a BS char and lowercase r with a break.
Since the script is a Windows Batch File, I was wondering if I can force to escape those characters via query (like mysql_real_escape_string in PHP but directly in MYSQL) or set some option server side so the database take care of those cases.
Any ideas?
What you describe is the result we would expect, given a backslash character in the input stream.
Where '\b' occurs in a string literal, that will be interpreted as a backspace character.
Where '\r' occurs in a string literal, that will be interpreted as a carriage return character.
To avoid the interpretation of these (and other similar) special "escape sequence" characters, the normative pattern is to precede the backslash character with another backslash.
Where '\\' occurs in a string literal, that will be interpreted as a single backslash character.
So, to answer your questions... yes, MySQL has a builtin "escape sequence" for special characters; that sounds like the problem you are encountering; some of your backslash characters are being interpreted as "escape sequences".
And, no, there's no option in the mysql command line client to perform mysql_real_escape_string type functionality. The command line client passes the strings to the server as it receives them.
But it is possible to set sql_mode of the session to include the NO_BACKSLASH_ESCAPES option. That disables the interpretation of the backslash character as the start of special escape sequence, and a backslash character will interpreted as a literal backslash character.
To query the current sql_mode of the session:
SELECT ##SESSION.sql_mode ;
To set the current sql_mode, e.g.
SET SESSION sql_mode = 'NO_BACKSLASH_ESCAPES'
NOTE: the SET statement will overwrite the current sql_mode, not just change the one setting. So you may just want to add to the existing sql_mode, if it's already set to something other than blank, e.g.
If current sql_mode is set to 'ALLOW_INVALID_DATES,ANSI_QUOTES', you'd want to just add the option to the current setting, e.g.
SET SESSION sql_mode = 'ALLOW_INVALID_DATES,ANSI_QUOTES,NO_BACKSLASH_ESCAPES'
Related
This failed in Java 13 (JDBC) code so I went to MySQL Workbench to duplicate problem.
I run a simple query as:
START TRANSACTION;
SET SESSION sql_mode = NO_BACKSLASH_ESCAPES;
SELECT *, "x\\x", "y\y" from dirs
WHERE d_pathname like 'E:\\\\BOOKS\\\\Dictionaries_and_Encyclopedias\\\\%' ORDER BY d_pathname;
and I get 400 rows returned. The issue is, that I do not want to use double-backslashes.
Rows returned show a single backslash, not a double backslash.
Interestingly, the x\\x and y\y clauses appear just as represented in the SELECT statement.
When I remove the double backslashes in the LIKE clause, I get zero rows!
Why? I'd rather not have to double-up the backslashes, and run simple and clean code.
The NO_BACKSLASH_ESCAPES mode only affects how backslashes are treated in ordinary string literals. It doesn't change how they're processed in LIKE patterns.
However, you can use the ESCAPE option to specify a different character to use as the escape character in LIKE. Just use some other character that doesn't appear in your pattern.
WHERE d_pathname like 'E:\BOOKS\Dictionaries_and_Encyclopedias\%' ESCAPE '|'
Is there any way to perform a SQL injection when single quotes are escaped by two single quotes? I know the MySQL server is using this specific technique to prevent against an attack. I'm trying to log in as a specific user but all of the common injections I've tried for the password have not worked successfully (i.e. ' or '1'='1, ' or ' 1=1, etc.).
No, and yes.
There's no way to have an unsafe values "breakout" of literal values that are enclosed in single quotes, if the value being supplied is "escaped" by preceding single quotes by with an additional single quote.
That is, assuming that your statement is guaranteeing that string literals are enclosed in quotes, as part of the "static" SQL text.
example perl-ish/php-ish
$sql = "... WHERE t.foo = '" . $safe_value . "' ... ";
^ ^
I've underscored here that the single quotes enclosing the literal are part of the SQL text. If $safe_value has been "escaped" by preceding each single quote in the "unsafe" value with another single value to make it "safe"...
$unsafe_value $safe_value
------------- ------------
I'm going I''m going
'she''s' ''she''''s''
1'='1 -- 1''=''1 --
As long as the escaping is handled properly, that we guarantee that potentially unsafe values are are run through the escaping, then including single quotes in data values is not a viable way to "breakout" of a literal with the SQL text.
That's the "no" part of the answer.
The "yes" part of the answer.
One of the biggest problems is making sure this is done EVERYWHERE, and that a mistake has not been made somewhere, assuming that a potentially unsafe string is "safe", and is not escaped. (For example, assuming that values pulled from a database table are "safe", and not escaping them before including them in SQL text.)
Also, the single quote trick is not the only avenue for SQL injection. The code could still be vulnerable.
Firstly, if we're not careful about other parts of the statement, like the single quotes enclosing string literals. Or, if for example, the code were to run the $sql through some other function, before it gets submitted to the database:
$sql = some_other_function($sql);
The return from some_other_function could potentially return SQL text that was in fact vulnerable. (As a ridiculous example, some_other_function might replace all occurrences of two consecutive single quotes with a single single quote. DOH!)
Also, with the vast number of possible unicode characters, if we're ever running through a characterset translation, there's also a possibility that some unicode character could get mapped to a single quote character. I don't have any specific example of that, but dollars to donuts that somewhere, in that plethora of multibyte encodings, there's some unicode character somewhere that will get translated to a single quote in some target.
There's a default character in the target for unmapped characters in the source, and that's usually a question mark (or a white question mark in a black diamond.) It would be a huge problem if the default character in the target (for unmapped characters in the source) was a single quote.
Bottom line: escaping unsafe strings by replacing single quotes with two single quotes goes a long ways towards mediating (mitigating?) SQL injection vulnerabilities. But in and of itself, it doesn't guarantee that code is not vulnerable in some other way.
if the input accepts unicode and is implicitly converted to ascii in the database (not as uncommon as it sounds) then an attacker can simply substitute ʻ or ʼ (0x02BB or 0x02BC) in place of single tick to get around the escaping mechanism and the implicit conversion will map those characters to single ticks (at least that's the case in SQL Server)
Is there a way to specify Unicode character literals in MySQL?
I want to replace a Unicode character with an Ascii character, something like the following:
Update MyTbl Set MyFld = Replace(MyFld, "ẏ", "y")
But I'm using even more obscure characters which are not available in most fonts, so I want to be able to use Unicode character literals, something like
Update MyTbl Set MyFld = Replace(MyFld, "\u1e8f", "y")
This SQL statement is being invoked from a PHP script - the first form is not only unreadable, but it doesn't actually work!
You can specify hexadecimal literals (or even binary literals) using 0x, x'', or X'':
select 0xC2A2;
select x'C2A2';
select X'C2A2';
But be aware that the return type is a binary string, so each and every byte is considered a character. You can verify this with char_length:
select char_length(0xC2A2)
2
If you want UTF-8 strings instead, you need to use convert:
select convert(0xC2A2 using utf8mb4)
And we can see that C2 A2 is considered 1 character in UTF-8:
select char_length(convert(0xC2A2 using utf8mb4))
1
Also, you don't have to worry about invalid bytes because convert will remove them automatically:
select char_length(convert(0xC1A2 using utf8mb4))
0
As can be seen, the output is 0 because C1 A2 is an invalid UTF-8 byte sequence.
Thanks for your suggestions, but I think the problem was further back in the system.
There's a lot of levels to unpick, but as far as I can tell, (on this server at least) the command
set names utf8
makes the utf-8 handling work correctly, whereas
set character set utf8
doesn't.
In my environment, these are being called from PHP using PDO, for what difference that may make.
Thanks anyway!
You can use the hex and unhex functions, e.g.:
update mytable set myfield = unhex(replace(hex(myfield),'C383','C3'))
The MySQL string syntax is specified here, as you can see, there is no provision for numeric escape sequences.
However, as you are embedding the SQL in PHP, you can compute the right bytes in PHP. Make sure the bytes you put into the SQL actually match your client character set.
There is also the char function that will allow what you wanted (providing byte numbers and a charset name) and getting a char.
I have a php script that inserts values into mySQL table
INSERT INTO stories (title) VALUES('$_REQUEST[title]);
I checked the values of my request variables before going into the table and it's fine.
But when I add title=john to the table for example,
I get something like this:
title = "[][][][]john"
and when I extract the value, it's a newline then john.
I have my columns set to utf-8, I tried swedish character set as well.
Note: I don't get this error when inserting values from the phpMyAdmin commandline
You need {} around any array notation when used inside "".
$q="INSERT INTO stories(title) VALUES('{$_REQUEST['title']}')";
BTW, it would be better, when checking your $_REQUEST vars to store the sanitized versions in new variables, and to be sure to escape them with real_escape_string()
SET NAMES <encoding> query must be executed every time you connect to your database.
very simple rule.
where <encoding> is your HTML page encoding in mysql dialect (utf8 for the utf-8)
You need to check the character set of the database, the server, and the client.
Note that it's not a swedish character set, it's a swedish collation.
I have a couple escaped characters in user-entered fields that I can't figure out.
I know they are the "smart" single and double quotes, but I don't know how to search for them in mysql.
The characters in ruby, when output from Ruby look like \222, \223, \224 etc
irb> "\222".length => 1
So - do you know how to search for these in mysql? When I look in mysql, they look like '?'.
I'd like to find all records that have this character in the text field. I tried
mysql> select id from table where field LIKE '%\222%'
but that did not work.
Some more information - after doing a mysqldump, this is how one of the characters is represented - '\\xE2\\x80\\x99'. It's the smart single quote.
Ultimately, I'm building an RTF file and the characters are coming out completely wrong, so I'm trying to replace them with 'dumb' quotes for now. I was able to do a gsub(/\222\, "'").
Thanks.
I don't quite understand your problem but here is some info for you:
First, there are no escaped characters in the database. Because every character being stored as is, with no escaping.
they don't "look ilke ?". I's just wrong terminal settings. SET NAMES query always should be executed first, to match client encoding.
you have to determine character set and use it on every stage - in the database, in the mysql client, in ruby.
you should distinguish ruby strings representation from character itself.
To enter character in the mysql query, you can use char function. But in terminal only. In ruby just use the character itself.
smart quotes looks like 2-byte encoded in the unicode. You have to determine your encoding first.