Im trying to escape both parentheses using MySQL. It's giving me an error starting at A.T.
INSERT INTO `schools`(`schoolid`, `name`) VALUES (1,"A.T. Still University of Health Sciences(AZ / MO)")
You don't need to "escape" anything in that statement. You just need to use single-quotes for strings in SQL:
'A.T. Still University of Health Sciences(AZ / MO)'
It looks like double-quotes can be supported in MySQL, depending on the ANSI_QUOTES setting. But unless there's a compelling reason to use them, I'd recommend sticking with single-quotes as it's more standard and more commonly used.
You need to use single quotes for strings instead of double quotes. Try this:
INSERT INTO `schools`(`schoolid`, `name`)
VALUES (1,'A.T. Still University of Health Sciences(AZ / MO)')
SQL FIDDLE DEMO
Use Special Character Escape Sequences in the Insert statement
Escape Sequence Character Represented by Sequence
\0 An ASCII NUL (0x00) character
\' A single quote (“'”) character
\" A double quote (“"”) character
\b A backspace character
\n A newline (linefeed) character
\r A carriage return character
\t A tab character
\Z ASCII 26 (Control+Z); see note following the table
\ A backslash (“\”) character
\% A “%” character; see note following the table
_ A “_” character; see note following the table
Related
I have a field called myfield that contains this string:
{'Content-Language': 'en', 'X-Frame-Options': 'SAMEORIGIN', 'X-Generator': 'Drupal 7 (http://drupal.org)', 'Link': '<https://01.org/node>; rel="shortlink"', 'Some-Header-Key': 'max-age=31; addSomething', 'Content-Encoding': 'gzip'}
I want to capture the 'Some-Header-Key': 'max-age=31; addSomething' where:
1) 'Some-Header-Key', max-age are fixed values that should always be present.
2) The addSomething is a optional.
3) There may be one or more spaces between the double colon and the equal sign
4) The general formal is 'key': 'value', with either single or double quotes.
5) The ([^""|'])* to say: zero or more characters that are not single or double quotes. This to capture addSomething.
I wrote this query:
select myfield
from mytable
where mycol regexp "('|"")Some-Header-Key('|"")\s*:\s*('|"")([^""|'])*max-age\s*=\s*[0-9]+([^""|'])*('|"")";
But it does not return anything!! although myfield contains the above example string.
When I copied the field value into an external text file and run the regexp in grep, the regexp captured the string correctly.
What is wrong in MySQL? I use MySQL workbench 8.0 in Ubuntu 18.04.
Your problem is with the \s in your regex expression. Versions of MySQL prior to 8 do not support this notation, you need to use the character class [:blank:] instead i.e.
where mycol regexp "('|"")Some-Header-Key('|"")[[:blank:]]*:[[:blank:]]*('|"")([^""|'])*max-age[[:blank:]]*=[[:blank:]]*[0-9]+([^""|'])*('|"")"
In MySQL 8, you can use \s but you need to escape the backslash as MySQL uses C-style escape syntax in strings, thus \s just translates to s. So change the \s to \\s and it should work:
where mycol regexp "('|"")Some-Header-Key('|"")\\s*:\\s*('|"")([^""|'])*max-age\\s*=\\s*[0-9]+([^""|'])*('|"")"
Demo on dbfiddle
Not single or double quotes: [^'"]
Zero or more of such: [^'"]
Either a single quote or two double quotes: ('|"")
Either a double quote or two single quotes: ("|'')
One of either type of quote: ['"] or ('|")
A single-quoted string: '[^']*'
A double-quoted string: "[^"]*"
Either of the above: ('[^']*'|"[^"]*")
Next problem: How to quote a regexp string: If it contains ' or ", escape that with a backslash:
my_json REGEXP "('[^']*'|\"[^\"]*\")"
If you use something that does "binding" for you, you don't need to do the escaping. PHP has mysqli_real_escape_string and add_slashes.
But... I you are going to use JSON, you should upgrade to MySQL 5.7 or MariaDB 10.2 so you can use JSON functions instead of REGEXP.
I have a CSV containing the following:
id,homie_id,user_id,some_data,some_datetime,list_stuff,confirmed_at,report_id
1,57,1,,,"{\"assets\":[]}","2014-12-26 16:50:32",18
2,59,1,,,"{\"assets\":[]}","2014-12-26 16:50:46",18
When I run the COPY command, I get an error "Invalid quote formatting for CSV"
Why is that? It has the backslash before the quote, so it should be acceptable. I see Redshift says to use "" instead (https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html#copy-data-format-parameters) but is there a way to tell it to accept \" since that is a valid way to escape quotes with CSVs?
I've already tried Googling and don't see a reason this wouldn't work.
There isn't really a way to force Redshift to use backslash as the escape character.
You have to convert your input data to the format that Redshift can handle. One way is just to replace all backslahes with double quotes. For example,
"{\"assets\":[]}" turns into "{""assets"":[]}" which is then parsable by Redshift and in the end the actual data should look like {"assets":[]} for that field.
From the docs:
The default quote character is a double quotation mark ( " ). When the
quote character is used within a field, escape the character with an
additional quote character. For example, if the quote character is a
double quotation mark, to insert the string A "quoted" word the input
file should include the string "A ""quoted"" word"
I have a set of queries with randoms data that i want to insert in database. Randoms data may have any special characters.
for example:
INSERT INTO tablename VALUES('!^'"twco\dq');
Are there any scope that escape all special characters?
please help.
No, there is no "scope" in MySQL to automatically escape all special characters.
If you have a text file containing statements that were created with potentially unsafe "random values" like this:
INSERT INTO tablename VALUES('!^'"twco\dq');
^^^^^^^^^^^
You're basically screwed. MySQL can't unscramble a scrambled egg. There's no "mode" that makes MySQL work with a statement like that.
Fortunately, that particular statement will throw an error. More tragic would be some nefariously random data,
x'); DROP TABLE students; --
if that random string got incorporated into your SQL text without being escaped, the result would be:
INSERT INTO tablename VALUES('x'); DROP TABLE students; --');
The escaping of special characters has to be done before the values are incorporated into SQL text.
You'd need to take your random string value:
!^'"twco\dq
And run it through a function that performs the necessary escaping to make that value safe for including that as part of the the SQL statement.
MySQL provides the real_escape_string_function as part of their C library. Reference https://dev.mysql.com/doc/refman/5.5/en/mysql-real-escape-string.html. This same functionality is exposed through the MySQL Connectors for several languages.
An even better pattern that "escaping" is to use prepared statements with bind placeholders, so your statement would be a static literal, like this:
INSERT INTO tablename VALUES ( ? )
You can use \ character to escape special characters like below. See this DEMO if in doubt.
INSERT INTO tablename VALUES('\!\^\'\"twco\\dq');
Per MySQL documentation, below are the defined escape sequences
Table 9.1 Special Character Escape Sequences
\0 An ASCII NUL (0x00) character.
\' A single quote (“'”) character.
\" A double quote (“"”) character.
\b A backspace character.
\n A newline (linefeed) character.
\r A carriage return character.
\t A tab character.
\Z ASCII 26 (Control+Z). See note following the table.
\\ A backslash (“\”) character.
\% A “%” character. See note following the table.
\_ A “_” character. See note following the table.
I have a table where a column allows special characters like '/' (forward slash) and '' (back slash).
Now when I try to search such records from table, I am unable to get those.
For example: abc\def or abc/def
I am generating a search query like:
select * from table1_1 where column10 like '%abc\def%'
It is returning 0 rows, but actually there is 1 record existing that should be returned. How do I write the query in this case?
The trick is to double escape ONLY the backslash; for string escapes only a single escape is needed.
For example
The single quote ' only needs escaping once LIKE '%\'%'
But to query backslash \ you need to double escape to LIKE '%\\\\%'
If you wanted to query backslash+singlequote \' then LIKE '%\\\\\'%' (with 5 backslashes)
Explanation Source
excerpt:
Because MySQL uses C escape syntax in strings (for example, “\n” to
represent a newline character), you must double any “\” that you use
in LIKE strings. For example, to search for “\n”, specify it as “\n”.
To search for “\”, specify it as “\\”; this is because the
backslashes are stripped once by the parser and again when the pattern
match is made, leaving a single backslash to be matched against.
In MySQL, this works:
select * from Table1
where column10 like '%abc\\\\def%'
FIDDLE
Backslash is an escape prefix for both strings and LIKE patterns. So you need to double it once for LIKE, and again for string literal syntax.
You have to escape the \ with another \
select * from table1_1 where column10 like '%abc\\def%'
Escaping the LIKE value didn't work when I tried it on MySQL v5.5. After a few tries, what did work was a regex (and I had to escape there too, plus hide it in a character class). I finally got it to work like this:
select * from table1_1 where column10 rlike 'abc[\\]def'
These didn't work:
... column10 like '%abc\\def%'
... column10 like concat('%abc', char(92), 'def%')
... column10 rlike 'abc\\def'
Note you also have this tagged as PL/SQL, which is Oracle. The query you posted works as-is on Oracle. Is the PL/SQL tag a mistake?
Use escaping.
https://dev.mysql.com/doc/refman/5.0/en/string-literals.html
Table 9.1. Special Character Escape Sequences
Escape Sequence Character Represented by Sequence
\0 An ASCII NUL (0x00) character.
\' A single quote (“'”) character.
\" A double quote (“"”) character.
\b A backspace character.
\n A newline (linefeed) character.
\r A carriage return character.
\t A tab character.
\Z ASCII 26 (Control+Z). See note following the table.
\ A backslash (“\”) character.
\% A “%” character. See note following the table.
_ A “_” character. See note following the table.
If you use PHP sprintf() to put together your query where you want:
AND column10 LIKE '%/%'
This worked for me inside sprintf():
AND column10 LIKE '%%/%%'
(MySSQL 8 - WAMP)
We all know that we should use prepared statements or the appropriate replacement/formatting rules in order to prevent sql injection in our applications.
However, when taking a look at MySQL's list of character literals, I noticed that it includes the following characters:
\0 An ASCII NUL (0x00) character.
\' A single quote (') character.
\" A double quote (") character.
\b A backspace character.
\n A newline (linefeed) character.
\r A carriage return character.
\t A tab character.
\Z ASCII 26 (Ctrl+Z). See note following the table.
\\ A backslash (\) character.
\% A % character.
\_ A _ character.
Now, while the % and _ characters need to be escaped in order to prevent injection of unwanted wildcards into LIKE statements, and while the ' (single quote), \ (backslash), and " (double quote) all need to be escaped in order to prevent injection of arbitrary SQL - could having any of these other characters unescaped lead directly to a SQL injection vulnerability that would not otherwise be present? Does anyone have any real world examples of such an exploit?
Let's assume we are building our query like:
SELECT * FROM users WHERE username='$user'
Is there any value for $user where the only unescaped character literals are \b (backspace), \0 (NUL), \n (newline), \r (linefeed), \t (tab) or \Z (Ctrl+Z) that allows the injection of arbitrary SQL into this query?
Considering the below lines from mysql_real_escape_string() manual :
MySQL requires only that backslash and the quote character used to quote the string in the query be escaped. mysql_real_escape_string() quotes the other characters to make them easier to read in log files.
SQL injection in MySQL should not be possible with these special characters alone by themselves : \b \0 \n \r \t \Z .
However String Literals manual states the following but the reasons specified ( or not ) does not relate to SQL injection :
If you want to insert binary data into a string column (such as a BLOB column), you should represent certain characters by escape sequences. Backslash (“\”) and the quote character used to quote the string must be escaped. In certain client environments, it may also be necessary to escape NUL or Control+Z. The mysql client truncates quoted strings containing NUL characters if they are not escaped, and Control+Z may be taken for END-OF-FILE on Windows if not escaped.
Furthermore , in a simple test , irrespective of weather the above listed special characters are escaped or not , MySQL yielded same results . In other words MySQL did not even mind :
$query_sql = "SELECT * FROM `user` WHERE user = '$user'";
The above query worked similarly for non-escaped and escaped versions of those above listed characters as put below :
$user = chr(8); // Back Space
$user = chr(0); // Null char
$user = chr(13); // Carriage Return
$user = chr(9); // Horizontal Tab
$user = chr(26); // Substitute
$user = chr(92) .chr(8); // Escaped Back Space
$user = chr(92) .chr(0); // Escaped Null char
$user = chr(92) .chr(13); // Escaped Carriage Return
$user = chr(92) .chr(9); // Escaped Horizontal Tab
$user = chr(92) .chr(26); // Escaped Substitute
Test table and data used in the simple test :
-- Table Structure
CREATE TABLE IF NOT EXISTS `user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user` varchar(10) CHARACTER SET utf8 NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- Table Data
INSERT INTO `user` ( `user` ) VALUES
( char( '8' ) ),
( char( '0' ) ),
( char( '10' ) ),
( char( '13' ) ),
( char( '9' ) ),
( char( '26' ) );
An obligatory addendum from 2020:
Dealing with characters was proven to be inefficient and obsoleted
You must use prepared statements and forget about escaping, "dangerous characters" or any of that business.
Using parameterized queries is considered the only proper way to protect from SQL injections, for the reasons provided in the original answer below:
Which characters are actually capable of causing SQL injection in mysql
There are no such characters.
It is not "characters" that cause the SQL injections. But improper formatting. Any character, depends on the circumstances, could be either "dangerous" or absolutely harmless. Limiting your protection to some subset is a dangerous delusion that will actually lead to SQL injection sooner or later.
There are two wrong statements in your question that led you to a confusion:
We all know that we should use ... the appropriate replacement rules in order to prevent sql injection in our applications.
This statement is wrong. Not replacement but formatting. The difference is essential. Replacement alone does not protect from injections, while formatting does. Note that every distinct part of the query require different formatting which being useless for any other part. Say, there is another character, essential for injection protection - a backtick (`). But you didn't list it because it has nothing to do with string literals.
the ' (single quote), \ (backslash), and " (double quote) all need to be escaped in order to prevent injection
That's a gravely wrong statement. Escaping do not prevent injections. These characters need to be escaped in order to format strings and has absolutely nothing to do with injections. While it is true that properly formatted query part is invulnerable. But the truth is - you have to format dynamical query parts just for sake of it, to follow the syntax rules and not because of whatever injections. And you will have your query impenetrable just as a side effect.
Now you can see why your last statement,
why all of these other characters are vulnerable enough to be escaped via mysql_real_escape_string, as it is not immediately obvious to me.
is wrongly put:
It is string formatting rules require these characters, not whatever "vulnerability". Some of them are escaped just for convenience, some for readability, some for the obvious reason of escaping a delimiter. That's all.
To answer recent questions from comments:
I really want an answer to this, as PHP's mysql_real_escape_string does not quote these literals either.
Again: although in the mind of average PHP user mysql_real_escape_string() is strongly connected to whatever scaring injection, in reality it doesn't. There are no "dangerous" characters. Not a single one. There are some service characters with special meaning. They have to be escaped in some circumstances, depends on the context.
Thus, there is no connection between characters escaped by this function, and whatever "danger". The moment you start thinking that mysql_real_escape_string()'s purpose is to escape "dangerous" characters, you are indeed putting yourself in a danger. While as long as you are using this function only to escape strings (and doing it unconditionally) - you may consider yourself safe (of course if you don't forget to format all other literals too, using their respective formatting rules)
I want to know if the "%" character can lead to anything more than extra results in a LIKE clause.
No.