This failed in Java 13 (JDBC) code so I went to MySQL Workbench to duplicate problem.
I run a simple query as:
START TRANSACTION;
SET SESSION sql_mode = NO_BACKSLASH_ESCAPES;
SELECT *, "x\\x", "y\y" from dirs
WHERE d_pathname like 'E:\\\\BOOKS\\\\Dictionaries_and_Encyclopedias\\\\%' ORDER BY d_pathname;
and I get 400 rows returned. The issue is, that I do not want to use double-backslashes.
Rows returned show a single backslash, not a double backslash.
Interestingly, the x\\x and y\y clauses appear just as represented in the SELECT statement.
When I remove the double backslashes in the LIKE clause, I get zero rows!
Why? I'd rather not have to double-up the backslashes, and run simple and clean code.
The NO_BACKSLASH_ESCAPES mode only affects how backslashes are treated in ordinary string literals. It doesn't change how they're processed in LIKE patterns.
However, you can use the ESCAPE option to specify a different character to use as the escape character in LIKE. Just use some other character that doesn't appear in your pattern.
WHERE d_pathname like 'E:\BOOKS\Dictionaries_and_Encyclopedias\%' ESCAPE '|'
Related
This question already has answers here:
How should I escape characters inside this LIKE query?
(4 answers)
Closed last year.
I have a column called \'column. I am able to use it in SELECT statements just like any other column. However, when I try to perform SHOW COLUMNS FROM myTable LIKE '\\''column' I get no results. I observed that it works if I double escape the backslash: '\\\\''column'.
I tested this from MariaDB console, but I also observed the same behaviour in MySQL 8.
How does the escaping work? How should I properly escape the value so that I can fetch the column information?
DB Fiddle
From the mysql documentation
MySQL uses C escape syntax in strings (for example, \n to represent
the newline character). If you want a LIKE string to contain a literal
, you must double it. (Unless the NO_BACKSLASH_ESCAPES SQL mode is
enabled, in which case no escape character is used.) For example, to
search for \n, specify it as \\n. To search for \, specify it as \\\\;
this is because the backslashes are stripped once by the parser and
again when the pattern match is made, leaving a single backslash to be
matched against.
Having followed some tips on escaping apostrophes I am getting an unexpected combination of escape characters in the resulting sql statement. The following rails 4 active record statement is run against 5.5.42-MariaDB:
User.where(["surname LIKE ?", "%#{params[:search]}%"])
Where
params[:search] = "O'Keefe"
A .to_sql generates
SELECT * FROM users WHERE surname LIKE '%O\\'Keefe%'
MySQL/MariaBD expects an apostrophe to be escaped as two single apostrophes '' , or with a single backslash \' so this results in a syntax error. I am looking for help to understand why two backslashes \\' are appearing, and for a solution that will maintain protection against SQL injection.
UPDATE
After further investigation following suggestions below, it appears as though the console .to_sql output SELECT * FROM users WHERE surname LIKE '%O\\'Keefe%' is not what is passed onto MySQL. It failed for me 'cos I simply copied the statement into a mysql console to test execution. There is some black magic on route to the database that converts the double backslash \\' into a valid mysql escape sequence.
So problem 1/2 solved
User.where(["surname LIKE ?", "%#{params[:search]}%"])
is valid syntax that correctly auto-escapes the user input string. But can anyone shed any light on the reason for the generation of the double backslash and how it is modified on its way to database execution?
Try this:
User.where(["surname LIKE ?", "%#{params[:search].gsub("'", "''")}%"])
http://dev.mysql.com/doc/refman/5.0/en/string-literals.html#character-escape-sequences
The last person in my job has flooded column names with special characters such as (?,!, and /), as well as used many reserved keywords for column names (more often than not, timestamp or user is used).
Normally, I step around this by using double quotes or brackets to escape the SQL object. A subset of the full list of columns are below:
DriverID,
Department,
Odometer,
MerchantState,
MerchantCity,
gallons/Units,
timestamp,
tax
Inside my query, I wrap the two columns in question (gallons/units and timestamp) inside double quotes. Timestamp because it's a reserved keyword, and Gallons/units, because without the quotes, SQL reads the query, stops at the slash, and tells me "Gallons" is not a column inside the table.
If I do wrap double quotes around the column name, SQL returns a different error: "Operand should contain 1 column(s)".
I've tried every variant (only capturing the slash in quotes, quoting both, using brackets, mixing brackets and quotes, etc. but with to no avail).
Is there anything I can do to fix this query short of renaming the column name and changing the associated code in the program that pulls from it? (the really tedious task I'm trying to avoid).
In SQL Server, identifiers can be delimited using square brackets, e.g.
SELECT [gallons/units] ...
In MySQL, identifiers can be delimited using backticks, e.g.
SELECT `gallons/units` ...
(NOTE: If MySQL SQL_MODE includes ANSI_QUOTES, then double quotes are treated as delimiters for identifiers, similar to the way Oracle handles double quotes; absent that setting, double quotes are handled as delimiters for string literals. With ANSI_QUOTES included SQL_MODE, "gallons/units" will be interpreted as an identifier (column name). Without ANSI_QUOTES, MySQL will see it as a string literal, as if it were enclosed in single quotes.)
FOLLOWUP:
As far as an error "operand should contain only 1 column(s)", that's usually a problem with query semantics, not an issue with escaping identifiers.
A subquery in the SELECT list can return only a single expression, for example, this would throw an error:
Query: SELECT 'foo' , ( SELECT 'fee' AS fee, 'fi' AS fi )
Error Code: 1241
Operand should contain 1 column(s)
You can try backticks instead of double quotes
`gallons/units`
There are a couple of options. First, have you tried using %/ to escape the slash?
Example: "select * from 'gallons%/units';"
Second one I've found, which may or may not be helpful, is to provide an escape character definition, such as
http://blogs.msdn.com/b/zainala/archive/2008/08/17/using-and-escape-clause-in-sql-server-like-query.aspx
select * from MyTable where Description like '|[summary|]%' escape '|'
In your case
select * from 'gallons|/units' escape '|'
You indicate both mysql and sql-server in your tags, so I'm not sure which server support I should be looking for exactly.
Does MYSQL has a bult-in escaping mechanism?
I'm running some scripts and inserting values to a MYSQL Database. When monitoring the script everything seems ok, printing outputs shows everything is in order, but when checking the database, some values get inserted in a weird fashion, with some letters replaced with control characters or breakspaces (I´m doing set basename=!var:~7! to get the values to insert that present problems).
Specifically, the lowercase b is replaced with a BS char and lowercase r with a break.
Since the script is a Windows Batch File, I was wondering if I can force to escape those characters via query (like mysql_real_escape_string in PHP but directly in MYSQL) or set some option server side so the database take care of those cases.
Any ideas?
What you describe is the result we would expect, given a backslash character in the input stream.
Where '\b' occurs in a string literal, that will be interpreted as a backspace character.
Where '\r' occurs in a string literal, that will be interpreted as a carriage return character.
To avoid the interpretation of these (and other similar) special "escape sequence" characters, the normative pattern is to precede the backslash character with another backslash.
Where '\\' occurs in a string literal, that will be interpreted as a single backslash character.
So, to answer your questions... yes, MySQL has a builtin "escape sequence" for special characters; that sounds like the problem you are encountering; some of your backslash characters are being interpreted as "escape sequences".
And, no, there's no option in the mysql command line client to perform mysql_real_escape_string type functionality. The command line client passes the strings to the server as it receives them.
But it is possible to set sql_mode of the session to include the NO_BACKSLASH_ESCAPES option. That disables the interpretation of the backslash character as the start of special escape sequence, and a backslash character will interpreted as a literal backslash character.
To query the current sql_mode of the session:
SELECT ##SESSION.sql_mode ;
To set the current sql_mode, e.g.
SET SESSION sql_mode = 'NO_BACKSLASH_ESCAPES'
NOTE: the SET statement will overwrite the current sql_mode, not just change the one setting. So you may just want to add to the existing sql_mode, if it's already set to something other than blank, e.g.
If current sql_mode is set to 'ALLOW_INVALID_DATES,ANSI_QUOTES', you'd want to just add the option to the current setting, e.g.
SET SESSION sql_mode = 'ALLOW_INVALID_DATES,ANSI_QUOTES,NO_BACKSLASH_ESCAPES'
I'm trying to write a MySQL query to identify first name fields that actually contain initials. The problem is that the query is picking up records that should not match.
I have tested against the POSIX ERE regex implementation in RegEx Buddy to confirm my regex string is correct, but when running in a MySQL query, the results differ.
For example, the query should identify strings such as:
'A.J.D' or 'A J D'.
But it is also matching strings like 'Ralph' or 'Terrance'.
The query:
SELECT *, firstname REGEXP '^[a-zA-z]{1}(([[:space:]]|\.)+[a-zA-z]{1})+([[:space:]]|\.)?$' FROM test_table
The 'firstname' field here is VARCHAR 255 if that's relevant.
I get the same result when running with a string literal rather than table data:
SELECT 'Ralph' REGEXP '^[a-zA-z]{1}(([[:space:]]|\.)+[a-zA-z]{1})+([[:space:]]|\.)?$'
The MySQL documentation warns about potential issues with REGEXP, I'm unsure if this is related to the problem I'm seeing:
Warning The REGEXP and RLIKE operators work in byte-wise fashion, so
they are not multi-byte safe and may produce unexpected results with
multi-byte character sets. In addition, these operators compare
characters by their byte values and accented characters may not
compare as equal even if a given collation treats them as equal.
Thanks in advance.
If you're testing this in the mysql client, you need to escape the backslashes. Each occurence of \. must turn into \\. This is necessary because your input is first processed by the mysql client, which turns \. into .. So you need to make it keep the backslashes by escaping them.